The Over-Credited Destination Job Server

Would you get a little upset if someone else kept taking all the credit for the work you do. Would you get down right furious when that person couldn’t fix any issues relating to their ill-earned reputation when a real problem came along and then you had to come along and save the day without even getting an ounce of credit. You would have to be a saint not to get upset about that, wouldn’t you, or perhaps you would just have to be a Business Objects Reporting/Adaptive Job Server?

The truth of that matter is that the Destination Job Server is not really at fault. He was born with a very bad and confusing name. Then over time IT folk got confused with the name, fixed an issue by configuring destinations on every server and then credited the “Destination Job Server” with being the key solution. While one can learn a lot from forum discussions, some BO forums are ripe with incorrect advice to fix job destination errors by configuring the Destination Job Server’s destinations. So this little article is written in hopes of dispelling a myth and giving credit were credit is due.

Destination Job Server, What Is It Good For?

First of all, let it be known that the Business Objects XI Destination Job Server is only responsible for handling the requests submitted through the “Send To” command within InfoView. Yes, that is right. When you are in InfoView and you select a document (check the box next to it) and then proceed to select to send it to an inbox, email, or other destination. In Business Objects XI InfoView it looks like this:

When you use this InfoView “Send To” functionality you are in effect creating a job that only has the purpose of delivering a file to a destination. This, my friends, is what a Business Objects Enterprise XI Destination Job Server does; nothing more and nothing less.

So what about the “destination DLL disabled. CrystalEnterprise.Smtp:”error I get on my jobs server?

If you have this question still please read the above paragraph one more time and then read my previous article titled “Fixing the Business Objects XI “destination DLL disabled. CrystalEnterprise.Smtp:” Error” and now log in to CMC and configure your destination on all job servers in the cluster. Now you know, and knowing is half the battle.

Denying Security Access Explicitly in Business Objects XI 3.1

Maybe I am the only one, but I have struggled twice with this topic and so I thought that I would write this short article to make sure I remember the correct workflow and hopefully help someone else out too.

Sometimes You Just Want to Say “Stay Out”

There are times when you may want to refuse access to a specific user or user group. In my case, this is usually for a specific sub-group of users where the parent group is allowed access. To be honest, there are many reasons and some may be just to appease over-cautious administrators, customers, etc.

Most Restrictive Rights Always Take Precedence

Whatever the reason, the rule of Business Objects security is always that “the most restrictive rights always take precedence”. Knowing this, some choose to deny access explicitly, even though it was never granted in the first place, just to be sure that now and in the future that user/group does not get access.

Adding the Users or Groups

In our example we will use a folder, but this translates to any object, top-level rights, or even applications. Of course, you will want to log in to the Central Management Console (CMC) using an account that is a member of the “Administrators” group. Locate your object, we are using the “BOT Expenses” folder I created for this example. By default when I select the “User Security” option from the right-click drop-down menu on the folder I see this:

By default my environment denies access to the “Everyone” group. This is done by editing the top-level folder security in advance.
Next we need to select “Add Principal” and choose the groups to whom we want to deny access.

Assign Security – What? Why? I Want to Deny not Assign!

Click the “Add and Assign Security” button and you will see the following page:

Now here is where I was getting confused. At first I was looking for an Access Level called “No Access”, this was the BO XI R2 in me. Then I thought well, maybe it is implicit here and all I have to do is “Save” and be done. I clicked “Apply” and the screen just re-drew itself, making no changes. Then I clicked “OK” and I saw this message:

Click “OK” here just returned me to the screen depicted in the first screenshot above and erased all of the “principals”. Thank you CMC, I love you too.

So How Do You Make It Work?

OK, I will just tell you how to make it work. Return back to this screen:

Now either click the “Remove Access” button or manually un-check the boxes next to “Inherit From Parent Folder” and “Inherit From Parent Group”. Either action results in the same result. Now click “OK” and you will still get this prompt:

But this is “OK” this time… so click “OK”.

Successfully Explicitly Denying Security Access in Business Objects XI 3.1

Now you should see the groups that you select with the most desired access level of “No Access”.

Congratulations, to you if you figured this out on your own and can remember this less-than-intuitive CMC security workflow. Once you look at the removing inheritance aspect it makes sense, but it still seems like there should have been an access level called “No Access”. Perhaps, an overly cautious Business Objects XI 3.1 administrator would create a custom access level that has everything explicitly denied and call this one “No Access”, unless Business Objects XI reserves that name in which case you could call it “No Access – Thanks to BusinessObjectsTips.com!”. That would be a great name for sure.

Please read Marshall’s important note below for additional explanation.

Fixing the Business Objects XI “destination DLL disabled. CrystalEnterprise.Smtp:” Error

One of the most helpful error messages (sad very sarcastically) in Business Objects XI is “destination DLL disabled. CrystalEnterprise.Smtp:”. You get this one usually after recently configuring a new job server or setting up your first scheduled job in a Business Objects XI environment. This error exists in XI R2 and it is definitely present in XI 3. So what does it mean and how do you fix it?

What is the “destination DLL disabled. CrystalEnterprise.Smtp:” Error?

This error essentially means that you do not have an SMTP/Email destination configured on the job server that your job or action is attempting to use. In some cases, your job may not even being trying to send email to anyone and you will still get this error if you have not configured an SMTP/Email destination for the job server being used. In XI R2 you need to be sure that the destination is enabled as well as configured. In XI 3 your will need to add it as it is not there by default and this will also enable it. Don’t forget the “Destination Job Server” as well as this one handles non-scheduled job, such as distributions straight from InfoView (I think). Also since jobs are assigned seemingly on a random basis, if your cluster has multiple job servers check them (or the specific job error) to pinpoint to trouble-maker.

How do I fix this error?

Easy, just configure an SMTP or Email destination on the job server (CMC > Servers > Job Server > Destinations). I highly recommend that you configure the SMTP/Email destination on all reporting job server (WebI, DeskI, Crystal), adaptive jobs servers (XI 3), and destination job servers. The only information that you must enter is domain, server name and port; however, your email admins might require the user id and password fields as well. Make certain that in XI R2 the destination is enabled too. This should stop the error; however, you might want to add more default information for the From/To email fields to prevent errors caused by users who will accept the defaults. I usually use the variable for the users email ID in both fields.

Cleanly Stopping and Starting Business Objects Servers

I really cannot explain why it is that I have NEVER seen any documentation that advises Business Objects XI administrators on how to stop and start Business Objects properly. For those of us running part of our business on BO XI we are very concerned about minimizing errors for users and scheduled jobs while we are restarting Business Objects.

The Cleanest Method to Stop a Business Objects Environment

I have discussed this topic with my senior engineers and the following is based on the input I received from them and from my own experience and knowledge. I will label optional steps that will make your stop and start as graceful as possible; these are optional, but they are the best method to follow if you have the time to do so. Also when following the steps make certain that the step is complete and the server is completely stopped/started before proceeding to the next step. Additionally if you have a clustered environment, and you should if at all possible, then you can stop all servers of the same kind in any order or even simultaneously.:

1. Please first make note of any pre-existing disabled servers to be sure that you do not enable them mistakenly later on. Screenshots are useful and fast, just be sure to save them.
2. Shutdown your web/application layer. This will stop your users from launching new jobs and from getting strange errors as you are in the midst of your shutdown. You could just disable a proxy server (if you use one to cut off the access, but (Graceful Option) you may want to completely flush the system by completely stopping the web server.
3. (Graceful Option) Through the Central Management Console (CMC) or through the Central Configuration Manager (CCM) Disable all BO servers except for the CMS, Input FRS, Output FRS, and Destination servers.
4. (Graceful Option) Wait as long as reasonable/acceptable or until all user sessions/requests have cleared the disabled servers before proceeding to the next step. Usually 30 to 60 minutes is sufficient for any valid threads being processed.
5. Shutdown the Event Servers (use CCM or CMC). This would stop any related scheduled jobs from launching.
6. Shutdown all Job Servers (WebI, DeskI, Program…).
7. Shutdown the Destination Servers.
8. Shutdown all Report Servers (WebI, DeskI, Crystal…).
9. Shutdown any other non-CMS Servers that are still up and running.
10. Shutdown CMS. CMS should always be last. This is essential and it will make your CMS shutdown go much faster and more smoothly. It may also help reveal any problems that your CMS may be having (for example, if it won’t shutdown you’ll know it is not because of any other lingering servers in the cluster).
11. With the CMS completely shutdown you are official and completely down.

The Cleanest Method to Start a Business Objects Environment

It is assumed that EVERYTHING is down prior to beginning these steps. If your environment is only partially down we strongly recommend that you first shut it down completely before attempting a start/restart. You want to have a clean environment so do yourself and your users this favor. When starting groups of the same kind of server you can start them one by one or simultaneously, jsut be sure that all are started before proceeding to the next step. (Graceful Option) If you disabled any servers prior to the shutdown (as part of a graceful shutdown) then you should enable them immediately after they have been started:

1. Start the CMS servers. This will take a little while depending mostly on the number of objects in your environment.
2. Start Destination Servers, Input FRS, and Output FRS.
3. Start Event Servers. Also as a side note, please be aware that Event Servers must always be restarted following the restart of any CMS in the cluster.
4. Start all Report Servers.
5. Start all Job Servers.
6. Start any other servers that have not yet been started.
7. Start your web/application layer.

The Value of the (Graceful Stop/Start Method)

Disabling Business Objects XI servers allows the servers to retain and complete their current threads/work, but it stops it from accepting any new work. Nevertheless, this requires caution. If users retain access to the environment (web and application layer are up) while you are disabling servers and you disable ALL servers, or most of them to a point below capacity demands, then you will cause errors for users! Therefore, if the users do retain access to the environment disabling servers would only be done partially to the collection of similar servers (with reason). Also do not forget to enable the servers after the servers are restarted!!!

Please also see the article “The Best Way to Stop a Business Objects Server“.

Sizing Limits to Web Intelligence Report Server Maximum Simultaneous Connections

In Business Objects XI R2 newly created Web Intelligence Report Servers default to 50 “Maximum Simultaneous Connections” (editable in the Central Management Console, or CMC). In Business Objects XI 3.X the “Maximum Simultaneous Connections” setting is defaulted to 100! Can we read anything in to these changes in default settings? Do the default settings mean anything at all with regards to the settings you should have on your system?

Sizing Limits to Maximum Simultaneous Connections

General consensus of our sources tell us that XI 3.X is better coded and can handle more connections than XIR2, with regards to WebI Report Servers. Nevertheless, it would be quite a leap to say that all other things as equal as possible that XI 3.X WebI Report Servers can handle twice as much traffic. Seriously, don’t count on it.

I feel I should add personal experience and best practice here. I see the default setting for “Maximum Simultaneous Connections” on the WebI Report Server as a maximum setting. I have personally never exceeded it nor witness it exceeded by anyone. It is generally held that if you need more Simultaneous Connections then you ought to add another WebI Report Server to your environment. Of course, keep in mind that SAP Business Objects’ general guideline is that you have no more than 1 Web Intelligence Report Server per available CPU core (for example, a server with 4 quad-core CPUs has 16 CPU cores); so there is a limit there too.

Real-World Web Intelligence Report Server Sizing

A production system will generally run better having more Web Intelligence Report Servers with lower Maximum Simultaneous Connections. For example, a server with 4 dual-core CPUs would run better having 8 WebI Rpt Servers each set at 30 Maximum Simultaneous Connections than it would having 4 WebI Rpt Servers each set at 60 Maximum Simultaneous Connections in CMC. In the real-world tuning and balancing usually are based on observed performance within configuration guidelines.

To get your sizing in the right place you should know what you maximum concurrent users are (logged-in users plus concurrent schedule jobs). This number is essentially how many Maximum Simultaneous Connections you will need. So let’s say you never have more than 100 concurrent users, but you can at those peak times also have 25 scheduled jobs. Keep in mind that most users will only do one process at a time, but some like me will be refreshing one report while editing another simultaneously. A scheduled job will always only be one connection. So you can safely say that you require only 150 simultaneous connections (with some wiggle room).

Now your server had only 4 CPU cores in it. So this one is easy 150 divided by 4 will give you 37.5. Round that up, because I suggest you have at least 150 and that you keep the same number on each WebI Report Server. So you can set each one’s “Maximum Simultaneous Connections” to 38 using the CMC. In XIR2 this might be pushing the limits, but in Business Objects XI 3.X this should be a comfortable setting.

The Little Known Business Objects Program Object

Most people are only familiar with the various reporting tools in Business Objects. Of course, these are the reasons organizations set up a Business Objects instance in the first place. However, Business Objects XI has managed to include some additional functionality that goes beyond reporting. For example, BOXI can be used to publish “agnostic” documents, such as PDF, Word, Excel, or PowerPoint. In particular, nevertheless, I would like to discuss the unique functionality of the Business Objects Program Object.

A Business Objects Program Object can be used to execute a batch file, a shell script, or a binary executable. It can also run Java, JavaScript, and VBScript. Therefore, they is virtually no limit to what you can do with a BO Program Object, except for your imagination and standards. Once the association is made to the program object standard objects rights can be applied and most importantly you can schedule the object (which is where the power is).

There are a few caveats that you need to be aware of. The program will require a “Logon As” account that must have interactive login rights to the server on which the Program Job Server runs. Also, I should mention that you need a Program Job Server configured for your environment. When you create a program you are not creating a pointer to your script, but rather you are uploading your script/program to the CMS InfoStore, just like you would an Excel file if you were to load it to your Business Objects repository. This means that if you later alter the script you used to source the program object then you must create a replacement object and re-source the script.

Another couple points, when I create program objects I like to log in to CMC while logged in to the server since often I have created a driving batch file that calls other batch files on the server. Oh, and that brings up the very last point. If you are running multiple Program Job Servers and your script calls other scripts on the server you will need to place identical copies of the called scripts in the exact same path on all servers running a Program Job Server (anything less would be easier, but it would cause failures).

The following are some simple steps to create a functioning program object. They are basic, but they should get you on your way to lots of programmatic trouble. The grouping of the step correspond to the screenshots above. Click on each thumbnail to view the full-size image; I recommend opening them in another browser window (hint, right click the thumbnail image and select this option).

Step and Screenshot #1
a) In CMC, go to Objects and click the “New Object” button
b) Click on “Program” in the left-hand menu
c) Enter in the location of the program object source, the path needs to be local to the client you are using. You will need to create a test script to run if you don’t already have something in mind.
d) Select the “Program Type”
e) Select the target folder and/or categories
f) Click “Submit”

Step and Screenshot #2
a) You may edit the program object name which gets generated from the source file you loaded. The name can be anything.
b) Add other properties if desired. A good description is always helpful to others and your future self.
c) If you made changes, click “Update”.

Step and Screenshot #3
a) Click on the “Process” tab and the “Logon” sub-tab.
b) Put the account information for an account with log on rights, any rights less than interactive logon will result in a failed execution.
c) Click “Update” when done

You are welcome to experiment with the other tabs and sub-tabs, but please share you findings if you do.