1) Local Text Echo
When a user at a Citrix client device hits a character-keystroke, a lot has to happen before there is a corresponding character on the client device, by default. The keystroke gets put inside an ICA packet, and that inside a TCP/IP packet, which is sent over the wire to the Citrix server, where the keystroke is unpacked and used to update the session on the server. Then the updated bitmap-piece is placed inside an ICA packet – and this inside a TCP/IP packet, back over the wire to the client, where it finally appears on the screen, where it belongs, on the client device.
The problem is the user thinks something is wrong with the network, or the Citrix server, when they are able to notice the “latency” described. A simple fix that is available in every Citrix implementation, from standard to enterprise (or “Platinum”), is the “SpeedScreen Local Text Echo” feature.
SpeedScreen Local Text Echo makes the network seem faster, by going ahead and writing the character to the client device ahead of time, letting the whole ICA packet story happen in the background. When the actual bitmap-piece in an ICA packet from the server finally gets to the client, the user will have no idea, because they will have thought it had been there the whole time, ever since they fist typed it.
Turning this feature on is a few clicks at the server console. But before you go click these clicks, there’s a reason Citrix left this off by default.
There is a very similar feature, called “Mouse Click Feedback”, which is on by default. Citrix will place an hourglass on the client device immediately after the mouse is clicked, well before the actual hourglass arrives, to save the user from the mistake of clicking again – the number one action performed when a click does not lead to an hourglass immediately.
But the “Local Text Echo” feature is left off by default. It can be turned on per ICA client, per Citrix Presentation Server, per application on each server, or even per field in an application, and the server-specific settings can usually be successfully used after being replicated to the other servers in the farm, so in this way it can be set farm-wide.
The reason there is an option to set it on or off per “field within an application”, is because of the dilemma of password fields. If there is an application on the Citrix server, whose application logic has a password field rendered as asterisks, then the client device would be showing clear text passwords, temporarily, until the latent packet eventually arrived back from the server with asterisks.
So either “Local Text Echo” can just be turned on every server, or the password field-protected apps will have to be identified first, and only after careful testing and configuration, should you enable “Local Text Echo”, (using the SpeedScreen Latency Reduction Manager tool in the ICA toolbar on every Citrix server.)
2) Load evaluators
Any Citrix implementation running Presentation Server “Advanced”, or “Enterprise”, (or “Platinum” in 4.5), has the powerful Citrix Load Management feature built in. Most implementations assessed by Unitek however, are not utilizing Load Manager to its potential.
Citrix Load Management comes with two “Load Evaluators”, called “default” and “advanced”. Predictably, the “default” evaluator is tied to all the servers by default. The “advanced” evaluator is just sitting there available.
All a Citrix team has to do is configure the published app to run on more than one server, and the load management is in place. The problem is, the load is being gauged by one criteria only, in the “default” load evaluator. That one criteria is called “server user load”. So the servers are considered evenly balanced when there are an even number of sessions across them. Never mind that half the sessions are being run by power users, who know how to work the software, and the other half are newbies looking at the help- /splash- screens. Never mind that half the servers are brand-new IBM boxes with 4 GB of RAM, and half the fleet is from before your time and due to retire, probably capable of half the load of the new boxes.
So Citrix included another Load Evaluator, free with the farm, called “Advanced”. Five years ago, when the “Advanced” evaluator was invented, it was the best guess on what the bottlenecks would be on a typical Citrix server, and would gauge load based on these potential bottlenecks: Processor Time, Memory, and Page Swaps. Five years ago the best practice was to at least attach the advanced load evaluator to all the servers, so that the load was based on something a little more “real-world” than “server-user load”.
The only problem with putting a five-year old evaluator on today’s servers is that the evaluator cannot be modified, and the thresholds in the “Advanced” evaluator are dated. Apparently five years ago, one-hundred page swaps per second was a lot, and the server needed to be considered at full load, but today, machines have grown a whole lot stronger, and apparently a hundred page swaps per second is nothing these days. So now the advanced load evaluator is a liability, listing servers as inaccessible – at “full load” – just because they are running a simple hundred page swaps per second.
So what’s the solution? The ideal solution is to baseline the implementation, find out exactly what the bottle necks are for your environment, and implement a custom evaluator accordingly.
Still, even without the baseline, we can do a lot better than just the “Default” evaluator. The best practice is to create a custom evaluator, just like the “Advanced” evaluator, but without the “Page swaps” rule included, and to attach this custom evaluator to all the servers in the farm.
3) Roaming profiles
In the common situation where the Citrix users need to save files and/or application settings, and there is more than one Citrix server, the default situation is a potential mess. By default, users connecting to the Citrix servers from Active Directory create a local profile on any server they connect to. In a load-managed environment, this means multiple local profiles, all over the farm, and if the user is trying to store “My Documents” and “Desktop” items in the Citrix sessions, they are in reality being stored on local profiles all over the Citrix server farm.
Ideally, the users should be configured in Active Directory to use “Terminal Services Roaming Profiles”, and these profiles are stored on a file server, close to the Citrix servers. The changes are then saved up to the central location each time the user logs out of Citrix, and subsequently copied back down to different Citrix servers when the user logs in to those.
Though the idea of using roaming profiles in Active Directory has spread to many implementations by this point, there is still a large portion of implementations without the rest of the picture in place; Citrix advises having these cached roaming profiles deleted from the Citrix servers each time the user logs out, usually done by means of an AD GPO. Also, Citrix recommends setting the “Do not detect slow network connection” and “Wait for slow profile to load” GPO’s on the OU of the Citrix servers, when using roaming profiles.
A quick test at the Citrix server console is to go to “My Computer”, “properties”, “advanced”, “user profiles”. Any one in this list would be currently logged on to the server, and everyone, other than possibly the administrator, has a profile of type “roaming”. This is also a good place to do a spot check, and catch the occasional user account that fell through the cracks, and is connecting to Citrix servers without a roaming profile.
4) Data store backup and restore
The data store is the static configuration database of the entire Citrix implementation. Everything in the Presentation Server Console – the farm settings, the administrator configurations, the published apps, the Installation Manager packages, the Resource Manager configuration, the Isolation environments, the configured policies, the configured Load evaluators, server settings – all of this is stored in a single, central repository, called the “data store”.
The data store can be an Access “mdb” file on one of the Citrix servers, or is (more likely) a SQL database on a non-Citrix server somewhere in the data center. Either way, the data store is a 30-day fault-tolerant single-point-of-failure, and should be protected, backed up, and managed rigorously.
Though the SQL team regularly backs up the SQL servers, what if the Citrix Data store happens to become corrupt? How far back do you go? Have they got tapes that go that far back? How do you know the corruption isn’t still there?
Every Citrix implementation should have a separate flash-drive, with the data store – either in access, or SQL, (or DB2) format – and a documented and practiced plan in place for recovering from the catastrophic failure of a data store.
The SQL team may back up the SQL data store nightly, or, the Citrix servers may be getting backed up daily and with them the Access data store on the first server. But nightly is rarely necessary in data store backups. Again, the data store is everything in the Presentation Server Console, so only when significant changes are made to the data store – such as publishing an application, configuring a PSC policy, or adding a print driver – does the backup need to be updated.
With this backup in place, along with the apps and the data, the entire Citrix implementation can be recreated, if needed, for disaster recovery. This month’s online video presentation documents backing up, moving, and restoring an Access data store.
5) Print drivers
Possibly the number one threat to the stability of today’s Citrix implementations, is the presence of unauthorized print drivers on the Citrix servers. How did they get there? Maybe you have a policy that “auto loads” them from the windows CAB files any time a user connects with a printer that seems to need one, even though some of these drivers can cause your Citrix servers to become intermittently unstable! Maybe you have Citrix administrators who remember the way things were done in the old days, and throw any CD in the computer room that says “print driver” at all the Citrix servers, just to see if maybe that will solve the problem and keep the users from calling the helpdesk about printing and Citrix.
But these would be grave mistakes in the modern Citrix environment. Citrix recommends minimizing the number of print drivers on any Citrix server. Best practice involves researching each printer model / type, for what driver to use in a Terminal Services / Citrix environment, then to load that driver and replicate it via the IMA server-to-server protocol, then to “map” the server driver to whatever it shows up as on the client device.
Between this rigorous practice, and the phenomenon of the new “Citrix Universal Printer” that was released with Presentation Server 4 and the 9.x ICA clients, the situation has changed so much from the old days, that we now have quotes from Citrix that tell us it is reasonable to expect to see NO print drivers on any of the Citrix servers, instead the “Citrix Universal Printer” using EMF printing to redirect the spooling to the individual client devices, and providing bi-directional communication with network print devices so that the advanced printing features can be rendered available for the users. With this kind of functionality in a “Universal Printer”, users are no longer demanding native drivers in their ICA sessions, and the management of Citrix server farms is greatly simplified.
CM, Citrix Training Instructor
Unitek Citrix Training