Application Hangs when Scanning in Citrix XenApp

Another interesting issue today with an application that runs on Citrix XenApp.

Environment
Customer has a Citrix XenApp 5 environment running on Windows Server 2003. Clients are all Windows XP and run the Citrix Online Plugin 12.3 full screen.

RES Workspace Extender is used to integrate locally installed application into the XenApp Session. Users have no access to the local desktop.

Symptoms
This particular application scans invoices using a USB scanner attached to the client and runs them trough a workflow.
Whenever the Start scan button was pressed the application froze.

SNAGHTML48ec098

 

Troubleshooting
Since the application was no longer responding I could only kill the process. However when I did this, the whole session freezed.

The only thing I could do was reset the whole session via the management console.

In order to fix this problem we need to understand how twain redirection is implemented in XenApp.

Twain Redirection
When an application running in a XenApp session calls into the Twain API’s in order to obtain an image from a scanner the call is intercepted by a hook dll called twnhook.dll.

So my first step was to verify that twnhook.dll was correctly configured in the registry:

image

Then I verified that the hook dll was loaded into the target process with Process Explorer (enable the Lower Pane View and set it to DLL’s):

image

On the client side a process called CtxTwnPA.exe is launched, this executable calls the TWAIN driver and communicates with the XenApp server through a Virtual Channel.

I verified CtxTwnPA.exe was being launched and that this process had loaded the twain dll:

SNAGHTML4f2aa1d

I right-clicked the CtxTwnPA.exe process in Process Explorer and selected Window | Bring To Front and that showed this Window:

image

I am not sure why this window is appearing since there is only one scanner to select but clearly the application was not hanging but was waiting for this dialog to be closed!

In theory this shouldn’t be a problem since the RES Workspace Extender should display this window in the XenApp session but it doesn’t. This happens because the dialog window doesn’t have an associated icon in the taskbar.

This is by design when using the Workspace Extender (RES Virtual Desktop Extender (VDX) doesn’t have this limitation).

I figured that if I could force this dialog to have a taskbar icon it would solve my problem.

Solution
I am not going to explain in detail here how resources in c/c++ applications work, but in a nutshell a dialog is composed as a resource script that can be modified with tools such as Resource Hacker.

I loaded twain_32.dll into Resource Hacker and found the dialog:

SNAGHTML4f66669

In order for an application to have an icon on the taskbar we must make sure that the Window Style contains the WS_POPUP attribute and that the Extended Window Style contains the WS_EX_APPWINDOW attribute.

So I modified the resource script from:

101 DIALOG 64, 60, 204, 86
STYLE DS_MODALFRAME | WS_CAPTION | WS_SYSMENU
CAPTION "Bron selecteren"
LANGUAGE LANG_DUTCH, 0x1
FONT 8, "MS Shell Dlg"
{
   CONTROL "Selecteren", 1, BUTTON, BS_PUSHBUTTON | WS_CHILD | WS_VISIBLE | WS_GROUP | WS_TABSTOP, 151, 46, 50, 14
   CONTROL "Annuleren", 2, BUTTON, BS_PUSHBUTTON | WS_CHILD | WS_VISIBLE | WS_TABSTOP, 151, 65, 50, 14
   CONTROL "Bronnen:", -1, STATIC, SS_LEFT | WS_CHILD | WS_VISIBLE, 8, 6, 85, 8
   CONTROL "", 10, LISTBOX, LBS_STANDARD | WS_CHILD | WS_VISIBLE | WS_GROUP | WS_TABSTOP, 8, 15, 140, 65
}

To

 

101 DIALOG 64, 60, 204, 86
STYLE DS_MODALFRAME | WS_POPUP | WS_CAPTION | WS_SYSMENU
EXSTYLE WS_EX_TOPMOST | WS_EX_APPWINDOW
CAPTION "Bron selecteren"
LANGUAGE LANG_DUTCH, 0x1
FONT 8, "MS Shell Dlg"
{
   CONTROL "Selecteren", 1, BUTTON, BS_PUSHBUTTON | WS_CHILD | WS_VISIBLE | WS_GROUP | WS_TABSTOP, 151, 46, 50, 14
   CONTROL "Annuleren", 2, BUTTON, BS_PUSHBUTTON | WS_CHILD | WS_VISIBLE | WS_TABSTOP, 151, 65, 50, 14
   CONTROL "Bronnen:", -1, STATIC, SS_LEFT | WS_CHILD | WS_VISIBLE, 8, 6, 85, 8
   CONTROL "", 10, LISTBOX, LBS_STANDARD | WS_CHILD | WS_VISIBLE | WS_GROUP | WS_TABSTOP, 8, 15, 140, 65
}

Then I recompiled the script with the Compile Script button and saved the Executable.

Now the Select Source Dialog has a Taskbar Icon and is displayed correctly in the session by the RES Workspace Extender:

image

When replacing twain_32.dll on Windows XP please note that the file is protected by Windows File Protection. You can use my WfpReplace tool to easily overwrite it!

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *