Avoiding LiveCycle Kerberos based SSO problems for Active Directory users with large group memberships

Scenario:

Many LiveCycle customers implement kerberos based SSO in their deployments.

We recently came across an interesting customer situation where SSO did not work for some domain users.
To recap working scenario – If LiveCycle SSO is configured correctly, when domain user opens workspace URL from  domain machine, workspace should directly open without asking for user credentials again.

In this case, it wasn’t working for some domain users; but worked for others.

Investigating further we found that non-working users were part of 100+ AD groups resulting in a large kerberos token size.

Customer was using JBoss 4.2.1 with LiveCycle ES2.5.

HTTP traces for working versus non-working scenario revealed that authenticate header was getting truncated in case of non-working scenario. Example below:

Working Http header
header=WWW-Authenticate=Negotiate
oYGqMIGnoAMKAQChCwYJKoZIhvcSAQIComoEaGBmBgkqhkiG9xIBAgICAG9XMFWgAwIBBaEDAgEPokkwR6ADAgEXokAEPtWpZjlA1z6oofQdcYB583
MLxlmoHwCFOtQ3VaGkG38A6zyKf9JogE/KrLPTScZnVCjbEMIeh7cCgUaJTNjqoycEJWAjBgkqhkiG9xIBAgIBAREA/////+sKU3YfjaMD +LWYtwumSTY=

Non Working Http header
header=WWW-Authenticate=Negotiate

Analysis and Solution:

LiveCycle does not have any hard limit on kerberos token size per se. Also Livecycle does not use group information associated with kerberos token for authentication.

However, we found that application servers (in this case JBoss) typically set http header limit due to which packet containing large tokens was getting dropped.

By default Jboss sets maxHttpHeaderSize to 8192 (8KB).

There are couple of options to solve this.   In this case customer chose option 1) below; however I wanted to lay out both options for completeness.

Option 1)

Change Jboss config file to increase http header limit.
Modify maxHttpHeaderSize to say =”16384″ in location below and restart Jboss.

${jboss.home}/server/${jboss.server}/deploy/jboss-web.deployer/server.xml

<Connector port=”8080″ address=”${jboss.bind.address}”
maxThreads=”250″ maxHttpHeaderSize=”16384″

Option 2)

Microsoft AD allows a flag to be set on user object in AD, so that extra PAC information (containing  groups) won’t be sent across by AD. One could set this flag for “service” user id in AD which is used for LiveCycle SSO.

This flag is called “NO_AUTH_REQUIRED” and was introduced in a hot fix on Windows 2003 AD.

Here is the link to download the hot fix, and also it explains where the flag should be set.

http://support.microsoft.com/kb/832572

VN:F [1.9.22_1171]
Was this helpful? Please rate the content.
Rating: 9.0/10 (3 votes cast)
Avoiding LiveCycle Kerberos based SSO problems for Active Directory users with large group memberships, 9.0 out of 10 based on 3 ratings

About Santosh Tatke

Santosh Tatke is a Sr. Support Architect at Adobe Systems. He specializes in ADEP (formerly LiveCycle) technology stack and works closely with customers, solution consultants and partners to troubleshoot, analyze and resolve technical challenges during POC, implementation, deployment or upgrade. He is a Certified LiveCycle ES2.5 Process Management Expert.
This entry was posted in Adobe LiveCycle ES2 (9.0.x), Adobe LiveCycle ES3, General Interest and tagged , , , , , . Bookmark the permalink.

2 Responses to Avoiding LiveCycle Kerberos based SSO problems for Active Directory users with large group memberships

  1. Dan Blackman says:

    Nice write up Santosh!

  2. anu jacob says:

    hi Mr Santhosh,
    We have a requirement here in our office. we have a project where we need to implement SSO with adobe livecycle ES2 with the oracle portal . Could you please help me in this matter. Would really appreciate if you could send some document or reference material.

    thanks

    Anu