Dealing with Out Of Memory problems on LiveCycle Server

Scenario:

We recently worked with a customer who saw Native Heap Exhaustion on their Weblogic server. Weblogic9 running on Solaris with LiveCycle 821.x.

While majority of customers have upgraded to LiveCycle ES2.x or LiveCycle ES3 which require 64bit OS + JVM combinations, some customers are still on LiveCycle 821.x with 32bit OS/JVM combinations.

Out Of Memory exceptions in JVM logs could indicate either JVM heap exhaustion or Native Heap exhaustion. Closer look to the stacktrace could help determine the root cause.

On 32 bit systems, since max memory space that can be accessed by a process is 4GB; choice of max JVM heap setting could be critical.

If JVM heap is too high, it leaves less room for native heap, and you could see Out Of Memory exceptions in JVM logs.

In this particular scenario, exceptions we saw in WebLogic server logs were:

—–
<<anonymous>> <BEA1-53BF593214445136C2FE> <> <1318337670542> <000000> <stalling action-instance: 742271 with message: /u02/app/adobe/bea/jdk150_10/jre/lib/sparc/libawt.so: Can’t load Sparc 32-bit .so on a Sparc 32-bit platform: java.lang.UnsatisfiedLinkError: /u02/app/adobe/bea/jdk150_10/jre/lib/sparc/libawt.so: Can’t load Sparc 32-bit .so on a Sparc 32-bit platform
                at java.lang.ClassLoader$NativeLibrary.load(Native Method)
                at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1751)
                at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1668)
                at java.lang.Runtime.loadLibrary0(Runtime.java:822)
—–

Closer look at the stacktrace indicates that JVM was not able to load the native .SO file on Solaris due to native heap exhaustion.

Solution:

Their JVM was set to use 3GB of Xmx (max heap), leaving very limited room for the native heap.

In this case we resolved the problem by reducing the Max JVM heap to 2GB.

It might sound counter intuitive, but increasing JVM heap to deal with Out Of Memory scenarios might not be the right thing to do all the time. It depends…

VN:F [1.9.22_1171]
Was this helpful? Please rate the content.
Rating: 0.0/10 (0 votes cast)

About Santosh Tatke

Santosh Tatke is a Sr. Support Architect at Adobe Systems. He specializes in ADEP (formerly LiveCycle) technology stack and works closely with customers, solution consultants and partners to troubleshoot, analyze and resolve technical challenges during POC, implementation, deployment or upgrade. He is a Certified LiveCycle ES2.5 Process Management Expert.
This entry was posted in Adobe LiveCycle ES, General Interest and tagged , , , , . Bookmark the permalink.

One Response to Dealing with Out Of Memory problems on LiveCycle Server

  1. Ashok says:

    Hi,
    I work for Warner Bros, who has purchased Livecycle ES 2 SP 1 about a year and a half ago and is installed on a Websphere 5.1 SERVER. The client expects that the Production server not be restarted, but every 3 or 4 months once we are not able to submit the forms, in other words the Forms submitted does not create a process instance. we belive this is a issue with heap and every time we encounter this issue we restart the server to resolve this issue.