Posts tagged "Jboss"

Using JMX-Console for configuring and debugging LiveCycle applications

 JMX-Console ??

JMX(Java Managed Extensions) is a Java Technology which provides a way to manage running applications via various utilities and tools.
The running services are registered as mbeans and can be accessed/controlled remotely via JMX-Console.

LiveCycle currently supports 3 App Servers namely, JBoss, Weblogic and Webspehere.
While Weblogic and Websphere don’t provide a UI to connect to JMX managed bean instances,
Jboss provides a management console called JMX-Console to do the same.
In order to manage beans for Weblogic and Websphere,
Java provides a generic Utility called JConsole which can be connected to any running Application Server and relevant tasks can be performed.

Why JMX-Console

  • There are times when server gets stuck at some error, e.g. Server goes into hang state. In such cases, needs arises to get an insight into the running system so that the issue can be narrowed down.
  • At times one would like to get some information about the registered LiveCycle service, e.g. some metadata, some server information, some kernel information.
  • Sometimes one needs to change some settings in the server at run time.
  • One may need to redeploy a web application.
  • One might need to change the logging level for a particular service or decrease the verbosity of the logs.
  • Most of these changes mandate restarting of the server which can be very tardy with a LiveCycle bundled with too many components.
  • There are times when the host machine is not accessible directly to debug the issue.

Some benefactions of JMX-Console for debugging LiveCycle issues

I’ll be discussing about the following topics that can be helpful while debugging a LiveCycle application/service.
a. Login into JMX-Console
b. Redeploying a LiveCycle war
c. Configuring the log levels for a specific package
d. Generating Thread Dump
e. Stopping LiveCycle Server Instance
f. Starting/Stopping the CRX server
g. Command line JMX management

NOTE: the changes made on JMX-Console remain active till the server is running and vanish once the server is restarted.

Login into JMX-Console

Redeploying a LiveCycle web service

Whenever one needs to change some settings in a war, Jboss doesn’t allow to do so until the server is stopped.
Of course, one can always copy the war/ear outside and edit and then hotswap with the existing running war.
But the following seems like a safe and graceful way to modify a war without shutting down the whole running LiveCycle server.

  • Navigate to http://LCServer:LCPort/jmx-console
  • Under the section “Object Name Filter”, click on the link named jboss.web.deployment
  • To the right hand side under the section jboss.web.deployment, there will be a listing of context-roots to which various wars are associated.
  • Click on the context-root which that is associated with the concerned war.
  • This will open the JMX Mbean view of the associated war.
  • The mbean names are of format “war=/context_root_associated_with_war”.
  • Now one can simply start/stop/redeploy the war by clicking on the related “Invoke” button.

Configuring log levels for package(s)

The log levels for a package are changed in Jboss via log4jService located at, \\Jboss\server\server_instance\conf\jboss-log4j.xml.
Or by changing the located in a particular war.
JMX-Console provides a remote way of doing the same.

  • Navigate to http://LCServer:LCPort/jmx-console
  • Under the section “Object Name Filter”, click on the link named jboss.system
  • Then click on link service=Logging,type=Log4jService under the section “jboss.system” on the right hand side.
  • There are various settings one can alter for logging as per need.
    • Custom Logging File – As pointed out before, the log4J settings are controlled via the file named jboss-log4j.xml.
      One can even change the settings of the server to point to a custom made logging.xml file.
      Replace the value of the attribute named “ConfigurationURL” from “resource:jboss-log4j.xml” to the custom logging.xml file.
      The custom file however needs to be placed at the same location alongside the jboss-log4j.xml.
    • Get specific Log Level details – Have a look at the jboss-log4j.xml.
      It consists of tags named “categories” with different priority values.
      In Log4j world, the category is called “Logger” and the priority value is called the “Log Level”.
      If one wants to know the log level for a particular category specified in the jboss-log4j.xml,
      then enter the name of the Logger for the Operation named “getLoggerLevel” and click on the Invoke button.
    • Set/Edit new Logger and Log level – One can create new Categories/Logger on the fly and associate a particular package with a Log Level.
      Enter the name of the Logger and the Log Level for the Operation named “setLoggerLevel” and click Invoke.
      Till the server instance is up and running, the log level for the particular package will remain active.
    • Set/Edit multiple Loggers and Log level – One can also set multiple Loggers with a particular Log Level under the Operation named “setLoggerLevels”.
    • Reconfigure Logging – If the logging is not getting reflected for some unknown reason,
      then click on Invoke button for the Operation named “reconfigure”.
      This is analogous to editing of a jboss-log4j.xml where the saving the changes the logging is reconfigured.
    • Edit root threshold Log level – When no Categories are specified then the Log level of Root takes control.
      This is specified under as the value for the attribute named “DefaultJBossServerLogThreshold”.
      Change the value to a required level and click on “Apply Changes”.
      This will activate the Log Level for all those packages which don’t have a category explicitly specified.

Generating Thread Dump

A Java thread dump is a way of finding out what every JVM thread is doing at a particular point in time.
This is very helpful in cases where the server has become unresponsive or went into an unexplained hang state.
Thread Dump allows one to get an insight into the failing JVM process.
As an analysis of the dump, it can be known where exactly the threads are stuck.

  • In order to get the Thread Dump via JMX-Console,Navigate to http://LCServer:LCPort/jmx-console
  • Under the section “Object Name Filter”, click on the link named jboss.system
  • Then click on link type=ServerInfo under the section “jboss.system” on the right hand side.
  • Click on Invoke button of the “listThreadDump” operation to generate the ThreadDump.
  • The Thread Dump can then be saved to the file system for analysis.

Stopping LiveCycle Server Instance

The JBoss server instance can be stopped online via JMX-Console.
Especially useful when one doesn’t have direct access to the host machine.
i. Navigate to http://LCServer:LCPort/jmx-console
ii. Under the section “Object Name Filter”, click on the link named jboss.system
iii. Then click on the link type=Server under the section jboss.system on the right hand side.
iv. Click on Invoke button of the “shutdown” operation to shutdown the server instance.

Starting/Stopping the CRX server

Have a look at second section of the blog entry,

While JMX-Console provides UI way of debugging and changing the settings,
the same can be achieved through command line by another utility named twiddle that comes bundled with Jboss and is located in the bin folder of the same.
It can perform every task that a JMX-Console can do through UI.
I found a nice article depicting the usage of Twiddle with examples. Have a look at,

Note: The JMXConosle.war is not shipped with the turnkey installations previous to LC ES3.
In such cases, it needs to be downloaded and deployed to the deploy folder of the related Server_Instance.

Bookmark and Share

Restarting CRX Server without stopping a JBoss Server Instance

Since the release of ES3, the CRX Server comes integrated with the LiveCycle Document Services.(NOTE: SA Installer should be run on an existing LiveCycle Document Services Installation in order to get the benefits of CRX Server and Correspondence Management Solution)

The CRX wars namely, crx-explorer_crx.war and crx-launchpad.war are bundled inside the core EAR, i.e. adobe-livecycle-jboss.ear.

There are times when a restart of the CRX Server is required due to some changes in configuration/settings or issues related to bundle activation.
Since the wars are bundled inside the core ear, the whole of JBoss server instance needs to be restarted to get things done.
This ultimately consumes time and things on the doc-services side also get blocked due to some task pending at CRX end.

Let’s handle the two issues separately,

i. Issues related to Bundle Activation

There is an easy way out where one can just shutdown the CRX server and Restart it again without interfering with the LiveCycle Server Instance.

Go to, http://localhost:8080/lc/system/console/vmstat (Snapshot at the end of the post)
It has two buttons,
a. Restart – Clicking on this will restart all the bundles and the framework.
b. Stop – Clicking on this will stop all the bundles and the framework.
In order to start the framework and the bundles, Just refresh the browser or hit the above mentioned URL again.
The LiveCycle services running under the Jboss server instance won’t be affected at all.

The above settings solves the issue of Bundle Activation because it restarts CRX LaunchPad. But it doesn’t start the whole of CRX due to which changes in repository.xml or don’t get reflected.

ii. Changes in Configuration/Settings like repository.xml,, etc.

For such changes to get reflected, the best way is to re-deploy the CRX war.
JBoss provides a seamless way of doing that through JMX Console.
a. Go to http://localhost:8080/jmx-console. (For more information on JMX-Console handling I’ll write a separate blog post)
b . Once inside JMX-Console, search for jboss.web.deployment section.
c. Click on the link named war=/crx.
d. Now you can stop and start the war by clicking on the respective “Invoke” buttons to get the settings and configuration reflected. This won’t affect the other LiveCycle Document Services running in parallel.

Bookmark and Share

Customizing ports on Jboss for running multiple server instances

Many a times we need to run multiple JBoss server instances on the same sever, e.g. Vertical Cluster setups. At times when some other application has taken control of port 8080, then need arises to switch JBoss server instance to run on some other port.

Below mentioned are the ways one can follow to configure different ports on multiple JBoss server instances without going for changing 10s of ports which is firstly time consuming and also hard to maintain.

Jboss 4.x.x

JBoss by default uses 8080 for Http connection and 8443 for SSL connection.

So, in case one needs to switch port 8080 and it’s set of related ports of a server instance to something else, then Jboss provides another set of 3 pre-configured ports to choose from, i.e. 8180, 8280, 8380.

To change the default port settings,
a. Go to \\Livecycle_Installation\JBoss\server\Server_Instance_Name\conf\jboss-service.xml
b. Look for the following mbean instance:

<mbean code=”” name=”jboss.system:service=ServiceBindingManager”>
<attribute name=”ServerName”>ports-01</attribute>
<attribute name=”StoreURL”>${jboss.home.url}/docs/examples/binding-manager/sample-bindings.xml</attribute>
<attribute name=”StoreFactoryClassName”></attribute>

c. The above mentioned mbean code, is commented by default to allow users to use 8080.
Uncomment the above said mbean code and the server instance will be ready to run on ports-01, which is 8180.
This also take care of the other related ports and increment their value with 100, i.e. SSL port related to ports-01 will be 8543.

If one wants to use any other port. Just change the value for the attribute “ServerName” to ports-02(8280) or ports-03(8380). These are pre-configured ports by JBoss.
For any new custom port other than the ones talked above, one will have to add new configurations to the file, specified in the path given for the attribute “StoreURL”.

For more information please refer the following doc,

Jboss 5.x.x

JBoss 5 also comes with a set of 4 predefined ports, 8080, 8180, 8280 and 8380.
By default, any server instance is bind to run with 8080(for http) and 8443(for https).

But in order to run with the other ports mentioned above, one can start the server with the following command,

run -c server_instance_name -Djboss.service.binding.set=ports-01

The above command will start the JBoss server instance with port 8180.
The related port values will as well get incremented with 100.

If one wants to configure a custom port other than the ones mentioned, then one’ll have to edit the pre-configured bindings(or add new) in following file,

For more information please refer the following doc,

Bookmark and Share