Real-Time Messaging with LiveCycle Data Services ES

This article describes the variety of configurations and the considerations for how to use each mechanism. The following members of the LiveCycle Data Services development and quality engineering teams contributed to this content: Jeff Vroom, Seth Hodgson, and Kumaran Nallore.

LiveCycle Data Services ES supports a variety of deployment options that offer real-time messaging from Flex clients to Java 2 Enterprise Edition (J2EE) servers. Although efficient real-time messaging is difficult when client and application server are both behind firewalls, LiveCycle Data Services ES attempts to maximize the real time experience while keeping load on the server resources manageable.

The following issues prevent a simple socket from being used to create a real time connection between client and server in a typical deployment of a rich Internet application:

  • The client’s only access to the Internet is through a proxy server.
  • The application server on which LiveCycle Data Services ES is installed can only be accessed from behind the web tier in the IT infrastructure. LiveCycle Data Services ES offers a few approaches for creating efficient real time connections in these environments, but in the worse case, the server fall’s back to simple polling. The main disadvantages of polling are increased overhead on both client and server machines and reduced response times for pushed messages.

One design goal for LiveCycle Data Services ES is to create an environment where J2EE code can be run in a channel-independent manner. Your backend server code can either be used in a traditional J2EE context or with protocols that offer scalable real time connectivity currently not available in the J2EE platform. For example, LiveCycle Data Services ES provides the FlexSession, which mirrors the functionality available with the HttpSession in for both the Flash Player RTMP (real time messaging protocol) as well as the standard HTTP protocol that J2EE supports.

Clients can be configured using a list of channels that they can use to connect to the server. A client attempts to use the first channel in the list, and if that fails, it tries the next channels until it finds one it can connect to. You can use this to provide different protocol options so that a client can first try to connect using RTMP and if that fails, can fall back to HTTP. You also could use this mechanism to provide redundant server infrastructures to provide high reliability.

RTMP channels

The LiveCycle Data Services ES Real Time Messaging Protocol (RTMP) server is embedded in the J2EE application server process. RTMP is is a protocol that is primarily used to stream data, audio, and video over the Internet to Flash Player clienst. RTMP maintains a persistent connection with an endpoint and allows real-time communication. The protocol is a container for data packets that can be in Adobe Action Message Format (AMF) or raw audio or video data.

The server uses application-server-specific extensions when necessary, so that RTMP requests are handled with managed J2EE threads that support the use of the Java Transaction API (JTA) transactions. This lets RTMP requests use your existing J2EE service code.

The LiveCycle Data Services ES RTMP server is implemented using Java NIO facilities, which allows it to decouple threads from connections. This approach is very scalable when dealing with idle connections. A single RTMP server can support 10,000 or more simultaneous RTMP connections as long as the load generated by those clients does not exceed the server’s computational capacity.

The RTMP server supports a session that mirrors HTTP session functionality. It does not support failover the way many J2EE application servers do when an application server process dies. This is often not required in Flash applications because Flash Player itself can store state more conveniently to make seamless failover possible in many cases.

An RTMP connection be made in one of three ways depending upon the network setup of the clients:

  1. Direct socket connection.
  2. Tunnelled socket connection through a proxy server using the "connect" method.
  3. Active, adaptive polling using the browser’s HTTP implementation. This mechanism does place additional computational load on the RTMP server and so reduces the scalability of the deployment. A given server can handle roughly 500-1000 poll requests per second, so you should ensure the number of polling clients does not exceed the number that a given server can handle comfortably.

RTMP Direct socket connection

RTMP in its most direct form uses a simple TCP socket made from the client machine directly to the RTMP server.

RTMPT Tunnelling with Proxy Server Connect

This acts very much like a direct connection but uses the proxy server’s support for the "CONNECT" protocol if supported. If connect is not supported, the server falls back to using HTTP requests with polling.

RTMPT Tunnelling with HTTP requests

In this mode, the client uses an adaptive polling mechanism to implement bidirectional communication between the browser and the server. This mechanism is not configurable and is designed to use at most one connection between the client and server at a single time to ensure that Flash Player does not consume all of the connections the browser will allow to one server (Internet Explorer imposes a limit of two simultaneous connections to the server when the server provides HTTP 1.1 support).

The client issues an HTTP POST to the server with any messages that the client needs to send to the server. The server keeps the connection until it has a message to push to that client, or until its wait time expires. Once the wait interval expires, the response is written to the socket. The client immediately issues another POST if it has additional messages to send. If not, it waits for its poll time to expire and then polls again.

The server gives the client the next interval for the client to use as its poll time. The server doubles the time of the previous request if no messages have come in for that client since the last poll. The server keeps doubling the poll time until it hits a limit of four seconds. At that point, the client polls at an interval of four seconds until a message comes for that client at which point it reduces the interval to a very short timeout.

Similarly, the server doubles the wait time each time it receives a poll request that does not include any client to server messages. The server again reaches the limit of four seconds and stops increasing the wait time.

Connecting RTMP from the web tier to the application server tier

Currently all RTMP connections must be made from the browser, or the browser’s proxy server, directly to the RTMP server. Because the RTMP server is embedded in the application server tier with access to the database, in some server environments it is not easy to expose it directly to web clients. There are a few ways to address this problem in your configuration:

  1. You can put a load balancer which is exposed to web clients that can implement a TCP pass through mode to the LiveCycle Data Services ES server.
  2. You can put a reverse proxy server on the web tier that supports the HTTP CONNECT protocol so it can proxy connections from the client to the RTMP server. This provides real time connection support by passing the web tier. The proxy server configuration can provide control over which servers and ports can use this protocol so it is more secure than opening up the RTMP server directly to clients.
  3. You can put a reverse proxy server on the web tier which proxies simple HTTP requests to the RTMP server. This approach only works in RTMPT polling mode.

HTTP and AMF channels

The HTTP channel uses XML over an HTTP connection. The AMF channel uses the AMF format over the HTTP protocol. The HTTP and AMF channels support simple polling mechanisms that can be used by clients to request messages from the server. You can use these channels to get pushed messages to the client when the other more efficient and realtime mechanisms are not suitable. The polling interval is configurable or polling can be controlled more flexibly from ActionScript code on the client. This mechanism uses the application server’s normal HTTP request processing logic and works with typical J2EE deployment architectures. When you use the HTTP channel or AMF channel, the FlexSession maps directly to the underlying application server’s HTTP session so the application server’s session failover can be used to provide fault tolerant session storage. You typically should ensure your application server is configured to use sticky sessions so one clients requests go to the same application server process instance.

In the default configuration for a polling HTTP or AMF channel, the channel does not wait for messages on the server. When the poll request is received, it checks to see if there are any messages queued up for the polling client and if so, those messages are delivered in the response to the HTTP request.

In LiveCycle Data Service ES, there is a new configuration option that provides the ability to support more real time behavior using these channels. You configure a poll-wait-interval on the channel which indicates the amount of time the server thread should wait for pushed messages. If that interval is set to -1, it means to wait indefinitely – as long as the client can maintain that connection to the server. As soon as any messages are received for that client, we push them in the response to the request. The poll-interval on the client then is used to determine when next to poll the server. If you set poll-interval to 0 and poll-wait-interval to -1, you get real time behavior from this channel.

There are two additional caveats for using the poll-wait-interval with LiveCycle Data Services ES. One is utilization of available application server threads. Since this channel ties up one application server’s request handling thread for each waiting client connection, this mechanism is not nearly as scalable as RTMP. Each application server thread requires a certain amount of memory and may add some overhead to thread scheduling. Modern JVMs can typically support a couple of hundred threads comfortably if given enough heap space. You should check the maximum thread stack size (often 1 or 2M per thread) and make sure you have enough memory and heap space for the number of application server threads you configure.

To ensure Flex clients using channels with poll-wait-interval do not lock up your application server, LiveCycle Data Services requires that you set an additional configuration attribute specifying the maximum number of waiting connections that LiveCycle Data Services ES should manage. This is configured through the max-waiting-poll-requests configuration setting. This number must be set to a number smaller than the number of HTTP request threads your app server is configured to use. For example, you might configure the application server to have at most 200 threads and allow at most 170 waiting poll requests. This would ensure you have at least 30 application server threads to use for handling other HTTP requests. Your free application server threads should be large enough to maximize parallel opportunities for computation. Applications which are I/O heavy may need a large number of threads to ensure all I/O channels are utilized completely. Examples where you need application server threads include: # of threads simulataneously writing responses to clients behind slow network connections, # of threads used to execute database queries or updates, and # of threads needed to perform computation on behalf of user requests.

The other consideration for using poll-wait-interval is that LiveCycle Data Services ES must avoid monopolizing the available connections the browser will allocate for communicating with a server. The HTTP 1.1 spec mandates that browsers allocate at most 2 connections to the same server when the server supports HTTP 1.1. To avoid using more that one connection from a single browser, LiveCycle Data Services ES allows only one waiting thread for a given app server session at a time. If a new polling request comes in over the same HTTP session as an existing waiting request, the waiting request returns and the new request starts waiting in its place.

If you have an application that has more than one player instance on the same page, beware that only one of those instances can have a real time HTTP channel connection open to the server at the same time.

RTMP deployment scenarios

The following table shows deployment scenarios and channel configuration when using RTMP channels:

RTMP protocol allowed?

Port open?

Channel configuration

Yes

Yes

Default configuration

Yes

No

1) Endpoint URI port should be set to an open port.
2) The <bind-port> property in the RTMP channel configuration should be set. RTMP server listens on this port.
3) RTMP traffic from the endpoing URI’s port should be redirected to the port set in the <bind-port> property.

No

Yes

1) Default configuration. The RTMP requests fall back to RTMPT.
2) In the case of non-secure channels only, the protocol in endpoint URI could be hardcoded to use RTMPT directly.

No

No

1) The endpoint URI port should be set to an open port
2) The <bind-port> property should be set. The RTMP server listens on this port.
3) RTMPT traffic from the endpoint URI port should be redirected to the port set in <bind-port> property.
4)  In the case of non-secure channels only, the protocol in the endpoint URI could be hard coded to use RTMPT directly.

When a client-side RTMPChannel instance internally falls back to tunneling, it sends its HTTP requests to the same domain/port that is used for regular RTMP connections and these requests must be sent over a standard HTTP port so that they make it through external firewalls that some clients could be behind. For RTMP to work, you must bind your server-side RTMP endpoint for both regular RTMP connections and tunneled connections to port 80. When the client channel falls back to tunneling, it always goes to the domain/port specified in the channel’s endpoint URI.

If you have a switch or load balancer that supports virtual IPs, you can use a deployment as the following table shows using a virtual IP (and no additional NIC on the LiveCycle Data Services ES server) where client requests first hit a load balancer that routes them back to a backing LiveCycle Data Services ES server:

Public Internet    Load balancer LiveCycle Data Services ES
Browser client makes Http request my.domain.com:80 Servlet container bound to port 80
Browser client uses RTMP/T  rtmp.domain.com:80 (virtual IP address) RTMPEndpoint bound to any port; for example, 2037

The virtual IP/port in this example (rtmp.domain.com:80) is configured to route back to port 2037 or whatever port the RTMPEndpoint is configured to use)on the backing LiveCycle Data Services ES server (. In this approach, the RTMPEndpoint does not need to bind to a standard port because the load balancer publicly exposes the endpoint via a separate virtual IP address on a standard port.

If you do not have a switch or load balancer that supports virtual IPs, you need a second NIC to allow the RTMPEndpoint to bind port 80 on the same server as the servlet container that binds to port 80 on the primary NIC, as the following table shows:

Public Internet    LiveCycle Data Services ES NICs LiveCycle Data Services ES
Browser client makes Http request my.domain.com:80 Servlet container bound to port 80 on primary NIC using primary domain/IP address
Browser client uses RTMP/T  rtmp.domain.com:80 RTMPEndpoint bound to port 80 on a second NIC using a separate domain/IP address

In this approach the physical LiveCycle Data Services ES server has two NICs, each with its own static IP address. The servlet container binds port 80 on the primary NIC and the RTMPEndpoint binds port 80 on the secondary NIC. This allows traffic to and from the server to happen over a standard port (80) regardless of the protocol (HTTP or RTMP). The key take away is that for the RTMPT fallback to work smoothly, the RTMPEndpoint must be exposed to Flex clients over port 80 which is the standard HTTP port that is generally allows traffic (unlike nonstandard RTMP ports like 2037, which are generally closed by IT departments).

To bind the RTMPEndpoint to port 80, one of the approaches above is required. For secure RTMPS (direct or tunneled) connections, the information above is the same but you would use port 443. If an application requires both secure and non-secure RTMP connections with tunneling fallback you do no’t need three NICs; you need only two. The first NIC services regular HTTP/HTTPS traffic on the primary IP address at ports 80 and 443, and the second NIC services RTMP and RTMPS connections on the secondary IP at ports 80 and 443. If a load balancer is in use, it iss the same as the first option above where LiveCycle Data Services ES has a single NIC and the different IP/port combinations are defined at the load balancer.