Tunneling with RTMP encapsulated in HTTP (RTMPT) should be avoided as it causes latency

Tunneling with RTMP encapsulated in HTTP or RTMPT should be avoided as it causes latency that can have a negative impact on user experience in a Connect meeting. In rare circumstances,the latency commensurate with tunneling RTMP encapsulated in HTTP, can become so acute that it renders Connect unusable for affected clients. The performance hit commensurate with tunneling is one of the primary reasons we continue to deploy Connect Edge servers as they often can replace third-party proxy servers that are often the cause of tunneling latency..

While the amount of acceptable latency depends on what one is doing in the room; RTMPT tunneling affects most activities. Some activities, such as screen-sharing are more bandwidth intensive than other activities such as presenting an uploaded PowerPoint from within a meeting room; The high latency commensurate with RTMPT tunneling would affect the former more than the latter. VoIP is often the first thing to make the effects of high latency felt. Here is some feedback from a test I did while on site with a client dealing with tunneling because of their refusal to pipe RTMP around a third-party proxy:.

External user tunneling during test:
Spike at 3.10/3.02 sec
Latency 403/405 ms up to 3.53/3.52 sec up .064 down 118
When latency peaks to 2.6/2.4 sec I get a mild interruption to the audio V
Video pauses momentarily when the latency spikes

Internal user with direct connection:
2 msec / 1 msec Up 0.08 kbits down 127 kbits
No pauses, delays or spikes

Tunneling should only be considered as a fallback mechanism or safety net to allow connections when RTMP is blocked due to something unplanned or for a few remote clients who must negotiate specific network obstacles. When RTMPT is the default by network design, you will need to limit your activities within Connect to those feature that use the least bandwidth.

The picture below shows a direct connection over RTMPS on 443 is being blocked somewhere on the client’s network and the fallback mechanism built into Connect of tunneling RTMP encapsulated in HTTPS is the fallback path. This is usually caused by either proxy servers or firewalls or both – any application-aware appliance on one’s network that sees the RTMPS traffic on 443 and recognizes that it is not HTTPS is a potential obstacle; RTMPS traffic is on port 443 and while it is disguised as HTTPS, it still may be blocked. The result is tunneling, indicated by the “T” in the output:

.tunneljpgOctagon

Compare with a connection without tunneling:

no-tunnel

The recommended steps for anyone experiencing persistent tunneling, is for their network engineers to trust the source IP addresses of the Adobe hosted, ACMS, managed ISP or on-premise Connect/FMS server VIPs in order to prevent the blocking of RTMPS traffic. RTMPS is not supported by any third-party proxy server. Static bypass works well to solve this issue. The problem stems from network policies that require all traffic to go through a proxy. The result is tunneling with commensurate high latency and drops. RTMPS must be allowed to stream around a proxy to avoid the overhead and latency of tunneling encapsulated within HTTPS. Attempts to cache the stream add no value.

Other options will depend on the capabilities of the third-party proxy servers in the affected client infrastructure. Blue Coat ProxySG is one of the popular proxy server options in our niche. In cases of latency invoked by tunneling RTMP encapsulated in HTTP on a network that employs Blue Coat ProxySG servers, sniff tests done by support representatives have indicated that when an affected client attempts to connect to an Adobe Connect meeting, those clients would establish both explicit HTTP connections based on PAC file settings in the system registry to the Blue Coat ProxySG pool through a hardware-based load balancing device (HLD) and transparent HTTP and SSL connections through Blue Coat ProxySG via WCCP GRE redirect to several Adobe Connect servers. The problem manifests with RTMPS when the clients attempt to establish an SSL connection directly to the destination host without going through PAC file proxy settings. Since a Blue Coat ProxySG is commonly configured to perform an SSL intercept on both explicit and transparent HTTPS traffic, upon examining the content after decrypting the SSL payload from the clients, the Blue Coat ProxySG will return an exception and close the connection because the request doesn’t contain an HTTP component and cannot be parsed for policy evaluation. As a workaround, other than using static bypass, it is possible to create a proxy service with the destination set to the Adobe Connect server IP range on port 443 and to set the proxy setting to TCP-Tunnel with Early Intercept enabled. This will allows Blue Coat ProxySG to intercept and tunnel the traffic without considering whether it is RTMPS or HTTPS.

Watch for a more comprehensive article on this topic forthcoming.

Comments are closed.