Behind the Curtain: Making Multiple Connect Meetings or Seminars Appear as One

On those occasions when a Meeting invitation may attract more participants than expected or planned for at the last minute so that you are unable to increase Seminar capacity in a timely manner, a skilled host can use two or more Connect Meeting rooms and project them to participants as though it were one room as an emergency workaround. Here is a basic outline of how to split a large meeting onto multiple servers. It is prudent to not just have more than one Meeting in these cases, but also to make sure each Meeting is hosted on a separate server in a cluster to add robustness to the meeting. Load-balancing is a wonderful thing and you should always use it to its fullest.

Assume an example of a three-server cluster/pool of Connect servers and that you want to split a Connect Meeting onto all three servers; a simple 3-server cluster is depicted here to use as an example:

C9SSLCluster3Simple

For a working example, let’s place a Connect Meeting room hosted on each server; to do this you will need three separate URLs: One URL for each 1/3rd  of your attendees. Getting the attendees distributed among the three rooms can be tricky. One effective technique is to either send out three different invitations, with each targeting 1/3rd of your audience and each offering a different URL, or just point everyone to a page with  all three URLs and request/instruct the participants to alphabetically arrange themselves in subsets of users by URL selection. That way it is not random; I have seen this technique work fine; here are sample meeting URLs based on our picture above:

http://connect.domain.com/splitmeeting1
http://connect.domain.com/splitmeeting2
http://connect.domain.com/splitmeeting3

To make certain the each meeting is hosted on a separate server (rather than all three on one as load-balancing could easily prescribe), it will require some effort to keep entering and leaving the room until your meeting lands on the server you want. Using multiple browsers may be helpful as well. Working on this well in advance of the meeting is prudent as there is a session timeout factor to consider. The load balancing algorithm will eventually get the sessions distributed but it may take some effort.

The way to tell which server you are on is simple: In any meeting room click Help and while holding down the shift key click About Adobe Connect. This will pop up an RTMP string that will identify the server that Meeting is hosted on and also which server a client is coming through as each client can be using multiple servers (just to add not only to the complexity, but also the overall robustness).

Here is what the RTMP strings might look like for each of the three servers in our simple example above ( I am inserting some URL parameters from a hosted meeting as I write this in order to create our hypothetical example RTMP strings – rtmp://arfms3.adobeconnect.com:1935/?rtmp://pcparapp07:8506/meetingas3app/89676385/630888204/)

rtmps:// connectmtg01.domain.com/?rtmp://connapp01:8506/meetingas1app/847483075/1086833045/
rtmps:// connectmtg02.domain.com/?rtmp://connapp02:8506/meetingas2app/847483076/1086833046/
rtmps:// connectmtg03.domain.com/?rtmp://connapp03:8506/meetingas3app/847483077/1086833047/

The first name in the string (connectmtg0#) is the built-in Connect Edge server and the second name (connapp0#)  is the Connect origin server  hosting the meeting (each Connect servers runs both AMS/FMS and Tomcat together). The second name is the important one for our technique of splitting the attendees onto separate meeting servers.

In the hypothetical RTMP string samples above, I have made these artificially neat and tidy, the truth is that the first part of the string can be any of the three for any meeting participant regardless of the application server hosting the meeting. For example, you could come in to connapp01 through connectmtg03 – any combination is possible. Load balancing is done at more than one level as Connect leverages both a hardware-based load-balancing device and also its own internal clustering capabilities; combinations for various clients (including the hosts and presenters) in our example cluster depicted  above might include:

rtmps:// connectmtg01.domain.com/?rtmp://connapp02:8506/meetingas2app/847483076/1086833046/
rtmps:// connectmtg02.domain.com/?rtmp://connapp02:8506/meetingas2app/847483076/1086833046/
rtmps:// connectmtg01.domain.com/?rtmp://connapp03:8506/meetingas3app/847483077/1086833047/
rtmps:// connectmtg03.domain.com/?rtmp://connapp03:8506/meetingas3app/847483077/1086833047/
rtmps:// connectmtg02.domain.com/?rtmp://connapp01:8506/meetingas1app/847483075/1086833045/
rtmps:// connectmtg03.domain.com/?rtmp://connapp01:8506/meetingas1app/847483075/1086833045/

The key to remember is that the second name is the one that matters; a distribution of participants approximating 1/3rd on each server is the goal targeting: connapp01, connapp02 and connapp03. After this is set-up, the pre-meeting preparation part is complete (this should be done at least one hour prior to the meeting).

Next comes the creative hosting venture during the split meeting: As the host, you will need all three meetings open in front of you to manage them as one. From the perspective of the participants, there is only one meeting (ignore the host behind the curtain). Be sure to hide the Attendee List Pod in the Presenter-only area as it will only present those participants in that specific Connect Meeting thereby allowing a peek behind the curtain or misrepresenting the size of the entire three meeting combination.

And here is where the techniques are very much up to you:

  • Splitting video among the three rooms is possible using a third-party option, one we have used successfully is: Splitcam.com.
  • For audio, if using integrated audio, be sure to use the same integrated telephony number for all three rooms.
  • If using VoIP, then allow one speaker only at a time to send audio via VoIP.

Some ways in which you can limit the amount of data being processed in your room and to improve the overall performance of these sessions are:

  • Optimize room bandwidth. In a Connect Meeting, at the top of the screen click on MEETING > Preferences. Under the preferences menu you are able to adjust screen sharing, video and VoIP quality setting separately.
  • Turn off cameras whenever they are not in use.
  • When in use, multiple cameras should probably be set to SLOW images (depending on how many and other variables).
  • Turn off VoIP if not talking.
  • Participants should directly connect to the fastest internet connection available and be on a dedicated DSL connection, at a minimum.
  • No clients or hosts on wireless – allow no exceptions.
  • Shut down Email, instant messaging, and any programs NOT being used for the presentation.
  • Shut down any VPNs as a VPN will potentially destroy the possibility for success.

When large Connect Meetings or Seminars become commonplace in your enterprise, this cumbersome workaround quickly becomes impractical and you should increase your Seminar or Webinar licensed capacity as needed to avoid this complexity and manual work. With that said however, this technique will work in a bind and will provide a robust Connect Meeting experience for a very large audience even if it challenges a seasoned Connect Meeting host.

Changing the License Serial Key in Connect

This article applies to on-premise and managed ISP Connect users. It does not apply to multi-tenancy hosted or ACMS.

On rare occasions it may be necessary to change the serial key in Adobe Connect. Here are the steps:

  1. Navigate to: \Connect_installation_directory\appserv\conf\config.ini and change the value of  SERIAL_KEY=  to reflect the new serial number
  2. In \Connect_installation_directory\custom.ini,  if there’s a serial key value listed (SERIAL_KEY=), replace it there as well.
  3. Using MSSQL Studio Express (or your choice of SQL editing options), view the serial key currently being used by Connect by running this command: SELECT * from pps_accounts WHERE name=’Enterprise Account’
  4. To get Connect to accept the new license you must change the serial key that is currently in the database by running this SQL command: UPDATE pps_accounts SET serial_key = ‘NEW_SERIAL_NUMBER’ WHERE serial_key = ‘OLD_SERIAL_NUMBER’
  5. Restart the services: Application Server (Connect) and the Meeting Server (AMS or FMS depending on the version of Connect) services.fw
  6. Open the Administration Console (port 8510 locally on any Connect server)

connconfig.fw

7. Go to License Settings and upload the new license file.

connconfiglic.fw

8. Restart the AppServer (Connect) and the Meeting server (AMS or FMS depending on version) again and the  new license file will be applied

services.fw

Troubleshooting: If there are any problems, do the following to troubleshoot:

  • Shut down the Connect and AMS or FMS Services
  • Open and verify \Connect_installation_directory\appserv\conf\config.ini and update the entry for SERIAL_KEY
  • Open and verify  \Connect_installation_directory\custom.ini and update  the entry for SERIAL_KEY
  • Open SQL Server and choose the Connect database and run the following script (replacing the text as appropriate):

“Input New Serial Key Here” with the New Serial Key but leaving the quotes.
DECLARE @NEW_SERIAL VARCHAR(32)
SET @NEW_SERIAL=’Input New Serial Key Here’

UPDATE PPS_CONFIG
SET VALUE = @NEW_SERIAL
WHERE SECTION=’cps’ AND NAME=’serial_key’

UPDATE PPS_ACCOUNTS
SET SERIAL_KEY = @NEW_SERIAL
WHERE ACCOUNT_ID=7

UPDATE PPS_ENUM_DATA_HOSTS
SET LICENSE = @NEW_SERIAL
WHERE HOST_ID > 0

db.fw_

  • Start the Connect and FMS services

Problems will ensue when the license is reducing the allowed usage of Connect (if you are downsizing) and you leave an overage in place. For example, if you have 100 meeting hosts assigned, and you are changing to a license that only allows 50 named meeting hosts then when you  apply the license you will get an error unless you have reduced the number to accommodate the new licensed restriction.

How to Enable Users to view the Seminar Calendar

Some Adobe Connect customers have a very large user base that requests daily access to use Seminar Rooms by checking the seminar calendar for availability. Prior to 9.5, users could login and see the Seminar Calendar even if they weren’t Seminar Hosts and then they could request permission to use the Connect Seminar room from a Seminar Hosts or Administrator of the Connect account who would then assign them the necessary resources.

Many customers with a large user base may not want to enable all of their potential Seminar users as Seminar Hosts without any screening or prioritization because they may run the risk of multiple Seminar Hosts competing for resources that may be limited.

Solution: If you turn off all permissions for the Limited Administrator group and enter those potential Connect Seminar users who are not in the Seminar Hosts group into the Limited Administrator, they will be able to see the shared seminar calendar:

la.fw

How to Remove a Course End Date in Connect

This solution is only applicable to only on premise customers; hosted much-tenancy customers are not able to edit the needed file.

tngdate.fw

On the Connect server, go to:

<CONNECT_HOME>/<release>/appserv/apps/course/sco_edit.xsl

Search for:

<xsl:variable name=”disable-date-closed-sel”  select=”boolean(/results/sco/date-end or (/results/params/param[@name=’date-end’] and /results/params/param[@name=’date-end’]!=’unset’) or (/results/nav/@style=’wizard’) )”/>

Remove the section highlighted in bold italic: or (/results/nav/@style=’wizard’)

Restart the Connect Services

The course end date is removed.

tngdate1.fw

On-premise Connect without Webinars Throws an Error when Moving Meetings

On-premise Connect customers running version 9.5 who do not have the Webinar option enabled will see an error when trying to move meetings via the GUI in Connect Central:

Webinar95Bugon-prem.fw

This will be fixed in an upcoming release and in the meantime we have two workaround options:

CSO: Resolved and Restored – DATE (09DEC2015)

From 9:05 to 10:30 AM MST, all Adobe Connect customers hosted on NA1 would have been unable to login or use any Connect related services. This has been repaired and customers are now able to login

Adobe Connect ID’s are changing from INT to BIGINT

UPDATE: THIS ARTICLE IS FOR XML API DEVELOPERS ONLY.  If you are not using the XML API with Adobe Connect to write your own custom built applications that integrate with Adobe Connect, please do not worry about the message below as it will not apply to you.


Due to the growing popularity of Adobe Connect, more and more customers rely on Adobe Connect for their collaboration needs and this naturally requires that we make adjustments to accommodate the growth.  We want to make you aware of an upcoming adjustment to the ID values in Adobe Connect databases to support longer values which will accommodate growing customer use of Adobe Connect.

What is being changed?

Starting with release 9.5.2, Adobe Connect will migrate the ID values in Connect databases from INT to BIGINT/Long.

What is the impact?

This change will impact any applications using Adobe Connect Web Service APIs.

How will Adobe Connect Web Service APIs be affected?

The change is for those APIs which consume/return id values i.e. which have a parameter in request/response which is an Id. (examples: sco-id, account-id, folder-id, etc).  The id fields in such APIs would return/accept larger values. The support and behavior of existing APIs remains same.

What does this mean for you?

In case you are interpreting ids as integers the new values might overflow. We strongly recommend using strings to represent id values. In case you still need to represent/store ids as a numeral please use integral data type with higher capacity. (Preferably 64 bit)

What is the timing?

We expect to release Connect 9.5.2 in the first quarter of 2016, so the change should be made as soon as possible in order to be ready prior to the release.

CSO : DATE (20 NOV 2015) – Adobe Connect Access Intermittent for DA2 Datacenter – RESOLVED

UPDATE #4:

As of 1:05 PM ET,  while we are still waiting on Level 3 to confirm on their end that the issue is resolved, further testing by our Technical Operations team was able to confirm that network traffic has returned to normal levels and Adobe Connect services are no longer appearing to be impacted.
We are marking this as resolved and a full incident review will be conducted.  Out of this, a statement will be prepared and delivered.

 

UPDATE #3:

As of 12:50 PM ET, we are still waiting on the service provider.  Level 3 is experiencing major issues across possibly multiple locations although we still do not have confirmation outside of Chicago.

The issue is still intermittent depending on the network route that customers are taking to our Dallas datacenter.  Customers may have packet loss but still get to the Connect account.  They may experience slower load times and performance issues.  Other customers may experience timeouts.   Customers not going through affected areas will not see an issue.
Update to follow.

 

UPDATE #2:

As of 12:00 PM ET, we have identified this being a carrier issue currently in the Chicago area. It appears to be an outage on the Level 3 network where it is affecting customers routing to our Dallas datacenter (DA2) through Chicago.  It may possibly be impacting other routes as well, but currently the teams are waiting on official word from Level 3 on this issue.

 

UPDATE #1:

As of 11:15 AM ET, there is intermittent connectivity to Adobe Connect meetings and the Adobe Connect web application for all clusters in the DA2 datacenter.  This affects accounts on the following clusters:

NA3, NA4, NA5, NA7, NA10, NA11

The teams are currently engaged and investigating this issue.

Update to follow.

 

Connect 9.5 Edge Server Installation Instructions

Note: The upgrade installer described in this article below is deprecated. For instruction on the latest 9.5 installer, please see the following article: 9.5 Connect Edge Proxy Server Full Installer

Connect 9.5 server installation instructions:

  1. Create a folder <Installation_Directory>/950/edgeserver
  2. Download the Edge 9.5 (based on AMS 5) installer
  3. Run the self-extracting .exe file downloaded in step#2 to <Installation_Directory>/950/edgeserver
  4. Refer the following articles for deployment options:
    1. http://www.connectusers.com/tutorials/2011/06/edge_server_deploy/
    2. http://www.connectusers.com/tutorials/2011/06/edge_server_deploy2/
  5. Run <Installation_Directory>/950/edgeserver/win32/vcredist_x64.exe
  6. Run the following commands as administrator:
    1. cd <Installation_Directory>/950/edgeserver/win32AMSAdmin.exe -install
    2. AMSMaster.exe -install
    3. sc start amsadmin
    4. sc start ams
  7. Confirm that services “Adobe Media Administration Server” and “Adobe Media Server (AMS)” are running
  8. If services need to run using specific user credentials, then be sure to set the credentials in service properties and restart the services

Estimating Bandwidth Consumption in Connect Meetings:

Estimating bandwidth consumption in Connect 9.x Meetings:

The following tables may be used for estimating the amount of bandwidth that will be required for a Connect Meeting based on the features used.

Screen Share:

Screen Share Frame Rate Quality Bandwidth
(kbps)
Low 2 65 500
Medium 4 80 800
Standard 6 90 1200
High 8 100 2000

VoIP:

With reference to VoIP, it is independent of any settings. There is no metric comparable to frame rate or quality or bandwidth with VoIP;  a single metric of micRate is used. Currently we use following values:

Number of VoIP users Without UV With UV
<4 22 kHz 22 kHz
>=4 11 kHz 8 kHz

Note: In a simple test during which a song was played via VoIP for a single user, up bandwidth consumed was 44 kbps when the number of VoIP users was < 4.

Webcam video:

Webcam Video Frame Rate Resolution Quality Bandwidth
(kbps)
Normal Widescreen
Low 4 160×120 214×120 70 200
Medium 4 320×240 427×240 70 300
Standard 8 320×240 427×240 70 300
High 20 640×480 854×480 90 600