LCCS Audio/Video Limitations

[__DEBUG__]

The goal of our project is to have an Audio/Video chat of 10-20 users (most of the users in the chat stream low-quality video and have audio muted, while two groups with up to 5 users all streaming full audio and video). We decided to use the latest LCCS with Flash 10.1 as a platform for our solution, because this combination seemed that it will provide us with a transparent way to achieve p2p communication with a fail-over mechanism.

But in our attempts to reach 10 users with full Audio/Video chat we run into some issues that look like we've hit the current technology limitations. We tried all kinds of options, but whatever we've tried our current achievement is just 5 users in an AV chat. Trying to join another user to the conversation results either in browser hangup or crash, we even managed to get our local network wi-fi router to get sunk when we reached 6 users.

I'll give you a brief explanation of the 3 main options we've tried and I'll also supply you with a link from where you can download an isolated example we've used in our tests, so everyone can check it. I'll be happy to get any feedback on whether I'm using the LCCS framework components in the correct and intended way.

1. Using the default settings of LCCS, no changes to the StreamManager properties.

On a single Mac computer the limit is 3-5. On Macs it tends to crash with fewer users in comparison with Windows. We tested it also on the local network with different machines - again Safari on Mac is crashing when we try to join a 6th user to a chat with 5.

Almost identical results we're getting with the other options.

2. Increasing the number of streamManager.maxP2PStreamPublish to 20.

3. Using streamManager.maxP2PStreamPublish=20 and session.streamManager.streamMulticast=true

From the link below you can download the Flash Builder project. You can import it and start testing.

DOWNLOAD THE TEST APPLICATION

Some notes on the Test Application:

The test application consists of two mxml application - LccsLimitationsOwner and LccsLimitationsPublisher. You should modify LccsLimitationsOwner with your correct username and email. And you should run it before the publishers applications, because the owner sets the StreamManager properties. Make sure you edut the correct room URLs in both applications.Also make sure that you check Auto Promote Users from the Room Console of your room. The users should be at least publishers.

When you start the publisher app, you will be prompted for a username - here you can put whatever you want, later you'll refer to these usernames when you want to connect to someone with the test app.

The testing is basicaly connecting several users and then each of them adds the rest by typing a username and clicking the GO-button.

Before running the owner's application you can set in the mxml what kind og configuration you want to test, by changing the value of the lccsConfiguration-property to one of the following values:

[Bindable] public static var DEFAULT_CONFIGURATION:String = "default";
[Bindable] public static var HUB_SPOKE_CONFIGURATION:String = "hub-spoke";
[Bindable] public static var MULTICAST_CONFIGURATION:String = "multicast";

Let me know if you have any questions on the test application!

[Former Community Member]

Hi Vladimir,

Thanks for the detailed post (note to other users - posts like this, with sample code and detailed issue descriptions, really get our attention! This is a textbook case for how to report bugs).

  Hironmay has told me he's on the issue (I think you know this already). We're going to focus on this starting on Monday, and try to figure out what's happening. Hopefully we can get a reproducible case - if so,  chances are we can find a fix or workaround for this .

  thanks again for the report - we're always glad to have users helping us fix issues.

  nigel

[Former Community Member]

Hi Vladimir,

Just a heads up. I downloaded your application , made the required changes and started running.

As for information on component implementation, everyone has his own style of coding and implementation and I found nothing wrong in your way of implementation on a first glance while running your owner and publisher apps. And I was able to connect to room and publish audio/camera and subscribe to user streams also.

Right now, I was only running your app on windows and just as I reply, I ran 9 client users with publisher apps and using Multicasting. I could connect fine and couldn't get crash, but of course my machine became very very slow as I started running so many clients from flash builder debugger.

I just wanted to give you on update as of current status. This is just beginning and Tomorrow as week starts, i will get more users to run (internally) rather than just from my own machine as well as run apps on MAC and update you in coming days about the state.

Only last thing I forgot to mention earlier, If you have even once did multicasting = true in your room , the room would have created the multicast node and from then onwards, the default option remains multicast unless you explicitly set it to false. It is only when you are in a new room and there is no multicast node, the non-multicast option is the default. I wasn't sure if you were aware of this but thought of informing you.

More updates by mid week.

Thanks

Regards

Hironmay Basu

[__DEBUG__]

Hi Hironmay,

Just to make sure, so far you've been able to run 9 users on a single machine and all nine were connected to each other in many-to-many Audio/Video chat? It is not just 9 publishing their own streams, right?

I also ran some tests with the LCCS SWC for Flash 10 and got results similar to my previous experience with the SWC for Flash 10.1. In my home setup - 2 laptops (Mac and Windows) I managed to connect not more than 5 in a stable chat. The browser on the Mac hanged when attempting to connect the 6th user, while the Windows just started performing real slow.

[Former Community Member]

Hi Vladimir,

I have been running different cases of your app for the last two days along with some help from my teammates. I would like to summarize some of the findings

a) If you are running a number of clients from the same machine, with each publishing and subscribing to all other streams that are there in the room , then it becomes really slow after 5 users and around 7 or more , it does crash or become unresponsive. The problem is not I believe related to LCCS or multicast but sharing too many audio video streams on the same machine requires lot of bandwidth and even if you do the same thing with any flash app that is sharing audio/video from same machine, the results would be similar.

I believe this is certainly not the use case in real world that one user is opening 7 instances of flash client for an application in the same machine.

b) If you are spreading out your clients on multiple machines with 2-3 in each, then it runs fine. We took an example where we had 3-4 machines and every machine used 3 clients and every client published his stream and subscribed to every other stream( i.e. subscribed to around 8-10 streams ) that are there in the room. We never faced any crash or slowness if we limited our number of clients to 2-3 per machine.

c) Most common use case in any real life application would be i am publishing my stream in one client on my machine and also subscribing to all other users who are sharing their streams. In such a case , it works fine with ease. I will in fact attach a screenshot where we had some 3-4 machines and on one machine we had only one client which subscribed to almost 12 streams( most of which were on mac) and it was fine.

So, to summarize its more to do with machine and bandiwidth rather than LCCS. If you run 1-2 instances per client and then even if there are lot of users in the room( as per your requirement), you will be able to chat/audio/video with them. In fact, if you want , we can invite you in one such session where you have one-two client and you can subscribe to everyone of us and we can all talk together.

Lastly, there was a bug in your publisher file regarding timeout which can throw errors, especially in 10.1. Because in 10.1, lot of the times, you might get a flash peer settings dialog asking whether you want p2p or not, and unless you say yes/no and the timeout expires and it starts publishing, you will get an RTE. This particularly happens when are are more users and little slower to enter rooms. So, i wonder that could be also one of the reasons for you. I have fixed that by adding a button which calls the publish function instead of timer. To publish, you need to click on that. I am attaching your updated publisher file too along with this mail.

Things could be little slower for you than us as you are on a different geography and outside US but as long as you limit the number of clients in a machine, it shouldnt crash for you.

All these tests I did using multicast groups with p2p. So, I didnt find noticeable limitations there.

Let us know your thoughts and how it goes for you.

Thanks

Regards

Hironmay Basu

[__DEBUG__]

Hi Hironmay,

I do understand that this is not a real world scenario to run several instances from the same machine and I would expect issues in such a test setup.

But we did run the same app with multiple machines on a same network in a clean setup where each machine runs just one instance. Nevertheless, we were unable to reproduce the results you're describing - our current achievement is 5 clients tops.

In addition to the LCCS test, we developed a similar test for FMS4 using hub-spoke and we ran it in exactly the same setup. The behavior was totally different - we managed to grow the users to 8, and if we had more computers available I would expect that we could easily reach 10 and more without any significant degrade in performance.

I would like if we approach this issue in a more systematic manner. I'll list all factors I could think of that could affect the behavior and we can go through all of them one by one to eliminate the potential reasons and to detect what could actually cause the issue.

1. Our source code

I was hoping that the problem could be in my source code and the way I'm using the LCCS framework. But you're reporting that everything is OK with our code. Except for the exception which arises - currently it's an open problem for me. I'm afraid your description of why the exception happens isn't clear, so let me bring some light why I'm using the setTimeout.

The setTimeot is only needed in the case I'm using LCCS with p2p and multicast set to true - in all other cases there's no need to use the timeout. If you comment out the setTimeot and call the publish()-method directly,

//setTimeout(publish, 5000);
publish();

you'll run into the following runtime exception:

Error: Error #2154: The NetStream Object is invalid.  This may be due to a failed NetConnection.
    at flash.net::NetStream/invoke()
    at flash.net::NetStream/attachAudio()
    at com.adobe.rtc.collaboration::AudioPublisher/onStreamReceive()[C:\work\main\connect\cocomoPlayer10.1\src\com\adobe\rtc\collaboration\AudioPublisher.as:900]
    at flash.events::EventDispatcher/dispatchEventFunction()
    at flash.events::EventDispatcher/dispatchEvent()
    at com.adobe.rtc.sharedManagers::StreamManager/onItemReceive()[C:\work\main\connect\cocomoPlayer10.1\src\com\adobe\rtc\sharedManagers\StreamManager.as:1955]
    at flash.events::EventDispatcher/dispatchEventFunction()
    at flash.events::EventDispatcher/dispatchEvent()
    at com.adobe.rtc.sharedModel::CollectionNode/http://www.adobe.com/2006/connect/cocomo/messaging/internal::receiveItem()[C:\work\main\connect\cocomoPlayer10.1\src\com\adobe\rtc\sharedModel\CollectionNode.as:777]
    at com.adobe.rtc.messaging.manager::MessageManager/http://www.adobe.com/2006/connect/cocomo/messaging/internal::receiveItem()[C:\work\main\connect\cocomoPlayer10.1\src\com\adobe\rtc\messaging\manager\MessageManager.as:738]
    at com.adobe.rtc.session.managers::SessionManagerBase/receiveItem()[C:\work\main\connect\cocomoPlayer10.1\src\com\adobe\rtc\session\managers\SessionManagerBase.as:415]

This exception has nothing to do with the number of users. Simply, in the case when I'm using p2p with multicast, the user needs to allow the Peer Assisted Networking from the Privacy dialog, otherwise he can not connect succesfully to the NetGroup of the mesh. The exception happens when you try to publish your audio/video stream before you have connection to the NetGroup. If I have an event that notifies me that I have successfully connected to the NetGroup I would not use the timeout. Currently I'm using a 5 seconds timeout to give a chance to the user to quickly allow the setting from the Privacy dialog - if the users slows down and misses the 5 seconds interval he'll get the exception.

But this exception is not related at all to the unstability of the chat. Each of the machines we use for the chat is a developer machine with debug Flash player, so this exception wouldn't be left unnoticed.

If you can not report about other issues with the test app, that we can both conclude that the issue is not there. Let's move forward.

2. LCCS Client Framework

I've tried both SWCs - for Flash 10 and for Flash 10.1 - and I'm getting similar results. So if we leave out the multicast, for hub-spoke both SWCs let to the same result (at least in our test setup).

3. Flash Player

All tests we've run so far are with Flash 10.1. Let me know if it makes sence to try Flash 10 for the hub-spoke.

4. Client Machines

The machines we use in our test setup are 4 iMacs bought recently, my macbook which isn't a very new model, and some high-performant Windows machines. All this machines are developer's machines, so it is unlikely to be a hardware problem.

6. LCCS Service Cluster

Could it be a problem related to the cluster in which the LCCS service is running. This may hold some ground only in the hub-spoke scenario?

The underlying server for LCCS I guess is FMS4. We did run some hub-spoke tests with FMS4 and everything seems OK.

7. Bandwidth/Network

If we're able to have a chat ot 8 and more with FMS4 therefore there's enough bandwidth for the same thing to run with LCCS. Could it be some other network issue, since we're running the FMS inside our local network, while the LCCS is outside our network.

The next test we're going to perform is simple implementation of p2p and multicast with the FMS4 and I'll let you know about our results.

What's the plan from your side?

Message was edited by: Vladimir Tsvetkov

[Former Community Member]

Hi Vladimir,

I think it will be better if we are in the same room running tests together. I already sent you screenshot where I was subscribing to 12 streams , leave aside 5. And I have run here many many tests in which taking around 3 machines, we have easily reached 9 streams with 3 clients each machine. And most of the time, I am out of adobe network in a remote location.

So, I can't quite agree with the 5 number. If you want, I can run the tests with at least 5 users and you can join in or you run 3 clients and we can join in with 3-4 on our machines. I can initially run with player 10 swc without using NetGroups and then going forward, can use 10.1 with multicast or even vice versa. And all this while, I have been running your publisher app , and we will run the same app.

Thanks

Hironmay Basu

[__DEBUG__]

Hi Hironmay,

Don't get me wrong, I'm not doubting the results of your tests, just as you shouldn't doubt the results we got.

If there are no problems on your side, you should be able to at least give us some sort of suggestion what could be wrong on our side.

It's the easiest thing for me to join you in a room and comfort myself that everything is just fine, but when it comes to demonstrate our application to our customer and when things go wrong in terms of chat stability and browser crashes, then it is me and my team who look total incompetent in the eyes of the client.

I do expect a deeper approach on this issue. Assuming everything is alright on your side, what could be wrong on our side?

[Former Community Member]

Ok, then lets go step by step.

Start with player 9 SDK swc first. It doesn’t have p2p or multicasting. See the result on your side with same 5-6 clients and let me know. If things are fine on your side , we will assume it may be p2p features are giving trouble. If things are still the same, we can then look for alternate ways.

Thanks

Hironmay Basu

[__DEBUG__]

Hi Hironmay,

We tested the app with the SWC for Flash Player 9 and the test went quite well - we managed to have a stable chat with 9 to 9.

Then, we changed to the SWC 10.1 just to feel the difference - at around 6 the my Mac browser became unresponsive and eventually crashed.

Then, we tested again with SWC 9 to double check - we managed to get to 9, but while attempting to grow to 10 we got a crash, but this time on the Windows machine.

Before each test all machines used in the test were restarted to make sure no other apps have occupied RAM or affect the tests in some way.

My conclusion is that the version for 9 is more stable, but it is not 100% stable.

I'm interested whether you test by yourself or you have a team of QAs who can give it a test drive in some sort of a more controlled environment where more factors can be monitored and tweaked?

[aj-runtimelogic]

Hi Hironmay,

I've also been using LCCS to implement an audio chat feature within an existing RIA I'm working on.  I'd like to inquire specifically about your mention of the P2P dialog that is shown when a user logs into a LCCS room when streamMulticast == true.  I've been struggling with how to deal with the timing issue created by the display of this dialog and the need for the user to have responded to it before publishing the stream through the creation of the AudioPublisher class.

For the Microphone I can do something like:

     _currentMic.addEventListener(StatusEvent.STATUS, this.micPermissionChanged);

In Vladimir's project I can see that he refers to:

     session.streamManager.addEventListener(StreamEvent.STREAM_MULTICAST_CHANGE, ...

but StreamEvent.STREAM_MULTICAST_CHANGE is undocumented and this doesn't seem to work.  Is there a way to handle this situation currently, and if not, what are Adobe's plans for handling this?  Ideally, we wouldn't need to prompt the user to manage networking traffic they may not understand (doesn't make for a great user experience), especially if we have to subsequently prompt them for mic permission anyhow.

Thanks.

[Former Community Member]

Hi,

I can understand the difficulty since we don't provide the source for 10.1 yet. If you need multicasting and use player 10.1, you need to wait for p2p dialog to accept/reject before publishing your stream.

Just to give some background how it works, so when you are using 10.1 and you are doing multicast ( lets say multicast=true) , then when you enter, streamManager creates a multicast NetGroup on synchronization that prompts you the dialog. Now, if you have checked always allow even once, its taken care automatically, otherwise you would have to wait for the dialog and respond to it before continuing.

For your question, one way could be if streamManager.multicast == true, then check for NetStatusEvent.NET_STATUS event fired from streamManager, and once that gets fired ( either "NetGroup.Connect.Rejected" or "NetGroup.Connect.Success" depending on whether you selected allow or deny p2p ) , you should publish the stream in the handler.

This event isn't documented and so not exposed as code hinting but I will make that next time.

As an additional info, the multicast group that is created is named as DEFAULT_MULTICAST_STREAM_GROUP. It's a public static const in StreamManager.

So, you can also check using this API called streamManager. getMulticastGroupSpec(StreamManager.DEFAULT_MULTICAST_STREAM_GROUP) whether the group exists or not before doing any publishing when multicast is true.

Lastly, StreamEvent.STREAM_MULTICAST_CHANGE doesn’t work in this case because you are not actually changing the multicast property.

Thanks

Hironmay Basu

[aj-runtimelogic]

Thanks for your quick response!  It's really helpful to understand a little more about what is going on behind the scenes so thanks for including the background information as well.  I implemented the NET_STATUS listener and it's working great.  Definitely a lifesaver.

On the topic of showing the peer-assisted networking dialog, I'm curious what the difference is for a 2 person audio chat when not using stream multicasting, but still using an RTMFP connection for P2P data flow.  Is the reason the peer-assisted networking dialog isn't shown in this case because there can be no relaying of stream data in that scenario?  If so, I understand the distinction, but the user is still going to be sending data both upstream and downstream, and there hasn't been any restriction placed on the streaming data bandwidth in either scenario, so I'm still not sure why the user needs to be prompted at all.  Perhaps there is a security detail I'm overlooking?

StreamManager.getMulticastGroupSpec looks promising for creating private collaborative groups within a room.  By default it looks like anyone who joins a room with streamMulticast == true is put into the default NetGroup with the name StreamManager.DEFAULT_MULTICAST_STREAM_GROUP.  Can you point me in the right direction for creating a new NetGroup within the room and assigning users to it?  Does the StreamManager.allowPrivateStreams property have any relationship or interaction with the P2P NetGroups?

Thanks!

[bernard_h]

I am experiencing exactly the same problems since SWC 10. when we are more than 4-5 users at the same time.

LCCS is a little bit more stable with Mac 10.6 and some Windows machines.

Read this old stuff...

http://forums.adobe.com/thread/667742?tstart=0

Bernard

[Former Community Member]

Hi,

So, basically when you are using 10.1 you have three options

a) Using multicast when it is true and if you do that, you get the peer to peer dialog. The dialog gets prompted when you create NetGroups for doing multicast.

b) Using basic P2P just as in player 10, there is no multicast and peer dialog but there is some limit on number of p2p streams you can have. My suggestion is, if you are not using multicast, you should ideally use our player 10 swc. Not only you will have a much more stable build, but also source code with you for any debugging.

c) Not using p2p and using hub-spoke. If you are using a lot of streams and users, hub-spoke works best in terms of quality and stability.

You are right that whenever we do multicasting, and users enter, they are assigned to the default multicast group. However, you have the freedom to create your own groups too. In StreamManager, we provide a concept of groups, and whenever you create a stream group, it also creates a multicast group ( if multicasting is true). The createGroup API in StreamManager creates multicast groups and if you want your audio pub/sub or camera pub/sub to belong to that group, you need to assign that component.groupName = "your new group name" . This is make sure those audio/camera components only contains streams for that group.

In fact, I don’t think any user has used this custom group part with multicasting. So, we aren't sure how well it works . You can try out and if it doesn’t work, I will fix it :).

For reference , You can look at the createGroup inside player 10 source code also , only thing is it will not contain the creating of the multicast group which is in 10.1 only.

StreamManager.allowPrivateStreams property doesn’t have any relationship or interaction with the P2P NetGroups. It is just for allowing private streaming inside a room.

Thanks

Hironmay Basu

[aj-runtimelogic]

Hi Hironmay,

Thanks for breaking that down for me.  I see now in the Beta AS3 Reference entry for NetGroup where it mentions the "peer-assistend networking" dialog:

http://help.adobe.com/en_US/FlashPlatform/beta/reference/actionscript/3/flash/net/NetGroup.html#NetG...

I'm trying to implement the use of custom NetGroups now, but I'm running into an issue.  As soon as the user accepts the "peer-assted networking" prompt I'm calling streamManager.getGroupNames() to get this list of existing groups and if "TestGroup" isn't in the Array then I'm attempting to create this group by calling streamManager.createGroup("TestGroup").  On that call I'm getting some kind of internal reference error within the rtc.session.sessionClasses package:

ReferenceError: Error #1069: Property http://www.adobe.com/2006/connect/cocomo/session/internal::streamPeerIDs not found on com.adobe.rtc.session.sessionClasses.SessionInternals and there is no default value.

at com.adobe.rtc.sharedManagers::StreamManager/createGroup()[C:\work\main\connect\SDKApp\payload\libs\flashOnly\player10.1\src\com\adobe\rtc\sharedManagers\StreamManager.as:1354]

I'm going to try the same code in a room with streamMulticasting set to false and will report back on that to see if it's specific to calling createGroup() with multicasting enabled.

[aj-runtimelogic]

It appears this error is only encountered when streamManager.allowPrivateStreams and streamManager.streamMulticasting are both true and one tries to create a NetGroup.  I disabled both and it worked fine, and then turned streamMulticasting back on and it also appears to work.  Still sounds like a possible bug to look into though.

[aj-runtimelogic]

Final update for the night: The exception I was getting when creating the group with multicasting on is still a problem.

I thought it was working because I created a "test" NetGroup with multicasting off, then turned multicasting on and was able to run with no runtime errors.  Well it turns out I had an error in my code and the audioPublisher was never having it's group set to the "test" NetGroup.  When I fixed the code I began getting a different error from the audioPublisher initialization when using the "test" NetGroup.  So I went ahead and tried creating a different NetGroup (with multicasting on) and when streamManager.createGroup is called I get the same error I posted above.

This looks like a bug with creating NetGroups with multicasting on, like you said might be the case.  Please let me know what you find and if there is anything else you would like me to try.

Thanks.

[Former Community Member]

Ok I will investigate today creation of custom netgroup and let you know the result. In the meanwhile, can you send me a small test code you are using ?

Thanks

Hironmay Basu

[aj-runtimelogic]

Certainly.  Our app is an all ActionScript project compiled under Flex Builder 3.  The project builds against the Flex 3.2 SDK, lccsFlash.swc 10.1, and playerglobal.swc 10.1.

Here is the method in my LCCS test code that creates the custom NetGroup if it doesn't already exist and then creates the audio pub/sub.  If you need more than this let me know, but this contains the key code for reproducing the failure.  Just make sure the room has streamMulticasting set to true.

private static const GROUP_NAME: String = "awesome";

private function initAudioComponents(): void
{
     var groupNames: Array = _currentSession.streamManager.getGroupNames();
     var groupNameMatched: Boolean = false;
     for (var i: uint = 0; i < groupNames.length; i++)
     {
          if (groupNames[i] == GROUP_NAME)
          {
               groupNameMatched = true;
               break;
          }

     }

    
     if (! groupNameMatched)
     {
          // the group doesn't exist so create it
          _currentSession.streamManager.createGroup(GROUP_NAME);
     }

    
     _audioSubscriber = new AudioSubscriber();
     _audioSubscriber.connectSession = _currentSession;
     _audioSubscriber.groupName = GROUP_NAME;
     _audioSubscriber.addEventListener(StreamEvent.CONNECTION_TYPE_CHANGE, this.audioConnectionTypeChanged);
     _audioSubscriber.subscribe();

     _audioPublisher = new AudioPublisher();
     _audioPublisher.connectSession = _currentSession;

     _audioPublisher.groupName = GROUP_NAME;
     _audioPublisher.codec = SoundCodec.SPEEX;
     _audioPublisher.subscribe();
     _audioPublisher.publish();
}

[aj-runtimelogic]

Hi Hironmay,

Just wanted to follow up on the error when creating a custom NetGroup with streamMulticasting on.  This is the last piece of our prototype that I need to confirm is functional before we can begin integration of LCCS based audio chat into our product.

Please let me know if my sample code is what you wanted and if you are able to repro the error.

Thanks.