[Red5] Multiplayer Game using RTMP

Tue Feb 19 17:49:07 PST 2008

Hi Thomas,
Thanks for the test report. I would like to know if you are willing to
contribute your stress test app to the community. :-)

On Feb 20, 2008 7:11 AM, Thomas Williams <thomas.williams at dubitlimited.com>
wrote:

> Hi chaps,We have been load testing red5 with this usage configuration, and
> we've found it has some serious problems with deadlocks. We've been using a
> java client that was implemented using the red5 RTMP libraries, and we've
> been sending collection data (maps, arrays & nested combinations of the
> two). Initially we planned to test from multiple machines, however this did
> not prove necessary, as we could not achieve sufficient traffic from one
> machine to warrant using a second. Our test client was a multi-threaded
> model which tries to emulate a flash connection (all it was doing was
> connecting, not performing any operations), the interval listed is the time
> between starting each connection:
>
> Threads :: Connections :: Interval :: Successful connections
> 1 :: 1 :: 1 sec :: 1
> 1 :: 10 :: 1 sec :: 3
> 1 :: 10 :: 10 secs :: 5
> 10 :: 10 :: 1 sec :: 5
> 10 :: 100 :: 1 sec :: 19
> 10 :: 100 :: 10 secs :: 14
> 100 :: 100 :: 1 sec :: 17
> 100 :: 100 :: 10 secs :: 11
> 100 :: 200 :: 1 sec :: 13
>
> The remaining connections failed to connect within 10 seconds. This was
> with red5 version 0.6.3. We got more success with red5 version 0.6.2(approximately 200 connections) but we found that connections weren't stable
> and there seemed to be some message loss (this may simply have been to do
> with threads being starved).
>
> With regard to memory usage we found that red5 0.6.2 with maximum
> connections can consume up to 2GB of ram (we've experienced this under real
> load), however this mainly seemed to consist of queued outgoing messages,
> which were being produced at a faster rate than the server could send them.
>
>
> We've had much more success using the binary sockets provided in flash 9,
> and have been peaking at around 6000 concurrent sessions on a single server,
> with a message throughput of about 25,000 messages per second (this seems to
> be a hardware limit on the server, it's 100% utilisation on all cpus).
>
>
> We've got lots of other stats if you want to contact me with specific
> enquiries.
>
> Tom Williams
>
> PS We ran the test server on a quad Xeon server with 4GB of ram, and
> strangely the server it did not seem to use up that many resources.
>
>
>
> On 14/02/2008, Walter Tak <walter at waltertak.com> wrote:
> >
> > > Walter,
> > >
> > > See my comments below.
> > >
> > > On Feb 12, 2008, at 6:02 AM, Walter Tak wrote:
> > >
> > >> I tend to disagree here a bit.
> > >>
> > >> You don't need to encapsulate your game-data in bloated xml-
> > >> messages. Just
> > >> invent your own "protocol" with formats like:
> > >>
> > >> v=200+x=829.82+y=837.22+hp=8222+mana=6252+shot=1+param1 etcetera
> > >>
> > >> No need to serialize, just split the message based on your
> > >> delimiter. Also
> > >> the overhead is near zero so your bandwidth usage on client and
> > >> server drops
> > >> dramatically and latency increases.
> > >
> > > Well splitting there is definitely a parsing routine, all be it a
> > > simple one. But it's still de-serializing your string message.
> >
> >
> > I was illustrating this since many ppl tend to go for XML right away
> > while
> > alternatives are around and aren't difficult and often suit the project
> > better.
> >
> > If performance matters don't use XML, that's the basic rule of thumb I
> > use.
> > If you want flexibility, portability and interoperability please do use
> > XML
> > ofcourse.
> >
> >
> > >> If you understand binary format you can even use encode data in bit-
> > >> format
> > >> to minimize packetsize. (In the above example each parameter is
> > >> represented
> > >> by a character. If you'd only have 8 different types of parameters
> > >> then 3
> > >> bits would be enough for the parameter-type. Numbers could be
> > >> recoded to
> > >> hexadecimal like v=C8 instead of v=200 ; saves another byte (which
> > >> is pretty
> > >> dramatically ; 4 chars instead of 5 chars is a 20% gain in this
> > >> particular
> > >> case ; imagine 20.000 users sending 5 of these messages per second
> > >> to your
> > >> server, think big. Now think of the xml-formatted messages if you'd
> > >> do it
> > >> the 'right' way.)
> > >
> > > You can still do the same thing over RTMP.
> >
> >
> > I know that. We're using Red5 for more than half a year in production
> > environments. And still do.
> > However ; quote:
> > "Real Time Messaging Protocol (RTMP) is a proprietary protocol developed
> > by
> > Adobe Systems for streaming audio, video and data over the Internet,
> > between
> > a Flash player and a server." (from
> > http://en.wikipedia.org/wiki/Real_Time_Messaging_Protocol )
> >
> > RTMP is still a non-open protocol and if you want to develop "some game"
> > then a more or less protected protocol might not be the best choice.
> >
> >
> > >>
> > >>
> > >> Furthermore a single socketserver is able to serve more connections
> > >> than
> > >> Red5 since Red5 is more like a heavyweightboxer ; it has a lot of
> > >> stamina
> > >> and features but isn't agile with a small memoryfootprint.
> > >
> > > That's true. But, having these extra features is quite nice, and most
> > > people end up using them anyway. Like database access with remoting
> > > for example. Now you don't need another server for that stuff. And you
> > > can't deny that using native binary objects is way easier to program
> > to.
> >
> >
> > Well if you'd setup a simple php-socket-server ( I helped the PEAR
> > developer
> > a bit a few years ago with some performance issues) then you'd have all
> > the
> > power of databasing etc you need.
> >
> > To be exact about remoting ; We're currently tryign to get remoting
> > working
> > (from PHP to Red5 and from Red5 to PHP) but we fail to so (from PHP to
> > Red5,
> > using the url:5080/gateway way ) and remoting from Red5 to PHP lets us
> > crash
> > the server (well java tbh) in minutes under heavy load :(
> >
> >
> > >> A native c++ socketserver would be fine but you could even setup a
> > >> lowbudget
> > >> and 'low knowledge required' scripting (php/perl/python)
> > >> socketserver until
> > >> your game gets hit by zillion people per day and you gain enough
> > >> revenue to
> > >> let someone with skills develop a proprietary (c++?) socketserver
> > >> for you.
> > >>
> > > Or you could just use Red5, it's already built, and there's a solution
> > > for when your game gets huge numbers of users.
> >
> >
> > I'm not sure if you want to setup edge/origin and a dozen servers just
> > to
> > end up having the same performance but a huge overhead of extra servers,
> > control, management and not to mention, cost versus some proprietary
> > 'own
> > made server-code' running on a single-system.
> >
> > Though we (my team) have quite a bit of knowlegde of
> > php,perl,javascript,asm, vb, asp, dotnet, Flash as2, as3, designing
> > large
> > scale systems, databases and stuff we still struggle to "do things
> > right" in
> > java and the way Red5 operates.
> >
> > I can imagine that unless you know Java AND Flash/actionscript by heart,
> > Red5 still isn't an easy option, unless you really need some of the
> > features
> > of Red5 like the videostreaming ; which is imho the best (and only?)
> > unique
> > selling point of Red5. Only Wowza and FMS are comparable competitors for
> > that feature.
> >
> >
> > >
> > >> The edge/origin option of Red5 is really cool but pretty much in
> > alpha
> > >> state. I wouldn't rely on that for the coming year unless you really
> > >> know
> > >> what you are doing and/or have an extensive developing background
> > >> with Red5.
> > >>
> > > yeah, or talk to us at Red5Server where we are providing hosting for
> > > such a thing now.
> >
> >
> > I wonder what the costs would be if a simple game would have 1k users
> > online
> > and eating 1 Tb traffic each month :)
> >
> >
> > >> Regards,
> > >> Walter Tak
> > >
> > > anyway, overall good points Walter. I like your simple approach to
> > > this, but sometimes people want the extra features and don't want to
> > > have to write a socket server specific to their app.
> >
> >
> > Tbh we are experiencing a lot of overhead on Red5 projects (doing a 4th
> > commercially currently) due to the fact that documentation, examples and
> > help is very fragmented. This mailinglist though is VERY helpful and I
> > really want to say thanks to all that contribute to it but it isn't
> > enough
> > sometimes .
> >
> > The product is under heavy construction and something relative simple
> > like
> > installing Red5 from Trunk and gettings things done like logging to a
> > file,
> > accessing a remote database, accessing remote scripts or letting other
> > servers access Red5 cannot be found even when searching the
> > list-archive,
> > many sites and faqs. It requires a time consuming trial-and-error
> > approach,
> > which isn't an option for every developer unfortunately.
> >
> > I'm soon able to stresstest Red5 on two brandnew quad-core servers on
> > several points ; # simultaneous streams, # calls per second (from a
> > client
> > to Red5) , # of max simultaneous users , # of max simultaneous rooms,
> > memory-footprint etc. (Part of the job) I hope to be able to publish
> > those
> > results here for further discussion.
> >
> > Regards,
> >
> > Walter
> >
> >
> >
> > _______________________________________________
> > Red5 mailing list
> > Red5 at osflash.org
> > http://osflash.org/mailman/listinfo/red5_osflash.org
> >
>
>
>
> --
> Thomas Williams
> Java Programmer
> Dubit Limited
> +44 1133 947920
> www.dubitlimited.com
>
> _______________________________________________
> Red5 mailing list
> Red5 at osflash.org
> http://osflash.org/mailman/listinfo/red5_osflash.org
>
>

-- 
Best Regards
Steven Gong

InfraRed5 Red5 Consultant: http://www.infrared5.com, steven at infrared5.com

Red5 Developer: http://osflash.org/red5,
http://jira.red5.org/confluence/display/~steven/Home

Javaflash Project Founder and Maintainer: http://osflash.org/javaflash

Modesty is an overrated quality in men of no great accomplishment.  -- Ricky
Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://osflash.org/pipermail/red5_osflash.org/attachments/20080220/76b8b7f0/attachment.html 