[Red5] Question about your setup

Fri Mar 21 12:09:16 PST 2008

Interesting idea Walter.

Lenny

On 3/21/08, Walter Tak <walter at waltertak.com> wrote:
>
>  This is a wild guess but Red5 does have some keepalive mechanics afaik.
>
> I case the capacity of the user is fully used the keepalive pings might
> not be able to get through (lost) or might arrive with a delay.
>
> Either one of those might trigger the 'inactivity threshold' inside the
> server and fire the disconnect-routine for that particular client.
>
> Wild guess tho.
>
> If your connection however has enough cpu-power, bandwidth and a decent
> latency (ping) you might want to start an interval on the (Flash) client
> after a succesful connect and call a keepalive function on your server
> (empty function, returns nothing). Call it once every minute and R5 should
> see activity and never disconnect that client. Idea ?
>
>
> Regards,
> Walter
>
>  ----- Original Message -----
> *From:* Lenny Sorey <lrsorey at gmail.com>
> *To:* red5 at osflash.org
> *Sent:* Friday, 21 March 2008 16:29
> *Subject:* Re: [Red5] Question about your setup
>
>
> I have seen the inactivity notice in the logs from time to time.
>
> Has anyone really explained exactly what causes this.
>
> Does this fire when a user is still connected but nothing
> no VOD, No live streaming, no red5 activity from that user
> for the specified amount of time. Does this then disconnect
> the inactive user?
>
> Lenny
>
>
> On 3/21/08, Dan Rossi <spam at electroteque.org> wrote:
> >
> > I believe the inactivity error is a global one. Ill try and get an mrtg
> > report of the network traffic. It's prob not worth looking further until all
> > the P4 machines have been migrated over to xeon. I believe one of the
> > problems is the metadata cache. ive put the max cache setting to 10000 as we
> > have more than 500 different files being served obviously. the edge / origin
> > is the goal, but from tests i havent seen a reason to use it, as the loads
> > stayed the same. Once that is corrected. We'll prob look at 3 edge xeon
> > servers then two origin servers maybe quad core from the looks of it, thats
> > what i need to find out how big they really need to be ?  There is a mysql
> > running on our storage machine which gets connections from two windows media
> > servers so will prob move that off and run a mysql on each server in
> > replication mode i think, at least cross that off as a problem of
> > bottlenecks serving files to the machines.
> >  On 22/03/2008, at 1:00 AM, Lenny Sorey wrote:
> >
> >  Hey Dan,
> >
> > Thanks for your response.
> >
> > Your answers will provide info for the rest of us as we continue to
> > grow with RED5.
> >
> > Lenny
> >
> >
> > On 3/21/08, Dan Rossi <spam at electroteque.org> wrote:
> > >
> > >
> > > On 21/03/2008, at 11:09 PM, Lenny Sorey wrote:
> > >
> > > > Dan,
> > > >
> > > > Hope things are better with your RED5 setup these days.
> > > >
> > > > I do have a question.
> > > >
> > > > Many of us are not having the same type of problems you and this is
> > > > causing me some concern.
> > > >
> > > > As I understand, I believe you setup for Streaming VOD is not one
> > > > that many of us would be using
> > > > as small developers and providers.
> > > >
> > > > Would you mind describing you setup for others (including me) so
> > > > that we might try to understand
> > > > and plan for the future as we start to grow and move into an
> > > > environment like yours.
> > > >
> > > > What type of servers are you using?
> > >
> > > 3 x Windows 2003 Server Standard Edition with P4 3Ghz 2GB of ram. The
> > > new machine going to replace the older P4 machines is Windows 2003
> > > Server Standard Edition 64bit with dual core Xeon 3Ghz 4GB of ram, I
> > > believe the new machine coming will be dual dual core xeon, because im
> > > seeing 50% usage spikes even out of a dual core xeon and have the
> > > process high priority using all its 4 cpus. I was able to successfully
> > > turn off the red5 services on the other machine and the new xeon
> > > server took all the connections fine, but it did take up about 50-60%
> > > cpu. For such bigger processors i would be expecting 1-10% usage
> > > maximum.
> > >
> > >
> > >
> > >
> > > >
> > > > What flavor of RED5, Jetty standalone or War version.
> > >
> > > Jetty standalone, the older machines are running inside the java
> > > service wrapper, the new machine is running in the 64bit JRE, there
> > > was no native java service wrapper windows 64bit dll binary, and it
> > > was impossible to build, so the service is managed via firedeamon now.
> > >
> > > >
> > > > Are you currently using the Edge /Origin Serve Red5 solution.
> > >
> > > None of these but looking into it once we clear all the problems. Have
> > > tested many times before, and load is equal on the edge and origin
> > > systems, and very worried that the origin machine will be the load
> > > bottleneck and not knowing how big the machine needs to be after
> > > having to upgrade all our edge machines to dual core xeon !
> > >
> > > >
> > > > What kind of load balancing do you employ to manage the Servers?
> > >
> > > It's a shared load balancer switch solution that the hosts provide.
> > > They manage it for us and allocate a class C ip address on the public
> > > ethernet which the machines binds to, and the load balancer will point
> > > to whichever machine has the service running on the configured port
> > > and won't if its down.
> > >
> > > >
> > > > Do you run any other applications RED5 on the same server?
> > > > (IIS, Programs running under IIS, Database server with Red5 apps,
> > > etc)
> > >
> > > We use to have windows media running also, but we reduced them back to
> > > two machines running P4. They are all now dedicated with no other
> > > services runnining. The storage machine does have mysql running
> > > however.
> > >
> > > >
> > > > If I remember correctly, You employ a SAN system. Please, briefly
> > > > explain to the userlist
> > > > What the SAN is and how it is used with RED5.
> > >
> > > A P4 controller machine has a Dell MD1000 rack attached with about 3TB
> > > of storage. They weren't exactly able to increase the RAID drive so
> > > there became two attached drives sadly when we had to increase the
> > > space. We'll be moving to a new modular shared SAN solution being
> > > provided for us hopefully so it can scale quite easily with not much
> > > fuss as we had to move files all over the place having two attached
> > > drives to work with. So the controller machine has windows shares
> > > setup for these drives, and the red5 machines loads the files from
> > > this machine over the network using the filenamegenerator bean and
> > > providing the playback path.
> > >
> > > >
> > > >
> > > > Do you allow users to upload their videos to your servers, if so how
> > > > do you handle the metatdata generation
> > > > for uploaded videos?
> > >
> > > The producers upload video, and i have now built a standalone
> > > application which i posted to the list, that when double clicked will
> > > run through the drive and any flv missing a metadata file will get
> > > generated, sometimes it misses one, then will timeout connections when
> > > attempting to build the meta file on the fly !  This is what happens
> > > anyway it takes quite a while to build these files, the timeout will
> > > cause the fallback system to switch to another server and will go
> > > through the list until its not possible to re-attempt to play.
> > >
> > > >
> > > > I noticed in one of the Emails that you store more than 500 Videos,
> > > > I assume for VOD. Again, are these user supplied
> > > > and what is the general size of the videos in either GB or length.
> > > > Smallest to largest.
> > >
> > > Largest would be 40 mins so 200MB. It's all single owned content.
> > >
> > > >
> > > > What kind of Internet connectivity are you using for Red5 and any
> > > > other apps on these servers, 100 Mbps, 300 Mbps, etc?
> > >
> > > I don't understand sorry.
> > >
> > > > Is your bandwidth burstable for heavy loads?
> > > >
> > >
> > > Good question. There is a big pipe I believe.
> > >
> > > > Does your RED5 app allow for live streaming? If so, do you allow
> > > > recording of the live stream.
> > >
> > > Live server is seperate from the cluster. We had issues with VOD
> > > hogging the resources and causing live streams to drop frames all the
> > > time.
> > >
> > >
> > > >
> > > > I am just a little perplexed of the number of problems you are
> > > > having and just want to help if anyone else has
> > > > a setup nearly like yours and can offer any help.
> > > >
> > > > In addition, I also looking at this as I grow with RED5 what to be
> > > > wary of and what to look for as either me
> > > > or others reach the size of your operation.
> > > >
> > > > By the way, what kind of a video streaming business is this? What
> > > > kinds of user loads do you experience?
> > > > (average number of viewers, max number of viewers)
> > >
> > > I believe its only 80-100 concurrent users on each server maybe ? But
> > > it's a persistent number 24hrs a day I believe.
> > >
> > >
> > > _______________________________________________
> > > Red5 mailing list
> > > Red5 at osflash.org
> > > http://osflash.org/mailman/listinfo/red5_osflash.org
> > >
> >
> > _______________________________________________
> > Red5 mailing list
> > Red5 at osflash.org
> > http://osflash.org/mailman/listinfo/red5_osflash.org
> >
> >
> >
> >
> > _______________________________________________
> > Red5 mailing list
> > Red5 at osflash.org
> > http://osflash.org/mailman/listinfo/red5_osflash.org
> >
> >
>  ------------------------------
>
> _______________________________________________
> Red5 mailing list
> Red5 at osflash.org
> http://osflash.org/mailman/listinfo/red5_osflash.org
>
> ------------------------------
>
> No virus found in this incoming message.
> Checked by AVG.
> Version: 7.5.519 / Virus Database: 269.21.8/1337 - Release Date: 20-03-08
> 20:10
>
>
> _______________________________________________
> Red5 mailing list
> Red5 at osflash.org
> http://osflash.org/mailman/listinfo/red5_osflash.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://osflash.org/pipermail/red5_osflash.org/attachments/20080321/6200435f/attachment.html 