[Red5] Question about your setup

[Dan Rossi]

I believe the inactivity error is a global one. Ill try and get an  
mrtg report of the network traffic. It's prob not worth looking  
further until all the P4 machines have been migrated over to xeon. I  
believe one of the problems is the metadata cache. ive put the max  
cache setting to 10000 as we have more than 500 different files being  
served obviously. the edge / origin is the goal, but from tests i  
havent seen a reason to use it, as the loads stayed the same. Once  
that is corrected. We'll prob look at 3 edge xeon servers then two  
origin servers maybe quad core from the looks of it, thats what i need  
to find out how big they really need to be ?  There is a mysql running  
on our storage machine which gets connections from two windows media  
servers so will prob move that off and run a mysql on each server in  
replication mode i think, at least cross that off as a problem of  
bottlenecks serving files to the machines.

On 22/03/2008, at 1:00 AM, Lenny Sorey wrote:

> Hey Dan,
>
> Thanks for your response.
>
> Your answers will provide info for the rest of us as we continue to
> grow with RED5.
>
> Lenny
>
>
> On 3/21/08, Dan Rossi <spam at electroteque.org> wrote:
>
> On 21/03/2008, at 11:09 PM, Lenny Sorey wrote:
>
> > Dan,
> >
> > Hope things are better with your RED5 setup these days.
> >
> > I do have a question.
> >
> > Many of us are not having the same type of problems you and this is
> > causing me some concern.
> >
> > As I understand, I believe you setup for Streaming VOD is not one
> > that many of us would be using
> > as small developers and providers.
> >
> > Would you mind describing you setup for others (including me) so
> > that we might try to understand
> > and plan for the future as we start to grow and move into an
> > environment like yours.
> >
> > What type of servers are you using?
>
> 3 x Windows 2003 Server Standard Edition with P4 3Ghz 2GB of ram. The
> new machine going to replace the older P4 machines is Windows 2003
> Server Standard Edition 64bit with dual core Xeon 3Ghz 4GB of ram, I
> believe the new machine coming will be dual dual core xeon, because im
> seeing 50% usage spikes even out of a dual core xeon and have the
> process high priority using all its 4 cpus. I was able to successfully
> turn off the red5 services on the other machine and the new xeon
> server took all the connections fine, but it did take up about 50-60%
> cpu. For such bigger processors i would be expecting 1-10% usage
> maximum.
>
>
>
>
> >
> > What flavor of RED5, Jetty standalone or War version.
>
> Jetty standalone, the older machines are running inside the java
> service wrapper, the new machine is running in the 64bit JRE, there
> was no native java service wrapper windows 64bit dll binary, and it
> was impossible to build, so the service is managed via firedeamon now.
>
> >
> > Are you currently using the Edge /Origin Serve Red5 solution.
>
> None of these but looking into it once we clear all the problems. Have
> tested many times before, and load is equal on the edge and origin
> systems, and very worried that the origin machine will be the load
> bottleneck and not knowing how big the machine needs to be after
> having to upgrade all our edge machines to dual core xeon !
>
> >
> > What kind of load balancing do you employ to manage the Servers?
>
> It's a shared load balancer switch solution that the hosts provide.
> They manage it for us and allocate a class C ip address on the public
> ethernet which the machines binds to, and the load balancer will point
> to whichever machine has the service running on the configured port
> and won't if its down.
>
> >
> > Do you run any other applications RED5 on the same server?
> > (IIS, Programs running under IIS, Database server with Red5 apps,  
> etc)
>
> We use to have windows media running also, but we reduced them back to
> two machines running P4. They are all now dedicated with no other
> services runnining. The storage machine does have mysql running  
> however.
>
> >
> > If I remember correctly, You employ a SAN system. Please, briefly
> > explain to the userlist
> > What the SAN is and how it is used with RED5.
>
> A P4 controller machine has a Dell MD1000 rack attached with about 3TB
> of storage. They weren't exactly able to increase the RAID drive so
> there became two attached drives sadly when we had to increase the
> space. We'll be moving to a new modular shared SAN solution being
> provided for us hopefully so it can scale quite easily with not much
> fuss as we had to move files all over the place having two attached
> drives to work with. So the controller machine has windows shares
> setup for these drives, and the red5 machines loads the files from
> this machine over the network using the filenamegenerator bean and
> providing the playback path.
>
> >
> >
> > Do you allow users to upload their videos to your servers, if so how
> > do you handle the metatdata generation
> > for uploaded videos?
>
> The producers upload video, and i have now built a standalone
> application which i posted to the list, that when double clicked will
> run through the drive and any flv missing a metadata file will get
> generated, sometimes it misses one, then will timeout connections when
> attempting to build the meta file on the fly !  This is what happens
> anyway it takes quite a while to build these files, the timeout will
> cause the fallback system to switch to another server and will go
> through the list until its not possible to re-attempt to play.
>
> >
> > I noticed in one of the Emails that you store more than 500 Videos,
> > I assume for VOD. Again, are these user supplied
> > and what is the general size of the videos in either GB or length.
> > Smallest to largest.
>
> Largest would be 40 mins so 200MB. It's all single owned content.
>
> >
> > What kind of Internet connectivity are you using for Red5 and any
> > other apps on these servers, 100 Mbps, 300 Mbps, etc?
>
> I don't understand sorry.
>
> > Is your bandwidth burstable for heavy loads?
> >
>
> Good question. There is a big pipe I believe.
>
> > Does your RED5 app allow for live streaming? If so, do you allow
> > recording of the live stream.
>
> Live server is seperate from the cluster. We had issues with VOD
> hogging the resources and causing live streams to drop frames all the
> time.
>
>
> >
> > I am just a little perplexed of the number of problems you are
> > having and just want to help if anyone else has
> > a setup nearly like yours and can offer any help.
> >
> > In addition, I also looking at this as I grow with RED5 what to be
> > wary of and what to look for as either me
> > or others reach the size of your operation.
> >
> > By the way, what kind of a video streaming business is this? What
> > kinds of user loads do you experience?
> > (average number of viewers, max number of viewers)
>
> I believe its only 80-100 concurrent users on each server maybe ? But
> it's a persistent number 24hrs a day I believe.
>
>
> _______________________________________________
> Red5 mailing list
> Red5 at osflash.org
> http://osflash.org/mailman/listinfo/red5_osflash.org
>
> _______________________________________________
> Red5 mailing list
> Red5 at osflash.org
> http://osflash.org/mailman/listinfo/red5_osflash.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://osflash.org/pipermail/red5_osflash.org/attachments/20080322/cd14d652/attachment-0001.html