[Red5] Memory Leak - Test Case
Tyler Kocheran
rfkrocktk at gmail.com
Fri Jun 12 10:35:53 PDT 2009
Ok, after much deliberation, it's working.
Here's my test criteria that it's currently passing.
1. It must pass all unit tests (duh)
2. It must be able to start via "ant server"
3. It must be able to start via startup scripts
4. It must be able to be used outside of Red5 as part of RTMPClient or as a
part of another application.
The only dependency that it has is EhCache, which is already included in
Red5, so it shouldn't be a problem.
If the System property "red5.root" is set, it will try to find the EhCache
configuration file at "red5.root/conf/ehcache.xml". If a CacheException is
thrown there, it will fail over to a default CacheManager. If "red5.root" is
not set, it will use a default CacheManager. If it uses the default
CacheManager, it will try to find ehcache.xml on the classpath, but if that
file is not found, it will fail over to the default configuration file
"ehcache-failsafe.xml" inside the EhCache JAR.
It just works :) Let me know what you think, Paul.
On Fri, Jun 12, 2009 at 9:20 AM, Tyler Kocheran <rfkrocktk at gmail.com> wrote:
> I think Walter is right. For the project I'm working on, caching totally
> makes sense, both of my data objects using EhCache with Hibernate and with
> encoded Strings in this example. And EhCache is really nice, I swear. If you
> don't want to cache, you can configure EhCache externally with an XML
> configuration file to tell it to not cache (set objects in memory to 1 or
> 0). It's a really flexible approach that definitely has lots of untapped
> power.
>
> What I'm going to be working on today is making sure it'll run no matter
> what. Currently, it's working great when run as a server via one of the
> startup scripts, and when using "ant server" to compile and start the
> server. It's *not* working when using the Output classes to serialize
> objects outside of Red5, but I have a few tricks up my sleeve to make it
> "just work."
>
>
> On Fri, Jun 12, 2009 at 6:44 AM, Walter Tak <walter at waltertak.com> wrote:
>
>> Isn't the caching meant to be able to repeat the message to all connected
>> clients, say 50, without the need to re-encode an in memory object 50 times
>> per individual send ?
>>
>>
>> I hate to play devils advocate here, but does caching even make sense
>>> there? In what situation would an identical AMF message be repeated a
>>> significant amount of times? And how high is the cost of encoding the
>>> string compared to finding the string in a large cache?
>>>
>>>
>>>
>>> Tyler Kocheran wrote:
>>>
>>>> That one isn't working the way I want. For using the Output class in
>>>> external applications (such as RTMPClient), the last patch throws
>>>> Exceptions since it can't really find the ehcache.xml file. I'll address
>>>> this tomorrow and post the patch. Sorry for so many posts, I'm on a roll
>>>> here :)
>>>>
>>>> - TK
>>>>
>>>
>>
>> --
>> I am using the free version of SPAMfighter.
>> We are a community of 6 million users fighting spam.
>> SPAMfighter has removed 659 of my spam emails to date.
>> Get the free SPAMfighter here: http://www.spamfighter.com/len
>>
>> The Professional version does not have this message
>>
>>
>>
>>
>> _______________________________________________
>> Red5 mailing list
>> Red5 at osflash.org
>> http://osflash.org/mailman/listinfo/red5_osflash.org
>>
>
>
>
> --
> And do this, knowing the time, that now it is high time to awake out of
> sleep;
> for now our salvation is nearer than when we first believed.
>
--
And do this, knowing the time, that now it is high time to awake out of
sleep;
for now our salvation is nearer than when we first believed.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://osflash.org/pipermail/red5_osflash.org/attachments/20090612/41522a1d/attachment-0001.html>
-------------- next part --------------
### Eclipse Workspace Patch 1.0
#P red5_server
Index: src/org/red5/io/amf/Output.java
===================================================================
--- src/org/red5/io/amf/Output.java (revision 3662)
+++ src/org/red5/io/amf/Output.java (working copy)
@@ -20,6 +20,7 @@
*/
+import java.io.File;
import java.lang.reflect.Array;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
@@ -30,9 +31,12 @@
import java.util.Map;
import java.util.Set;
import java.util.TimeZone;
-import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.ConcurrentMap;
+import net.sf.ehcache.Cache;
+import net.sf.ehcache.CacheException;
+import net.sf.ehcache.CacheManager;
+import net.sf.ehcache.Element;
+
import org.apache.commons.beanutils.BeanMap;
import org.apache.mina.core.buffer.IoBuffer;
import org.red5.annotations.Anonymous;
@@ -58,12 +62,49 @@
protected static Logger log = LoggerFactory.getLogger(Output.class);
/**
- * Cache encoded strings.
+ * Cache encoded strings... the TK way...
*/
- protected static final ConcurrentMap<String, byte[]> stringCache = new ConcurrentHashMap<String, byte[]>();
- protected static final Map<Class<?>, Map<String, Boolean>> serializeCache = new HashMap<Class<?>, Map<String, Boolean>>();
- protected static final Map<Class<?>, Map<String, Field>> fieldCache = new HashMap<Class<?>, Map<String, Field>>();
- protected static final Map<Class<?>, Map<String, Method>> getterCache = new HashMap<Class<?>, Map<String, Method>>();
+ private static Cache stringCache;
+
+ private static Cache serializeCache;
+
+ private static Cache fieldCache;
+
+ private static Cache getterCache;
+
+ private static CacheManager cacheManager;
+
+ private static CacheManager getCacheManager() {
+ if (cacheManager == null) {
+ if (System.getProperty("red5.root") != null) {
+// we're running Red5 as a server.
+ try {
+ cacheManager = new CacheManager(System.getProperty("red5.root") + File.separator + "conf"
+ + File.separator + "ehcache.xml");
+ } catch (CacheException e) {
+ cacheManager = constructDefault();
+ }
+ } else {
+// not a server, maybe running tests?
+ cacheManager = constructDefault();
+ }
+ }
+
+ return cacheManager;
+ }
+
+ private static CacheManager constructDefault() {
+ CacheManager manager = new CacheManager();
+ if(!manager.cacheExists("org.red5.io.amf.Output.stringCache"))
+ manager.addCache("org.red5.io.amf.Output.stringCache");
+ if(!manager.cacheExists("org.red5.io.amf.Output.getterCache"))
+ manager.addCache("org.red5.io.amf.Output.getterCache");
+ if (!manager.cacheExists("org.red5.io.amf.Output.fieldCache"))
+ manager.addCache("org.red5.io.amf.Output.fieldCache");
+ if (!manager.cacheExists("org.red5.io.amf.Output.serializeCache"))
+ manager.addCache("org.red5.io.amf.Output.serializeCache");
+ return manager;
+ }
/**
* Output buffer
@@ -322,15 +363,19 @@
buf.put(AMF.TYPE_END_OF_OBJECT);
}
+ @SuppressWarnings("unchecked")
protected boolean serializeField(Serializer serializer, Class<?> objectClass, String keyName, Field field, Method getter) {
- Map<String, Boolean> serializeMap = serializeCache.get(objectClass);
+// to prevent, NullPointerExceptions, get the element first and check if it's null.
+ Element element = getSerializeCache().get(objectClass);
+ Map<String, Boolean> serializeMap = (element == null ? null : (Map<String,Boolean>)element.getObjectValue());
+
if (serializeMap == null) {
serializeMap = new HashMap<String, Boolean>();
- serializeCache.put(objectClass, serializeMap);
+ getSerializeCache().put(new Element(objectClass, serializeMap));
}
Boolean serialize;
- if (serializeCache.containsKey(keyName)) {
+ if (getSerializeCache().isKeyInCache(keyName)) {
serialize = serializeMap.get(keyName);
} else {
serialize = serializer.serializeField(keyName, field, getter);
@@ -339,12 +384,15 @@
return serialize;
}
-
+
+ @SuppressWarnings("unchecked")
protected Field getField(Class<?> objectClass, String keyName) {
- Map<String, Field> fieldMap = fieldCache.get(objectClass);
+// again, to prevent null pointers, check if the element exists first.
+ Element element = getFieldCache().get(objectClass);
+ Map<String, Field> fieldMap = (element == null ? null : (Map<String,Field>)element.getObjectValue());
if (fieldMap == null) {
fieldMap = new HashMap<String, Field>();
- fieldCache.put(objectClass, fieldMap);
+ getFieldCache().put(new Element(objectClass, fieldMap));
}
Field field = null;
@@ -366,13 +414,15 @@
return field;
}
-
+
+ @SuppressWarnings("unchecked")
protected Method getGetter(Class<?> objectClass, String keyName) {
-
- Map<String, Method> getterMap = getterCache.get(objectClass);
+// check element to prevent null pointer
+ Element element = getGetterCache().get(objectClass);
+ Map<String, Method> getterMap = (element == null ? null : (Map<String, Method>)element.getObjectValue());
if (getterMap == null) {
getterMap = new HashMap<String, Method>();
- getterCache.put(objectClass, getterMap);
+ getGetterCache().put(new Element(objectClass, getterMap));
}
Method getter = null;
@@ -497,12 +547,13 @@
* @return encoded string
*/
protected static byte[] encodeString(String string) {
- byte[] encoded = stringCache.get(string);
+ Element element = getStringCache().get(string);
+ byte[] encoded = (element == null ? null : (byte[])element.getObjectValue());
if (encoded == null) {
ByteBuffer buf = AMF.CHARSET.encode(string);
encoded = new byte[buf.limit()];
buf.get(encoded);
- stringCache.put(string, encoded);
+ getStringCache().put(new Element(string, encoded));
}
return encoded;
}
@@ -552,4 +603,36 @@
clearReferences();
}
+ protected static Cache getStringCache() {
+ if (stringCache == null) {
+ stringCache = getCacheManager().getCache("org.red5.io.amf.Output.stringCache");
+ }
+
+
+ return stringCache;
+ }
+
+ protected static Cache getSerializeCache() {
+ if (serializeCache == null) {
+ serializeCache = getCacheManager().getCache("org.red5.io.amf.Output.serializeCache");
+ }
+
+ return serializeCache;
+ }
+
+ protected static Cache getFieldCache() {
+ if (fieldCache == null) {
+ fieldCache = getCacheManager().getCache("org.red5.io.amf.Output.fieldCache");
+ }
+
+ return fieldCache;
+ }
+
+ protected static Cache getGetterCache() {
+ if (getterCache == null) {
+ getterCache = getCacheManager().getCache("org.red5.io.amf.Output.getterCache");
+ }
+
+ return getterCache;
+ }
}
\ No newline at end of file
Index: src/conf/ehcache.xml
===================================================================
--- src/conf/ehcache.xml (revision 3606)
+++ src/conf/ehcache.xml (working copy)
@@ -12,7 +12,7 @@
Subdirectories can be specified below the property e.g. java.io.tmpdir/one
-->
- <diskStore path="java.io.tmpdir"/>
+ <diskStore path="java.io.tmpdir/red5"/>
<!--
Specifies a CacheManagerEventListenerFactory, be used to create a CacheManagerPeerProvider,
@@ -285,4 +285,35 @@
diskExpiryThreadIntervalSeconds="120"
memoryStoreEvictionPolicy="LFU"
/>
+
+ <!-- org.red5.io.amf.Output.stringCache
+ Caches strings for output via AMF serialization. Keeps up to 1000 elements in memory,
+ does not overflow to disk, is not eternal, and expires elements after 20 minutes of
+ inactivity. -->
+ <cache name="org.red5.io.amf.Output.stringCache"
+ maxElementsInMemory="1000"
+ eternal="false"
+ timeToIdleSeconds="1200"
+ overflowToDisk="false"/>
+
+ <!-- org.red5.io.amf.Output.serializeCache
+ -->
+ <cache name="org.red5.io.amf.Output.serializeCache"
+ maxElementsInMemory="200"
+ eternal="false"
+ timeToIdleSeconds="1200"
+ overflowToDisk="false"/>
+
+ <cache name="org.red5.io.amf.Output.fieldCache"
+ maxElementsInMemory="200"
+ eternal="false"
+ timeToIdleSeconds="1200"
+ overflowToDisk="false"/>
+
+ <cache name="org.red5.io.amf.Output.getterCache"
+ maxElementsInMemory="200"
+ eternal="false"
+ timeToIdleSeconds="1200"
+ overflowToDisk="false"/>
+
</ehcache>
Index: src/org/red5/io/amf3/Output.java
===================================================================
--- src/org/red5/io/amf3/Output.java (revision 3662)
+++ src/org/red5/io/amf3/Output.java (working copy)
@@ -29,6 +29,8 @@
import java.util.Map;
import java.util.Set;
+import net.sf.ehcache.Element;
+
import org.apache.commons.beanutils.BeanMap;
import org.apache.mina.core.buffer.IoBuffer;
import org.red5.annotations.Anonymous;
@@ -142,12 +144,13 @@
}
protected static byte[] encodeString(String string) {
- byte[] encoded = stringCache.get(string);
+ Element element = getStringCache().get(string);
+ byte[] encoded = (element == null ? null : (byte[])element.getObjectValue());
if (encoded == null) {
ByteBuffer buf = AMF3.CHARSET.encode(string);
encoded = new byte[buf.limit()];
buf.get(encoded);
- stringCache.put(string, encoded);
+ getStringCache().put(new Element(string, encoded));
}
return encoded;
}
More information about the Red5
mailing list