[osflash] Object Databases

[Evert | Rooftop]

Hi João,

An ODBMS will not solve your problems. 900M is quite a bit of records,  
so I can see that you'll quickly run into problems there. I would  
recommend approaching this problem with the usual optimization tricks.

Start off with your requirements for the reporting you'll be needing  
and work your way down. A simple example.

Your 900.000.000 records will require time-based reports and a  
granularity of 5 minutes is required. You receive 500 records every  
second on average. This will effectively reduce your entire dataset to  
from 900M to 6000 records. Calculate these summary tables every 5  
minutes, and your reports should be delivered under 20ms.

In practice you'll end up creating separate summary tables for every  
field you reference in a GROUP BY statement.

If your dataset is much more complex than simple statistics, looking  
into a map-reduce type scheme like Hadoop.

I hope this helps,
Evert

P.s.: recommended reading material:
http://oreilly.com/catalog/9780596101718/

On 28-Oct-08, at 12:18 PM, João Saleiro wrote:

> Hello,
>
> I am researching on Object Databases to integrate with our RIA  
> solutions. We have to build a statistics system to handle complex  
> databases that can have tables with more than  900.000.000 records.  
> The system needs to be VERY efficient (i.e. return the result of a  
> calculation in less than 4 seconds). We are curious about Object  
> Databases, especially the Versant one. We have read all their  
> "marketing" material, but I would like to prefer the opinion of  
> someone who has used ODBMS. Does someone here has experience with  
> this kind of solutions?
>
> Thanks,
>
> João Saleiro
>
> CTO @ Webfuel
> Adobe Community Expert
> Co-founder of the Portuguese RIA Community
>
> <btn_profile_greytxt_80x15.gif>
> _______________________________________________
> osflash mailing list
> osflash at osflash.org
> http://osflash.org/mailman/listinfo/osflash_osflash.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2433 bytes
Desc: not available
URL: <http://osflash.org/pipermail/osflash_osflash.org/attachments/20081028/d99e4bd9/attachment.bin>