Multiple approaches to memory-centric analytics
Memory-centric analytic processing is in the spotlight.
- Microsoft’s big analytics announcement for the week (one of them, anyway), is “Gemini,” which evidently amounts to some kind of in-memory, cube-based analytics, but with columns rather than true cubes as the in-memory data structure.
- That sounds at lot like SAP’s BI Accelerator, which is a way to manifest SAP InfoCubes in-memory in a columnar architecture.
- QlikTech is going gangbusters with memory-centric business intelligence.
- IBM/Cognos’ Applix, which has a rather unique approach to memory-centric cubes, has never lived up to its potential. But now people are being reminded it exists.
- Exasol has made some sales with a highly memory-centric approach to data warehousing. Kognitio’s story is somewhat disk/RAM hybrid (disk is certainly involved, but the best parts of the technology deal with what happens once the data gets into RAM).
- Most of what the CEP (Complex Event Processing, aka event/stream processing) industry does is memory-centric analytics, both via tight integration with operational apps seems and for conventional BI.
So what about Oracle? In his excellent article on yesterday’s Microsoft announcements, Doug Henschen ties Oracle’s future in the area to its TimesTen in-memory DBMS. I don’t agree. More relevant might be Oracle’s new feature of materialized views in the form of OLAP cubes, which may or may not live in cache much of the time just as other materialized views might.
And that’s just the query-centric part. Actual parallel analytics — from in-database data mining or spatial analysis primitives all the way to full in-memory MapReduce, — is a whole other set of subjects. The point is to do at least somewhat sophisticated calculations quickly, and do to so in parallel when that’s desirable or necessary. Teradata, Netezza, Greenplum, Aster Data, and Oracle have stories in this regard, as does Pervasive with Datarush.
I’m not so sure about Microsoft, however. Microsoft has lots of research investments and so on that sound highly parallel, but in its shipping products Microsoft appears to be a parallel analytics laggard.
Comments
3 Responses to “Multiple approaches to memory-centric analytics”
Leave a Reply
I think HP Neoview does much of its mpp in an in-memory style, am I right?
Luis,
I don’t know of any big differences for Neoview in that regard vs. Netezza, Greenplum, Oracle Exadata, whoever.
Are you basing this on the claim “Teradata materializes intermediate result sets to disk but some other systems have better pipelining?”, or are you talking about something else?
CAM
IBM has had in-database data mining for a while now as part of InfoSphere Warehouse, so you can count us into your “parallel analytics” camp.
We can’t rely on all data being present in main memory when we create the mining models – “memory-centric” analytics would actually be a bad idea here. But when applying your model to new data, this model better be in memory to achieve the performance requirements for scoring new, unseen records.