Memory-centric data management
Analysis of technologies that manage data entirely or primarily in random-access memory (RAM). Related subjects include:
- Oracle TimesTen
- solidDB
- QlikTech
- SAP‘s BI Accelerator
- Exasol
- Solid-state memory as a replacement for disk
QlikTech numbers update
I chatted with QlikTech again yesterday. The update on their numbers is that they ended 2006 with 5,436 customers in 68 countries. Of those, 3,200 were added over the year. (I.e., they only had 2,200 or so at the end of 2005.) Revenue growth was slightly more than 80% for the year, for the third straight year over 80%. (I think their real goal is to double.) That should put them at $40 million or so in license fees, for classical BI only. (Budgeting/planning features are apparently slated for QlikView Release 8 in May.) Read more
Are row-oriented RDBMS obsolete?
If Mike Stonebraker is to be believed, the era of columnar data stores is upon us.
Whether or not you buy completely into Mike’s claims, there certainly are cool ideas in his latest columnar offering, from startup Vertica Systems. The Vertica corporate site offers little detail, but Mike tells me that the product’s architecture closely resembles that of C-Store, which is described in this November, 2005 paper.
The core ideas behind Vertica’s product are as follows. Read more
Introduction to Kognitio WX-2
Kognitio called me for a briefing this morning on their WX-2 product. Technical highlights included:
- Their core technology is MPP/shared-nothing data warehousing.
- Unlike most other vendors (but like Greenplum), they are available software-only.
- Like DATallegro and Netezza, they have no global indexing.
- Unlike the other MPP players, they don’t hash partition the data and lead with hash joins. Rather, they have local compressed bitmap indices on every node.
- Similarly, they distribute data utterly randomly and have no concept of range partitioning whatsoever.
- Probably for that reason, WX-2 reads data in small 32K blocks. This forfeits the benefit of sequential reads, unless David Aldridge is correct that Linux can take care of that on its own.
- They seem more chip-heavy than DATallegro and Netezza. A dual-core Opteron blade with 16 or 32 gigabytes of RAM talks to 144, 288, or in some cases 600 gigabytes of disk (before mirroring).
- The position themselves somewhat as being a memory-centric product supplier. While I suspect this is exaggerated, it probably indicates that they’ve put some work into managing RAM as well as disk.
Much like the other “new” MPP data warehouse vendors, Kognitio claims to never have knowingly been outbenchmarked, whether on performance or on TCO factors such as ease of installation.
Read more
| Categories: Data warehouse appliances, Data warehousing, Greenplum, Kognitio, Memory-centric data management | 11 Comments |
SAP’s BI Accelerator
I wrote about SAP’s BI Accelerator quite a bit in my white paper on memory-centric data management, but otherwise I seem not to have posted much about it here. In essence, it’s a product that’s all RAM-based, and generally geared for multi-hundred-gigabyte data marts. The basic design is a compression-heavy column-based architecture, evolved from SAP’s text-indexing technology TREX. Like data warehouse appliances, it eschews indexing, relying instead on blazingly fast table scans.
I asked Lothar Schubert of SAP how BIA was doing in the market in its early going. This was his response:
| Categories: Analytic technologies, Business intelligence, Data warehouse appliances, Data warehousing, Database compression, Memory-centric data management, SAP AG | 8 Comments |
Myths about DATallegro, Ingres, open source, etc.
Sometimes, when one talks to a company about a close competitor, what one hears may not be 100% strictly accurate. Yesterday, I more than once heard claims that sounded oddly like “DATallegro has to open source whatever software it develops.” Today, DATallegro CEO Stuart Frost clarified as follows:
• DATallegro has no (little?) legal obligation to open source anything. Even the version of Ingres they use is not the GPL one.
• They do give a few enhancements back to Ingres (via open source?) rather than maintain them themselves.
• The whole MPP technology is proprietary, in every sense of “proprietary.” (For example, they use a whole different optimizer than Ingres’s. I’ve forgotten whether the Ingres optimizer is also left in place.)
| Categories: Actian and Ingres, Data warehouse appliances, DATAllegro, Memory-centric data management, Open source | 1 Comment |
Is data warehousing now all about sequential access?
A lot of evidence is pointing to a major paradigm shift in data warehouse RDBMS, along the lines of:
Old way: Assume I/O is random; lower total execution time by improving selectivity and thus lowering the amount of I/O.
New way: Drive the amount of random I/O to near zero, and do as much sequential I/O as necessary to achieve this goal.
Examples include:
- Data warehouse appliances (see especially this discussion of DATallegro’s architecture)
- Columnar systems (see Nathan Myer’s first comment in this discussion of the much-hyped Required Technologies prototype)
- Memory-centric systems, notably SAP’s BI Accelerator
| Categories: Data warehouse appliances, DATAllegro, Memory-centric data management, SAP AG, Theory and architecture, TransRelational | 4 Comments |
QlikView – a leader in memory-centric BI
QlikTech — the vendor of QlikView — contacted me to tell their memory-centric BI story. A Swedish company with >$23 million in estimated license revenue last year and a 100%ish growth rate, they claim to be the leader in that space, pulling ahead of Applix. But for now, I’ll call them “a” leader, and say that their story sounds like a hybrid between those of Applix (TM1 product) and SAP (BI Accelerator).
| Categories: Business intelligence, Cognos, Memory-centric data management, QlikTech and QlikView, SAP AG | 1 Comment |
ANTs’ memory-centric characteristics to the fore?
An eWeek article suggests that ANTs is repositioning with a strong emphasis on memory-centricity. ANTs’ website, frankly, doesn’t support this theory, giving a more balanced tech overview in line with how they pitched me in a briefing last November. Still, it’s an interesting possibility to watch.
The main focus of the article actually wasn’t ANTs, but rather SAP’s wildest dreams in expanding the scope of its BI Accelerator technology. But the new-to-me part was the positioning of ANTs.
| Categories: ANTs Software, Memory-centric data management, SAP AG | 2 Comments |
Solid’s MySQL engine
Solid Information Technology is making the beta of its MySQL engine available for download midday on Tuesday. So I talked with them today, mercifully unembargoed. Here’s the story.
| Categories: Memory-centric data management, Mid-range, MySQL, OLTP, Open source | 4 Comments |
Good DATallegro/Intel white paper
I really like this short white paper, which carries the personal byline of Stuart Frost. Stuart is DATallegro’s CEO, and also the guy who does analyst relations for them (at least in my case). Part of it just does a concise job of spelling out some of the DATallegro story. But the rest is about the comparison between Intel’s new dual-core “Woodcrest” Xeons and their single-core predecessors. Not only does it give credible statistics, it gives understanding of the reasons behind them.