In-memory, (hybrid) memory-centric DBMS — three analytic glossary draft entries
These are three closely-related draft entries for the DBMS2 analytic glossary. Please comment with any ideas you have for their improvement!
1. We coined the term memory-centric data management to comprise several kinds of technology that manage data in RAM (Random Access Memory), including:
- In-memory DBMS (DataBase Management Systems).
- Hybrid memory-centric DBMS.
- Other kinds of in-memory data stores, such as:
- Caching layers.
- In-memory data stores that are tightly tied to specific analytic tools, for example the in-memory data management part of QlikView.
- Complex event/stream processing.
Related link
- Many examples of memory-centric data management (April, 2012)
2. An in-memory DBMS is a DBMS designed under the assumption that substantially all database operations will be performed in RAM (Random Access Memory). Thus, in-memory DBMS form a subcategory of memory-centric data management systems.
Ways in which in-memory DBMS are commonly different from those that query and update persistent storage include:
- Data access processes which include a larger number of individually cheaper steps. In-memory database access is orders of magnitude cheaper than disk access, so it’s not as important to minimize the number of accesses.
- Reduced locking. At RAM speeds, the cost of database locks can be significant, so in-memory DBMS are designed to minimize their use.
If what otherwise appears to be an in-memory DBMS routinely queries data from disk, then we refer to it as being hybrid memory-centric. However, even true in-memory DBMS may copy data into persistent storage, so as to keep it safe.
Examples of in-memory DBMS include:
- SAP HANA.
- Oracle TimesTen.
- IBM TM1.
- Several NewSQL systems, such as VoltDB.
- Several NoSQL systems, such as Citrusleaf.
3. Hybrid memory-centric DBMS is our term for a DBMS that has two modes:
- In-memory.
- Querying and updating (or loading into) persistent storage.
It is difficult to make the boundaries of this category precise, because:
- Almost any DBMS runs faster when the entire database is kept in RAM (Random Access Memory).
- For some DBMS, especially in high-volume short-request processing use cases, it is a best practice to keep one’s entire data working set in RAM.
That said, we prefer to reserve the term “hybrid memory-centric” for DBMS designed according to the same principles as in-memory DBMS, for example IBM solidDB.
Hybrid memory-centric DBMS form a subcategory of memory-centric data management systems.
Comments
7 Responses to “In-memory, (hybrid) memory-centric DBMS — three analytic glossary draft entries”
Leave a Reply
I wonder if the key here is that the DBMS explicitly manages data in memory as opposed to implicit management via buffers of via the OS? Explicit management might include some of the concepts you suggest like reduced locking or like techniques to increase utilization of the CPU caches.
Question: If Postgres was disk-based but added explicit management of some data in-memory does it become a hybrid? If HANA added a disk-based data management layer would it become a hybrid?
Rob,
If there’s a disk-based layer we should say it becomes a hybrid.
As for explicit vs. implicit memory management — good idea, but I’m concerned about going down the rabbit hole of ascertaining which DBMS does exactly what, and how much benefit it gives them.
All this will be clearer if I also put in an entry for memory mapping, actually. A lot of DBMS that happen to often be used with their working sets in RAM are the ones that get data on and off disk by delegating the problem to the OS via memory mapping. At least, I think that’s how memory mapping works …
Curt,
Thanks for taking the time to develop the glossary. I think this is going to be useful for a long time.
I recommend that at some point, you consider how flash memory is going to alter the in-memory vs. persistent storage tradeoffs. In a few years, all databases might qualify as “hybrid”, and the idea of needing a RAM-centric variant for performance may seem quaint.
Dear all,
I’m working at Altibase Corporation.
If you have a question of In memory DBMS as well as Hybrid DBMS (Memory + Disk DBMS), send us by email.
ALTIBASE Provides fast, reliable, data management systems in all kinds of business critical operating systems.
In your memory…
[…] focus on in-memory database operations; memory-centric systems are not commonly referred to as “data warehouse […]
[…] coined “memory-centric” as an alternative. Then I relented 1 1/2 years ago, and defined in-memory DBMS as DBMS designed under the assumption that substantially all database operations will be performed […]
[…] all database operations will be performed in RAM (Random Access Memory).” [“In-memory, (hybrid) memory-centric DBMS — three analytic glossary draft entries,” A Monash Research Publication, 20 August 2012] The article […]