Database management system choices — overview
This is the first in a 5-part series of posts on data management product choices. By pre-arrangement, Mike Stonebraker is responding on The Database Column, starting with his own taxonomy of DBMS types.
In the 1990s, most database management experts believed that a single general-purpose DBMS could meet substantially all needs. If you just kept adding in enough datatypes and data access methods (e.g., specialized indexes), your DBMS could eventually do a good job of meeting almost any requirement. And so, from the late 1990s into the beginning of this decade, it seemed that technology was supporting business trends, and the DBMS industry was inexorably consolidating. There was an oligopoly of high-end vendors, who sold increasingly similar super-sophisticated database management systems. Nothing else in database management seemed to matter.
Well, we were wrong. The big thing we overlooked is that database optimizations go down to the level of actual storage. It makes a huge difference how you arrange the data, and even what kinds of devices you store it on. High-end data warehouses run best on shared-nothing massively multi-parallel (MPP) systems. Smaller ones may in the future do best with solid-state disks. Classic online transaction processing (OLTP) systems still do well in a shared-everything architecture.* And that’s just for the relational systems; some kinds of data shouldn’t be arranged in rows and columns at all.
*Even that may be over-generous to traditional shared-everything. Oracle RAC, high availability wide-area replication, and the H-Store research project all suggest that shared-everything’s dominance of high-end OLTP is at road’s end.
The plot thickens further. Most of these technical categories are populated by small companies, with relatively immature products – and in immaturity there is diversity. Thus, there is a broad range of viable data management products, each the best choice in at least some specific application and deployment scenarios.
Recently, one more alternative has emerged – create your own DBMS, or don’t use one at all. There also are half-and-half solutions, in which (commonly) MySQL is used to manage a variety of metadata, but media files might be left just in the file system. This underlies much of the buzz around Amazon and Google services or technologies such as EC2, SimpleDB, and MapReduce. But in most cases, especially for enterprise uses, the best way to go is with a DBMS, or else some DBMS-like data management technology such as a search engine or complex event/stream processing tool.
The database diversity series so far
- Part 1: Database management system choices – overview
- Part 2: Database management system choices – 4 categories of relational
- Part 3: Database management system choices – relational data warehouse
- Part 4: Database management system choices – mid-range-relational
- Part 5: Database management system choices – beyond relational
- Mike Stonebraker’s first response
- My first rejoinder
- Mike Stonebraker’s second response
- My second rejoinder
- My first post on H-Store
- My follow-up post on H-Store and its assumptions
- My ZDnet guest post on H-Store and its timing
- My most recent 11-category data management technology taxonomy
Comments
14 Responses to “Database management system choices — overview”
Leave a Reply
Hi Curt,
I noticed you mentioned H-Store a couple of times in your “Database
management system choices” series. Your readers might want to find out more about this project. The H-Store Website can be found at:
http://db.cs.yale.edu/hstore/
Thanks, Daniel. I have a post coming on the subject of H-Store. Links will be edited in accordingly. 🙂
CAM
Or if you’re available to talk, I’d be happy to talk with you first and THEN write the post. So far I’ve only discussed the project with Mike.
CAM
Thanks. H-Store post is now up, with links inserted into older posts as appropriate.
http://www.dbms2.com/2008/02/18/mike-stonebraker-calls-for-the-complete-destruction-of-the-old-dbms-order/
Talk with you soon!
CAM
[…] Where can I learn more? Academically, H-Store is described in a paper and slide presentation. I examined H-Store and its assumptions at some length over on DBMS2, where you’ll also find an extensive analysis of other specialized database technologies. The H-Store team writes for the Database Column blog, and H-Store discussion is expected over there soon. Also related is a series of posts Stonebraker and I have done on general database diversity. […]
[…] l’articolo di Diamond Notes. Se siete indecisi nella scelta del DBMS adatto a voi un post di DBMS2 potrebbe esservi molto utile. Gli ultimi movimenti di SUN vi lasciano perplessi? Non sapete se la […]
could u publish some few research projects done in dbms
probably with ms access
James,
I’m not sure what you’re asking.
CAM
[…] This is the fourth of a five-part series on database management system choices. For the first post in the series, please click here. […]
[…] Part 1: Database management system choices – overview […]
[…] Part 1: Database management system choices – overview […]
[…] This is the fifth of a five-part series on database management system choices. For the first post in the series, please click here. […]
[…] This is the third of a five-part series on database management system choices. For the first post in the series, please click here. […]
[…] This is the second of a five-part series on database management system choices. For the first post in the series, please click here. […]