Storage
Analysis of storage technologies, especially in the context of database management. Related subjects include:
How will SSDs get incorporated into data warehousing?
SSDs (Solid-State Drives) have gotten a lot of recent attention as an eventual replacement for spinning disk. I haven’t researched expected timelines in detail, but George Crump offered a plausible scenario recently in a highly visible Information Week blog post. After the great recent (and still ongoing!) discussion in the SAN vs. DAS comment thread, I’d like to throw some questions out for discussion, including:
- Just how much faster than disk will SSDs be than disk for random reads?
- Will SSDs be faster or slow than disk for sequential reads, and by how much?
- What will the speed comparison be on SSDs between sequential and random reads?
- How many times will it be possible to write to an SSD? Will this be a problem?
- Will DBMS — which today invariably assume that storage is homogeneous — need to take account of storage heterogeneity?
- What are the implications of SSDs for database and DBMS architecture?
I commented on some of these issues a year ago. Now it’s your turn. 🙂
| Categories: Data warehousing, Solid-state memory, Storage | 5 Comments |
SANs vs. DAS in MPP data warehousing
Generally speaking:
- SANs (Storage Area Networks) are pulling ahead of DAS (Direct Attached Storage).
- Much of the growth in storage is due to data warehousing.
- MPP (Massively Parallel Processing) is pulling ahead of SMP (Symmetric MultiProcessing) for high-end data warehousing.
- MPP architectures are commonly shared-nothing.
- Shared-nothing entails DAS.
But if you think about it, those facts don’t exactly add up. Read more
| Categories: Calpont, Parallelization, Storage, Vertica Systems | 24 Comments |
Head to head blog debate between EMC, NetApp, and HP
Chuck Hollis of EMC started a fierce debate with a blog post on how to measure effective storage capacity. Competitors from NetApp and HP responded in often sarcastic detail in the comment thread, Hollis shot back, and the volleying continued for quite a while.
I’m not a storage maven, and I don’t understand all the details of that stuff. If you’re like me in that regard, you may find the post worth skimming just to see what some of the choices, trade-offs, and complications are in designing and measuring storage systems. Stephen Foskett’s related post is also worth a look in that regard.
My recent foray into measuring disk storage pales by comparison.
| Categories: Storage, Theory and architecture | 3 Comments |
Database management system architecture implications of an eventual move to solid-state memory
I’ve pointed out in the past that solid-state/Flash memory could be a good alternative to hard disks in PCs and enterprise systems alike. Well, when that happy day arrives, what will be some of the implications for database management software architecture?
- Compression will be even more important. Cost per terabyte of storage will spike up for that storage that is moved from disk to solid-state.
- The sequential-rather-than-random reading strategy of data warehouse appliance makers may become less relevant. The one way to get rid of the disk-speed bottleneck is to get rid of disks.
- DBMS will need to write data as rarely as possible. Solid-state memory tends to wear out if you keep writing over it. Assuming this problem gets better over time (if it doesn’t, this whole discussion is moot) but isn’t totally solved, architectures which have fewer writes are on the whole better.
| Categories: Data warehouse appliances, Data warehousing, Database compression, Netezza, Solid-state memory, Theory and architecture | Leave a Comment |
Solid state (Flash) memory vs. RAM vs. disks
I just wrote a column and a blog post on the potential for diskless PCs based on flash drives. It was a fun exercise, and I think I kept it general enough that my lack of knowledge about hardware technology details didn’t lead me into significant error.
The first vendor response I got was from Bit Micro Networks, who seem to sell such drives for PCs and enterprise storage alike. One of their press releases touts an Oracle implementation. Interesting idea. It’s far from a substitute for full memory-centric data management, but it’s kind of an intermediate way of getting some of the benefits without altering your traditional software setup much at all.
| Categories: Memory-centric data management, Oracle, Solid-state memory | 1 Comment |