FlexStore and the rest of Vertica 3.5
Today, Vertica is announcing its 3.5 release, timed in line with a TDWI conference. Vertica 3.5 is scheduled to go into beta test in mid-August and be released to general availability in early October. Vertica 3.5 highlights include:
- Vertica/MapReduce integration, which I’m covering in a separate post.
- A new storage architecture called Vertica FlexStore, which seems to boil down essentially to three things:
- A sort of row/column hybridization — Vertica would probably prefer to call it something like a column clustering feature — that I’m also covering in a separate post.
- The beginnings of a multi-temperature capability, somewhat akin to Teradata Virtual Storage.
- Enhancements to Vertica’s WOS (Write-Optimized Store, the in-memory part of Vertica that first receives updates). I don’t understand WOS architecture well enough to write about that yet.
- Load-balancing, to route queries evenly among Vertica nodes — probably just round-robin — rather than having them just be processed by whichever node happens to receive them.
And Vertica 3.5 surely includes some lesser features as well.
Like Teradata Virtual Storage, Vertica FlexStore wants to partition data among different parts of storage automatically. But, unlike Teradata’s technology, it will grudgingly let you do it by hand if you insist. Since Vertica installations seem to generally have only one kind of disk each — and that kind spinning rather than solid-state — early tests concentrate on allocating data among the inner and outer tracks of a disk. Vertica said that one typically gets 80% of the benefit by dividing the data into just two partitions, and 90% of the benefit if one divides it into three. However, I don’t recall getting a clear estimate of just how large that benefit is.
Comments
11 Responses to “FlexStore and the rest of Vertica 3.5”
Leave a Reply
[…] Vertica 3.5 introduces what Vertica calls “FlexStore.” A key part of FlexStore is the ability to store data not just in pure columnar format, but also to group columns together in what amounts to sub-rows. This is advantageous when data is retrieved together and, I presume, when it is updated. There’s a tradeoff in giving up column stores’ compression advantages, however, and use of this feature is not recommended for columns that are frequently retrieved independently. Vertica also notes that since it typically uses 1 megabyte block sizes, any table smaller than that shouldn’t be broken into columns at all. […]
[…] with Omer Trajman of Vertica Monday night about Vertica’s MapReduce integration, part of its Vertica 3.5 release. Highlights […]
Well, I AM puzzled. It was stonebraker who critisized MapReduce a while ago and went feature by feature comparing Hadoop MapReduce implementation with DBMS … and now comes full circle to adopt MapReduce into a DB … WOW. Interesting. But in a way it is good because when MapReduce has been successfully implemented by Aster and Greenplum who are Row based, it was a matter of time before some column based jumped into using this trick. I expected ParAccel…but it is Vertica. Great job vertica team !
For starters, Stonebraker != Abadi != Vertica. Beyond that, we’re talking about a bunch of very smart men and women, and if evidence leads them to tweak their opinions, then they’re open to doing so.
Thanks Balaji – we’ve been getting lots of customer interest in MR and specifically hadoop. It’s a great community to be involved with.
@Balaji – That’s true. Good that you pointed it out. They were so vocal against MapReduce and a month later they have this 🙂
This is all very confusing. Daniel Abadi works on VoltDB. Isn’t he also the same guy who is working on HadoopDB – MapReduce?
Column Grouping.
Bigtable’s column family is a good abstraction.
For starters, Stonebraker != Abadi != Vertica. Beyond that, we’re talking about a bunch of very smart men and women, and if evidence leads them to tweak their opinions, then they’re open to doing so.
Yes Stonebreaker is entitled to change his opinion. The problem is that he spent a great deal of time and energy bashing, and invalidating the other opinions before he changed his own. That makes him a hypocrit and you an apologist for his hypocrisy
“Bill”,
Unless you’re personally acquainted with some or all of Mike Stonebraker, Dave DeWitt, Daniel Abadi, and Vertica execs — and unless you’ve talked about MapReduce with some of them — you don’t know what the hell you’re talking about.
[…] http://www.dbms2.com/2009/08/04/flexstore-and-the-rest-of-vertica-35/ […]