Why you would want an appliance — and when you wouldn’t
Data warehouse appliances are booming. But Hadoop appliances are a non-starter.
Data warehouse and other data management appliances are on the upswing. Oracle is pushing Exadata. Teradata* is going strong, and also recently bought Aster Data. IBM bought Netezza. Greenplum and Vertica were bought by EMC and HP respectively. All those moves are favorable for appliances.
*As far as I’m concerned, all Teradata hardware-included systems are appliances.
In essence, there are two kinds of reasons to prefer appliances over software-only offerings:
- Technology.
- Internal politics.
“Technology” can include performance, price/performance, ease of installation, ease of administration, and so on. I write about those issues a lot. Even more to the point, certain software only runs — or only runs well — in appliance configurations; e.g. Netezza’s, or Teradata’s if you have more than a couple terabytes of data, or Oracle’s if you’re in the double-digit terabyte range.
But it turns out technology isn’t the only reason to like appliances. Sometimes, internal politics hold sway. Suppose central IT is in control of normal computing installations, but departments can buy and control their own appliances? Then those departments might avoid bureaucracy, internal chargeback costs, and all the other things they hate about IT … but only if they go the appliance route. It turns out that this can be a very real consideration.
Of course, similar considerations arise with SaaS (Software as a Service).
But I’ll tell you one area where appliances seem to make little sense — running web-oriented, often scale-out, often open-source data management software. To the extent this software is commonly designed to play nicely with low-end commodity hardware, tailored appliances are a bit of a mismatch. If it is designed to be elastic, appliances are an awkward fit. If users value the flexibility of open source, appliances might get in the way; if users value the low prices of open source, appliances may be even less appropriate. If they’re running their software offsite, appliances may be the opposite of convenient. And if a system is big enough for the data-center-as-computer approach (e.g. Facebook or Google), appliances don’t make sense at all.
Kickfire failed, Schooner pulled back from appliance sales, Clustrix doesn’t seem to be accomplishing as much as its funding and early technical reputation might suggest, and I’m not optimistic for Hadoop appliances either. Appliances aren’t elastic, they aren’t free-as-in-speech, and they certainly aren’t free-as-in-beer. I like Hadoop, and I like appliances; but I don’t see them being good together.
Comments
10 Responses to “Why you would want an appliance — and when you wouldn’t”
Leave a Reply
Commodity hardware is kind of a misnomer. As we have scaled out our Hadoop cluster to 100’s of nodes. We are finding we need to have a very specific configuration to reduce the amount of operational overhead.
Interesting — what’s that configuration like? And is the main point the exact configuration, or is it more that everything have the SAME configuration?
Hardware can be fully commodity.
The issue is the engineering required to get the Hadoop cluster to perform well.
In a balanced system, no one component (CPU, disk, network, memory) is consistently limiting performance. How do you choose components to get a well balanced configuration? Are your engineers going to do this work?
With an appliance, the vendor has done this work and the customer can benefit. Add a support organization that is supporting the Hadoop cluster, not just a bunch of linux server and a Hadoop appliance may be an attractive option. right choice.
I hear you saying that internal politics is much less of a problem in the Hadoop world, and as such technology is the only decision point for Hadoop?
Now what if Hadoop starts getting traction in Financial Services and other industries, wouldn’t that change, and suddenly these guys have a point again?
No argument against the idea that you want support for your Hadoop software.
But Hadoop is rapidly evolving. E.g., I hope and trust that Hive will get a lot more performant, fast. So the details of ideal configurations will change fast. What’s more, different workloads have different ideal configurations anyway. And when I look at DBMS vendors’ appliances, it’s not clear what the big benefit of proprietary hardware is, unless you go to an extreme like Netezza’s FPGA, or Oracle’s refusal to sell you its storage tier software unless you buy Exadata as well. (Even if we grant Teradata as an exception, that would be ONE exception.)
Bottom line: I’m not sure what Greenplum could give you by choosing hardware that you also couldn’t get from asking Cloudera or IBM what it recommends. As a bonus, at least Cloudera would probably recommend something a lot cheaper than Greenplum would.
Marcus,
The political point is that a department might be “allowed” to run its own appliance but not its own other computing system. If IT shops really insist on centralizing Hadoop clusters AND on over-tightly controlling departments’ use of them, there could be a problem. I’m guessing, however, that at most enterprises that particular problem isn’t arising soon.
If I’m wrong, it will probably be because IT shops insist on expensive Hadoop implementations where cheap ones would actually suffice.
[…] suggesting that there’s little point to Hadoop appliances, it occurred to me to look into what kinds of hardware actually are used with Hadoop. So far as I […]
Curt, we’ve had this conversation before, but I still find the entire conversation around “appliance” to be humorous. Appliance, in this context, is a) a pure and meaningless marketing term, and b) a misnomer.
There is room for an appliance in Hadoop world, same way that Netezza was successful in RDBMS world, some business buyers just like that model better. The main selling point would be simplified management, and not having to secure internal approvals from multiple departments. Also, Appliance doesn’t always = proprietary hardware, it can be commodity hardware with a proprietary firmware/software appliancification layer.
— amr
I think you raise some good points Curt. Hadoop does aim to solve a lot of problems in software on relatively low cost hardware. This means buying more commodity hardware is going to be an option for many organizations. So appliance vendors are going to have to make their case to the market.
That said, There is definitely space for organizations to provide value by packaging and selling integrated systems.
One thing I’m certain of is that we are going to see more Hadoop offerings in the next few years, so we will see real test cases of appliance offerings and healthy competition from commodity solutions. I look forward to seeing what buyers select.
E14