Netezza’s chip story
In addition to its software story, Netezza of course has a rather unique chip story. Where other vendors might have standard disk controllers and high-powered microprocessors, Netezza respectively has a FPGA (Field-Programmable Gate Array) and lesser microprocessor (PowerPC). Netezza claims that two major advantages of these choices are:
- 5X throughput/performance improvement
- Much lower heat and power consumption.
The main function of the FPGA, other than generically getting data on and off disk, is to restrict and project tables (i.e., execute single-table WHERE clauses). Netezza claims that their FPGAs can perform these operations on the streaming data at least as quickly as an expensive, hot, power-hungry top-end microprocessor would, and indeed faster. The key word is “streaming”, which they contrast to the microprocessor’s need to get the data in and then back out of RAM (cache or otherwise).
I’ll be interested to see whether somebody can muster a ringing refutation to Netezza’s claims.
Comments
12 Responses to “Netezza’s chip story”
Leave a Reply
[…] There’s another cool-sounding part to the Netezza story, which straddles their chips and their software: The FPGA takes over the work of assuring database consistency. If the system attempts to read and write a record at the same time, the FPGA keeps thing straight. This eliminates the need for locks — at least if you don’t care about transactional integrity — and some of the reason for logs. (I guess that in lieu of any kind of rollback/rollforward they just rely on failover to mirrored disks.) […]
Curt,
According to your post, Netezza claims two major advantages from their use of FPGAs: 5X performance and lower heat and power consumption. Both are very easy to refute.
On performance, our latest version pretty much maxes out the disks at around 70MBps each. This is when running queries, not just simple table scans. Based on recent proof of concept projects, we’ve seen that Netezza does roughly the same. However, our performance is achieved through the use of smart software on commodity CPUs, rather than their combo of a CPU and an FPGA.
IMHO, our performance is even more impressive when you consider that we have just two dual-core CPUs for every twelve disks. In contrast, Netezza uses one CPU and one FPGA for every disk. Hence, our CPU:disk ratio is 1:3 compared with Netezza’s 2:1.
This goes a long way in explaining our price advantage – our CPU cost per disk is much lower and this feeds directly into a significant advantage on both price/performance and price/capacity.
It also leads to a refutation of their supposed heat/power advantage. A single FPGA certainly does use less power than a dual core Xeon CPU. However, a typical Netezza installation uses far more chips for a given performance and capacity level.
To be specific, a Netezza 8650z has 33TB capacity and uses 672 SPUs with one Power CPU, an FPGA and one disk each. That’s a total of 1,344 ‘CPUs’ and 672 disks. In contrast, a DATAllegro P33 has 33TB capacity and uses 66 nodes with 2 dual core CPUs and 12 disks each for a total of 132 dual core CPUs and 792 disks.
With a similar number of disks being scanned at a similar rate, the two systems will have similar performance (until we take into account our sophisticated multi-level partitioning, which gives us a huge performance advantage on most queries). However, our system has less than one tenth the number of CPUs. I don’t know how much power Netezza’s FPGAs use, but I doubt it’s less than a tenth of that used by one Xeon.
Our P33 is also less than half the price.
Stuart
Curt,
Just read another article that put Netezza’s power consumption at around 30W per SPU. Since each SPU has one disk, that equates to 30W per disk.
Our nodes use around 420W each and have 12 disks, so that’s 35W per disk. Our P Series gets slightly more usable space per disk than Netezza, so the consumption per TB is about the same.
If we compare to our C series, we get around four times the capacity per disk, so power consumption per TB is much lower.
Stuart
DATAllegro
[…] Philip Howard has published a write-up based on Netezza’s user conference, entertainly mixing fantasy and reality in his usual manner. Notably, he confuses Netezza’s zone maps, which are basically a very limited form of range partitioning, with something that can substitute for real indices. And the mind boggles at his implication that Netezza has neglected the FPGA in its overall market messaging. More understandable is his regurgitation of Netezza’s claims about heat and power, but although I must confess to not having checked either side’s arithmetic, I find Stuart Frost’s rebuttal in the comments to this thread pretty interesting. […]
[…] As for the hardware/platform side, it’s similar to what I’m hearing from other appliance vendors (Netezza and their emphasis on an FPGA are a real outlier). Namely: […]
[…] Netezza is Type 0 all the way. The product, if not the company, is built around its FPGA. […]
I’ve seen many articles like this where Stuart tries to position Datallegro over Netezza, the problem is Datallegro has hardly any customers on board, at least hardly any willing to talk about their success publicly, compare that to the momentum Netezza has in the marketplace, that speaks volumes to me !
Up until the second half of last year, I used to complain that Netezza would talk to me about its customers but not its technology, and DATAllegro would talk about its technology but not its customers.
They still both have those orientations, but neither is nearly as extreme any more as they were.
CAM
At one of Netezza’s User Conference technical breakout sessions this week they basically opened the kimona.
They discussed in great detail with animated slides the technology utilized starting from the user query to
the front end software optimization and scheduling, how the query is distributed & executed on the SPUs, how
data is manipulated, and ultimately sent back to the host and to the user for the answer to their query.
Also several of their customers gave presentations at their business track sessions disclosing their DW
deployment and reported performance gains of 2 and 3 orders of magnitude. So something is missing in the
technical analysis just comparing disk data rates which on average cannot go faster than 70 MB/s. Any clues??
Hiya.
Lots of clues elsewhere in this blog. Index-light MPP architectures such as Netezza’s or DATAllegro’s are waaaay better for large data warehouses than traditional index-heavy SMP systems like Oracle or Microsoft SQL*Server.
More debatable is the choice between FPGAs (Netezza) and more conventional chips (pretty much everybody else). Both choices have a lot of merit.
CAM
[…] Netezza всю дорогу является Типом 0. Продукт, если даже не компания, построен вокруг их FPGA. […]
[…] for core SQL processing, after the data is ingested via conventional I/O. This is different from Netezza’s approach to FPGA-based data warehouse appliances, in which the FPGA sits in the place of a disk controller […]