WibiData and its Kiji technology
My clients at WibiData:
- Think they’re an application software company …
- … but actually are talking about what I call analytic application subsystems.
- Haven’t announced or shipped any of those either …
- … but will shortly.
- Have meanwhile shipped some cool enabling technology.
- Name their products after sushi restaurants.
Yeah, I like these guys. 🙂
If you’re building an application that “obviously” calls for a NoSQL database, and which has a strong predictive modeling aspect, then WibiData has thought more cleverly about what you need than most vendors I can think of. More precisely, WibiData has thought cleverly about your data management, movement, crunching, serving, and integration. For pure modeling sophistication, you should look elsewhere — but WibiData will gladly integrate with or execute those models for you.
WibiData’s enabling technology, now called Kiji, is a collection of modules, libraries, and so on — think Spring — running over Hadoop/HBase. Except for some newfound modularity, it is much like what I described at the time of WibiData’s launch or what WibiData further disclosed a few months later. Key aspects include:
- A way to define schemas in HBase, including ones that change as rapidly as consumer-interaction applications require.
- An analytic framework called “Produce/Gather”, which can execute at human real-time speeds (via its own execution engine) or with higher throughput in batch mode (by invoking Hadoop MapReduce).
- Enough load capabilities, Hive interaction, and so on to get data into the proper structure in Kiji in the first place.
More specifically, 5 of the 6 Kiji modules are:
- KijiSchema, the oldest of the bunch (not that any have been out for long). A “direct wrapper” for HBase, KijiSchema provides HBase with a DDL (Data Description Language), metadata (I haven’t asked details), and serialization (via Avro).
- KijiMR, released in February. KijiMR is the batch part of Kiji data processing — notably MapReduce integration (as you might have guessed from the name) and bulk import.
- KijiScoring,* released very recently. That’s the (human) real-time part of Kiji processing.
- KijiHive, which is — you guessed it! — Kiji’s Hive adapter.
- KijiREST.* I bet you can guess what that does too.
*It’s a roadmap item to “blur” KijiScoring and KijiREST together, and have them run in the same process.
And then there’s the confusing one — KijiExpress, which has something to do with running models. Right now KijiExpress seems to be mainly a Scalding/Cascading interface to the rest of Kiji. However:
- Just about anything else you might imagine that could help with specifying, running or managing analytic models sounds like it’s at least a possibility for eventual inclusion in KijiExpress …
- … but it seems like an exaggeration to say that any particular direction for the project is already on the roadmap.
Finally, on the business side:
- WibiData has very few production customers to date, in each case for a rather custom system.
- WibiData knows what its first application vertical market will be, and will probably tell you under NDA if you’re in that segment and you ask.
- WibiData will gladly support Kiji developer seats in the usual commercial open source way.
- WibiData will not exactly support Kiji runtimes in the usual commercial open source way. Rather, it will support entire applications — or perhaps “applications” — built in/on Kiji. Presumably, WibiData professional services would be involved in building those apps, but that’s not a strict requirement.
- Based on mailing list activity and so on, WibiData believes there are serious Kiji users with whom it has no financial relationship.
- I counted 30 distinct headshot photos on WibiData’s team page, in a 9:1 primate:canine ratio.
Comments
5 Responses to “WibiData and its Kiji technology”
Leave a Reply
Curt,
Thanks for the great overview. We’re pretty excited about how the Kiji project is coming together and the great feedback from early users. All of the components are available in an easy to download and use SDK called the BentoBox. Like everything else in Kiji, it’s Apache licensed and available at http://www.kiji.org. We’re also hosting a one day workshop and hackathon following HBaseCon this year. It’s being sponsored by Cloudera and Opower and folks can sign up at http://kijicon.eventbrite.com.
-Omer
[…] Curt’s recent blog post, “WibiData and its Kiji Technology” he […]
[…] number of my clients are focused on such scenarios, including WibiData, Teradata Aster (e.g. via nPath), Platfora (in the imminent Platfora 3), and others. And so I get […]
[…] In other news, WibiData has had some executive departures as well, but seems to be staying the course on its strategy. I continue to think that WibiData has a really interesting vision about how to do large-data-volume interactive computing, and anybody in that space would do well to talk with them or at least look into the open source projects WibiData sponsors. […]
[…] it doesn’t have much in the way of exact or head-to-head competitors, but cites Spring and WibiData/Kiji as coming […]