Notes from a long trip, July 19, 2016
For starters:
- I spent three weeks in California on a hybrid personal/business trip. I had a bunch of meetings, but not three weeks’ worth.
- The timing was awkward for most companies I wanted to see. No blame accrues to those who didn’t make themselves available.
- I came back with a nasty cough. Follow-up phone calls aren’t an option until next week.
- I’m impatient to start writing. Hence tonight’s posts. But it’s difficult for a man and his cough to be productive at the same time.
A running list of recent posts is:
- As a companion to this post, I’m publishing a very long one on vendor lock-in.
- Spark and Databricks are both prospering, and of course enhancing their technology as well.
- Ditto DataStax.
- Flink is interesting as the streaming technology it’s now positioned to be, rather than the overall Spark alternative it used to be positioned as but which the world didn’t need.
Subjects I’d like to add to that list include:
- MemSQL, Zoomdata, and Neo Technology (also prospering).
- Cloudera (multiple topics, as usual).
- Analytic SQL engines (“traditional” analytic RDBMS aren’t doing well).
- Microsoft’s reinvention (it feels real).
- Metadata (it’s ever more of a thing).
- Machine learning (it’s going to be a big portion of my research going forward).
- Transitions to the cloud — this subject affects almost everything else.
I’ll edit these lists as appropriate when further posts go up. Last update: August 23, 2016.
Let’s cover some other subjects right here.
1. While Kafka is widely agreed to be the universal delivery mechanism for streams, the landscape for companion technologies is confused.
- Back in January I wrote that the leaders were mainly Spark Streaming, followed by Storm.
- I overlooked the fact that Storm creator Twitter was replacing Storm with something called Heron.*
- If there’s any buzz about Confluent’s replacement for distant-third-place contender Samza, I missed it.
- Opinions about Spark Streaming are mixed. Some folks want to get away from it; others like it just fine.
And of course Flink is hoping to blow everybody else in the space away.
*But that kind of thing is not necessarily a death knell. Cassandra inventor Facebook soon replaced Cassandra with HBase, yet Cassandra is doing just fine.
As for the “lambda architecture” — that has always felt like a kludge, and various outfits are trying to obsolete it in various ways. As just one example, Cloudera described that to me during my visit as one of the main points of Kudu.
2. The idea that NoSQL does away with DBAs (DataBase Administrators) is common. It also turns out to be wrong. DBAs basically do two things.
- Handle the database design part of application development. In NoSQL environments, this part of the job is indeed largely refactored away. More precisely, it is integrated into the general app developer/architect role.
- Manage production databases. This part of the DBA job is, if anything, a bigger deal in the NoSQL world than in more mature and automated relational environments. It’s likely to be called part of “devops” rather than “DBA”, but by whatever name it’s very much a thing.
I had a moment of clarity on this point while visiting my clients at DataStax, and discussing their goal — shared by numerous companies — of being properly appreciated for the management tools they provide. In the room with me were CEO Billy Bosworth and chief evangelist Patrick McFadin — both of whom are former DBAs themselves.
3. I visited ClearStory, and Sharmila Mulligan showed me her actual sales database, as well as telling me some things about funding. The details are all confidential, but ClearStory is clearly doing better than rumor might suggest.
4. Platfora insisted on meeting circumstances in which it was inconvenient for me to take notes. So I have no details to share. But they sounded happy.
Edit: On July 22, it was announced that Workday is acquiring Platfora. Now I understand why Platfora gave me a bit of a runaround.
5. Pneubotics — with a cool new video on its home page — has found its first excellent product/market fit. Traditional heavy metallic robots are great at painting and related tasks when they can remain stationary, or move on rigid metal rails. Neither of those options works well, however, for large curved or irregular surfaces as might be found in the aerospace industry. Customer success for the leading soft robot company has ensued.
This all seems pretty close to the inspection/maintenance/repair area that I previously suggested could be a good soft robotics fit.
Comments
7 Responses to “Notes from a long trip, July 19, 2016”
Leave a Reply
[…] management of even NoSQL DBMS is a big issue, and help in that area has high cash value for […]
In DBA world we had two kinds of DBA – “production DBA” who’s role is obvious and development DBA who’s role was to make sure that software developers are using database efficiently.
In case of NoSQL second role became very important. NoSQL databases are much less tolerant to misuse than RDBNS where a lot can be done to improve things after SQL’s where hard coded into the application.
In my opinion NoSQL DBA should have deep understanding of the internals and to help developers to decide how their goals can be met with given NoSQL.
Above will greatly affect production side of the story. If NoSQL was used in exactly right way – it will perform well in production. Opposite is much more true – if NoSQL was bend to do things not natural for it – it will be real challenge to make it happy in production.
I do remember back in da dayz Larry Ellison talking about Oracle 10G as self managed database that does NOT require a DBA or something like that
One can push a button on Oracle, do a quick install, and never administer it.
Perhaps a small app or two might even run under those circumstances. 🙂
[…] learned some newish terms on my recent trip. They’re meant to solve the problem that “data scientists” used to be folks with […]
[…] visited DataStax on my recent trip. That was a tipping point leading to my recent discussions of NoSQL DBAs and misplaced fear of vendor lock-in. But of course I also learned some things about DataStax and […]
[…] a July visit to DataStax, I […]