Use cases for low-latency analytics
At various times I’ve noted the varying latency requirements of different analytic use cases, which can be as different as the speed of a turtle is from the speed of light. In particular, back when I wrote more about CEP (Complex Event Processing), I listed some applications for super-low-latency and not-so-low-latency CEP alike. Even better were some longish lists of “active data warehousing” use cases I got from Teradata in August, 2009, generally focused on interactive customer response (e.g. personalization, churn prevention, upsell, antifraud) or in some cases logistics.
In the slide deck for the Teradata 6680/solid-state drive announcement, however, Teradata went in a slightly different direction. In its list of “hot data use case examples”, Teradata suggested:
- BI dashboards
- 7 X 24 real time operations
- Financial peak periods
- Month end, quarter end
- Cyber Security
- Short and long term threats
- Operational reporting
- Claims processed
- Inventory instant status
- Machine generated data
- Rapid response
- Fast analytics
To me, four things stand out about that list. The first two are:
- Customer response cases are glaringly absent.
- Few of those applications are satisfied by single-row or other tiny result sets; most require real analytic queries. That is, they’re on the analytic side of the short-request/analytic boundary.
Since a lot of customer response applications use tiny result sets, I imagine those two features are not coincidental. Even so, some customer-response applications can benefit from serious real-time analysis, such as graph-analytic techniques, which can be applied to antifraud and influencer-identifying anti-churn alike.
And thus I’ve shown that a list of bullet points, sized to fit on a single marketing slide, is imperfect. Oh, the humanity!
The two other notable points are:
- Multiple references to machine-generated data (in general and to cyber security in particular).
- Multiple references to low-latency monitoring (dashboards in general and analyzing machine-generated data in particular).
Frankly, I think low-latency monitoring is going to be one of the hot areas over the next few years. “Real-time” is cool, and big monitors with constantly changing graphics are cooler yet.
Comments
2 Responses to “Use cases for low-latency analytics”
Leave a Reply
A frustrating limitation of low-latency or “real-time” analytics is the absence of visualization systems designed to be driven that way for many application domains.
If you have a logically contiguous model of reality that supports a very high update rate — some geo-sensing databases are like this — then it becomes a problem of detecting intersections with a visualization viewport on the backend and pushing that event to the frontend. For these applications it is not efficient to poll the database nor to let the visualization client decide what to render by drinking from the firehose. The backend needs to decide if an update needs to be rendered in a current visualization context and push it there.
While it is possible to build backends like this it turns out that it is quite difficult to find existing visualization frontends that are designed to support this “real-time” use case. Some of the closest software designs architecturally are server-rendered game engines. Real-time is cool but there seems to be little support for it on the frontend except for the trivial cases that fit inside the old models, which is a limiting factor.
[…] Use cases for low-latency analytics (April, 2011) […]