Cool analytic stories
There are several reasons it’s hard to confirm great analytic user stories. First, there aren’t as many jaw-dropping use cases as one might think. For as I wrote about performance, new technology tends to make things better, but not radically so. After all, if its applications are …
… all that bloody important, then probably people have already been making do to get it done as best they can, even in an inferior way.
Further, some of the best stories are hard to confirm; even the famed beer/diapers story isn’t really true. Many application areas are hard to nail down due to confidentiality, especially but not only in such “adversarial” domains as anti-terrorism, anti-spam, or anti-fraud.
Even so, I have two questions in my inbox that boil down to “What are the coolest or most significant analytics stories out there?” So let’s round up some of what I know.
Financial trading is highly dependent on analytics, except perhaps for low-speed dealing in some traditional securities classes.* Specific algorithms are highly competitive, and can change within weeks. A broad range of analytic techniques have at least been tried. Some have failed so spectacularly as to precipitate a global financial crisis.
*Equities, corporate bonds, maybe municipal bonds — i.e., the stuff that’s still governed by “fundamental” — as opposed to generic quantitative — analysis.
It’s tough to be sure just what governments are doing with all that questionably-legal data they’ve assembled to fight terrorism, but we can be pretty sure that there’s a lot of relationship analytics involved. And whatever they’re doing seems to be working. As Glenn Greenwald frequently points out, there are substantially no terrorist plots in the United States, except for the ones that law enforcement itself invents. Anti-fraud applications have used relationship analytics for the past few years as well.
Also in the graph area, LinkedIn is doing a good job with People You May Know.
Also in crime-fighting, anti-spam techniques have gotten really good. Akismet blocks well over 99% of the spam comments to this blog, and I can’t recall the last false positive I saw. Google Mail approaches 99% in spam-catching (ironically, the ones that get through commonly come from mailing list sellers), and it’s been a long time since I noticed a false positive worth worrying about.
Shifting gears now, a huge fraction of analytic efforts are devoted to being more effective in one-on-one consumer marketing offers. Some of the most visible examples are in telecommunications, specifically in churn prevention. My favorite may be the case of multilingual text analytic integration in Switzerland:
I once confirmed that SPSS customer Cablecom‘s statistical models for churn and the like absolutely included text data; Cablecom even assigned different weights to the same apparent level of emotion depending on whether the text was in German, French, or Italian.
Other noteworthy adventures in marketing analytics include:
- Harrah’s successes in targeting gambling customers, even though I’m having trouble validating something I thought I’d read, namely that over 100% of Harrah’s profits came from analytics on loyalty card data.
- Various creepily effective efforts in de-anonymization.
- The recently famous story of Target targeting pregnant women.
So is it all about trading, marketing, and crime fighting? The answer is closer to being “Yes” than I would like. That said:
- Biomedical research is a hugely important area for analytic progress, and bioinformatics has gone from being a minor task to a full-time laboratory job. I hope we’ll have a bunch of great stories soon.
- Just as medicine needs to be personalized, so does education. Perhaps some low-hanging analytic fruit will be plucked soon.
- General optimization and operations research have been around for decades. They’re still going strong, with a lot more data points.
- Ditto process control. And the UIs are cooler now.
- Planning is a mess. Conceptual breakthroughs are sorely needed.
Related posts
Comments
6 Responses to “Cool analytic stories”
Leave a Reply
How about google? Graph based algorithm (plus other stuff) used to determine most relevant sites, making life easier for all of us in the hunt for specific web info?
ZestCash and Wonga doing interesting things with lending and predictive modeling/big data. If better analytics does spur more lending and increases money supply, than it has a macro effect.
“Planning is a mess. Conceptual breakthroughs are sorely needed.”
Curt, can you explain this further?
Do you mean forecasting and financial prediction for enterprise performance management? Or something else?
Thanks!
Signet/Capital One is the classic test-and-learn story.
Re “beer and diapers” — I first heard this at a 1996 Data Mining conference in San Francisco, where a Teradata presenter used the story to tout the capabilities of Teradata.
A year or so later, Forbes ran a piece quoting the head of merchandising for Wal-Mart saying that even if the finding were true he wouldn’t know what to do with it.
That’s an important point. Suppose that it’s true that shoppers tend to purchase beer and diapers on Friday nights. Does this mean retailers should:
(1) Place beer and diapers next to one another in the store, for shopper convenience;
(2) Place beer and diapers far apart in the store, to maximize time in store;
(3) Issue a coupon for beer when purchased with diapers;
(4) Don’t issue a coupon, since shoppers buy beer and diapers anyway.
There is no way to answer the question without a follow-up test and learn. In the absence of an experimental design, observed associations have little or no value for decision-making.
[…] Monash writes in Cool analytic stories There are several reasons it’s hard to confirm great analytic user stories. First, there […]
[…] explores the legend in 1998 also here Urban Legend Not true The promise of big data Let the data speak It’s complicated but yes, Virginia… Insights […]