“Freeing business analysts from IT”
Many of the companies I talk with boast of freeing business analysts from reliance on IT. This, to put it mildly, is not a unique value proposition. As I wrote in 2012, when I went on a history of analytics posting kick,
- Most interesting analytic software has been adopted first and foremost at the departmental level.
- People seem to be forgetting that fact.
In particular, I would argue that the following analytic technologies started and prospered largely through departmental adoption:
- Fourth-generation languages (the analytically-focused ones, which in fact started out being consumed on a remote/time-sharing basis)
- Electronic spreadsheets
- 1990s-era business intelligence
- Dashboards
- Fancy-visualization business intelligence
- Planning/budgeting
- Predictive analytics
- Text analytics
- Rules engines
What brings me back to the topic is conversations I had this week with Paxata and Metanautix. The Paxata story starts:
- Paxata is offering easy — and hopefully in the future comprehensive — “data preparation” tools …
- … that are meant to be used by business analysts rather than ETL (Extract/Transform/Load) specialists or other IT professionals …
- … where what Paxata means by “data preparation” is not specifically what a statistician would mean by the term, but rather generally refers to getting data ready for business intelligence or other analytics.
Metanautix seems to aspire to a more complete full-analytic-stack-without-IT kind of story, but clearly sees the data preparation part as a big part of its value.
If there’s anything new about such stories, it has to be on the transformation side; BI tools have been helping with data extraction since — well, since the dawn of BI. The data movement tool I used personally in the 1990s was Q+E, an early BI tool that also had some update capabilities.* And this use of BI has never stopped; for example, in 2011, Stephen Groschupf gave me the impression that a significant fraction of Datameer’s usage was for lightweight ETL.
*Q+E came from Pioneer Software, the original predecessor of Progress DataDirect, which first came to fame in association with Microsoft Excel and the invention of ODBC.
More generally, I’d say that there are several good ways for IT to give out data access, the two most obvious of which are:
- “Semantic layers” in BI tools.
- Data copies in departmental data marts.
If neither of those works for you, then most likely either:
- Your problem isn’t technology.
- Your problem isn’t data access.
And so we’ve circled back to what I wrote last month:
Data transformation is a better business to enter than data movement. Differentiated value in data movement comes in areas such as performance, reliability and maturity, where established players have major advantages. But differentiated value in data transformation can come from “intelligence”, which is easier to excel in as a start-up.
What remains to be seen is whether and to what extent any of these startups (the ones I mentioned above, or Trifacta, or Tamr, or whoever) can overcome what I wrote in the same post:
When I talk with data integration startups, I ask questions such as “What fraction of Informatica’s revenue are you shooting for?” and, as a follow-up, “Why would that be grounds for excitement?”
It will be interesting to see what happens.
Related link
- Analytics for everybody! (March, 2014)
Comments
12 Responses to ““Freeing business analysts from IT””
Leave a Reply
I think Tamr ( I am not familiar with the rest of DI startups) is much more than simply taking share from Imformatica.
There is paper written by Dr. Stonebraker about Tamr.
It attempts to solve very difficult problem using mach. learning to automatically integrate data.
Ranko,
That’s still an attempt to take share from Informatica or whoever by providing better technology.
Good post. Two previous similar articles on this very significant topic:
1) http://www.zdnet.com/big-data-why-it-departments-mustnt-be-a-drag-on-analytics-7000027428/
2) http://www.qas.com/data-quality-news/why_it_offices_drop_the_ball_on_analytics_and_how_they_can_stop_9875.htm
“We can’t solve problems by using the same kind of thinking we used when we created them” – Albert Einstein.
An alternative followup question might be, “What fraction of the combined Informatica and Informatica consulting ecosystem revenue are you shooting for?” The big bucks in BI are still in the ETL-related professional services, just as it did in the 1990s.
I thought Tamr was a data curation tool.
What has anybody said to contradict Tamr’s decision to use the category name “data curation”?
The whole “data prep” movement is not about taking share away from Informatica (or any ETL vendor).
The aim is to reduce time and dependency on MS Excel.
There are three main problems with Excel:
1. It’s error prone (too much cutting and pasting)
2. It’s hard to repeat steps (because cutting and pasting)
3. It doesn’t scale well.
What I often tell people is that the “dirty little secret” about BI platforms is that they are essentially little more than data extraction engines for most business analysts (or any business user really – even admins).
All the real “work” is done in Excel.
For example, let’s say your boss wants to see a histogram of CRM opportunities bucketed as:
Under 1 month
Under 2 months
Under 3 months
Under 6 months
Under 1 year
Open
The analyst will just go to the CRM BI tool, extract out all opportunities to Excel (with the Open and Close Dates).
She will then spend her entire morning doing “data prep” in Excel, to essentially create a new “bucket” data element, derived from the Start and End date.
Then she’ll spend about 10 minutes creating a Pivot table (in Excel of course).
She’ll then spend some time on the presentation – getting fonts and colours right.
Then she’ll send off to her manager.
Next week the boss will come back and ask for it all over again, and it will still take a big chunk of time, not to mention those errors.
So that’s what tools like Paxata and Trifacta are trying to improve on.
That said, I can see some of these tools eventually subsuming traditional ETL tools. But that won’t happen for several years.
I think free business analysis capabilities from reliance on IT is a universal goal. However, there is still some significant work before this becomes a reality. According to a recent IDG SAS survey, only 10 percent of organizations feel they are extremely capable at interpreting data to find meaningful insight and the same is true when turning insights into action. Long term yes, but right now IT still needs to play a crucial role.
Peter Fretty
[…] “Freeing business analysts from IT” (August, 2014) […]
[…] “Freeing business analysts from IT” (August, 2014) […]
@ Neil Hepburn
THat was really good explanation about data preparation. I was really confused about why these tools are trying to do the very same thing as what ETL does. Thank you for making it clear and differentiating between data preparation tools and ETL!
[…] the target users of Cloudera Data Science Workbench can be characterized the same way BI-oriented business analysts are. They’re people […]