Business intelligence
Analysis of companies, products, and user strategies in the area of business intelligence. Related subjects include:
- Data warehousing
- Business Objects
- Cognos
- QlikTech
- (in Text Technologies) Text mining
- (in Text Technologies) Text analytics/business intelligence integration
- (in The Monash Report) Strategic issues in business intelligence
- (in Software Memories) Historical notes on business intelligence
Upcoming webinar on investigative analytics
I recently coined the phrase investigative analytics to conflate
- Statistics, data mining, machine learning, and/or predictive analytics.
- The more research-oriented aspects of business intelligence tools:
- Ad-hoc query.
- Drilldown.
- Most things done by BI-using “business analysts”
- Most things within BI called “data exploration.”
- Analogous technologies as applied to non-tabular data types such as text or graph.
This will be be basis for my part of a webcast on March 10 at 11 am Pacific/2 pm Eastern time. The other main part of the webcast will be a demo by the webcast’s joint sponsors Aster Data and Tableau Software.
Some of Aster’s verbiage in describing and titling the webinar is so hyperbolic that I do not want to give the impression of endorsing it. But I am very hopeful that the webinar itself will be interesting and informative, and will point people at least somewhat in the direction of the benefits Aster is claiming.
| Categories: Analytic technologies, Aster Data, Business intelligence, Data warehousing, Presentations, Tableau Software | 3 Comments |
Mega-trends driving data warehousing and business intelligence
Philip Russom opines (emphasis mine):
What’s driving change in data warehousing (DW) and business intelligence (BI)? There are obvious scalability issues, due to burgeoning data, reports, and user communities. Plus, end-users need more real-time and on-demand BI. For many organizations, integrating existing systems into DW/BI is a higher priority than putting in new ones. And the “do more with less” economy demands more BI at lower costs. Hence, most drivers of change in BI and DW concern four Mega-Trends: size, speed, interoperability, and economics.
Depending on which universe of enterprises and vendors you’re looking at, Philip’s claim of “most” may be technically true. But from where I sit, Philip omitted two other crucial trends: new kinds of data and increased analytic sophistication.
A year ago, I divided data into three kinds:
- Human/tabular, which is what Philip’s comments seem to be focused on.
- Human/nontabular, e. g. what is best handled via text analytics.
- Machine-generated, such as web log or sensor data.
Most organizations on the planet could benefit from better understanding or exploiting their human-generated tabular data. But even so, many of the best opportunities to add analytic value come from capturing and analyzing fundamentally newer kinds of information.
I further would suggest that analytic sophistication is going up, for at least two reasons:
- New kinds of data call for or at least allow new kinds of analytics.
- Better price-performance (on bigger data sets) allows for more sophisticated analytic techniques.
Some of the best examples of these trends, especially the second one, may be found in what I recently called analytic profiling.
| Categories: Analytic technologies, Business intelligence | 4 Comments |
The six useful things you can do with analytic technology
I seem to be in the mode of sharing some of my frameworks for thinking about analytic technology. Here’s another one.
Ultimately, there are six useful things you can do with analytic technology:
- You can make an immediate decision.
- You can plan in support of future decisions.
- You can research, investigate, and analyze in support of future decisions.
- You can monitor what’s going on, to see when it necessary to decide, plan, or investigate.
- You can communicate, to help other people and organizations do these same things.
- You can provide support, in technology or data gathering, for one of the other functions.
Technology vendors often cite similar taxonomies, claiming to have all the categories (as they conceive them) nicely represented, in slickly integrated fashion. They exaggerate. Most of these categories are in rapid flux, and the rest should be. Analytic technology still has a long way to go.
In more detail: Read more
| Categories: Analytic technologies, Business intelligence, Cognos, Data warehousing, RDF and graphs, Text | 13 Comments |
The Workday architecture — a new kind of OLTP software stack
One of my coolest company visits in some time was to SaaS (Software as a Service) vendor Workday, Inc., earlier this month. Reasons included:
- Workday has forward-thinking ideas about SaaS enterprise applications and the integration of business intelligence into same.
- Workday has highly innovative ideas in how it manages data.
- Companies founded by Dave Duffield tend to feature smart, likeable people who talk to one pleasantly and forthrightly. Workday is no exception; CTO Stan Swete and the other Workday folks present were a delight to talk with.
- I’d invited Merv Adrian to come along with me. He asked great questions, and I could gather myself a bit despite how sleep-deprived I was for the first part of that trip.
Workday kindly allowed me to post this Workday slide deck. Otherwise, I’ve split out a quick Workday, Inc. company overview into a separate post.
The biggie for me was the data and object management part. Specifically: Read more
The substance of Pentaho’s Hadoop strategy
Pentaho has been talking about a Hadoop-related strategy. Unfortunately, in support of its Hadoop efforts, Pentaho has been — quite insistently — saying things that don’t make a lot of sense to people who know anything about Hadoop.
That said, I think I found four sensible points in Pentaho’s Hadoop strategy, namely:
- If you use an ETL tool like Pentaho’s to move things in and out of HDFS, you may be able to orchestrate two more steps in the ETL process than if you used Hadoop’s native orchestration tools.
- A lot of what you want to do in MapReduce is things that can be graphically specified in an ETL tool like Pentaho’s. (That would include tokenization or regex.)
- If you have some really lightweight BI requirements (ad hoc, reporting, or whatever) against HDFS data, you might be content to do it straight against HDFS, rather than moving the data into a real DBMS. If so, BI tools like Pentaho’s might be useful.
- Somebody might want to use a screwy version of MapReduce, where by “screwy” I mean anything that isn’t Cloudera Enterprise, Aster Data SQL/MapReduce, or some other implementation/distribution with a lot of supporting tools. In that case, they might need all the tools they can get.
The first of those points is, in the grand scheme of things, pretty trivial.
The third one makes sense. While Hadoop’s Hive client means you could roll your own integration with your own favorite BI tool in any case, having somebody certify it for you themselves could be nice. So if Pentaho ships something that works before other vendors do, good on them. (Target date seems to be October.)
The fourth one is kind of sad.
But if there’s any shovel-meet-pony aspect to all this — or indeed a reason for writing this blog post — it would be the second point. If one understands data management, but is in the “Oh no! Hadoop wants me to PROGRAM!” crowd, then being able to specify one’s MapReduce might be a really nice alternative versus having to actually code it.
| Categories: Analytic technologies, Business intelligence, Hadoop, MapReduce, Parallelization, Pentaho | 10 Comments |
Teradata’s future product strategy
I think Teradata’s future product strategy is coming into focus. I’ll start by outlining some particular aspects, and then show how I think it all ties together.
Read more
| Categories: Business intelligence, Data warehouse appliances, Data warehousing, Kickfire, MicroStrategy, Solid-state memory, Storage, Teradata | 5 Comments |
Advice for some non-clients
Edit: Any further anonymous comments to this post will be deleted. Signed comments are permitted as always.
Most of what I get paid for is in some form or other consulting. (The same would be true for many other analysts.) And so I can be a bit stingy with my advice toward non-clients. But my non-clients are a distinguished and powerful group, including in their number Oracle, IBM, Microsoft, and most of the BI vendors. So here’s a bit of advice for them too.
Oracle. On the plus side, you guys have been making progress against your reputation for untruthfulness. Oh, I’ve dinged you for some past slip-ups, but on the whole they’ve been no worse than other vendors.’ But recently you pulled a doozy. The analyst reports section of your website fails to distinguish between unsponsored and sponsored work.* That is a horrible ethical stumble. Fix it fast. Then put processes in place to ensure nothing that dishonest happens again for a good long time.
*Merv Adrian’s “report” listed high on that page is actually a sponsored white paper. That Merv himself screwed up by not labeling it clearly as such in no way exonerates Oracle. Besides, I’m sure Merv won’t soon repeat the error — but for Oracle, this represents a whole pattern of behavior.
Oracle. And while I’m at it, outright dishonesty isn’t your only unnecessary credibility problem. You’re also playing too many games in analyst relations.
HP. Neoview will never succeed. Admit it to yourselves. Go buy something that can. Read more
Microstrategy technology notes
Earlier this week, Microstrategy made Mark LaRow available to talk about technology. The proximate reason was my recent mention of Microstrategy’s mobile BI emphasis, but we also touched on Microstrategy’s approach to in-memory business intelligence and some other subjects. We didn’t go into the depth of a similar conversation I had recently with Qlik Technologies, but I found it quite interesting even so.
Highlights of the in-memory BI discussion included:
- Microstrategy’s in-memory BI data structure is some kind of simple array, redundantly called a “vector array.” A more precise description was not available.
- While early versions of the capability have been around since 2002, Microstrategy’s in-memory BI capability only got serious with Microstrategy 9, which was released in Q1 of 2009. In particular, Microstrategy 9 was the first time in-memory BI had full security.
- Mark says a core reason for having their own in-memory BI is because Microstrategy has more smarts to predict which aggregates will or won’t be needed. Strictly speaking, that can’t be argued with. Vendors like Infobright would argue they come close enough to that ideal as to make little practical difference – but I’m also cheating by naming Infobright, which is particularly focused in that direction.
- Microstrategy in-memory BI compresses data by about 2X. Mark didn’t know which compression algorithm was used.
- The limitation on what’s in-memory is, of course, how much RAM you can fit on an SMP box. Microstrategy has seen up to ½ terabyte deployments.
- In-memory Microstrategy data structures are typically built during the batch window, for performance reasons. This is not, strictly speaking, mandatory, but I didn’t get a sense that Microstrategy was being used for much that resembled real-time business intelligence.
- Mark said Microstrategy has no interest in using solid-state memory to expand the reach of its in-memory BI. Frankly, if Microstrategy doesn’t change that stance, it’s in-memory BI capabilities are unlikely to stay significant for too many years.
Another key subject we discussed was Microstrategy’s view of dashboards. Read more
| Categories: Business intelligence, Data warehousing, Memory-centric data management, MicroStrategy | Leave a Comment |
How should somebody teach themselves database and programming skills?
From time to time, I get in a conversation with somebody who is:
- Unemployed, underemployed, or otherwise desirous of having more commercial skills.
- Not a programmer, but desirous of having some technical skills.
- Astute enough to realize s/he will never be a serious techie.
I generally have two models in mind when guiding such a person:
- Analytics/business intelligence/stats.
- Website building.
Those are both useful skill sets for people who aren’t full-time techies, the first perhaps best for those who are more quantitative and big-company-friendly, the second perhaps better for the creative and/or rebellious types.
So what SPECIFICALLY should one guide them to do? My initial thoughts include: Read more
| Categories: Business intelligence, MicroStrategy, MySQL, Open source | 35 Comments |
False-positive alerts, non-collaborative BI, inaccurate metrics, and what to do about them
I’ve been hinting at some points for quite a long time, without really spelling them out in written form. So let’s fix that. I believe:
- “Push” alerting technology could be much more granular and useful, but is being held back by the problem of false positives.
- Metrics passed down from on high didn’t work too well in Stalin’s USSR, and haven’t improved sufficiently since.
- A large, necessary piece of the solution to both problems is a great engine for setting and modifying metrics definitions.
I shall explain. Read more