The software vendors that have dominated the business intelligence market for the last 15 years or so have hit the wall and they haven’t a clue how to scale it. They’re stuck because they insist on applying the skills and methods that helped them successfully build data warehouses and production reporting systems to a radically different problem: data sense-making. Their past achievements were grand feats of engineering, solved almost entirely with technology, but data sense-making (also known as, analytics) requires a different approach—one that leads with design, not engineering, and focuses on people, their needs and abilities, not technology. Attempts of big BI companies to open the data repository for exploration have produced some embarrassing tools, and they keep on coming. One of the newest examples is SAP BusinessObjects Explorer.
In an article written for SAP Insider, Jeff Veis, Vice President of Industry Solutions and Strategic Initiatives for SAP BusinessObjects, sets expectations for BusinessObjects Explorer:
When most users think of business intelligence (BI), they think of it in a very traditional sense: predefined reports that can’t account for real-time market fluctuations and that don’t allow business users to truly engage with the information.
SAP BusinessObjects Explorer software changes all that. Using the tool, companies can extend the reach of BI to all business users — not just a small subset of expert data analysts…SAP BusinessObjects Explorer enables deep exploration of vast amounts of data, enabling users to identify, manipulate, and act on insights that a pre-structured, traditional BI tool would be hard pressed to deliver.
(”Transform the Way Your Company Thinks about Business Intelligence”, Jeff Veis, SAP Insider, Jan-Feb-Mar 2010)
In the same issue of SAP Insider, Jonathan D. Becher, Senior Vice President of Marketing, answered the question “Can you contrast SAP BusinessObjects Explorer to the business intelligence (BI) tools our readers now have in place?” as follows:
SAP BusinessObjects Explorer is about data exploration, not report generation.
The second big differentiator is accessibility. Everybody who needs access to your company’s business data can use SAP BusinessObjects Explorer. While many of your readers work for companies that are now running BI solutions, most of the employees who need access to business data can’t use the tools. Mastering their requirements and interfaces just isn’t practical, so the BI tools remain the exclusive purview of a relatively scarce number of power users, analysts, and IT department members, and the broader business user community has to go through one of these intermediaries to get their questions answered.
It’s analogous to the way people made phone calls a few generations back. To place a call, you would pick up the receiver and wait for an operator to get on the line and ask to whom you’d like to speak. If that person was in a different part of the country, a series of local operators, each covering specific regions of the country, worked to facilitate the connection. There was nothing self-service about it. In a very real sense, this is the way BI — and frankly, decision making — works today.
SAP BusinessObjects Explorer changes this. It is so easy to use, it democratizes access to data.
(”Better Answers through Better Questions”, Jonathan D. Becher, SAP Insider, Jan-Feb-Mar 2010)
This is roughly the same explanation that big BI companies have been giving for the last 15 years. Terms like “self-service BI” and the “democratization of data” have been used in association with every new product that they’ve introduced since the day that the term “business intelligence” was coined. Obviously, however, none of their past products have achieved this, which is why they keep coming out with new ones to cure the ills caused by every unnecessarily complicated under-performing product that they’ve delivered in the past. But if the previous generation of products didn’t achieve these goals, why should we believe that BusinessObjects Explorer will?
Let’s look at an example of how Becher thinks this new tool will operate in the workplace.
It’s pretty common for participants to show up at planning meetings with their go-to PowerPoint presentation, replete with their favorite metrics about what strategic concerns the business faces. And it’s extraordinarily common for these metrics not to match up — at all. Consider a simple question: How many new customers did we acquire last quarter?
The operations organization posits that people who bought and then returned products do not constitute “new customers.”
Metrics from the head of marketing, who views returns as a quality issue, not a sales issue, do count customers who bought and subsequently returned products last quarter as “new customers.”
The Large Enterprise Sales organization recognizes “new customers” as only those with orders in excess of US$10,000.
Given that the purpose of the meeting is to devise or refine plans, do you really want to lose another planning cycle sending participants off in pursuit of a new definition of the term “new customer,” asking them to regenerate their figures? With SAP BusinessObjects Explorer, this could be done in real time, with all stakeholders looking at the same data.
So, what this new product will finally put within our reach is the earth-shattering ability to get an answer to the question “How many new customers did we acquire last quarter?” without having to involve the IT department. Be still my heart; it’s all a-flutter. A few paragraphs later in the same article, still referring to this data sense-making miracle, Becher states: “If SAP BusinessObjects Explorer sounds revolutionary, it’s because it is revolutionary.” [Long dumbfounded silence] Huh?!!!
Lest we be accused of missing the real miracle here, let’s take into account Becher’s claim that this new tool will eliminate the confusion and roadblocks to consensus caused by the fact that the Operations, Marketing, and Sales departments each define new customers differently, and therefore come up with different new customer counts. This must be some magical tool if it somehow puts everyone on the same page, despite their different perspectives. If this sounds to you like marketing smoke and mirrors, you’re getting the picture.
And it gets better.
Take the example one step further. Let’s say that I am the head of the Large Enterprise Sales organization, and I want to compare sales in select regions of the country. I throw in a few other requirements, and we’re no longer dealing with a standard query — so I have to enlist the help of an analyst. The analyst needs certain warehouse statistics, but finds the right data isn’t loaded, so a call goes out to a data architect, who in turn enlists the help of others to cleanse and load the data. Eventually, I get the report. And 99 times out of 100, the experience ends with something like this: “Oh! That’s not the question I meant to ask. I meant to specify New York City, not New York state, and I actually needed to account for sales that took place in the wake of a new promotional campaign.”
With BusinessObjects Explorer, according to Becher, problems like these will go away. How? Through a new interface that will allow you to ask questions of your data similar to the way you search the Web with Google today. Anyone who understands BI, however, knows that no interface, no matter how magical, will give you access to data that isn’t available, will clean data that is dirty, or will simplify the navigation of complicated operational databases. These improvements are accomplished by a whole lot of hard work on the back end (probably done by someone in IT, because only they have access) to prepare the data for use.
Enough of these same old hollow claims by the big BI vendors that have been frustrating and angering users for years. Are we going to let them continue to raise our hopes and dash them forever, never going elsewhere for answers?
Let’s forget what SAP BusinessObjects is saying about Explorer and take an honest, objective look at it ourselves.
Caution
Don’t mistake what I’ve written as a case against Big BI in favor of Small BI. It is entirely possible for large BI vendors to provide effective tools for data sense-making. To do this, they need to switch from a technology-centric engineering-focused approach to a human-centric design-focus approach, and base their efforts on a deep understanding of data sense-making. Most of the small BI vendors have done no better in cracking this nut than the big guys. They might be more agile due to their small size and thus able to bring a new product to market more quickly, but when they approach the problem in the same dysfunctional way as the big guys, they fail just as miserably. Just like politicians who sell themselves as “not like the guys in Washington,” new players in the BI space often point to the failures of the big guys and then go on to do exactly the same. I am not making a case of small vs. big, but of clear-headed, informed, and effective vs. an old paradigm that doesn’t work for the challenges of data sense-making.
Review of BusinessObjects Explorer
As quoted above, Jeff Veis claims that “SAP BusinessObjects Explorer enables deep exploration of vast amounts of data, enabling users to identify, manipulate, and act on insights that a pre-structured, traditional BI tool would be hard pressed to deliver.” To test these claims, I asked Bryan Pierce who works with me here at Perceptual Edge to access an evaluation copy of the tool on SAP’s website and put it through its paces. The following are Bryan’s findings.
Basically, I didn’t really find anything good about SAP BusinessObjects Explorer. If it was all you had, you could use it to perform some analysis, and it might be a little easier for certain types of exploratory analysis than a tool like Excel, but compared to other tools that are actually designed for exploratory analysis, it’s a joke. Here is an example of the BusinessObjects Explorer interface:
1) Perhaps the single biggest problem with SAP BusinessObjects Explorer is that it only allows you to view one graph at a time. In addition to this, it only allows a maximum of three measures in a graph, so you can only have three lines in a line graph and three segments in a stacked bar graph. There is a notable—but not useful—exception to the single graph rule. If you select two or three measures and choose a pie chart or a radar chart it will create two or three graphs next to each other, although I couldn’t find a way to make them share a quantitative scale (this would only be applicable for radar charts). Unfortunately, I couldn’t find any way to get multiple versions of any useful graph types.
2) Although you can view up to three measures at once, there’s even less functionality when viewing categorical variables. For instance, the dataset I analyzed had three years’ worth of quarterly data. Using a line graph, I could view the values for the three years and I could look at a particular year and see the quarterly values, but I couldn’t find a way to view all three years at a quarterly level simultaneously (either by using a single line that spanned twelve quarters or by using three lines that each spanned four quarters). Similarly, while I could get two lines or stacked-bar segments for Profit and Expenses (both measures), I couldn’t find a way to get separate lines or stacked-bar segments for States or Cities.
3) In attempting to determine why I couldn’t get a line graph to display quarterly sales for more than one year at a time, I uploaded a custom dataset of time-series data. It appears that BusinessObjects Explorer handles time-series data very poorly. The dataset I uploaded contained daily sales data for two products over 90 days. When I first opened it, this is how the date variable was displayed
As you can see, the variable has been sorted by the total sales for each date, rather than the dates themselves. To fix this, I clicked the little down arrow in the top-right corner of the image and told it to re-sort by date. This is what appeared:
For some reason, every date except for one of them disappeared; I had to close and reopen the dataset to get the other dates to reappear. So, apparently there’s something buggy with the way BusinessObjects Explorer handles dates. In fact, the only way I could get time-series data to work correctly was when I separated years, months, and days into different variables. If you do this, just make sure that you format the months as numbers, because that’s the only way they’ll sort correctly. Unless, of course, you like alphabetical time:
4) With BusinessObjects Explorer, you’re not able to customize the appearance of graphs in any way. This isn’t as important as it would be for a data presentation tool, like Excel, but even an analysis tool should let you do things like disable the gridlines or remove the data points from a line graph.
5) Like most web-based analysis tools, it responds too slowly for seamless interaction. After a filter is applied there is short delay while the application contacts the server for the new data. This delay is only about one or two seconds long, but that’s still more than enough to hamper an analyst’s train of thought.
6) There were several times when I switched over to time-based views where the graphs weren’t sorted in chronological order. For example, I was viewing a bar graph that showed Margin and Quantity Sold by State, which was sorted by the Margin values. I then switched from viewing the graph by State to viewing it by Quarter. The sort by Margin was still in effect so the graph displayed the bars in this order: Q4, Q2, Q1, Q3. It’s one thing to allow people to arrange time-series information in non-chronological order for those extremely rare cases when that might be useful. It’s quite another thing to allow time-series data to be arranged in this way by the software.
7) The program includes all the standard graph types and a few unexpected ones, such as treemaps (although, how useful is a treemap when it only takes up about 1/3 of your screen space?), but it doesn’t include box plots.
Speaking of treemaps, they should be used to navigate hierarchical data, for instance, to view sales and margin data at the regional, state, and city levels. However, the treemaps in BusinessObjects Explorer appear to only allow a single level of hierarchy. Here is a treemap that is displaying Sales and Margin by State:
In a functional treemap, I should be able to display each city’s data as a smaller square within the corresponding state square and, potentially, to select a particular city and drill into it to see even more detailed data (such as sales by individual stores). Unfortunately, I could find no way to do any of this. As a side note, red and green are the worst two colors to use to encode data (in this case they’re encoding margin values from highest to lowest), because most people who are colorblind (10% of males and 1% of females) can’t distinguish between the two colors. I would have switched to more suitable colors, but, as I mentioned before, I couldn’t find a way to modify the appearance of any of the graphs.
9) The user doesn’t have enough control of the layout of the display. When viewing both the filter controls and the visualization, the filter controls take up as much space as the visualization. You can hide the filter controls, which gives the visualization more space, but there’s no way to just reduce the filter controls’ size. In addition to using too much space, the filter controls make poor use of the space they require. The filters were designed so that each filter column has the same width. As a result, there are some filters that contain large amounts of wasted space, like the Quarter filter below:
If the filters had been designed to size themselves based on their contents, more filters could have fit on the screen at the same time, which would make filtering more efficient.
10) The shape of the graph is also a problem sometimes. Because the plot area is about four times wider than it is tall, it makes certain types of graphs awkward to read, such as scatterplots (which should usually be roughly square in shape) or vertical bar graphs that only have a few bars (in which case, the bars might be wider than they are tall).
11) In the filter section, totals are displayed next to each categorical value. For instance, here is the quarter variable:
As you can see, the totals aren’t all written to the same precision. The Q1, Q3, and Q4 totals are written to the tenth of a dollar, while the Q2 total is written to the whole dollar. This means the decimal points don’t line up, which makes the numbers harder to read.
These are the problems that Bryan found while spending two hours with the product. A deeper look would no doubt produce a longer list, but Bryan was only trying to spot the big problems that most severely undermine the products use for data exploration and analysis. Had we taken the time to compare BusinessObjects Explorer to one of the good data exploration and analysis products that are available today, such as Tableau or Spotfire, the claim by BusinessObjects that Explorer is “revolutionary” would be exposed more clearly for what it is: a sad statement about this Big BI company’s understanding of data sense-making. BusinessObjects is struggling to catch up with human-centered, design-focused companies like Tableau and Spotfire, which are running circles around them, and making them look pathetic. SAP BusinessObjects and most other Big BI companies haven’t taken the time to understand data sense-making in general, data visualization in particular, or even the real needs of their customers. They need a new mindset, but learning to see the world with new eyes is hard. By the time they figure this out and make the shift, will it be too late?
Take care,
