How Tableau Saved Enterprise BI

When I started using Tableau in 2006 Enterprise Data Warehouses were the sine qua non of Business Intelligence.

Common wisdom, backed by the opinion of experts and years of practice, held that the only meaningful way to achieve value from an organization's data was to collect, organize, homogenize, restructure, and present it in enterprise-spanning industrial scale data stores from which meaningful and valuable reports, dashboards, balanced scorecards, strategy maps, and their ilk could be developed to deliver business information to those few, those lucky few, who could make use of it.

Notwithstanding the obvious self-defeating biases of this hierarchically-stratified decision-making paradigm, creating these massive data cathedrals and their information-exposing front ends turned out to be one of the most difficult, expensive, and error-prone business undertakings. Failure rates of enterprise BI projects have historically been embarrassingly high. There are many causes for this, the primary being the almost inevitable disconnect between business needs for timely data-origin information and the inward-looking technology focus of the BI development effort.

Tableau, used appropriately, can rescue enterprise BI through the application of its fundamental ability to immediately access and instantly and effectively analyze and understand business data wherever it lives. The key being: data wherever it lives.

Tableau provides a previously impossible, and largely unrecognized, opportunity to change the Enterprise Data Warehouse-based BI model. It does this by enabling several fundamentally new approaches to applying direct and pervasive data analysis to the full spectrum of activities.

Business Intelligence.
The Business View.

From a business stakeholder's perspective the whole point of BI is to deliver accurate information when it's needed, in a form that makes it easy to understand.

The business stakeholder should not be concerned with, nor adversely affected by technological factors—these should be safely hidden behind the curtains.

Using Tableau, it's possible to work intimately with the business stakeholders to analyze their data, in their local context, and achieve immediate, even instantaneous insights into the data they need to understand to make effective decisions.

This intimate coupling of people with their data observes the simple truth that all BI is local—people are first and foremost concerned with the data within their immediate business horizon; without understanding that data they cannot do their jobs properly. This is the truth that traditional enterprise BI failed to recognize.

Creating and delivering these analyses up front and continually achieves an immediate knowledge benefit to the business stakeholders, which invests them in the process and builds their trust in the ability of the initiative to deliver the goods, which is a tremendous incentive for them to remain engaged.

These high-value business analytics are then the gold standard requirements for the analytics the enterprise BI system needs to deliver (if it needs to deliver them is a legitimate question). Having been already vetted by the business stakeholders who need them, they provide the bright line path that can guide the whole program, from the design of the ODS and Data Warehouse, to the final operationalized reports, dashboards, etc.

So do we need Data Warehouses?

Given the rosy scenario painted above, the elephant in the room is the question: do we still need data warehouses and the benefits of enterprise BI?

Tableau enables Business Intelligence Delivery.

Yes. We still need Enterprise BI and Data Warehouses.

The problems and failings with traditional enterprise BI are of practice and implementation, not of principle. Attempting to build the universal answer-anything universal data store and query engine is a doomed approach. But as long as an organization's data is collected by disparate systems, and this will likely always be true, the need to collect, organize, and align it into a common context so that sense can be made of it at the enterprise level will exist. This is the raison d'etre for enterprise data warehouses and the enterprise platforms that support them.

How then do we conduct Enterprise BI?

I'm glad I asked.

It's simple: build outward from the business stakeholders' information needs. As shown above, and in proven in practice, starting with live, valuable stakeholder-approved analytics works. Using these analytics, every enterprise BI element can be conceived of, designed, and implemented with a clear idea of exactly how it will support delivery of one or more of the business analytics. Anything that doesn't support this goal is unnecessary, even harmful in its pollution of the BI space.

But isn't Tableau just an eye candy toy tool, good for making pretty pictures?
It doesn't fit in the industrial strength tool arena where real developers build real data infrastructure.
Does it?

It's a big surprise to many BU traditionalists schooled in the industrial production process approach, but introducing Tableau into an enterprise BI project can provide tremendous benefits in the velocity, accuracy, effectiveness, and quality of the activities across the project process spectrum. I've been successful in rescuing multiple enterprise BI projects that had gone off the rails by bringing Tableau into the mix. At first it was for my own use in understanding the data in play. It didn't take long for my ability to rapidly and effectively understand the data to be noticed, at which point other people would become interested and start either using Tableau for themselves or asking me to provide them with the analytics from their data. Eventually, Tableau found a place across the full range of activities.

This might seem strange, as it did at first to many of the technical people. After all, enterprise BI projects are conducted by "real" developers using "real" SQL-based data analysis tools, sometimes native to the technology in use, sometimes standalone tools like Toad. Which is one of the problems at the heart of traditional enterprise BI - the people who need to understand what's going on inside the data processing systems are hampered by their tools. SQL query tools are by their very nature low level mechanisms ill suited for the higher level cognitive activities involved in data analysis. This isn't to say that SQL query tools don't have their place—they do, but it's not as the primary tool for understanding bodies of data, and assessing them in terms of expectations and realities. That's what Tableau was designed for, and does so exquisitely well.

Here's a link to a diagram showing how Tableau fits into a typical enterprise BI project. It's too large to fit into a web page, much less a blog post. But you should look at it, preferably at poster size.

The diagram includes the Business View from above (which was really extracted from the main diagram), connecting that perspective to everything behind the curtain that's out of the business person' sight (or should be). When used effectively Tableau, can be used by everyone to help them understand their data, as shown in the shaded areas in the diagram.

Used in this way Tableau provides the opportunity to introduce enormous benefits and gains in the conduct of those processes. Since the entire process chain is at heart a series of data processing stages, Tableau's ability to instantly, flexibly, and effectively analyze any and all of the data means that the internal project people can lift their efforts above writing SQL queries or looking at table-based tools to the realm of Tableau's visual analytical approach. It's not unreasonable to expect that by using Tableau effectively enterprise BI projects can be delivered successfully, with greater value and stakeholder satisfaction, in 40% of the time, at about the same fraction of cost, as projects conducted in the traditional manner.

Tableau isn't a silver bullet.

Tableau provides the opportunity to improve the conduct of enterprise BI projects. It doesn't guarantee results. The effectiveness of any project rlies upon the skill and effort of the people involved. All Tableau does is provide the means whereby enterprise BI projects can be directed towards delivering what the business stakeholders want and avoid working on anything they doesn't contribute to that effort, while removing the internal data analytical barriers that can impede progress.

Whether, when, and how Tableau actually replaces data warehouses and enterprise BI for the delivery of valuable business information are questions undergoing vigorous debate. There are no clear lines segregating the realms—the boundaries are and will remain flexible. We'll cover at least some of this territory in future posts.

"Parameters" is an invalid Data Connection Name – here's why.

A Parameters Data Connection.

Suppose you've put together a data set that serves as the parameters you need in to analyze your data. Such a data set might be contained in the file Parameters.csv and look like this:
  Parameter,Date,Value
  QWERTY,8/2/2012,1357
  AEIOU,7/32/2012,2468

Now suppose you try and open this data source in Tableau, and that you would, quite naturally, want to name it "Parameters".

You'd open up Tableau's Text File Connection and name your new data connection "Parameters".

No dice.

Tableau won't let you use "Parameters" for your data connection name, and shows you this message:

The message is somewhat unclear. Here are its problems:

• Data source or data connection?
  "Parameters" was the name we provided in the previous dialog for "the connection", but we're being told here that "Parameters" is the name of an existing data source. Whichever it is, it should be named consistently.
• Where's "Parameters"? There's no evidence in the UI that any data sources exist, much less one named "Parameters". This is a mystery.
• Although the message asks politely for another name to be entered, there's no definitive statement that "Parameters" can't be used. (OK, this is picky, but friction comes in little bits.)

Why "Parameters" isn't allowed.

Tableau reserves "Parameters" for internal use as the data source that contains the individual user-created Parameters. It's not clear why Tableau implemented Parameters this way, with the same semantic naming structure as external data sources, whatever name it used for Parameters is necessarily unavailable for end user use as a data connection (source?) name. It's unfortunate that Tableau used "Parameters" instead of a technical name that would be highly unlikely to be used by users.

The evidence: an empty Tableau workbook TWB file (partial):

<?xml version='1.0' encoding='utf-8' ?>
<workbook version='8.1' xmlns:user='http://www.tableausoftware.com/xml/user'>
  <!-- build 8000.13.0319.1225                -->
  <preferences>
  </preferences>
  <datasources>
  </datasources>
  <worksheets>
    <worksheet name='Sheet 1'>
      <table>
        <view>
          <datasources>
          </datasources>
          <aggregation value='true' />
        </view>
        <style>
        </style>
        <panes>
          <pane>
            <view>
              <breakdown value='auto' />
            </view>
            <mark class='Automatic' />
          </pane>
        </panes>
        <rows></rows>
        <cols></cols>
      </table>
    </worksheet>
  </worksheets>

This file was created by saving a newly opened workbook. As we can see in lines #6 & #7, there are no data sources at all in place. If this workbook is closed and then re-opened, the structure will be the same. This brings up the question: if Tableau uses the name "Parameters" for its internal Parameters data source, where is it? Well, Tableau hasn't created it yet, but it reserves the right to "Parameters" for when it will. Which begs the question: when does Tableau create the Parameters internal data source? As far as I can tell, Parameters isn't created until Tableau creates the first parameter, as shown below:

One data source, no Parameters workbook.
This workbook has a single data source open: "Dummy Data.csv". There's no evidence of a Parameters data source.

<?xml version='1.0' encoding='utf-8' ?>
<workbook version='8.1' xmlns:user='http://www.tableausoftware.com/xml/user'>
  <!-- build 8000.13.0319.1225                -->
  <preferences>
  </preferences>
  <datasources>
    <datasource caption='Dummy Data#csv (Dummy Data.csv)' 
                name='csv.41568.014063067130' ... >
      <...>
    </datasource>
  </datasources>

Same workbook with one Parameter created.
This workbook has a single data source open: "Dummy Data.csv", and one parameter—"Parameters" has been created.

<?xml version='1.0' encoding='utf-8' ?>
<workbook version='8.1' xmlns:user='http://www.tableausoftware.com/xml/user'>
  <!-- build 8000.13.0319.1225                -->
  <preferences>
  </preferences>
  <datasources>
    <datasource hasconnection='false'
                inline='true'
                name='Parameters' version='8.1'>
      <aliases enabled='yes' />
      <column caption='Parameter 1'          datatype='real' 
              name='[Parameter 1]'           param-domain-type='any' 
              role='measure'                 type='quantitative' 
              value='1.0000000000000000'>
        <calculation class='tableau' formula='1.0' />
      </column>
    </datasource>
    <datasource caption='Dummy Data#csv (Dummy Data.csv)' 
                name='csv.41568.014063067130' ... >
      <...>
    </datasource>
  </datasources>

More interesting Parameter things.

One of the interesting things that comes out of this exploration are some hints that there's quite a bit of information that we can tell about parameters in the TWB. Which leads into a post or two, or few, in which we'll take a look and see what we can distill out of them.

About Parameters - Introducing The Phantom Parameter

A Mystery.
Or perhaps a Perplexment.

In the Workbook shown here, where does Parameter 1 come from? Why is it there? What use is it? What can be done with, for, and to it?

We're used to seeing Parameters listed in their own panel in the Data Window, but there is no such panel here.

This is one of those odd corner cases in Tableau that causes one to stop and wonder: "Now what on earth is going on here, and how did it happen?"

A Challenge.

Without looking below (no peeking), create a Tableau Workbook just like the one shown, with its own phantom Parameter. It's really not all that hard. Maybe.

Well, yes, it's odd, but does it really matter? If so, why?

This situation is problematic because it sits outside normal Tableau space—that set of configurations and behaviors that constitute the standard Tableau interface. By presenting an abnormal configuration that confounds expectations Tableau confuses the user, which in turn leads to, at least: time wasted in trying to recover from the situation (even if it's not strictly a fault or problem); effort and energy spent trying to understand what's happened, and why (some users won't care, some will dig into it until the mystery is fully solved); and a sense of mistrust, a suspicion that Tableau isn't as simple, straightforward, easy, and reliable as it could be. It's the accretion of these elements that ultimately undermines the whole product.

Why bother?

Why should I—or you—spend any time figuring out what's going on here? Particularly if it's not really a going to cause Tableau to crash, do something erroneously, or be a major usability problem.

Fair question. I've found that chasing down these rabbit holes reveals a lot about Tableau, how it's put together and how it works, and this deeper understanding leads to being able to use it better. And it's fun, in the way puzzling things out is.

Creating your own Phantom Parameter.     It's really not all that difficult, just follow these easy steps.

Open a new Tableau workbook.

You don't have to name it, although I did here, and it's a good practice.

As with all new workbooks there are no data connections, no dashboards, no parameters, and only the initial "Sheet 1" worksheet.

To this we'll add a single parameter, which will exist all by itself.

But not just yet.

Tableau doesn't provide any way for us to create or otherwise interact with Parameters in an empty workbook.

1. The Analysis menu's "Parameters" option is inactive.

2, 3. Right-clicking in either the Dimensions or Measures panel of the Data Window is the normal route to creating a Parameter.
As we see here, in the absence of any Data Connections, the right-click menu only offers the "Paste" option. (and it's not entirely clear at this point what can be pasted)

There's a reason for this behavior, which I'll identify below. Here's a clue for those of you who like puzzling this sort of thing out: it hinges upon what "empty" means.

Connect to a data source.

As it turns out, Parameters are dependent upon Data Connections for their existence. It's impossible to create a Parameter unless at least one Data Connection is open in the workbook.

So you'll need to connect to one.

I'm connecting here to my saved "Dummy Data" data source. It doesn't really matter what's in the data, how many or what types of fields it has, nor how much or little data it contains. The advantage "Dummy Data" has here is that it only contains one Dimension and one Measure, so that it takes up very little space in the Data Window, making it easier to see what's happening in the following steps.

Here's Dummy Data in the Data Window.

Create a Parameter.

Parameters are created via the Data Window's right-click menu available in either the Dimensions or Measures panel.

Accessing this menu can be a little tricky since each of the individual Dimensions and Measures has it's own right-click menu. Care must be taken to right-click in either the Dimensions or Measures panel in a space that's separate from the Dimensions and Measures.

Go ahead and create a Parameter. I let my new Parameter use the default configuration values.

Please note that I've adjusted these images from the actual Tableau UI, moving their elements into the width shown here. This has no effect on functionality, but makes it possible to show them in this blog's limited width.

Here's newly created Parameter 1

Tableau has created a Parameters panel for the Data Window and is showing Parameter 1 in it.

You can experiment with Parameter 1, although all you can really do is one of: left-click on it, in which case Tableau highlights it, colored green or blue depending upon its type; or right-click on it, which brings up the functional menu that gives you the opportunity to, among other things, choose the "Show Parameter Control" option.

Since we want to show the Parameter in the analysis space, you can go ahead and choose the "Show Parameter Control" from Parameter 1's right-click menu.

Or

You can choose "Parameter 1" from the Analysis menu's "Parameters" option, as shown here.

Of course, if you named your Parameter something other than "Parameter 1" that name will show up here.

Showing Parameter 1.

Now let's make it a phantom.

Close "Dummy Data".

Closing "Dummy Data" will remove it from the workbook.

From above, when there are no Data Connections in the workbook, Tableau doesn't provide any presentation for Parameters in the Data Window. Given this, what seems reasonable to you, in terms of Parameter 1?

Let's see what Tableau does – go ahead and close Dummy Data from its right-click menu.

Holy Cow!
We have a Phantom Parameter.

Interesting. But what, really, is a Phantom Parameter?

Simple, it's one that cannot be accessed to configure or manage it. We can't change anything about it—not its type, format, whether it accepts All values, a List or values, or values within a Range. We can't delete it.

We can toggle its visibility via the Analysis menu's "Parameters" option. I used it above because in this scenario it's the only way to show/hide the Parameter control.

If the Parameter's control is visible we can operate it, which is really pretty meaningless in this context, since there's nothing for it to affect. Is this a bad thing? I think it's moderately ungood. Not actively harmful, it's an oddity that introduces friction where there needn't be any.

As an interesting exercise, go ahead and try this:

• Hide Parameter 1
  use either the "Analysis|Parameters" menu or select the "Hide Card" option from Parameter 1's context menu (via the down-triangle glyph visible when you mouse onto Parameter 1).
• Once Parameter 1 is not visible, save and close the workbook.
• Now re-open the workbook — Parameter 1 isn't visible, nor is there a Parameters panel in the Data Window.
• Select the "Analysis|Parameters" menu option. There's Parameter 1, ready for presentation.

What's going on here?

The short version is that Parameters were implemented as a special of data source internal to the workbook. This makes sense from the perspective of the implementation being able to leverage Tableau's existing data management mechanisms, but it has consequences that are sometimes less than optimal for the Tableau user.

For example, the Parameters internal data source is named "Parameters", so Tableau won't let you name a Data Connection "Parameters"; if you try it Tableau shows you this message:

The message is misleading in at least two ways: the user has no visibility to the "Parameters" data source that Tableau claims already exists—Tableau never reveals "Parameters" in the user interface; and it's not always technically correct—Tableau reserves the "Parameters" Data Connection name even before it creates the internal Parameters data source (this is admittedly a very fine hair to split, but I think it's significant).

Is this dangerous?

Given this odd behavior, what does this mean? Does it signal anything about Tableau that's dangerous, or could cause problems for ordinary Tableau users?

Not to worry. This one little wrinkle doesn't, as far as I can tell, indicate that there's anything amiss that is cause for real concern.

It is, however, a manifestation of deeper, substantive aspects of how Parameters were implemented in Tableau that do have consequences that surface in different places. I'll be examining these in future posts.

A better design proposal.

The basic problem with Parameters that this situation reveals is that they've been implemented as a shadow data source that's only partially surfaced to the Tableau user interface. We've seen here one scenario in which there's the opportunity for real confusion, and confusion in a UI is invariably a bad thing.

It would make sense if parameters were first class Tableau citizens. The set of Parameters should occupy their own space separate and apart from the data sources. This would alleviate the situation of the name "Parameters" not being available for a user-named data connection.

As Parameters have an existence apart from other data sources— e.g. they persist in a workbook even if no data sources exits, they should be represented in the UI separate and apart from data sources. I think I see why their presentation within the Data Window might have seemed to make sense when Parameters were introduced, but it's one of those things that needs to be adjusted so that the UI accurately reflects Tableau's components' semantics. The counter-argument is that this would overload and complicate the user interface, that they're working OK as implemented, and that there's no real cry from the user community to change the way things are. This is a powerful argument, one that hard to contest. It's also the "things are good enough" argument that's led many once-successful endeavors down the path to obsolescence.

Clearly, improving how Parameters are handled in Tableau isn't likely to be trivial, and it may have wide ranging implications, up to and including reconsidering the set of uses they're put to and contemplating whether there isn't a better way to satisfy these needs. The specter of complexity is an inhibitor, but Tableau would be a better tool if at least the wrinkles identified here were ironed out.

Tableau Friction — Into the Future

It's several weeks now since TCC 2013 in Washington, D.C. and I haven't posted a thing. Not for a lack of topics—I left immediately after the conference for a sailing trip in the Cyclades and have very limited internet access. This post is a short placeholder until I get back in mid October.

About TCC 2013

I loved TCC 2013, got to reconnect with many people I know and respect, learned a lot, and wished only that I could be in more than one place at a time to take it all in.

Tableau Software and the Tableau community keep moving the wave front forward. The data analytical world is a better place because of Tableau, and Tableau is better because of the contributions of everyone who puts so much time, effort, energy, and brilliance into exploring and understanding it, and in extending a welcome and helping hand to all.

From what we've seen of the new 8.1 and 8.2 features Tableau isn't sitting still. The universe of possibility continues to expand.

The future of Tableau Friction.

This blog started out as a way for me to keep track of of Tableau's idiosyncrasies so that I could have a handy reference for them, and for workarounds where they exist.

My goals have broadened.

In the seven years I've been using Tableau I've watched it grow from an innovative disrupter to a powerful shaper of a new way of thinking about data analysis. Tableau has been the biggest agent of change in the data analytical landscape—the relationship between people and their data, and with other people to whom they need to communicate data-based insights and information is much different than it was ten years ago. More importantly, Tableau has led the way in changing the mindscape of people with data to understand, particularly in corporate environments where the simple straightforward activity of data analysis had become hostage to too-large, too-cumbersome, too-industrial processes that actively impeded obtaining information and insights from data in its native context.

One of the consequences of Tableau's success is that the door to data analytical innovation is wide open. Tableau introduced a new way of analyzing data, one that made it simple and easy for people without database and programming skills to access and analyze their information. Now that the idea that data analysis should be simple and easy has taken root and flourished more people are exploring ways to broaden the reach of data analysis, to new types of data, new types of analytical functionality, and new types of analytical modeling and visualizations. The data analysis horse is out of the barn. I intend to explore and cover as much of the ongoing innovation as I can, with an eye towards how it intersects with Tableau's realm.

Continuing coverage of Tableau's friction points.

I will continue to point out those aspects of Tableau that don't work as well as they should, from atomic point problems to systemic design flaws—including where a lack of coherent design impedes functionality and usability. These things are within the original Tableau Friction horizon. There's quite a queue of these topics; the challenge will be to identify the individual friction points, how they relate to one another, and how they reflect higher level organizational characteristics.

As Tableau has grown it really does appear that it's been extended by cobbling together new features in relative isolation from one another, sometimes by employing existing machinery for new purposes, other times by developing entirely new mechanisms. On the whole, there's a lack of coherence across the full Tableau functional spectrum, and this coherence is itself a major source of friction in the product, with passive and active consequences that impede people in their data analysis.

The evolving universe of Machine Assisted Data Analysis (MADA).

When Tableau was invented the machine-stored data universe was dominated by a particular paradigm, one whose dominance was so comprehensive that a whole generation of people knew only it and assumed that it was the one, the only true way to store data. I refer of course to the relational model. And not to Ted Codd's relational model, but of the more general, degenerate concept of data stored in tables of rows and columns. The hows and whys of this model achieved dominance are interesting and informative, but tangential to this post (and I'm sorely tempted to go off on it), but the upshot is that Tableau was designed (I believe) to access tabular information, with the limited exception of analytically less-flexible access to "non-relational" data sources. This initial design space has, like all paradigm-bound endeavors, limitations and boundaries that may be hard or soft horizons. OK, I hear you thinking: "What? Nice sentence, but does it mean anything to me?" Fair enough. Put another way, Tableau's original creation to access tabular data with particular analytical visual semantics may make it unable to adapt to the analysis requirements of other (non-tabular) types of machine-stored data.

Tableau is already limited by a lack of the ability to represent and analyze tablular, flat data, and has been for quite some time. This is a bold claim, and controversial in some quarters. But I hold it to be true. There has long been a need within Tableau for data analysts and dashboard authors to be able to represent non-relational data, e.g. the different components of their workbooks in ways that make it easy to see, understand and manage them. To cite just one example: the difficulty of understanding the relationship between data sources and worksheets has always existed and was exacerbated with the introduction of global filters, which benefited and vexed us, and which have a long train of follow-on effects. The main reason I created TWIS, and (I think) Andy Cotgreave created his TWB Auditor, is to answer these and related questions. The main problem with these solutions is that they exist outside of Tableau, and even though it's possible to create relational data sets covering all of the relationships between workbook elements so that Tableau can be used to analyze them, there are a prohibitively large number of individual tables required—one for each distinct path in the relationship graph.

The post-relational MADA universe.

The future is here, and always has been.

The relational model for storing data was preceded by other models that combined the structure and content of data. The relational model became the dominant approach because of a convergence of factors, none of which were it being a superior way to store data so that humans could make sense of it. In fact, from the perspective of real human people wanting to understand their data the relational model was, and is, a curse and a pox and an abject failure.

Fortunately, non-relational data is making a comeback. The real question for the purposes of this blog is: "Will Tableau adapt and become the best tool for helping make sense of all this new data?" Time will tell.

Non-relational data.

Data is increasingly found in a wide variety of non-relational forms, from XML files to (mostly) web-related structures such as JSON and YAML, to noSQL and big data technologies. Tableau workbooks are XML files. Tableau Server uses YAML internally. JSON is rapidly becoming a significant, even leading data format on the web. noSQL data is becoming more and more prominent.

The common basic characteristic of the non-relational forms is that they accommodate structural relationships in the data, from the simplest two level master-detail hierarchies to arbitrarily complex networks. The great benefit from the analytical point of view is that it's possible to model data naturally, with the structure and content reflecting real-world relationships.

If Tableau is to evolve into a tool that has a place in the post-relational MADA universe it will need to provide a way to model, represent, and analyze post-relational data as clearly and easily as it has relational data. The good news is that Tableau's inventors cracked a tough nut once in bringing a visual interface to what had previously been a programmatic or mechanical endeavor, that Tableau is recognized as the leader in post-Big BI, and Tableau has demonstrated the ability to gather the resources necessary to be in the game for the long term.

I'm preparing a post on the characteristics of the first step up from the relational model—a master-detail hierarchical relationship, specifically of an organization's departments and the employees working in each. This simple example introduces many of the concepts any tool will need to express in order to successfully help people access and understand their data.

I also plan on extending these concepts and principles to higher level data structures. There are some tough problems to solve – one way to look at the situation is that the world isn't exactly awash in highly effective visual XML editors, and XML has been with us for quite some time now.

Evolving technologies.

There is a highly vibrant, even aggressive, evolution going on in data analysis technologies. The most visible of these leverage web-associated technologies, with Javascript-based implementations manipulating SVG and the HTML 5 canvas out in front of the parade. Examples include: D3.js/; Raphaël; and Crossfilter.

The best of these technologies can, in the hands of someone appropriately skilled in their programmatic use and analytical information design, provide data visualizations equal in quality to Tableau's. This might sound like hearsay, but it's prudent to remember that this has always been possible, and that Tableau's great gift was removing the technological programming expertise from the analytical process.

Technologies vs tools.

As the new technologies evolve they're gaining higher and higher levels of abstraction, bringing the data analytical operations closer and closer to the surface. It's really only a matter of time until the final step is taken and tools appear that meld the technologies' capabilities with human-oriented interfaces, resulting in the next generation of data analytical tools. In this scenario these new tools will be the next wave of disrupters, particularly given that they're starting out with an intrinsic understanding with post-relational data, which will make the development of the tooling less a conceptual hurdle that needs to overcome an entrenched paradigm and more a pure design problem centered on finding an effective visual language for representing the essential data analytical operations.

New tools.

Tableau and its cousins have spawned a host of imitators. Companies are springing up striving to be the next great thing in rapid, effective data visualization. Many of them have been founded by hungry young people who believe that they're onto the next great idea, and some of them have very good ideas indeed. Tableau has, by virtue of its own success in leading the creation of an entire new product space, demonstrated that it's possible to bring a good idea to life and, with hard work and luck, upend the conventional wisdom and make a real mark.

I'll cover some of these new tools features that I think are interesting and valuable, and will relate them to Tableau's way of doing things whenever possible. A couple of features that I've seen recently bear mentioning because they highlight valuable data-analytical functionality that aren't in Tableau's wheelhouse.

The first is something everybody wants: search. Simple, plain, ordinary search for all of the data that contains a word or phrase. One new product provides a simple Google-like text input are into which the user can type whatever they want and the data will be filtered to contain only those records containing the text entered. The wrinkle is that the user doesn't need to specify which fields will be examined, by default all of the fields in the current context will be evaluated to determine if they contain the content. This is a really impressive feature. Yes, there are a whole host of background things that come into play, things that matter and need to be addressed, but right up front the user doesn't need to know about or deal with them. And isn't that the whole point?

The second interesting feature is natural language recognition. One product provides, again, a simple text input in the UI into which the user can type simple English language that the tool can interpret as meaningful data analysis instructions, like "show employees with salary over 5,000". Watching someone see this for the first, and seeing it work, is interesting – the machine is responding to them in a way that they find easy and natural. It's worth noting that this isn't really a new feature. There have been commercially successful English-based data analytical languages for decades, since at least 1970. The difference now is that the use of the query language is, or can be, integrated into a tool that presents the opportunity to use the language as an element of and adjunct to the analytical process, not as the whole shebang.

Wrap up.

The universe of machine assisted data analytical possibilities is expanding. And it's expanding at an accelerating rate. There hasn't been this much fertile activity in, well forever. As the universe expands the opportunities to provide the best possible tool for helping people access and understand their data grow in parallel.

Tableau is in the catbird seat. The future is wide open.