Monday, December 17, 2012

Changing the Database Server Name in multiple Workbooks

It's good practice to use Tableau against a non-production database in many circumstances, e.g. when the database is under active development, you're early in the data exploration phase, or access to the production database is restricted.

Tableau workbooks tend to multiply, and their data connections in them multiply apace.

Changing the properties for a data connection or two isn't difficult, nor is it terribly burdensome. But it's a mechanical process, and after making the same change through the UI more than a couple or few times becomes boring, and when there's more than a few Workbooks it's really difficult to keep track of all the connections that need to be changed, those that have been, and those that have not yet been.

Here's a way to change the database server for a whole collection of Workbooks all at once.

Simple and easy, and in keeping with the small-apps approach, I've created a small Ruby app that scans all the Workbooks in a directory, locates those data connections with a server specified by name, and change the servers' names to a new one.

How it works — the "before" picture:

In the top section of the image below is a list of the Workbooks in the current directory. The bottom section shows the server names of the various database connections in these Workbooks.

For this post, we're going to change the "Ora.development.env.server" server names to "Ora.production.env.server"

How it works — making the changes:

In the first section of the image below I've executed the Ruby script: TTC_cds.rb. It first asks if I want to change database server names, and upon getting the answer "yes" asks for the existing (old) server name, and the new name to change it to.

Some important points — TTC_cds.rb will:

  • ONLY change the names for those servers with the one it's scanning for;
  • make a backup of those Workbooks that contains data connections to change to the new names; and
  • keep a record of the Workbooks it has scanned, and the changes it's made, in the CSV file TCC_ChangedDataSources.csv, where it is available for reviewing with Tableau.

The middle and bottom sections below correspond to the image sections above. We see that the server names have been changed according to our specifications.

A Tableau viz of the changed server names

The Magic Sauce

OK, it's just plain old Ruby code, but it works some magic.


# TCC_cds.rb - this Ruby script Copyright 2012, Christopher Gerrard require 'nokogiri' require 'open-uri' require 'fileutils' $oldServer $newServer def getInput work = false print "\n\tChange data sources? " if gets.downcase =~ /y|yes/ then print "\t Old Server Name: " $oldServer = gets.chomp if !$oldServer.nil? print "\t New Server Name: " $newServer = gets.chomp if !$newServer.nil? work = true end end end return work end def init work = getInput if work then $f = File.open("TCC_ChangedDataSources.csv",'w') $f.puts 'Workbook,Server Changed?,Server - old,Server - new' unless $f.nil? end return work end $xmlDoc = nil def oldServer? twb $xmlDoc = Nokogiri::XML(open(twb)) sNodes = $xmlDoc.xpath("/workbook/datasources/datasource/connection[@server=\"#{$oldServer}\"]") if sNodes.empty? return false else return sNodes end end def backupTWB twb original = File.basename(twb) $origCopy = original + '.originalDS' FileUtils .cp(original, $origCopy) end def changeServer twb, serverNodes serverNodes.each do |node| node['server'] = $newServer end nsf = File.open(twb,'w') nsf.puts $xmlDoc nsf.close puts "\t\t changed: #{twb}" puts "\t\t original: #{$origCopy}" end def changeDataSources twb serverNodes = oldServer? twb if serverNodes backupTWB twb changeServer twb, serverNodes $f.puts "#{twb.gsub(/"/,'""')},true,#{$oldServer},#{$newServer}" else $f.puts "#{twb.gsub(/"/,'""')},false" end end if init then puts Dir.glob("*.twb") {|twb| changeDataSources twb } end

 

No Workbooks are harmed during the database server renaming.

Your workbooks with server name changes are safely backed up.

How to use TTC_cds.rb

  • Prerequisites
    • Minimal technical skills.
    • Have Ruby installed and ready to run.
    • Have the Nokogiri Ruby gem installed—it's used in the XML parsing.
    • Have TTC_cds.rb in place—it doesn't matter where, or what name you use, as long as you know where it is.
      You can copy the code above and paste it into your favourite text editor.
  • Running TTC_cds.rb
    • Open a command prompt.
      (you can run it otherwise, but this is simple and straightforward)
    • CD to the directory containing the Workbooks you're interested in.
    • Run it: "[path to]\ruby    [path to]\TTC_cds.rb"
    • Note, unlike some of my other apps,TTC_cds.rb only looks in the current directory.
  • Presto. You now have successfully changed your Workbooks' database server names, where appropriate

The usual caveats.

TTC_cds.rb works fine with the limited testing it's been through, but it's definitely not bulletproof and hardened. It's entirely possible that it won't work on your Workbooks, or somehow mangles them. So use carefully. Always back up your Workbooks, or only run TTC_cds.rb against copies. .

I hope it works for you, but make no guarantees. If you do use it and make improvements I hope that you'll post them back here as comments so I can learn from them, and hopefully other people can benefit from them too.

Saturday, December 15, 2012

Understanding tabcmd with Tableau

Clarifying the tabcmd documentation

The Tableau Public-published workbook below is an example of using Tableau to improve upon something that's not generally considered to be analyzable data: the Tableau Server tabcmd online help documentation.

Although the online doc is reasonably comprehensive it's not readily digestible. The content's presentation space is very large and sparse — it's harder than it could be to get to the particular command you need, even harder to achieve a comprehensive understanding of the scope and details of the things tabcmd can do. If this were a normal Tableau Friction post, or a web content design critique, I'd point out the flaws in the online content. But it's not, and I'm not.

Instead, this post celebrates Tableau's ability to take a bunch of web stuff and render it in a way that actively supports our ability to find what we need, when we need it, and to gain a richer information context.

All in all, this is a pretty handy way to get to the tabcmd command you want, to understand what it does, how it does it, and how to get it to work.

If you find this interesting or valuable, please let me know.

If you find a way to improve it, I would really like hearing about it.

Tuesday, December 4, 2012

Technical-, not Human-Oriented Field Calculations

Tableau's Field Calculations — Sometimes a Mystery

Tableau sometimes does things in a way that makes sense from a particular technical perspective, but leaves non-technical people bewildered, wondering what happened. If they're lucky enough to realize that what did happen wasn't in line with their perfectly reasonable expectations.

This post illustrates a situation where Tableau's Calculated Field evaluation results in values for the calculated field that are in some circumstances not what the User intended or expected.

In this scenario, there are a set of records containing Budget, Released, and Spent values for US and Canadian States and Provinces. The User is interested in determining how much of the budget is unspent, or surplus, which from his/her perspective is exactly what the common meaning of that statement is.

Arithmetically the relationship is Budget – Spent, and in the case where no money has been spent the unspent amount is the full Budget amount. As we see below, when the data is in an Excel spreadsheet and the formula {Budget} - {Spent} is evaluated the result is exactly as intended. However, the arithmetically equivalent formula in a Tableau Calculated Field will produces an erroneous, from the human perspective, result when there is no data for the relevant Spent field.


Tableau's great grace is its human-oriented design that lets the User do the thing most natural to their experience and expectation with the confidence that Tableau will do the right thing.

As this example shows, Tableau sometimes doesn't live up to this promise. But that can, and should, be remedied.

Monday, November 26, 2012

Dashboard Imaging — see beneath your Dashboard's skin.

When building Tableau dashboards it's helpful to know what's already in them – placing and organizing components is affected by the components already in there.

Where you can place things, and how they behave when you do is particularly sensitive to the presence of layout containers. And there are multiple types of layout containers—horizontal and vertical containers are available in the dashboard window so you can place them into the dashboard, although their behaviors aren't well, if at all, documented. For example, a vertical container pushes it's contained components into the minimal space at the top, which is handy as it's the only way one can top-align text, the regular dashboard text component lacking that ability.

There are at least two other containers that Tableau puts into dashboards as you add and manipulate components. Tableau uses these for its own purposes, and even though they affect the dashboard's layout and behavior, Tableau doesn't let you know that they're there, although in some cases (I haven't quite puzzled this through) they can be seen via a component's "Select Layout Container" right-click option.

It would be nice to see what component panels are in a dashboard.

For example, this Tableau Public-published workbook contains "Dashboard 8", which has a variety of components.

Dashboard 8 is nothing special, there appear to be a dozen or so visible components in it, but how to tell for sure, and how do we tell what (layout containers are in it, abd what their types are?

Imaging Dashboard 8

Dashboard 8 has 20 components.

The image above shows Dashboard 8's components laid out so as to show them in context, with their essential characteristics including type, dimension, and placement visible. The diagonal lines span the horizontal and vertical containers, forward-leaning for horizontal, backward-leaning for vertical. The image shows a single HTML page with the schematic on the left, and Dashboard 8's components in the table on the right, one component per row.

The static image is nice, but there's more.

When one mouses over a table row the corresponding component is highlighted in the schematic. The video below shows this in action. (The screen capture didn't grab the mouse.)

How'd your get that schematic/HTML page?

I'm glad I asked. The HTML page is generated by my latest Ruby tool. When run it locates all of the TWBs in the local directory, and in all sub-directories, and creates a separate HTML page for every dashboard in every workbook. Running on my local machine it takes a minute or so to find and create the HTML for 1308 Dashboards in 662 Workbooks.

Live Action Highlighting

Sunday, November 11, 2012

Finally - you can use the Tableau data colors in your formatting.

Yes, it's true. You can now harmonize your color schemes by using the Tableau data colors in your formatting.

The short version — use the Worksheet below to find the RGB values for the Data Color Palette colors you need and then use those values in the Format coloring of the objects you need colored.
One of Tableau's great strengths is its exceptional proficiency at coloring the various visualization elements. The defaults it chooses have been carefully crafted to work very well under a wide variety of circumstances, and Tableau provides a good set of mechanisms for configuring the colors where needed..
But...
It's really frustrating that Tableau uses two different mechanisms for configuring colors, and the colors aren't consistent across them.

Coloring data.

The colors contained in the data-coloring Palettes are both excellent choices for visualization and fixed sets that can't readily, if at all, be extended or otherwise changed.

Format coloring.

Tableau's formatting systems actually provide two different palettes, but share a common mechanism for specifying the RGB values that specify the full set of available colors.
So...
It's possible to make the formatting colors match the data Palette colors via the RGB specification mechanism
if only.. the RGB values of the data Palette colors were known. But Tableau doesn't reveal them. At least not through its User Interface.

The Tableau Data Color RGB Values

The Tableau Public-published Workbook embedded below provides the RGB values for every color in every Tableau data palette so you can use them in configuring the colors in your Worksheets and Dashboards. Everything that can be color-formatted can now use the same colors as Tableau makes available for data coloring.

But wait, there's more

There's quite a lot more to the Tableau color story, e.g.
  • Many of the data colors are used in multiple palettes.
  • Many of the colors are visually nearly indistinguishable from one another.
  • Colors are configured to elements at the data source ‐ explaining why changing the colors in one worksheet changes the colors in others. This is very similar to other scoping situations where Tableau doesn't really notify one that local actions have non-local side effects.
  • When the Automatic Palette is in effect, no specific values are used or recorded.
  • The Automatic Palette uses the same colors as the Tableau 10 Palette.
  • It's not visible, but once Tableau is configured to color field elements it remembers those colorings even when the field is no longer in the data source. This could lead to serious downstream problems.

Look through the Dashboards and Worksheets to see more.


Thursday, November 8, 2012

Dashboard Images Revealed

Some interesting dashboard image facts:

You can embed images in your dashboards. OK, that's not news. Things you may not realize:

  • You can embed images in your dashboards. OK, that's not news.
  • You can specify a PDF as the image file, but Tableau won't produce an image.
    (you can embed the PDF in a Web Page component)
  • Once you've inserted the image Tableau doesn't show you the image's source, so there's now way to tell what file is being shown.
  • Tableau doesn't forget the images you've put in the Image component—if you add a second image, Tableau remembers the name of the first; add a third, the second is remembered, etc.
    So be careful about using placeholder images, their ghosts will be lurking about.

Revealing your Dashboards' Images

The Ruby script below will scan the current and all descendant directories for TWBs and for each one found will identify the images and report on them with the Workbook, Dashboard, and Image details. The information is captured as data in the CSV file "TCC_DI.csv", which you can analyze with Tableau.


# TCC_DI.rb - this Ruby script Copyright 2012, Christopher Gerrard # - show the images in the Workbooks' Dashboards require 'nokogiri' require 'open-uri' $twbCnt, $twbNum, $recNum = 0, 0, 0 def init $f = File.open("TCC_DI.csv",'w') $f.puts 'Record #,Workbook,Workbook Dir,Dashboard,Image,Image Dir' unless $f.nil? print "\n\tScanning for Dashboard images\n\t" end def showImages twb doc = Nokogiri::XML(open(twb)) $twbCnt += 1 dashes = doc.xpath('//workbook/dashboards/dashboard' ) if dashes.length > 0 then $twbNum += 1 twbName = File.basename(twb) twbDir = File.dirname(twb) end dashes.each do |d| dName = d.xpath('./@name').text images = d.xpath('//zone[@type="bitmap"]') images.each do |i| iName = i.xpath('./@param').text print '.' imgName = File.basename(iName) imgDir = File.dirname(iName) $f.puts "\"#{$recNum+=1}\",\"#{twbName}\",\"#{twbDir}\",\"#{dName.gsub('"','""')}\",\"#{imgName}\",\"#{imgDir}\"" if $recNum.modulo(72) == 0 then print "\n\t" end end end end init Dir.glob("**/*.twb") {|twb| showImages twb } $f.close unless $f.nil? plural = if $twbNum == 1 then '' else 's' end puts "\n\tDone.\n\tFound #{$recNum} images in #{$twbNum} Workbook#{plural} of #{$twbCnt} Workbooks scanned" if $recNum > 0 then puts "\tTCC_DI.csv contains the image names & the Dashboards containing them." end

 

No Workbooks are harmed during the revealing of the Images.

Your workbooks are unaffected, nothing is done to them.

How to use TCC_DI.rb

  • Prerequisites
    • Minimal technical skills.
    • Have Ruby installed and ready to run.
    • Have the Nokogiri Ruby gem installed—it's used in the XML parsing.
    • Have TCC_DI.rb in place—it doesn't matter where, or what name you use, as long as you know where it is.
      You can copy the code above and paste it into your favourite text editor.
  • Running TCC_DI.rb
    • Open a command prompt.
      (you can run it otherwise, but this is simple and straightforward)
    • CD to the directory containing the Workbooks you're interested in.
    • Run it: "[path to]\ruby    [path to]\TCC_DI.rb"
    • Note, TCC_DI.rb also looks through all the directories below the current one.
  • Presto. You now have a TCC_DI.csv file containing the information about your Dashboard images, if any.

The usual caveats.

TCC_DI.rb works fine with the limited testing it's been through, but it's definitely not bulletproof and hardened. It's only been run against Tableau v7 Workbooks, and it's entirely possible that images in other Tableau version Workbooks, or images coded in the TWBs in a manner other than the one in my sample TWBs not found might not be detected.

I hope it works for you, but make no guarantees. If you do use it and make improvements I hope that you'll post them back here as comments so I can learn from them, and hopefully other people can benefit from them too.

Tuesday, November 6, 2012

Fallacious Overspecificity

This is a happy day. I'm writing about a situation where Tableau is the solution to a particularly pernicious problem. The lubricant, if you will allow me, that frees the stuck gears of informed discourse by elimination a particular point of friction.

I attended a session today at the annual Tableau Customer Conference, TCC 2012—Stat Spotting: A Field Guide to Identifying Dubious Data, by Dr. Joel Best, Professor, University of Delaware.

Dr. Best covered a number of ways in which numbers are misused, often to great effect, in support of a wide variety of claims across the culture. Many of them are only too familiar to those of us who were raised numerate, and it was a great comfort that these abuses of numeracy are a formal topic of rigorous study.

I chose to work in my profession because I believe that facts and data are important aids that people can take advantage of and employ to make better informed decisions. With more than 25 years in BI one recurring theme I've seen is one Dr. Best identified, which goes something like this:

New! Improved! 106.7% cleaner in independent blind evaluations!

703% ROI in just 11 months.

16.7 people are injured every month in their slippers.

Raise your child's IQ by 14.8% in only 16 short weeks!

We can implement your Enterprise BI solution in 23.4 weeks, starting when we begin the discovery phase.

All of the above employ the all too common tactic of using precise values as a persuasive tool. All of them are also lying, using values with a degree of precision that's either impossible to exist, impossible to measure within the standard conventions of arithmetic precision, or simply monkeys-out-the-butt fabrication.

They're all examples of fallacious overspecificity—the use of a numerically precise value for a deceptive purpose without regard for whether the value is or could be real or true.

One of the great advantages of using Tableau is that it dramatically reduces the effort to "check the numbers" and see what, if any, data evidence supports claims that smell of fallacious overspecificity.

Tableau's great virtue is that is makes it simple to explore and examine data. This directly challenges fallacious overspecifiers because of the ease with which the facts and number behind the numbers can be examined to validate or refute the truth of the claim. This benefit is cumulative, as data analysis becomes the norm numeracy increases and eventually the penalties for misreporting or misrepresenting numeric values tilt the cost-benefit assessment strongly in favor of truth and accuracy.

Monday, November 5, 2012

Unhide that Worksheet!

Or – Worksheet? What Worksheet? I don't see no stinking Worksheets.

Hidden Worksheets are one of Tableau's enduring mysteries. Once hidden, a Worksheet's only visible presence in Tableau is as a component of the Dashboard(s) in which it appears.

Their stealthy nature makes them difficult to work with. But there are ways to find them:

  • the simplest is to look through the Dashboards until they're spotted but this is tedious at best and since well-designed Dashboards tend to camouflage their individual Worksheets it's also very error-prone.
  • You can use an automated Workbook analysis approach, e.g.
    • The Tableau Companion (aka TWIS) is a full featured Workbook analysis and inventorying system. Its unique feature is the creation of diagrams—PDFs, GIFs, and SVGs, showing the Dashboards and Worksheets in your Workbooks:
      If TCC is a bit much, you could use...
    • Andy Cotgreave's TWB Auditor is a very nice, polished, easy to use way to get the information, or
    • This post presents a short (37 line) Ruby script that will pull the Workbook-Dashboard-Worksheet information from your Workbooks so that you can see what's where. And the code is right there so you can be sure of what it's doing, or even improve it if you want.

So that's where the Worksheets are. Now what?

Knowing where the Worksheets are is very nice, but it only addresses part of the problem. The hidden Worksheets are still hidden, and unhiding them means opening up their Workbooks with Tableau Desktop, and for each hidden Worksheet navigating to a Dashboard containing the hidden Worksheet, "Going to" the Workbook, and right-click 'unhiding' it.

Unless...

You download or copy/paste and run the following Ruby script in the directory containing the Workbooks you want the Worksheets unhid in. ('Unhid'–is that a real word?)


# TCC_sW.rb - this Ruby script Copyright 2012, Christopher Gerrard require 'nokogiri' require 'open-uri' $hwTwbs = 0 $recNum = 0 def init $f = File.open("TCC_HWs.csv",'w') $f.puts 'Record #,Workbook,Worksheet,class,auto-hidden,maximized' unless $f.nil? puts "\nHidden Windows found for Workbooks:\n-----------------------------------" end def showWindows twb doc = Nokogiri::XML(open(twb)) h1 = handleHiddenNodes(twb, doc, '//workbook/windows/window[@class="hidden-worksheet"]' ) h2 = handleHiddenNodes(twb, doc, '//workbook/windows/window[@auto-hidden="1"]' ) if h1 || h2 writeTWB(doc, twb) end end def handleHiddenNodes(twb, doc, xPathStr) nodes = doc.xpath(xPathStr) if nodes.length > 0 then unhideWindows(nodes, twb) hidden = true else hidden = false end end def unhideWindows(wNodes, twb) puts twb wNodes.each do |n| puts n wName = n.xpath('./@name') wClass = n.xpath('./@class') autoH = n.xpath('./@auto-hidden') max = n.xpath('./@maximized') $f.puts "#{$recNum += 1},#{twb},#{wName},#{wClass},#{autoH},#{max}" unless $f.nil? n.set_attribute('class','worksheet') n.set_attribute('auto-hidden','0') n.set_attribute('maximized','0') end end def writeTWB(doc, twb) newTwb = File.open("#{twb.sub(/.twb$/,'')}.unhidden.twb",'w') newTwb.puts(doc) newTwb.close end init Dir.glob("*.twb") {|twb| showWindows twb } $f.close unless $f.nil?

What TCC_sW.rb does.

Each of the Workbooks in the current directory is opened up and examined for the presence of hidden Worksheets. If any are found they are unhidden and a copy of the Workbook is made with the unhidden Worksheets.

No Workbooks are harmed during the unhiding of the Worksheets.

The name of the new copy Workbook is the same as the name of the original Workbook with the trailing ".twb" replaced with ".unhidden.twb"

For example, the Workbook "FinancialAnalyses.twb" is copied, Worksheets unhidden, to ""FinancialAnalyses.unhidden.twb""

How to use TCC_sW.rb

  • Prerequisites
    • Minimal technical skills.
    • Have Ruby installed and ready to run.
    • Have the Nokogiri Ruby gem installed—it's used in the XML parsing.
    • Have TCC_sW.rb in place—it doesn't matter where, or what name you use, as long as you know where it is.
      You can copy the code above and paste it into your favourite text editor.
  • Running TCC_sW.rb
    • Open a command prompt.
      (you can run it otherwise, but this is simple and straightforward)
    • CD to the directory containing the Workbooks you're interested in.
    • Run it: "[path to]\ruby    [path to]\TCC_sW.rb"
  • Presto. You now have a set of your Workbooks with all of their Worksheets unhidden.

The usual caveats.

TCC_sW.rb works fine with the limited testing it's been through, but it's definitely not bulletproof and hardened. It's only been run against Tableau v7 Workbooks, and it's entirely possible that hidden Worksheets in other Tableau version Workbooks, or other types of Worksheet-hiding conditions might not be detected.

I hope it works for you, but make no guarantees. If you do use it and make improvements I hope that you'll post them back here as comments so I can learn from them, and hopefully other people can benefit from them too.

No Workbooks are harmed during the unhiding of the Worksheets.

The name of the new copy Workbook is the same as the name of the original Workbook with the trailing ".twb" replaced with ".unhidden.twb"

For example, the Workbook "FinancialAnalyses.twb" is copied, Worksheets unhidden, to ""FinancialAnalyses.unhidden.twb""

How to use TCC_sW.rb

  • Prerequisites
    • Minimal technical skills.
    • Have Ruby installed and ready to run.
    • Have the Nokogiri Ruby gem installed—it's used in the XML parsing.
    • Have TCC_sW.rb in place—it doesn't matter where, or what name you use, as long as you know where it is.
      You can copy the code above and paste it into your favourite text editor.
  • Running TCC_sW.rb
    • Open a command prompt.
      (you can run it otherwise, but this is simple and straightforward)
    • CD to the directory containing the Workbooks you're interested in.
    • Run it: "[path to]\ruby    [path to]\TCC_sW.rb"
  • Presto. You now have a set of your Workbooks with all of their Worksheets unhidden.

The usual caveats.

TCC_sW.rb works fine with the limited testing it's been through, but it's definitely not bulletproof and hardened. It's only been run against Tableau v7 Workbooks, and it's entirely possible that hidden Worksheets in other Tableau version Workbooks, or other types of Worksheet-hiding conditions might not be detected.

I hope it works for you, but make no guarantees. If you do use it and make improvements I hope that you'll post them back here as comments so I can learn from them, and hopefully other people can benefit from them too.

No Workbooks are harmed during the unhiding of the Worksheets.

The name of the new copy Workbook is the same as the name of the original Workbook with the trailing ".twb" replaced with ".unhidden.twb"

For example, the Workbook "FinancialAnalyses.twb" is copied, Worksheets unhidden, to ""FinancialAnalyses.unhidden.twb""

How to use TCC_sW.rb

  • Prerequisites
    • Minimal technical skills.
    • Have Ruby installed and ready to run.
    • Have the Nokogiri Ruby gem installed—it's used in the XML parsing.
    • Have TCC_sW.rb in place—it doesn't matter where, or what name you use, as long as you know where it is.
      You can copy the code above and paste it into your favourite text editor.
  • Running TCC_sW.rb
    • Open a command prompt.
      (you can run it otherwise, but this is simple and straightforward)
    • CD to the directory containing the Workbooks you're interested in.
    • Run it: "[path to]\ruby    [path to]\TCC_sW.rb"
  • Presto. You now have a set of your Workbooks with all of their Worksheets unhidden.

The usual caveats.

TCC_sW.rb works fine with the limited testing it's been through, but it's definitely not bulletproof and hardened. It's only been run against Tableau v7 Workbooks, and it's entirely possible that hidden Worksheets in other Tableau version Workbooks, or other types of Worksheet-hiding conditions might not be detected.

I hope it works for you, but make no guarantees. If you do use it and make improvements I hope that you'll post them back here as comments so I can learn from them, and hopefully other people can benefit from them too.

Friday, November 2, 2012

So, was that extract refreshed or not?

The need: have a Tableau Server-published data extract refreshed

Here's what the Tableau Server documentation says about using tabcmd for the job:


refreshextracts workbook-name or datasource-name

Performs a full or incremental refresh of extracts belonging to the specified workbook or data source. This command takes the name of the workbook or data source as it appears on the server, not the file name when it was published.
Examples
tabcmd refreshextracts --datasource sales_ds
tabcmd refreshextracts --workbook "My Workbook"
tabcmd refreshextracts --url SalesAnalysis
Option (short)Option (long)ArgumentDescription
 --incremental Runs the incremental refresh operation.
 --synchronous Runs the full refresh operation immediately in the foreground.
 --workbookName of a workbookThe name of the workbook containing extracts to refresh. If the workbook has spaces in its name, enclose it in quotes.
 --datasourceName of a data sourceThe name of the data source containing extracts to refresh.
 --projectName of a projectUse with --workbook or --datasource to identify a workbook or data source in a project other than Default. If not specified, the Default project is assumed.
 --urlURL name of a workbookThe name of the workbook as it appears in the URL. A workbook published as “Sales Analysis” has a URL name of “SalesAnalysis”.


The documentation seems to say that executing tabcmd with the proper options will cause the extract to be refreshed.

So I gave it a go, and here's the result:

C:\Program Files (x86)\Tableau\Tableau Server\7.0\bin>tabcmd refreshextracts --url "DashboardWithTDE" ===== Continuing previous session ===== Server: http://[TableauServer]/ ===== Username: adminName ===== Scheduling extracts for workbook 'DashboardWithTDE' to be refreshed now... ===== Succeeded ===== Workbook DashboardWithTDE' scheduled to have its extracts refreshed.

As it turns out, there was no visible evidence that the extract did get refreshed, including checking Tableau Server's maintenance section and looking at the published Dashboard.

So I had to download the Workbook, manually refresh the extract, and then republish the Workbook in order to make sure the published Dashboard contained up to date information. Which is what I was hoping to avoid doing all along.

Wednesday, October 31, 2012

Inconsistent Chart/Table Formatting

I'm frequently asked to provide an auxiliary table containing the specific numbers for the marks in a specific chart. And it's quite often a good analytic information design choice, better than simply displaying the chart's mark labels.

Creating the table should be an easy, even trivial chore: duplicate the sheet, drag the measure from the Rows or Columns shelf onto the Marks' Text shelf, and Presto! we have a table matching the chart with numbers in the cells instead of marks.

Only it doesn't quite work out the way I intended.

True, the cells do now contain the measure's numeric values, which is the primary point.

But the overall table formatting is very dissimilar to the original chart formatting, and getting them all polished up and in sync in order to present a Dashboard showing a coherent, consistent design requires a fair bit of fiddling about.

Notable elements that need to be changed include:

  • Name alignment – right-aligned in the chart, left-aligned in the table. Either one may be suitable in a given situation, but they need to be the same.
  • Row shading in the table – the default row shading is one of my little itches, and its presence here is particularly irksome.
  • Column dividers – present in the chart, absent in the table. Again, they should be the same.

All in all, adjusting the table formatting to match the chart isn't that big a deal, but when one is building out a fair number of dashboards the amount of persnickety work can really interfere with the creative flow.

Possible fixes

  • Duplicate a sheet's formatting along with the sheet itself.
  • Provide a mechanism for establishing and managing the default formatting for the various via types. Tableau already keeps and knows how to apply these, it would be a relatively simple matter to surface the ability. (of course, we;ve been waiting years for the surfacing of similar internal mechanisms, e.g. data source management, including global filters)
  • Provide the ability to copy and paste the formatting from one viz to another – this goes hand in hand with the previous format management point. (to be honest I have a hint of a suspicion this might already exist, but I don't know how to do it)

Monday, October 29, 2012

Tableau: Know Thyself. And Please Tell Me.

Is there any way to get a list of all the Worksheets and Dashboards in a Workbook as an output from Tableau?

This question was posed on the LinkedIn Tableau Software Fans and Friends discussion group recently. It echoes one of the fundamental needs that arises once Tableau's been in use for a while - knowing what's in your Workbooks, and where it is.

The simple answer to the question is: No.

Or at least not directly. But you you can find out. If you want to get right to the get-the-answer part, jump down to the answer.

Tableau Desktop has no self-analysis abilities, no way to let you know what's in the Workbook you're looking at other than by browsing through it with the UI. One can simply walk through the list of Dashboards and open the Worksheets in each, recording their names as you go, but that's a boring, mechanical, error-prone process that's maddening in its ratio of effort to information achieved.

Tableau exists to make it easy to understand data. And the information about the relationships between Workbooks, Dashboards, and Worksheets is easily rendered as data, so it seems a natural fit for Tableau to be the vehicle for understanding the relationships. The fly in the ointment is that Tableau doesn't provide the data.

Solutions

Five or so years ago I was faced with the problem of determining what Dashboards were in the Tableau Workbooks my client had. Some of them I'd built, some of them were other people's creations. With a little poking around, and some connecting with people in the Tableau forums I found out Tableau Workbooks are XML files, and that other people were looking into how to tease this information out of them. Andy Cotgreave was exploring XSLT, which I'd started with but had to abandon when the cross- and back-references got too complicated for my limited skills. I built the Tableau Workbook Inventory System with Java, and over time bundled all sorts of functionality into it. Andy continued on and created his Access-based TWB Auditor.

TWIS and the TWB auditor both do the job, and lots more. But they might be too much tool for the many people who simply want the simple facts about what's where.

Simple and Easy - No Muss, No Fuss, Just the Basics.

I've been using Ruby quite a lot lately for data gathering and munging. Ruby makes it easy to browse the WWW and pull out lots of data that's not otherwise directly addressable by Tableau—e.g. HTML tables with embedded tags that Tableau sees as data delimiters.

It was pretty straightforward to put together a Ruby script to go through a set of TWBs and pull out their dashboards, worksheets, and the relationships between them.

The Answer: 37 lines of Ruby code.

Give Tableau some Ruby slippers and watch it dance.

The code immediately below is a Ruby script that will open every TWB in the current directory and pull out the Dashboards and Worksheets, and their relationships, into a CSV file named "TCC_BDW.csv". A description of how to use it and what it produces follows the code.

If you would like a ready-made script, along with a Workbook containing a starter Dashboard showing the Workbooks, Dashboards, and Worksheets, you can download it from here.


# TCC_BDW.rb # This Ruby script Copyright 2012, Christopher Gerrard require 'nokogiri' require 'open-uri' $recNum = 0 def init $f = File.open("TCC_BDW.csv",'w') $f.puts 'Record #,Workbook,Dashboard,Worksheet' unless $f.nil? end def pullSheets twb doc = Nokogiri::XML(open(twb)) sheetNodes = doc.xpath("//workbook/worksheets/worksheet").to_a sheetNames = sheetNodes.to_a.collect! { |s| s.xpath("./@name").text} dashNodes = doc.xpath("//workbook/dashboards/dashboard") dashNames = dashNodes.to_a.collect! { |d| d.xpath("./@name").text} dashNodes.each do |d| dashName = d.xpath('./@name').text zones = d.xpath('.//zone[@name]') dashNames.delete(dashName) unless zones.empty? zones.each do |z| zName = z.xpath('@name').text putCSV twb, dashName, zName sheetNames.delete(zName) end end dashNames.each do |d| putCSV twb, d, 'z...' end sheetNames.each do |s| putCSV twb, 'z...', s end end def putCSV(workbook, dashboard, worksheet) $recNum += 1 $f.puts "#{$recNum},\"#{workbook.gsub('"','""')}\",\"#{dashboard.gsub('"','""')}\",\"#{worksheet.gsub('"','""')}\"" unless $f.nil? end init Dir.glob("*.twb") {|x| pullSheets x } $f.close unless $f.nil?

How to use TCC_BDW.rb

  • Prerequisites
    • Minimal technical skills.
    • Have Ruby installed and ready to run.
    • Have the Nokogiri Ruby gem installed—it's used in the XML parsing.
    • Have TCC_BDW.rb in place—it doesn't matter where, or what name you use, as long as you know where it is.
      You can:
      • Copy the code above and paste it into your favourite text editor, or
      • Download the existing one, along with the Dashboard/Worksheet Tableau workbook, from here.
  • Running TCC_BDW.rb
    • Open a command prompt.
      (you can run it otherwise, but this is simple and straightforward)
    • CD to the directory containing the Workbooks you're interested in.
    • Run it: "[path to]\ruby    [path to]\TCC_BDW.rb"
    • If you downloaded TCC_BDW.zip, unzip it and copy its TCC_BDW.twb to the current directory.
  • Open TCC_BDW.twb and see your Workbooks, Dashboards, and Worksheets all nicely organized.

What's in TCC_BDW.csv?

–or– what ARE the relationships between Dashboards and Workbooks?

The standard idea is that Dashboards contain Worksheets. This is true, but it's not complete. Some Dashboards contain no Worksheets, these are childless. Some Worksheets are contained in no Dashboards, these are naked.

In collecting the TWB information, TCC_BDW accounts for childless Dashboards and naked Worksheets thus:

  • For every childless Dashboard, its record in TCC_BDW.csv contains the value "z..." for the Worksheet field. "z..." was chosen because Tableau's data sorting places after the other alphanumeric and special characters, and therefore it gets put at the bottom of Tableau's natural listings.
  • Similarly, each naked Worksheet gets recorded with "z..." in the CSV Dashboard field.
Here are TCC_BDW.csv's first dozen lines when run against the Tableau v7 Sample Workbooks:
Record #,Workbook,Dashboard,Worksheet 1,"Finance.twb","Economic Indicators","sp1" 2,"Finance.twb","Economic Indicators","sp2" 3,"Finance.twb","Economic Indicators","sp3" 4,"Finance.twb","Economic Indicators","sp4" 5,"Finance.twb","Economic Indicators","sp1" 6,"Finance.twb","Investing in DJIA","Investment Growth" 7,"Finance.twb","Investing in DJIA","Investment Growth" 8,"Finance.twb","Investing in DJIA","Investment Growth" 9,"Finance.twb","Tale of 100 Entrepreneurs","Top IPOs 1" 10,"Finance.twb","Tale of 100 Entrepreneurs","Top IPOs 2" 11,"Finance.twb","Tale of 100 Entrepreneurs","Top IPOs 1"

Here's a screen shot of TCC_BDW.twb showing how their Workbooks, Dashboards, and Worksheets are related:


Other Mysteries –or– what else can we get Tableau to tell us?

There are plenty of other things that Tableau can tell us, e.g. where and how data sources are used, field calculations–useful for ensuring consistency in metrics, where parameters are used. The list is long.

The list is a very long one, and as long as Tableau won't tell us on its own there will be ways to figure things out for ourselves.

Saturday, October 27, 2012

Disappearing Input - Data Extract Dialog "Top" Value

I meant what I said, and I said what I meant.

Does Tableau forget what I just told it? Or does it decide that I didn't really mean it? Either way, I meant what I said, and I said what I meant, so Tableau Should remember and not make me say it again.

What I want

In this case, I'm trying to extract a percentage of the source data, and I want to pull out 10% of it, so I invoke the Extract Data dialog and enter "10" into the "Top" input area, as shown on the left in the image below. However, I want 10 percent of the data, not 10 rows, so I pull down the pulldown, as shown by the arrow pointing to the pulldown.

Tableau forgets

As shown on the right side, once "percent" is selected from the pulldown, Tableau forgets that I entered "10", and the input area is cleared.
(Or worse, Tableau decides that i didn't really want 10 percent, so it makes me enter the value I really wanted—this would be very not good and I choose not to think it's the case)

General Principle

The User should not have to repeat themself—Tableau should remember and not need to be told again.

Monday, October 22, 2012

Miscellaneous Mapping Vexations

Tableau doesn't recognize the following as legitimate geographic entities:

  • England
  • Wales
  • Scotland
  • Ireland
  • Northern Ireland
  • Antilles
  • Ascension Island
  • Azores
  • Canary Islands
  • Herzegovina
  • Northern Marianas

It's really hard to accept that England, Wales, Scotland, Ireland, and Northern Ireland aren't geographically legitimate.

While we're here, the UI for providing lat/long values is pretty weak.

  • It doesn't accept vales in the form "13°10′N 61°14′W", or at least it didn't work for me. Tableau should be smart enough to parse this form.
  • And even when the lat/long are available in the form "51.481581,-3.17909" you can't simply paste them from the clipboard into the up front field, you have to go through the bother of entering them individually into their separate lat and long text areas as a secondary action.

Tableau has made great progress in its mapping abilities in the past few releases, but there's still plenty of lubricating to be done.

Where's my viz? When Desktop hides the current viz's tab.

It's pretty frustrating to create a new viz—worksheet or dashboard, and have Tableau decide that it's not going to make the viz's tab visible in the tab bar.

This happens pretty frequently, particularly when the workbook has a fair number of dashboards and worksheets in it, and is most annoying when building a new viz and need to change its name from the default.

Similar to the previous post about the need to show the active element in the table list, this is a violation of the usability principle of providing the affordances with which the User can manipulate the objects in their current context.

Friday, October 19, 2012

Umm, what's that data source?

It should be easier to find out what data is being accessed.

Here's the Tableau Data Connection right-click menu with its options to see and configure the data connection:

Selecting the "Edit Connection..." option, you see this:

It would be better to see this:

In the improved version on the right the current table is shown in the list when the dialog is presented.

This might seem like a little thing, but when working with real business databases there may be many, many tables in the list and navigating to the currently selected one becomes an exercise in frustration.

General Principle

Whenever possible, present the current state rather than make the User hunt for it.

Monday, October 1, 2012

Text Editing Idiosyncracies

This is going to be a very live post - I've not written this stuff out before because the individual bits seem too insignificant to note separately, but then they never get recorded.

Editing viz names in thumbnail workbook view doesn't observe normal navigation keyboard shortcuts, e.g. ctrl-<right> & ctrl-<left> don't work but ctrl-<home> & ctrl-<end> do.

Title editors - don't accurately copy/paste text contents, sometimes adds an extra line.

The Annotation Editor:

  • Text selection: select all text, set font size, close/save editor, open editor, select all text (<ctrl>-a, text is selected but previously set font size is not honored.

Saturday, September 29, 2012

Reconsidering previous IFNULL post

This post is an extension to my original post on IFNULL.

The comments it received (thanks to everyone) got me thinking a little differently and somewhat deeper (I think). The upshot is that I propose this as a workable improvement to the current IFNULL function:

new IFNULL

IFNULL(expr1, expr2, expr3)

Its structure matches the language:

IF expr1 IS NULL
THEN expr2
ELSE expr3

Benefits

  • The logic is clear and transparent.
  • It's more powerful and flexible, with the opportunity to provide an explicit non-NULL value when expr1 IS NULL.
  • It avoids backwards compatibility problems—a new function, it doesn't conflict with existing calculations.

Drawbacks

  • In the normal (maybe) case, expr3 will be the same as expr1; this will be a bit if a cognitive burden, but not all that much.

About the current IFNULL

new - Mapping IFNULL parsing

I'm adding this section because I'm pretty sure that my objection to the current IFNULL operation is clear. So, pictures being better than words in some circumstances, I mapped it out:

Even though these formulations are logically equivalent, I'm confident that the second one is clearer than the first, and it matches the current IFNULL documentation in the function editor.

The problem with the second reading is that the function's name is the reading's negation. It's this collision that I'm urging be fixed.

I understand that the current formulation is readable by programmers and technical people who are experienced in interpreting these things (I've been at it since 1974). But, and this really is the whole point of this blog, Tableau was invented to move data analysis and visualization from the technical-only realm and open it up to non-technical people who need to understand their data but shouldn't be burdened with the arcane technical aspects. Every single thing that's not as clear and transparent as it could be is an impediment to real human people. Forcing non-technical people to adapt to the technical conventions is contrary to the whole idea of eliminating the friction between between people and the information and insights in their data.

addendum to original post

Note: in my original post's comments, this section was unclear because I used angle brackets—< and >, and these were interpreted as invalid tags and not shown. Sorry about that.

In a nutshell, my objection to the current IFNULL(expr1,expr2) function is that from a human standpoint, the reader has to figure out:

  • what, exactly, is being evaluated to see if it's null, and depending upon the assessment:
    • what positive action is being taken when it is null, and
    • what positive action is being taken when it's not null.

If I have it right (it's been a long time since undergraduate logic class), the two semantic constructions for {FUNCTION}(expr1,expr2) are:

  1. If {IS NOT NULL -expr1-}, then return expr1 else return expr2
  2. If {IS NULL     -expr1-}, then return expr2 else return expr1

  1. parses strictly right to left and is easy to understand
  2. requires a forward leapfrog when -expr1- IS NULL, followed by a backreference when -expr1- IS NOT NULL, and is, at least for me, twisty and unclear

I believe that:

  1. 1. is better than 2., and
  2. the documentation describes 1., although it's not as clearly worded as it could be.

I understand the problems associated with 1., most notably that a negative expression is typically harder to parse and a negative–null coupling is doubly difficult to wrap one's head around, and changing the expression causes backward compatibility problems (and Tableau hasn't yet implemented self-analysis).

Footnote: I'm sure someone could do a better job on the logical formalisms.

Thursday, September 20, 2012

Putting Measures in the Middle of a Viz

The Tableau Public-published dashboard "Proposed Placement of Measures in Interior Viz Columns" (below) illustrates one of my favorite alternate vizzes, shown in the "Measures in the Middle" worksheet in the top of the dashboard.

In it, the sums of the measures of interest are placed in the body of the viz, not hanging out way over at the right end, as is the standard Tableau layout.

There's a lot of value in promoting the measures to the more cognitively valuable left-hand side of the via, e.g. they're picked up more quickly in the standard visual scanning pattern; and they're closer to and thus more intimately linked to the Name as ID of the Dwarf to whom they belong.

Even though "Measures in the Middle" and "Measures to the Right - Tableau's Layout." contain the same data there's a very large visual difference between them the effect of moving the Measure's bars from the right side to near the left side is dramatic.

I had intended to write up some more about this (published it late in the evening). As luck would have it, Joe Mako commented with an alternate example worksheet that's visually cleaner than my faked-up dashboard. In responding to his example and comment I covered much or most of what I intended to add here. So I'm pulling it up from the comments to make it a little easier to see. But you should see Joe's example.

My emphasis is on how to make Tableau better by making it easy to do reasonable things. And I believe that liberating the measures' presentation from its current rightmost-only position in the viz is reasonable and valuable.

I wrote "there's no data-structural reason that this placement can't occur." and I stand by this.

There's nothing in the structure of the source data, nor in the structure of the data in the viz that precludes what I want Tableau to do. The elements on display are identical in my two views - they're all at the same level in the data hierarchy. (this is a simple case, because there's only one record per Dwarf, but the principle holds for multi-dimensional data that's aggregated to a common level)

Re: Tableau's VizQL foundation

I wish I knew how VizQL works. I don't. I've never seen documentation on it. Not a grammar, not a write-up, nada. One of the things that drew me to Tableau was its basis in a data management and presentation language, but Tableau has seemingly stopped promoting, even surfacing, VizQL as its underpinning.

I do know very well the original 4GLs that invented the data structuring and presentation space. FOCUS was created for exactly these purposes, and had all of the data-operational constructs necessary to access, organize, aggregate, and present data, some of which were much clearer than Tableau's analogs. Unfortunately, since it originated in the age of mainframes, COBOL reporting, and data processing, it lacked Tableau's deliberate central focus on data visualization capabilities. I've often wondered what the structural similarities, and dissimilarities, are between them.

I don't think that VizQL is somehow incapable of accomplishing what I'm asking for, either as-is or with a modicum of adjustment. I see no structural barrier for it, and think I see the shape of the problem domain well enough to be confident in my assessment. If nothing else, if VizQL isn't capable of being improved in this way, or Tableau is unwilling or unable to do it, it opens the door for the innovator who will.

Is this visualization, or is it reporting?

Where's the bright line between what Tableau does and what reporting software does? I don't see one. Tableau is a data visualization product. Reports are data visualizations.

The biggest differences I see between Tableau tables and normal reporting software reports are things I consider to be deficiencies in Tableau and hope to see corrected soon.

Some examples:

  • The strict left-right ordering of dimensions in the same row is a big waste of space compared to dimensional folding, with each succeeding dimension underneath and slightly offset from its predecessor.
  • Strict sub-totaling for every most-granular dimension (when sub-totaling is on), even those with only one record. This is worse than useless, it dramatically increases the cognitive load on the user who has to sort out the true sub-totals from simple restatements of a single data value.
  • Granular formatting of the elements; it would be extremely helpful to be able to provide richer formatting, with control over the individual formats for each structural element in the viz.

Warts on a toad.

Are you still reading this?

Why? Most people don't care about warts on a toad. And why should they? Toads are warty; it's their natural state, normal and expected. If, when handling a toad, one encounters a wart, it's no surprise. If a toad grows another wart, nobody really cares; after all, what's another wart on a creature already festooned with them?

(I know that toads' skin lumps aren't really warts; I'm invoking poetic license)

An Ode to Warts

–or–

A Fractured BI Fairy Tale

The traditional BI products are warty. Toady. Awkward, even unpleasant to handle, they're difficult to use, putting lots and lots of crufty operational controls, gadgets, layers, and other lumpy stuff on top of your data, making it very difficult to get at, and even harder to make sense of.

And yet, in a world populated only by toads, the population of people seeking to understand their data thought that toadiness was an essential characteristic of their BI tools—all the tools were like that, so it must be good, eh? They even came to embrace their essential wartiness, even to the point or proclaiming that wartiness/toadiness was a desirable characteristic of BI tools. In advanced cases some kool-aid drinking souls grew to love their tools' toady wartiness and declared it a virtue without which BI could not exist, much less flourish.

Wart-Free BI

–or–

Enter Tableau

When Tableau came upon the BI scene it was revolutionary. Designed from the outset to make it as easy as possible to connect to, organize, and quantitatively visualize data, it was virtually wart-free. Devoid of the usual lumpiness and bumpiness, it provided a smooth unblemished palette upon which data could be painted in a dynamic model that encouraged and fostered exploration of the data.

Close but not quite.

–or–

Why Tableau's warts are so irksome.

Tableau has its warts. Very few compared to it's enterprise BI brethren, but there are some. This blog is in one sense an inventory of the warts I encounter that vex me enough to keep track of.

Because Tableau is so remarkably unwarty those that it does have stand out like, well a wart on the tip of someone's nose. They're prominent because of their rarity, sometimes disproportionally so in relationship to their real effect.

Encountering a Tableau wart is jarring. It interrupts the normally fluid user-Tableau interaction and imposes upon the user the cognitive cost of recognizing and interpreting it, and the effort to adjust and accommodate the wart and its effect.

Why we need to worry about warts.

–or–

Removing warts is important.

There are two aspects to addressing Tableau's warts: guarding against the introduction of new warts should be a primary principle of designing and implementing Tableau's new features, and existing warts must be aggressively hunted down and eliminated.

Evolving a commercial software product takes a lot of time, energy and attention. Adding new features, expanding the breadth and depth of functionality are the whole point. When adding the new features it's critical to be constantly on the lookout for new warts and ruthlessly eliminate them. As Tableau has evolved it appears that this principle hasn't always been given the attention it needs.

Guarding against the introduction of new warts isn't enough. It's even more important to relentlessly focus on hunting down and removing the existing warts. Doing this will continue to improve the product in a different dimension than adding new features, but one that's arguably more important to the overall product quality. Releasing a new version that consists only of wart removal is good and valuable, and sends a clear signal to the product's users that their investment in the product is well made and worth continuing.

Evolving a software product by only considering the new features and functionality, and not also considering fixing existing problems—removing existing warts, is shortsighted and inevitably leads to an increasingly warty product. Eventually, the product evolves into a toad.

Tableau's future.

–or–

Tableau's future?

As Tableau continues to evolve it's becoming increasingly warty.

If the trend continues Tableau will become a toad. If this happens Tableau will have become one of the products it successfully competed against by being simple, easy, and transparent in use, and with enough depth and breadth of functionality to let people achieve their data analysis goals with a minimum of friction. At this point Tableau will be vulnerable to new products that provide the same basic functionality without Tableau's warts. Tableau the disruptive BI tool will become itself displaced by newer, less warty competitors.

A request for Tableau Software.

Don't let Tableau become a toad.

My Tableau Wish List

With TCC 2012 coming up, there's not enough time to write separate blog posts for all the individual friction points and areas where Tableau should (IMHO) make improvements, so I'm listing all the big ones here. I'll elaborate on them as time and opportunity permit

Have something to add to the list? Leave a comment or sent me a message and I'll add it in if it fits, with attribution unless you wish not.

Eliminate all the friction identified in this blog –or– Don't let Tableau become a toad..
(naturally)

Provide a dashboard layout manager capable of producing real publication-quality material.
Tableau's improved the dashboard layout manager with each release, but it's still like drawing with crayons.

Provide Tableau Desktop automation.
Routine repetitive tasks shouldn't require us to do work a trained monkey could handle.

Incorporate a true programming language, preferably Ruby or Python.
For calculated field coding, automation (per previous point), data munging etc.
Along with this and automation, add the ability for Tableau to access data from a standard pipe, this would enable a tremendous expansion of the high-value Tableau Use Cases.

Create and publish a bona fide Tableau philosophy.
It would explain the guiding principles behind the product, and be the standard to hold the product to.

Add the ability to access hierarchical, multi-path data.
This is a significantly harder problem to solve from the user experience perspective, but Tableau's inventors cracked this nut with single-path data visualization, to their eternal credit. This ability is necessary for Tableau to implement the next item in this list:

Add introspection and self-analysis.
Tableau has no general ability to expose its internals for analysis.
Workbooks are collections of data sources, worksheets, dashboards, real fields, calculated fields, etc., that have multiple interrelationships. It's extremely helpful to be able to see, from a data perspective, what's in a workbook, and how the parts are related.
With this ability we'd be able to answer questions such as:
  •"Is this data source used, and if so where?"
  •"Which dashboards does this worksheet appear in"
  •"How many different ways is Profit calculated in my workbooks, and in which worksheets and dashboards do they appear?"
  •"If I change this database, how much work will I need to do to update my Tableau workbooks?"
I built The Tableau Companion (TCC, formerly TWIS) to be able to answer these questions, but it would be far, far better to have it be an integral part of Tableau.

Add visualizations for hierarchical, multi-path data.
Extensions of the previous items, this ability is at least useful in visualizing the structure and content of Tableau workbooks. A visualization of the Tableau Science Sample Workbook can be seen here, and other examples can be seen here.

Hire someone to ensure that Tableau honors the Tableau philosophy.
Someone responsible for ensuring that the various Tableau products adhere to the Tableau philosophy in their broad strokes and details. Ideally, this would be me.

...

Tuesday, September 18, 2012

IFNULL - is not "IF NULL", is "IF NOT NULL"

Here's one that really twists my head around.

I frequently need to assign a placeholder value to a field when the Tableau interprets the data value as NULL. In most cases it's assigning a zero to a null value. Using a calculated field as a proxy for the real data field allows me to do this, and all I need is a way to assign the value zero (0) to the calculated field when the data value is null. The logic of this is:
    to the calculated field:
        if the data field is null assign zero
        else assign the data field's value

The Tableau IFNULL()function does this, and in exactly the way that it should. Here's the Tableau calculated field dialog showing IFNULL() (edited for space):

Spot the problem?

The description of the function is the accurate description of the function's behavior. When the data field's value is NOT null, its value is used for the calculated field, otherwise the value of expr2 provides the value. The function's description and behavior match each other, and are exactly what we need.

But the function's name is wrong.

It should be IFNOTNULL
—NOT—
IFNULL

This misnaming is at least confusing, and has the potential to lead people to making poor or outright wrong decisions when they rely on the results of employing it based on the semantics of its name.

Oddly, the logically inverted naming makes it harder for experienced programmers to find the right function to use than for inexperience people.

Monday, September 17, 2012

Why this blog?

I loved Tableau when I found it better than 5+ years ago because it embraced a novel approach: removing the friction from the process of accessing, organizing, and analyzing data.

Compared to the products that were dominant in the marketplace it was an absolute revelation. But it's not perfect—there are lots of little grains of sand in the gears that might not be too bad individually, but in combination impose a substantial cognitive load.

As Tableau moves forward adding functionality it's going to be very interesting to see how it evolves. Will time and attention be spent finding and removing the sand, or will the pressure to add new features dominate, leaving the existing irritations in place, and adding new ones.

So far it feels like new features and functionality are being added by individual developers, each perhaps reasonable in isolation, but out of synch enough with the other parts to be just a little, and sometimes a fair bit, jarring.

I started writing this blog as a place to keep track of all the little things that irritate me as I use Tableau.

Part of the reason is because I'm something of a purist—I believe that the best tools should be as good as they can be.

Partly it's because I have an attachment to Tableau—it's what made BI interesting again after a decade and a half of boredom listening to the droning on and on of data warehousing's blathering at the alter of big-Big-BIGGER-BIGGEST tooling as the only way to make sense out of data.

Some of it was a shameless attempt to get Tableau to hire me to come and help fix their product—if I had a good, solid comprehensive list of things I could fix I could talk my way into getting them to let me do it. I've talked to a fair number of Tableau people, many of whom are very sympathetic. But nobody at Tableau seems interested in hiring someone who's only going to be cleaning and tidying up. I get the sense that the emphasis is on moving the product forward by adding more to it.

But why do I care, really?

Because I believe that computers should help us, and get out of the way absolutely as much as possible.

I once worked for a company that had a product much like Tableau, that busted down the barriers between people and data and made it easy and straightforward to explore and find information and insights in the data. But that company forgot what made then product special - its simplicity and ease of use - and went down the path of adding more and more cruft and crap onto the basic product, layering more and more friction onto it until it lost its way and failed to live up to its promise.