Goldilocks Criteria: Business Intelligence Platforms

This is first of a new series of posts dedicated to defining the features of the perfect product for different use cases.  Of course these products rarely exist, but if they did, I’d use them! First up: business intelligence platforms.

There are many fiefdoms in the kingdom of data – from product analytics to predictive models to advanced data-driven applications to quality of service reports to user facing product interfaces – and no single platform will address every need.  For the purpose of this discussion, let’s define Business Intelligence platforms as data platforms for business users with unknown quantitative skills.  This tools should give these users data to inform and improve their workflows and should not require a STEM degree or General Assembly workshop to operate.

Business Intelligence (BI) platform date as far back to the ‘60 – the 1860s when Richard Devens coined the term in his Cyclopædia of commercial and business anecdotes when Devens used the term to describe how Sir Henry Furnese, a banker, gained an advantage over his competitors by using and acting upon information surrounding him.  Over the past few decades BI tools have matured much more rapidly, shifting from beastly on-premise data warehouses with text-focused UIs to cloud-based, mobile-first, lithe data platforms designed for non-technical users.

https://www.sales-i.com/a-history-of-business-intelligence

BI is defined, generally, as tools for data analysis and report generation on top of data aggregated from multiple disparate systems.  Some BI platforms sit on top of separate data aggregation tools, and some modern platforms serve as the data platform as well. BI tools pack a ton of functionality, but are typically narrow scoped.  You don’t “do” anything within your Business Intelligence platform, instead you investigate, learn and report on how other systems are “doing”. BI surfaces data to guide decisions made elsewhere.

You will see BI or Embedded Analytics within various tools – like your CRM system or your Web Analytics platform – but by in large Embedded Analytics help steer micro tasks like which email subject performed best vs. providing a holistic view of data across multiple sources.  The best BI tools pull in all of your data to support cross functional views and insights.

So how does this work in practice?  A great use case for BI platforms is to create easy to digest OKRs dashboards for your company, teams and individuals.  A great platform should allow teammates to pull up a live view of their progress towards their outcomes / goals on their phones – right before they go to sleep every night!

OK, enough preamble.  Here are the goldilocks (aka “just right”)  criteria I look for in BI platforms:

Integrated data platform

Traditionally, BI tools sat on top of separate data platforms managed by IT teams.  Recently, a new class of products have emerged that allow you to upload / connect to your data without engineering support.  I find this to be a huge advantage as it allows semi-technical users to get up and running without distracting / relying on external parties.  Self service also leads to challenges with data governance but that’s another story.

Parallel: imagine uploading all of your data to Google Drive / Dropbox and then being able to easily join them together and visualize them as you choose.  That’s what these new platforms do. And if you’re missing a bit of data – simply upload it then and there or create a persistent authenticated connection to pull down the data in perpetuity.

Data engineering for dummies

Some of the best data scientists I’ve worked with estimate that they spend 80-90% of their time on data hygiene before they can begin analysis and exploration.  

https://www.forbes.com/sites/gilpress/2016/03/23/data-preparation-most-time-consuming-least-enjoyable-data-science-task-survey-says/#183675d26f63

What does that mean for BI tools?  Any functionality that support easy data manipulation for the sake of improved clarity is awesome.  That means – joining data together via drag and drop, changing data types with a click, deduplicating rows without writing SQL is all a huge value add, extending the range of users who can go deep with the data without external assistance.

Live data! From the cloud! On your phone!

Data that arrives embedded within emails or as an excel attachment is Dead on Arrival.  That is one of my absolute pet peeves. Further, once people begin to discuss and edit the data set, the risk of multiple versions / views of the same data becomes legitimized.  

BI tools need to pull from a live server at all times.  When I pull up a link to view a dashboard the data should be (pseudo) real-time up to date or time stamped clearly with the data last run.

This also means the platform should be mobile-centric.  Old timers still want their desktop-focused printouts, but there is nothing more powerful than conversing with colleagues and pulling out live data views on your phone à la minute.  

AI / ML aware

I don’t want to overstate this one as we’re in the very earliest of innings, but your platform should have the foundation of supporting automated machine-learning driven insights.  You may not find these valuable right away (they rarely are) but in a few years you should be getting voice alerts when your data spikes unpredictably. There is not sense in investing in a platform that is ignorant to this coming trend.

To start, I’d like to see a platform present basic statistics around the data that I’ve onboarded.  This means basic distribution and correlations information.  As you play with these basic metrics you’ll be able to more easily wrap your arms around the data at hand, informing deep analysis and insights.  Simple predictive analytics is another good baby step before full blown AI.

This all said, you separately need to invest in training of your teams to take advantage of these statistical insights.  Leveling up the data fluency of your team is often more worthwhile than the data platforms that they utilize.

Narrative & collaboration focused

A perfect platform would allow for metrics-backed storytelling, and not just the sharing of data dashboards.  That means as a product owner, I could use a platform to explore a set of data and then build a coherent, sharable narrative around it.  That could manifest itself as a online presentation with live charts (naturally) surrounded by text, images, video and other added insights.  It also means that I should be able to drawn / pin annotations to the data itself.

This also means that the presentation platform should support conversation around what’s being presented.  Unlimited named user accounts, threaded comments, open annotations, tasks lists, @ mentions and more are a natural fit here.

Governance gone wild

Sad to say, this is critical.  Like supercritical. Like, as soon as you create your second dashboard you need this otherwise you’ll never find / know which data is most recent, best, approved and official.  I’ve seen smart approaches here and they center around clear labeling of the data, it’s origins, similar / duplicative data and more. Having easy way to validate data / views as “best” or “official” helps too.  Ultimately, Machine Learning will be a huge help in this arena.

An integrated, dynamic “data catalog” that shows you the breadth of your data, its lineage, validations and error reporting is also must-have.

User-level data FTW

BI tool typically play in the aggregated, anonymous altitude.  You can see how all your site visitors behave, customer acquisition by location, sales by campaign, etc.  Data is viewed on the content, page, campaign, location level – rarely at user level. In a perfect world, a graph model would be deployed at the atomic data layer allowing pivots by the above altitudes but also on the user level.

A new breed of system called Customer Data Platforms is jumping into the fray here, promising a single view of the user.  These CDPs are being leveraged today by Marketing and Sales team but the application of this granular view to more typical BI use cases is immense.  Perhaps CDPs are the topic of the next post in this series…