AgileData App UX Capability Maturity Model

TD:LR

There are always more than one step

When I think about adding a new capability to the AgileData App I use an internal maturity model to determine the level of user experience we will initially build and deploy, plus an idea of where we will take it in the future.

The Data Domain is messier than the Software Domain

One of the challenges of building an App that deals with the messy business of data is that the data domain is a very different beast compared to building an app in say the eCommerce domain.

In the eCommerce domain there are a set of commonly known and repeatable core business processes.  For example people list products on the store, people add a product to their cart, people pay for the product.  The boundaries for those things (apart from the users behaviour of course) are well known and the creators of the app are able to control a lot of things within those boundaries.  They control the interfaces on how the data is created, they control the interfaces on how the data is stored and they control the interfaces on how the data is represented.

When we look at the data domain, we have a lot less control.  

Yes the Data Value Stream steps for data is common, we Collect the data, we Store the data so we can reuse it time and time again (or we can virtualise the data to remove this step), we Change the data so it matches business reality and is fit for purpose, we apply Semantics to the data to help non technical people understand it, we define Metrics so we get the same answer to the same question, and then we Present the data so it can be easily Consumed.

But within each of these steps in the Data Value Stream there are many variations, and those variations are often driven by the Data Domain, the Industry, the Systems of Capture, the completeness and quality of the data, the Data Team Design and the level of expertise of the people working with the data.

We are highly opinionated on purpose 

When designing the AgileData App we need to be opinionated on all of these steps in the Data Value Stream, while also catering for the large variations required in those steps.  That variation comes with every different industry, domain and set of data.  For instance we need to cater for data that is heavily curated in a System of Capture and data that is lightly curated in a spreadsheet.  We have to cater for fully centralised data teams and teams where the analysts are decentralised in each business unit.  We have to cater for data that is from an eCommerce domain and data from an Insurance domain.

Yes we could build an App that does one data thing well, for instance allow you to apply a rule on some data to check its quality and tell you if it has nulls or not.  We could build an App that just orchestrates your code.  We could build an App that just  catalogs your data, no matter where you store it.  We could build an App that just lets you ask a question and converts that question into SQL code to answer it.  We could build an App that only collects data from Google Analytics and presents prebuilt Dashboards in a flash.  

But our aim is to not to automate just one small part of the Data Value Stream, our aim is to reduce the complexity of all data management work in a simply magical way.

Our goal is always magic

I think of our nirvana as the “Magic Sorting Hat” from the Harry Potter books. Nirvana is when any person, fluent in data or not, can drop their data into the sorting hat and with no cognitive effort the data is made fit for purpose for the task they intended.

Even with the latest iteration of Large Language Models and ChatGPT, given the variability of all the things I have mentioned already, I believe we are a long way off from this being a reality.  But hey ask me in a years time and see if I was right or wrong.

AgileData App UX Capability Maturity Model

So how does this all relate to my internal maturity model for adding capabilities to the AgileData App.

Well each time I think about automating data work by adding a feature in our app I think about these maturity levels.

Automation lens

Let’s look at these maturity levels based on the different ways we can automate the Data Work.

• Level 0 – Manual effort outside of the AgileData App

   

• Level 1 – Code outside the AgileData App

   

• Level 2 – Code inside the AgileData App

   

• Level 3 – User Interface in the AgileData App, No Assistance

   

• Level 4 – User Interface in the AgileData App, Machine Assisted

   

• Level 5 – Fully Automated

Level 0 – Manual effort outside of the AgileData App

For Level 0, we have identified Data Work that needs to happen and we have created a manual way of working or manual templates to achieve that work, but these sit outside the AgileData App and Platform.

A current example of this is the Information Product Canvas, which is a Google Slide (and PDF) template we use to define the Information Products we need to deliver.

This template and process is part of our AgileData Way of Working but is not catered for in our App.

Level 1 – Code outside the AgileData App

For Level 1, we have code that can deliver the required Data Work, but the code is not able to be run directly inside our App or directly on our Platform. However the code can be interfaced to use the data stored in the AgileData Platform.

A historical example, in the very early days we used to run Panda’s profiling on the data in the AgileData Platform as a separate capability.  We have now embedded data profiling into both our Platform and surface the results in our App.

Level 2 – Code inside the AgileData App

For Level 2, we provide the capability to embed code in our App that is executed by our Platform.

We will always provide this feature in specific capability areas within the App as a “get out of jail for free option”, for example in the Change Rules and Trust Rules capabilities. 

But we also use this pattern as an interim way to deliver a feature before we move it up the maturity levels.   A current example we are working on is the ability to define a Data Contract in pseudo code (think the equivalent of a YAML) file which you will store against a Data Tile and we will use that contract to validate any data collected for that Tile.

Level 3 – User Interface in the AgileData App, No Assistance

For Level 3, we bring the capability into the AgileData App where the Data Magician can use the Graphical User Interface to do the Data Work.  This removes the need for them to write code, but they still need a reasonable level of data fluency to do the Data Work.

An example of this is the Data Design and Change Rules capabilities.  The Data Design capability allows the Data Magician to define a logical data model by dragging and dropping tiles on the screen, but they still need to be fluent in modeling Core Business Concepts and Events.  The Rule Step screen allows Data Magicians to transform the data, by following our natural language interface:

• Given this data from Shopify
• And we want to filter it based on number of orders in last 3 months
• Then store it as Concept of an Active Customer.  

But they still need to be fluent in the different ways you can transform data to make it match business reality and become fit for purpose.

Level 4 – User Interface in the AgileData App, Machine Assisted

For Level 4, the AgileData Platform executes magical models and uses the results to recommend an action for the Data Magician to accept or override within the AgileData App.

An example of this is when data is first collected into a History tile, a set of Change Rules are automagically generated to automated the loading of this data in the future.  The Change Rules that are generated are set as Draft, so the Data Magician can accept the recommendation by publishing those rules.

Another example we are exploring is when data is collected into the History Tile, the App would recommend the Core Business Concepts, Events and Details that could be created from that data.

Level 5 – Fully Automated

For Level 5, the Platform automatically does the work and the results are visible in the App.

One example of this is FileDrop.  When a Data Magician drops a file into AgileData, lets say a csv file, the Platform automagically parses the file, determines the schema of the data, creates a Draft History Tile for this data, creates the Change Rules to collect this data and finally executes the Change Rules to populate the data into the Tile.  All this happens with no effort or interaction from the Data Magician, but at any time they can see what is happening in the AgileData App.

Persona lens

We can look at these maturity levels based on the level of expertise the person is required to have to do the Data Work.

• Level 0 – Anyone

   

• Level 1 – DataOps Engineer

   

• Level 2 – Data Engineer

   

• Level 3 – Experienced Data Analyst

   

• Level 4 – Any Analyst

   

• Level 5 – Anyone

 Level 0 – Anyone

Level 0 features can be used by any Persona who is fluent in the manual data work at hand.  For example almost anyone can complete the Information Product Canvas.

Level 1 – DataOps Engineers

Level 1 features can be used by DataOps Engineers (or as we call them Data Plumbers).  They need to be fluent in writing complex code and interface this code with third party systems, in this case the AgileData App or Platform.

Level 2 – Data Engineers

Level 2 features can be used by Data Engineers.  They need to be fluent in writing data transformation code, typically SQL or Python.  They paste these snippits of code into the AgileData App and it gets executed for them on the Platform.

Level 3 – Experienced Data Analysts

Level 3 features can be used by experienced Data Analysts.  They need to be fluent in the way data works, how to decide when data is a Concept,  Detail of a Concept, or an Event.  They need to know how to map data from a System of Capture to the Designed Data and how to define Metric Calculations.

Level 4 – Any Analyst

Level 4 features are for both experienced and inexperienced analysts.  They have to know when to accept the recommendation from the machine and when to override it.

Level 5 – Anyone

Level 5 is Nirvana, all features that can be used by anybody with any level of data fluency.  The feature acts like the magic sorting hat and is bullet proof regardless of the data being used. 

Slowly, slowly catchy monkey. 

Often we will progress our features through these maturity levels in a linear fashion, but often we will jump levels where we can.  

DataSync Notifications

For example we initially exposed the metadata for all loads of History Tiles as data, so it could be used by last mile tools to visualise when data had been loaded on time or a Data Contract missed.

We then automated the notifications of the loads within the AgileData Platform and pushed these to Slack channels.

Next we created a Notification screen within the AgileData App so the Data Magicians can see what had run, what completed successfully and what failed.

Then we created a feature that automagically flagged Tiles with a predicted DataSync schedule, based on historical loads.  An automated process looks at when a Tile gets refreshed with new data, and then sets that as the expected DataSync frequency.  This frequency can be overridden by the Data Magician if desired.  Each day the Platform automagically checks every Tile to see if the DataSync expectations have been meet, and notifies the Data Magician in the App if any expectations have not been met.

File Drop

Another example is the FileDrop capability. 

The usual development path for this feature in a Data App is to provide a screen where the user manually parses a preview of the file to determine the fields within that file.

This would result in a Level 3 feature. 

We decide to jump straight to a Level 5 feature where the file is automagically parsed by the AgileData Platform. 

This means instead of putting our time and effort into designing and implementing a parsing interface in the AgileData App, we spend our time iterating the FileDrop Parser everytime a Data Magician drops in a file that breaks the parser.

Its all about Patterns and Context

Every time we start to develop a new capability or set of features I quickly apply this maturity pattern to these features and make an initial decision on what path I we think we will take.

And then we start the work.

As they say, “no plan survives the first battle”.

Which is why we adopt a very light way of working for this step in our App and Platform Value Stream.

But as they also say, “without a plan, you are planning to fail”.  

For us it’s all about using a set of agile patterns that have value given a certain context, but which can be easily adapted when required.

Our AgileData App UX Capability Maturity Model helps us achieve that.

Keep making data simply magical