#AgileDataDiscover weekly wrap No.3

TL;DR

Let’s get into it!

#AgileDataDiscover weekly wrap No.3

Let’s build an initial prototype

We have already done some prototyping as part of the initial McSpikey. We can provide specific inputs into the LLM, we can give it a specific language to use, we can get the outputs we think have value. 

But in the McSpikey we did that manually. Next step is building out a prototype app that allows customers to do that with or without us.

Here is the first cut of what we built:

https://guides.agiledata.io/demo/clx2hlkru0ibi35xebw371v0f

In this demo example we used a file that contains the Consume query log for our demo AgileData Tenancy. As part of this step we learnt a lot about the feasibility of building the potential product.

Application Framework:

We need to build, deploy and run a browser based application, this capability is a core part of the AgileData App.

Secure Sign-in:

We need to allow customers to securely sign in, this capability is a core part of the AgileData Platform.

File Upload:

We need to upload input files and store them for use by the LLM, this capability is a core part of the AgileData Platform and App.

Data Security / Privacy:

Even though we wouldn’t be using Customers actual data for the prototype we would be using their logs etc at the MVP stage, we need to make sure whatever we use is stored securely. The AgileData Platform architecture provides a Private data plane for every customer tenancy, we can reuse this for the prototype and deploy a new AgileData Tenancy for each MVP customer in the future.

LLM:

We need to access an LLM via API so we can run it from within the prototype app. AgileData is built on Google Cloud, Google Gemini is the obvious choice of LLM service for this.

We have already used Google Gemini as part of the Customer Trust Rules feature we recently released, we have a lot of the plumbing (engineering) patterns in place for it already.

Prompt Storage:

We need a way to store the prompts we are using so they can be used across all prototype Tenancy’s. The AgileData App stores everything in Google Spanner which is then accessed by the AgileData Platform, so we can use this to store the prompts.

We were presently surprised how many reusable patterns we had built as part of the AgileData, that we could reuse for this prototype and the fact they are all production ready.

Let’s show a few people the initial prototype and see what feedback we get.

We show a few people the initial prototype for feedback

We are lucky to have a network of talented data peeps across the world, people we have connected with over the years, people whose opinion we respect.

As part of the No Nonsense Agile Podcast we have a series of awesome guests on the show, and one of those was Teresa Torres. One of the key takeaways from that chat was the pattern of continuous discovery and the power of talking to three people per week, if you can. 

We started asking people about their data problems and showing them the prototype to elicit some feedback. The conversations were enlightening. We can see a few problems we might be able to solve with this thing. We’ll explore those in more depth later.

One bit of feedback we got as we started showing people the prototype was that having all the outputs in one screen was confusing. We need to iterate that.

Another thing that frustrated us from day one with the prototype was trying to find the url to demo it. Especially when we have a couple of different demo Tenancies. It’s a small thing, but it’s one thing that has caused frustration and a surprising amount of wasted effort and time in the past, so we’rei’m very conscious of it.

We were always going to have a bunch of decisions to make about where this fits in our current product Information Architecture aka our menuing system. Let’s deal with that menu problem next.

Encountering and solving a top menu problem (for now, anyway) 

It’s not a big thing, and I know I shouldn’t be sweating the small stuff but I also know that if I don’t it’s going to really needle me until I scratch this itch.

 A problem shared and all that, so a Slack huddle with Nigel it is.

When talking through the problem Nigel mentioned what I had done to the header area in our marketing website. I have iterated it to use each of our core colours depending on what part of the AgileData product we are talking about, Network, Wow, Platform or App. Why can’t we do that for the App header?

Intriguing thought, we use those core colours heavily in our course material as well to indicate a transaction from one course module to another.

We could spin on this decision for months if we let ourselves, but one of the key things a Product Leader has to do is make decisions quickly if they can, and I can.

So we will introduce a pattern where the App header changes colour when you are in the Discover area.

Now how to get to it?

Well we always talk about Personas, and again we teach data and analytics teams how to apply the persona pattern in the data domain (hell it might even become the second AgileData Guide book!) 

Let’s add the ability to quickly change personas in the app and as a result change the menus that are presented to us so that we can get the job that personas needs to do, done.

And speaking of jobs, on the menu options, job done (for now).

One response to rule them all, not!

As we started showing people the prototype one bit of feedback we consistently got was that having all the outputs in one screen was confusing. There are a few other problems with the single output model we have initially built into the prototype.

• The number of output tokens the LLM will generate is limited, which means the LLM reduces the number of words used as it moves through the output types. 
• We want the definition of the Data Dictionary to be as verbose as the Information Product, even when the LLM produces it as the last part of the output.
• We want to be able to regenerate just one of the output types in isolation, not all the output types.  A single output response type makes this difficult and costly.
• We want to make it easy to copy and paste one of the output types to another medium.

The single output pattern we have built into the prototype won’t allow these things to happen easily. And also scrolling down a long list of output text to find what you are looking for is a pain.

So we need to break the various output types into their own unique objects.

Luckily it’s an easy enough change to experiment with (well Nigel made it look easy enough). 

We have already decided to break the Discover menu for the prototype out into its own menu for now, so it’s “just” a case of adding new menu options that only display the response for that output type.

We have split the outputs into the following screens / menus / output types:

Information Product

Models

• Business Model
• Conceptual Model
• Physical Model
• Reporting Model

Glossaries

• Business Glossary
• Data Dictionary

Metrics

• Facts
• Measures
• Metrics

Matrix

• Bus Matrix
• Event Matrix
• Concept Matrix

One of the interesting benefits of this change is it makes it much clearer to explain to people what we are producing and what they can be used for.

I have spent the last decade finding and iterating pattern templates that have value in the data domain. That knowledge and content has been instrumental in defining these output types. And that clarity has resulted in getting much clearer feedback on what output types we are missing.

So which approach are we going to take for #AgileDataDiscover? Follow along as we build in public.

Keep making data simply magical