When you work in a data team you have to split your time between building and delivering new Information Products and maintaining the ones you have already delivered.
DataOps patterns can help reduce the time you spend on the maintenance work.
Shane Gibson Blogs
Because sharing is caring
Building the Data Plane while flying it
In the data domain you typically have to balance between building the right thing and building the thing right.
The days of being able to spend 6 months or a year on “Sprint Zero” creating your data platform have gone.
One team I worked with called it “building the airplane as you fly it”
Here are 5 patterns I have seen data teams adopt to help them do this.
2024 the year of the Intelligent Data Platform
AI was the buzzword for 2023 and it will continue to be the buzzword for 2024.
I have been thinking about our approach to AI in our product for a while and landed on 3 patterns that I use as a reference.
Ask AI
Assisted AI
Automated AI
Adopting these patterns moves a data platform from being a manual data platform, towards a data platform that can do some of the data work for you.
An Intelligent Data Platform.
The 3 patterns of AgileData AI
Having AI embedded in your product have become table stakes it seems.
I have been thinking about our approach to AI in our product for a while and landed on 3 patterns that I use as a reference.
Ask AI
Assisted AI
Automated AI
The magic of DocOps
TD:LR Patterns like DocOps provide massive value by increasing collaboration across team members and automating manual tasks. But it still requires a high level of technical skills to work in a DocOps way. For the...
Iterations create milestone dates, milestone dates force trade off decisions to be made
Data teams struggle to not “boil the ocean” when doing data work.
Use milestones as a pattern to help the data team to focus on what really needs to be built and manage the trade-off decisions for what doesn’t.
Your data team are mercenaries, define your ways of working based on this
Modern data teams are transient, often staying less than 5 years, unlike past decades of long-term loyalty.
Companies should adapt by defining robust Ways of Working (WoW) that endure beyond individual tenures.
Balancing in-house teams with reliable data vendors for continuity and efficiency may also be a useful pattern as part of your WoW.
I’m getting pedantic about semantics
TD:LR Having a shared language is important to help a data team create their shared ways of working. When we talk about self-service, we should always highlight which self-service pattern we are talking about.I'm...
Most of your data is rotten and it’s not your fault, but it is your problem
Data quality is an expectation, not an exception.
While data quality is crucial, it’s not always directly our fault when issues arise, nevertheless, it remains our problem to solve.
Data Contracts are one pattern that can help us solve this problem.
Are you delivering drills, holes or outcomes?
TD:LR Whether you're a Data Entrepreneur or an organisation looking for actionable insights, its the business outcome these insights help you achieve that is the most important thing. Yes you need a data platform and...
Data Asset, Data Product, Data Service?
TD:LR Should we treat data as an Asset, a Product, a Service or a hybrid combination of all three? Data Asset, Data Product, Data Service? There has been a lot of discussions on LinkedIn, lots of podcasts, lots of...
Understanding Concepts, Details, and Events: The Fundamental Building Blocks of AgileData Design
Reducing the complexity and effort to manage data is at the core of what we do. We love bringing magical UX to the data domain as we do this.
Every time we add a new capability or feature to the AgileData App or AgileData Platform, we think how could we just remove the need for a Data Magician to do that task at all?
That magic is not always possible in the first, or even the third iteration of those features.
Our AgileData App UX Capability Maturity Model helps us to keep that “magic sorting hat” goal at the top of our mind, every time we add a new thing.
This post outlines what that maturity model is and how we apply it.
AgileData App UX Capability Maturity Model
Reducing the complexity and effort to manage data is at the core of what we do. We love bringing magical UX to the data domain as we do this.
Every time we add a new capability or feature to the AgileData App or AgileData Platform, we think how could we just remove the need for a Data Magician to do that task at all?
That magic is not always possible in the first, or even the third iteration of those features.
Our AgileData App UX Capability Maturity Model helps us to keep that “magic sorting hat” goal at the top of our mind, every time we add a new thing.
This post outlines what that maturity model is and how we apply it.
The Hitchhikers guide to the Information Product Canvas
TD:LR In mid 2023 I was lucky enough to present at The Knowledge Gap on the Information Product Canvas. Watch The Information Product Canvas, is an innovative pattern designed to capture data requirements visually and...
New Google Cloud feature to Optimise BigQuery Costs
This blog explores AgileData’s use of Google Cloud, specifically its BigQuery service, for cost-effective data handling. As a bootstrapped startup, AgileData incorporates data storage and compute costs into its SaaS subscription, protecting customers from unexpected bills. We constantly seek ways to minimise costs, utilising new Google tools for cost-saving recommendations. We argue that the efficiency and value of Google Cloud make it a preferable choice over other cloud analytic database options.
5E’s
As Data Consultants your customers are buying and outcome based on one of these patterns – effort, expertise, experience or efficiency.
We outline what each of these are, how they are different to each other and how to charge for delivering them.
Agile DataOps
TD:LR Agile DataOps is where we combine the processes and technologies from DataOps with a new agile way of working, to reduce the time taken and increase the value of the data we provide to our customers What's in a...
The “Killer” Feature
One feature to rule them all As product managers we are always looking for the next “killer feature” for our product. You know the one, that feature that will become the magical thing that will have customers flooding...
3 types of product features
Our UX/UI journey is accelerating We are currently full steam into the development of the initial User Interface for AgileData.io. The team have done some awesome work on the UX designs for a bunch of the core screens,...
Reducing Manual Effort, Everywhere, Every-time
Some tasks seem really small and only take minutes, but multiply that effort by completing that task a hundred times and you have found a task that should be automated. Collect your data In AgileData we automate the...
Why assumptions are just that
When we first sketched out our plans for AgileData we were pretty clear what AgileData would do and what it wouldn’t do. Those assumptions didn’t last long. Combine with magic We knew we wanted to focus on what we call...