Blogs
Because sharing is caring
Combining agile and data five years on with Blair Tempero
Guests Blair TemperoShane GibsonResources Join Shane Gibson as he chats with Blair Tempero on that last five years since they started the AgileData Podcast together.Listen on your favourite Podcast Platform | Apple Podcast | Spotify | Google Podcast | Amazon...
The Heart of Agile with Mike Leber
Join Murray Robinson and Shane Gibson as discuss business agility and the heart of agile with Mike Leber.
We talk about how the heart of agile liberates you from rigid process frameworks by focusing on collaboration, delivery reflection and improvement. We talk about how the agile industrial complex has turned agile into a heavyweight waterfall process. And how innovation is happening outside the traditional agile bubble. And lastly, we talk about agile leadership. If you’re interested in getting to the heart of agile and exploring business agility.
The state of product management with Jason Knight
Join Murray Robinson and Shane Gibson as they chat with Jason Knight on the state of product management.
Jason shares his insights and experiences on the responsibilities of a product manager, the evolving nature of the role. Differentiating buyer and user features the importance of testing and validating ideas. Supporting cross-functional teams and fostering a mindset of continuous improvement. Jason shares insights from his experiences and discussions with thought leaders and provides advice for those navigating the product management field.
There are 3 strategic / macro data use cases
I often ask which of these three macro data use cases the Organisations believed were its priorities to achieve their business strategy:
Providing data to Customers
Supporting Internal Processes
Providing data to External Organisations
Each of these three strategic / macro data use cases come with specific data architectures, data work and also impact the context of how you would design your agile data ways of working.
Eventually the data maintenance Tortoise will catch the new data work Hare
When you work in a data team you have to split your time between building and delivering new Information Products and maintaining the ones you have already delivered.
DataOps patterns can help reduce the time you spend on the maintenance work.
Building the Data Plane while flying it
In the data domain you typically have to balance between building the right thing and building the thing right.
The days of being able to spend 6 months or a year on “Sprint Zero” creating your data platform have gone.
One team I worked with called it “building the airplane as you fly it”
Here are 5 patterns I have seen data teams adopt to help them do this.
Dysfunction Mapping with Michael Lloyd
Join Murray Robinson and Shane Gibson as they chat with Michael Lloyd about dysfunction mapping.
Whether you are new to the agile world or a seasoned professional, you’ll find valuable insights as we delve into understanding team dynamics, honing the coaching process. And the essential principle of iterative hypothesis testing. Join us as we learned the power of mapping dysfunctions to unveil powerful solutions in agile environments.
2024 the year of the Intelligent Data Platform
AI was the buzzword for 2023 and it will continue to be the buzzword for 2024.
I have been thinking about our approach to AI in our product for a while and landed on 3 patterns that I use as a reference.
Ask AI
Assisted AI
Automated AI
Adopting these patterns moves a data platform from being a manual data platform, towards a data platform that can do some of the data work for you.
An Intelligent Data Platform.
Developing digital products in China with Bianca Grizhar
In this episode, we talk to Bianca Grizhar about developing digital products in China. We discuss the cultural differences between Germany and China, and how Scrum improved team collaboration. We talk about the benefits of working directly with people offshore in cross functional, cross organisational teams. Whether you’re a product manager or a developer, this episode is packed with practical advice that will help you work more successfully with your offshore teams.
AgileData App
Explore AgileData features, updates, and tips
Network
Learn about consulting practises and good patterns for data focused consultancies
DataOps
Learn from our DataOps expertise, covering essential concepts, patterns, and tools
Data and Analytics
Unlock the power of data and analytics with expert guidance
Google Cloud
Imparting knowledge on Google Cloud's capabilities and its role in data-driven workflows
Journey
Explore real-life stories of our challenges, and lessons learned
Product Management
Enrich your product management skills with practical patterns
What Is
Describing data and analytics concepts, terms, and technologies to enable better understanding
Resources
Valuable resources to support your growth in the agile, and data and analytics domains
AgileData Podcast
Discussing combining agile, product and data patterns.
No Nonsense Agile Podcast
Discussing agile and product ways of working.
App Videos
Explore videos to better understand the AgileData App's features and capabilities.
Data Products
Join Shane and guest Eric Broda as they discuss data products.
Using a manifest concept to run data pipelines
TD:LR … you don’t always need to use DAGs to orchestrate Previously we talked about how we use an ephemeral Serverless architecture based on Google Cloud Functions and Google PubSub Messaging to run our customer data...
“Serverless” Data Processing
TD:LR When we dreamed up AgileData and started white-boarding ideas around architecture, one of the patterns we were adamant that we would leverage, would be Serverless. This posts explains why we were adamant and what...
A Data Engineer an Agile Coach and a Fish walk into a bar…
This is the first of a series of articles detailing how we built a platform to make data fun and remove complexity for our users
Analysts can model democratising data modeling
In 2022 Shane Gibson was lucky enough to present “Analysts can model democratising data modeling” at the Knowledge Gap Conference
Watch the presentation.
Data Mesh Podcast – Finding useful and repeatable patterns for data
TD:LR I talk to Scott Hirleman on the Data Mesh Radio podcast on my thoughts on Data Mesh and the need for resuable patterns in the data & analytics domain. My opinion on Data Mesh I am not a fan of the current...
The Enchanting World of Data Magicians: Marketing Analytics vs. Product Analytics
Marketing Analytics involves analysing data from various channels, such as social media, email, and websites, to assess the performance of marketing efforts.
Product Analytics focuses on understanding and improving user experience and satisfaction with digital products or services.
What is Data Lineage?
TD:LR AgileData mission is to reduce the complexity of managing data. In the modern data world there are many capability categories, each with their own specialised terms, technologies and three letter acronyms. We...
Agile Security
Join Shane and guest Laura Bell as they discuss how you can eat the security elephant in an agile way.
Data Mesh 4.0.4
TD:LR Data Mesh 4.0.4 is only available for a very short time. please ensure you scroll to the bottom of the article to understand the temporal nature of the Data Mesh 4.0.4 approach.This article was published on 1st...
Abracadabra! Unravel the Mysteries of Data Catalog
Data catalogs are comprehensive inventories of an organisations data assets, helping data analysts and information consumers to quickly find, understand, and utilise relevant information. They foster collaboration, maintain data governance, and ensure compliance.
Catalog & Cocktails Podcast – agile in the data domain
Early in 2022 Shane Gibson was lucky enought to talk to the Catalog and Cocktails podcast crew about agile in the data domain. Watch or listen to the episode.
Whats the hottest new data thing in 2022 — Data Mesh or Metric Store
There is a lot of vendor washing going on A lot of data vendors are vendor washing their technologies to pretend they enable "Data Mesh" as they are punting on Data Mesh being the new thing for 2022. I think they are...
Data Observability Uncovered: A Magical Lens for Data Magicians
Data observability provides comprehensive visibility into the health, quality, and reliability of your data ecosystem. It dives deeper than traditional monitoring, examining the actual data flowing through your pipelines. With tools like data lineage tracking, data quality metrics, and anomaly detection, data observability helps data magicians quickly detect and diagnose issues, ensuring accurate, reliable data-driven decisions.
Analyst vs Analytics Engineer – Benn Stancil
Join Shane and guest Benn Stancil as they discuss the difference between the age old analyst role and the new emerging role of analytics engineer (amongst a few other interesting things)
3 agile architecture things
Join Shane and guest Brian McMillian as they discuss the art of architecture in an agile data world. We discuss 3 things: 1. the 4x approach2. data vault3. everything is code
AgileBI – Raphael Branger
Join Shane and guest Raphael Branger as they discuss combining agile ways of working with the world of data and business intelligence (BI).
Agile and Analytics – Shaun McGirr
Join Shane and special guest Shaun McGirr as they discuss the combining of agile ways of working with analytics teams.
AgileData >>> Modern Data Stack
TD:LR AgileData's mission is to reduce the complexity of managing data. A large part of modern data complexity is selecting, implementing and maintaining a raft of different technologies to provide your "Modern Data...
A selection of practical agile patterns when using Data Vault
In 2021 Shane Gibson was lucky enough to present “A selection of practical agile patterns when using Data Vault” at the Knowledge Gap Conference
Agile DataOps
TD:LR Agile DataOps is where we combine the processes and technologies from DataOps with a new agile way of working, to reduce the time taken and increase the value of the data we provide to our customers What's in a...