Blogs
Because sharing is caring
Bridging Data and Product Management Practises and Patterns with Juha Korpela
Join Shane Gibson as he chats with Juha Korpela on how to adopt patterns and practises from Product Management and apply them to the data domain.
Innovation and disaster with Ron Westrum
Join Murray Robinson and Shane Gibson as they chat with Ron Westrum about how organisational culture drives innovation and disaster. We discuss generative, bureaucratic and pathological cultures, using real-world examples from NASA, Boeing and others. This episode highlights the importance of leadership integrity, openness, honesty, the free flow of information, and a mission-focused approach to achieving organisational goals. Listen for insights on identifying and promoting a generative culture, dealing with bad news, the role of technical maestros, and much more.
Introducing Hai, AgileData 2024 Data Intern
I’m Hai, a name that intriguingly means “hi” in English. Originally from Vietnam, I now find myself in Australia, studying Data Science and embracing an internship at AgileData.io. This journey is not just about academic growth but also about applying my knowledge in practical, impactful ways. Join me as I explore the blend of technology and community, aiming to make a difference through data.
Unmanaged with Jack Skeels
Join Murray Robinson and Shane Gibson as they chat with Jack Skeels about agile for digital and marketing agencies. Jack advocates for reducing the managerial overhead in organisations to promote productivity and improve communication. We discuss common hurdles in agile implementations. The value of a manager as a sports coach. The importance of scoping the work. Over managing and how real change must start from the top. This episode is full of practical advice and critical insights into organisation dynamics. This conversation is especially useful for those seeking to implement agile in a more holistic, efficient, and outcome oriented way in a service provider.
Patterns for being a successful internal data consultancy with Dylan Jones
Join Shane Gibson as he chats with Dylan Jones on how to adopt patterns used by successful data consultancies and apply them in your organisation as an internal data team.
Defining self-service data
Everybody wants self service data, but what do they really mean when they say that.
If we gave them access to a set of highly nested JSON data, and say “help your self”, would that be what they expect?
Or do they expect self service to be able to get information without asking a person to get it for them.
Or are they expecting something in between.
I ask them which of the five simple self service patterns they want to find, which form of self service they are after.
OKR’s with Adrian Howard
Join Murray Robinson and Shane Gibson as they discuss objectives and key results with Adrian Howard.
Adrian explains how OKR’s can be used as a strategic instrument for aligning goals within your organisation. We walk through common pitfalls and misconceptions emphasizing the importance of using OKR’s focused on outcomes rather than outputs. We discuss the potential for using OKR’s with vendors. The idea of paired metrics. And how OKR’s can serve as indicators of underlying problems within your organisations culture and structure.
Combining agile and data five years on with Blair Tempero
Guests Blair TemperoShane GibsonResources Join Shane Gibson as he chats with Blair Tempero on that last five years since they started the AgileData Podcast together.Listen on your favourite Podcast Platform | Apple Podcast | Spotify | Google Podcast | Amazon...
The Heart of Agile with Mike Leber
Join Murray Robinson and Shane Gibson as discuss business agility and the heart of agile with Mike Leber.
We talk about how the heart of agile liberates you from rigid process frameworks by focusing on collaboration, delivery reflection and improvement. We talk about how the agile industrial complex has turned agile into a heavyweight waterfall process. And how innovation is happening outside the traditional agile bubble. And lastly, we talk about agile leadership. If you’re interested in getting to the heart of agile and exploring business agility.
AgileData App
Explore AgileData features, updates, and tips
Network
Learn about consulting practises and good patterns for data focused consultancies
DataOps
Learn from our DataOps expertise, covering essential concepts, patterns, and tools
Data and Analytics
Unlock the power of data and analytics with expert guidance
Google Cloud
Imparting knowledge on Google Cloud's capabilities and its role in data-driven workflows
Journey
Explore real-life stories of our challenges, and lessons learned
Product Management
Enrich your product management skills with practical patterns
What Is
Describing data and analytics concepts, terms, and technologies to enable better understanding
Resources
Valuable resources to support your growth in the agile, and data and analytics domains
AgileData Podcast
Discussing combining agile, product and data patterns.
No Nonsense Agile Podcast
Discussing agile and product ways of working.
App Videos
Explore videos to better understand the AgileData App's features and capabilities.
Information Products – What and Why with Tammy Leahy
Join Shane GIbson and Tammy Leahy as they discuss what the Information Product pattern is and Why you would use it.
App Engine and Socket.IO
We wanted to be able to dynamically notify Data Magicians when a task had completed, without them having to refresh their browser screen constantly. Implementing websockets allowed us to achieve this.
I can write a bit of code faster
TD:LR To get data tasks done involves a lot more than just bashing out a few lines of code to get the data into a format that you can give it to your stakeholder/customer. Unless of course it really is a one off and...
The Focus Podcast – Agile Data Governance Patterns
Early in 2022 Shane Gibson was lucky enough to talk to the Focus podcast crew about agile governance in the data domain. Watch or listen to the episode.
ELT without persisted watermarks ? not a problem
We no longer need to manually track the state of a table, when it was created, when it was updated, which data pipeline last touched it …. all these data points are available by doing a simple call to the logging and bigquery api. Under the covers the google cloud platform is already tracking everything we need … every insert, update, delete, create, load, drop, alter is being captured
Three Agile Testing Methods – TDD, ATDD and BDD
In the word of agile, there are three common testing techniques that can be used to improve our testing practices and to assist with enabling automated testing.
Scaling data teams – Tammy Leahy
Shane Gibson chats to Tammy Leahy about how she helped scale the data teams she leads.
Data Products
Join Shane and guest Eric Broda as they discuss data products.
Using a manifest concept to run data pipelines
TD:LR … you don’t always need to use DAGs to orchestrate Previously we talked about how we use an ephemeral Serverless architecture based on Google Cloud Functions and Google PubSub Messaging to run our customer data...
“Serverless” Data Processing
TD:LR When we dreamed up AgileData and started white-boarding ideas around architecture, one of the patterns we were adamant that we would leverage, would be Serverless. This posts explains why we were adamant and what...
A Data Engineer an Agile Coach and a Fish walk into a bar…
This is the first of a series of articles detailing how we built a platform to make data fun and remove complexity for our users
Analysts can model democratising data modeling
In 2022 Shane Gibson was lucky enough to present “Analysts can model democratising data modeling” at the Knowledge Gap Conference
Watch the presentation.
Data Mesh Podcast – Finding useful and repeatable patterns for data
TD:LR I talk to Scott Hirleman on the Data Mesh Radio podcast on my thoughts on Data Mesh and the need for resuable patterns in the data & analytics domain. My opinion on Data Mesh I am not a fan of the current...
The Enchanting World of Data Magicians: Marketing Analytics vs. Product Analytics
Marketing Analytics involves analysing data from various channels, such as social media, email, and websites, to assess the performance of marketing efforts.
Product Analytics focuses on understanding and improving user experience and satisfaction with digital products or services.
What is Data Lineage?
TD:LR AgileData mission is to reduce the complexity of managing data. In the modern data world there are many capability categories, each with their own specialised terms, technologies and three letter acronyms. We...
Agile Security
Join Shane and guest Laura Bell as they discuss how you can eat the security elephant in an agile way.
Data Mesh 4.0.4
TD:LR Data Mesh 4.0.4 is only available for a very short time. please ensure you scroll to the bottom of the article to understand the temporal nature of the Data Mesh 4.0.4 approach.This article was published on 1st...
Abracadabra! Unravel the Mysteries of Data Catalog
Data catalogs are comprehensive inventories of an organisations data assets, helping data analysts and information consumers to quickly find, understand, and utilise relevant information. They foster collaboration, maintain data governance, and ensure compliance.
Catalog & Cocktails Podcast – agile in the data domain
Early in 2022 Shane Gibson was lucky enought to talk to the Catalog and Cocktails podcast crew about agile in the data domain. Watch or listen to the episode.
Whats the hottest new data thing in 2022 — Data Mesh or Metric Store
There is a lot of vendor washing going on A lot of data vendors are vendor washing their technologies to pretend they enable "Data Mesh" as they are punting on Data Mesh being the new thing for 2022. I think they are...
Data Observability Uncovered: A Magical Lens for Data Magicians
Data observability provides comprehensive visibility into the health, quality, and reliability of your data ecosystem. It dives deeper than traditional monitoring, examining the actual data flowing through your pipelines. With tools like data lineage tracking, data quality metrics, and anomaly detection, data observability helps data magicians quickly detect and diagnose issues, ensuring accurate, reliable data-driven decisions.