The pattern of writing a data book with Shane Gibson and Ramona C Truta

Join Ramona C Truta as she interviews Shane Gibson about the patterns of writing a data book (and they discuss a whole lot of other data things)

Read along you will

Shane: Welcome to the AgileData Podcast. I’m Shane Gibson. 

Ramona: And I’m Ramona C Truta. 

Shane: Hey, Ramona. Thank you for coming on the podcast today. Today we have something slightly different because today I’m gonna be the guest and you are gonna be the host. So this is gonna be a really interesting experience for me. But before we do that, why don’t you give the audience a bit of background about yourself.

Ramona: Okay. I have started in data since ever I joke that I came on this earth doing data, and I’m gonna explain. I’ve always been the kid who arranged all the fundraising, did all the financials. I was the trusted kid who really did every year, all of that. And so it was data, right? Not Excel spreadsheets, but with pen and paper, and I loved math and always dreamed of becoming a math teacher.

At that time in Romania, we had to pass an admittance exam, and I went to fill in the forms and I had three separate options. One was in computer science. One was math and the other one was applied math. And the idea is that everybody would write the same exam and based on the mark that you get, you’ll be distributed in your section.

So as I was. Filling in that form. A friend of mine told me, why don’t you put computer science first? Because you can do with computer science, everything that you can do with math and some, of course, I wrote that first. I got a very high mark. I got into the computer science program, clueless as to what that meant.

My first reaction to the first. Computer science class was this X equals x plus one. Now, for a mathematician like me, I was that in the water. I just, let’s just say I had a very terrible first computer science teacher and I struggled. I struggled with a computer science for a while, and luckily for me, I excelled at math.

So that really counterbalance and all the logic and all those other courses. Long story short, I get to the databases course and coincidentally at the same time with the probability and statistics. So that saved me fast forward. My first job out of university was working with Data and Fox Pro. I worked for high school and I was preparing everything from design to implementation.

The next job for a multinational, it was with. Progress, another database job, and my first project that I was working on was doing black box testing. So this application that we’re developing was for a Dutch company and everything was in Dutch. I had to use two separate dictionaries because there was no Romanian to Dutch translation.

I had to use English as an intermediary. I love that job. And they advanced me very quickly, put me in charge for the project, and that’s the time when I got to a job in academia, which was actually my dream. And, uh, funny tidbit, I had to do a lecture, part of the series of interviews, and my lecture was in data modeling.

I always remember this and I got to teach that, and then I met my future reason to move to Canada. I got a scholarship to come and do a PhD program at University of Toronto, and during that time I was in the database group. So I continued with that and my research was in Xlx Query. So we did that kind of research at that time and I started teaching the database courses at the computer science department at University of Toronto, and I’ve got to teach other courses, software engineering and other courses to make up the schedule.

I’ve been doing that for a while. And then I stopped teaching and moved to a different university to do research and I was a data person, and that is where I really got to practice all the things that I’ve been teaching and more. And to this day, I guess that was my absolute favorite job. Fast forward to today.

What I am now is a facilitator of knowledge. This is what I’m calling myself these days. I just love to do that in the sense that I take the tech talk, like really what the technical people talk and translate it into business talk and vice versa. And I talk a lot of strategy, especially in the data and AI era that we live.

There’s a lot of. Translation that is needed to be done. You probably have witnessed this yourself. A lot of people talk. Like they know everything, but it’s just what I call the foam on the top of the cappuccino. And I’m the person that goes down to the caffeine. So I remove the fluff, I remove the foam, and I speak caffeine, like really strong espresso, caffeine 

Shane: as a coffee lover, I’m there with you.

So yeah, that idea of being a translator, I think we see. People fall into that role over time, and it is typically people, we’ve talked about our ages before we started recording, but people of our generation, our age, because we’ve spent years observing and learning and training our brains to recognize patterns and be able to go, oh, actually I can describe that quite clearly.

Because I have all these frameworks and patterns that I can anchor to. I have these examples and I can bring those in so that I can explain something that’s complex in a relatively simple way to somebody else who’s never dealt with that complexity. And I think that is a really important role, especially now with all the AI stuff coming.

That ability to be able to provide that role is gonna become. Far more valuable than it’s ever been. 

Ramona: Absolutely, Shane, and I’m really glad that you brought up our conversation prior to recording because what we said, we’ve lived several decades and we’ve collected a lot of data points and data. We have these massive data sets, and now we can build those knowledge graphs of everything that we have accumulated during this time.

And while the new technologies. May look new, but they are built on similar principles and we have a system thinking, and then we just apply all those frameworks. And I think this is where we are really valuable because we have all this body of knowledge that we’ve accumulated and we’ve built and we shaped.

Our minds. 

Shane: Hopefully it’s true because I’m really bad with a hammer and a screwdriver or any of those kind of, so I’m gonna have to rely on the last decade of my working life, still being knowledge, not hard work. Okay. Like I said, we’re gonna flip the script. You’re gonna be the host. I’m gonna be the guest.

Yeah. So 

Ramona: first of all, Shane, huge congrats and I applaud you for. Finishing up the book and putting it out there, and I hope the book does well. 

Shane: Lots of people have asked me why did I decide to write a book and it was the scratching itch. It was just something that I decided I wanted to do. And so yes, there’s lots of knowledge in there.

I hope people adopt. I hope it’s successful. Somebody told me really early on, you don’t make money out of a book and looking at how much you get paid. You don’t. And so for me, it was about scratching a itch and the other thing people said was. It’s incredibly hard, like lots of people start and never finish, and it’s been a few years and now I understand why.

It’s really interesting. You don’t really understand how hard it is. It doesn’t seem that hard, like to just write a book. It doesn’t seem that daunting at the beginning, 

Ramona: especially with Judge GPT. Right. You just prompted there and then the book writes itself, right? Yes. 

Shane: Yeah, 

Ramona: and we know that there are books like that.

Tell us a little bit about, first of all, the title and how you decided on this particular book. I know it’s all about patterns, and I shared with you earlier that ever since I’ve met you and I’ve known you, it was all. Pattern, this pattern, that pattern the other. And come on, what, why does everything has to be a pattern with you?

And it took me a while to understand what was a pattern to you. So take us to this journey, if you will. Yeah. So let’s 

Shane: start off with patterns, right? There’s a book that was written many years ago, a patent language, and it was about patents for building houses. And the concept of a patent really is there’s a common problem.

There’s a common solution that fixes that problem most of the time, and therefore, if you can articulate what that solution is to that problem, it becomes a pattern. If we think about a house, when we want to build a house, there’s inherent set of patterns in that house. If I want to build a lounge, I am typically gonna have a, a room that is a certain minimum size.

It won’t be a cupboard, it’ll be bigger than that. It will typically have, depending where I live, lots of windows and light. It’ll have a place for couches and seats that probably in the modern world, we have a place for a television. It may have a heating source, like a fire. If I compare the design of that room to a bathroom, a bathroom’s typically gonna have one window.

That window’s probably gonna be opaque because you don’t want people pairing in, if I live in a sunny place, I’m probably gonna have a porch. I’m gonna have an open space outside where I can sit and it has some form of shelter from the sun, but it’s open, or it has a series of windows I can open. Each one of those is a pattern.

It’s a solution to where I wanna spend most of my time during the day where I want to go clean myself, where I wanna sit when it’s sunny and fresh. And that’s what the. Book was all about, was describing these patterns for architecture. And if we think about that one more step is if we were gonna build a house nine times outta 10, we’d do architectural drawings, we’d do a blueprint, we would sketch some details that make us think about the layout and the patterns we’re gonna apply, and also give us something to follow and iterate as we learn some more things.

And so for me. That was really intriguing. It was intriguing because a few decades ago I had a consulting company, so the usual five to 20 people, data and analytics consulting company, and I used to get really frustrated that whenever we went into a new customer, we seem to be reinventing the wheel. We are like, oh, let’s start from scratch.

And I saw a massive amount of waste and inefficiency when we did that. And so my natural reaction. Was to go to methodologies. Yeah. Oh, we’ll write a methodology. We’ll write this book of rules, and everybody would follow the rules. It was a fricking epic failure. Nobody followed the rules. The rules didn’t work half the time because every context was slightly different.

The amount of money we spent trying to write the methodology was waste compared to how much we got out of it, and so I got really frustrated and then I lucked into this thing called agile, which completely changes the way you think. Aligned with that came this idea of patterns of small chunks, of proven solutions for a common problem, and then what we call antipas.

The times that we know that patent, actually I juices more waste than the problems it solved. So when I first started to write, I thought about what I wanted to write because I, I had many goes at starting that failed. I sat back and went, oh, I’ll just write this big book of patterns for data. And that’s actually the first thing I tried to do.

It was an epic failure in terms of the outcome, but it was brilliant in terms of the learning that helped me finally finish a book. 

Ramona: Oh, okay. Why do you think that happened? What exactly made that failure happen, or what have you learned from that? Aside from shrinking the knowledge and trying to fit something, did it change your perspective or your approach?

What influence. The most. And what made you change what you took with you and you carry with you in the future? 

Shane: So I think for me it was understanding the way that I’m driven. Like I said, I had a whole bunch of false starts many years ago. Over many years was like, okay, I wanna write a book. It’s a niche.

And then I start to do something and nothing would happen. And one of the things everybody tells you to do is write the outline of the book. You know, write the table of contents and. For some people that is a good approach. For me, it was an epic failure because what I kept doing was I kept rewriting the table of contents.

I’d write the table of contents and I go, yep. And I’d write two paragraphs of actual content and I’d go, oh, that doesn’t fit the structure of the book. So I’d go and seriously waste more time restructuring the book. And after a while I realized that was an anti-patent for me. Create the table of contents.

The other problem I had was I would write when I felt like it and then I would not write. And that consistency was not a good thing. So I decided that what I needed to do was a forcing function that would make me write consistently and not worry about the content yet. And so when I do forcing functions, what works for me is make them public.

So I said I’m gonna write 60,000 words in six months. So that’s around about 10,000 words a month. I am going to, on LinkedIn, publish my word count. As a recurring theme so that I could hold this metric to myself of how many words I wrote. I am not gonna care about the quality of the words. I’m not gonna care about what the words are.

I am just gonna prove that in six months I could write 60,000 words. And I did that, and I used Substack and I created this substack site called the Agile Data Book Book of Ways of Working. And that was just 60,000 words. Now, what was interesting about that is it forced me to research. So I would write it about a pattern for say, team design.

And I’d go, yep, okay. This is the way I design a team. Or actually the example I used is because I’d go, I can’t describe it. It’s one of those things you go, what’s the agile when you feel like it and it, that’s not a great piece of content. So I started doing research and then I found this research around fixed mindset and growth mindset and a book by Carol Duck all around that.

And what I could do by doing that research and doing that reading is I could see the patterns that were anchoring me and where they might have come from. ’cause I can’t remember where I read them. I read lots of books. I watched lots of things. I talked to lots of people, I experimented, but I could, couldn’t see the genesis of where the idea came from.

And that actually made my content better in my view because now I, I could think about it more. I could rewrite and write some stuff. And for me, writing those 60,000 words. Well, six months. It gave me a whole lot of. Process that I’ve used from then on, but also gave me a whole lot of content ideas and a whole lot of research.

So that was the first step. 60,000 words of absolute crap to prove I could write 60,000 words. 

Ramona: And I think that’s when we first met right around that time. And I remember. I was supposed to contribute to some substack and you said, oh, this pattern, and you gave it a name, and I read the thing and I said, oh, I know this thing.

I just didn’t know that it has a name. And I think in the perfectionist in me, I. Was upset that how come there is a thing that has a name and I do the thing, but I don’t know the name. So this is where my frustration with everything, pattern and name came to some context because you know that perfectionist, why don’t you know this thing?

And I started beating myself up. Anyways, that, that’s an aside. But it sounds to me like even in the first iteration where you were trying to come up with that. Table of contents, it really sounds to me like you are trying to figure out a pattern in the knowledge that you wanted. It’s meta the way I see it, that you try to approach everything from a pattern point of view, including the thing that describes the pattern.

I don’t know. In my mind it looks like that. 

Shane: So I 

Ramona: think now 

Shane: in hindsight, after a couple years. 

Ramona: Yeah, I 

Shane: think that’s what I was doing, but I didn’t know it at the time. Oh. And actually, again, so forcing functions are really important for me, otherwise I just won’t get stuff done. I was the kid at school that was finishing his essay at 2:00 AM in the morning because he hadn’t done it for the last four weeks.

Yeah. And that really hasn’t changed in how it worked. Yeah. Know. One of the things that was intriguing me was, why is it so hard to write a book? And one of the things I’ve got to now is actually the first book’s hard, but the second book’s harder. And so I’ve got a challenge on, can I write a second book in six months using the patterns that I’ve created for this book?

So we’ll see how that one goes. I don’t think linearly, I don’t think in a structured way, I think in chaos and randomness, and that’s just the way I’ve always worked and I’m constantly distracted, so I had to find a writing technique. For that. And then I had to learn some new skills because I lasted six months at university when I was a kid, and then I left and went to work.

So I’d never really learned the research skills that you get from a good university. So what I found was I could write, I. But I couldn’t justify what I was writing and therefore I had to learn this whole new pattern of research. Whereas for you, I think you understand research intrinsically ’cause you’ve done it all your life.

And so you had that foundational piece that I didn’t have. I think the second thing is, over the last couple of years. I, I tend to prefer to work with other people. I find it more invigorating. I find it more fun. I find having somebody else that I’m working with gives me those deadlines to force me to do that work.

It’s a forcing function and I have tried multiple times to write this book with many different people, and every one of them was an abstract failure and, and it’s really interesting. It’s like, why is it me? Is it the book actually? Collaborating on a book is actually harder than writing it by yourself? I think so.

I’ve kinda resigned myself now that every other book I’m gonna write is just gonna be me. Yeah. Well, but having said that, in the book that I did publish, there were a whole lot of example canvases that people generously spent their time creating without me. They’ve gone with the book. So actually the book is a collaboration.

It just wasn’t co-written. In the typical way you would co-write a book, which again is an intriguing pattern. 

Ramona: You ju I was just gonna comment that. Eventually you found your pattern to write this book, something that worked for you and the collaborators, and we have a few friends who wrote books and probably they are agreeing with you on the side of having.

Co-authors and collaborators, and it’s hard in general when you work with somebody and having different styles and it’s not for everyone and it’s a different set of skills that is required and always having to. Push towards your own issues. The other person issues. I 

Shane: think the other thing is I’m highly opinionated and I had a certain vision for the book, like the layout.

The book is about the information product canvas, and that is an iteration of the business canvas and I love the Business Canvas book. When it came out, just the style, the form, and the function of it. So this book is an iteration of that style. It is more coffee table than it is textbook. And I think if I ended up with a true co-author, there would’ve been a hell of a lot of tension because I don’t think I’d be willing to give up the structure of that book.

And in fact, the early feedback I got. On the structure of that book was that it was crap and it’s gonna be really interesting. Now it’s published to see whether that’s still true. But it’s the book I wanted to write and like I said, I wrote it to scratch a itch. 

Ramona: Did you succeed? Hard Scratch? 

Shane: Yeah. Yeah, it’s definitely a hard scratch.

Yeah, I, I like the book more than I thought I would, which for me is a big compliment. ’cause normally I do something and I hit it. 

Ramona: Okay. So Diane, it’s a success. You achieved what you set out to do. Yes. And you were happy with the result. What? Yeah, 

Shane: but a couple years late. 

Ramona: But that, let’s go back a little bit in the process and see what other patterns we are identifying.

You say several years later, but there was a purpose or since you studied that fixed versus. Growth mindset. You had to transition towards more growth mindset than the fixed one. So that gave you plenty of opportunities. So just consider how much growth there is and how much you got out of that process on its own to achieve that kind of shift, because that is huge.

And especially as we grow older, if we are a fixed mindset. Than the older we are. The older we get, the much harder it is to shift to the other side. So I wanna. Congratulate you on that and give yourself grace for it. It’s a huge achievement. 

Shane: And I think, yeah, fact that actually I finished it. ’cause you know, I do tend to start some things and not finish them on a regular basis.

Ramona: Shane, let’s admit that’s a pattern that. I don’t know anyone who doesn’t follow it, let’s put it that way. And especially now with AI is so easy to experiment and, oh, I have a weekend. Let me vibe code something. I know your Opinionated did about that. I read what you wrote because everything is so easy.

And then you start something and then the mind goes to the next thing and the next thing and at the end of the weekend, okay, what do I have to show for? And this is something that I bring up more. I wanna talk about the new paradigm or pattern to use your preferred term of learning how to learn.

There’s a lot of shift in how we learn now in this context and for us all timers, we have a lot of knowledge as we established at the beginning and now with the vibe coding and all these tools that do a lot for us. We immediately can make connections, oh, it’s doing this even though it’s something that we never saw before.

Code we never wanna learn and do it ourselves, but we have anchors and there are things that we have knowledge about. We may not know syntax and stuff, but we understand what’s happening, and then we can go. And ask in a chatbot, Hey, tell me more about this and teach me more about this, and then check my knowledge.

I do that for myself. It’s a lot of fun and I don’t know that I’ve ever been this astic about learning new things as I am now 

Shane: we’ve got access to a knowledge base. In a way we’ve never had before. I’m old enough to remember the Encyclopedia Britannica Uhhuh as the thing you used to have to go and look up when you wanted to understand what the thing was.

Yeah. You used to have to go to the library at school and get a book out and read it and everything’s changed. I think there’s some interesting concepts out there now that with the advent of the LMS and the gene AI and the access to that knowledge, that curators of that knowledge, humans that actually tell a story.

Based on that knowledge in a different way will actually become the more successful people having the knowledge is no longer good enough because that knowledge is now available at a click of a button type of a key. It’s the way you tell the story that becomes intriguing. So we’re going back to thousands of years ago to storytellers being the most valuable part of the tribe.

And that’s intriguing. So 

Ramona: intriguing how? 

Shane: Because it’s, it goes back to an old pattern. It goes back to the storytellers and the way they told the story. Becoming more important than the holding of the knowledge, the articulation, the ability to have somebody else understand what you understand to coach and mentor and teach.

Yeah. ’cause the knowledge is just available to any, 

Ramona: and it’s just the way we transfer that knowledge. Going back to what I was saying at the beginning, that I’m a facilitator of knowledge and. I’m a visual thinker and a systems thinker and all sorts of other things, and what I observed a pattern, I observed this pattern that I have conversation with many people, just as you said, right?

You talk to a lot of people. I read a lot, I see a lot. I consume a lot of knowledge, and suddenly it’s all that care, chaos. Gets into an equilibrium and I get a visual or a simple sentence, but I get a picture, and that picture can be described in so many words and I can use it. It’s a pattern or a framework on its own that I keep using and I keep finding usage for it.

Like the cappuccino thing, or another one I came with was the puzzle versus. Lego in this highly siloed world we live, everybody’s working on their own tiny piece of puzzle and then they expect that they all get glued together somehow, or everything feeds and expect to wear like Lego, which no matter which Lego you buy, which Lego set they will fit on top of.

Another, and you can build something, but that’s not the reality. You know how Joe encourages everyone to write an article explaining why they write or their process? So I was 13. I. And in Romania we had to give an exam at the end of grade eight to go to high school. So we had to decide what kind of high school we wanna go, et cetera.

And at 13, I remember vividly I had this conversation in the schoolyard with my Romanian literature teacher, and she asked me, so Ramona, what are you gonna do? Storytelling or math? I was. Exceptionally good at storytelling. I know how that comes across, but it was effortless for me, and I would go to competitions, olympiads.

I could never qualify for math to higher levels, but for storytelling, I would just go and I would always win. So I had to make that decision. And what hold me back for going onto the humanity side of training was the fact that I didn’t feel. Good enough for a foreign language. So my first foreign language that I had to get in school was Russian, then French.

But I had such a terrible teacher that we were clueless with French. I’ve learned French going to high school, so I picked it up English when I went to university. So I go into the sciences. I did math and physics, and I remember years later in grade 11. I mean this class with these kids, everybody who was anybody in my little town, and we have the Romanian literature exam, grade 11, and we are given a fragment of one of the books that we’ve been studying that trimester and to write an essay.

And when the teacher brings back the exams, she. Acknowledges me in front of the whole class. She highlights my work and my essays and my storytelling and all of that. And I thought, huh, how interesting. Because I was never acknowledged in that way for that particular subject, and it was the only time she ever gave me the 10 out of 10 mark that stuck with me.

And here I am. Decades later, finding myself in that passion of telling stories again and wanna tell stories about so many things. 

Shane: I remember years ago I used to work for SaaS, the software company. Mm-hmm. Statistical company, and I was in pre-sales, so I wasn’t in the implementation side, but I spent a lot of time with, back then they were called data miners, and one of the techniques they had that still sticks with me was they would go and build an analytical model using a neural net.

They’d do it with a bunch of different stats techniques. Often it was marketing related, so they’d look at the lift chart and they’d go, that model with those parameters is the best fit for solving that business problem. And sometimes the neural net would be the one that wins. The problem was you could never explain to the customer what the hill it was doing, because how does it work?

We basically push all this data in, we set these three flags, we let it run, and it came back with the best answer. So what they used to do was they used to produce a decision tree model at the same time, and they used to use the decision tree to explain to the customer what’s happening. And they would say, this is not the model, but the model is doing something like this.

It’s looking at customer and it’s looking at length of time they’ve been with us, and then total spend and the number of. Contact center calls to come up with these clusters. And although it wasn’t the actual model, it was able to tell the story that the model was probably doing. And so for me, that’s always stuck with me is that we have to be able to take these technical things we do and explain them by just telling stories.

And when you’re writing a book, for me it became actually, how do I tell a story? Because I know the techniques, the patterns that I was using. ’cause I’ve used them for a decade with lots of people. I knew they worked. But there’s this really weird thing that when you are forced to write. Actually, it helps enhance your understanding of what you know.

And so on. This podcast, what I do now for this year, I will say to somebody who’s coming on as a guest, write an article about the pattern we’re gonna talk about. Not because I need to see it, but that art of writing, that process of writing the article about what you think you know will make you know it so much better.

And for me, that was something that I didn’t expect because I, my whole life has been talking and waving hands and drawing pictures compared to math or writing. Mm. And so moving into the skill that I didn’t have, and I still don’t think I have it very well, but it helped me articulate and understand what I thought I knew.

A lot more because once you’ve written it and you go, that’s his pants, I don’t understand what I wrote. How does anybody else do it? I think that is actually part of the value of writing. So again, something new for me. 

Ramona: You’ve explained that really well because we have so much in here, right? So much as we’ve accumulated, and I see this in all the conversation, especially with tech people, and they immediately go to tech jargon and.

What I want is I wanna hear a story that is tech agnostic. Can you distill what you’re saying without all the jargon? And I’ve been in meetings sometimes we were in those meetings together and all times I had to figure out a translation in my head, oh, this person is talking about dots and this person is talking about the other thing.

But I think they talk about the same thing, but they can’t figure out that they’re talking about the same thing because. They use their own particular jargons. So that was a big shift for me, understanding that. People around me are talking pretty much the same thing. They just don’t use the same vocabulary or they’re very particular and very opinion opinionated on their respective jargons.

That’s why I became so passionate. I’ve always been a teacher. I’ve. Taught thousands of students and had to explain and distill and all of that. But it gives me so much more passion right now, realizing how people just touch the surface of things and don’t know what’s underneath, but they would like to know, and they just need a different way of explaining that without all the jargon and all the technical details.

You just wanna understand what’s in that box. At a level that allows you to have a conversation with somebody else or a convincing conversation. 

Shane: And one of the interesting things for me was, so I started off with 60,000 words. I looked and said, actually, if I documented every patent that I think I know it’ll be 600,000, 6 million, and it’s never gonna happen.

I then picked one of those areas, which was the canvas, and I started writing the book and it got too big. Again, because a patent, especially in the data world, doesn’t live on its own. So the canvas is a pattern, but actually there’s a work shopping pattern. There’s the who does what pattern, the core business event.

Model patterns already in there, prioritization personas, there were all these other patterns that it used that leveraged. And so I ended up again with a book that was too big and I had to stop myself and refocus. And then the other thing was, whenever I used a term, I started to go into too much detail.

For me, it was like, oh, I’m gonna go and explain it to the degree. And again, that made the book too big. I was going, if I keep writing at that level, I’m never gonna finish. And so what I decided was to write a book for people that were inquisitive. So it tells you what you need to know, but actually you still have to go and do more work.

And it’s really interesting. As part of our startup, we have people that are in the middle of their degrees from around the world, come and do work with us for three months. And I remember I had one student and we give him a bunch of choices of what they can work on, and he said he wants to get into product analytics.

And I said, okay, you’ve got two choices. You can do product analytics or marketing analytics for a product. You need to pick which one that you’re gonna do. And he goes, what’s the difference? And I went, actually, your first task is to go and give me back one page that describes the difference between product analytics and marketing analytics and.

I waited and I waited, and a week later I went, where is it? And he goes, I couldn’t find anything. And 10 seconds later, being grumpy old man that I am, I’d Googled it. I found 20 references and I sent it back to them. But what that meant was he wasn’t interested. He wasn’t inquisitive. He was only doing this to pass a tick in the box.

He didn’t actually wanna learn. And so I think again, that’s part of it is that. Books are there to help you learn the next thing you need to learn. But that second step, going from the 60,000 words to what I thought was very focused and then starting to write and then realize, no, it’s still too big, forced me to go to that third iteration, which was, what am I not gonna write?

That was more important. The constraints based model of, I am not gonna write about these things, was the last thing that I had to understand in my writing process. So start really big, lots of stickies, complete chaos. Find something that looks like it coalesced into a great story that I could tell that was different to every other story that I’d read.

Start to write that story and realize it’s still too big. There’s too many 10 rules, the graphs got too many nodes. And then a constraints based model to say, what am I not gonna do? And then that was the process that let me get to 193 pages in a couple of years. And the test, like you said, is I think I have a pattern to write the next book.

I know what the subject is. Can I reuse that writing process that I think I have to do 193 pages in six months? That’s gonna be the test for me. ’cause that’s when these are patterned, not just a bunch of ad hocness.

Ramona: So Shane, during the process of writing this book, have you uncovered some new pattern that you didn’t know before, something that really excited you so much that you wanted to include, and maybe something that is one of your preferred patterns right now? 

Shane: Yeah, there’s a couple. So one of the things that.

Happened when I was writing the book was I had created a course for the Canvas, which I used to teach private lead to organizations, and I made public on Maven, and what I didn’t realize was running that course helped me refine the content, and it also helped me get feedback on that content because effectively I was practicing delivering that content time and time again.

So what I’m gonna do next time is I’m gonna write the course first. Then write the book. And the second thing is I’m gonna focus a lot more on the questions that have been asked via the course because I didn’t record them this time. And what I found was actually the questions that I get asked is the content the book should focus on.

So that’s the first thing in terms of the writing process, create that course, run it, iterate the content on the courses. I’m journey, the content of the book. Actually document or record the questions I get asked by the students, because that’s telling me what I’m not articulating. The second part was the example Canvases in the book have been done by a whole lot of collaborators based on projects they’ve done.

And I hadn’t written the book, so I had nothing to give them, and they hadn’t been on the course. So what I did was I took them through the canvases, a quick half hour, here’s how you fill it out, and then let them go off on their own and do their own canvas. And that was intriguing because they did it in ways I didn’t expect.

But there were also. Patterns across every collaborator. So for example, in the canvas, there’s a vision statement, and the last part of the vision statement is what’s called unlike. So if we didn’t build this information product, if we didn’t deliver this data and information to the organization, what would happen?

What’s the alternative negative thing that would happen? And pretty much for every collaborator who worked on their own, the answer was manual process using Excel. 

Ramona: You are kidding me. No. 

Shane: Because every project they worked on, because they’re all practitioners in the data domain, that was what they were replacing.

Wow. And it made me think. We talk about the value of data and information, and we’ve talked about that a lot lately. We talk about the value of data teams. We talk about writing business cases and justification for this work to be done, but actually maybe we’re overthinking it because really the alternative is do you wanna run that process in Excel or do you not?

Because that’s actually the thing we’re replacing nine times outta 10. And that was intriguing for me. That spawned a whole lot of thinking about value of data in an organization. Are we replacing a process that’s inefficient and pe do people really care? So that was another kind of surprised learning for me as part of that process.

Ramona: So really, what do you do with that? So which answer did you settle on? You ask these questions, right? So are we over exaggerating things or are we over emphasizing on the value that we bring? How do you settle seeing this pattern emerging from your collaborators? And those are the people at the ground level, right?

Ground zero. They are doing this. How did you settle that pattern emerging? 

Shane: I haven’t answered that. It started me thinking, it started me thinking data teams are like a finance team. Data teams are like a finance team. Without the mandate, a finance team, a bunch of financial people in the financial part of the organization are a shared service.

And they’re mandated. You can’t typically run an organization without a chief financial officer, without people doing the accounts In the general ledge room producing profit and loss and balance sheet statements, they don’t articulate their value to the organization. They don’t go. Would you like to have a balance sheet in a profit and loss?

But for some reason, data people do. We’re a shared service. And then we have to go and justify our existence and that seems like an oxymoron to me. That’s intriguing. Maybe the answer is actually we do get embedded with the operational teams. We no longer become a centralized shared service. We become decentralized.

I don’t know. There’s lots of questions there, and when we do data team design book, the guide for, that’ll probably, I’ll have to answer them at that stage. That’s not the next book, by the way. That one’s a hard one. Yeah. Again, the good thing about it is that. As you write, you learn ’cause you are thinking, you are asking questions and then you are researching and exploring and collaborating.

And so just the process itself is invigorating. 

Ramona: This was a really perplexing pattern to emerge, right? I know we always joke that, oh, Excel will never die, and the Excel is all you need, right? So not attention. And Transformers, Excel is all you need and. To actually have this end result of all this research and the work that you and your collaborators have done.

You validated that, which we took as not empirical, but we as anecdotal information. So you actually proved that is a fact. It’s it in reality. It’s really fascinating. 

Shane: And these are. People that are professionals and typically consultants that get paid to do that work, an organization has paid them or their teams or their companies to replace a manual Excel spreadsheet process with something that’s quite costly because data teams are costly.

Intriguing. Yeah, 

Ramona: very. And so in this context, Shane, you just proved that data teams are a cost center essentially. So Yeah, it’s, it’s not the conclusion that you would have hoped to arrive at, but now we have AI and we need data teams to put the data in order so we can consume all that money to generate AI to fix something that we can do in manual Excel.

Is that what you’re saying? 

Shane: I’m not convinced. That ai, the Gen AI lms and the new way of working with those tools actually equates to us structuring our data more. I have a hypothesis that I haven’t got around to testing that actually will structure our data less, but that’s probably a different and very longer podcast.

Ramona: Okay, so maybe we’re looking at it from different angles. Let’s clarify this a little bit because. Structuring or cleaning the data. Those are D different aspects of it. So for any data to be relevant and to get patterns, good patterns out of it so we can learn something from it, it’ll have to be better than just plain garbage.

I’m not 

Shane: sure it does. 

Ramona: Please, uh, elaborate. 

Shane: So this is unstructured thinking. I haven’t gone through and done the work, the research, the experimentation to get the core patterns out. So I’m very early in my thought process, but the LMS have been trained on patent recognition of large text repositories.

That’s what they do. So why do we think that our data factories, our way of working where we put all these processes in to change the structure and the context and the content of data increase the quality is actually a better way of feeding those models than just giving it the chaos and asking it to find the pattern?

That’s my underlying hypothesis, and I dunno the answer to it. 

Ramona: As you said, this should be a podcast on its own, and Mr. Snarky, snarky came out to play. Oh yeah. I was wondering when he will come out to play. I 

Shane: think we have a whole new set of. Tools that have a whole new set of patterns and there’s a real risk that we are trying to apply legacy patterns that haven’t worked for us anyway into this new world.

And I just think that’s dangerous. So I don’t know, again, we’ll find out in the next year or two. ’cause as of course, things accelerating so fast, 

Ramona: we are talking about data in general and how it’s shaped and all of that. But in the context of a business that data is. Pretty structured or we wanna believe that it is right and the all the documentation, everything that we can think is digitized already or about to be digitized.

It is in some shape and form better than how we imagine everything that was scraped off the internet. It’s a different kind of data volume sites, whatnot, when you talk in the context of the business and the information that is pertain inside the business and the rest of it. So let me give you a.

Shane: Concrete example that keeps coming back to my head Again, I don’t come from the library services domain, and so whenever I hear ontology and taxonomy, I get a whole bunch of technical words that I don’t quite understand. That’s not true. I can’t understand them more now. ’cause I had one on one of the podcasts to explain to me what the hell are those things mean.

But they’re about putting a form of structure. Against some things, a classification to a degree. And my question is that a better pattern for using that type of data with Gen AI and LMS versus just chaos tagging. If I went in and just tagged the snot out of every piece of data that had any kind of value with any tag that it might be, and then put that chaos into the LLM, will I get a better result?

And that’s my question is that actually there’s more unstructured chaos to let it find the patterns. Is that better? For example, it’s a difference between a K means where I just stroll the data at it and say, tell me four clusters versus a sequel statement where I put a 50 and Nelson to say, here’s four clusters.

I’m just intrigued on the difference of those patterns right now, and I think we are applying the tools that we know to a problem that we haven’t proven. That’s my view at the moment. 

Ramona: It’s the entire non-deterministic nature of it, and there have been so many papers coming out and there are. Examples of one shot learning when it saw it once, and that is what it learn as opposed to other things that have seen it many times and it didn’t learn.

So that is an issue because you don’t know in the end what it’ll learn, what it’ll be, the end result among the structure, part of the equation. And this is just how my brain works, and that is what makes. Sense to me, but I understand, and I know there is order and structure in chaos. You just have to find it.

And that requires a different technique, different set of patterns. 

Shane: Yeah. And once Cicada challenged that versus, uh, non-deterministic model in a really interesting way, what he said was, because we know the machine is not deterministic, because we know we will ask it a question and get a different answer.

We don’t trust it. We expect the machine to be a hundred percent right. However, when we ask a human, when we ask a data analyst to analyze that data and give us the answer, we don’t expect a hundred percent accuracy. We don’t ask to see their workings. We don’t ask them to prove it. We trust that they know what to do with their job.

So there’s this economy between not trusting a machine but trusting a human. Whereas actually a human is probably far more non-deterministic than a machine to a degree. And an example today on the. Practical data modeling discord that I posted was, I saw somebody post something on LinkedIn that said the technical tools that a data scientist use is Excel and Tableau, like bollocks, the term data scientists should be the clue that the system study science involved.

And they actually listed, they listed the stat skills that a data scientist needed. So they said the skills you need is stats and they listed a whole lot of stats, capability and patterns you needed. And then they said the tools. Excellent tableau. And so I can look at that and I can go, that’s a disconnect.

That doesn’t ring true to me. I don’t believe that based on my experience, but how many other people would, and I think we seem to think that humans are infallible and we should trust ourselves, where actually we can’t yet we won’t trust the machine because we know it hallucinates. I’m intrigued again by that dichotomy.

Ramona: I don’t know who created that. And there were some other issues there. If you went to the right most column with the Gen AI engineer and then it, there were some other typos there, probably it was generated, who knows? But back to this idea of humans being more non-deterministic and the machines being non-deterministic.

So why do we trust humans more than machines? I’ve heard Cassie talking about this. She wrote several articles on this idea that it’s not that it’s more non-deterministic, but there are a lot more correct answers to go back to that. Which one do we choose better? Out of all these? Good answers, correct answers.

Which one is more correct? So they are all correct. So how do you choose, how do you make decisions? And especially from a leadership standpoint, what do you do with all of that? You have to make decisions. You have to entrust in the end in something. Then anchor and ground. What is the ground through, what do you do?

And it’s an interesting point with humans, we have such a large spectrum. You put two different people with the same title next to each other and you’ll need a translator in between them. 

Shane: We just gotta take the arguments we have about what is a semantic layer. Oh my god. Our, our semantic argument on semantics.

Ramona: So you were in that meeting? Yes, 

Shane: I think so. And then you got the what’s a data product argument, 

Ramona: and then in that meeting with the semantics. That. Okay. What exactly are we talking about? Yeah, from where I’m coming from the academic side and the research, this is what I think of semantics. What are you talking about?

And then it was complete silence and I didn’t know what to do with that silence. But 

Shane: because you can speak multiple languages, you are doing a form of semantic mapping to be able to take Romanian and make it French or English or Russian. Your brain has been wired to that mapping. And one of the things that I look back on my life and go, I wish I learned a language at five years old, a second language, because I think fundamentally that would wire your brain.

You know, maybe three. I don’t know what the age is for the brain wiring, but to learn a second language fluently at a really young age, I think would wire your brain completely different. And actually allow your ability to semantically map different patterns in the world to happen a lot faster than if you hadn’t learned multiple languages.

I’m intrigued by that one. 

Ramona: My daughter, she is so interested in languages, well languages, but, and more into symbols and alphabets and the scripts. So she had taught herself more than time scripts. She wanted to call herself a poly alphabet. So she is now a poly alphabet and she explains it scientifically.

So she did all this research prior to child GPT on her own, and she would tell me that sound and that phatic and this and that and wow. So it’s all very systematic. And she found the rules, patterns. I know I kept throwing patterns at you as an inside joke between us. Now I’m serious. She uncovered patterns and she could figure out connections between a pattern in some script and patterns in the other script and how they translate.

I’m realizing this in real time in my conversation with you right now. 

Shane: Oh, and she’s mamo, right? ’cause she’s doing it with both text and audio. So she’s got the ability to be a multimodal. 

Ramona: Yeah. And she really wants to stay away from ai. And Joe talks about his own kids and how no, they insist on not being influenced by AI and what is this doing and that.

And we on the other side, we embrace it and we see how much help we get and that’s another pattern merging. 

Shane: I think because we’ve had to do the hard work ourselves, therefore I think we can leverage the tool. I wonder if you grow up with knowing nothing other than Gen, AI and LLMs as your way of getting access to the patents in the world.

I. What’s gonna happen? The world will be a different place as that happens. I don’t know if it’s a waste or a bad place, but it’s gonna be a different place. 

Ramona: How do you learn the critical thinking? Do you need to be exposed to those things? And you can keep asking a machine and then you get the answer, and then you get the satisfaction?

Oh, I got the answer. That’s it. I’m done. How do you know that you. Can do more and actually how do you break down a problem? How do you develop that system thinking we cannot let it happen that way because then what are we training next generation of brainwashed kids? 

Shane: Having opened up about the fact that you’ve worked for universities, you think about the fundamental change they have to do now because again, I only spent six months there.

I spent very little time. But my view of those education facilities is they are running legacy education sessions. They are about giving you some knowledge and testing whether you got the knowledge, not giving you some tools and testing whether you are, can systematically apply those tools to solve a problem.

And so I think our education facilities have to change. 

Ramona: Absolutely. And. There are reasons for happening in the way that you describe and one of the reasons being how do you mark if you test them on critical thinking at all times? You need massive power of assistance to work to be able to mark those exams.

It was always a challenge. I’ll give you an example for the advanced database course. Where I was teaching them query optimization and all those techniques, so to test them, I would give them an SQL query and ask different query plans, different evaluations, and I would always provide them with the calculations.

I don’t care for them to memorize those things. And I just wanted to test how they apply that knowledge and how they go about it and do rewrites and whatnot. It, it’s always very challenging to mark those kind of questions because they’re open-ended. How do you compare this solution with that solution when they’re both good solutions and now we can tie it back to.

Non-deterministic. 

Shane: Well, maybe we talk it back to the pattern of education, which is based on a test that was graded to see whether you’d learn is actually a pattern that no longer survives. 

Ramona: Exactly. And 

Shane: what is the pattern? Is it actually they can tell you a story, they would solve a problem and that actually is how you grade them?

Or do you need to grade people anymore? Again, all the patterns that are foundational for the last wee while are probably about to change. And so maybe the answer is more people write good books that tell better stories, and that becomes the content that all the universities actually use rather than the textbooks of old, I don’t know.

Ramona: How do we teach them to learn to be better storytellers? Because that’s art. It’s an art form, and you need to practice. It’s these kids, they don’t read that many books anymore. They are creative in their own ways. We cannot expect them to behave in the way that we did because especially me growing up. A kid in communist Romania.

We’d get 10 minutes of cartoons Saturday at lunch, and we had to run back from school. I was going to school on Saturday two, so I would run back from school just to catch that 10 minutes of Bambi, and throughout half a year I would see the whole movie books where the salvation. They grow differently and they’re exposed to different things and they manifest that creativity in different ways, and it’s all amazing.

As long as it keeps manifesting and they bring that into the world, that is a good thing. I completely agree with you. It’s gonna be a big shift, and I think we are not the only ones talking about this, but we just get more vocal. About it. It’s something that is on many people’s minds. Yeah. They are the future, so we need to prepare them for that future and.

We are not sure exactly of that pattern of the future. So we have to prepare them for something that we guess how it’s gonna be. But who knows? 

Shane: One of the patterns I teach a lot of teams is skills, not roles. When we look at data teams, they often want roles and data modeling. My favorite one to be snarky, mix snarky on, right?

That’s the data modeler, but why can’t everybody model? They can’t be experts at it, but they can be proficient. So if we teach everybody data modeling skills and we no longer have a data modeling role, then the teams are better team. I think that’s what we’re gonna get to is that. The education system has to teach skills, not roles.

So actually your example of going in and having make a choice about going through arts or science, it’s gonna disappear. You’re gonna have some storytelling skills and some science system thinking skills, and that’s gonna allow you to craft yourself to fill one or many roles, and the ability to put those skills together.

As a set of patterns to solve certain complex problems is actually where I think we’ll end up. But who knows? We did it with our parents where we were like, we’re not gonna work the way you did. The world’s gonna change, and every generation’s gonna do that. Exciting time. 

Ramona: Yes. Exciting. I we’re not even thinking five years down the road.

Right now, we’re shortening that period of the forecasting. We just don’t know. But it is exciting to live and start from scratch and learn these new things just at the same time as the new grad. 

Shane: As I said in my LinkedIn post, when I announced that the book was finally live, I now know far more about Adobe InDesign bleeds and Amazon Kindle Direct Publishing bleed formats than I’ve ever wanted to know and ever really needed to know, probably in hindsight, one of the skills I should have given to somebody else who had those skills rather than trying to work it out myself.

Just close it out, I suppose. I had niche. I finally got round to fulfilling it, and I learnt so many new things in doing it because I had so many new problems to solve. That was intriguing for me. That was part of the journey. 

Ramona: So Shane, if you were to summarize in one sentence, what advice would you give yourself at the beginning of the process now that you are on the other side of it?

Shane: That’s a hard one. I’d naturally go back to, now I know some of the patterns. Don’t do them the wrong way, don’t do the antipas, but actually. That wouldn’t have been, it’s actually the journey of 

Ramona: that brought you to where? To the end of it. Yes. So you had to go through that process. It was a transformative process and the flywheel and everything.

So it would’ve been a different book if you didn’t go through those iterations, don’t you think? 

Shane: It might’ve been the same book, but actually it would be a different learning experience for me. Oh, okay. I got told right at the beginning, writing books are hard. Most people don’t finish. And now I understand why.

I got told at the beginning, don’t write a book ’cause you wanna make money. Because you won’t, and that’s true. Don’t write a book ’cause you wanna be famous because actually nobody’s gonna find it. Very rarely do you get the books that we all read and aspire to where it becomes effectively viral. There are so many books out there that probably like my one will get published and very few people will read it.

So if you’re gonna go and write a book, do it for yourself. Because that’s the one thing that’ll be sticky is that it’s something you want to do. And if you happen to make money out of it, great. If you happen to become that thought leader book that everybody reads and loves and you get massive positive feedback, great.

There are good benefits that you can achieve maybe. But if it’s not something you really want to do, then you’re not gonna finish it. 

Ramona: I love that. Write the book for yourself. I love that. ‘

Shane: cause once you’ve got a printed copy and it’s sitting on your desk and you pick it up in six months time, it should be something you’re happy with.

’cause it’s a lot of blood, sweat, and tears to get those bloody words on a page. 

Ramona: I wrote a book when I was very young and there are two copies, mine and the coauthor. And that is because the publishers screwed us over big time. 

Shane: I gotta say that I’ve self-published. That was hard, but I think it’s the right choice.

Hearing some of the horror stories about publishers, and again, it was one of the learnings I had was I somehow thought if you used a publisher, they did a whole lot of heavy lifting for you. But all the research I’ve done, it’s not true. They don’t go and sell it for you. You have to sell it yourself. Yes.

You get an editor if you get a good editor. There is massive value in that. I think that’s something that people have told me, but yeah, all the things you think a publisher does. They actually don’t. Do you still have to do it? I think there’s value in publishers. I think the publishers would’ve given me a forcing function.

They would’ve forced me to create the content at a certain cadence and that may have been valuable. But yeah, I’m quite glad I self-published. So we’ll see. 

Ramona: Once again, congrats on scratching that each learning so much in the process. And thank you for sharing with everyone what you’ve learned and the process.

I hope this will help. Everyone aspiring to write a book and to learn more about this? 

Shane: Oh yeah, definitely. If people are struggling to capture data requirements, then dual data guide to the Information product Canvas is the best book in the world right now for understanding how to capture data requirements.

Ramona: Excellent. 

Shane: There’s the last sales pitch. Thanks for taking over as the host. If people wanted to catch up with you and talk to you, where’s the best place for them to find you? 

Ramona: The best place and probably the only place is on LinkedIn. That’s where I spend the time. And yes, I’m on some Discords, but officially LinkedIn.

Shane: Excellent. I’ll put your , LinkedIn link in the show notes and thanks for coming on and taking the seat of Host and I hope everybody has a simply magical day. Thank you, Shane. This was a lot of fun. It was a lot of fun. Excellent. Catch you later.