Join Murray Robinson and Shane Gibson in a conversation with Chris Butler about agile Product Management at Microsoft, Facebook and Google. Three core things, that product managers should do. Wrangle uncertainty, help stakeholders make good decisions and build alignment through communication with roadmaps, strategy documents, priorities and specifications. The difference between a Product Manager, Product Owner and Project Manager. Dealing with uncertainty. Map your assumptions and test them with qualitative research. Develop a strategy. Determine value. Decide what to do. Power dynamics. Hippos. Product Operations. Lean UX and Continuous Discovery. How to work with engineering teams. Product Management Patterns. Copying vs learning. Product Canvases.

Read along you will

Shane: Welcome to the no nonsense agile podcast. I’m Shane Gibson. 

Murray: And I Murray Robinson. 

Chris: And I’m Chris Butler. 

Murray: Hi, Chris. Thanks for coming on. We wanna talk to you about product management, what it is, how to be good product manager and so on. Why don’t you kick us off by telling us about your background and experience in product management. 

Chris: Yeah, that sounds good. I mean, I’ve been doing product management for an awful long time. Probably 20 something years at this point. Places that you would’ve heard of, that I’ve worked at. Been Microsoft ways, kayak, Facebook, reality labs, cognizant, and now Google and at Google, I’m the lead product manager for the core machine learning group. And so our group specifically builds all the tools, services platforms that both Google internally, that includes researchers or other engineers as well as third party people using open source. So that could be TensorFlow, Caris the TPU chips really anything from software to hardware when it comes to machine learning.

Murray: So what is product management? 

Chris: I think an awful lot about the right type of job for a product manager to do. And I’ve been dabbling in this realm that we would call product operations, which I would summarize it as PMing, the PM experience. I would say that there’s really three core things, that product managers end up doing. And so one of them is they tend to wrangle the uncertainty of the world. And I would say this is not necessarily taming it or reducing but being able to dance with the bear. How do we understand what we should be doing, what we should be reducing uncertainty around, what we should be embracing uncertainty around.

The second thing is really how do we help facilitate better decisions for the team? And so it may look product managers make a lot of decisions, but the reality is we’re putting time stops on it, or requiring a particular way to make a decision based on certain stakeholders, things that.

And then finally, really building alignment through communication. And so the things that this ends up looking like inside of teams are things like roadmaps, maybe a strategy document, maybe helping groom the backlog a little bit. Writing specifications or PRDs writing the equivalent of a one pager or we’re experimenting with things the PR F AQ. So there’s a lot of these artifacts, that product managers end up building, which end up being canonical documentation, usually customer centric, canonical documentation. And so that’s what I’d say product managers tend to do. 

We’re not really the executors of the organization. So we work very closely with designers and engineers really to make sure that they’re keeping one, a customer centric mindset, but then also to helping them weigh trade offs within their roles. So engineers could build something that is perfect, but it takes way too long to get to market and may not actually meet the needs of the customer in the end. And so talking about that type of trade off is really important. And I would say that’s the key thing that we end up doing with other kind of groups is talking about these trade off. 

Shane: If I go back to those three things, that you were talking about, they sound like a product owner from scrum. I help the team make trade off decisions and time box and figure out when’s good is good enough. And I communicate with the organization that’s a product owner from scrum. Isn’t it?

Chris: Yeah. I think the problem with the product owner title and this was something I saw a lot at cognizant is that we would hire a lot of people that were a titled as product owners. But what they’d end up being is actually just fancy card shuffler. And they would not be given access to the strategic stuff. The reason why this product owner exists is that they’re supposed to have a really close relationship with the end customer. They’re supposed to know the end goal and the strategy of all these types of things. But what I’ve found in practice is that product owner ends up being as part of the scrum team, more of an organizer rather than someone that actually has that deep kind of understanding.

 I’ve heard, scrum practitioners say that the PO should actually not be in the team most of the time. They should be out talking to customers is what the end goal was supposed to be, but they end up sitting there and babysitting the team. That’s why I tend to shy away from wanting to give people the title of PO. I saw someone who is titled a user story writer in a client project. And it was it’s ridiculous. so BAS end up assuming that everything is figured out, that all we need to do is write the right spec or write the right list of user stories and the right acceptance criteria and we’re done. Project managers I see as being really about risk reduction. So known risks. Product managers should be much more focused on the uncertainty. And so it’s not one or the other. And I was a program manager at Microsoft. I actually had to do both jobs because I would still do the product management job, but then I had to have a Gantt chart in Ms. Projects. And so I think this doesn’t work because you don’t have the right tension between these two different roles. And I actually like tension a lot. Mark Abraham wrote a really interesting book about product management is tensions. And so to me, this tension between project and product manager is really important, cuz we want to reduce known risks. We wanna screw up the same way over and over again.

But if we only did that, we would never do anything. And so the product manager is the one who’s pushing for that new stuff and trying to embrace new things and try to embrace those trade offs. So I think the same thing ends up happening with engineers and designers and QA and management and compliance and legal and privacy. So I think that tension is really important.

Murray: How do you know what customers want? 

Chris: So I believe a lot in qualitative research. When I was at a company called philosophy, that was a design consultancy. I had a lot of hands on ability to not only do very generative user research, asking people about the problem that they’re dealing with, having me walk through it, job to be done, style interviewing all the way through to rapid prototyping and usability studies. And so I gained a, really big respect for that qualitative aspect of it, because I can ask follow up questions. I can understand something that’s more core about that. I shy away from surveys. I feel they don’t give you the full story. They, can work in a pinch for certain aspects and there’s definitely people that are professionals there. And then I’d finally say behavioral information. This is where data comes into the job. Being data centric means that you’re trying to understand the behavior of what people are doing, but you then also need to bridge. If we think about what. Product manager doing it’s not induction or deduction. It’s actually abduction, which is that we have some evidence and we have to make a leap, an imaginative leap of faith to something that we think is actually going on. And so I’d say that the qualitative stuff ends up helping a lot with that. I actually taught the spring semester at the art center college of design I X, D three, which is a prototyping and user research class.

And so I think this idea of putting something together could be a beta product. It could be even something fake but then putting it in front of someone to get their reaction, I think is the way that you end up learning an awful lot. Just asking people questions would you do this or do that. Those are bad ways to ask questions. So I, think qualitative view of research is incredibly important. 

Shane: So, I’ve been working through UX research, lead practices and the whole MVP, lean startup, build something and then get some feedback. You sometimes pick an approach that is UX research led. You don’t know what the problem you’re gonna solve is. So you go out and you jobs to be done. You look at what out there you find a problem. You need to solve. You ideate around how you might solve it. You do some design and then you go and prove whether that design works. And then the opposite of that is you think, you know what the problem is. So you’re gonna build an MVP that solves that problem, go out and use quantitative information to test whether you’ve solved it. So What I’m thinking is that you often wanna use both depending on the fidelity, where you are in your organizational state, where you are in your product maturity state, so that sometimes you want to do that research UX first, to find the problem. And sometimes you want to use that MVP And you just gotta be really clear which one you are using and why. Is that how you see it? That they’re both valuable, but it’s just right. Tool for right time.

Chris: Yeah, I think you’re right. Companies that have a lot of uncertainty about what they’re trying to do should rely more on this generative research that usually is spearheaded by user research. We have an amazing team at Google within the KMEL team. And we have a researcher That focuses on APIs. So these people have a really deep understanding of the way people use things. I think, it’s valuable to do that. What I’ve generally done as well is David J bland and his technique around assumption mapping, I think is a really interesting way to look at are we still trying to figure out assumptions about the problem space and the customers themselves, or do we have a lot of assumptions about the solution that we think is good?

And so you can actually do both. In some of these 45 minute sessions I used to do back at philosophy. Very early on, it was very generative. It was about the problem because we didn’t have enough information there. As we started to move through the process, we would have prototypes, but we’d still double check with people, whether we were solving a problem that made sense to them.

The Nielsen, Norman group has a really good blog post about the number of people you should be talking to. We were in a very small group doing very rapid research. And so we talked to like three to five people, but I think when we get to the scale of something like Google, you’re talking to tens of people to be able to validate one thing. So I think it’s all about collecting evidence. And it’s trying to understand, what is this question actually about? Is it about the customer? Is it about their problems? Is it about the way that a solution could situate there? Or is it the solution? And one thing that I think is interesting is that we still have opinions about what we should be doing. I’ve really loved this concept of discursive design or provocative prototyping, which is you put something that you think is purposely bad in front of someone to find out what matters. And so maybe that’s in between this generative and usability testing.

Shane: So that concept of assumption mapping, I haven’t heard that before. Can you give us a minute or two about what it is, how it works, why we’d do it?

Chris: So David J bland wrote a book about experimentation. And he created this technique, which essentially is you get a cross-functional team together; the engineers, the designers, the product, people, user researchers even like legal and compliance. And you start to ask usability, feasibility, desirability value questions.

And you start just writing down, what are all of the assumptions that you have right now based on these different questions. And then you map it on basically a two by two where the vertical is usually , how important is this? And then the horizontal is , how much evidence do we have about this?

And so the thing that you wanna look at is what’s the most important stuff that we have little evidence about. And so we should go and we should go do generative user research around that type of thing. If there are things that are very important and we know an awful lot about, we should just go do it.

 What I’ve found is , whenever I’ve worked with a lot of user researchers in doing this type of technique, they actually know an awful lot about the customers. So I feel like product managers, we do not do a good enough job, involving them all the time because whenever I’ve done it, where it’s just a product manager, designer, an engineer, there’s a lot of stuff in the middle to unknown space. Whereas when I bring in UXR, everything shifts to the left towards we have a lot of evidence around this type of thing. It’s a really amazing technique. And I think it’s a really good kickoff activity for an organization that is maybe getting started.

You’re gonna kick off a new project. It really helps level set what it is that we all understand about the project and how do we now move forward and what we should be working on or experimenting with, or even researching. So I love that technique.

Murray: I see modern product companies using tools like user voice to ask customers what features they want to give feedback, ask for feature requests, there’s discussion on them, there’s voting on them. So the idea is that your customers will tell you what you need to build. And so you should just listen to your customers and do what they want, if enough of them want it. What do you think about that idea?

Chris: I feel like you just cut out the product manager from this whole process, we just pipe that into JIRA and then engineers start building these things and we have a perfect system, basically. There’s no more hard questions. 

Murray: But seriously though, won’t customers tell you what you need to do for them? 

Chris: No I don’t think so. And the reason why I say that is I’ve usually thought of the product manager as being, the embodiment of the customer understanding, but then bridging that to how do we align that with the strategy of the business and the type of thing that we’re trying to achieve. Customers are experts in their own problems, and especially when you work with people that are real experts within a field, they know exactly where the problems are. And that’s great. That’s what you do want to know. You want, know where the problems are, but for them to be able to imagine a different world than what they have today is very hard for them.

Job, to be done, talks about this as cobbling together of solutions. They’re going to pull together the things that will work for them in that time. But they have a very hard time, imagining something different. And this is why when we say you are not the user, the moment you start thinking about these things, you’re no longer that person in the realm of just the problems.

And so I don’t think it’s good for us to take direct stuff from customers and pipe it directly into stories or to, tasks or work items. There needs to be some type of filtering and transformation in some way. We need to do a better job in part of what product management is really trying to help with is how do we do a better job of prioritizing things based on what is the most important thing for us to do. Customers may want one thing, but it could be that they’re actually seeing a problem that could be fixed by actually just getting rid of everything. I think systems thinking service design, they think about this concept of that, okay, we have to fix this one report. That’s handed off between these two people. And so we fix that report but it still takes really long to finish this whole process. And so if we just looked at all of the different stages in this process, we would see that this report was not necessary if we plugged these two things together.

And so it’s that type of thinking, that idea of creativity within the product management field, I think is a lot about framing and reframing in some way. Being able to understand at the strategic level, the roadmap level and the backlog level, those things need to be in alignment.

I believe that strategy is not about some future thing. I wrote a medium post about how strategy is now, but I think when we need to do a better job of prioritization, we need to use all those things as being in alignment in some way. And that is part of the product manager’s job is to make sure that all those things in alignment, and even if a customer wants something even if it’s your biggest customer, but the strategy, you actually have says that will not be a thing that will be valuable.

You should still not do it. And this is where a lot of startups get burnt very early on. They build a bunch of things for really important customers, but they can’t get out of that. They end up becoming professional services organizations and the reality is maintenance of these things, that you end up building for these important customers is 10 X, what you’re actually charging them probably. And you’re not charging them for maintenance. You’re not doing all these things. The product manager is there to filter things and reinterpret them in a way that is in line with the strategy. And ideally that’s where the imagination or creativity comes in is reimagining and reframing the problem in a way that is even more helpful than just say, fixing that one problem for that customer. 

Murray: So is what you are doing, trying to find a balance between what’s desirable. To the customer what’s technically feasible and what’s valuable to the business. 

Chris: I hate that, Venn diagram. Trying to simplify what you do down to a ven diagram feels like a bad idea. So the way I’ve drawn it, or the way I’ve tried to get people to think about it, is that product managers have to work with a lot of different types of stakeholders and yes. Technical business or management and design, which is desirability in my opinion. Those are people that we need to work with. But in today’s environment it’s much more than that too. It’s customer engagement. It’s legal compliance. When I was at Facebook, rowdy labs, there were a bunch of stakeholders I had to work with security safety, integrity. These are all groups that we need to be doing a better job of working with. And so I wouldn’t just limit to those three is what I would say.

Product management is about how do we help the experts in all these different fields come together to make a decision. And this is something I’m working on a lot with our team. How do we decide how to decide? Because Google and Facebook are both very consensus driven organizations, but a place like Philosophy, we were much more kind of consent driven which is the idea of balanced teams and decision making authority.

And so there’s some interesting challenges, especially in large organizations, we talk about how to decide. But that’s why I think it’s more than just those three and it’s more about , how do we pull in or actually move out the people that are no longer necessary for a decision and who should actually make this decision who should give feedback and how do we say thank you for all the feedback, but I’m making this decision. If it is the product manager’s decision. If something’s just not maintainable in the code or there’s no future path or it’s just a lot of technical debt that is the engineer’s job to say actually this is not a good thing for us to do. We’re not gonna build it this way.

We can build it a different way. We can build a different thing. And that’s the trade off discussion, same thing with design and then same thing with legal or privacy is that this is really bad for these customers. And so this is not acceptable. But it’s about those trade offs that you need to have a discussion. 

Murray: I’ve found in most organizations I’ve worked in that the decisions about, what to do with the product are very internally driven by senior managers by the highest paid person’s opinion.

Chris: yeah, and actually that’s something that’s happened to me a lot is I’ve gotten a lot of emails that are , Hey, my friend was using this product, from an executive that does not work in my org at all And, gets forwarded to my executive. And they’re , Hey, this is not working.

Can you please look into this? And it’s you want, my hourly is are you sure you want me to be taking time away from everything else we’re doing? But yes I totally get what you’re saying about that. This is where I think leaders, especially within agile teams, they don’t seem to get sometimes what their actual work output is.

And so a leadership team is there to provide vision strategy acceptable or not acceptable behaviors through incentive programs hiring people, those are the types of things that leadership teams should be doing. But I actually hope that my VP and my VP’s manager have zero access to the backlog or task list for our individual product teams.

I don’t want them making those decisions because that is not what they’re paid to do. They’re paid to make really hard strategic decisions, easy for everybody else in the organization. And then the teams that are closest to the customer, those are the people that are actually supposed to be making these minute decisions about, should we do this or that, as a work item. I’ve done a lot around OKRs, for example. But whatever goal setting methodology you want to use, there’s usually two sides to a goal, there’s what is this end state we’re trying to get to for someone in the world, ideally, that is aspirational that helps direct the team that provides a trade off of some type.

And then how do we know that we’re getting there and how do we know we got there? And so in OKRs, that’s the objective versus the K. And I would say there’s third piece, which is the initiatives. And I’ve found an awful lot of the time. People are constantly mistaking initiatives for the goal.

when that’s maybe this outcome over output issue that we wanna talk about. I think that’s a big problem. And so I don’t want leaders to tell actually the team, what are the key results. The leader in the team should decide on what is this objective that is really worthwhile going that kind of networks in to the other OKRs that are there not a hierarchy, but a network.

And how do we branch that to the strategy? How do we use this goal setting as a lens, to look at our roadmap, and then it’s up to the team based on their agency and autonomy to say, what are the key results, and if the leader disagrees with what those key results are, there’s a clear disconnect between what the team thinks it can do and what the actual leader thinks that team can do.

And so I think that’s an important discussion. So I almost think of goal setting as an API, there has to be this way to query the API or push something into the API. But if you end up allowing these leaders down into that lower levels of the APIs they’ll screw everything up in my opinion.

Shane: Yeah. I agree with you. I think about it as a service, and there’s this complete disconnect. So if a senior leader was buying a service, a supplier. They either have a, well describe service they’re buying. So they have sure about what it is, or if there’s any uncertainty, they have a set of outcomes or measures. They go if you do this, then we’ll pay you that amount of money. And it is normally a clear, because it’s an external party and money’s changing hands. There’s a one form of behavior. When we talk about internal teams, for some reason, we don’t use that same behavior, we don’t go we’re paying a stomp load of money for your time. Let’s have an agreement where you, I’m very clear on what I want for that money and that time. And you are very clear what you need to do to deliver it. And then that’s effectively what I’m buying, but we don’t do that. We go, I just got a team and I’ll make them happy and then they’ll do some work and then potentially the works, what I wanted. So why is there that disconnect, 

Chris: I think part of it is power dynamics within teams, you’re you’ve made it up to this point. And so you deserve the decision making authority. There was a really great study back in the eighties in a management science journal that was about something called dominant logic. And so the dominant logic theory is this concept that people that are in very senior leadership positions they’ve gotten there because of past successes. And so what happens is they bring all the mental models and the way they do their work to new environments, but there’s a mismatch. And so the person who is super successful, rising up the ranks, but then in that particular context that are now at right, you only get promoted to the level of your incompetence. The problem is that the dominant logic dictate dictates that these people. We’ll try to continue to do the things they did. And so the big problem, especially in technology is that generally practitioners get forced into management tracks a lot of the time. That’s why in a lot of very large organizations Microsoft Google, Facebook, there’s very senior IC engineers I don’t think this happens enough for product managers though, honestly. But I think that is part of the problem.

Shane: Yeah an interesting lens on it. So do you see that there are different product management practices, different ways of working different patterns that are successful. And that it’s almost like being an agile coach and you say there are things more than scrum. There are some other patterns we can apply, which of those agile mindset pattern. Can be applicable in here. Is that the same product management where people come in and go this is the way you do product management. Versus I have a product management toolkit and a bunch of skills, and let’s work through which one may fit for the context of the organization.

Chris: Yeah. I think that’s one way I can tell how experienced a product manager is. If they come in and say that we’re just gonna use this methodology and it’s gonna work a hundred percent time, they’re just wrong. There’s gonna be something that it’s gonna screw up. We just don’t know how we don’t know that yet. So I would just say that it is a lot like coaching, because what you’re trying to understand is what is the bottleneck or friction that we’re dealing with, and it’s not gonna be me just telling them what to do to fix it. But it’s gonna be, how does this work in their context in some way? And so we definitely apply a huge number of tool sets processes, frameworks, techniques, et cetera. I think an expert product manager has a huge number of these to be able to, apply. 

Murray: You’ve talked about product operations a couple of times, what is product operations? 

Chris: So product operations is a newer thing I would say in the product management world. But operations is just, making sure that what is business as usual is a well oiled machine. In say a consultancy, it would be assigning people to roles inside of projects. It would be making sure they’re rolling off correctly, getting paid a bunch of these things that are overlapping with finance. A lot of the time once DevOps came out was this idea that. Engineers throwing stuff over the wall to operations. Wasn’t a good thing. And so how do we have a better kind of handoff and transition to do a better job of smoothing out some of these rough edges or gaps?

And so we then had things like research ops, design ops, where research ops is all about, how do we collect all this stuff and create good reports and make sure that recruiting is super easy or design op is how do we do a better job of creating design systems and providing the capabilities for That inside the world of product management, product operations is applying all the things that you would do as a product manager. But doing that on the product management role itself. And so I think an awful lot about some of the key artifacts, are documents their meetings. Their communication. And I think about all these things for the people that I work with. And some of it looks like coaching senior leaders My peers within my org are directors of product management that may have multiple layers of management under them.

But it’s about helping them figure out what should their all hands meeting look like? Is the all hands meeting right now that are doing actually valuable to anybody. If it’s just them talking or it’s just about people waiting to speak and give their status update, that’s bad. And so even mixing things up with something as simple as a lean coffee approach, or just even asking people what do you hope to get out of this meeting?

What do you hoping to convey to other people? What’s the context you wanna get from them? That’s really, my job is to ask those types of questions. And so a lot of what I end up doing is thinking about process. But I would say I’m definitely on the agile manifesto side of people over process.

So my job is thinking about all the interactions between individuals and Emily Tate basically wrote a really great book about communities of practice. And this concept of communities of practice is something that a lot of people have been thinking about. But I think about my product as a product operations person is really the product manager community of.

 And then it’s also the individual experience of a product manager. And that’s what I care about. If I think about my customers, it’s product manager, it’s partners to product managers, it’s my manager. And then it’s customers way down below, super far away from me. I think product operations though, I’ve seen anti patterns and I’ve actually done a course with the product led Alliance about product operations.

People should check that up. They’re interested and I’ve written a lot of articles with the product led Alliance about product operations. But I think when people mistake the end customer as their customer, as a product ops person, you end up doing the bullshit work. The bullshit work is what you do not want product operations, people doing, they’re not doing the stuff that product managers don’t feel like doing.

And I would say the same thing, product managers should not be doing the bullshit work that engineers don’t wanna do. There been times I’ve seen where the engineers. just don’t feel like updating my JIRA cards. Can you do that for me? And I always wanna step in and say, no, we will not be doing that for you. That is your job. As part of this team is to update your JIRA cards. And even back in the day at Microsoft, it was joking that I was there as a program manager to order pizza and get sodas for the engineers when we were working late. And it was a joke. I still did it probably once every couple months.

Because we have this amorphous kind of role. That exists in the world. And so that’s interesting. A great book I love referencing is Fontan the uncertainty mindset. I used to work at restaurant back of house and I had a restaurant tech startup. But it’s all about super highend dining, R and D and the way that they form their teams. And I think there’s a lot of really interesting analogies to the way product managers work and the way we wrangle uncertainty. But the thing is, that you may have a set of skills. You have a background around product management, but when you join a team, you actually have to actively negotiate with the other people on that team to figure out , where you fit?

Is the problem that there’s just no, specifications, is it something as simple as that? Or is it that there’s not actually a good discussion happening where we’re being more customer centric? There’s no one here as on behalf of the customer pushing engineers in that direction. And so it’s always amorphous and you always have to figure this out as you actually join a team in some way. And so I, I think that’s a very interesting part of product management shoot. 

Murray: Yeah. So in agile, we are talking a lot about working out what’s valuable and prioritizing features by value to build and so on. How do you determine what is valuable to build, not just to customers, but to the organization as.

Chris: Yeah. This is an incredibly hard question and the reason why is that there is no one way to judge value and it’s highly specific to the type of product you’re making the customers. You have. There’s a lot about business model in there. I do think that this is what strategy is supposed to help you do.

And so when I say strategy in this case we probably should declare bankruptcy on the terms, vision, mission, strategy, principles, values, all those things we should probably just say what do we actually mean by these things? We’ll start talking about that. But what I mean about strategy is that there is some end goal that you’re trying to go after, or in the case of Richard, roels good strategy, bad strategies strategy kernel. There’s a challenge. There’s a way you believe you can solve that challenge. And then you link it to day. And what I would argue is that value is what you’re trying to create in some way through your strategy.

And so I guess value is an end state, it’s maybe more of a vision than anything. But I would say that what is helpful in strategy is a couple things. So one is leaders should be making harder decisions. They should be saying what we’re gonna do and what we’re not gonna do.

And that should then turn into, depending on the type of environment or landscape that you’re within. If it’s a very Rocky landscape or, what conne would talk about as complex or, Simon Worley who just had on recently would talk about the pioneer model, 

 All those spaces We probably want to give people rules that they can apply to their day to day, but then we trust them to make the right types of decisions. And so I would say these trade offs are at the core of that. And then the thing you need to do is you need to now think about what are signals that those trade offs are actually working.

And so I think of that as metrics that are helping you understand whether you’re doing a good job or a bad job. And so that’s how I would LA us up to value in some way, but I don’t think it’s a simple question. 

Shane: So its interesting I just recorded a agile data podcast this morning and it was with somebody. Who’s worked in the ML space and the financial services space for ages. And one of the things that came up was this idea of blast radius. So the idea went that if you have an ML model, that’s doing a recommendation of your shopping cart, right next this product. If their model’s slightly inaccurate, the blast radius says, okay, they’re not gonna buy it. If that model is actually driving your trading behavior it’s actually doing financial trades on the market. You can lose a hundred million, with a bad model. Do you ever see that idea of the blast radius of those product features or work to be done being used as a way of doing the value and the prioritization. 

Chris: Yeah, I’m most familiar with last radius through chaos engineering. think what we’re talking about though is what I would refer to as opportunity cost. And this is one of those things that we need to think about, we could do anything, and we should do certain things because they’re in line with the strategy. And so I think the question then becomes , is there an opportunity cost to us actually taking this on right now versus something else that’s more important. And I’ve seen a lot of teams really struggle with this because they try to do this thing that is very urgent right now, but they are not getting to the thing that is actually going to get them to a better place, longer term.

And that’s why, I really dislike effort versus value or impact types of two by twos. And the reason why is that upper corner that is lots of effort and lots of value or impact people are like, we can’t do that. That’s too pie in the sky when the reality is you should break this down, you should validate those things. You should experiment with them. You should see how you can do something. That’s slightly better than that. And so I really hate that type of prioritization. I also get triggered whenever I read a prioritization article that has Moscow, not a prioritization framework, it’s a tagging framework.

And it’s basically up to whatever the hippo says. Conno, which is something I have never seen a team actually use in real life to actually use, to do prioritization of delightful versus baseline stuff. And then rice, which is basically again, just licking your finger and putting it in the air to see which direction everybody and everybody games the system.

So I don’t mean to be , go down this rabbit hole about prioritization methodologies, but we have to do a better job of understanding the difference between opportunity costs and value and how that connects to decision making today towards the strategic goals that we need to go to. 

Shane: Oh, I definitely agree. I tend to like Prioritization frameworks outta gamified. Outta their job community, one of the ones is 500. You give them 500 fake dollars and they basically be on the one they want done most. And the thing with the most value, the most money on it, that’s the thing gets done next because the decision makers in the room have collaborated. And it’s short, it’s quick. We know what to work on next. It gets our goal of what’s next. 

Chris: Yeah. I think those buy a feature types of systems are really interesting, but I’ve seen gamification even in that idea of buying a feature or.voting. But, there has to be someone that has to make a final decision about what is the actual stack ranking. And so whatever you want to do to turn that crank in the machine, to pop out a list of prioritize things and then having the right person that is a decision maker, then change that priority based on their intuition, their gut feeling, their abductive thinking capability, all that type of stuff I think is really important.

Unfortunately it ends up being the hippo most of the time. But that’s, I think to your point, we’re always trying to build these things to de-bias, but the reality is actually being in groups de-bias is an awful lot when power dynamics are not an issue. So that’s the little catch I would say 

Shane: Yeah. And they’re just techniques to say what’s next. So as long as we are not, baking into 90 day fixed scope plans for delivery. As long as we’ve got little iteration points and we can get constant feedback and change, it’s all. Okay. Yeah, it’s the same as when you say there’s seven things we could do and you say to the stakeholder, what’s the most important and they go sevens. So my approach is cool. I’ll let the team pick the one that’s the most fun. Cause you’re telling me you don’t care and sure. As shit, as soon as the team picks one, the stakeholder comes back and goes, oh, that’s not the most important. And so the answer is okay, so you did care, which one is it? So again, it’s just about what’s next. Let’s get it done. Let’s get some feedback. Let’s change our mind and let’s do the next thing. 

Chris: Exactly right. 

Murray: Yeah. I wanna talk about uncertainty and decision making because everything you’ve been talking about so far there’s seems to me that there’s a lot of uncertainty in it. There’s a lot of uncertainty in what people really would buy. There’s a lot of uncertainty in what’s really gonna be profitable for the business. There’s a lot of uncertainty about, how long things are gonna take to build and so on. And nearly, always what happens is that, we build a consensus within the organization and then we go ahead and we do that, but that consensus may not be reflective of what’s really going on. At all. We’re just kidding ourselves. Yeah. We’ve come to an agreement about what we’re going to do. Good. Let’s go. That’s usually enough for a product manager or any sort of manager but may not be what, people out there in the world are really going to want. So how do we deal with that uncertainty?

Chris: Yeah. I don’t think that actually consensus driven decision making most of the time is a good thing. So I would argue that we don’t want to build humongous things that take a lot of time that are not having a lot of ways to check with the market and do experimentation and understand. So I think that is something that’s very important. But I think maybe we should disentangle alignment and consensus. Because I think alignment is one of those things that is incredibly costly, but valuable in certain cases versus this idea of consensus is that everybody agrees. The Ready consulting group, have a really cool app called the decider. And it basically outlines eight different ways to do decisions. And there’s a lot of different ways I actually have done whole talks about just randomness in decision making.

And so I think that’s a very valuable or interesting thing to think about. But I would just say consensus is one, essentially that we all have to agree that this is the right thing. But there may be other ones where all that we actually want is we just wanna make sure no one Vitos this.

And that could be very different than the idea of people that are engaged actually make the decision or that there’s only one decision maker and we decide who that is. And so those are all possibilities. What we need to do usually is actually decide how to decide. And that’s where I think the alignment comes in is that we can all align to do something and move forward.

Amazon calls this disagree and commit. I would call it consent over consensus. And it doesn’t always have to be. Everybody is yes, I want, I, 100% agree with this. It could be that, the engineer disagrees with the prioritization of which customers are most important. It could be that the PM disagrees with the engineer and the architecture of the way this is gonna be built. And the engineer could disagree with the designer about the way that this interface will work. They could all do this, but it doesn’t mean that they should make that decision and they should be able to stop progress in some way, even if they decide that. We do get a benefit when talking to different people there’s been some interesting research I’ve been diving into recently about all the cognitive biases that we have, so the thinking fast and slow Canman type of discussion, there was one particular exercise they did where based on a logical fallacy, only 20% of individuals would get it.

But the moment you put people into a group of four or five people, 80% of the groups get it right. And so there’s something really valuable about the way we work in teams. And so that idea of tension within teams is incredibly important to foster. And I think team to apologies also tries to talk about this a little bit. But the idea of trying to create the right types of tension within teams through experts in different domains is really important. And so that means that not all the time will this group be in a hundred percent agreement. And actually I’ve argued to people. To get strategic alignment. You actually don’t even need 50% of the people to agree to the strategy. You just, you need to get people that trust the leaders and move forward. That’s the thing you need 

Murray: I agree that there’s a lot of value in getting, people with different views together and, respectfully discussing the pros and cons and then coming up with a decision based on that. But what I wanted to ask you about was how do you test your decisions? Are you a supporter of the lean uX type of approach or lean startup approach where we, break everything down into small things and we test them constantly.

Chris: I definitely agree with that. And I know Josh Satan and I love a lot of the stuff that he does and I definitely agree that learning faster is important. The issue becomes how do we learn things that are very large. So I think that’s the part that I tend to struggle with. There’s a lot of intuition that goes into the way that product managers work.

 When we were at Microsoft, we would talk about how for the first two years of a program manager’s life, they weren’t really worth that much. And part of the reason why is because of. This tacit knowledge or intuition or gut feeling is built up over time through experiences. And I’ve actually been diving really deep into this other realm of naturalistic decision making, which is how the military fire people, all these people end up making decisions in split seconds.

But actually are using very deep knowledge in some way. And then way you actually transfer this easier to people. And so I’ve been doing some things around decision forcing cases and how we train people. But I think part of the product manager’s job is to use intuition. And that’s the abduct of thinking a little bit, but I think allowing for the gut feeling to happen on smaller slices, this is probably the right way. So why don’t we go out and build a prototype and go out and talk to a bunch of people that we think would be the right users of this in some way, or right customers and ask them really unbiased questions, see if this actually works or not. I am actually a big fan of lean UX and four steps to the epiphany by Steve blank. I think is getting outta the building is what you should be doing. So that is the last way that you end up breaking down these biases. The way I’ve thought about bias, ladder is individuals, your team talking to the customers. And then I think of kind of the universe as the last place. And that’s where I think randomness actually comes in is provoking you with kind of things that maybe you wouldn’t have thought of. And so I think there’s some interesting, randomization techniques to get at that. Those are the ways that I end up trying to de-bias or make decisions safer, but there’s still gonna be times where there’s core leader that believes something and they could be wrong. But they’re driven by what they believe. And I think that’s valuable to also have, I just, hopefully they’re not as much of an asshole as say Steve jobs was right. 

Murray: I wanted to ask you how a product manager should work with engineering. So I see two models out there. One is a traditional model where the product manager does strategy and research and marketing, and then engages a technical project manager to manage the engineers. And they write, a 80 page product spec and they get involved in super detailed discussions about. Features, but basically it’s at a distance. Do you know what I mean? And then there’s the agile model, which is that the product manager’s intimately involved in the engineering team and making, trade offs almost on a daily basis. What’s your view on how you should work with engineering? 

Chris: Since I’m a PM, I’m gonna say it depends. But what the thing that’s interesting about kind of the distance, I think actually comes down to ratios of product managers to engineers more than anything. Because if you can actually have a ratio of say one product manager to one designer to one researcher to one engineer that’s awesome.

I think you’ll get an awful lot of benefit from that type of thing. But not a lot of teams work that way. And I would say usually the teams that work that way tend to be startups. They tend to be consulting project groups, things like that. I found that at very large organizations, sometimes the ratio of product manager to engineer will be one to thirty.

And these are for highly technical things. We’re talking about compilers for machine learning models that have to interface with TPU chips, it’s a very complicated thing. It’s highly technical. There still is a customer, and the customer may be just a few hundred people in the world that write these compiler extensions, but there’s still a customer.

And so there still needs to be trade offs. There’s still a strategy. And so I think it’s more of a function of how capable, is the org of actually supporting these types of models more than anything. I do think that for new projects, having something that is a much tighter group is more valuable because it allows you to make decisions much faster.

And maybe one tweak I would make about that model. You were talking about where the person writes a really long spec. And by the way I did write a spec when I was at Hotmail. We were doing calendar integration with MSN calendar exchange and outlook on pop. And we also had to worry about Lotus notes, I think.

And I had to write a specification that was about 550 pages long because it was trying to map three different RFCs to all of our internal systems. So I definitely know what you’re talking about with regards to a long specification. I would never do it again. I don’t wanna do that to anybody, but I think. That distance is in some way required because of the fact that the PM is not going to be able to do as much one-on-one in those particular high ratio cases. I would say though, it’s not necessarily TPMS that are managing the engineers, the TPMS that are there, at least in our teams to make sure we don’t make common mistakes.

We don’t do anything that is known risk. We reduce the known risks of things. It’s actually the technical leads or engineering management that end up managing the engineers, end up helping them. And honestly, Keaton Shaw, who is a product manager that talks a bit about product management, also startups. He had said will product management be around in 10 years? Or is it just something that , gets broken apart and put into different organizations, an engineering manager, a TPM, a designer and things like that. And of course for job security, I would say, no, that’s a bad idea.

But I would also argue that I think there’s an interesting tension. That is helpful by having someone else that is just focused on uncertainty and decision making is helpful. When I was at Facebook reality labs, I barely ever saw the tasks that were actually being worked on. I inherited a list of 100 wishes. So I was the person that was politely declining all of these hundreds wishes to people all through the company. But it was very rare that it would actually be brought in to think about the tasks themselves, what was more important was , is this the right type of thing for us to do for the customer? And so those discussions I would have but a lot of them were perfectly fine for engineers to make the decision about.

Shane: So one of the things we see come back and forward and back and forth again, is this idea of hyper specialization. We have the fan companies, facebook, alphabet, apple and yeah, they have scale and they do some cool shit and we want to copy them. And so what we do is we look at the way they work and because they’re scaling so much, they tend to have hyper specialization because that’s one way of managing scale. And then we see smaller organizations that can’t scale start to copy the hyper specialization behavior because they think that’s the answer. Is that what you see that we often take patents from an organization that has context, and we try and imply them to ours when we don’t have the same context and therefore our chance of that patent working is low.

Chris: Yeah, I think that does happen a lot. Is this idea of the perfect pattern is in this organization and we’re just gonna adopt it wholesale? I think the specialization thing is really interesting because I guess I have two minds of this, I think that product management is a fairly generalizable set of skills and experiences that I’ve actually found that by working in different industries, working on different types of products, even working in a consultancy I gained exposure to so many different, patterns that I think I can come in for a lot of things, even in industries that I don’t know. And I can talk about problems that are probably there. I think that product management is a very generalizable skill. And I’ve railed against this idea of product managers being hyper technical. I don’t think they should be. 

That’s fine. If you understand the terminology and you can have a good discussion about trade offs, that’s one thing. But this idea of expecting people to be X engineers, I don’t think is a good thing. I think that you end up building a different type of group think and then, specialization within the product management world.

I’ve seen some people break apart. A lot of things platform, product management this idea of product marketing manager, which is about go to market but is different from marketing. I’m struggling and working with someone on a set of content just around AI and machine learning, product manager. And what are the differences there, I still think you have to be customer centric. You’re still trying to go after an outcome. I think the question becomes, how do you end up talking about trade offs with the different groups? I don’t think that there’s actually specialization for product managers. And I actually think that highly experienced in a particular industry may overly bias them towards certain kind of reframing. 

Murray: Do you have any product canvases that you use for planning? 

Chris: I use all types of campuses. I created one personally five or six years ago, focused a lot on strategy because I just found that people were struggling in creating a framing around their strategy. A lot of the strategy documents that I’d read just say, we’re gonna be the best in class in this industry.

And it doesn’t mean anything it’s just bullshit. I think the thing that I ended up focusing on was really an adoption of the Richard Roel good strategy, bad strategy, kernel which includes a challenge, a guiding policy, or what is the thing you believe about the world and the way to address this challenge.

And then he just talks about connecting it to day to day. And so I created a canvas that, that tries to do a better job of understanding the idea of investments or best and then an actual linkage to day to day initiatives. But I think what we’re missing here is historical context for the organization, because that is really meaningful when it comes to strategic discussions.

And then this idea of , how do we actually judge metrics of success? So my friend, Adam Thomas is doing a lot of stuff right now around something called survival metrics. For me, this idea of survival comes from John Boyd, who, is a military strategist. From the us he was an air force pilot and then became helped with the Marines doctrine. I’m a big fan of his stuff. But the thing that he came out with is his observe orient decide act loop, which is how do you make decisions and understand the world faster than your adversary. 

He said strategy was actually. Really the ability to continue on in some way that allowed you to have freedom to make choice. And so it’s almost like optionality. And so to me, I think inside of this canvas, and I think what Adam talks a lot about with survival metrics is not so much what is the perfect thing that we hope we will, gain like we wanna have 20% year over year growth, or we wanna make a million dollars or whatever that is, that doesn’t matter.

What actually matters is what are the things that we know around the way our business works, the way that our customers are, the way that our monetary position is that keeps us alive to continue making decisions. And so that survival level is actually way more important to me from a strategic standpoint, because what you then have to talk about. And, Vontan had some great points in this post about something he calls Boris, which is a goal setting workshop he has is really, it’s not just about trade offs that you’re willing to make, but what are you willing to sacrifice to actually get the thing done that you’re doing. And so that’s where I think this all ties into this idea of there’s a base level of survivability that you need to do as an organization, as a startup, it’s probably burn rate, and some type of attachment to the way you make money or raise more money. And so that could be either. We need to make money now, or we need to hype so we can raise another round. Or inside of organizations politically, what do you have to do to continue to look like you’re being successful or learning something.

Anyways, the canvas tries to tie this strategic context together. It’s a little bit old at this point. Every product manager of a certain tenure has to have a canvas. It’s a Rite of passage, I think a lot of the time. But I think there’s something really valuable from that thinking of what are you gonna give up to actually go after what you think is most important and doing that across your leadership team, and then finding a core set of trade offs, I think is really core to this because strategy is never the decision between a good and a bad option, 

You should just take the good option. That’s not a strategy, but this idea of here are two things that are perfectly valid that we could do, that, that are even in our industry. It’s what customers expect from us. It’s something that could both make us money. But what leadership is supposed to do is they’re supposed to say actually based on where we want to go, eventually what our vision is, what our mission is about the way we do our work, what our values are. We should go after this thing, not this thing. Even though they both are good options.

Shane: Yeah, I like the idea of BES. I’ve struggled with vision, strategy, roadmap, blueprints, all that crap for years. So looking forward into the future where the uncertainty is, we’re betting that if we do this, we will survive. We think this is the way we should go. And if we get it wrong we failed fast and we learned that was the key.

Murray: I feel we should go to summaries. Shane, what do you. 

Shane: Excellent look. One of my tests for the podcast is how many other tabs I open up for things I gotta read. And how many of them relate to patterns? And I think I can adopt I got lots of tabs open, so that means it was a good F for me. 

So we started out and you said there’s three things that product manager, does they help rank uncertainty. They help facilitate better decisions for the team. Or help the team make better decisions. And they build alignment through communication and they help weigh those trade offs . As I said, I still think that’s a definition of product owner from scrum, but I take your point that we’ve devalued the term product owner, just like we devalue the term of business analyst.

We talk about research and UX led research. And we talk about MVP and quantitative proving of a hypothesis. And what I got outta this was that actually they’re both acceptable patterns. It’s around that level of uncertainty that drives which approach you should use when and I’m really keen to read about that assumption mapping, cause it’s a patent that I haven’t used before. So I think that’s gonna help me understand, the level of uncertainty, the level of assumptions around that uncertainty and which approach to use. 

I like that you talked about, as a customer I’m an expert in what I do. And I typically have a solution in mind of how I’m gonna fix a problem, but it’s very hard for them to step back and imagine a different way to solve that problem. We had, jobs to be done conversation a while ago on the podcast, and it’s this idea of making coffee, I don’t want a faster kettle I wanted an espresso machine where I could just put a pot in and that kind of thing.

So that ability for a product leader to step back and look at that is interesting. And so I don’t , I don’t like the term manager anymore product manager, project manager. I, they don’t manage anything. I like the term product leader, but then we started talking about this idea of a toolkit.

And so I can see a lot of similarity between agile coaches who have a toolkit and product coaches who have a toolkit. And they’ve built the toolkit over time. So you’re either a product leader where you’re leading the development of a product, or you’re a product coach where you have a toolkit and you’re helping people understand their way of working.

 I’m glad that you struggled with a can approach. I’ve tried it a few times and it’s been an epic failure. And so it’s not one for me. 

And then the other thing that you talked about, there’s a difference between alignment and agreement. We can have people align, but not agree or we can ideally try and get them to agree, more than one person in the room, that’s always hard. And so again the underlying theme for me that came outta this was this idea of uncertainty, as a product leader or a product coach, we’re trying to reduce the uncertainty and we have ways of helping people reduce their uncertainty.

And then once we’ve reduced the uncertainty, we can help them make decisions on tradeoffs or make the decision ourselves depending what our role is. Yeah, for me, it was great. It was reinforced lots of things that we’ve had on the podcast before, but given me a bunch of other patents to go and look at and see how I can apply those. Murray, what do you got

Murray: It, depends. That was the thing I. heard most often. 

Chris: I tried to say it as few times as possible. I feel I just jumped into the, it depends part without saying it, but yes, I. basically gave you nonstop answers of it. Depends. 

Murray: We talked a lot about what is a product manager? And I think from what you are saying it’s a very holistic view of the product. There’s so many things to think about. There’s the customer, the stakeholders, the engineers, the marketing. And a lot of what you seem to be doing is helping people make good decisions. Because you work in consensus organizations and I think we, or collaborative organizations, let’s say, I think we are very much supportive of collaborative organizations, but how do you help people make decisions effectively? Partially it’s about getting diverse voices together and having discussions and bringing in a lot of ideas.

And you said that’s a way you can go from 20% to 80% success in decisions. And I think also a big part of it is by testing your assumptions. So you talked about identifying assumptions so that we can test them. I agree with you that there’s a lot of uncertainty in product and product management, a lot of consensus building and a lot of trial and error, and maybe what’s uniting it all is having empathy for your customer and what their problem is and what they’re trying to achieve. And from that, you can build up a mental model of what’s gonna work and what isn’t.

Chris: Yeah I totally agree. I think the key thing about being a product manager is you’re gonna be wrong so many different ways at so many different times. We spend a lot of time in school learning about certainty through facts and math and all this type of stuff.

When the reality is the way the world actually works is there may be causation, but we have no idea how to actually understand. Things are constantly changing. And so I think just the assumption that we’re always gonna be right. Is something that, I try to push back on within my teams. There’s the terminology around product excellence, and what product excellence means is that you’re going to execute in a perfect way, and that’s gonna give you great outcomes when the reality is you could execute as perfectly as possible and you still lose.

And so I don’t know think it’s about a much longer game than that. I think, even just outside of your own company the idea of you as an individual, constantly figuring things out. And if you look at my background, it’s very novelty driven. For me it’s a lot about kind of figuring out new things, all the time.

And so that I think is really important to me. And I think a lot of product managers end up having that type of curiosity. And maybe just to add one more point about uncertainty, I also wanna make it very clear that I don’t think we can actually reduce uncertainty. I think what we can do is we can make decisions about what uncertainty we think is probable or possible, and we can make decisions about what uncertainty we’re going to address or not.

But part of the product manager’s job is actually to hold off the team from making a decision in some cases, and I think real options is a really amazing book and a story about that idea of, making a decision right now versus holding off on making a decision is still a decision.

And sometimes it makes more sense to do something in the future than it does to make a decision right now. And so when’s the last responsible moment to make this decision. And another book I just always recommend to product managers is, how to measure anything, but this idea of the way that information over time becomes more certain, but becomes less valuable is really at the crux of what product managers need to balance all the time. And so I think those are some really important points. I think about uncertainty. 

Murray: Yeah, that’s great. All right. So where can people find out more , about what you think

Chris: Absolutely. So on LinkedIn, I’m always happy to connect with anybody. I’m not the Chris Butler that is a hockey player or a parliamentary figure, or I think there’s a new musician and maybe a producer I’m, I think I’m the top result for Chris Butler in technology. And then I’m on Twitter under sbot with a Z that’s where I tweet my musings. I’ve done a lot of talks. I tend to attend a lot of conferences and I’ve written on medium as well, so I’m happy to provide links to all that stuff, but again, I’m just excited to engage with people. I always wanna know, have you tried this and if you tried it, did it work or not for you? And that to me is super interesting. So I’d love to connect with anybody about that. 

Murray: All right. Thanks very much for coming on. We really appreciate it.

Chris: Now thank you for all the great questions. It was really great to be here. Thank you.

That was the no nonsense agile podcast from Murray Robinson and Shane Gibson. If you’d like help with agile product development, contact At evolve.co that’s evolved with the Thanks for listening.