Product Management with Scott Baldwin

In this enlightening episode, Murray Robinson and Shane Gibson host Scott Baldwin from Productboard for a deep dive into value-based product management.

Key points of episode include:

• The shift in focus: From ‘features shipped’ to ‘value created’, recognising that many built features often go unused.
• Harvesting customer feedback: Utilising every interaction with customers as an opportunity to identify problems that need solving.
• The ideas backlog: Analyzing proposed ideas based on their value to the customer, the organisation, and considering technical feasibility and cost before actual development.
• The role of a product manager: Comparable to an orchestra conductor, orchestrating different elements of the product development process.
• Outcome-based success metrics: Setting up key performance indicators based on the desired outcomes of the product.
• Early and frequent involvement of engineers and designers: Encouraging collaboration from the early stages of product development.
• Regular customer interactions: Engaging with real customers every week to ensure a user-centered approach in decision-making.

Tune in to gain valuable insights into effective product management.

Read along you will

 Shane: Welcome to the No-nonsense agile podcast. I’m Shane Gibson. 

Murray: And I’m Murray Robinson. 

Scott: And I’m Scott Baldwin 

Murray: hi Scott. Thanks for coming on. So wanna talk about product management with you today. Could you start by telling our listeners a bit about who you are and what your experience with product is?

Scott: Sure. I’m the community lead and product evangelist at Product board. prior to this role, I was a longtime product management leader and I’m a member of the product community here in Vancouver, Canada. I’ve got about 20 years experience building lots of different kinds of products across EdTech, FinTech media. And a few other sectors worked at companies such as Think Finning, Digital Central One, Credit Union, and a few others. And I’m now wearing that marketing hat as part of a product and marketing organization with a piece of software that I really love and got a chance to use for a number of years. 

Murray: Okay, so what is product board? 

Scott: So product board’s a product management platform that helps product teams understand what customers need, helps them prioritize what to build next, and helps them get everyone aligned around the roadmap and the plan. We serve about 6,000 plus customers globally and have a number of different organizations and companies utilizing the software really to help them build products, that matter to their customers and that are customer centric. And thinking first about the needs of customers and the work that they need to accomplish with your product and the success and factors that you need to contribute to your business. 

Murray: Ok, good So I’d like to start by asking you what you think product management is, cuz I think there’s quite a bit of confusion about this and different ideas.

Scott: Yeah, it’s an interesting. Yet loaded question in some ways. And I think one easy way sometimes to answer that is what product management isn’t. What a lot of people are practicing as product management today, it is to a large degree focused on delivery, versus actually thinking about the problems and needs of your customers and the value you can add to them.

Murray: Okay, so you’re talking about delivery of features and the development side of the product. 

Scott: Yeah. So this historical focus on maybe the output versus actually the outcomes of the work that we do. The product manager is really accountable for understanding is this actually valuable functionality for us to build for our customers. And Is this something that’s gonna deliver value for our business? And maybe what we’ve over indexed on is the output and the delivery of a thing at the expense of understanding whether this is actually something a customer wants. And actually whether this is actually gonna help our business. So we’re doing a really great job, maybe in lot of cases, shipping stuff, which is good. We’re checking boxes off there and look at all the stuff we’re shipping, look at all the cool things we’re putting in production, look at all the code and the new feature releases that we’ve written and all the excitement that we’ve generated on our social media platforms with announcements and things like that. But is it driving value for the customer and is it driving value for the business?

Murray: Yeah, I looked at some research by Pendo. So Pendo are a SAS platform for products and their research showed that only about 20 or 30% of product features are regularly used and the other 70% or so are not used. The Standish Group have been saying the same thing in their chaos report for many years. So given how much it costs to build product features, that’s a tremendous amount of waste.

Scott: Yeah. We’ve actually got some similar stats on that side. I think it’s upwards of 80, 80 or 90% of features aren’t delivering any real world value to the business or to the customers. 

Murray: How does that happen? 

Scott: I think it happens a few different ways. The first part of this is we’re not often fueled by understanding the actual core customer needs and problems. So as product managers we’re all somewhat guilty of not being fueled by real, customer learning and customer insights. 

 Ultimately we’ve got a job to do as product managers, as product people to really validate and understand whether our customers have these needs. Whether that need is actually big enough and important enough for us to actually solve in the context of the work that we’re doing. And we need to truly, understand that problem, not just asked a few questions and done a little bit of discovery. The second piece, I think that’s really critical has been the prioritization and the decision making about what to build. Product teams struggle with having a clear vision strategy, objectives and, an articulate framework for making those prioritization choices. They’re not really assessing the things that they’ve got in front of them and really coming to the hard choices about whether this is something worth making the investment in.

And then I think if we’re going to deliver value, we need to deliver that through an effective method of execution and getting the work delivered getting that feature to market. And a lot of teams are struggling there. If we can’t execute well then we create a lot of waste and a lot of inefficiency. 

Murray: I think there’s a lack of willingness to research things because people are sure that they’re right. 

Scott: The best product teams have clear systems for funneling a steady stream of user feedback to their product team. And the simple fact is your entire organization, your support team, your executive, your sales team, your go to market team, they’re gonna be learning continually through their interactions with customers about the needs and problems that they’re having with your solution or with the space in which they operate. One of the biggest challenges we see is product teams aren’t doing a really good job of centralizing that feedback. All that input showing up through those conversations and calls aren’t really ending up in a single place that can be actionable by the product team. And that’s really a core problem that we’ve actually solved with product board. It’s really identifying patterns and starting to see some of those core underlying problems. And then being able to use, those different inputs to help formulate a plan about where your product needs to go, the problems it needs to solve, and which problems that you could solve are actually the ones gonna have the most impact, both for your business and your customers, and the business you’re trying to get.

Murray: Yeah, so it’s a good point. As a product manager, you’re getting lots of input from lots of people talking to customers, and some of them are very senior and you’ve gotta have them on side . And yet everything they say to you is really just a hypothesis about what customers want. And I think it’s very common for product managers just to accept what they’ve been told and become order takers for senior managers. Do you see that? 

Scott: You definitely see that. show up, but if I’m like a senior manager or senior leader, and I’m bringing my opinion to the table, then it’s our job here as product managers to bring the evidence of what we’ve learned through the conversations and discussions we’re having. The evidence of what we’re hearing from the market, from our teams and from our customers. And utilizing that to have an honest conversation with those people about where it is we’re gonna invest the dollars. If you can start having some honest conversations, Armed with real information and that real understanding of user needs, you can start to shift that conversation a little bit in a different direction and cause organizations to make a different set of choices. 

Murray: So how do we go and test these ideas? How do we get that information about user needs? What’s the best way to do that? 

Scott: I don’t think we test it any way different than we test any other hypothesis. We have an idea about something we think we could do. We’ve got some meaningful input and measures around that. We go off and we do some discovery in the problem space to validate that this is really truly an actual valid problem worth us solving. And then we spend some time, with that set of people that have given us that feedback, and we start exploring potential solutions. And that needs to be rather iterative. We refer to that as the double diamond. There’s an opportunity here to iterate through that solution space to land on the right features that are gonna solve for those customer needs that you’ve been hearing about through that problem exploration.

Murray: I see two methods of product development being discussed. One is a traditional consumer goods marketing approach to product development. You have some ideas internally. You engage a market research agency to research it. You write a product brief, which is quite long. You start designing your product. You engage a technical project manager, they engage an engineering team. You build it and 18 months later you put it out into the market with a multimillion dollar marketing campaign. And then the other approach I see from the Agile community, is the lens startup type of approach, which is based around we don’t really know, so we gotta do a lot of experimentation and we wanna do that early and often and continuously. So where do you sit in that spectrum of approaches? 

Scott: It’s definitely more in the build measure, learn quick iteration type approach rather than the Big Bang approach. Most of my product experience is definitely been there, but in the companies where I’ve worked on that larger big bang approach, most of those things we delivered while they were well marketed and well trumpeted weren’t really delivering end user value and in the end really didn’t get utilized in the software. We can all pat ourselves on the back but in the end we haven’t delivered business value and we haven’t delivered end user value. And back to Kagan’s thing of the valuable and viable, that’s the core job here for product manager. If we’re not, achieving that outcome, then did you need a product manager to do that? 

Murray: Yeah. So how do you define your outcomes for your product? And how do you keep those outcome measures important and involved all the way through? 

Scott: A big part of this is having clear strategy. A clear product strategy sets a direction that keeps a product team from being really reactive and putting out fires continually and just shipping a bunch of things to instead constantly asking themselves , is this actually something that’s gonna solve a particular needs of a specific kind of user or market segment that we’re actually trying to service? Is it actually gonna help us drive growth, competitive differentiation regulatory compliance, and is it gonna really help us drive end user value? 

For our own company objectives and key results have been a really effective tool in helping to set that direction around the outcomes that we’re hoping to meet. And keeping our teams aligned on those outcomes. And so it’s a clear, measurable, inspiring goal with a specific outcome that you’re trying to achieve for your customer, for your product or your business. And so everyone’s gonna describe that in different ways, but they’re definitely gonna be qualitative And time bound. And they’re gonna energize your team towards, its mission and its goal, 

Murray: shane, do you have any of these success measures for your product?

Shane: So we’ve got a bunch of outcomes and they are based on the bets we are making. So we’re still not at product to market fit. So we still don’t know who our true customer is. So we have , a bunch of bets and then we’ve looked at which of those bets are most likely to get us to the outcomes that we want. And then we’re trying to figure out how we measure those bets. And the key thing with a bet for us is when do we stop betting? Cause that’s hard. You get bought into it, right? You start believing that bet has value cause you spent money on it. And that’s always the hard part to go. Yeah, that one didn’t work. And then back off and do something else. 

Murray: Scott, can you give some examples of success measures that you’ve seen work well for product teams? 

Scott: Yeah what would we care about, might depend a little bit, on the team that we’re based on? so if I was in a growth based product team my goal here is to drive self-serve, and get people to use the product on their own. I’m gonna look at things like people that are registering to use the product, activation rate within the product, seven day retention that we’ve got around the product, and overall conversion rate. If I’m in a core product team, I’m gonna probably look at things like adoption of our product, adoption of our features engagement with our platform, like how they’re using it, how often they’re coming back, how they’re working with it. I’m probably gonna be looking at churn, overall satisfaction with the product. Those would be some common ones I think I could probably do. We often utilize what we call a north star on this side. Northstar can be more a broader lens, a way to zoom out and really understand whether your product’s actually generally hitting in the right direction and going the right place. And could be quite polarizing and strong for a product team and helping them see whether the product’s actually succeeding or failing. 

Murray: Can you give an example?

Scott: Yeah, Amplitude’s North Star, was the weekly number of users who run at least one query and amplitude. So they called it the wq Weekly querying users. That’s essentially the pulse for them around their overall use of the product. That’s the one metric that really, truly mattered that the entire business could look at as a pulse on whether it was actually contributing to the right decisions and the right directions, and the right resourcing decisions and the right trade offs and product bets essentially, that they were making. 

Murray: You’d have to be quite careful about choosing the right one

Scott: yeah. exactly. You can incentivize the wrong behaviors and end up building towards the wrong actions. 

Murray: All right. How do you integrate research designers, engineers, product people, so that you can get fast learning on a continuous basis.

Scott: Yeah, I think one of the biggest opportunities we have as product people is to bring them in early. I love methods like the design sprints and other iterative ways of working early on to form a clear sense of understanding across the product team. And bringing in to that product team. That triad of design, product and engineering, And a range of other voices and perspectives whether it’s a data analyst who can share some really great data inputs. Or a CS team member or a support team member. They can give us a different lens on the customer problems and the needs that we’re solving on that side. This is really the critical action that a lot of product teams miss. And it really changes the dynamic. 

Murray: So how do you do that? Do you have all those people in the same team? Do you have a user researcher, a user designer, the product manager, the engineers, and a customer service person all sitting in the same team virtually, perhaps these days physically? 

Scott: Yeah. You do. Those teams are multi-functional. They’re crossing all those boundaries and they’re bringing each of those people to the table as part of that journey. One of the biggest misses that product teams make is they don’t involve their engineers in product discovery. They don’t take the moment to actually let them go out and hear and talk to customers. And I’ve had the pleasure of bringing engineers into that product discovery process, that understanding process, and sometimes the output of that has actually been a much faster time to market, a much clearer solution path than we would’ve thought about as a product team if that engineer had not been there. 

Murray: Yeah. Skilled and experienced engineers, know different ways of solving a problem. There’s easy ways, there’s hard ways, and very often engineers get given something extremely specific to do that’s been designed by somebody else.

Scott: Yeah. They get an order, 

Murray: They get an order and they have to fulfill it,

Scott: that’s right. 

Murray: and yet if you went to them with a problem, they might be able to say why are you trying to solve it that way? This way would be much easier. Why don’t I just use this API and plug this in? 

Scott: I’ve always said engineers are magicians. I’ve worked some really great ones and you bring them in at that stage, help them really understand what the core underlying issue is. They’ve been able to pull out of that really magical ways in which to solve it. And in a lot of cases that dramatically cut down the time and effort. And so we need to be able to give them the room both in the problem space to be informed about the problem we’re trying to solve. And we also need to give them room in the solution space to be able to do their best work. And that comes through a participatory process that involves the stakeholders that need to be part of that, and a team that works closely together and interacts with one another to form that shared understanding of both the problem and the solution and how best to solve those customer needs. 

Murray: Yeah. Most agile teams these days will have a product owner. What do you think is the difference between a product manager and a product owner?

Scott: Oh man. Really the product manager is responsible for prioritizing what to build next. And so they discovering what the user’s needs are and unveiling some of those kind of critical insights. They’re aligning a team around a products vision and strategy, the roadmap helping to make that decision about what features are gonna be built next and what functions we’re gonna deliver. 

The product owner’s responsible for maximizing the value of the product resulting from the work of the development team. And so the product owner in a lot of cases is responsible for that backlog grooming. And they turn those customer pains that we’ve discovered into actionable user stories, prioritize stories, they arrange them in the product backlog. They work with the agile team through those agile processes, those scrum meetings and all that kind of stuff. They keep the development stuff on track. They communicate some of that voice of customer that we’re learning and some of the feedback that we’re getting. But they’re really different roles. 

Now, the interesting thing of this in the product space is we often confuse the two. So some product owners are really product managers doing a lot of product owner work with a little bit of product manager work. We have a lot of product managers that are doing product owner stuff that call themselves product managers, but they’re really not. 

If I’m in a small startup where I’ve got one, maybe two product managers, their chances are they’re probably doing all of this. Because that’s just the nature of how the team is structured. Probably have five developers and one PM and I’ve gotta get all this work done and somebody’s gotta do it. A lot of bigger teams and bigger organizations have the bandwidth to have things like product owner roles. The product manager would never have the time to get the user stories done cuz they’re so busy on trying to understand the problem space and work with customers. Just the domain is so large, right?

So, these to me are complimentary roles. They’ve got a lot of benefits to each other. But not every team has them. We don’t have product owners at product board, we have product managers and we do not have product owners, but that product owner role is sometimes shared by other team members.

Murray: So some people describe the product manager as the CEO of the product. And others described them as being more like an orchestra conductor. What do you think. 

Scott: I do see it a lot as a conductor. It’s a bit of an orchestration job. It is our job to help put shape around a lot of things that are really messy across different groups to ensure that all of those pieces are coming together. But I think for a long time, yeah, we’ve had this historical thing of us being the CEO of the product and that hasn’t really helped us. It’s probably hindered us.

Murray: So you’ve been doing this for 20 years. What are the big things you’ve learned while you’ve been doing this? What were you wrong about that you’ve changed your mind about? 

Scott: Yeah, one that really stands out maybe from a recent example is that we sometimes, believe that we can grab a framework, a way of working a methodology and apply it on top of our business. And somehow that will solve all that ails us. 

Sometimes what people do is they grab this model, like the Spotify model of how teams are gonna be constructed and they apply it on top of their organization and they believe that’s gonna solve all that nails them. And so I think what I’ve learned is that isn’t actually the case.

What you need to do is find out what actually works best for your organization and find your right path that’s gonna get you to the right value. In the end, it come down to those core functional pieces of what we call product excellence. We really truly understand what the customers need, that we’ve got an effective way to get aligned around our vision, our goal, our strategy, and our plans, and that we can effectively execute on the work that we need to do. That can be done lots of different ways by lots of different teams and they might have different lenses on how they view that. There’s not like a one size perfect fits all on top of that. And I think a lot of teams unfortunately buy into this is the way we’re gonna do it. It worked over here at this other place and therefore it’s gonna work for us. And the outcome of that in a lot of cases isn’t really super. 

Murray: Yeah, I would call that experimenting with your process in the same way that you experiment with your product features or your marketing and tailoring to your situation.

Scott: I think we sometimes anchor in a process so strongly that the process takes over the way we think about getting work done and gets in the way of, it doesn’t actually help a team be efficient.

Murray: Shane, let’s go to summaries. What do you got? 

Shane: Yeah. So one of the things is that there are a large number of features that are built but not used. And often we focus on the number of features shipped, not the value created by them. We also have a real reluctance to remove the feature cuz we’ve invested in it, we spent money on it may not have value. In fact, it may decrease the value because it makes the product more complex and nobody’s using it. But we still want that little menu option there because we invested in it. 

I like this idea of harvesting from everybody that engages with customers what they’re learning, how do we harvest all those customer interactions from people that aren’t in the product team and bring it back. Where are the problems that need to be solved? How do we understand what those problems are, And observing problems that customers have is a great way of figuring out what the highest priority is. Versus, just thinking without that context.. 

Scott: Yeah, understanding what customers need comes from listening. How we actually get to the conclusion what’s most valuable to work on is more of a prioritization decision, and that to me is largely a mix of a few different factors. One is the problem big enough? Is this actually a painful enough problem for us to solve? And is our product good enough to solve this today? And if not, then we shouldn’t be doing it. If yes, then how can we dive into that a little bit further and start to get clarity on that problem, that need and everything else informed by those insights and those learnings. 

Shane: So just because a customer says they want it or need it, doesn’t mean it’s actually a good idea to invest in it. But then the alternative of that one is we never actually talk to any customer. We never really listen to them. We might do some very light research at the beginning to self justify our prioritization that we’ve already made in our head. So this idea of just using it as another input, there’s a valuable set of conversations and data that an organization has outside of the product team that they can leverage.

And then the role of the product manager really is to bring that evidence of what they’ve learned from the markets, from the customers, from their research, and use that to inform and prioritize what is the next most valuable thing? 

So you listed off a really great set of metrics for a growth team versus metrics for a core product team. And we often just like to read somebody else’s key metrics and say, Oh, that fits us without really understanding why. So matching those key result areas, at least around the type of organization you are and the state that you are at or stage your organization is as important.

 The fact that there is a massive overlap between UX research, product management and product owners. So we just gotta be clear when, if we have those three roles in an organization where the work that they are doing and the work they’re not doing, and how we have conversations or handoffs between them.

 I like, the idea of a product manager is that conductor. They’re bringing all the pieces together and then making sure the next most valuable thing is done. 

And that there are a set of product patents. And what you do is you build your own way of working with the patents that are valuable to you. So you find a patent, you apply it experiment with it. If it has value, keep doing it that way. If it doesn’t trash it out, right? Read the next book. But there is no methodology. We see the term agile coach a lot in the agile community. I don’t often see the term product coach but I see lots of product training. And not as much agile training. But you were saying there is term Yeah. That product coach is a role out there. 

Scott: Yeah, it’s definitely a big roll out there. There’s definitely lots of practitioners in that space. I would say it takes two angles predominantly. One is a focus actually on the leader themselves. So back to be able to line your organization strategically and tell the story of where you’re going. I think there’s a lot of effort on the coaching side being put into help product teams mature in their capabilities. And then there’s a lot of time being spent at the team level helping the team develop bottoms up into more successful product people.

Shane: All right, Excellent. Murray, what have you got?

Murray: Yeah, I was thinking a lot about prioritization as you were talking because as we were saying, you get a lot of ideas coming in from all over the place, and a lot of them come from those hippos, high influence people. And yet the research tells us that 80% of the things that we build are just not being used. So there’s this massive gap going on there as an enormous source of inefficiency. And I think the answer is, as you said, to collect those ideas, put them into a type of a backlog, then research them work out whether customers do want them or not, and whether it’s valuable to your organization, whether it’s feasible.

So there’s balance. I talk about in design thinking, you’re looking for the balance between what’s valuable, what’s desirable, and what’s feasible. The feasibility side means you’re gonna have a lot of discussions with your engineers, so you wanna bring them into, to sue that. Prioritization as well. And sometimes it means doing proof of concepts and getting your designers showing things to people, seeing what I think.

 So I think that there’s a ideas backlog for product managers, which is a combination of features and it should also include marketing as well. I don’t see why we treat marketing any different. There’s also considerable uncertainty about what marketing works and what doesn’t. And then there’s a build backlog.

Once you’ve prioritized things, you’ve done some research, then you can prioritize them for development. Cuz that costs a lot, obviously to develop things. What often happens in engineering teams is that. You only ever see the build backlog and the ideas backlog is all very vague and often not properly managed or done at all. I see a lot of product managers who are just order takers for senior stakeholders, really.

Scott: Yep. 

Murray: So we shouldn’t do that. We should say things like, that’s an interesting idea. How can we validate that? How can we research that? How can we find out what customers think about that before we invest heavily in building it? And I think it’d be really good for a lot more product than just to have those discussions. That’s not confrontational. That’s very sensible. How do we know that this particular idea that the CEO had is going to provide a good return on investment? And then I like the model of the product manager as orchestra conductor.

 Yeah so prioritization critical theme for me coming outta this. And in order to do that, you have to know what your objectives are. You have to know what your measures are of success, and you’ve gotta be doing some, serious talking to customers. On a regular basis. I think every week you should be talking to some real customers about ideas and feedback, and then looking at data as well. Once you’ve got a product in market, there’s tons of data about what people are doing. And there’s also tools where you can watch people using your product or service and see where they get confused. 

Scott: Yeah, I think it’s also about segmentation. Who is the actual customer you’re trying to go after and what are the attributes and factors , that make up that customer and do you really, truly understand and can you slice and dice on that? Are you delivering something that’s for a certain geography or a certain industry or for certain type of customer or makeup of customer or sub 

Murray: but also you need to keep in mind, Scott, that you could be wrong about that. Like the customer that you think is the one for your product 

Scott: yeah. 

Murray: not be. It may, it’s somebody else.

Scott: And they may not be the buyer potentially as 

Murray: Yeah. 

Scott: Yeah.

Murray: So all these things are up in the air. We’ve gotta try them out. Same as processes. So that’s basically what I got out of it. Now, how can people find out more about your ideas? Do you write, Do you blog?

Scott: I, I think probably the best place where we can find out some of the ideas is to visit our blog@productboard.com. I have some writings up there, but I think also a lot of the materials that we produce, like the eBooks and resources and stuff like that, that are on product, I have my hand in those, I’m involved in the co-creation of those. And so a lot of the perspectives of how I think about product are being reflected in those materials and resources as well 

Murray: Okay, great. All right, thank you very much for coming on. 

Scott: No problem. Thanks for having me.

Exit: That was the No Nonsense Agile podcast from Murray Robinson and Shane Gibson. If you’d like help with Agile contact Murray Evolve. That’s evolve with zero. Thanks for listening.