Join Murray Robinson and Shane Gibson in a conversation with Christian Crumlish about the intersection between UX Design and Product Management. The risk of designing the wrong product is high. Customer research reduces that risk and uncertainty early. Product Development is a process of continuously learning, building and adapting. You can start anywhere in the cycle if you keep looping through it. Good products come from cross-functional product teams that are focused on measurable outcomes. Be outcome-focused rather than output-focused. One team rather than silos. Empowered teams rather than bureaucracies. Tailor your design process for your situation rather than religiously following a framework. Let the problem lead you to the solution. Product Managers facilitate constant communication between stakeholders, customers, designers and engineers. Retrospectives are a powerful way to improve.

Read along you will

Murray: This is the No Nonsense Agile Podcast. Each week we have in-depth conversations with experts in agile product development and technology leadership. Our guests share their insights and recommendations on how you can build great teams that produce high value digital products and services. To the podcast on Podbean, Spotify, Apple Podcast, or your favorite podcast app. 

Shane: Welcome to the No Nonsense Agile Podcast. I’m Shane Gibson. 

Murray: And I’m Murray Robinson.

Christian: And I’m Christian Crumlish.

Murray: Hey, Christian thanks for coming on.

Christian: Thanks for having me. 

Murray: We wanted to talk today about user experience design and product management. Why don’t you kick us off by telling us about your background and experience?

Christian: Sure. I came up in the UX profession, along with the profession itself, in, in the sense that I was doing the work before even the term UX had become the standard term for what you do. So I had titles like information architect and content strategist early on during the.com period. 

Worked at companies like Yahoo, where I was the last curator of the Yahoo Design Pattern Library. And then later on at AOL where we were trying to do a turnaround effort, where I spent some time as the director of product on AOL Instant Messenger, which is now I think in the museums. And so I started out that journey as a UX person, but I came out the other end in product leadership roles. I went to startups and ran product in UX together for a while and then tried to separate them out again. I was the head of product at a Mental Health Startup called Seven Cups for about four years. And we got that company to be profitable. We broke even, which was a nice milestone in the life of any startup. And most recently I was product managing the Covid website for the state of California. And I’ve ended up taking an appointment from the governor of the state to lead the product practice for the office of data and innovation for the state of California right now. And along the way, I wrote this book because I’ve been on this kind of career path that took me from UX to product and I’ve had a lot of other people on the way ask me about how to do that or whether to do that.

UX people trying to understand where they fit in a product world or what product means, or product management, what it is. And over time, I started hosting a little community to scale the conversation a bit and just realized that there’s at least a niche book on that topic. My consulting has revolved around helping teams improve, the way their team is organized, set up their product ops or product strategy or product practice.

I’ve often been brought in to help with strategy and ended up helping more on the operational side. And it sounds like some of your consulting is also about helping teams get their head straight and the way they work together. But what I find is that no one’s got this figured out. No one quite has the perfect model of how does UX relate to product management. It’s a conundrum. Everybody thinks that they’re the only people who haven’t got it right and that they’re behind the eight ball. But it’s really that there’s a lot of unspoken assumptions about each other’s roles and a lot of conversations that haven’t happened yet. Maybe uncomfortable conversations but if you sort it out I think there’s a lot of ways to resolve these issues. It’s just most people haven’t done the work.

Murray: Yeah. So what are the similarities between UX design and product management? 

Christian: Product managers will say that UX is something that they do or that they care about, or that’s one of the pillars of their area. Now, if you’re a UX professional, you might say your version of UX is a shorthand. It’s not really full thing. At least they’re using exactly the same language and concepts. Sometimes other times you have more like analogous concepts.

So for instance, product managers will talk about a customer quite frequently and something like customer obsession, like trying to deeply understand the customer’s pain points and do product discovery to figure out where, their journey could be better. If you come from a UX background, like I came up through, then this is very similar to doing ux research to, discover the user’s needs and pain points. So there’s even a little bit of elbowing around who’s in charge of understanding those things. Whether it’s a market or a user or a need or an opportunity, depending on whether you’re using business language or design language to describe the problem and the solution. You may find you’re talking about the same thing, but just through different lenses. 

Murray: Well, what are the differences then? 

Christian: There’s a number of differences. Different types of people are often drawn to these roles or come into them. Although people can come from engineering into both of those roles. You can come from a business background. Fewer designers have gone into product management, but that’s something I’m trying to change. I think the product managers often been thought of as more of a businessy role. It speaks the language of business. It puts a lens on things around trying to create business value, trying to sustain or grow a business. more concerned with how will we make money and, are we using data to steer the ship correctly. UX has more of a background, in design and human factors research and that sort of academic style research. So it’s a little more philosophical. Sometimes it’s a little bit more purist. Hey, the user really needs this one thing and we should fight for this one thing. Whereas a product manager may say Well, that would be great for the user, but we can’t afford that. Or, We have other things we need to do first. So the lenses are somewhat different. What you’re accountable for is different the kind of story that you’re telling the rest of the company, but what you do is quite different.

Murray: Yeah. So do user experience designers make good product managers?

Christian: I would say that there’s a certain type of UX designer who has a good foundation for becoming a product manager if that’s what they want to do. And the kind I would say is the sort of person who’s always been a bit of a busy body, a bit of a orchestrator, a bit of a person who steps up when nobody is quite defining the thing that’s going on. Person who grabs a marker during a meeting and starts writing on a whiteboard to say, Are we talking about something that looks like this or this. And they’re making boxes in little stick figures and arrows and things like that. There, there’s a kind of a leaderly ux role again that steps into the breed sometimes and says, Hey, are we asking the right questions?

Or is this even the right problem space? And I think if you’re that temperament of UX person and maybe a little bit less as somebody who wants to be a production designer for your whole career, Then product management can be a way to be at the center of that conversation about what’s the actual thing we should be focusing on and where’s the value and orchestrating the team.

So that type of UX person, can make a very good product manager. There are still some stumbling blocks because you can thrive as a UX professional these days and still not really grapple with all the more businessy things that product managers have to. It helps if they understand, the financial basis for what they’re doing and the need for revenue or the need for growth. But in truth you can be a little bit sheltered from that and left to make a slightly more theoretical design in the way UX is often practiced these days. So if you come into a product role from a design background or UX background, it could be a rude awakening that, hey, we have to pay for things, and we have to make sure the numbers are going in the right direction.

Murray: Yeah, a product manager is trying to find a balance between what’s good for the customer. What’s good for the company and what the engineers can do. Desirable, viable, feasible. 

Christian: Yeah, that’s a good model. That works pretty well, especially that idea of balancing those factors. Cause often it, can be more complicated under the hood. It’s also what the CEO really wants today. It’s also what a major client is asking for. There’s often this standing in the middle trying to find, just the right balance of all those different forces to, create something, that’s valuable enough to make a place for itself in a competitive market.

Murray: Yeah. There’s a considerable amount of stakeholder management. So I found as a product manager that often, you have a lot of pressure on you from senior managers to do this, that, or the other thing. And maybe you really don’t have a lot of choice except to execute the vision of those stakeholders. 

Christian: I think that’s the case in A lot of teams and it actually leads to this funny situation where the other people on the team, like the UX designer who comes asking for something or asks why priorities are the way they are and is just told this is how it’s coming down from the chief. In that environment, the product manager is read as tyrannical as a sort of a powerful product manager who says no to everybody and doesn’t have to explain why things are the way they are. And typically that kind of product manager doesn’t feel powerful at all, but is actually a ticket taker who’s just passing down what the bosses want. And typically, if you’re in that position, you haven’t been empowered, you don’t have a lot of power. You’re essentially more just like a person managing a project. Sometimes it might be that there’s some agency and it’s just not being expressed. So sometimes you have a bad product manager because they don’t push back. They don’t manage up at all. They just agree to whatever they’re asked to do, and then they come into the room and say, Okay, here’s the tickets for the next sprint. When they ought to be saying, Okay, boss, that’s great, but remember what we said last week. We’re working on this other thing. It’s really important for the company, or, to have those difficult conversations. Really if a product manager’s doing their job at all they’re supposed to be standing in between and as you say, managing those stakeholders.

And yeah, a UX designer or UX strategist. Is often insulated from that and may not have that muscle, built, but it can depend. You could have worked in agencies and dealt with stakeholders a lot. There’s no saying you don’t have those skills. Rallying people is often like the same thing as the ability to have a workshop and pull a bunch of people together and get them all to agree to something or to sign onto some manifesto.

I’ve seen UX people thrive in these roles, but it is a different set of skills. And something you point out too with the stakeholder management, it’s a lot more communication work. I think a designer can put headphones on and listen to techno and, be in Figma for hours all day. Whereas I feel like product managers have to answer Slack and email and fill out Jira and GitHub tickets and write up one pagers and do a lot of communicating all the time. 

Murray: Yeah. I have worked with designers who were very frustrated, sometimes quite angry with product managers and project managers because they felt that, what they did wasn’t being respected. But on the other hand, those designers just didn’t care about budget or commercial issues. Feasibility. They saw their role as being fighting for the best possible design for users and everyone can just stuff off. Have you seen that?

Christian: Yeah. Some UX people have a savior complex or, evangelical, they’ve really drunk the Kool-Aid or, they really believe in this in a passionate way, like it’s an ethical thing and they believe on some level that they’re better than other people, because they’re the only person looking out for the poor user. Not only is that a not very wise way of, thinking about one’s own role in the work, but it’s self-defeating because it’s off putting, people don’t like someone to come into the room and be like, Okay, dummies, now the smart person’s here and I’m gonna fix everything for you people who apparently don’t care about users like I do. I think it’s a journey in a lot of the UX professional’s lives to go through that enthusiasm and then realize, okay, I’ve become a little bit of a person who just converted to some sort of new belief system and is way into it. It talks about it too much and you have to start listening again and not come into the room shouting, this is what I have to do for you, but more like, I’m here to understand your needs and figure out how to be helpful. People joke about ux, people believing they have really great empathy, and then I push back and say, you’re not always showing that much empathy for your coworkers. You’re just saving it all for the end user.

Shane: In the product leadership space, there seems to be a lot more material and training and coaching to help you acquire skills. Is there actually a whole lot of coaching capability out there in the UX space?

Christian: Yeah I think that UX has a pretty evolved community of practice . It’s subdivided many times now, so that can sometimes, I think, make it hard to find the specific, say, UX research, which it would’ve just been uX or IA or something at one point, and now you have to be in the research community, the research ops community, or the research leadership community, or the leadership coaching community or, so I suspect it’s out there, but maybe not as directly visible if you aren’t already embedded in it and probably generally harder for people to find those things.

I would say that one of the particular flavors of frustration, UX people have, in their relationships with product management these days comes from UX researchers. If you’ve been around UX researchers, it’s not a new lament.

Product managers are just the newest people that they’re complaining about. Which is, we do all this research, we have these recommendations, and then nobody listens to us. We don’t get put on the roadmap. We’re told it’s already been decided. They look at the deck and then nothing happens.

So researchers sometimes feel artificially separated from design, let alone from shipping product. And the thing is, sometimes this is, a failure to really make use of good research and a failure for the whole organization to be on the same page about what the research is for. But there is room for UX researchers to, reflect and say, if they’re not listening to us, maybe our slide decks have too much detail in them about how we did it and not enough about what our recommendations are. Maybe they’re not pithy enough.

Shane: We’ve had a lot of people from the product side, and a few people from the UX side. And I think there’s a chasm between what I call the lean MVP approach and the true UX research approach. If you take a UX research approach, you’re doing the UX research to discover a problem, and once you’ve discovered the problem, then you may go and figure out how to fix it. If I take the Lean Startup MVP approach, you think you know what the problem is, so you’re gonna build something quickly, get it in front of the users, get some feedback. And whether A, that problem actually exists, and B, whether what you are doing is gonna fix it. And it seems to be a fairly different way of working, depending which one you pick. And then what I see is people get confused. They look like they’re taking a lean MVP approach, but then they have this idea of UX research, which doesn’t actually fit into their flow. Is that actually you need to pick one of those two as your primary way of working and then figure out where everybody’s skills and roles fit in it? Or can you blend those two approaches well?

Christian: I don’t think there is a single recipe that’s gonna work for everybody. I personally believe these things can be blended to some extent. I’m not a big fan of design sprint or parallel sprints that are off by X weeks, which is just like a mini waterfall happening all the time. So I don’t think that is inherently great. One thing is I do believe in the whole team being involved all the time. So there’s researchers on the team and they know what the research questions are and what the problem space is just like everybody else does. Different roles are more engaged at different stages. So engineers listening in and giving thoughts about their architecture, but not coding early on. Often in certain projects than designers are doing more QA and just refinement close to shipping time. So we all know how the pistons come up and down. The different parts of the team are firing away at different times in different degrees. Similarly I think research is about de-risking things and I think the lean approach and the agile lean startup model and the build measure, learn thing, I think had a refreshing truth, which is you don’t have to do a UX research project before everything that you ship.

You can have a good enough idea, get it out there and learn in the field. Admittedly, you’re doing UX research at that point in the sense of you’ve just got live users, you’re just using as subjects for a giant study on some level, so it’s shifting around the definition of how you’re learning. Because the learn part is the research. Now you could say we don’t do it the way, it’s not as longitudinal. It doesn’t take as long. And I think UX research has learned to be a little bit less academic and a little faster and a little bit more getting to the point. Usability testing can be done handly with a very small number of people.

It’s more, a matter of having all these different arrows in your quiver or, I don’t know, like golf clubs or whatever for different shots. There’s times when you go, we need to dig into this, we need to get some real people together and talk to them and do some research to understand this problem space better or this possible opportunity.

And other times you can say, we’ve got a pretty good hunch we’ve seen this one. That cycle is going on all the time. So where it starts is meaningless because once you’re building, whether you say we didn’t do any research, we built first, But even that very first build, what did you build?

Somebody, some founder or whatever, they didn’t do research, but for a year or two they were like, I have an idea. They kept thinking that was the research. It was just this qualitative self-directed, not very scientific scratching of an itch until they decided, I think I know what I wanna make, and then they hire someone to build it and , start learning.

I think that model shows that you don’t have to reflexively do elaborate UX research projects like big enterprise UX teams all the time. But I do think that there’s a lot of UX research modalities that are absolutely still useful, throughout the life of products and depending on the product.

Murray: My experience is that user experience research is ignored because the process of getting support for something within an organization is largely political. If you’re a product manager or a senior product manager, you are being told that certain things are important to senior people, and you are building consensus for a business case to get money. And that business case could be based on wrong assumptions about what users want. It very often is because nobody bothered to ask the users. It’s just a bunch of managers sitting around saying, Oh, it’s definitely this, or it’s obviously that, or our competitor’s doing this other thing. So there’s all those baked in assumptions by the time you get the budget to do anything. And then if the user research comes up with something else, it says those assumptions are wrong. There can be a very negative reaction from the stakeholders who’ve , put their reputations on the line to back something. So the research contradicts. What they’re saying, they can actually get really upset about it. And, derail the whole thing sometimes, or just ignore it. 

Christian: Right Now, if you take this to its logical conclusion, what’s the alternatives? You ignore that information, you ship anyway. I guess there’s some cases where you get lucky and whenever you all have moved on to new jobs before it collapses or something. Typically unwelcome news from the researcher that the premise doesn’t make sense or the users don’t really want this new button or whatever is just an early indication of something that’s probably gonna happen anyway.

I a hundred percent agree with you that the failure often takes the shape of unwelcome information that comes in a time that isn’t really helpful cuz you’re trying to ship the product and it’s too late. it’s like, what do you mean?

So the point is we shouldn’t do this. Like that ship is sailed, buddy. It’s not a really helpful bit of advice at that point. So that, point’s the one problem, which is why is research presenting to the product person at this stage, like, why don’t they already both know about this stuff?

Why didn’t they do the research together? Why wasn’t this part of an earlier part of the project to vet this idea. There’s a failure that’s happened already at that point, But even if I think you’ve done research and you’ve oh, we found something that was what people didn’t wanna know, it actually makes sense to figure out a way to share this information in a constructive way.

To be aware that it’s gonna land wrong, that it’s going to be very distressing to people, and to try to come in with something that will mitigate the problem, to say, Hey, look we’ve noticed a big risk. It turns out people may not want this. There are communication skills that could help in making your findings useful and sometimes I think it’s better understanding what you’re describing there, that kind of political situation.

If I need to give you ammunition to win this political argument, it’s not what researchers think what they’re doing. They wanna do something relatively pure. Again, it’s just academically correct and that’s fine. You can do your math correctly or talk to the right number of people, but couching your advice in a usable form is also important.

Murray: Yeah, and I think that this is a very important role for a product manager. If the product manager is good, then they will lead this discussion with the stakeholders and influence them to make some changes. And they may decide to just accept it because it’s too hard and prove that it doesn’t work after it goes live use that to say go in a different direction. But this is a core thing that product managers do is balance all those stakeholder needs and try to bring in the information from customers and users. 

Shane: I think. also it depends on the size of the company and the life stage of the company. So I’m currently doing the Reforge program and I did the data for product managers course . And what surprised me was it came out of, big companies like Atlassian and the whole focus of it was, how to convince your supervisor was the team they used, that you’re doing a good job as a product manager. And for us, we’re a two person, bootstrapping startup. There ain’t, a whole hierarchy of product managers. So I think that context is really important is where’s the organization, how many product managers they have, where are they in their product life cycle? Are they early? Are they late? Are they VC funded? Are they bootstrapping? And that affects what your role is because if you are small and bootstrapping, the product management skills are only one 10th of what you need. But if you’re in an organization that has, a bunch of product managers and senior product managers and director of product management, then the roles that you’re doing or the things you’re doing is slightly different.

Christian: They can be more specialized, they can be reduced to, a smaller set of tasks. And as you said, when I was in a relatively early stage startups, the fact that I could shift back and forth between the UX and product sensibilities was a plus. In the large organizations, they’re like, Pick lane, what are you, that’s too confusing. In a small place, they’re like, You can do something else on Thursday. We don’t have to hire a second person for that. Great. Let’s do it.

Murray: In your book you had a section on product analytics. So what should you be looking at? 

Christian: Anything that a user may, tap to start a process or go to a new screen or enter information, can be instrumented, meaning that, you can be capturing that event in a standard way and logging it and associating it with that person’s session. And therefore understanding more or less what they did during the session, how long they were around, whether they completed actions that they started. I would’ve loved to have had this when I was a designer. Like really high fidelity insight into how it’s actually working. Before we had instrumentation in our products it was like flying blind, Like how did we ever land the plane? Just looking out the front window and hoping we weren’t too close to the ground. Now we have these panels that tell us, Oh, you’re this high up. Oh, you’re doing well, you’re losing altitude, whatever. 

Largely speaking, you capture the number of visitors, the number of people who actually stick around and don’t bounce away. Or if you’ve got an app who downloaded and certain metrics like install it or run it for the first time or set up an account. You account for people who are engaged by some definition. And then you see of those people what percentage of them come back in a certain timeframe, a week later, a day later, a month later, depending on how you were doing your cohort analysis and there’s a normal drop off at just human nature.

Not everybody comes back. But you look at that curve, you try to keep that curve from dropping off too fast. You try to understand what sort of behaviors are most strongly associated with the people who do come back more often, and what it can take to maybe nudge your beginning users into having that set of experiences that tend to correlate.

Now, sometimes you just found a correlation and by making more people do the one thing, it doesn’t cause them to do the other thing at all, because they were just coincidentally caused by the same thing. So, people play around with trying to maximize engagement, they look very closely at the retention curves and try to understand, how to make sure more of the people who, who come for the first time come back a second time.

If you’ve got a product with revenue like subscription, you’re looking for like how many people complete the trial and actually subscribe. How many people canceled the subscription, how long the average subscription. There’s just endless. In the end, you get way more data than you can consume because inevitably you ask so many questions and you’ve got it flooding in.

But I think typically a product manager has a couple north star metrics, some key metrics that you’re checking almost every day, or certainly every time they’re up to date that are what you’re steering by. If this is go in the right direction, we’re generally going the right way.

I can always dive in to figure out what happened over the weekend with that strange spike or drop off or whatever. But if you do have a couple of the sort of like stars to steer by and if you delve into your data frequently I believe you get like a feel for it after a while. Like it’s almost like data has a kind of grain to it or certain patterns or waves in it. And if you looked at enough of these charts and sliced them over and over again you start to intuit a little bit about it and recognize when it’s acting funky. When the weather is weird.

Murray: So we’ve talked to quite a few people now who’ve said that rather than focus on your backlog of features that you wanna build, what you should really do is focus on your objectives and the key success indicators or measures that you want to move. You should, for example, Commission a team 10 people for the next three months to focus on increasing retention. And that’s it. Just put in your measures, talk about it. Work it out amongst yourselves and every week look at your retention numbers rather than, do these big design up fronts, these big, political business cases, these big architectures just be completely focused on the objective that you’re trying to move and get everybody in the team to contribute to ways to move it. What do you think about that approach? 

Christian: I think that could be a very powerful approach. I think , it works when you’re trying to get, up over a mountain, have the whole team accomplish something that has to accomplish. It could be getting from zero to one, like getting to launch. It could be, you need to get something to start paying off or to meet some important goal. And I think that this approach of saying, Hey, we know what the goal is and we’re gonna just iterate throughout this time period towards it as much as possible, is much more powerful than coming up with a bunch of ideas and just executing on them. I think when you started the question, I felt like it, I was gonna have to point out that, We all agree that you should focus on outcomes, and that’s more important than just like building a backlog and creating a set of features and then just building them out.

But there’s always a little hand waving because in the end, you still come up with some idea and you build it, it sounds like we’re saying you never build any features, but of course you do that, but you do it because , the team agreed that this was the most likely way to achieve the outcome.

And we’re gonna try this first. And this is the sketch we came up with, It’s good enough to get this thing going. I can think of a very specific example where this definitely worked really well. This approach that you’re describing. When I was at Seven Cups, this mental health company where we were offering something for free.

It was volunteer support. People would just come online and they’d chat with other people and give them support because it was fulfilling volunteer work to do. But we still had to keep the lights on and figure out a way, some way to make money. We tried a bunch of things. The thing that eventually paid off was that we hired professional therapists and had a kind of a paid tier, like a freemium model where you get the free service, but you could subscribe and pay somebody.

But to even get that to break even, it wasn’t automatic. The conversion rate wasn’t enough, The number of people sticking around where it wasn’t long enough and we had a little task. It wasn’t the whole company, but we had about four or five of us, and we met once a week and we looked at, those critical data points and we said, Okay, what are we gonna do in this next week?

What’s the next experiment we’re gonna do? We’re gonna change the onboarding, we’re gonna tweak what the bot says. We’re gonna give the therapist a new dashboard that shows them how well they’re doing. Oh no, they hated that. Let’s take that down again. That demoralized them. We’re going, and we’re going to drop the least performing people we’re going to, offer free trial.

We went through all these different things and systematically cranked the conversion rate up a little bit at a time, a little bit of time. Sometimes it dropped, sometimes it went back up. But over time, it went from something like of the people who entered the funnel, less than 1% converted, and later on it was more than 1.6.

So it was a 50% improvement over time of the rate of people who were converting. That along with getting people to stick around a little bit longer was how we achieved that break even point. So I think it absolutely can work that way, that you can have a focus team that just has a goal and has a mandate to try what they can to try to achieve that goal. But that understands that you have to have the people on the team who can design something, write code, put in some copy, all the craft skills you need to make software are still needed.

Murray: Yeah. And research as well. So, a cross-functional team made up of, product manager people with skills in product management, user research design, front end, back end, software development, marketing, copy everything that you need. And those people are basically discussing amongst themselves what could we do? What about if we did this well, we dunno if that’ll work well, Would it be better to talk to our users and see what they think? Or should we just do it and see what happens? Oh, it’s gonna be really cheap to do, so why don’t we just do it and so it happens? Or, No, that’s gonna be really expensive, so let’s get some people in and walk them through it and see what they think. Because we are gonna do, continuous discovery. That’s like our, we’re getting towards a process now and how teams are working together. So you got everybody in one team. They’re working together at the same time. They’re taking ownership. That’s the ideas that we’re hearing from people now on how you should do this. 

Christian: Yeah, I think broadly speaking, that’s the right approach. As I said, I think some people are more involved at some points than others. And I agree that there are times when you say, let’s just build this we can build this faster than we can prototype it. And other times you say, Look, we’ve got some figma screens. We can make a quick prototype, set up something with remote user testing or something like that, and let’s just quickly validate or de-risk this idea a little bit. Other times, we, I’m working on a project with a state where we’re trying to recommend people who are looking for benefits that they might wanna be checking out some other benefits they may be available for, if people who got this benefit also maybe got this benefit, as you can imagine.

And we did a certain amount of research and data analysis to understand who visiting the site is seeking benefits. Like how can you detect from where they’ve been what their intent is so that the recommendation is timely and not interrupting them and things like that. So it, it wasn’t like we did a multi-year research project, but we put in some time trying to understand the available data and we did some probes with ad placement to see whether we could target people at the right income levels and we talk to other, like entities in the field that have already worked in this space.

So there’s like qualitative research of learning from other people’s experience. UX research is really like a large grab bag of different techniques that are appropriate at, at different times. Some of them you’ll never pull it out and use them, diary studies or whatever. And then sometimes you’d be surprised and it’s the appropriate thing for what you need to figure out. 

Murray: There is something that worries me about continuous discovery and that is that it can be too focused on the minutiae. And it could be that, you don’t understand the big picture, the big need Well, enough. And if you did, you could make a big pivot, which would take you from 1% conversion to 10% because you had a much deeper understanding of what people needed. What do you think about that?

Christian: I’ve definitely seen examples of diminishing returns of polishing a local maximum as sometimes call it, you’ve optimizing something, to heck, but that thing itself is only a small percentage of the biggest impact possible.

So even if bring it to its ultimate it’s not gonna make a big difference. And you may be ignoring a much bigger lever elsewhere that could be really making a difference. I think the question is, what’s the track record of other methods for finding those groundbreaking, game changing insights. That’s what innovation’s supposed to be, is not just small, incremental tweaks, but complete game changers, things that that, that upset the table and all of that. And I think we’d all love those kinds of things, but I think that maybe the idea that there’s a ritual set of processes that generate that might not be true.

Like it may not be possible to do it that easily. Maybe a certain amount of your team’s energy ought to be focused on fundamental things, on things that, question the whole basis of what you’re doing. I don’t think a lot of teams spend time on that cuz it’s probably like existentially upsetting.

It’s the extreme form of what you’re saying before. When the researcher says, Hey, the premise of this project is actually wrong, And you’re like great. Now what do I do? So if you do that fundamentally Hey, the whole company, the whole product we need to rethink it from first principles not everybody’s ready for that every day, but that’s why, people stay in a rut or don’t notice the huge opportunity. So it’s a good question about maybe how can a business or any kind of organization, have some part of its intelligence with feelers out to those breakthrough ideas, next level ideas and so they don’t get ignored.

Murray: I wanted to talk about processes and frameworks. What do you think about double diamond design thinking, design sprints and dual track design. What processes do you like and what don’t you like? 

Christian: Yeah, that’s a jumble of things I’d say to me the double diamond is like a lot of diagrams and models a useful idea conveys something important. It shouldn’t be made into kind of like a cult or anything too rigid, but the idea that you can go wide and then narrow again on understanding a problem space and then go wide and then focus in on a solution seems like not that controversial notion, if it’s not turned into a dogma.

I think that design thinking to me is like a marketing concept for business consultants, that has had some value in terms of socializing the language of design and the idea that design is a problem solving school of thought. That design offers a way of solving problems that’s somewhat different from the way say engineering solves problems or the way business leadership solves problems. It brings a different set of tools to the table. And I think design thinking maybe helped people understand that. But it also something that kind of takes the juice out of design, it’s reduces design to a set of moves in a business workshop , rather than a way of actually working.

So I’m a little suspicious of design thinking to be honest. And I don’t like parallel design sprint. I don’t like that. I think I mentioned this hinted at this before. I feel like it creates a waterfall situation where if you’re really doing a research and design sprint that is scheduled ahead of time for an engineering and building sprint that’s expected to come afterward, then inherently doing some design and then handing it down and then something’s being built and you’re not being agile and working together in the way that I think you should be.

But the problem is, some things are dependent on other things. It’s like you wanna see the design tell me what is the screen gonna look like? So the design has to be done before I start building it, or at least done enough. I think that this idea that the team is all together all the time and that different roles are more active or playing a different part, like being a decider at certain times and being more of a technical support at other times is more realistic.

So yes, you have to be researching stuff now that you’re gonna work on later. Precedence in time is a real thing and you are gonna have stuff that is sequential that’s not evil. Doesn’t mean you planned and now you’re not being agile cause something happened in a sequence. Things do sometimes take a little bit of preparation and I think that can’t be taboo. But I don’t like these two parallel sprint lines. And I know I’ve heard of some teams that use them and like them, so I can’t say they never work or they’re never useful, but I haven’t heard of many cases where that lasted all that long. It’s an attempt to solve this problem of agile lean thinking, not having a lot of room for traditional UX is trying fix that by putting one on top of the other without really resolving how they work together.

Murray: Yeah I agree. I was going to ask you about organizational structure for product development. The traditional way of doing it. is that everybody’s working in silos. You have a product team in marketing you have an engineering team in it. You have a design team sometimes they sit under marketing. But they tend to be quite separate. And people working in silos and, quite often led by managers who are jealous of their empires. How should you structure a product team to be effective? 

Christian: This is another one of those things where it, does depend on the maturity of the team and the size of the team. So I think at different stages, different structures may make sense. But I do agree, for instance, that siloing off the disciplines or having them be treated like specialties that just are assigned temporarily to one project or another isn’t a great way to do it, that, great products are made by cross-functional teams that are consistently working together over time. I like the idea of product teams that have engineers, designers, and product managers. And if other folks, whether subject matter experts or data people or QA people or whatever, in the mix as well.

On the UX side, I think generally nowadays you need content designers or content strategists and you need researchers. I don’t always see these teams reporting into marketing, but typically product and engineering are reporting into different leadership at the VP or chief level.

And often UX is reporting into product in some way or if not into product management, into a product leader, like a VP of product who has both the UX and product management under their belt, which isn’t not necessarily a good thing. As an ex designer I’m sympathetic with people who feel like it’d be good to have a VP of design or a Chief design officer or chief experience Officer or something like that, but you rarely have that.

I’ve also been on a very small team where I was the head of product and and even the engineers reported to me, which was super fun for me, but I think engineering teams do need to be self sufficient and shouldn’t report into the head of product.

Murray: Okay. All right. I think we need to support cross-functional teams. It gets more challenging when teams get bigger. There’s this whole product ops idea now. 

Christian: Yeah. I think that, people are starting to realize that when you get to a certain scale of practitioners of any kind you start to have these operational needs. And so just like design ops has become a thing more recently in research ops, and of course DevOps was always a thing.

The idea of saying, Okay, how do we train new people? How do we provision them? How do we equip people? How do we keep them learning their craft? Do they get to go to conferences? Like the idea of having a guild or a community of practice or something like that starts to evolve once the organization gets big enough. 

Murray: That just seems like pretty obvious leadership coaching support. 

Christian: But I think it’s more about pulling out an individual to do it. I don’t think it’s new things that nobody ever did before. I think it’s starting to recognize that at scale you can have people who are officially dedicated to doing those things.

I was just talking to Peter Burma the head of design ops At Miro. they work closely with the scrum masters, they’re all coaching training, they’re all helping professional development of their practices, but sometimes to like different traditions of how to do it.

Shane: So I’ve got a, a topical question. It’s topical because there was a whole rant and argument on LinkedIn by a bunch of people around it, and it’s around this idea of minimal viable product. So some people were taking the stance, That, you’re a startup, you’re doing mvp, you go use no code tools. You create a website, a little bit of a hacky thing, and then don’t build anything. And if people start giving you money, then you’ve got a minimum viable product. The other lens that I could take from there was you’re building a product that works, you’re just not adding all the features you want to. And then you’re testing the features you built had value were viable. And then, rinse and repeat. So what’s your take on mvp? What is it like, what’s your definition that you use?

Christian: Okay, sure. I mean that first example, that no code example, it’s a real thing. I know examples of that working. And I guess I would consider that maybe a flavor of mvp, but it’s certainly not what MVP meant to me in the past. I was taught by one of my product mentors, a guy named Jay Zari to, instead of saying viable, to say, valuable to say the minimum valuable product to help with that idea that product managers are focused on value and creating something of value, a value to the customer, a value to the company. Something that’s actually valuable enough that it pays its own way. So here’s the thing, mVP has become bastardized term that gets used very loosely to mean like a V one, any V1 of something.

I was working on a project where the executives started talking about MVP 2.0 or MVP 1.1, it just became this jargon word for some early version of a thing that wasn’t very well specified. I was taught that it’s the minimum thing that will actually work. You don’t hang around with that in the market for a long time because it’s only minimally valuable. It’s not really that great. I think the main criticism is this idea that there’s no version two, things are called an mvp, but often they ship it and you move on and you never come back to it.

You never refine it. So if you’re not really doing that, you’re not really doing that MVP process, which was never supposed to that you ship it and forget it, but we’re supposed to be, get something out there as quickly as you can. That starts to be any kind of semblance of the eventual thing to learn. Whether it’s at all valuable and then you can continue to refine it and, add jets to it and add a spoiler on the back or whatever it needs.

Murray: I agree with you. This came from the Lean Startup book. And also Steve Blank’s work . And I understood that it was about doing experiments in the market to see what people wanted. the classic example was the guy who started the Zappos Shoe store put up a website with photos of shoes from some shoe store. People bought them. It all looked okay on the front, but it was all email in the back. And then he went and bought the shoes from the shop and then sent them to them. Losing a lot of money all along the way, but just proving that people would buy shoes online. 

Christian: Yeah, I’ve heard that called the Wizard of Oz, prototyping technique where. Behind the front end, you’re just running around doing it all by hand. That’s fine cuz why write any code before you know if people want this thing or not? All these ideas have great value. They catch on cuz they really crystallize some important wisdom. But then they get a little bit too crystallized, they get ossified and they become like a, you have to do it. And it becomes very rigid what the idea means. And they lose some of their value when that happens. But if you think about it, the MVP is saying, get something out there. Don’t study and research and go to the library for six years. Get some part of your idea happening right away and start learning from real people. 

Murray: You mentioned before about content strategist and you said you were an information architect. Explain to me what that is.

Christian: Which one? Information architect or content strategist. 

Murray: Both.

Christian: When I started off I was working for a a.com consulting firm that was making web marketplaces for other startups and building them out for them and helping them figure out their ideas. And I was hired initially cuz I had a writing and editing background. I had been making my own websites onto the content team and they said, Oh, you’re a content manager. Cuz I guess cuz you do the content management, and I was like, Okay. But then we started thinking about what we were actually doing and we said really, we’re content strategists because we’re not just writing and editing some content for the site launch.

We’re setting up a process by which this website can be maintained over time. What kind of copy are we gonna use and what are gonna be the names for all the labels and what’s the messaging and communication strategy? And those are gonna be like, Oh, what’s new, page and with press releases on it, and who’s gonna, update the thing over time.

Which is kind a blind spot, even back then, it’s like putting a building up with no plan for heating it or keeping it, Cool in the, hot season and all that. So content strategy to me then was the sort of thing of saying, this is what all the content should be on this project and this is why, and this is how to keep doing it in the long run.

It may have evolved into other things since then. Our company got sold by some other company that didn’t have content strategists, and they said you sound more like an information architect. And I said, Oh, okay. That sounds like a cool title. And started trying to look up what does that mean?

And, information architecture at the time were people who were taking complex information systems and trying to map them out, clarify how they were structured, come up with visualizations or structural concepts that could be anything from a library science. Type of person with taxonomies and ontologies and synonym rings and things that became more like search indexing work.

And other people who were more like just trying to understand how the website should be organized and what the menu should be called and what the structure of the site should be. That’s what a lot of people who were called information architects were doing in like the early two thousands.

Those people, again, tended to evolve into interaction designers or UX designers over time and information architecture has come on websites and to a lesser extent mobile apps to mostly mean how is the site structured? What’s the organizational, what’s the way finding, what’s the menu, what’s the navigation?

If you talk to hardcore information architects, they work on stuff that goes beyond that. Especially now where information spaces and real spaces. crossing over each other. They do stuff that’s actually more architectural these days. There’s a whole subculture of information architects out there.

Murray: Is this part of user experience design and product management? 

Christian: Some people say the content is like 80% of the user interface. Sure matters the color, the look and feel, how professional the actual design elements are. But actually, what you use the terminology the messaging, the explanatory copy, that’s most of what people experience when they’re using an interface.

So I think there’s been more and more recognition that you need good content people on a good UX team. I think information architecture, unless you’re a certain kind of team that works on certain kinds of information. Products, you don’t tend to have a lot of dedicated information architects on UX teams anymore.

But I do think that people who have information architecture background and skills tend to be the type of UX designers who make good product managers. Because I think product managers also benefit from that ability make a picture or a map of the whole thing that we’re making, the landscape of the project, not necessarily the UI of it or the data model or the technical stack, but something that’s more like the user journey.

Like the what of it, Like what does a person do when they use this product? Where do they start? What happens in the middle? What’s it like when they come back later? Why are they using it? Where does it fit into the rest of their day? I think that there. An information architecture skill set that helps to , visualize that and make it into a map people can look at and debate and come to consensus on, and then agree, Okay, we’re tackling this part of it.

And I think product managers and product leaders like to have those kinds of maps too. Even if they don’t personally have the skill set , to generate them, they need to find people to help make them.

Murray: Yeah, so given you have a limited amount of time and money and resources, and you’ve got all this information coming in, all these ideas, how do you decide what to build? 

Christian: There are always, infinitely more ideas than you could possibly build. You could make your team 10 times as big and you would just have a hundred times as many ideas. The answer lies with the problem that you’re working on. The problem is the path. When you look at the biggest problem you have, the thing you’re most worried about, the problem you’re trying to solve or the opportunity that you’re trying to meet. It’s like going deeply into that, into the hardest, most difficult part of that is where you find the thing that you should be focused on. 

The generic answer is that you have to focus, that you have to find some way to throw out , 90%, 99% of your ideas and postpone them. Even if a end, you’re arbitrary. You have to resist the temptation to complexify things and to let extra ideas sneak in.

Cuz that’s the death usually of finding that one idea that’s really gonna work. On some level you have to have some sense of what is the most salient thing, what’s the heart of your idea, and that’s what you need to focus on first. But you also have to ruthlessly fight off attempts to like, make it more complex or add more things to it, especially at the beginning.

Murray: Yeah. Okay. That’s good. All right, let’s go to summaries. Shane, what do you got? 

Shane: Excellent. All right. So we started talking about, product managers. They talk in business language and the UX teams talk in design language. And that theme of language came through time and time again in a lot of the conversations we had. I like the idea that, often the UX kind of roles are more philosophical, they’re more part of the process. Whereas the product manager role tends to be a little bit more broad. They tend to have to worry about how they’re gonna make the money, they tend to do the stakeholder engagement about what will be built and what won’t be built, what’s feasible, what tradeoffs are we gonna make.

They’re typically talking to the engineering team about what’s possible to build within the constraints we have. I still see it as a tri factor where the UX skills, the product management skills and the engineering skills should have a conversation ongoing basis to find that blend of what can we build that has value in the constraints we have, and then test whether it worked.

I think the way you describe product managers aligns a lot with the definition of a product owner within Scrum, that ownership and trade off decision and being able to go across those other different roles and get consensus or make a call if you can’t. I love the idea of pistons come up and down as a way of describing the flow of work across a team, because there is a delay in a handoff often, unless you’ve got that unicorn that can do everything. I also like the idea that research is about de-risking, it’s about reducing uncertainty early. So we do the research because we want to reduce , the risk of that bit or the impact of that bit if we get it wrong. We’re constantly doing research, we just do it in different ways. So with UX research, we have some patterns, which are, get a group of people, ask them some questions, document, we have a certain way of working to do that research. But when we talk about, learn, build, and adapt, that’s the form of research as well. We’re just doing different patterns to do it well.

So like the idea that a while ago you were doing UX design and flying blind, there was no instrumentation. And now we talk about instrumentation with things like mix panel and amplitude. I also like the way that design offers a way to solve problems that’s different to business or engineering, there are design patterns that we can use for our way of working. So again, , that idea of combining, business engineering, design and agile together to, to give us a shared language.

Like the MVP discussion language is important. If we’re working from people from different domains, perhaps the first thing we should do is actually just agree what we’re gonna call a things a thing. it’s okay to call it minimum viable product. Minimal valuable product. Minimal lovable product. Minimal saleable product because each one of those different words may have a different set of techniques. So that’s okay. It’s when we use the same words for five different things across the teams that we hit problems. So yeah, that was, that’s what I’ve got. Murray, what about you?

Murray: Yeah, I’m thinking about the similarities and differences between user experience design and product management. I think it’s critical for product managers to understand user experience research and customer research. And it’s critical for them to test their hypotheses or the organization’s hypotheses. There should be a funnel of hypotheses where, senior stakeholders and the CEO and everybody else can put their hypotheses into a funnel that the product manager can prioritize and test before investing heavily in, and that testing could be done in many ways through some user experience research interviews, through mocking stuff up, putting it out there and seeing how people respond and just by doing it and seeing how they respond.

That doesn’t seem to happen very much at all, and we should be doing more of it. I think there, is a good career path from user experience design into product management. So they’re not the same, They’re quite different sort of roles, but it’s very helpful for successful product management. It bothers me a lot when they’re fighting with each other. I don’t think they should. They should be helping each other. 

I like your approach, the process stuff that, these are just frameworks and things you should consider. Let’s not treat them as being recipes that you have to follow. There’s a lot of people in our world who just want a recipe to follow and they try and institutionalize it in their company’s processes. And people are busy writing processes. You must do this way, you must do it that way. We are big fans of tailoring letting the problem lead you, as you said. So when we are helping teams become more effective at building products and software and so on. We start with what are we trying to achieve, where have we got to, What’s the problems that we’re trying to solve? And I always get a huge amount out of doing retrospectives with the teams. The teams will tell you what the problem is and when you need to focus. And I think the customers will as well if you talk to them in a structured way and talk to them often. So I would like teams to be doing customer interviews every week, just book in two or three and just talk to those customers about whatever your thinking of doing next. And surprise me that people don’t know this, but the way you get customers to come to interviews is you pay them. I used a great tool called Askable to find people.

Christian: I just wanted to second your enthusiasm for retrospectives. I actually think that they’re the most agile thing you can do. More so even than saying, Hey, what did we build last sprint? How can we build something better next sprint, but more like, how did we work together last sprint? How can we work together better next sprint? You know, In the little ways like, Oh, it would’ve been nice to know this thing had changed, Or, if you put the Figma layers in alphabetical order, that’s easier for me. all these little things that you do, that you, that help a team learn how it specifically needs to work together. Retrospectives are just this very powerful way to keep getting better and better at what you do.

Murray: Yeah, I agree. If you do this with teams, you’ll find that they will be able to improve the way they’re working quite quickly. The retrospective will also tell you about organizational changes that you need to make to unblock the teams. Like for example, having the designer in your team rather than in the separate team where you need to submit the design request and wait weeks and weeks. So yeah, retrospectives will tell you a lot on how to get better. I think customer interviews do the same thing and probably, just talking to your stakeholders, frequently, communicating with them. These are our internal stakeholders I’m talking about. So it’s all about getting information regularly from those different groups, what’s working and what’s not, and acting on it. 

Christian: Yeah. And that, over-communication theme, which has come up before, really everybody ought to have the access to the insights from instrumentation and from the data that’s flowing in, but of course it’s hard if you haven’t not practiced at it and you don’t know what, where to look or what’s the most salient part of it.

So then I put that back on the product people to put together little reports to say, Here’s the interesting stuff from the data this last month or this week or today, or look at what happened. And to be a guide to that stuff for the rest of the team, essentially to cultivate their interest. Cause what you’ll find is one of the designers is super geeky about numbers and does wanna know that stuff and they start to become your data buddy.

And then, of course some of the engineers are too, obviously. But I do put that on the product people to be the ones to talk up to stakeholders, talk out to the team, talk to customers outside of the organization entirely.

Murray: Yeah, and let’s have a cross-functional. Product team that is focused on measurable outcomes and is looking at those every week rather than a team that is just trying to build a long list of features that somebody thought was a good point at some time. Just be outcome focused rather than output focused. One team rather than silos. Empowered teams rather than bureaucracies.

Christian: It’s always great when you tell a team that you’re gonna throw away their 300 item backlog. 

Murray: question. You’ve written a book called Product Management for UX People. I think people can find that on Amazon. How else can people find out more about your ideas?

Christian: I have a website, design in product.com or dnp.xyz if you wanna type it shorter. I’ve got a personal site just for a little more information about me. crum.me, just that’s my last name, Crumms. then. december 6th. I’m collaborating with Rosenfeld Media, my publisher on a one day remote conference called Design in Product. Looking at this intersection between UX and product and sort of what UX people need to know about, the product world we’re all in now. So there’s a lot of ways to get in touch or stay in touch. 

Murray: All right. That’s great. Thank you very much for coming on.

Christian: Oh, thanks for having me. It was a great conversation. Great summaries too.

Murray: That was a no nonsense. Agile podcast from Murray Robinson and Shane Gibson. If you’d like help to build great teams that create high value digital products and services, contact Murray evolve.co. Let’s evolve with a zero. Thanks for listening.