Lean software development
Join Murray Robinson and Shane Gibson in a conversation with Mary and Tom Poppendieck about Lean and Software Development. Organizations have queues because they don’t care about the customer. The three rules of lean, customer focus, flow and highly efficient expert teams. Scale your organization like a micro services architecture. Its not about doing the work right its about doing the right work. Don’t focus on utilization focus on the value we provide. Don’t copy practices copy the principles behind the practices. Don’t centralize processes. Set goals and let teams develop their own processes.
Read along you will
Shane: Welcome to the no nonsense podcast. I’m Shane Gibson.
Murray: And I’m Murray robinson.
Mary & Tom: And I’m Mary & Tom Poppendieck and I’m Tom Poppendieck,
Murray: Hi guys. Thanks for coming on. I really appreciate it. I’m a big fan of your work. I bought all your books when they first came out, which is quite a while ago. And it inspired me to read, a whole lot of books about lean. I really appreciate what you’ve done there.
Murray: So a bit of background for our audience. Mary & Tom and Tom are husband and wife team. Who’ve been teaching people about lean software development for the last 20 years or so. Mary & Tom started a career as a process control programmer and moved on to manage the it department of a manufacturing plant, and then ended up in product development. Where she was a product champion and departmental manager.
Murray: And Tom was a physics professor who spent 25 years in it as an enterprise analyst and architect. His early work was in it infrastructure, product development and manufacturing support. And he spent many years consulting in healthcare, logistics, banking, and travel services. Over the last 20 years, Mary & Tom and Tom have been writing about lean software development and teaching people how to implement it.
Murray: So a couple of questions. When I look around at organizations today, big telcos, big banks, I see huge bottlenecks of work everywhere not just in software, but in everything to do with the services that they provide. Why is that what’s going on there?
Mary & Tom: In some sense, organizations don’t really have to be customer orientated. Queues don’t matter. They maybe optimize work for the people who are doing it. And the fact that customers have to stand in a queue to get taken care of. Doesn’t seem important to a bunch of organizations. I found that out just this morning. We have a motor vehicle division and it gives out driver’s licenses and license plates for cars. And we have a little local shop where you can go and you can get whatever work done if you’re willing to wait in line for an hour, maybe two, you have no idea, least that was true before the pandemic. And now it turns out that you can make an appointment and they have the same process, long queues. If you walk in, you get in a queue, but if you come in on time for your given appointment, you get to the front of the queue.
Mary & Tom: so now I can schedule and I can get in there and I can get out and I can go straight home. And they did that. First they closed during the pandemic, and then they started opening just at a window. And then they realized they had to find a way not to have the cues. It’s a very simple solution, but all of a sudden people sitting around in doors became a bad thing and overnight they figured out how not to make people stand in cues.
Mary & Tom: And if you look around that happened in lots of places during the pandemic, we have discovered all kinds of ways not to have physical cues, . Because whoa, we don’t want people standing next to each other in an indoor space. And yet we still haven’t applied any of those queue limiting strategies to other queues. And that is frustrating. All kinds of things are messed up because demand is exceeding supply and you get queues and there are lots of pretty classic ways to deal with queues.
Mary & Tom: Most it organizations have always had more work than they can handle. In the 20 years we’ve been doing this, I’ve actually never run into an it department that didn’t have that problem.
Mary & Tom: If you can make stuff flow through your organization, rather than weight in piles, weight in queues everything works better because you’re not multitasking. You’re not putting something down, forgetting about it. Stuff stays fresh in your mind. There’s lots and lots of advantages of rapid work through a system. But when people discover that demand exceeds supply they still keep accepting the demand, even though they can’t take care of it. And one of the fundamental things with any strategy to deal with more demand than you have the capability to deal with is to stop accepting excess work.
Mary & Tom: You know what very few companies, very few it organizations even thought of that as a concept. When we heard people start talking about agile, they said, just put work in a backlog. established a queue. These are bad things because they don’t let work flow and letting work flow through your organization is the best way to get people focused on just one thing and getting it done and to get that thing done really well.
Mary & Tom: Yeah. Mary & Tom missed the major point. Oops. Yep. And the major point is not how to get work done efficiently, get a lot of work through the system, but getting the right work done. And the question is, how do you figure out what the right work is? And it’s not what somebody asked you for. We have been to a number of very successful organizations. The founding leaders said my principal job is to ensure that my technical people don’t do what the customers ask them for. My principal job is to make sure that my technical people find out what problem our customers and potential customers are trying to solve and solve that problem for them.
Mary & Tom: The solution that the customers ask for is normally not the one that is going to really work for them. That’s what led to Apple’s success. They didn’t do what the customers asked for. They did a deep understanding of the problems that people faced and delivered brilliant solutions to those problems.
Mary & Tom: Now, software teams cannot do this. Neither can marketing teams, neither can any individual teams. What can do this is a product team. A product team includes the technical people. It includes support people. It includes operations people. It includes people with the authority to establish business policy.
Mary & Tom: It includes financial people. It includes many different perspectives that are need to meld their expertise into a brilliant product. Now that has nothing to do with scrum that has nothing to do with agile. And it has little to do with lean. It has to do with developing successful products and every organization, including banks and everything else is really developing products. They’re not developing software, nobody wants software. They want their solution solved the problem solved. That’s the direction we’re thinking these days?
Murray: Yeah. That’s what I was gonna ask you. What have you learned since you wrote your lean software development books.
Mary & Tom: So we’re talking two decades. I’ve been in the software industry since late sixties, very long time, like 60 years. And I have never seen anything that isn’t rapidly changing in my field. It changes constantly. Any thing that’s 10 years old is suspect. If something’s 10 years old, it’s either become table stakes or it’s obsolete one or the other, anything that’s 20 years old is pretty much outdated. And so we have to be careful not to be hanging onto ideas that we came up and wrote about 15 or 20 years ago, but there are basic principles, things that you can think about to help you think along the right directions. Like you have to focus on what the customer problem is, not what somebody is telling you to do. That’s a principle that’s probably gonna become, that’s become table stakes rather than, rather than something that we don’t think of and a large amount of lean comes from the principles that were developed in the fifties and in the sixties to rapidly flow work through manufacturing plants in Japan. And the concept of rapid flow through a complex environment is what it’s all about. And I’ve learned that there’s no set of practices, any set of practices that you can expect to last for 20 years. You have to start thinking about what are today’s problems, what are today’s technologies? What are today’s issues and what kinds of principles can I learn from the past? That’ll help me tackle today’s problems and issues.
Shane: If, we look at 30, 40, 50 years ago we were basing a lot of the way we work on industrial factory thinking. If we look at our education system, especially in our part of the world, it’s around chicken farming. It’s around teaching hierarchies and not teaching thinking or self organizing. But, my view is over the last 50 or 60 years, the things we are trying to do, the ways we are working the ways we organize people and ourselves has fundamentally changed from that factory concept to more self organizing, more solving problems. And for me leans always seem to be applicable to the idea of a factory, because whenever I hear about lean, I always hear about stations and flow and movement. Work in progress just in time of those , Toyota car manufacturing concepts, but fundamentally that’s wrong. Actually those patents and principles, a lean are still applicable when we are solving complex problems, we just have to apply them in a different way. Or am I completely barking?
Mary & Tom: Right now we’re trying to get solar panels on our roof. So this is a very modern problem. In fact, the technology to be able to do that is very recent. And the impetus to do it comes from the fact that we had a complete electrical meltdown in Texas a year or so ago. And we have a massive oil crisis right now because of a war. So energy is not just costly one place, but around the world and less reliable because of demand. And all of a sudden people want solar panels and a battery in their house. So they are way more independent. . So that’s a brand new problem and the technology is brand new, but the problem is very old and that is, we don’t have the engineers, the project managers, the designers the installers. To do that. We don’t have enough. We have a lot of little startup solar companies that would love to do it. And in our environment, every one of them is totally overwhelmed.
Mary & Tom: So we signed a contract in November and they’re still not getting it. And every two weeks, there’s a step forward in another queue to go into and the queues are all sequential. They’re not even in parallel. And sometimes it’s a queue at the energy company. Sometimes it’s a queue at the building permit place, but the queues just keep coming and they’re 2, 3, 4 weeks .
Mary & Tom: And all of those cues are what we’re talking about when we’re talking about flow through a complex system. Those solar companies are gonna compete with each other. And one or two of them is gonna figure out a faster way to get work done. That’s gets more work out of the people that they can get. Cuz you can’t hire enough. When you get into that a deep hole, you have to figure out how to do more with what you can get your hands on because there’s just not enough people. And so if you think about lean as something that allows everybody who’s involved to be way more efficient in their work, whether it’s machines to be more efficient so I can get more stuff through my plant or whether it’s to allow the project managers and engineers at the solar company to be more efficient so they can spend more time concentrating on their problem rather than juggling all of the unhappy customers or whatever it’s how do I get more?
Mary & Tom: Work out of the intelligent people that I can get my hands on. And in factories it was, how do I get more work out of those machines? Cuz I only can get my hands on so many. And there’s three, I would like to say three fundamental principles that we have come up with over these last 20 years that we keep coming back to.
Mary & Tom: And if your problem is you’ve got too much demand for the resources, you can get your MI on. Then you’ve got probably three things that you need to think about. And the first thing is you have to start focusing on the customers and their problems. You cannot start doing what people tell you. You have to make sure you’re solving the right problem. You have to make sure you understand what the problem is. As Tom said, or you’ll start spending all of your time doing stuff. That doesn’t matter. And so you have to develop an ear for listening to the people and asking them the right questions or not hearing what they say, but seeing what their problem is.
Mary & Tom: You have to be able to understand what the issue is. Take our solar company, they. Are not really worried about the fact that I’m losing months and months of opportunity costs, cuz I’m not getting my solar installed because they have so many other problems. They can’t, worry about what I’m thinking because they have so much stuff on their hand.
Mary & Tom: So you’ve got to figure out a way to make sure that the people doing the work are focusing on the customers, what their experience is, what they really need as opposed to what they think they need. And there’s no formula for that. Somebody says product owner fine. In one case that might work, but that’s not a universal solution. That’s a for this particular environment, it might be a good idea solution. You need to have ways to be sure you, the customers are heard, they’re asked the right questions, their feedback is obtained and acted on rapidly.
Mary & Tom: And then the second thing that you have to do is you have to make sure that you don’t have cues. Because the queues are bottlenecks, as stuff goes through your company, all that is, is work piling up for somebody to have to handle, to have to prioritize, to have to call the customer and say, I’m sorry, we’ve got another wait, that’s all wasted work. Any time you spent putting something on a list and prioritizing it is not getting the work done that the customer needs.
Mary & Tom: That’s actually the technical definition of waste. It’s not work that’s contributing to the problem being solved, to the customer, being happy. All of those bottlenecks do nothing but make people that are supposed to be doing work, have to juggle problems and have to figure out where the work is and have to drop something and pick it up later and forget what they were working.
Mary & Tom: And all of those things that make the creative people trying to actually do the work way less efficient. So there’s ways to think about how to get rid of queues, but the most important way is to not handle things individually. Take shipping for an example. What made shipping so extraordinarily efficient containers. In the 1950s, we started having containers and container ships. You put all your stuff from say, Australia, that’s coming to the us and you put it all in one container. And then you never have to think about it again, because it’s gonna arrive at the designated spot.
Mary & Tom: We ordered a little piece of something from a furniture company in Norway and our local company, three miles downstream. They had a container on order from this factory in Norway. Factory filled up their container, put it on a ship and voila, three weeks later, like clockwork, it ended up there with nobody paying any attention to all the individual orders.
Mary & Tom: If you think about that concept, when you have way too much work to do, you need to figure out how to get groups of things to flow you through your company without demanding a lot of attention. Because the attention is wasted effort. So if our solar company has the project manager that has to make every single step happen, every single task, then what you’ve got is somebody spending an awful lot of time on one thing. And that’s just not scalable. When Amazon discovered that it couldn’t buy a computer big enough to handle all of the transactions that needed to handle and all of the databases that needed to have databases for. It said, let’s stop letting databases be responsible for transactions. Let’s figure out how to take an order and move it through all these services. Forget the database because we’re gonna move it from service to service. We’ll tag it with the things that it needs and then we’ll have it moved through by itself. And so that was a similar concept. To putting stuff in a container stop, having individual things tracked and start figuring out how to group things and get them to move through together so that, as a solar order comes in, you put the steps that it has to go through into a computer and the computer moves it along from one step to the next.
Mary & Tom: And what’s going on with all the smart people that you have, what they’re doing. This is the third thing is you’ve got people that I’m gonna call single responsibility teams. You have teams that are supposed to do one thing, in, Amazon, you have teams that handle one service, one step in the transaction chain.
Mary & Tom: And so you want to have a team that does just this part of the electrical design or just the installation scheduling or something like that. So that those teams can spend all of their time doing the areas that they really understand really well. And you can’t have a long queue in front of any of these teams.
Mary & Tom: So what you have to do is you have to make sure that the work, when it is ready to move from this place to this place, it’s ready to move. And when it comes into my area, it’s ready for me to work on. I don’t have five other things I have to check, and I don’t have to be interacting with and waiting for all these other people to complete their thing.
Mary & Tom: So the brilliance of services in software is that it takes all the work and focuses it on a single responsibility team that does one thing and one thing Now that’s also the concept the Toyota had in its flow through a factory. And what it found when it applied its lean concepts to engineering is that the engineers could have, their thing, their single responsibility, they did one thing and one thing and they could spend 70 to 80% of their time doing engineering rather than worrying about cues and worrying about multitasking and all of that.
Mary & Tom: So what you’re trying to do is have it so that the people who actually add value can spend all of their time or the vast majority of their time doing stuff that really matters to customers. And the way that this is often. Time’s done is by involving what I’m gonna call single responsibility teams, that have a service that they’re gonna supply.
Mary & Tom: And if you think about it from a service orientated architecture point of view, you can tell that service orientated architectures are the foundation of the cloud. And they’re the thing that allows vast majority of work to move through the cloud in a very efficient manner without bottlenecks. And it’s the same concept.
Mary & Tom: The same concept applied to lean manufacturing, the same concept applied to our little solar company. Where you have more demand than you. capacity. You have to figure out how to make everybody who’s doing really valuable, useful work, a way to spend their time doing what they do well and doing it most of their time.
Mary & Tom: And yet, if you look into our solar company, you’ll find it a great number of their really bright people are project managers. And the project manager doesn’t actually make anything happen. He just tries to get other people to make something happen, but you know what computers can do that.
Mary & Tom: You could set up if you have, 15 different things that need to be done in your little startup company, so that any work that moves through your organization will move through rapidly through teams that handle those 15 things and is automatically directed to the next one. If you’ve set it up That way. And if you’re in a competitive environment where nobody can hire enough people and there’s just too much work to do, you’re gonna win because you will be able to get so much more work done so much faster. If you keep your cues short and make sure that your people are focusing on doing good engineering or doing the work that they do almost all the time and it’s set up so that the work flows through these efficient focused teams that do really good in the area that they’re responsible for. A service orientated architecture is a good model for this. Then you’re going to actually be able to scale one whole lot more than any of your competitors.
Mary & Tom: So I think it’s the three things. Are we really focusing on what our customers want and need? Do we have flow rather than great big cues, which do very little, but come up the work at slow stuff down and make customers unhappy. And do we have ways that people can focus with their team on doing this batch of things very well. And that’s their focus. When they don’t have to worry about interacting with other teams because when stuff comes, it should be ready for them to do it. And then when it’s done, they should be ready to pass it on so that it’s ready for the next step. And when it comes to the team, it should not have waited in a large queue.
Mary & Tom: Now, if you wanna imagine this environment, which you’re managing is the flow. How big are your queues? You know what? This is the bottleneck right here. Step seven. . We need to get more people doing step seven, because step seven is where everything plugs up. And if you have decent flow, there will be a bottleneck.
Mary & Tom: The one that gets the biggest cues and you blast that bottleneck. You focus on it. You add some more people or decide that they’re gonna do only half of the work, whatever it takes, then that bottleneck goes away and you’ll get another one. But at that point in time, you’ve taken their cue, whatever length it is and you’ve shrunk it. So the stuff moves through there faster and something else has the longest queue. And if you know about queuing theory and you think about it, and you have these workstations that do important steps on your path, through your company they each have a queue and typically those queues are, the average time it takes to get through that work step times the number of things in the queue. Say you have 20 things in the queue takes two days to get through. Then your average, wait, time is 10 days. And if 10 days is too long, because you’re accepting work faster than, one thing every two days then you have to figure out how to make that one step, be able to do something every day. Because if everything coming into your company needs that step and that step takes two days, then you cannot accept, work into your company more than one thing every two days, otherwise all you’re doing is just building up a backlog and that’s exactly what you don’t want to do. I’m sorry, scrum, but that’s exactly what you don’t wanna do. What you wanna do is figure out where is your bottleneck and then accept work at that rate at which stuff can get through your bottom link. You wanna accept more work, then fix your bottle. Make it go faster and you can accept more work. So you need to throttle the input, work to match your bottleneck process, which that work has to move through. And if you do those things, that’s what flow is all about.
Murray: . So I have seen manager’s attempt to do this, but with very negative results. I’ll give you an example. So I worked for a large telco where if you wanted to get a firewall rule changed, it would typically take 12 weeks. Now firewall rule changes, take five minutes of work to do in your configuration system and maybe if it’s complicated, they might take a day to work out what you gotta do first. But the reason it consistently took 12 weeks was because they took the process that one or two people would do, and they broke it up into 20 steps and gave each step to a highly specialized team of very specialized people who were just do that step. And they outsourced a lot of the steps to different companies and places in India and so on. So you had high specialized teams and then each team was, given KPIs focused on efficiency. So they all minimized their staffing and they all did queue management. So reject things coming in by demanding that you fill out enormous paperwork. That was their way of slowing work coming in.
Mary & Tom: So, that’s not lean That is exactly the opposite of lean .
Murray: but they thought they were doing it.
Mary & Tom: OK but remember lean starts with customers. Now in our environment it used to be that the telco would get requests and it would take forever to do ’em. And their goal was to say, how can we have no more than 24 hours from receiving an order to having them up? Because then we get revenue faster and we have much happier customers. So that was the first question they asked. It was a customer focused question. This customer is moving into the area. They need this tomorrow. We are gonna say maximum 48 hours, but we’re trying to gonna get it done in 24. What do we have to do to make that happen? And when they figured out how to do that, they were way more efficient than any other way. Cuz they had no queue management, no send stuff off to India, no, break it up into pieces and manage the flow.
Mary & Tom: It was what can we do to get it done in 24 hours? Then you start thinking about what steps. If I got an hour of work, I don’t need to have four trips. That’s just wasted time. So the logic behind the break it up into small pieces is not so that we can focus people’s energy it’s so that we can solve the customer’s problems. And what we don’t wanna do is have a team that has. All kinds of interactions with other teams and stuff like that.
Mary & Tom: The flawed concept, there is the idea of efficiency as being meaningful. Efficiency is not what you’re after it’s effectiveness and you can be very efficiently ineffective. And if you are ineffective, you’ve totally wasted time.
Mary & Tom: The idea is not to do each step efficiently. The idea is to change the steps so that each one makes a difference in solving the problem for the customer. Now that has two important effects. One effect is it gets the problem solved for the customer quicker. And the other is that everybody working on the problem knows they are making a difference.
Mary & Tom: And that is the most critical thing in making good use of your people, making sure they don’t waste their time, making sure they understand and believe that they’re making a difference for the customer that they are serving. Indeed Ono said that the essence of lean is respect people don’t waste their time. Don’t waste their energy and everything that he taught and everything that has come out of lean since then is about helping people work together effectively to delight customers, both the immediate customers, the next step in the process, the next step on the assembly line, the next step, whatever, and the end customer.
Mary & Tom: And if there’s a conflict between the next step in the process and the end customer, guess who wins the end customer. Because that is what makes people believe that they are doing something meaningful. Now, in the last couple of years, we’ve seen a explosion that has affected the entire world.
Mary & Tom: And that is called the great resignation . Even our solar company loses people because they have people doing a lot of work and not seeing a big difference made. So they go someplace else where at least they can get paid better, but maybe they can make more of a difference. So a lot of smart people around the world are realizing that it matters whether or not your time is wasted. It’s not just a salary.
Mary & Tom: Yeah. There’s many things about lean ideas that are self contradictory. So when I say single responsibility teams I do mean that you have an expert team that focuses on one area, but I don’t mean that you divide it up into such little bitty pieces that they’re like automatons .
Mary & Tom: When you have a service orientated architecture, you have single responsibility services, but you would never divide those single responsibility services up into such little pieces that they’d be terribly inefficient. The reason why you’re dividing it up is so that people can focus on what they do well.
Mary & Tom: So the idea of a t-shirt shaped person or a T-shaped team is that it’s a team that goes very deep into a particular area. They have clear expertise in a particular area, but they also interact very well with the areas around them and beside them. I had cataract surgery in my eyes, that’s why I don’t have glasses on right now. And there’s only so many doctors that do cataract surgery and they do it very well. It’s very rapid, very well known, very safe process these days, but still I would want the surgeon and the team that’s doing it to be really good at just that cataract surgery, not any other surgery, just cataract surgery.
Mary & Tom: Now, I don’t want them to just do this little piece of it. I want them to know me and cover the whole thing and be able to deal with, problems and exceptions. And if I have problems later to be able to follow up on them, but I don’t want somebody that does another surgery doing my cataract surgery.
Mary & Tom: and that’s the specialty that I’m talking about. There does have to be deep expertise when you have very critical stuff being done and how you divide your organization up so that you have the deep expertise, you need, and the cooperation across the teams that have the deep expertise is interesting?
Mary & Tom: The example I like to think about is Amazon web services. They actually take this concept of single responsibility teams as one of their fundamental principles, but a single responsibility team is fractal. So you can have a single responsibility team for a service and beneath them. They can have some that deal with subsets of that service. But at the top of a major service, somebody at Amazon told me. There will be a small team whose sole job is to get that service solving customer problems. And then they can have teams beneath them that do subsets of that. But their job is here’s the customer problem. Your only job, nothing else is to solve this customer problem.
Murray: I wanted to just say what I think was going wrong in that firewall. Situation I was talking about, cause I actually see that sort of problem happening all the time in big corporations. I think what’s happening there is that they’re applying your third principle of single responsible expert teams, but they’re not applying the first two principles of customer focus and flow because the way the management have set it up is that there’s nobody responsible for the end to end flow, nobody responsible for what the customer thinks. They don’t give a damn what the customer thinks. And all the KPIs are focused on minimizing cost and maximizing utilization in each of the single responsible units. So if you just have that without the other things, the result good.
Mary & Tom: Well, focusing on profit rather than customers is a sure way to go downhill.
Mary & Tom: Utilization is one of the worst metrics that’s ever been invented because it assumes that what you’ve utilized something for is worth doing, which it usually is not. It’s a case of sub optimization. The catastrophe that’ve seen in the global supply chain crash that was triggered by COVID is sub optimization.
Mary & Tom: They’ve gone too far. They have not thought to balance the efficient use of resources with the robustness that you need to be reliable. So the things that you see that you described in a large company with this firewall rule change is focused on resource utilization. Now, how many resources are there? There are switches and wires and so forth. Those are the resources. The people are not resources, but you measure them as if they were machines whose utilization mattered. One of the key insights in lean manufacturing is that utilization of machines is a bad metric, too. It’s not what percent of the time you have a machine active doing something that leads to huge amounts of waste and unwanted outputs.
Mary & Tom: It leads to huge inventory costs. It leads to tremendous amounts of waste. When you focus on utilization. You don’t focus on utilization. You focus on throughput, you focus on what comes out at the end. And that metric is a metric that works because that’s the metric that generates revenue. That’s the metric that delights customers. If everybody focuses on throughput rather than utilization of for an expertise or a machine or whatever, you’ll get much better results.
Mary & Tom: One of the key lean metrics, is you take the amount of time that it actually takes to do the useful work necessary to get something done. And you figure out what percentage is that of the total time from the time the customer had the problem until the problem was solved. And you’re looking for that to be better than 50%, which it almost never is. If you look at any inefficient organization, I don’t care how efficient they pretend that they are they’re gonna have that number down below five below one. . And that was what you were just describing to me.
Mary & Tom: Your firewall example was probably one 10th or a hundredth of a percent.
Mary & Tom: And that’s the fundamental lean metric is when you’re talking about getting valuable work done. What percentage of the overall time from customer annoyed to customer happy. You need to figure out how to get that metric up high into the, double digits for sure. The best way to annoy customers is to have them give you a job, which they have a fairly good idea of how much work it takes. And have you take 10, 20, 30, 50, a hundred times that long to actually get it done. That is not lean.
Shane: So I work a lot in the data space. And what we’ve seen in there is this idea of hyper specialization, we’re seeing people that had great skills, get their roles broken down more and more into hyper specialized skills and lots of handoffs and lots of very small moving parts and lots of inefficiency. And I think you just articulated an anti patent for lean, which is that breaking all the work down into two smaller chunks where the cost of handoff is actually higher than the cost of doing the work.
Shane: So I have a theory that the reason we don’t see a lot of lean adoption is because actually you have to learn something, there are things you need to know. You need to know about how big the queue is, how long it’s taking, constraining what’s coming into the queue, focusing on removing the bottlenecks. Yet with scrum, we can, read a couple of principles and have a kumbaya and all play together. What’s your view on why lean isn’t as dominated?
Mary & Tom: Lean has internal contradictions. It means that lean is not a silver bullet. It’s tough. You have to think, you have to understand how it applies in your area. On the other hand, scrum is a silver bullet. And so you have scrum being adopted by silver bullet seekers and lean being adopted by people who wanna figure out how to get their hands around customer issues and energizing employees, and doing really, effective work as opposed to following somebody else’s textbook.
Mary & Tom: I don’t say that about all of agile, cuz I don’t equate agile with scrum, but I definitely say that scrum is a textbook for a 1990s problem and it did a sort of half baked job of solving a 1990s problem. But it’s not something that solves today’s problems and people who are trying to use scrum today. I’m sorry that there are definitely the laggards and they’re way behind the times. And, I hope they don’t have a whole lot of competitors because their competitors will probably be smarter than that.
Murray: I wanted to ask you about. Quality Management, because when you think about lean, it’s always quality managers talking about it. And yet quality managers don’t tend to know anything about software engineering.
Mary & Tom: So what is the job of a quality manager?
Murray: Apparently it’s to make sure that the company has standard processes for everything that I follow consistently so that you can pass your quality audit.
Mary & Tom: Thats not a quality manager. In the lean world quality is something that must be built in at the time of building something. . You don’t have separate managers worrying about it. In our manufacturing plant, when we did just in time, the most important thing was that we stopped having the quality department responsible for quality and had the individual production areas responsible for the quality of their own product and the quality department helped them to do the necessary testing and other work to make sure that their work was up to the quality that was necessary.
Mary & Tom: So I don’t understand the job of a quality manager, but sure as heck is not telling other people how to run their processes.
Mary & Tom: The standard process is a alternate way of implementing command and control management. It has nothing to do with agile. In lean manufacturing standard processes are important, but that’s not a contradiction. The standard is the best way we know how to do it so far as a baseline for figuring out how to do it better. It’s not the goal to do the standard process. This goal is to improve continuously the standard process, the standard way of doing things. Now that is in a manufacturing world where the same thing gets done repeatedly.
Mary & Tom: Mary & Tom spent a lot of time in the manufacturing world and she knows damn well that even in the manufacturing plant, you don’t do the same thing repeatedly. It just, the raw materials change, the formulas change, everything changes continuously and you have to continuously adapt how the work gets done in a design world.
Mary & Tom: There’s never a repeated thing. It’s always unique problems that need to be solved. And the idea of a standard process, except for small pieces. How do you go about deploying something when you’re ready to test it out, perhaps? Is not a helpful concept. It is a way of generating a lot of waste, a lot of waiting, a lot of useless activity, a lot of frustration, a lot of quitting of your people who can’t stand it anymore.
Mary & Tom: Furthermore, in a lean world, workers are responsible for the quality of their own process. They don’t have somebody else telling them what to do. The idea that there is a standard process is insulting to people. The idea that people figure out how to do what they do better and better. All the time is much more respectful of people and Taichi Ohno once got really annoyed. He was going through a factory and their standard work procedure up on the walls was a month old. And he said, what aren’t you improving this process every day? Why aren’t you having constant changes to what’s up here on the wall. You shouldn’t have the same thing you do all the time.
Mary & Tom: You should be constantly making it better by doing experiments and figuring it out. The people that are at the workstation doing the work are the ones responsible for figuring out what’s the best way we know how to do it now. And then how do we do it better? And what experiments can we run to figure out how to do it better?
Mary & Tom: And the constantly figuring out how to solve the problem of how can we, do things better measured against, are we making our customers happier? Are we solving their problems faster? Are we making our work safer? Are we making our work something that we can do easier? Those are the things that you’re looking for and the people who know how to do that are the people who are doing the job, not somebody else.
Mary & Tom: So that’s why I asked you, what’s the job of quality manager.
Shane: So isn’t it again, another case though, of people taking an idea and implementing it badly. So I’ll give you one, one of my pet peeves is the idea of best practice it’s just bollocks. I see that there are valuable patterns that given a certain context adopting that patent to start with may have value.
Shane: A quality manager changing from being a commander control, at the end of the line, testing the product, when it comes off to becoming a coach that helps the line, understand what they can do to increase the quality, but making the people, doing the work in charge of it. I can align that with the whole DevOps practice of test first and then write code second. So I can see some practices and patents being adopted in the software world, out of that manufacturing world. Again, I can see some abuse, I can see the idea of best practice and this is the way we standardize the way we develop software and going to that sub optimization, area that you talked about.
Mary & Tom: most of the things people think they’re adopting outta manufacturing come from a very bad understanding of what manufacturing’s actually like
Shane: But there are patterns in manufacturing that we can adopt software.
Mary & Tom: Principles.
Shane: Principles. And some patents, some ways of working that may have value given a context or do you not agree? Do you think that there is no reusability.
Mary & Tom: There are lots of very successful lean companies in the manufacturing world and they are perfectly happy to have anybody that wants to copy them, come and look and see what they do. And copy exactly the practices that they have. The reason they’re very happy is that those practices don’t make any difference to their copiers because those practices are the best solution that the good company has been able to figure out to the problems that they themselves are facing in the context that they themselves are competing with the people and the machinery and everything else that defines their business.
Mary & Tom: And if anybody else tries to copy the practices, it will have marginal impact. What they need to copy instead is not the practices. The practices are not what makes the difference. It’s the way of thinking the way of analyzing the way of constantly improving in your context to solve your problems with your people, your background, your history. And that will make a difference. That’s what needs to be copied, not practices, not procedures, not processes,
Murray: so I worked with a very famous lean manufacturing company who I won’t name and in their office, in their it department, they were old school command and control, heavily documented processes, highly centralized processes. They talked about some of the lean ideas. They had this one about how everything has to go up multiple levels and everybody has to agree to it.
Mary & Tom: Thats not lean
Murray: Anyway, my point was that they hadn’t been able to make the transition from the lean principles in the factory to applying the lean principles in knowledge work. They just could not understand it at all.
Mary & Tom: Correct. I’ve seen that too. I think we’ve been in that company. And if it’s the one I’m thinking of, they were very successful at translating the ideas, the ways of thinking to their engineering work, but not yet to their software work when we were there, but that was 10 years ago.
Murray: I think it’s still the case. And one of the things that’s really very different about software is that there’s a very high degree of uncertainty in everything you do. Like you said, Tom, everything’s new and it’s a design process. So there’s a lot of uncertainty about the requirements, the design the code, the implementation. And that requires a very different way of thinking because it’s not all the same thing all the time.
Mary & Tom: Yep. I have found that people that try to apply stuff that might work in a operations environment to a development environment, if they try to copy practices, virtually always fail. The only thing you can copy is the principles behind the practices, not the practices themselves. One of the places where I have seen people try to apply lean principles is in hospitals.
Mary & Tom: And we have some hospitals that have been very successful and others that have not. And the ones that have been very successful are the ones that have figured out how to let the people there given a few basic principles, figure out how to make their work better, how to not have errors in the operating room, how to not have errors in the pharmacy, how to get the pharmaceuticals to the bedside rapidly and securely.
Mary & Tom: And it’s going to be different if your pharmacy is in a different location, if you have different kinds of people, the kinds of errors that one environment will generate is very different than the kinds of problems and errors that another will generate. So we have this concept in operating rooms of a checklist is a very good concept.
Mary & Tom: It’s works in airplanes and it works in operating rooms and all that, but there’s no standard checklist. Each hospital tends to have a different set of things in detail that they’re going to check out before they start surgery, depending upon the surgery and the surgeons. And the way that they get the doctors and the nurses to go through this checklist will differ from one hospital to the next. And the successful ones will have figured out how to get their own people, to understand it, buy into it and get it right rather than have somebody from outside comes and does this to us.
Mary & Tom: And it’s only when people figure out how to optimize their area with their environment and their problem. So you take the people to there and you respect that they’re intelligent people. And they can understand the problems and they can understand the customers and they can understand the concept of flow and they can understand what constitutes good work, and they can be engaged in doing something for a purpose.
Mary & Tom: If you’d only let them, instead of telling them here is exactly what to do down to every last detail and you just better do it and do it fast and efficiently. How many people wanna work in that world.
Mary & Tom: That approach just simply cannot work because things change. The foundation on which you plan is going to be incomplete. And the only people that are close enough to perceive that there’s something that’s changed. That there’s something that’s been overlooked are the people on the front lines, doing the work themselves. If they’re the ones that develop their approach to doing the work, if they’re the ones that define the practices.
Mary & Tom: Then they’re the ones that know when it can be improved to deal with changes that have happened or oversights that have been
Shane: The example of a checklist is a perfect example of a pattern. Where we say that given this context in the past, other organizations have applied a checklist patern and it. Given them some value. So experiment with it. It doesn’t say here is the checklist. Here’s the form of all the things. It’s.
Mary & Tom: I do not wanna get in an airplane if the pilot isn’t going through a checklist
Shane: Yep. so it’s a patent that has value in a certain context, but picking up on Murray’s point in the software world. We often have this theory that everything’s complex and everything’s never been seen before. And I don’t agree with that, or I agree that there are a bunch of patterns that are table stakes that you should look at first. Don’t just put them in, but you should experiment with them. You should see whether your context, they have value. And then if they don’t replace ’em with something else, if they do then iterate on them to do something that’s better.
Mary & Tom: Yes. There are lots of good ideas out there, but they can’t be just taken over to your area and poured in. They have to be taken into your area and understood why they work and then try to figure out how to make the same kinds of things work in your world. And that’s especially true in a design environment.
Murray: Yeah, I’m a big fan of pat and Shane, you know that cuz we talked about it before. And I think that we should try all these different patterns and see what works for you. And sometimes they will work in your situation and sometimes they won’t. So that’s why I’m opposed to a highly centralized way of working like safe, for example
Mary & Tom: Huh. Have you see SAFE been successful anywhere? I have yet to.
Shane: There’s some training companies that are highly successful teaching it.
Murray: it’s a great way to make money for your consulting company.
Mary & Tom: yeah, but that doesn’t count. How about actually using it?
Mary & Tom: It’s also highly successful at preserving the way you always used to work. Just renaming parts of it. It. preserves the same hierarchical structure just gives it different names.
Mary & Tom: If you need to scale, you have a system architecture problem, and you have an organizational architecture problem since, based on Conway’s law they’re basically the same. And unless, and until you attack your architecture problem. As for example, Amazon did, when it finally moved to services, you’re not gonna solve the problem. And using some great big standard process to paint over it. All it does is delay your attacking the essence of your problem. The essence of your problem is not gonna be solved by more process. It’s gonna be solved by some other way of breaking up your work and thinking about it completely different, like from a different angle, as Deming said, think of your system, it’s not working for you. Then you have to change how the system works and safe is not doing that. It’s giving people cover and distracting them from the real work, which is to fix the architecture. If you have a scaling problem of any type, you have an architecture problem. There’s a lot of companies out there that are way bigger than you that have no scaling problem. How is it that AWS has pretty much wiped out the enterprise software world with services? How did they do that?
Mary & Tom: They’ve introduced 10, 12 major software services every single year in and year out. I go to AWS reinvent and it’s 12 or 15 every year that are really major new changes. And every year they’re enterprise scale, they work, they don’t break down and they solve real problems and they start gathering whole bunches of enterprise companies. It happens time and again, and they’re not the ones that say, how do we scale? . They don’t say that because they figured it out. They have single responsibility teams that have a defined customer focused problem, and they’re supposed to be obsessed with that customer and their problem.
Mary & Tom: And they have ways to make sure that the teams do not have to interact with each other, rather than having a process like safe to help them figure out dependencies. They don’t have dependencies. So the single responsibility teams can do their thing without having dependencies on other teams. And they have more software used by more companies at this point than you could ever have dreamed of back when people were saying, how do we scale?
Mary & Tom: How you scale is not with some new process that’s gonna help you manage your dependencies. It’s gonna be rethink your architecture guys, cuz it ain’t working for you. And that’s what people have to do. If they think they need something like safe, they have to say, why is it that we even think we need something to scale?
Mary & Tom: Because we’re doing individual things through our process rather than doing, containers of work flowing through our process. The concept of moving work through something like AWS is basically orthogonal to the concept of moving work through a process like safe completely hundred and, 90 degrees, the opposite direction.
Mary & Tom: And so companies that think they need to do that are simply not even beginning to grapple with the real problem that they have. And they say, how can we transition? How can we transform? Stop thinking that what you have is the only process in the world. Start thinking completely different about how work is going to flow through your organization. And get done by focusing on customers, thinking about how do I move groups of stuff through my organization, rather than having every individual thing have to be shepherded by somebody and go into Q after Q.
Mary & Tom: There’s a book called working backwards, which talks about how they organize in order to make that happen. And they get scale. Google gets scale too, and they don’t do it the same way. I think space X scales pretty well too. They have pretty much blown the whole rocket boosting industry out of water by completely rethinking how you do booster development. So those companies have very large, massive problems and they have. Completely rethought the mechanism whereby they do it. And what they tend to do is they tend to create teams with single responsibilities that own their responsibilities.
Mary & Tom: So in a service orientated like AWS, you will either own a service and there will be a service leader that organizes that team and they own responsibility for it. And they may have subteams and those subteams may own responsibility for a subset of that. And that’s actually the same way that SpaceX is organized.
Mary & Tom: It’s organized by the components of the booster. And there are teams that own complete responsibility for a component. There’s not hardware teams and software teams, they’re component teams and all the hardware and software and fuel and all that sort of stuff necessary to make this component work. That’s the responsibility of that component team. And there may be sub components. And that’s what I’m thinking about with single responsibility teams. They own responsibility to make one thing happen well, and you move work through these teams rather than thinking about how do I get this thing that customer wants through all the steps that it has to go through.
Murray: But they must have people or teams who are responsible for the end to end process of the value stream that who are looking at the customer and making sure that things are flowing as it goes from one team to another.
Mary & Tom: You definitely have people looking at what a customer is trying to achieve. There’s no question about that. That is the fundamental thing that they do. They say what are our customers asking for next? . And then there will be a person who owns a responsibility, take that customer problem and come up with a solution for it. Absolutely. And then that person will form a team and that’s their job come up with a solution for it. But the way that they come up with a solution for it is that they figure out how to. Take existing services and put them together in a stream or they figure out what new things have to be done. And then they will charter sub teams to make those things happen.
Mary & Tom: But in the end, they do not end up with a great big team around that service. The team around the service will be relatively small. And it will have that responsibility to solve that one customer problem. Now, if there are other teams that have figured out how to do the database portion of that, or the, whatever other portion of that, they will lean on those already existing services.
Mary & Tom: And they will hand off responsibility from one team to the next. And they’ll say, here’s where we need this database service to do this. And here’s where we need this other service to do this. Here’s where we need to develop a service, cuz it doesn’t exist. Those teams are chartered, always based on this customer needs this thing to happen and it’s not something we do right now.
Mary & Tom: Figure out how to solve this category of problems for this customer. And there is a single responsibility leader. Who then assembles a team. Who’s one and only one job is to solve that customer problem. That’s how they get stuff ready to announce once a year. Cause then the person says, we want something ready to solve this particular customer problem that we’ve identified nine months from now, when we have our reinvent conference, figure out what it takes to solve this customer problem. Here’s instances of it. Here’s customers that are willing to work with it.
Mary & Tom: You can work with all of the other teams, as long as you don’t get them as part of your team. You use their services with a proper interface and figure out how to solve that customer problem. If you need to have people in your team, be responsible for sub teams. Fine. You have the charter and the responsibility to solve this problem and do what it takes. And that’s the only thing you’re gonna focus on nothin else between now and when you’ve done something. And we’re not gonna tell you what we’re gonna have that customer tell you if you’re solving their problem.
Mary & Tom: And that’s how they charter teams. That’s the equivalent of safe in AWS. That’s how they do it.
Murray: Alright. So couple of closing things, I should say, when I was talking about quality managers before Shane, I was not talking about test managers. I was talking about, manufacturing type of quality managers. And what is their goal? Apparently it’s to pass quality audits, as far as I can tell as their main goal.
Mary & Tom: my manufacturing plant it wasn’t
Murray: It’s not what it should be, but yeah. Alright. So just in closing then, what is it that software development people and agile people most misunderstand about lean.
Mary & Tom: They view it as a set of practices rather than a way of thinking. It’s easy to copy practices. It’s not so easy to think differently. It’s not so easy to think first about customers to think first about. Respecting your people. It’s not so easy to think about. Do we really need to do this? That we’ve always done copying practices, putting cards on a wall, having lots of meetings, all of that stuff.
Mary & Tom: The practices may or may not have value, but it’s very easy to copy. It’s very easy to measure. The wrong measures will give you wrong results every time. When you measure things like utilization, you’re going to be led to very bad decisions. When you measure things like how many stories did you do? Output measures like that? You’re going to end up with lots of stories that are not worth anything. If you measure how much impact does this have on the customer for. Then you will have a difference. Way back in the eighties, Tom Gilb was writing about this sort of thing. He was saying, you start out with no more than 10 business metrics that you want to impact .
Mary & Tom: And the measures of the impact on those business metrics are the measure of the success of your project, of your work, your product work. And then as you decide what to do you look at how much impact do you expect to have on one or more of these metrics and your progress metric is how much have these metrics improved.
Mary & Tom: So it’s been known for a very long time that. The impact on the people that matter is the most important metric of all until the financial people got ahold of it and started introducing all kinds of other metrics that lead to bad decisions that lead to tremendous amounts of waste.
Murray: That was a good answer. Thanks, Tom. Let’s go to summaries then. Shane what have you got.
Shane: So we started out saying that, queues are there for the people doing the work, not optimize the work for the customer and that, was an insight for me. And then we follow that up where says, if you’ve got a queue stop accepting work, because now there’s a bottleneck. And so don’t accept work until the bottleneck’s gone and then start focusing on how you remove the bottlenecks.
Mary & Tom: Don’t accept work at a rate faster than you can get through the. It’s the rate that matters not is the queue going. So it’s pace thing.
Shane: You’ve got a queue stop getting new work, fix the queue problem, and then figure out the rate of the queue and then start accepting work to match that.
Shane: The next thing for me was it’s not about work we’re doing, it’s about doing the right Work. So, what is the problem? Are we solving it? That’s what we should focus on. It’s not about a marketing team. It’s not about a software team, it’s about a product team.
Shane: And, there are a bunch of themes that come out around set goals, make sure everybody knows what they’re working towards. Look at the value that we add, not the tasks that we do work together with cross skilled people to get the job that needs to be done, but don’t have big teams experiment, early, get feedback all that product thinking come through.
Shane: So those are the core themes that I really like. And it came back to this idea that everybody who’s involved, we wanna focus on them being more efficient in the work they’re doing so they can add more value to that goal, that customer that they’re working for.
Shane: So that’s what we’re focused on. I like the three principles. Solve problems, remove queues and provide focus. focus on the customer and their problem. Don’t implement their solution, implement a solution that solves the problem.
Shane: And the reason we wanna remove a queue because every piece of effort we do to manage the queue is waste it’s overhead. It’s just moving the widget. So remove the need to do that. , I still struggle a little bit with the definition of batch work versus flow. Cuz for me flow never seemed to be batch orientated, but I liked your example of a shipping container. That sometimes we can batch work where it increases the flow of the work across the organization, because we’re not dealing with individual things we’re automating or moving multiple things at the same time.
Shane: Don’t focus on Utilization focus on the value we provide. That’s a core theme that’s come through for a while. This idea of lean and factories. So you still gotta think, you still gotta solve your own problems. There are just a bunch of things other people have done that may get you started earlier and they’re useful before you start iterating.
Murray: So for me the three principles were very clear and a nice, concise sumMary & Tom of the way to look at this. Focus on the customer’s goal, the outcome, very aligned with user experience, design thinking and modern product thinking. Second one focus on the flow, manage the flow. Think about, how long does it take to go from end to end? What is your flow efficiency. If something takes one day to do it, shouldn’t take more than two or three days in total. Whereas in most organizations, if something takes one day to do it’ll take a year to get completed. so flow efficiency and you need somebody, or maybe that’s management’s job is to focus on flow efficiency.
Murray: And then the third one is responsible specialized teams that can do each of their part very productively very much aligned with a microservices architecture way of thinking. So that was all good. When you started off, you said that these things are table stakes, in the organizations I talk to, they’re not doing any of this or very little of it, or they’re very confused about it and doing it in a bad way. So even though things may have moved on, most big organizations are quite a mess inside. And I think it’s mainly because of the way executives think about. Work and their focus on certainty and utilization and cost. Maybe it comes from their finance and accounting people. I’m not quite sure
Mary & Tom: Are you talking about it organizations?
Murray: a, range of organizations that build software products and organizations that, do banking and finance and insurance and telephones.
Mary & Tom: How are they measured? What is the priMary & Tom metrics of the people who are driving things in that direction? Are They measured on cost?
Murray: They tend to be measured on cost. Yes.
Mary & Tom: So fundamentally they’re operating like cost centers.
Murray: Yeah. And very often software development is, or software engineering is a cost center. It’s got no revenue attributed.
Mary & Tom: Yeah. And that’s just wrong. That’s a nice 1970s concept. But that’s not today. When software is the fundamental essence of most strategies of most companies, there is no way it belongs in the cost center.
Murray: I agree.
Mary & Tom: That’s the fatal flaw right there cost center thinking is what leads to all these other inappropriate metrics. All these other bad behaviors, bad decisions, all this other waste of people’s time.
Mary & Tom: When the executives are measured based on cost and utilization, there’s no way that everybody else isn’t going to be too. They all just pass it down. So I think the fundamental issue to think about is what are the priMary & Tom metrics. Everybody’s a person and they have metrics and expectations that they are responsible for. Even if they’re a manager or an executive, there are metrics. What are the metrics that the people who are making the decisions are responsible for? And then you’ll figure out why they’re behaving the way that they are.
Mary & Tom: Lean is not something that starts at the bottom of the organization.
Mary & Tom: It’s not something that starts on the factory floor. It’s a mindset in the executive suite about how to communicate, what matters to the organization all the way down. And it’s not by cost center management. It’s by creating
Murray: yeah, I agree with you. So that’s what I got out of it and some of the problems I see in organizations
Mary & Tom: I’ve never actually had a really good experience working with cost centers.
Murray: you haven’t.
Mary & Tom: Nope. Never
Mary & Tom: because the way I think doesn’t match the way they are required to think
Murray: We can all agree. It’s all the accountants fault. There
Mary & Tom: There are plenty of companies in the world that don’t think that way. And they’re the ones that everybody is looking at jealously and say, how do they do that?
Murray: there is a whole beyond budgeting movement. Maybe we should get somebody from there, to come on and talk to us.
Murray: So thanks very much for coming on. How can people find you?
Mary & Tom: We are retired, so they’re not gonna get too much from finding us, but we have a website poppendieck.com. It points to what I call essays, it’s essentially a blog site on blogger called lean essays.com. And it’s the last 20 years of stuff that I’ve written. 60 plus blogs. And that’s pretty some, free sumMary & Tom of the stuff we’ve been thinking about over the years.
Murray: Yeah. All right. Well, thanks very much for coming on. We appreciate it.
Mary & Tom: well, thanks a lot for having us
Exit: that was the no nonsense agile podcast from Murray Robinson and Shane Gibson. If you’d like help with agile contact Murray evolve, that’s evolve with zero. Thanks for listening.