I’ve lived through several management or computer programming process fads; Six Sigma, XP, Lean, Agile, Kanban, TDD, and many others. They come and go, but the outcome is the same. Applying a new process either fails for the same reason the old process failed, or it allows people the reset to restart development.

The new failed process

I have a ScrumMaster certification, which I find worse than helpful. There’s no better way to ostracize yourself in a group than to know how things are supposed to work. Most people don’t even bother to read the Agile Manifesto before they buy into it, and then they don’t review it to check that they haven’t fallen into old habits.

Every place I’ve experienced Scrum does it wrong. That’s a different post, but the basic idea was that in an environment with rapidly changing conditions, the development team should organize itself. That’s too scary for managers, so they use it as just another way to assign tasks and expect them to be done by the end of the sprint. These are the same managers who weren’t performing before the change, but someone expects them to perform after.

In the short term, the new process is rocky because it’s a new way of doing things and people need to coordinate their interactions and draw lines of responsibility (yes, even though you shouldn’t have those). Then, there might be a couple of good weeks before people start falling back on old habits for the same reasons they did them before. See Change Is Hardest in the Middle, also known as Kanter’s Law.

Restarting development always shows benefit

A new fad can help a team look at things in a new way, discard things they held too long, and embrace new ideas.

But here’s the crux: any new process will allow that. The time you take to decide to develop in a new way is the same respite you could use to improve your current process.

You probably know someone who’s gone on a diet. They probably saw some immediate results. But, they’d see some results no matter which diet they chose, including those diets that are harmful in the long term. It’s not the particular diet that is helpful; it’s the change away from previously bad behavior. See the meta analysis in Comparison of Weight Loss Among Named Diet Programs in Overweight and Obese Adults doi:10.1001/jama.2014.10397.

The new, new process

Once one process doesn’t play out, managers choose another one. And, these periods might last as long as a manager’s tenure. Change the manager and you often get a new process.

Why did the previous process fail? It worked at Google, or Facebook, or whatever. Why didn’t it work here?

That’s simple: it’s survivorship bias. Those companies would have succeeded despite the way they did it. Their success wasn’t due to their management process. It’s probably more attributed to social connections, accidents of timing, and changes in customer preferences. A friend likes to tell me that Lance Armstrong on a Schwinn Huffy would still beat most bike riders. Although for other reasons, Lance himself has written that its Not About the Bike. The first step in being a champion is choosing better parents.

Consider financial investing; why do some mutual funds do so much better than others? Given some collection of random funds, statistically, some are going to do better (Survivorship Bias and Mutual Fund Performance). You don’t even need to know anything about the funds. The same funds might do worse the next time period; see Long Term Capital Management for an extreme example. Something like 15% of funds survive longer than 15 years. That’s why good advisors tell you to choose index funds, which is the average of how well you’d do over a sufficiently long time period. See Vanguard’s What is ‘survivorship bias’ and why does it matter?.

How many companies used the same management or development process and were never heard of? They are doing it like the surviving players, but they still fail. Process is no silver bullet (nod to Frederick Brooks’s Mythical Man Month). If you knew that 95% of practitioners failed with a particular process, would you still want to use it?

So you fail to improve with one process and change to another. You end up perpetually process shopping.

Why is there process?

Process is something that substitutes for thought. It’s a shortcut. When you’re Captain Sully putting a plane down in the Hudson River, you don’t want to think too much. Pull out the checklists and go through them. Perform from muscle memory the tasks you’ve practiced in the simulators. You fall back to process because you have two minutes. There’s no time for discussion.

That’s one reason for process: highly competent people under extreme stress and time pressure.

I first learned about this when a Top Gun pilot visited my high school. He was the guy who was flying the Tomcat that did a roll off the carrier in that movie (and was quick to point out that he’d get in trouble if he did that roll in normal flying). He showed us his flight checklist, a small, tabbed, and color coded book that is strapped to his leg during flight. If something went wrong, he went to the right tab in the book. There he found the instructions on what he needed to do. Some of the things were in bold; those things he could not skip. If he had time, he went through all the steps. If he was in deep shit, we did only the bold items. You don’t see Maverick or Goose going through their checklists during their fatal flat spin, though.

But, there’s another application for process: low skilled workers. These are not competent people and you don’t hire them to think. You want them to carry out a task just like they are told, with no deviations. Anthony Bourdain mentioned this in Kitchen Confidential: you want the Guatemalans to make the risotto because they’d make it the same way every time, as opposed to the Italian worker who thinks he knows better based on his grandmother’s version.

If you’ve dealt with offshore teams, you’ve probably experienced this. You may have experienced this with onshore teams. At some point in your career, you may have been the problem. They do exactly what they are told with no deviations or understanding.

The process allows new people to contribute. You’ve given them training wheels. Continuous Integration, Quality Assurance, and many other things are process to let low-skilled people to participate.

Consider Frederick Taylor, who applied an engineer’s eye to organizing labor and might even be one of the fathers of management fads in what he called “scientific management”. Arrange humans like a machine and give them very little latitude, and measure everything. He literally wanted to remove thought from the menial work, and assign that work to educated people. Harvard Business School’s first year (1908) focused on Taylorism. You may have heard of Gantt Charts. Gantt and Taylor worked together at Bethlehem Steel (which shut down in 2001 despite all that process).