Project Management

I recently had a revelation: the first time that I worked with a Project Manager — a person whose sole role is maintaining a schedule and coordinating the tasks on that schedule — was 2002. For nearly 20 years of my career, I worked on teams where project management was a subsidiary role of the team lead or development manager. True, my career has mostly been spent at small companies, some that couldn't afford a dedicated project manager. But there were also a few larger ones — including GE, which you'd expect to be a bastion of project management and rigorous checklist checkers.

Before continuing, I want to say that, unlike many developers, I don't disdain project management per se. I've worked on projects that have succeeded (or at least failed less badly) because a talented project manager pulled together people with diverging goals, people who might have otherwise ignored or actively undercut one-another. I've also worked on projects where the project manager seemed to be actively inflaming the participants. Either way, it's a role with impact, one that cannot be ignored.

So why did I spend two thirds of my career without every seeing a project manager? I think the answer is that the structure of software development organizations changed over that time, along with the companies where they reside. And that's not necessarily a Good Thing.

But first, a little history. Corporate management, as we know it today, didn't exist before the mid-1800s. Prior to that time, business were small and generally confined to a single location; a few hundred employees was an industrial giant. The railroads changed all that: they hired thousands of employees, for a myriad of functions, and those employees were dispersed across the thousands of miles of terrain served by the railroad.

Up to that point, management relied on instant, face-to-face communication between front office and factory floor. This simply was not going to work for the railroads. In response, they adopted and adapted the hierarchical structure of the military, and even some of its terminology. The corporation was now composed of semi-autonomous divisions, which took strategic direction from the home office, but had freedom in tactical operations. Each division had its own complement of functional organizations such as maintenance shops, and those functional organizations kept largely to themselves.

This model worked well for the railroads, and for the giant industrial corporations that followed. You can even see the functional structure embodied in the layout of a manufacturing plant. And it permeated the thinking of the people working for those corporations: at GE in the late 1980s I received a five minute dressing-down from a mid-level manager, for daring to use a photocopy machine that belonged to his group. Even in the software industry, the hierarchical mindset prevailed: as you read The Mythical Man-Month, you won't find “project” managers, just managers.

So where do these project managers come from? I think the answer is construction.

Whether you hire a general contractor for your home remodel, or Bechtel for a billion dollar highway project, you get a project manager. And they're necessary: the construction industry is fragmented into dozens different trades and specialties within trades, even at the level of home repair. Carpenters, electricians, plumbers, masons, sheetrock installers, painters, tilers, landscape designers, and so on … you need all of them, and none of them do the other's job. And more important, each works for only a small part of the project schedule, and then they're gone. And if they don't start at exactly the right time, the whole project gets delayed.

It works for construction, so why not software?

In the 1980s and 1990s, corporations started to adopt “matrix management.” The reason was simple economics: self-contained organizations waste money. Just as you wouldn't want to pay a sheetrock crew to sit idle while the carpenters are building stud walls, most organizations don't want to pay a DBA to sit idle while the developers write front-end code. So the DBA team gets matrixed to the project team: when the project team needs a DBA, one will be assigned.

From the company's perspective, this maximizes employee utilization. And from the DBA's perspective, it's a better career path: rather than being isolated in a product-specific development, she gets to work with her peers and have her work recognized by a manager who doesn't think that mauve databases have more RAM. Everybody wins.

But something I noticed, when working with matrix organizations, is that you could never find a DBA when you needed one — or, as matrix management spread, any other specialist. They always seemed to have other projects demanding their time. Perhaps that was really true: for a corporation wanting to reduce costs, why stop with sharing people, why not understaff as well? But I also noticed that you could always get a DBA to turn up for meetings where there were donuts present.

And what I inferred from this is that matrix management creates a disincentive to project loyalty. After all, the specialist career path depends more on pleasing the specialist manager than the project lead. In the best of cases, specialists can cherry-pick projects that catch their interest, ignoring the rest. In the worst cases, there are lots of places to hide in a matrix organization.

This effect goes deeper than a few DBA's, however. In a fully-matrixed organization, project teams are ad hoc. You no longer have developers who are working on a product, they work on a project. And when it's done — or fails — they move on to another project. Taking with them the in-depth knowledge of what they did and what they should have done. Long term loyalty simply doesn't exist.

And with the creation of ad hoc teams, you need an ad hoc manager: the project manager. So to reiterate, it's not that project managers are bad per se, it's what their presence says about the organization that disturbs me.