Software projects often fail to meet deadlines — that seems like just a fact of life. And when you see your own projects starting to get off track, you want to do everything in your power to try to finish on time. Generally, that means you're going to expand the team to add a few more people so you can pump out the rest of your project before the deadline. After all, many hands make light work — right?
Wrong, according to Frederick Brooks Jr, author of The Mythical Man-Month, a seminal text on production in software engineering. There are many reasons why projects get off track, and adding engineers to a project is one of them.
Why is this, and what can workplaces do to actually keep the deadlines they make for their software projects? That's what we'll cover today in our overview of Brooks's eponymous essay from The Man-Month.
Optimism and Estimates
Missed deadlines start before you've even begun working on a project — that's the key takeaway Brooks sets forth in two sections of this essay. The biggest reason for this, especially where software is involved, is that we tend to assume that everything will go well when we plan out our projects.
This optimism gets people into trouble. No project is going to go off without a hitch. Taking a look at what needs to get done and deciding exactly how long it will take based on internal feelings means setting yourself up to miss deadlines.
When creating a deadline, be realistic about the fact that you are going to run into trouble. That isn't to say always plan for the absolute worst scenario, but make a habit of building in hours of troubleshooting and delays when you make your deadlines.
Another reason estimates can fail is that clients want optimistic deadlines met. This can be particularly frustrating if your client is outsourcing software work to you that they know nothing about, and seem to just pull due dates out of thin air.
Software doesn't care when your clients want things done. You need to be in charge of making sure that your clients understand how long something is going to take and how long it takes you to complete the project:
- One way to do this with authority is to use a time tracking tool to build project estimates based on actual hours from other similar undertakings: With a sophisticated tool like Harvest (and its sister app, Forecast), your time tracker may even be able to plot estimates for you.
- Using a project tracking method, like Teamwork Project's Gantt chart, can help you have an easy-to-access file on how long every project took. Tracking exactly how long subtasks take to complete in such a meticulous way gives you a huge amount of data to base future decisions on.
- Establish a good post mortem process to understand how each project reached completion. What roadblocks cropped up? How many deadlines were missed, and why does the team think they weren't able to stay on track? Keep your findings from your post mortems concise and accessible.
When you've looked back at your data and determined what a reasonable time frame is (and curbed your own optimism about your project), then be upfront with your client about whether or not their deadlines are realistic. The truth is that you can't deliver software that's 75% done, so working towards a reasonable deadline is in everyone's best interest.
One of the most important points put forth by Brooks is the relationship between manpower and time.
We think that manpower and time have a linear relationship to each other. That is, if you have a project and you want it to get done faster, you can stick more people on that project and voila! it's finished on time:
This is just not true. The only case where you can be sure that this type of relationship exists is in a project where production for each individual is completely independent — like reaping wheat, says Brooks, not software engineering.
If a task requires that individuals work together to produce the product, things become fuzzier. Large projects may go faster if more people work on them — provided they all start at the same time and there is an impeccable distribution of tasks.
Otherwise, software projects may be hindered by adding extra workers after a project has already begun, because:
- subtasks being readjusted and communicated takes time.
- aligning and training new team members takes time.
- communicating about progress in a systems effort takes more time the more people you have.
Brooks lays out various permutations of how long a project may take for any number of scenarios, but it is sufficient to say that in almost every case, it's not actually more efficient to just throw more workers at the problem.
The more people that need to communicate with one another, for instance, the more connections that need to be maintained—and the more overhead you'll have.
Two people only need one line of communication, from one to the other. As you increase the amount of people on a team, however, the number of lines you need increases faster, creating an ever-growing communications burden:
Mathematically this is:
Your number of connections is the sum of all your nodes minus the number of nodes. So, for three nodes, 3+3 equals 6, minus 3 equals 3. The problem is the first part of this equation increases faster than the second. With just the 33 people/nodes we have at Gliffy, this makes 528 possible connections.
Regenerative Schedule Disaster
The last part of Brooks' essay deals with rescheduling. Although Brooks lays out a much more extensive analysis, let's just take one of his examples and see how it would play out, according to Brooks' principals about time and manpower.
You have a task that would take 12 months for a single engineer, and you assign it to three people. They are projected to finish the project in four months, splitting up tasks and aligning from the start of the project. During planning, you make four milestones, one at the end of each month for four months, with the last being total project completion:
But something goes wrong, and your team takes two months to reach the first milestone instead of one. You're now scrambling to keep the project on track to mitigate your miscalculation and keep your client's, or your own, internal deadlines.
So you want to do what we already know Brooks advises against: add more people. It seems like you could just add one person and still get things done on time. After all, if the original project took four months for three people, it would take three months for four, effectively making up your lost time.
Still, you know that workers can't magically jump into a project and contribute as though they've been working on it the whole time.
Instead, they need to be onboarded to the project, and all the subtasks of the project need to be restructured. You have to get the project done on time, so you'd better just figure out exactly how many people you will need to add to the project:
- You consider that you only miscalculated the time that it took you to reach the first milestone. Given the work already completed and this assumption, you have 9 man-months of effort left, and two months to complete the project, so you add two people to the three already working.
These new people will require training, which takes time out of working on the project. You add four people instead of two to make up for this additional lost time, and you're still squeezing the last three milestones into almost no time.
- Now the task will also require repartitioning and system tests, so you have to entirely restructure the project, adding more time to a project on which you now have seven engineers toiling away. In order to keep the proposed deadline you add even more men, pushing your milestones back and —
— Well, things get out of hand. You should have just redone the timeline in the first place. Instead, you gave into the temptation to throw more and more people at the project, trying desperately to make things faster when they're only becoming slower and more gummed up.
And all this without factoring in anything along the lines of employee satisfaction, putting other projects on hold to siphon engineers into finishing one project on time, or other factors that must be considered when running a business.
And thus we arrive at Brooks' Law: adding manpower to a late software project makes it later.
Systems Testing: It Takes More Time Than You Thought
The last fatal flaw Brooks uncovers in structuring deadlines is underestimating the time it takes to test and debug systems. When coming up with a timeline, people tend to seriously underestimate the amount of time they need to spend planning and testing, ending up thinking they'll spend the most time coding, relative to all other tasks:
But this is not generally a realistic view of software projects.
Brooks revises the breakdown of time spent on each task to suggest something more like this, where:
- Planning takes up 1/3 of your time, the largest chunk
- Coding takes up a mere 1/6
- Systems testing, in total, takes up 1/2 of your time, with equal time dedicated to early and final full systems tests
Debugging takes way more time than we allot for it. It also makes for better software — code that is stress tested and given a run for its money before it needs to be used is going to be cleaner, with fewer errors that pop up unexpectedly.
If you can accurately predict the amount of time it takes you to code, then use that as a starting point to build in more time for debugging and planning than instinct tells you. If you gather historical data on how long various parts of a project take, you may be able to ballpark better estimates for yourself in the long run.
But for now, know that underestimating the time that testing takes — 1/2 of total time spent on a project, according to Brooks — is going to wreak havoc on your deadlines.
Fighting your own instincts
We'd all like to get software projects out on time, but that means being smart about your estimates and your effort, not demanding all available crew members report to a sinking ship. If a project is falling behind, set up a new timeline and don't scramble to add more people. As we now know, number of men working and time to completion do not always benefit each other, but being smarter about your scheduling and the time it takes to get through all steps required to complete a project benefits everyone.
For more of Brooks' essays, or for a full reading of the essay that we've summarized today, check out his classic text The Mythical Man-Month: Essays on Software Engineering, originally published in 1975 by Addison-Wesley and still widely available and totally applicable today.