I couldn't sleep last night so I started reading Tufte's The Visual Display of Quantitative Information, and tried to think if there were any stats that would be useful to a software development project, and if so, how those stats could best be shown. FWIW, my recent experience is working on one (1) software project for the last 4.5 years, so I feel reasonably well qualified to think about metrics in this area.

Possible graphs

Some of the statistics I thought about are shown below. In this group I'd have a Y2 axis on the right of the graph that would show the number of developers, or maybe overall team members.

• Source lines of code (SLOC) per day
• Total SLOC
• SLOC churn/day
• Bugs opened/day
• Bugs closed/day
• Total bugs
• Project cost versus time
• (Many more deleted here; you'll see why)

(If you're read some of my writings before you know I don't believe in SLOC, and do believe in Function Points combined with complexity factors. I use SLOC above to simplify this initial discussion.)

I considered creating all of these graphs, and many more, but -- I don't think any of them are important. I believe Mr. Tufte would call them "chartjunk".

What stakeholders need

Having worked with project stakeholders very closely over the years I think the only important questions from their perspective are:

• Are you done yet?
• If not, when do you really think you'll be done?
• How much is this going to cost?

If you agree with these questions you can skip this paragraph. If you disagree with me, think about it this way: You walk into the car repair place, and they've had your car for two days already. What questions are you going to ask them? Now look at those questions again. Project stakeholders are incredibly vulnerable when it comes to software development projects, much more vulnerable than you are walking into the auto repair place (at least in terms of cost). It's all about trust. They're basically saying "Here's a big pile of money, please don't screw me." When they keep hearing things like "Just two more weeks" over and over again, pretty soon they know they're getting screwed. And once they lose confidence in you you're gone, and they're going to find someone who really knows what they're doing.

One graph that is relevant for software development projects

Now, given that those are the only important questions to project stakeholders, what metrics are really relevant?

IMHO the only thing you can do -- get ready for it -- is use Function Point Analysis (FPA) to make an initial estimate of the project size, add in the complexity factors, and then prepare a graph at the end of each development iteration showing the functionality delivered versus the overall estimate, like this:

  ^
  |
e |
s |
t |
i |............................................. 
m |                 (original baseline estimate)
a |
t |
e |         .. (current delivered functionality)
  |      ...
  |  ....
  |.. 
  +-------------------------------------------->
                   (time)

Of course with 99% of all software projects what really happens is that you end up chasing a moving target. It's like if you were to start running a 5K race, but then as you're running along the finish line is moved to 6K, 7K, etc. So that simple graph turns into something a little more complicated, like this:

  ^
  |            ............................  (2)
e |            . 
s |       ......  (1)
t |       .
i |........         (original baseline estimate)
m |                 
a |
t |
e |         .. (current delivered functionality)
  |      ...
  |  ....
  |.. 
  +-------------------------------------------->
                   (time)

(I hope Mr. Tufte won't condemn me for these fancy ASCII graphics. :)

In this chart "(1)" refers to the new overall project estimate following a change in application functionality, and "(2)" represents the second change. You continue to show the baseline, but in this case, line (2) is the new current project estimate.

This graph is a lot more like a real project, but, it's what a stakeholder rarely sees. He hears about such things from his goal donors and project managers, but he never sees this in a graph.

And I think this is the only chart that's really needed. It's like Martin Fowler has said about UML: As a trained engineer he thought UML was important, but at the end of the day, did it really contribute to more accurate code getting produced faster? Same thing here. You can have all the fancy graphs you want showing SLOC, bugs/defects, developers -- and all those things don't matter very much. The only thing that matters is "Are you done?", followed by "If not, how much longer?", followed by "How much is this going to cost?"

Summary

Wow, this could easily turn into a Master's thesis, but to keep it moderately short I'm going to bring this to a quick close and be a little anti-Tufte and summarize my thoughts in bullet points. (Actually, these are detailed, full-sentence bullet points, not six-word PPT bullets, and I think he'd be okay with these.)

• Using FPA you can take very educated guesses at overall application size very early in the project lifecycle. Please, please, please, take those "off the cuff" estimates and put them in your back pocket, and use a technology like FPA.
• Note #1: Using FPA can be very fast, and usually accounts for less than 1% of a project's overall budget.
• Note #2: If you use FPA over time at your company you'll also know the average scope creep of previous projects at your company. (Hmm, that could come in handy ...)
• Now, regarding that second chart, yes, this trend happens on 99% of all software development projects.
• No, there isn't much the industry has to offer to keep this from happening. Users touch the software and realize what they thought they wanted isn't what they really want. (And, as the Scrum guys will tell you, software development is an empirical process -- it has to be this way.)
• While this makes it very hard to accurately predict total cost and project duration -- especially on very large projects -- people that run projects have to try, it's the only way you'll get better and give the sponsors what they need.
• For project sponsors, the people that can be your best friends at times like these are experienced goal donors, project managers, and developers. And when I say "experienced", I mean experienced on a project of this size and complexity level. They should already know that you don't really want what you said you wanted, and they should be able to use their best practices to try to get through that problem.
• You need to mix in less-experienced people on these projects so they can gain experience for the future.

Afterward

From what I've seen the Scrum guys have things about right. They have stand-up meetings every day where they ask developers to answer three questions that are remarkably close to what I just wrote:

• What are you working on?
• When will you be done?
• What impediments are in your way? (Roadblocks the Scrum-master can clear for them.)