Is there any reason I'll need to think about this code again in the near future?

If you didn't quite implement all the unit tests you think you should have, the answer is yes and you're not done. If you cut a corner or two to squeeze it in before the end of the iteration, the answer is yes and you're not done. If you're uncomfortable with the implementation and would like to refactor it, the answer is yes and you're not done. If there are a few details on the UI you left out in the interest of time, the answer is yes and you're not done.

Too many times I've seen people close out a task only to revisit it the next week to polish up some loose ends, write a few more tests, refactor a confusing piece, etc. Don't do it.

Software development is hard enough without having to keep mental notes of all the needed tweaks and adjustments to things you've already done. Your goal should be to knock out a task so totally and completely that you can get it out of your brain and focus on the next one. Doing so will do wonders for your sense of accomplishment.

The problem is that burndown charts lack two essential pieces of information. First, how much work was actually accomplished during a given iteration (as opposed to how much work remains to be completed) and second how much total work the project contains (or if you prefer how much scope has increased each iteration).

What you might not know is how flexible they can be. In particular the example I gave last time has a problem. Can you spot it?

The problem is that the scope increase of 20 points in iteration six might very well have had zero impact on the actual deployment date. Those points might have all been low priority tasks that the team will work on after they deploy the initial 100 points. Wouldn't it be nice if the burn up chart could convey priority information as well?

So that's exactly what I did on my current project last week, and it appeared to be a big hit. And with the addition of some Excel trend lines the chart was able to convey a lot of insight into the project:

Obviously all that data is made up, but the chart still tells an interesting story:

• The customer hasn't been adding high priority tasks, perhaps because there is a hard deadline approaching
• Because of the customer's restraint in adding high priority tasks the team can be fairly confident that they will finish all the high priority tasks by Iteration 3
• While the customer hasn't added high priority tasks they have still been able to be agile by adding normal and low priority tasks

What's really wonderful about this way of reporting is that it gives the customer the flexibility to add scope to future iterations while maintaining near term deadlines. In short it supports what agile is supposed to be about.

But why stop there? Depending on the story you need to tell or the scenario you need to evaluate perhaps you could incorporate team member contributions. For example:

In this example we can try to show what might happen if we remove a particular team member or if we'd never had a particular team member. What's unique about this way or reporting is that it's incorporating the backlog's increasing trend into account. While it's possible that the backlog may increase or decrease at a different rate with a different team makeup, it's a better scenario than ignoring the rate of increase of the backlog all together.

Perhaps we could incorporate other information. Maybe bug vs. feature information. Or planned vs. actual (e.g. task slippage) information. It seems like there is a lot of potential. Any other ideas out there? Feel free to post in the comments or via twitter. If there's enough interest I'd be happy to post another screencast similar to the last one I did on how to produce burn up and burn down with SharePoint and Excel except this time with trends and priorities.

So overall while these charts may be a little more complicated than a traditional burn down chart they would make an excellent talking point during a PowerPoint presentation, or as part of your iteration close out. And with a little training I bet just about any customer would learn to love the insight they provide.

It's widely accepted that refactoring is a best practice for keeping code simple, DRY, and maintainable. What doesn't seem so widely accepted is the habit of applying the same practices at a broader system level.

Why don't we refactor systems? I started thinking about this very question during a recent all-day system design presentation, one of several in a series mind you. As I ingested slide after slide of barely visible text accompanied by Visio diagrams packed with boxes and arrows pointing in every direction, I wondered how things became so complicated.

In my experience, there seems to be some kind of stigma around reworking (aka refactoring) interconnected systems. New features are always bolted on to our existing systems, but nothing is ever redesigned, reorganized, or re-anything'ed. Over time, we end up with bolts on top of bolts until the thought of reworking any part of the system is completely overwhelming.

So why is the concept of refactoring an interconnected system so foreign when it's been used so successfully for more discrete pieces of software? Is it possible?

This 13 minute screencast shows you how to create a user stories list in SharePoint and build burn up and burn down charts off of the data using Microsoft Excel. It includes the custom fields you'll need in SharePoint, the formula's you'll have to create in Excel, and shows tips along the way like how to simplify refreshing your reports when data changes.

Once you've watched the screencast if you want to implement this yourself here is the information you'll need:

Custom SharePoint Columns

• Iteration (Choice column e.g. Iteration00, Iteration01, zBacklog)
• Released (Date - the date the iteration was closed out)
• Points (Number)
• Created Date (for demo only, you should be able to use "Created" on your project)

Calculated Columns in Excel

• Total Points to Date
  • The total points on a specific date including backlog, current iteration and all completed stories
  • =SUMIF([Created Date],CONCATENATE("<=",Table_owssvr_1[[#This Row],[Released]]),[Points])
• Completed To Date
  • Total points for stories released prior to current story's release date
  • =SUMIF([Released],CONCATENATE("<=",Table_owssvr_1[[#This Row],[Released]]),[Points])
• Backlog
  • What remains to be completed at a point in time (e.g. don't include user stories added to the backlog in iteration 4 if this is for a task in iteration 03)
  • =Table_owssvr_1[[#This Row],[Total Points to Date]]-Table_owssvr_1[[#This Row],[Completed to Date]]

Burndown Pivot Table Values

• Row: Released
• Filter: Released After [Some Date]
• Values: Max(Backlog)

Burnup Pivot Table Values

• Row: Released
• Filter: Released After
• Values
  • Max(Total Points to Date)
  • Max(Completed To Date)

Summary

I hope you found the presentation useful and if you are currently using SharePoint for tracking user stories please comment on your experiences.

As a consultant (or as a person who is learning someone else's business) your job is to understand your customers existing business better than they do themselves. And you need to be on the same page with others regarding how you are going to change their day to day functions. And that's what business process modeling is for.

You may ignore this document after the first week on the project, in fact I would encourage you to. But even if you throw it away immediately after creating it, the process of developing the document will still:

• Flush out important questions
• Show your customer you understand their world
• Help document the project to other developers
• Facilitate communication (especially with the people who pay the bills)
• Generate user stories (requirements)
• Identify the entities that can feed into an Entity Relationship Diagram (or database I suppose)
• Aid making good choices for the decisions it be hard to undo later (like whether to manually code workflows or use Windows Workflow Foundation)
• More clearly identify pain points and areas where your software can help end users; and
• Identify metrics that can help determine project success from an ROI and product owner's perspective

Convinced this is a tool you need in your toolbelt yet? As long as the answer isn't "I only write code leave me alone" then check out this quick how-to:

Start with the AS-IS diagram

As soon as possible on a new project you should plan to interview your future end users. This shouldn't be an unstructured Q&A session. You should start with a rough draft of what you think their business process is and ask questions that identify the places where you were wrong.

In order to start the diagram you need to identify both the end users and existing systems and put those in "swimlanes" down the left side of a diagram. Swimlanes are important because they help you be explicit about who is performing what activities. Then flush out activities in the happy path in appropriate swimlanes like so:

As soon as you talk to end users (like the clerk in the example above) you will discover unhappy paths. What if the warehouse is out of a product? What if the payment was declined? What happens if a customer needs to return something?

Obviously you document these using standard activity diagram notation:

Identifying Pain Points/Opportunities

The interesting part about unhappy paths is how often they occur, how they affect the customers of your customer, and ultimately how they affect your customer's bottom line. For instance, how often is payment declined in the example above? Fairly frequently? Can you get an actual percentage? If it turns out it's because the original application failed to validate credit cards, then a small amount of programming effort can provide a large benefit to length of time until order fulfillment. Sometimes you can make the process more efficient without writing a line of code.

And at the end of your project if you can show that you've decreased the average length of time it takes to go through the process, ideally by showing less time was spent in unhappy paths, you will likely have made yourself one happy customer. This can be especially effective if your new system can show statistics (like percent of orders that are declined) and compare those to the old process.

Developing the TO-BE diagram

AS-IS process diagrams are completely useless unless you can effectively describe what the process will look like after you implement your new system. The process of developing these documents is pretty similar to the AS-IS diagram except of course you're trying to fix the pain points.

One technique I like for generating a backlog of user stories (tasks) is to put unique numbers with each step in the TO-BE process, then ensure there is at least one user story per step in the process. You may be surprised at how easily activities in your new project's swimlane map to user stories.

Summary

I'm a fan of writing code as soon as possible on a project, but if you fail to understand an end user's business process you may be writing the wrong code. Business process modeling can ensure you're writing the right code to solve the most important problems first. And that may be the difference between project success and project failure.

Agile has always espoused the importance of transparency within the development team, but it is also important to be as transparent as possible with your customer. In fact, the use of transparency can build the foundation of your customer relationship.

At Near Infinity most of our projects are consulting efforts where we build software for other people. In these situations, our customers place an enormous amount of trust in us, and it is incumbent upon us to do everything that we can to earn this trust.

Most Agile development methodologies have clear roles that serve to simplify the development process. In Scrum, the development team is responsible for delivering complete, tested, working software each iteration. The product owner is responsible for making decisions about which features should be developed at what time, and for deciding when there is enough new functionality in the software to make a release to production.

For anything not related to feature development the product owner is the decision maker on the project, but many times the software developers have more information about the state of the software than the product owners do. This puts the development team in a position of power. As we all know, with great power comes great responsibility. You have the knowledge that your product owner needs to effectively drive the project to success. You should share as much of this information as your product owner can stomach - whether it is good news or bad news. Especially when it is bad news.

Too many times in our industry, development teams will hide bad news from decision makers, hoping that they can adjust resources or shuffle the order of features that need to be developed in order to make up schedule time. We rationalize this by saying that our customers wouldn't want to know all of the "implementation details" and there is no reason to get the customer involved with something that may not even be an issue. This idea of the development team insulating their customers from development issues is flat out wrong. Nothing good can happen from this. In software development, we almost never make up schedule time and things don't magically get better because we shuffle around resources.

 So instead, share the bad news. You may think that it makes the development team look bad, but it won't. Software development is not an exact science. Pretending that it is can be a recipe for disaster. A good product owner will know this and should expect some setbacks during the course of the project. Sharing these setbacks shows your customer that you are their partner in this effort. It gives your product owners the valuable information that they need in order to make the critical decisions that will affect the overall success of your project.

As software developers, we want to prove our expertise. This is our profession; we are supposed to know what we are doing, right? (Right!) So then doesn't admitting our mistakes and pointing them out to our customer make us look incompetent? Absolutely not. The truth is that by owning our mistakes and exposing them, we look more confident in our abilities. It shows your customer that making mistakes is part of the development process and is normal. Confidently owning mistakes, learning from them, and addressing them with better solutions shows your customer that you are experienced and you know how to deal with adversity. It shows that you can be trusted to make decisions that will benefit the project and the customer, and that you are not just looking out for yourself.

It is easy to fall into the trap of always wanting to please our customers. This is a very shortsighted view of software development. If you aren't concerned with repeat business, then by all means, always tell the customer what they want to hear. The truth of the matter is that sharing information that upsets your customer is almost always the right thing to do. It may mean having an awkward meeting or a tense few days, but it allows your customers to do their jobs more effectively, to plan appropriately and it better enables them to meet their expectations in the long run.

Don't mistake honesty for transparency. Customers expect honesty. They expect truthful responses to their questions. Transparency goes one step further. It means sharing information when you don't have to. It means sharing information when things go wrong. It means partnering with your customer in order to deliver the best software possible in order to return value to the business.

Being transparent isn't always easy. Just remember, the most important time to be transparent is when it is the hardest.

The Problem with Burndown

A typical burndown for the life of a project might look like this:

It shows the project started with 100 points of work in the backlog; it's completed eight iterations; the team accomplishes about ten points an iteration; and if everything continues at the current velocity the project will complete all work within another two iterations. Great.

But what happened in iteration six? Very little appears to have been accomplished. Maybe the team all took a vacation. Maybe there was a major problem or the team incorrectly estimated complexity. Or perhaps a large set of new requirements were added to the backlog because the customer decided what they thought they wanted wasn't what they really needed: namely the exact scenario agile was designed for.

Burnup Charts

The problem is that burndown charts lack two essential pieces of information. First, how much work was actually accomplished during a given iteration (as opposed to how much work remains to be completed) and second how much total work the project contains (or if you prefer how much scope has increased each iteration). A burnup chart for the exact project above might look like this:

We can now clearly see that the team did not take a breather in iteration six. They continued to complete about ten points per iteration, but during the sixth iteration the scope increased by about twenty points.

One could imagine the opposite happening as well. Later in the project the team might delete old user stories that were envisioned during project inception and thus decrease the total scope. The burndown chart would incorrectly show such a scenario as a sudden increase in velocity.

Summary

Either way the burndown chart hides essential information. I propose we throw it away and show the slightly more complicated, but infinitely more useful burnup chart. After all you wouldn't want upper management thinking you were lazy in iteration six would you?