Learning from Software Disaster

Disasters fascinate us. We slow down to look at auto accidents even on the other side. While a disaster is nice to watch, we can also learn from them. The new healthcare.gov web site offers some good lessons on how not to do software development. Some of these lessons are: Bad news needs a way to reach the top. In a disaster, shut things down. When trying a major innovation, use a new set of resources. All of these were not followed in this situation.

Bad news needs a way to reach the top. Every project will have bad news. Managers are being paid to handle the bad news. But bureaucracies often will hide the bad news instead of dealing with it. In this case, the bad news was hidden from the top. The president learned the bad news from the press instead of from his own staff.

We saw the same thing with the Challenger Space Shuttle. The bureaucracy hid bad news. The result in both cases was a small problem becoming a major disaster.

In order to prevent bad news from becoming a disaster, management often needs to seek out the bad news. One major goal of "management by walking around" is to go around the bureaucracy and find the problems that are being hidden.

When things are going wrong in a project, there is a time when it needs to be shut down. We see this in highways. Somewhere there is a highway under reconstruction. You know the situation. If they could shut the highway down, the construction effort would take less than half the time. When there is a real disaster, the highway gets shut down. When a truck fire damages a bridge, the road is shut down.

The same is true of software development projects. When a real disaster happens, shut the system down. Trying to patch a system into working order while at the same time allowing customers to use it is a recipe for a huge mess. We saw that with how the daily changes still left people unsatisfied.

Innovation needs a new organization. Building a new system needs a group of people who are communicating well and regularly. Typically, that means using a smaller project team.

In this case, the government used the people they have used in the past. These contractors had the systems in place for normal government development. A brand new system performing a function new to government was outside of their experience.

The standard way for government projects is to split the project up between many subcontractors. Those subcontractors split it further and on down the line. It has been estimated that there might have been as many as five layers of corporations between the government and the person actually doing the code. A small, cohesive, well communicating team this was not.

There are many more ways that project can fail. But keeping these ways from happening will help.

Results Oriented Work

Which are you more concerned about, whether or not an employee made it to work on time or what that employee accomplished while working for you? Far too often, a manager measures the minutes in the office and fails to measure the accomplishments. There is a story about an employee who acted as if he was always busy, running between places in the office and staying late, accomplishing nothing, but getting promoted. All that effort was not checked on.

Measuring accomplishments is difficult. In order to do that, the job has to be well defined. It takes serious management effort to define measurable outcomes for each position. It is far easier to measure the hours sitting in the office.

“Results Oriented Work Environment" is one style of changing a company to performance instead of presence. Another style is "outsourcing everything that isn't strategic". We are seeing far more companies outsource than adopt results oriented internally. The process for outsourcing is often better set up than the process for defining internal work.

Performance based job descriptions change the nature of recruiting as they change from "find me this purple squirrel" to "find me someone who can do this". It isn't what a person has done in the past that is important, it is what they will do for you in the future. Lou Adler posts quite a bit about the value of changing to performance based job descriptions.

Managing for results has been described as trusting the employees. If the results are not properly defined, then the company can wind up in a lot of trouble.

Risky World

The routing over the Internet is being hacked. If you are sending any unencrypted traffic over the Internet, sometimes that traffic is going through places like Belarus and can be inspected. If you send any credit card data over the Internet, encrypt it.