Does Not Compute
By Nicholas G. Carr
Carlisle, Mass. — The Federal Bureau of Investigation has officially entered what computer professionals call "software hell." After spending $170 million to create a program that would give agents ready access to information on suspected terrorists, the bureau admitted last week that it's not even close to having a working system. In fact, it may have to start from scratch.
Shocking? Not at all. A look at the private sector reveals that software debacles are routine. And the more ambitious the project, the higher the odds of disappointment. It may not be much consolation to taxpayers, but the F.B.I. has a lot of company. Software hell is a very crowded place.I had the good fortune of working for a company several years ago the corporate leadership of which decided that it needed to keep up with the competition and in so doing, allocated $1 billion to a computer system transformation. What the company ended up with several years later can best be described as a complete mess. An impressive mess but a mess just the same. The new system was slower, more complex, and less user-friendly. It required that the company maintain 700 programmers on their employ. The mainframe people within the headquarters building couldn't
Consider Ford Motor Company's ambitious effort to write new software for buying supplies. Begun in 2000, the goal of the project, code-named Everest, was to replace Ford's patchwork of internal purchasing systems with a uniform system that would run over the Internet. The new software was supposed to reduce paperwork, speed orders and slash costs. But the effort sank under its own complexity. When it was rolled out for testing in North America, suppliers rebelled; according to Automotive News, many found the new software to be slower and more cumbersome than the programs it was intended to replace. Last August, Ford abandoned Everest amid reports that the project was as much as $200 million over budget.
A McDonald's program called Innovate was even more ambitious - and expensive. Started in 1999 with a budget of $1 billion, the network sought to automate pretty much the entire fast-food empire. Software systems would collect information from every restaurant - the number of burgers sold, the speed of customer service, even the temperature of the oil in the French fry vats - and deliver it in a neat bundle to the company's executives, who would be able to adjust operations moment by moment.
Or so it was promised. Despite the grand goals, the project went nowhere. In late 2002, McDonald's killed it, writing off the $170 million that had already been spent.
communicate with the PC people. In some cases, mainframe people couldn't communicate with each other. As time went on, upgrades and patches were added, brought in by different providers, some of whose consultants didn't speak English. And we all prayed that, when we had need of a programming change, that someone was still with the company that knew something about the original code.
And we talked in terms of "man-months" to get a change made. "You want me to alter a field in this report? I can do it. It will take nine man-months." The most frustrating moment I remember in this regard was on a day when I asked a senior programming department head for some rather major changes. I laid out for her what I needed, in great detail. She took it all in, asked a few questions, took lots of notes and told me, without expression , it will take five years to accomplish. What?!!!
"Never mind. We'll make do."
Believe it or not, developing our corporate computer system was not our core business. We actually sold stuff to customers.
Research by the Standish Group, a software research and consulting firm, illustrates the troubled fates of most big software initiatives. In 1994, researchers found, only 16 percent were completed on time, on budget and fulfilling the original specifications. Nearly a third were canceled outright, and the remainder fell short of their objectives. More than half of the cost overruns amounted to at least 50 percent of the original budget. Of the projects that went off schedule, almost half took more than twice as long as originally planned. A follow-up survey in 2003, however, showed that corporate software projects were doing better; researchers found that the percentage of successful projects had risen to 34 percent.We learned it the hard way. Such wasted effort. And scarce resources.
But I learned from that experience. A few years later, I was working for a different company and was asked by senior management to take over a troubled department. Morale was poor. Training was substandard. We had your standard personnel issues relating to productivity, discipline, absenteeism, and motivation - or lack thereof.
And the computer program in use was unsatisfactory. It was purchased from a development firm that came in and designed it for us, at great expense. The company provided consultants and a help desk should we have need, and the software was sophisticated enough to put a man on Mars. I remember too that the company allowed as many users to log onto the system, after we paid - for each one of our employees - $10,000 per year in the way of a license fee.
The problem was, the software was so complex that only one person in the department could run reports from this monster. We had apparently sent her off at great expense to learn its intricacies. Everyone else would input data (it was a multi-user networked system) but only one woman, who would close her office door and work her magic in secret, could actually get this elaborate system to provide us with any meaningful information. Naturally, we had to accept this woman's peculiarities and workplace demands. Without her, we were doomed.
Worse yet, I realized, having sat down with one of these consultants for several precious days, that we were only utilizing about 5% of this software's capabilities. It had functionality that we had absolutely no need of.
So I scrapped it.
I went out and bought a new program. Off the shelf. Actually manufactured by Microsoft. No consultants. Inexpensive to use. Easy to operate. It integrated perfectly with Excel, Word, and PowerPoint. It allowed for multiple users to input data simultaneously. We generated meaningful and sophisticated reports with it. And anyone could learn to configure the software to spit out whatever data we felt we needed. It worked. It was sufficient. Not being our core business, it allowed us to free up funds for our core business - selling stuff.
What happened between 1994 and 2003? The Internet boom went bust. Stung by wasted investments in complicated software systems, business executives began taking a more skeptical view of such projects. They scaled back their expectations, pursuing more modest software enhancements with narrower goals - and far higher chances of success.
Equally important, they stopped trying to be creative. Rather than try to customize their software, they began looking for cheaper, off-the-shelf programs that would get the job done with a minimum of fuss. When necessary, they changed their own procedures to fit the available software. Old, generic technology may not be glamorous, but it has an important advantage: it works.
It may well turn out that the F.B.I.'s biggest problem was its desire to be innovative - to build a new wheel rather than use an old one within easy reach. When it comes to developing software today, innovation should be a last resort, not a first instinct.I prefer to think of it as graduating from "the school of hard knocks."
