Economic Progress requires “Creative Destruction”—Joseph Schumpeter
Economics is about the Right Livelihood. In Chapter 5, we look at the economic aspects of systems, taking some principles from the Dismal Science and applying them to prevent dismal systems. Economics is called the Dismal Science, for good reason. It’s about limits: Satisfying unlimited human wants with limited resources. Economics is the study of allocation of scarce resources. And in any competitive business environment, resources will be limited. I was an Economist before I got into the computer business. We were fond of saying: “There is no such thing as a free lunch”, often while we were eating one.
We always hear how rarely computer projects are completed on time and within budget. Should we blame it on technologists? Here’s the story of one “late and over budget project.”
Scene 1: The office of Manny the Programming Manager in a Fortune 500 firm.
Lily the Lead Programmer enters stage left. Manny looks up expectantly.
Lily: I have the estimate, Manny. It will take approximately 24 months to build that new system.
Manny: Under his breath, but loud enough for Lily to hear: Gee, I thought she was a better programmer than that! To Lily: Whew, that’s two whole years! Dina the Director will never accept that. But wait! I know! What if I got you and your team brand new workstations, and a concierge service to handle some of your daily chores, and sent you all to that productivity seminar in Las Vegas so you can learn the latest programming techniques. Under those conditions, do you think you could do it in, say, 18 months?
L: Hesitantly Well, I suppose that might be possible….
M: Good, it’s settled then! Writes down: “18 Months”. Does not write down workstations, or concierge, or Vegas. I’ll take this to Dina right away. Exeunt, stage left.
Narrator: On the way to Dina’s office, through that phenomenon totally unexplained by science but frequently experienced by programming managers, a warp in the Universe causes the ink on the paper to re-arrange its molecules so that “18 Months” now reads “12 Months”.
Scene 2: The office of Dina the Development Director.
Manny the manager enters stage left. Dina looks up expectantly.
Manny: I have the estimate, Dina. It will take approximately 12 months to build that new system.
Dina: Under her breath, but loud enough for Manny to hear: Gee, I thought he was a better manager than that! To Manny: Whew, that’s a whole year! Vince the Vice President will never accept that. But wait! I know! What if I got you a bigger office, and a reserved parking space so you won’t waste time trying to park, and sent you to that management seminar in Maui so you can learn the latest motivation techniques. Under those conditions, do you think you could do it in, say, nine months?
M: Hesitantly Well, I suppose that might be possible….
D: Good, it’s settled then! Writes down: “9 Months”. Does not write down office, or parking space, or Maui. I’ll take this to Vince right away. Gets up, they both exit, stage left.
This is one play you can safely leave during Intermission. We all know the outcome. A six-month project is approved and funded, a deadline set, and sure enough, 24 months later the project is eventually finished. Management decries the undependability of a programming staff that came in Three Hundred Percent over budget. They’ll blame it on the miserable programmers. But in fact, we actually hit the programmer’s original estimate right on the head. As Robert Glass points out in Software Runaways, “most cost and schedule targets are set by marketers or customers, next most often by managers, and least often by the technologists who will do the work.”
In some cases, management originates the tendency to underestimate. When it’s subtly clear a low estimate is expected, not surprisingly you’ll get one. And when enough pressure is brought on technical employees to cave and agree to an unreasonable schedule, they’ll probably do so.
In other cases, it’s self-delusion of management, and to be fair, the programming staff as well, that leads to this problem: “We want it done by a certain date, therefore it must be possible to do it by that date.” Or, “I really want to do this project, and if it can’t be done within a certain cost it won’t be approved, so I’m sure we can do it for that amount.” The estimate is inversely proportional to the desire to do the project. This will be especially true if missed deadlines are easily forgiven; the corporate culture becomes “Tell them what they want to hear, the missed deadlines won’t matter anyway.”
In some cases, the management uses this tendency. In I Sing the Body Electronic, Fred Moody describes a project at Microsoft where the team agrees to an impossible schedule with an arbitrarily shortened schedule under pressure from Bill Gates himself. Moody concludes: “while giving his employees the means to win he also ensured that they would interpret their victory as defeat. There would be no laurels for them to rest upon; instead they would dive immediately into their next project hoping to redeem themselves.” If Moody is right, this strategy might be cold-blooded, it might even be dishonest, but it’s hard to argue it’s not effective given Microsoft’s commercial success.
[ [ [
Many deadlines are set arbitrarily with no actual relation to value, so overrunning the budget is really just missing a meaningless goal, anyway. American President Lines was developing a new computer system to track cargo in the early 1980s. The project got into the usual problems associated with a project in trouble: missed deadlines, cost overruns, and an overall lack of tangible progress. Eventually the manager was fired, and a consultant was interviewed to take over the project. After careful study, the consultant concluded the project was feasible, but that the schedule and budget would need some serious rework. At the meeting where he presented his proposal, the executives were pleased to hear the good news, and asked what modifications he would like. He replied: “Two, Two and Two.” What did he mean by that? “Multiply the staff by two and the time by two, and divide the features by two.” Naturally, pandemonium broke out, but after everything settled down, the executives ruefully agreed to the scheme. The consultant was as good as his word, and indeed delivered half the originally planned system with twice the staff in twice the time. But the results were extraordinary, the system assured APL a position as the leader in its industry for more than a decade. The original estimate had no relation to the eventual value, which was many times the actual cost even after accounting for the underestimate.
As Economists are wont to emphasize:
We can do it Faster …
We can do it Cheaper…
We can do it Better …
—choose no more than two of the above.
Why do you want to put in a new computer system? Let me guess: To make it Faster, Cheaper, and Better. The first rule of economics is, “You can’t have it all”. Often making something faster will require you to spend more money and or sacrifice some feature that makes it better, cheaper will require slower or less good, and better might slow it down or cost more. You should emphasize one, and choose no more than two, objectives. Be clear upfront and all along which one will be sacrificed if necessary to achieve success. If you have an innovation that supposedly gives all three, beware. Alfred P. Sloan at General Motors once presided over a meeting where a proposal was presented. No one could think of any disadvantages to the proposal. Sloan said if no one can think of anything wrong with the proposal, they’d better not go ahead. He’d never heard of an action with absolutely no downside, so if they hadn’t thought of one, they hadn’t thought enough. At the next meeting, many reasons were presented why the proposal would have been disastrous, and it was scrapped.
Efficiency is a measure of output versus input, focusing on reducing waste. Most companies don’t care as much about the efficiency of an analysis project as they do about its effectiveness, i.e. producing the best system by meeting the business need. Companies are willing to spend hundreds of thousands of dollars on tools and training which will probably not reduce the cost of analysis in any way, if they believe it will give them a better system. Some of the tool vendors claim their analysis techniques will deliver a system at lower cost, but generally because of the efficiency of the programming phase, not the analysis phase.
The two most important economic principles you need to know about costs are sunk costs and opportunity costs. They frequently occur together, but have also been sighted apart. Sunk costs look to the past: “What’s done is done” and opportunity costs look to the future: What could be done.
There is a human reluctance to accept loss. Often in systems development, a project turns out to be too expensive, or takes too long, or just goes off the rails. The sensible thing is to cut your losses and cancel the project. Someone will protest: “We’ve put umpteen gazillion dollars into this project, we can’t throw that away!” But if the project is destined for failure, unless you’re buying time to find another job, there is no logic in losing even more money because you don’t want to lose face.
Don’t throw good money after bad.
And remember, past consideration is no consideration. “What have you done for me lately?” is the motto. If it’s a bad project, pull the plug.
One of the most important concepts from economics is the concept of opportunity cost. Opportunity cost is calculated as the theoretical cost of the profits lost from not taking an alternate course of action, that is the cost of the money you could have made had you pursued another opportunity. For example, if you have a vacant apartment, it is incorrect to say there is no cost in not renting it out. If you rent your apartment to your brother-in-law at below market rate, the extra rent he didn’t pay is the opportunity cost of not renting it to someone else. And there is an opportunity cost measured as the difference between the rent your brother-in-law actually pays and what you would get at market rate. In systems, opportunity cost is the other system you could build, or the cost of waiting on a system and its effect on TTM—Time To Market. So don’t lose opportunity on unproductive projects.
While you keep sinking, you’re losing the opportunity. Retailers learn this early. If you stock and item that doesn’t sell, novice retailers have trouble cutting the price below cost and getting them out of the store. What you must do is ruthlessly cut the price, even if you will lose money on those items, in order to free up shelf space for items that will sell. The reluctance to lose the sunk cost in the unpopular item is costing you even more in opportunity cost of the profit you would make on the new item.
It’s a postulate of systems analysis is that the cost of changing a program increases exponentially through the SDLC, as illustrated by the Boehm (rhymes with “Game”) Curve, from Barry W. Boehm’s classic, Software Engineering Economics, which is still a surprisingly valuable book despite being over two decades old. A design change that would have cost $1 during early analysis might cost hundreds of dollars after the system has been installed.
This used to be unquestioned dogma, and was very much reinforced by my experience: frequently a trivial change in analysis will be all but impossible to fix after implementation. Some developers, however, now question this assumption, arguing modern development methods can reduce the Boehm effect. This debate about shortening development cycles starts with two letters: XP. It stands for eXtreme Programming, and was originally articulated by Kent Beck, but has been endorsed in books on the subject by such luminaries as Martin Fowler, Tom DeMarco, Ward Cunningham, and Erich Gamma (of Design Patterns fame). There is a growing series of over a half dozen books and have actually been conferences devoted to this approach. In Extreme Programming Explained: Embrace Change, Beck explains the exponential growth could flatten: “If we can flatten the curve, old assumptions about the best way to develop software no longer hold.”
XP asserts the Boehm Curve no longer holds, and recommends a development cycle characterized by many iterations of small increments. I personally have no direct experience with the XP approach but have spoken with several developers who tried it with favorable results. They say it works well for the right project. Beck himself points out that this approach is not for every project (something few other methodologists have been honest enough to do). For example, it must be small: five or ten programmers is doable, twenty probably not. Likewise, if you are tied to a complex legacy system the Boehm curve probably does apply and XP probably won’t work.
In England in the early days of the Industrial Revolution, the workers in the wool and cotton trades became, in the title of Kirkpatrick Sale’s book, Rebels Against the Future. As work moved from skilled workers at home to unskilled workers in factories, Industrialization ruined their way of life by making it possible for greedy employers to drive wages down and exploit masses of workers for obscene profits. Or so the myth of the Luddites would have us believe. But imagine a 60 Minutes investigator interviewing one of the downtrodden factory workers:
Mike Wallace: “So tell me, how does it feel to work at these wages?”
Worker: “Great!”
Mike: “Uh, I thought you’d be angry. You don’t mind making less than what wool croppers used to make?”
Worker: “Hell, no: this is more money than I ever made in my life before.”
Mike: “But surely you’ve heard of the Luddites…”
Worker: “Look, those guys controlled a trade and artificially kept their own wages high at the expense of those of us they considered beneath them. They passed the job from father to son, and were sure to keep the likes of me out of work. Their closed guild would never have allowed me or my children to join. So you see, they aren’t just fighting those above them on the economic ladder; they are also keeping us down. And now, thanks to the factories, thousands of us have jobs where only a few of them had jobs before. Serves them right!”
Mike (to cameraman): “Woops, looks like there’s no story here. Let’s go over and interview some of those poor, displaced Luddites.”
Veblen’s Principle:
ALL changes help some people and hurt others.
So technological innovation will probably bring at least some harm to some people, and some of the historically accepted examples of injustice may have had some justice in them. I’m sure most of us have our qualms about where technology is taking us, and whether it’s for good or ill. I’ve thought long and hard and have concluded it is indeed good, and offer three steps in the argument. First, we humans in a technological society live significantly longer than our non-technological counterparts, both past and present. The second observation is that given a choice, the vast majority of people choose the technological lifestyle over the primitive one, and those that reject it find, often to their horror or disgust, that their children embrace it. So although Jacques Ellul argues in The Technological Bluff that although we live longer lives, they are not better lives, we must accept that the technological life is better since those with a free choice freely choose it. And third, our lives have less drudgery, more entertainment, and are more interesting than without technology. So the bottom line is: I’m glad I was born into a technologically advanced time and place, and those that disagree might turn off the power to their homes for a week and see if it changes their perspective. You’ll face similar quandaries on a smaller scale as a computer systems developer, but I am convinced that on balance, technology improves our lives.
Metcalfe’s Law is stated as: Network value increases with the square of the number of nodes attached to it. It is somewhat uncertain whether Bob Metcalfe, after whom it is named, ever actually said it, and if he did, it was in reference to LANs, not the Internet, but I’m quibbling. Carl Sagan insists that he never said “Billions and Billions” (in his book of the same name); Sherlock Holmes never said “Elementary, my dear Watson”; and Bogart never said, “Play it again, Sam”. But they might as well have, and even if they didn’t, they probably should have. And Bob Metcalfe could have expressed his law as “Billions and Billions”, because that’s what his namesake formula computes to.
Yes, this is the same Bob Metcalfe who predicted collapse of the Internet in his column “From the Ether: Predicting the Internet's catastrophic collapse and ghost sites galore in 1996”:
“Almost all of the many predictions now being made about 1996 hinge on the Internet’s continuing exponential growth. But I predict the Internet, which only just recently got this section here in InfoWorld, will soon go spectacularly supernova and in 1996 catastrophically collapse. Here’s why there soon will be only World Wide Web ghost pages.”
But I’m quibbling, again, aren’t I? Apparently George Gilder actually stated the law, attributing it to Metcalfe, who gamely accepted credit. Not content with the amazing increases implied by the original formulation, in his book Telecosm, Gilder restates Metcalfe’s Law as The Law of the Telecosm: “The value of a network grows by the square of the processing power of all the terminals attached to it.” Since processing power increases with Moore’s Law, Gilder’s Law of the Telecosm would imply you need to factor a doubling every 18 months on top of the squaring. Not to be outdone, Kelly, in New Rules for the New Economy argues Metcalfe understates the value of relationships, that the formula is truly exponential, N to the Nth power!
[ [ [
Wait! Stop it! Time for a reality check!
Do the math: Between about 1994 and 1999, the Internet grew about 100 fold, from about a million to 100 million nodes. If Kelly were correct, that would mean the value of the network increased by 100 to the 100th power. That is an extremely large number: it’s a googol googols, a one followed by 200 zeroes. A googol is 1 followed by a mere 100 zeroes. A googol is a number so large there is nothing to use it on: it’s larger than the number of atoms in the universe, and Kelly expects us to accept a googol of googols’ increase. In fact, according to Newman, if the entire Universe were filled with protons and electrons, so that no vacant space remained, the total number of protons and electrons would only be 100 to the 55th power. Irrational exuberance indeed, if dollars were the size of atomic particles, you would need a billion, billion, billion, billion, billion, billion, billion, billion, billion, billion (Carl Sagan would love this, bless his heart!) universes the size of ours to cram in 100 to the 100th dollars! Methinks Mr. Kelly might have overstated just a wee bit.
But even Metcalfe’s Law in its original statement, without Kelly’s or Gilder’s embellishments, cannot hold long mathematically before the Ponzi-scheme effect kicks in. Internet growth has been phenomenal. When you take a phenomenal increase, and then square it, you get absurdity. It’s probably slowed of late, but for several years the Internet was doubling in size every nine months, a compound rate of about 150 percent per annum. Metcalfe’s Law implies value increased at a compound rate of 535 percent per annum! Set yourself up a spreadsheet and see the result of that kind of growth. Let’s begin at the beginning. In 1971, ARPANet, the Internet precursor, linked 23 sites. ARPA spent thousands creating this humble beginning, but let’s just say each connection was worth only what an AOL or MSN connection would cost today, about $23 per month. That means today the average node is worth over $300 million a month, and the Internet is worth $90 thousand trillion per month in total! There ain’t that much money in the World: it’s about 80,000 US economies! Okay, so maybe the squaring didn’t start at the beginning. At the 150% rate we’ve been growing, we’re increasing 10 fold about every three to four years, but squaring gives us a 100-fold increase in value in that same period. Those crybaby customers who objected to the $2 per month increase at AOL are wrenching, grasping, ungrateful cheapskates, since the value of each connection must have increased by well over two hundred dollars per month—why were they complaining about a lousy two bucks?!?
[ [ [
So the math doesn’t work, but how about the theory? It is possible to question a key underlying assumption, to wit: that the Law of Diminishing Returns—which states that as a factor increases, there will be a decrease in the marginal benefit from a given increase that factor of production—doesn’t apply to a network. As Shapiro & Varian say in Information Rules: “Technology changes, economic laws do not.” Consider an individual Net user, such as myself. If Internet growth is proceeding at anything like its historic rate, several hundred new users are joining the wired world every second, even as I sleep! Is the value of my node increasing geometrically as a result? I will never have any contact with the vast majority of these individuals. And truth to be told, I don’t want to have any contact with most of them. So, no, they aren’t increasing the value of the Internet to me at all. Not everyone has something worth listening to. In Emergence: the Connected lives of Ants, Brains, Cities and Software, Johnson considers the familiar analogy of the Internet as a huge brain and concludes its ratio of growth to order is characteristic of not a brain, but a brain tumor.
How about business on the Internet? Surely the e-commerce community will derive value from these newbies. Perhaps, but certainly not at a geometrically increasing rate; in fact, the rate is diminishing at the margin. How’s that? The value of new entrants to the world of e-business is clearly only the discounted value of the eventual profits derived from them. The average entrant has a lower income than the average person already on line, so the potential marginal contribution to e-commerce is declining, not increasing at all, much less increasing geometrically. We’ve long passed Pareto optimality with the 20% of customers representing 80% of the income. In fact, considerably more of the world’s disposable income is already on-line than still off-line. So if Metcalfe’s Law ever did hold, it doesn’t anymore. You wouldn’t expect it to, rather than continuing steeply upward we expect to see an S-shaped curve with growth leveling off.
Originally Metcalfe was clearly talking about networks and computer nodes, but it’s gone beyond that. The X-Economy: “The value of a network is equal to the square of its members—whether computers, phones or value chain participants.” Shapiro &Varian in Information Rules apply the concept of network externalities, which they discuss with Metcalfe’s Law, to fax machines, and even buyers of a given product, such as Mac Users as a network. The authors’ don’t actually say Metcalfe’s Law would apply, but this again argues against taking the Law literally: these kinds of networks always existed, so that would suggest the Law has been operating for centuries, and would also operate on the off-line competitors to the Internet, thus removing any special advantage. Wouldn’t spiders be the riches creatures on earth?
Note the phenomenal increases in Internet users can’t continue much longer in any event: We’re less than two doublings away from everyone with a telephone. A further four to five doublings will require us to go to literally universal access, because it will exceed the population of Earth, so we’ll have to boldly go out into the Universe to find more users. Of course if we talk nodes or computers instead of people, the doubling could continue indefinitely. It’s said there are already several times as many computers (microprocessors) as people on Earth, but they are not as valuable as people since they don’t have their own credit cards (yet).
[ [ [
But I always interpreted Metcalfe’s Law metaphorically rather than as an actual statement of mathematical certitude. In Mr. Metcalfe’s defense, it appears he made a casual remark that Gilder hyperbolically stated as a law; the Law was never mathematically demonstrated, and wasn’t intended to be. Nonetheless, Metcalfe’s Law should not be simply dismissed. Murphy’s Law, “Anything that can go wrong, will go wrong” is a Law, a useful Law, and yes, even a true Law. It is irrefutably empirically demonstrable that the vast majority of things that can go wrong, do NOT go wrong, but Murphy’s Law is a truth that needs no proof to anyone who has ever worked on a complex system.
In The Lexus and the Olive Tree, Friedman refers to Moore’s Law, not Metcalfe’s, but describes the difference between the old Cold War paradigm and globalization in network terms: “In the Cold War, the most frequently asked question was: ‘Who’s side are you on?’ In globalization, the most frequently asked question is: ‘To what extent are you connected to everyone?’ In the Cold War, the second most frequently asked question was: ‘How big is your missile?’ In globalization, the second most frequently asked question is: ‘How fast is your modem?’ ” Both Friedman and Metcalfe are right: the network might not be the computer, but the computer that is not on the network may become irrelevant. And the business. And even the person.
One of my pet peeves is the popular misunderstanding of the Learning Curve. Like The Ugly American, people always get it backwards. The “Ugly American” in the book was a kind, caring individual, as opposed to the Handsome American, who was obnoxious. Whenever you hear someone describe an American behaving boorishly abroad as an ugly American, you can be sure they haven’t read the book: It was the handsome American who was the lout. The Learning Curve is likewise the opposite of what most people think.
A Learning Curve shows a decline in unit cost as we gain experience. Note the original learning curve, from Economics: It slopes down to the right, so you don't “climb the learning curve”, you slide down it. And a “steep learning curve” means it’s easy to learn. In everyday speech, these characteristics are often reversed, “a steep learning curve” intended to mean hard to learn.
The Pareto Principle, also known as the 80/20 Rule, is named after Vilfredo Pareto, an Italian economist, who noted that in distributions 80 percent of the dividend usually goes to 20 percent of the observations. For example, 80 percent of the wealth is held by the richest 20 percent of households, 80 percent of the workers are employed by the largest 20 percent of companies, etc. By this reckoning, 80 percent of your problems will be caused by 20 percent of your bugs, or cases, or whatever.
In many circumstances the 80/20 rule becomes the 90/10 rule: 90 percent of the problems are caused by 10 percent of the items. So the opportunity is to be sure to find all of the 10 to 20 percent of items that will cause most of the problems, and then reduce the other problems as much as you can. Include the critical items you can’t operate without, look inside complicated processes in which the chance of error is high, and consider the past history of problems you've encountered from each system.
Self-fulfilling Prophecies are an interesting phenomenon. They happen when people believe something is going to happen, and then because of these beliefs, unintentionally cause the event to actually happen. For example, if people believe a bank is going to fail, they rush to withdraw their money from the bank. If enough people do this, the bank will fail, even if its finances are sound. The stock market has a tendency to go where everyone thinks it will: if most investors think prices will rise, they probably will, because buyers will be willing to pay more and sellers will expect more.
The same can be true of your systems development effort. In many cases, whether you think you can, or you think you can’t, you’re right. If the team becomes convinced they are going to fail, they’ll stop putting in serious effort, their morale and effort will flag, and sure enough, they’ll fail.
Whether you believe you can, or
Believe you can’t
You are right.
I once worked on a project where we were all inexperienced. None of us new what we were doing, so we committed to what we now all know was an impossible task. Fortunately, we were so dumb we didn’t know enough to know the task was impossible, so we just did it! Only after we’d done it did we realize it was impossible.
Okay, you’ve been working hard on this stuff, so now I’m going to tell you to take a break, this time to see a movie. Every Systems Analyst must see Mr. Blandings Builds His Dream House, based on the book by Eric Hodgins, with Cary Grant as Jim Blandings, and Myrna Loy as Muriel Blandings. Although it was made in 1948, and is about building a house, not a system per se, it’s probably the best movie on projects and systems ever made. I like to play the “Changes in Closet” sequence to my class at UC Berkeley Extension.
The notation “Changes in Closet” is written on an outrageously large bill for modifications supposedly authorized by Mrs. Blandings. It turns out there were four pieces of flagstone left over from the porch that were going to be thrown away. Mrs. Blandings asked the contractor to put them on the floor of the pantry closet she called her “flower room”. She said she wanted it to be nice and dry. “Well, you’re the doctor” he said. The contractor interpreted “nice and dry” to require a drain. This needed a carpenter to rip out the floor, which went under the wall so they knocked that out, too. Then they needed to chop off a joist to make room for a cradle, weakening said joist, and requiring iron straps for a large pan to hold the cement. Because of the added weight, they needed a lally column to support the floor. The pan would sit on the hot and cold water pipes and the 220-volt electrical cable. So they needed a plumber to re-angle the drainpipes under the entire house and move the hot and cold water pipes, and an electrician had to reroute the 220-volt electrical cabling. Then a carpenter and a plasterer put the wall and floor back. Cost for the four old stones that were to be thrown away added something like a fifth of what the entire house, including 35 acres of land, had cost!
The lessons? A seemingly simple change can be very costly; and something that would have been trivial at the beginning is extremely expensive after a lot of work has been done. Don’t forget technicians will assume you know what’s involved when you ask for something, so beware.
Copyright ©2003, 2008 Patrick McDermott