The Death of Software Engineering?

8/4/2009 1:07 PM EDT

Jack,
Certainly Tom's paper is provocative! I saw a few things that kind of gave me the "flavor" of what he's really saying:
"This leads us to the odd conclusion that strict control is something that matters a lot on relatively useless projects, and much less on useful projects" What this tells me is that a project with low profit potential is "useless" - unless, of course, it's something necessary! Virtually all of the software I write could be considered useless by his definition, as it goes into little 8-bit microcontrollers. My designs will never change the world, but they do provide a nice income for my employer (and, by extension, me!). 'Useful' and 'useless' are very slippery metrics, and ones which Tom might do well to drop!
The second assertion:
"The more important goal is transformation, creating software that changes the world or that transforms a company or how it does business." Apparently, Tom doesn't (and maybe hasn't) written software for washing machines, or portable air conditioners, or refrigerators, or automobile engine controls, or any one of thousands of products that just make it possible for us to live our lives. It is unfortunate that so many of these products have small profit margins, thus requiring some level of control, but in the end, they are just necessary (if not more) than the iPods, cell phones, and Linuxes of the world.
It is likely that no software I write will ever make a press release, but it's still fun to do!

8/4/2009 1:32 PM EDT

I do think Software Engineering is dead. The idea really was kind of pointless to start with. I define software as creative problem solving process, not an assembly line process that requires stats on how fast or cheap you can make a product. Do keep stats on writers on how many lines of English they write vs errors they make? Software development is closer to writing a book then it is engineering.
Lets face it if you are going to make software that has a low profit margin you are going to do it overseas for half the price you can do it in the United States to make the software more profitable. If you are an EE or software developer in the United States making a product you cannot compete cost wise against India or China what is left is you have to be creative in order to make a profit.

8/4/2009 6:09 PM EDT

I think I have found our next generation of "Killer Apps".

Given that my colleagues and I work extensively in safety related/critical software systems, let us just take a lead from Tom DeMarco and remove all that strict engineering discipline. We are guaranteed a lot of media attention with the results!

8/6/2009 7:19 AM EDT

> Wasn't the pursuit of dot-com and mortgage jackpots the source of our two most recent recessions?

No, it was not. It was a result of increase in money supply.

As for software engineering, it is not dead, it is in its infancy.

8/6/2009 11:54 AM EDT

DeMarco, apples, oranges, and weird analogies. The article seems to be intended as thought provoking, the result of senility, or a chasm between him and reality.
I do agree with the goal of incremental development. Ironically, his description includes several metrics. Many grand 'intergalactic projects', like Google Earth, would have nothing to show if it were not for some project[s] that acquired the images. He also seems to ignore the business/problem solving component [like why/how products are made and why/how engineers are paid]. I am confident that his previous banking system customers would balk at a software development proposal modeled around Project B.
Like most things that fall under the topic of 'software', there are many 'plants, fruits, and vegetables' = most need water and sunlight, but there is not a single prescription for each one.

8/6/2009 11:57 AM EDT

I'm a hardware engineering, and I always hear that hardware engineering is dead. All the value is in the software. I agree with Ganssle, though. There's a lot of mundane but important hardware and software to be done.

8/8/2009 4:20 PM EDT

The question always is, who will be responsible for support of the product, and how will that responsibility be discharged? Software Engineering practices aim to build quality into the product in the first place, but also to provide the trail of breadcrumbs that will be necessary to provide for support and further development down the road. The transition point between the experimental/artistic phase of software development and the engineering phase is always tricky to recognize, just as its management is tricky to handle. In the beginning, you need to be able to shape the clay freely, and try out various ideas and forms. Once you are clear about the thing you want to make, however, you need to go about the making in a responsible way so that the resulting is useful and reliable, and can be supported and further developed later.

The developers of most of the "transformative" products I have seen essentially shipped their betas, and were lucky enough that the application domain was sufficiently forgiving of their mistakes in design and execution, as to allow for many years of clean-up development (no doubt involving a lot more "software engineering" than the original creation). How fortunate when we can stumble upon such applications, but as Ganssle pointed out, that is winning the lottery.

The methods of software engineering are valuable -- even necessary, I think -- but the tricky part is to know when to start using them, and to have something in hand that can be reliably and efficiently improved BY using them. If you start with regimen too soon, the "transformative" factor can often be engineered right out of the project (if for no other reason than the creators quail and bolt for another project). But if you start too late, you may end up with a corpus of unwieldy, dangerous, unmaintainable, unsupportable code.

Too bad Mr. DeMarco never found a metric for determining where that "sweet spot" of transition is located.

8/10/2009 10:34 AM EDT

This is a good example of why I don't pay a great deal of attention to well-meaning, "fascinating pundits". What they say sounds pretty good, but nine times out of ten ends up distracting me from my work more than helping me to get it done.

1. Software-development metrics have never told the whole story, and I haven't seen a metric yet that doesn't add more in "process" than it takes away in required work.

2. There never truly was any such thing as "software engineering" as down and close to the hardware as most embedded systems developers work, but people felt the need to call it something that fit reasonably within the vocabulary of the time.

3. When I buy, for example, a wireless router, I don't want "transformational". I want "reliable". We can't all be Julia Child or Wolfgang Puck. Some of us find gainful, sustainable work at Subway or Panera or any of a large number of places that turn out the same decent food day after day, often at thin margins the superstars wouldn't touch with a three-meter pole. The millions waiting in line for their five-dollar tuna sandwich couldn't care less how tasty Julia's fifty-dollar-a-plate salmon is. They need to eat. Now.

8/12/2009 5:02 PM EDT

The DeMarco's separation of projects into 2 camps is worthless. Every project I have been involved in was not going to cost much and was going to generate millions in revenue. Its only after the fact that you can historically go back and do that separation and apply his necessary level of project management.

In this 40th anniversary year of the Apollo moon landing, what if President Kennedy had said: "send a man to the moon and return him safely to earth before the decade is out, for under 2 billion dollars." There would be no 40th year anniversary...

8/14/2009 4:39 PM EDT

Looking at Mr. DeMarco's 2 software projects. The projected return on investment on 1 is 4900 percent and the return on investment in the other project is "only" 10 percent. If you can predict this return, then all other things being equal anyone will take the 4900 percent return. But projects like these are not common. Most investors and companies, on a day to day basis, would accept the 10 percent return. This is actually a very good return, depending on the length of the project.

In my 25+ years of work, I have only worked on 1 project, where the projected return was many times the investment, and on that project, we knew from the outset what the return would be. Was it a big important project? No, it was a small technical change, but the savings to the company was $300K per year, every year. Well worth the investment of 1 software engineer for a whole summer.

So the point of his article must not be the return of the projects, but when is control of software necessary and how much is necessary. I think he is trying to say, there are some point where the control can be minimal and there is a sweet spot where control and return balance out.

8/28/2009 10:03 AM EDT

"As for software engineering, it is not dead, it is in its infancy." --Andrew B

I agree whole heartidly.

Jack, great piece.