Making "Software Engineering" Engineering

12/22/2009 8:11 AM EST

Everybody agrees:

One thin September soon
A floating continent disappears
In midnight sun....

The shepherd cries
The hour of choosing has arrived
Here are your tools.

Political science = software engineering.

12/22/2009 12:42 PM EST

We talk about other engineering disciplines as if they are akin to software engineering. For example, electrical engineers have equations and laws that govern their work, therefore it follows that we should develop the equations and laws that govern software engineering. But what if software was more like the laws of physics, rather than a specific engineering discipline? Electrical engineers deal with tha laws of physics in terms of electron flow, properties of materials, temperature effects, etc. Basically a subset of the total body of physical laws. A civil engineer has to deal with properties of materials, gravity, stress, strain, etc. - also a subset of physical laws, some overlapping those an EE has to contend with. What if software engineering is really another way of saying "laws of physics engineering", which of course is silly. Perhaps we'd make more progress if we considered for example, Embedded Operating System Engineering and Graphical User interface Engineering as distinct disciplines, separate from each other but with some overlap. I think that the myriad methods and formal approaches to software development have more specific application then a general discipline called "software engineering". Perhaps AGILE is perfect and appropriate for specific applications, but not everything. Perhaps it can be developed further with provable results - but only when applied to specific application genres. We may be attempting to extend our collective arms around something to big to grasp. And that might explain the slow progress seen to date.

Respectfully submitted as my humble opinion.
Steve Ciricillo

12/24/2009 2:41 PM EST

We really don't have to look for the science that far - programming is just a restricted (digital) case of sequencing, such as we find in this Rube Goldberg contraption: http://www.youtube.com/watch?v=s0jheZLhJB0

Or if you want the ultimate, Divine, version of this, watch these!:

http://aimediaserver4.com/studiodaily/videoplayer/?src=ai4/harvard/harvard.swf&width=640&height=520


http://www.youtube.com/watch?v=4PKjF7OumYo&feature=player_embedded

Of course, in these cases each of these little blobs is a pre-programmed digital computer in its own right, but it uses base-4, and - the hardware IS the software!

1/11/2010 9:01 AM EST

I tuned into this debate about 15 years ago, when Yourdon was writing about the "Death March" of the software industry. With respect, I think this whole issue is BS.

EE types have equations to prove the correctness of individual components and circuits. We have Boolean Algebra. Both solutions are best applied to limited subsets of what we actually practice. I would like someone to show me a formula for the correctness of the pinout of an automotive wiring harness, or a differential equation for the correctness and robustness of a low level messaging protocol. Both things are commonly designed by EE's, and neither has a Spice module to help do it. Above a certain level of complexity, EE's handle things the same way we do - simulations, inspections, testing and rework cycles. They tend to do the first things more, because their rework cycles are more expensive than ours.

If there is a difference between software design and the classic engineering disciplines, it is in the level of complexity. Over the past 20 years, as electronic and mechanical components have become more modular, more and more functionality has been pushed into software. Of course the software looks like the weak point in the system - everyone else's leavings have rolled downhill into the software hole. I think we do ok, mostly.

But my main point is this: the vaunted analysis tools of the true engineering disciplines apply only to narrow slices of the overall requirement. Everything else comes down to smart people caring about results.

I have two success stories, neither of which involves formal engineering analysis. At my last dayjob, as software manager of an RFID startup, my department nearly always had software ready before hardware, with only minor bugs. We did it because one other programmer and I both had enough hardware knowledge to create breadboards for advance development. Nobody in our group ever had to wait for the first board spin. That fad was called codesign. We made it work.

More recently, for one of my contract software clients, I created a simulation of their machine on a Gumstix, which I wired to their controller. That let us test their software on their controller, without having any of their machines in house. I guess that fad was called simulation.

In my less than humble opinion, the real issue is complexity, and it spans more disciplines than just software.

1/12/2010 6:22 AM EST

Bad programmers will make bad code no matter what process is used.

Good programmers make good code even if there is no process.

The split between good and bad programmers is about 20%/80% or maybe 10%/90% so make you hire the good 10%.

1/12/2010 6:27 AM EST

If you want true engineered code to EE standards then use the limited C that can be compiled to FPGAs. This is available now!

So no OS, no recursion, no allocation of memory, etc. Just KIS I/O rules all running in parallel with some state machines. If you are not sure what KIS means try KISS.

svik

1/12/2010 6:55 AM EST

My Favourite Rules:

Use Modules. Hide you mess inside.

Define the architecture hierarchy in terms of the same modules. Define a linear order from drivers to main to match your .H files.

Oh and do define an interface to each module (i.e. .H or tpu file). This is obvious but usually never done fully! NO files like global.h, functions.h or syswide.h allowed.

Prove that your logic or function calls is not recursive. All fcn calls should be downstream. Document each exception where an upstream fcn call is used.

KIS or KISS if you don't underestand KIS.

Do code reviews. If another programmer can't understand it quickly, fail it.

Do a simulator at the driver level and test your code on the desk top. A partial simulation is fine as it usually still allows you to test 90% of the code.

Compile you embedded code for Win32 using free GNU or VisualC. Fix all those warnings.

Build in testability, log files and stats trackers.

I owe, I owe, so off to work I go!

1/22/2010 6:26 PM EST

Agreed with almost everything "svik" said above. Looked like he belonged to the 10%. But sorry about the statement "Compile you(r) embedded code for Win32 using..."
Why Win32 or other cross compiling? Shouldn't that be "test what you run, run what you test?"
For sure you can cross-compile millions of combination of compilers and targets (include simulated ones) to "make believe" your code is perfect. It takes only one failure to be fatal, which happens to be on the actual target. In this case, it is still a big success because 99.9999% test cases passed.

3/13/2010 2:02 PM EST

I saw no mention of the tools used in chip design for layout, synthesis, or timing analysis. They are all involved in finding paths through logic networks in order to make the physical circuits work. This is aside from the EE physics involved in each circuit.
Where are the tools that trace the flow through the logic of a program to measure how many paths are actually exercised?
I am an EE, computer architect/designer, and understand enough about programming to sense the analogies. Software engineers need to learn a bit about the problems of chip design and pick up on how the design tools work. No, not learn chip design, just peek over the fence.

5/9/2011 1:29 PM EDT

Software Engineering (SE) is not really a science and neither is it an art.

Software Engineering is really a system of logic. However, in its current practice it is very disorganized because no SE standard approaches are set. This is why SE is such a mess. (The current SE approaches gives programmers enough rope and loose ends to hang ourselves several times over.)

The solution is not to call upon a Newton or Einstein. What is needed is not a scientific genius. But rather a "organizational" genius that organizes the hodge-podge sets and variety of tools, techniques, approaches, etc. into a highly organized system of logic (which includes all the valuable elements of combinational logic, sequential logic, conditionals, object oriented contructs, etc. that make up the current programming landscape and set up a easy to understand and disciplined approach that will theoretically and in a practically verifiable way guarantee a highly reliable logical system.

Since SE is a logical system I suspect the ideal system would have to be amenable to be fully described with mathematics or a very rigorous form of logic in some way. This might put some constraints on what software elements are allowed but if properly designed I think we may be able to provide 99% of what is really needed in a software system. And the benefit will be a mathematically or logically provable system in the end.

Any "organizational" genious out there that want to take up this challenge?

5/11/2011 5:45 AM EDT

What I've been saying for the last thirty years or so is that software "engineering" isn't engineering at all; it's a craft, a skilled trade. We're not even on the level of architecture at the time of the Cathedral at Rheims; ridiculously over-engineered by today's standards as it was, its builders could confidently and with reason say that it was fit for purpose. Of how many of our artefacts can we say the same?

I agree with the symptoms cited in the original post. I think what is going to continue to kill software as a would-be profession is precisely its open, anybody-can-jump-in nature. Until we have a few spectacular software-driven disasters, modern analogues to the New London, Texas school explosion of 1937, we'll continue to do things the way we are, or have things done to us the way they are now. The bean-counters are In Charge, and no matter how pathological the process is, attempts at fundamental change will be futile so long as that remains the status quo.

1/23/2012 9:01 PM EST

Hello,

I would like to make a quick subtle point without overly simplifying things.

In EE, E=IR, KCL, KVL, etc. are derived from Maxwell's Equations, and from this fundamental physical law, which states mathematically what had been observed and thoroughly measured in the physical world, we have the basis of physical "passive" electrical phenomenon. Quantum Mechanics/Field Theory extends this to cover diodes and "active" electrical phenomenon. (Note that for our purposes, I am not including relativity, because relativity is built into Maxwell's Equations.)

Similarly, the ME world is overwhelmingly dominated with the applied physical applications of F=ma.

Software is different. Software is a form of crystallized thought, if you will, with a healthy use of Mathematics(especially Logic), at least so we would hope.

While Physics, and thereby it's applied offspring EE, uses the language of Mathematics to communicate observed and measured physical phenomenon, I would argue there is no fundamental physical parallel nor counterpart for our friend software.

Whereas Physics is the pinnacle of "Hard", objective Science, which makes ample use of Mathematics, Software is far more of a Humanity, just like its friend Mathematics.

While I applaud the desire to consider taking a highly "observation and experimental" approach to s/w, I think it might be more prudent to approach s/w and s/w fundamentals as individual proofs which need to be undertaken (just like in Mathematics), and then later refine and heavily simplify the results to arrive at a series of practices for practitioners, just as Engineering tries to do with taking fundamental physics and putting it in a more "user friendly" form for its applied practitioners.

This will take some time, if even possible.

1/24/2012 7:26 AM EST

Dear colleague,
dear friends,

I fully agree with your article.

The software engineering scene is corrupt.
I submitted a "view point paper" to Comm. ACM.
It has been rejected within less than 24 hours.

Meanwhile we have also new aspects:
The dominance (or, the monopoly ?) of the von Neumann paradigm cannot be tolerated in the future. We have to leave the Aristotelian world model, where the CPU is the center of the computing world.

The supercomputing scene has started anumber of new conference series, which have the word "heterogeneous" in the conference series title. They have learnt, that also accelerators (typically non-vN) are needed

The von Neumann syndrome is the reason of massive inefficiency, so that we have an electricity consumption problem.

vN-based software packages and even modules are much more complex then a human being is capable to master. Sure, you know "Nathan's Law".

Burton Smith (ex-CTO of Cray) said: we have to reinvent computing. Also see http://www.inf.pucminas.br/sbc2010/anais/pdf/semish/st03_02.pdf

The software-only programmer is a disappearing species. Programmers have to cope with multi paradigm thinking: software & configware & flowware (datastreams). We now have the dichotomy of two machine paradigms: 1) one for software, controlled by the program counter, and 2) another one for data streams, controlled by data counters. In a heterogeneous system both kinds of machines have to communicate with each other.

We have to reinvent curricula since programmers have to be ready to cope with this new world model, where the CPU is not the center of the world.

Best regards,
Reiner