Break Points
Programmers are people, too
Jack Ganssle
10/1/2008 12:00 AM EDT
The Agile Manifesto reads, in part, that the signatories value "individuals and interactions over processes and tools."1 The field of management was revolutionized by "empowering" people to make decisions and take charge of their work, which led companies to realize the importance of hiring wisely.
People matter.
In my experience, most companies do prize their engineers. No, they're not given CEO-like rock-star status, and we all wish salaries were higher. But engineers do make a decent middle-class wage, and in these days of nearly full employment in our industry, businesses fearful of losing their developers generally treat the engineers well. I do hear some dramatic exceptions, but most correspondents claim to be satisfied with their jobs.
But there are persistent complaints that never seem to go away. The first of these is overtime; the 40-hour work week is but a dream to many, and panicked overtime is de rigueur at many outfits in the last few months of a project.
Tired people make mistakes. You'd think it would be self-evident that overtime leads to buggier products. Or that safety issues in dangerous environments will escalate when workers take a shortcut that they'd never consider in a less sleep-deprived state.
In my collection of embedded disasters, a common theme is tired engineers. Investigating bodies routinely cite "60 to 80 hours of work a week for months on end" as a contributing cause to a system failure. And, since only the most expensive systems, like space missions, are investigated when something goes wrong, the dollars lost when a weary worker enters the decimal point in the wrong place are staggering.
Most professions suffer from the curse of OT. Doctors get midnight calls from the hospital, lawyers work late in the night to respond to an unexpected judicial ruling, and accountants dread the arrival of tax season. Is this good? I doubt it, and mounting evidence suggests medical mistakes, no doubt at least partially driven by tired people, kill.
Maybe we should plan better, but even the best planning fails when something unexpected occurs. And the unexpected is one of the things we should expect most when designing a new product.
"Unexpected" sometimes comes because we're confused about R&D. One of my top ten reasons why projects fail is because the science is bad: engineers are developing the science in parallel with making the product. For instance, the algorithm changes constantly as we play with the system's chemistry to get meaningful data. That's a sure-fire route to scheduling disaster, and often results in cancellation of the project. I have seen companies go out of business because engineering is so wrapped up in R&D they never get a product to market.
We don't do R&D--or, if we do, we shouldn't. It's either R, or it's D. Sure, development contains an element of research but is mostly about achieving a pretty well-understood objective using known science or algorithms.
It's possible to plan D; no one can schedule R since that intrinsically explores the depths of the unknowns. If research could be scheduled, we'd know when the cure for cancer would be available.
Although it's possible to plan development, it's impossible to be perfect, so overtime will never go away. Wise managers, though, understand the costs.
Circadian's Shiftwork Practices 2005 survey found that productivity can decrease by as much as 25% when workers put in 60+ hour weeks.2 Clearly overtime leads to diminishing returns for everyone involved.
|
|
|
|
|
klenkka
10/1/2008 2:45 PM EDT
Maybe I'm not one of those superprogrammers, because I didn't know what OT, CS, EE, ME or CE meant.
Well, know I know. Hooray almighty Wikipedia.
Sign in to Reply
Theckyam
10/2/2008 9:11 AM EDT
That's a wonderful take. I should just agree completely that high activity is no substitute for prevention. And the Chinese parable itself is a great analogy. Its wonderful to have Jack just speak *my* heart out.
I only wish that some or any of the leads/managers read this and see what they think about the costs of not preventing & the rewarding of solving problems that should have been completely prevented in the first place.
Saravanan T S
Sign in to Reply
Crous
10/2/2008 10:53 AM EDT
Jack,
I now understand why Micrium is not as known as it should be!
Micriums message to the embedded industry is that there are best practices that can be used and will help you develop better products faster and at less cost.
Micriums naming conventions, coding standards, source code quality and more are like the eldest brother abilities in your China family of healers tale.
It is always a pleasure reading your column.
Regards,
Christian Legare
Sign in to Reply
Don McCoy
10/2/2008 11:28 AM EDT
Interesting that people with BS degrees weren't mentioned, specifically Physics and especially Applied Physics. I think about half the really good software engineers I know have a background in the sciences.
Communication is a huge issue in this industry as I've learned from working with EEs. I can see how someone with good skills can help bridge the gap between those writing the code and those designing the hardware.
Regarding the Dilbert cartoon: I remember when that came out and we laughed over it. Funny how that works -- the laughter covers the pain from knowing exactly what it is like to be in that situation where overtime is over-valued as compared to front-end planning and good design methodology. After failing to change the culture where I worked, I changed jobs...
Great article. Thanks!
Sign in to Reply
robert.berger
10/2/2008 1:19 PM EDT
Hi Jack,
Great article as usual and it's fully inline with my way of thinking!
I am traveling the globe promoting "people over things" (tools, processes,...) and it's hard for me to comprehend all this insane amount of overtime engineers make. I'm officially against it!
Obviously tired people make more mistakes than people who had a good rest, but let me throw in another reason for people to do overtime:
I believe the boss likes people who work many hours due to the fact that it's easy to count minutes for evaluation purposes. In order to evaluate performance he would need a much more elaborate process and tools and on top of it put extra effort into it.
A working environment like this of course doesn't provoke any kind of performance improvement, but people working long hours.
Regards,
Robert
---------------
Robert Berger
Embedded Software Specialist
http://www.ReliableEmbeddedSystems.com
Sign in to Reply
muttusagara
10/3/2008 1:02 AM EDT
While its true that we are all pushed to our limits, in my opinion it is our own making..... Some of us (the super achievers among developers / managers / want to prove we are the best and will be prepared to bend over backwards to prove it. This makes an efficient average performer to also be pushed in to this leage and in the end we mess with our own lives... we live an unbalanced life and further mess it in our personal life too.
Sign in to Reply
daleinaz
10/6/2008 1:31 PM EDT
Robert Berger is correct, many companies value "face time" over productivity. That is one reason (there are others) why telecommuting hasn't become more common. I'm not averse to putting in OT during the bringup/debug phase of a program, once or twice a year, but not on a continual, week in, week out basis. I've seen companies spend more time justifing and defining a product than they allocate to design and debug.
Sign in to Reply
Nancy Van Schooenderwoert
10/10/2008 1:52 PM EDT
Jack - you mention teams that must do research *while* doing development, and how this inevitably gets them into trouble. But companies are increasingly having to figure out products as they develop. There are 2 paths one can take: separate R from D, or find a legitimate way to do them together. In the 1990's I worked on finding some techniques for embedded developers to do both together without compromising quality, and without blowing the budget or schedule. Working with various contractor teams I picked up a number of great ideas that I believed gave this ability to do both R and D legitimately together.
On a real project that had to implement an algorithm which was in flux, my team succeeded in delivering releases on time, and they averaged close to a 40 hour work week throughout the project. There were other significant unknowns handled successfully too. Quality was documented at being an order of magnitude better than the best teams that Capers Jones had ever measured. I collected data all through the project because I wanted to have more than just anecdotes when it was over.
The whole documented story appears in a paper I gave at the Agile 06 conference entitled "Embedded Agile Project by the Numbers with Newbies" which is available at http://www.leanagilepartners.com/publications.html or by emailing me at vanschoo at acm dot org.
A long time ago when I worked at GE I remember feeling quite put out when management announced to us that we all were to manage ourselves from now on, and by the way they wanted us to strive to produce ten times as much product as the year before! Now after all these years, I know what was really happening - they were up against competition that knew how to legitimately and gracefully combine R and D to get real work done fast. But they didn't understand fully the mechanisms that made it work. So all they could do was to just pile even more pressure onto the working engineers and everyone in the management layers, too. They were trying to copy "lean thinking" by misunderstanding it to just mean severe cost cutting.
We are in the middle of a change that will sweep away all the businesses that are trying to compete by just ratcheting up the pressure. A few companies are discovering how to *really* use lean and agile concepts in their competitive strategy. They will be able to outmaneuver the competition without even breaking a sweat, and will displace many current industry leaders.
- njv
Sign in to Reply
EREBUS
9/1/2011 8:41 PM EDT
Hi Jack,
You hit on a good point. In my experience, no highly skilled coder or process can compensate for an bad system design. Developing on the fly may get you a product, but will it be the right product and does it all work? Also, I have found that if you rely on the "super" coder, you also die by the super coder. When they are good, you win, but it is only a matter of time before they have an off day.
Give me a group of dedicated capable people and a solid process and I can get the product out without much difficulty. Super Stars can be a distraction if not a detriment to the "team". Teams build good reliable products.
As for the abuse of OT, I made it clear to my managers that for every hour I worked past 8, they lost 25% of my productivity the next day. They could either have a fully capable engineer the following day or a brain dead hazard. The choice was theirs.
Sign in to Reply