Embedded Linux: With friends like these, who needs enemies?

5/5/2008 1:30 PM EDT

I couldn't really tell what you were trying to say in your column until I read the author info that said you work for Green Hills. While I think highly of your products as well as the overall quality of Embedded.com I think your piece should have been clearly marked as "op-ed". Having said that, it's fair to wonder what Embedded Linux vendors are bringing to the table. I've evaluated Embedded Linux vendors more than once and have come to the conclusion that time and money would be better spent learning kernel details than costly proprietary tools.

5/6/2008 3:28 AM EDT

Well they gave you the tinder and you lit the fire ! If both companies used reverse psychology to get newer customers, you've taken them to task generalising their non-official statements as company strategy.
This article has the quality of a blog post more than that of an embedded.com article.

Morever even though I don't support either of these companies, I do use and propagate embedded linux. Its still a boon for a large amount of companies who require embedded systems (not just devices).

Green Hills itself sells dev tools that are prohibitively expensive to mid-sized users or smaller companies. Embedded linux liberates those users who still have a blatant need to put an OS in their systems.

Doug, as a CEO, you could use embedded .com in a much better way by telling us " How to build an LOW COST Embedded System with the Green Hill SDK". I think that would be much more readable.

5/8/2008 9:33 AM EDT

Rich, you forgot the summary. Allow me:

"Competitor takes embedded Linux vendor sound bites out of context, spins them for maximum FUD."

Had Green Hills just come out and said that this was a shill piece, it would have actually been an improvement.

5/8/2008 12:15 PM EDT

$19,623,750 for three year of deployment? Try less than $200,000.

We started with a snapgear distribution for a reference board and a copy of Linux Device Drivers and have used linux for multiple projects shipping 5000 per year for 4 years. This took about 1 1/2 man years for linux configuration and support including the time my sharp CS partner took to learn how to configure and use it and for me to learn how to write drivers.

A commericial operating system might have been faster due to telephone support but even it would probably have take 1 man year.

5/8/2008 5:10 PM EDT

Why do you waste our time with the drivel from this clown? This is the guy who two years ago said:

"Linux in the defense environment is the classic Trojan horse scenario -- a gift of 'free' software is being brought inside our critical defenses. If we proceed with plans to allow Linux to run these defense systems without demanding proof that it contains no subversive or dangerous code waiting to emerge after we bring it inside, then we invite the fate of Troy,"

Oh and my the way his company sells
their own proprietary OS, geez what a coincidence.

I'm coining a new term: FUBS (F*$!#ing Untrue Bull S*$*)

5/8/2008 8:52 PM EDT

A paid advertisement disclaimer as the first paragraph would have been helpful!

BTW: My employer has spent over 80K with MontaVista. Over 6 months later, Still can't use their product.

I've built a kernel and tools from free and available source off the web that runs a heck of a lot better.

fredskber's post tells a good success story too.

All of these company's marketing is just hype.

Chuck Wright

5/9/2008 11:30 PM EDT

RE: Embedded Linux is the most hyped embedded operating system ever.
i don't think so. Ever hear of Microsoft's Auto-PC/Windows-CE/Windows-Mobile/ ???

As for Linux costs: (following in the articles style of self-promotion) we build, ship and support tens of thousands of Linux systems a year spread across multiple products for each of multiple customers.
We design custom products (Linux on ARM) for each customer, and ship a custom Linux image for every product.
Custom HW designs cost $10k to $25k, and porting/customizing of the Linux image is typically $5k PER PRODUCT.
We deliver working protos on custom HW with a custom Linux image in 6 to 10 weeks from contract signing.
Our customers write their own applications that run on our Medallion Linux. (in C/C++ or Python or whatever they want, or we supply our custom port of IBM's J9 VM for J2ME/CDC applications)
We've been shipping 2.6.24 since last year (we keep current)

For products with an LCD, touch-panel, GUI, Ethernet and/or cellular comms, Linux is the lowest cost and most robust option out there.
If there was something better, we'd be using it.

And please don't discuss companies such as Wind River who are completely new to the game as a basis for your arguments.
Finally, there is no such thing as Embedded Linux .... its just Linux!

5/12/2008 7:31 AM EDT

An update for you, Dan.

Linux, embedded and otherwise, *is* being used in the defense industry. Ever here of SELinux? I have a relative in the service, and he's told me twice about systems he uses that show a Linux boot screen.

Why is Linux so popular?

Because it's *open*. You can see exactly what you're getting, get help from others using it, and make it do exactly what you want it to do. Really, it's not that hard.

Hmmm...I wonder if Dan might have a vested interest in running down embedded Linux?

5/13/2008 10:05 AM EDT

After reading this article I convinced myself to never purchase any products from Green Hills. A CEO with such an arrogance may actually doom Green Hill. It sounds as though he never read "Mein Kampf" by Adolf Hitler? By the way, Hitler never made it.

5/15/2008 10:06 AM EDT

This is shameful anti-marketing. Get off the soapbox, Mr. O'Dowd. Indeed, if anyone makes a case for Linux, it's Mr. O'Dowd himself. If engineers were running from embedded Linux, the article would be unnecessary. Rather, the drama is put forth to sell proprietary OS's, not on their advantages but by highlighting the competition's sensationalized disadvantages -- falsly implying that GHS does not suffer from these problems. Is O'Dowd saying that GHS never patches anything, or that no one waits, or that he has not used OSS code in any way? I have run Linux and embedded Linux for years and years, and with all the patches and whatnot, it is still a better OS and the applications and utilities are still better than anything I could buy, with a certain few exceptions. We have a project now for a $400,000 Broadcom development kit written for VxWorks tha will not compile without a 3rd-party patch that can no longer be purchased and its publisher will not supply for licensing reasons, so the kit is useless but we need it and it is seriously hurting our business. Now *that* is a tragedy. Closed-source and proprietary code are no less prone to trouble, bugs, security patches, etc. One positive aspect for Linux is that if it is critical for you, and no one else is fixing a bug or adding a feature, you can do it and supply the change to the upstream project. With GHS et al, you're at their mercy and their "business focus" and their "business needs", not yours. They are after your money, not your success. Linux, and OSS, flaws and all, are the polar opposite.

5/15/2008 3:00 PM EDT

Reader comment submitted via e-mail:

What the heck is "embedded.com" thinking with this article? I have to say that I am not a major embedded Linux fan, and when I
first saw the article title, I thought it would be a good read. Then, the first thing I scanned to was the author, and being from
Green Hills, that immediately raised a flag regarding the obvious fact that this guy has vested interest in bashing embedded Linux. The content of the article just made it more obvious.

I realize that there are probably months that you guys have trouble trying to fill the pages of the magazine with good technical stuff, but this article is pitiful. Even if the negative Linux hype is all true, just the fact that the article was written by someone from Green Hills discredits it.

Maybe if there was a version of the "Enquirer" for embedded systems, this article would fit well there. The editors need to keep a clear separation between "articles" and "advertisements" otherwise, I personally think you're discrediting the magazine.

--Ed Sutter

5/15/2008 3:06 PM EDT

Reader comment submitted via e-mail:

I find it very sad to find this article in your otherwise high-quality magazine under the heading "guest editor". Under that heading, people may actually put some stock into it. A more appropriate heading might have been "Opinion" or even "Company shill" or "Paid programming". I can understand that Mr. O'Dowd wants to downplay the competition, but he actually may have done more damage to his own image and that of his company by resorting to scare tactics, out of context and partial quotes, and exaggerations instead of some real, solid arguments. Since he is CEO of a company that makes a product called "Integrity", I had expected more from him.

MontaVista and Wind River Systems are both companies that are trying to make money off of software that is available for free. How can they convince people to pay them money for something that is free? By offering extra tools and services, and by offering their expertise. It is much easier to sell your tools if you can make your potential customers believe that the tools that are freely available are inadequate. It is also much easier to sell your expertise if you can make your potential customers believe that something is difficult. These marketing messages are intended to benefit the respective companies, they are not intended to show Linux in the most favorable light. After all, how would you convince anyone to pay you money if you would advertise that embedded Linux is a breeze? The marketing departments of these companies have to thread a fine line in promoting Linux while at the same time convincing people that it isn't easy, and that they will need help with it.

Now Mr. O'Dowd takes these moderately "negative" marketing messages and twists them into such strong statements as "terrible", "horrors", "monster" and "nightmare". I'm sorry, but I haven't seen these words used by the companies he is basing his rant on. I haven't read the complete MontaVista article he quotes, but it seems to be highlighting the work they have to do in order to support many architectures and software packages to make a full distribution. If you're using Linux to develop a product, you are using a single architecture and a select set of packages. Often, your development board even comes with a Linux dev kit and Linux loaded on the board and ready to run. Making the work that MontaVista does, and the work that someone making a Linux product has to do, sound as if they are the same thing is more than a stretch. I won't even comment on the Wind River "CHAOS" ad. I have seen the ad and talking about only half of it and not the other half should make it clear how genuinely Mr. O'Dowd is interested in being fair.

As to his claim that these companies are in trouble, and somehow equating that to embedded Linux being in trouble, this argument would only sway those who are ignorant about how open source works. Unfortunately, this is still a large percentage of developers, so that's why I thought it important to send this response. If these companies aren't making as much money as they hoped, could it be that it is because embedded Linux is simple enough for most companies to tackle without their help? Due to the nature of open source, these companies might fold tomorrow and developers would continue to put Linux in their devices just as well. Could the same be said about "Integrity" if Green Hills were to fold tomorrow?

Of course there has to be a survey thrown in for good measure, so I'll contribute one too (http://www.linuxdevices.com/news/NS4920597981.html). It is well known that anyone can make a survey say what they want it to say. The one I found says Linux is the most popular embedded OS for the fourth year running. The one Mr. O'Dowd quotes says less people are considering using Linux because more are actually using it (those are separate survey options... get it?). Doesn't sound to me like we're talking about a "dwindling number of disenchanted embedded Linux users" as Mr. O'Dowd claims. By the way Mr. O'Dowd, what are the stats on your embedded OS? Your article has all the twisted information, exaggerations and other telltale signs of someone who is desperately trying to tear down the competition. Is fair competition not doing it for you anymore?


--Patrick Van Oosterwijck
Senior Software Engineer
EcoWater Systems

5/15/2008 3:27 PM EDT

Reader comment submitted by e-mail:

Dear Mr. Nass,

I'm sure I am just adding to a barrage of response you must have received to this article, but I feel that it is necessary for you to know how badly it has reflected on the otherwise good quality of your magazine.

Although I get the point that two (previously unheard of) proponents of 'embedded linux' systems may have decided to condemn it, that doesn't mean an article should allow it's tone to verge so far off course. Dan
O' Dowd is obviously a man of FUD and I hope his misuse of Embedded Systems Design has allowed ESD to create new filters for such
uninformative and misleading information.

It's a shame that MontaVista and Wind River Systems can't get their act together, but there are plenty of us out here who most certainly can. We will enjoy inexpensive development of embedded linux systems on a
wide range of processors and hardware configurations where these others have failed.

Kind Regards,
--Wes Cravens

5/15/2008 6:18 PM EDT

Embedded.com editors -- This bickering doesn't belong here. This is an online magazine, not a boxing ring for companies to take whacks at each other. I'm ashamed to find this journalistic style on an otherwise fine website. Do us a favor, editors - please don't allow it any longer. Please raise the journalistic bar back to where it belongs.

Green Hills - I've been thinking that Green Hills might be a good RTOS for our next-gen product. I'm going to have to reconsider based on your apparently-flawed value system.

5/16/2008 4:50 PM EDT

FUD or not FUD, let us wait for their next move.
I'm sure that it will explain us the trick.

If a next move will fail to run, than we'll be reasonably inducted to evaluate them as a kindergarten crew.

5/22/2008 10:40 AM EDT

Reader comment sent by e-mail:

The article mentions Wind River and MontaVista and details some of the
numerous outrageous claims by these companies.

Thing is, Green Hills is also actively making similar claims and the
article doesn't debunk them, it just states them. This is a way to join
the others and get out their negative message about Linux.

In passing it should be noted that both companies were dragged into
Linux by customer demands. Both companies continued to run their
business as a proprietary venture while supporting Linux. As the article
points out this doesn't work.

I worked in a company that investigated buying development tools and
kernel from various vendors. In the end we found the companies difficult
to work with, limited in what they would support, and expensive. It
should be noted the developers were not interested in the tools since
they already had development tools that came with their desktop
distributions. Desktop Linux and Embedded Linux has grown to be pretty
self sufficient.

The embedded designs that I have been involved with revolved around
ASICs that included a CPU. All of the chip vendors provided a supported
Linux kernel, root filesystem, and cross development toolchain. In
several years of working in this environment there were remarkably few
problems that went back to the vendor. Actually, most of the problems
have to do with the code we are writing, there are seldom problems with
Linux.

Beyond that, a web search will find projects that focus on Linux for
specific processors. These projects do processor specific development
work and feed the results to the kernel project. You can tap into the
developer community and get a lot of support. There are also (usually
gcc based) cross development tool chains supported by (or linked to by)
the projects.

The patching and security issues don't relate to us. The vendor provides
us a kernel from kernel.org, 2.6.18 at present, and a set of patches. We
have made a few additional patches to support our hardware. We rarely
get a new patch set from the vendor. Security is of limited concern. The
users can only interact with the device through a GUI that is part of
the product and the product doesn't connect to the outside world.

Their cost figures are stunning. For $192,000,000 you can write your own
RTOS that is similar in features and quality to VxWorks, QNX, LynxOs, or
any of the other RTOSes.

You don't have to scour the internet for patches. Many distributions and
CPU projects do that for you. There is a lot of practical support on the
internet and through vendors. Actually, I think the last security patch
for a core Linux module was a couple of years ago.

Linux is very modular. That is why it is a good fit for a wide range of
systems. The core Linux is pretty small and you only need the
architecture support, device drivers, network stack,and filesystems
that are used in your product. This is, in our case, a minor fraction of
the complete Linux.

It would be very good for you to publish an article that debunks this
article and explains why Linux is so important in the embedded space.

--Philip Cameron

5/24/2008 10:48 AM EDT

Capt Frito said it very well. It's ridiculous to propose that one is less likely to encounter uncontrollable expense or unsolvable problems by relying on any closed source software company rather than on source code over which he has complete control.

6/3/2008 10:21 AM EDT

Linux is very fast changing OS in the market. It is because it is open. If you increase production you likely decrease the quality.
Linux has the problem of maintaining. But over all Linux is the cheapest solution.
If I am to design in Linux I wouldn't choose purchasing it from Windriver or Montavista.

I also worry about licensing issues of GPL.
Will every linux user agree the terms ?
As a RTOS, Linux and derivatives will never be my choice in my serious projects.

8/21/2008 3:45 PM EDT

If I were to develop my custom system in my garage i don't see even second option to Linux.
Debugging the issues in opensource Linux gives me more insight in to the system software and helps to develop/evolve my expertize.
Experts in community are there to help me for free.

After a team does so for a specific product from scratch, they already got many experts and then who needs paid expertize from external sources.
Then who understands your product better than this team! Isn't this so difficult to baseline most stable system software of the product for mass production.

I wonder if these costly(in many ways) RTOS and even more costlier tools will ever help me with above goals.

8/24/2008 10:58 PM EDT

Linux is a great system. It is free and comes in a thousand flavors. It has been good for the embedded systems industry and certainly good for QNX. We sell QNX and SMX so stop here if you want.

What worries me in any discussion relating to Linux VS commercial is the emotion that Linux devotees apply.

Building an embedded system should be a clinical process. Embedded systems are generally built out of a profit motive and the companies concerned want a number of things. The lowest possible development costs. The quickest time to market. The Highest possible product success. And a great market reputation that builds future product success.

Forward thinking companies need to be able to reuse and build on previous developments at the same time as allow for whatever requirements may be thrown into the pot later. Linux helps with some of these things.

We all “bag” companies like Windriver and IBM for one reason or another because they are big and have broad shoulders. What they offer for their dollars is significant. Reliability of supply and a wealth of professional support that “TARGETS YOUR PROBLEM”.

I have accepted for years that it is clearly possible to build successful embedded systems with Linux. Clearly, and from experience, it is not always easy and developers should be honest with their bosses about this. Linux is a vehicle. One vehicle is not good for all occasions. Devotees of any sort often ignore this fundamental truth.

It is certainly true that starting out in your garage to build a widget is easier with Linux. No up front costs and lots of assistance. But when you get to 98 percent (the well known barrier) you want to be resolving application and product issues not fundamental OS issues.

The article under discussion represents comments and criticisms of Linux that are widely discussed (valid or not) and it in no way diminishes the magazine.