@Andrew: I think this is a good point. As a guy that works with a program that, while complex, is amazingly simple compared to an operating system, I’m constantly impressed that these OS’s work as well as they do. Trying to create a system that effectively lets anyone hook any device up to it running any sort of driver and accessing any sort of software is a monumental challenge. True, for a commercial product there is a certain expectation of quality, but frankly I abuse my OS so much that I’m surprised it runs at all.

And on a side note – using Vista here and at home since it came out – one crash ever – every peripheral worked the first time – all my software works – and on top of that, better UI experience than any I’ve had. *shrug* For what it’s worth!

I don’t think Linux is any better then Windows when it comes to testing software and then removing it. I have had plenty of issues with missing dependencies and incorrect package versions. Linux suffers from the same type of “DLL Hell” as windows.

The real issue is that this is difficult stuff to do. If Application A uses libX and Applicaiton B uses libXv2 you can easily be in a situation with 2 different versions of the library on the same machine and both are used by different applications.

Also, both Linux and Windows allow the user to install and remove apps outside the package management system, so accurate counts of which apps use which libraries is hard to maintain.

Sorry to get off topic, but i get Angry when the Linux/Apple/Windows (name your OS) fanboys say “My OS doesn’t have this problem that yours does” and usually it is “Windows sucks, Linux/Mac is better”. Guess what, they all have similar design and similar issues. Apple controls OS hardware and software so they get tighter integration and have to worry less about driver compatibility, but Apple boxes do crash, and people do have issues with shared libraries. All OS have pros and cons, but all get the job done.

Marc — keep up the good work man. Take your time, it’ll be ready when it’s ready, though I am waiting with bated breath.

As for the experiment you mention in your post I completely agree. There is no purely scientific way they can go about this. It’ll be ‘interesting’, but nothing more. Unless these guys plan to completely dissect the source of each and every anomaly, all they will do is shovel meaningless dreck on the ill-informed masses and give honest developers a black eye (Rupert Murdoch?). I can only hope that they preface any findings with disclaimers about the ambiguity of the results. I think we can all agree that there are substantial shortcomings in all operating systems, and also that there are some very poorly written software packages written by good developers working on unrealistic projects. All you guys can do is keep writing great applications and respond to your customers’ needs. Let the herd cull itself, you guys will be around for years to come.

Take care,

-ben

Interestingly, since I enabled the nightly backup+reboot, I rarely need to reboot during the day.

I don’t think it’s reasonable to blame third party applications for needing to reboot. That’s the whole reason why I have a Windows Mobile device — so I can use third party apps and do more with my device. That’s a significant feature.

The problem is the design of the third party applications. On my Linux box I can install, try and remove applications trivially, and with no affect to the system as a whole. The difference is that there’s features that curb and help avoid poor application design issues. Priviledge separation helps avoid applications abusing system function and resources, and a solid package manager keeps install/removal clean. These aren’t unreasonable expectations for a mobile OS.

Problems can be avoided with careful selection of applications… but that’s hard when I want to buy a new product. I’ll try out the various competitors – and once in a while I’ll get a dud that causes problems.

James

Great post! You are correct about what I am hoping to show by doing this challenge. I know it’s going to be hard to prove beyond a doubt, but hopefully this is a solid start. You raised a number of key concerns and issues. There’s just so many factors involved that I can’t possibly rule them all out and create a truly controlled environment for this test.

Ultimately, it would be best if the OS protected us from unsafe and instable applications. But the truth is that we are dealing with a mobile OS and there just isn’t the resources to handle this correctly. Heck, the full blown desktop OS still doesn’t do an amazing job at this. Although, it’s getting better but there’s a long way to go.

I didn’t mean to imply that all 3rd party applications are bad. In fact, the reason I selected a few apps to install is because I wanted to show that there are good developers and that you can find good products that won’t compromise the stability of your device. IMHO, Ilium is one of those companies that I know I can trust to build a solid app.

Cheers!

My experience with the company has been first rate all these years. In fact I jumped on eWallet for the iPhone just as soon as it was released. (I even did this before I found out about the upgrade allowance on their web site).

The point is this company has an outstanding track record with their installed base and has consistently well publicized upgrades to the product. Sure, I would love to be able to sync with my Mac today, however, I am absolutely confident Ilium will deliver a first rate product that will work “out of the box”.

Blew Smoke Up Our Rears: Maybe its because for so many years companies have been taking advantage of their customers that make people assume the worst. We very honestly described what we did and why. That’s the truth whether folks believe it or not. I can’t offer something different. I won’t repeat it all again but it isn’t double speak or BS. I’ve been shooting straight right from the start.

Truly Half-Baked: There are a ton of applications in the store AppStore that don’t have a desktop component. This includes other secret keeper apps. eWallet as it is right now is an app I’m extremely proud of. It does it’s job extremely well, even without sync. It is a fully functional, complete application. Will it be better when sync comes out? Absolutely! Is it a great app as is? A lot of folks thing so!

Bad Taste for Loyal Customers: We never promised anything that wasn’t true. In fact we rewarded folks who bought even without sync. And in the end, we’re coming out with sync pretty darn close to our predicted mark. I can tell you’re frustrated and I’m sorry, but there is no way we will release an app that doesn’t do exactly what it should exactly AS it should. If that means a few extra days of development, then we’ll do it. There are many reasons we have loyal customers, and a big one is that when we release something, by god it works and works right…and we don’t manage that by releasing something we aren’t 100% satisfied with just to shave a few days off a calendar date.

My SP2XP box has almost uptime on it as my friends Linux box, and many Windows servers (at work) also have uptimes that equal Linux and Sun boxes. Most boxes are only rebooted for patches, and some of these we put off for safety sake (Oracle and Sun patches are scariest of all).

My Windows Mobile devices have been pretty solid. The only issue i ever really had was pocket IE hosing things up, but I can crash a BlackBerry by going to a javascript filled site.

I would love to see an OS not crash, but as long as we the users expect to have plug and play and other conveniences where user code interacts with kernel, then we have to take some chances on stability.

I think all OS manufacturers truly strive for a bug free, rock solid OS. But we all know that bugs do happen, and sometimes it takes a user doing something a developer never envisioned to make that bug appear.

eWallet never once crashed any OS, and, on second thought, has never itself crashed on me.