Here we are, celebrating:
OK, not really. The party is next week. But I made everyone put on a hat and pose for the picture so I could use it in this post.
Check out our home page for Julie’s fabulous falling hat show.
Share this article:
This picture illustrates two things. One is that I have terrible handwriting, especially on a white board. It also gives a glimpse into what software testing is like here at Ilium Software.
I’ve been meaning to talk about testing, but between testing eWallet 5.0, InScribe 2.0 and just finishing up on a Keep Track update, not to mention supporting our existing and new software… there just wasn’t time. If you have some time, read on!
Aside from technical support, I do a lot of the software testing for Ilium Software. I don’t do all of it because that would be impossible. Not just figuratively impossible, but literally impossible. Thanks to math - which I absolutely hate and am terrible at - 100% ’test coverage’ of a product is intractable.
Digression warning: 100% test coverage for something like a full PC operating system such as Microsoft Windows XP requires some insanely large number of test cases. I could try to explain why this is, but I’d fail miserably as my computer science math knowledge has trickled out of my brain. I think it involves a lot of factorials. 100% complete testing requires exponentially larger numbers of test cases for even a small increase in the number of variables, until it becomes impossible to test completely before entropy makes everything the same anyway.
More digression: I’m going to harp on the possible magnitude of test cases for just a moment, for a little scale. Here’s a micro-lesson in binary math: computers store data in binary format, meaning there is just 0 or 1. You make larger numbers by increasing the number of bits. For a 2-bit binary number, you have 4 numbers you can represent: 0, 1, 2, 3. For a 3-bit number, you have 8 numbers you can represent. For a 4 bit number, you have 16 numbers. Every time you add one ‘bit’ of information to the length of the number, you double the number of possible numbers. The same is true of test cases. If you have 4 variables you’re testing, that means there are twice as many test cases as with three variables. This quickly gets out of hand if you consider all possible test cases, not just likely ones.
So, I don’t test everything. No tester ever tests everything unless “everything” is ten different cases, period. What I do test usually falls into two categories:
Verification testing is the often very repetitive and frankly boring task of making sure that a function does what it’s supposed to do, a bug is fixed, etc. This involves lots of checklists and tasks like creating every possible card in eWallet and filling up every possible field, or manually forcing a trial version to expire after 30 days. It’s not very fun and exciting. It isn’t supposed to be fun and exciting. It’s just required.
Exploratory testing is almost self-explanatory. You explore a product and hopefully find bugs. You do weird things, such as entering in 10000000000000000000000000000000000000 into the number of minutes to wait; you set your coffee cup down on your keyboard to enter in letters into an eWallet card; you try to drag a program file where a picture’s supposed to go; you open up a data file in a text editor and change one character to see if the program notices; you pretend to be The User Who Manages To Do Everything And Break All Software and throw the book at the product. [Exploratory testing is often ‘black box’ testing, where the tester doesn’t have any knowledge of how a particular item is implemented or exactly what it should do.]
Like verification testing, this can be really boring, especially if you aren’t finding any bugs. There’s a very awful feeling when you do everything you can imagine and nothing bad happens. There’s an even more awful feeling when you send the supposedly-well-tested and bug-free product out to users, and within ten minutes get a report of some glaring and awful bug that internal testing didn’t find (that’s what beta testers are for!)
However, exploratory testing can also be exciting, especially with alpha software. [Alpha software is completely unusable by the public, whereas beta software is usable but could have some big bugs still. Then you have Release Candidates, which theoretically can become the released product if no more bugs are found.]
For example, I might get an alpha release that crashes every time you click on a menu item, or crashes when run on my testing PC but not my main PC.
Don’t panic - there’s a reason we don’t let the public use alpha software! I don’t want to horrify any users with tales of eWallet lighting my PC on fire and making my cat’s fur stick up all over, I’m just being transparent about what it’s like to be on the inside of software testing. If you think you’ve found a bug, we’ve found thousands. That’s what software testing is for. The upside of horrifying failures in testing is that they make life exciting.
Another aspect of being a tester is that I get to see things before anyone else does. I get to see what’s in the next version of ListPro, for example. As a tester, I know a lot, but as a support person, I have to ‘know’ only some things. This puts me in the odd support position of taking a request for a Feature That Will Make Your Kitten Cute and Fluffy even though I know it will be in the next version. Or worse, I get to take a request for a feature that won’t be in the next version, but I still can’t say anything. We’ve had features removed right before release for various reasons, and it’s no fun finding out that the super cool feature you always dreamed of (and that a person told you would be in the next version) is nowhere to be seen.
Testing is also more than the actual tests - there’s maintenance of the test cases that we re-use, all kinds of upkeep of bug information in our bug tracking software, research into new ways to test, and keeping an eye on customer support issues for possible bugs.
In fact, one of the things I like about doing both technical support and testing is that it removes the intermediary between a user with a problem and the person in charge of writing it up as a bug: if you tell me about a bug, I’ll try it out, look it up in our bug tracking system, and enter it if it’s not there. I know there’s a problem because I’ve seen it myself. I field user requests and feedback, so it’s always fresh in my mind when I’m trying out some new or changed feature to see how it works.
Hopefully, this gives more of an idea as to what it’s like to do software testing for a PDA software company. If anyone has any specific requests for things you’d like you know about, just leave a comment and we’ll see what the blog post fairy brings!
Share this article:
Ellen walked into my office this morning and told me I’d been tagged, and I had no idea what she was talking about. It sounded like a bad thing at first, but I’ve found out that it’s just my turn in an online game of tag that’s been going on for a few weeks now. Doug over at Just Another Mobile Monday chose me. The rules are pretty simple, I have to tell everyone five things about myself, and then I get to choose five unsuspecting bloggers to tag next. It’s like a big ol’ chain letter. So here are my five things; I’ll try to make it as fascinating as possible:
So, there you have it. That’s me, in a nutshell. So now I get to tag five more people. And let me tell you, it wasn’t easy trying to think of five people - this game has been going on for a few weeks now, and a lot of bloggers have already been tagged! I chose:
Tag, guys, you’re it now!
Share this article:
So apparently there’s this new game among bloggers where someone “tags” another blogger. If you’re tagged, you have to post 5 facts about yourself that most people don’t know, and then tag 5 other bloggers. I just got tagged by spmwinkel at solsie.com who had been tagged by Patrick from JustAnotherMobileMonday.
OK, this is going to be pretty easy, and fun. First, the interesting people I tag are:
Dan Hanttula a hugely interesting person who has more credentials than I’ll even try to list.
Hal Goldstein, editor-in-chief of Pocket PC and Smartphone Magazine.
Jen Edwards, the Pocket Goddess, which title I wish I could have.
My good buddy and sometime competitor Alex Kac (I hope this counts; he calls it a blog though it’s a blog in a forum format).
Howard Tomlinson, founder and CEO of Astraware.
And 5 things about me, in no particular order, are:
One, I’m in the small group of people who are huge fans of musicals. Oddly enough, both Dan and Hal, above, are also in that group, so when I write them that I’m going on vacation to watch people kill people while singing, they know just what I’m talking about.
Two, I am one of the original founders of Ilium Software, which, to a large extent, I started because I didn’t want to work for anyone any more. It’s managed to keep me out of a “real” job for over 9 years, which I’m pretty happy about.
Three, I got interested in mobile computing when I worked at what’s now AT+T, and they handed me a Newton and sent me to the first (and, I think, only) Newton Developers’ Conference. I think I still have the t-shirt, though the Newton was stolen in a break-in at our “headquarters” at the time. (Working at AT+T was a big factor in convincing me that I never wanted to work for anyone again.)
Four, I am also involved with Bits du Jour, a one-deal-a-day site for Windows and PDA software.
And five, I consider the name “Ilium Software” as my biggest business mistake. Here’s what I learned: never, never, never start a company or a website with the letters i L i. Or even i L, or L i unless it’s a really well-known word. Nobody can distinguish ili from illi from lli from lil in those little tiny fonts without serifs. Ilium is Greek for Troy - I was working in Troy, Michigan at the time, and liked the sound of the name Ilium, but never thought about the readability issue until it was way, way too late. It’s not my biggest life mistake, but definitely a candidate for the bad company name awards.
Share this article:
Marc Tassin, our Product Manager (who introduced himself here a while ago) has just had a story published in a real book! The book is called Furry Fantastic
Again, congratulations Marc!
Share this article:
Not, as you might think, that some guy in the street tried to strike up a conversation, but that Randy Eisenmen, Handango’s CEO, for some reason decided to lift me up about 2 feet when he gave me the Handango Champion Award we won for eWallet. To say I was shocked doesn’t even begin to cover it - no one has tried that in many years. Apparently I looked so freaked out that he had to send someone over to ask if I was OK (I was fine. I was worried about him - I was having visions of him falling over, and seeing headlines like “Ilium Software Exec Kills Handango CEO”. I’m short, but I’m really not all that small). Anyway, it was lots of fun. Someone asked Randy why he’d picked me up (everyone else just got hugs or handshakes) and he was blaming Ken Landau from LandWare, who won the award before ours (congratulations, Ken), but I’m pretty sure that was just an excuse. If companies get their personalities from their CEOs, I can certainly see why Handango is so high-energy.
Of course I was delighted that eWallet was again recognized by the judges as one of the top mobile apps (and by the user community as being worth the nomination). And of course I was overjoyed that InScribe (which we’ll be releasing this month) was picked by the judges of the UMPC contest as one of the 3 best programs submitted for the UMPC. Both Marc and I are really excited by the UMPCs, and very happy that InScribe was recognized. And now that we’ve won another UMPC (courtesy of Microsoft, thank you to them), I might actually get a chance to use it.
The whole Summit was lots of fun - thanks, everyone at Handango. I got to meet some very interesting people, catch up with some old friends, and even listen to a few presentations. As well as eat and drink too much. Everything made it back, including the new hardware, all the swag (what was that thing from Verizon?) and of course the Champion Award, which is now in its place of honor on our awards bookcase. It’s a good thing my suitcase expands, but not so good that I do.
Once again, many thanks to Handango, the sponsors and the judges.
Share this article: