When I announced our open beta I promised that I’d let everyone know how it went. I mean this is a ‘behind the scenes at Ilium Software’ blog after all! I’ll start by saying that overall, this was a great experience. We got some really excellent insights into how people reacted to changes, caught a number of bugs, and found out how folks liked the new features. Without question I plan to run additional Open Betas in the future.
Statistics
Here are a few statistics about the Open Beta:
Number of People Who Viewed the Page: 4000
Number of People Who Downloaded the Beta: 1500
Number of People Who emailed Comments: 28
Although I don’t have much to compare it to, this struck me as a pretty good response! Particularly considering how short a time the beta ran. While we didn’t get many email comments, what we did get proved very useful. Those 28 emails alone were worth the effort of setting this up!
A few more interesting statistics for you number crunchers out there:
Pocket PC Downloads: 912
Smartphone Downloads: 266
Palm OS Downloads: 172
Windows Only Downloads: 167
Duration
What about the length? Well we knew from the start that this was going to be quick. In the future I’d like to run a longer open beta. One of the things we weighed was just how “beta” of a version we wanted to send out there. We’re sort of perfectionists over here and we hate to send out broken software. Still working on this one but I’m thinking at least three weeks next time. We’ll see!
Conclusions
The conclusions I draw about the Open Beta in general is that I’m really glad we did it. Not sure that it would work for other developers but it certainly worked for us. My thanks to everyone who downloaded and especially to those that sent us comments!
Looking at the other numbers (the fact that Smartphone downloads well outnumbered Palm downloads for instance, when we have very strong sales for both platforms) leaves one wondering as well. Does it mean something? Is it a sign of use patterns? Or maybe Palm people are just less likely to try betas? Feel free to throw in your 2 cents! We’d be happy to hear it!