Users Make Awful Robots!

May 30, 2006 on 1:02 am | In General Programming | No Comments

Ok folks, I have a lesson learned for you. I wouldn’t say that I’m a perfectionist, but this was a crazy experience that’s left a long lasting impression on me.
A week after declaring an app I wrote “ready for production” I got a list of 10 items the client considered to be a bug. Nine of these turned out to be minor, quick fix, standalone bugs. “Standalone bugs,” you ask? One of the things that’s bad about knowing the list of flaws in your system is that you’re certain of where it will break, and what circumstances will cause it to flaw or crash entirely. One of the good things about knowing the list of flaws is that you’re certain of where it will break, and what circumstances will cause it to flaw or crash entirely. With every fix you will always introduce the uncertainty that your new code will break something else that seemed to have been working before. So, check ten items off the list, and you’re still not sure that your list of bugs is at zero. I did just that, and got burned pretty badly.  The final item on the list turned out to affect the ability to edit existing data, and I looked really stupid.  I wouldn’t have myself ever discovered the bug, because it arose out of them using the application in such a way I wouldn’t have ever imagined.  They claimed that they do this all the time, and needed to be able to, so I went down that road.
There are millions of ways to try and combat this, unit testing, very granular design (with unit testing) and so on, but still you will always leave out something. Some circumstance your client will undoubtedly trigger, perhaps more often than any other state, the deadly sequence of events or data items and you’ll have a mess on your hands at some point.

A good, and arguably more intelligent, friend of mine sums up the scenario in the cute yet true phrase that “users make awful robots.” If they’d only do the same things with your app that you’d written tests for, there would never be any issues. Of course, that’s never going to be the case, and it’s so painfully obvious to anyone that does this on a regular basis, that I might not even say that… but I had to learn this lesson… hard, so I believe it deserves mention.
Before I get into it here, and start making quick enemies, I’m not saying that you will let some bugs “live forever,” just that you may need to leave some at the time of relase.

So, when do we fix it before release, and when do we keep it on the ‘good list’ of bugs at launch time? I can’t say that I have an easy answer for you, but there are some basic points that I like to keep in mind.

How big of an impact on the apps overall function is this bug?

If your bug doesn’t really do anything bad other than annoy you, then it may be one to keep and study for a service release. If your product produces some official printout as a deliverable to a government agency (which in my case was exactly what the product does) you can’t leave anything affecting the printouts in there or you might wind up in court, or send your customer up the creek, but … There are times when a bug that doesn’t officially screw anything up still warrants fixing before release. Logging for example, or error reporting. Those types of bugs will throw off your ability to fix others down the line… I think it’s pretty clear that I can’t go into a list of yes’s and no’s on this, but you get the idea. My general rule of thumb is, does this affect the ultimate function of my application, and does this obscure my view of what else is happening in my application? If the answer is no, then you can possibly live with it.

How easy is it to trigger the bug

This one is a little more complicated than it sounds… Each client may indeed use your code for slightly, or entirely different things than each other, so if the bug is common to a certain category of clients, and not to others, then you might want to give a little less weight to the ‘easy to trigger’ factor. It need not necessarily be a true average. (Math and debugging don’t always mesh, in my experience. Though many have tried to make a mathematical model to describe everything from development time to phase iteration, sometimes you just have to squeeze a bit of reality over your numbers.) If the use of your code though is fairly universal, and the feature that’s affected, or the exact circumstances required to trigger it are pretty obscure, (I mean in real use, not simply on paper) then you can probably put this one off.

Is this going to impact existing data

One thing that I find myself running into is a change that doesn’t really break anything properly, but will leave previous versions of the data created with the old code either in an arbitrary or undertermined state or leave behind extraneous bits altogether. For one example, one thing to consider is “Data Determinism”. Does your code use the data itself to determine how to store the incoming information (or modify existing information), or is the method of input what determines it’s persistent representation? If you choose the former, you’ll have to account for regressively changing your old data to reflect the new structure, if the latter, you might be ok, but still have to check very thoroughly…

The ultimate question

Does this make people not want to use my product over an available alternative? If you happen to be microsoft, you probably don’t have to think much about this one, but for the rest of us, this is probably the most critical thing to consider. You only get good answers for this one from doing careful beta testing. Alpha testing will of course leave you with a bunch of ‘of course not, this is our great product’ but the beta people, by definition, won’t be employed by you and will tell you if they are turned off enough by whatever the bug happens to be. You may want to construct this sort of question directly for your beta testers. Give them a list of popular, or similar alternatives, and try to ask something that seems heartfelt and honesty inspiring like ‘Considering your experiences with our product, which of the following products are you most likely to choose: Alternative B, Alternative A, Our Product’ If you get people NOT choosing yours, call them back a second time and ask why. Putting it on the same survey may make them not want to answer truthfully in order to avoid having to fill out a ‘please explain box.’ (I’m guilty of that myself…)

Anyway, I know I haven’t given a lot of instruction, but at least I hope that I have illustrated why sometimes it might not be worth your while to try and perfect a product before it goes out the door. Especially if you’re writing something groundbreaking and novel, and possibly a competitor is working on it at the same time… You don’t want to let your perfectionism to allow your competition to become a ‘de facto’ standard, because getting someone to invest in your product after they’ve already shelled out for a similar solution is worse than pulling healthy teeth — even if yours really does happen to be better. (For example apple vs microsoft…)

TINAI (this is not an introduction)

May 27, 2006 on 2:50 am | In Uncategorized | No Comments

Hi I’m Dave. Jack of lots of trades, master of … eh.. well… not all of them. I’m pretty good a few of them. Not sure what a master would be in any case.

I’m a sysadmin for my primary paycheck. I do lots of coding work though in the course of my sysadminship. I’ve done perl, php, c, c++, a bunch of many sorts of shell scripting even a bit of pascal (delphi for you modern folk) but my focus of late is C#. I know what you’re thinking. Microsoft. Right. Well, I’m also a capitalist, so sparing the details of the situation, it’s a Microsoft world, and I’m living in it, and lately starting to like it. .NET is quickly becoming my platform of choice, simply because it’s so accessible, and lets face it Visual Studio 2005 is a great product, even with it’s myriad of bugs and quirks. If Microsoft is delaying the release of Vista in order to provide the same amount of completeness and user facility that they did for VS 2k5, then I’m all for it. C# is also great because of the vast community of developers that share their wisdom, trials, tribulations, and code samples all over the element.
I’m a recent convert to the world of real Object Oriented Programming, and in terms of design strategy that makes me a newbie, but make no mistake my coding skills are worth at least the weight of these words in cyberspace. There are few times in ones life when they can claim their life was actually changed by a book, but for me, “Design Patterns” rewired my brain. I had to read it twice to realize the stupifying power that a rational approach to separatation and delegation of duty offers a developer such as myself. I was the type of developer who would be reading about the newest features that come out in some grand new planned system (.NET 2.0 for example) and before I could finish the first page of Docs, I was three links away in another tab trying to figure out how to implement a custom hook on it. So, sometimes I ended up in the predicament of having implemented some fancy aspect of something that’s supposed to be quite simple before I’ve ever tried the simple approach. I’m starting to change that habit, but till now it’s taken me to places I’ve been and quite enjoyed the ride as well as the results. Often times I’ll find myself saying ‘man I’d never do it that way again, but I’m sure glad I did that time.’

I learn a lot from my mistakes, and I make a ton of them. I’m not a guru, and I don’t claim to be one, but I do feel that I have a lot to offer the developer community from my experiences, contacts, adventures and misadventures. So, if you like that kind of thing, or just want someone to read who may be experiencing the same amount of frustration that you are with certain things, I’m your man. I do usually manage to find my way out of every coding jam in which I find myself, so if you come along, you’ll get there too. I don’t promise, but I propose.