Budget Driven Automated Acceptance Tests

Any oft repeated manual process is a good candidate for automation.
The question for me isn't whether to automate acceptance tests or not, but how we keep them relevant and manageable over time.

One way is to acceptance test the features that are most important to you, but how do you discover what features are most important to you?

If parts of your product enabled different revenue streams, you could bias your acceptance tests towards features that generated the most revenue.

If you had statistics on how your product was used out in the wild, you could bias your acceptance tests towards the most commonly used features.

Or I guess you could just bias your tests towards the stuff that keeps bloody breaking !

But how do we trim out the fat?

One possible answer could be to define a budget for the automated acceptance tests, you could perhaps define a ratio of lines of code to minutes of execution time for acceptance testing.
You now have a simple metric to allow you to discuss what is most important to you.
Expensive test, not much value, drop it.
Expensive test, pretty valuable, see if you can get it cheaper without losing to much value (refactor).

A Recipe for Budget Driven Bug Fixing

Our team at the guardian is really getting into the spirit of being budget-driven, this is the recipe we are using roll bugs into everyday development:

Take the number of bugs to be completed in an iteration.
Divide by the number of stories to be completed in an iteration.
This number is your Bug Debt.
You must fix the number of bugs denoted by your bug debt in order to commence a story.

mmm... the sweet taste of programming new features juxtaposed with the sour taste of humble pie