1) The more rigorous the test, the less you’ll be able to compare them to anyone else’s numbers. Three nines is great for this type of site as measured by the type of testing you’re doing (looking for a string on the home page) but expect that number to go down should you start scripting a “click through” or transactional type of test. Ironically, when you start doing these types of tests you see more problems, fix them faster (than most other sites) but your overall numbers go down because you’re looking for (and probably finding) more problems–more problems than those “other guys” anyway.

2) For a better reliability test (but again will probably get you worse numbers) consider a separate “health check” page. This would be a page that for a sanity check, self-checks your site’s home page as you’re doing now, but also self checks any major site subsystems (or anything else) that are required to keep your site running (e.g, check a backend database connection, check that the RSS system can be read from, etc). Either report the results of each or overall site home page and subsystem responses as HTML strings or different HTTP return codes (200=all good, 201=1 error, 202 = 2 errors, 500 = major problems). The page could even have some fancy script to automatically alert you if/when (major) problems occur. Ours does and annoys the heck out of us but about every 1 in 3 or so times, saves our backsides.

Just for reference big firms have server farms behind load balancing switches which access a health check page as I’ve mentioned (along with other health checks). What’s nice about these arrangements is that when a server starts acting up, the load balancer marks it as unhealthy and disallows any more traffic to/from it until all of the health checks start passing again. That’s why I say your site is doing great at 99.9% availability. The big guys, with all of the big iron I’ve described should really be getting 4 or 5 nines availability.

Here’s a pretty good listing of a lot of testing/automation resources for the type of stuff we’re talking about. I think I include it mostly to back up my claim about how vast the field is:

http://www.softwareqatest.com/qatweb1.html

(I can’t speak to too many of these, since I mostly script in native Python sometimes with the Pamie module).

Thanks again for the site; I usually don’t have too much to add, since I’m either usually less experienced or way too late to throw my 2 cents into the discussions but today’s discussion was right up my alley.

And…I’ve always thought performance was adequate if not downright snappy (note that web professional jargon).

All the best!

-ard