Editor’s Note: This post was originally published on Seattle 2.0, and imported to GeekWire as part of our acquisition of Seattle 2.0 and its archival content. For more background, see this post.

By Anthony Stevens

Today’s installation in my Tech Startups 101 series is about instrumentation. (Previous installations in the series can be found here, here, here, and most recently, here).

What’s instrumentation?  It’s the ability for you and your team to get useful data about what’s going on with your software.  A classic example is Google Analytics: it’s a tool that tells you, quickly and accurately, how many people visit your website (among many other things).  Useful information for sure.  An entire industry grew up in the late ‘90s and through the ‘00s around analysis of site traffic numbers, and many companies have been acquired mostly or solely based on their visitor counts and/or trends.

I’d like to make a distinction between external instrumentation and internal instrumentation.  External instrumentation can be leveraged after the fact: Google Analytics, site-uptime services, e-mail newsletter delivery tracking, PayPal reports, and others.  You don’t have to modify your software too much, if at all, to take advantage of these instrumentation services.  For example, Google Analytics just requires you to add a simple bit of JavaScript to your web pages.

Internal instrumentation, on the other hand, is baked in to your software and typically provides a deeper, more technical, and more detailed look at what’s going on.  Some examples:

Automatic exception notifications.  You can include handlers that automatically e-mail the team when an unhandled exception occurs.

Method logging.  You can flip a switch and start writing out very detailed log files that show what methods in your software get called, and with what parameters.

Performance auditing.  You can write in hooks that tell you in near-real-time how long certain key portions of your code take to execute.

Special event notifications.  Do you have a business-specific rule in place? You can alert yourself when the rule is triggered.  An example might be “more than one account created from the same IP address in the same hour.”

Customized dashboards.  Your company has special reporting needs.  A custom dashboard might be very useful.  If you’re an online retailer, an example might be “show me how many people each day clicked on our upsell call to action, and, of those, how many people ended up purchasing the higher-priced product.”

Potential downsides to instrumentation?

Time and cost.  It can take time and money to add instrumentation to your software, or pay someone else to use their 3rd-party services.

Information overload. Did you know that the term “information at your fingertips” was coined by none other than Bill Gates of Microsoft, 20 years ago? (I didn’t, until now).  The key hypothesis is this: the more information you have, and the easier it is to get it, the better.  There’s a definite controversy here: the term “information overload” sums up the competing argument nicely.  But it’s not a black-and-white, either-or decision: “All infomation” vs. “No information”.  There are certain key things about your business that you want or need to know, and you can draw the line wherever you want.

Measuring the wrong things.  This isn’t a fault of instrumentation per se, but if you’re measuring something, you are more likely to act on it.  What if the thing you’re measuring is an ineffective measure of your business?  There’s a lot of thought – offline thought – that needs to be done before you start throwing in instrumentation.  Think of instrumentation as arthroscopic surgery, not open-heart surgery.  Think “selective” and “targeted” – and rethink your choice of measurements often.

Used in the right way, instrumentation can add a lot of value to your startup.  But instrumentation is not a panacea, and more is not always better.  What are your experiences with instrumentation?  Share it in the comments!

By the way, did you know I’ll be speaking at Deploy 2010 on June 24th?  Click the link below to register!

Register for Deploy 2010 in Seattle, WA  on Eventbrite
Like what you're reading? Subscribe to GeekWire's free newsletters to catch every headline

Job Listings on GeekWork

Find more jobs on GeekWork. Employers, post a job here.