Controlled Experimentation

The purpose of an experiment is to learn. Experiments start small and in controlled environments, then expand: Virgin Galactic could have sent 50 people on its first commercial flight to space, but it’s probably better it started with one.

We think we know, and we do. That is, we know things to a degree and, even then, not for sure. So why not test?

Why not give ourselves room to explore an answer? A safe place, where we can’t do too much damage. A space big enough to be valuable and real enough to be viable.

What is the risk of rushing forward versus the reward of exploration (What is the potential cost, impact to customers, the team, outside relationships, capital reserves, etc.? Can we find a good balance?

Tests are started in a controlled environment. Once we have a safe setting, we can use this test to think differently, to build off of the work of others while pushing further to unlock new avenues of growth. There is no reason to set up a test just to do more of the same; for that, we simply copy and paste.

Tests meander their way towards answers and expand once answers are found. But once we have completed a test, what’s the next step? How do we continue to move forward? How do we build on what we’ve learned and not get stuck in the safety of a test or get complacent with the results of the test? Can we prevent the perfect from becoming the enemy of good? Can we commercialize and scale this?

The tests itself, however, doesn’t need to be scalable; they only need to provide answers to questions or offer hunches. Once we have a baseline, what variances do we need to account for in a live environment? What from the test can’t be executed at scale and how do we solve for that? What impact might this new environment and scale have on the net result?

If the experiment is a complete failure, then we start over. No harm done and a valuable lesson learned.
If there was a spark, we explore that spark further, seeing if we can get it to ignite.
If there is fire, we pour a little fuel on the fire to see how bright it can burn and how far it can spread.
At every step, we are learning– always learning. At the same time, we are always mitigating unnecessary risk in an attempt to make the education more valuable than the potential cost.

Leave a Reply

%d bloggers like this: