On learning from Lean Experiments

Lean experimentation is a great way to de-risk innovative and entrepreneurial ventures. It offers us a way to systematically test our assumptions, generate trustworthy insights, and make better decisions.

Lean experimentation is not a goal in itself, but rather means to get us to our destination faster. I see no point in running lean experiments just because it is (currently) cool, or others are doing it.

As I often say, every good lean experiment start with a clearly defined learning goal, and concludes with a clear decision on what to do next. But how does one reach that decision?

Once the experiment has been conducted, the following should be done:

• Capture. All the data generated from the experiment should be captured, ideally in raw form. Try to keep it as "pure" as possible, i.e. no interpretations or analysis yet. If there are numbers—just write them down as they are. If they are observations and conversations, just write down what you've literately seen and heard, not what you think you've seen or heard (e.g. "...he slumped in his chair, lowered his voice, and looked to the side..." vs "...he became dejected and melancholic..."). Capturing all data in its raw form will allow you to re-analyse and re-interpret it later, or in different contexts, if needed.

• Analyse. Now is the time to crunch the quantitative data, if you collected any. You don't need a PhD in statistics to do some exploratory data analysis (a fancy term for looking at data in a spreadsheet software), chart few basic plots (e.g. bar chart and scatter-plot), and calculate basic descriptive statistics (mean, median, mode, standard deviation, variance minimum and maximum variables, kurtosis, and skewness; all available in standard spreadsheet software). And remember, you might not have to do any of that if you collected very simple quantitative data like "x out of n said/did."

• Interpret. This is the most important step, as this is where you combine the data you've collected with your knowledge and lived experience, from your specific context and environment. Think of it as going on a learning adventure, narrating your own story. In the beginning you had a set of assumptions and a learning goal, then you went on a journey and collected data, and now you are making sense out of it; all of that in service of making an informed decision on how to move forward.

• Decide. By this moment you have everything you need to make a decision: raw data, analysis, and your interpretation. Will you continue? Stop? Change direction? What, specifically, will you do? Remember, decisions can be both small and big. "We decide to do another experiment" is as valid as "We decide to invest everything we have."

• Document. A critical step for creating reusable organisational knowledge. Organisations don't learn, people do. You might have learned a great amount from running this specific experiment, but unless you document all of the above it will be nothing but transient knowledge. It will perish as quickly as it came. This is a particularly important point for large companies, as they can increase their experimentation ROI simply by properly documenting and disseminating their learning.

Above makes for a mean-sounding CAIDD acronym, but hey, I set out to write something useful, so sounding pleasant wasn't necessarily one of the top priorities...

For a complete guide to designing, conducting, and learning from Lean Experiments, head to:

Step-by-step guide for designing Lean Experiments

Designing sound experiments is critical to creating valid and reusable knowledge. This guide will help you achieve that.

Bruno PešecBruno Pešec