Building an evidence-led culture
As with all cultural change, there has to be clear commitment from the top for the change to permeate the whole organisation. This means the Board and CEO always asking the question “what data do we have to back up this idea?” and repeatedly giving the instruction “go and get the data we need to inform this decision”. If the Board and CEO do this regularly then the rest of the organisation will change to think this way.
More formally, this commitment is substantiated through the following:
- Including “evidence-based decision making” or equivalent sentiment as one of the key organisational values and ensuring that this value is followed.
- Pushing the key insights that the organisation has collected to all staff as part of an ongoing education programme on the evidence that drives the organisation.
- Training staff in the basics of data interpretation so that they can understand and interpret dashboards and figures.
- Broadening access to tools that allow people to explore and interpret data themselves and the raw data for those tools, beyond the traditional business intelligence team, to support power users.
- Using “data as evidence” in all the top-level organisational documents – the strategy, the annual report and board papers.
- Providing sufficient budget for the ongoing use of external survey companies and/or survey tools to ensure that the collection of new data becomes habitual.
- Designing instrumentation into all automated systems and adding it into key legacy systems.
Aligning staff around key evidence
If the staff of an organisation are to fully align behind the strategy, then they need to see the evidence behind that strategy so that they can make the same intellectual connection between the evidence and the strategy that the organisational leaders have.
Some of this can be achieved by opening up access to corporate dashboards, reports and other data insights, which are often locked up within personal or department silos. However, key insights need to be pushed to staff as part of a company-wide programme to ensure 100% coverage.
In 1903 the famous author HG Well wrote the following, which even now over a hundred years later, seems ground-breaking:
… for complete initiation as an efficient citizen of one of the new great complex world wide states that are now developing, it is as necessary to be able to compute, to think in averages and maxima and minima, as it is now to be able to read and to write.Mankind in the Making, http://www.gutenberg.org/files/7058/7058-h/7058-h.htm
All office workers are now effective users of word-processing software, but that was not always the case. Only thirty years ago there were still many organisations with pools of secretaries or typists who wrote up documents from dictation tape. It did not take long for the benefits of everyone having access to a word-processor to become the standard, and the same approach needs to be taken to data-literacy. The quicker that we can put data into the hands of most staff and help them to become data-literate, then the quicker the benefits will flow through.
Achieving a data-literate workforce requires training in such basics as:
- calculating representative percentages (what is counted, what population, what values can be aggregated, etc)
- understanding margins of error and statistical significance
- judging correlation and causation
- recognising common forms of cognitive bias
The core mechanism for evidence gathering about people, their motivations, how they make decisions, what they are influenced by, and related questions, should be through surveys – surveys of your customers or those who use your services, surveys of potential customers, surveys of staff, surveys of the public and so on. These are the best ways to provide direct and unfiltered evidential data, but getting the best out of surveys requires expertise.
Most people know what they want a survey to ask and can make a reasonable attempt at writing the questions without confusing, leading or disengaging the reader. However, they soon find out that understanding what the results actually mean is far more complex than it appeared at the outset and there’s a good chance the questions weren’t asked in quite the right way needed to join up the dots. Quite often people get knowledge from surveys but not actionable insights and the survey becomes a wasted opportunity.
To get the most from a survey we start with the definitive results we want the survey to deliver and work backwards from there to create questions that will provide an ‘evidence chain’, where each step in the reasoning to the result links to the next. This ensures that when your results say “customer/stakeholder segment A prefers Y over X” you are not left asking “but would they prefer Z if it were offered to them?” or “what do we need to change with X to make that preferred?”. That way, the results are actionable, not just interesting.
All IT systems collect and log data but this is nearly always for operational and not strategic reasons. The common problems that come from this are:
- Not collecting the full data to support strategic analysis
- Difficulty in viewing the data as at a specific point in time in the past, and a related difficulty in generating historical trend data
Instrumentation of systems for strategic reasons has a number of components:
- Specifying additional data points in addition to the technical logging to provide a fuller picture.
- Creating point in time snapshots of data to enable historical analysis.
- Building hooks for automated survey distribution and data collection.
How we can help you become an evidence-led organisation
- Advising your board on the benefits of this approach.
- Planning the cultural change to become an evidence-led organisation.
- Training or advising on training and tools for staff.
- Implementing a data governance framework.
- Designing surveys.
- Analysing survey results and produce a full report of actionable insights and other key information.
- Reviewing the results already produced from a survey to check they stack up and see what other insights can be inferred.