By Dr Ryan Young, Principal at Queritas Pty Ltd
There are various initiatives underway to improve the quality of evidence that underpins policy decisions – including explicit moves to add more scientific rigour to policy-making. These are important but genuine, embedded change may fail to materialise if these efforts fail to understand how science genuinely works. It is common to focus too narrowly on identifying what policy approaches work, rather than understanding why and how policies work.
At the core of scientific practice is a disciplined effort to build cohesive theories to explain how the world works. These theories underpin experiments, data-collection and the ongoing reliability of science. Theory matters because we don’t just want to understand what happens in particular circumstances, but need to understand how and why the world works.
This same focus is important for rigorous policy-making. When designing a policy, we don’t want to just focus on what we are going to do and what our objectives are. We need to also be clear about how government actions are going to deliver our desired outcomes. If we aren’t clear about what our theory of change, or in scientific terms our background theory, our chances of achieving good outcomes decrease. Many poor decisions can be traced to this gap – we know what we want to do and what we want to achieve, but are very fuzzy about how the two are connected in the real world.
Being clear about how we expect a policy will work in the real world – setting out our underpinning theory of change – before making a decision is a simple and very reasonable requirement. Yet doing it routinely can lead to important policy outcomes: it will make the development of policy more scientific, improve the rigour of our analysis and lead to more robust decisions.
For one, clearly setting it out helps ensure flaws in our logic or plans are easier to spot. For example, it is surprisingly common to bring in new legislation or regulations but pay little thought to enforcement or compliance. This gap would become clear if all the expected steps from legislation to real world outcomes were explicitly detailed.
Similarly, stating the underpinning theory makes it easier for others with different perspectives or insights to challenge our thinking and point out dynamics that we hadn't thought about. It also makes evaluation - another current priority - easier and more useful. If we know what we wanted to do, how it was meant to work and what we hoped it would achieve, we have an immediate structure to use to evaluate how a policy or program went. If any of these are missing, what evidence we need and what we are evaluating gets murky.
In scientific practice, all of these elements – goals, plans, theory – are codified in the default structure of scientific reports. This ensures all steps are explicit, we understand why something did or didn’t work and so we can improve as we learn more. Policy making and program design would benefit from adopting this scientific practice and clearly articulating all parts of the thinking behind the policy design.
More information about the scientific practices can be found at a longer article this blog post was based on. The author is interested in working with agencies on simple and efficient ways to build greater rigour into policy-making.
