Defining Impact: The Challenge

Innovation is widely lauded as the key to economic growth but the challenge of measuring the impact of new ideas has received much less attention in research literature, and in the dialogue with key stakeholders: governments, businesses, and investors.

Defining and measuring impact requires a systematic approach based on explicitly addressing three tiers of the eco-system: policy definition, resource allocation, and intervention management. Policies usually translate into how resources are allocated, which in turn drives the design and implementation of specific intervention programmes.

Tier 1: Policy Definition

Many of the challenges in defining and measuring impact stem from macro-economic priorities which reflect ideological positions in different eco-systems. Broadly speaking there are 3 general positions:

  • The ideology of the ‘free marketers’ who believe in minimising the role of the state in commercialising innovation: this viewpoint says that the state should stay out of the way, only getting involved when there is a ‘market failure’. This attitude is hard to sustain given the dramatic structural changes in markets resulting from new technologies, as a result of which we see many contortions in defining ‘industrial strategy.

  • Eco-systems which emphasize the primacy of the state in defining overall priorities, initially represented by so-called ‘socialist’ countries: this is now a minority position in most eco-systems.

  • Hybrid positions, based on ‘re-inventing’ capitalism, which emphasize the need for ‘public-private’ partnerships in translating ideas into impact: at a macro-economic level. There is much debate about striking the right balance, which in practice means we can see many flavours of this overall thinking. This ‘third way’ now dominates most policy discussions but is hamstrung by the limitations of macro-economic thinking, so, for example, it is hard to un-pack the implications of the UN Sustainable Development Goals, ‘levelling-up’ agendas, or the value and desirability of ‘Unicorns’ vs a ‘Mittelstand’ strategy.

Tier 2: Resource Allocation

Ultimately the real test of any policy is how this translates into the actual allocation of resources. This is the real challenge in implementing hybrid policies which cover all the main eco-systems of interest now. True free marketers believe the market will fix this; the statist viewpoint says everything is defined centrally. In broad terms, most of these policy prescriptions adopt different balances between direct and indirect incentives: advocates of direct funding focus on the provision of grants and related instruments; the advocates of indirect support emphasize the role of the state in creating the right environment to encourage private investment, for example through R&D tax credits. The problem with this is that macro-economic thinking does not provide the right level of granularity to properly model, understand, and compare the impact of different approaches.

In particular, a quantitative approach to the allocation of resources requires:

  • ‘Richer’ characterisation of markets and businesses within them which models the outcome of interactions between the wide range of existing, new, and emerging actors, ownership structures, and regulatory environments

  • More dynamic and precise understanding of how innovation is actually translated into impact, in particular for ‘non-equilibrium’ growth

  • Understanding the critical role of the public sector in the innovation process: creation of economic value is a collective process, not simply something that the private sector can tackle on its own.

Tier 3: Intervention Management

Policy clarity can enable resource allocation to be tackled precisely using the meso-economic model of growth. This more granular definition of the key variables enables the impact of different interventions to be accurately measured and compared, provided the following issues are tackled head-on:

  • Many conventional measures used to assess impact focus strongly on ‘inputs’ rather than ‘outputs’, largely because these metrics are much easier to measure in most economic settings; for example, financial inputs (grants, investments )are often reported in some detail with little measurement and discussion of outputs generated. Where output metrics are defined, these tend to be aggregate measures, which only make sense at a macro-economic level, so it is hard to measure the impact of a specific intervention: for example, the ‘impact’ measures adopted by most European governments tend to focus on the number of new jobs created, and the impact of the new firms on the tax base. It is very hard to apply these kinds of measures to understand the short, medium, and long term impact of investment at a more granular level which allows impact to be measured more precisely.

  • The focus on translation and impact is very strongly based on conventional financial criteria such as return on investment and shareholder value rather than any broader view of impact: this uni-polar view can result in an obsession with finding and supporting the big ‘winners’, resulting, for example, in the unhealthy fascination with so-called ‘unicorns’, which largely owe their success to manipulation of the financial system, rather than any inherent value creation.

  • We need a systematic approach which enables the detailed meso-economic profile of any innovation to be assessed at different points along the commercialisation journey, so that changes can be defined more precisely, and related to the impact of the interventions made. This approach also needs to explicitly address any issues related to different ideological orientations, in particular sustainability stance which reflects the Green Agenda.

To learn more about the Triple Chasm approach and research, visit our Research Hub.