In March 2021 John Rahilly, a Research Associate in the Medical Research Council Epidemiology Unit at the University of Cambridge began designing a research project to produce the first large-scale analysis of the impact on community health of local authority policies aimed at restricting the opening of takeaway food outlets.
The project was intended to address an area where very little detailed research had been carried out. Such evidence-based research could play an important role in helping to inform policy targeting the improvement of community health. Currently, around 60,000 shops in the UK sell hot food to takeaway, and such foods tend to be high in salt, fat and calories. Previous research has established that people who live and work in areas with the most takeaways eat more takeaway food and are more likely to live with obesity than those that live elsewhere. 
Local authorities are increasingly concerned about the impact of takeaways on health, and around a quarter of those in England have applied planning rules to new takeaways – often through using 400-800m ‘exclusion zones’ around schools or other community facilities frequented by young people, where planning permission for takeaways is denied. Since 2009, around 44 local authorities have implemented such zones, while around 70 others are considering introducing them. While zones are centred around schools, the policies are often also intended to target local communities surrounding them. 

Finding controls

To undertake the research into the impacts of preventing new takeaways from opening, Rahilly’s team intended to analyse changes to rates of new takeaway outlets opening within and around exclusion zones in both subject and paired ‘control’ local authorities, using a ‘controlled interrupted time series’ (CITS) methodology. Effectively, this meant looking at around 35 subject local authorities that had implemented exclusion zones over a certain time period, as well at a control series of local authority areas with a similar profile – in terms of socioeconomic factors, demographics, infrastructure and so on – that were not subject to these interventions. The method – and therefore research outcomes – was dependent on the validity and comparability of the subject and control local authorities. That’s where the CIPFAstats+ suite of products came in.

“The problem is that local authorities are so varied, and there are significant issues in terms of identifying a control in a scientific and robust way,” explains Rahilly. “Commonly, other studies or other fields may use broadly comparable cities for their models – for example, if they’re studying Manchester, they may also look at Birmingham. However, we needed to justify being able to identify a control that we knew is most likely to be statistically similar in a more robust way. It’s very difficult – local authorities can be such different organisations in so many respects, so it’s very challenging normally to identify a comparator.”

Nearest Neighbours

However, Rahilly and his team were aware of a potential solution to their research dilemma – the CIPFAstats+ Nearest Neighbours Model tool. They had experience of it through Public Health England (PHE), which uses it for part of its PHE Fingertips website that enables local authority comparisons to be made.
“Early on in the project, we decided to explore using the Nearest Neighbours Model tool, with all the criteria available within it. Within the tool, there’s the capacity to narrow down parameters for modelling by selecting from around 40 different metrics, which cover a wide range of socioeconomic indicators.

“It was a tool we as unit had knowledge of,” Rahilly says. “It was one that we knew would be useful, and which had already been employed in a similar field.”

The Nearest Neighbours Model tool is a popular part of the CIPFAstats+ product suite. The tool uses statistical processes, but the factors upon which classifications are based reflect the need to provide a balanced representation of the local authority’s traits. The variables employed are therefore descriptive of characteristics of the area each authority administers. The outputs returned by these data calculations – underpinned by key indicators and complex mathematical modelling – presents this information in a simple to understand way on the tool’s dashboard. This helps the user to interpret results and visualise the statistical relationships – including similarities and differences – between local authorities. 
As well as around 40 predefined indicators, users can select which datasets they want to compare, plus as many – or as few – of the indicators as they wish via a user-friendly dashboard. The tool then provides a list of comparable authorities in order of the statistically nearest. 

By April 2021, the team had decided to use the Nearest Neighbours Model tool to establish the study’s control local authorities. “The identification of those controls was a really important stage, in terms of setting us up for the rest of the research,” Rahilly says. “We approached CIPFA directly, had a discussion with the people there about our needs. They gave us a demonstration of how the model worked and discussed with us whether it seemed appropriate for our needs before we committed.” 

The Cambridge University research team felt that using the full menu of metrics available would enable them to establish the most robust control sample “in terms of the most likely impact you'd see, and the most likely similarity you'd see between planning decisions in each authority”.

Rahilly continues: “The Nearest Neighbours Model gave us the means with which to access possible controls for our research, which we wouldn't have been able to do otherwise in a structured way. I originally tried to create my own control sampling method with open access data, but you simply can't get the same level of detail and comparators compared with CIPFAstats+.”

Cause and effect

The researchers signed up for a yearly subscription for the Nearest Neighbours Model tool, which provided not only a means of selecting controls in the early days of the modelling, but also a resource for other researchers within the faculty. “It was the logical starting point for us,” says Rahilly. “Control identification was more complex than just using the most similar as there were additional criteria that were relevant (such as the control not having implemented a policy), but the format meant that I could easily code additional criteria into a formalised identification algorithm.”

Rahilly built a flow chart and algorithm that pulled out all those local authorities that exceeded a particular similarity score. “With the Nearest Neighbour Model, there is essentially a ‘similarity score’ for each authority. That meant that we could select for different authorities a relevant control that met a certain research threshold for us, which meant we weren’t dependent on, say, the top three or four, for example. Some had only one control that would meet the threshold, and having access to that data – and that level of statistical significance and value – was really useful.”

Consequently, through the tool, the team were able to identify up to three viable controls for each of its subject local authorities. “It’s the key element for the for the techniques we are using. It’s given us the means to do what we wanted to do using a method we wanted to use. It was very complex, but the CIPFAstats+ solution enabled us to establish good controls, and that gives us a more scientifically valid method of analysis and research. As a result, that also gives us a scientifically robust means of creating causal inference, which is vital, because if you want to analyse a policy, you need to be able to identify that the cause and effect can be linked.”

The research project is scheduled to run until March 2023, with findings published shortly after. 

Contact us to find out more on customerservices@cipfa.org or 020 7543 5600.