The assessment of the scope for Network Rail to improve on its cost efficiency is central to our work, since it enables us to establish efficient levels of track access charges when we periodically review Network Rail's outputs as part of a balanced package.
The use of international benchmarking is necessary as Network Rail is a national network monopoly, and hence there are no straightforward direct domestic comparators we can compare it to. The international group of railway infrastructure managers we use in our efficiency analysis is made up of European mainline rail infrastructure managers.
Our analysis of Network Rail builds on our work from the 2008 periodic review (PR08), which set Network Rail's outputs and funding levels for control period 4 (CP4). Comparing Network Rail's maintenance and renewal cost efficiency directly with that of other rail infrastructure managers shows, at least partially, how the company is progressing on its cost efficiency over time against the base assumption we made in PR08 that the company should be able to improve on the efficiency of its controllable operating, maintenance, and renewal costs by at least 21% over the five years covered by CP4.
The 2013 periodic review (PR13) developed the approach used in PR08 by drawing on recent developments in the literature on efficiency benchmarking and employing a wider range of models. We also opted to present the results of a set of four models: we considered all of these models to be sufficiently robust from an econometric and engineering perspective, and to provide a reasonable model of a reality which is fundamentally unknown. Rather than choosing one of these specifications as the "preferred" approach, we instead accepted that there is inherent uncertainty as to the true model and have carried all of these models through to our results. The analysis produced a distribution of possible efficiency gaps for Network Rail in 2010 ranging from 13% to 24%.
We intend to carry out an update of our econometric benchmarking work each year, because regular updates provide important cost efficiency information to us, Network Rail, and interested stakeholders.
Econometric benchmarking
Collapse accordion
Open accordion
PR13 Econometric benchmarking using the LICB
Our PR13 econometric analysis used a subset of the Lasting Infrastructure Cost Benchmarking (LICB) dataset developed and maintained by the International Union of Railways (UIC) for 14 European rail infrastructure managers, including Network Rail, covering the period 1996 to 2008. We are very grateful to the UIC, its members, and Network Rail for making the dataset available and for the discussions and presentations we had with Network Rail and the UIC.
This paper documents the process followed to reach the international top down benchmarking results documented in the 2013 Periodic Review Draft Determination. It records ORR's work on improving the quality and reliability of our econometric estimates of Network Rail's efficiency gap, and the steps taken to address questions put forward by Network Rail and recommendations from reviews of the 2008 Periodic Review (PR08) work.
We have commissioned a peer review of this work by Dr. Michael Pollitt at the Judge Business School, University of Cambridge. His comments on this process are available below.
Updates to PR08 econometric benchmarking using the LICB
Our 2010 update included extensive econometric model development and sensitivity testing. Our results showed that, in 2008, Network Rail was between 34 to 40% less cost-efficient than the top European infrastructure managers in the peer group. This result broadly confirms the econometric analysis we undertook in PR08 which showed that, compared to the top European infrastructure managers, Network Rail was around 40% less cost-efficient.
By comparison with the best 25% of international companies, the cost efficiency difference for Network Rail is 29 to 37%, considering the full sample of international comparators over thirteen years from 1996 to 2008 inclusive. This represents an improvement against our PR08 econometric modelling results, where we estimated that there was a difference of 37% measured from the overall frontier made up of the best 25% of international comparators.
We recruided Oxera to review the econometric models used to undertake international benchmarking for the 2008 periodic review (PR08). The findings of their assessment are detailed in the report below:
Review of econometric benchmarking methodology
Top-down, and specifically econometric, cost benchmarking of regulated network activities is routinely carried out by other regulators in Europe and beyond. Together with the Centre on Regulation in Europe (CERRE), we have summarised the main techniques used in regulatory benchmarking worldwide (econometrics, linear programming) and prepared a survey of international benchmarking practices in other network industries, with a number of case studies (Australia, New Zealand, Austria, Germany, the Netherlands, and Norway).
For each country, we considered the object of benchmarking (operating expenditure, capital expenditure, total expenditure), the techniques adopted, model specification (inputs, outputs, and context variables), whether different approaches are employed and cross-checked in practice, and how benchmarking results are incorporated into the regulatory framework. In particular, the report considers how and whether efficiency estimates feed through the price/revenue allowance process.
- Download the CERRE report
- Download the September 2011 CERRE Workshop presentations
Other benchmarking
Collapse accordion
Open accordion
Gap analysis
We have conducted further work to understand the drivers of the difference in maintenance and renewal cost efficiency between Network Rail and its international comparators.Two studies undertaken by Balfour Beatty Rail for us show that the cost efficiency difference between Network Rail and four international comparators is driven by a number of activities, including procurement strategy, possessions strategy, and approaches to asset management. We will continue to develop this work with Network Rail in PR13.
Frontier and catch-up efficiency
In our Determination of Network Rail's outputs and funding for 2009-14, we explained that our efficiency assessment for Network Rail covered three elements: 'catch-up efficiency', 'frontier-shift efficiency', and input price inflation.
The catch-up element related to the expectation that Network Rail could achieve cost savings by catching up to the levels of performance of more cost-efficient companies. The frontier-shift element related to the expectation that, in addition to any catch-up, Network Rail should be able to improve on its cost efficiency over time. Our decisions on frontier-shift and catch-up elements were made in light of consultancy reports which provided analysis of historical changes in measures of unit costs and productivity for other UK companies and sectors.
We commissioned Reckon LLP in 2011 to carry out an update of that analysis. Their report does the following:
- They provide estimates of historical changes in measures of unit operating expenditure for some regulated network industries in the UK. They use recent data to update an analysis of unit operating expenditure that was carried out for us at the last periodic review. They also provide some further analysis using alternative data sources and output measures.
- They provide estimates of historical changes in measures of productivity growth, based on data for sectors of the UK economy. They use recent data to update an analysis of 'Total Factor Productivity (TFP) composite benchmarks' for Network Rail which was carried out by our consultants at the last periodic review. They also provide estimates for some alternative measures of productivity growth for sectors of the UK economy.
- They review the way in which we used this type of information at the last periodic review, drawing on the approaches used by other GB regulators. They suggested potential improvements to us and, in light of their suggestions, they carried out some further quantitative analysis.