Dubai Metro, Dubai, United Arab Emirates.

+ Evaluation of projects such as the Dubai Metro should attempt to better represent the wider benefits.

I think we need to find ways of better evaluating the economics of new designs, ideas or projects. 

Traditionally, major projects are subject to financial and economic evaluation based on quantifiable parameters. So, for example, you use factors like travel time reductions or accident rates to evaluate a transport project. 

Qualitative aspects are included in common methodologies such as WebTag in the UK, the Federal Transit Administration approach for major capital investment in the US and World Bank guidance. But they’re often treated as a by-product and have little impact on decision-making.

Why is this a problem? Well, city planning, particularly investment in transport schemes and new development, is undergoing major changes. Intangible objectives such as supporting innovation and smart initiatives, creating sustainable and resilient projects, promoting social inclusion and liveability, and influencing behaviour are becoming the main drivers for new projects, technology and investment.

This has created a dilemma because standard assessment procedures for financial, and in particular economic returns, are struggling to adequately quantify whether these largely qualitative objectives have been achieved. Even existing methods for estimating environmental benefits can be inconclusive. They’re often not robust and hence not accepted or are given less weight to get them accepted. 

To support smart, sustainable and social planning objectives, I think it is crucial to rethink the standard approaches. New ideas and concepts are needed for measuring how well a project is meeting these new types of objectives.

Commercial and economic impacts are even more important now that abundant financing – even in the Middle East – is a thing of the past. More and more projects are financed by investors, and they want to see an estimate of the likely returns.

So what needs to be improved? I can see three key areas: 

Firstly, existing evaluation methods need to be more accurate and more reliable. This means further research into the parameters and impacts, and enhancing the modelling tools used for deriving appraisal data. For example, there is a method for reporting the physical activity impact of walking and cycling but the evidence for the health benefits is limited to a few studies and locations. And transport-modelling tools are struggling to accurately forecast walking and cycling trips and distances.

Secondly, new appraisal techniques need to be developed to quantify impacts that are currently not evaluated using available data. This would make it easier to measure sustainability benefits, which could be linked to the resource consumption and savings from recycling materials. For example, the environmental benefit of using recycled asphalt for road construction could be quantified. Similarly, the existing economic concept of time savings could be expanded beyond travel time reductions by considering other time savings and efficiencies. This could include time saved through smart applications, improved logistics and operations.

Thirdly, an accepted, standardised approach is needed for parameters that cannot be quantified, instead of simply discussing these impacts qualitatively. This is certainly the most difficult area and only achievable through international consent and extensive research. An index, for example, similar to the Human Development Index or Quality of Life Indicator defined for the European Union, could be produced to capture and compare benefits. This could evaluate things like public realm, access to schools and entertainment, or the level of security of a project. Counterfactual evaluation could help us validate such an index.

Do you think better evaluation will improve decision making? What things that matter are currently not measured appropriately?