Bias, Group Think, Agency Theory and Bidding
It is interesting to note that forecast performance correlates to who commissions the forecasts: the 70% average becomes 82% (so, not so bad) when lenders commission forecasts, but just 66% if commissioned by others (Bain, R. & Wilkins, M. ?Credit Implications of Traffic Risk in Start-Up Toll Facilities?, Standard & Poor?s, September 2002). This suggests that forecast bias is, to at least an extent, influenced by clients despite the profession?s protestations of objectivity and neutrality. Indeed, research I previously conducted found practitioners to have only weak acceptance of bias in their work despite acknowledging that over-forecasts are more prevalent than under-forecasts (Di Bona, R.F. What are the Key Risks Associated with Private Investment in Start-Up Toll Road Projects in Developing East Asian Economies?, MBA Dissertation, Henley Management College, 2006).
Planners and engineers alike are keen to create solutions and see them implemented. And although having a study team all ?buying into? helps focus attention, there are dangers of descending into Group Think. Excessive optimism can blind those involved to a scheme?s potential weaknesses. Team members may be unwilling and uncomfortable to question their colleagues, bosses and friends. So rather than identifying (and addressing) downside risks, they are overlooked (and unaddressed).
After having formulated the ?big idea?, comes the challenge of getting it approved. It may be competing against other schemes for funding (be it from government or the private sector). There may be incentive to overstate benefits and downplay risks: after all, a rival scheme might do that to obtain funding. But an oft quoted statistic should be noted: every 5 years 80% of businesses fail ? so transport planners are not alone in such errors of optimism.
Optimism, Skyscraper Theory and Economic Cycles
Typically during (and particularly towards the end) the boom times, confidence in an endless boom often takes hold. Politicians might proclaim that they have beaten economic cycles (like Chancellor Gordon Brown declared in 2007 in the UK: ?We will not return to the old boom and bust?) as development projects are becoming aggressively more grandiose. Skyscraper Theory observes that the world?s latest tallest building typically opens as an economic crash engulfs its location (e.g. New York?s Empire State Building in 1931, Kuala Lumpur?s Petronas Twin Towers in 1998, Dubai?s Burj Khalifa in 2009/2010). These become quite visible manifestations of previous excessive optimism. Similar problems face transport infrastructure: Figure 1 shows how projects conceived prior to a boom, opening in the early phases thereof, are likely to be relatively successful (conditions improve whilst implementation occurs). These successes encourage more projects, as conditions continue to improve; these likewise may be successful. Towards the end of a boom, the number of projects being planned increase markedly. But owing to lead times between conception and completion, by the time this larger set of projects are completed, economic conditions have deteriorated. These projects are not deemed successful, leading to fewer projects being undertaken.
Infrastructure is still Important
The experience of railway development in the UK provides a good illustration of infrastructure remaining important: networks were developed privately. Often, early investors did quite well; but later investors did not. However, small towns in UK with branch lines prospered (even if the investors in the branch lines did not), whilst those towns without railways declined. The problem is that infrastructure development can be hampered by perceptions related to the many projects conceived relatively during boom times which open after the boom has gone.
Back to those State-of-the-Art Techniques
Models are, by definition, simplifications. And different models can hav
e different uses. The State-of-the-Art techniques should not be ruled out per se. Rather, it should be understood where and how such techniques might be worthwhile.
Traffic micro-simulation can be useful at determining in more detail the likely performance of traffic management and engineering measures. However, given its sensitivity to traffic flows, such analysis is best used for just short-term assessment. Forecast uncertainties should preclude this from longer-term assessments in rapidly developing environments. Also, driver behaviour can vary a lot between countries and between different parts of the same country (both rural versus urban and between different cities). So such models require very careful calibration to local conditions. Of particular concern in an Indian environment is the wide mix of vehicle types and determining how they interact with one another.
Trip-chaining and activity-based models look at daily travel patterns and linkages between different trip purposes; this is in contrast to the traditional trip-based models. Increasingly popular amongst many (though by no means all) practitioners in western economies, these require much more data to establish robust and meaningful functions compared to traditional trip-based models. Consequently, they are quite vulnerable to error in rapidly developing environments, if relied upon to give the ?best? answer. Nevertheless, they may have a role to play if paired with land use-transport interaction (?LUTI?) models.
Such LUTI models look at how land will develop (within land use zoning constraints which are usually assumed to be strictly enforced) taking into account transport accessibility and linkages with activity-based models. They iterate between land use and transport models. These models can be useful at a very strategic level to evaluate different ways in which cities may develop. However, they too require a lot of assumptions such as the length of behavioural lags (how long people take to adapt to new land availability or transport options). As such, these models could be used perhaps to set general development policy, rather than to assess transport requirements in detail.
So what should be done?
Much of the above may appear confusing or even contradictory. To an extent that is an inherent problem of trying to give broad advice: most situations have their own exceptions to generic rules. The key is perhaps to remember that our duty is to give the best possible practical advice to decision makers. And perhaps the first step is to warn of the limitations of the advice we can give. Nevertheless, to give more constructive advice, I would suggest:
- Keep things as simple as possible. Especially in rapidly developing environments: the more variables and assumptions, the greater the scope for error.
- Fit-for-purpose means best suited to the project?s requirements. Adopting state-of-the-art techniques for personal vanity or CV building is likely to backfire. For sure, appreciate the range of techniques available, but the best way to develop a profile and capabilities is to be successful through finding the best solution. This does not necessarily require sophisticated modelling.
- Explain any assumptions made which can be critical; also identify which parameters are excluded from the analysis. For advice to be cogent, its own limitations must be explained. Excluding less relevant factors based upon the principle of Occam?s Razor is good practice, but document what was explicitly chosen to exclude together with reasons for same.
- Tailor analyses to the question at hand, even if that means having to change the models for each project (and quite possibly cutting out as much detail as new detail gets included). For policy level studies, LUTI might be feasible (subject to data availability and reliability of course). Restrict highly detailed analyses to short-term, small area situations.
- Always evaluate alternative scenarios. Not just simply a Base Case with a slightly different economic and/ or population growth rate to develop a Low (Conservative) and High (Optimistic) Case: that is merely sensitivity analysis (important in its own right but this does not constitute scenario-based analysis). Think about possible development paths (social, economic and land use) and transport policy regimes which are qualitatively different. Develop a range of possible cases.
- Think critically and sceptically. Ensure that your group comes up with a set of conflicting outlooks on how things may develop. Though some form of consensus is required to work together, if your group agrees on every aspect then it is most likely that you are overlooking some very real eventualities.
- Rather than simply adopting a solution which seems to give the highest revenue, ridership, NPV, IRR or whatever on your Base Case, find a family of solutions which perform well across a range of outcomes deemed most likely (and ideally which would also perform satisfactorily under less likely outcomes). If need be, this may require the setting of ?trigger points? to implement some measures rather than necessarily pegging implementation to Year 2020 or Year 2025.
- And perhaps most important of all, remember the adage ?we learn from our mistakes.? Learn the good and the bad from past experience, rather than downplaying errors to convince people of one?s infallibility. These will often be the best pointers one can get ? from direct personal experience ? when seeking the most appropriate approaches and hence, outcomes on our future assignments.
rience gained in projects in over 25 countries. He has undertaken a wide range of transport demand forecasts ? urban, interurban, highways and public transport, multi-criteria evaluation of both infrastructure projects and policy initiatives. He peer reviews and audits of forecasts prepared by other practitioners. He may be contacted at rfdibona@yahoo.com).