Transportation planners often use statistical metaphors for describing the credibility of the travel demand models they develop and use. The process of model estimation, calibration, and validation is largely a quantitative exercise. A model is considered successful if it meets preconceived levels of accuracy in replicating observed data and empirical knowledge. Planners can typically assess ``how good a model is'' in relation to most others by simply comparing graphical and tabular summaries of observed counts versus modeled link flows. Formal standards for model accuracy are sometimes used, and far more extensive comparisons are often made to appropriate targets for each model component. In some cases more detailed comparisons with models from similar areas, consultant reviews, and peer review panels are used to provide additional validation of the models or forecasts developed using them. Such levels of scrutiny typically satisfy the public and federal sponsors, and are the norm in our profession.
The situation changes dramatically when the models or forecasts derived from them are contested. Models then come under at least three unaccustomed levels of scrutiny: public, political, and judicial. While many of the actors in these realms are well-educated and intellectually sophisticated most view models in a much different way than transportation planners do. The former may seem indifferent to the statistical proofs offered by the latter, and believe that the latter seem narrowly fixated upon them. As a consequence the two groups often talk past one another.
Having again found themselves in the middle of controversy over models the authors have recently interviewed dozens of political leaders to gain insight into the factors they use to evaluate travel demand models. Not surprisingly, the concept of statistical validity is far less important to them than credibility. The results of their interviews will be summarized, with recurring themes and issues highlighted. The main thrust of the presentation, however, will focus upon how modeling processes and tools might be better designed in order to enhance their credibility from the outset. Examples from recent contested models and forecasts will be used to illustrate these points.