Bringing in model estimation and implementation projects on time budget and to the agreed scope is a challenge.
There are good reasons for this. I have described almost 80 tasks and sub-tasks required to set up a city model, each of which will involve a whole series of subsidiary activities, many of great complexity. Consequently determining at the outset the detailed scope of the model, the resources required to implement it and the time this will take is extremely difficult.
But there are also bad reasons for this. For most of these tasks, sub-tasks and activities, there are many alternatives ways of doing them, only a few of which will actually prove to be appropriate. Poor technical decision-making will mean that resources are wasted on inappropriate solutions ("blind alleys"). Model development involves large amounts of data manipulation and processing, for all of which there are risks of error, and it is common to find tasks being redone many times. The costs of this repetition are magnified by the sequential nature of these models, in that errors made at an earlier stage may mean not only redoing the original tasks but also all subsequent tasks. It is not difficult to see that poor technical decision-making and insufficient attention to accuracy can easily double or triple the resources and time required for model development.
In the following paragraphs I make some suggestions on what are the key technical issues to focus on in managing modelling projects, and there is more in my section on modelling in the textbook Traffic Engineering and Management published by Monash University (2004). In the following I adopt a typical "project lifecycle" sequence.
Given the impossibility of accurately assessing the resources required for these major and complex projects, include resource contingencies (say 10-15%) so that you have spare resources available when you hit problems. Ideally make sure that your customer/client is aware that when you get down to detailed planning, there may be a need to change the balance of resources, discuss adjustments in scope etc.
At this stage, your customer/client will have responses to your original proposal, which will need to be adapted so that you can meet all requirements. The Statement of Requirements document should be produced, often as an outcome of a customer workshop, and signed off by both your client and yourself. Unless there is a clear, shared and documented appreciation of the project scope, you can be sure that there will be difficulties, confusion and misunderstandings later on which will undermine your relationship with your customer. Note that this appreciation can include uncertainty - clients with a real understanding of the complexities may be willing to acknowledge uncertainties over the scope of future activities.
The main technical activities here are to produce the Functional Design of the model and then, once this is agreed, the Technical Design of the model.
The aim of the Functional design is to set up a broad modelling structure, with the functionality to address the agreed customer requirements. If these turn out to be over-ambitious, then the Functional Design can stimulate a dialogue leading to something more sensible.
The purpose of the Technical Design is to think through the model implementation process in detail and thus to exclude inappropriate methods and so avoid resources being wasted on blind alleys. It is also to identify and plan for the most risky tasks. A well-specified Technical Design is also very effective as a management tool, enabling staff to carry out tasks using the specification, thus needing less direct technical management.
With the best will in the world it is not feasible to anticipate all technical details at the outset of the project and the Technical Design is therefore likely to be somewhat sketchy for later tasks. Consequently the design should be reviewed and updated before these tasks are commenced.
The key to keeping a modelling project on track is to make technical design and implementation decisions which are efficient in the use of resources and effective in solving problems and meeting client requirements more or less continuously through the project. While project monitoring (including the packages) helps in identifying cost or time overruns, it is the management decisions which actually keep the project to plan. Managers who spend too much time on monitoring systems, rather than taking the management decisions, are useless.
For a technical manager, the implementation phase is one of great activity because the process of tuning the Technical Design into a working model is one of refinement, adjustment, innovation and problem-solving. The more clever and experienced the team, the more chance there is of a successful implementation phase.
This is all about model acceptance tests, demonstrating suitability for purpose (and also of course hand-over to the customer and training of his staff). The tests should not be shirked, a comprehensive and successful programme will provide a secure foundation for later model applications.
The process of model estimation, often disaggregate and usually on an individual sub-model basis, may give little insight into the performance of the final model system once all of the components are combined. This means that the process of acceptance testing can be fraught, and it is best to plan for re-estimating the individual sub-models in a final tuning process.
It is however my experience that if these models are put together carefully and skilfully, recognising the accummulated international experience, then they prove to be accurate, reliable and robust. But it is common to find the opposite, models carelessly put together with little appreciation of the history of development and research in the literature and, as a result, providing poor and unacceptable performance.
Skills and experience are the key to successful modelling projects. The Technical Design, which is the basis of the approach advocated here, can only be written by very experienced staff. No single person is likely to have sufficient experience of all aspects of the more complex models, for which reason it is advisable to draw on other, perhaps external, expertise. That said, independent experts are unlikely to have the same concerns over resources, timescales etc, so it is essential that the project team has sufficient skills to negotiate and interpret their advice.
None of these experts may actually do much of the work, they are responsible for advising on what should be done and how it should be done. Those who do the work need to be highly numerate and for model estimation projects, to have a real understanding of statistics. All should be computer literate and some should also be skilled programmers, because there are inevitably processes or procedures that are not easily handled within the confines of transport modelling packages. These people are managed by a technical project manager or team leader, and this is perhaps the key role in the team.