Design Machine Learning System Outputs to Match User Capacity and Operational Goals when Designing AI Applications for Transportation.
Case Studies from Five States Showed Deployment Lessons for Future AI Applications for Transportation.
Statewide, Nebraska, United States
Statewide, Missouri, United States
Statewide, Delaware, United States
Implementing and Leveraging Machine Learning at State Departments of Transportation
Summary Information
Artificial intelligence (AI) offers the potential to enhance transportation systems by improving safety, reducing congestion, and enabling more efficient use of resources. This project focused on raising awareness, documenting successful applications, and developing practical tools and guidance to support AI integration aimed at improving transportation safety and efficiency. It involved a nationwide survey of state departments of transportation (DOT), in-depth case studies with five selected agencies, and a synthesis of current practices and available AI resources. The research team also compiled publicly available code and toolkits and created three prototype AI applications tailored to transportation needs. A key outcome of the study was a comprehensive guide featuring a 10-step roadmap designed to assist agencies in evaluating, implementing, and managing AI initiatives. Among the case studies included were the experiences of Nebraska, California, Iowa, Missouri, and Delaware DOTs.
Some key lessons learned are presented below.
- Design machine learning (ML) system outputs to match user capacity and operational goals. The Missouri Department of Transportation (MoDOT) improved the effectiveness of their predictive analytics tool by limiting crash risk alerts to five locations per 3-hour period—down from 70, to ensure Transportation Management Center (TMC) operators could act on the information, balancing usability with system coverage.
- Use Small Projects with Clear Goals to Build Executive Support for AI Implementation. Nebraska DOT’s AI pilot demonstrated that starting with small, well-scoped projects helped gain leadership buy-in and justified further investment in AI initiatives.
- Incorporate critical system capabilities during the initial design phase. The Delaware Department of Transportation (DelDOT) noted that features like modularity must be embedded from the beginning, as retrofitting such capabilities later is often difficult or impossible.
- Plan for ongoing operations, maintenance, and retraining of ML models. DelDOT emphasized that ML models can degrade over time without consistent updates, monitoring, and retraining to reflect current transportation conditions. Design detector systems with future maintenance and data fusion in mind is also essential for long-term success.
- Evaluate the trade-offs between in-house development and vendor procurement. Balancing cost, expertise, and control when choosing between internal solutions and external providers is important; mixed lessons were learned from both investing in internal development teams to enhance agility and reduce dependency and partnering with specialized AI/ML vendors for greater flexibility and innovation.
- Use human-verifiable outputs to communicate AI performance. Caltrans found that visual results from computer vision models helped non-technical stakeholders better understand and trust results, aiding regulatory approval and executive buy-in.
- Prioritize minimizing false positives in safety-critical applications. Iowa DOT’s experience showed that even models with near-perfect accuracy can generate disruptive false alarms at scale, emphasizing the need for careful evaluation of model precision.
- Evaluate integration complexity before selecting off-the-shelf AI/ML solutions. MoDOT found that off-the-shelf tools were not always easier or faster, due to significant integration hurdles with agency systems.
- Develop a strategy to assess data quality and eliminate sources that do not add value to system performance. Not all data contributes value to AI models; MoDOT improved efficiency by eliminating less useful inputs after evaluating their impact.
