User acceptance testing with Iowa DOT staff was conducted in October 2018 to compare the accuracy and utility of three modeling tools used to predict road weather conditions.
Research and testing participants included:
- 9 Maintenance field staff from around the state
- 5 Central Office project team members
- 2 Staff from Iowa State/CTRE
Several models were tested before the Decision Tree model was selected as the preferred approach over Regression and Neural Networks models based on the number of correctly predicted road conditions. Of approximately 242,000 road condition observations, about 237,000 were correctly classified by the decision tree model, a 97.8 percent accuracy rate. This high-accuracy model can help field maintenance staff plan more efficiently for winter road condition changes. The relative simplicity of the model compared to the other approaches was also viewed as an asset.
Data Governance is very important
- Alongside the development of the algorithm, a user-interface was developed to communicate information produced by the algorithm. During testing, it was discovered that the data provided to SAS were incorrectly time-stamped, yielding some unexpected results in the visualizations that did not match real conditions at all. This experience illustrated the importance of carefully collecting, storing, and sharing data. Accurate data were then provided, which brought the results of both approaches in alignment.
- Further exploration into historical data and mis-matches between the expected and actual outcomes are underway, as are efforts to improve the usability of the user interface.
Decision Tree model is best for modeling winter road condition
- Three Model Building Methodologies were assessed: Decision Tree, Regression, and Neural Networks.
- The decision tree is the best model based on the number of correctly predicted road conditions and that there are simpler and less variables without sacrificing accuracy.