Final Evaluation Report: Evaluation of the Idaho Transportation Department Integrated Road-Weather Information System
The project created a new page, the Road-Weather Integrated Data System (http://itd.idaho.gov/apps/rwids_public/), or "RWIDS," on the Road Report traveler Web site. The Web page consolidated data from several relevant sources: ITD's two different brands of ESS (previously accessible only to ITD maintenance personnel via two separate user interfaces) as well as a number of ESS, within and bordering Idaho, operated by a wide range of organizations. The Web page also incorporated additional types of weather information drawn from sources including: non-transportation weather data from Meso West consortium, ITD closed-circuit television camera views, and National Weather Service satellite and radar images, watches and warnings. A password-protected version of the RWIDS Web page, containing printing and e-mail alert features not available to the public, was made available to ITD maintenance personnel. This lesson is based on a Federal Highway Administration sponsored independent evaluation of the ITD RWIS Integration project.
This project demonstrated the importance of conducting follow-on training with maintenance personnel when providing them with a new winter road maintenance information tool, and then monitoring their utilization of that tool. The ITD experience provides two specific suggestions for deployers of winter road maintenance information resources:
- Provide hands-on, increasingly sophisticated follow-on training. The ITD provided an initial orientation to the RWIDS tool to maintenance personnel throughout the state. Although most personnel that were interviewed found the initial training to be useful, nearly all of them felt that additional training was necessary in order to get the most out of the resource. Specifically, the follow-on training should include hands-on exercises, with each trainee having access to the webpage, and should delve into how information could be used to support various types of winter maintenance decisions. Training should be an on-going, long-term activity, with periodic (e.g., annual) refresher courses. Training interactions also should be viewed as an opportunity for two-way communication between those who designed the RWIDS and those who would use it for roadway maintenance duties. In addition, training sessions should provide an opportunity to solicit users’ suggestions for how the system could be improved.
- Build-in reliable mechanisms to monitor system usage, including usage by individual maintenance offices. Qualitative feedback from the maintenance staff is critical in ascertaining the effectiveness of an information resource, but it is also important to independently and quantitatively monitor utilization, including the frequency of access, duration of website sessions, and information consulted. Coupled with direct, subjective user input, such data provide a powerful insight for refining the information resource and flagging the potential need for additional training. It is therefore important to build into the system the ability to automatically capture usage data and the ability to conveniently generate useful analytical reports based on those data. If possible, usage monitoring should include the ability to examine usage parameters by individual maintenance office or even by individual personnel. This allows operators to better understand how locally variable factors such as the type of Internet access, the number and type of ESS, specific local storm events, and local maintenance practices and policies impact utilization.
This lesson suggests that the effectiveness of a winter road weather information tool, in terms of its ability to improve maintenance personnel’s productivity and to provide safer and more efficient travel conditions, is enhanced with adequate staff training and utilization monitoring. Most of the maintenance personnel valued the RWIDS tool but felt that additional training would result in greater benefits.