Ensure that ITS field operations tests use technologies and applications that are proven to be deployment ready.

Experience with a Field Operational Test for the Intelligent Vehicle Initiative in Minnesota

Date Posted
08/10/2010
TwitterLinkedInFacebook
Identifier
2010-L00533

Evaluation Methods and Lessons Learned from the Minnesota Department of Transportation Intelligent Vehicle Initiative Field Operational Test

Summary Information

The Minnesota Department of Transportation (Mn/DOT), under sponsorship of the USDOT and as part of the Intelligent Vehicle Initiative (IVI), conducted a field operational test (FOT) from December 22 2001 to March 31 2002 to establish the safety benefits of providing lateral guidance and collision avoidance systems to support drivers of specialty vehicles including snowplows, ambulances and State Patrol squad vehicles when driving in low visibility conditions. For the FOT, four snowplows, a state patrol car, and an ambulance were equipped with a Driver Assist System (DAS). The DAS integrated ITS technologies for maintaining lane position and avoiding collisions with objects in harsh weather conditions such as blowing snow, fog or rain. The FOT was conducted in the winter to exploit the Minnesota weather but unfortunately for the outcome of the project, the winter was one of the mildest on record for the state. Nonetheless, the independent evaluation of the FOT revealed lessons learned relevant to technology deployments in general and FOTs in particular.

Lessons Learned

Specialty vehicles such as snowplows, police vehicles and ambulances must operate in nearly all conditions, including during inclement weather when visibility is compromised by heavy snow or rainfall, increasing safety risks. There is the potential to improve the safety of specialty vehicles by installing ITS detection technologies that warn the operator when the vehicle begins to move out of its lane or when it approaches an object in the road. To explore this potential, the Minnesota Department of Transportation (Mn/DOT), in partnership with the USDOT ITS Program, conducted a research and development program on on-board safety systems referred to as Intelligent Vehicle Safety Systems (IVSS). IVSS consisted primarily of a lane departure warning system, a collision avoidance system and a driver assistance system (DAS) for showing objects on the forward road via a head-up display and for warning the operator with auditory, visual and haptic cues when the vehicle began to deviate from the lane or encountered an object in the forward path.



The IVSS research culminated in a Field Operational Test (FOT), conducted in Minnesota from December 2002 to March 2003, of the safety benefits and driver acceptance of the systems. The FOT deployed four snowplows, an ambulance and a State Patrol squad vehicle that were equipped with the IVSS. The FOT occurred in the winter months to take advantage of the typically heavy snowfall in Minnesota and test the systems in the condition for which they were designed, namely, very low visibility. However, two limitations to the FOT emerged that diminished the capability of determining the safety benefits of IVSS. First, the winter turned out to be one of the mildest on record for the state without producing the desired (for the FOT) low visibility. Second, it emerged during the FOT that not all of the on-board technologies were not deployment ready and did not perform reliably. The evaluation of the FOT revealed lessons that are relevant to the design of ITS FOTs and include the following recommendations.

  • Ensure the ITS technologies to be evaluated are deployment-ready. The IVSS FOT produced inconclusive results because the test conditions did not materialize due to the mild winter but also because the technologies did not perform as designed or reliably. One lesson from this experience is that an FOT will be compromised when the technologies under assessment are not deployment-ready. Although this point seems obvious, its importance must be reinforced. The pressure to continue a project as planned may compel agencies or personnel to move forward with a schedule even if systems are not demonstrably ready to be tested in the field. From a project manager's point of view, delaying a field test can incur negative consequences. However, there are lost opportunity costs and perhaps safety costs associated with following through with scheduled tests when systems are not ready. FOT plans should include steps and requirements for confirming that technologies meet a minimum standard of reliability and performance before deployment. It is also worth noting that the problem of system unreliability appeared to have damaged operator acceptance of the system. Interviews and questionnaire data of the operators show that operators were initially optimistic about using the system, but they lost trust in the system due to technical problems such as the intermittent signal loss and unstable images on the head-up display, among other problems.
  • Develop contingency plans for field tests requiring specific weather conditions. The premise of the IVSS FOT was that the winter months in Minnesota would generate significant amounts of snowfall and create the conditions necessary for the evaluation of IVSS. The winter, having been atypically mild, did not generate the conditions necessary for the evaluation. Thus, the results were of limited value. This outcome underscores the risks involved when research designs are dependent upon externally-controlled events or conditions (such as weather). Thus, to the extent possible, FOT designs must include alternative test scenarios and options. For example, although inconvenient, an FOT requiring particular weather conditions should have flexible start dates so that testing can occur under the desired weather conditions. Test scenarios could utilize simulation and modeling opportunities if possible until the desired environmental conditions emerge. Another consideration is to partner with an agency that is located in an area that has the test conditions. For example, several operators in the IVSS FOT expressed the idea or concern that the IVSS was better suited for snowplows in the rural areas of Minnesota that has more severe and frequent low visibility conditions, as opposed to the location designated for the FOT.
  • Involve end users early in the design phase for the operator interface. The snow plow, ambulance and patrol car operators in the FOT indicated in interviews and questionnaires that they approved of the concept of the IVSS but that they experienced problems with the interface and lost trust in the system due to its unreliable performance. For example, operators noted that the IVSS equipment installed in the cab was bulky and awkward, with the projector located in a way that meant the operator's shoulders frequently hit it. The ambulance operators in particular viewed the equipment as inhibiting communication with other emergency personnel in the vehicle and elected not to use the technologies (although it is possible that they may have had there been low visibility conditions). Similarly, operators disliked the fact that the auditory warning shuts the truck radio off, and many operators tried to lower the volume of the auditory warning and said that they would have disabled it if they could have. Involving end users in the design phase in straight-forward usability sessions or focus groups can prevent or mitigate problems with the interface that will be too late to change after a system is built, installed and deployed.



On-board ITS applications have the potential to improve the safety of specialty vehicles by assisting operators in low visibility situations due to inclement weather. However, realizing the safety benefits of these technologies requires further research and development, and field evaluations. This lesson learned entry highlights the importance of conducting FOTs with systems that are proven to be ready for field deployment, using flexibility in schedules and/or locations to conduct tests in the weather conditions needed for testing, and involving end users early in the design process to produce better operator interfaces.