Knowledge Resources

The FHWA Office of Operations Mobility Monitoring Program team gathered archived traffic detector data from more than 20 different agencies. In doing this, the team encountered a wide variety of data management and archiving practices. Some of the data management practices could be described as sloppy resulting in misleading or inaccurate data for performance-monitoring applications. Some data collection and management practices are simply not documented well enough, leading to confusion or uncertainty during data analysis.

• Establish careful and accurate data collection and management practices and document them thoroughly. The devil is in the details; seemingly minor data management practices can have significant consequences when using archived data for performance monitoring. Examples of such practices include the following:
  • When does zero really mean zero? Most traffic detector systems report zero values for volume, occupancy, and speed when no vehicles are detected during the reporting period. This can be a common occurrence if the reporting period is short (one minute or less) and the road has low traffic volumes (during early morning hours). However, some data archives average these zero speeds (which are really missing or null speeds) with other measured speeds when summarizing data for permanent storage. For example, a speed of 60 mi/h is averaged with a speed of 0 mi/h (no traffic) to get an incorrect average of 30 mi/h. The result of this practice is that several cities appear to have slow speeds in the early morning hours.
  • Is zero used as an error code? In other cases, traffic detector systems report zero values as error codes (for various hardware or software failures). Thus, the data analyst is typically unable to determine if the data are missing because of a detector malfunction, or simply because there was light traffic and no vehicles were detected during the reporting period.
  • How many samples are contained in the summary statistics? Several data archives summarize detailed traffic detector data before storage. For example, the data archive may retrieve one-minute data samples but only store five-minute summary statistics. Problems arise when the data archive does not record how many data samples were actually used in the calculation of summary statistics. Assume that we have one-minute data samples, but an intermittent communication failure prevented the collection of three of the five possible minutes of data. When the two available one-minute data samples are aggregated into the five-minute summary statistics, the volume subtotals will be inaccurate because three of the one-minute samples are not available. Analysts will not be aware of this inaccurate subtotal unless metadata are kept to record the number of data samples used in calculating summary statistics. The result of this practice is that detectors may appear to be undercounting, and system-wide vehicle-miles of travel (VMT) estimates may also appear to be lower than normal.
• Strengthen interest in data archiving systems among traffic managers. Traffic managers may be in the best position to champion and implement data archiving systems; they collect the data, maintain the equipment, and are most familiar with data collection devices and protocols. Greater awareness of the tangible benefits of developing and maintaining data archives would encourage traffic managers to champion such systems. Current trends and anecdotal evidence indicate that more traffic managers are beginning to take an interest in developing and maintaining data archives. There appear to be at least two applications that provide tangible benefits to traffic managers:
  • Performance monitoring helps traffic managers preserve or expand funding for operations
  • Detector status reporting helps traffic managers diagnose and troubleshoot extensive data collection systems
  Of these two applications, performance monitoring appears to be the most compelling application that is likely to strengthen traffic managers' interest in developing data archiving systems. Another application that could highlight the need for better data archives is short-term traffic forecasting that uses historical traffic patterns in algorithms..
  

As demonstrated in this lesson, when archiving data, it is important to make sure that the data are archived correctly, and that both the data and data management procedures can be easily understood by data users. With the availability of well-archived data, users of the data will have to spend less time discerning the relevance and accuracy of the data, thus improving the efficiency of their work. Additionally, increasing traffic manager involvement in data archiving may improve the quality of the archived data and provide operational benefits.