The City of Columbus Assessed Benefits from User Feedback and Usage Metrics on the Deployment of the Smart Columbus Operating System for Data Exchange, Analytics, and Visualization.
Columbus, Ohio, United States
Primarily funded by the USDOT’s Smart City Challenge, the Smart Columbus Program is a collection of eight transportation, mobility and data projects aimed at improving access to jobs, enhancing tourism, stimulating the economy, connecting residents to safe and reliable transportation, and supporting efficient and sustainable movement of people and goods throughout Columbus. The Smart Columbus Operating System (SCOS) forms the backbone of the other seven Smart Columbus projects. By ingesting, visualizing and sharing open data, the SCOS was crafted to host performance metrics, serve the data needs of public agencies, researchers, and entrepreneurs, and serve as the foundation for other program projects. The goals of the SCOS include containing accessible and user-friendly analytics and visualization tools, ensuring an open-source format to allow a scalable, extensible, portable, and sustainable platform for other cities to leverage, supporting regional initiatives and data sharing, interoperability with other regions and transportation systems, and ensuring management compliance with respect to security and data privacy. The SCOS was initiated in 2016 and the SCOS data platform 2.0 was launched on April 23, 2019, heralding the start of the project’s demonstration period, which was completed on March 31, 2021.
METHODOLOGY
The SCOS team employed the Agile development process, characterized by iterative releases of software and software updates, to evaluate the successes and shortcomings of each package released. The team created “personas” to understand the work performed by user agencies and understand the end users’ needs, itemized a “backlog” of all the development tasks that would be needed by a user. These features were then integrated into the software package continuously and were then tested and evaluated with user feedback. The SCOS was developed in a modular manner with cloud technology, which allows other cities to implement a custom version of the OS in a matter of only weeks, reducing subscription and development costs due to the open-source code.
During the demonstration of the SCOS project, Google Analytics was used to monitor user behavior and performance of the system. Google Analytics metrics were collected monthly and shared on the Smart Columbus Program SharePoint website, as well as amongst stakeholders and the USDOT. Data points that were collected included user visits, visit durations, page views, dataset downloads (number and which ones), queries created, API calls made, popular pages, and the visualizations that were created from certain datasets. User feedback on the SCOS was mainly collected through a survey on the Smart Columbus website, two Hackathons, and Technical Working Group (TWG) meetings.
FINDINGS
| Metric | Number |
|---|---|
| Public Datasets Ingested | 2,018 |
| Private Datasets Ingested | 64 |
| Users | 58,784 |
| Page Views | 193,980 |
| Dataset Downloads | 15,569 |
| Dataset API Queries | 67,529 |
Survey Findings:
- Implementing SCOS improved data sharing ability, reduced amount of time accesses the data and resulted a high number and frequency of data retrievals.
- SCOS received an overall good rating in terms of customer satisfaction. Seventy-six percent of agency users rated various functions of SCOS as “good” or “very good” with respect to data quality, freshness, and completeness.
- Ingestion methods, especially the preview mode, received good reviews from agency users, with 69 percent of users rated data extraction experience “good” or “very good”. Based on the findings from the SCOS demonstration, it can scale to handle large streams of data from multiple project sources (e.g., data exceeded two terabytes).
- For data discoverability, 59 percent of agency users rated their experience with finding data “good” or “very good”.
