The Valley Metro Mobility on Demand (MOD) Sandbox project was created to provide an updated version of their previously developed mobile application with innovative features. These enhanced features included a fully accessible transit app, the Pass2Go® app, which provides a mobile, single ticketing and payment system and connects users with multiple public and private transportation options. Three-hundred and thirty-two (332) participants were involved in testing and evaluating the app, first taking a pre-study survey, using the Pass2Go app, afterward reporting their experiences in a post-study survey. Paired observations from both surveys as well as app activity and payment data, the project’s open data platform, and insights from expert (stakeholder/project partner) interviews during January 2017–December 2019 were used for project evaluation.
Different lessons learned were observed during project planning, technical development, data collection, user engagement and technical expertise, as summarized below:
- Recruit twice the number of participants needed to ensure a statistically viable sample size, especially if the research requires participants to commit to sustained engagement in order to meet all study requirements. Identify participants who are willing to engage in sustained participation.
- Develop a strategy with potential partners at the beginning of the project to enhance capability and provide subject matter expertise, as well as clearly outline partnership parameters to expedite agreements in the future.
- Limit the size of the project management team in order to effectively manage response times and gain consensus. A small project management team with a larger advisory group that could still be involved and provide feedback about the project is found to be effective.
- Proactively share knowledge nationwide with other agencies and projects to ensure lessons learned are spread and duplication of unproductive efforts and strategies is minimized.
- Confirm integration capability between old and new technologies, which may not always be feasible. Conducting an overall review of legacy technology to ensure compatibility before moving forward is recommended for future projects.
- Document the data collection and management approach in the evaluation plan and to communicate regularly with the independent evaluator. Conduct frequent initial reviews of the data produced followed by periodic reviews is also suggested.
- Engage expertise either through hired internal staff or third-party support to assist in pilot projects as having technical support close at hand is necessary to troubleshoot technical issues that can arise.