By DataKind San Francisco
DataKind San Francisco partnered with Starting with Today on a data advisory project to streamline how they measure the economic and social impact of its programs through improved survey assessments. Previously, DataKind DC worked with SWT on a DataCorps® project on building data maturity for community empowerment. Through these collaborations, DataKind was able to identify areas with high needs and provide actionable surveys to evaluate program impact.
Background
Customer feedback tools are critical for nonprofit evaluation but as prior HBR and SSIR articles have shown, “only 13 percent of social sector leaders believe they have been able to turn their feedback aspirations into a reality” to drive continuous program improvement.
Starting With Today (SWT) leads community-based programming and advocacy training to improve Black wellbeing in the DC Metropolitan area, working in close partnership with local Black businesses and community hubs. SWT’s mission is to help participants with career development and economic empowerment by offering culturally relevant programming. SWT was interested in augmenting its customer feedback and surveying methodology to improve participant and community outcomes.
Data Advisory Project Execution
For this project, SWT wanted comprehensive recommendations on how to streamline their surveys to aggregate insights and deploy the tool in an automated, cost-effective manner. The goal of this data advisory project was to streamline the questions asked in the program evaluation tool and gauge the impact of their workshops to provide recommendations on which tools SWT should choose for their platform. Upon implementing these recommended tools, SWT will have a simplified evaluation process for their programs, with the ability to automate key insights.
The project began with preliminary scoping to understand the current state of SWT’s data collection and surveying process and their key bottlenecks in the existing workflow. This involved a deep-dive into analyzing the current infrastructure, and identifying the specific key performance indicators (KPIs) that SWT wanted to measure as outcomes through their surveys. Through multiple conversations, we were able to deeply understand how SWT envisioned the key outcomes to be measured from their program evaluation deployment.
Survey Preparation
Our first main deliverable was to streamline SWT’s surveys to measure participant experience and to have a better understanding of the impact their workshops are making on members of the community.
As a part of their programming, SWT offers diverse workshops including Barbershop Talks on mental health, sessions on financial literacy, and professional networking- all specifically designed for the Black community in DC to address local racial disparities and economic challenges. Initially, their surveys, conducted before and after workshops, consisted primarily of binary questions tailored to each session’s theme (“check the statements you agree with”), complemented by a single rating question (“How useful was this session on a scale of 1-5?”). However, in collaboration with DKSF, SWT recognized the need for more nuanced feedback. They revised their survey approach to include a broader spectrum of response options and expanded the focus to cover more comprehensive community and personal questions, moving beyond solely workshop-related inquiries.
Image above courtesy of Starting With Today
The revised survey template introduced several changes: a five-option response scale (Strongly Disagree to Strongly Agree), questions on community inclusion and personal well-being, fewer workshop-specific questions, one open-ended question for additional feedback, and a format suitable for both print and digital use. Additionally, survey frequency was reduced to post-event only, focusing on the impact of SWT services. Despite being slightly longer, the new format, which includes about ten scaled, one open-ended, and a few demographic questions, has been well-received in initial sessions. Over time, this template aims to yield a more detailed understanding of participants’ experiences and growth, offering clearer insight than the previous binary format.
A snippet of the new survey template can be seen below.
Survey Platform Recommendations
Our second deliverable was to identify and recommend a survey platform that would seamlessly integrate with SWT’s unique requirements. In particular, SWT was looking for digital survey tools with intuitive user interface for building survey templates, skip logic capabilities, QR code functionality for quick survey deployment, and built-in tools for analyzing participant engagement and analyzing survey results. The team also prioritized budget implications on SWT and researched if any long-term nonprofit discounts were offered.
With these guidelines in mind, we explored eight potential platforms. By conducting thorough research through demos, reviews, and detailed feature comparisons, the team narrowed down to two platforms- TypeForm and SurveyMonkey. Both excelled in key areas but also offered distinct advantages.
Typeform stood out with its visually appealing interface, intuitive design, and focus on user experience, aligning well with SWT’s budget. However, it offered limited built-in support for advanced analysis of survey results, relying on Google Analytics as an alternative tool.
SurveyMonkey, in contrast, featured a broader range of functionalities, powerful reporting tools, and extensive customization options, catering to more complex survey needs. However, its pricing required careful consideration.
Illustrated sample logos of sample survey vendors
In addition to Survey Platforms, we researched available options to convert SWT’s existing paper format surveys to digital format using OCR (Optical Character Recognition). We concluded that SWT will need to subscribe to services that are privacy compliant such as Adobe or Google Cloud to use OCR software .
We’re confident that our recommendations will empower SWT to gather valuable feedback, gain critical insights, and ultimately achieve their organizational goals.
As the DKSF Data Advisory team grows, we hope to partner with more nonprofits in need of data infrastructure guidance! We hope this project can be a useful framework for future data advisory projects requiring impact measurement and survey deployment. If your organization is looking to partner with us, reach out to sfbayarea@datakind.org to learn more.
The project team and those who co-wrote this blog post included:
- Starting With Today: Charlyn Anderson (Founder and CEO) and Zuri Hunter (Board of Directors, Data and Evaluation)
- DataKind San Francisco Volunteers: Padma Chandramouli, Jillian Penaflor, Ramya Vishwanath, Anjana Sundaram, Jaya Pokuri
We’d like to also thank Natalie Jackson, Meareg Hailemariam, Vishnu Nair, Melinda Tellez (DKSF), Caroline Charrow (Datakind), and Melissa DiLoreto (DataKind) for their support and guidance throughout this project.
Header image courtesy of Starting With Today.
Join the DataKind movement.
- Interested in supporting our work? Donate here.
- Interested in sponsoring a project? Partner with us.
- Interested in volunteering with DataKind? Look no further.
- Interested in working at DataKind? We’re hiring!
- Interested in submitting a project? Go for it!
