iCouldBe | 
Nashville, TN
Objectives
- Understand to what extent youth mentees successfully complete the program to better target program improvements
- Understand how effective the mentor/mentee engagements are so iCouldBe can identify areas for improvement, better train mentors and improve the mentorships overall
Findings
- Understood how data science can help show new ways to define “effective engagement” in the iCouldBe program beyond simply dropping out
- Understood how grouping mentors and mentees by different factors can provide a whole new level of insight into what leads to an effective engagement
- Found that positive and timely mentor responses cause appreciative mentee response
- Completed first steps to measure mentee sentiment
- Found that more words in mentor responses correlates to mentees dropping out of program
Question
Every nine seconds a student drops out of school. iCouldBe’s e-mentoring program has served over 19,000 at-risk youth since 2000, providing middle and high school students with an online community of professional mentors that empowers them to stay in school, plan for future careers and achieve in life.
Internally, iCouldBe had been working on their organizational goals and metrics to improve their program, but had taken it as far as they could go. With extensive data from 15 years of mentor/mentee interactions on their online platform that allows mentees to complete learning tasks or “quests” and interact with mentors virtually, they wanted to understand what makes a mentoring engagement successful so they could improve their curriculum, training and better serve students.
What Happened
At the Teradata PARTNERS DataDive, held in October 2014, Data Ambassadors Elisia Getts and Mahdi Moqri led their volunteer team to first define what a “successful” mentoring engagement looks like, then uncover what factors are indicators of success or failure.
By the end of the two-day DataDive, the team defined a “successful” mentee/mentor engagement as one where a mentee completes at least 3 “quests” or learning modules in 3 months. For many nonprofits, defining metrics of success can be a challenge so just confirming this was a powerful first step.
After confirming these definitions of success, they worked to identify the characteristics of engagements and interactions between mentees and mentors that were the most determinative of success or failure. The team examined the length of their posts to one another, the timeliness of their responses and whether the overall sentiment of the conversation was “positive” or “negative.” They then ran a logistic regression model to determine if any of these factors were indicators of success or failure.
They found positive and timely mentor responses cause appreciative mentee response and that, for some reason, more words in mentor responses was correlated to mentees dropping out of program. While these are interesting initial findings, iCouldBe will need to dig in deeper to understand why this is the case.
One team member looking at mentee/mentor conversations on successful engagements noticed a pattern that mentors expressed support to their mentee in their messages by saying “I’m here for you” or other words of encouragement.
“When I heard that my hair stood up on end,” said executive director Kate Schrauth. “Now we can start really training mentors on how to be successful based on more than just anecdotes.”
Ultimately, the team built a predictive model that iCouldBe can use to identify key predictors of successful and unsuccessful mentee/mentor engagements and built a framework for a text analysis model to analyze mentee/mentor communications to discover how the online mentor relationships develop and ultimately to benefit at-risk youth.
This visual showing number of posts per activity/quest and participation by mentees and mentors will allow iCouldBe to begin a full analysis of the curriculum.
What’s Next
The team’s findings will serve as a springboard for iCouldBe to find other indicators of success and do further analysis into specific words and phrases and the affect they can have on mentor/mentee conversations.
The findings will also help iCouldBe improve its program and curriculum as well as advise the teachers and schools they work with to better use data to achieve their outcomes.
“This is the best experience I have ever had working with a group of expert volunteers,” said executive director Kate Schrauth. “Our team’s clarity of purpose and diversity of expertise is exactly what we needed to uncover the hidden gems in our data to better serve our students.”
A huge thanks to our sponsor, Teradata Cares, for making this DataDive possible and for their generous support of DataKind.
