txt 4 success! - cfwv connect · 2017-08-31 · text messaging during the summer statistically...

Txt 4 Success! Utilizing text messaging to shift students’ college-going behaviors Text in: (202) 751-2123

Road Map

•  Technology •  Messaging •  Implementation

•  Introductions •  Research Review •  Case Studies •  Needs assessment

Introductions Let’s Get to Know Each Other! •  Name •  Department/Organization & Role/Title •  Currently using texting outreach? •  One thing you hope to learn

Research review •  Pioneering studies and trials •  Principles of behavioral economics •  Other policy interventions •  Key publications and resources

Pioneering Studies & Trials

Session informed by lead researchers in this field - Dr. Ben Castleman and Dr. Lindsay Page

●  Use principles from behavioral economics & psychology to improve college outcomes

●  Focus their work on supporting low-income

& non-traditional students

●  Powerful, exciting results have garnered national attention & widespread application

Pioneering Studies & Trials

2010: uAspire began partnership with Castleman & Page to study summer melt 20% drop-off

2011: First uAspire summer melt intervention - successful but difficult to reach students

Summer Melt: the phenomenon whereby college-intending students do not make the transition from high school to college enrollment

Summer Melt Trial Results ★ Summer obstacles threaten more students than predicted,

with more serious consequences than imagined

★ Student lack of awareness can create false sense of security; “don’t know what they don’t know”

★ Obstacles can be overcome with effective advisor support, but students don’t often answer phone or check email

Pioneering Studies & Trials

Pioneering Studies & Trials

First Texting Studies (2012, 2013): Applied key ingredients of summer melt intervention to texting

Personalized outreach around

discrete, time-bound tasks

Real-time 1:1 advisor support

Student-centered mode of

outreach: TEXTING

Used  randomized  design  in  each  tex2ng  project:    

One group randomly assigned to receive texting

One group randomly assigned to business as usual

Pioneering Studies & Trials

Pioneering Studies & Trials

Text messaging during the summer statistically increased on-time college enrollment by 7 percentage points. Among first-generation students, we saw an 8 percentage point increased enrollment in 4-year colleges.

Summer Melt Text Messaging Results

Reading list

•  Castleman and Page: Summer Nudging: Can Personalized Text Messages and Peer Mentor Outreach Increase College Going Among Low-Income High School Graduates (2013)

•  Castleman: Prompts, Personalization, and Pay-offs: Strategies to Improve the Design and Delivery of College and Financial Aid Information (2013)

•  Ross et al: Using Behavioral Economics for Postsecondary Success (2013)

•  Pew Research: Internet Project (see the myriad of reports related to mobile internet access and use of text messaging among various populations)

•  Castleman and Page: Summer Melt: Supporting Low-Income Students Through the Transition to College (2014)

•  Castleman and Page: Freshman Year Financial Aid Nudges (2014)

Research insights

Source: Castleman (2013)

In planning for college, students and families face…

Complex information

Complicated processes

Research insights Key behavioral principles:

•  Limited attention

•  Planning fallacy

•  Social norms

•  Identity

•  Status quo bias

Key behavioral principles

Sources: Castleman (2013); Ideas42 (2014)

Limited attention and planning fallacy Adolescents are particularly prone to put off hard choices/complex tasks in favor of pleasurable pursuits — and their attention often is divided.

Key behavioral principles

But adults also procrastinate when faced with complexity. Sources: Castleman (2013); Ideas42 (2014)

Key behavioral principles

Sources: Ideas42 (2014); Iyengar and Lepper (2000); Sivers (2009)

Status Quo Bias •  When overwhelmed, people

tend to do nothing.

•  “Preset” or default options can create monumental shifts in behavior.

•  “Choice is demotivating.”

Key behavioral principles

Sources: Ideas42 (2014); Castleman (2013)

Identity and Social Norms •  Individuals are highly influenced by their perceptions of

what’s normal for their peers.

•  Individuals are motivated to meet clear expectations that they deem to be standard or usual.

•  The way we view and reflect on our personal identities or roles has a powerful influence over our behaviors.

Key behavioral principles

Sources: Ideas42 (2014); Castleman (2013)

Coping with complexity •  Simplifying strategies

•  Following the crowd

•  Paralysis

Prior policy interventions •  Simplifying information increases participation in retirement

programs and the quality of school and college choices (Beshears et al, 2012; Hastings & Weinstein, 2008; Hoxby & Turner, 2013)

•  Reducing hassles increases completion of federal financial aid applications (Bettinger et al, 2012)

•  Shifting perceptions of social norms increases tax compliance and reduces home energy use (Alcott, 2011; Coleman, 1996)

•  Prompts increase financial savings and flu vaccination rates (Karlan et al, 2010; Stockwell et al, 2012)

Prior policy interventions •  Simplifying information increases participation in retirement

programs and the quality of school and college choices (Beshears et al, 2012; Hastings & Weinstein, 2008; Hoxby & Turner, 2013)

•  Reducing hassles increases completion of federal financial aid applications (Bettinger et al, 2012)

•  Shifting perceptions of social norms increases tax compliance and reduces home energy use (Alcott, 2011; Coleman, 1996)

•  Prompts increase financial savings and flu vaccination rates (Karlan et al, 2010; Stockwell et al, 2012)

Other interventions •  Weight Loss & Fitness

o  Reify Health used text messaging in clinical trials o  Focused on weight-loss & fitness related interactions

•  Ebola Outbreak

o  The WFP is using texting to survey food supplies in Sierra Leone (The Guardian, October 14, 2014)

•  India Rural Farming o  Texting is used to communicate to rural villagers about subsidized

farming products

•  Literacy in Niger o  World Bank Cell-Ed intervention increased basic literacy by 5 years

Shifting perceptions of social norms increases tax compliance and reduces home energy use (World Bank, ‘Teach Literacy by Text. Really.”, 7/7/2014

Questions? Comments?

Context •  Organizations using texting •  Audiences targeted through texting

Where is this happening? •  State Education Departments

– Louisiana, Arizona, Delaware, Montana, Massachusetts, Wisconsin, Virginia, and many others in process

•  US Department of Education – Two funded research projects including an IES research grant and an Outward Bound

research study

– Inclusion in GEAR UP grant applications

• Major Colleges & Universities – Research: Stanford, Harvard, Penn, U of Chicago, Michigan State, and Others – Admissions & Student Affairs: U of Washington - Bothell, Youngstown State University, etc.

• College Access Orgs, School Districts, & Charters – Philadelphia USD, KIPP Philly, Mastery, and many CAO’s

How is texting being used in Education? •  College Going Process and Summer Melt

– College Access Organizations, K-12 districts – To Increase Matriculation and Persistence Rates

•  Admissions & Financial Aid & Admissions – Related to summer melt, improve engagement with students through their admittance – Provide financial guidance up-front and renewal/compliance information throughout

– For Financial Aid, reduce loan default rates (CC in particular)

• College Persistence & Student Affairs – Communicate with college freshman (Louisiana) to maintain good standing and persist – Expanding to more general dialogue with students at the college/university level

•  K-12 Attendance - (University of Chicago Research)

•  Values Affirmation (Middle School) - (Stanford Research)

•  Early Childhood Reading - (Stanford Research)

Case studies •  uAspire •  College Foundation of West Virginia •  Other Signal Vine projects

uAspire Case study:

uAspire: College Affordability Non-Profit

uAspire Texting: Summer Melt

Text messaging to support the high school to college transition 2012: Piloted text messaging for summer melt using online platform

•  Students received 10 customized text reminders July-August

•  Could text “HELP” but nothing else •  Advisor required to call student to respond

uAspire Texting: Summer Melt

Identified need for a two-way, student-centered texting platform.

2013-2015: New platform allows students to interact “live” via text message

•  Customized reminders sent out •  Students texted back for support •  Advisors responded immediately

uAspire Texting: Summer Melt

Customized content based on student data:

Student City Data

Intended College Data Student City Data

•  Goals:  –  Re-­‐establish  student-­‐advisor  contact                a1er  high  school  gradua5on    –  Nudge  students  on  specific  financial    aid  and  college  planning  tasks    

–  Customize  advising  using  data              from  student  records  and  surveys    

uAspire Texting: Summer Melt

uAspire Texting in Practice

Do students engage? Yes!

Real responses: •  I do need help. How do I file a loan?

•  Yes, thank you! I have a lot of questions & the internet is giving me little answers.

•  I didn’t get accepted to any of the schools I applied to, do I need to do FAFSA again for community college?

•  Actually it’s great that you texted me bc I really do need some help concerning my financial aid.

•  OMG this came at just the right time. I need to electronically sign my loan but I’m not sure how to do it.

But  can  advising  happen  via  text?  Yes!  

uAspire Texting in Practice

Advising  via  text  

uAspire Texting in Practice

Advising  via  text  

uAspire Texting in Practice

What does this all mean for student outcomes?

★ Through all texting interventions, we see statistically significant impact on enrollment and persistence, from 5-12 percentage points

★ With better data in the last two years, we’ve been able to compare impact across different student groups. We find the benefits of texting are greatest among students with:

★ $0 Expected Family Contribution (FAFSA) ★ Mid-range GPA (High school record) ★ Undefined college plans (12th grade survey) ★ First-generation college status (12th grade survey)

uAspire Evaluation & Results

Summer  melt  preven2on  tex2ng  (4th  year)  •  5,000  students  served    •  Focus  on  financial  aid  &  college  enrollment  tasks  •  65%  response  rate    

Postsecondary  tex2ng  (3rd  year)  •  3,000  students  served  •  Focus  on  financial  aid  renewal  &  connec5ng  

students  to  on-­‐campus  advisors  •  40%  response  rate    

11th-­‐12th  grade  tex2ng  (4th  month)    •  29,000  students  served  •  Focus  on  college  planning  &  financial  aid    •  47%  response  rate  

uAspire Texting Expanded

Sent reminders to renew financial aid and check SAP Advisors texted back real-time responses to student questions Texting increased persistence by 12 percentage points

College Texting Studies: Restarted texting with summer program students in December of freshman year

uAspire Texting Expanded

uAspire Texting Expanded

Freshman Year Text Messaging Results

●  Text messaging into the freshman year of college doubled the rate at which community college students sought help from uAspire with their FAFSA renewal

●  What’s more, freshman year texting increased sophomore persistence by 12 percentage points among community college students

Building your audience Opt-out •  Utilizes an existing

database to start sending messages.

•  Students can choose to stop messaging after an introductory message is sent/received.

Opt-in •  Students sign up to

start receiving messages.

•  Sign-ups might include written, digital or verbal agreements.

vs

Behavioral Economics: Default Austria (Opt-out)

99% Source: Complete College America (2012). Guided pathways to success: Summary.

Germany (Opt-in)

12%

Organ Donation

Rates

Opt-out model When students have already agreed to be contacted

Potential data sources: •  School administrative data •  Student surveys (online or paper) •  Scholarship applications •  College applications •  College registration paperwork

College Foundation of West Virginia

Case study:

$225,000 grant from the Kresge Foundation Three-year pilot project serving 14 high schools in Year 1 (GEAR UP)

West Virginia’s project

Primary functions Nudge Urge students to complete college-related tasks

Counsel Provide access to one-on-one counseling with a professional

Align Provide a continuous contact point from high school into college

Project goals Financial aid awareness •  Increase the number of students

applying for and utilizing financial aid

Project goals Campus connections •  Increase students’ use of campus

resources

•  Position campus and state-level staff as a continuous source of support

•  Address “summer melt” by guiding students through transitional process

Project goals Capacity building •  Design a low-cost, high-impact service

model allowing colleges and universities to improve access and retention

•  Expand on the body of research surrounding next-generation student supports and engagement

How it works

How it works Students opt in •  College admissions applications

•  Statewide merit-based scholarship application

•  State-level college access web portal

•  Inquiry forms collected at community events and college fairs

•  Direct sign-up

Opt-in model Potential data collection points: •  Applications

–  College admissions –  Scholarship programs

•  Events –  During the event –  During pre-registration

•  Online –  Websites –  Social media pages –  Email blasts

•  Text to join systems

Sign-up field

<<< Source: oregongoestocollege.org

How it works Students receive scheduled messages according to a pre-planned campaign.

Students can text our number at any time for help.

A college counselor (a real person) responds!

How it works State-level campaign: •  Messages are relevant to all

college-intending students. •  All students receive messages.

Example: Did u know u can register 4 fall classes now? Call 304-792-7098 & ask 4 a campus counselor 2 make an appointment.

Example: When reviewing your college schedule... Remember '15 to finish.' On average, you have to take 15 credit hours each semester to finish college on time.

Campus campaigns: •  Messages provide information

specific to a particular college. •  Some students receive messages.

Eight college partners: •  Bluefield State College (suburban public four-year, HBCU)

•  Concord University (rural public four-year)

•  Fairmont State University (suburban public four-year)

•  Marshall University (urban public four-year)

•  New for 2016: Shepherd University (rural public four-year located near the DC and Baltimore metro areas)

•  Southern West Virginia Community and Technical College (rural public two-year)

•  New for 2016: West Virginia Northern Community College (urban public two-year)

•  West Virginia State University (urban public four-year, HBCU)

How it works

How it works Campus-level messaging At various points in the campaign, students receive texts asking if they’ve made a college selection…

•  Students who select a partner college are added to the campaign for that particular college.

•  Students who select a non-partner college continue receiving only state-level messaging.

•  Students who do not respond or text back “undecided” continue receiving state-level messaging and are asked again about their college choice later in the campaign.

HEPC commitments •  Provide text-messaging platform

•  Provide technical support, training and guidance

•  Coordinate statewide messaging

Campus commitments •  Collect and share student data with

HEPC (name, cell number, email, high school, and blind SSN)

•  Align student supports to provide a single point of contact

•  Commit to following the statewide schedule (with options of 2 custom messages/month)

Campus commitments •  Provide staff to check messages

daily and within one hour of any outgoing message

•  Assist in the evaluation by providing feedback and limited data

Campus staffing Campus-level teams/support Consider engaging…

•  Admissions/recruitment officers •  Enrollment personnel •  Student affairs officers •  Student success/retention officers •  Academic/advising personnel •  Financial aid representatives •  Students (but only as mentors, not content experts!) •  Public information officers (keep them in the loop!)

Early outcomes

923 opt-ins •  Approximately 40% of eligible students

signed up

•  10% opted out

•  61% engagement

•  2,200 student responses

•  351 have indicated college choice

Class of 2014… completed 18 month intervention

Early outcomes

1899 opt-ins •  Approximately 46% of eligible students

signed up

•  8% opted out

•  48% engagement

•  3,013 student responses

•  397 indicated college choice

Class of 2015… 6 months in...

Early outcomes

5,200+/- student responses •  Questions about processes

Ex: When is the deadline to apply for WV financial aid?

•  Requests for guidance Ex: I can’t decide on a college... Help!

Since January 2014…

Early outcomes

Early outcomes

Early outcomes

Early outcomes

•  Data pollution due to student error (e.g. students reporting the wrong grade level or high school on sign up)

•  Cell phone turnover (e.g. students change numbers or pass the phone off to younger siblings)

•  Lack of alignment with career and technical pathways

•  Fewer than expected responses regarding college choice

•  Declining engagement

Early challenges

Cell phone data can be used to tailor Twitter and Facebook outreach ●  Upload your cell phone list

to Twitter and/or Facebook

●  Automatically identify individuals on your list for targeting on these platforms

●  This works for email addresses too!

Bonus feature

Refined audience segmenting

Detailed in-platform analytics

“How to” toolkit

Counseling guide and service training

Evaluative study and white paper

New pathways/ intervention models

In the works

Evaluative study Comparison groups:

•  GEAR UP students who did not receive messages

•  GEAR UP students who received messages (segmented by length of participation… e.g. four months, 12 months)

•  Non-GU students who did not receive messages

•  Non-GU students who received messages

Evaluative study Variables:

•  Demographic (gender, race/ethnicity, Pell eligibility, zip code, high school)

•  Academic (ACT scores, GPA, highest high school math course completed)

•  Process (FAFSA completion, ACT/SAT score submission, payment of deposits)

•  Outcomes (Fall and spring enrollment, course completion, course grades, degree objective, persistence)

Questions? Comments?

Let’s recharge! We’ll start back in five minutes...

Needs assessment •  Problem definition •  Goals and outcomes •  Texting intervention applications

Needs assessment •  What are the concrete challenges your

organization could potentially target with a text messaging program?

Let’s get to work! Work through sections one, two and three of your worksheet.

Questions? Comments?

Technology

Send  Reply  Message   2  

Live Demo A tour of the Signal Vine platform

Questions? Comments?

Content and messaging •  Anatomy of a text message •  Auto-responses, message pathways and

audience segmentation •  Style and tone •  Sample content

Best Practices Credibility – Students need to recognize and trust texting source

–  Intro text is critical to establishing legitimacy

–  Choose name carefully & explain program’s relevance

–  Personalize as much as possible

I’ll be available by text at this # all summer to help with financial aid tasks for college. I’ll also send 1-2

texts/week w/ info. Text back anytime for help.

Awesome, thanks! Nice to meet you Jacqui and okay that sounds good to

me.

Hi Nadia, this is Jacqui from uAspire. You worked with my coworker Mr. Williams to do your financial aid at

Washington this year.

Best Practices

Engagement – Create opportunities that prompt student response

–  Use neutral responses (“Ok, thanks!”) as an opportunity to engage

–  Probing questions reveal critical issues sitting under the surface

–  Demonstrates real person is on the other end

Alright, thanks Arman!

No problem! I know we reviewed your SJSU financial aid award last month.

Do you have any questions at this time?

Actually, I do. On my SJSU page it says I owe about $6000 for housing! So is my financial aid going to cover

it??

Hi, it’s Arman from uAspire. We worked on financial aid together at

Central HS. I’ll be available by text all summer.

Best Practices

Clarity – Conveying information in 160 characters requires thoughtful content & phrasing –  Pilot texts with students in

advance –  Recognize confusion in

student responses & remedy immediately

–  Elevate to phone conversation if needed

No, it’s supposed to come in the mail?

Hey Dan! Sorry that text was misleading ☺ No, it’s the one posted

in your online student portal.

Ohh, okay. Yeah, I saw financial aid info there but I wasn’t sure how much

I actually owe.

Hi Dan, it’s Elsa from uAspire. Reviewing your financial aid award

letter is critical to see what you might owe. Has UMass sent a final award

letter? Reply Yes or No

Best Practices

Responsiveness – Texting sets expectation for immediate response –  Ensure texting schedule

aligns with staff capacity –  Majority of text responses

arrive within a few minutes –  Use away message to

reset expectations when necessary

Thanks for texting. Our office is closed for the holiday weekend but I’ll get back to you asap on Monday,

July 6. Thanks for being patient!

Hey, I have a question about the health insurance waiver…

Ok, thx

Best Practices

Frequency – Resist urge to send outbound texts too often or goals may be diluted –  Tell students how often you’ll be

texting –  Text to keep momentum, not so

much that students tune out –  Be prepared for candid feedback

Best Practices How often would you prefer to receive college planning text messages?

Source: survey of students who attended West Virginia’s 2014 College Goal Sunday event

Anatomy of a text

Message pathways Message personalization: •  Based on data fields (think MS mail merge!)

– Ex: Student first name – Ex: Student’s advisor name – Ex: Financial aid office phone number at the student’s

chosen college

Hi  {first  name}!  Get  free  help  comple5ng  your  FAFSA  during  College  

Goal  Sunday  on  Feb.  15!  There’s  a  loca5on  near  you  at  {closest  College  

Goal  Sunday  site}.  Register:  h[p://samplelink.com    

Message pathways Audience segmenting: •  By campaign:

–  Ex: FAFSA renewal campaign –  Ex: Junior year college-planning timeline

•  By student variables: –  Ex: Students on academic probation –  Ex: Students who haven’t yet registered for orientation

•  Management tool: By group or caseload: –  Ex: Marshall University students –  Ex: Scott High School students

Action automation •  Types of automation:

–  Generate a message response •  Ex: “thanks” might trigger “you’re welcome!”

–  Trigger a system action •  Ex: Stop messaging •  Ex: Move to a group or add to a campaign •  Ex: Launch message series •  Ex: Change future message schedule

Action automation Example: When WV students say they’ve selected a college, the system…

–  Launches a series of response messages to gain more information from the student.

–  Automatically moves them to the appropriate caseload based on their college choice.

–  Automatically enrolls them in their particular college’s supplementary messaging campaign.

–  Automatically removes any further messages related to college choice from their message schedule.

Style and tone considerations

Personality: •  Do you have a sense of humor? •  Are you informal or formal?

Style and tone considerations

Counseling approach: •  Proactive or reactive? •  How would you respond to this exchange?

CFWV: Don’t forget to register for orientation! Student: I’m not going to college.

Style and tone considerations

Stop protocols: •  Honoring opt-outs •  Handling inappropriate messaging

Style and tone considerations

Brand and communications strategies: •  Style and use of terms

•  Integration of organizational goals and complementary campaigns

•  Integration of partnerships

•  Managing “hot button” issues and conflicts

Questions? Comments?

Let’s get to work! Work through section four of your worksheet.

Measuring Engagement •  In-system analytics •  External analytics

In-System analytics

External analytics Link trackers •  Bitly •  Tiny url •  Google

Meeting tracking •  Program database •  Internal record system •  Google calendar

External analytics Google analytics

External analytics •  Social views

–  YouTube views –  Twitter follows –  Facebook likes

•  Actions taken –  Call center calls –  Deposits or forms submitted –  Event registrations

•  Surveys and polls –  How did you hear about us? –  Where do you get college planning information?

Questions? Comments?

Policies and Procedures •  Legal requirements and responsibilities •  Partnership agreements •  Messaging and counseling guidelines

Legal requirements •  FCC rules prohibit SPAM and other

unwanted communications – Nonprofit rules are more lenient than commercial

regulations

•  Age limits – COPPA restrictions require parental consent for

students under age 13 – School districts and other partners may have

individual policies

Legal requirements •  Student confidentiality and privacy

– FERPA regulations often apply – Data sharing agreements should be in place

among partners and vendors

•  Counseling policies – Responsibilities to report

Legal requirements •  Purchasing

– Sole source justifications vs. bidding processes

Policy considerations •  Formalizing partnerships

– Memoranda of understanding

•  Organizational policies – Communications standards and guidelines – Counseling guidelines – Data sharing policies and agreements

Questions? Comments?

Let’s get to work! Work through section five of your worksheet.

Conclusion •  Key takeaways and next steps •  What else?

Questions? Comments?

Contact Alexandra Chewning Vice President of Research and Evaluation uAspire (617) 778-7195 x125 alexandrac@uaspire.org www.uaspire.org @uAspire

Dr. Adam S. Green Vice Chancellor for Student Affairs WV Higher Education Policy Commission (304) 558-0655 adam.green@wvhepc.edu www.wvhepc.edu @adamscottgreen @wvgearup @cfwv

Brian Kathman CEO Signal Vine, LLC (703) 338-1046 brian@signalvine.com www.signalvine.com @signalvine

Jessica Kennedy Director of Communications and Outreach WV Higher Education Policy Commission (304) 558-0655 jessica.kennedy@wvhepc.edu www.wvhepc.edu @jackennedy @wvgearup @cfwv