www.nacacnet.org/USNews

An Update from the NACAC/U.S. News and World Report Ad Hoc

Committee

COLLEGE RANKINGS

www.nacacnet.org/USNews

COLLEGE RANKINGS •  Committee Formation Timeline

•  NACAC Dialogue with Other Entities

•  Committee Challenges

•  Committee Meetings

•  US News and World Report Representatives

www.nacacnet.org/USNews

COMMITTEE MEMBERS Pete Caruso Lee Melvin Boston College University of Connecticut Mike Sexton Bruce Chamberlain Santa Clara University Georgetown University Irene Loan Pam Horne Virginia State University Purdue University Joe Prieto Ralph Figueroa Hinsdale Central HS Albuquerque Academy

www.nacacnet.org/USNews

COLLEGE RANKINGS

•  1983: US News begins its annual rankings of American Colleges and Universities.

•  1987: First edition of newsstand book-”America’s Best

Colleges.” •  1995: The debut of US News Online

www.nacacnet.org/USNews

CURRENT METHODOLOGY

•  Academic Reputation 22.5% •  Faculty Resources 20% •  Graduation and Retention 20% •  Student Selectivity 15% •  Financial Resources 10% •  Graduation Rate Performance 7.5% •  Alumni Giving 5%

www.nacacnet.org/USNews

COLLEGE RANKINGS

•  Establish working dialogue •  Develop and implement survey to current members •  Report survey findings to members and US News •  Establish final recommendations to NACAC Executive

Board and membership

Committee Goals

www.nacacnet.org/USNews

•  Administered to NACAC membership in May of 2010

•  Members notified of survey options by various mediums

•  Members were given a three week window to complete survey

Survey

COLLEGE RANKINGS

www.nacacnet.org/USNews

•  A total of 9,660 members were sent the survey

•  Survey results presented to members in three stages from October 2010-January 2011

•  Media Coverage

COLLEGE RANKINGS Survey

www.nacacnet.org/USNews

COLLEGE RANKINGS

2,649 respondents College 1176 High School 1157 Independent Counselor 135 Organization 84 Other 97

Survey Respondents

www.nacacnet.org/USNews

COLLEGE RANKINGS •  Survey posed fifteen statements

•  Reaction scale

•  Members were also given an opportunity to provide additional narrative comments

•  Questions focused on use, perception, methodology, influence, marketing and admission practices

www.nacacnet.org/USNews

COLLEGE RANKINGS

0 20 40 60 80 100

Average Perceptions of US News Rankings on "Feeling Thermometer"

College

High School

Total

www.nacacnet.org/USNews

COLLEGE RANKINGS

0.0% 5.0%

10.0% 15.0% 20.0% 25.0% 30.0% 35.0% 40.0% 45.0% 50.0%

Agree Somewhat agree

Somewhat disagree

Disagree

Rankings Are a Helpful Resource for Students and Families Interested in College

Information

College

High School

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Agree Somewhat agree Somewhat disagree Disagree

Rankings Create Confusion for Students and Families

College

High School

Total

www.nacacnet.org/USNews

COLLEGE RANKINGS

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

Agree Somewhat agree

Somewhat disagree

Disagree

Do USNWR Rankings cause counter-productive behavior at colleges and universities?

College

High School

Total

www.nacacnet.org/USNews

COLLEGE RANKINGS

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

Yes, consistently Yes, occasionally No

Puts Pressure on Institutions to Invest in Strategies to Maintain/Improve Ranking?

College

High School

Total

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Yes, consistently Yes, occasionally No

School or Institution Makes Programmatic Changes Because of Rankings?

College Results Only

www.nacacnet.org/USNews

GENERAL SURVEY REFLECTIONS

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0%

35.0%

40.0%

45.0%

Good indicator Fair indicator Poor indicator Not an indicator

Figure 1. Peer Assessments

College

High School

Total

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

Good indicator Fair indicator Poor indicator Not an indicator

Figure 2. Graduation and Retention Rates

College

High School

Total

www.nacacnet.org/USNews

GENERAL SURVEY REFLECTIONS

•  Strong Opinions in regards to Academic Reputation portion

•  Encouragement to focus on more value added methodology: graduation and retention

•  More focus on student engagement and satisfaction

•  Constructive suggestions to remove the “rankings variable” and let families interpret data for themselves

www.nacacnet.org/USNews

•  Creation of a student “Fit Index”

•  Integration of NSSE data into the rankings

•  Unhealthy reliance by international students on rankings

GENERAL SURVEY REFLECTIONS

www.nacacnet.org/USNews

GENERAL SURVEY REFLECTIONS •  Best for whom?

•  Increase in prominence over past five years

•  What’s in a rank?

•  Conflicting messages about utility and usefulness for families

•  Positive comments on editorial and featured content

www.nacacnet.org/USNews

COLLEGE RANKINGS

•  Presentation of Results in Spring of 2011

•  Presentation of Final Committee Report in New Orleans in September of 2011

•  Report will include recommendations in use, methodology changes and a model for a standing committee on College Rankings

www.nacacnet.org/USNews

COLLEGE RANKINGS

•  For a review of the Survey findings, please visit this link:

www.nacacnet.org/USNews

www.nacacnet.org/USNews

DISCUSSION POINTS •  What have been the intended and unintended

consequences of College Rankings?

•  How has methodology influenced marketing strategies of colleges and universities?

•  How do colleges and universities market results of US News and other rankings?

•  How do we counsel families on the use of rankings in our process?

B215: The Findings of NACAC's U.S. News & World Report Advisory Committee National Issues and External Influences in Admission

Robert Morse, U.S. News and World Report, DC Members of NACAC's U.S. News and World Report Advisory Committee

Members of NACAC's U.S. News & World Report Advisory Committee

•  Peter Caruso, Chair, Boston College, MA •  Bruce Chamberlain, Georgetown University, DC •  Rafael S. Figueroa, Albuquerque Academy, NM •  Pam Horne, Purdue University, IN •  Irene Logan, Virginia State University •  Lee Melvin, University of Connecticut •  Joseph Prieto, Hinsdale Central High School, IL •  Kris Getting Roach, University of St. Thomas, MN •  Michael Sexton, Santa Clara University, CA

Survey Respondents

2,649 respondents College 1176 High School 1157 Independent Counselor 135 Organization 84 Other 97

Overarching Themes

•  Methodology Matters •  Distorting Effects and Professional Practice •  Taking the Next Step

Committee’s Recommendations For NACAC

•  Develop professional education resources for members about rankings. •  Work with education publishers and data outlets to encourage development

of do-it-yourself lists for consumers.

Committee’s Recommendations For Rankings Publications

•  Remove the “class rank” and “standardized testing” metrics from rankings methodologies in favor of factors that measure student satisfaction and engagement.

•  Reduce the weight of the reputational survey. •  Encourage emphasis on fit through customized rankings. •  Continue to evolve rankings methodologies through the association’s

communication channels.

For More Information

•  http://www.nacacnet.org/AboutNACAC/Governance/Comm/Pages/NACACUSNewsAdHocCommittee.aspx