final meta analysis 1

Running head: Emerging Predictor of Academic Performance1

Emerging Predictor of Academic Performance: Meta-Analysis of Conscientiousness on GPA

Bruce A. Keller

Lafayette College

EMERGING PREDICTOR OF ACADEMIC PERFORMANCE2

Abstract

Previous research suggests that Continuousness could be a good predictor of academic

performance that would not be subject to many of the same limitations as traditional predictors,

like previous GPA and standardized tests. We Meta-Analyzed indices of Contentiousness on

overall college GPA, and found Contentiousness to be extremely reliable and to have an overall

average correlation of .215. Our variable of existence of an institutional tutoring program was a

strong moderator, which contrary to our expectations, yielded a higher correlation between

Conscientiousness and GPA for institutions without tutoring programs.


Emerging Predictor of Academic Performance: Meta-Analysis of Conscientiousness on GPA

Education is a significant component in our society as greater education benefits

everyone. Greater education results in a greater Total Factor of Productivity (TFP) in

economics, is vital to having a knowledgeable electorate in republics, and is the avenue of

improving and maintains our increasingly computerized farming (Dorfman, 2009). However,

despite the great importance of education, current predictors of college performance are limited,

and could be benefited by further research on the relationship between the personality

characteristic Conscientiousness and college performance.

Grade point average or GPA is the most commonly used indicator of academic

performance. High school GPA is considered a great predictor of college GPA (Betts &

Morell,1999), and it is no surprise that it is used by practically every college in making decisions

regarding admissions and distribution of financial aid resources. It is also a very common metric

used by employers for screening applicants. GPA also holds considerable prestige as it is

associated with many titles, such as valedictorian and Cum Laude. Despite its prevalence and

utility it remains a somewhat flawed measure of academic ability. There are countless

confounds that make it hard to compare one GPA to another.

Differences in teaching quality, curriculum, test form, and objectivity make GPA a

theoretically unreliable criterion. As early as 1912 research demonstrated that grades can be

unreliable. One of these century old studies submitted hundreds of copies of identical papers to

different English teachers at different schools that assigned the same essay, which also used the

same numbers for passing scores. The responses took the form of a normal distribution in which

there did seem to be a point of central tendency in the scores, but at the same time the range of


scores was very high, some teachers would give a paper a low sixty while others gave the same

paper a high ninety (Starch & Elliot, 1912). This same study was replicated and published in

2011, and the result was similar with a range of 46 percentage points (Brimi, 2011). Then and

now, the same paper can receive an almost perfect score, or a failing score, just depending on

who is grading it. This problem with consistency of grading was known of 100 years ago, and

has still to been ameliorated; but GPA is comprised of more than essays so this alone is not

enough to discredit the use of GPA substantially.

Previous research indicates that along with inconsistency of grading, that GPA

substantially varies across college departments and institutions while holding ability constant.

Bigger high schools offer a wide range of courses in different disciplines, and some even have

majors, so we believe that this research is generalizable to high schools as well. The rational that

previous researchers have proposed and supported, is that people of higher ability (those with

higher standardized test scores) tend towards more demanding fields (usually STEM); where the

department normalizes their grades around the students in the department. This causes someone

of higher than average cognitive ability and performance, to receive lower, more average grades

in his/her more demanding department. This has been supported by creating indices that

accurately standardized grades for students taking courses across different departments. For

example, a Computer Science major may be averaging a 2.5 in his/her major, but may take a few

Philosophy classes and get 4.0s in them; and vice versa. This would indicate that a 2.5 in

computer science is the equivalent of 4.0 in Philosophy. These findings suggest that grades are

assigned somewhat on relativistic criteria as opposed to absolute, but this could be ameliorated

by using these same indices before making comparisons. Unfortunately, research has found

cardinal differences in the indices across institutions (Elliott & Strenta, 1998). Essentially we


can make accurate comparisons between GPAs using these indices, but these indices would

require extensive collection of data between all departments in all institutions one would want to

make comparisons between, making it difficult to use. GPA is a somewhat impractical criterion

to use for measuring future academic performance due to unreliability of grading in essays, and

difficulty in comparing grades across intuitions and disciplines.

While GPA has several limitations due to unstandardized criteria for grading and

relativistic scoring, standardized tests solve many of the deficiencies that GPA has. For instance,

as previously mentioned although there was a large range of scores for the same essays, the

grades did seem to be normally distributed. Most standardized tests such as the SAT (Scholastic

Aptitude Test) and College Board’s AP (Advanced Placement) tests have multiple readers grade

essays. This should result in a more reliable measure of academic ability. Standardized tests are

also highly predictive of future academic performance (Noftle & Robins, 2007). The SAT for

instance is reliable in grading for the most part given that the questions are the same and

multiple-choice, so that there is no subjectivity in grading or easier questions for some people

over others. In addition, multitudes of statistics for the test are readily accessible, percentile

ranks, for instance, allow for easy comparisons of where an individual stands relative to his/her

peers.

Unfortunately, these standardized tests have their critics as well. One consistent finding

is that SAT scores are positively correlated with family income. We can speculate that a family

with greater income can more easily afford preparatory classes and test retakes (Zwick & Greif,

2007). These are particularly valid criticisms as the SAT allows one to use the highest scores on

the subsections of different testings to report to colleges. This gives a distinct advantage to those

who are more willing and able to repeatedly take the SAT. In addition, despite its ability to


predict college GPA, the SAT has been discarded by many colleges. There exists controversy as

to whether the SAT is racially and sexually biased, which seems to arise, in part, from the afore

mentioned correlation between socioeconomic status and test score (Young & Fisler, 2000).

Many admissions directors think valuing an individual on a single test so highly is wrong, and

the cost to diversity associated with its use outweighs its benefits.

An alternative avenue of predicting academic performance is through the use of

personality tests. Using personality as a predictor of academic ability is not a novel idea.

College essays and interviews are intended to measure personality to some extent. However,

these methods are limited. Interviews, despite their prevalence, are not very good indicators of

job performance, although structured interviews are substantially better (Dipboye, 1994); and

while there is limited research on interviews on academic performance the same trend is likely

true. In addition, the type of information an interviewer may be processing is likely not being

done through the Big 5 personality types, which is the most widely supported personality theory

at the moment. The same would likely be true of the college essay, but be confounded by the

same unreliability from admissions directors that was present from English teachers. This is not

to say that the use of college essays and structured interviews should be discarded as they surely

are somewhat useful, but it would not be a suitable substitute for a personality inventory.

Personality inventories are used in employment screening, and would likely cost less time

and money to perform than the four hour, $60 SAT. Using a personality inventory of the Big

Five has many distinct advantages. One of the most important is that it is not biased towards any

race or sex, like some suspect the SAT to be. The other major advantage is that it explains

different variation in college GPA, than high school GPA and standardized test scores. The

biggest disadvantage is that the effect sizes of these personality traits have been very low in


previous research. Previous research has looked to examine what personality traits are correlated

with academic performance. The only personality trait that has been consistently found to be

significant and have a substantial effect size is Conscientiousness. However, the effect size has

varied widely depending on the study. Moderating variables have been proposed, and some

argue that the effect is only due to one of the facets of Conscientiousness. Despite considerable

research on the topic no conclusion has yet been reached on the population effect size of

contentiousness and/or what moderators are causing these differences (Noftle & Robins, 2007).

The situation in regard to the uncertainty of this trait sounds like the textbook circumstances for

Meta-Analysis.

Education is important and improvements in predicting college performance could help

better the educational system by giving educational opportunities to those most likely to succeed.

GPA although still a useful predictor has limitations from unreliableness in grading and is

difficult to accurately compare between institutions and subjects. Standardized tests while very

predictive of college performance may be biased towards certain groups, reducing diversity in

exchange for greater prediction. More generalized personality assessments through essays and

interviews are somewhat effective but are no substitute for a thorough personality inventory.

Personality inventories measuring Conscientiousness may provide a valid unbiased measure of

academic performance. Our study is the use of Meta-Analysis to determine the correlation of

conscientiousness on overall college GPA, and based on previous research we expect a low

positive coefficient. Some previous research suggests that there is no effect of Contentiousness

when moderating for tutoring (Farsides & Woodfield, 2003), so our analysis will moderate for

tutoring as well.


Method

Literature Review

Studies were found by searching relevant key words in the google scholar search engine.

Then once a substantial amount of studies related to our research question were found, relevant

studies cited in these papers were searched and the process was repeated until we had a large

enough sample.

Criteria for Inclusion

12 studies were selected for the Meta-Analysis under several criteria. Our criterion for

our dependent variable was limited to overall college GPA. As already mentioned previous

research suggests that there is considerable unreliability in teacher’s assignment of grades, but

that it does seem to follow a normal distribution. Therefore, with a high N, characteristic of

Meta-Analyses, GPA should function as a good indicator of college performance. We also

limited ourselves to overall college GPA in case there is a different effect of Conscientiousness

between high school and college GPAs, and to account for differences in transitioning into

college.

We used a variety of personality tests as our independent variable. Due to limits of

previous research we were restricted to self-report measures of conscientiousness. We accepted

studies using different well established personality tests including variations of the NEO and PF

inventories. These well-established personality tests should all validly measure

Conscientiousness and therefore should yield the same correlation, although longer versions

would be more thorough then their shorter counterparts.


Coding

All data collected already used the r correlation coefficient so no conversions were

necessary. Some of our longitudinal studies included individual GPAs for each semester, in

which case we recoded these by averaging them into an overall college GPA. For our tutor

moderating variable we classified a school as having tutoring if it had an institutional tutoring

service that was not part of regular courses (labs) and was one on one.

Meta-Analytic Procedure

The data was analyzed using the Hunter & Schmidt meta-analysis method, which corrects

for sampling error and range restriction.

Results

Table 1. Summary data

The overall weighted mean

correlation was .215. From previous research

we expected a low correlation coefficient so this result matches our predictions. All of our

analysis had very little variance as can be seen from the small range of our confidence intervals,

suggesting that Contentiousness is a very reliable predictor of GPA.

N r bar CI

Overall 361

8

.215 .226 - .204

Tutor 216

8

.177 .190 - .164

No Tutor 145

0

.272 .276 - .268


Tutoring did moderate this relationship considerably. Our tutor group had a much lower

correlation coefficient of .177 than our no tutor group of .272. This did not match our

predictions. Previous research had suggested that more Contentious students would seek out

extra academic resources (Farsides & Woodfield, 2003), and would have the higher correlation.

Discussion

Answers to Research Questions

Our study’s results partly matched our expectations. We expected low average

correlation coefficients since that has been the norm in previous research (Noftle & Robins,

2007). However we expected that more Contentious students would be more likely to use tutors

and would then obtain better grades and therefore have a larger correlation, in line with previous

research (Farsides & Woodfield, 2003). A possible explanation is that if the institution offers

tutoring, then less Contentious students would also be more likely to use it since they would not

have to search for tutors. In investigating what institutions had dedicated tutoring services we

encountered many more ads for 3rd party tutors for the institutions that lacked tutoring programs.

Grades tend to normalize based on student performance (Elliott & Strenta, 1998) and so it would

stand to reason that in one of these institutions without their own tutoring program, a more

contentious individual would be more likely to search for a tutor than his/her less Contentious

classmates and would do relatively better and obtain a higher grade, resulting in a higher

correlation than in institutions where tutoring is readily available.

Our results do indicate that Conscientiousness is a useful predictor of college GPA. Our

overall average correlation coefficient for Contentiousness (.215) is similar to what previous

research has found for the SAT (.22). While high school GPA seems to be a stronger predictor

with a correlation coefficient of .33 (Noftle & Robins, 2007), our correlation for no tutor


institutions had an average coefficient of .272, which is close but was also extremely reliable

having very little unexplained variance (.00357).

Limitations

There are 2 primary limitations to our study. The first is that we used different

personality tests as dependent variables, under the assumption that they have convergent validity.

The second is that in moderating our variables we grouped our studies depending on if the

institution of the participants had a one on one tutoring program. This excludes institutions like

the University of Iowa, that was limited to group tutoring, but that would certainly be better than

no tutoring. In addition, this coding fails to take into account the quality of the tutoring

programs which may be an important factor.

Future Research

Future research can attempt to explain our paper’s unexpected findings, look to improve

our ability to predict academic performance with Contentiousness, and apply Conscientiousness

as a predictor of college performance. As previously mentioned the results of moderating our

analysis with tutoring was the opposite of what we predicted. While we did purpose a possible

explanation future research could test our hypothesis or investigate other explanations for the

unexpected result. Also availability of tutoring is likely not the only moderating variable, for

Conscientiousness on GPA. Future research could explore other possible moderating variables.

Finally a personality inventory measuring Contentiousness could be applied for college selection

and be evaluated to test if it results in colleges selecting better students.


References

(* Indicates studies that were used in the Meta-Analysis)

*Barchard, K. (2003). Does Emotional Intelligence Assist in the Prediction of Academic

Success? Educational and Psychological Measurement, 63(5), 840-858.

doi:10.1177/0013164403251333

Betts, J., & Morell, D. (1999). The Determinants of Undergraduate Grade Point Average: The

Relative Importance of Family Background, High School Resources, and Peer Group

Effects. The Journal of Human Resources, 34(2), 268-293. Retrieved April 19, 2015,

from http://econweb.ucsd.edu/~jbetts/Pub/A21 Betts Morell 1999 JHR.pdf

*Busato, V., Prins, F., Elshout, J., & Hamaker, C. (1999). Intellectual Ability, Learning Style,

Personality, Achievement Motivation and Academic Success of Psychology Students in

Higher Education. Personality and Individual Differences, 29, 1057-1068.

*Chamorro-Premuzic, T., & Furnham, A. (2003). Personality Predicts Academic Performance:

Evidence from Two Longitudinal University Samples. Journal of Research in Personality,

(37), 319-338.

*Conard, M. (2006). Aptitude is not Enough: How Personality and Behavior Predict Academic

Performance. Journal of Research in Personality, 339-346. Retrieved April 14, 2015

Dipboye, R. (1994). Structured and Unstructured Selection Interviews: Beyond the Job-Fit

Model. Research in Personnel and Human Resources Management, 12, 79-123.

Dorfman, J. (2009, December 12). Fields of Automation. Retrieved April 21, 2015, from

http://www.economist.com/node/15048711

http://www.economist.com/node/15048711

http://econweb.ucsd.edu/~jbetts/Pub/A21%20Betts%20Morell%201999%20JHR.pdf


*Duff, A., Boyle, E., Dunleavy, K., & Ferguson, J. (2004). The Relationship Between

Personality, Approach to Learning and Academic Performance. Personality and

Individual Differences, (36), 1907-1920.

Elliott, R., & Strenta, A. (1998). Effects of Improving the Reliability of the GPA on Prediction

Generally and on Comparative Predictions for Gender and Race Particularly. Journal of

Educational Measurement J Educational Measurement, 25(4), 333-347. Retrieved May 9,

2015, from http://www.jstor.org/stable/1434965?seq=1#page_scan_tab_contents

*Farsides, T., & Woodfield, R. (2003). Individual Differences and Undergraduate Academic

Success: The roles of Personality, Intelligence, and Application. Personality and

Individual Differences, 34, 1225-1243.

*Fruyt, F., & Mervielde, I. (1998). Personality and Interests as Predictors of Educational

Streaming and Achievement. Eur. J. Pers. European Journal of Personality, 10(5), 405-

425. doi:10.1002/(SICI)1099-0984(199612)10:53.0.CO;2-M

*Gray, E. K., & Watson, D. (2002). General and Specific Traits of Personality and their Relation

to Sleep and Academic Performance. Journal of Personality, 70, 177–206.

*Langford, P. (2003). A One-Minute Measure of the Big Five? Evaluating and abridging

Shafer's (1999a) Big Five markers. Personality and Individual Differences, 35, 1127-

1140.

Noftle, E., & Robins, R. (2007). Personality Predictors of Academic Outcomes: Big Five

Correlates Of GPA And SAT Scores. Journal of Personality and Social Psychology,

93(1), 116-130. Retrieved April 14, 2015, from

https://www.gwern.net/docs/conscientiousness/2007-noftle.pdf

*Oswald, F., Schmitt, N., Kim, B., Ramsay, L., & Gillespie, M. (2004). Developing a Biodata

https://www.gwern.net/docs/conscientiousness/2007-noftle.pdf

http://www.jstor.org/stable/1434965?seq=1#page_scan_tab_contents


Measure and Situational Judgment Inventory as Predictors of College Student

Performance. Journal of Applied Psychology, 187-207.

*Ridgell, S. D., & Lounsbury, J. W. (2004). Predicting Academic Success: General intelligence,

“Big Five” Personality Traits, and Work Drive. College Student Journal, 38, 607–619.

*Wolfe, R. N., & Johnson, S. D. (1995). Personality as a Predictor of College Performance.

Educational and Psychological Measurement, 55, 177–185.

Young, J., & Fisler, J. (2000). Sex Differences on the SAT: An Analysis of

Demographic and Educational Variables. Research in Higher Education, 41(3), 401-16.

Retrieved April 16, 2015

Zwick, R., & Greif Green, J. (2007). New Perspectives on the Correlation of SAT Scores, High

School Grades, and Socioeconomic Factors. Journal of Educational Measurement, 44(1),

23-45. Retrieved April 17, 2015, from http://www.jstor.org/stable/20461841?

seq=5#page_scan_tab_contents


Appendix

Calculations

Overall

Study Ni ∑Ni ri r bar (ri-r bar)^2 Ni(ri-r bar)^2 ∑[Ni(ri-r bar)^2] (1-r bar^2)^2 ∑Ni Ni*ri ∑(Ni*ri)Barchard (2003) 150 0.327 0.012540905 1.881135684 49.05Busato et al. (2000) 409 0.06 0.024029284 9.827977289 24.54Conard (2006) 186 0.35 0.018221269 3.389156006 65.1de Fruyt & Mervielde (1996) 714 0.26 0.002023756 1.444962077 185.64Duff et al. (2004) 146 0.21 2.51384E-05 0.003670205 30.66Farsides & Woodfield (2003) 432 0.04 0.030629837 13.23208963 17.28Furnham et al. (2003) 93 0.39 0.030620163 2.847675184 36.27Grey & Watson (2002) 300 0.36 0.021020992 6.306297736 108Langford (2003) 203 0.31 0.009022374 1.831542009 62.93Oswald et al. (2004) 644 0.21 2.51384E-05 0.016189122 135.24Ridgell & Lounsbury (2004) 140 0.15 0.004226797 0.591751547 21Wolfe & Johnson (1995) 201 0.21 2.51384E-05 0.005052816 42.21

3618 0.21501382 41.37749931 0.909675415 3618 777.92

TutorStudy Ni ∑Ni ri r bar (ri-r bar)^2 Ni(ri-r bar)^2 ∑(ri-r bar)^2∑[Ni(ri-r bar)^2](1-r bar^2)^2∑Ni Ni*ri ∑(Ni*ri)

Busato et al. (2000) 409 0.06 0.013671089 5.591475355 24.54Conard (2006) 186 0.35 0.029955498 5.571722718 65.1

Farsides & Woodfield (2003) 432 0.04 0.018748026 8.0991473 17.28Furnham et al. (2003) 93 0.39 0.045401624 4.222351027 36.27

Langford (2003) 203 0.31 0.017709373 3.595002723 62.93Oswald et al. (2004) 644 0.21 0.001094059 0.704574233 135.24

Wolfe & Johnson (1995) 201 0.21 0.001094059 0.219905933 42.212168 0.176923 28.00418 0.938376 2168 383.57

Equations VariancesSr^2 = ∑[Ni(ri-r bar)^2] / ∑Ni 0.012917δe^2 = (1-r bar^2)^2 / N 0.000259δp^2 = Sr^2 - δe^2 0.012658



No TutorStudy Ni ∑Ni ri r bar (ri-r bar)^2Ni(ri-r bar)^2∑(ri-r bar)^2∑[Ni(ri-r bar)^2](1-r bar^2)^2∑Ni Ni*ri ∑(Ni*ri)Barchard (2003) 150 0.327 0.003029 0.454319 49.05

de Fruyt & Mervielde (1996) 714 0.26 0.000143 0.102226 185.64Duff et al. (2004) 146 0.21 0.00384 0.5606 30.66

Grey & Watson (2002) 300 0.36 0.00775 2.325021 108

Ridgell & Lounsbury (2004) 140 0.15 0.014876 2.082582 21

1450 0.271966 5.524748 0.85754 1450 394.35


final meta analysis 1

Documents