personality at three levels of analysis

Open Science Open statistics, open materials, open methodology, open data Part I: Open Statistics: R

Personality at three levels of analysis

William Revelle

Department of Psychology Northwestern UniversityEvanston, Illinois USA

Keynote Address to the 2nd World Conference on PersonalityBuzios, Brazil

April, 2016Slides at http://personality-project.org/sapa.html

Partially supported by a grant from the National Science Foundation: SMA-1419324

1 / 101

http://personality-project.org/sapa.html


Outline

Two alternative titles:1. Personality at three levels of analysis.

1.1 Within individuals1.2 Between individuals1.3 Between groups of individuals

2. Personality research: an open and shared science

The importance of open science for personality research can beshown at each of these three levels of analysis.

1. Open source statistics, e.g.,The R project

2. Open source materials, e.g., IPIP and ICAR

3. Open source methodology, e.g., Synthetic AperturePersonality Assessment (SAPA)

4. Open source data, e.g., Journal of Open Psychology Dataand DataVerse

2 / 101


Open Science

• Science is an international collaborative endeavor thatbenefits when more people from more countries participate.

• Scientific societies were started (e.g, the Royal Society inLondon in 1660) as an “invisible college” to facilitatecommunication and the sharing of ideas.

• Traditionally we collaborate by publishing our results inscientific journals and by sharing our ideas at national andinternational conferences or giving guest lectures to ourcolleagues.

• More recently, there is a trend towards sharing our materials,our methods, and our results, even our data, on the web.

• This makes for better science.

3 / 101


Open Science and the problem of replication

• The last several years has seen a plethora of papers reportingfailures to replicate results. This has lead some to worry aboutthe strength of our findings and others to question what doesit mean to “replicate” or reproduce a result.

• Others have suggested that we should be more open in ourdesigns, publishing what we plan to do independent of whatwe actually find.

• This is an important problem that should not be ignored,although pre-registering might inhibit exploratory research.

• But, open science is much more than protecting us from type Ierrors. It is a philosophy of collaboration. That is what I wantto emphasize today.

4 / 101


A Short History of Science: Instrumentation and Modeling

New methods of data collection led to new models and theories.Instrumentation

1. Telescopes (Galileo)

2. Sailing ships (e.g., Beagle)

3. Depth sounders

4. CO2 measurement (Keeling)

5. The internet (Al Gore?)

6. WWW (Tim Berners-Lee)allows for remote assesment

7. Cellphones (e.g., SteveJobs) allow for repeatedmobile assessment

Theories and Models

1. Newton (Principa )

2. Darwin/Wallace

3. Plate Tectonics

4. Climate change

5. Open Science

6. Modeling the data acrossvery large sample

7. Modeling within subjectvariation over time (processmodels)

5 / 101


A Short History of Science: Part 2: Mathematics and Statistics

1. Calculus (Newton/Leibniz)2. Data visualization (Playfair to Tukey to Cleveland to Tufte)3. Probability theory (Fermat/Pascal) and the normal curve

(Gauss/Quetelet)4. Correlation (Galton/Pearson)5. Factor analysis (Spearman/Thurstone) and Principal

Components analysis (Pearson/Hotelling)6. Discrete (experimental) conditions and the t and F (seeds x

manure) distributions (Gossett and Fisher)7. Main frame computation (Ada Lovelace, John von Neumann,

Grace Hopper)8. Randomization and resampling (empirical distributions, not

idealized)9. Open statistical software for all of us, e.g., R.

We have new methods for collecting, analyzing and sharing ourdata, lets use them to improve our science.

6 / 101


Four types of openness:

1. Open source software: The R project (R Core Team, 2015)

2. Open source materials:• The International Personality Item Pool (IPIP) (Goldberg, 1999)

• The International Cognitive Ability Resource (ICAR) (Condon &

Revelle, 2014)

3. Open source methodology: The Synthetic AperturePersonality Assessment Project (Revelle, Wilt & Rosenthal, 2010; Revelle, Condon,

Wilt, French, Brown & Elleman, 2016)

4. Open source data:• Data from the ICAR project (Condon & Revelle, 2016, 2015a)

• Data from SAPA studies (Condon & Revelle, 2015c,b)

In the process of summarizing the last several years of mystudents and my research, I will show how we use open sourcesoftware, items, and methods and then share them with the world.

7 / 101






Revelle, 2014)

3. Open source methodology: The Synthetic AperturePersonality Assessment Project (Revelle et al., 2010, 2016)

4. Open source data:

• Data from the ICAR project (Condon & Revelle, 2016, 2015a)


In the process of summarizing the last several years of research, Iwill show how we use open source software, items, and methodsand then share them with the world.

8 / 101

Part I: Open Statistics: R R What is R Use R for replications and extensions Getting and using R Part II: Open Materials

Part I Open Statistics: R

Part I: Open Statistics: R

R: open source statistical system

What is R

Use R for replications and extensions

Getting and using RUseful packages

Part II: Open Materials

9 / 101


10 / 101


R: What is it?

1. R: An international collaboration for applied statisticalresearch

• Originally developed in New Zealand in 1991-93• Comprehensive R Archive (CRAN) run out of Vienna• Core R members in Austria (2), Canada, Denmark, France,

Germany (2), India, New Zealand (3), Switzerland, US (6), UK

2. R: The open source - public domain version of S+

3. R: Written by statisticians (and some of us) for statisticians(and the rest of us)

4. R: Not just a statistics system, also an extensible language.• This means that as new statistics are developed they tend to

appear in R far sooner than elsewhere.• R facilitates asking questions that have not already been

asked.• “Easy” to learn, harder to master.

11 / 101


Statistical Programs for Psychologists

• General purpose programs• R• S+• SAS• SPSS• STATA• Systat

• Specialized programs• Mx• EQS• AMOS• LISREL• MPlus• Your favorite program

12 / 101


Statistical Programs for Psychologists

• General purpose programs• R• $+• $A$• $P$$• $TATA• $y$tat

• Specialized programs• Mx (OpenMx is part of R)• EQ$• AMO$• LI$REL• MPlu$• Your favorite program

13 / 101


R: A brief history

• 1991-93: Ross Dhaka and Robert Gentleman begin work onR project for Macs at U. Auckland (S for Macs).

• 1995: R available by ftp under the General Public License.• 2001-2016: Core team continues to improve base package

with a new release every 6 months (now more like yearly).• Many others contribute “packages” to supplement the

functionality for particular problems.• 2003-04-01: 250 packages• 2004-10-01: 500 packages• 2007-04-12: 1,000 packages• 2009-10-04: 2,000 packages• 2011-05-12: 3,000 packages• 2012-08-27: 4,000 packages• 2014-05-16: 5,547 packages (on CRAN) + 824 bioinformatic packages on

BioConductor• 2015-05-20 6,678 packages (on CRAN) + 1,024 bioinformatic packages +

?,000s on GitHub/R-Forge• 2016-03-31 8,182 packages (on CRAN) + 1,104 bioinformatic packages +

?,000s on GitHub/R-Forge (increased by 60 in last 10 days)

14 / 101


Rapid and consistent growth in packages contributed to R

15 / 101


R as a way of facilitating replicable science

1. R is not just for statisticians, it is for all research orientedpsychologists.

2. R scripts are published in psychology journals to show newmethods:

• Psychological Methods• Psychological Science• Journal of Research in Personality

3. R based data sets are now accompanying journal articles:• The Journal of Research in Personality now accepts R code

and data sets.• JRP special issue in R.• The replicability project has released its data and R scripts.

4. By sharing our code and data the field can increase thepossibility of doing replicable science.

16 / 101


Reproducible Research: Sweave and KnitR

Sweave is a tool that allows to embed the R code forcomplete data analyses in LATEXdocuments. The purposeis to create dynamic reports, which can be updatedautomatically if data or analysis change. Instead ofinserting a prefabricated graph or table into the report,the master document contains the R code necessary toobtain it. When run through R, all data analysis output(tables, graphs, etc.) is created on the fly and insertedinto a final LATEXdocument. The report can beautomatically updated if data or analysis change, whichallows for truly reproducible research.

Friedrich Leisch (2002). Sweave: Dynamic generation of statistical reports using literate data analysis. I

Supplementary material for journals can be written inSweave/KnitR so that others can redo or extend the analyses.

17 / 101


What is so great about reproducible research?

1. Allows us to share methods with our collaborators.

2. This can be other labs who want to know what you did. It canbe your students, it can even be you.

3. David Condon has suggested that your closest collaborator isyou, six months ago, but you don’t answer your emails.

4. That is, scripted analyses are for you.

5. The Reproducibility Project (https://osf.io/ezcuj/ hasreleased their 100 replication data set and the R code toanalyze it. If any one finds errors or needs more information,they are happy to provide it.

6. See, for instance Dan Gilbert et al. critique (Gilbert, King, Pettigrew & Wilson,

2016) and the response (Anderson, Bahnık, Barnett-Cowan, Bosco, Chandler, Chartier & Cheung,

2016). Also see the recent denial of the need to worry aboutreplication by Roy Baumeister.

18 / 101

https://osf.io/ezcuj/


Misconception: R is hard to use

1. R doesn’t have a GUI (Graphical User Interface)• Partly true, many use syntax.• Partly not true, GUIs exist (e.g., R Commander, R-Studio).• Quasi GUIs for Mac and PCs make syntax writing easier.

2. R syntax is hard to use• Not really, unless you think an iPhone is hard to use.• Easier to give instructions of 1-4 lines of syntax rather than

pictures of menu after menu to pull down.• Keep a copy of your syntax, modify it for the next analysis.

3. R is not user friendly: A personological description of R• R is Introverted: it will tell you what you want to know if you

ask, but not if you don’t ask.• R is Conscientious: it wants commands to be correct.• R is not Agreeable: its error messages are at best cryptic.• R is Stable: it does not break down under stress.• R is Open: new ideas about statistics are easily developed.

19 / 101


What makes R so powerful are the > 8, 100 contributed packages

psych A general purpose toolkit for psychological researchwith a particular emphasis upon• Basic descriptive statistics and basic graphical

tools.• Basic psychometric procedures including

functions for finding α (= λ3), ωh, and ωt• More advanced data reduction techniques using

factor analysis, principal components analysis,and cluster analysis.

• Introductory Item Response Theory andMulti-level modeling

lavaan Basic and advanced structural equation modeling(“The gateway package to R”).

sem Structural equation modelinglme4 Multilevel modeling.

20 / 101


Short courses and workshops emphasize training in basic andadvanced R

1. Symposia• International Society for Study of Individual Differences (2005)• 1st World Conference on Personality (2013)• Society for Personality and Social Psychology (2015)

2. Short courses• European Conference on Personality (2012, 2014)• Association for Research on Personality (2012)• Association for Psychological Science (2013, 2014, 2015,

2016)• 1st World Conference on Personality (2013)• STuP 2015 preconference: Confirmatory factor analysis in the

lavaan package (R)

3. Summer schools• Summer school sponsored by ISSID, EAPP and SMEP (2014)

21 / 101




22 / 101






Revelle, 2014)






23 / 101

Part II: Open Materials TAI IPIP ICAR Part III: Open Methods



Temperament, Abilities, and Interests: considering appetites andaptitudes

Temperament, Abilities, and Interests: TAI

IPIP: The International Personality Item PoolLew Goldberg and the development of the IPIPExtending the IPIP to include more domains

ICAR: International Cognitive Ability ResourceAn international collaboration to measure ability with opensource itemsAnalysis of ICAR items

Part III: Open Methods

24 / 101


Personality, prediction, and life outcomes

1. It has long been known that to predict real world outcomes weneed to study more than just ability (Kelly & Fiske, 1950, 1951; Deary, 2008;

Roberts, Kuncel, Shiner, Caspi & Goldberg, 2007).

2. Level of education and jobs differ in their intellectualrequirements (Gottfredson, 1997).

3. My colleagues and I have shown that there are alsotemperamental requirements for educational and job choice(Condon & Revelle, 2014; Revelle & Condon, 2012, 2015a; Revelle, Wilt & Condon, 2011; Wilt & Revelle, 2015)

4. We consider individual differences in Temperament, Ability,and Interests (TAI) as they relate to niche selection in choiceof college major and in occupational choice (Bouchard, 1997; Hayes, 1962;

Johnson, 2010). as well as to the second and third level ofpersonality analysis (between individuals and between groupsof individuals (Revelle & Condon, 2015b)).

25 / 101


Measuring Affect, Behavior, Cognition and Desire within subjects

1. Personality is not just how people differ from each other, it isalso how people differ in their Affect, Behavior, Cognition andDesire (ABCDs) over time and space.

2. This can be done using simple text messaging technology, ormore deligtfully elegant systems such as the Big EAR (Mehl,

Pennebaker, Crow, Dabbs & Price, 2001; Mehl, Vazire, Ramirez-Esparza, Slatcher & Pennebaker, 2007; Mehl &

Robbins, 2012) or Ryne Sherman’s “Big Eye”.

3. Most potential subjects carry around a very powerful datarecorder: their cell phone.

4. Apps have been developed for smart phones to measure mostanything

5. Even dumb phones can receive text messages.

26 / 101


Measuring the ABCDs with text messaging

1. In the past, if you wanted to measure the within personstructure of affect, you needed to use paper and pencil diaries(Bolger, Davis & Rafaeli, 2003).

• This was tedious for the participant• Difficult for the investigator.

2. Josh Wilt, Katie Funkhouser, Wiebke Bleidorn and I haveused cell phone text messaging to measure affect, behavior,and meaning over time (Wilt, Funkhouser & Revelle, 2011; Wilt, Bleidorn & Revelle, 2016).

3. The basic observation is that the structure of affect withinpeople is stable, is predicable, and differs across people. (seealso (Rafaeli, Rogers & Revelle, 2007).

4. While for some people, Energetic Arousal is positivelycorrelated with Tense Arousal, for others they are orthogonal,and for yet others they are negatively correlated. Thiscorrelation is, itself, predictable.

27 / 101


Structure of Affect within subjects varies by orientation to world

28 / 101


Measuring individual differences1. A basic problem in the study of individual differences is that

there are so many different constructs that interest us. Theseinclude constructs from at least four broad domains

• Temperament• Ability• Interests• Character

2. Each domain has many constructs:• Dimensions of Temperament 2-3-5-6-15?• Structure of Ability (g - gf , gc , V-P-R)?• Hierarchical structure of interests people-things, RIASEC .• Range of possible measures of character.

3. But many important measures are proprietary.4. In addition, showing the utility of TAIC measures requires

criterion variables, and should include demographics.5. Our solution: Use and/or develop open source temperament,

ability, and interest items.29 / 101


The International Personality Item Pool (Goldberg, 1999)

1. Perhaps one of the greatest contributions from Lew Goldberghas been his release of the International Personality Item Poolor IPIP (Goldberg, 1999) s http://ipip.ori.org.

2. The IPIP adapted a short stem item format developed in thedoctoral dissertation of Hendriks (1997) and items from theFive Factor Personality Inventory developed in Groningen(Hendriks, Hofstee & De Raad, 1999).

3. Goldberg (1999) used about 750 items from the Englishversion of the Groningen inventory, and has sincesupplemented them with many more new items in the sameformat.

4. The IPIP items have been translated into at least 39languages by at least 65 different research teams. Thisincludes Arabic, German, Farsi, Icelandic, Indonesian,Japanese, Korean, Mandarin, Polish, Portuguese, Russian,Serbian, Spanish, Turkish, Urdu, Slovenian and Swedish.

30 / 101

http://ipip.ori.org


IPIP and other personality inventories

1. The IPIP was originally meant to be short stems to measurethe Abridged Five Factor Circumplex structure of adjectives(Hofstee, de Raad & Goldberg, 1992) but also includes items targeted at mostmajor personality tests.

2. Using a panel of roughly 1000 residents fromEugene-Springfield, Oregon, Goldberg administered hisoriginal IPIP items along with the NEO-PI-R (Costa & McCrae, 1992), theCPI (Gough & Bradley, 1996), the 16PF (Cattell & Stice, 1957), the MPQ (Tellegen &

Waller, 2008), the Hogan PI (Hogan & Hogan, 1995), the TCI (Cloninger, Przybeck & Svrakic,

1994), the JPI-R (Jackson, 1983), and the 6FPQ (Jackson, Paunonen & Tremblay, 2000).3. Goldberg then developed item stems that were highly

correlated to the commercial inventories and put these intothe public domain with the formation of the IPIP.

4. The items are available at http://ipip.ori.org and theEugene-Springfield data are available from Goldberg.

31 / 101

http://ipip.ori.org


What are the “Big 5”?: Some representative itemsSemantic analysis of many (although primarily European)languages suggest 5 broad factors of the ways in which wedescribe others.

Conscientiousness Complete my duties as soon as possible. Dothings according to a plan. Like order.

Agreeableness Take advantage of others. (R) Am concernedabout others. Sympathize with others’ feelings.

Neuroticism Get upset easily. Get overwhelmed by emotions.Have frequent mood swings.

Openness/Intellect Am able to come up with new and differentideas. Am full of ideas. Have a rich vocabulary.

Extraversion Like mixing with people. Enjoy meeting new people.Am a talkative person. Am rather lively.

These are sometimes organized as the OCEAN of personality,alternatively, the CANOE of personality.

32 / 101


Extending the IPIP

1. In addition to the basic temperament items at the IPIP site,there are additional items to measure vocational interests (theORVIS) (Pozzebon, Visser, Ashton, Lee & Goldberg, 2010) as well as avocationalinterests (Goldberg, 2010).

2. David Condon has expanded 2500 IPIP items to include theoriginal IPIP items, the ORVIS, the ORAIS, as well as itemsfrom the EPQ (Eysenck, Eysenck & Barrett, 1985), the O*NET interest profilescales (Rounds, Su, Lewis & Rivkin, 2010). These, and other items make atotal set of 4,300 items.

3. These are available athttps://sapa-project.org/MasterItemList/.

4. Condon has also noted that although 18 different inventories(with 168 scales) have what appear to be 1,894 items, thereare actually just 696 unique items. In addition, those “magic696” cover between 57% to 85% of 10 additional inventorieswith 235 additional scales.

33 / 101

https://sapa-project.org/MasterItemList/


The International Cognitive Ability ResourceICAR: Extending the IPIP to ability: IPIP:Personality::ICAR:Ability

1. ICAR is an international collaboration to develop open sourcecognitive ability items.

2. Information at http://www.icar-project.com/3. News letter at http:

//www.icar-project.com/ICAR_News_Issue_One.pdf

4. Key organizers who are coordinating the project:Germany Phillip Doebler (Munster and Ulm) and Heinz

Holling (Munster)U.K. Luning Sun and John Rust (Cambridge)

U.S.A William Revelle and David Condon(Northwestern)

5. Everyone is welcome to join this international collaboration.6. Supported by Open Research Area (ORA) for the Social

Sciences which includes participation from national fundingagencies (Germany:DFG), (UK:ESRC), (US:NSF).

34 / 101

http://www.icar-project.com/

http://www.icar-project.com/ICAR_News_Issue_One.pdf

http://www.icar-project.com/ICAR_News_Issue_One.pdf


ICAR: Proof of concept1. About 60 items were developed as part of a honors thesis at

Northwestern by Melissa Liebert (Liebert, 2006) and meant to be“Google resistant” (answers are not available on the web).

• This set was reported at conference in Krakow and in asubsequent book chapter (Revelle et al., 2010).

2. Subsequently David Condon developed some 3 Dimensionalrotations items and did some extensive item analysis of thetotal set.

3. Condon & Revelle (2014) examined the first 60 publiclyavailable items and validated them against self reported SATexam scores as well as a small sample given the Shipley-2(Shipley, 2009).

4. The original data set has been released to DataVerse (Condon &

Revelle, 2015a) and has been published in the Journal of OpenPsychology Data (Condon & Revelle, 2015c).

5. An example data set of 16 items with N = 1, 525 is includedas the ability dataset in the psych package.

35 / 101


Sample ICAR itemsMatrix Reasoning Verbal Reasoning

What number is one fifth of one fourth of one ninth of 900?

(1) 2 (2) 3 (3) 4 (4) 5 (5) 6 (6) 7

If the day after tomorrow is two days before Thursday,

then what day is it today?

(1) Friday (2) Monday (3) Wednesday

(4) Saturday (5) Tuesday (6) Sunday

Letter and Number SeriesIn the following alphanumeric series, what letter comes next?

I J L O S

(1) T (2) U (3) V (4) X (5) Y (6) Z

In the following alphanumeric series, what letter comes next?

Q S N P L

(1) J (2) H (3) I (4) N (5) M (6) L

Three-Dimensional Rotation

36 / 101


Sample analysis of ICAR items

1. Using basic R functions in the psych package (Revelle, 2015) we canevaluate the factor structure of the ICAR items.

2. irt.fa will do a factor analysis of the items and report thestatistics in terms of those statistics more commonly used inItem Response Theory.

• The two parameters from factor analysis are item difficultytaken from the τ parameter from the tetrachoric correlationand the item factor loading λ of the matrix of tetrachoriccorrelations.

a =λ√

1− λ2δ =

τ√1− λ2

• The hierarchical structure of the ability items may be shown byfactoring the factor intercorrelations.

• Loadings on a general factor may then be found by using theomega function which applies a Schmid Leiman transformationto the resulting higher level solution.

37 / 101


Item information curves for the 16 ICAR sample set

-3 -2 -1 0 1 2 3

0.0

0.2

0.4

0.6

0.8

1.0

Item information from factor analysis

Latent Trait (normal scale)

Item

Info

rmat

ion

reason.4

reason.16

reason.17

reason.19

letter.7

letter.33

letter.34letter.58

matrix.45matrix.46

matrix.47

matrix.55

rotate.3

rotate.4

rotate.6

rotate.8

38 / 101


Structure of sample ICAR 16 items shows a clear 4 factor hierarchicalsolution ωh = .87

Omega Hierarchical for ICAR Sample Test

LN.07

LN.34

LN.33

LN.58

R3D.03

R3D.08

R3D.04

R3D.06

MR.45

MR.46

MR.55

MR.47

VR.17

VR.04

VR.16

VR.19

LetterNum

0.80.70.60.5

3D Rotate

0.8

0.8

0.7

0.5

Matrices 0.70.70.50.4

Verbal R0.8

0.7

0.6

0.5

g

0.9

0.7

0.9

0.9

39 / 101


Structure of ICAR 60 items shows a messier 4 factor hierarchicalsolution ωh = .76

Hierarchical structure of ICAR60 items

VR.18VR.39VR.14VR.17LN.01VR.42VR.19VR.09VR.04VR.16LN.03VR.26VR.36LN.34R3D.07R3D.02R3D.17R3D.24R3D.18R3D.14R3D.19R3D.13R3D.12R3D.11R3D.01R3D.03R3D.21R3D.04R3D.08R3D.15R3D.10R3D.23R3D.09R3D.16R3D.05R3D.22R3D.06R3D.20MR.48MR.46MR.45MR.56MR.53MR.55MR.44MR.47MR.43LN.05LN.33MR.50LN.06LN.07MR.54LN.58LN.35VR.23VR.13VR.31VR.32VR.11

Verbal R

10.90.80.60.60.50.50.50.50.50.40.40.40.3

0.40.30.3-0.3

3D Rotate

0.80.70.70.70.70.70.70.70.70.70.60.60.60.60.60.60.60.60.50.50.50.50.40.4Matrices

0.30.3

0.3

0.60.60.60.60.60.60.60.50.50.50.50.40.40.40.40.40.30.3LetterNum

0.30.30.3

0.3

0.3

0.3

0.3

g

0.8

0.6

1

0.4

40 / 101


Open materials

1. The International Personality Item Pool items (Goldberg, 1999) aswell as the extended IPIP are in the public domain and areavailable to anyone for free.

2. The items from the International Cognitive Ability resource arealso in the public domain and are available to registeredusers. (We are trying to keep the items relatively secure anddo not put all of the actual items up on the web.)

• We have a basic set of 60 ICAR items (Condon & Revelle, 2014) and theICAR group is developing and validating item generators toautomatically produce hundreds of each of a growing numberof item types.

• We encourage others to join us in this mission.• Go to http://www.icar-project.com/ to register and for

more information.

41 / 101

http://www.icar-project.com/

Part III: Open Methods SAPA SAPA theory SAPA: practice Part IV: Open Data

Part III: Open Methods

Method: Synthetic Aperture Personality Assessment (SAPA)Measuring individual differences: the tradeoff between breadthversus depthSynthetic Aperture Astronomy

SAPA theorySample items as well as peopleCovariance algebra

SAPA: practiceOpen source software comes to the rescue

Part IV: Open Data

42 / 101






Revelle, 2014)






43 / 101


Measuring individual differences

1. A basic problem in the study of individual differences is thatthere are so many different constructs that interest us. Theseinclude constructs from at least four broad domains

• Temperament• Ability• Interests• Character

2. Each domain has many constructs• Dimensions of Temperament 2-3-5-6-?• Structure of Ability: g - gf , gc , V-P-R ?• Hierarchical structure of interests people-things RIASEC• Range of possible measures of character

3. In addition, showing the utility of TAIC measures requirescriterion variables

44 / 101


Breadth vs. depth of measurement

1. Factor structure of domains needs multiple constructs todefine structure.

2. Each construct needs multiple items to measure reliably.

3. This leads to an explosion of potential items .

4. But, people are willing to only answer a limited number ofitems.

5. This leads to the use of short and shorter forms (theNEO-PI-R with 300, the IPIP big 5 with 100, the BFI with 44items, the TIPI with 10) to include as part of other surveys.

45 / 101


Many items versus many people

1. Not only do want many items, we also want many people.

2. Resolution (fidelity) goes up with sample size, N (standarderrors are a function of

√N)

σx =σx√

N − 1σr =

1− r2√

N − 2

3. Also increases as number of items, n, measuring eachconstruct (reliability as well as signal/noise ratio varies asnumber of items and average correlation of the items)

λ3 = α =nr

1 + (n − 1)rs/n =

nr(1− nr)

4. Thus, we need to increase N as well as n. But how?

46 / 101


A short diversion: the history of optical telescopes

Resolution varies by aperture diameter (bigger is better)

47 / 101


A short diversion: history of radio telescopesResolution varies by aperture diameter (bigger is still better)

Aperture can be synthetically increased across multiple telescopesor even multiple observatories

48 / 101


Can we increase N and n at the same time?

1. Frederic Lord (1955) introduced the concept of samplingpeople as well as items.

2. Apply basic sampling theory to include not just people (wellknown) but also to sample items within a domain (less wellknown).

3. Basic principle of Item Response Theory and tailored tests.

4. Used by Educational Testing Service (ETS) to pilot items.

5. Used by Programme for International Student Assessment(PISA) in incomplete block design (Anderson, Lin, Treagust,Ross & Yore, 2007).

6. Can we use this procedure for the study of individualdifferences without being a large company?

7. Yes, apply the techniques of radio astronomy to combinemeasures synthetically and take advantage of the web.

49 / 101


Subjects are expensive, so are items

1. In a survey such as Amazon’s Mechanical Turk (MTURK), weneed to pay by the person and by the item.

2. Why give each person the same items? Sample items, as wesample people.

3. Synthetically combine data across subjects and across items.This will imply a missing data structure which is

• Missing Completely At Random (MCAR), or even moredescriptively:

• Massively Missing Completely at Random (MMCAR)

4. This is the essence of Synthetic Aperture PersonalityAssessment (SAPA).

50 / 101


3 Methods of collecting 256 subject * items dataa) 8 x 32 complete4621363452114345344364533121241421243623166421516154432261516513516613511551654636222244356233441114134336233221561215213561452225353121264561433433232246526411613351545664241146126412253535162463434215153624242541351343511611554654453123111162423325516334

b) 32 x 8 complete4632311425443314433154232631414541435614422361536242134435234443345141666341515444441342135143216636566312264546314661353264551466151251144114416244363633316236633254251153112661155546332453615224165463212356244146636366141445555223143644332146141633232365

c) 32 x 32 MCAR p=.25..3..2..6.....4.55.......44................4..6..45..3.4..6....16..3.......6.1.....6.2.......5.6....3522......5.3...3......5........3.2.2.......3..2......65..5......51....324.........23......5....552............25...54.5.......44.4.5....3..6...6........3......61.523.2....2...........3...5.............42.4..6.5......61.....3....3.6..1.4...1..5......5.1....54..........2.4.33..6......4.....52..6.....44.3...........2..44...1........1..42....5..1.....1..3.......2..3.521.......6..........3.142.........22......12..4...2..........3..162...4.....4..4..6..3.4...1....5.33.........5..........243..5....41......1....5..3..4...4.4..5..1.........4......4.......3..5.2.....64.4..4....1.1.2...6....4......55....2.......3..2..53.....2..2.3.3............1...2..43...3.13........5....2.........4..54...2.3..62....22.......332..1.....5......6.......5..3.4.....3....5.241..............63.1.......6...5..4..2...5..2.4..5..........52.4.....44...2.55.....2.....6.....6.....55.....5..........4....6341.4..2.........55......5.......45....3..32.51 / 101


Synthetic Aperture Personality Assessment

1. Give each participant a random sample of pn items takenfrom a larger pool of n items.

2. Find covariances based upon “pairwise complete data”.3. Find scales based upon basic covariance algebra.

• Let the raw data be the matrix X with N observationsconverted to deviation scores.

• Then the item variance covariance matrix is C = XX ′N−1

• and scale scores, S are found by S = K ′X .• K is a keying matrix, with K ij = 1 if itemi is to be scored in the

positive direction for scale j, 0 if it is not to be scored, and -1 ifit is to be scored in the negative direction.

• In this case, the covariance between scales, Cs, is

Cs = K ′X (K ′X )′N−1 = K ′XX ′KN−1 = K ′CK . (1)

4. That is, we can find the correlations/covariances betweenscales from the item covariances, not the raw items.

52 / 101


SAPA standard errors and effective sample size

1. When forming synthetic scales from MMCAR based items, thestandard error of correlations decreases as a function of theTotal number of subjects (N), the percentage of items samples(p), and the number of items forming the scale (n).

2. Ashley Brown has shown this quite clearly by simulation (Brown,

2014).

3. A good way to visualize this is to examine the standard errorof correlations as a function of N, p, and n.

4. An even more dramatic way is to plot the Effective SampleSize (Neff ) which because

σr =1− r2√

N − 2is merely Neff =

(1− r2)2

σ2r

+ 2

53 / 101


Effective sample size varies by the size of the composite scale.Simulating N= 10,000 with probability of any item (Brown, 2014)

(p = .125, .25, .5, or 1) and items in the composite 1 , 2, 4, 8, 16.

Rw: 0.3

p=0.125

p=0.25

p=0.5

p=1

0

2500

5000

7500

1000010000

Rb: 0

1 2 4 8 16n (items per scale)

Effe

ctive

Sam

ple

Size

54 / 101


SAPA is not magic: We can obtain high accuracy at the structure levelbut accuracy is much lower at the single subject level

1. Reliability of composite scales is high when formed fromsynthetic matrices Cs = K ′CK because the number of itemsper scale/per subject is the nominal amount.

2. Reliability of single scores is much less because very fewitems measuring a single trait are given to a single subjectS = K ′X .

3. However, the precision of the estimate of subject means (x) ishigh because σx = σx√

Np−1 and Np is large.

4. SAPA technique is very powerful for research of structure, butless powerful for research based upon single subjects.

5. Particularly useful in web based surveys with many subjectswhen we are limited in the number of items we can administerand where we are trying to increase our domain validity.

55 / 101


How does it work?1. Give our basic belief in open science, we use public domain

items, open source software:• Apache webserver, MySQL data bases, PHP and HTML5 web

tools, R for statistics.• Extensive coding in PHP and MySQL to present item sets in

random fashion (Joshua Wilt, David Condon, Jason French)• Code written for psychometric measurement and scale

construction as implemented in the psych package (Revelle, 2015)

using R (R Core Team, 2015)

2. Domains measured and item sources• Temperament items taken from International Personality Item

Pool (IPIP) (Goldberg, 1999) (ipip.ori.org) and supplemented withother items.

• Ability items have been validated (Condon & Revelle, 2014) as part of theInternational Cognitive Ability Resource Project(ICAR-project.org). (ICAR:Ability::IPIP:Temperament)

• Interest items taken from Oregon Vocational Interest Survey(ORVIS) (Pozzebon et al., 2010)

56 / 101

ipip.ori.org

ICAR-project.org


SAPA overview

1. A “Personality Test” is included as a resource at thehttp://personality-project.org and gives feedback toall participants.

2. Some participants then link their feedback to their socialmedia sites which then appeals to yet more to take it.

3. Some professors assign it to their students in various classes.

4. About 120-180 people per day from around the world visit thepersonality-project.org or sapa-project.orgwebsites. This does not sound like much, but over a year, weget around 40,000-50,000 participants.

5. This is much less than the 6 ∗ 106 subjects that Gosling andothers are reporting, but we are talking about structure basedupon 700-1000 items instead of a limited 44 BFI items.

6. This the benefit of random sampling.

57 / 101

http://personality-project.org

personality-project.org

sapa-project.org


10/28/15, 12:43 PMThe SAPA-Project: Explore Your Personality

Page 1 of 2https://sapa-project.org/

FAQ about the testIs it long? (not really) Is it free? (yes)

We won't sort you into a house or match

you to a harmonious date. But we will

give you feedback based on modern

psychological theory. Learn more...

The research behind SAPAHow was the test developed?

Each customized report is generated on

the basis of participant's responses, but

each participant gets a slightly different

subset of all 3,800 items. Learn more...

Individual DifferencesLearn more about differential psychology.

Why do individuals differ in the ways they

think, feel, and act? How do people differ

in response to the same situations? Learn

why individual differences matter...

Temperament Ability

Another personality test?Well, yes, but we like to think ours is a little more sophisticated than most.

The SAPA Project is a collaborative data collection tool for assessingpsychological constructs across multiple dimensions of personality. Thesedimensions currently include temperament, cognitive abilities, and interestsbecause research suggests that these three dimensions cover a large percentageof the variability among people without too much redundacy. Other broaddimensions like motivation and character may be included later depending onhow much incremental benefit they provide in terms of predicting outcomes.

The SAPA ProjectTake the test.Explore your personality.Advance the study of individualdifferences.

Start the test

More info

58 / 101


How does it work?: part II

1. Participants find us by searching web for “personality tests”,etc. and find personality-project.org orsapa-project.org

2. Each participant is given a number of web pagesConsent Form Basic description of project and question

whether they have taken test before.Demographics Age, sex, height, weight, education, parental

education, country, state, ZipCode (if US), ...TAIC questions Temperament/Ability/Interest questions (25

per page, 21 T/I, 4 Ability per pageContinuation pages After each page, told that feedback will

be more accurate if they keep going.Optional modules Creativity, Peer ratings, interests, ...Feedback Personality feedback based upon scores on

temperament items.3. Results are stored (page by page) on the MySQL server.

59 / 101


sapa-project.org


How does it work: part III

1. Various data cleaning scripts run using the SAPA-toolspackage (French & Condon, 2015) in R.

• Screen for duplicate responses based upon a RandomIdentification Number issued when subjects start the page. Wedrop all subsequent pages.

• Screen for subjects < 14 or > 90.

2. Subsequent analyses are done primarily using functions inthe psych package (Revelle, 2015) for R.

3. Analyses are done at multiple levels:3.1 At the item and scale covariance level to examine the structure

of items3.2 At the multiple levels of aggregation: zip code, state, college

major, occupation. This requires finding individual level scoresand then examining the structure of group means throughbasic multi-level techniques.

60 / 101

Part IV: Open Data Demographics Structure TAI and niche selection Summary: Open Science

Part IV: Open Data

Part IV: Open Data

Demographics of the SAPA dataSample characteristicsWeb data are not random samples

Personality StructureAre the “Big Five” really big?Temperament, Ability and Interests,

Temperament, Ability and Interests: Occupational ChoicePooled correlations 6= within group or between groupcorrelationsOccupational Choice as niche selection

Summary: Open Science

61 / 101






Revelle, 2014)






62 / 101


Our data

Time Frame Data collected at personality-project.org andsapa-project.org from August 18, 2010 toDecember 22, 2015

Subjects N = 207,002 (77,550 males, 129,451 females)Materials 953 items (696 temperament, 60 ability, 152

interests, 45 demographic)Scales used 15 Temperament, 4 Ability, 6 InterestsN in workforce N =80,317N students N = 86,348

Occupations 973 separate occupations, following a Paretodistribution with ≈ 80% represented by the top 20%of occupations

N ≥ 75 195 occupations for 55,902 participantsN ≥ 100 116 college majors for 130,584 participants

63 / 101


sapa-project.org


The top 35 countries account for 89% of the sample

USACANGBRAUSINDDEUPHLMYSSWENLDNORMEXNZLSGPCHNBRAZAFFINFRAIRLPOLROUIDNHKGSRBPAKNGADNKITAKORTURCHEBELESPRUS

150000 160000 170000 180000

The top 35 countries account for 89% of the sample

1418398895

55774024

22421986198119261362959869856853794765668654625610547506500498488472440434433419346330329325316283

64 / 101


Where do they come from? US SAPA data by zipcodes

65 / 101


US lights (from NASA)

66 / 101


Not a random sample of either education or gender (62% female)

Participants Education by Gender

count

Participants Education by Gender

60000

40000

20000

0

20000

40000

60000

<12 High Sch. In Coll <16 BA/BS In grad Grad/Pro

Female

Male

13% 7% 43% 5% 13% 5% 7%

67 / 101


Not a random sample of either age or gender (62% female)

Participants' Age by Gender

count

Age

Participants' Age by GenderAge

020

4060

80

10000 5000 0 5000 10000

Male Female

68 / 101


86,348 students, 80,317 employed)

Employment by Gendercount

Employment by Gender

40000

20000

0

20000

40000

Student Seeking work Employed Retired69 / 101


Are the “Big Five” really big?

1. There is a “consensus” about the proper number offactors/components of personality (Goldberg, 1990, 1992; Hofstee et al., 1992).

2. This seems to match life challenges of Getting Along andGetting AheadConscientiousness WorkAgreeableness LoveNeuroticism Effective functioning in many domainsOpenness/Intellect PlayExtraversion Leadership

3. Additional work has been done on the same 800-1000 personEugene-Springfield sample and suggests a hierarchicalstructure (e.g., the BFAS) (DeYoung, Quilty & Peterson, 2007).

4. But what happens with a larger and more diverse sample?

70 / 101


Multiple solutions to the dimensionality of temperament

• 2) Digman “alpha and beta” (Digman, 1997), DeYoung “stability andplasticity”(DeYoung, Peterson & Higgins, 2002)

• 3) Eysenck “Giant 3” (Eysenck, 1994) PEN• 5)The “Big 5” (Digman, 1990; Goldberg, 1990) CANOE/OCEAN/ I ... V (Norman,

1963; Tupes & Christal, 1961)

• 6)The HEXACO 6 (Lee & Ashton, 2004; Ashton, Lee & Goldberg, 2007)

• 7-9)Tellegen 7-9 (Tellegen & Waller, 2008)

• 8-9) Comrey 8-9 (Comrey, 2008)

• 16) Cattell 16 Personality Factors (Cattell, 1957)

1. Condon (2014, 2015) examined 696 non-overlapping itemsfrom IPIP:100, IPIP:NEO, IPIP:MSQ, BFAS, EPQ, etc. (Goldberg,

1999; DeYoung et al., 2007; Eysenck et al., 1985)

Found meaningful 3, 5, 15, (and now) 27 factor solutions.2. The Condon 3/5/15/27 form a heterarchical and non

hierarchical structure (i.e., lower levels are not cleanly nestedin higher levels.)

71 / 101


Personality shows a heterarchical even fractal structure

1. David Condon (2014, 2015) and in prep has shown:

2. The structure of 696 personality items given to 100-200,000participants does not show a clean organization.

3. The number of factors problems (and its non-solutions) willbreak the heart of most investigators.

4. No clear structure at any level.

5. Equally compelling fits at 3/5/15/27 factors.

6. The larger the sample size, the greater the resolution andthere is always something more there.

7. Articles showing “incremental validity beyond the Big Five” arejust as likely to show incremental validity beyond any numberof factors.

8. We think of this as showing a fractal structure: equal level ofcomplexity at any level of resolution.

72 / 101


The best items from the 15 scale solution

Table : Sample items from the Short Personality Inventory 15 factorsolution

Each scale has 8-10 itemsSPI Item ItemFear Panic easily. Begin to panic when there is danger.Volatility Get irritated easily. Lose my temper.Outlook Dislike myself. Feel a sense of worthlessness or hopelessness.Compassion Sympathize with others feelings. Am sensitive to the needs of others.Trust Trust others Trust what people say.Easygoing Let things proceed at their own pace. Take things as they come.Industrious Start tasks right away. Get chores done right away.Mach Use others for my own ends. Cheat to get ahead.Impulsivity Act without thinking. Do things without thinking of the consequences.Sociability Am mostly quiet when with other people. Tend to keep in the background on social occasions.Boldness Love dangerous situations. Take risks.Serious Seldom joke around. Am not easily amused.Conventional Don’t like the idea of change. Prefer to stick with things that I know.Intellectual Am quick to understand things. Catch on to things quickly.Open Enjoy thinking about things. Love to reflect on things.

73 / 101


The correlations of 696 personality items

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

1920

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38 39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

5960

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152153

154

155

156

157

158

159

160

161

162163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238239

240

241

242

243

244

245

246

247

248

249

250

251

252

253254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495

496

497498

499

500

501

502

503

504

505506

507

508

509

510 511

512

513

514

515

516

517

518

519520

521

522

523

524

525

526

527

528

529

530

531

532

533

534

535

536

537

538

539

540

541

542

543

544

545

546

547

548

549

550

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

566 567

568

569

570

571

572

573

574

575

576

577

578

579

580

581

582

583 584

585

586

587

588

589

590

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607 608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

629

630

631

632

633

634

635

636

637

638

639

640

641

642

643

644

645

646

647

648

649

650

651

652

653

654

655

656

657

658

659

660

661

662

663

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

679

680

681

682

683

684

685

686

687

688

689

690

691

692

693

694

695

696

74 / 101


Factoring the items on the first factor of the 696

●●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

8081

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260261

262

263

264

75 / 101


And doing it again

1

2

3

4

5 6

7

8

9

10

11

12

13

14

15

16

17

18

1920

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

5556

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

7576

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

76 / 101


And again

1 2

3

4

5

6

7

8

9

10

11 12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

3233

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

77 / 101


6 factors of interests

1. 6 factors from the O*NET interest profiler scales (60 items; Rounds

et al., 2010)

2. 8 factor Oregon Vocational Interest Scales (92 items; Pozzebon et al., 2010)

3. Oregon Avocational Interest Scales (199 items; Goldberg, 2010)

4. Formed into 6 scales fitting a “RIASEC” structure (60 items)Realistic “Like to work with tools and machinery.”

Investigative “Would like to do laboratory tests to identifydiseases.”

Artistic “Would like to write short stories or novels.”Social “Would like to help conduct a group therapy

session.”Enterprising “Would like to be the chief executive of a large

company.”Clerical “ Would like to keep inventory records”

78 / 101


The correlational structure of scales are found from the SAPA itemcorrelations

1. Given the raw data matrix, we can find the covariances (usingpairwise complete data) and the find the scaleintercorrelations.

2. The correlations with scales with overlapping items can becorrected for overlap scoreOverlap using a correction byBashaw & Anderson Jr (1967); Cureton (1966)

3. We can do this for 696 Temperament items, 60 Ability items,and 152 Interest items.

4. Although we have 150 different scales, I am reporting just IPIPbased Big 5, 4 ICAR ability scales, and the 6 RIASEC scales.

5. We can also drill down to the “Aspects Level” (DeYoung et al., 2007), the“facet level” (Costa, McCrae & Dye, 1991), the “nuances” (Mottus, 2016; ottus, Kandler,

Bleidorn, Riemann & McCrae, 2016) or items.

79 / 101


Personality at 3 levels of analysis (Revelle & Condon, 2015b)

Personality can be examined at three levels of analysis1. Personality as a unique temporal signature of one’s Affect,

Behavior, Cognition and Desires (ABCDs) as they changeover time and space within a single individual.

• Measuring within person patterning requires repeatedmeasures on single subjects over time. We do this with opensource text messaging procedures e.g., (Wilt et al., 2011; Wilt, 2014).

2. Personality is also how people differ in their patterning of theABCDs between people.

• This can be multilevel modeling of data collected withinsubjects showing that the correlational structure withinsubjects differs across subjects (Wilt et al., 2011; Revelle & Wilt, 2015).

• It is also the more conventional structure of personality itemsas collected from the SAPA project.

3. But people choose groups such as college major oroccupation based upon their unique aptitudes and appetites.

• We can analyze this niche selection in terms of the covarianceof the mean personality of the group.

80 / 101


TAI for groups is not the same as TAI for individuals

1. How do occupational groups or college majors differ on TAI?• The mean scores for groups allow us to compare the groups• But it is the structure of these group means that are particularly

interesting for they allow us to examine niche selection.

2. Overall correlation is a function of within group correlationsand between group correlations.

3. Correlations of aggregate scores rxybg (between groups) 6=aggregate of correlations rxywg (within groups)

4. The overall correlation rxy is a function of the within and thebetween correlationsrxy = etaxwg ∗ etaywg ∗ rxywg + etaxbg ∗ etaybg ∗ rxybg

5. These multi level correlations sometimes lead to what isknown as the Yule-Simpson paradox (Kievit, Frankenhuis, Waldorp & Borsboom,

2013; Simpson, 1951; Yule, 1903)

• These are independent and useful information.

81 / 101


Temperament, Ability, and Interests – within and between groups

1. Examined the factor structure of the TAI scales at the normal,between subjects (across groups) level.

• This produces the normal factor structure of temperament, ofability and of interests

• Can show these correlations as a “heatmap”

2. But when analyzing the structure of the mean scores for eachof 148 majors or 196 occupational groups (minimum size of75 members), the structure is drastically different.

• Several dimensions of temperament and interests are nownegatively correlated with ability, others are orthogonal

• Can also show these correlations as a “heatmap”.• In that some correlations change sign, this is not an artifact of

aggregation.

82 / 101


Subject Level data of 8 criteria, 5 personality scales, 4 ability, 6interests,

Within group correlations of Criteria, Temperament, Ability, Interests

ConventEnterpriSocialArtisticInvesti

RealisticVerbal3Drotmatrix

Let-NumICAR60

Open/IntelExtraStabAgreeConscincomeBMI

p2edup1edu

educationexercise

agegender

gender

age

exercise

education

p1edu

p2edu

BMI

income

Consc

Agree

Stab

Extra

Open/Intel

ICAR60

Let-Num

matrix

3Drot

Verbal

Realistic

Investi

Artistic

Social

Enterpri

Convent

-0.01 0.01 -0.02 0.02 -0.02 -0.01 0.02 0 0.03 0.01 -0.01 -0.01 0.02 0.09 0.08 0.08 0.07 0.07 0.21 0.15 -0.01 0.04 0.21 1

-0.06 -0.03 0.01 0 0 0.01 0 0 0.01 0.01 0.01 0.05 0.04 0.07 0.07 0.06 0.05 0.06 0.19 0.12 0.03 0.02 1

0.03 0.02 0 0.01 0 0 0.01 -0.01 0.01 0.1 0 0.05 0.04 -0.01 0 -0.01 -0.02 -0.01 -0.02 0.05 0.09 1

-0.01 0.02 0 0.02 0.02 0.02 -0.01 0.01 -0.01 0.02 0 0.01 0.07 0.01 0.01 0.01 0.03 0 0.1 0.1 1

-0.07 0 0 0.04 0.01 0.01 -0.01 0.02 -0.01 -0.02 0 -0.01 0.08 0.12 0.1 0.1 0.12 0.09 0.27 1

-0.11 0 0.01 0.02 0 0.01 0.02 0.01 -0.01 -0.01 0.01 -0.01 0.06 0.13 0.11 0.11 0.12 0.1 1

-0.07 0.03 -0.11 0.11 -0.07 0 0.06 0.02 -0.02 0.08 -0.07 -0.03 0.14 0.84 0.67 0.62 0.49 1

-0.1 0.02 -0.06 0.11 0 0.04 0 0.04 -0.02 0.04 -0.04 -0.02 0.1 0.75 0.59 0.57 1

-0.07 0.02 -0.12 0.1 -0.08 -0.01 0.05 0.02 -0.01 0.08 -0.07 -0.02 0.13 0.87 0.74 1

-0.07 0.02 -0.12 0.11 -0.07 0 0.05 0.02 -0.01 0.08 -0.07 -0.02 0.13 0.9 1

-0.09 0.03 -0.12 0.13 -0.07 0.01 0.05 0.03 -0.02 0.09 -0.08 -0.03 0.15 1

-0.08 0.11 0 0.13 -0.02 0.01 0.05 0.05 0.13 0.2 0.15 0.24 1

0.01 0.03 0.04 0.01 0.02 0.02 -0.01 0.02 0.18 0.36 0.27 1

-0.13 0.08 0.05 0.02 0.03 0.02 -0.02 0.04 0.2 0.15 1

0.12 0.09 0 0.01 -0.03 0 0.04 0 0.21 1

0.06 0.1 0.04 0.02 -0.02 -0.02 -0.01 0.03 1

-0.04 0.22 0.03 0.26 0.02 0.02 0.02 1

-0.03 0.2 -0.1 0.04 -0.16 -0.13 1

-0.01 -0.12 0.07 0.03 0.51 1

0 -0.11 0.09 0.03 1

-0.04 0.45 0.03 1

-0.05 0.02 1

-0.01 1

1

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

83 / 101


Group Level data of 8 criteria, 5 personality scales, 4 ability and 6interests”

Between group correlations of Criteria, Temperament, Ability, Interests



Let-NumICAR60


p2edup1edu

educationexercise

agegender

gender

age

exercise

education

p1edu

p2edu

BMI

income

Consc

Agree

Stab

Extra

Open/Intel

ICAR60

Let-Num

matrix

3Drot

Verbal

Realistic

Investi

Artistic

Social

Enterpri

Convent

-0.72 0.31 -0.01 0.42 0.14 0.17 -0.14 0.75 -0.03 -0.65 0.52 -0.32 0.19 0.51 0.53 0.51 0.52 0.45 0.67 0.22 -0.1 -0.62 0.73 1

-0.64 0.21 0.23 0.36 0.18 0.2 -0.22 0.72 0.07 -0.54 0.61 0.1 0.11 0.31 0.33 0.3 0.3 0.29 0.42 -0.04 -0.14 -0.47 1

0.73 -0.09 0.11 -0.41 -0.27 -0.35 0.24 -0.67 0.27 0.83 -0.31 0.49 -0.35 -0.6 -0.6 -0.6 -0.61 -0.55 -0.72 -0.31 -0.08 1

-0.06 -0.05 -0.04 0.33 0.42 0.44 -0.36 -0.09 -0.72 -0.24 -0.49 -0.3 0.6 0.2 0.15 0.15 0.19 0.31 0.11 -0.16 1

-0.39 -0.37 0 0.14 0.38 0.41 -0.37 0.27 -0.26 -0.43 0.15 -0.56 0.38 0.66 0.66 0.68 0.69 0.6 0.54 1

-0.86 0.02 -0.08 0.46 0.36 0.44 -0.32 0.7 -0.4 -0.79 0.4 -0.58 0.48 0.79 0.79 0.81 0.81 0.73 1

-0.75 -0.14 0.01 0.69 0.71 0.79 -0.63 0.62 -0.73 -0.79 0.11 -0.73 0.82 0.98 0.96 0.95 0.95 1

-0.79 -0.16 -0.04 0.59 0.64 0.71 -0.57 0.65 -0.63 -0.8 0.22 -0.72 0.72 0.99 0.98 0.98 1

-0.79 -0.14 -0.08 0.58 0.58 0.67 -0.53 0.66 -0.59 -0.78 0.22 -0.72 0.7 0.99 0.99 1

-0.8 -0.13 -0.06 0.61 0.6 0.69 -0.53 0.68 -0.6 -0.79 0.23 -0.71 0.71 0.99 1

-0.79 -0.14 -0.04 0.62 0.64 0.72 -0.57 0.66 -0.65 -0.8 0.2 -0.73 0.75 1

-0.51 -0.07 0.01 0.61 0.66 0.75 -0.51 0.35 -0.8 -0.59 -0.16 -0.57 1

0.46 0.08 0.34 -0.41 -0.38 -0.43 0.33 -0.35 0.63 0.65 0.21 1

-0.58 0.25 0.27 0.16 -0.08 -0.05 0.09 0.54 0.36 -0.26 1

0.86 0.03 0 -0.59 -0.54 -0.59 0.51 -0.75 0.53 1

0.37 0.28 0.02 -0.53 -0.72 -0.76 0.65 -0.17 1

-0.84 0.27 -0.04 0.55 0.27 0.32 -0.24 1

0.33 0.54 -0.39 -0.42 -0.87 -0.84 1

-0.47 -0.37 0.35 0.59 0.95 1

-0.39 -0.41 0.4 0.56 1

-0.55 0.38 0.17 1

-0.02 -0.11 1

-0.09 1

1

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

84 / 101


Compare the two correlation matrices–college majors8 criteria, 5 personality scales, 4 ability and 6 interests

Within group (below diagonal), Between groups (above diagonal



Let-NumICAR60


p2edup1edu

educationexercise

agegender

gender

age

exercise

education

p1edu

p2edu

BMI

income

Consc

Agree

Stab

Extra

Open/Intel

ICAR60

Let-Num

matrix

3Drot

Verbal

Realistic

Investi

Artistic

Social

Enterpri

Convent

-0.01 0.01 -0.02 0.02 -0.02 -0.01 0.02 0 0.03 0.01 -0.01 -0.01 0.02 0.09 0.08 0.08 0.07 0.07 0.21 0.15 -0.01 0.04 0.21

-0.06 -0.03 0.01 0 0 0.01 0 0 0.01 0.01 0.01 0.05 0.04 0.07 0.07 0.06 0.05 0.06 0.19 0.12 0.03 0.02 0.73

0.03 0.02 0 0.01 0 0 0.01 -0.01 0.01 0.1 0 0.05 0.04 -0.01 0 -0.01 -0.02 -0.01 -0.02 0.05 0.09 -0.47 -0.62

-0.01 0.02 0 0.02 0.02 0.02 -0.01 0.01 -0.01 0.02 0 0.01 0.07 0.01 0.01 0.01 0.03 0 0.1 0.1 -0.08 -0.14 -0.1

-0.07 0 0 0.04 0.01 0.01 -0.01 0.02 -0.01 -0.02 0 -0.01 0.08 0.12 0.1 0.1 0.12 0.09 0.27 -0.16 -0.31 -0.04 0.22

-0.11 0 0.01 0.02 0 0.01 0.02 0.01 -0.01 -0.01 0.01 -0.01 0.06 0.13 0.11 0.11 0.12 0.1 0.54 0.11 -0.72 0.42 0.67

-0.07 0.03 -0.11 0.11 -0.07 0 0.06 0.02 -0.02 0.08 -0.07 -0.03 0.14 0.84 0.67 0.62 0.49 0.73 0.6 0.31 -0.55 0.29 0.45

-0.1 0.02 -0.06 0.11 0 0.04 0 0.04 -0.02 0.04 -0.04 -0.02 0.1 0.75 0.59 0.57 0.95 0.81 0.69 0.19 -0.61 0.3 0.52

-0.07 0.02 -0.12 0.1 -0.08 -0.01 0.05 0.02 -0.01 0.08 -0.07 -0.02 0.13 0.87 0.74 0.98 0.95 0.81 0.68 0.15 -0.6 0.3 0.51

-0.07 0.02 -0.12 0.11 -0.07 0 0.05 0.02 -0.01 0.08 -0.07 -0.02 0.13 0.9 0.99 0.98 0.96 0.79 0.66 0.15 -0.6 0.33 0.53

-0.09 0.03 -0.12 0.13 -0.07 0.01 0.05 0.03 -0.02 0.09 -0.08 -0.03 0.15 0.99 0.99 0.99 0.98 0.79 0.66 0.2 -0.6 0.31 0.51

-0.08 0.11 0 0.13 -0.02 0.01 0.05 0.05 0.13 0.2 0.15 0.24 0.75 0.71 0.7 0.72 0.82 0.48 0.38 0.6 -0.35 0.11 0.19

0.01 0.03 0.04 0.01 0.02 0.02 -0.01 0.02 0.18 0.36 0.27 -0.57 -0.73 -0.71 -0.72 -0.72 -0.73 -0.58 -0.56 -0.3 0.49 0.1 -0.32

-0.13 0.08 0.05 0.02 0.03 0.02 -0.02 0.04 0.2 0.15 0.21 -0.16 0.2 0.23 0.22 0.22 0.11 0.4 0.15 -0.49 -0.31 0.61 0.52

0.12 0.09 0 0.01 -0.03 0 0.04 0 0.21 -0.26 0.65 -0.59 -0.8 -0.79 -0.78 -0.8 -0.79 -0.79 -0.43 -0.24 0.83 -0.54 -0.65

0.06 0.1 0.04 0.02 -0.02 -0.02 -0.01 0.03 0.53 0.36 0.63 -0.8 -0.65 -0.6 -0.59 -0.63 -0.73 -0.4 -0.26 -0.72 0.27 0.07 -0.03

-0.04 0.22 0.03 0.26 0.02 0.02 0.02 -0.17 -0.75 0.54 -0.35 0.35 0.66 0.68 0.66 0.65 0.62 0.7 0.27 -0.09 -0.67 0.72 0.75

-0.03 0.2 -0.1 0.04 -0.16 -0.13 -0.24 0.65 0.51 0.09 0.33 -0.51 -0.57 -0.53 -0.53 -0.57 -0.63 -0.32 -0.37 -0.36 0.24 -0.22 -0.14

-0.01 -0.12 0.07 0.03 0.51 -0.84 0.32 -0.76 -0.59 -0.05 -0.43 0.75 0.72 0.69 0.67 0.71 0.79 0.44 0.41 0.44 -0.35 0.2 0.17

0 -0.11 0.09 0.03 0.95 -0.87 0.27 -0.72 -0.54 -0.08 -0.38 0.66 0.64 0.6 0.58 0.64 0.71 0.36 0.38 0.42 -0.27 0.18 0.14

-0.04 0.45 0.03 0.56 0.59 -0.42 0.55 -0.53 -0.59 0.16 -0.41 0.61 0.62 0.61 0.58 0.59 0.69 0.46 0.14 0.33 -0.41 0.36 0.42

-0.05 0.02 0.17 0.4 0.35 -0.39 -0.04 0.02 0 0.27 0.34 0.01 -0.04 -0.06 -0.08 -0.04 0.01 -0.08 0 -0.04 0.11 0.23 -0.01

-0.01 -0.11 0.38 -0.41 -0.37 0.54 0.27 0.28 0.03 0.25 0.08 -0.07 -0.14 -0.13 -0.14 -0.16 -0.14 0.02 -0.37 -0.05 -0.09 0.21 0.31

-0.09 -0.02 -0.55 -0.39 -0.47 0.33 -0.84 0.37 0.86 -0.58 0.46 -0.51 -0.79 -0.8 -0.79 -0.79 -0.75 -0.86 -0.39 -0.06 0.73 -0.64 -0.72

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

85 / 101


Niche selection

1. College majors and occupations differ systematically in theintellectual Ability they require.

2. But they also differ in the Interests and Temperament theyrequire.

3. Examine the structure of the mean personality scale scoresfor college majors and for occupations.

4. The space is 2-3 dimensional.

5. A simple two factor solution shows that high ability can tradeoff for low Industry or Conscientiousness and that Boldness(low Anxiety) and Realistic interests differs from high Anxietyand Social interests.

6. We can examine the extent these two dimensions relate to avariety of criteria using factor extension (Dwyer, 1937; Horn, 1973)

86 / 101


Biplot of a two factor solution to the group level major data

-3 -2 -1 0 1 2 3

-3-2

-10

12

3TAI factors of aggregated college majors

MR1

MR2

Industryorder

comp

polite

volatile

withdrawn

assert

enthus

intel

open

Let-Nummatrix3Drot

Verbal

Realistic

Investi

Artistic

Social

EnterpriConvent

-1.0 -0.5 0.0 0.5 1.0

-1.0

-0.5

0.0

0.5

1.0

87 / 101


Add 8 criteria to the extended factor solution of the group data

-3 -2 -1 0 1 2 3

-3-2

-10

12

3TAI factors of aggregated majors extended to criteria variables

MR1

MR2

Industryorder

comp

polite

volatile

withdrawn

assert

enthus

intel

open

Let-Nummatrix3Drot

Verbal

Realistic

Investi

Artistic

Social

EnterpriConvent

gender

exercise

education

p1edup2edu

BMIage

income

-1.0 -0.5 0.0 0.5 1.0

-1.0

-0.5

0.0

0.5

1.0

88 / 101


Occupational choice

1. Just as people assort themselves into college majors basedupon the appetites and their aptitudes, so they assortthemselves into occupations.

2. Some occupations require getting along, some emphasizegetting ahead, some require both

3. I show a 2 dimensional solution and then using factorextension, add in gender to the model

4. The final figure is not meant to be read, but rather seen as therange of occupations covered.

89 / 101


Biplot of a two factor solution to the group level data

-3 -2 -1 0 1 2 3

-3-2

-10

12

3

Biplot of TAI scores at group level

MR1

MR2

Fear

Volatility

Enthus

Compassion

Trust

Easygoing

Industry

Mach

Impulsivity

Sociability

Bold

Serious

Habit

Int

Open LetNumMatrix3DRot

Verb

Real

Invest

Arti

Social

EnterClerical

-1.0 -0.5 0.0 0.5 1.0

-1.0

-0.5

0.0

0.5

1.0

90 / 101



-3 -2 -1 0 1 2 3

-3-2

-10

12

3


MR1

MR2

Fear

Volatility

Enthus

Compassion

Trust

Easygoing

Industry

Mach

Impulsivity

Sociability

Bold

Serious

Habit

Int


Verb

Real

Invest

Arti

Social

EnterClerical

gender

-1.0 -0.5 0.0 0.5 1.0

-1.0

-0.5

0.0

0.5

1.0

91 / 101



-3 -2 -1 0 1 2 3

-3-2

-10

12

3


MR1

MR2 Actor

Art Director

Coach and/or Scout

Copy Writer

Editor

Graphic Designer

Musician and/or Singer

Photographer

Producer and/or DirectorPublic Relations Specialist

Writer

Other - ArtistOther - Designer

Other - Entertainer and/or PerformerOther - Media Related Worker

Janitor and/or Cleaner (except Maid and/or Housekeeping Cleaner)

Landscaping and/or Groundskeeping Worker

Maid and/or Housekeeping Cleaner

AccountantAuditor

Budget AnalystBusiness Operations Specialist

Credit Analyst

Financial Analyst

Human Resources and/or Training

Loan Officer

Management Analyst

Personal Financial AdvisorOther - Financial Specialist

Other - Business Worker

Clergy

Counselor - Educational, Vocational, and SchoolCounselor - Mental Health

Probation Officer and/or Correctional Treatment Specialist

Social and Human Service Assistant

Social Worker - Child, Family and/or SchoolSocial Worker - Mental Health and/or Substance Abuse

Other - Community and/or Social Service Specialist

Other - Counselor

Other - Religious WorkerOther - Social Worker

Business Intelligence Analyst

Computer and Information Scientist

Computer Programmer

Computer Security Specialist

Computer Software Engineer

Computer Specialist

Computer Support Specialist

Computer Systems Engineers/Architect

Database AdministratorInformation Technology Project Manager

Network and Computer Systems Administrator

Software Quality Assurance Engineer and/or TesterWeb Developer

Other - Computer Science Worker

Carpenter

Construction Laborer

ElectricianSupervisor/Manager of Construction and/or Extraction Workers

Other - Construction and Related Work

Adult Literacy, Remedial Education, and/or GED Teacher and/or Instructor

Elementary School Teacher (except Special Education)

Graduate Teaching Assistant

Instructional CoordinatorInstructional Designer and/or Technologist

Kindergarten Teacher (except Special Education)Librarian

Library Technician

Middle School Teacher (except Special and Vocational Education)

Postsecondary Teacher - Business

Postsecondary Teacher - Education

Postsecondary Teacher - English Language and/or Literature

Postsecondary Teacher - Psychology

Preschool Teacher (except Special Education)

Secondary School Teacher (except Special and Vocational Education)

Special Education Teacher - Preschool, Kindergarten, and Elementary School

Special Education Teacher - Secondary School

Teacher Assistant

Tutor

Other - Education, Training, or Library WorkerOther - Teacher and/or Instructor

Architect

Civil Engineer

Electrical EngineersMechanical Engineer

Other - Engineer

BaristaBartender

Chef and/or Head Cook

Cook - Fast Food

Cook - Institutional and Cafeteria

Cook - Other

Cook - Restaurant

Cook - Short Order

Dishwasher

Food Preparation Worker

Food Server

Food Service Attendant/Helper

Host/HostessSupervisor/Manager of Food Preparation and Serving Workers

Waiter/Waitress

Other - Food Preparation Worker

Acute Care Nurse

Dental Assistant

Emergency Medical Technician and/or Paramedic

Family and General Practitioner

Home Health Aide

Licensed Practical Nurse and/or Licensed Vocational Nurse

Massage Therapist

Medical and Clinical Laboratory Technician and/or Technologist

Medical Assistant

Medical Records and Health Information Technicians

Nurse Practitioner

Nursing Aid, Orderly and/or Attendant

Pharmacist

Pharmacy Technician

Physical Therapist

Psychiatrist

Radiologic Technologist and/or Technician

Registered Nurse

Surgical TechnologistOther - Health Technologist and/or TechnicianOther - Healthcare Support Worker

Other - Therapist

Automotive Mechanic and/or Service Technician

Electrical and Electronics Installer and/or RepairerOther - Installation, Maintenance, and Repair Worker

Law Clerk

Lawyer

Paralegal and/or Legal Assistant

Other - Legal Worker

Other - Legal Support WorkerBiologists

Psychologist - ClinicalPsychologist - Counseling

Psychologist - Other

Social Science Research Assistant

Other - Social Scientist

Administrative Services Manager

Chief Executive Officer

Computer and Information Systems ManagerFinancial Manager

Food Service Manager

General and Operations Manager

Human Resources Manager

Marketing Manager

Sales ManagerTraining and Development Manager

Other - Manager

Supervisor/Manager of Production Workers

Other - Production Worker

Air Crew Member

InfantrySupervisor/Manager of All Other Tactical Operations Specialists

Other - Military Enlisted Special and Tactical Operations Crew Member and/or Specialist

Other - Military Officer Special and Tactical Operations Leader/Manager

Billing Clerk

Bookkeeping, Accounting, and Auditing Clerks

Customer Service Representative

Data Entry KeyerExecutive Secretary and/or Administrative Assistant

Hotel, Motel, and Resort Desk Clerk

Human Resources AssistantInsurance Claims and/or Policy Processing Clerk

Medical Secretary

Office Clerk, General

Receptionist and/or Information ClerkSecretary

Supervisor/Manager of Office and Administrative Support Workers

Other - Office and Administrative Support Worker

Child Care Worker

Hairdresser, Hairstylist, and/or Cosmetologist

Nanny

Personal and Home Care Aide

Other - Personal Care and Service Worker

Correctional Offices and/or Jailer

Security Guard

Other - Protective Service Worker

Advertising Sales AgentCashierCounter and/or Rental ClerkFinancial Services Sales Agent

Insurance Sales Agent

Real Estate Sales AgentRetail Salesperson

Sales Representative - Services

Sales Representative - Wholesale and ManufacturingSupervisor/Manager of Retail Sales Workers

Supervisors/Manager of Non-Retail Sales Workers

Telemarketer

Other - Sales RepresentativeOther - Sales Worker

Driver/Sales Worker

Laborer - Freight, Stock, and/or Material Moving

Other - Material Moving Worker

Other - Transportation Workers

Fear

Volatility

Enthus

Compassion

Trust

Easygoing

Industry

Mach

Impulsivity

Sociability

Bold

Serious

Habit

Int


Verb

Real

Invest

Arti

Social

EnterClerical

gender

-1.0 -0.5 0.0 0.5 1.0

-1.0

-0.5

0.0

0.5

1.0

92 / 101

Replication Studies Summary: Open Science References

Part VI

Open Scientific Research

93 / 101


Replicability of personality research

1. To what extent is personality research replicable?

2. By using open methods and open data, we allow ourselves totest for replicability

94 / 101


State level personality differences: Are they replicable?

1. Rentfrow and Gosling have reported data from several largeinternet surveys (much larger than ours).

2. Are they replicable?

3. With data shared from Gosling and Rentfrow, we were able toexamine how much the 5 studies they report match ourfindings.

4. State differences are small, but reliable for some measures,not all.

5. We have looked at the basic state differences in personalitymean scores

6. Also examined 18 different behavioral criterion (e.g., crimerates, state income, state well being, state liberalism, etc.)

95 / 101


Replicability of state differences depends upon the trait

−1 0 1

Sam

ple

1

Sam

ple

2

Sam

ple

3

Sam

ple

4

Sam

ple

5

SAPA

2010

SAPA

2015

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

SAPA2010

SAPA2015

Neuroticism

80 77

66

74

67

64

54

45

47

63

53

41

48

53

40

76

66

59

72

49

64

−1 0 1

Sam

ple

1

Sam

ple

2

Sam

ple

3

Sam

ple

4

Sam

ple

5

SAPA

2010

SAPA

2015

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

SAPA2010

SAPA2015

Openness

85 70

62

72

85

67

53

68

34

65

43

48

19

29

27

43

36

9

18

22

63

−1 0 1

Sam

ple

1

Sam

ple

2

Sam

ple

3

Sam

ple

4

Sam

ple

5

SAPA

2010

SAPA

2015

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

SAPA2010

SAPA2015

Conscientiousness

72 37

45

70

61

14

30

31

23

27

26

27

12

21

11

37

47

37

41

43

41

−1 0 1

Sam

ple

1

Sam

ple

2

Sam

ple

3

Sam

ple

4

Sam

ple

5

SAPA

2010

SAPA

2015

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

SAPA2010

SAPA2015

Extraversion

70 56

59

20

21

12

−6

−11

−20

46

27

20

36

35

17

28

25

21

37

9

65

−1 0 1

Sam

ple

1

Sam

ple

2

Sam

ple

3

Sam

ple

4

Sam

ple

5

SAPA

2010

SAPA

2015

Sample 1

Sample 2

Sample 3

Sample 4

Sample 5

SAPA2010

SAPA2015

Agreeableness

76 53

49

62

28

37

26

31

5

19

−2

19

15

−16

−23

11

41

22

−16

11

47

96 / 101


Replicability of personality by state demographics depends upon trait

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

−1.0 −0.5 0.0 0.5 1.0

−1.0

−0.5

0.0

0.5

1.0

Neuroticism

Original Partial Correlations

Rep

licat

ion

Parti

al C

orre

latio

ns r=0.73

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●●

−1.0 −0.5 0.0 0.5 1.0−1

.0−0

.50.

00.

51.

0

Openness


Rep

licat

ion

Parti

al C

orre

latio

ns r=0.76

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

−1.0 −0.5 0.0 0.5 1.0

−1.0

−0.5

0.0

0.5

1.0

Conscientiousness


Rep

licat

ion

Parti

al C

orre

latio

ns r=0.84

●

●

●

●

●

●●

●

●

●

●

●●

●

● ●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

● ●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

● ●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●●

●●

●

●

●

●

●

●

●

−1.0 −0.5 0.0 0.5 1.0

−1.0

−0.5

0.0

0.5

1.0

Extraversion


Rep

licat

ion

Parti

al C

orre

latio

ns r=0.12

●●●

●

●

●

●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

● ●● ●

●

●

●

●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

● ●●●

●

●

●

●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●● ●

●

●

●

●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

−1.0 −0.5 0.0 0.5 1.0

−1.0

−0.5

0.0

0.5

1.0

Agreeableness


Rep

licat

ion

Parti

al C

orre

latio

ns r=−0.05

97 / 101


State level: Income varies by iq (weighted r = .51)

0.15 0.20 0.25 0.30 0.35

40000

45000

50000

55000

60000

65000

State Income by State Ability -- weighted r = .51

ICAR60 estimate of state ability

Med

ian

inco

me

Alabama

Arizona

Arkansas

California

Colorado

Connecticut

Delaware

Florida

Georgia

Idaho

Illinois

Indiana

Iowa

Kansas

Kentucky

Louisiana

Maine

Maryland

Massachusetts

Michigan

Minnesota

Mississippi

Missouri

Montana

Nebraska

Nevada

New Hampshire New Jersey

New Mexico

New York

North Carolina

North Dakota

OhioOklahoma

OregonPennsylvania

Rhode Island

South Carolina

South Dakota

Tennessee

Texas

Utah

Vermont

Virginia

Washington

West Virginia

Wisconsin

Wyoming

98 / 101


State level: Well being varies by neuroticism (weighted r = -.59)

3.30 3.35 3.40 3.45 3.50 3.55

5960

6162

6364

Personality x Demographics -- Weighted r = -.59

State Neuroticism

Sta

te W

ellb

eing

Alabama

Arizona

Arkansas

California

Colorado

Connecticut

Delaware

Florida

Georgia

Idaho

Illinois

Indiana

Iowa

Kansas

Kentucky

Louisiana

Maine

Maryland

Massachusetts

Michigan

Minnesota

Mississippi

Missouri

Montana

Nebraska

Nevada

New Hampshire

New Jersey

New Mexico

New York

North Carolina

North Dakota

Ohio

Oklahoma

Oregon

Pennsylvania

Rhode Island

South Carolina

South Dakota

Tennessee

TexasUtah

Vermont

Virginia

Washington

West Virginia

Wisconsin

Wyoming

99 / 101


State level: Liberalism varies by Conscientiousness (r= -.42)

4.1 4.2 4.3 4.4

0.10

0.12

0.14

0.16

0.18

0.20

0.22

Personality x Demographics

State Conscientiousness

Sta

te L

iber

alis

m

Alabama

Arizona

Arkansas

California

Colorado

Connecticut

Delaware

Florida

Georgia

Idaho

Illinois

Indiana

Iowa

Kansas

Kentucky

Louisiana

Maine Maryland

Massachusetts

Michigan

Minnesota

Mississippi

Missouri

Montana

Nebraska

Nevada

New Hampshire

New Jersey

New Mexico

New York

North Carolina

North Dakota

Ohio

Oklahoma

Oregon

Pennsylvania

Rhode Island

South Carolina

South Dakota

Tennessee

Texas

Utah

Vermont

Virginia

Washington

West Virginia

Wisconsin

Wyoming

100 / 101


Summary and Conclusions

1. Ability, temperament and interests all provide usefulinformation about human personality.

2. Intellectual and personality development is the process ofexperiencing and choosing niches.

3. When we describe the intellectual requirements of aprofession or a college major, we should not ignore thatappropriate interests and temperaments guide occupationalchoice.

4. We need to consider appetites along with aptitudes.

5. The statistics, materials, methods, and data from all of thesestudies are done using Open Source Science.

6. Join us in this journey.

7. For more information and for these slides go tohttp://personality-project.org/sapa.html

101 / 101

http://personality-project.org/sapa.html


Anderson, C. J., Bahnık, S., Barnett-Cowan, M., Bosco, F. A.,Chandler, J., Chartier, C. R., & Cheung, F. (2016). Response tocomment on “estimating the reproducibility of psychologicalscience”. Science, 351(6277), 1037–1037.

Anderson, J., Lin, H., Treagust, D., Ross, S., & Yore, L. (2007).Using large-scale assessment datasets for research in scienceand mathematics education: Programme for InternationalStudent Assessment (PISA). International Journal of Scienceand Mathematics Education, 5(4), 591–614.

Ashton, M. C., Lee, K., & Goldberg, L. R. (2007). TheIPIP-HEXACO scales: An alternative, public-domain measure ofthe personality constructs in the HEXACO model. Personalityand Individual Differences, 42(8), 1515–1526.

Bashaw, W. & Anderson Jr, H. E. (1967). A correction for replicatederror in correlation coefficients. Psychometrika, 32(4), 435–441.

Bolger, N., Davis, A., & Rafaeli, E. (2003). Diary methods:101 / 101


Capturing life as it is lived. Annual Review of Psychology, 54,579–616.

Bouchard, T. (1997). Experience producing drive theory: howgenes drive experience and shape personality. Acta Paediatrica,86(S422), 60–64.

Brown, A. D. (2014). Simulating the MMCAR method: Anexamination of precision and bias in synthetic correlations whendata are ‘massively missing completely at random’. Master’sthesis, Northwestern University, Evanston, Illinois.

Cattell, R. B. (1957). Personality and motivation structure andmeasurement. Oxford, England: World Book Co.

Cattell, R. B. & Stice, G. (1957). Handbook for the SixteenPersonality Factor Questionnaire. Champaign, Ill.: Institute forAbility and Personality Testing.

Cloninger, C. R., Przybeck, T. R., & Svrakic, D. M. (1994). TheTemperament and Character Inventory (TCI): A guide to its

101 / 101


development and use. center for psychobiology of personality,Washington University St. Louis, MO.

Comrey, A. L. (2008). The Comrey Personality Scales. In G. J.Boyle, G. Matthews, & D. H. Saklowfske (Eds.), Sage handbookof personality theory and testing: Personality measurement andassessment, volume II (pp. 113–134). London: Sage.

Condon, D. M. (2014). An organizational framework for thepsychological individual differences: Integrating the affective,cognitive, and conative domains. PhD thesis, NorthwesternUniversity.

Condon, D. M. (2015). The many little items of “big five” measures:Hierarchy, heterarchy, and predictive utility in personalitystructure. Long Beach, California. Society of Personality andSocial Psychology.

Condon, D. M. & Revelle, W. (2014). The International CognitiveAbility Resource: Development and initial validation of apublic-domain measure. Intelligence, 43, 52–64.

101 / 101


Condon, D. M. & Revelle, W. (2015a). Selected ICAR data fromthe SAPA-Project: Development and initial validation of apublic-domain measure. Harvard Dataverse.

Condon, D. M. & Revelle, W. (2015b). Selected personality datafrom the SAPA-Project: 08dec2013 to 26jul2014. HarvardDataverse.

Condon, D. M. & Revelle, W. (2015c). Selected personality datafrom the SAPA-Project: On the structure of phrased self-reportitems. Journal of Open Psychology Data, 3(1).

Condon, D. M. & Revelle, W. (2016). Selected ICAR data from theSAPA-Project: Development and initial validation of apublic-domain measure. Journal of Open Psychology Data.

Costa, P. T. & McCrae, R. R. (1992). NEO PI-R professionalmanual. Odessa, FL: Psychological Assessment Resources,Inc.

Costa, P. T., McCrae, R. R., & Dye, D. A. (1991). Facet scales foragreeableness and conscientiousness: A revision of the neo

101 / 101


personality inventory. Personality and Individual Differences,12(9), 887–898.

Cureton, E. (1966). Corrected item-test correlations.Psychometrika, 31(1), 93–96.

Deary, I. (2008). Why do intelligent people live longer? Nature,456(7219), 175–176.

DeYoung, C. G., Peterson, J. B., & Higgins, D. M. (2002).Higher-order factors of the big five predict conformity: Are thereneuroses of health? Personality and Individual Differences,33(4), 533–552.

DeYoung, C. G., Quilty, L. C., & Peterson, J. B. (2007). Betweenfacets and domains: 10 aspects of the big five. Journal ofPersonality and Social Psychology, 93(5), 880–896.

Digman, J. M. (1990). Personality structure: Emergence of thefive-factor model. Annual Review of Psychology, 41, 417–440.

Digman, J. M. (1997). Higher-order factors of the big five. Journalof Personality and Social Psychology, 73, 1246–1256.

101 / 101


Dwyer, P. S. (1937). The determination of the factor loadings of agiven test from the known factor loadings of other tests.Psychometrika, 2(3), 173–178.

Eysenck, H. J. (1994). The big five or the giant three: Criteria for aparadigm. In C. F. Halverson, G. A. Kohnstamm, & R. P. Martin(Eds.), The developing structure of temperament and personalityfrom infancy to adulthood (pp. 37–51). Hillsdale, N.J.: LawrenceErlbaum Associates.

Eysenck, S. B. G., Eysenck, H. J., & Barrett, P. (1985). A revisedversion of the psychoticism scale. Personality and IndividualDifferences, 6(1), 21 – 29.

French, J. A. & Condon, D. M. (2015). SAPA Tools: Tools toanalyze the SAPA Project. R package version 0.1.

Gilbert, D. T., King, G., Pettigrew, S., & Wilson, T. D. (2016).Comment on “estimating the reproducibility of psychologicalscience”. Science, 351(6277), 1037.

101 / 101


Goldberg, L. R. (1990). An alternative “description of personality”:The big-five factor structure. Journal of Personality and SocialPsychology, 59(6), 1216–1229.

Goldberg, L. R. (1992). The development of markers for thebig-five factor structure. Psychological Assessment, 4(1), 26–42.

Goldberg, L. R. (1999). A broad-bandwidth, public domain,personality inventory measuring the lower-level facets of severalfive-factor models. In I. Mervielde, I. Deary, F. De Fruyt, &F. Ostendorf (Eds.), Personality psychology in Europe, volume 7(pp. 7–28). Tilburg, The Netherlands: Tilburg University Press.

Goldberg, L. R. (2010). Personality, demographics and selfreported acts: the development of avocational interest scalesfrom estimates of the mount time spent in interest relatedactivities. In C. Agnew, D. Carlston, W. Graziano, & J. Kelly(Eds.), Then a miracle occurs: Focusing on the behavior insocial psychological theory and research (pp. 205–226). NewYork, NY: Oxford University Press.

101 / 101


Gottfredson, L. S. (1997). Why g matters: The complexity ofeveryday life. Intelligence, 24(1), 79 – 132.

Gough, H. G. & Bradley, P. (1996). Cpi manual . palo alto.

Hayes, K. (1962). Genes, drives, and intellect. PsychologicalReports (Monograph Supplement 2), 10, 299–342.

Hendriks, A. A. J. (1997). The construction of the five-factorpersonality inventory (FFPI). PhD thesis, RijksunivsiteitGroningen, Groningen, The Netherlands.

Hendriks, A. A. J., Hofstee, W. K., & De Raad, B. (1999). Thefive-factor personality inventory (FFPI). Personality andIndividual Differences, 27 (2), 307 – 325.

Hofstee, W. K., de Raad, B., & Goldberg, L. R. (1992). Integrationof the big five and circumplex approaches to trait structure.Journal of Personality and Social Psychology, 63(1), 146–163.

Hogan, R. & Hogan, J. (1995). The Hogan personality inventorymanual (2nd. ed.). Tulsa, OK: Hogan Assessment Systems.

101 / 101


Horn, J. L. (1973). On extension analysis and its relation tocorrelations between variables and factor scores. MultivariateBehavioral Research, 8(4), 477 – 489.

Jackson, D. N. (1983). JPI-R: Jackson Personality Inventory,Revised. Sigma Assesment Systems, Incorporated.

Jackson, D. N., Paunonen, S. V., & Tremblay, P. F. (2000). SixFactor Personality Questionnaire. Port Huron, MI: SigmaAssesment Systems.

Johnson, W. (2010). Extending and testing tom bouchard’sexperience producing drive theory. Personality and IndividualDifferences, 49(4), 296 – 301. Collected works from theFestschrift for Tom Bouchard, June 2009: A tribute to a vibrantscientific career.

Kelly, E. L. & Fiske, D. W. (1950). The prediction of success in theVA training program in clinical psychology. AmericanPsychologist, 5(8), 395 – 406.

101 / 101


Kelly, E. L. & Fiske, D. W. (1951). The prediction of performance inclinical psychology. Ann Arbor, Michigan: University of MichiganPress.

Kievit, R. A., Frankenhuis, W. E., Waldorp, L. J., & Borsboom, D.(2013). Simpson’s paradox in psychological science: a practicalguide. Frontiers in Psychology, 4(513), 1–14.

Lee, K. & Ashton, M. C. (2004). Psychometric properties of theHEXACO personality inventory. Multivariate BehavioralResearch, 39(2), 329–358.

Liebert, M. (2006). A public-domain assessment of musicpreferences as a function of personality and general intelligence.Honors Thesis. Department of Psychology, NorthwesternUniversity.

Lord, F. M. (1955). Estimating test reliability. Educational andPsychological Measurement, 15, 325–336.

Mehl, M. R., Pennebaker, J. W., Crow, D. M., Dabbs, J., & Price,J. H. (2001). The electronically activated recorder (ear): A

101 / 101


device for sampling naturalistic daily activities andconversations. Behavior Research Methods, Instruments, &Computers, 33(4), 517–523.

Mehl, M. R. & Robbins, M. L. (2012). Naturalistic observationsampling: The electronically activated recorder (ear). In M. R.Mehl & T. S. Conner (Eds.), Handbook of research methods forstudying daily life. New York, NY: Guilford Press.

Mehl, M. R., Vazire, S., Ramirez-Esparza, N., Slatcher, R. B., &Pennebaker, J. W. (2007). Are women really more talkative thanmen? Science, 317 (5834), 82.

Mottus, R. (2016). Towards more rigorous personalitytrait-outcome research. European Journal of Personality.

Norman, W. T. (1963). Toward an adequate taxonomy ofpersonality attributes: Replicated factors structure in peernomination personality ratings. Journal of Abnormal and SocialPsychology, 66(6), 574–583.

101 / 101


ottus, R. M., Kandler, C., Bleidorn, W., Riemann, R., & McCrae,R. R. (2016). Personality traits below facets: The consensualvalidity, longitudinal stability, heritability, and utility of personalitynuances. Journal of Personality and Social Psychology, (inpress).

Pozzebon, J. A., Visser, B. A., Ashton, M. C., Lee, K., & Goldberg,L. R. (2010). Psychometric characteristics of a public-domainself-report measure of vocational interests: The oregonvocational interest scales. Journal of Personality Assessment,92(2), 168–174.

R Core Team (2015). R: A Language and Environment forStatistical Computing. Vienna, Austria: R Foundation forStatistical Computing.

Rafaeli, E., Rogers, G. M., & Revelle, W. (2007). Affectivesynchrony: Individual differences in mixed emotions. Personalityand Social Psychology Bulletin, 33(7), 915–932.

Revelle, W. (2015). psych: Procedures for Personality and101 / 101


Psychological Research.http://cran.r-project.org/web/packages/psych/: NorthwesternUniversity, Evanston. R package version 1.5.8.

Revelle, W. & Condon, D. M. (2012). Multilevel analysis ofpersonality: Personality of college majors. Presented at theannual meeting of the Society of Multivariate ExperimentalPsychology.

Revelle, W. & Condon, D. M. (2015a). Ability, temperament, andinterests: their joint predictive power for job choice. InternationalSociety for the Study of Intelligence. Albuquerque, New Mexico.

Revelle, W. & Condon, D. M. (2015b). A model for personality atthree levels. Journal of Research in Personality, 56, 70–81.

Revelle, W., Condon, D. M., Wilt, J., French, J. A., Brown, A., &Elleman, L. G. (2016). Web and phone based data collectionusing planned missing designs. In N. G. Fielding, R. M. Lee, &G. Blank (Eds.), SAGE Handbook of Online Research Methodschapter 37: Mobile Methods. Sage Publications, Inc. (in press).

101 / 101


Revelle, W. & Wilt, J. (2015). The data box and within subjectanalyses: A comment on Nesselroade and Molenaar.Multivariate Behavioral Research (in press).

Revelle, W., Wilt, J., & Condon, D. (2011). Individual differencesand differential psychology: A brief history and prospect. InT. Chamorro-Premuzic, A. Furnham, & S. von Stumm (Eds.),Handbook of Individual Differences chapter 1, (pp. 3–38).Oxford: Wiley-Blackwell.

Revelle, W., Wilt, J., & Rosenthal, A. (2010). Individual differencesin cognition: New methods for examining thepersonality-cognition link. In A. Gruszka, G. Matthews, &B. Szymura (Eds.), Handbook of Individual Differences inCognition: Attention, Memory and Executive Control chapter 2,(pp. 27–49). New York, N.Y.: Springer.

Roberts, B. W., Kuncel, N. R., Shiner, R., Caspi, A., & Goldberg,L. R. (2007). The power of personality: The comparative validityof personality traits, socioeconomic status, and cognitive ability

101 / 101


for predicting important life outcomes. Perspectives onPsychological Science, 2(4), 313–345.

Rounds, J., Su, R., Lewis, P., & Rivkin, D. (2010). O* NET R©interest profiler short form psychometric characteristics:Summary.

Shipley, W. C. (2009). Shipley-2: manual. Western PsychologicalServices.

Simpson, E. H. (1951). The interpretation of interaction incontingency tables. Journal of the Royal Statistical Society.Series B (Methodological), 13(2), 238–241.

Tellegen, A. & Waller, N. G. (2008). Exploring personality throughtest construction: Development of the multidimensionalpersonality questionnaire. The Sage handbook of personalitytheory and assessment, 2, 261–292.

Tupes, E. C. & Christal, R. E. (1961). Recurrent personality factorsbased on trait ratings. Technical Report 61-97, USAF ASD

101 / 101


Technical Report, Lackland Air Force Base (reprinted in Journalof Personality (1992) 60: 225–251).

Wilt, J. (2014). A new form and function for personality. PhDthesis, Northwestern University.

Wilt, J., Bleidorn, W., & Revelle, W. (2016). Finding a life worthliving: Meaning in life and graduation from college. EuropeanJournal of Personality, (in press).

Wilt, J., Funkhouser, K., & Revelle, W. (2011). The dynamicrelationships of affective synchrony to perceptions of situations.Journal of Research in Personality, 45, 309–321.

Wilt, J. & Revelle, W. (2015). Affect, behaviour, cognition anddesire in the big five: An analysis of item content and structure.European Journal of Personality, 29(4), 478–497.

Yule, G. U. (1903). Notes on the theory of association of attributesin statistics. Biometrika, 2(2), 121–134.

101 / 101

personality at three levels of analysis

Documents