using 360 degree peer review to validate self‐reporting in human capital measurement
TRANSCRIPT
Using 360 degree peer review tovalidate self-reporting in human
capital measurementPeter Massingham and Thi Nguyet Que Nguyen
Centre for Knowledge Management, University of Wollongong, Wollongong,Australia, and
Rada MassinghamSchool of Accounting, University of Western Sydney, Penrith, Australia
Abstract
Purpose – The purpose of this paper is to address the subjectivity inherent in existing methods ofhuman capital value measurement (HCVM) by proposing a 360-degree peer review as a method ofvalidating self-reporting in HCVM surveys.
Design/methodology/approach – The case study is based on a survey of a section of the RoyalAustralian Navy. The sample was 118 respondents, who were mainly engineering and technicalworkers, and included both civilian and uniform.
Findings – The research may be summarised in three main findings. First, it confirms previousresearch demonstrating that correlations between self- and other-ratings tend to be low. However,while previous research has found that self-rating tends to be higher than other-rating, it wasfound to be the opposite: other-rating was higher than self-rating. Second, personality is discountedas an influencing variable in self-rating of knowledge. Third, there are patterns in the size of thediscrepancy by knowledge dimension (i.e. employee capability, employee sustainability) that allowgeneralisation about the adjustment necessary to find an accurate self-other rating of knowledge.
Research limitations/implications – The findings are based on a single case study and aretherefore an exercise in theory development rather than theory testing.
Practical implications – The 360-degree peer review rating of knowledge has considerableapplication. First, use the outcomes in the way 360-degree feedback has been traditionally used; i.e.identifying training needs assessment, job analysis, performance appraisal, or managerial andleadership development. Second, use it for performance appraisal – given the method’s capacity toidentify issues at a very finite level: e.g. are you building effective relationships with customers?Third, identify knowledge gaps, at a strategic level, for recruitment and development targets.Finally, in terms of financial decisions investors might be able to compare knowledge scores byorganization.
Originality/value – Traditionally, researchers and practitioners have used other-ratings as a tool foridentifying training and development needs. In this paper, other-ratings have been introduced as amethod for validating self-rating in the measurement of knowledge. The objective was to address oneof the weaknesses in existing methods – subjectivity. The solution to this problem was to use threedata points – self-reporting, 360-degree peer review, and personality ratings – to validate themeasurement of individuals’ human capital. This triangulation method aims to introduce objectivity tosurvey methods, making it a value measurement rather than value assessment.
Keywords Human capital, Intellectual capital, Management information, Intellectual property,Human resource accounting
Paper type Case study
The current issue and full text archive of this journal is available at
www.emeraldinsight.com/1469-1930.htm
Using 360 degreepeer review
43
Journal of Intellectual CapitalVol. 12 No. 1, 2011
pp. 43-74q Emerald Group Publishing Limited
1469-1930DOI 10.1108/14691931111097917
IntroductionWhile there is now widespread acceptance that knowledge has economic value for thefirm (Pike et al., 2002), there is little consensus on how to measure this. Previous literaturereviews identified a wide range of methods: Sveiby (1997) found 21 models, andAndriessen (2004) found 30 models. In the authors’ review for this paper, they identified46 different models. We need to consider why there is such an interest in knowledgevalue measurement (KVM) and then why no method has been widely accepted.
From an academic perspective, interest in KVM has increased since strategicmanagement theorists argued that knowledge is now the firm’s most importantstrategic asset (Grant, 1996), that knowledge resources contribute to competitiveadvantage (Teece, 1998), and may be converted to organizational competencies (e.g.Mowery et al., 1996). Intellectual capital (IC) theory is most commonly used as thestarting point in measuring knowledge value (Edvinsson and Malone, 1997; Sveiby,1997; Stewart, 1998; Bontis, 1998. IC comprises three dimensions: human capital (HC),structural capital, and relational capital (Edvinsson and Malone, 1997). Researchersargue that HC is the firm’s most important asset because it is the source of creativityand, therefore, innovation, change, and improvement (Bozbura, 2004; Carson et al.,2004). HC represents the human factor in the organization: the combined intelligence,skills, and expertise that give the organization its distinctive character (Bontis, 1998).Therefore, the authors focus on measuring the value of HC. Interest in measuring thevalue of IC is multi-disciplinary and has been investigated from various perspectivesincluding economic (Augier and Teece, 2005), strategic (Marr and Roos, 2005),accounting (Lev et al., 2005), financial (Sudarsanam et al., 2005), reporting (McEneryand Blanchard, 1999), marketing (Fernstrom, 2005), human resources ( Johanson, 2005),information systems (Peppard, 2005), legal (Cloutier and Gold, 2005) and intellectualproperty (Sullivan, 2005).
From a practitioner perspective, the knowledge-based economy places theimportance for creating economic value with knowledge. The main change in thetransition from industrial age (twentieth century) to the knowledge economy(twenty-first century) has been the way investors assign value to firms. In theindustrial age, greater value was placed on the ownership of tangible assets. In theknowledge economy, investors estimate the future value of a firm’s present knowledgeand knowledge-generating capacity (Housel and Bell, 2001). Bozbura (2004) providedempirical evidence that there is a positive relationship between the HC and thebook/market value of the firm. This research found that increases in employeecapabilities are seen to directly influence financial results, leading to a directrelationship between HC and organization performance (Bozbura, 2004). The economicvalue of knowledge, therefore, may be described as knowledge capital and defined asthe value that a customer or potential buyer places on a firm over and above its bookvalue (Marsick and Watkins, 2003, p. 137). This can be thought of as the value of thefirm’s knowledge. Financial statements still focus on tangible assets, and accountantsand financers have not yet come to terms with how to measure intangible assets, suchas knowledge. Given this context, investors, management, customers, and regulatorshave a need for knowledge value metrics that are reliable and acceptable to thecertifying bodies that have traditionally supplied financial data (Housel and Bell, 2001,p. 77). As Bontis (1998, p. 63) points out – therein lies one of the greatest challengesfacing business leaders and academics today and tomorrow: how to value knowledge?
JIC12,1
44
Measurement theory suggests that a suitable method for valuing HC (HC) must beobservable, measurable, auditable, and objective (Pike et al., 2002, p. 660). The mainweakness in existing methods is their subjectivity, which downgrades most of theapproaches to value assessment rather than value measurement (Andriessen, 2004).This article proposes 360-degree peer review as a method to validate self-reporting inHC value measurement. In this way, the authors develop a method that aims to reducethe subjectivity inherent in existing methods, leading towards a value measurement ofindividuals that may be used by investors and management as a reliable and auditablemeasure of IC. In measuring tacit knowledge, the authors have developed aquestionnaire based on survey instruments from the IC literature and psychometricliterature. However, using a questionnaire to measure tacit knowledge raises theproblem of self-reporting/evaluation. There is, of course, a respondent bias inself-evaluation, particularly in such an emotive subject as valuing your ownknowledge. Self-reporting, therefore, is a major problem for questionnaire-basedmeasurements of HC, and reflects the problem of subjectivity in existing methods.
The purpose of the paper is, therefore, to examine whether using three data points –self-reporting, 360-degree peer review, and personality ratings – may validate themeasurement of individuals’ HC. This triangulation method aims to introduceobjectivity making it a value measurement rather than value assessment (seeAndriessen, 2004). Based on a case study with the Australian Department of Defence,the authors look first at the differences between self and other-rating to determinewhether 360-degree peer review can validate self-reporting in terms of HC evaluation.The authors then look at whether 360-degree peer review ratings may be used to adjustself-ratings to provide a more accurate HC measurement. Finally, the authors look atwhether personality may be used as a further adjustment of self-ratings. Differencesbetween the perceptions of self-raters and those of other observers are often found inthe literature (Nilsen and Campell, 1993). This is problematic and raises severalresearch questions. First, is there significant discrepancy between self andother-ratings in measuring HC value? Second, if so can a contextual factor such aspersonality be used to explain this discrepancy? Third, if there is significantdiscrepancy between self-other evaluations, how may the correct HC value of anindividual be determined? The study attempts to answer these questions.
Literature reviewCriterion for measuring HC valueThe knowledge economy has created two main challenges in the measurement of firmperformance and market value:
(1) people are now considered assets with an indeterminable value as far asstandard accounting is concerned; and
(2) the knowledge embodied into firm processes has a value but when traded thevalue is dependent on the context of use of the buyer (i.e. new owner of the firm)which varies from buyer to buyer (Pike et al., 2002, p. 659).
It is difficult to measure the value of people and the processes that make a firmsuccessful. Indeed, Bontis (1998) suggests that a formula for determining how much ICis worth may never exist.
Using 360 degreepeer review
45
What criterion may be used to evaluate methods for measuring HC value andovercome the challenges listed above? The answer lies in the reasons explaining whymeasurement is necessary. Managers need to manage knowledge resources effectivelyand increase the performance and competitive position of the firm via decisionsregarding HC, and to communicate the performance, position, and potential of the firmexternally to shareholders and the wider investment community (Pike et al., 2002,pp. 659-60). Andriessen (2004) argues that measuring knowledge would serve thefollowing purposes:
. Internal management. Inform managerial decision making.
. External reporting. Inform investors about firm performance and growth.
. Regulatory reporting. Meet legal requirements for transparency, ethics, andregulation.
. Increase IC.
Following on from this, Pike et al. (2002, p. 660) suggest that suitable criteria to assessmethods for measuring HC value are that:
. It is auditable and reliable.
. It does not impose a large measurement overhead.
. It facilitates strategic and tactical management.
. It generates the information needed by shareholders and investors.
Evaluation of methods needs to be grounded in measurement theory. Pike et al. (2002,p. 660) identify six key requirements for compliance with measurement theory:
(1) complete in coverage;
(2) distinct and free from overlaps;
(3) preference independent with respect to one another;
(4) observable;
(5) measurable; and
(6) agreeable in that they are an agreed measure of the attribute.
Andriessen (2004) summarises measurement theory’s contribution by explaining that amethod is necessary that measures knowledge that is useful (desirable) based in anon-monetary criterion which can be translated in observable criteria (objective,testable and provable). Figure 1 explains this view.
The key to this framework is the distinction between assessment and measurement.Measures of HC value which are subjective are classified as merely assessments;whereas objectivity provides value measurement. From the authors’ perspective, thedevelopment of an observable and objective measure of HC value is the most importantcriteria for developing an agreeable model. Objectivity would ensure that managersand external stakeholders could trust the value measurements. An ideal model wouldallow a scenario where auditing firm x would evaluate the value of individual a andproduce a measurement (e.g. a point score) that would be replicated by auditing firm yif they followed the same method.
JIC12,1
46
The importance of objectivity is highlighted further when the history of HC valuemeasurement is considered. Bontis (1999) explained that current attempts to measureIC flow on from work in the field of human resources accounting (HRA), which evolvedfrom earlier attempts to measure employees as assets. In a review of this literature,Roslender (2000) explained that the field began in the mid-1960s with Hermanson’s(1964) finance-oriented human asset accounting, this was followed in the late 1960swith Flamholtz (1973) who introduced the first management-oriented human resourceaccounting system. This was concerned with human resource cost and revenueinformation for the management processes of control, planning and decision. This wasthen followed by the sociology-oriented human worth accounting, which was basedgreatly on a range of subjective employee worth assessments, and on softermeasurement metrics such as retention rates. Finally, this led to the current view,which is referred to as human competence accounting.
HRA models have two further important advantages. They can be calculated asassets in financial terms (Harrison and Sullivan, 2000). It is reasonably easy to trace thecost of HC through salaries, training, and so on. Firms can also assess the “value” ofemployees through tools such as the Hay methodology, which is commonly used tocompare and determine salary scales, and as a way to identify high performance andappropriate rewards (Robinson and Kleiner, 1996).
For an extensive review of HRA and the various attempts to apply the models, seeSackmann et al. (1989). HRA has been around a long time, well before the boom ininterest in IC measurement that began in the 1990s with Stewart (1995). It is alsointeresting to note that theoretical development in this area seemed to cease in themid-1970s. Despite numerous attempts to understand and measure the value of peopleto organizations (e.g. see Sackmann et al., 1989), HRA has never really caught on. Themain reason for this is that valuations of people using IC methods – i.e. an individual’scompetence or knowledge – are “soft’ measures rather than objective auditablenumbers (Bontis, 1999).
Figure 1.Framework for
discriminating betweendifferent types of
construct in intangibleasset assessment
Using 360 degreepeer review
47
The HRA method has three fundamental weaknesses. First, it is subjective and,therefore, a value assessment, rather than valuation measurement. Thus, even thoughHRA can yield a solid numerical figure of value of IC, the necessary estimations madewhile using HRA, can bring about inaccuracies that will lead to false value measures(Bontis et al., 1999). Second, HC is very difficult to define in a single framework,because it consists of a range of qualitative measures, including employees’ knowledgeacumination, leadership abilities, and risk-taking and problem-solving capabilities(Bozbura, 2004). For example, the large amount of assumptions such as the projectedcompany size in the future and the tenure per employee, turnover, and salary increaseson which the HRA models are based, is seen as a disadvantage (Bontis et al., 1999).Third, HRA values HC despite the fact that the firm does not explicitly own it ( Johnson,2002). Organizations do not own people and employees may leave at any time and taketheir knowledge with them. Therefore, it is considered unreasonable to list employeesin financial statements as firm assets.
This paper addresses the first weakness of HRA and the most important criteriaidentified by researchers in value measurement: subjectivity. In doing so, it proposes amethod for valuing employees’ knowledge based on empirical research with theAustralian Department of Defence.
Multiple-source feedback methodRatings by multiple sources, known as 360-degree feedback, have become afundamental tool in personnel and human resource management. It is mainly used inidentifying training needs assessment, job analysis, performance appraisal, ormanagerial and leadership development (Tornow, 1993; Yammarino and Atwater,1993). Traditional performance appraisal involves managers rating the jobperformance of subordinates/staff; 360-degree feedback extends the informationboundaries to include others who may be reasonably asked to comment on theperformance of the individual being rated. In addition to manager assessment, the360-degree method requires a self-assessment, assessment from below (where thesubject has subordinates/staff), from peers or co-workers, and from externalrelationships (e.g. customers/clients and consultants) (Church, 2000).
Largely due to its capacity to increase objectivity in performance and other appraisalof individuals, 360-degree feedback has grown in popularity. The traditional methodwhereby managers evaluate staff to provide feedback on performance for training, careerplanning, and work improvement is vulnerable to manager’s cognitive bias. The managermay not have full awareness of the staff member’s performance and other factors, such asthe manager’s personal relationship with the staff member, may influence ratings.Therefore, the concept of including others in the evaluation of an individual’s performancehas gained substantial traction amongst both academics and practitioners.
Despite this growing popularity, researchers have identified two concernsregarding 360-degree feedback. The first major concern involves how theinformation is used. Although considerable studies have found a relatively lowcorrelation between self-other ratings, it is suggested that those estimates can be moreuseful sources for training and development than for performance evaluation andadministrative decisions. The reason is that leniency and restriction of range may notbe the significant issues they appear to be under evaluation conditions (McEnery andBlanchard, 1999; Shore et al., 1992). Moreover, the multiple source feedback method has
JIC12,1
48
been primarily employed under the assumption that increasing one’s level of self-otheragreement will facilitate the implementation of the necessary changes (Goodstone andDiamante, 1998). This suggests that agreement over training and development issuesleads to organizational legitimacy (funding support) and participant acceptance(willingness to attend staff training).
While the effectiveness of multiple source rating methods for development purposesis strongly supported in the literature (Hazucha et al., 1993; Reilly et al., 1996), researchto date has not used this method for measuring HC value. The authors argue thatmeasuring HC value fits with the use of 360-degree feedback methods, as the purpose isto identify the needs for HC training and development and tracking changes based on agap found between the actual and desired HC value for each job position. Inaccurateevaluations can misdiagnose an individual’s strengths and weaknesses, leading toineffective development and management decisions of this intangible asset.
The second major concern in the literature of multisource feedback systems is thedegree of inter-rater agreement (Furnham and Stringfield, 1994), which typicallydetermines the accuracy of self and other estimates (Yammarino and Atwater, 1993).
Measurement of self-other agreementThere has existed some debate over the most appropriate way by which self-otheragreement is determined (Halverson et al., 2002; Newhouse, 2008). Different approachesinclude the assessment of measurement equivalence, the use of congruence indices andcategories of agreement, as well as polynomial and multivariate regression procedures.Newhouse (2008) argues that although the measurement equivalence of the ratinginstrument across rating sources is necessary in multisource feedback, this approachdoes not directly assess self-other agreement.
This research examines the self-other agreement in using 360-degree feedback tomeasure a HC value and the effects of personal attributes on the agreement.Agreement, regardless of the source of other-ratings, functions as the dependent oroutcome variable and thus, the algebraic difference score approach was consideredappropriate and chosen to operationalise self-other agreement in the current study. Thescores computed by subtracting the self-ratings from the mean other-ratings for eachdimension will be either zero or have a sign, positive or negative (Newhouse, 2008).Zero values indicate that self-ratings are the same with the ratings of the relevantothers. While positive values indicate that the others give the focal individual higherratings than the focal individual gives him/herself, negative values indicate that thefocal individual inflate his/her ratings compared to their other-ratings.
There have been mixed results on the generalisation of self-observer agreement ordiscrepancies across dimensions. Some research found different agreement ordiscrepancies depending on the type of dimensions being rated (Brooks andStuhlmacher, 1999; Kulas and Finkelstein, 2007) as well as the level of difficulty toobserve of rated characteristics (Wohlers and London, 1989). Meanwhile, other studiesfound a relative consistency of self-observer discrepancies across ratings of managerialskills and personality traits (Nilsen and Campell, 1993) and managerial performance(Furnham and Stringfield, 1994). Given that the dimensions of interest measuredifferent aspects of knowledge resource value and the nature of this intangible asset ishard to observe and evaluate, the stability of self-observer agreement acrossdimensions is not expected in the current study.
Using 360 degreepeer review
49
Antecedents of self-other agreementResearchers in the field of multisource feedback systems have tried to identify thefactors which affect self-ratings and other-ratings and hence, self-other agreement(Newhouse, 2008; Yammarino and Atwater, 1993). Atwater and Yammarino (1997)argue that accurate self-perception, self-rating accuracy, and other ratings accuracy arekeys to ensuring self-other agreement. Specifically, their model specifies five potentialinfluences on self-ratings, including personal attributes, biographical characteristics,job-relevant experiences, cognitive processes, and context. Numerous studies haveconcentrated on the effect of the ratee’s personality on the ratee’s and other’sperceptions, leading to accurate or inaccurate self and other ratings, which in turninfluences self-other ratings agreement (Brutus et al., 1999), Gaylord (2003), Goffin andAnderson (2007), Newhouse (2008), Sinha (2003) and Vecchio and Anderson (2009). Inthis paper, the authors examine whether personality affects self-otheragreement/difference and, if so, whether self-ratings may be adjusted to produce amore accurate self-evaluation of knowledge.
Among a variety of personality measurement scales, the Big Five factor model, aglobal measure of personality, has been extensively validated and used in theprediction of job performance and other organizational criteria (Barrick and Mount,1991; Digman, 1997; Hogan et al., 1996; Tett et al., 1991). However, the role of Big Fivefactors within the research examining feedback systems is rarely discussed (Gaylord,2003). For example, to investigate personality variables as predictors of congruence inmulti-source ratings, Brutus et al. (1999) use the California Psychological Inventory(CPI); Atwater and Yammarino (1992), Fletcher and Baldry (2000) and Newhouse (2008)employ the 16 Personality Factors Test (16PF); Goffin and Anderson (2007) considerthe full set of 20 traits contained in the Personality Research Form (PRF) and the fullset of 15 traits contained in the Jackson Personality Inventory (JPI); Sinha (2003)examines the ASSESS expert system; and Vecchio and Anderson (2009) examine thetwo theoretically relevant personality subscales from the Leadership Circle Profile (i.e.social dominance and social sensitivity). Gaylord (2003) is one of very few studies thatemploy the Five Factor model of personality to examine the moderating effects ofpersonality between self-other rating differences and feedback reactions. The presentresearch, therefore, attempts to examine the ratee’s personal characteristics as a keyantecedent of self-other agreement in HC value ratings, using an IPIP version of theNEO PI-R known as the 100-item Big Five factor markers (Costa and McCrae, 1992;Goldberg, 1992).
The first Big Five factor, Neuroticism is defined by the characteristics of beinganxious, worried, angry, depressed, embarrassed, and insecure. Goffin and Anderson(2007) have shown that low scorers on anxiety tend to inflate self-ratings due to theirstrong sense of self-worth and self-confidence and thus, have positive judgements ofthemselves. In addition, they may ignore the effects of their behaviour on their peersand subordinates and the feedback they may receive from these others. The second BigFive factor, Extroversion is defined by the characteristics of being sociable, friendly,talkative, assertive, and expressive. One of the first studies to examine personalitycharacteristics as antecedents of self-other agreement found that Introverts ratedthemselves more similarly to subordinate ratings of leadership than Extroverts (Roushand Atwater, 1992). This is because Introverts may spend more time in self-reflectionand tend to think things through in a deliberate and objective manner, resulting in
JIC12,1
50
higher levels of self-other agreement than Extraverts (Atwater and Yammarino, 1997).The next Big Five factor, Conscientiousness is defined by the characteristics of beingorganized, systematic, responsible, thorough, hardworking, and achievement striving.Bernardin et al. (2000) hypothesised that conscientiousness would be negatively relatedto rating leniency since highly conscientious individuals set high performancestandards and naturally give their best efforts on task. In terms of the last two factors,Agreeableness (defined by the characteristics of being sympathetic, cooperative,trustful, flexible, and soft-hearted) and Openness to Experience (defined by thecharacteristics of being intellectual, creative, artistic, imaginative, and emotional),previous research has found no strongly persuasive conceptual links betweenself-raters’ scores on these attributes and self-other agreement or discrepancies(Newhouse, 2008).
Attribution theory might also help identify influencers on self and other-ratings ofHC value. Attribution theory examines the determinants of behaviour within anachievement-oriented model (e.g. see Gooding and Kinicki, 1995). It helps individualsunderstand “why” events, or situations, occur (Weiner, 2000). It suggests that to helpmake sense of their world, individuals attempt to uncover the causality of events.Individuals assess the outcomes of past behaviours and adapt strategies to increase theprobability of success in future endeavours, thus minimising the risk of failure (Belket al., 1981). As such, current behaviour is based on the causal attributions of pastevents (i.e. the application of new knowledge). Attribution theory has become widelyused by business researchers to enhance understanding of individual andorganizational behaviour (Schaffer, 2002). The theory might be considered in furtherresearch on HC value measurement, as its underlying foundation of motivation andemotion could influence self and other-ratings, particularly within the context of anachievement-oriented model such as HC value.
Data collection methodAn invitation and cover letter explaining the study and assuring confidentiality weresent via e-mail to all 178 engineering and technical staff at Navy Systems Branch(NAVSYS) of the Royal Australian Navy. Respondents were asked to complete andsubmit the surveys online. Among 124 returned responses, six cases were deleted dueto many missing values, leading to the final sample of 118 usable responses. Theaverage age of respondents was 42 with a range from 23 to 62 years old. The conditionsunder which the questionnaires were implemented were that the responses wereconfidential, participation was voluntary, but respondents were encouraged bymanagement to complete the survey. These guidelines were governed by theuniversity’s ethics committee and subject to approval of an ethic application.Respondents were allowed to save their survey, and were given three weeks tocomplete. The data reported in this study are part of a larger survey.
First, respondents were asked to evaluate their own HC value across 11 dimensions,including employee capability, employee sustainability, employee satisfaction,colleagues’ attitude, network structure, network quality, currency, usage,contribution, formal ties, and informal ties. Most constructs were drawn fromexisting Likert scales validated by the IC and psychometric literatures, where possible,while the measurement scales of some constructs were not available and were,therefore, newly developed specifically for this study by the authors. Given the main
Using 360 degreepeer review
51
focus of this study is on the method of validation, rather than the value measurementitself, the focus is on the former due to space constraints. Further details are availablefrom the authors on request.
In addition, the knowledge resource value of each staff was also evaluated by theirfive colleagues using a shorter version of the questionnaire used for self-reports. Thecolleagues might be superiors, subordinates and peers of a focal individual so thatratings could include the entire 360-degree range of feedback sources to that individual.Other-ratings on each dimension were computed by taking the average of all otherratings about a focal individual. Finally, respondents were asked to evaluate their ownpersonality characteristics across five ten-item factors on a five-point likert scale,including neuroticism, extroversion, openness to experience, agreeableness, andconscientiousness. This short version of personality inventory was adapted from Costaand McCrae (1992) and Goldberg (1992) that has been strongly validated in theliterature. Cronbach’s alpha was calculated and principal component analysis (PCA)followed by a varimax rotation where necessary was conducted to assess the validityand reliability of the measurement scales. Items with low item-total correlations (lessthan 0.3) and low factor loadings (less than 0.5) were deleted (Gerbing and Anderson,1988; Hair et al. 2006; Nunnally, 1978). All measurement scales were found to havesatisfactory coefficient alpha (ranging from 0.84 to 0.93). The PCA for compositevariables extracted only one underlying component with an eigenvalue greater than 1explaining from 46.73 per cent to 58.58 per cent of the total variance in the original setsof variables. Thus, the reliability and validity of the measurement scales wereconfirmed.
Using data collected from the two sets of questionnaire measuring individuals’knowledge resource value (by self and colleagues) respondents were then allocated twotypes of scores on a 100-point scale for each construct, namely self-rating scores andother-rating scores. The self-other difference scores were computed by subtracting theself-rating scores from the mean other-rating scores for each of the 11 dimensions. Thefollowing section examines the validity of self-ratings in relationship to those ofrelevant others and test the hypothesised effects of ratees’ personality characteristicson self-other agreement.
Data analysis resultsDifference between self and other ratingsFirst, the difference between self and other-ratings is examined to answer the firstresearch question, stating that whether or not a significant discrepancy exists betweenself and other- ratings in an individual’s HC value. Table I provides descriptivestatistics for the 11 dimensions of knowledge resource value. When the means of selfand other ratings throughout these dimensions is looked at, it is interesting that mostconstructs, except Colleague attitude, have lower self-ratings than other-ratings.
To test the validity of self and other ratings, inter-correlations between the two setsof scores on the 11 dimensions of knowledge resource value were calculated. Theresults are presented in Table II. The validity coefficients were significant in one-thirdof the cases, ranging from 0.21 to 0.41. These correlations were consistent with pastresearch demonstrating that correlations between self and other ratings tend to be low(Harris and Schaubroeck, 1988; Mount, 1984). Moreover, when comparing the meanratings between two rater groups using Levene’s test, this typical pattern was also
JIC12,1
52
confirmed. Specifically, there were significant differences between the mean scores ofself and other ratings on all 11 dimensions of knowledge resource value (Table II). Thismeans that on average, the ratings of an individual’s knowledge (here measured bycapital types and the underlying constructs) between self and other groups aresignificantly different. These results answered research question one.
Dimension n Minimum Maximum Mean Std deviation
Human capital dimensionsSelf employee capability 118 40.48 84.27 58.3041 7.17240Other employee capability 95 33.33 91.67 63.2977 12.88872Self employee sustainability 118 2.78 100.00 61.2994 20.43414Other employee sustainability 95 33.33 100.00 72.4548 14.45750Self employee satisfaction 118 37.22 90.23 65.3364 7.73834Other employee satisfaction 95 33.33 100.00 75.9242 13.59193
Social capital dimensionsSelf colleagues’ attitude 118 13.89 100.00 88.0650 13.41946Other colleagues’ attitude 95 16.67 100.00 63.6462 18.77894Self network structure 118 30.56 84.72 52.6660 10.22677Other network structure 94 33.33 100.00 71.3127 11.54370Self network quality 118 25.00 81.02 54.5355 11.54405Other network quality 95 25.00 100.00 72.4951 12.50735
Structural capital dimensionsSelf currency 118 22.22 83.33 56.4888 13.43630Other currency 94 33.33 100.00 73.8011 10.68800Self usage 118 11.11 97.22 32.0386 14.30609Other usage 94 16.67 100.00 52.9943 17.60095Self contribution 118 40.76 89.29 63.1253 9.82547Other contribution 95 33.33 100.00 73.3713 11.29666
Relational capital dimensionsSelf formal ties 118 10.00 70.95 26.5707 18.41207Other formal ties 95 16.67 100.00 64.6140 18.31610Self informal ties 118 5.16 72.86 27.8374 21.60228Other informal ties 95 16.67 86.67 58.1775 17.13030
Table I.Descriptive statistics for
HC value dimensions
Dimension Self mean Other mean t df p
Employee capability 58.3017 63.2977 23.880 94 0.000Employee sustainability 61.5789 72.4548 23.992 94 0.000Employee satisfaction 65.6055 75.9242 25.729 94 0.000Colleague attitude 88.7427 63.6462 10.515 94 0.000Network structure 52.8369 71.3127 212.421 93 0.000Network quality 55.7895 72.4951 211.853 94 0.000Currency 57.3760 73.8011 210.607 93 0.000Usage 33.0674 52.9943 28.804 93 0.000Contribution 63.7992 73.3713 27.018 94 0.000Formal ties 27.3372 64.6140 215.259 94 0.000Informal ties 28.8029 58.1775 210.265 94 0.000
Table II.Results of t-tests
comparing means of selfand other ratings
Using 360 degreepeer review
53
When the means of the 11 dimensions are compared, the HC dimensions – employeecapability, sustainability and satisfaction – have the lowest level of difference betweenself and other rating. This is an interesting finding. It tells us that there is lessdisagreement between individuals and others when rating HC compared to the othercapitals. It suggests that others can more accurately evaluate an individual’s HCcompared with other types of knowledge. The explanation for this finding lies with themeasurements defining each capital dimension.
The smallest difference between self-other rating is on employee capability. Employeecapability is measured by the following constructs: activity, qualifications, experience,skills, and knowledge. These constructs involve the type of questions typically asked injob interview or promotion situations: i.e. what work do you do, how long have you donethat, what has your performance been? This criterion is readily observed by people youwork with and this is reflected by the low difference between the self and other rating.
On the other hand, the highest difference between self-other rating is on relationalcapital – formal and informal ties – where the differences in the mean scores for selfand other ratings was very large: 38 per cent and 31 per cent, respectively. Formal tieswas measured by constructs such as number and importance of external contacts andfrequency of contact; while informal ties was measured by constructs such as depth ofrelationship with external contacts, nature of knowledge flows, and outcomes of theinteraction. These measurements are likely to be invisible to people you work with orat least very difficult to evaluate, which is reflected by the lack of agreement betweenthe self and other ratings.
The authors draw several conclusions from these findings. First, the accuracy ofother ratings will vary by type of knowledge, i.e. here measured by capital dimensions.Second, the more visible the knowledge construct is to others, the more accurate theother rating. Third, the accuracy of other ratings for the various capital dimensionsand their underlying constructs may be generalised as follows (based on differencesbetween self mean and other mean (see Table III).
Capital dimension Rank
Degree to which you can have confidence in the accuracy of other ratings by capital dimensionHuman capital 1stStructural capital 2ndSocial capital 3rdRelational capital 4th
Degree to which you can have confidence in the accuracy of other ratings by capital constructEmployee capability 1stContribution 2ndEmployee satisfaction 3rdEmployee sustainability 4thCurrency 5thNetwork quality 6thNetwork structure 7thUsage 8thColleagues’ attitude 9thInformal ties 10thFormal ties 11th
Table III.Self-other ratings –ranking by accuracy
JIC12,1
54
The purpose in deriving a ranking of accuracy in the other ratings is to provide ameasure of the face validity of the model by construct. From Table III, you can see thatothers have a reasonably accurate perception of their colleagues’ capability, whetherthey are willing to share their knowledge (contribution), how happy they are, andwhether they will stay; but they have less accurate perceptions about colleagues’ socialinteractions, particularly with people outside the organizational (relational capital).
There are also interesting patterns in the differences. Others tend to rate their peershigher than their self-rating in all areas except colleague attitude (a social capitaldimension). This means that people tend to under-rate themselves in most areas. Thetendency by others to over-rate the focal individuals increases in relation to thevisibility of the knowledge dimension. In other words, areas where peers have goodvisibility of respondents’ work, such as their capability (seen via their workperformance), contribution (seen via their reports, documents and other codified workevidence), and satisfaction (seen via their work attitude and behaviour) have muchcloser self and other-ratings; whereas areas where peers may have very littleawareness, such as colleagues’ attitude (how people really feel about others may behidden within their own minds), and informal ties (frequency of interaction and depthof relationship with people outside the organization) have bigger differences betweenself and other-ratings (Table IV). It might be that self and other-ratings may need to befurther disaggregated into attitude dimensions such as affective, cognitive andbehavioural, with the latter being more visible for other-ratings. This may be an areafor further research.
The effect of personality on self-other agreementIn this section, the impact of the ratee’s personality on self-rating and other-ratings andthe differences between self-other ratings on 11 HC value dimensions was examined toanswer research question two.
Table V displays the descriptive statistics for self-other difference scores across the11 dimensions. The mean of difference scores ranged from 225.09 to 37.27 and thesign of difference implied that either focal individuals underestimate their knowledgeresource value or colleagues overestimate the focal individuals’ HC value, except forthe colleague attitude dimension. The later assumption might present a leniency effecton other ratings that was extensively discussed in the multisource feedback literature.
Descriptive statistics for five factors of personality were calculated and aredisplayed in Table VI. The mean scores indicated that on average, the ratees scoredlower than the mid-point of the scale on neuroticism (2.1 compared with 2.5) but scoredhigher than that on other personality factors, including extroversion, openness toexperience, agreeableness, and conscientiousness (from 3.6 to 3.87 compared with 2.5,respectively).
In order to test the effects of ratees’ personality characteristics on self-otheragreement or discrepancies, Pearson correlation coefficients were calculated. Table VIIpresents the correlations between the five personality factors and self-ratings,other-ratings and self-other algebraic difference scores of 11 knowledge valuedimensions. Specifically, for each of the knowledge value dimensions, the first lineshows the correlations between the personality factors and self-ratings, the second lineshows the correlations between the personality factors and other-ratings, and the thirdline shows the correlations between the personality factors and self-other differences.
Using 360 degreepeer review
55
Sel
f–
oth
erco
nst
ruct
s1
23
45
67
89
1011
1PeerECap
Cor
rela
tion
coef
fici
ent
0.342
**
20.
140
20.
056
0.14
70.
132
0.374
**
0.37
8*
*0.206
*0.275
**
0.11
70.
080
Sig
.(t
wo-
tail
ed)
0.00
10.
175
0.58
80.
155
0.20
40.
000
0.00
00.
046
0.00
70.
259
0.44
3n
9595
9595
9595
9595
9595
95
2PeerESus
Cor
rela
tion
coef
fici
ent
20.
013
20.
114
20.
112
0.07
80.225
*0.400
**
0.278
**
0.12
50.202
*0.
151
0.13
9S
ig.
(tw
o-ta
iled
)0.
904
0.27
10.
279
0.45
30.
028
0.00
00.
006
0.22
60.
049
0.14
40.
178
n95
9595
9595
9595
9595
9595
3PeerESat
Cor
rela
tion
coef
fici
ent
0.12
72
0.07
32
0.14
40.
026
0.15
40.
218
*0.
276
**
0.03
40.
102
0.06
82
0.01
4S
ig.
(tw
o-ta
iled
)0.
221
0.47
90.
162
0.80
30.
136
0.03
40.
007
0.74
40.
326
0.51
00.
891
n95
9595
9595
9595
9595
9595
4PeerCAtt
Cor
rela
tion
coef
fici
ent
0.03
82
0.13
72
0.05
20.
021
0.289
**
0.303
**
0.273
**
0.17
50.217
*0.
071
0.08
4S
ig.
(tw
o-ta
iled
)0.
712
0.18
60.
614
0.84
30.
005
0.00
30.
007
0.09
00.
035
0.49
60.
418
n95
9595
9595
9595
9595
9595
5PeerNStr
Cor
rela
tion
coef
fici
ent
20.
008
20.
201
20.
197
0.10
90.
134
0.11
30.
247
*0.
195
0.10
10.
017
20.
014
Sig
.(t
wo-
tail
ed)
0.94
30.
052
0.05
70.
295
0.19
90.
279
0.01
70.
059
0.33
10.
874
0.89
0n
9494
9494
9494
9494
9494
94
6PeerNQua
Cor
rela
tion
coef
fici
ent
0.02
32
0.207
*2
0.15
60.
118
0.261
*0.360
**
0.260
*0.
119
0.213
*0.
076
0.06
6S
ig.
(tw
o-ta
iled
)0.
822
0.04
40.
132
0.25
30.
010
0.00
00.
011
0.24
90.
039
0.46
50.
522
n95
9595
9595
9595
9595
9595
7PeerCurr
Cor
rela
tion
coef
fici
ent
20.
009
20.314
**
20.213
*0.
050
0.17
90.290
**
0.247
*0.
044
0.05
30.
082
0.07
9S
ig.
(tw
o-ta
iled
)0.
935
0.00
20.
039
0.63
20.
084
0.00
50.
016
0.67
40.
615
0.42
90.
449
n94
9494
9494
9494
9494
9494
(continued
)
Table IV.Correlations between selfand other ratings on HCvalue dimensions
JIC12,1
56
Sel
f–
oth
erco
nst
ruct
s1
23
45
67
89
1011
8PeerUsa
Cor
rela
tion
coef
fici
ent
20.
067
20.
125
0.01
52
0.11
40.206
*0.228
*0.
062
0.17
20.
108
0.13
10.
076
Sig
.(t
wo-
tail
ed)
0.52
20.
229
0.89
00.
272
0.04
70.
027
0.55
00.
097
0.30
00.
209
0.46
7n
9494
9494
9494
9494
9494
94
9PeerCon
Cor
rela
tion
coef
fici
ent
0.02
22
0.12
42
0.06
20.
109
0.08
90.346
**
0.249
*0.
152
0.272
**
0.09
10.
021
Sig
.(t
wo-
tail
ed)
0.83
50.
231
0.54
80.
294
0.39
30.
001
0.01
50.
141
0.00
80.
382
0.83
9n
9595
9595
9595
9595
9595
95
10PeerFTie
Cor
rela
tion
coef
fici
ent
0.02
72
0.14
92
0.09
90.
065
0.251
*0.422
**
0.332
**
0.11
70.267
**
0.15
00.
096
Sig
.(t
wo-
tail
ed)
0.79
60.
149
0.33
90.
531
0.01
40.
000
0.00
10.
259
0.00
90.
148
0.35
6n
9595
9595
9595
9595
9595
95
11PeerInTie
Cor
rela
tion
coef
fici
ent
20.
020
20.
090
20.
200
20.
102
0.18
90.331
**
0.204
*2
0.04
40.
035
0.08
20.
068
Sig
.(t
wo-
tail
ed)
0.84
40.
385
0.05
20.
324
0.06
70.
001
0.04
70.
670
0.73
50.
427
0.51
3n
9595
9595
9595
9595
9595
95
Table IV.
Using 360 degreepeer review
57
These self-other differences are the algebraic difference scores between self-ratings andother-ratings for each of the knowledge value dimensions.
The results show no correlations between neuroticism with any of the differencescores and all correlation coefficients were very low (under 0.2). However, thecorrelations between neuroticism with self-ratings and other-ratings indicate that thereis a negative relationship between neuroticism and self-ratings throughout all HC valuedimensions. In other words, the higher neuroticism scores, the lower self-ratings.Meanwhile, the correlations between neuroticism and other-ratings are hardlysignificant. These results imply that if a sample has high proportions of highneuroticism, the self-rating scores would be expected to be lower than a sample withhigh proportions of low neuroticism.
The results show that extroversion was negatively correlated with only onedimension of HC value – currency – meaning that low scores on extroversion wererelated to high difference score, where high difference scores indicate discrepanciesbetween self and other ratings. However, the correlations between extroversion withself-ratings and other-ratings indicate a positive relationship between extroversion andself-ratings on eight HC value dimensions and a positive relationship betweenextroversion and other-ratings on six HC value dimensions. These results imply that ifa sample has high proportions of high extroversion, the self-rating and other-ratingscores would be expected to be higher than a sample with high proportions of lowextroversion. However, self-ratings were found to be more strongly related, withextroversion than other-ratings in five out of six pairs of correlations. in other words,the higher extroversion scores are more associated with higher self-ratings thanother-ratings.
Factor n Minimum Maximum Mean Std deviation
Neuroticism 118 1.00 4.00 2.10 0.61Extroversion 118 1.88 4.75 3.60 0.53Openness to experience 118 2.00 5.00 3.81 0.54Agreeableness 118 2.00 5.00 3.87 0.41Conscientiousness 118 2.00 5.00 3.87 0.44
Table VI.Descriptive statistics forBig Five factors ofpersonality
Dimension n Minimum Maximum Mean Std deviation
Employee capability 95 224.47 31.79 4.9959 12.55168Employee sustainability 95 263.89 65.28 10.8758 26.55273Employee satisfaction 95 256.89 62.78 10.3187 17.55574Colleague attitude 95 283.33 69.44 225.0965 23.26230Network structure 94 215.28 54.86 18.4758 14.42197Network quality 95 233.33 50.69 16.7056 13.73752Currency 94 230.09 47.92 16.4251 15.01378Usage 94 247.22 80.56 19.9270 21.94422Contribution 95 239.29 38.23 9.5722 13.29358Formal ties 95 219.00 81.67 37.2769 23.81105Informal ties 95 256.19 78.17 29.3747 27.89259
Table V.Descriptive statistics foralgebraic differencescores
JIC12,1
58
The results show no significant correlations between conscientiousness with any of thedifference scores and all correlation coefficients were very low (under 0.2). However,the correlations between conscientiousness with self-ratings and other-ratings indicatea positive relationship between conscientiousness and self-ratings on nine HC valuedimensions and a positive relationship between conscientiousness and other-ratings onfour HC value dimensions These results imply that if a sample has high proportions ofhigh conscientiousness, the self-rating and other-rating scores would be expected to behigher than a sample with high proportions of low conscientiousness. The level ofassociation between conscientiousness and self-ratings and between conscientiousnessand other-ratings is not clear as the former relation is stronger on two dimensionswhile the later is stronger on the other two dimensions.
In terms of the remaining personality factor, it is interesting that openness toexperience was found to be negatively correlated with two out of 11 HC value dimensions,indicating that high scores on openness to experience were related to low difference scoreswhere low difference scores referred to self-other ratings agreement. Moreover, the
Neuroticism Extroversion Openness to experience Agreeableness Conscientiousness
Self ECap 2 0.278 * * 0.006 0.281 * * 0.285 * * 0.291 * *
Peer ECap 2 0.219 * 0.208 * 0.176 0.234 * 0.298 * *
ECap Diff 20.033 0.143 20.087 0.022 0.079Self ESus 2 0.219 * 0.224 * 20.038 0.183 * 0.141Peer ESus 20.184 0.280 * * 0.058 0.224 * 0.240 *
ESus Diff 0.072 0.009 0.098 0.027 0.072Self ESat 20.157 0.279 * * 0.179 0.246 * * 0.246 * *
Peer ESat 20.198 0.151 20.015 0.231 * 0.349 * *
ESat Diff 20.072 20.034 20.123 0.029 0.137Self CAtt 2 0.243 * * 0.314 * * 0.297 * * 0.297 * * 0.329 * *
Peer CAtt 20.185 0.220 * 20.043 0.149 0.151Catt Diff 20.025 0.052 20.160 0.007 20.016Self NStr 20.160 0.264 * * 0.057 0.123 0.166Peer NStr 20.169 0.162 20.026 0.161 0.113NStr Diff 20.001 20.030 20.049 0.041 0.007Self NQua 2 0.259 * * 0.277 * * 0.199 * 0.236 * 0.259 * *
Peer NQua 20.130 0.222 * 0.019 0.203 * 0.252 *
NQua Diff 0.112 20.041 20.136 20.037 20.001Self Curr 2 0.322 * * 0.478 * * 0.308 * * 0.309 * * 0.384 * *
Peer Curr 20.173 0.207 * 0.110 0.124 0.287 * *
Curr Diff 0.180 2 0.281 * * 20.081 20.133 20.121Self Usa 2 0.243 * * 0.129 0.150 0.166 0.229 *
Peer Usa 20.135 0.128 20.149 20.091 20.049Usa Diff 0.008 0.060 2 0.241 * 20.128 20.185Self Con 2 0.478 * * 0.372 * * 0.307 * * 0.397 * * 0.449 * *
Peer Con 20.178 0.219 * 0.059 0.186 0.210 *
Con Diff 0.173 20.022 20.145 20.110 20.102Self FTie 2 0.187 * 0.269 * * 0.103 0.176 0.289 * *
Peer FTie 20.190 0.200 20.111 0.092 0.124FTie Diff 0.036 20.051 20.185 20.052 20.145Self InTie 2 0.238 * * 0.252 * * 0.239 * * 0.183 * 0.319 * *
Peer InTie 2 0.249 * 0.090 20.103 0.098 0.151InTie Diff 0.052 20.180 2 0.272 * * 20.054 20.178
Table VII.Correlations between
personality anddifference scores
Using 360 degreepeer review
59
correlations between openness to experience with self-ratings and other-ratings indicate apositive relationship between openness to experience and self-ratings on six HC valuedimensions while no significant relationship exists between openness to experience andother-ratings on any HC value dimensions. These results imply that if a sample has highproportions of high openness to experience, the self-rating scores would be expected to behigher than a sample with high proportions of low openness to experience.
The correct HC valueAt the start of the paper, it was explained that the authors’ objective was to propose amethod for validating self-reporting when measuring the value of HC. This can be donein several ways. First, if the 360-degree peer review ratings were the same or very similarto self-reporting ratings, then the self-reporting ratings may be accepted. Second, if thereis a significant difference between the other-ratings and self-report, either can beaccepted depending upon which is believed more reliable or accurate. Third, theself-report ratings can be adjusted to take into account the difference between theself-report and the other-ratings. Fourth, if the self-report ratings need to be adjusted, cana moderating variable, such as personality profiles, improve the validity or accuracy ofthe adjusted figure? The authors look at each of these scenarios separately.
If self-rating and other-rating are similar?If the self-rating is similar to the other-rating, then the best decision is to accept theself-rating. In this case, the other-rating validates the self-rating. The results of the dataanalysis showed that there was a significant difference between self and other ratingson all 11 dimensions of knowledge resource value or HC. Thus the authors look at thesecond scenario.
Should we accept self-rating or other-rating?If there is a significant difference between self-rating and other-rating, should one beaccepted instead of the other? Which should be trusted? Is one more accurate than theother? The results of the study are somewhat inconclusive. In terms of trust, previousresearch suggests that due to the presence of a leniency or self-favouritism effect,self-ratings tend to be significantly greater than other-ratings. There is alsoself-interest when rating something as emotional as our knowledge, creating acognitive bias towards over-rating. However, the results contradict previous research,as self-ratings were actually lower than other-ratings. This suggests that theself-ratings should be trusted more than the other-ratings, given the cognitive bias ofover-rating does not appear in the study.
In terms of accuracy, the analysis is much clearer. It suggests that the self-ratingsshould be accepted when compared with other-ratings. The self-rating questionnairecompleted by respondents contained more than 1,000 questions and represents acomprehensive audit of HC. In contrast, the questionnaire completed by respondents torate their peers contained 18 questions, which summarised the 11 dimensions using aLikert scale. Given the rigor of the self-rating questionnaire, it is reasonable to accept thatself-rating is a more accurate indicator of capability than other-rating in the case study.
However, in the authors’ minds there was still concern that other-ratings weresignificantly different to self-ratings. This led to the next scenario: adjusting theself-ratings.
JIC12,1
60
How then should we adjust self-rating to take other-rating into account?There were several options that were explored. First, is there a clear pattern in thedifferences? Second, should a midpoint between the self-rating and other-rating betaken? The authors look at each of these options separately.
PatternsTables VIII and IX summarise the descriptive statistics of the knowledge differencescores for the 95 respondents using the difference value (positive when other rating islarger than self-rating and vice versa). Meanwhile, Tables X and XI summarise thedescriptive statistics of the knowledge difference scores for the 95 respondents usingthe difference percentage (positive when other rating is larger than self-rating and viceversa).
The results show that there are clear patterns in the difference between self-ratingand other-rating across dimensions of HC value. This is a significant finding whichmay be generalisable for other industry contexts and with managerial application. Inlooking for patterns, the first thing the authors checked was whether there weresignificant differences in the various columns. The authors then draw the followingconclusions:
. Others tend to significantly over-rate relational capital by a factor of at least 30per cent. The evidence for this is based on 74 per cent of respondents over-ratingformal ties by .30 per cent and 66 per cent of respondents over-rating informalties by .30 per cent. The authors then generalise these results by suggestingthat self-ratings in relational capital and its two dimensions – formal ties andinformal ties – may be increased by a factor of 30 per cent.
. Similar results for usage (a structural capital dimension), means self-ratings mayalso be increased in that area by a factor of 30 per cent.
. Other results are less conclusive but useful generalisations may still be derived.Network structure and network quality (social capital dimensions) and currency(a structural capital dimension) have less significant percentages than thedimensions already listed above (40 per cent, 27 per cent, and 35 per cent
Difference scores(out of 100)
None 0.1-5 5.1-10 10.1-20 20.1-30 .30No. % No. % No. % No. % No. % No. %
Employee capability 0 0 13 14 12 13 24 25 10 11 2 2E sustainability 1 1 5 5 6 6 20 21 8 8 23 24Employee satisfaction 0 0 10 11 14 15 28 29 15 16 8 8Colleague attitude 1 1 3 3 2 2 4 4 0 0 1 1Network structure 2 2 7 7 8 8 26 27 23 24 20 21Network quality 1 1 4 4 10 11 37 39 16 17 17 18Currency 1 1 6 6 7 7 27 28 27 28 15 16Usage 1 1 8 8 4 4 17 18 11 12 37 39Contribution 0 0 5 5 17 18 34 36 12 13 4 4Formal ties 0 0 1 1 4 4 14 15 15 16 55 58Informal ties 0 0 3 3 4 4 13 14 16 17 44 46
Table VIII.Descriptive statistics of
self-other differences(positive
value ¼ other . self)
Using 360 degreepeer review
61
Dif
fere
nce
scor
es(o
ut
of10
0)2
5!
20.
12
10!
25.
12
20!
210
.12
30!
220
.1,2
30N
o.%
No.
%N
o.%
No.
%N
o.%
Em
plo
yee
cap
abil
ity
1819
66
66
44
00
Em
plo
yee
sust
ain
abil
ity
66
77
1011
33
66
Em
plo
yee
sati
sfac
tion
66
44
66
11
33
Col
leag
ue
atti
tud
e5
55
521
2220
2133
35N
etw
ork
stru
ctu
re2
22
24
40
00
0N
etw
ork
qu
alit
y4
43
32
20
01
1C
urr
ency
33
22
44
11
11
Usa
ge
44
33
66
22
11
Con
trib
uti
on8
88
86
60
01
1F
orm
alti
es2
22
22
20
00
0In
form
alti
es4
43
35
50
03
3
Table IX.Descriptive statistics ofself-other differences(negativevalue ¼ other , self)
JIC12,1
62
respectively in terms of over-rating by 30 per cent or more). However, when thefinal columns are combined to identify proportions who have over-rated by .20per cent, the figures are significant. Network structure then has 61 per cent,network quality has 58 per cent, and currency has 56 per cent of respondentsover-rating by .20 per cent. The authors then generalise these results bysuggesting that self-ratings in network structure, network quality, and currencymay be increased by a factor of 20 per cent.
. For the remaining dimensions (except colleague attitude), the authors adopt aconservative approach – based on aggregating the differences until a significantproportion of the sample is reached – and suggest that self-ratings may beincreased by a factor of 10 per cent for employee capability, employeesustainability, employee satisfaction, and contribution. There may be a case toincrease this to 15 per cent for employee sustainability.
. Colleague attitude (a social capital dimension) is the only dimension where otherstend to significantly under-rate. 69 per cent of respondents under-rated by .20per cent. The authors then generalise this result by suggesting that self-ratingsin colleague attitude may be decreased by a factor of 20 per cent.
The findings can be aggregated by dimension into their respective knowledge capitaltypes (e.g. HC), and also the knowledge capital types into an overall HC value, toprovide similar guidelines at these higher levels.
Mid-pointAnother only option is to take the midpoint between self and other-ratings. Table XIIshows that the correlations between self-rating and mid-point rating and betweenother-rating and mid-point rating are highly significant, indicating a reasonableinternal consistency of mid-point ratings with self and other ratings. This suggeststhat the midpoint may be a suitable alternative, particularly if the general guidelinesabove are not adopted.
Difference scores(Percentage)
None 0.1-5 5.1-10 10.1-20 20.1-30 .30No. % No. % No. % No. % No. % No. %
Employee capability 0 0 8 8 9 9 25 26 13 14 6 6E sustainability 1 1 3 3 6 6 16 17 10 11 27 28Employee satisfaction 0 0 7 7 10 11 22 23 23 24 13 14Colleague attitude 1 1 1 1 4 4 4 4 0 0 1 1Network structure 2 2 4 4 4 4 18 19 20 21 38 40Network quality 1 1 1 1 4 4 24 25 29 31 26 27Currency 1 1 6 6 5 5 18 19 20 21 33 35Usage 1 1 2 2 6 6 3 3 4 4 62 65Contribution 0 0 4 4 4 4 31 33 20 21 13 14Formal ties 0 0 0 0 2 2 6 6 11 12 70 74Informal ties 0 0 2 2 3 3 4 4 8 8 63 66
Table X.Descriptive statistics of
self-other differences(positive
% ¼ other . self)
Using 360 degreepeer review
63
Dif
fere
nce
scor
es(P
erce
nta
ge)
25!
20.
12
10!
25.
12
20!
210
.12
30!
220
.1,2
30N
o.%
No.
%N
o.%
No.
%N
o.%
Em
plo
yee
cap
abil
ity
99
1011
44
44
77
Em
plo
yee
sust
ain
abil
ity
33
55
66
77
1112
Em
plo
yee
sati
sfac
tion
44
33
55
44
44
Col
leag
ue
atti
tud
e3
35
511
1212
1353
56N
etw
ork
stru
ctu
re1
12
22
21
12
2N
etw
ork
qu
alit
y3
32
22
22
21
1C
urr
ency
33
11
22
11
44
Usa
ge
00
22
44
11
99
Con
trib
uti
on7
72
29
92
23
3F
orm
alti
es1
11
11
11
12
2In
form
alti
es2
22
23
31
17
7
Table XI.Descriptive statistics ofself-other differences(negative% ¼ other , self)
JIC12,1
64
MP
emp
loy
eeca
pab
ilit
yM
Pem
plo
yee
sust
ain
abil
ity
MP
emp
loy
eesa
tisf
acti
on
MP
coll
eag
ue
atti
tud
e
MP
net
wor
kst
ruct
ure
MP
net
wor
kq
ual
ity
MP
curr
ency
MP
usa
ge
MP
con
trib
uti
on
MP
form
alti
e
MP
info
rmal
tie
Selfem
ployee
capability
Cor
rela
tion
coef
fici
ent
0.69
0*
*0.
100
0.21
7*
0.14
5-0
.010
0.02
60.
023
0.03
30.
033
0.04
70.
015
Sig
.(t
wo-
tail
ed)
0.00
00.
334
0.03
40.
160
0.92
60.
804
0.82
30.
752
0.75
20.
650
0.88
3n
9595
9595
9495
9495
9595
95
Peerem
ployee
capability
Cor
rela
tion
coef
fici
ent
0.89
5*
*0.
282
**
0.52
1*
*0.
687
**
0.41
9*
*0.
689
**
0.62
6*
*0.
528
**
0.52
8*
*0.
643
**
0.50
5*
*
Sig
.(t
wo-
tail
ed)
0.00
00.
006
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
0n
9595
9595
9495
9495
9595
95
Selfem
ployee
sustainability
Cor
rela
tion
coef
fici
ent
-0.0
290.
774
**
0.22
6*
0.01
7-0
.117
-0.1
41-0
.241
*-0
.087
-0.0
87-0
.060
-0.0
10S
ig.
(tw
o-ta
iled
)0.
782
0.00
00.
028
0.87
20.
260
0.17
30.
019
0.40
10.
401
0.56
50.
927
n95
9595
9594
9594
9595
9595
Peerem
ployee
sustainability
Cor
rela
tion
coef
fici
ent
0.50
0*
*0.
496
**
0.53
5*
*0.
594
**
0.59
6*
*0.
808
**
0.71
1*
*0.
651
**
0.65
1*
*0.
714
**
0.64
5*
*
Sig
.(t
wo-
tail
ed)
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
0n
9595
9595
9495
9495
9595
95
Selfem
ployee
satisfaction
Cor
rela
tion
coef
fici
ent
0.01
80.
379
**
0.40
1*
*0.
091
-0.1
53-0
.130
-0.2
02-0
.071
-0.0
71-0
.088
-0.1
52S
ig.
(tw
o-ta
iled
)0.
859
0.00
00.
000
0.37
80.
141
0.21
00.
051
0.49
60.
496
0.39
90.
141
n95
9595
9594
9594
9595
9595
Peerem
ployee
satisfaction
Cor
rela
tion
coef
fici
ent
0.47
1*
*0.
334
**
0.80
4*
*0.
608
**
0.44
0*
*0.
664
**
0.56
1*
*0.
499
**
0.49
9*
*0.
499
**
0.47
6*
*
Sig
.(t
wo-
tail
ed)
0.00
00.
001
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
0n
9595
9595
9495
9495
9595
95
Selfcolleagueattitude
Cor
rela
tion
coef
fici
ent
0.20
7*
0.34
5*
*0.
235
*0.
431
**
0.14
20.
171
0.09
80.
120
0.12
00.
108
-0.0
27S
ig.
(tw
o-ta
iled
)0.
044
0.00
10.
022
0.00
00.
172
0.09
80.
345
0.24
50.
245
0.29
90.
794
n95
9595
9594
9594
9595
9595
Peercolleagueattitude
Cor
rela
tion
coef
fici
ent
0.48
4*
*0.
297
**
0.63
1*
*0.
858
**
0.57
9*
*0.
793
**
0.51
8*
*0.
522
**
0.52
2*
*0.
720
**
0.55
7*
*
Sig
.(t
wo-
tail
ed)
0.00
00.
003
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
0n
9595
9595
9495
9495
9595
95(continued
)
Table XII.Correlations between
self-rating, other-ratingsand mid-point ratings
Using 360 degreepeer review
65
MP
emp
loy
eeca
pab
ilit
yM
Pem
plo
yee
sust
ain
abil
ity
MP
emp
loy
eesa
tisf
acti
on
MP
coll
eag
ue
atti
tud
e
MP
net
wor
kst
ruct
ure
MP
net
wor
kq
ual
ity
MP
curr
ency
MP
usa
ge
MP
con
trib
uti
on
MP
form
alti
e
MP
info
rmal
tie
Selfnetworkstructure
Cor
rela
tion
coef
fici
ent
0.09
30.
154
0.10
40.
365
**
0.15
80.
278
**
0.18
40.
090
0.09
00.
277
**
0.20
2*
Sig
.(t
wo-
tail
ed)
0.37
00.
135
0.31
50.
000
0.12
90.
006
0.07
60.
384
0.38
40.
007
0.05
0n
9595
9595
9495
9495
9595
95
Peernetworkstructure
Cor
rela
tion
coef
fici
ent
0.32
6*
*0.
189
0.40
0*
*0.
532
**
0.92
9*
*0.
614
**
0.50
3*
*0.
625
**
0.62
5*
*0.
517
**
0.35
4*
*
Sig
.(t
wo-
tail
ed)
0.00
10.
068
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
0n
9494
9494
9494
9494
9494
94
Selfnetworkquality
Cor
rela
tion
coef
fici
ent
0.34
5*
*0.
271
**
0.29
4*
*0.
361
**
0.10
30.
406
**
0.32
9*
*0.
353
**
0.35
3*
*0.
399
**
0.35
2*
*
Sig
.(t
wo-
tail
ed)
0.00
10.
008
0.00
40.
000
0.32
50.
000
0.00
10.
000
0.00
00.
000
0.00
0n
9595
9595
9495
9495
9595
95
Peernetworkquality
Cor
rela
tion
coef
fici
ent
0.49
3*
*0.
265
**
0.61
5*
*0.
729
**
0.51
4*
*0.
917
**
0.57
6*
*0.
647
**
0.64
7*
*0.
661
**
0.51
5*
*
Sig
.(t
wo-
tail
ed)
0.00
00.
010
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
0n
9595
9595
9495
9495
9595
95
Selfcurrency
Cor
rela
tion
coef
fici
ent
0.45
8*
*0.
337
**
0.39
3*
*0.
375
**
0.21
0*
0.25
4*
0.29
7*
*0.
233
*0.
233
*0.
304
**
0.22
2*
Sig
.(t
wo-
tail
ed)
0.00
00.
001
0.00
00.
000
0.04
20.
013
0.00
40.
023
0.02
30.
003
0.03
0n
9595
9595
9495
9495
9595
95
Peercurrency
Cor
rela
tion
coef
fici
ent
0.47
8*
*0.
121
0.44
3*
*0.
429
**
0.44
2*
*0.
619
**
0.92
1*
*0.
523
**
0.52
3*
*0.
512
**
0.49
5*
*
Sig
.(t
wo-
tail
ed)
0.00
00.
245
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
0n
9494
9494
9494
9494
9494
94Selfusage
Cor
rela
tion
coef
fici
ent
0.27
0*
*0.
078
0.12
70.
222
*0.
210
*0.
177
0.06
60.
153
0.15
30.
131
0.03
8S
ig.
(tw
o-ta
iled
)0.
008
0.45
40.
221
0.03
00.
042
0.08
60.
528
0.13
80.
138
0.20
70.
715
n95
9595
9594
9594
9595
9595
Peerusage
Cor
rela
tion
coef
fici
ent
0.19
40.
043
0.04
50.
201
0.10
20.
272
**
0.18
10.
250
*0.
250
*0.
309
**
0.24
1*
Sig
.(t
wo-
tail
ed)
0.06
00.
679
0.66
90.
052
0.33
20.
008
0.08
30.
015
0.01
50.
002
0.01
9n
9494
9494
9394
9394
9494
94(continued
)
Table XII.
JIC12,1
66
MP
emp
loy
eeca
pab
ilit
yM
Pem
plo
yee
sust
ain
abil
ity
MP
emp
loy
eesa
tisf
acti
on
MP
coll
eag
ue
atti
tud
e
MP
net
wor
kst
ruct
ure
MP
net
wor
kq
ual
ity
MP
curr
ency
MP
usa
ge
MP
con
trib
uti
on
MP
form
alti
e
MP
info
rmal
tie
Selfcontribution
Cor
rela
tion
coef
fici
ent
0.40
5*
*0.
341
**
0.27
7*
*0.
297
**
0.05
40.
184
0.07
40.
289
**
0.28
9*
*0.
216
*0.
044
Sig
.(t
wo-
tail
ed)
0.00
00.
001
0.00
70.
003
0.60
50.
074
0.47
60.
005
0.00
50.
036
0.67
1n
9595
9595
9495
9495
9595
95
Peercontribution
Cor
rela
tion
coef
fici
ent
0.43
6*
*0.
278
**
0.43
0*
*0.
495
**
0.57
0*
*0.
656
**
0.54
0*
*0.
981
**
0.98
1*
*0.
625
**
0.36
9*
*
Sig
.(t
wo-
tail
ed)
0.00
00.
006
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
0n
9595
9595
9495
9495
9595
95
Selfform
altie
Cor
rela
tion
coef
fici
ent
0.17
30.
222
*0.
084
0.11
40.
090
0.14
70.
180
0.11
90.
119
0.23
1*
0.18
5S
ig.
(tw
o-ta
iled
)0.
093
0.03
00.
416
0.27
30.
387
0.15
60.
083
0.24
90.
249
0.02
40.
072
n95
9595
9594
9594
9595
9595
Peerform
altie
Cor
rela
tion
coef
fici
ent
0.50
5*
*0.
288
**
0.51
4*
*0.
709
**
0.52
7*
*0.
772
**
0.59
6*
*0.
709
**
0.70
9*
*0.
907
**
0.62
8*
*
Sig
.(t
wo-
tail
ed)
0.00
00.
005
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
00.
000
0.00
0n
9595
9595
9495
9495
9595
95
Selfinform
altie
Cor
rela
tion
coef
fici
ent
0.15
10.
175
0.02
20.
101
0.09
40.
158
0.15
70.
051
0.05
10.
207
*0.
189
Sig
.(t
wo-
tail
ed)
0.14
30.
090
0.83
00.
328
0.36
60.
125
0.13
00.
621
0.62
10.
044
0.06
7n
9595
9595
9495
9495
9595
95Peerinform
altie
Cor
rela
tion
coef
fici
ent
0.37
7*
*0.
275
**
0.42
4*
*0.
478
**
0.32
1*
*0.
579
**
0.53
0*
*0.
380
**
0.38
0*
*0.
616
**
0.89
5*
*
Sig
.(t
wo-
tail
ed)
0.00
00.
007
0.00
00.
000
0.00
20.
000
0.00
00.
000
0.00
00.
000
0.00
0n
9595
9595
9495
9495
9595
95
Table XII.
Using 360 degreepeer review
67
ConclusionsTraditionally, researchers and practitioners have used other-ratings as a tool foridentifying training and development needs. In this paper, other-ratings have beenintroduced as a method for validating self-rating in the measurement of knowledge.Our objective was to address one of the weaknesses in existing methods – subjectivity.Our solution to this problem was to use three data points – self-reporting, 360-degreepeer review, and personality ratings – to validate the measurement of individuals’ HC.This triangulation method aims to introduce objectivity to the method making it avalue measurement rather than value assessment (see Andriessen, 2004).
The research may be summarised in three main findings. First, previous researchdemonstrating that correlations between self and other-ratings tend to be low (Harrisand Schaubroeck, 1988; Mount, 1984) is confirmed. However, while previous researchhas found that self-rating tends to be higher than other-rating, the authors found theopposite: other-rating was higher than self-rating. This means that how to adjustself-ratings to take the discrepancy with other-ratings into account must be considered.Given other-ratings are higher than self-ratings, the problem becomes how much toincrease self-ratings.
Second, personality is discounted as an influencing variable in self-rating ofknowledge. Numerous studies have examined the effect of the ratee’s personality onthe accuracy of self-other ratings agreement (Brutus et al., 1999) Gaylord (2003), Goffinand Anderson (2007), Newhouse (2008), Sinha (2003) and Vecchio and Anderson (2009).However, these studies have not used an instrument as comprehensive as the Big Fivefactor model of personality (Costa and McCrae, 1992; Goldberg, 1992). Our resultsshowed only limited effect of personality on self-other ratings. This means that it maybe unnecessary to consider respondent personality when adjusting self-ratings to takethe discrepancy with other-ratings into account.
Our third major finding is that there are patterns in the size of the discrepancy byknowledge dimension (i.e. employee capability, employee sustainability) that allow usto generalise about the adjustment necessary to find an accurate self-other rating ofknowledge. As a general guide, self-ratings may be adjusted as follows:
(1) Overall HC value: self ratings may be increased by 10 per cent.
(2) By knowledge capital type:. HC: self ratings may be increased by 5 per cent.. Social capital: self ratings may be increased by 5 per cent.. Structural capital: self ratings may be increased by 10 per cent.. Relational capital: self ratings may be increased by 30 per cent.
(3) By knowledge dimension:. Formal ties, informal ties, and usage may be increased by 30 per cent.. Network structure, network quality, and currency may be increased by a
factor of 20 per cent.. Employee capability, employee sustainability, employee satisfaction, and
contribution may be increased by 10 per cent.. Colleague attitude may be decreased by a factor of 20 per cent.
JIC12,1
68
In terms of the authors’ academic objective, they aimed to reduce subjectivity inmeasuring HC value, therefore, improving on existing methods as a valuemeasurement framework. This was the main benefit against measurement theory’scriteria (see Pike et al., 2002). The model’s performance against other criteria isdiscussed in the final paragraph. The authors have found that using a self-other ratingmethod provides useful context. While there is a discrepancy between self andother-ratings in the measurement of knowledge, it is relatively small. For example, theauthors recommend adjusting self-ratings by only 10 per cent to take overallother-ratings into account (i.e. the overall HC value). However, the method becomesparticularly interesting as the authors dug deeper into conceptual layers used todetermine overall HC value. Traditionally, organizations have defined the value ofpeople in a very narrow sense, i.e. their qualifications, skills, experience. This iscaptured in the model by the construct employee capability. The context provided bythe 11 dimensions of knowledge allows us to consider the value of staff in a moreobjective way than proposed by earlier models. The guidelines on how to adjustself-rating in each of these dimensions suggest progress towards a value measurementframework.
In terms of managerial application, the main benefit of the method presented in thispaper is confidence in the objectivity of the HC scores. Managers would be suspiciousof an evaluation based on self-reporting alone. The use of other-ratings and theadjustment guidelines presented in this paper may provide validity and reliabilitymissing from other HC evaluation methods. Once satisfied with the data themselves,managers can then look at with the broader perspective of why measurement of HCvalue is useful. Researchers have found that there is a positive relationship between theHC and the book/market value of the firm (Bozbura, 2004). The measurement ofintangible assets, such as HC, has become a very important research agenda for bothacademics and practitioners. These methods seek to quantify the economic value ofpeople to the organization (Sackmann et al., 1989) and to assist management andfinancial decisions. In terms of management decisions, the 360-degree peer reviewrating of knowledge proposed in this paper has considerable application. A first step isto use the outcomes in the way 360-degree feedback has been traditionally used; i.e.identifying training needs assessment, job analysis, performance appraisal, ormanagerial and leadership development (Tornow, 1993; Yammarino and Atwater,1993). A second step might be to use it for performance appraisal (Church, 2000) giventhe method’s capacity to identify issues at a very finite level: e.g. are you buildingeffective relationships with customers? – Informal ties score. A third step would be toidentify knowledge gaps, at a strategic level, for recruitment and development targets,e.g. to increase knowledge levels by dimension. Finally, in terms of financial decisionsinvestors might be able to compare knowledge scores by organization in each of theareas of overall knowledge, knowledge capital type, and knowledge dimensions. Forexample, one would expect a sales-focused organization to have high relational capitalscores and poor performance in this area might lead investors to consider otheralternatives.
This paper is an exercise in theory development, rather than theory testing, and assuch the method and the findings would be enhanced by further research. The paperhas several limitations. First, one of the criteria required by measurement theory is thatthe method must be auditable (Pike et al., 2002, p. 660). It might be argued that the
Using 360 degreepeer review
69
measurement process is auditable, but not the HC value scores. One’s view on thispoint will be determined by whether it is felt subjectivity has been addressed by themodel presented here. Second, the findings are based on a single case study only; andone with a hierarchical organizational structure (e.g. RAN), characterised by lineauthority and authoritarian leadership. This might induce homogenous perceptionsand behaviours, which could influence the self and other ratings. However, the samplecontained a mix of uniform and civilian staff, and the latter group is hierarchical onlyin the sense of subject matter expertise i.e. status and authority is attributed to experts.The type of work conducted by the case study organization, i.e. a knowledge “factory”of technical experts, may moderate the hierarchical structure. Third, another criteriarequired by measurement theory is that the method must not impose a largemeasurement overhead (Pike et al., 2002, p. 660). Clearly a survey with more than 1,000questions taking up to seven hours to complete may be considered impractical withunbearable implementation costs, particularly for small-to-medium sized enterprises.Bearing in mind that this was an academic study which aimed to adequately test thevalidity and reliability of a comprehensive set of measurement scales, a commercialversion of the survey with repeatability could be designed so that the survey mighttake only two hours. Finally, more studies using the model across different industriesand organizational contexts will further enhance the model’s validity and reliability.Further details on the survey instrument are available on request.
References
Andriessen, D. (2004), Making Sense of Intellectual Capital, Elsevier Butterworth-Heinemann,Burlington, MA.
Atwater, L.E. and Yammarino, F.J. (1992), “Does self-other agreement on leadership perceptionsmoderate the validity of leadership and performance predictions?”, Personnel Psychology,Vol. 45 No. 1, pp. 141-64.
Augier, M. and Teece, D. (2005), “An economic perspective on intellectual capital”, in Marr, B.(Ed.), Perspectives in Intellectual Capital, Elsevier Butterworth-Heinemann, Oxford,pp. 3-27.
Barrick, M.R. and Mount, M.K. (1991), “The Big Five personality dimensions and jobperformance: a meta-analysis”, Personnel Psychology, Vol. 44 No. 1, pp. 1-26.
Belk, R., Painter, J. and Semenik, R. (1981), “Preferred solutions to the energy crisis as a functionof causal attributions”, Journal of Consumer Research, Vol. 8 No. 3, pp. 306-12.
Bernardin, H.J., Cooke, D.K. and Villanova, P. (2000), “Conscientiousness and agreeableness aspredictors of rating leniency”, Journal of Applied Psychology, Vol. 85 No. 2, pp. 232-4.
Bontis, N. (1998), “Intellectual capital: an exploratory study that develops measures and models”,Management Decision, Vol. 36 No. 2, pp. 63-76.
Bontis, N. (1999), “Managing organizational knowledge by diagnosing intellectual capital:framing and advancing the state of the field”, International Journal of TechnologyManagement, Vol. 18 Nos 5-8, p. 433.
Bontis, N., Dragonetti, N.C., Jacobsen, K. and Roos, G. (1999), “The knowledge toolbox: a reviewof the tools available to measure and manage intangible resources”, EuropeanManagement Journal, Vol. 17 No. 4, p. 391.
Bozbura, F.T. (2004), “Measurement and application of intellectual capital in Turkey”,The Learning Organization, Vol. 11 Nos 4/5, p. 357.
JIC12,1
70
Brooks, J.H. and Stuhlmacher, A. (1999), “Personal characteristics and rater congruence inmulti-rater feedback”, paper presented at The Annual Convention of the Society forIndustrial and Organizational Psychology, Atlanta, GA.
Brutus, S., Fleenor, J.W. and McCauley, C.D. (1999), “Demographic and personality predictors ofcongruence in multi-source ratings”, Journal of Management Development, Vol. 18 No. 5,p. 417.
Carson, E., Ranzijn, R., Winefiel, A. and Marsden, H. (2004), “Intellectual capital: mappingemployee and work group attributes”, Journal of Intellectual Capital, Vol. 5 No. 3, p. 443.
Church, A.H. (2000), “Do higher performing managers actually receive better ratings? Avalidation of multirater assessment methodology”, Consulting Psychology Journal: Practiceand Research, Vol. 52 No. 2, pp. 99-116.
Cloutier, L.M. and Gold, E.R. (2005), “A legal perspective on intellectual capital”, Perspectives onIntellectual Capital, Elsevier Butterworth-Heinemann, Burlington, MA, pp. 125-36.
Costa, P.T. and McCrae, R.R. (1992), Revised NEO Personality Inventory (NEO-PI-R) andNEO-FFI Inventory Professional Manual, PAR, Odessa, FL.
Digman, J.M. (1997), “Higher-order factors of the Big Five”, Journal of Personality and SocialPsychology, Vol. 73 No. 6, pp. 1246-56.
Edvinsson, L. and Malone, M.S. (1997), Intellectual Capital: The Proven Way to Establish YourCompany’s Real Value by Measuring Its Hidden Brainpower, Piatkus, London.
Fernstrom, L. (2005), “A marketing perspective on intellectual capital”, in Marr, B. (Ed.),Perspectives on Intellectual Capital, Elsevier Butterworth-Heinemann, Burlington, MA,pp. 82-95.
Flamholtz, E.G. (1973), “Human resources accounting, measuring positional replacement cost”,Human Resource Management, Vol. 12 No. 1, pp. 8-16.
Fletcher, C. and Baldry, C. (2000), “A study of individual differences and self-awareness in thecontext of multi-source feedback”, Journal of Occupational and Organizational Psychology,Vol. 73 No. 3, pp. 303-19.
Furnham, A. and Stringfield, P. (1994), “Congruence of self and subordinate ratings ofmanagerial practices as a correlate of superior evaluation”, Journal of Occupational andOrganizational Psychology, Vol. 67, pp. 57-67.
Gaylord, T.W. (2003), “Feedback acceptance within leadership development: effects of self-otherrating discrepancies, conscientiousness and neuroticism”, PhD thesis, Kansas StateUniversity, Manhattan, KS.
Gerbing, D.W. and Anderson, J.C. (1988), “An update paradigm for scale developmentincorporating unidimensionality and its assessments”, Journal of Marketing Research,Vol. 25 No. 2, pp. 186-92.
Goffin, R.D. and Anderson, D.W. (2007), “The self-rater’s personality and self-other disagreementin multi-source performance ratings: is disagreement healthy?”, Journal of ManagerialPsychology, Vol. 22 No. 3, pp. 271-89.
Goldberg, L.R. (1992), “The development of markers for the Big Five factor structure”, Journal ofPersonality and Social Psychology, Vol. 59 No. 6, pp. 1216-29.
Gooding, R.Z. and Kinicki, A. (1995), “Interpreting event causes: the complementary role ofcategorization and attribution processes”, Journal of Management Studies, Vol. 32 No. 1,pp. 1-21.
Goodstone, M.S. and Diamante, T. (1998), “Organisational use of therapeutic change:strengthening multisource feedback systems through interdisciplinary coaching”,Consulting Psychology Journal: Practice and Research, Vol. 50 No. 1, pp. 152-63.
Using 360 degreepeer review
71
Grant, R.M. (1996), “Toward a knowledge-based theory of the firm”, Strategic ManagementJournal, Vol. 17, Winter, special issue, pp. 109-22.
Hair, J.F., Anderson, R.E., Tatham, R.L. and Black, W.C. (2006), Multivariate Data Analysis,6th ed., Prentice-Hall, Englewood Cliffs, NJ.
Halverson, S.K., Tonidandel, S., Barlow, C. and Dipboye, R.L. (2002), “Self-other agreement on a360 degree leadership evaluation”, paper presented at The 17th Annual Conference of theSociety for Industrial and Organizational Psychology, Toronto.
Harris, M. and Schaubroeck, J. (1988), “A meta analysis of self-supervisor, self-peer andpeer-supervisor ratings”, Personnel Psychology, Vol. 41 No. 1, pp. 43-61.
Harrison, S. and Sullivan, P.H. (2000), “Profiting from intellectual capital: learning from leadingcompanies”, Journal of Intellectual Capital, Vol. 1 No. 1, p. 33.
Hazucha, J.F., Hezlett, S.A. and Schneider, R.J. (1993), “The impact of 360-degree feedback onmanagement skills development”, Human Resource Development, Vol. 32 No. 2, pp. 325-51.
Hermanson, R.H. (1964), “Accounting for human assets”, Occasional Paper No. 14, MichiganState University, East Lansing, MI.
Hogan, R., Hogan, J. and Roberts, B.W. (1996), “Personality measurement and employmentdecisions: questions and answers”, American Psychologist, Vol. 51 No. 5, pp. 469-77.
Housel, T. and Bell, A.H. (2001), Measuring and Managing Knowledge, McGraw-Hill, New York,NY.
Johanson, U. (2005), “A human resource perspective on intellectual capital”, in Marr, B. (Ed.),Perspectives on Intellectual Capital, Elsevier Butterworth-Heinemann, Burlington, MA,pp. 96-105.
Johnson, W.H.A. (2002), “Leveraging intellectual capital through product and processmanagement of human capital”, Journal of Intellectual Capital, Vol. 3 No. 4, pp. 415-29.
Kulas, J.T. and Finkelstein, L.M. (2007), “Content and reliability of discrepancy-definedself-awareness in multisource feedback”, Organisational Research Methods, Vol. 10 No. 3,pp. 502-22.
Lev, B., Canibano, L. and Marr, B. (2005), “An accounting perspective on intellectual capital”,in Marr, B. (Ed.), Perspectives on Intellectual Capital, Elsevier Butterworth-Heinemann,Burlington, MA, pp. 42-55.
McEnery, J.M. and Blanchard, P.N. (1999), “Validity of multiple ratings of business studentperformance in a management simulation”, Human Resource Development Quarterly,Vol. 10 No. 2, pp. 155-72.
Marr, B. and Roos, G. (2005), “A strategy perspective on intellectual capital”, in Marr, B. (Ed.),Perspectives on Intellectual Capital, Elsevier Butterworth-Heinemann, Burlington, MA,pp. 28-41.
Marsick, V. and Watkins, K. (2003), “Demonstrating the value of an organisation’s learning”,Advances in Developing Human Resources, Vol. 5 No. 2, pp. 132-51.
Mount, M.K. (1984), “Psychometric properties of subordinate ratings of managerialperformance”, Personnel Psychology, Vol. 37, pp. 687-701.
Mowery, D.C., Oxley, J.C. and Silverman, B.S. (1996), “Strategic alliances and interfirmknowledge transfer”, Strategic Management Journal, Vol. 17, Winter, pp. 77-91.
Newhouse, N.K. (2008), The Role of Ratee Personality in Self-Boss Agreement on InterpersonalDimensions of Performance, Illinois Institute of Psychology, Chicago, IL.
Nilsen, D. and Campell, D.P. (1993), “Self-observer rating discrepancies: once an overrater,always an overrater?”, Human Resource Management, Vol. 32 Nos 2-3, pp. 265-81.
JIC12,1
72
Nunnally, J.C. (1978), Psychometric Theory, McGraw-Hill, New York, NY.
Peppard, J. (2005), “An information systems perspective on intellectual capital”, in Marr, B. (Ed.),Perspectives on Intellectual Capital, Elsevier Butterworth-Heinemann, Burlington, MA,pp. 106-24.
Pike, S., Rylander, A. and Roos, G. (2002), “Intellectual capital management and disclosure”,in Choo, C.W. and Bontis, N. (Eds), The Strategic Management of Intellectual Capital andOrganizational Knowledge, Oxford University Press, Oxford, pp. 657-71.
Reilly, R.R., Smither, J.W. and Vasilopoulos, N. (1996), “A longitudinal study of upwardfeedback”, Personnel Psychology, Vol. 49 No. 3, pp. 599-612.
Robinson, G. and Kleiner, B.H. (1996), “How to measure an organization’s intellectual capital”,Managerial Auditing Journal, Vol. 11 No. 8, pp. 36-9.
Roslender, R. (2000), “Accounting for intellectual capital: a contemporary managementaccounting perspective”, Management Accounting, Vol. 78 No. 3, pp. 34-7.
Roush, P. and Atwater, L.E. (1992), “Using the MBTI to understand transformational leadershipand self-perception accuracy”, Military Psychology, Vol. 4 No. 1, pp. 17-34.
Sackmann, S.A., Flamholtz, E.G. and Bullen, M.L. (1989), “Human resource accounting:a state-of-the-art review”, Journal of Accounting Literature, Vol. 8, pp. 235-64.
Schaffer, B.S. (2002), “Board assessments of managerial performance: an analysis of attributionprocesses”, Journal of Management Psychology, Vol. 17 No. 2, pp. 95-115.
Shore, T.H., Shore, L.M. and Thornton, G.C. (1992), “Construct validity of self and peerevaluations of performance dimensions in an assessment center”, Journal of AppliedPsychology, Vol. 77 No. 1, pp. 42-54.
Sinha, N. (2003), “Self-other rating discrepancy and personality antecedents”, PhD thesis, FloridaInternational University, Miami, FL.
Stewart, T.A. (1995), “Getting real about brainpower”, Fortune, Vol. 132 No. 11, pp. 201-3.
Stewart, T.A. (1998), Intellectual Capital: The New Wealth of Organizations, Nicholas BrealeyPublishing, London.
Sudarsanam, S., Sorwar, G. and Marr, B. (2005), “A finance perspective of intellectual capital”,in Marr, B. (Ed.), Perspectives on Intellectual Capital, Elsevier Butterworth-Heinemann,Burlington, MA, pp. 56-68.
Sullivan, P.H. (2005), “An intellectual property perspective on intellectual capital”, in Marr, B.(Ed.), Perspectives on Intellectual Capital, Elsevier Butterworth-Heinemann, Burlington,MA, pp. 137-51.
Sveiby, K.E. (1997), The New Organizational Wealth: Managing andMeasuring Knowledge-basedAssets, Berrett-Koehler Publisher, San Francisco, CA.
Teece, D.J. (1998), “Capturing value from knowledge assets: the new economy, markets forknow-how, and intangible assets”, California Management Review, Vol. 40 No. 3, pp. 55-79.
Tett, R.P., Jackson, D.M. and Rothstein, M. (1991), “Personality measures as predictors of jobperformance: a meta-analytic review”, Personnel Psychology, Vol. 44 No. 4, pp. 703-42.
Tornow, W.W. (1993), “Perceptions or reality: is multi-perspective measurement a means or anend?”, Human Resource Management, Vol. 32 Nos 2/3, pp. 221-9.
Vecchio, R.P. and Anderson, R.J. (2009), “Agreement in self-other ratings of leader effectiveness:the role of demographics and personality”, International Journal of Selection andAssessment, Vol. 17 No. 2, pp. 165-79.
Weiner, B. (2000), “Attributional thoughts about consumer behavior”, Journal of ConsumerResearch, Vol. 27 No. 3, pp. 382-7.
Using 360 degreepeer review
73
Wohlers, A.J. and London, M. (1989), “Ratings of managerial characteristics: evaluationdifficulty, subordinate agreement, and self-awareness”, Personnel Psychology, Vol. 42 No. 2,pp. 235-61.
Yammarino, F.J. (1997), “Do managers see themselves as others see them? Implications ofself-other rating agreement for human resources management”, Organizational Dynamics,Vol. 25 No. 4, pp. 35-45.
Yammarino, F.J. and Atwater, L.E. (1993), “Understanding self-perception accuracy: implicationsfor human resource management”, Human Resource Management, Vol. 32 Nos 2-3,pp. 231-46.
Further reading
Church, A.H. (1997), “Managerial self-awareness in high performing individuals inorganisations”, Journal of Applied Psychology, Vol. 82 No. 2, pp. 281-92.
Friedman, A. and Lev, B. (1974), “A surrogate measure for the firm’s investment inhuman-resources”, Journal of Accounting Research, Vol. 12 No. 2, pp. 235-50.
Hekimian, J.S. and Jones, C.H. (1967), “Put people on your balance-sheet”, Harvard BusinessReview, Vol. 45, January/February, pp. 105-13.
Likert, R. (1967), The Human Organization: Its Management and Value, McGraw-Hill, New York,NY.
Likert, R. and Bowers, D.G. (1973), “Improving the accuracy of P/L reports by estimating thechange in dollar value of the human organization”, Michigan Business Review, Vol. 25No. 2, p. 15.
Mouritsen, J., Bukh, P.N. and Marr, B. (2005), “A reporting perspective on intellectual capital”,in Marr, B. (Ed.), Perspectives on Intellectual Capital, Elsevier Butterworth-Heinemann,Burlington, MA, pp. 69-81.
Corresponding authorPeter Massingham can be contacted at: [email protected]
JIC12,1
74
To purchase reprints of this article please e-mail: [email protected] visit our web site for further details: www.emeraldinsight.com/reprints