The Journal of Systems and Software 83 (2010) 574–584

Contents lists available at ScienceDirect

The Journal of Systems and Software

journal homepage: www.elsevier .com/locate / jss

An empirical examination of application frameworks success basedon technology acceptance model

Gregor Polancic *, Marjan Hericko, Ivan RozmanUniversity of Maribor, Faculty of Electrical Engineering and Computer Science, Smetanova 17, 2000 Maribor, Slovenia

a r t i c l e i n f o a b s t r a c t

Article history:Received 16 September 2008Received in revised form 17 August 2009Accepted 24 October 2009Available online 29 October 2009

Keywords:Application frameworksTechnology acceptance modelPost adoption behaviorInformation system successSurveyStructural equation models

0164-1212/$ - see front matter � 2009 Elsevier Inc. Adoi:10.1016/j.jss.2009.10.036

* Corresponding author. Tel.: +386 2 220 7421.E-mail address: gregor.polancic@uni-mb.si (G. Pol

Framework-based development is currently regarded as one of the most promising software develop-ment approaches when it comes to improvements in lead time, productivity and quality. However, manyframeworks and projects based on frameworks still report failures, which indicate that there are prob-lems related to both frameworks technology and frameworks usage. The objective of our research wasto examine the major drivers that have an impact on a framework’s acceptance and a framework’s suc-cess. We used the technology acceptance model (TAM) and Seddon’s information systems success modelas our underlying theory. Data collected from an online survey of 389 active framework users was used totest hypothesized models. Data analysis was performed via structural equation modeling. Our findingssupport the post-adoption version of TAM and the relationship between continuous use and the success-ful use of systems, where more use also means more net benefits. We found that the successful use offrameworks is mainly dependent on two factors: continuous framework usage intention and the per-ceived usefulness of the framework. Several practical and theoretical implications can be foreseen includ-ing advances in framework development guidelines and insight into the relationship between theacceptance and success of frameworks.

� 2009 Elsevier Inc. All rights reserved.

1. Introduction

Application frameworks (or ‘‘frameworks” for short) are amature technology for reusing software designs and implementa-tions in order to reduce costs and improve the quality of developedsoftware (Mamrak and Sinha, 1999; Morisio et al., 2002b). Frame-works are semi-completed systems that contain certain fixed as-pects common to all applications in the problem domain, andcertain variable aspects unique to each application made from it(also known as ‘‘framework instances”) (Srinivasan, 1999). Ob-ject-oriented frameworks are the most prevalent. They are definedas ‘‘a set of classes that embodies an abstract design for a solutionto a family of related problems” (Johnson and Foote, 1988).

Frameworks differ from other reuse techniques, such as compo-nents, libraries or design patterns, because they aim to reuse largergrained components and higher-level designs (Fig. 1). Because theydefine the flow of control, they act as the main program for instan-tiated applications.

Frameworks play a central role in the software developmentcommunity (Manolescu et al., 2006), especially when it comes toinstantiating software within product lines and product families(Batory et al., 2000; Cunningham et al., 2006). Frameworks also

ll rights reserved.

ancic).

act as an extension of generic programming languages and allowdevelopers to make gains from commonalities in the domain theyact (using domain frameworks), development practices theyimplement (using support frameworks) or applications they devel-op (using application frameworks).

Besides the positive effects, developing, instantiating andmaintaining frameworks continues to be a difficult endeavor(Bosch et al., 1999; Srinivasan, 1999; van Gurp and Bosch, 2001).Due to this, software developers may not decide to develop oruse frameworks despite their availability. Or they might developor use frameworks in an inappropriate way, which leads to projectfailures.

Because there are problems that make framework developmentand instantiation difficult, practitioners and researchers haveproposed several improvements to them, ranging from documen-tation improvements (Johnson, 1992), technical improvements(van Gurp and Bosch, 2001) and general improvements forsuccessful framework development and instantiation (Landin andNiklasson, 1995).

While these improvements do stimulate new ideas, most ofthem are based on personal experiences. Additionally, they donot include proven theoretical foundations and empirical research,which is regarded as one of the main problems of software engi-neering (Shaw, 1990). Measurement and experimentation forproduct line engineering has been identified by Frakes and Kang

Framework

Core

HotSpot

ClassLibrary

FrozenSpotHook

Fig. 2. Framework structure, presented in UML class diagram.

Class / class library

Design pattern

Framework

Application

Fig. 1. Framework elements and their relationships (Sangdon et al., 1999).

G. Polancic et al. / The Journal of Systems and Software 83 (2010) 574–584 575

(2005) as an area where considerably more work is necessary. Sec-ondly, most of the framework improvements are only technologi-cally oriented. However, while frameworks are intensively usedby application developers, it is reasonable to incorporate frame-work developers and users into research on the subject. This sup-position is consistent with the results of the Morisio et al.(2002a) study, in which the authors found that human factors haveplayed a focal role in the success of software reuse. Finally, giventhe importance of frameworks and the extent of their impact onsoftware development projects, there is still a lack of research thatidentifies and addresses fundamental issues, such as how differentframework-related improvements influence frameworks and theperceptions of application developers who instantiate frameworks.However, without a clear understanding of the dynamics of aframework’s success, those improvements can only be speculative.

Compared with existing framework research, our research doesnot propose concrete improvements to frameworks but investi-gates the major drivers that have an impact on a framework’sacceptance and its successful use. We investigate frameworkacceptance as a special case of IT acceptance where a solid theorywith reliable measurement scales has been developed and empir-ically validated over the past 15–20 years. This theory is commonlyexpressed in the ‘‘technology acceptance model” or TAM. Our re-search builds on TAM and adapts it to the context of frameworks.The reason for adapting TAM specifically to frameworks insteadto all reusable assets is that researchers in previous studies re-ported problems with ‘‘not distinguishing among different typesof reusable assets” (Mellarkod et al., 2007).

The original outcome of this article is an empirically validatedcausal model of factors that impact a framework’s acceptanceand its successful use. Based on the model, several theoreticaland practical implications can be foreseen and are presented inthe conclusion of this paper.

2. Theoretical background

2.1. Application frameworks

As presented in the introduction, a framework is a partial designand implementation for an application in a given problem domain.Frameworks are expressed in a programming language, so theycommonly consist of a set of cooperating classes and libraries thatmake up a reusable design. Certain methods of these classes areleft unspecified or abstract. In this way they expose details thatvary among framework’s implementations. A framework instanceprovides the missing details. It is a pairing of a concrete subclasswith each abstract class of the framework to provide a completeimplementation. These areas of variability within a framework

are called hot spots. A hot spot can contain several hooks, whichrepresent actual places (methods) in the framework that can beadapted or extended in order to provide application specific func-tionality (Fig. 2).

In contrast, framework’s frozen spots capture the commonali-ties across framework instances. These remain unchanged in anyinstantiation of the framework. In frozen spots, responsibility, col-laboration and thread of control are defined. These are commonlyexpressed with design patterns (Froehlich et al., 1998).

Despite the problem domain knowledge and large-scale-reuseoffered by framework-based development, application develop-ment based on frameworks continues to be difficult. The frame-work user must first understand the complex class hierarchiesand object collaborations embodied in the framework to use theframework effectively. Besides, frameworks are particularly hardto document since they represent a reusable design at a high-levelof abstraction implemented by the framework classes.

To improve the interaction between frameworks and theirusers, numerous framework development guidelines have beenproposed (Mattsson, 2000). For example, a cataloque of 71 frame-work-related guidelines has been defined by Landin and Niklasson(1995). These guidelines impact design, implementation and use offrameworks. However, the questions remain how these guidelinesimpact frameworks, their users’ perceptions and project outcomes.

2.2. Information technology acceptance

Understanding what influences users to use specific technology,is a major issue in the IT success area (Sharp, 2007). Within thetheories which examine the acceptance and use of IT, Davis’s(1989) Technology Acceptance Model (TAM), remains one of themost cited, validated and often used theoretical models (Kingand He, 2006). As demonstrated with solid arrows in Fig. 3, a keyassumption of TAM is that external variables (EV) influence thedecision to use particular IT only indirectly through their impacton users’ beliefs: i.e. the perceived ease of use (PEOU) and per-ceived usefulness (PU). These two beliefs both influence users’ atti-tude towards using IT (ATU). ATU sequentially has influence onwhich behavioral intention to use (BI), which is the key factor indetermining IT use (U). And, as Tonella et al. (2007) stated, TAMfactors must be fulfilled to efficiently exploit new tools.

The solid arrows in Fig. 3 show the initial TAM relationships asintroduced by Davis. The dotted arrows show the relationshipsinvestigated by other researchers, whereas the values on the ar-rows show the results of TAM meta-analysis, which was performedby Legris et al. (2003). The values on arrows indicate following: (1)number of significant positive relations identified, (2) number ofnon-significant relations identified, (3) number of significant neg-ative relations identified and (4) number of untested relations.

External Variables

(EV)

Perceived Usefulness

(PU)

Perceived Ease of Use

(PEOU)

Attitude Toward

Using (ATU)

Behavioral Intention to

Use (BI)

Actual System Use

(U)

(12,1,1,14)

(21,5,0,2)

(11,2,0,15)

(16,3,0,9)

(10,3,0,15)

(7,4,0,17)

(3,0,0,25)

(10,1,0,17)

(8,5,0,15)

(4,5,0,19)

Fig. 3. Technology Acceptance Model – TAM (solid arrows).

576 G. Polancic et al. / The Journal of Systems and Software 83 (2010) 574–584

Although TAM has been further developed into a more elabo-rate model known as the Unified Theory of Acceptance and Useof Technology (UTAUT) (Venkatesh et al., 2003) there currently ex-ists only one study confirming UTAUT’s validity and robustness.TAM, on the other hand, has been tested by many researchers withdifferent populations of users and IT innovations. Different types ofsoftware have been investigated using TAM. However, we did notfind any research that would apply TAM to semi-completed orreusable software. Besides this, Hong et al. (2006) concluded thatTAM is the most simple and generic model that can be used tostudy both initial and continued IT adoption.

To summarize, several reasons can be identified for using TAMas the central theoretical model of our research: (1) its specific fo-cus on information technology; (2) its demonstrated validity andreliability; (3) its extensive application; (4) its accumulated re-search tradition (Sharp, 2007); and (5) its ability to be used in bothadoption and post-adoption behavior.

3. Research model and hypotheses

Based on the research problem and theoretical foundations, wedefined the following research questions: (1) Is TAM (post-adop-tion version) valid for frameworks? (2) Can we use TAM (post-adoption version) constructs for anticipating the successful use offrameworks?

Researchers already-demonstrated positive relationship be-tween the number of framework instantiations and frameworksuccess (Mattsson, 1999; Morisio et al., 2002b; Moser and Nier-strasz, 1996). What remains is to investigate what impacts theamount of frameworks instantiations. We commenced this prob-lem with TAM.

Frameworkease of use

(FEOU)

Frameworkusefulness

(FU)

Perceptions & Beliefs

fr

H3

Intent

H5

H1External variables

Domain of TAM

H2

H4

Fig. 4. Hypothesized

Following the logic of TAM, we anticipated that framework-related external variables (for example: framework quality charac-teristics, documentation quality and incorporated design patterns)influence user beliefs, whereas more positive beliefs increase thelikelihood of a framework’s acceptance and use. While the success-ful use of frameworks requires continued framework use, weinvestigated actual framework users on a post-adoption versionof TAM. Post adoption version of TAM demonstrated its validityin anticipating IT usage continuance with two constructs: PU andPEOU (Hong et al., 2006). Considering above we defined researchmodel as shown on Fig. 4.

In the proposed research model (Fig. 4) we adapted post-adop-tion TAM constructs to frameworks context. The two dependentconstructs of the research model (NBF and SAT) were taken fromSeddon’s IS Success model (Seddon, 1997). Both constructs, wereused to investigate success, as reported by framework users. Whilesystem-behavior related constructs are common in TAM and Sed-don’s IS Success model (Seddon, 1997) they can also serve as a ba-sis for their integration.

3.1. User beliefs

Booch (1994) stated that ‘‘The most profoundly elegant frame-work will never be reused unless the cost of understanding itand then using its abstractions is lower than the programmer’sperceived cost of writing them from scratch.” This statement corre-sponds to the cost-benefit paradigm, which represents one ofTAM’s theoretical foundations (Davis, 1989).

3.1.1. Framework usefulnessStudies have shown that PU is the main precondition for behav-

ioral intentions, system acceptance and continued usage intentions

Framework success

Satisfaction with framework

use(SAT)

Continued amework use

intention(CFUI)

Framework acceptance

(FA)

Net benefits of framework use

(NBF)

ions & Behavior

H7H9

Domain of IS Success model

H8

H6

research model.

G. Polancic et al. / The Journal of Systems and Software 83 (2010) 574–584 577

(Hong et al., 2006; Legris et al., 2003). We therefore believe that themain reason for ‘‘positive” user behavior is that users perceive aframework as useful.

In our case, the following user-dependent TAM constructs wereinvestigated: (a) ‘‘framework acceptance,” as defined by Avlonitisand Panagopoulos (2005) and (b) ‘‘continued framework usageintention,” as defined by Hong et al. (2006). In both cases, research-ers found a significant relationship between PU and the investi-gated dependent construct. Thus, we hypothesize the following:

H1: Framework usefulness (FU) will have a positive effect onframework acceptance (FA).

H2: Framework usefulness (FU) will have a positive effect on con-tinued framework usage intention (CFUI).

3.1.2. Framework ease of useEase of use refers to the property of a product that a user can

operate without having to overcome a steep learning curve. Exten-sive research of previous years has provided evidence of the signif-icant effect of perceived ease of use on user behavior, eitherdirectly or mediated by perceived usefulness (Davis et al., 1989;Dishaw and Strong, 1999; Lederer et al., 2000). While frameworksare generally recognized as the most complex reusable softwarestructures with a steep learning curve, we hypothesize thefollowing:

H3: Framework ease of use (FEOU) will have a positive effect onFramework usefulness (FU).

H4: Framework ease of use (FEOU) will have a positive effect onframework acceptance (FA).

H5: Framework ease of use (FEOU) will have a positive effect oncontinued framework usage intention (CFUI).

3.2. User behavior

Four different dependent constructs are common in TAM: (a)intention to use (b) actual use, (c) acceptance and (d) continuedusage intention. In our case, we found it reasonable to investigateframework acceptance and continuous framework usage intention.

User acceptance has been incorporated as a dependent variableinto the majority of the IT implementation research (Saga andZmud, 1994). The general premise is that user acceptance is criticalto successful implementation. Framework acceptance represents aspecial case of user acceptance for following reasons: (1) it is suitedto frameworks and (2) it is intended for application developers in-stead of application users.

In line with existing framework-related studies (Mattsson, 1999;Morisio et al., 2002a,b; Moser and Nierstrasz, 1996) and the productline cost model (Bockle et al., 2004), we have presumed that frame-work acceptance is not satisfactory for successful framework use.This is because of the experience that framework-based develop-ment starts with a productivity decrease and that cost savingsincrease with the number of framework instantiations. Consistentwith this, we expect that continuous use behavior is required forsuccessful frameworks use. So we hypothesize the following:

H6, H7: Frameworks acceptance (FA) will have an impact onframework success.

H8, H9: Continuous framework usage intention (CFUI) will have apositive impact on framework success (FS).

3.3. Framework success

‘‘IS success” is generally recognized as a multidimensional con-struct and is defined as ‘‘a measure of the degree to which the per-son evaluating the system beliefs that the stakeholder (in whose

evaluation is being made) is better off” (Seddon, 1997). Seddon fur-ther states that ‘‘if net benefits could be measured with precision,they would be equivalent to IS Success”.

Framework success (FS) represent dependent constructs of ourresearch. Based on the above, we investigated framework successwith two constructs: (1) net benefits of framework use (NBF) and(2) satisfaction with framework use (SAT). We defined NBF as aself-reported latent variable, comprised of the three most commonframework success measures: (1) improvements in the quality ofsoftware products, (2) improvements in the productivity of soft-ware development and (3) improvements in the lead time. SATwas defined as ‘‘a subjective evaluation of the various consequencesevaluated on a pleasant-unpleasant continuum” (Seddon, 1997).

4. Survey method

We used a web-based survey for testing the stated constructsand finding relationships between them. The methodology usedis described in the following subsections.

4.1. Measures and pretest

We developed a survey instrument that contained instructionsasking the respondent to identify a framework that they developand/or uses most frequently during application development.Focusing a subject’s attention on a specific framework is in linewith Churchill’s recommendation to define a unit of analysis in or-der to get a more precise response and greater validity (Churchill,1979). The development of the research instrument was primarilybased on existing measures, with validation and wording changesadapted to our context. The instrument included 32 questionswhich were categorized in the following major sections: (1) Demo-graphic questions about the respondent’s gender, age, experiencewith software development and frameworks; (2) Measures forFEOU and FU, obtained from TAM meta-analysis (Legris et al.,2003); (3) Measures for FA, adapted from Goodhue and Thompson(1995); (4) Measures of CFUI, adapted from Hong et al. (2006); (5)Measures for NBF, based on the most common benefits of usingframeworks; and (6) Measures for SAT, adapted from Hong et al.(2006).

Likert scales of 1–7 with end points of ‘‘strongly agree” and‘‘strongly disagree” were used for all items except for the itemsmeasuring SAT, which was defined on a semantic differential scale.To achieve the desired balance in the questionnaire, some ques-tions were worded with proper negation and all items in the ques-tionnaire were randomly sequenced to reduce any potential ceilingeffect. The constructs and scale items are presented in theappendix.

A pilot study of the survey instrument was conducted to ensurethat the subject could understand the items and measurementscales. Twenty-two students who used a framework in the labwork participated in the pilot study. Completed questionnaires,feedback from the subjects and observations by the authors re-sulted in minor changes to the survey instructions, refinement tothe wording of several items and an additional explanation of sometechnical terms.

4.2. Subjects and the sampling process

Surveys differ greatly in value according to how respondentsare sampled (McBurney and White, 2003) whereas the identifyingsample frame is an important step in ensuring that the populationof interest has been correctly identified. The ideal candidate for ourstudy was an experienced framework user who has already usedseveral frameworks, where the most experienced framework users

Table 1Profile of the respondents.

Variable Values Frequency Valid (%)

Gender Female 9 2.3Male 380 97.7

Education High school 53 13.6Bachelor 153 39.3Master 127 32.6Doctorate 30 7.7Other 26 6.6

Software developmentexperience (in years)

0–1 years 1 0.3

1–2 years 12 3.13–5 years 70 17.85–8 years 94 23.98–13 years 103 26.213–21 years 69 17.6>21 years 44 11.2

Number of frameworksexplored to date

1 13 3.3

2 24 6.23–5 119 30.65–8 83 21.3

578 G. Polancic et al. / The Journal of Systems and Software 83 (2010) 574–584

are usually those who participate in framework development. Thesample frame, which was accessible to us, was comprised ofSourceforge.net members. Sourceforge.net is the largest repositoryof open source projects with database snapshots available toresearchers. In the snapshot version ‘‘December 2007‘‘, 5216 pro-jects were classified into framework projects. So, our sample frameembraced 11,357 distinct members of these projects. We used aninteger random generator for random sampling, where everymember from the sample frame had an equal and independentchance of being selected in the sample.

The sample, which resulted from a systematic random samplingprocess, embraced 4000 different framework project members. Ofthe 4000 surveys mailed by an online survey system, 934 mailswere opened, 710 surveys actually started and 447 surveys com-pleted, representing a response rate of 11.2%. Fifty-eight surveyshad incomplete responses or they did not pass controlled questionsand were therefore not used. As a result, 389 surveys were actuallyused, with a resulting response rate of 9.7%. The survey resultswere obtained from the online survey system as a spreadsheet file,which was further managed in SPSS and AMOS (seewww.spss.com).

8–13 57 14.7>13 years 93 23.7

Framework experience (in years) <1 year 12 3.11–2 years 31 7.92–3 years 60 15.33–5 years 103 26.35–8 years 91 23.28–13 years 58 14.8>13 years 37 9.4

Table 2Analysis of evaluated frameworks use.

Variable Values Frequency Valid (%)

Evaluated framework Spring Framework 23 5.9.NET 20 5.1Jakarta Struts 15 3.8EclipseFrameworks

9 2.3

Django 7 1.8Hibernate 6 1.5Other frameworks 313 79.6

Type of use Voluntary 304 78.1Mandatory 85 21.9

4.3. Data analysis

We used structural equation modeling (SEM) (Anderson andGerbing, 1988) in order to test the fit of the proposed theoreticalmodel (Fig. 4) with empirical data. SEM was applicable because itencourages confirmatory rather than exploratory modeling. Thus,it is suited to theory testing rather than theory development. Weused a strictly confirmatory approach, where a model is testedusing SEM goodness-of-fit tests to determine if the pattern of vari-ances and co-variances in the data is consistent with the hypothe-sized model (Fig. 4). SEM consists of two main parts (Anderson andGerbing, 1988): the measurement model showing relationships be-tween latent variables and their indicators, and the structuralmodel showing potential causal dependencies between endoge-nous and exogenous variables. The measurement model was esti-mated using confirmatory factor analysis to test whether theproposed constructs possessed sufficient validation and reliability.To ensure data validity and reliability, internal consistency, conver-gent validity, and discriminate validity were demonstrated. Thestructural model was estimated using path analysis. We usedAMOS as the SEM software for data analysis. AMOS is a covari-ance-based approach similar to LISREL (Gefen et al., 2000), inwhich the covariance structure obtained from the observed datais used to simultaneously fit measurement and structural equa-tions specified in the model. AMOS estimated both the measure-ment and structural models using the maximum likelihoodestimator.

Number of completed projects 0 28 7.21 54 13.82 71 18.23–5 118 30.25–8 63 16.18–13 26 6.613–21 8 2.0>21 23 5.9

Duration of framework use <1 year 68 17.31–2 years 102 26.02–3 years 100 25.53–5 years 81 20.75–8 years 30 7.7>8 years 11 2.8

Frequency of framework use Daily 199 51.0Weekly 120 30.8Monthly 31 7.9Less than monthly 40 10.3

5. Results

5.1. Descriptive statistics

The characteristics of the respondents are presented in Table 1.The typical respondent is a 31.7-year-old male with a bachelor’sdegree. The respondent has an average of 8–13 years of experiencein software development. He has so far explored 3–5 frameworksand has 3–5 years of experiences using frameworks.

Only 33% of the respondents decided to evaluate the frameworkfrom the corresponding Sourceforge.net framework project, where67% of the respondents decided to evaluate another open source orproprietary framework. In most cases, respondents decided toevaluate a framework that they used on a voluntary and daily

basis. On average, they already instantiated this framework 3–5times.

Further, Table 2 shows that most respondents have evaluatedthe Spring framework, followed by the .NET framework. In mostcases, respondents used the evaluated framework voluntarily for

Table 3Reliability of constructs.

Construct Number of items Mean Standard deviation Cronbach’s alpha Composite reliability Average variance extracted

FEOU 5 2.42 1.19 0.87 0.88 0.60FU 4 1.93 1.13 0.95 0.95 0.83FA 4 2.59 1.50 0.89 0.89 0.68CFUI 3 2.22 1.37 0.88 0.88 0.72SAT 3 2.12 1.09 0.84 0.85 0.65NBF 3 2.10 1.23 0.76 0.79 0.56

G. Polancic et al. / The Journal of Systems and Software 83 (2010) 574–584 579

a 2–3 year period. They have already instantiated 3–5 projectsusing the framework. The respondent typically uses a frameworkdaily.

Table 5Descriptive statistics of items.

Mean Std. Deviation Factorloading

Squared multiplecorrelations

FEOUFEOU1 2.28 1.17 0.68 0.46FEOU2 2.73 1.30 0.89 0.79FEOU3 2.35 1.09 0.81 0.65FEOU4 2.46 1.21 0.68 0.47FEOU5 2.27 1.18 0.79 0.62

FUFU1 1.87 1.07 0.89 0.80FU2 2.03 1.21 0.93 0.86

5.2. Measurement model results

The measurement model is the part of an SEM model that dealswith latent variables and their indicators. To ensure the validityand reliability of the measurement model, the internal consistency,convergent validity and discriminant validity were examined.

The reliability (convergent validity) of the items for each con-struct was computed using Cronbach’s alpha (Table 3). Cronbach’salpha is a commonly used measure that tests the extent to whichmultiple indicators for a latent variable belong together. Cron-bach’s alpha ranged from 0.76 to 0.95, meaning that all constructsexceed the threshold of 0.7 for field research. Moreover, the aver-age variance extracted (AVE) value for each construct was between0.56 and 0.83 (see Table 4), exceeding the minimum value of 0.5 asproposed by Fornell and Larcker (1981), and also indicating goodinternal consistency. Fornell and Larcker also suggested that dis-criminant validity criteria are determined by the AVE value,namely, whether it exceeds the squared correlation between theconstructs. The findings (see Table 4) revealed good discriminantvalidity.

Finally, Anderson and Gerbing (1988) proposed that the assess-ment of convergent validity requires assessing the loading of eachobserved indicator on its latent construct (see Table 5). The CFA re-sults indicated that all loadings were significant (p-value <0.01).Thus, the evidence revealed satisfactory convergent validity. Giventhat the required assessment of reliability and validity in the mea-surement model is satisfactory, the subsequent process of identify-ing the structural model that best fits the data can be detailed asfollows.

In addition to factor loadings, Table 5 presents mean values,standard deviations and squared multiple correlations of observedvariables.

FU3 1.99 1.18 0.94 0.88FU4 1.83 1.06 0.89 0.79

FAFA1 2.40 1.46 0.91 0.82FA2 2.75 1.58 0.84 0.71FA3 2.36 1.48 0.87 0.76FA4 2.84 1.48 0.64 0.42

5.3. Structural model results

Two subactivities were performed in structural model estima-tion. First, we examined the goodness-of-fit of the hypothesizedmodel (FS-Initial) and its submodels (TAM-FA, TAM-CFUI). After-

Table 4Discriminant validity.

FEOU FU FA CFUI SAT NBF

FEOU 0.60FU 0.34 0.83FA 0.25 0.49 0.68CFUI 0.36 0.38 0.21 0.72SAT 0.19 0.26 0.30 0.36 0.66NBF 0.23 0.33 0.41 0.40 0.25 0.56

Note: Diagonals represent average variance extracted (AVE). Other entries representthe shared variance.

wards, improvements to the models, based on AMOS modificationindices, were performed as necessary and a revised model was de-fined (FS-Revised).

Some common fit indices reported in SEM are designed to iden-tify model goodness-of-fit. These indices include the goodness-of-fit index (GFI), the adjusted goodness-of-fit index (AGFI), the com-parative fit index (CFI), the normed fit index (NFI), the Tucker–Le-wis coefficient (TLI), the root mean squared residual (RMR), and theroot mean square error of approximation (RMSEA). The same fitindices were used to analyze the fit of structural models. Thresholdvalues were resumed from Hong et al. (2006). Table 6 shows modelfit indices for evaluated structural models. Values, which exceededtheir limits, are bordered. Table 6 points out that three of the fourinvestigated models fit well with the data. Additionally, Table 6summarizes the squared multiple correlations or explained vari-ance (R2 value) of each construct to demonstrate its explanatorypower.

5.3.1. Tam submodelThe hypothesized structural model (Fig. 4) included two TAM

submodels: (1) the TAM-FA model used ‘‘framework acceptance”for its dependent construct, whereas (2) the TAM-CFUI model(Fig. 5) used ‘‘continuous framework use intention” for its depen-

CFUICFUI1 2.64 1.47 0.75 0.56CFUI2 2.04 1.25 0.92 0.85CFUI3 1.99 1.37 0.86 0.74

NBFNBF1 1.97 1.10 0.91 0.83NBF2 2.01 1.12 0.73 0.53NBF3 2.31 1.48 0.55 0.31

SATSAT1 1.86 0.97 0.88 0.77SAT2 2.26 1.26 0.69 0.48SAT3 2.25 1.06 0.84 0.71

Note: All factor loadings are significant at p < 0.01.

Table 6Overall fit and explanatory power of investigated structural models.

Rec. criteria Submodel TAM-FA Submodel TAM-CFUI FS-Initial FS-Revised

Fit indexGFI P0.90 0.91 0.94 0.83 0.91AGFI P0.80 0.87 0.97 0.78 0.88NFI P0.90 0.92 0.96 0.86 0.94TLI (NNFI) P0.90 0.91 0.96 0.90 0.95CFI P0.90 0.95 0.97 0.91 0.96RMSR 60.10 0.10 0.07 0.14 0.08RMSEA 60.08 0.09 0.07 0.09 0.06X2 (df, p) – 253.98 (62, p < 0.01) 155.52 (51, p < 0.01) 795.97 (200, p < 0.01) 343.04 (128, p < 0.01)

Explanatory power (squared multiple correlations)

R2SAT

– – 0.46 0.59

R2NBF

– – 0.55 0.77

R2FA

0.44 – 0.50 –

R2CFUI

– 0.39 0.46 0.40

R2FU

0.34 0.34 0.34 0.37

Frameworkease of use

(FEOU)

Frameworkusefulness

(FU)R2=0.34

Continuous framework use

intention(CFUI)R2=0.39

0.58***

0.36***

0.34***

Fig. 5. Results of the TAM-CFUI submodel.

Table 7The moderating effect of voluntary and mandatory use in a TAM-CFUI submodel.

Voluntary users Mandatory users All users

FEOU ? FU 0.55*** 0.64* 0.58***

FEOU ? CFUI 0.38*** 0.27*** 0.34***

FU ? CFUI 0.36*** 0.31* 0.36***

R2FU

0.30 0.41 0.34

R2CFUI

0.42 0.28 0.39

* p < 0.05.*** p < 0.001.

580 G. Polancic et al. / The Journal of Systems and Software 83 (2010) 574–584

dent construct. In both models, all hypothesized paths were signif-icant (hypotheses H1–H5), where the TAM-FA explained highervariance (44%) in user intentions compared with the TAM-CFUI(39%). However, the TAM-CFUI model resulted in a better modelfit. We therefore continued to use TAM-CFUI (Fig. 5).

We further used TAM-CFUI to analyze two common factors thatmoderate user intentions (Venkatesh et al., 2003; Venkatesh andDavis, 2000) – user experiences and voluntariness of use. In thecase of user experiences with frameworks, the ANOVA was signif-icant, F (6,385) = 1.95, indicating a significant difference betweenthe groups (levels of experiences as defined in Table 1) at a signif-icant level (p < 0.01).

Second, a T-test was performed on two independent groups ofusers: voluntary and mandatory users. While the sample sizewas high (N > 100), we were able to perform T-test, despite of highskewness and kurtosis of the sample data. The T statistics for allCFUI indicators were significant at p < 0.01, indicating significantdifferences between voluntary and mandatory users when deter-mining CFUI. Table 7 outlines the differences between voluntaryand mandatory users in significance levels, factor loadings and ex-plained variances of TAM-CFUI submodel.

TAM based concepts and measures have demonstrated to bereliable and valid in determining the continuous usage behaviorintention of frameworks - especially within the context of volun-tary users. All the hypothesized relationships of TAM were foundto be significant (hypotheses H1–H5), whereas the strongest rela-tionship was related to H3 (b = 0.58). These findings match withthe results of TAM meta-analysis, performed by King and He(2006). In addition, a TAM version that used a CFUI construct,proved to be a better fit with the data, compared with a TAM ver-sion that used a FA construct.

More, we can confirm the strong and significant impact of FEOUon CFUI (b = 0.34). This finding is a diversion from most TAM stud-ies, which found the relationship weaker and not significant (aver-age b = 0.18). However, our results are similar to the findings ofHong et al. (2006), who examined TAM in a similar, post-adoptioncontext. We therefore presume that the strength of this relation-ship is also due to the complexity of the examined object –frameworks.

The proportion of explained variance (R2) was 0.37 for FU and0.40 for CFUI. This means that most of the variance belongs to fac-tors that were not examined. We believe that TAM external vari-ables and factors incorporated from other theories should beexamined in order to increase TAM’s predictive power.

Summarizing above we found following: (1) CFUI depends onframework users’ experiences and (2) post-adoption version ofTAM can be applied to frameworks, especially if framework useis voluntary. TAM-CFUI model answers positively our first researchquestion.

5.3.2. Framework success modelThe entire hypothesized structural model (Fig. 4) was defined as

the initial framework success model (FS-Initial). The resultsshowed that one hypothesized path (H4) was not significant(FEOU ? FA). The impacts (b) for other paths ranged between0.35 (FA ? SAT) and 0.59 (FEOU ? FU). Comparing these resultsto stated hypotheses it means that all hypotheses except H4 wereconfirmed. The goodness-of-fit for the hypothesized model waspoor, with only two out of seven indices fitting within the recom-mended values. When considering this, and the modification indi-ces reported by AMOS, we modified the initially hypothesizedmodel. As presented on Fig. 6, the resulting model (FS-Revised) re-moved the FA construct, non-significant or weak relationships andincluded two direct relationships (FU ? NBF, FU ? SAT).

The FS-Revised proved a good fit to the data with all fit indexeswithin recommended values. Moreover, the model resulted inbetter explanatory power (R2) compared with the hypothesized

Frameworkease of use

(FEOU)

Frameworkusefulness

(FU)R2=0.37

Continuous framework use

intention(CFUI)

R2=0.400.61***

0.35***

0.35***

Satisfaction(SAT)

R2=0.59

Net benefits(NBF)

R2=0.770.22***

0.26***

0.74***

0.59***

Initially hypothesized relationshipsInitially NOT hypothesized relationships

User beliefs & perceptions User intentions

Framework success measures

Fig. 6. Results of FS-Revised model.

Table 8The moderating effect of voluntariness in the FS-Revised model.

Voluntary users Mandatory users All users

FEOU ? FU 0.57*** 0.65*** 0.61***

FEOU ? CFUI 0.37*** 0.28* 0.35***

FU ? CFUI 0.37*** 0.31* 0.35***

CFUI ? NBF 0.18*** 0.24** 0.22***

CFUI ? SAT 0.24*** 0.25** 0.26***

FU ? NBF 0.77*** 0.72*** 0.74***

FU ? SAT 0.58*** 0.62*** 0.59***

R2FU

0.32 0.42 0.37

R2CFUI

0.43 0.29 0.40

R2NBF

0.79 0.74 0.77

R2SAT

0.56 0.60 0.59

*** p < 0.001.** p < 0.01.* p < 0.05.

G. Polancic et al. / The Journal of Systems and Software 83 (2010) 574–584 581

model (Table 6) especially in the context of voluntary use (Table 8).Comparing FS-Revised relationships to stated hypotheses, themodel confirms following hypotheses: H2, H3, H5, H8 and H9.

Based on the FS-Revised model, the following issues are ofinterest. First, NBF and SAT have demonstrated to be reliable andvalid constructs for framework success. Moreover, self-reportedNBF measures: productivity, quality and lead time have suitedthe criteria for convergent validity, meaning that they all measurethe same construct. When also considering the high proportion ofexplained variance (77% for NBF and 59% for SAT), we believe thatwe have found major antecedents for a framework’s success.

Second, we found that the TAM-CFUI model can be used todetermine a framework’s success. This is because of the already-demonstrated relationship between user intentions and actualbehavior. So, we implicitly confirmed the presumption that systemuse can be used as a system success predictor in systems wheremore use means more net benefits.

Third, we found that a CFUI construct does not completelymediate relationships between user perceptions (FEOU, FU) andframework success constructs (NBF, SAT). Initially, the un-hypoth-esized direct relationships FU ? NBF and FU ? SAT proved to havea strong and significant impact on dependent constructs. In addi-tion, the mediating role of CFUI was less significant within the con-text of mandatory use.

Finally, we found that the most important construct for deter-mining NBF and SAT is FU, where FEOU should be considered asan antecedent of FU.

To summarize, FS-Revised model (Fig. 6) supports our secondresearch question. This means that TAM constructs, as defined byHong et al. (2006), can be used for anticipating framework success.

5.4. Research limits

Interpreting the results of this study should concern followinglimitations. First, we used TAM for the theoretical background ofour study. However, we are aware that other models and corre-sponding concepts could also have a significant impact on frame-works success.

Second, the sample frame used in our survey was obtained fromSourceforge.net database snapshot. Despite the fact that surveyparticipants also evaluated proprietary frameworks, there is noevidence that the sample frame is a typical representative of thepopulation of interest, i.e. all frameworks users. Thus, future stud-ies need to be conducted to investigate and compare different sam-ple frames.

Third, according to software licenses, both major types offrameworks were investigated – open source and proprietary.Despite there are significant differences between these two groupsof frameworks we did not investigate those differences in thisresearch.

The SEM approach was the most reasonable choice in our re-search. If properly employed, it holds great potential for theorydevelopment and constructs validation. However, the readershould be aware that SEM approach has its own limitations, forexample: possible over-fitting, testing of suspected non-lineareffects and suspected influential outliers (Gefen et al., 2000). So,our findings should consider them.

We are also aware that we have only demonstrated that ourmodel has valid constructs, causal relationships and is a good fitwith the empirical data. However, as other unexamined modelsmay fit the data just as well or better, we are aware that an ac-cepted model is only a not-disconfirmed model.

6. Discussion

The objective of this study was to apply the post-adoption ver-sion of technology acceptance model (TAM) to application frame-works in order to explain the acceptance and success of usingframeworks. The data collected from an online survey of 389 activeframework users was used to test the hypothesized models. Over-all, the hypothesized and revised research models demonstrated anacceptable fit with the data. This allowed us to answer stated

582 G. Polancic et al. / The Journal of Systems and Software 83 (2010) 574–584

research questions: (1) we demonstrated that post-adoption ver-sion of TAM can be applied to frameworks and (2) we demon-strated that post-adoption TAM constructs can be used foranticipating framework success.

The major theoretical contributions are that we demonstratedpost-adoption TAM validity in a new context of semi-completedsoftware – frameworks. Besides this, we defined framework suc-cess measures and established relationships between them andTAM constructs. We assume that these relationships are generallyapplicable in systems where more use means more net benefits.

According to research results, several implications for practitio-ners and researchers can be foreseen. First, framework developersshould be aware that the acceptance of frameworks is dependenton framework users perceptions. It means that framework usersshould be incorporated into framework design and development.This is actually the case in open source frameworks developmentand might explain the success of open source frameworks (John-son, 2005).

Second, as researchers already proposed a multitude of frame-work development guidelines (Landin and Niklasson, 1995), ourresults should be incorporated into the analysis of the impacts ofthose guidelines. In other words, these guidelines should be evalu-ated according to the factors, which were investigated in this arti-cle (FEOU, FU and CFUI).

Third, some researchers and practitioners already proposed cri-teria and guidelines for framework selection and evaluation (Fayad

TAM

Prior factors

Perceived usefulness

Perceived ease of use

Contextual factors

suo

1

2

3

Fig. 7. TAM and four categories of m

Analyze FEOU

Develop / maintain a framework

Analyze FU

Framework development

model

Users' feedbacks

Fig. 8. A high-level framewor

and Hamu, 2000; Meier, 1998). According to our research results,we believe that selection of an appropriate framework should in-volve evaluating framework’s FEOU and FU.

Forth, managers in framework based projects should be awarethat the success of using a framework can be anticipated with fu-ture framework usage intentions and with user’s perceptions aboutthe framework. This means that project managers should inquiryfeedbacks from framework users, to anticipate the success of cur-rent and forthcoming projects.

We plan future research in several directions. To increase theapplicability and accuracy of the model, we plan to extend and spe-cialize it into the context of frameworks. According to King and He(2006) this is possible in four directions (see numbered factors onFig. 7).

For researchers and practitioners, knowledge of the determi-nants of FEOU and FU is essential when investigating or developingframeworks. So, we intend to identify major framework-relatedprior factors that have an impact on FEOU and FU. In this area,we will primarily focus on software quality factors, reusability fac-tors and individual differences factors.

Additional, we plan to define a framework improvement pro-cess (Fig. 8) which will act as an extension of current frameworkdevelopment models, such as ‘‘evolving frameworks” (Robertsand Johnson, 1997).

The major idea behind the framework improvement process liesin an iterative process, which analyzes framework users’ feedbacks,

Behavioral intention

Consequest factors

Use

Factors ggested from ther theories 4

odifications (King and He, 2006).

Apply guidelines which improve

FEOU

Apply guidelines which improve

FU

Improvements required

Improvements required

k improvement process.

G. Polancic et al. / The Journal of Systems and Software 83 (2010) 574–584 583

based FEOU and FU. According to the results of the analysis, theprocess identifies appropriate guidelines for improving the investi-gated framework.

Acknowledgments

The authors express their appreciation to the University of No-tre Dame for enabling access to Sourceforge.net research data. Theauthors would also like to thank the survey participants for theiranswers and critical feedback.

Appendix

Constructs, items and sources of measures.

� FEOU: Framework Ease of use – Adapted from (Moore and Ben-basat, 1991)s FEOU1: The framework is rigid and inflexible to interact with.s FEOU2: I find it is easy to get the framework to do what I

want it to do.s FEOU3: Overall, I believe that the framework is easy to use.s FEOU4: Learning to operate the framework is easy for me.s FEOU5: I find it takes a lot of effort to become skillful at using

the framework.

� FU: Framework Usefulness – Adapted from (Moore and Benba-sat, 1991)s FU1: I believe that using the framework will further increase

my productivity.s FU2: I believe that using the framework will further increase

my job performance.s FU3: I believe that using the framework will further enhance

my job effectiveness.s FU4: Overall, I believe the framework will be further useful in

my job.

� FA: Framework Acceptance – Adapted from (Goodhue andThompson, 1995)s FA1: I have fully accepted the framework in my daily work.s FA2: I feel that the framework constitutes an integral part of

my daily work.s FA3: I consider myself a frequent user of the framework.s FA4: I fully use the capabilities of the framework.

� CFUI: Continued framework usage intention – Adapted from(Hong et al., 2006)s CFUI1: I intend to increase my use of the framework in the

future.s CFUI2: I intend to continue my use of the framework in the

future.s CFUI3: I am not going to use the framework in the future.

� NBF: Framework’s net benefits – New scaless NBF1: The framework increases the productivity of software

development.s NBF2: The framework increases the quality of software

development.s NBF3: The framework lowers the lead time for developing

software.

� SAT satisfaction – Adapted from (Hong et al., 2006)s SAT1: How satisfied do you feel about your overall experi-

ence with the framework use?s SAT2: How contented do you feel about your overall experi-

ence with the framework use?

s SAT3: How delighted do you feel about your overall experi-ence with the framework use?

References

Anderson, J.C., Gerbing, D.W., 1988. Structural Equation Modeling in Practice – AReview and Recommended 2-Step Approach. Psychological Bulletin (3), 411–423.

Avlonitis, G.J., Panagopoulos, N.G., 2005. Antecedents and consequences of CRMtechnology acceptance in the sales force. Industrial Marketing Management (4),355–368.

Batory, D., Cardone, R., Smaragdakis, Y., 2000. Object-Oriented Frameworks andProduct Lines. In: Donohoe, P. (Ed.), First Software Product Line Conference, pp.227–247.

Bockle, G., Clements, P., McGregor, J.D., Muthig, D., Schmid, K., 2004. Calculating ROIfor software product lines. IEEE Software 21 (3), 23–31. Available from: ISI:000220988700010.

Booch, G., 1994. Design an application framework. DR Dobbs Journal (2),24–32.

Bosch, J., Molin, P., Mattsson, M., Bengtsson, P., 1999. Object-Oriented Frameworks –Problems and Experiences. In: Fayad, Schmidt, Johnson (Eds.), BuildingApplication Frameworks: Object Oriented Foundations of Framework Design.Wiley and Sons.

Churchill, G.A., 1979. Paradigm for developing better measures of marketingconstructs. Journal of Marketing Research (1), 64–73.

Cunningham, H.C., Liu, Y., Zhang, C.H., 2006. Using classic problems to teach Javaframework design. Science of Computer Programming 59 (1–2), 147–169.Available from: ISI: 000233946500010.

Davis, F.D., 1989. Perceived Usefulness, Perceived Ease Of Use, And User AcceptanceOf Information Technology. MIS Quarterly, 13 (3), 318–331. Aavailable from:http://business.clemson.edu/ISE/html/perceived_usefulness__perceive.html.

Davis, F.D., Bagozzi, R.P., Warshaw, P.R., 1989. User Acceptance of Computer-Technology – A Comparison of 2 Theoretical-Models. Management Science (8),982–1003. Available from: ISI: A1989AL89000005.

Dishaw, M.T., Strong, D.M., 1999. Extending the technology acceptance model withtask-technology fit constructs. Information and Management (1), 9–21.Available from: ISI: 000080707400002.

Fayad, M., Hamu, D., 2000. Enterprise frameworks: guidelines for selection. ACMComputing Survey (1), 4.

Fornell, C., Larcker, D.F., 1981. Evaluating structural equation models withunobservable variables and measurement error. Journal of MarketingResearch (1), 39–50.

Frakes, W.B., Kang, K., 2005. Software reuse research: status and future. IEEETransactions on Software Engineering (7), 529–536.

Froehlich, G., Hoover, J., Liu, L., Sorenson, P., 1998. Designing object-orientedframeworks. In: CRC Handbook of Object Technology. CRC Press.

Gefen, D., Straub, D.W., Bordeau, M.-C., 2000. Structural Equation Modeling andRegression: Guidelines for Research and Practice. Communications of AIS (7), 1–70.

Goodhue, D.L., Thompson, R.L., 1995. Task-Technology Fit and Individual-Performance. MIS Quarterly (2), 213–236. Available from: ISI:A1995RH14500005.

Hong, S.J., Thong, J.Y.L., Tam, K.Y., 2006. Understanding continued informationtechnology usage behavior: a comparison of three models in the context ofmobile internet. Decision Support Systems (3), 1819–1834.

Johnson, R.E., 1992. Documenting frameworks using patterns. Sigplan Notices (10),63–76. Available from: ISI: A1992JU12100006.

Johnson, R.E., 2005. J2EE development frameworks. Computer (1), 107–110.Available from: ISI: 000226193800020.

Johnson, R.E., Foote, B., 1988. Designing Reusable Classes. Journal of Object-OrientedProgramming (2), 22–30. Available from: ISI: A1988Q223800003.

King, W.R., He, J., 2006. A meta-analysis of the technology acceptance model.Information and Management (6), 740–755.

Landin, N., Niklasson, A., 1995. Development of Object-Oriented Frameworks. LundInstitute of Technology, Lund University, Department of CommunicationSystems.

Lederer, A.L., Maupin, D.J., Sena, M.P., Zhuang, Y., 2000. The technologyacceptance model and the World Wide Web. Decision Support Systems (3),269–282.

Legris, P., Ingham, J., Collerette, P., 2003. Why do people use information technology?A critical review of the technology acceptance model. Information andManagement (3), 191–204. Available from: ISI: 000180179200005.

Mamrak, S.A., Sinha, S., 1999. A case study: productivity and quality gains using anobject-oriented framework. Software-Practice and Experience (6), 501–518.Available from: ISI: 000080323300001.

Manolescu, D., Noble, J., Voelter, M., 2006. Patterns for Successful Object-orientedFramework Development. In: Pattern Languages of Program Design 5, first ed.Addison-Wesley Professional, pp. 401–431.

Mattsson, M., 1999. Effort distribution in a six year industrial applicationframework project. IEEE Computer Society, 326–334.

Mattsson, M., 2000. Evolution and Composition of Object-Oriented Frameworks.Department of Software Engineering and Computer Science – University ofKarlskrona/Ronneby.

McBurney, D.H., White, T.L., 2003. Research Methods, sixth ed. WadsworthPublishing.

584 G. Polancic et al. / The Journal of Systems and Software 83 (2010) 574–584

Meier, P., 1998. Critical Success Factors of Object Oriented Frameworks. In: STJA 98,pp. 1–10.

Mellarkod, V., Appan, R., Jones, D.R., Sherif, K., 2007. A multi-level analysis of factorsaffecting software developers’ intention to reuse software assets: an empiricalinvestigation. Information and Management (7), 613–625.

Moore, G.C., Benbasat, I., 1991. Development of an instrument to measure theperceptions of adopting an information technology innovation. InformationSystems research (3), 192–222.

Morisio, M., Ezran, M., Tully, C., 2002a. Success and failure factors in software reuse.IEEE Transactions on Software Engineering (4), 340–357. Available from: ISI:000174814100002.

Morisio, M., Romano, D., Stamelos, I., 2002b. Quality, productivity, and learning inframework-based development: an exploratory case study. IEEE Transactionson Software Engineering (9), 876–888. Available from: ISI: 000177805300006.

Moser, S., Nierstrasz, O., 1996. The effect of object-oriented frameworks on developerproductivity. Computer (9), 45–52. Available from: ISI: A1996VF54200012.

Roberts, D., Johnson, R., 1997. Patterns for evolving frameworks. In: Patternlanguages of program design 3. Addison-Wesley Longman Publishing Co. Inc.,pp. 471–486.

Saga, V.L., Zmud, R.W., 1994. The nature and determinants of it acceptance,routinization, and infusion. Diffusion, Transfer and Implementation ofInformation Technology 45, 67–86.

Sangdon, L., Hansuk, C., Youngjong, Y., Sangduck, L., 1999. Storage and managementof object-oriented frameworks. In: International Conference on Systems, Man,and Cybernetics, pp. 762–767.

Seddon, P.B., 1997. A respecification and extension of the DeLone and McLeanmodel of IS success. Information Systems Research (3), 240–253. Availablefrom: ISI: A1997YA62700002.

Sharp, J.H., 2007. Development, extension, and application: a review of thetechnology acceptance model. Information Systems Education Journal (9), 1–9(Accessed 2007).

Shaw, M., 1990. Prospects for An Engineering Discipline of Software. IEEE Software(6), 15–24. Available from: ISI: A1990EE94200004.

Srinivasan, S., 1999. Design patterns in object-oriented frameworks. Computer32(2), 24–+. Available from: ISI: 000078322200016.

Tonella, P., Torchiano, M., Du Bois, B., Systa, T., 2007. Empirical studies in reverseengineering: state of the art and future trends. Empirical Software Engineering(5), 551–571.

van Gurp, J., Bosch, J., 2001. Design, implementation and evolution of objectoriented frameworks: concepts and guidelines. Software-Practice andExperience (3), 277–300. Available from: ISI: 000167468300004.

Venkatesh, V., Davis, F.D., 2000. A theoretical extension of the technologyacceptance model: four longitudinal field studies. Management Science (2),186–204. Available from: ISI: 000086130700002.

Venkatesh, V., Morris, M.G., Davis, G.B., Davis, F.D., 2003. User acceptance ofinformation technology: toward a unified view. MIS Quarterly (3),425–478.

Dr. Gregor Polancic is an assistant professor at the Institute of Informatics. Hereceived his PhD in Computer Science from the University of Maribor in 2008. Hismain research interests are: (1) analysis and design of software, (2) web applica-tions, communities, architectures and patterns, (3) FLOSS software, projects anddevelopment models, (4) business process modelling, informatization and re-

engineering and (5) computer mediated communication and collaboration. Dr.Polancic has been a work co-ordinator/member of several applied projects andwork co-ordinator/member in several international research projects. Dr. Polancichas appeared as an author or co-author in six peer-reviewed scientific journals. Inall, his bibliography contains over 90 records.

Dr. Marjan Hericko is a full professor at the Institute of Informatics. He is the headof the information systems laboratory and deputy head of the Institute of Infor-matics. He received his PhD in Computer Science from the University of Maribor in1998. His main research interests include all aspects of information systemsdevelopment, software and service engineering, agile methods, process frame-

works, software metrics, functional size measurement, SOA, component-baseddevelopment, object-orientation, software reuse and software patterns. He is thescientific coordinator of NEESI Slovenia – Slovenian technology platform for soft-ware and services. Dr. Hericko has been a project/work co-ordinator in severalapplied projects, project/work co-ordinator in several international research pro-jects and a committee member and chair of several international conferences. Dr.Hericko has published papers (as author or co-author) in 46 peer-reviewed scien-tific journals. In all, his bibliography embraces over 380 records.

Dr. Ivan Rozman is a full professor at the Institute of Informatics. He received thePhD degree from University of Maribor in 1983. His interests include qualityassessment, project management, software metrics and software design. He hasbeen involved in numerous commercial and government projects in many of themas a project manager. Dr. Rozman is author and co-author of numerous articles

published in different scientific journals and a member of several program com-mittees at domestic and international conferences. Dr. Rozman is currently therector of the University of Maribor and head of the Institute of Informatics.