Beyond Task/Technology Fit: How Information Technology
Affects PerformanceBy Transforming the Task
Dale L. Goodhue
Stefano Grazioli
Barbara D. Klein
Outline
• A comparison of the previous way of conceptualizing TTF with a new way -- Different technologies present the task doer with different options for task completion processes – some of which are more “attractive” than others.
• An experiment applying these ideas to the task of accessing information from integrated or non-integrated databases
Expanding the Task/Technology Fit Perspective• Original insight from TTF: technology
improves performance when the technology “fits” the task
• Use alone is not enough!• What is “fit”, and how does it improve
performance?• How does a technology improve
performance at a task?
IndividualCharacteristics
TaskCharacteristics
Use
Individual Performance
TechnologyCharacteristics
The Technology-to-Performance Chain (Goodhue and Thompson, 1995)
Task-Technology
Fit
IndividualPerceptions/ Beliefs
A Different Perspective:Two Different Tasks?
• Organizational researchers don’t distinguish between technology and task; see task as presented to the task doer (after the application of technology)
• TTF researchers see task as existing before the application of technology
• There are two tasks! The underlying task and the task as presented to the task doer.
• Technology changes the task as presented to the task doer
Task As Underlying
Problem or Motivation
Technology
Task As Sequence
of Actions Used To Meet the
Task Need
Perrow, Fry and Slocum Actions used to transform inputs into outputs
Wood
Required acts and info cues, etc.
Jarvenpaa (89) Choose a restaurant using different choice rules.
Vessey & Galleta (91) Determine point values vs. relationships
Goodhue (95) Meet different mgmt info requirements
McGrath (1984) Task circumplex
Task as Problem Task as Solution
There are two tasks!
Task As Underlying
Problem or Motivation
Technology
1
Sequence A for Actions
To Meet the Task Need
Sequence B for Actions
To Meet the Task Need
Technology
2
Sequence C for Actions
To Meet the Task Need
Sequence D for Actions
To Meet the Task Need
Sequence E for Actions
To Meet the Task Need
Changing the technology,Changes the strategy options for task completion (the possible action sequences)
Different strategy options have different “attractiveness”
A technology that makes possible an “attractive” strategy option has high TTF
Task as Problem
Task as Solution
A Simple Example: Which Technology Will be Chosen, Which
Gives Better Performance and Why? • Task -- Decide if either of two divisions is making
excessive use of high cost shipping alternatives.
• Three Different Technologies – Paper based systems with all original documents
– Division specific accounting database systems
– Integrated accounting database system
How conceptualize and measure TTF of 3 Systems?
The Old Way: • Decide what the task requirements are. For information
access, they might be:
– right data, – right level of detail, – easy to locate, – understandable meaning, – accessiblity, – reliable systems, – training, – assistance, – accuracy, – currency, – compatibility, – Etc.
TTF the Old Way• Now, rate the three technologies on meeting task needs for these dimensions. The
best technology has highest TTF Div Spec Integr.
Paper DB DB
– right data, high high med– right level of detail, high low low– easy to locate, low high high– understandable meaning, high high high– accessiblity, low high high– reliable systems, high high high– training, high med low– assistance, high med med– accuracy, high high high– currency, high high high– compatibility, low low high– Etc.
• Examine the task process (the actions needed) when using each of the three systems to carry out the task.
• Characterize the “attractiveness” of the three ways of accomplishing the task.
• The “attractiveness” is the TTF of each technology for that task.
How conceptualize and measure TTF of 2 Systems?
A New Way:
Technologies Change the Processing Options
Presented to Task Doer 3 Different Technologies
Situation 1: Paper Documents for Each Transaction
Decide if either of two divisions is making excessive use of high cost shipping alternatives.
(Recover aggregate info for each division from records of shipping transactions)
For both divisions, manually select all shipping transactions, translate to problem categories, and consolidate.
Task Presented to Task Doer(or Technology/Processing Options)
Above, or: For each division separately: translate acctg DB system categories to problem categories, use queries to gather totals for relevant acctg categories, consolidate.
Above, or: For both divisions combined: translate acctg DB system categories to problem categories, use queries to gather totals for relevant categories, consolidate.
Situation 2: Separate Accounting DB Systems for Each Division
Situation 3: Integrated Accounting DB Systems Across Both Divisions
Underlying Task
How Characterize the “Attractiveness” of Different Processing Options?
• Narrow the focus to “intellective tasks” (McGrath 1984): solving a problem that has a correct answer (not psycho-motor, creativity, planning, etc.)
• What is it about a processing option that is changed by technology and task, and affects performance?– Task complexity (Wood 1986):
• Component complexity: How many distinct actions, information cues?• Coordinative complexity: How many precedence relationships?• Dynamic complexity: How fast is the underlying reality changing
– Difficulty (Campbell 1988): reliance upon skills, abilities, experience of individual task doer
– Task complexity is independent of the task doer, difficulty is dependent on task doer.
How Characterize the “Attractiveness” of Different Processing Options?
• Question: Can we really capture the essential differences between strategy options using task complexity and difficulty?
How Characterize the “Attractiveness” of Different Processing Options?
• Question: Can we really capture the essential differences between strategy options using task complexity and difficulty?
• Answer: Perhaps, if the strategy options are not too different.
Why Go to All this Trouble?
• Humans choose technologies on the basis of the most attractive task processing option, not the best technology characteristics
• The more they know about how the technology works, the more this is true
• Individual performance as well is a function of the technology/task process option chosen and its attractiveness
IndividualCharacteristics
TaskCharacteristics
Choice of One Processing Option (and the associated Technology)
Individual Performance
TechnologyCharacteristics
The Task Transformation Model
A Set of Processing
Options, Each With It’s
Attractiveness(Task
Complexity, Difficulty?)
IndividualPerceptions/
Beliefs
Task as problem Task as Solution
Data Integration• Definition: standardization of data definitions and
structures across a collection of data sources
• Assumption: when questions require data from multiple sources, DI should reduce manual and intellectual retrieval effort
• Important part of value of ERP and DW is provision of integrated data
• No scientific assessment of how, or if assumption is true!
Non-Integrated EnvironmentDivision A Division B Comments
1. Codes for PartNumbers:Codes for 3/4"BOLT
115899 337189 Potentiallydifferent codesfor same part
2. Codes forCustomer_ID:Codes for ABC, Inc.
42765 42675,49345,47293
Potentiallydifferentstructure ofcodes
3. Codes andDefinitions forAccounts Showing
Sales and Sales Expenses
301 GROSS SALES (net of returns and allowances)302 SALES DISCOUNTS726 ADVERTISING 727 PROMOS, MAILINGS
301 GROSS SALES (net of sales discounts)401 RETURNS AND ALLOWANCES713 SML ACNT SALES EXPENSES: advert./promo.723 LRG ACNT SALES EXPENSES: advert./promo.
Potentiallydifferentdefinitionschemes
Examples of Non-Integrated Data
How Does Data Integration Change the Task Complexity of the Information Retrieval Task?
• To understand this we need a model of the information retrieval task.
• Then focus on the impact of DI on component complexity and coordinative complexity of different sub-processes in that overall task
Example Task
• Management is concerned about ratio of advertising and promotions expenses to sales revenue in two divisions
• task doer is asked to find (for each division)– year-to-date advertising and promotion
expenses – year-to-date sales (net of discounts, returns and
allowances)
Division A (65 Account Codes) Division B (65 Account Codes)..301 GROSS SALES (net of returns and allowances)302 SALES DISCOUNTS..726 ADVERTISING 727 PROMOS, MAILINGS..
.
.301 GROSS SALES (net of sales discounts)..401 RETURNS AND ALLOWANCES..713 SML ACNT SALES EXPENSES: advert./promo.723 LRG ACNT SALES EXPENSES: advert./promo.
Account Codes in A Non Integrated Data Environment
2. Semantic Specification
3. Syntactic Specification
Query Processor
5. Error Repair
4. Error Detection
Overall Result: -accuracy -time required
Process Model of Information Retrieval
1. Split Off One Subtask
7. ConsolidateSubtask Results
Problem Statement Data Environment
6. Any More Subtasks?
Error DetectedNo ErrorsDetected
No
Yes, Repeat Steps 1-6
Query Displayed Results
Hypotheses For Non Integrated Data• Subprocess 1: greater component complexity (more data items to consider)
will encourage task doers to sub-divide the task into more subtasks.• Subprocess 2: greater component complexity (more data items to consider)
makes it more likely task doer will misclassify at least one data item, in total task.
• Subprocess 3: less component complexity (fewer elements in a less complex query) makes it less likely task doer will make syntax or logic errors in any given query.
• Subprocess 3, more precedence requirements (keeping straight which database) make it more likely task doer will confuse or mis-specify the database.
Impact of Number of Sub-tasks and Error Profiles on Performance
• Time to complete will increase with the number of sub-tasks
• Time to complete will increase with the number of high feedback errors (syntax and logic) in the total set of queries used
• Likelihood of totally correct answers will decrease with the existence of one or more low feedback errors (neglecting a needed account code or included a non-needed account code)
Impact of Data Integration on Performance
Different Mix of Processing Options
Performance - accuracy - time
Choice of OptionWith Given Task Complexity
UnderlyingTask
Integrated vs. non Integrated DB
Number and Type of Errors
Size of Sub-Queries
Method• 107 student pairs
• Given: managerial questions, SQL query processor, and either integrated or non integrated database
• 4 sessions: 2 training, 2 with treatments
• Captured time to complete, accuracy, and every query submitted (1164 queries)
• LISP program determined subtasks used and error profiles of each query
• (Kappa coefficient of agreement between LISP and human coders: .93 or excellent)
Type ofError
Description Examples Frequency>= 1 error in214 Sessions
High FeedbackErrorsSyntax Violation of syntax
rules, misspellings,etc.
WEHRE acctcode = ‘301’ oracctcode = ‘302’) (Should beWHERE)
63%
Logic Misusing the logicof the WHEREclause
Where acctcode < ‘300’ and acctcode> ‘303’ (No account codes in range)
34%
Database Looking for Div Aacct code in Div Bdatabase
Where acctcode = ‘401’ (whenseeking Div A info – acctcode 401exists only on the Div B database)
9%
Low FeedbackErrorsSelectLowFeedback
Specifyingplausible butwrong field
Select acctcode, Month_to_Date(instead of “Select Year_to_Date”)
43%
UnderSpec.
Leaving out aneeded category
Where acctcode = ‘301’(when 302 or 401 is also needed)
42%
OverSpec.
Adding anincorrect category
Where acctcode = ‘301’ or acctcode =‘302’ or acctcode = ‘920’(when 920 should not be included)
38%
Interesting Aspects of Analysis• Analyzed only the first query attempt at any
subtask. (Remainder are error correction queries and much harder to predict.)
• Distribution of errors was highly skewed. Many made no errors. Inappropriate as dependent variable in regression. Used Logistic Regression for those analyses
• Used pair characteristics as additional explanatory variables: tendency to speed, tendency to accuracy
Impact of Integrated Data
Fewer Subtasks
At the Individual Query Level
At the Total Task Level
Mixed Impact on Erroneously Includingor Excluding Account Codes (Low Feedback Errors)
On UltimatePerformance
Fewer Subtasks Leads to Shorter Time to Complete
Fewer Erroneously Included or ExcludedAccount Codes (Low Feedback Errors)
Many More Logic Errors But Only a Hint of More Syntax Errors (High Feedback Errors)
More Logic Errors But No More Syntax Errors (High Feedback Errors)
Fewer Low FeedbackErrors Leads to Greater Accuracy
More Logic Errors Has No Impact On Time To Complete(More Syntax Errors Does Impact Time)
Research Implications• We can understand task doers’ choice of technologies and the impact
on individual performance by considering the “attractiveness” of the processing options provided by different technologies
• We should focus on the process of carrying out the task, not on the characteristics of the technology
• We can use “task complexity” to understand the better TTF of DI for multi-division tasks
• When we do, we see that DI does not improve performance at the query level, but allows task doers to “take larger bites” with each query, and use fewer queries, hence making fewer hard to catch errors and taking less time overall.
• In this way DI reduces time and increases accuracy for data retrieval