cresst onr/netc meetings, 17-18 july 2003 17 july, 2003 onr advanced distributed learning greg...
TRANSCRIPT
CRESST ONR/NETC Meetings, 17-18 July 2003
17 July, 2003
ONR Advanced Distributed LearningONR Advanced Distributed Learning
Greg Chung Allen Munro
Bill Bewley Quentin Pizzini
Girlie Delacruz USC/BTL
Terry Vendlinski
Zenaida Aguirre-Munoz
UCLA/CRESST
USN EDO Application
2003 Regents of the University of California
2CRESST ONR/NETC Meetings, 17-18 July 2003
CRESST/BTL/USN EDO CRESST/BTL/USN EDO CollaborationCollaboration
• Research to develop new kinds of assessment and instructional tools to support USN EDO school
• Initial domain: Risk management within the integrated case study
• Tool prototypes:– Knowledge mapping (assessment)– Decision-analysis (instruction)
3CRESST ONR/NETC Meetings, 17-18 July 2003
Assessment FocusAssessment Focus
• Knowledge mapping as a way to measure problem solving– Examine students’ understanding
of the decision analysis process– Examine the extent to which
domain-independent links can be used to measure problem solving processes
4CRESST ONR/NETC Meetings, 17-18 July 2003
ContextContext
• Integrated Case Study (ICS)– Problem-based learning– Teams are given broad
requirements– Deliverable is a professional-
level briefing•Problem solving, teamwork,
communication, prior job experience
– Concepts covered in instruction are put into practice in ICS
5CRESST ONR/NETC Meetings, 17-18 July 2003
DomainDomain
• Risk Management– Complex — multiple variables to
consider, broad requirements, imperfect information, changing conditions
– No “correct” answer — trade-offs among cost, schedule, and performance variables
– Reasoning and analytical processes are critical to success
6CRESST ONR/NETC Meetings, 17-18 July 2003
The ToolsThe Tools
• Knowledge maps– Students depict their understanding of
the decision process underlying the refueling at sea (RAS) “hiccup”
– Students’ maps can be compared to an expert’s map for assessment purposes
• Decision-analysis– Tool permits quantitative or qualitative
views of problem solution alternatives– Instruction can be delivered in the
context of the tool
7CRESST ONR/NETC Meetings, 17-18 July 2003
Decision Analysis TemplateDecision Analysis Template
Options
Analysis
Selection
Domain Variables
Environment
8CRESST ONR/NETC Meetings, 17-18 July 2003
Risk ManagementRisk Management
Environment
Selection
Options
Analysis
Cost
Schedule
Performance
9CRESST ONR/NETC Meetings, 17-18 July 2003
Risk Management: Risk Management: Refueling at Sea HiccupRefueling at Sea Hiccup
• Existing system• Recompete• Full inventory buy• Technical data plan buy• Full + TDP buy
Selection
Analysis
Cost
Schedule
Performance
• No. ships• Time
constraints
• Config. Mgt• Manufacturing• Contracting• Supportability• T&E
OptionsEnvironment• Requirements• Regulations• Legal issues
10CRESST ONR/NETC Meetings, 17-18 July 2003
Assessment ToolsAssessment Tools
• How do we measure an EDO’s problem solving processes within the risk management domain?– Knowledge maps– Online processes and decisions
with decision-analysis simulation tool
11CRESST ONR/NETC Meetings, 17-18 July 2003
Refueling At Sea (RAS) ScenarioRefueling At Sea (RAS) Scenario
12CRESST ONR/NETC Meetings, 17-18 July 2003
Refueling At Sea (RAS) ScenarioRefueling At Sea (RAS) Scenario
Options
Analyze
Select
Schedule
Cost
Performance
Environment
13CRESST ONR/NETC Meetings, 17-18 July 2003
Refueling At Sea (RAS) ScenarioRefueling At Sea (RAS) Scenario
14CRESST ONR/NETC Meetings, 17-18 July 2003
Current StatusCurrent Status
• Instructor consensus knowledge map created for RAS scenario
• Domain-independent links– Harder for processes (vs. declarative)– Level of abstraction issues
• Domain-independent: context• Domain-dependent: requirement, legal,
regulation
15CRESST ONR/NETC Meetings, 17-18 July 2003
Next StepsNext Steps
• Pilot test at EDO with EDO instructor and EDO class (July/Aug)
• Evaluate and revise for next class
16CRESST ONR/NETC Meetings, 17-18 July 2003
Decision Aiding Toolhttp://btl.usc.edu/WS/rasProb/rasProb.jnlp
17CRESST ONR/NETC Meetings, 17-18 July 2003
Decision Making and Risk Management
• A major aspect of EDO education• The Refueling at Sea (RAS) problem
– Evaluating options systematically
• A tool that promotes systematic assessments of risks and remediations
• A tool that teaches about itself and about risk and decision-making
18CRESST ONR/NETC Meetings, 17-18 July 2003
Decision Aiding Tool Demo
TDP = Technical Data Package
TDP = Technical Data Package
T&E = Testand Evaluation
T&E = Testand Evaluation RAS = Refueling
at Sea systemRAS = Refuelingat Sea system
Decision ThresholdsController
Decision ThresholdsController
User created nodeshave arbitrary names.User created nodeshave arbitrary names.
User can set estimatedprobabilities, utilities.User can set estimatedprobabilities, utilities.
RAS = Refuelingat Sea system
RAS = Refuelingat Sea system
19CRESST ONR/NETC Meetings, 17-18 July 2003
Decision Aiding Tool Instruction
20CRESST ONR/NETC Meetings, 17-18 July 2003
Analyzing the UserAnalyzing the User
• Application of Chung’s ‘Click-through’ interface.
21CRESST ONR/NETC Meetings, 17-18 July 2003
General ApplicationGeneral Application
• Instructional approach can provide opportunities for students to engage in systems engineering thinking
• Assessment approach can provide information on what people know and understand about processes