chief chefs of z to alloy: using a kitchen example to ... - tarkan/tfm09tarkan.pdf · introduction...
TRANSCRIPT
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Chief Chefs of Z to Alloy: Using A KitchenExample to Teach Alloy with Z
Sureyya Tarkan Vibha Sazawal
University of Maryland
November 6, 2009
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Outline
Introduction
Z to Alloy Tutorial
Case Studies with Novice Students
Discussion
Conclusion
Related Work
ReferencesTarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
The Problem
I Z is a well-defined and well-known specification language.
I Model checkers enable automatic checking of specifications.
I The lack of student interest in Formal Methods causeseducators to focus only on one aspect.
I Educators need a technique to master tool usage besides Zwithout burdening the students to learn a new language.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
The Problem
I Z is a well-defined and well-known specification language.
I Model checkers enable automatic checking of specifications.
I The lack of student interest in Formal Methods causeseducators to focus only on one aspect.
I Educators need a technique to master tool usage besides Zwithout burdening the students to learn a new language.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
The Problem
I Z is a well-defined and well-known specification language.
I Model checkers enable automatic checking of specifications.
I The lack of student interest in Formal Methods causeseducators to focus only on one aspect.
I Educators need a technique to master tool usage besides Zwithout burdening the students to learn a new language.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
The Problem
I Z is a well-defined and well-known specification language.
I Model checkers enable automatic checking of specifications.
I The lack of student interest in Formal Methods causeseducators to focus only on one aspect.
I Educators need a technique to master tool usage besides Zwithout burdening the students to learn a new language.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Contribution
I Alloy is a model checking tool that provides a languagesubstantially similar to Z.
I There is a lack of examples and educational material for thetransition from Z to Alloy.
I To populate the materials, we suggest the use of an easy andyet comprehensive and self-explanatory example.
I We share our experiences with novice students using ouronline tutorial designed around a Kitchen Environmentreal-life example.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Contribution
I Alloy is a model checking tool that provides a languagesubstantially similar to Z.
I There is a lack of examples and educational material for thetransition from Z to Alloy.
I To populate the materials, we suggest the use of an easy andyet comprehensive and self-explanatory example.
I We share our experiences with novice students using ouronline tutorial designed around a Kitchen Environmentreal-life example.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Contribution
I Alloy is a model checking tool that provides a languagesubstantially similar to Z.
I There is a lack of examples and educational material for thetransition from Z to Alloy.
I To populate the materials, we suggest the use of an easy andyet comprehensive and self-explanatory example.
I We share our experiences with novice students using ouronline tutorial designed around a Kitchen Environmentreal-life example.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Contribution
I Alloy is a model checking tool that provides a languagesubstantially similar to Z.
I There is a lack of examples and educational material for thetransition from Z to Alloy.
I To populate the materials, we suggest the use of an easy andyet comprehensive and self-explanatory example.
I We share our experiences with novice students using ouronline tutorial designed around a Kitchen Environmentreal-life example.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Tutorial Content
Birthday Book
I It records people’s birthdays and places a reminder when thatday comes.
I It can be populated and depopulated, and there is a searchoption.
I Its Z specification reviews concepts from set theory andteaches the notation.
I An Alloy model is used to automatically check someassertions.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Tutorial Content
Birthday Book
I It records people’s birthdays and places a reminder when thatday comes.
I It can be populated and depopulated, and there is a searchoption.
I Its Z specification reviews concepts from set theory andteaches the notation.
I An Alloy model is used to automatically check someassertions.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Tutorial Content
Birthday Book
I It records people’s birthdays and places a reminder when thatday comes.
I It can be populated and depopulated, and there is a searchoption.
I Its Z specification reviews concepts from set theory andteaches the notation.
I An Alloy model is used to automatically check someassertions.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Tutorial Content
Birthday Book
I It records people’s birthdays and places a reminder when thatday comes.
I It can be populated and depopulated, and there is a searchoption.
I Its Z specification reviews concepts from set theory andteaches the notation.
I An Alloy model is used to automatically check someassertions.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Tutorial Content
Kitchen Environment
I It simulates the actions of a kitchen chef to direct othercook(s) in the preparation of a dish.
I The primitive object types and built-in functions are given.
I We define a list of assertions that are crucial for our system.
I An event-driven programming approach is suggested.
I We develop an approximate Z specification to later convert itinto a full Alloy model.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Tutorial Content
Kitchen Environment
I It simulates the actions of a kitchen chef to direct othercook(s) in the preparation of a dish.
I The primitive object types and built-in functions are given.
I We define a list of assertions that are crucial for our system.
I An event-driven programming approach is suggested.
I We develop an approximate Z specification to later convert itinto a full Alloy model.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Tutorial Content
Kitchen Environment
I It simulates the actions of a kitchen chef to direct othercook(s) in the preparation of a dish.
I The primitive object types and built-in functions are given.
I We define a list of assertions that are crucial for our system.
I An event-driven programming approach is suggested.
I We develop an approximate Z specification to later convert itinto a full Alloy model.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Tutorial Content
Kitchen Environment
I It simulates the actions of a kitchen chef to direct othercook(s) in the preparation of a dish.
I The primitive object types and built-in functions are given.
I We define a list of assertions that are crucial for our system.
I An event-driven programming approach is suggested.
I We develop an approximate Z specification to later convert itinto a full Alloy model.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Tutorial Content
Kitchen Environment
I It simulates the actions of a kitchen chef to direct othercook(s) in the preparation of a dish.
I The primitive object types and built-in functions are given.
I We define a list of assertions that are crucial for our system.
I An event-driven programming approach is suggested.
I We develop an approximate Z specification to later convert itinto a full Alloy model.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Tutorial Content
Kitchen Schema and Signature
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Tutorial Content
Kitchen Environment
I At each step, we point out the differences and similaritiesbetween the two versions.
I We explain what the limitations of model checking are.
I We formalize a guarded event implementation.
I Our final Alloy model is long but once a rough Z specificationis at hand, Alloy development is not cumbersome.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Tutorial Content
Kitchen Environment
I At each step, we point out the differences and similaritiesbetween the two versions.
I We explain what the limitations of model checking are.
I We formalize a guarded event implementation.
I Our final Alloy model is long but once a rough Z specificationis at hand, Alloy development is not cumbersome.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Tutorial Content
Kitchen Environment
I At each step, we point out the differences and similaritiesbetween the two versions.
I We explain what the limitations of model checking are.
I We formalize a guarded event implementation.
I Our final Alloy model is long but once a rough Z specificationis at hand, Alloy development is not cumbersome.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Tutorial Content
Kitchen Environment
I At each step, we point out the differences and similaritiesbetween the two versions.
I We explain what the limitations of model checking are.
I We formalize a guarded event implementation.
I Our final Alloy model is long but once a rough Z specificationis at hand, Alloy development is not cumbersome.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Implementation
Design
I With Kolb’s theory [4], we decided to have multiple-choicequestions within the tutorial.
I The questions ask the student to think beyond what is givenrather than passively reading the text.
I The students reveal the correct answer once they are satisfiedwith their answer.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Implementation
Design
I With Kolb’s theory [4], we decided to have multiple-choicequestions within the tutorial.
I The questions ask the student to think beyond what is givenrather than passively reading the text.
I The students reveal the correct answer once they are satisfiedwith their answer.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Implementation
Design
I With Kolb’s theory [4], we decided to have multiple-choicequestions within the tutorial.
I The questions ask the student to think beyond what is givenrather than passively reading the text.
I The students reveal the correct answer once they are satisfiedwith their answer.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Implementation
The Online Interactive Tutorial
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Basics
I Task & ProcedureI The students were asked to answer all the questions in the
tutorial and fill out the surveys.I Each student spent 1-3 hours working on the tutorial.
I Student BackgroundI 8 students took our tutorial.I Students strongly agreed that they have experience with small-
and medium-sized software projects.I Students strongly agreed that they have not had much
experience with formal methods.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Basics
I Task & ProcedureI The students were asked to answer all the questions in the
tutorial and fill out the surveys.I Each student spent 1-3 hours working on the tutorial.
I Student BackgroundI 8 students took our tutorial.I Students strongly agreed that they have experience with small-
and medium-sized software projects.I Students strongly agreed that they have not had much
experience with formal methods.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Results
Tutorial Questions
I Initial questions reviewed the math background but later, morequestions became related to the specifics of the examples.
I Both the Birthday Book specification and model werewell-received.
I Excluding 2 students who dropped out of the KitchenEnvironment, there were 45/58 correct answers on average.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Results
Tutorial Questions
I Initial questions reviewed the math background but later, morequestions became related to the specifics of the examples.
I Both the Birthday Book specification and model werewell-received.
I Excluding 2 students who dropped out of the KitchenEnvironment, there were 45/58 correct answers on average.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Results
Tutorial Questions
I Initial questions reviewed the math background but later, morequestions became related to the specifics of the examples.
I Both the Birthday Book specification and model werewell-received.
I Excluding 2 students who dropped out of the KitchenEnvironment, there were 45/58 correct answers on average.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Results
Assessment
I The quiz questions asked the student to type one line of Alloycode to complete and correct an Address Book model.
I The questions were evaluated based on legitimate rationalerather than exact syntax.
I Students received 2/4 on average.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Results
Assessment
I The quiz questions asked the student to type one line of Alloycode to complete and correct an Address Book model.
I The questions were evaluated based on legitimate rationalerather than exact syntax.
I Students received 2/4 on average.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Results
Assessment
I The quiz questions asked the student to type one line of Alloycode to complete and correct an Address Book model.
I The questions were evaluated based on legitimate rationalerather than exact syntax.
I Students received 2/4 on average.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Results
Student Feedback
I Students’ appreciation of formal methods, formalspecification, and model checking were moderate.
I Students strongly agreed with our statements on the ease oftransitioning from Z to Alloy with the tutorial.
I Students agreed that the teaching material and the exampleswere good and that they were satisfied.
I Students wanted visualizations (pictures, diagrams, tables).
I Students needed examples that are between the BirthdayBook and the Kitchen Environment in complexity.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Results
Student Feedback
I Students’ appreciation of formal methods, formalspecification, and model checking were moderate.
I Students strongly agreed with our statements on the ease oftransitioning from Z to Alloy with the tutorial.
I Students agreed that the teaching material and the exampleswere good and that they were satisfied.
I Students wanted visualizations (pictures, diagrams, tables).
I Students needed examples that are between the BirthdayBook and the Kitchen Environment in complexity.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Results
Student Feedback
I Students’ appreciation of formal methods, formalspecification, and model checking were moderate.
I Students strongly agreed with our statements on the ease oftransitioning from Z to Alloy with the tutorial.
I Students agreed that the teaching material and the exampleswere good and that they were satisfied.
I Students wanted visualizations (pictures, diagrams, tables).
I Students needed examples that are between the BirthdayBook and the Kitchen Environment in complexity.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Results
Student Feedback
I Students’ appreciation of formal methods, formalspecification, and model checking were moderate.
I Students strongly agreed with our statements on the ease oftransitioning from Z to Alloy with the tutorial.
I Students agreed that the teaching material and the exampleswere good and that they were satisfied.
I Students wanted visualizations (pictures, diagrams, tables).
I Students needed examples that are between the BirthdayBook and the Kitchen Environment in complexity.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Results
Student Feedback
I Students’ appreciation of formal methods, formalspecification, and model checking were moderate.
I Students strongly agreed with our statements on the ease oftransitioning from Z to Alloy with the tutorial.
I Students agreed that the teaching material and the exampleswere good and that they were satisfied.
I Students wanted visualizations (pictures, diagrams, tables).
I Students needed examples that are between the BirthdayBook and the Kitchen Environment in complexity.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Findings
I The student performance did not show a huge drop betweenKitchen Environment (77.5%) and Birthday Book (88%).
I Students believe that Z and Alloy are similar and can betaught together but that textbooks using toy examples for Zor Alloy are insufficient.
I Students criticized the length of the tutorial but not thecontent.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Findings
I The student performance did not show a huge drop betweenKitchen Environment (77.5%) and Birthday Book (88%).
I Students believe that Z and Alloy are similar and can betaught together but that textbooks using toy examples for Zor Alloy are insufficient.
I Students criticized the length of the tutorial but not thecontent.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Findings
I The student performance did not show a huge drop betweenKitchen Environment (77.5%) and Birthday Book (88%).
I Students believe that Z and Alloy are similar and can betaught together but that textbooks using toy examples for Zor Alloy are insufficient.
I Students criticized the length of the tutorial but not thecontent.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Final Remarks
I Novice students are able to learn Z and Alloy and answerquestions about an intermediate-sized example after only acouple hours of study.
I We recommend educators to focus on the Alloy tool whileteaching the Z language.
I It is necessary to provide well-explained and interestingintermediate-level examples for the Z to Alloy transition.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Final Remarks
I Novice students are able to learn Z and Alloy and answerquestions about an intermediate-sized example after only acouple hours of study.
I We recommend educators to focus on the Alloy tool whileteaching the Z language.
I It is necessary to provide well-explained and interestingintermediate-level examples for the Z to Alloy transition.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Final Remarks
I Novice students are able to learn Z and Alloy and answerquestions about an intermediate-sized example after only acouple hours of study.
I We recommend educators to focus on the Alloy tool whileteaching the Z language.
I It is necessary to provide well-explained and interestingintermediate-level examples for the Z to Alloy transition.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
I Interactive application of Formal MethodsDean’s interactive case e-study [3], Pandora’s e-tutor [2]
I Similar Educational MaterialsPiaget’s genetic epistemology theory [8], Z to SPIN module [9]
I Content-wiseReal-life examples such as a Bluetooth communicationprotocol, a Eurovision Song Contest [1, 5, 6, 7]
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
Brakman, H., Driessen, V., Kavuma, J., Bijvank, L.N., Vermolen, S.Supporting formal method teaching with real-life protocols.In Formal Methods in the Teaching Lab: Examples, Cases, Assignments andProjects Enhancing Formal Methods Education, pages 59–68, McMasterUniversity, Canada, 2006.
Broda, K., Ma, J., Sinnadurai, G., Summers, A.Friendly e-tutor for natural deduction.In TFM: Practice and Experience, London, 2006. BCS-FACS.
N. Dean.Development of an interactive case e-study.In TFM: Practice and Experience Workshop, pages 13–20, Oxford BrookesUniversity, 2003. BCS-FACS.
D. Kolb.Experiential learning: Experience as the source of learning and development.Prentice-Hall, Englewood Cliffs, NJ, 1984.
P. Larsen.Two courses on vdm++ for embedded systems: Learning by doing.In Formal Methods in the Teaching Lab: Examples, Cases, Assignments andProjects Enhancing Formal Methods Education, pages 21–26, Canada, 2006.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy
Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References
D. Lightfoot.‘voici les votes!’ – formal specification as light entertainment: An example ofaudience participation in developing a specification.In TFM: Practice and Experience Workshop, pages 71–75, Oxford BrookesUniversity, 2003. BCS-FACS.
Reed, J.N., Sinclair, J.E.Motivating study of formal methods in the classroom.LNCS, 3294:32–46, 2004.
S. Rosa.Designing algorithms in high school mathematics.LNCS, 3294:17–31, 2004.
J. Rudall.From z to spin in one module.In TFM: Practice and Experience Workshop, pages 71–75, Oxford BrookesUniversity, 2003. BCS-FACS.
Tarkan & Sazawal UMD
Chief Chefs of Z to Alloy