presentación de powerpoint...bctt administration: participants and procedure 299 primary school...
TRANSCRIPT
![Page 1: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/1.jpg)
Beginners Computational Thinking TestBCTt
1
![Page 2: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/2.jpg)
2
María Zapata Cáceres Computer Science Department URJC@maria__z_c
Estefanía Martín BarrosoComputer Science Department URJC@EstefaniaURJC
Marcos Román González Faculty of Education UNED@elpsycomago
![Page 3: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/3.jpg)
Computational Thinking (CT)
Human problem-solving process that requires abstract
thinking
Many definitions
Thinking skills that precede programming, and are applied
in understanding a problem and formulating a solution like
a computer scientist
Many breakdowns
Shute, V. J., Sun, C., & Asbell-Clarke, J. (2017a). Demystifying computational thinking.Educational Research Review, 22,
142-158 doi://doi.org/10.1016/j.edurev.2017.09.003
Abstraction
Decomposition
Algorithms
Debugging
Iteration
Generalization
BROAD DEBATE
![Page 4: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/4.jpg)
Computational Thinking (CT)
Fundamental skill required to adapt to the future
Should be taught at schools
Programming exposes students to CT
CONSENSUS
![Page 5: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/5.jpg)
5
Computational Thinking (CT)
3D Framework (Brenan & Resnick*)
* K. Brennan, M. Resnick and MIT Media Lab, "New frameworks for studying and assessing the development of computational thinking," American Educational Research Association Meeting, Vancouver, BC, Canada, 2012.
CT Concepts
Many frameworks
CT Practices
CT Perspectives
![Page 6: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/6.jpg)
Howto teach?
Whatto teach?
Howto assess?
Whento teach?
Questions
![Page 7: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/7.jpg)
Howto teach?
Whatto teach?
Howto assess?
Whento teach?
Questions
![Page 8: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/8.jpg)
How toteach?
What toteach?
How toassess?
When toteach?
8
Assessment
Types ofassessment
Traditional test
Portfolio
Survey
Interview
Focused on middle school grades and specific programming environments
![Page 9: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/9.jpg)
3D Framework
9
Computational Thinking Test (CTt)
* M. Román-González, J. Pérez-González and C. Jiménez-Fernández, "Which cognitive abilities underlie computational thinking? Criterion validity of the Computational Thinking Test," Computers in Human Behavior, vol. 72, pp. 678-691, 2017.
Computational Thinking Test
(CTt) *
Stand-alone assessment instrument
Aimed at 10 to 16 years old
Reliability and criterion validity, psychometric approach
Aligned with the international standards
CT concepts
CT practices
CT perspectives
![Page 10: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/10.jpg)
10
Beginners Computational Thinking Test (BCTt)
5 to 10 years old
Based on CTt
Substantial improvements
Beginners Computational Thinking Test
(BCTt) Form / content adaptation
Stand-alone assessment instrument
![Page 11: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/11.jpg)
11
BCTt v.1 Designv.1
25 items long 40 minutes
3 alternative responses 3D Framework computational concepts
Test
items
Computational concepts in BCTt
1.
Sequences
Loops Conditionals
2. Simple
loop
3. Nested
loop
4. IF-
then
5. If-then-
else
6.
While
1 - 6
7 - 11
12 - 18
19 - 20
21 - 22
23 - 25
![Page 12: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/12.jpg)
12
BCTt v.1 Designv.1
Least possible textSelf explanatory symbols
Emotional connection
Canvas and maze type
![Page 13: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/13.jpg)
13
BCTt v.1 Designv.1
![Page 14: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/14.jpg)
If-else example
CA B
![Page 15: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/15.jpg)
If-else example
CA B
![Page 16: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/16.jpg)
CA B
If-else example
![Page 17: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/17.jpg)
C
If-else example
A B
![Page 18: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/18.jpg)
C
If-else example
A B
![Page 19: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/19.jpg)
C
If-else example
A B
![Page 20: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/20.jpg)
C
If-else example
A B
![Page 21: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/21.jpg)
BCTt Expert Judgement procedure & results
45 experts
Content validation
Item difficulty level
v.1
Item relevance to measure CT
Test length adequacy
Graphic interface adequacy
Improvements adequacy: e.g. transitions
Other / suggestions / comments
66 items form
![Page 22: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/22.jpg)
22
BCTt Expert Judgement procedure & resultsv.1
adequate
too long
too short
y = 0,1063x + 1,4239
R² = 0,8883
0,00
0,50
1,00
1,50
2,00
2,50
3,00
3,50
4,00
4,50
5,00
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
y = 0,0786x + 3,6857
R² = 0,5818
0,00
0,50
1,00
1,50
2,00
2,50
3,00
3,50
4,00
4,50
5,00
0 1 2 3 4 5 6 7
BCTt item difficulty perceived by experts
BCTt computational concept relevance to measure CT, perceived by experts
Adequacy to evaluate CT
very good
good
intermediate
bad
very bad
Test length
83% → Transitions are positive
![Page 23: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/23.jpg)
23
BCTt Expert Judgement procedure & resultsv.1
Comments / suggestions
“transitions are easily associated to arrows in the answers”
“the allowed paths are clear with transitions,
because it excludes diagonal movements”
“In the design without transitions, doubts are generated about when
a character reaches another (either when it reaches the
previous square or when it reaches the other character square?)”.
“It is not clear if two chicks can move together after meeting”
“the test is TOO HARD”
![Page 24: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/24.jpg)
24
BCTt v.2 Design
Form and content modifications
v.2
4 alternative responses
If-else reformulation
If-then-else reformulation
Oral explanation
![Page 25: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/25.jpg)
25
BCTt v.2 Designv.2
Colour blindness adaptation
![Page 26: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/26.jpg)
BCTt Administration: participants and procedure
299 Primary School Students
v.2
5 to 12 years old
School Educational stage Grades Students ages
Colegio Público Carlos Ruiz 1st 1st and 2nd 5 - 8
Colegio Los Escolapios 2nd 3rd and 4th 7 -10
CEIP León Felipe 3rd 5th and 6th 9 -12
Action protocol
Test printed in paper form
BCTt
BCTt variation
transitions
No-transitions
![Page 27: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/27.jpg)
27
BCTt Administration: participants and procedurev.2
Educational stage
Grade Identifier BCTtBCTt
variation
1st1 A A1: n=522 B B1: n=18 B2: n=18
2nd4 C C1: n=544 D D1: n=28 D2: n=28
3rd5 E E1: n=516 F F1: n=25 F2: n=25
Time 1
B2, D2, F2 A1, B1, C1, D1, E1, F1 D1
Time 2: 5 weeks later
Procedure
BCTtBCTt
variationBCTt
![Page 28: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/28.jpg)
28
BCTt Administration: resultsv.2
TransitionsStudent’s
t-testSignificant difference in test scores
(p=0.005< 0.01) in lower grades
Sample Entire sample A1 B1 C1 E1 F1
Grade 1-6 1 2 4 5 6
N 200 52 18 54 51 25
Mean 19.92 16.52 16.78 21.57 21.84 21.72
Median 20.00 16.00 18.00 23.00 23.00 22.00
Std. Deviation 3.79 3.31 2.49 3.044 2.61 2.62
Variance 14.36 10.96 6.183 9.268 6.815 6.88
Minimum 8.00 8.00 11.00 14.00 13.00 15.00
Maximum 25.00 24.00 20.00 25.00 25.00 25.00
Percentiles
25 17.00 14.00 15.75 19.00 20.00 19.50
50 20.00 16.00 18.00 23.00 23.00 22.00
75 23.00 19.00 18.00 24.00 24.00 24.00
Descriptive statistics
![Page 29: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/29.jpg)
29
BCTt Administration: resultsv.2
0,00
0,50
1,00
1,50
2,00
2,50
3,00
3,50
4,00
4,50
5,00
Sequences Simple loop Nested loop If-then If-then-else While
1st 2nd 4th 5th 6th
Item analysis
Computational concept by grade
y = -0,0155x + 1,0083
R² = 0,6344
0
0,2
0,4
0,6
0,8
1
1,2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Item difficulty index for each BCTt item
y = -0,0173x + 0,9249
R² = 0,3904
0
0,2
0,4
0,6
0,8
1
1,2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Item difficulty index
![Page 30: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/30.jpg)
30
BCTt Administration: resultsv.2
Sample Reliability Statistics Item Statistics
NN of
ItemsCronbach's Alpha
Cr. 's Alpha Based on
Stand. Items Mean Min. Max. Variance
200 25 0.824 0.829 0.807 0.576 0.976 0.021
Subsamples Reliability Statistics Item Statistics
Ed. stage Grade Id. n Cronbach's AlphaCr. 's Alpha Based on
Stand. Items Mean Variance1st 1 A1 52 0.833 0.838 0.742 0.0411st 2 B1 18 0.793 0.801 0.630 0.0422nd 4 C1 54 0.771 0.735 0.837 0.0223rd 5 E1 51 0.660 0.683 0.863 0.0123rd 6 F1 25 0.657 0.648 0.844 0.015
very strong significant correlation (rs=0.93; p<0.01).
Task and re-task method (D1 subsample)
Non-parametric Spearman’s test
![Page 31: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/31.jpg)
✓BCTt is adequate for the assessment of CT in Primary School
✓Transitions between maze squares are a relevant improvement for young students
✓BCTt seems to be aimed at 1st to 4th grades (5 to 10 years old)
✓Reliability is high and higher in younger students
✓Recommended to use in parallel with other tools → system of assessments
31
Conclusions
3D Framework
CT concepts
CT practices
CT perspectives
![Page 32: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/32.jpg)
✓It could be used as a pre-test / post-test instrument
Conclusions
Blue Ant Code (Android and IOs)
![Page 33: Presentación de PowerPoint...BCTt Administration: participants and procedure 299 Primary School Students v.2 5 to 12 years old School Educational stage Grades Students ages Colegio](https://reader033.vdocuments.us/reader033/viewer/2022051605/600e9e8f7a4740413f7f66f0/html5/thumbnails/33.jpg)
✓BCTt lower limit
✓Other countries: Portugal, Chezch republic, Germany, France, …
✓Populations
✓BCTt new version → Cornell University NY
33
Conclusions