course content analysis: an initiative step toward ... · for mooc learners yiling dai, yasuhito...
TRANSCRIPT
![Page 1: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/1.jpg)
Course Content Analysis: An Initiative Step toward Learning Object Recommendation Systems for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University
EDM 2016
![Page 2: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/2.jpg)
Background
2
Much easier to challenge new subjects
Rising needs for diversity in one’s knowledge
![Page 3: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/3.jpg)
No guidance
Free to combine courses
Similar courses and difficult syllabi
Background
3
A supporting system may help!
For novices:
![Page 4: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/4.jpg)
A course map
that presents the courses based on the knowledge they cover
Intuition
4
Categories of domain knowledge
An image of the course map
![Page 5: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/5.jpg)
Interact with the learner
estimate his learning goals and ability
Provide a path of learning objects (i.e. course lectures).
Recommendation system
5
An image of the recommendation system
Advanced functions:
![Page 6: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/6.jpg)
Task recognition
6
Final goal
Subtasks: • Course analysis
• Domain knowledge representation • Course content analysis
• Learner analysis • Background knowledge estimation • Learning goal recognition • …
![Page 7: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/7.jpg)
Course content analysis /Basic idea
7
Target: Locating a course to the domain knowledge base
Curriculum guideline Course 1 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Computer Science Curricula 2013 (IEEE & ACM)
Course syllabus
![Page 8: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/8.jpg)
8
Operating Systems
Concurrency …
Course 1 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Intelligent Systems
Fundamental issues
…
Course 1
An initiative step
Labeled LDA + Snippet Complementation
![Page 9: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/9.jpg)
CS2013: Body of Knowledge (BoK)
9
AL-Algorithms and Complexity
AR-Architecture and Organization
CN-Computational Science
DS-Discrete Structures
…
Knowledge Area (KA) Basic Analysis
Algorithmic Strategies
Fundamental Data Structures and Algorithms
Basic Automata, Computability and Complexity
…
Knowledge Unit (KU) • Brute-force algorithms • Greedy algorithms • Divide-and-conquer • …
Topics
…
• … • …
…
…
…
Course content analysis /Dataset
![Page 10: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/10.jpg)
10
AL SDF PD …
30 1.5 1.5 …
Knowledge coverage
---------------------------------
Course syllabus
Course content analysis /Dataset
CS2013: Sample Courses
![Page 11: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/11.jpg)
11
--------------
…
--------------
… …
--------------
--------------
--------------
--------------
…
--------------
--------------
nDCG(0~1)
Inference
Learn
Input Output
Labeled LDA
Course content analysis /Method
Term - Label
![Page 12: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/12.jpg)
• In Labeled LDA, the quality of the set of documents for learning (training set) affects the result remarkably.
12
--------------
…
--------------
… …
--------------
--------------
--------------
-------------- Inference
Learn
Input Output
Labeled LDA
Course content analysis /Method
![Page 13: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/13.jpg)
13
--------------
…
--------------
Learn
Course content analysis /Method
Documents for training: A part of syllabi of sample courses
• Multi-labeled • Combination of written
language and spoken language Body of Knowledge (CS2013)
• Single -labeled • Concise language
Why? How?
Complement it with snippets
![Page 14: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/14.jpg)
14
• Brute-force algorithms • Greedy algorithms • Divide-and-conquer • …
Algorithmic strategies (AL)
The sentence is too concise to contain enough representative terms of this KU.
We integrate external texts that are explaining this KU.
E.g.
![Page 15: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/15.jpg)
15
• Brute-force algorithms • Greedy algorithms • Divide-and-conquer • …
Algorithmic strategies (AL)
①Throw queries into search engine
• These are the easiest and simplest… •Some Algorithm Strategies, …
②Append snippets into the original text
![Page 16: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/16.jpg)
16
Query Result
The title of the KU snippet1
The title of the KU and the title of the KA it belongs to snippet2
Top three representative terms of the KU (tf-idf value) snippet3
Patterns of queries to extract snippets:
![Page 17: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/17.jpg)
17
--------------
…
--------------
Learn
Course content analysis /Method
Documents for training: A part of sample courses (SC) Body of Knowledge (BoK)
BoK_Snippet1(KU title) BoK_Snippet2 (KU+KA title) BoK_Snippet3 (KU top terms)
--------------
… …
--------------
--------------
-------------- Inference
Labels to inference: KA KU
![Page 18: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/18.jpg)
18
Course content analysis /Method
No snippet Snippet1 Snippet2 Snippet3
BoK BoK BoK_snippet1 BoK_snippet2 BoK_snippet3
BoK+SC BoK+SC BoK_snippet1+SC BoK_snippet2+SC BoK_snippet3+SC
Experiments: We want to observe: • Which type of documents is suitable for training? • Does snippet work? • Does this model work for both KA and KU?
![Page 19: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/19.jpg)
Result Discussion
19
0.797 0.788
0.865 0.858
0.742 0.736
0.796 0.838
0.326 0.299
0.439 0.439
0.306 0.279
0.375 0.367
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
BoK BoK_snippet1 BoK_snippet2 BoK_snippet3
nDCG(KA)_BoK nDCG(KA)_BoK+SC nDCG(KU)_BoK nDCG(KU)_BoK+SC
Course content analysis /Result
Result 1: BoK > BoK + SC Discussion 1: The documents of SC are multi-labeled, which brings some noise in Labeled LDA.
![Page 20: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/20.jpg)
Result Discussion
20
0.797 0.788
0.865 0.858
0.742 0.736
0.796 0.838
0.326 0.299
0.439 0.439
0.306 0.279
0.375 0.367
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
BoK BoK_snippet1 BoK_snippet2 BoK_snippet3
nDCG(KA)_BoK nDCG(KA)_BoK+SC nDCG(KU)_BoK nDCG(KU)_BoK+SC
Course content analysis /Result
Result 2: Snippet2, Snippet3☑ (KU+KA title, top 3 terms) Discussion 2: We can complement an insufficient domain knowledge base by information retrieval techniques
![Page 21: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/21.jpg)
Result Discussion
21
0.797 0.788
0.865 0.858
0.742 0.736
0.796 0.838
0.326 0.299
0.439 0.439
0.306 0.279
0.375 0.367
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
BoK BoK_snippet1 BoK_snippet2 BoK_snippet3
nDCG(KA)_BoK nDCG(KA)_BoK+SC nDCG(KU)_BoK nDCG(KU)_BoK+SC
Course content analysis /Result
Result 3: KA☑ KU? Discussion 3: It needs to be improved when extracting knowledge coverage at a smaller granularity.
![Page 22: Course Content Analysis: An Initiative Step toward ... · for MOOC Learners Yiling Dai, Yasuhito Asano, Masatoshi Yoshikawa Graduate School of Informatics Kyoto University EDM 2016](https://reader034.vdocuments.us/reader034/viewer/2022043006/5f8f0006da6fa762f561962b/html5/thumbnails/22.jpg)
SUMMARY
Recommendation system design
Course content analysis
22
Course 1 -----------------------------------------------------------
Labeled LDA + Snippet Complementation