a simple and effective approach to organizing in …...git-based ctf: a simple and effective...
TRANSCRIPT
![Page 1: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/1.jpg)
Git-based CTF:A Simple and Effective Approach to
Organizing In-Course Attack-and-Defense Security CompetitionSeongIl Wi, Jaeseung Choi, Sang Kil Cha
KAIST
![Page 2: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/2.jpg)
Capture The Flag (CTF)
2https://ctftime.org/ctfs
CTF: Cybersecurity competition that involves capturing a flag
![Page 3: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/3.jpg)
Types of CTF
3
Attack-and-Defense Style
JeopardyStyle
Real time, Realistic
One-way, Problem solving
![Page 4: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/4.jpg)
Number of CTF Events in 2018
https://ctftime.org/event/list/past 4
Attack-and-Defense Style
JeopardyStyle
8 times 73 times
![Page 5: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/5.jpg)
In-Course Attack and Defense CTF
5
• Class Capture-the-Flag Exercises, USENIX 3GSE ’14
• Build It, Break It, Fix It: Contesting Secure Development, ACM CCS ’16
Difficult to organize!
![Page 6: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/6.jpg)
6
Game Server
Team1 VM Team5 VM
Attack & Defense CTF Infrastructures
Team1 Team5
TeachingAssistant
Flag
![Page 7: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/7.jpg)
7
Challenges
Game Server
Team1 VM Team5 VM
Team1 Team5
TeachingAssistant
Need interaction between teamsC1: Interactivity
Challenge
![Page 8: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/8.jpg)
8
Challenges
Game Server
Team1 VM Team5 VM
Team1 Team5
TeachingAssistant
C1: Interactivity Challenge
Network, VM, DB, Scoreboard, etc.C2: Configuration
Challenge
![Page 9: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/9.jpg)
9
Challenges
Game Server
Team1 VM Team5 VM
Team1 Team5
TeachingAssistant
C2: ConfigurationChallenge
C1: Interactivity Challenge
Need monitoring and administering
continuously
C3: MonitoringChallenge
![Page 10: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/10.jpg)
10
Challenges
Game Server
Team1 VM Team5 VM
Team1 Team5
TeachingAssistant
C2: ConfigurationChallenge
C1: Interactivity Challenge
C3: MonitoringChallenge
Need to invent new problems for every competition
C4: Contents Creation
Challenge
![Page 11: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/11.jpg)
11
Challenges
Game Server
Team1 VM Team5 VM
Team1 Team5
TeachingAssistant
C2: ConfigurationChallenge
C1: Interactivity Challenge
C3: MonitoringChallenge
Need to invent new problems for every competition
C4: Contents Creation
Challenge
![Page 12: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/12.jpg)
• C1: Interactivity Challenge− SWPAG, USENIX ASE ’17− InCTF, USENIX ASE ’16
• C2: Configuration Challenge− SWPAG, USENIX ASE ’17− CTFd, USENIX ASE ’17
• C3: Monitoring Challenge− VM-based Framework, USENIX 3GSE ’15
• C4: Contents Creation Challenge− BIBIFI, ACM CCS ’16− SecGen, USENIX ASE ’17
Handle only a subset of the challenges
12
Recent Researches:
![Page 13: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/13.jpg)
Previous Work: BIBIFI, ACM CCS ’16
13
Build-It Break-It Fix-It
Does not allow real-time attack and
defense exercise
Teacher should manually check
every fix
DefenseAttack
C1: Interactivity Challenge
C3: Monitoring Challenge
![Page 14: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/14.jpg)
14
Challenges in BIBIFI
Game Server
Team1 VM Team5 VM
Team1 Team5
TeachingAssistant
C2: ConfigurationChallenge
C4: Contents Creation
Challenge
C1: Interactivity Challenge
C3: MonitoringChallenge
![Page 15: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/15.jpg)
Can we handle all the challenges?
15
![Page 16: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/16.jpg)
We propose
Git-based CTF
16
![Page 17: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/17.jpg)
GitHub as a CTF Framework
17
Local Repository
Local Repository
Developer A Developer B
Pull Pull
PushPush RemoteRepository
![Page 18: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/18.jpg)
GitHub = DB
18
Local Repository
Local Repository
Team 1 Team 2
Pull Pull
PushPush DefenseAttack
Service
RemoteRepository
OrganizerLocal Repository
Organize
Pull
Push
![Page 19: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/19.jpg)
19
Our Goal: Handle All the Challenges
Game Server
Team1 VM Team5 VM
Team1 Team5
TeachingAssistant
C2: ConfigurationChallenge
C1: Interactivity Challenge
C3: MonitoringChallenge
Need to invent new problems for every competition
C4: Contents Creation
Challenge
![Page 20: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/20.jpg)
20
Handle Interactivity Challenge
Game Server
Team1 VM Team5 VM
Team1 Team5
TeachingAssistant
C2: ConfigurationChallenge
C1: Interactivity Challenge
C3: MonitoringChallenge
Need to invent new problems for every competition
C4: Contents Creation
Challenge
Real time attack and defense
![Page 21: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/21.jpg)
21
Real Time Attack and Defense
Attacker Defender
Round System
Periodically award points until it is fixed by the defending team
![Page 22: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/22.jpg)
Repository as Scoreboard
22
Local Repository
Local Repository
Team 1 Team 2
Pull Pull
Check score
Scoreboard
RemoteRepository
OrganizerLocal Repository
Evaluate
Push
Check score
![Page 23: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/23.jpg)
23
Handle Configuration Challenge
Game Server
Team1 VM Team5 VM
Team1 Team5
TeachingAssistant
C2: ConfigurationChallenge
C1: Interactivity Challenge
C3: MonitoringChallenge
Need to invent new problems for every competition
C4: Contents Creation
Challenge
Real time attack and defense
Git-based infrastructure
![Page 24: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/24.jpg)
24
Game Server
Team1 VM Team5 VM
Git-based Infrastructure
Network, VM, DB, Scoreboard, etc.
![Page 25: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/25.jpg)
25
Game Server
Team1 VM Team5 VM
Git-based Infrastructure
![Page 26: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/26.jpg)
26
Game Server
Team1 VM Team5 VM
Git Repository as Service
GitHub Repository
VulnerableProgram
![Page 27: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/27.jpg)
27
Distributed System
GitHub Repository
VulnerableProgram
Cloned GitHubRepository
VulnerableProgram
Exploit(1) Clone (2) Find
(3) Submit Exploit as a GitHub issue
Attacker
![Page 28: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/28.jpg)
28
Handle Monitoring Challenge
Game Server
Team1 VM Team5 VM
Team1 Team5
TeachingAssistant
C2: ConfigurationChallenge
C1: Interactivity Challenge
C3: MonitoringChallenge
Need to invent new problems for every competition
C4: Contents Creation
Challenge
Real time attack and defense
Git-based infrastructure
Automated verification
system
![Page 29: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/29.jpg)
29
>_
gitctf.pyTeaching Assistant
• Verify exploit in each round• Manage the game score
Execute
Automated Verification System
![Page 30: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/30.jpg)
30Scalable and Lightweight CTF Infrastructures Using Application Containers, USENIX ASE ’16
VulnerableProgram
Docker Container
Exploit
Docker Container
Random String
Execution Result
Copy & Execute
Exploit
Flag
Automated Exploit Verification
![Page 31: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/31.jpg)
Public Verification
31
Local Repository
Local Repository
Team 1 Team 2
Pull Pull
PushPush
Service
RemoteRepository
OrganizerLocal Repository
Verify exploit
Pull
Push
Verify exploit Verify exploit
Publicly Accessible!
![Page 32: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/32.jpg)
32
Handle Contents Creation Challenge
Game Server
Team1 VM Team5 VM
Team1 Team5
TeachingAssistant
C2: ConfigurationChallenge
C1: Interactivity Challenge
C3: MonitoringChallenge
Need to invent new problems for every competition
C4: Contents Creation
Challenge
Real time attack and defense
Git-based infrastructure
Automated verification
system
Shifting creation burden to student
![Page 33: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/33.jpg)
Shifting Creation Burden to Student(BIBIFI, ACM CCS ’16)
33
Hands-onDevelopment
PreparedProgram
![Page 34: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/34.jpg)
GitHub Repository
Injecting Vulnerabilities
34
PreparedProgram
VulnerableProgram
Intended Vulnerability
Unintended Vulnerability
Injection
GitHub Repository
![Page 35: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/35.jpg)
35
Our Goal: Handle All the Challenges
Game Server
Team1 VM Team5 VM
Team1 Team5
TeachingAssistant
C1: Interactivity Challenge
C3: MonitoringChallenge
Need to invent new problems for every competition
C4: Contents Creation
Challenge
Real time attack and defense
Git-based infrastructure
Automated verification
system
Shifting creation burden to student
![Page 36: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/36.jpg)
Evaluation Setup (Preliminary)
36
• IS521 Information Security Laboratory 2018 in KAIST, Korea
• 21 students (11 of them had no experience in CTF), 6 teams
• Preparation: Develop a simple secure messaging application (use either C or C++)
• Injection (Individual): Inject at least one vulnerability• Exercise (Individual): Report unintended vulnerabilities or
functionality bugs
![Page 37: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/37.jpg)
Diversity of Injected Vulnerabilities
37
The students introduced 28 vulnerabilities in the 6 distinct applications
Help the instructors prepare a diverse set of CTF challenges
![Page 38: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/38.jpg)
• 14 vulnerabilities and 18 functionality bugs were reported
• Each team had at least one unintended vulnerability
• Unintended vulnerabilities are found mostly by experienced students
Exercise with Unintended Vulnerability
38
![Page 39: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/39.jpg)
<source/>
PreparedSource Code
010111010011101110010010Compiled
Binary
Discussion: Binary-Only CTF
39
Compile
GitHub Repository
![Page 40: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/40.jpg)
010111010011101110010010Compiled
Binary
Discussion: Binary-Only CTF
40
GitHub Repository
Attack & DefensePlayer
![Page 41: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/41.jpg)
Open Science
https://github.com/SoftSec-KAIST/GitCTF
41
![Page 42: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/42.jpg)
Demo
42
![Page 43: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/43.jpg)
Demo Scenario
43
Local Repository
Local Repository
Pull Pull
PushPush (3) Defense
Service
RemoteRepository
Local Repository
(1) Turn on evaluator
Pull
Push
Team 1(Attacker)
Team 2(Defender)
Organizer
(2) Attack
![Page 44: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/44.jpg)
44
![Page 45: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/45.jpg)
Open Science
https://github.com/SoftSec-KAIST/GitCTF
45
![Page 46: A Simple and Effective Approach to Organizing In …...Git-based CTF: A Simple and Effective Approach to Organizing In-Course Attack-and-Defense Security Competition SeongIl Wi, Jaeseung](https://reader035.vdocuments.us/reader035/viewer/2022070718/5ede4470ad6a402d666996f0/html5/thumbnails/46.jpg)
Question?
46