smt solvers in it security - deobfuscating binary code ...€¦ · deobfuscating binary code with...
TRANSCRIPT
![Page 1: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/1.jpg)
SMT Solvers in IT Security -Deobfuscating binary code
with logic
barbieauglend @ BlackHoodie17 - LuxembourgR [email protected] • 7 barbieauglend
![Page 2: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/2.jpg)
DISCLAIMER
This research was accomplished by me in my personalcapacity. The opinions and views expressed in this talk andarticle are my own and do not necessarily reflect the official
policy or view of my employer.
![Page 3: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/3.jpg)
WHO AM I?
![Page 4: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/4.jpg)
Overview:
• Introduction to Constraint Logic Programming• Applications of CLP in IT Security• Binary Obfuscation• Malware deobfuscation using CLP
![Page 5: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/5.jpg)
CONSTRAINTS
![Page 6: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/6.jpg)
”Constraint programming represents one of the closestapproaches computer science has yet made to the Holy Grail ofprogramming: the user states the problem, the computer solves
it.”
Eugene C. Freuder, Constraints, April 1997
![Page 7: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/7.jpg)
![Page 8: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/8.jpg)
Automated Theorem Proving
• Hardware and Software→ Large-scale verification
• Languages specification and Computing proof obligations
![Page 9: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/9.jpg)
SYMBOLIC EXECUTION
![Page 10: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/10.jpg)
APPLICATIONS
![Page 11: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/11.jpg)
Bug Hunting
• Fuzzing
• Verification
• Analysis
![Page 12: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/12.jpg)
Exploit Generation
• Automatic Exploit Generation
• Proof of Concept
• Automatic Payload Generation
![Page 13: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/13.jpg)
Malware Analysis
• Obfuscation
• Garbage-code elimination
• Compilation
• Packing
• Anti-debugging
• Crypto analysis
![Page 14: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/14.jpg)
BINARY OBFUSCATION
![Page 15: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/15.jpg)
Malware Obfuscation
SW Property Protection
![Page 16: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/16.jpg)
HOW DOES IT WORK?
![Page 17: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/17.jpg)
• Compiled
• Packed
• Obfuscated
• Anti-debugging
![Page 18: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/18.jpg)
Garbage Code
• Unnecessary instructions• Jumps that are never taken
![Page 19: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/19.jpg)
The exclusive or operation
![Page 20: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/20.jpg)
Packers
• UPX, NSIS• self implemented
![Page 21: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/21.jpg)
Malware Analysis• Practical:
Techniques tothwart analysis
• Theoretical:Rice’s Theorem
![Page 22: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/22.jpg)
![Page 23: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/23.jpg)
• Symbols as arguments⇒ any feasible path
• Program states• Symbolic values for memory
locations• Path conditions
![Page 24: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/24.jpg)
![Page 25: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/25.jpg)
CONCLUSION
![Page 26: SMT Solvers in IT Security - Deobfuscating binary code ...€¦ · Deobfuscating binary code with logic barbieauglend @ BlackHoodie17 - Luxembourg R barbieauglend@chaosdorf.de 7 barbieauglend](https://reader035.vdocuments.us/reader035/viewer/2022063006/5fb7501a2eba8b307c51f121/html5/thumbnails/26.jpg)
THANK YOU!