CLIP OS: Building a defense-in-depth OSaround Linux kernel security improvements
Timothée Ravier, Mickaël Salaün
Agence nationale de la sécurité des systèmes d’information (ANSSI)
September 26, 2018
About the ANSSI
I Agence nationale de la sécurité des systèmes d’information
I French authority in the area of cyberdefence, network and informationsecurity
I We are not an intelligence agency
ANSSI CLIP OS: Building a defense-in-depth OS 2/27
Overview
CLIP OS ?
I Linux distribution developed by the ANSSI
I Initially only available internally
I Now open source, mostly under the LGPL v2.1+I Code and issue tracker hosted on GitHub:
I Version 4: available as reference and for upstream patch contribution1
I Version 5: currently developed version, alpha status2
1https://github.com/CLIPOS-Archive2https://github.com/CLIPOS
ANSSI CLIP OS: Building a defense-in-depth OS 4/27
Hardened OS
I Hardened Linux kernel and userspace
I Confined services
I "Unprivileged" admin, audit and update roles:⇒ the root account is not usable
I Automatic updates using A/B partition model (similar to Android 7+)
ANSSI CLIP OS: Building a defense-in-depth OS 5/27
Multilevel security OS
I Provide two isolated user environments: low and highI Interactions follow the Bell-LaPadula model:
I Write up: upload documents from low to highI Read down: high has read only access to untrusted USB devicesI Trusted write down: encrypt documents from high to write them in low
I Level high can only access network through a VPN
I Per level user device assignment
ANSSI CLIP OS: Building a defense-in-depth OS 6/27
Multilevel from the end user point of view
ANSSI CLIP OS: Building a defense-in-depth OS 7/27
Admin panel: devices assignment per level
ANSSI CLIP OS: Building a defense-in-depth OS 8/27
Differences with Qubes OS
CLIP OS development began 5 years earlier than Qubes OS
Goal of CLIP OSI We target non-expert usersI Bell-LaPadula model with two levelsI We favor a defense-in-depth approach
Technical point of viewI Hypervisor vs. supervisor isolationI Limited access right, even for the administrator
ANSSI CLIP OS: Building a defense-in-depth OS 9/27
Differences with Qubes OS
CLIP OS development began 5 years earlier than Qubes OS
Goal of CLIP OSI We target non-expert usersI Bell-LaPadula model with two levelsI We favor a defense-in-depth approach
Technical point of viewI Hypervisor vs. supervisor isolationI Limited access right, even for the administrator
ANSSI CLIP OS: Building a defense-in-depth OS 9/27
Differences with Qubes OS
CLIP OS development began 5 years earlier than Qubes OS
Goal of CLIP OSI We target non-expert usersI Bell-LaPadula model with two levelsI We favor a defense-in-depth approach
Technical point of viewI Hypervisor vs. supervisor isolationI Limited access right, even for the administrator
ANSSI CLIP OS: Building a defense-in-depth OS 9/27
Architecture
ANSSI CLIP OS: Building a defense-in-depth OS 10/27
CLIP OS 4
Hardening mechanisms
Gentoo HardenedI Hardened toolchainI Flexible patching
Linux-VServerI Linux namespaces with additional constraintsI Unique container and network IDs: XIDs and NIDs
grsecurity/PaXI Kernel self-protection (e.g., memory protection, CFI)I Multiple userspace hardening features (e.g., chroot, TPE)
CLIP LSMI Complement the Linux permission modelI Leverage Linux-VServer and grsecurity/PaX
ANSSI CLIP OS: Building a defense-in-depth OS 12/27
Hardening mechanisms
Gentoo HardenedI Hardened toolchainI Flexible patching
Linux-VServerI Linux namespaces with additional constraintsI Unique container and network IDs: XIDs and NIDs
grsecurity/PaXI Kernel self-protection (e.g., memory protection, CFI)I Multiple userspace hardening features (e.g., chroot, TPE)
CLIP LSMI Complement the Linux permission modelI Leverage Linux-VServer and grsecurity/PaX
ANSSI CLIP OS: Building a defense-in-depth OS 12/27
Hardening mechanisms
Gentoo HardenedI Hardened toolchainI Flexible patching
Linux-VServerI Linux namespaces with additional constraintsI Unique container and network IDs: XIDs and NIDs
grsecurity/PaXI Kernel self-protection (e.g., memory protection, CFI)I Multiple userspace hardening features (e.g., chroot, TPE)
CLIP LSMI Complement the Linux permission modelI Leverage Linux-VServer and grsecurity/PaX
ANSSI CLIP OS: Building a defense-in-depth OS 12/27
Hardening mechanisms
Gentoo HardenedI Hardened toolchainI Flexible patching
Linux-VServerI Linux namespaces with additional constraintsI Unique container and network IDs: XIDs and NIDs
grsecurity/PaXI Kernel self-protection (e.g., memory protection, CFI)I Multiple userspace hardening features (e.g., chroot, TPE)
CLIP LSMI Complement the Linux permission modelI Leverage Linux-VServer and grsecurity/PaX
ANSSI CLIP OS: Building a defense-in-depth OS 12/27
Write ⊕ Execute policy
Avoid arbitrary code execution and persistent attacks, improve multilevelisolation
Memory (PaX)Deny writable memory to be executable, throughout the system lifetime
DevctlEnforce and extend W ⊕ X from devices to mount points
Mount pointsEnforce W ⊕ X thanks to mount options: ro ⊕ noexec
The O_MAYEXEC flagEnforce and extend W ⊕ X from mount points to scripts (via interpreters)
ANSSI CLIP OS: Building a defense-in-depth OS 13/27
Write ⊕ Execute policy
Avoid arbitrary code execution and persistent attacks, improve multilevelisolation
Memory (PaX)Deny writable memory to be executable, throughout the system lifetime
DevctlEnforce and extend W ⊕ X from devices to mount points
Mount pointsEnforce W ⊕ X thanks to mount options: ro ⊕ noexec
The O_MAYEXEC flagEnforce and extend W ⊕ X from mount points to scripts (via interpreters)
ANSSI CLIP OS: Building a defense-in-depth OS 13/27
Write ⊕ Execute policy
Avoid arbitrary code execution and persistent attacks, improve multilevelisolation
Memory (PaX)Deny writable memory to be executable, throughout the system lifetime
DevctlEnforce and extend W ⊕ X from devices to mount points
Mount pointsEnforce W ⊕ X thanks to mount options: ro ⊕ noexec
The O_MAYEXEC flagEnforce and extend W ⊕ X from mount points to scripts (via interpreters)
ANSSI CLIP OS: Building a defense-in-depth OS 13/27
Write ⊕ Execute policy
Avoid arbitrary code execution and persistent attacks, improve multilevelisolation
Memory (PaX)Deny writable memory to be executable, throughout the system lifetime
DevctlEnforce and extend W ⊕ X from devices to mount points
Mount pointsEnforce W ⊕ X thanks to mount options: ro ⊕ noexec
The O_MAYEXEC flagEnforce and extend W ⊕ X from mount points to scripts (via interpreters)
ANSSI CLIP OS: Building a defense-in-depth OS 13/27
Write ⊕ Execute policy
Avoid arbitrary code execution and persistent attacks, improve multilevelisolation
Memory (PaX)Deny writable memory to be executable, throughout the system lifetime
DevctlEnforce and extend W ⊕ X from devices to mount points
Mount pointsEnforce W ⊕ X thanks to mount options: ro ⊕ noexec
The O_MAYEXEC flagEnforce and extend W ⊕ X from mount points to scripts (via interpreters)
ANSSI CLIP OS: Building a defense-in-depth OS 13/27
O_MAYEXEC
ANSSI CLIP OS: Building a defense-in-depth OS 14/27
Partitioning
Hardened containersI Leverage Linux-VServer admin and watch (audit) conceptsI New capability bounding sets: for root and per containerI Hardened chroot
Container content and interactionI Tailored filesystem layouts per serviceI Container management with vsctl and clip-libvserver (self-jailing)
ANSSI CLIP OS: Building a defense-in-depth OS 15/27
Partitioning
Hardened containersI Leverage Linux-VServer admin and watch (audit) conceptsI New capability bounding sets: for root and per containerI Hardened chroot
Container content and interactionI Tailored filesystem layouts per serviceI Container management with vsctl and clip-libvserver (self-jailing)
ANSSI CLIP OS: Building a defense-in-depth OS 15/27
Veriexec and permissions (CLIP-LSM)
GoalI Split Linux capabilities (e.g., Fuse, unshare)I Add new permissions (e.g., network, XFRM)I Can be tied to an XIDI Does not use xattr (thus independent from the filesystem)
Configuration example: /etc/verictl.d/chromium
/usr /.../ chrome - sandbox 1002 eSETUID | SETGID | SYS_CHROOT SETUID | SETGID | SYS_CHROOT -cUP sha256 45 bcbd1 ...
ANSSI CLIP OS: Building a defense-in-depth OS 16/27
Veriexec and permissions (CLIP-LSM)
GoalI Split Linux capabilities (e.g., Fuse, unshare)I Add new permissions (e.g., network, XFRM)I Can be tied to an XIDI Does not use xattr (thus independent from the filesystem)
Configuration example: /etc/verictl.d/chromium
/usr /.../ chrome - sandbox 1002 eSETUID | SETGID | SYS_CHROOT SETUID | SETGID | SYS_CHROOT -cUP sha256 45 bcbd1 ...
ANSSI CLIP OS: Building a defense-in-depth OS 16/27
Veriexec example
ANSSI CLIP OS: Building a defense-in-depth OS 17/27
CLIP OS 5
General Linux kernel hardening
I Strict whitelist of kernel options, but easily composable setsI Paranoid command line
I iommu=force, pti=on, spectre_v2=on, etc.
I Strict sysctl defaultsI kernel.kptr_restrict, kernel.yama.ptrace_scope, etc.
ANSSI CLIP OS: Building a defense-in-depth OS 19/27
Enabling Linux kernel hardening
GoalsI Protecting the kernel from itself and from userspaceI Include additional features for userspaceI Being able to test kernel and userspace coordinated changes
Security may come firstI We can handle minor compatibility breakage in our userspaceI Will accept changes that upstream may reject
Interaction with upstream & KSPPI Include in-progress or ready-for-upstream patchesI Integrate and validate patches in a single treeI Maintain hardening patches for latest stable kernel
ANSSI CLIP OS: Building a defense-in-depth OS 20/27
Enabling Linux kernel hardening
GoalsI Protecting the kernel from itself and from userspaceI Include additional features for userspaceI Being able to test kernel and userspace coordinated changes
Security may come firstI We can handle minor compatibility breakage in our userspaceI Will accept changes that upstream may reject
Interaction with upstream & KSPPI Include in-progress or ready-for-upstream patchesI Integrate and validate patches in a single treeI Maintain hardening patches for latest stable kernel
ANSSI CLIP OS: Building a defense-in-depth OS 20/27
Enabling Linux kernel hardening
GoalsI Protecting the kernel from itself and from userspaceI Include additional features for userspaceI Being able to test kernel and userspace coordinated changes
Security may come firstI We can handle minor compatibility breakage in our userspaceI Will accept changes that upstream may reject
Interaction with upstream & KSPPI Include in-progress or ready-for-upstream patchesI Integrate and validate patches in a single treeI Maintain hardening patches for latest stable kernel
ANSSI CLIP OS: Building a defense-in-depth OS 20/27
Patch series: linux-hardened
FeaturesI Memory hardening improvements, including:
I better userspace ASLRI slab allocators hardening (mostly SLUB)I simpler page sanitizing
I Various restrictions: TIOCSTI ioctl, perf subsystem, device timing sidechannels, etc.
I Miscellaneous additions: more BUG_ONs, more __ro_after_init, etc.
I Development status: In progressI CLIP OS status: MergedI Upstream status: Most changes unlikely to be merged upstream
ANSSI CLIP OS: Building a defense-in-depth OS 21/27
Patch series: linux-hardened
FeaturesI Memory hardening improvements, including:
I better userspace ASLRI slab allocators hardening (mostly SLUB)I simpler page sanitizing
I Various restrictions: TIOCSTI ioctl, perf subsystem, device timing sidechannels, etc.
I Miscellaneous additions: more BUG_ONs, more __ro_after_init, etc.
I Development status: In progressI CLIP OS status: MergedI Upstream status: Most changes unlikely to be merged upstream
ANSSI CLIP OS: Building a defense-in-depth OS 21/27
Upstream contribution integration: Lockdown
FeaturesI Reduce options for root to run untrusted code in kernel context
I Development status: Feature completeI CLIP OS status: MergedI Upstream status: Ready for upstream integration
ANSSI CLIP OS: Building a defense-in-depth OS 22/27
Upstream contribution integration: Lockdown
FeaturesI Reduce options for root to run untrusted code in kernel context
I Development status: Feature completeI CLIP OS status: MergedI Upstream status: Ready for upstream integration
ANSSI CLIP OS: Building a defense-in-depth OS 22/27
Upstream contribution integration: STACKLEAK
FeaturesI Reduce information leaks and block attacks using uninitialized kernel
stack variables:I Erase the stack before returning from system calls
I Improve runtime detection of kernel stack overflows (e.g. Stack Clash):I Instrument calls to alloca()
CLIP OS specific changesI Kept alloca()-related changes (dropped for upstream in v15)
I Development status: Feature completeI CLIP OS status: MergedI Upstream status: Ready for upstream integration
ANSSI CLIP OS: Building a defense-in-depth OS 23/27
Upstream contribution integration: STACKLEAK
FeaturesI Reduce information leaks and block attacks using uninitialized kernel
stack variables:I Erase the stack before returning from system calls
I Improve runtime detection of kernel stack overflows (e.g. Stack Clash):I Instrument calls to alloca()
CLIP OS specific changesI Kept alloca()-related changes (dropped for upstream in v15)
I Development status: Feature completeI CLIP OS status: MergedI Upstream status: Ready for upstream integration
ANSSI CLIP OS: Building a defense-in-depth OS 23/27
Upstream contribution integration: STACKLEAK
FeaturesI Reduce information leaks and block attacks using uninitialized kernel
stack variables:I Erase the stack before returning from system calls
I Improve runtime detection of kernel stack overflows (e.g. Stack Clash):I Instrument calls to alloca()
CLIP OS specific changesI Kept alloca()-related changes (dropped for upstream in v15)
I Development status: Feature completeI CLIP OS status: MergedI Upstream status: Ready for upstream integration
ANSSI CLIP OS: Building a defense-in-depth OS 23/27
Upstream contribution: Landlock
FeaturesI Enables seccomp-bpf -like self-sandboxing for unprivileged processesI Stackable LSMI Powered by eBPFI Dynamic filesystem access control using whitelists & blacklistsI See landlock.io
I Development status: Initial feature set readyI CLIP OS status: PlannedI Upstream status: Work in progress
ANSSI CLIP OS: Building a defense-in-depth OS 24/27
Upstream contribution: Landlock
FeaturesI Enables seccomp-bpf -like self-sandboxing for unprivileged processesI Stackable LSMI Powered by eBPFI Dynamic filesystem access control using whitelists & blacklistsI See landlock.io
I Development status: Initial feature set readyI CLIP OS status: PlannedI Upstream status: Work in progress
ANSSI CLIP OS: Building a defense-in-depth OS 24/27
Upstream contribution: VServer-like LSM
FeaturesI Adds a single kernel enforced indentifier for confined environmentsI Similar in principle to VServer XID or to "Container IDs"I Inspired by the VServer patchI Integrated as a stackable LSM
I Development status: Early development stageI CLIP OS status: Planned
ANSSI CLIP OS: Building a defense-in-depth OS 25/27
Upstream contribution: VServer-like LSM
FeaturesI Adds a single kernel enforced indentifier for confined environmentsI Similar in principle to VServer XID or to "Container IDs"I Inspired by the VServer patchI Integrated as a stackable LSM
I Development status: Early development stageI CLIP OS status: Planned
ANSSI CLIP OS: Building a defense-in-depth OS 25/27
Conclusion
Take awayI Hardened Linux distro and kernelI Coordinated userspace and kernelspaceI Support multilevel security
Ongoing projectI Contributions welcomeI Browse the doc and the sources to find more interesting features:
docs.clip-os.org
ANSSI CLIP OS: Building a defense-in-depth OS 26/27
Conclusion
Take awayI Hardened Linux distro and kernelI Coordinated userspace and kernelspaceI Support multilevel security
Ongoing projectI Contributions welcomeI Browse the doc and the sources to find more interesting features:
docs.clip-os.org
ANSSI CLIP OS: Building a defense-in-depth OS 26/27
Thanks!
� clip-os.org B [email protected]
� v4: github.com/CLIPOS-Archive
� v5: github.com/CLIPOS
We’re hiring! (but not directly for CLIP OS)
Linux system security experthttps://www.ssi.gouv.fr/emploi/expert-en-securite-des-systemes-linux/
Boot chain and root partition integrity protection
1 UEFI Secure Boot support:I Custom keys (i.e. not signed by Microsoft)I Requires enrollment in hardware
2 Minimal bootloader (gummiboot/systemd-boot)3 EFI bundle:
I Linux kernelI initramfsI kernel command line
4 DM-Verity partition:
I DM-Verity root hash set in kernel command lineI Forward error correction support (FEC)I Read only uncompressed SquashFS root
filesystem
ANSSI CLIP OS: Building a defense-in-depth OS 28/27
Boot chain and root partition integrity protection
1 UEFI Secure Boot support:I Custom keys (i.e. not signed by Microsoft)I Requires enrollment in hardware
2 Minimal bootloader (gummiboot/systemd-boot)
3 EFI bundle:
I Linux kernelI initramfsI kernel command line
4 DM-Verity partition:
I DM-Verity root hash set in kernel command lineI Forward error correction support (FEC)I Read only uncompressed SquashFS root
filesystem
ANSSI CLIP OS: Building a defense-in-depth OS 28/27
Boot chain and root partition integrity protection
1 UEFI Secure Boot support:I Custom keys (i.e. not signed by Microsoft)I Requires enrollment in hardware
2 Minimal bootloader (gummiboot/systemd-boot)3 EFI bundle:
I Linux kernelI initramfsI kernel command line
4 DM-Verity partition:
I DM-Verity root hash set in kernel command lineI Forward error correction support (FEC)I Read only uncompressed SquashFS root
filesystem
ANSSI CLIP OS: Building a defense-in-depth OS 28/27
Boot chain and root partition integrity protection
1 UEFI Secure Boot support:I Custom keys (i.e. not signed by Microsoft)I Requires enrollment in hardware
2 Minimal bootloader (gummiboot/systemd-boot)3 EFI bundle:
I Linux kernelI initramfsI kernel command line
4 DM-Verity partition:I DM-Verity root hash set in kernel command lineI Forward error correction support (FEC)I Read only uncompressed SquashFS root
filesystem
ANSSI CLIP OS: Building a defense-in-depth OS 28/27