what's new in freebsd 10
DESCRIPTION
My talk at ruBSD 2013 at Yandex on new features in upcoming FreeBSD 10.0-RELEASETRANSCRIPT
What’s new in FreeBSD 10?
Gleb [email protected]
ruBSD 2013YandexMoscow
December 14, 2013
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 1 / 24
Outline
1 Introduction2 Userland changes
Packaging systemToolchainDevelopers toolsDNS toolsOther userland updates
3 Kernel: virtualizationbhyveguest improvements
4 Kernel: securitycapsicum(4) update/dev/random improvements
5 Kernel: general improvementscallout(9) new generationunmapped I/Omemory management
6 Kernel: I/O and storagechangesfilesystems
7 Kernel: networkingCARPchangespacket filters
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 2 / 24
Introduction
Two years of development
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 3 / 24
Userland changes Packaging system
New generation packaging system
pkg(1)
Replaces pkg_tools in FreeBSD 10.0Updates packages from remote repositoryIs developed as a library + command line frontend
Don’t miss section at 11:40 by Vsevolod Stakhov!
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 4 / 24
Userland changes Packaging system
New generation packaging system
pkg(1)
Replaces pkg_tools in FreeBSD 10.0Updates packages from remote repositoryIs developed as a library + command line frontend
Don’t miss section at 11:40 by Vsevolod Stakhov!
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 4 / 24
Userland changes Toolchain
Compiler change
LLVM/Clang 3.3 is default compiler(amd64, arm and i386)
Why?BSD licensed (gcc > 4.2.1 is GPLv3)
Fully C++11 compliant. Includes LLVM libc++.Always cross compiler.
We still support gcc 4.2+ to build tier 2 arches.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 5 / 24
Userland changes Toolchain
Compiler change
LLVM/Clang 3.3 is default compiler(amd64, arm and i386)
Why?BSD licensed (gcc > 4.2.1 is GPLv3)
Fully C++11 compliant. Includes LLVM libc++.Always cross compiler.
We still support gcc 4.2+ to build tier 2 arches.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 5 / 24
Userland changes Toolchain
Compiler change
LLVM/Clang 3.3 is default compiler(amd64, arm and i386)
Why?BSD licensed (gcc > 4.2.1 is GPLv3)
Fully C++11 compliant. Includes LLVM libc++.Always cross compiler.
We still support gcc 4.2+ to build tier 2 arches.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 5 / 24
Userland changes Toolchain
Toolchain
Moving towards external toolchain.
Portable make(1) imported from NetBSDTools updated:
patch(1): GNU BSD licensed fork of original Larry Wallsort(1): GNU own implementationbyacc for yacc(1)flex for lex(1)
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 6 / 24
Userland changes Toolchain
Toolchain
Moving towards external toolchain.Portable make(1) imported from NetBSD
Tools updated:patch(1): GNU BSD licensed fork of original Larry Wallsort(1): GNU own implementationbyacc for yacc(1)flex for lex(1)
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 6 / 24
Userland changes Toolchain
Toolchain
Moving towards external toolchain.Portable make(1) imported from NetBSDTools updated:
patch(1): GNU BSD licensed fork of original Larry Wallsort(1): GNU own implementationbyacc for yacc(1)flex for lex(1)
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 6 / 24
Userland changes Developers tools
Developers tools
CVS -> subversion (lite)ATF/kyua from NetBSDWork in progress: gdb -> lldb
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 7 / 24
Userland changes DNS tools
DNS tools
Recursive resolver & toolsBIND -> unbounddig(1) -> drill(1)new host(1) implementationnslookup
LDNS libraryFeature rich API, providing control over recursion,DNSSEC, TSIG, etc.Utilized by OpenSSH, drill(1)
Plan for FreeBSD 11: caching, validating, secureresolver library with standard API
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 8 / 24
Userland changes DNS tools
DNS tools
Recursive resolver & toolsBIND -> unbounddig(1) -> drill(1)new host(1) implementationnslookup
LDNS libraryFeature rich API, providing control over recursion,DNSSEC, TSIG, etc.Utilized by OpenSSH, drill(1)
Plan for FreeBSD 11: caching, validating, secureresolver library with standard API
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 8 / 24
Userland changes DNS tools
DNS tools
Recursive resolver & toolsBIND -> unbounddig(1) -> drill(1)new host(1) implementationnslookup
LDNS libraryFeature rich API, providing control over recursion,DNSSEC, TSIG, etc.Utilized by OpenSSH, drill(1)
Plan for FreeBSD 11: caching, validating, secureresolver library with standard API
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 8 / 24
Userland changes Other userland updates
Other userland updates
freebsd-version(1) tool introducedlibyaml added to base
Citrus iconv(3) in libcnewest jemalloc 3.4.1 in libcnvi editor supports wide character localeswpa_supplicant/hostapd updated to 2.0OpenSSH updated to 6.4OpenSSL updated to 1.0.1e
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 9 / 24
Userland changes Other userland updates
Other userland updates
freebsd-version(1) tool introducedlibyaml added to baseCitrus iconv(3) in libcnewest jemalloc 3.4.1 in libc
nvi editor supports wide character localeswpa_supplicant/hostapd updated to 2.0OpenSSH updated to 6.4OpenSSL updated to 1.0.1e
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 9 / 24
Userland changes Other userland updates
Other userland updates
freebsd-version(1) tool introducedlibyaml added to baseCitrus iconv(3) in libcnewest jemalloc 3.4.1 in libcnvi editor supports wide character locales
wpa_supplicant/hostapd updated to 2.0OpenSSH updated to 6.4OpenSSL updated to 1.0.1e
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 9 / 24
Userland changes Other userland updates
Other userland updates
freebsd-version(1) tool introducedlibyaml added to baseCitrus iconv(3) in libcnewest jemalloc 3.4.1 in libcnvi editor supports wide character localeswpa_supplicant/hostapd updated to 2.0OpenSSH updated to 6.4OpenSSL updated to 1.0.1e
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 9 / 24
Userland changes Other userland updates
Installer
bsdinstall features ZFS root installationRemoved old installer sysinstall and auxiliary toolslibdisk, libftpio, sade
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 10 / 24
Kernel: virtualization bhyve
bhyve(4) hypervisor
BSD hyper visor(pronounced as “bee hive”)
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 11 / 24
Kernel: virtualization bhyve
bhyve(4) hypervisor
BSD hyper visor(pronounced as “bee hive”)
Requirements:host is amd64: Intel CPU with VT-xfeature or AMD CPU with AMD-Vfeatureno BIOS provided
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 11 / 24
Kernel: virtualization bhyve
bhyve(4) hypervisor
BSD hyper visor(pronounced as “bee hive”)
Results in:12k lines of code in kernel14k lines of code in userland
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 11 / 24
Kernel: virtualization bhyve
bhyve(4) hypervisor
BSD hyper visor(pronounced as “bee hive”)
Guest OSes supported:FreeBSD, OpenBSDGNU/Linux
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 11 / 24
Kernel: virtualization guest improvements
Guest improvements
Xen and Xen HVM in GENERIC kernelMicrosoft Hyper-V drivers addedVMware VMXNET3 driver added
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 12 / 24
Kernel: security capsicum(4) update
capsicum(4) update
Capsicum - hybrid capability + UNIX accesscontrol model. Introduced in FreeBSD 9.0.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 13 / 24
Kernel: security capsicum(4) update
capsicum(4) update
Capsicum integrates further into FreeBSD:notions of “capability” and “filedescriptor” mergenew APIs: cap_new(2)cap_rights_limit(2)capsicum(4) in GENERIC by defaultsandboxed applications: tcpdump(1),dhclient(8), rwhod(8), kdump(8),hastd(8), auditdistd(8), ctld(8),iscsid(8)
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 13 / 24
Kernel: security capsicum(4) update
capsicum(4) update
Future integration in 10.1-RELEASE:casperd(8) daemonlibcapsicum(3) librarysandboxing a lot of applications
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 13 / 24
Kernel: security /dev/random improvements
better random
Problem: hardware assisted randomness (RDRAND andPadlock) no longer trusted.Solution: run them through Yarrow.
Problem: not enough entropy on early boot.Solution: we can get some from device attach time.
Problem: not enough entropy on first boot.Let bsdinstall save an entropy cookie for future boot.
FreeBSD 11.0 plan: substitute Yarrow with Fortuna.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 14 / 24
Kernel: security /dev/random improvements
better random
Problem: hardware assisted randomness (RDRAND andPadlock) no longer trusted.Solution: run them through Yarrow.
Problem: not enough entropy on early boot.Solution: we can get some from device attach time.
Problem: not enough entropy on first boot.Let bsdinstall save an entropy cookie for future boot.
FreeBSD 11.0 plan: substitute Yarrow with Fortuna.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 14 / 24
Kernel: security /dev/random improvements
better random
Problem: hardware assisted randomness (RDRAND andPadlock) no longer trusted.Solution: run them through Yarrow.
Problem: not enough entropy on early boot.Solution: we can get some from device attach time.
Problem: not enough entropy on first boot.Let bsdinstall save an entropy cookie for future boot.
FreeBSD 11.0 plan: substitute Yarrow with Fortuna.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 14 / 24
Kernel: security /dev/random improvements
better random
Problem: hardware assisted randomness (RDRAND andPadlock) no longer trusted.Solution: run them through Yarrow.
Problem: not enough entropy on early boot.Solution: we can get some from device attach time.
Problem: not enough entropy on first boot.Let bsdinstall save an entropy cookie for future boot.
FreeBSD 11.0 plan: substitute Yarrow with Fortuna.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 14 / 24
Kernel: general improvements callout(9) new generation
callout(9) improvements
callout(9) - kernel subsystem to schedule delayed events.
New improvements:ticklessevent coalescingdirect execution
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 15 / 24
Kernel: general improvements callout(9) new generation
callout(9) improvements
callout(9) - kernel subsystem to schedule delayed events.New improvements:
ticklessevent coalescingdirect execution
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 15 / 24
Kernel: general improvements unmapped I/O
unmapped I/O
Problem: kernel doing I/O on behalf of userland processmaps the I/O region into kernel address space.
Change ofvirtual memory map requires notification of other CPUs.
Solution: unmapped I/O. Required modification of filesystem layer, GEOM classes, disk drivers.
Result: 30% of system CPU time saved in I/O boundtasks.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 16 / 24
Kernel: general improvements unmapped I/O
unmapped I/O
Problem: kernel doing I/O on behalf of userland processmaps the I/O region into kernel address space. Change ofvirtual memory map requires notification of other CPUs.
Solution: unmapped I/O. Required modification of filesystem layer, GEOM classes, disk drivers.
Result: 30% of system CPU time saved in I/O boundtasks.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 16 / 24
Kernel: general improvements unmapped I/O
unmapped I/O
Problem: kernel doing I/O on behalf of userland processmaps the I/O region into kernel address space. Change ofvirtual memory map requires notification of other CPUs.
Solution: unmapped I/O. Required modification of filesystem layer, GEOM classes, disk drivers.
Result: 30% of system CPU time saved in I/O boundtasks.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 16 / 24
Kernel: general improvements unmapped I/O
unmapped I/O
Problem: kernel doing I/O on behalf of userland processmaps the I/O region into kernel address space. Change ofvirtual memory map requires notification of other CPUs.
Solution: unmapped I/O. Required modification of filesystem layer, GEOM classes, disk drivers.
Result: 30% of system CPU time saved in I/O boundtasks.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 16 / 24
Kernel: general improvements memory management
memory management changes
Kernel memory maps:vmem(9) generic allocator from NetBSDkernel memory map allocation backed by vmem(9)
Mach VMradix tree instead of splay tree for vm_pages invm_object
UMAperformance/efficiency improvementsper-CPU zoneslog warning when a zone hits limit
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 17 / 24
Kernel: general improvements memory management
memory management changes
Kernel memory maps:vmem(9) generic allocator from NetBSDkernel memory map allocation backed by vmem(9)
Mach VMradix tree instead of splay tree for vm_pages invm_object
UMAperformance/efficiency improvementsper-CPU zoneslog warning when a zone hits limit
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 17 / 24
Kernel: general improvements memory management
memory management changes
Kernel memory maps:vmem(9) generic allocator from NetBSDkernel memory map allocation backed by vmem(9)
Mach VMradix tree instead of splay tree for vm_pages invm_object
UMAperformance/efficiency improvementsper-CPU zoneslog warning when a zone hits limit
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 17 / 24
Kernel: general improvements memory management
atomic close-on-exec
Prevents descriptor leak in presence of threads orsignalsSuggested for future POSIX
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 18 / 24
Kernel: I/O and storage changes
storage changes
NAND flash supportNAND controller/chip/bus APIsNAND disk GEOM classNAND file system
Resizinggeneral support of “resize” notion in GEOMresizing of GEOM mirror (in 10.1-RELEASE)growfs(1) works on mounted filesystems
legacy ATA layer removed
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 19 / 24
Kernel: I/O and storage changes
storage changes
NAND flash supportNAND controller/chip/bus APIsNAND disk GEOM classNAND file system
Resizinggeneral support of “resize” notion in GEOMresizing of GEOM mirror (in 10.1-RELEASE)growfs(1) works on mounted filesystems
legacy ATA layer removed
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 19 / 24
Kernel: I/O and storage changes
storage changes
NAND flash supportNAND controller/chip/bus APIsNAND disk GEOM classNAND file system
Resizinggeneral support of “resize” notion in GEOMresizing of GEOM mirror (in 10.1-RELEASE)growfs(1) works on mounted filesystems
legacy ATA layer removed
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 19 / 24
Kernel: I/O and storage changes
GEOM: work in progress
Targeted for 10.1-RELEASE:
direct dispatch in GEOM instead of two threadsfine grained locking of CAM layer
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 20 / 24
Kernel: I/O and storage changes
GEOM: work in progress
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 20 / 24
Kernel: I/O and storage changes
GEOM: work in progress
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 20 / 24
Kernel: I/O and storage filesystems
FUSE
FUSE moved to base from ports to improve stability
Giant-locked and GPL-contaminated filesystemsremoved from kernel: hpfs, ext2fs, ntfs, reiserfs, coda,xfs, nwfs, portalfs.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 21 / 24
Kernel: I/O and storage filesystems
FUSE
FUSE moved to base from ports to improve stabilityGiant-locked and GPL-contaminated filesystemsremoved from kernel: hpfs, ext2fs, ntfs, reiserfs, coda,xfs, nwfs, portalfs.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 21 / 24
Kernel: networking changes
networking changes
ZERO_COPY_SOCKETSsendfile(2) on shared memory fdnetwork byte order throughout the stackIP/TCP/UDP dtrace(1) providersraceless and cheap statistic per-CPU counters
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 22 / 24
Kernel: networking CARP
new CARP
CARP isn’t pseudo-interface anymore.% ifconfig igb0 10.0.0.112/27 vhid 112% ifconfig igb0igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 00:25:90:03:0e:fainet 10.0.0.112 netmask 0xffffffe0 broadcast 10.0.0.127 vhid 112media: Ethernet autoselect (1000baseT <full-duplex>)status: activecarp: BACKUP vhid 112 advbase 1 advskew 0
Redundant address is configured directly on a realinterface.
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 23 / 24
Kernel: networking packet filters
packet filters
pf(4): fork off OpenBSD, bringing in multithreadingipfilter(4): update to 5.1.2 (BSD license pledged)ipfw(4): no significant changes
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 24 / 24
Kernel: networking packet filters
Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 25 / 24