what's new in freebsd 10

54
What’s new in FreeBSD 10? Gleb Smirnoff [email protected] ruBSD 2013 Yandex Moscow December 14, 2013 Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 1 / 24

Upload: gleb-smirnoff

Post on 05-Jul-2015

439 views

Category:

Software


1 download

DESCRIPTION

My talk at ruBSD 2013 at Yandex on new features in upcoming FreeBSD 10.0-RELEASE

TRANSCRIPT

Page 1: What's new in FreeBSD 10

What’s new in FreeBSD 10?

Gleb [email protected]

ruBSD 2013YandexMoscow

December 14, 2013

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 1 / 24

Page 2: What's new in FreeBSD 10

Outline

1 Introduction2 Userland changes

Packaging systemToolchainDevelopers toolsDNS toolsOther userland updates

3 Kernel: virtualizationbhyveguest improvements

4 Kernel: securitycapsicum(4) update/dev/random improvements

5 Kernel: general improvementscallout(9) new generationunmapped I/Omemory management

6 Kernel: I/O and storagechangesfilesystems

7 Kernel: networkingCARPchangespacket filters

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 2 / 24

Page 3: What's new in FreeBSD 10

Introduction

Two years of development

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 3 / 24

Page 4: What's new in FreeBSD 10

Userland changes Packaging system

New generation packaging system

pkg(1)

Replaces pkg_tools in FreeBSD 10.0Updates packages from remote repositoryIs developed as a library + command line frontend

Don’t miss section at 11:40 by Vsevolod Stakhov!

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 4 / 24

Page 5: What's new in FreeBSD 10

Userland changes Packaging system

New generation packaging system

pkg(1)

Replaces pkg_tools in FreeBSD 10.0Updates packages from remote repositoryIs developed as a library + command line frontend

Don’t miss section at 11:40 by Vsevolod Stakhov!

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 4 / 24

Page 6: What's new in FreeBSD 10

Userland changes Toolchain

Compiler change

LLVM/Clang 3.3 is default compiler(amd64, arm and i386)

Why?BSD licensed (gcc > 4.2.1 is GPLv3)

Fully C++11 compliant. Includes LLVM libc++.Always cross compiler.

We still support gcc 4.2+ to build tier 2 arches.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 5 / 24

Page 7: What's new in FreeBSD 10

Userland changes Toolchain

Compiler change

LLVM/Clang 3.3 is default compiler(amd64, arm and i386)

Why?BSD licensed (gcc > 4.2.1 is GPLv3)

Fully C++11 compliant. Includes LLVM libc++.Always cross compiler.

We still support gcc 4.2+ to build tier 2 arches.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 5 / 24

Page 8: What's new in FreeBSD 10

Userland changes Toolchain

Compiler change

LLVM/Clang 3.3 is default compiler(amd64, arm and i386)

Why?BSD licensed (gcc > 4.2.1 is GPLv3)

Fully C++11 compliant. Includes LLVM libc++.Always cross compiler.

We still support gcc 4.2+ to build tier 2 arches.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 5 / 24

Page 9: What's new in FreeBSD 10

Userland changes Toolchain

Toolchain

Moving towards external toolchain.

Portable make(1) imported from NetBSDTools updated:

patch(1): GNU BSD licensed fork of original Larry Wallsort(1): GNU own implementationbyacc for yacc(1)flex for lex(1)

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 6 / 24

Page 10: What's new in FreeBSD 10

Userland changes Toolchain

Toolchain

Moving towards external toolchain.Portable make(1) imported from NetBSD

Tools updated:patch(1): GNU BSD licensed fork of original Larry Wallsort(1): GNU own implementationbyacc for yacc(1)flex for lex(1)

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 6 / 24

Page 11: What's new in FreeBSD 10

Userland changes Toolchain

Toolchain

Moving towards external toolchain.Portable make(1) imported from NetBSDTools updated:

patch(1): GNU BSD licensed fork of original Larry Wallsort(1): GNU own implementationbyacc for yacc(1)flex for lex(1)

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 6 / 24

Page 12: What's new in FreeBSD 10

Userland changes Developers tools

Developers tools

CVS -> subversion (lite)ATF/kyua from NetBSDWork in progress: gdb -> lldb

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 7 / 24

Page 13: What's new in FreeBSD 10

Userland changes DNS tools

DNS tools

Recursive resolver & toolsBIND -> unbounddig(1) -> drill(1)new host(1) implementationnslookup

LDNS libraryFeature rich API, providing control over recursion,DNSSEC, TSIG, etc.Utilized by OpenSSH, drill(1)

Plan for FreeBSD 11: caching, validating, secureresolver library with standard API

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 8 / 24

Page 14: What's new in FreeBSD 10

Userland changes DNS tools

DNS tools

Recursive resolver & toolsBIND -> unbounddig(1) -> drill(1)new host(1) implementationnslookup

LDNS libraryFeature rich API, providing control over recursion,DNSSEC, TSIG, etc.Utilized by OpenSSH, drill(1)

Plan for FreeBSD 11: caching, validating, secureresolver library with standard API

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 8 / 24

Page 15: What's new in FreeBSD 10

Userland changes DNS tools

DNS tools

Recursive resolver & toolsBIND -> unbounddig(1) -> drill(1)new host(1) implementationnslookup

LDNS libraryFeature rich API, providing control over recursion,DNSSEC, TSIG, etc.Utilized by OpenSSH, drill(1)

Plan for FreeBSD 11: caching, validating, secureresolver library with standard API

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 8 / 24

Page 16: What's new in FreeBSD 10

Userland changes Other userland updates

Other userland updates

freebsd-version(1) tool introducedlibyaml added to base

Citrus iconv(3) in libcnewest jemalloc 3.4.1 in libcnvi editor supports wide character localeswpa_supplicant/hostapd updated to 2.0OpenSSH updated to 6.4OpenSSL updated to 1.0.1e

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 9 / 24

Page 17: What's new in FreeBSD 10

Userland changes Other userland updates

Other userland updates

freebsd-version(1) tool introducedlibyaml added to baseCitrus iconv(3) in libcnewest jemalloc 3.4.1 in libc

nvi editor supports wide character localeswpa_supplicant/hostapd updated to 2.0OpenSSH updated to 6.4OpenSSL updated to 1.0.1e

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 9 / 24

Page 18: What's new in FreeBSD 10

Userland changes Other userland updates

Other userland updates

freebsd-version(1) tool introducedlibyaml added to baseCitrus iconv(3) in libcnewest jemalloc 3.4.1 in libcnvi editor supports wide character locales

wpa_supplicant/hostapd updated to 2.0OpenSSH updated to 6.4OpenSSL updated to 1.0.1e

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 9 / 24

Page 19: What's new in FreeBSD 10

Userland changes Other userland updates

Other userland updates

freebsd-version(1) tool introducedlibyaml added to baseCitrus iconv(3) in libcnewest jemalloc 3.4.1 in libcnvi editor supports wide character localeswpa_supplicant/hostapd updated to 2.0OpenSSH updated to 6.4OpenSSL updated to 1.0.1e

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 9 / 24

Page 20: What's new in FreeBSD 10

Userland changes Other userland updates

Installer

bsdinstall features ZFS root installationRemoved old installer sysinstall and auxiliary toolslibdisk, libftpio, sade

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 10 / 24

Page 21: What's new in FreeBSD 10

Kernel: virtualization bhyve

bhyve(4) hypervisor

BSD hyper visor(pronounced as “bee hive”)

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 11 / 24

Page 22: What's new in FreeBSD 10

Kernel: virtualization bhyve

bhyve(4) hypervisor

BSD hyper visor(pronounced as “bee hive”)

Requirements:host is amd64: Intel CPU with VT-xfeature or AMD CPU with AMD-Vfeatureno BIOS provided

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 11 / 24

Page 23: What's new in FreeBSD 10

Kernel: virtualization bhyve

bhyve(4) hypervisor

BSD hyper visor(pronounced as “bee hive”)

Results in:12k lines of code in kernel14k lines of code in userland

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 11 / 24

Page 24: What's new in FreeBSD 10

Kernel: virtualization bhyve

bhyve(4) hypervisor

BSD hyper visor(pronounced as “bee hive”)

Guest OSes supported:FreeBSD, OpenBSDGNU/Linux

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 11 / 24

Page 25: What's new in FreeBSD 10

Kernel: virtualization guest improvements

Guest improvements

Xen and Xen HVM in GENERIC kernelMicrosoft Hyper-V drivers addedVMware VMXNET3 driver added

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 12 / 24

Page 26: What's new in FreeBSD 10

Kernel: security capsicum(4) update

capsicum(4) update

Capsicum - hybrid capability + UNIX accesscontrol model. Introduced in FreeBSD 9.0.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 13 / 24

Page 27: What's new in FreeBSD 10

Kernel: security capsicum(4) update

capsicum(4) update

Capsicum integrates further into FreeBSD:notions of “capability” and “filedescriptor” mergenew APIs: cap_new(2)cap_rights_limit(2)capsicum(4) in GENERIC by defaultsandboxed applications: tcpdump(1),dhclient(8), rwhod(8), kdump(8),hastd(8), auditdistd(8), ctld(8),iscsid(8)

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 13 / 24

Page 28: What's new in FreeBSD 10

Kernel: security capsicum(4) update

capsicum(4) update

Future integration in 10.1-RELEASE:casperd(8) daemonlibcapsicum(3) librarysandboxing a lot of applications

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 13 / 24

Page 29: What's new in FreeBSD 10

Kernel: security /dev/random improvements

better random

Problem: hardware assisted randomness (RDRAND andPadlock) no longer trusted.Solution: run them through Yarrow.

Problem: not enough entropy on early boot.Solution: we can get some from device attach time.

Problem: not enough entropy on first boot.Let bsdinstall save an entropy cookie for future boot.

FreeBSD 11.0 plan: substitute Yarrow with Fortuna.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 14 / 24

Page 30: What's new in FreeBSD 10

Kernel: security /dev/random improvements

better random

Problem: hardware assisted randomness (RDRAND andPadlock) no longer trusted.Solution: run them through Yarrow.

Problem: not enough entropy on early boot.Solution: we can get some from device attach time.

Problem: not enough entropy on first boot.Let bsdinstall save an entropy cookie for future boot.

FreeBSD 11.0 plan: substitute Yarrow with Fortuna.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 14 / 24

Page 31: What's new in FreeBSD 10

Kernel: security /dev/random improvements

better random

Problem: hardware assisted randomness (RDRAND andPadlock) no longer trusted.Solution: run them through Yarrow.

Problem: not enough entropy on early boot.Solution: we can get some from device attach time.

Problem: not enough entropy on first boot.Let bsdinstall save an entropy cookie for future boot.

FreeBSD 11.0 plan: substitute Yarrow with Fortuna.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 14 / 24

Page 32: What's new in FreeBSD 10

Kernel: security /dev/random improvements

better random

Problem: hardware assisted randomness (RDRAND andPadlock) no longer trusted.Solution: run them through Yarrow.

Problem: not enough entropy on early boot.Solution: we can get some from device attach time.

Problem: not enough entropy on first boot.Let bsdinstall save an entropy cookie for future boot.

FreeBSD 11.0 plan: substitute Yarrow with Fortuna.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 14 / 24

Page 33: What's new in FreeBSD 10

Kernel: general improvements callout(9) new generation

callout(9) improvements

callout(9) - kernel subsystem to schedule delayed events.

New improvements:ticklessevent coalescingdirect execution

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 15 / 24

Page 34: What's new in FreeBSD 10

Kernel: general improvements callout(9) new generation

callout(9) improvements

callout(9) - kernel subsystem to schedule delayed events.New improvements:

ticklessevent coalescingdirect execution

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 15 / 24

Page 35: What's new in FreeBSD 10

Kernel: general improvements unmapped I/O

unmapped I/O

Problem: kernel doing I/O on behalf of userland processmaps the I/O region into kernel address space.

Change ofvirtual memory map requires notification of other CPUs.

Solution: unmapped I/O. Required modification of filesystem layer, GEOM classes, disk drivers.

Result: 30% of system CPU time saved in I/O boundtasks.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 16 / 24

Page 36: What's new in FreeBSD 10

Kernel: general improvements unmapped I/O

unmapped I/O

Problem: kernel doing I/O on behalf of userland processmaps the I/O region into kernel address space. Change ofvirtual memory map requires notification of other CPUs.

Solution: unmapped I/O. Required modification of filesystem layer, GEOM classes, disk drivers.

Result: 30% of system CPU time saved in I/O boundtasks.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 16 / 24

Page 37: What's new in FreeBSD 10

Kernel: general improvements unmapped I/O

unmapped I/O

Problem: kernel doing I/O on behalf of userland processmaps the I/O region into kernel address space. Change ofvirtual memory map requires notification of other CPUs.

Solution: unmapped I/O. Required modification of filesystem layer, GEOM classes, disk drivers.

Result: 30% of system CPU time saved in I/O boundtasks.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 16 / 24

Page 38: What's new in FreeBSD 10

Kernel: general improvements unmapped I/O

unmapped I/O

Problem: kernel doing I/O on behalf of userland processmaps the I/O region into kernel address space. Change ofvirtual memory map requires notification of other CPUs.

Solution: unmapped I/O. Required modification of filesystem layer, GEOM classes, disk drivers.

Result: 30% of system CPU time saved in I/O boundtasks.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 16 / 24

Page 39: What's new in FreeBSD 10

Kernel: general improvements memory management

memory management changes

Kernel memory maps:vmem(9) generic allocator from NetBSDkernel memory map allocation backed by vmem(9)

Mach VMradix tree instead of splay tree for vm_pages invm_object

UMAperformance/efficiency improvementsper-CPU zoneslog warning when a zone hits limit

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 17 / 24

Page 40: What's new in FreeBSD 10

Kernel: general improvements memory management

memory management changes

Kernel memory maps:vmem(9) generic allocator from NetBSDkernel memory map allocation backed by vmem(9)

Mach VMradix tree instead of splay tree for vm_pages invm_object

UMAperformance/efficiency improvementsper-CPU zoneslog warning when a zone hits limit

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 17 / 24

Page 41: What's new in FreeBSD 10

Kernel: general improvements memory management

memory management changes

Kernel memory maps:vmem(9) generic allocator from NetBSDkernel memory map allocation backed by vmem(9)

Mach VMradix tree instead of splay tree for vm_pages invm_object

UMAperformance/efficiency improvementsper-CPU zoneslog warning when a zone hits limit

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 17 / 24

Page 42: What's new in FreeBSD 10

Kernel: general improvements memory management

atomic close-on-exec

Prevents descriptor leak in presence of threads orsignalsSuggested for future POSIX

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 18 / 24

Page 43: What's new in FreeBSD 10

Kernel: I/O and storage changes

storage changes

NAND flash supportNAND controller/chip/bus APIsNAND disk GEOM classNAND file system

Resizinggeneral support of “resize” notion in GEOMresizing of GEOM mirror (in 10.1-RELEASE)growfs(1) works on mounted filesystems

legacy ATA layer removed

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 19 / 24

Page 44: What's new in FreeBSD 10

Kernel: I/O and storage changes

storage changes

NAND flash supportNAND controller/chip/bus APIsNAND disk GEOM classNAND file system

Resizinggeneral support of “resize” notion in GEOMresizing of GEOM mirror (in 10.1-RELEASE)growfs(1) works on mounted filesystems

legacy ATA layer removed

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 19 / 24

Page 45: What's new in FreeBSD 10

Kernel: I/O and storage changes

storage changes

NAND flash supportNAND controller/chip/bus APIsNAND disk GEOM classNAND file system

Resizinggeneral support of “resize” notion in GEOMresizing of GEOM mirror (in 10.1-RELEASE)growfs(1) works on mounted filesystems

legacy ATA layer removed

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 19 / 24

Page 46: What's new in FreeBSD 10

Kernel: I/O and storage changes

GEOM: work in progress

Targeted for 10.1-RELEASE:

direct dispatch in GEOM instead of two threadsfine grained locking of CAM layer

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 20 / 24

Page 47: What's new in FreeBSD 10

Kernel: I/O and storage changes

GEOM: work in progress

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 20 / 24

Page 48: What's new in FreeBSD 10

Kernel: I/O and storage changes

GEOM: work in progress

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 20 / 24

Page 49: What's new in FreeBSD 10

Kernel: I/O and storage filesystems

FUSE

FUSE moved to base from ports to improve stability

Giant-locked and GPL-contaminated filesystemsremoved from kernel: hpfs, ext2fs, ntfs, reiserfs, coda,xfs, nwfs, portalfs.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 21 / 24

Page 50: What's new in FreeBSD 10

Kernel: I/O and storage filesystems

FUSE

FUSE moved to base from ports to improve stabilityGiant-locked and GPL-contaminated filesystemsremoved from kernel: hpfs, ext2fs, ntfs, reiserfs, coda,xfs, nwfs, portalfs.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 21 / 24

Page 51: What's new in FreeBSD 10

Kernel: networking changes

networking changes

ZERO_COPY_SOCKETSsendfile(2) on shared memory fdnetwork byte order throughout the stackIP/TCP/UDP dtrace(1) providersraceless and cheap statistic per-CPU counters

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 22 / 24

Page 52: What's new in FreeBSD 10

Kernel: networking CARP

new CARP

CARP isn’t pseudo-interface anymore.% ifconfig igb0 10.0.0.112/27 vhid 112% ifconfig igb0igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500

ether 00:25:90:03:0e:fainet 10.0.0.112 netmask 0xffffffe0 broadcast 10.0.0.127 vhid 112media: Ethernet autoselect (1000baseT <full-duplex>)status: activecarp: BACKUP vhid 112 advbase 1 advskew 0

Redundant address is configured directly on a realinterface.

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 23 / 24

Page 53: What's new in FreeBSD 10

Kernel: networking packet filters

packet filters

pf(4): fork off OpenBSD, bringing in multithreadingipfilter(4): update to 5.1.2 (BSD license pledged)ipfw(4): no significant changes

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 24 / 24

Page 54: What's new in FreeBSD 10

Kernel: networking packet filters

Gleb Smirnoff [email protected] What’s new in FreeBSD 10? December 14, 2013 25 / 24