COSCUP 2016

The Origin: Init

Tzung-Bi Shih <penvirus@gmail.com>

COSCUP 2016

Introduction• some particular jobs are required to execute during

system boot up

• more or less in a specific order

• service: bind on I/O ports for serving requests

• daemon: "never-dead" background process

➡ How does OS manage services/daemons to be launched at appropriate time?

2

COSCUP 2016

Glance at Booting• CPU executes the first instruction at a fixed address after

reset

• in the past, BIOS[1] / MBR

• initialize peripherals

• POST

• beep

• now, UEFI[2] / GPT

• attempt to find boot loader

3

COSCUP 2016

Boot Loader Example Grub2 in Linux[3]

• grub2

• mount temporary root file system

• load vmlinuz

• with kernel parameters[4]

• load initrd

• execute /init in initrd

• initrd[5]

• initialize essential environment, e.g., sysfs, procfs, device drivers

• remount real root file system

• /sbin/init, /etc/init, /bin/init, /bin/sh

4

set root='hd0,msdos1'if [ x$feature_platform_search_hint = xy ]; then search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 \ --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 \ 86b2cdf5-17d5-4d82-9f7b-933f1329209celse search --no-floppy --fs-uuid --set=root 86b2cdf5-17d5-4d82-9f7b-933f1329209cfilinux /vmlinuz-4.2.0-16-generic root=UUID=a06139f7-ad29-45fd-a759-882c7cdbe488 roinitrd /initrd.img-4.2.0-16-generic

init has been responsible for run control (rc) for 20+ years

# blkid/dev/sda1: UUID="86b2cdf5-17d5-4d82-9f7b-933f1329209c" TYPE="ext4" PARTUUID="2ef8dba4-01"/dev/sda2: UUID="a06139f7-ad29-45fd-a759-882c7cdbe488" TYPE="ext4" PARTUUID="2ef8dba4-02"# mount | grep sda/dev/sda2 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)/dev/sda1 on /boot type ext4 (rw,relatime,data=ordered)

COSCUP 2016

History Run Control in Init[6][7][8][9]

• Research UNIX

• /etc/rc

• 4.3BSD

• /etc/rc.local

• NetBSD 1.5, FreeBSD 5.0

• /etc/rc.conf, /etc/rc.d/

• SysVInit

• run level

• /etc/inittab

5

COSCUP 2016

RC Example inittab[10][11]

6

# Note, BusyBox init doesn't support runlevels. The runlevels field is# completely ignored by BusyBox init. If you want runlevels, use sysvinit.## Format for each entry: <id>:<runlevels>:<action>:<process>::sysinit:/etc/init.d/rcS

# Start an "askfirst" shell on the console (whatever that may be)::askfirst:-/bin/sh# Start an "askfirst" shell on /dev/tty2-4tty2::askfirst:-/bin/shtty3::askfirst:-/bin/sh

# /sbin/getty invocations for selected ttystty4::respawn:/sbin/getty 38400 tty5tty5::respawn:/sbin/getty 38400 tty6

# Example of how to put a getty on a serial line (for a terminal)#::respawn:/sbin/getty -L ttyS0 9600 vt100#::respawn:/sbin/getty -L ttyS1 9600 vt100

# Example how to put a getty on a modem line.#::respawn:/sbin/getty 57600 ttyS2

# Stuff to do before rebooting::ctrlaltdel:/sbin/reboot::shutdown:/bin/umount -a -r::shutdown:/sbin/swapoff -a

id:3:initdefault:

si::sysinit:/etc/rc.d/rc.sysinit

l0:0:wait:/etc/rc.d/rc 0l1:1:wait:/etc/rc.d/rc 1l2:2:wait:/etc/rc.d/rc 2l3:3:wait:/etc/rc.d/rc 3l4:4:wait:/etc/rc.d/rc 4l5:5:wait:/etc/rc.d/rc 5l6:6:wait:/etc/rc.d/rc 6

ca::ctrlaltdel:/sbin/shutdown -t3 -r now

1:2345:respawn:/sbin/mingetty tty12:2345:respawn:/sbin/mingetty tty23:2345:respawn:/sbin/mingetty tty34:2345:respawn:/sbin/mingetty tty45:2345:respawn:/sbin/mingetty tty56:2345:respawn:/sbin/mingetty tty6

x:5:once:/etc/X11/prefdm -nodaemon

busybox SysV

COSCUP 2016

RC Example BSD rc.d

7

# cat /etc/rc.d/cron#!/bin/sh## $FreeBSD: releng/10.2/etc/rc.d/cron 240336 2012-09-11 05:04:59Z obrien $#

# PROVIDE: cron# REQUIRE: LOGIN FILESYSTEMS# BEFORE: securelevel# KEYWORD: shutdown

. /etc/rc.subr

name="cron"rcvar="cron_enable"command="/usr/sbin/${name}"pidfile="/var/run/${name}.pid"

load_rc_config $nameif checkyesno cron_dstthen cron_flags="$cron_flags -s"firun_rc_command "$1"

should be defined in /etc/rc.conf or /etc/defaults/rc.conf

specify dependencies

Note: service will not be re-spawned if got crashed

COSCUP 2016

# ls -l /etc/rc1.d/lrwxrwxrwx 1 root root 14 Mar 23 2015 K20atop -> ../init.d/atop*lrwxrwxrwx 1 root root 15 Sep 5 2014 K20rsync -> ../init.d/rsync*lrwxrwxrwx 1 root root 24 Sep 5 2014 K20screen-cleanup -> ../init.d/screen-cleanup*lrwxrwxrwx 1 root root 27 May 30 2015 K80nfs-kernel-server -> ../init.d/nfs-kernel-server*-rw-r--r-- 1 root root 369 Mar 13 2014 READMElrwxrwxrwx 1 root root 19 Sep 5 2014 S30killprocs -> ../init.d/killprocs*lrwxrwxrwx 1 root root 19 Sep 5 2014 S70dns-clean -> ../init.d/dns-clean*lrwxrwxrwx 1 root root 18 Sep 5 2014 S70pppd-dns -> ../init.d/pppd-dns*lrwxrwxrwx 1 root root 16 Sep 5 2014 S90single -> ../init.d/single*

RC Example Linux init.d

8

# ls -ld /etc/rc*.ddrwxr-xr-x 2 root root 4096 May 30 2015 /etc/rc0.d/drwxr-xr-x 2 root root 4096 May 30 2015 /etc/rc1.d/drwxr-xr-x 2 root root 4096 May 30 2015 /etc/rc2.d/drwxr-xr-x 2 root root 4096 May 30 2015 /etc/rc3.d/drwxr-xr-x 2 root root 4096 May 30 2015 /etc/rc4.d/drwxr-xr-x 2 root root 4096 May 30 2015 /etc/rc5.d/drwxr-xr-x 2 root root 4096 May 30 2015 /etc/rc6.d/drwxr-xr-x 2 root root 4096 Sep 5 2014 /etc/rcS.d/

hybrid of SysV and BSD style

helper utilities (e.g. chkconfig, service, update-rc.d) manipulate the symbolic links directly

Note: service will not be re-spawned if got crashed

COSCUP 2016

We've Got a Problem Traditional Init is Insufficient[12][13][14]

• boot process is time-consumed

• some jobs can be launched simultaneously

• event-driven is now a common discipline

• especially in mobile devices and laptops

• e.g., PnP of USB drive

• too many similar but not comprehensive subsystems

• e.g., inetd, at, cron

9

COSCUP 2016

Modern Init• Mac OS X

• launchd[15][16][17]

• Ubuntu

• upstart[18]

• start from 15.04, Ubuntu defaults to systemd

• other mainstream Linux distributions

• systemd[19][20][21][22]

10

COSCUP 2016

launchd (1/9) Introduction

# launchctl load /Library/LaunchDaemons/com.example.xxx.plist# launchctl start /Library/LaunchDaemons/com.example.xxx.plist

• pid 1 of OS X 10.4 and later

• XML configuration file format

• daemon: runs on behalf of root

• agent: runs on behalf of the logged in user

11

COSCUP 2016

launchd (2/9) Traditional Daemon

12

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><dict> <key>Label</key> <string>com.example.hello</string> <key>ProgramArguments</key> <array> <string>hello</string> <string>world</string> </array> <key>KeepAlive</key> <true/></dict></plist>

unconditionally running at all time, i.e., will be re-spawned if the process got crashed

argv

COSCUP 2016

<key>SockServiceName</key><string>23</string>

launchd (3/9) On-demand Network Socket

13

<key>Sockets</key><dict> <key>Listeners</key> <dict> <key>SockServiceName</key> <string>bootps</string> <key>SockType</key> <string>dgram</string> <key>SockFamily</key> <string>IPv4</string> </dict></dict>

port number

from /etc/services

COSCUP 2016

launchd (4/9) Debugging

14

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><dict>[snip] <key>StandardOutPath</key> <string>/var/log/myjob.log</string> <key>StandardErrorPath</key> <string>/var/log/myjob.log</string> <key>Debug</key> <true/> <key>SoftResourceLimits</key> <dict> <key>Core</key> <integer>9223372036854775807</integer> </dict> <key>HardResourceLimits</key> <dict> <key>Core</key> <integer>9223372036854775807</integer> </dict></dict></plist>

set rlimits for core dumps

redirect stdout and stderr

COSCUP 2016

launchd (5/9) Periodic Job

15

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><dict> <key>Label</key> <string>com.example.touchsomefile</string> <key>ProgramArguments</key> <array> <string>touch</string> <string>/tmp/helloworld</string> </array> <key>StartInterval</key> <integer>300</integer></dict></plist>

in seconds

COSCUP 2016

launchd (6/9) Cron Job

16

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><dict> <key>Label</key> <string>com.example.touchsomefile</string> <key>ProgramArguments</key> <array> <string>touch</string> <string>/tmp/helloworld</string> </array> <key>StartCalendarInterval</key> <dict> <key>Minute</key> <integer>45</integer> <key>Hour</key> <integer>13</integer> <key>Day</key> <integer>7</integer> </dict></dict></plist>

wildcards if unspecified

COSCUP 2016

launchd (7/9) Monitor Directory for Changing

17

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><dict> <key>Label</key> <string>com.example.watchhostconfig</string> <key>ProgramArguments</key> <array> <string>syslog</string> <string>-s</string> <string>-l</string> <string>notice</string> <string>somebody touched /etc/hostconfig</string> </array> <key>WatchPaths</key> <array> <string>/etc/hostconfig</string> </array></dict></plist>

target directories

COSCUP 2016

launchd (8/9) Monitor Directory for Emptiness

18

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><dict> <key>Label</key> <string>com.example.mailpush</string> <key>ProgramArguments</key> <array> <string>my_custom_mail_push_tool</string> </array> <key>QueueDirectories</key> <array> <string>/var/spool/mymailqdir</string> </array></dict></plist>

• start the job whenever the directories are non-empty • keep running as long as the directories are not empty

COSCUP 2016

launchd (9/9) Inetd

19

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><dict> <key>Label</key> <string>com.example.telnetd</string> <key>ProgramArguments</key> <array> <string>/usr/libexec/telnetd</string> </array> <key>inetdCompatibility</key> <dict> <key>Wait</key> <false/> </dict> <key>Sockets</key> <dict> <key>Listeners</key> <dict> <key>SockServiceName</key> <string>telnet</string> <key>SockType</key> <string>stream</string> </dict> </dict></dict></plist>

This flag corresponds to the "wait" or "nowait" option of inetd. • If true, then the listening socket is passed

via the standard in/out/error file descriptors. • If false, then accept(2) is called on behalf of

the job, and the result is passed via the standard in/out/error descriptors.

COSCUP 2016

upstart (1/2) Introduction

• pid 1 of Ubuntu 6.10~14.10

• non-standard configuration file format

• intended to be default init for all Linux distributions

• has frozen release and replaced by systemd[22][23]

20

upstart has obsoleted; no need to pay too much attention on it

COSCUP 2016

upstart (2/2) Example: cron

21

# cat /etc/init/cron.conf# cron - regular background program processing daemon## cron is a standard UNIX program that runs user-specified programs at# periodic scheduled times

description "regular background program processing daemon"

start on runlevel [2345]stop on runlevel [!2345]

expect forkrespawn

exec cron

# start cron# stop cron# status cron# initctl status cron# initctl emit xxx

the process will call fork exactly once

dependency events

respawn if the process has crashedtarget executable

symbolic links to initctl

COSCUP 2016

systemd (1/6) Introduction

• pid 1 of most Linux distributions

• young but huge project

• ini configuration file format

• everything is an "unit"

• e.g., target, service, timer, socket

• run level has replaced by target unit

22

# systemctl start xxx# systemctl stop xxx# systemctl enable xxx# systemctl disable xxx# systemctl status xxx# journalctl

COSCUP 2016

systemd (2/6) Service Unit

23

# cat /lib/systemd/system/sshd.service[Unit]Description=OpenBSD Secure Shell serverAfter=network.target auditd.serviceConditionPathExists=!/etc/ssh/sshd_not_to_be_run

[Service]EnvironmentFile=-/etc/default/sshExecStart=/usr/sbin/sshd -D $SSHD_OPTSExecReload=/bin/kill -HUP $MAINPIDKillMode=processRestart=on-failureRestartPreventExitStatus=255Type=notify

[Install]WantedBy=multi-user.targetAlias=sshd.service

specify belonging target

associated executable

the service will be restarted when the process exits with a non-zero exit code

COSCUP 2016

systemd (3/6) Socket Unit

24

# cat /lib/systemd/system/sshd.socket[Unit]Description=OpenSSH Server SocketDocumentation=man:sshd(8) man:sshd_config(5)Conflicts=sshd.service

[Socket]ListenStream=22Accept=yes

[Install]WantedBy=sockets.target

If a unit has a Conflicts= setting on another unit, starting the former will stop the latter and vice versa.

something like placeholder# systemctl list-unit-files | grep sshssh.service enabledssh@.service staticsshd.service enabledssh.socket disabled

COSCUP 2016

systemd (4/6) Target Unit

25

# cat /lib/systemd/system/multi-user.target[Unit]Description=Multi-User SystemDocumentation=man:systemd.special(7)Requires=basic.targetConflicts=rescue.service rescue.targetAfter=basic.target rescue.service rescue.targetAllowIsolate=yes

# systemctl isolate multi-user.target# systemctl isolate graphical.target

used for grouping units and as well-known synchronization points during start-up

COSCUP 2016

# cat /lib/systemd/system/systemd-tmpfiles-clean.service[Unit]Description=Cleanup of Temporary DirectoriesDocumentation=man:tmpfiles.d(5) man:systemd-tmpfiles(8)DefaultDependencies=noConflicts=shutdown.targetAfter=local-fs.target time-sync.targetBefore=shutdown.target

[Service]Type=oneshotExecStart=/bin/systemd-tmpfiles --cleanIOSchedulingClass=idle

systemd (5/6) Timer Unit

26

# cat /lib/systemd/system/systemd-tmpfiles-clean.timer[Unit]Description=Daily Cleanup of Temporary DirectoriesDocumentation=man:tmpfiles.d(5) man:systemd-tmpfiles(8)

[Timer]OnBootSec=15minOnUnitActiveSec=1d

COSCUP 2016

# cat /lib/systemd/system/systemd-networkd-resolvconf-update.service[Unit]Description=Update resolvconf for networkd DNSConditionPathIsSymbolicLink=/etc/resolv.confConditionPathExists=/run/resolvconf/enable-updatesAfter=resolvconf.service

[Service]Type=oneshotStartLimitBurst=20# we might be triggered several times in short succession during restarting networkd, so wait until we get a DNS entryExecStart=/bin/sh -c 'for timeout in `seq 30`; do out=$(sed -n "/^DNS=/ { s/^DNS=/nameserver /; p}" /run/systemd/netif/state); [ -z "$out" ] || break; sleep 1; done; echo "$out" | /sbin/resolvconf -a networkd'

systemd (6/6) Path Unit

27

# cat /lib/systemd/system/systemd-networkd-resolvconf-update.path[Unit]Description=Trigger resolvconf update for networkd DNSConditionPathIsSymbolicLink=/etc/resolv.conf

[Path]PathChanged=/run/systemd/netif/state

COSCUP 2016

Conclusion• launchd is excellent

• systemd covers a lot of business

• much more than what we mentioned before

• systemd won the war

• although it is controversial[25]

• start to learn and use systemd

• whether you like it or not

• If systemd is not qualified, it shall be replaced again soon. But you need to use it before judging systemd.

28

COSCUP 2016

References

29

[1]: https://en.wikipedia.org/wiki/BIOS [2]: https://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface [3]: https://en.wikipedia.org/wiki/Linux_startup_process [4]: https://www.kernel.org/doc/Documentation/kernel-parameters.txt [5]: https://www.kernel.org/doc/Documentation/initrd.txt [6]: http://aplawrence.com/Basics/unix-startup-scripts-1.html [7]: http://aplawrence.com/Basics/unix-startup-scripts-2.html [8]: http://linuxmafia.com/faq/Admin/init.html [9]: https://en.wikipedia.org/wiki/Init [10]: http://linuxembedded.blogspot.tw/2006/11/understanding-busybox-inittab.html [11]: http://linux.vbird.org/linux_basic/0510osloader/0510osloader-fc4.php#startup_init [12]: http://aplawrence.com/Basics/unix-startup-scripts-3.htm [13]: http://www.tecmint.com/systemd-replaces-init-in-linux/ [14]: http://www.pcworld.com/article/2841873/meet-systemd-the-controversial-project-taking-over-a-linux-distro-near-you.html [15]: http://launchd.info/ [16]: http://paul.annesley.cc/2012/09/mac-os-x-launchd-is-cool/ [17]: https://developer.apple.com/library/mac/documentation/MacOSX/Conceptual/BPSystemStartup/Chapters/CreatingLaunchdJobs.html [18]: http://upstart.ubuntu.com/ [19]: https://www.certdepot.net/rhel7-get-started-systemd/ [20]: http://www.freedesktop.org/wiki/Software/systemd/ [21]: https://fedoraproject.org/wiki/Systemd [22]: https://wiki.archlinux.org/index.php/Systemd [23]: http://www.zdnet.com/article/debian-init-decision-further-isolates-ubuntu/ [24]: http://www.zdnet.com/article/after-linux-civil-war-ubuntu-to-adopt-systemd/ [25]: http://without-systemd.org/wiki/index.php/Arguments_against_systemd