Adam Leventhal, @ahl

A Brief History of System Calls

Who Am I?

•  Co-inventor of DTrace

•  Founder ZFS Storage Appliance team

•  CTO at Delphix

•  EIR at Sutter Hill Ventures

•  Recent founder

What’s a System Call?

•  User call into the kernel to do some work

•  Typically implemented with a trap

•  Mostly standardized in terms of function

•  Fundamental to multi-user systems

USENIX Summer 1986

Time Spent in Syscalls Matters

Syscalls Tell You About The System

ULTRIX Trace

/dev/trace

… If (tracing)

... /* do the syscall */ If (tracing)

...

Syscall Handler

call exec(...) return exec(...) call open(...) return open(...) call read(...) return read(...)

Buffer

Aside: How Great Is This?

Syscall Tracing For All

•  Truss 1988 –  “If your program doesn't work, put it in a truss.”

– Roger Faulker –  Used then-new structured /proc –  SunOS / UNIX

•  Strace 1991 –  “an alternative syscall tracer”

– Paul Kranenburg on comp.sources.sun –  SunOS then, famously, Linux in 1992 –  Non-Linux code removed in 2012

Breakpoint Tracing

process invoke syscall execute syscall proceed

scheduler stop process wake tracer stop tracer wake process stop process wake tracer stop tracer wake process

tracer record arguments resume process record return value signal process

Slow

# Slow the target command and print details for each syscall:strace command

# Slow the target PID and print details for each syscall:strace -p PID

# Slow the target PID and any newly created child process, printing syscall details:strace -fp PID

# Slow the target PID and record syscalls, printing a summary:strace -cp PID

# Slow the target PID and trace open() syscalls only:strace -eopen -p PID

Brendan Gregg

Do No Harm

•  Unsafe

•  Chase away the problem

DTrace

•  Dynamic tracing framework –  Solaris, macOS, FreeBSD, some Linux distros

•  Safe for production by design

•  Concise answers to arbitrary questions

•  Systemic in scope

•  Syscalls are a common place to start

Syscalls with DTrace

Aside: Java

“Write once, run anywhere”

Java Java Java

Servers in1996

Write Once Run Anywhere + 20yrs

Primacy of Syscalls

•  Highly significant events

•  30 years later, still the right granularity

•  Well-understood domain for tracers

•  Ubiquitous

•  De facto standard in Linux

Thank You!