woo: writing a fast web server @ els2015

Woo: Writing a fast web server

8th European Lisp Symposium @ London, UK April 21, 2015

Eitaro Fukamachi Somewrite Co., Ltd.

I’m Eitaro Fukamachi @nitro_idiot fukamachi

My Products

• Clack

• Caveman2

• CL-DBI

• quickdocs.org

• datafly

• SxQL

...and 25 other libraries

We’re hiring!

Woo

• HTTP server written in Common Lisp

• HTTP/0.9, HTTP/1.x

• Clack compliant

Woo

Clack?• Abstraction layer of HTTP servers

Web server

Application

Web server Web server

Application Application

Clack?• Abstraction layer of HTTP servers

Web server

Application

Web server Web server

Application Application

Clack

ex) Caveman2• built on top of Clack

Web server Web server Web server

Caveman2(Web framework)

Clack

ex) RESTAS (without Clack)• directly built on top of Hunchentoot

Web server

Depends on Hunchentoot. Can’t switch the backend!!

Supports only Hunchentoot

Web server Web server

RESTAS(Web framework)

Clack-compliant

(woo:run (lambda (env) ‘(200 (:content-type “text/plain”) (“Hello, World”))))

(clack:clackup (lambda (env) ‘(200 (:content-type “text/plain”) (“Hello, World”))) :server :woo)

Run with Clack

Fast

https://github.com/fukamachi/woo#benchmarks

Fast — in the real worldre

q/se

c

0

33

65

98

130

Node.js Woo

127.18

76.73

1.6 times faster!

Let me tell why I had to write a fast HTTP server.

Wookie is slower than Node.js

• Wookie is 2 times slower than Node.js

Wookie is slower than Node.js

• Wookie is 2 times slower than Node.js

IS COMMON LISP SLOW???

Wookie is slower than Node.js

• Wookie is 2 times slower than Node.js

IS COMMON LISP SLOW??? NO WAY!

So, I decided to write a faster one.

How could it be fast?

3 difficulties in writing an HTTP servers

3 difficulties in HTTP server

• Network I/O is the largest bottleneck

• Need to handle a vast amount of requests at once

• Need to handle various HTTP clients (fast / slow / unstable)

3 tactics (no silver bullet)

1. Better architecture

2. Fast HTTP parsing

3. The libev event library

Tactic 1: Better architecture

2 architectures: Prefork vs Event-driven

Prefork

Worker process

Worker process

Worker process

master process

Requests

accept connections

Responses

• Simple

• Fast for little simultaneous connections

• ex) Hunchentoot, Unicorn, Apache

Prefork

Worker process

Worker process

Worker process

master process

Requests

accept connections

Responses • Slow client can cause performance issue.

• like Mobile users

• Slowloris attack

blocking!

･･･

Problem

Event-driven• Handle many

clients at the same time

• Asnyc ACCEPT/READ/WRITE

• ex) Woo, Wookie, Tornado, nginxServer process

(single-threaded)

Event loop

Event-driven

• Single-threaded

Problem

Server process (single-threaded)

Event loop

Woo took another way: Multithreaded event-driven

Multithreaded event-driven

Server process

Event loop

listen on the same file descriptor

Server process

Event loop

Server process

Event loop

Tactic 2: Fast HTTP parsing

HTTP parsing can be a bottleneck

• Wookie's largest bottleneck is HTTP parsing

• http-parse (uses regular expression)

• fast-http (byte by byte parser)

• 6000 times faster than http-parse

A brief introduction of HTTP

HTTP request look like…

GET /media HTTP/1.1↵ Host: somewrite.jp↵ Connection: keep-alive↵ Accept: */*↵

↵

HTTP request look like…

GET /media HTTP/1.1↵ Host: somewrite.jp↵ Connection: keep-alive↵ Accept: */*↵

↵

First Line

Headers

Body (empty, in this case)

HTTP request look like…

GET /media HTTP/1.1↵ Host: somewrite.jp↵ Connection: keep-alive↵ Accept: */*↵

↵ CR + LF

CRLF * 2 at the end of headers

HTTP is…

• Text-based protocol. (not binary)

• Lines terminated with CRLF

• Very lenient.

• Ignore multiple spaces

• Allow continuous header values

And, there’s another difficulty.

HTTP messages are sent over a network.

Which means, we need to think about long & incomplete HTTP messages.

There’s 2 ways to resolve this problem.

1. Stateful (http-parser)

http-parser (used in Node.js)

• https://github.com/joyent/http-parser

• Written in C

• Ported from Nginx’s HTTP parser

• Written as Node.js’s HTTP parser

• Stateful

http-parser (used in Node.js)for (p=data; p != data + len; p++) { … switch (parser->state) { case s_dead: … case s_start_req_or_res: … case s_res_or_resp_H: … } }

http-parser (used in Node.js)for (p=data; p != data + len; p++) { … switch (parser->state) { case s_dead: … case s_start_req_or_res: … case s_res_or_resp_H: … } }

Process char by char

Do something for each state

http-parser (used in Node.js)for (p=data; p != data + len; p++) { … switch (parser->state) { case s_dead: … case s_start_req_or_res: … case s_res_or_resp_H: … } }

Process char by char

Do something for each state

Executed for every char!!

2. Stateless (PicoHTTPParser)

PicoHTTPParser (used in H2O)

• https://github.com/h2o/picohttpparser

• Written in C

• Stateless

PicoHTTPParser (used in H2O)

• https://github.com/h2o/picohttpparser

• Written in C

• Stateless

• Reparse when the data is incomplete

• Most HTTP request is small

And fast-http is…

fast-http is in the middle

• Stateful

• Track state for every line, not every char

• 1.25 times faster than C http-parser.

Tactic 3: The libev event library

libev

• Wrapper of epoll, kqueue, POSIX select, poll

• Thin

• Fast

libev

• Wrapper of epoll, kqueue, POSIX select, poll

• Thin

• Fast

• Poor Windows support

• Windows isn’t popular for running an HTTP server

Goal

Goal: What is fast enough?

• The initial goal was “Beating Node.js”

• Done

• Being the fastest HTTP server in Common Lisp

• Done

Got a great Pull Request

See Also

• Clack: clacklisp.org

• fast-http: github.com/fukamachi/fast-http

• libev

• Woo: github.com/fukamachi/woo

You may be interested in

• Dexador: github.com/fukamachi/dexador

• HTTP client library built on top of fast-http and usocket.

Thanks.

EITARO FUKAMACHI 8arrow.org @nitro_idiot fukamachi