woo: writing a fast web server @ els2015
TRANSCRIPT
![Page 1: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/1.jpg)
Woo: Writing a fast web server
8th European Lisp Symposium @ London, UK April 21, 2015
Eitaro Fukamachi Somewrite Co., Ltd.
![Page 2: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/2.jpg)
I’m Eitaro Fukamachi @nitro_idiot fukamachi
![Page 3: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/3.jpg)
My Products
• Clack
• Caveman2
• CL-DBI
• quickdocs.org
• datafly
• SxQL
...and 25 other libraries
![Page 4: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/4.jpg)
![Page 5: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/5.jpg)
We’re hiring!
![Page 6: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/6.jpg)
Woo
![Page 7: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/7.jpg)
• HTTP server written in Common Lisp
• HTTP/0.9, HTTP/1.x
• Clack compliant
Woo
![Page 8: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/8.jpg)
Clack?• Abstraction layer of HTTP servers
Web server
Application
Web server Web server
Application Application
![Page 9: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/9.jpg)
Clack?• Abstraction layer of HTTP servers
Web server
Application
Web server Web server
Application Application
Clack
![Page 10: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/10.jpg)
ex) Caveman2• built on top of Clack
Web server Web server Web server
Caveman2(Web framework)
Clack
![Page 11: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/11.jpg)
ex) RESTAS (without Clack)• directly built on top of Hunchentoot
Web server
Depends on Hunchentoot. Can’t switch the backend!!
Supports only Hunchentoot
Web server Web server
RESTAS(Web framework)
![Page 12: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/12.jpg)
Clack-compliant
(woo:run (lambda (env) ‘(200 (:content-type “text/plain”) (“Hello, World”))))
(clack:clackup (lambda (env) ‘(200 (:content-type “text/plain”) (“Hello, World”))) :server :woo)
Run with Clack
![Page 14: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/14.jpg)
Fast — in the real worldre
q/se
c
0
33
65
98
130
Node.js Woo
127.18
76.73
1.6 times faster!
![Page 15: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/15.jpg)
Let me tell why I had to write a fast HTTP server.
![Page 16: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/16.jpg)
![Page 17: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/17.jpg)
Wookie is slower than Node.js
• Wookie is 2 times slower than Node.js
![Page 18: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/18.jpg)
Wookie is slower than Node.js
• Wookie is 2 times slower than Node.js
IS COMMON LISP SLOW???
![Page 19: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/19.jpg)
Wookie is slower than Node.js
• Wookie is 2 times slower than Node.js
IS COMMON LISP SLOW??? NO WAY!
![Page 20: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/20.jpg)
So, I decided to write a faster one.
![Page 21: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/21.jpg)
How could it be fast?
![Page 22: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/22.jpg)
3 difficulties in writing an HTTP servers
![Page 23: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/23.jpg)
3 difficulties in HTTP server
• Network I/O is the largest bottleneck
• Need to handle a vast amount of requests at once
• Need to handle various HTTP clients (fast / slow / unstable)
![Page 24: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/24.jpg)
3 tactics (no silver bullet)
1. Better architecture
2. Fast HTTP parsing
3. The libev event library
![Page 25: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/25.jpg)
Tactic 1: Better architecture
![Page 26: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/26.jpg)
2 architectures: Prefork vs Event-driven
![Page 27: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/27.jpg)
Prefork
Worker process
Worker process
Worker process
master process
Requests
accept connections
Responses
• Simple
• Fast for little simultaneous connections
• ex) Hunchentoot, Unicorn, Apache
![Page 28: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/28.jpg)
Prefork
Worker process
Worker process
Worker process
master process
Requests
accept connections
Responses • Slow client can cause performance issue.
• like Mobile users
• Slowloris attack
blocking!
・・・
Problem
![Page 29: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/29.jpg)
Event-driven• Handle many
clients at the same time
• Asnyc ACCEPT/READ/WRITE
• ex) Woo, Wookie, Tornado, nginxServer process
(single-threaded)
Event loop
![Page 30: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/30.jpg)
Event-driven
• Single-threaded
Problem
Server process (single-threaded)
Event loop
![Page 31: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/31.jpg)
Woo took another way: Multithreaded event-driven
![Page 32: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/32.jpg)
Multithreaded event-driven
Server process
Event loop
listen on the same file descriptor
Server process
Event loop
Server process
Event loop
![Page 33: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/33.jpg)
Tactic 2: Fast HTTP parsing
![Page 34: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/34.jpg)
HTTP parsing can be a bottleneck
• Wookie's largest bottleneck is HTTP parsing
• http-parse (uses regular expression)
• fast-http (byte by byte parser)
• 6000 times faster than http-parse
![Page 35: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/35.jpg)
A brief introduction of HTTP
![Page 36: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/36.jpg)
HTTP request look like…
GET /media HTTP/1.1↵ Host: somewrite.jp↵ Connection: keep-alive↵ Accept: */*↵
↵
![Page 37: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/37.jpg)
HTTP request look like…
GET /media HTTP/1.1↵ Host: somewrite.jp↵ Connection: keep-alive↵ Accept: */*↵
↵
First Line
Headers
Body (empty, in this case)
![Page 38: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/38.jpg)
HTTP request look like…
GET /media HTTP/1.1↵ Host: somewrite.jp↵ Connection: keep-alive↵ Accept: */*↵
↵ CR + LF
CRLF * 2 at the end of headers
![Page 39: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/39.jpg)
HTTP is…
• Text-based protocol. (not binary)
• Lines terminated with CRLF
• Very lenient.
• Ignore multiple spaces
• Allow continuous header values
![Page 40: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/40.jpg)
And, there’s another difficulty.
![Page 41: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/41.jpg)
HTTP messages are sent over a network.
![Page 42: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/42.jpg)
Which means, we need to think about long & incomplete HTTP messages.
![Page 43: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/43.jpg)
There’s 2 ways to resolve this problem.
![Page 44: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/44.jpg)
1. Stateful (http-parser)
![Page 45: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/45.jpg)
http-parser (used in Node.js)
• https://github.com/joyent/http-parser
• Written in C
• Ported from Nginx’s HTTP parser
• Written as Node.js’s HTTP parser
• Stateful
![Page 46: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/46.jpg)
http-parser (used in Node.js)for (p=data; p != data + len; p++) { … switch (parser->state) { case s_dead: … case s_start_req_or_res: … case s_res_or_resp_H: … } }
![Page 47: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/47.jpg)
http-parser (used in Node.js)for (p=data; p != data + len; p++) { … switch (parser->state) { case s_dead: … case s_start_req_or_res: … case s_res_or_resp_H: … } }
Process char by char
Do something for each state
![Page 48: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/48.jpg)
http-parser (used in Node.js)for (p=data; p != data + len; p++) { … switch (parser->state) { case s_dead: … case s_start_req_or_res: … case s_res_or_resp_H: … } }
Process char by char
Do something for each state
Executed for every char!!
![Page 49: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/49.jpg)
2. Stateless (PicoHTTPParser)
![Page 50: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/50.jpg)
PicoHTTPParser (used in H2O)
• https://github.com/h2o/picohttpparser
• Written in C
• Stateless
![Page 51: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/51.jpg)
PicoHTTPParser (used in H2O)
• https://github.com/h2o/picohttpparser
• Written in C
• Stateless
• Reparse when the data is incomplete
• Most HTTP request is small
![Page 52: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/52.jpg)
And fast-http is…
![Page 53: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/53.jpg)
fast-http is in the middle
• Stateful
• Track state for every line, not every char
• 1.25 times faster than C http-parser.
![Page 54: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/54.jpg)
Tactic 3: The libev event library
![Page 55: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/55.jpg)
libev
• Wrapper of epoll, kqueue, POSIX select, poll
• Thin
• Fast
![Page 56: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/56.jpg)
libev
• Wrapper of epoll, kqueue, POSIX select, poll
• Thin
• Fast
• Poor Windows support
• Windows isn’t popular for running an HTTP server
![Page 57: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/57.jpg)
Goal
![Page 58: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/58.jpg)
Goal: What is fast enough?
• The initial goal was “Beating Node.js”
• Done
• Being the fastest HTTP server in Common Lisp
• Done
![Page 59: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/59.jpg)
Got a great Pull Request
![Page 60: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/60.jpg)
See Also
• Clack: clacklisp.org
• fast-http: github.com/fukamachi/fast-http
• libev
• Woo: github.com/fukamachi/woo
![Page 61: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/61.jpg)
You may be interested in
• Dexador: github.com/fukamachi/dexador
• HTTP client library built on top of fast-http and usocket.
![Page 62: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/62.jpg)
Thanks.
![Page 63: Woo: Writing a fast web server @ ELS2015](https://reader030.vdocuments.us/reader030/viewer/2022020307/55aa88dd1a28ab50028b45a3/html5/thumbnails/63.jpg)
EITARO FUKAMACHI 8arrow.org @nitro_idiot fukamachi