fscons scalable appplication transfers
Post on 16-Apr-2017
2.859 Views
Preview:
TRANSCRIPT
Scalable application layer transfers
Daniel Stenberg
Free Software
Network hacker, code and IETF
Embedded developer
Consultant
Email: daniel@haxx.seTwitter: @bagderWeb: daniel.haxx.seBlog: daniel.haxx.se/blog
Agenda
Traditional network client
applied to libcurl
C10K
applied to libcurl
Protocol stack
ContentsApplicationTransportInternetLink
libcurl
curl.haxx.se
Very portable C library
Application protocol transfers: DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP
Roots from 1997
MIT licensed
Daniel is lead developer
A typical client
Create socket
Connect socket
Select() for data to read or write
Read or write data
Loop
pseudo C code
WARNINGthe following source code examples are not complete and they resemble C code
int main(int argc, char *argv[]){ int sock; /* Socket descriptor */
if ((sock = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0) DieWithError("socket() failed");
/* it would resolve the host name here */
if (connect(sock to server)) DieWithError("connect() failed");
do {
select(maxfd+1, sock, ); /* wait for socket to become readable or writable */
if(check sock if writable) send(sock, string, string_len, 0);
if(check sock if readable) recv(sock, string, string_len, 0);
} while(everything is fine);
close(sock); exit(0);}
School book example
1
2
1
3
4
5
Simple case works fine
Easy to read
Fast to write
Easy to learn and teach
int main(int argc, char *argv[]){ int sock[NUM_CONNS]; /* Socket descriptors */
for(i=0; i< NUM_CONNS; i++) { if ((sock[i] = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0) DieWithError("socket() failed");
if (connect(sock[i] to server[i])) DieWithError("connect() failed");
do { /* set up the bitmasks for select() */
select(maxfd+1, sock, ); /* wait for any socket to become readable or writable */
for(i=0; i< NUM_CONNS; i++) { if(check sock[i] if writable) send(sock, string, string_len, 0);
if(check sock[i] if readable) recv(sock, string, string_len); }
} while(everything is fine);
close(sock); exit(0);}
More connections
1
2
1
3
4
5
int main(int argc, char *argv[]){ int sock[NUM_CONNS]; /* Socket descriptors */
for(i=0; i< NUM_CONNS; i++) { if ((sock[i] = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0) DieWithError("socket() failed");
if (connect(sock[i] to server[i])) DieWithError("connect() failed");
do { /* set up the bitmasks for select() */
select(maxfd+1, sock, ); /* wait for any socket to become readable or writable */
for(i=0; i< NUM_CONNS; i++) { if(check sock[i] if writable) send(sock, string, string_len, 0);
if(check sock[i] if readable) recv(sock, string, string_len); }
} while(everything is fine);
close(sock); exit(0);}
More connections
Many connections
Quickly degrades
select() and poll() suck
select() slightly more due to FD_SETSIZE
Threads help to some degree, but...
what about libcurl
examples in C
has bindings for 40 languages
libcurl simple app
int main(void){ CURL *curl; CURLcode res;
curl = curl_easy_init(); if(curl) { curl_easy_setopt(curl, CURLOPT_URL, "http://example.com"); res = curl_easy_perform(curl);
/* always cleanup */ curl_easy_cleanup(curl); } return 0;}
easy interface
easy to write
synchronous
supports many protocols in one go
made for single transfers
libcurl multi app
int main(int argc, char **argv){ handle1 = curl_easy_init(); handle2 = curl_easy_init();
curl_easy_setopt(handle1, CURLOPT_URL, "http://www.example.com/"); curl_easy_setopt(handle2, CURLOPT_URL, "http://fscons.org/");
multi_handle = curl_multi_init();
curl_multi_add_handle(multi_handle, handle1); curl_multi_add_handle(multi_handle, handle2);
curl_multi_perform(multi_handle, &still_running);
while(still_running) { curl_multi_timeout(multi_handle, &curl_timeo); curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);
rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout);
if(rc says timeout or readable/writable sockets) curl_multi_perform(multi_handle, &still_running); }
curl_multi_cleanup(multi_handle);
curl_easy_cleanup(http_handle); curl_easy_cleanup(http_handle2);
return 0;}
1
2
5
4
6
3
7
8
libcurl multi app
int main(int argc, char **argv){ handle1 = curl_easy_init(); handle2 = curl_easy_init();
curl_easy_setopt(handle1, CURLOPT_URL, "http://www.example.com/"); curl_easy_setopt(handle2, CURLOPT_URL, "http://fscons.org/");
multi_handle = curl_multi_init();
curl_multi_add_handle(multi_handle, handle1); curl_multi_add_handle(multi_handle, handle2);
curl_multi_perform(multi_handle, &still_running);
while(still_running) { curl_multi_timeout(multi_handle, &curl_timeo); curl_multi_fdset(multi_handle, &fdread, &fdwrite, &fdexcep, &maxfd);
rc = select(maxfd+1, &fdread, &fdwrite, &fdexcep, &timeout);
if(rc says timeout or readable/writable sockets) curl_multi_perform(multi_handle, &still_running); }
curl_multi_cleanup(multi_handle);
curl_easy_cleanup(http_handle); curl_easy_cleanup(http_handle2);
return 0;}
multi interface
several easy handles dealt with combined
made for simultaneous transfers
still easy to write and read
but: relies on select()
select with many
Scale to c10K
10000 simultaneous connections
not as easy as it may sound
A better way
event-based
react only on the sockets that have changed actions
no check for action
no loops
really scalable
not that portable
There's no POSIX for event-based
The portable libs don't support all platforms
epoll (Linux), kqueue (FreeBSD, NetBSD, OpenBSD, Darwin), /dev/poll (Solaris, HPUX), pollset (AIX), Event Completion (Solaris 10), I/O Completion Ports (Microsoft Windows)
libev and libevent
potent event libraries we base the examples on in this presentation
they support a subset of platforms
the examples are not complete now either
event-based basics
detail what sockets we work with and what do we wait for on the sockets
timeouts
wait until something happens
simple event client
int function(int socketfd){ev = ev_default_loop();
ev_timer_init(..., timer_callback, );
ev_io_init(ev, event_callback, socketfd, WHAT);
ev_loop(ev); /* sit here until done */}
int timer_callback(...){/* timer expired! */}
int event_callback(..., int socketfd, ...){/* something happened on my socket, now we recv or send etc */
}
event based compared
different code-flow
might require a different state machine for the protocol
again: not widely portable
Other ways to avoid wrongs
one thread per connection with blocking sockets
threads with non-blocking i/o to handle N connections each
using libcurl scalable
multi_socket API added 2006
tells the app what sockets to wait for what on
app then speaks with event library
avoids select()
event system agnostic
libcurl multi_socket bindings
not always supported
a matter of writing the glue code correct you can help!
multi_socket example
int main(){CURLM *multi = curl_multi_init();CURL *easy;
curl_multi_setopt(..., CURLMOPT_SOCKETFUNCTION, socket_cb);curl_multi_setopt(..., CURLMOPT_TIMERFUNCTION, timer_cb);
easy = curl_easy_init();curl_easy_setopt(..., CURLOPT_URL, );
curl_multi_add_handle(..., easy);
event_dispatch(); /* start the libevent dispatch loop */}
int event_cb(...){/* event library found event on libcurl's socket, act */curl_multi_socket_action(..., action, ...);}
int socket_cb(...){/* called when libcurl adds, removes or changes a socket that we must deal with */
event_set(..., event_cb); /* tell event lib about setup */}
Scales?
done 70,000+ connections
port numbers are 16 bit
callbacks take time thus limit throughput
Which protocols?
(almost) all network based ones libcurl support! (DICT, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS and TFTP)
Caveats?
yeah sure, but...
c-ares for name resolving
not everything is non-blocking
More factors
Network stack inefficiency
Stack size?
Memory / mallocs
Other code may slow down things
Copy copy copy copy ...
For you who fell asleep
old style socket applications don't scale
use event based concepts
libcurl can too
libcurl is...
entirely open source
in need of your help!
at http://curl.haxx.se/libcurl/
top related