Download - Introduction to asyncio
![Page 1: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/1.jpg)
@saghul
Introduction to asyncioSaúl Ibarra Corretgé
PyLadies Amsterdam - 20th March 2014
![Page 2: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/2.jpg)
github.com/saghul
![Page 3: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/3.jpg)
![Page 4: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/4.jpg)
Sockets 101
import socket
server = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM)server.bind(('127.0.0.1', 1234))server.listen(128)print("Server listening on: {}".format(server.getsockname()))
client, addr = server.accept()print("Client connected: {}".format(addr))
while True: data = client.recv(4096) if not data: print("Client has disconnected") break client.send(data)
server.close()
![Page 5: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/5.jpg)
Scaling Up!
import socketimport _thread
def handle_client(client, addr): print("Client connected: {}".format(addr)) while True: data = client.recv(4096) if not data: print("Client has disconnected") break client.send(data.upper())
server = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM)server.bind(('127.0.0.1', 1234))server.listen(128)print("Server listening on: {}".format(server.getsockname()))
while True: client, addr = server.accept() _thread.start_new_thread(handle_client, (client, addr))
![Page 6: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/6.jpg)
Scaling Up! (really?)
Threads are too expensive
Also, context switching
Use an event loop instead!
![Page 7: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/7.jpg)
The Async Way (TM)
Single thread
Block waiting for sockets to be ready to read or write
Perform i/o operations and call the callbacks!
Repeat
(Windows is not like this)
![Page 8: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/8.jpg)
Why asyncio?
asyncore and asynchat are not enough
Fresh new implementation of Asynchronous I/O
Python >= 3.3
Trollius: backport for Python >= 2.6
Use new language features: yield from
Designed to interoperate with other frameworks
![Page 9: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/9.jpg)
Components
Event loop, policy
Coroutines, Futures, Tasks
Transports, Protocols
![Page 10: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/10.jpg)
asyncio 101
import asyncio
loop = asyncio.get_event_loop()
@asyncio.coroutinedef infinity(): while True: print("hello!") yield from asyncio.sleep(1)
loop.run_until_complete(infinity())
![Page 11: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/11.jpg)
Coroutines, futures & tasks
![Page 12: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/12.jpg)
Coroutines, futures & tasksCoroutine
generator function, can receive values
decorated with @coroutine
Future
promise of a result or an error
Task
Future which runs a coroutine
![Page 13: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/13.jpg)
FuturesSimilar to Futures from PEP-3148
concurrent.futures.Future
API (almost) identical:
f.set_result(); r = f.result()
f.set_exception(e); e = f.exception()
f.done(); f.cancel(); f.cancelled()
f.add_done_callback(x); f.remove_done_callback(x)
![Page 14: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/14.jpg)
Futures + Coroutines
yield from works with Futures!
f = Future()
Someone will set the result or exception
r = yield from f
Waits until done and returns f.result()
Usually returned by functions
![Page 15: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/15.jpg)
Undoing callbacks
@asyncio.coroutinedef sync_looking_function(*args): fut = asyncio.Future() def cb(result, error): if error is not None: fut.set_result(result) else: fut.set_exception(Exception(error)) async_function(cb, *args) return (yield from fut)
![Page 16: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/16.jpg)
Tasks
Unicorns covered in fairy dust
It’s a coroutine wrapped in a Future
WAT
Inherits from Future
Works with yield from!
r = yield from Task(coro(...))
![Page 17: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/17.jpg)
Tasks vs coroutines
A coroutine doesn’t “advance” without a scheduling mechanism
Tasks can advance on their own
The event loop is the scheduler!
Magic!
![Page 18: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/18.jpg)
Echo Serverimport asyncio
loop = asyncio.get_event_loop()
class EchoProtocol(asyncio.Protocol): def connection_made(self, transport): print('Client connected') self.transport = transport def data_received(self, data): print('Received data:',data) self.transport.write(data) def connection_lost(self, exc): print('Connection closed', exc)
f = loop.create_server(lambda: EchoProtocol(), 'localhost', 1234)server = loop.run_until_complete(f)print('Server started')
loop.run_forever()
![Page 19: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/19.jpg)
Echo Server Reloadedimport asyncio
loop = asyncio.get_event_loop()clients = {} # task -> (reader, writer)
def accept_client(client_reader, client_writer): task = asyncio.Task(handle_client(client_reader, client_writer)) clients[task] = (client_reader, client_writer) def client_done(task): del clients[task] task.add_done_callback(client_done)
@asyncio.coroutinedef handle_client(client_reader, client_writer): while True: data = (yield from client_reader.readline()) client_writer.write(data)
f = asyncio.start_server(accept_client, '127.0.0.1', 1234)server = loop.run_until_complete(f)loop.run_forever()
![Page 20: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/20.jpg)
HTTP Serverimport asyncioimport aiohttpimport aiohttp.server
class HttpServer(aiohttp.server.ServerHttpProtocol):
@asyncio.coroutine def handle_request(self, message, payload): print('method = {!r}; path = {!r}; version = {!r}'.format( message.method, message.path, message.version)) response = aiohttp.Response(self.transport, 200, close=True) response.add_header('Content-type', 'text/plain') response.send_headers() response.write(b'Hello world!\r\n') response.write_eof()
loop = asyncio.get_event_loop()f = loop.create_server(lambda: HttpServer(), 'localhost', 1234)server = loop.run_until_complete(f)loop.run_forever()
![Page 21: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/21.jpg)
Redis
import asyncioimport asyncio_redis
@asyncio.coroutinedef subscriber(channels): # Create connection connection = yield from asyncio_redis.Connection.create(host='localhost', port=6379) # Create subscriber. subscriber = yield from connection.start_subscribe() # Subscribe to channel. yield from subscriber.subscribe(channels) # Wait for incoming events. while True: reply = yield from subscriber.next_published() print('Received: ', repr(reply.value), 'on channel', reply.channel)
loop = asyncio.get_event_loop()loop.run_until_complete(subscriber(['my-channel']))
![Page 22: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/22.jpg)
More?
We just scratched the surface!
Read PEP-3156 (it’s an easy read, I promise!)
Checkout the documentation
Checkout the third-party libraries
Go implement something cool!
“I hear and I forget. I see and I remember.
I do and I understand.” - Confucius
![Page 23: Introduction to asyncio](https://reader036.vdocuments.us/reader036/viewer/2022081400/554a0edbb4c9055c598b4957/html5/thumbnails/23.jpg)
Questions?
@saghulbettercallsaghul.com