stm on pypy
TRANSCRIPT
The Free Lunch Is Over: A Fundamental Turn Toward Concurrency in Software
Herb Sutterhttp://www.gotw.ca/publications/concurrency-ddj.htm
Inside the Python GIL
David Beazleyhttps://www.youtube.com/watch?v=ph374fJqFPE
Removing the GIL
● Fine-grained locking● Shared-nothing● STM
BEGIN;
UPDATE accounts SET balance = balance - 100.00
WHERE name = 'Alice';
UPDATE accounts SET balance = balance + 100.00
WHERE name = 'Bob';
COMMIT;
def update_account(src_account_id, dst_account_id, amount):
with atomic:src_account = accounts[src_account_id]dst_account = acconts[dst_account_id]dst_account.deposit(account.withdraw(amount))
● Hardware and Software support● Some modern processors (Haswell) support it
– https://software.intel.com/en-us/blogs/2012/02/07/transactional-synchronization-in-haswell
● Transaction is started● Memory read are logged into read set● Memory writes are logged into write set● Commit is attempted
PyPy approach
Illustration taken from Armin Rigo
Problems
● Too experimental● Only runs in 64 bit Linux and patched version of
Clang (just for translation)● Limited to 1.5 GB of Ram (Configurable)● JIT warm up still pretty bad● Different GC from regular PyPy (Less efficient)● Long running processes crash
Demo
References
● Rajwar, Herlihy and Lai: Virtualizing Transactional Memory
● Herlihy and Moss: Transactional Memory: Architectural Support for Lock-Free Data Structures
● Dice, Shalev, Shavit: Transactional Locking II
Donations
● http://pypy.org/tmdonate2.html