building and breaking a python sandboxweb.mit.edu/jesstess/www/pytennessee_sandbox.pdfexamples in...

Building and breaking a Python sandbox

Director

Organizer

@jessicamckellarhttp://jesstess.com

http://jesstess.com

http://jesstess.com

Why?

• Learning a language

• Providing a hosted scratch pad

• Distributed computation

• Inspecting running processes safely

Examples in the wild

• Seattle’s peer-to-peer computing network

• Google App Engine’s Python shell

• Codecademy’s empythoned

• CheckIO.org’s online coding game

Building a sandbox

• Language-level sandboxing (pysandbox)

• OS-level sandboxing (PyPy’s sandbox)

Question: How do we execute arbitrary code?

How do we execute arbitrary code?

exec: compiles and evaluates statements

>>> exec "print 'Hello world'"Hello world

eval: compiles and evaluates expressions

>>> eval("1 + 2")3

class Sandbox(object): def execute(self, code_string): exec code_string

sandbox.py

from sandbox import Sandbox

s = Sandbox()

code = """ print "Hello world!" """

s.execute(code)

test_sandbox.py

$ python test_sandbox.pyHello world!


s = Sandbox()

code = """ print "Hello world!" """

s.execute(code)

What should we disallow?


• Resource exhaustion

• Information disclosure

• Running unexpected services

• Disabling/quitting/erroring out of the sandbox


s = Sandbox()

code = """ file("test.txt", "w").write("Kaboom!\\n")"""

s.execute(code)

>>> __builtins__.__dict__.keys()['bytearray', 'IndexError', 'all', 'help', 'vars', 'SyntaxError', 'unicode', 'UnicodeDecodeError', 'memoryview', 'isinstance', 'copyright', 'NameError', 'BytesWarning', 'dict', 'input', 'oct', 'bin', 'SystemExit', 'StandardError', 'format', 'repr', 'sorted', 'False', 'RuntimeWarning', 'list', 'iter', 'reload', 'Warning', '__package__', 'round', 'dir', 'cmp', 'set', 'bytes', 'reduce', 'intern', 'issubclass', 'Ellipsis', 'EOFError', 'locals', 'BufferError', 'slice', 'FloatingPointError', 'sum', 'getattr', 'abs', 'exit', 'print', 'True', 'FutureWarning', 'ImportWarning', 'None', 'hash', 'ReferenceError', 'len', 'credits', 'frozenset', '__name__', 'ord', 'super', '_', 'TypeError', 'license', 'KeyboardInterrupt', 'UserWarning', 'filter', 'range', 'staticmethod', 'SystemError', 'BaseException', 'pow', 'RuntimeError', 'float', 'MemoryError', 'StopIteration', 'globals', 'divmod', 'enumerate', 'apply', 'LookupError', 'open', 'quit', 'basestring', 'UnicodeError', 'zip', 'hex', 'long', 'next', 'ImportError', 'chr', 'xrange', 'type', '__doc__', 'Exception', 'tuple', 'UnicodeTranslateError', 'reversed', 'UnicodeEncodeError', 'IOError', 'hasattr', 'delattr', 'setattr', 'raw_input', 'SyntaxWarning', 'compile', 'ArithmeticError', 'str', 'property', 'GeneratorExit', 'int', '__import__', 'KeyError', 'coerce', 'PendingDeprecationWarning', 'file', 'EnvironmentError', 'unichr', 'id', 'OSError', 'DeprecationWarning', 'min', 'UnicodeWarning', 'execfile', 'any', 'complex', 'bool', 'ValueError', 'NotImplemented', 'map', 'buffer', 'max', 'object', 'TabError', 'callable', 'ZeroDivisionError', 'eval', '__debug__', 'IndentationError', 'AssertionError', 'classmethod', 'UnboundLocalError', 'NotImplementedError', 'AttributeError', 'OverflowError']

How do we disallow execution of

problematic builtins?

Idea: keyword blacklist

class Sandbox(object): def execute(self, code_string): keyword_blacklist = ["file", "open", "eval", "exec"] for keyword in keyword_blacklist: if keyword in code_string: raise ValueError("Blacklisted") exec code_string

Idea: keyword blacklist


s = Sandbox()


s.execute(code)

Testing: keyword blacklist


s = Sandbox()


s.execute(code)

$ python test_sandbox.pyTraceback (most recent call last): File "test_sandbox.py", line 11, in <module> s.execute(code) File "/Users/jesstess/Desktop/sandbox/sandbox.py", line 86, in execute raise ValueError("Blacklisted")ValueError: Blacklisted


How can we get around a keyword

blacklist?

Circumvention idea: encryption

func = __builtins__["file"]func("test.txt", "w").write("Kaboom!\n")

Circumvention idea: encryption

func = __builtins__["file"]func("test.txt", "w").write("Kaboom!\n")

func = __builtins__["svyr".decode("rot13")]func("test.txt", "w").write("Kaboom!\n")


s = Sandbox()

code = """ func = __builtins__["svyr".decode("rot13")] func("test.txt", "w").write("Kaboom!\\n")"""

s.execute(code)


Kaboom

Observation: if I can get a reference to something

bad, I can invoke it.

How can we remove all references to

problematic builtins?

Idea: builtins whitelist

builtins_whitelist = set(( # exceptions 'ArithmeticError', 'AssertionError', 'AttributeError', ... # constants 'False', 'None', 'True', ... # types 'basestring', 'bytearray', 'bytes', 'complex', 'dict', ... # functions '__import__', 'abs', 'all', 'any', 'apply', 'bin', 'bool', ... # block: eval, execfile, file, quit, exit, reload, etc.))

import sys

main = sys.modules["__main__"].__dict__orig_builtins = main["__builtins__"].__dict__builtins_whitelist = set(( ... ))

for builtin in orig_builtins.keys(): if builtin not in builtins_whitelist: del orig_builtins[builtin]


s = Sandbox()


s.execute(code)

Testing: builtins whitelist


s = Sandbox()


s.execute(code)

$ python test_sandbox.pyTraceback (most recent call last): File "test_sandbox.py", line 9, in <module> s.execute(code) ... File "<string>", line 2, in <module>NameError: name 'file' is not defined


Circumvention idea: import something

dangerous


s = Sandbox()

code = """ import os fd = os.open("test.txt", os.O_CREAT|os.O_WRONLY) os.write(fd, "Kaboom!\\n")"""

s.execute(code)


Kaboom

How do we disallow problematic imports?

Idea: import whitelist

>>> importer = __builtins__.__dict__.get("__import__")>>> os = importer("os")>>> os<module 'os' from '/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.pyc'>>>> os.getcwd()'/Users/jesstess/Desktop/sandbox'


How does importing a module work in Python?

>>> help(__builtins__.__dict__["__import__"])

__import__(...) __import__(name, globals={}, locals={}, fromlist=[], level=-1) -> module


What is the expected function signature for the importer?

>>> def my_importer(module_name, globals={}, ... locals={}, fromlist=[], ... level=-1):... print "Using my importer!"... return __import__(module_name, globals, ... locals, fromlist, level)... >>> os = my_importer("os")Using my importer!>>> os.getcwd()'/Users/jesstess/Desktop/sandbox'


Cool, let’s write our own importer

def _safe_import(__import__, module_whitelist): def safe_import(module_name, globals={}, locals={}, fromlist=[], level=-1):! if module_name in module_whitelist: return __import__(module_name,! ! ! ! ! ! ! ! ! ! ! ! globals, locals, fromlist, level) else: raise ImportError( "Blocked import of %s" ( module_name,)) return safe_import

import sys

main = sys.modules["__main__"].__dict__orig_builtins = main["__builtins__"].__dict__

for builtin in orig_builtins.keys(): if builtin not in builtins_whitelist: del original_builtins[builtin]

safe_modules = ["string", "re"]orig_builtins["__import__"] = _safe_import( __import__, safe_modules)


s = Sandbox()


s.execute(code)

Testing: import whitelist


s = Sandbox()


s.execute(code)

Testing: import whitelist

$ python test_sandbox.pyTraceback (most recent call last): File "test_sandbox.py", line 11, in <module> ... raise ImportError("Blocked import of %s" % (module_name,))ImportError: Blocked import of os

Circumvention idea: modifying builtins

Idea: make builtins read-only

How can we make an object read-only in

Python?

class ReadOnlyBuiltins(dict): def __delitem__(self, key): ValueError("Read-only!")

def pop(self, key, default=None): ValueError("Read-only!")

def popitem(self): ValueError("Read-only!") ... def setdefault(self, key, value): ValueError("Read-only!")

def __setitem__(self, key, value): ValueError("Read-only!")

def update(self, dict, **kw): ValueError("Read-only!")

main = sys.modules["__main__"].__dict__orig_builtins = main["__builtins__"].__dict__

for builtin in orig_builtins.keys(): if builtin not in builtins_whitelist: del original_builtins[builtin]

safe_modules = ["string", "re"]orig_builtins["__import__"] = _safe_import( __import__, safe_modules)

safe_builtins = ReadOnlyBuiltins( original_builtins)main["__builtins__"] = safe_builtins

Observation redux: if I can get a reference to something bad, I can

invoke it.

Circumvention idea: exploiting the

inheritance hierarchy

>>> dir([])['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__delslice__', '__doc__', '__eq__', '__format__', '__ge__', ...]>>> [].__class__<type 'list'>

What can we find out about an object’s base classes?

>>> [].__class__<type 'list'>>>> [].__class__.__bases__(<type 'object'>,)>>> [].__class__.__bases__[0]<type 'object'>

What can we find out about an object’s base classes?

list subclasses object

>>> [].__class__.__subclasses__()[]>>> int.__subclasses__()[<type 'bool'>]>>> basestring.__subclasses__()[<type 'str'>, <type 'unicode'>]

What can we find out about an object’s subclasses?

subclasses of basestring

>>> [].__class__.__bases__(<type 'object'>,)>>> [].__class__.__bases__[0]<type 'object'>

>>> [].__class__.__bases__[0].__subclasses__()[<type 'type'>, <type 'weakref'>, <type 'weakcallableproxy'>, <type 'weakproxy'>, <type 'int'>, <type 'basestring'>, <type 'bytearray'>, <type 'list'>, <type 'NoneType'>, <type 'NotImplementedType'>, <type 'traceback'>, <type 'super'>, <type 'xrange'>, <type 'dict'>, <type 'set'>, <type 'slice'>, <type 'staticmethod'>, <type 'complex'>, <type 'float'>, <type 'buffer'>, <type 'long'>, <type 'frozenset'>, <type 'property'>, <type 'memoryview'>, <type 'tuple'>, <type 'enumerate'>, <type 'reversed'>, <type 'code'>, <type 'frame'>, <type 'builtin_function_or_method'>, <type 'instancemethod'>, <type 'function'>, <type 'classobj'>, <type 'dictproxy'>, <type 'generator'>, <type 'getset_descriptor'>, <type 'wrapper_descriptor'>, <type 'instance'>, <type 'ellipsis'>, <type 'member_descriptor'>, <type 'file'>, <type 'PyCapsule'>, <type 'cell'>, <type 'callable-iterator'>, <type 'iterator'>, <type 'sys.long_info'>, <type 'sys.float_info'>, <type 'EncodingMap'>, <type 'fieldnameiterator'>, <type 'formatteriterator'>, <type 'sys.version_info'>, <type 'sys.flags'>, <type 'exceptions.BaseException'>, <type 'module'>, <type 'imp.NullImporter'>, <type 'zipimport.zipimporter'>, <type 'posix.stat_result'>, <type 'posix.statvfs_result'>, <class 'warnings.WarningMessage'>, <class 'warnings.catch_warnings'>, <class '_weakrefset._IterationGuard'>, <class '_weakrefset.WeakSet'>, <class '_abcoll.Hashable'>, <type 'classmethod'>, <class '_abcoll.Iterable'>, <class '_abcoll.Sized'>, <class '_abcoll.Container'>, <class '_abcoll.Callable'>, <class 'site._Printer'>, <class 'site._Helper'>, <type '_sre.SRE_Pattern'>, <type '_sre.SRE_Match'>, <type '_sre.SRE_Scanner'>, <class 'site.Quitter'>, <class 'codecs.IncrementalEncoder'>, <class 'codecs.IncrementalDecoder'>, <class 'string.Template'>, <class 'string.Formatter'>, <type 'operator.itemgetter'>, <type 'operator.attrgetter'>, <type 'operator.methodcaller'>, <type 'collections.deque'>, <type 'deque_iterator'>, <type 'deque_reverse_iterator'>, <type 'itertools.combinations'>, <type 'itertools.combinations_with_replacement'>, <type 'itertools.cycle'>, <type 'itertools.dropwhile'>, <type 'itertools.takewhile'>, <type 'itertools.islice'>, <type 'itertools.starmap'>, <type 'itertools.imap'>, <type 'itertools.chain'>, <type 'itertools.compress'>, <type 'itertools.ifilter'>, <type 'itertools.ifilterfalse'>, <type 'itertools.count'>, <type 'itertools.izip'>, <type 'itertools.izip_longest'>, <type 'itertools.permutations'>, <type 'itertools.product'>, <type 'itertools.repeat'>, <type 'itertools.groupby'>, <type 'itertools.tee_dataobject'>, <type 'itertools.tee'>, <type 'itertools._grouper'>, <type '_thread._localdummy'>, <type 'thread._local'>, <type 'thread.lock'>, <class 'sandbox.Protection'>, <type 'resource.struct_rusage'>, <class 'sandbox.config.SandboxConfig'>, <class 'sandbox.proxy.ReadOnlySequence'>, <class 'sandbox.sandbox_class.Sandbox'>, <class 'sandbox.restorable_dict.RestorableDict'>]

All of the subclasses of object!


>>> obj_class = [].__class__.__bases__[0]>>> for c in obj_class.__subclasses__():... print c.__name__...wrapper_descriptorinstanceellipsismember_descriptorfilePyCapsulecellcallable-iteratoriterator...


>>> obj_class = [].__class__.__bases__[0]>>> for c in obj_class.__subclasses__():... print c.__name__...wrapper_descriptorinstanceellipsismember_descriptorfilePyCapsulecellcallable-iteratoriterator...

!!!


s = Sandbox()

code = """ obj_class = [].__class__.__bases__[0]obj_subclasses = dict((elt.__name__, elt) for \ elt in obj_class.__subclasses__())func = obj_subclasses["file"]func("text.txt", "w").write("Kaboom!\\n")"""

s.execute(code)

Testing: read-only builtins

Kaboom

Idea: don’t expose dangerous

implementation details

>>> type.__bases__(<type 'object'>,)>>> del type.__bases__

Let’s delete __bases__ and __subclasses__

>>> type.__bases__(<type 'object'>,)>>> del type.__bases__Traceback (most recent call last): File "<stdin>", line 1, in <module>TypeError: can't set attributes of built-in/extension type 'type'


Imposed by the underlying C implementation!

from ctypes import pythonapi, POINTER, py_object

_get_dict = pythonapi._PyObject_GetDictPtr_get_dict.restype = POINTER(py_object)_get_dict.argtypes = [py_object]del pythonapi, POINTER, py_object

def dictionary_of(ob): dptr = _get_dict(ob) return dptr.contents.value

cpython.py


from cpython import dictionary_of

main = sys.modules["__main__"].__dict__

...

safe_builtins = ReadOnlyBuiltins( original_builtins)main["__builtins__"] = safe_builtins

type_dict = dictionary_of(type)del type_dict["__bases__"]del type_dict["__subclasses__"]

Circumvention idea: would a function by any

other name smell as sweet?

>>> def foo():... print "Meow"... >>> dir(foo)['__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__', '__doc__', '__format__', '__get__', '__getattribute__', '__globals__', '__hash__', '__init__', '__module__', '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'func_closure', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name']

>>> def foo():... print "Meow"... >>> dir(foo)['__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__', '__doc__', '__format__', '__get__', '__getattribute__', '__globals__', '__hash__', '__init__', '__module__', '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'func_closure', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name']

???

>>> foo.func_code<code object foo at 0x100509d30, file "<stdin>", line 1>>>> dir(foo.func_code)['__class__', '__cmp__', '__delattr__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'co_argcount', 'co_cellvars', 'co_code', 'co_consts', 'co_filename', 'co_firstlineno', 'co_flags', 'co_freevars', 'co_lnotab', 'co_name', 'co_names', 'co_nlocals', 'co_stacksize', 'co_varnames']>>> foo.func_code.co_code'd\x01\x00GHd\x00\x00S'

>>> def foo():... print "Meow"... >>> def evil_function():... print "Kaboom!"... >>> foo()Meow>>> foo.__setattr__("func_code", evil_function.func_code)>>> foo()Kaboom!

Kaboom

Idea redux: don’t expose dangerous

implementation details

from cpython import dictionary_offrom types import FunctionType

...

type_dict = dictionary_of(type)del type_dict["__bases__"]del type_dict["__subclasses__"]

function_dict = dictionary_of(FunctionType)del function_dict["func_code"]

Delete func_code

Whew. Let’s recap tactics:• Keyword blacklist

• Builtins whitelist

• Import whitelist

• Making important objects read-only (builtins)

• Deleting problematic implementation details (__bases__, __subclasses__, func_code)

• Deleting the ability to construct arbitrary code objects

We have run out of tricks!

We’ve implemented 80% of a full-fledged Python sandbox

builtins_whitelist = set(( # exceptions 'ArithmeticError', 'AssertionError', 'AttributeError', 'BufferError', 'BytesWarning', 'DeprecationWarning', 'EOFError', 'EnvironmentError', 'Exception', 'FloatingPointError','FutureWarning', 'GeneratorExit', 'IOError', 'ImportError', 'ImportWarning', 'IndentationError', 'IndexError', 'KeyError','LookupError', 'MemoryError', 'NameError', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError','PendingDeprecationWarning', 'ReferenceError', 'RuntimeError', 'RuntimeWarning', 'StandardError', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'TabError', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError','UnicodeError', 'UnicodeTranslateError', 'UnicodeWarning', # constants 'False', 'None', 'True', '__doc__', '__name__', '__package__', 'copyright', 'license', 'credits', # types 'basestring', 'bytearray', 'bytes', 'complex', 'dict', 'float', 'frozenset', 'int', 'list', 'long', 'object', 'set', 'str', 'tuple', 'unicode', # functions '__import__', 'abs', 'all', 'any', 'apply', 'bin', 'bool', 'buffer', 'callable', 'chr', 'classmethod', 'cmp', 'coerce', 'compile', 'delattr', 'dir', 'divmod', 'enumerate', 'filter', 'format', 'getattr', 'globals', 'hasattr', 'hash', 'hex', 'id', 'isinstance', 'issubclass', 'iter', 'len', 'locals', 'map', 'max', 'min', 'next', 'oct', 'ord', 'pow', 'print', 'property', 'range', 'reduce', 'repr', 'reversed', 'round', 'setattr', 'slice', 'sorted', 'staticmethod', 'sum', 'super', 'type', 'unichr', 'vars', 'xrange', 'zip', ))

def _safe_import(__import__, module_whitelist): def safe_import(module_name, globals={}, locals={}, fromlist=[], level=-1): if module_name in module_whitelist: return __import__(module_name, globals, locals, fromlist, level) else: raise ImportError("Blocked import of %s" % (module_name,)) return safe_import

class ReadOnlyBuiltins(dict): def clear(self): ValueError("Read-only!")

def __delitem__(self, key): ValueError("Read-only!")

def pop(self, key, default=None): ValueError("Read-only!")

def popitem(self): ValueError("Read-only!")

def setdefault(self, key, value):! ValueError("Read-only!")

def __setitem__(self, key, value): ValueError("Read-only!")

def update(self, dict, **kw): ValueError("Read-only!")

class Sandbox(object): def __init__(self):! import sys! from types import FunctionType! from cpython import dictionary_of

! original_builtins = sys.modules["__main__"].__dict__["__builtins__"].__dict__

! for builtin in original_builtins.keys(): if builtin not in builtins_whitelist:! ! del sys.modules["__main__"].__dict__["__builtins__"].__dict__[builtin]

original_builtins["__import__"] = _safe_import(__import__, ["string", "re"]) safe_builtins = ReadOnlyBuiltins(original_builtins) sys.modules["__main__"].__dict__["__builtins__"] = safe_builtins

! type_dict = dictionary_of(type)! del type_dict["__bases__"]! del type_dict["__subclasses__"]

! function_dict = dictionary_of(FunctionType)! del function_dict["func_code"]

def execute(self, code_string):! exec code_string

builtins whitelist

import whitelist

read-only builtins

deleting __bases__, __subclasses_, and func_code

Building a sandbox

• Language-level sandboxing (pysandbox)

• OS-level sandboxing (PyPy’s sandbox)


• Resource exhaustion

• Information disclosure

• Running unexpected services

• Disabling/quitting/erroring out of the sandbox

Food for thought

Is this level of reflectiveness good

or bad?

Do other languages have these sandboxing

concerns?

If you were designing a new language, how would you do this?

Experiments

• How does an alternative Python implementation like PyPy handle these issues?

• How does the CPython interpreter compile and run bytecode?

• What does the Python stack look like?

• How do ctypes work?

• How can the operating system help provide a secure environment?

Bedtime reading

• The full pysandbox implementation: https://github.com/haypo/pysandbox/

• A retrospective on pysandbox’s challenges: https://lwn.net/Articles/574215/

• PyPy’s sandbox implementation: http://pypy.readthedocs.org/en/latest/sandbox.html

• How PythonAnywhere’s sandbox works: http://blog.pythonanywhere.com/83/

https://github.com/haypo/pysandbox/

https://github.com/haypo/pysandbox/

https://lwn.net/Articles/574215/

https://lwn.net/Articles/574215/

http://pypy.readthedocs.org/en/latest/sandbox.html

http://pypy.readthedocs.org/en/latest/sandbox.html

http://blog.pythonanywhere.com/83/

http://blog.pythonanywhere.com/83/

Thank you!

Let’s talk!O’Reilly booth, 3pm

Thank you!

building and breaking a python sandboxweb.mit.edu/jesstess/www/pytennessee_sandbox.pdfexamples in...

Documents