Python Objects Mykola Kharechko, Developer

© Quintagroup, 2008

Table of contents

• Objects in memory• Objects at python level• Objects in python virtual machine

Objects in memory

• Object structure• Python memory management• Garbage collection

Object structure

Objects with same class

Python MemoryManagement

Allocation Strategy

For small requests, the allocator sub-allocates <Big> blocks of memory.Requests greater than 256 bytes are routed to the system's allocator.

Pymalloc structure

Arena256kb

Pool4kb

Fixed size block

Fixed size block

* Request in bytes Size of allocated block Size class idx* ------------------------------------------------------------------------------* 1-8 8 0* 9-16 16 1* 17-24 24 2* 25-32 32 3* 33-40 40 4* 41-48 48 5* 49-56 56 6* 57-64 64 7* 65-72 72 8* ... ... ...* 241-248 248 30* 249-256 256 31

Garbage Collection

Python performs garbage collection viareference counting and a cyclic garbagecollector that is able to detect and breakreference cycles.

Example>>> import gc>>> gc.disable() # 2.9mb>>> a = range(1000000) # 18mb>>> del a # 14mb>>> gc.enable() # 14mb>>> gc.collect() # 14mb0>>> gc.disable()>>> a = range(1000000) # 18mb>>> a.append(a) # 18mb>>> del a # 18mb>>> gc.enable() # 18mb>>> gc.collect() # 14 mb1>>>

Weak references

• weakref module• weakref.ref class• weakref.proxy class• weakref.WeakKeyDictionary

weakref example>>> from weakref import ref, proxy>>> class A(object):pass...>>> a = A()>>> ref_a = ref(a)>>> a.a = 16>>> ref_a().a16>>> proxy_a = proxy(a)>>> proxy_a.a16>>> proxy_a<weakproxy at 0x94e25f4 to A at 0x94e0fcc>>>> del a>>> ref_a()>>> ref_a().aTraceback (most recent call last):File "<stdin>", line 1, in <module>AttributeError: 'NoneType' object has no attribute 'a'>>> proxy_a<weakproxy at 0x94e23c4 to NoneType at 0x81479b8>

List object

typedef struct {PyObject_VAR_HEAD // ob_size/* Vector of pointers to list elements. list[0] is ob_item[0], etc. */PyObject **ob_item;// 0 <= ob_size <= allocated// len(list) == ob_sizePy_ssize_t allocated;} PyListObject;

List creation

PyObject *PyList_New(Py_ssize_t size){PyListObject *op;size_t nbytes;....nbytes = size * sizeof(PyObject *);.....op->ob_item = (PyObject **) PyMem_MALLOC(nbytes);memset(op->ob_item, 0, nbytes);...op->ob_size = size;op->allocated = size;...}

Integer objects

typedef struct {PyObject_HEADlong ob_ival;} PyIntObject;

Integeroptimization.

Problem>>> a = -2>>> b = -2>>> id(a) == id(b)True>>> a = 300>>> b = 300>>> id(a) == id(b)False

Integer optimization.Solution

#define NSMALLPOSINTS 257#define NSMALLNEGINTS 5/* References to small integers are saved in this arrayso that they can be shared.The integers that are saved are those in the range-NSMALLNEGINTS (inclusive) to NSMALLPOSINTS (not inclusive).*/static PyIntObject *small_ints[NSMALLNEGINTS + NSMALLPOSINTS];

Objects at pythonlevel

• Objects creation• Object attributes• Specific attributes

Objects creation

Constructor:obj = type->tp_new(type, args, kwds);type = Py_TYPE(obj);type->tp_init(obj, args, kwds);

Classes created by metaclasses

Object attributes

Attributes stored in __dict__ or in objectwhen __slots__ used

Attribute can be accessed via descriptor protocol

The most famous descriptor - property

Specific attributes

• __getattribute__ -access to all attrs• __setattr__ -opposite to __getattribute__• Descriptor protocol: __get__, __set__,

__delete__• Pickler protocol: __getstate__,

__setstate__, __reduce__

Objects in PVM

typedef struct {PyObject_HEADint co_argcount; /* #arguments, except *args */int co_nlocals; /* #local variables */int co_stacksize; /* #entries needed for evaluation stack */int co_flags;/* CO_..., see below */PyObject *co_code; /* instruction opcodes */PyObject *co_consts; /* list (constants used) */PyObject *co_names;/* list of strings (names used) */PyObject *co_varnames;/* tuple of strings (local variable names) */PyObject *co_freevars;/* tuple of strings (free variable names) */PyObject *co_cellvars; /* tuple of strings (cell variable names) *//* The rest doesn't count for hash/cmp */PyObject *co_filename;/* string (where it was loaded from) */PyObject *co_name;/* string (name, for reference) */int co_firstlineno;/* first source line number */PyObject *co_lnotab;/* string (encoding addr<->lineno mapping) */void *co_zombieframe; /* for optimization only (see frameobject.c) */} PyCodeObject;

CodeObject example

File test.py

a = 17b = a + 3print b

CodeObject Example

>>> import test34>>> pyc = open('test.pyc')>>> pyc.read(8)'\xb3\xf2\r\n>C\x8eH'>>> from marshal import load>>> co = load(pyc)>>> dir(co)[... 'co_argcount', 'co_cellvars', 'co_code', 'co_consts', 'co_filename', 'co_firstlineno', 'co_flags', 'co_freevars', 'co_lnotab', 'co_name', 'co_names', 'co_nlocals', 'co_stacksize', 'co_varnames']>>> co.co_consts(17, None)>>> co.co_names('a', 'b')

CodeObject example

>>> from dis import dis>>> dis(co)1 0 LOAD_CONST 0 (17)3 STORE_NAME 0 (a)

2 6 LOAD_NAME 0 (a)9 LOAD_CONST 0 (17)12 BINARY_ADD13 STORE_NAME 1 (b)

3 16 LOAD_NAME 1 (b)19 PRINT_ITEM20 PRINT_NEWLINE21 LOAD_CONST 1 (None)24 RETURN_VALUE