Introduction Scripting The JavaScript days Python Bindings Future Examples
Python usage in Samba and OpenChange
Jelmer Vernooij
Samba OpenChange
PUN December 7, 2008
Introduction Scripting The JavaScript days Python Bindings Future Examples
Agenda
1 Introduction
2 Scripting
3 The JavaScript days
4 Python
5 Bindings
6 Future
7 Examples
Introduction Scripting The JavaScript days Python Bindings Future Examples
What is Samba ?
Free (GPLv3) implementation of the SMB protocol andothers
DCE/RPC, NetBIOS, LDAP, CLDAP, Kerberos, . . .
“Network Neighborhood” (and more) for POSIXTraditionally written in CExtremely portableAbout the same age as Python (’91)Developed by a team of 25 peopleOriginally developed thru network analysis
Introduction Scripting The JavaScript days Python Bindings Future Examples
What is Samba ? - Branches
Samba 3Proven, stable, codebase
Samba 4Our very own “Duke Nukem Forever”Originally started in 2003, as an effort to improve the fileserverStrong focus on the right infrastructure
Introduction Scripting The JavaScript days Python Bindings Future Examples
What is OpenChange ?
Free implementation of the MAPI protocolAs used by Microsoft Outlook/ExchangeWorks on top of the DCE/RPC protocol
Being build on top of Samba 4French project, started in 2003Evolution OpenChange plugin will ship with GNOME 2.26
Introduction Scripting The JavaScript days Python Bindings Future Examples
Who am I?
CS StudentFOSS developer, working on:
Samba, mostly Samba 4OpenChangeBazaar
Introduction Scripting The JavaScript days Python Bindings Future Examples
Samba and scripting
We are all C programmersScripting: awk, shell or perl
Samba had Python bindings for a whileRemoved after several years because they wereunmaintained
Introduction Scripting The JavaScript days Python Bindings Future Examples
Why scripting?
Within SambaQuicker developmentEasier to debugEasier to understand
Lower barrier for contributions?
For usersEasier to customizeEasier to use (administrator scripts, etc)
Introduction Scripting The JavaScript days Python Bindings Future Examples
And the winner is...
Original choice: JavaScriptSmall, fast engine (small enough to include with Samba)Familiar to a lot of developers out thereA lot like C (familiar to developers)
Introduction Scripting The JavaScript days Python Bindings Future Examples
JavaScript: Problems
A lot like CSucks as a scripting language
No exceptionsPoor string manipulation functionsNo keyword arguments
No bindings for standard librariesThe library we were using was different from the standardlibrary
Contributors had to learn yet another language (dialect)
No development toolsHard to write unit tests
Introduction Scripting The JavaScript days Python Bindings Future Examples
Why Python?
Comes “with batteries included”No need to reimplement utility functions and bindings forSamba
Easy to create bindingsMost existing libraries already have Python bindings
GTK+, Qt, HTTP, .ini-parsers...Large existing developer base
Potential contributorsBetter scripting language
Nested functionsModularity
More development tools availableDebugger, profiler, code coverage analyser, ...
Introduction Scripting The JavaScript days Python Bindings Future Examples
Why not something else?
Several developers already knew (and liked) PythonAlternatives
Perl: Hard to use C API, silly syntaxRuby: Not well known enoughLua: Not really well suited for application development, justsmall snippetsScheme: well, . . .
Introduction Scripting The JavaScript days Python Bindings Future Examples
Concerns
Our first ever mandatory build-dependency other thanlibc. . .A lot of code to migrate, requiring effort that could beuseful elsewhereMaintainability
Unit tests mandatoryShould be used by core code
Introduction Scripting The JavaScript days Python Bindings Future Examples
SWIG
Doesn’t require a lot of effort to generate simple bindingsCan generate bindings for multiple languages at the sametimeGenerates portable code
However...
Tends to create very C-like Python bindingsCustomization language is hard to graspUnreadable generated C code
Introduction Scripting The JavaScript days Python Bindings Future Examples
SWIG
Doesn’t require a lot of effort to generate simple bindingsCan generate bindings for multiple languages at the sametimeGenerates portable code
However...
Tends to create very C-like Python bindings
Customization language is hard to graspUnreadable generated C code
Introduction Scripting The JavaScript days Python Bindings Future Examples
SWIG
Doesn’t require a lot of effort to generate simple bindingsCan generate bindings for multiple languages at the sametimeGenerates portable code
However...
Tends to create very C-like Python bindingsCustomization language is hard to grasp
Unreadable generated C code
Introduction Scripting The JavaScript days Python Bindings Future Examples
SWIG
Doesn’t require a lot of effort to generate simple bindingsCan generate bindings for multiple languages at the sametimeGenerates portable code
However...
Tends to create very C-like Python bindingsCustomization language is hard to graspUnreadable generated C code
Introduction Scripting The JavaScript days Python Bindings Future Examples
Pyrex/Cython
Very Python-like
But...
Needs to be run on the developer machine (extrabuild-dependency)Doesn’t support certainUnreadable generated C code
Introduction Scripting The JavaScript days Python Bindings Future Examples
Pyrex/Cython
Very Python-like
But...
Needs to be run on the developer machine (extrabuild-dependency)
Doesn’t support certainUnreadable generated C code
Introduction Scripting The JavaScript days Python Bindings Future Examples
Pyrex/Cython
Very Python-like
But...
Needs to be run on the developer machine (extrabuild-dependency)Doesn’t support certain
Unreadable generated C code
Introduction Scripting The JavaScript days Python Bindings Future Examples
Pyrex/Cython
Very Python-like
But...
Needs to be run on the developer machine (extrabuild-dependency)Doesn’t support certainUnreadable generated C code
Introduction Scripting The JavaScript days Python Bindings Future Examples
Manual bindings
The Python C API really isn’t that badAllows close integration between our memory managerand PythonsMuch more flexible than autogenerated Python bindings
Introduction Scripting The JavaScript days Python Bindings Future Examples
Partially generated from IDL
IDL CodeNTSTATUS unixinfo GetPWUid ([in,out,ref,range(0,1023)] uint32 *count,[in,size is(*count)] hyper uids[],[out,size is(*count)] unixinfo GetPWUidInfo infos[*]);
Python APIS.GetPWUid(uids) − > infos
Introduction Scripting The JavaScript days Python Bindings Future Examples
Current state of affairs
Mostly used for administrative tools:provisioning the databases after installationweb service? (wsgi compatible)Server functionality and performance-dependentSome nifty GUI tools based on GTK+
Popular for writing testsPerformance-dependent code is still all in C
And most importantly:Developers seem reasonably happy
Introduction Scripting The JavaScript days Python Bindings Future Examples
More crazy Samba Python hacks
Full Python coverage of our librariesMore GNOME integration in Pythonwin32com on Linux?Port to Samba 3?Server partially in Python?
Introduction Scripting The JavaScript days Python Bindings Future Examples
OpenChange
Provisioning already uses PythonMost client tools will be in PythonBindings still to be done
Introduction Scripting The JavaScript days Python Bindings Future Examples
Remaining and new concerns
No good standard mechanism for asynchronous functions(yet?)Some users are running Python older than 2.4Python3000 will drop support for some of our platformsSupporting all combinations of platforms with Pythoninstalled turned out to be quite a challenge
Introduction Scripting The JavaScript days Python Bindings Future Examples
Reading TDB files
1 import tdb , sys23 db = tdb . Tdb ( sys . argv [ 1 ] )4 for ( k , v ) in db . i tems ( ) :5 pr in t ” { ”6 pr in t ” key(%d ) = %r ” % ( len ( k ) , k )7 pr in t ” data(%d ) = %r ” % ( len ( v ) , v )8 pr in t ” } ”
Introduction Scripting The JavaScript days Python Bindings Future Examples
Using LDB
1 # ! / usr / b in / python23 import l db45 conn = ldb . Ldb ( ”msg . tdb ” )67 conn . add ({ ” dn ” : ” dc=samba , dc=org ” , ” a t t r 1 ” : ” foo ” } )89 for msg in conn . search ( ” dc=samba , dc=org ” ) :
10 pr in t s t r (msg . dn )
Introduction Scripting The JavaScript days Python Bindings Future Examples
Connecting to LDAP using LDB
1 # ! / usr / b in / python23 import l db45 # Connect to the LDAP server6 conn = ldb . Ldb ( ” ldap : / / ldap . abmas . org / ” )78 for msg in conn . search ( ” dc=samba , dc=org ” ) :9 pr in t s t r (msg . dn )
Introduction Scripting The JavaScript days Python Bindings Future Examples
Adding users
1 # ! / usr / b in / python2 import samr , l sa34 # Connect to the l o c a l SAM5 conn = samr . samr ( ” nca l rpc : ” , ” s t / dc / e tc / smb . conf ” )67 # Get SAMR connect handle8 samr handle = conn . Connect (0 , 0 x f f f f f f f )9
10 domainname = lsa . S t r i n g ( )11 domainname . s t r i n g = u ”SAMBADOMAIN”1213 s id = conn . LookupDomain ( samr handle , domainname )14 pr in t ” Found s id %s f o r SAMBADOMAIN” % s id1516 conn . Close ( samr handle )
Introduction Scripting The JavaScript days Python Bindings Future Examples
Unit tests
1 import winreg2 from samba . t e s t s import RpcInterfaceTestCase34 class WinregTests ( RpcInterfaceTestCase ) :5 def setUp ( s e l f ) :6 s e l f . conn = winreg . winreg ( ” nca l rpc : ” , s e l f . get loadparm ( ) , s e l f . g e t c r e d e n t i a l s ( ) )78 def t es t hk lm ( s e l f ) :9 handle = s e l f . conn .OpenHKLM(None ,
10 winreg .KEY QUERY VALUE | winreg .KEY ENUMERATE SUB KEYS)11 s e l f . conn . CloseKey ( handle )
Introduction Scripting The JavaScript days Python Bindings Future Examples
More information
http://www.samba.org/
http://www.openchange.org/
IRC: #samba-technical / #openchange on Freenode
If you have ideas about asynchronous function usage, pleaselet me know.