python to go
TRANSCRIPT
![Page 2: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/2.jpg)
About Myself• Coding python since 2006
• Speaker @ PyCon China 2013
• Architect @ zalora.com
• https://github.com/Wuvist
!
• Switching to Go last year
![Page 3: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/3.jpg)
“Switching”• Before:
• Love Python
• python for all projects
• After:
• Still love python
• python for hobby projects
• go for serious projects
![Page 4: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/4.jpg)
“Serious Projects”
• Performance is critical
• Code Quality is important
!
• OK, mostly web projects
![Page 5: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/5.jpg)
Performance
![Page 7: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/7.jpg)
Do you use pypy?
![Page 8: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/8.jpg)
“benchmark could be very misleading”
![Page 9: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/9.jpg)
Stress test on real Go API
![Page 10: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/10.jpg)
Requests/sec: 18071.20 Transfer/sec: 1.19GB
![Page 11: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/11.jpg)
• Just anything
• Can python?
• Can nginx?
![Page 12: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/12.jpg)
Why Go is so fast?
• Async
• Compiled
• Multi-cores
![Page 13: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/13.jpg)
Python
nginx (4 workers)
tornado tornadotornadotornado
![Page 14: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/14.jpg)
nginx <-> tornado
• Inter-process communication is slow
• network latency
• memory copy
• parsing
• Process overhead
![Page 15: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/15.jpg)
Python
nginx (4 workers)
tornado tornadotornadotornado
memcached
![Page 16: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/16.jpg)
Go
Go binary with GroupCache
https://github.com/golang/groupcache
![Page 17: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/17.jpg)
Performance
• Care abut performance? You must consider go
• http://blog.cloudflare.com/go-at-cloudflare
![Page 18: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/18.jpg)
Code Quality
![Page 19: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/19.jpg)
Error
• Python: scripting language
• deal with run-time errors
• Go: compiled language
• deal with compile-time errors
![Page 20: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/20.jpg)
Exception Handling
https://twitter.com/yinwang0/status/417899535344472065
![Page 21: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/21.jpg)
Python• All exception checked?
• Which one will crash the process?
!
• Dunno?
• Service running in “unknown state” is the worse thing could happen on production server!
![Page 22: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/22.jpg)
Exception in Go
• error
• Must check
• panic
• Crash
![Page 23: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/23.jpg)
Hacky
• Go doesn’t enforce error checking
• Go enforce all variable usage
![Page 24: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/24.jpg)
![Page 25: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/25.jpg)
➜ ~GOPATH git:(feature/rpccache) ✗ go run h.go # command-line-arguments ./h.go:8: err declared and not used
![Page 26: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/26.jpg)
![Page 27: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/27.jpg)
Test
• Python
• Are you actually using pyunit?
• Go
• go test
![Page 28: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/28.jpg)
https://github.com/Wuvist/gophpserialize
![Page 29: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/29.jpg)
https://github.com/Wuvist/gophpserialize/blob/master/gophpserialize_test.go
![Page 30: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/30.jpg)
➜ gophpserialize git:(master) go test -v === RUN TestUnmarshal --- PASS: TestUnmarshal (0.00 seconds) === RUN TestPhpToJson --- PASS: TestPhpToJson (0.00 seconds) === RUN TestPhpToJson2 --- PASS: TestPhpToJson2 (0.00 seconds) === RUN TestPhpToJsonComplex --- PASS: TestPhpToJsonComplex (0.00 seconds) PASS ok _/Users/wuvist/source/gophpserialize 0.021s
![Page 31: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/31.jpg)
Performance Tuning
• Python:
• from timeit import Timer ?!
• import cProfile ?
![Page 32: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/32.jpg)
Go pprof
• http://blog.golang.org/profiling-go-programs
![Page 33: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/33.jpg)
![Page 34: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/34.jpg)
![Page 35: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/35.jpg)
Productivity
![Page 36: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/36.jpg)
• go is just like python
• easy
• concise
• Yes, go needs more code than python
• Usually exception checking
• Don’t really need to trade productivity for performance & code quality
![Page 37: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/37.jpg)
Dependency Management
• Python
![Page 38: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/38.jpg)
• pip install MySQL-python
• may work, if you have the prerequisites
• prerequisites are system wide
• MySQL-python is system wide
• So, vitualenv?
![Page 39: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/39.jpg)
go
• go get = pip (kind of)
• gopath = per project virtualenv
!
• Both are built-in tools
![Page 40: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/40.jpg)
Deployment
• Deploy python(Django?) project
• WSGI, FastCGI, SCGI, or AJP?
• python version?
• dependencies?
![Page 41: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/41.jpg)
GO
• Copy & run the binary
![Page 42: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/42.jpg)
Concurrency
• Python
• twisted
• gevent
• not as node.js’s callback hell, but…
![Page 43: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/43.jpg)
Tornado
@gen.coroutine!def fetch_json(url):! response = yield AsyncHTTPClient().fetch(url)! raise gen.Return(json_decode(response.body))!
http://www.tornadoweb.org/en/stable/gen.html#tornado.gen.coroutine
![Page 44: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/44.jpg)
gevent
>>> from gevent import monkey; monkey.patch_socket()!>>> import urllib2 # it's usable from multiple greenlets now
http://www.gevent.org/intro.html#monkey-patching
![Page 45: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/45.jpg)
• How about MySQL-python/pylibmc?
• Do you really want monkey patch?
• Explicit is better than implicit. - Zen of python?
• Or wait for python 3.4?
• beta 3, 26 January 2014
![Page 46: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/46.jpg)
why not just go?
![Page 47: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/47.jpg)
Q & A
![Page 48: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/48.jpg)
Go tips
• Start with Go's official libs
• Look through their source code
• Consider use/create a framework
• http://beego.me
![Page 49: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/49.jpg)
cool projects
!
• GroupCache
• Docker
• raft
• beego
![Page 50: Python to go](https://reader034.vdocuments.us/reader034/viewer/2022052618/554a1110b4c90507558b4cdf/html5/thumbnails/50.jpg)
Gotcha• Template support is poor
• Coding with static Type could be tedious
• No generic type
• reflection is awful
• JSON (parsing) is slow! Much slow than python
• https://code.google.com/p/go/issues/detail?id=5683
• Only fields with capital letters are exported