msrks’s dev notes documentation - read the docs · 13 raspberry pi 31 14 read the docs 33 i. ii....

Msrks’s Dev Notes DocumentationRelease 1.0

Masahiro Rikiso

May 10, 2017

Scientific Python

1 Deep Learning 1

2 Mosaik 3

3 Python 7

4 Docker 9

5 Heroku 13

6 Vagrant 15

7 Bootstrap 17

8 D3JS 21

9 JQuery 23

10 Atom 25

11 Linux 27

12 Local Server 29

13 Raspberry Pi 31

14 Read the Docs 33

i

CHAPTER 1

Deep Learning

• Deep Learning

• Personal Page

1

https://msrks.github.io/deeplearning/

http://msrks.github.io

Msrks’s Dev Notes Documentation, Release 1.0

2 Chapter 1. Deep Learning

CHAPTER 2

Mosaik

mosaik is a simulator manager!

resource

• https://mosaik.offis.de

• http://mosaik.readthedocs.io/en/latest/index.html

installation

mosaik:

$ cd mosaik$ pyenv virtualenv 3.5.0 mosaik35$ pyenv local mosaik35$ pip install mosaik

mosaik-demo:

$ brew install hg gcc hdf5$ pip install numpy scipy h5py$ hg clone https://bitbucket.org/mosaik/mosaik-demo$ cd mosaik-demo$ pip install -r requirement.txt$ python demo.py

Concept

4 main component

3

https://mosaik.offis.de

http://mosaik.readthedocs.io/en/latest/index.html


• mosaik Sim API(communicate between simulators and mosaik)

• Scenario API

• Simulator Manager(handle simulator processes)

• simulator(schedule)

4 module

• mosaik-core

• mosaik-componets(example simulator)

• mosaik-tools(analysis tools)

• Odysseus(big-data-analysis)

How to use

1. create simulation wrapper

1. define Meta Data:

META = {'models': {

'ExampleModel': {'public': True,'params': ['init_val'],'attrs': ['delta', 'val'],

},},

}

2. define Simulator Wrapper:

class ExampleSim(mosaik_api.Simulator):def __init__(self):

super().__init__(META)self.simulator = simulator.Simulator()self.eid_prefix = 'Model_'self.entities = {} # Maps EIDs to model

# additional simulator initializaitondef init(self, sid):

return self.meta

# model instance initializaitondef create(self, num, model, init_val):

entities = []## model creation#entities.append({'eid': eid, 'type': model})return entities

def step(self, time, inputs):## peform simulation step

4 Chapter 2. Mosaik


#return nexttime

def get_data(self, query):## construct returndata#return data

Note: simulator wrapper is subclass of mosaiki_api.Simulator

3. making it executable:

def main():return mosaik_api.start_simulation(ExampleSim())

if __name__ == '__main__':main()

2.4. How to use 5


6 Chapter 2. Mosaik

CHAPTER 3

Python

Python

pyenv

upgrade:

$ brew update$ brew upgrade pyenv

os:

import osfor f in os.listdir('txt/'):# do something

import osif not os.path.exists(FOLDER):

os.makedirs(FOLDER)

pandas

thousands:

df = pd.read_csv('txt/'+f, sep='\t',names=['delay','count'],thousands=',',dtype={'delay':str, 'count':int})

7


save csv:

df.to-csv("filename", index=False)

matplotlib, seaborn

rc:

matplotlib.rc('font', size=12)matplotlib.rc('axes', labelsize=16, titlesize=16)matplotlib.rc('xtick', labelsize=16)matplotlib.rc('ytick', labelsize=16)plt.rc('text', usetex=True)#sns.set(font_scale=1.4)

savefig:

fig.savefig("hoge.png")

8 Chapter 3. Python

CHAPTER 4

Docker

Docker

:

$ docker images$ docker run --name NAME -i -t (-d)\ # -d:

-v HOST-DIR:CONTAINER-DIR \-p HOST-PORT:CONTAINER-PORT \IMAGE /bin/bash

$ docker rmi IMAGE$ docker ps (-a)$ docker start CONTAINER$ docker stop CONTAINER$ docker attach CONTAINER #$ docker rm CONTAINER

Build:

$ mkdir mydockerbuild$ cd mydockerbuild$ touch Dockerfile#https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/$ atom Dockerfile

.. FROM docker/whalesay:latest

.. MAINTAINER Masahiro Rikiso <[email protected]>

.. RUN apt-get -y update && apt-get install -y fortunes

.. ADD ..

.. ENV ..

.. EXPOSE 80

.. CMD /usr/games/fortune -a | cowsay$ docker build -t <image-name> .

Commit:

9


$ docker commit <container-id> <image-name>

Tag & Push:

$ docker tag <image-id> msrks/<image-name>$ docker login --username=msrks [email protected]$ docker push msrks/<image-name>

:

$ docker save IMAGE > i-name.tar$ docker load < i-name.tar$ docker export CONTAINER > c-name.tar

:

$ docker cp CONTAINER:filename ./

:

$ docker diff CONTAINER

gitbucket

$ docker run --name gitbucket-8001 -d -p 8001:8080 takezoe/gitbucket

tensorflow:

$ docker run --name jupyter-8002 -d -p 8002:8888 gcr.io/tensorflow/tensorflow

caffe:

$ docker run --name jupyter-8003 -d -t -p 8003:8888 msrks/caffe-demo \/usr/local/bin/jupyter notebook --notebook-dir=/opt/caffe/examples/ --ip=0.0.0.0

caffe-webdemo:

$ docker run --name webdemo-8004 -d -t -p 8004:5000 msrks/caffe-demo \/usr/bin/python /opt/caffe/examples/web_demo/app.py

sphinx-latexpdf:

$ sudo docker run --name "sphinx" -v=/HOST/doc:/docs \-it higebu/sphinx-latexpdf /bin/bash(docker) # pip install -U sphinx_rtd_theme --proxy=158.202.41.115:8080(docker) # cd docs(docker) # make latexpdfja

10 Chapter 4. Docker

https://github.com/takezoe/gitbucket-docker


:

$ docker ps -a

:

$ docker start <container-id>

4.2. 11


12 Chapter 4. Docker

CHAPTER 5

Heroku

resource:

• https://devcenter.heroku.com/articles/getting-started-with-python#introduction

• https://github.com/heroku/python-getting-started

• http://kennmyers.github.io/tutorial/2016/03/11/getting-flask-on-heroku.html

Basic

installation

1. create account on heroku

2. download and instal heroku toolbelt

3. heroku login

deploy python app:

$ mkdir heroku-app$ cd heroku-app$ pyenv virtualenv 2.7.10 venv$ pyenv local venv$ echo .python-version > .gitignore$ git init

$ pip install gunicorn$ pip freeze > requirements.txt$ echo python-2.7.10 > runtime.txt$ echo web: gunicorn app:app --log-file=- > Procfile

#$ heroku local web

$ heroku create <app-name>

13

https://devcenter.heroku.com/articles/getting-started-with-python#introduction

https://github.com/heroku/python-getting-started

http://kennmyers.github.io/tutorial/2016/03/11/getting-flask-on-heroku.html


$ heroku buildpacks:set heroku/python

$ git add -A$ git commit -m "initial commit"$ git push heroku master$ heroku ps:scale web=1$ heroku open

log:

$ heroku logs --tail

14 Chapter 5. Heroku

CHAPTER 6

Vagrant

resource:

https://www.vagrantup.com

boxes:

https://atlas.hashicorp.com/boxes/search

vagrantfile

example:

# -*- mode: ruby -*-# vi: set ft=ruby :

Vagrant.configure("2") do |config|config.vm.box = "ubuntu/trusty64"config.vm.network "forwarded_port", guest: 80, host: 8080config.vm.network "private_network", ip: "192.168.33.10"# config.vm.network "public_network"# config.vm.synced_folder "../data", "/vagrant_data"

# config.vm.provider "virtualbox" do |vb|# # Display the VirtualBox GUI when booting the machine# vb.gui = true## # Customize the amount of memory on the VM:# vb.memory = "1024"# end

# config.vm.provision "shell", inline: <<-SHELL# apt-get update# apt-get install -y apache2

15

https://www.vagrantup.com

https://atlas.hashicorp.com/boxes/search


# SHELLend

16 Chapter 6. Vagrant

CHAPTER 7

Bootstrap

<!doctype html><html lang="ja"><head><meta charset="utf-8"><title>test</title><meta name="viewport" content="width=device-width, initial-scale=1.0"><link href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/css/bootstrap.min.css"

→˓rel="stylesheet" media="screen"><script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js"></

→˓script><script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/js/bootstrap.min.js"></

→˓script><style>body {

font-family: Verdana, Arial, sans-serif;-webkit-font-smoothing: antialiased;

}h1,h2,h3 {

font-family: Times, serif;line-height: 1.5em;border-bottom: 1px solid #ccc;

}</style>

</head><body></body></html>

17

body

• container

• col-md-X (row): grid system

• header navbar

:

<div class="container"><div class="row"><nav class="navbar navbar-default navbar-fixed-top"><div class="navbar-header"><a href="" class="navbar-brand">Home</a>

</div><ul class="nav navbar-nav"><li class="active"><a href="">menu1</a></li><li><a href="">menu2</a></li><li><a href="">menu3</a></li><li><a href="">menu4</a></li>

</ul></nav>

</div><div class="row" style="padding:60px 0 0 0"><div class="col-md-3" align="center">left</div><div class="col-md-9" align="center">center</div>

</div><hr><p>© Masahiro Rikiso</p>

</div>

table:

<table class="table table-striped table-boarderd table-hover"><thead><tr><th>ID</th><th>Score</th></tr>

</thead><tbody><tr><tb>python</tb><tb>100</tb></tr>

</tbody></table>

Note: class=”table table-striped table-boarderd table-hover”

form:

<form><div class="form-group">

18 Chapter 7. Bootstrap


<label class="control-label" for="email">Email</label><input type="text" id="email" class="form-control" placeholder="email">

</div><div class="form-group"><input type="submit" value="submit" class="btn btn-primary">

</div></form>

Note: class=”form-group” input text “form-control” input submit “btn btn-primary”

7.3. 19


20 Chapter 7. Bootstrap

CHAPTER 8

D3JS

Setting

d3js :

<script src="http://d3js.org/d3.v3.min.js" charset="utf-8"></script>

Tips

DOM:

select, selectAll

:

text, attr, style, append, remove

:

data, enter, exit

:

<body><p>1</p><p>2</p>

<script>var dataset = [1,2,3]var p = d3.select("body").selectAll("p")var update = p.data(dataset)var enter = update.enter().append(p)//var exit= update.exit()

21

http://ja.d3js.node.ws


</script></body>

svg:

<body><script>var dataset = [1,2,3,4,5]var svg = d3.select("body").append("svg")

.attr("width", 500)

.attr("height", 200);</script></body>

animation:

transition, delay, duration, ease,each("start",function(){}), each("end",function(){})

event:

on("mouseover", function(){})on("mouseout", function(){})on("mousedown", function(){})on("mouseup", function(){})on("click", function(){})

22 Chapter 8. D3JS

CHAPTER 9

JQuery

id=classifyfiledisabled=trueid=imagefileformsubmit:

<head><script type="text/javascript">$(document).ready(function(){$('#classifyfile').attr('disabled',true);$('#imagefile').change(

function(){if ($(this).val()){

$('#formupload').submit();}

});

});

</script></head><body><form id="formupload" class="form-inline" role="form" action="classify_upload"

→˓method="post" enctype="multipart/form-data"><div class="form-group"><label for="imagefile">Or upload an image:</label><input type="file" name="imagefile" id="imagefile">

</div></form>

</body>

23


24 Chapter 9. JQuery

CHAPTER 10

Atom

Proxy

Linux

@160506:

$ apm config set http-proxy http://<ip>:<port>$ apm config set https-proxy https://<ip>:<port>$ apm config set strict-ssl false$ export ATOM_NODE_URL=http://gh-contractor-zcbenz.s3.amazonaws.com/atom-shell/dist

Windows

@160623:

# 1) $HOME/.atom/.apmrchttp-proxy=http://<ip>:<port>https-proxy=https://<ip>:<port>strict-ssl=false# 2)$ setx ATOM_NODE_URL http://gh-contractor-zcbenz.s3.amazonaws.com/atom-shell/dist /M# 3)$ apm config set http-proxy http://<ip>:<port>$ apm config set https-proxy http://<ip>:<port>$ apm config set strict-ssl false

25


26 Chapter 10. Atom

CHAPTER 11

Linux

ip:

$ sudo ifconfig eth0 inet 10.0.0.100 netmask 255.0.0.0 up$ sudo route add default gw 10.0.0.1

SNMP server

netsnmp:

$ sudo apt-get install snmp sniped$ snmpwalk -v 2c -c public <ip-addr> <mib-dir>

DHCP server

isc-dhcp-server:

$ sudo apt-get install isc-dhcp-server$ sudo leafpad /etc/dhcp/dhcp.conf

# dhcp-relaysubnet 172.16.0.0 netmask 255.255.255.0 {}subnet 192.168.0.0 netmask 255.255.255.0 {

range 192.168.0.2 192.168.0.250;option routers 192.168.0.1;option subnet-mask 255.255.255.0# dns setting##

27


}$ sudo leafpad /etc/default/isc-dhcp-server

# INTERFACE = "eth0"$ sudo touch /var/lib/dhcp/dhcpd.leases$ sudo systemctl stop isc-dhcp-server.service$ sudo systemctl start isc-dhcp-server.service$ sudo systemctl enable isc-dhcp-server.service

Packet Capture

wireshark:

$ sudo apt-get install wireshark$ sudo groupadd wireshark$ sudo usermod -a -G wireshark pi$ sudo chgrp wireshark /usr/bin/dumpcap$ sudo chmod 750 /usr/bin/dumpcap$ sudo raspi-config$ sudo setcap cap_net_raw.cap_net_admin=eip /usr/bin/dumpcap$ sudo chmod 4711 'which dumpcap'$ sudo chmod 4711 /usr/bin/dumpcap

28 Chapter 11. Linux

CHAPTER 12

Local Server

python2(port=8000):

$ python -m SimpleHTTPServer

python3(port=8000):

$ python -m http.server

python2:

$ wget https://gist.github.com/msrks/18125b9a57e69e0309c8f743c87904d6/raw/→˓ff5a398ce7d828137869f91796446300b64c9941/server2.py$ python server2.py USER:PASS

python3:

$ wget https://gist.github.com/msrks/3fe0e3287f548382f442a9c79d5f5ec9/raw/→˓3416832d494feec0dda1b05b0190986f59878153/server3.py$ python server3.py USER:PASS

29


30 Chapter 12. Local Server

CHAPTER 13

Raspberry Pi

OS Installation @ Mac OS X

@160501

1. RPI official RASPBIAN JESSIE()

2. zip:

$ cd Downloads$ tar zxvf 2016-05-27-raspbian-jessie.zip

3. SD(Mac book /dev/disk3 ):

$ diskutil list

4. SD:

$ sudo diskutil eraseDisk FAT32 RPI /dev/disk3Started erase on disk3Unmounting diskCreating the partition map...Finished erase on disk3

5. SD:

$ sudo diskutil unmountDisk /dev/disk3Unmount of all volumes on disk3 was successful

6. SD:

$ sudo dd bs=1m if=2016-05-27-raspbian-jessie.img of=/dev/disk33125+0 records in

31

https://www.raspberrypi.org/downloads/raspbian/


3125+0 records out3276800000 bytes transferred in 2784.822319 secs (1176664 bytes/sec)

7. RPISD:

pi@raspberrypi ~ $ sudo raspi-config

ip

/etc/network/interfaces dhcp‘‘/etc/dhcpcd.conf‘‘ :

$ sudo leafpad /etc/dhcpcd.conf$ sudo /etc/init.d/dhcpcd reload

32 Chapter 13. Raspberry Pi

CHAPTER 14

Read the Docs

githubrepoclone:

$ git clone <github.repo>$ cd <project>

build-dir:

$ touch .gitignore$ echo docs/_build > .gitignore

doc:

$ mkdir docs$ cd docs$ sphinx-quickstart

conf.py:

# defaulthtml_theme = "default"

githubpush:

$ cd ..$ git add -A$ git commit -m "[add] docs"$ git push

Read the Docs <project>importbuild

33

https://readthedocs.org

msrks’s dev notes documentation - read the docs · 13 raspberry pi 31 14 read the docs 33 i. ii....

Documents