![Page 1: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/1.jpg)
How to backup files in the middle of (almost) nowhere
Petr Pulc
Faculty of Information Technology,Czech Technical UniversityPrague, Czech republic
7. 10. 2017
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 1 / 36
![Page 2: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/2.jpg)
What you want:- portable device- automatic backup on media insertion- as cheap as possible
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 2 / 36
![Page 3: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/3.jpg)
Assumptions:- You have some parts lying around- You are willing to bodge a little
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 3 / 36
![Page 4: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/4.jpg)
Solution?- Single board PC- HDD (SSD if rich)- LEDs (segment display better)
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 4 / 36
![Page 5: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/5.jpg)
Single board PC:
- Versatile- Affordable- Small- GPIO!
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 5 / 36
![Page 6: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/6.jpg)
Banana Pi+ Dual Core+ SATA+ Able to be powered from battery
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 6 / 36
![Page 7: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/7.jpg)
Challenge 1 − Power
[BAT 3,6V] --> [BANANA]
? STEP-UP ?
[HDD 5V]
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 7 / 36
![Page 8: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/8.jpg)
Compromise 1 − USB Power
[5V SOURCE] --> [BANANA]
vvv
[HDD 5V]
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 8 / 36
![Page 9: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/9.jpg)
Challenge 2 − Minimise user interface
- Use what you have lying around- Show when ready- Sync on media insertion- Show progress without full display- Show amount of free space on HDD
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 9 / 36
![Page 10: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/10.jpg)
Solution 2.1 − 7 Segment display
__________|____11____|_ _| | | | Common anodes:|1| | | first digit - 12|0| |7| second digit - 9|_| |_| third digit - 8__________ fourth digit - 6|_____5____|_ _| | | || | | ||1| |4||_| |_|__________|_____2____| _
|3|
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 10 / 36
![Page 11: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/11.jpg)
- 12x GPIO required (up to 17 available on CONN3)- 4 to select digit (raise to 3,3V)- 8 to select segments (drop to 0)
- 8x 110 Ohm resistors- Piece of old IDE cable- Solder
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 11 / 36
![Page 12: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/12.jpg)
display .py
#!/usr/bin/env python3import time
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BOARD)
SEGMENTS = [18, 16, 12, 10, 3, 8, 5, 7]NUMBERS = [19, 15, 13, 11]
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 12 / 36
![Page 13: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/13.jpg)
CHARS = {’ ’: (1, 1, 1, 1, 1, 1, 1, 1),’0’: (0, 0, 0, 1, 0, 0, 0, 1),’1’: (1, 1, 0, 1, 1, 0, 1, 1),# ...’,’: (1, 1, 1, 1, 1, 0, 1, 1),’%’: (1, 0, 1, 1, 1, 0, 1, 1),’K’: (1, 0, 1, 0, 0, 1, 1, 1),’M’: (0, 1, 1, 1, 0, 0, 1, 1),’G’: (0, 0, 1, 1, 0, 0, 0, 1),’T’: (1, 0, 1, 0, 0, 1, 0, 1),’d’: (1, 1, 0, 0, 0, 0, 0, 1),’o’: (1, 1, 1, 0, 0, 0, 0, 1),’n’: (1, 1, 1, 0, 0, 0, 1, 1),’e’: (0, 0, 1, 0, 0, 1, 0, 1),# ...}
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 13 / 36
![Page 14: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/14.jpg)
def init():for p in SEGMENTS:
GPIO.setup(p, GPIO.OUT, initial=1)for p in NUMBERS:
GPIO.setup(p, GPIO.OUT, initial=0)
def out_zip(pins, vals):for p, v in zip(pins, vals):
GPIO.output(p, v)
def clear():for p in SEGMENTS:
GPIO.output(p, 1)for p in NUMBERS:
GPIO.output(p, 0)
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 14 / 36
![Page 15: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/15.jpg)
def show(text, duration=1):for _ in range(0, duration * 50):
number = 0for char in text[:4]:
out_zip(SEGMENTS,CHARS.get(char,
(1, 0, 0, 0, 0, 0, 0, 1)))
GPIO.output(NUMBERS[number], 1)time.sleep(0.004)GPIO.output(NUMBERS[number], 0)number += 1
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 15 / 36
![Page 16: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/16.jpg)
show.py
#!/usr/bin/env python3import displayimport sysdisplay.init()display.show(sys.argv[1].rjust(4), 2)
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 16 / 36
![Page 17: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/17.jpg)
Solution 2.2 − Show when ready − /etc/rc.local:
/usr/local/bin/gpio mode 2 out/usr/local/bin/gpio write 2 0# same for GPIO 3, 7, 12
/usr/local/bin/gpio mode 0 out/usr/local/bin/gpio write 0 1# same for GPIO 1, 4, 5, 8, 9, 15, 16
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 17 / 36
![Page 18: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/18.jpg)
Solution 2.3 − Sync on media insertion
/etc/udev/rules.d/99_sync.rules:
KERNEL=="sd?1", SUBSYSTEMS=="usb", ACTION=="add",RUN+="???"
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 18 / 36
![Page 19: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/19.jpg)
Challenge 2.3.1 − What should we run?
- Long scripts get killed- Multiple USB devices at once, one only once
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 19 / 36
![Page 20: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/20.jpg)
Solution 2.3.1.1 − Service
/etc/udev/rules.d/99_sync.rules:
KERNEL=="sd?1", SUBSYSTEMS=="usb", ACTION=="add",RUN+="/bin/systemctl --no-block start sync.service"
/etc/system.d/system/sync.service:
[Unit]Description=SD backuper
[Service]Type=simpleExecStart=/opt/mount_n_sync.sh
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 20 / 36
![Page 21: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/21.jpg)
Solution 2.3.1.2 − Pass device identifier
/etc/udev/rules.d/99_sync.rules:
KERNEL=="sd?1", SUBSYSTEMS=="usb", ACTION=="add",RUN+="/bin/systemctl --no-block start sync@%k.service"
/etc/system.d/system/[email protected]:
[Unit]Description=SD backuperBindsTo=dev-%i.device
[Service]Type=simpleExecStart=/opt/mount_n_sync.sh %I
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 21 / 36
![Page 22: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/22.jpg)
Challenge 2.3.2 − Mount and sync
- Sync media independently (different id on format)- Mount manually- (Show remaining space)- Copy only new files- Unmount and clean-up- (Show remaining space)
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 22 / 36
![Page 23: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/23.jpg)
Solution 2.3.2.[ˆ4]
/opt/mount_n_sync.sh:
# Get information on block deviceexport ‘blkid -o export /dev/$1‘
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 23 / 36
![Page 24: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/24.jpg)
# Mountmkdir "/media/$UUID"mount -o ro "$DEVNAME" "/media/$UUID"
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 24 / 36
![Page 25: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/25.jpg)
# Show free before backup/opt/show.py Free/opt/show_df.py
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 25 / 36
![Page 26: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/26.jpg)
# Do backup/opt/rsync.py $UUID
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 26 / 36
![Page 27: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/27.jpg)
# Unmount and clear mount pointumount "$DEVNAME"rmdir "/media/$UUID"
sync
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 27 / 36
![Page 28: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/28.jpg)
# Show free after backup/opt/show.py Free/opt/show_df.py
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 28 / 36
![Page 29: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/29.jpg)
show df.py
Almost as simple as df -h /mnt/backup...
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 29 / 36
![Page 30: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/30.jpg)
import subprocessimport display
with subprocess.Popen([’df’, ’-h’, ’/mnt/backup’],stdout=subprocess.PIPE) as p:
data = p.stdout.readlines()
display.init()display.show(str(data[1]).split()[3].rjust(4), 2)
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 30 / 36
![Page 31: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/31.jpg)
Challenge 2.3.2.4
The display.show IS BLOCKING
... for now it is helpfull, but:
We need to get progress DURING the sync!
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 31 / 36
![Page 32: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/32.jpg)
Solution 2.3.2.4
/opt/rsync.py:
import subprocessimport sysfrom threading import Threadimport display
display.init()
def print_thread():global textwhile True:
display.show(text, 1)if text == ’done’ or text == ’FAIL’:
display.show(text, 5)break
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 32 / 36
![Page 33: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/33.jpg)
def rsync_thread():global texttty = open(’/dev/tty1’, ’w’)
with subprocess.Popen([’rsync’, ’-a’,’--info=progress2,flist0’,’/media/’+sys.argv[1],’/mnt/backup/’],stdout=subprocess.PIPE,bufsize=1,universal_newlines=True) as p:
for line in p.stdout:tty.write(line)out = line.split()if len(out) > 2:
text = out[1].rjust(4)if p.returncode == 0:
text = ’done’else:
text = ’FAIL’
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 33 / 36
![Page 34: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/34.jpg)
text = ’init’thread1 = Thread(target=print_thread)thread2 = Thread(target=rsync_thread)thread1.start()thread2.start()thread2.join()thread1.join()
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 34 / 36
![Page 35: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/35.jpg)
DEMO
Incomplete source code can be found on Github:
https://github.com/petrpulc/segment-display
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 35 / 36
![Page 36: How to backup files in the middle of (almost) nowhere€¦ · How to backup les in the middle of (almost) nowhere Petr Pulc Faculty of Information Technology, Czech Technical University](https://reader034.vdocuments.us/reader034/viewer/2022050612/5fb33e1d9242572635463bc9/html5/thumbnails/36.jpg)
Thank you
_ ___ _ _ _ ___ __| | |_ _| \ | | | | \ \/ / ^| | | || \| | | | |\ / | \| |___ | || |\ | |_| |/ \ | \|_____|___|_| \_|\___//_/\_\ \ \
\ \_________ _ __ ______ \ |
/| | _ \ / \\ \ / / ___| \___ |// | | | |/ _ \\ V /\___ \ | |// | |_| / ___ \| | ___) | | |
// |____/_/ \_\_| |____/ / /// / /// ^ ^ / // / /\____^__ /\ | | | | _/\_ ^ | |
/ L_| || |_ _| /\ |__/ \| |__| || v || |\ _____~~~\ ____~~~\ _____~~~~~\ ______~~~~L____~~~~
Petr Pulc (FIT CTU) File backuper 7. 10. 2017 36 / 36