welcome to linux & shell scripting small group how to learn how to code workshop small-group

1

Welcome to Linux & Shell ScriptingSmall Group

How to learn how to Code Workshop

http://onish.web.unc.edu/how-to-learn-how-to-code-linux-small-group

/

Erin Osborne Nishimura

2

Pop quiz!

3

Why should biologists learn to code?

• Power

• Efficiency

• Reproducibility

4

What are the barriers to learn to code?

• Where to begin?

• This feels awkward

• Sporadic use

• Hitting a wall/bug

• Finding help

• Cultural barriers

5

Where to begin?computing environments and languages

• Linux – an open-source operating system.

• bash – A command-writing language. The “language” of linux. Good for tasks, automated workflows, and general management.

• Python – A general purpose, high-level programming language. Highly readable and writable.• Perl – A general purpose, high-level programming language. Great with text files.• Javascript – a general purpose, high-level programming language. Specialized for web

applications, apps, commonly used in plugins (ImageJ).• Ruby -- A very high-level programming language.

• R – A high level programming language and software environment specialized for statistics and large data management.

• MATLAB – A computing environment and programming language. Good integration for image analysis. Costs money.

• MYSQL – Database organization

• Others?

OS

PR

OG

RA

MM

ING

LA

NG

UA

GE

SM

AT

H-y

LA

NG

UA

GE

SD

B

6

A HANDS-ON INTRO TO KILLDEVIL

How can I use some of these resources?

7

Killdevil is a high-performance computing environment

• Linux operating system• 1 login node• 774 compute nodes

– 48 – 96 GB memory per node.– 12 – 16 CPU’s cores per node.

• 2 large memory nodes (1 TB)• 12 Graphics Processors (GPUs)

nodes• File systems for storage

8

No seriously, what is Killdevil?

9

Getting onto Killdevil

• MAC OS & Linux machines:– Link to killdevil through

“Terminal”– Open “Terminal” (in

Applications -> Utilities)– Type this:ssh <yourOnyen>@killdevil.unc.edu

-- Add password when prompted

• PC– Open SSH Secure Shell

Client– Click on “Quick Connect”– Hostname =

killdevil.unc.edu– Username = <yourOnyen>– Port Number = 22– Add password when

prompted$ ssh erinosb@killdevil.unc.edu

10

WHAT IS LINUX?WHAT IS BASH?

11

Linux is a lifestyle

• Linux is an operating system• born out of UNIX• Linux comes in many flavors• Linux is the most prominent example of

the free, open-source movement• Linux embodies a philosophy of respect

for its users

12

bash is the shell for most linux distros

13

NAVIGATING THE LINUX ENVIRONMENT

14

Getting oriented

• Commands• Manuals

• Your first two commands:– whoami– Date

• Learn more about built-in utilities with man

$ whoamierinosb

$ dateThu Apr 9 13:24:09 EDT 2015

$ man whoami

15

Navigating – paths and directories

• pwd – Print Working Directory

• cd – Change Directorycd <directoryname>

• ls – List Contents

$ pwd

$ cd /nas02/home/

$ ls

16

Typical File Structures and Paths

17

The Killdevil file structure

• Directories and sub-directories are “folders”• Some important directories on Killdevilhttp

://help.unc.edu/help/getting-started-on-killdevil/#P63_6342

– ms/ mass storage – netscr/ scratch space– ~ home directory

• Making a new directorymkdir <directoryname>

• Removing a directoryrm –ri <directoryname>

$ mkdir 01_HTLHTCode$ cd 01_HTLTCode$ ls$ mkdir 01_week1$ cd 01_week1$ ls

18

Exercise #1

19

A few key tips and tricks

• Naming conventions– NO SPACES!

• Auto complete with TAB• Beginning of code line

– CTRL + A

• End of code line– CTRL + E

• What if I get stuck?– CTRL+C

• &#$%! Get me out of here!– Q– CTRL+C– CTRL+D– quit– logout– logoff– logout()– bye– quit()– q()– exit

• What if I need help?– man <command>– <command> -h– <command> --

help– <command>

• GOOGLE it!– Use language

name in search

20

Making and Removing files

• Making a file$ touch <filename>

• Removing a file$ rm –i <filename>

$ command [-OPTIONS] <requiredfiles> </required/path/>

$ cd$ ls$ cd 01_HTLTCode$ ls$ cd 01_week1$ ls$ touch 150615_NOTES.txt

21

Editing files

• nano, vi -- OK, but cumbersome– CTRL + X to exit– Type “y” to save

• SFTP client – better!– Cyberduck, Mozilla – SSH/SFTP

• Set it up, then drag and drop

$ nano 150615_NOTES.txt

22

Getting files onto and off of Kure

• sftp clients– Cyberduck, Mozilla – SSH/SFTP

• Set it up, then drag and drop

• scp

scp <filename> <onyen>@killdevil.unc.edu:/path/

• sftp, wget

23

Copying

• copying a file$ cp <filename> <newfilename>

OR

$ cp <oldfilename> </path/newfilename>OR

$ cp <filename> </path/>

• Copying a directory$ cp –r <dirname> </path/dirname>

OR$ cp –r <dirname> </path/>

24

moving/renaming

Renaming:$ mv <oldfilename> <newfilename>

Moving:$ mv <oldfilename> </path/newfilename>

OR$ mv <filename> </path/>

EXERCISE 3

25

Exploring files

• diff – difference between two files

• wc – count lines in files

• * is useful – wild card. Zero, one or more of any character.

26

bash

• Hello World

27

Capturing information -- variables

$ x=42$ echo x$ echo $x$ x=“onyen”$ echo $x$ echo ${x}$ echo “$x”$ echo ‘$x’

28

Capturing information -- arrays

$ starks=( “robb” “sansa” “arya” “bran” “rickon” )

$ echo $starks$ echo ${starks}$ echo ${starks[0]}$ echo ${starks[1]}$ echo ${starks[*]}$ starks[1]=“SANSA”$ echo ${starks[1]}

29

Don’t use all caps to name variables

• printenv shows environment variables