introduction to linux - mcgill university - mcgill... · 2015-09-28 · bash bash is a shell: gives...
Post on 27-Mar-2020
16 Views
Preview:
TRANSCRIPT
Introduction to Linux
September 24, 2015
1
By: Simon Nderitu
Partners and Sponsors
2
3
Schedule and Outline● 9:00-10:00 The Terminal● 10:00-11:30 File Systems
○ 10:20-10:30 Break● 11:30-13:30 Running Commands
○ 12:00-13:00 Lunch● 13:30-14:30 Processes
○ 14:20-14:30 Break● 14:30-15:00 How to Compile● 15:00-15:30 Remote Connections● 15:30-16:00 Graphical User Interfaces in Linux
Short History of Linux● Reference:
○ http://www.cs.cmu.edu/~awb/linux.history.html● Project : started in 1991 by Linus Torvalds● Context :
○ Unix vs DOS○ New Intel 386 architecture
● Linux kernel:○ Open source○ Under the GNU GPL
● Linux used in high-performance computing clusters
4
The Terminal
5
The Terminal● How to Open a Terminal● Bash● How to Exit● Previous Commands in Bash● Auto-completion in Bash● User Manuals● Help Topics● Environment Variables (1-2)● Exercises
6
How to Open a Terminal● How to log in a local computer
○ Username: csuser##○ Password: ___@[S##
■ Replace ## with both digits on the first line of the small piece of paper
■ “___” are 3 secret letters provided in classroom● In the Ubuntu interface, open a Terminal
○ On top left of the screen, click on the Ubuntu button○ Type: terminal○ Select the Terminal application
7
Bash● Bash is a shell:
○ Gives access to file systems○ Runs commands as a specific user○ Has per-session environment variables
● Started from a terminal, a parent shell or SSH.○ In Ubuntu, search for application “Terminal”.
● The Bourne-Again shell (Bash) is the default shell on Guillimin and on many other linux systems.
● Other shells : Bourne shell (sh), C shell (csh), Tenex C shell (tcsh), Korn shell (ksh), etc.
8
How to Exit● Type “exit” (w/o quote marks) and press Enter
○ The “exit” command will go to the history● Or use Ctrl+D
○ Bash receives commands from the STDIN stream○ Ctrl+D closes the STDIN stream○ Bash will exit itself as soon as STDIN is closed○ This exit will not be logged in the history
● If Bash was started by:○ A terminal : the terminal will close as well○ A parent shell : the user will return to the parent shell○ SSH : the connection will be closed
9
Previous Commands in Bash● To get all the logged previous commands, type:
history
● To get a previously executed command, use the Up arrow key until you find it
● You may search for a command in the history:○ Use Ctrl+R○ Start typing parts of the command you are looking
for○ Use the Left or Right arrow keys to select and edit
the command (if needed)○ Use Ctrl+C to cancel
10
Auto-completion in Bash● The first word in a command is usually an
executable:<executable>
● The other words (or arguments) may be options or short or long paths to files or folders:
<executable> -a -b --option --option2<executable> -ab -- <filename>
● Knowing that, Bash can auto-complete what you are typing or give you some suggestions
● Use the Tab key to trigger the auto-completion function
11
User Manuals● Some programs have one or many manual pages
man <program-name>
● Most of the man pages are also on the Web● In the man page, you can search for a keyword:
○ Type “/”○ Type your keyword and press Enter○ All instances of the keyword will be highlighted. Man
will go to the first instance. Press the “n” key to go to the next one.
● To quit the man page, press the “q” key○ Man pages are displayed (piped) through a program
called 'less'
12
Help Topics● Some programs provide additional help topics on
the command line:
executable --help
executable -h
executable help
13
Environment Variables● To print a list of the current environment
variables:
env
● Notable variables:○ HOSTNAME: computer's name○ PWD: current directory○ SHELL: default shell (Bash)○ LANG: session language, text encoding
14
Environment Variables (2)● Other notable variables:
○ LOGNAME, USER: session user name○ GROUP_ID: groups of current user○ HOME: home directory○ PATH: paths to executable files○ LIBRARY_PATH: paths to libraries (mainly for
compilation)○ LD_LIBRARY_PATH: paths to libraries (mainly for
execution)
15
Exercises● Use the auto-completion to reach the following
executable and print the help topics:
/usr/bin/svn
○ /u<Tab>b<Tab>...● Connect to Guillimin and get the hostname:
ssh guillimin.hpc.mcgill.ca -l class##
env
● What is Python? Read its manual page:
man python
16
File Systems
17
File Systems● Where am I?● Structure of Directories● Mounted File Systems● Files and File Attributes (1-4)● Changing File Attributes● Directories and Directory Attributes (1-2)● Changing Directory Attributes● Symbolic and Hard Links● Exercises (1-2)
18
Where am I?● Print working directory:
pwd
● List the files and folders in the current directory (# for comments):
ls # Same as ls * where * could be anything
ls -a
● Hidden files and folders start with a dot “.”● The “.” folder is the current directory● The “..” folder is the parent directory● To change the directory (use “;” between commands on
one line):cd <directory_name_or_full_path>
cd .. ; cd - # parent, previous
19
Structure of Directories● / : root directory of everything in Linux● /home : contains all home directories● /tmp : temporary files and folders● /bin, /usr/bin, /usr/local/bin, etc. :
○ see the PATH variable (directories for executable files)
● /dev : devices and virtual devices○ /dev/null, /dev/random, /dev/urandom, /dev/zero
● /proc : processes information○ /proc/meminfo, /proc/cpuinfo
20
Mounted File Systems● Directories are mount points
/media/USB-DRIVE
● Print the mounted file systems:mount
● Usage of the mounted file systems:df -h
● Your personal quota (if any):quota # or myquota on Guillimin
● Disk usage of a file or a directory:du -sh <file_or_directory>
21
Files and File Attributes● Create a file or change its time stamps:
touch <filename>
● Show the attributes of a file:stat <filename>
ls -l <filename>
● Rename or copy a filemv <filename> <newname> # No confirmation
cp -a <filename> <newname> # Preserves all
● Remove a file permanently (no trash/recycle bin):rm <filename> # No confirmation
22
Files and File Attributes (2)● Current language (LANG):
locale
● Edit a new or an existing text file (nano, emacs, vi):
nano -w <filename>
● Keyboard shortcuts in nano:○ Ctrl+X : Exit○ Ctrl+O : Save○ Ctrl+K : Cut the current line. Repeat for consecutive
lines○ Ctrl+U : Paste the multiple lines
23
Files and File Attributes (3)● Get the type of file (text encoding, image format, ...):
file <filename>
● View a text file:less <filename>
■ Press “q” to quit
more <filename>cat <filename>
● Word count and line count:wc <filename> # All statisticswc -w <filename> # Word count onlywc -l <filename> # Line count only
24
Files and File Attributes (4)● See the default file attributes:
ls -l # -rw-rw-r-- username group
umask # 0002
id # uid=... gid=... groups=...
● Permissions:○ - : regular file○ rw- : user has read and write access, no exec.○ rw- : group has read and write access, no exec.○ r-- : other users have read access only
25
Changing File Attributes● Changing the permissions:
chmod u+x,o-r <filename>
chmod 760 <filename> # -rwxrw----
● 760 is an octal number : 421 42- ---chmod 640 <filename> # -rw-r-----
● Changing the ownership:chown <username> <filename>
chown <username>:<group> <filename>
chgrp <group> <filename>
26
Directories and Directory Attributes● Go to your home directory and list its attributes
cd ~
cd
ls -ld . # drwx------ username group
ls -la
● To access inside a directory, the execute “x” permission must be enable.
● Hidden directories start with a “.”
27
Directories and Dir. Attributes (2)● Create a directory:
mkdir <dirname> # mkdir -p very/long/new/path
● Show the attributes of the directory:stat <dirname> ; ls -ld <dirname>
● Rename or copy a directorymv <dirname> <newname>
cp -r <dirname> <newname> # cp -pr OR cp -a
● Move a file or a directory into another directory:mv <file_or_dir> <dirname>/[<newname>]
● Remove a directory (no trash/recycle bin)rmdir <emptydir>
rm -r <dirname>28
Changing Directory Attributes● Remove other accesses recursively:
chmod -R o-rwx <dirname>
● Add the group sticky bit recursively:chmod -R g+w <dirname>
find <dirname> -type d -exec chmod g+s {} \;
● Changing the ownership recursively:chown -R <username> <dirname>
chown -R <username>:<group> <dirname>
chgrp -R <group> <dirname>
29
Symbolic and hard links● Create a symbolic link (files and directories):
ln -s <original_file_or_dir> <linkname>
ls -l
● Create a hard link (files only):ln <original_file> <linkname>
ls -li # inode_ID ... num_links
● Remove a link:rm <linkname>
30
Exercises● Create a text file. Write something in it.● Create a directory. Move the file into the directory.● Block all other users access (rwx) to this directory.● Create a second directory. Move the first directory
in this second directory.● Create a symbolic link to the first directory.● Create a hard link to the text file. Modify the file
from the new hard link.● Print the content of “both” files.● Remove everything created in these exercises.
31
Exercises (2)● Import a compressed archive from Internet:
mkdir www; cd www
wget http://www.hpc.mcgill.ca/downloads/
intro_linux/intro-linux201509.tar.gz
● Extract the content:tar -xvf intro-linux201509.tar.gz
● Create another tar archive:tar -cvf archive.tar intro-linux
● Compress the new archive:gzip archive.tar
32
Running Commands
33
Running commands● Which, Where?● What Time?● Hello World!● Filtering (1-2)● Bash Scripts● The .bashrc file● Exercises
34
Which, Where?● Which command is this?
which <executable>
● Where is my file? Where is this program?locate <file>
● Find a file in a directory:find <dirname> -name <file_or_dir>
35
What Time?● The current date and time:
date
● Time to go to sleep?time -p sleep 123.45
● Want to stop a command running?Ctrl+C
36
Hello World!● Say hello and world (two arguments):
echo "Hello" $(echo -n "World")
● Pipe between output and input of two processes:echo "Hello World" | less
● Output redirection to a file (if the file exists, its previous content will be lost):
echo "Hello World" > file.txt
● Output redirection appended to the end of a file:date >> file.txt
● Input from a file:less < file.txt
37
Filtering● Basic regular expressions / searching in files:
cat file.txt | grep -i hello # Case insensitive
grep -n -i hello file.txt # Show line number
grep some_string file1 file2 file3
● Extended regular expressions:grep -E "^k?[hH]el+o.{2}o.*$" file.txt
○ ^: beginning of line, $: end of line○ ?: 0 or 1 occurrence, *: 0 or many occurrences○ +: 1 or many occurrences, {n}: exactly n occurrences○ .: any character○ [a-zA-Z0-9]: list of characters (matches only one)
38
Filtering (2)● A .csv file (intro-linux/examples/):
Animal,Color,Age
cat,black,10
dog,black,5
dog,brown,8
● Show all dogs:awk -F, '/dog/ {print $3,$2}' file.csv
grep dog file.csv | awk -F, '{print $3,$2}'
39
Bash Scripts● Consider script.sh:
#!/bin/bashecho "------------"
date
ls -a
● Make script.sh executable:chmod +x script.sh
● Run script.sh:./script.sh > myfiles.txt
40
The .bashrc file● Located in your home directory:
cat ~/.bashrc
● This is a Bash script that is executed at the beginning of each Bash session
● Used to set environment variables and aliases● Example of an alias:
alias rm='rm -i' # Adds confirmation
41
Exercises● Use the .bashrc file, echo, hostname, date
and output redirection to log all new Bash sessions in ~/.mysessions with the following format:
New session on <hostname>: <date>
● Open and close Bash a few times.● Print only the date of the last 4 sessions (use
tail -n, the separator would be “: “ for awk).
42
Processes
43
Processes● Running Processes● How to Kill Processes● Processes in Background● Introduction to Threads● Exercises
44
Running Processes● Listing the running processes in your session:
ps
● Listing all the running processes on the server:ps -ef
● The init process (PID = 1)● The parent PID (PPID)● Monitoring the running processes (“q” to quit):
top
45
How to Kill Processes● Sending signals to a process:
kill -<number> PIDkill -s SIGNAL_NAME PID
● To list the different signals:kill -l
● The default termination signal is SIGTERM (15):kill PID
● The termination signal from a terminal (like Ctrl+C):kill -s SIGINT PID # or kill -2 PID
● To force the process to stop now:kill -s SIGKILL PID # or kill -9 PID
46
Processes in Background● Processes unattached to a terminal : they are running
in background● To start a process directly in background:
command arg1 ... > outputfile &jobs
● To resume in foreground:fg
● To suspend a foreground process (SIGSTOP): Ctrl+Z● To resume in background (SIGCONT):
bg
● To wait for all background processes to terminate:wait
47
Introduction to Threads● One or many threads in a process
○ CPU usage may be greater than 100%● Shared memory
○ False sharing● Synchronization concepts:
○ Barrier: synchronisation○ Mutex: one and only one thread○ Semaphores: wait and release
48
Exercises● Start a secondary bash session in your primary
session● Start the following command
dd if=/dev/urandom of=/dev/null
● Put it in the background○ Suspend the command○ Resume in the background
● While running, look at the CPU usage● Quit the secondary bash session● Get the new PPID of dd● Kill the dd process with SIGTERM
49
How to Compile
50
How to Compile● C with OpenMP Code● C with MPI Code● C++ Code● Makefiles● Perl Scripts
51
C with OpenMP Code● Code example
main.c
● Compilationgcc -o exec_ser main.c
gcc -o exec_omp main.c -fopenmp
● Executiontime -p ./exec_ser
time -p ./exec_omp
52
C with MPI Code● Code example
main.mpi.c
● Compilationmodule add ifort_icc # On Guillimin
module add openmpi
mpicc -o exec_mpi main.mpi.c
● Executiontime -p mpiexec -n 2 ./exec_mpi
53
C++ Code● Code example
main.c
● Compilation
g++ -o exec_cpp main.c
● Execution
time -p ./exec_cpp
54
Makefiles● A Makefile file
○ Variables (set: VAR=..., use: $(VAR))target: dependencies commands ...
○ Regular expressions%.o: %.cpp
○ Special variables ($@, $<, $+)● The make command
make -f <makefile>make # if Makefile or makefilemake <target>
55
Perl Script● Code example
main.pl (-rwxr-x---)
● Code validationperl -c main.pl
● Execution
perl main.pl
./main.pl
56
Remote Connections
57
Remote Connections● Using SSH● Using SCP● Using SSH Keys
58
Using SSH● From a terminal (Mac OS X, Linux):
ssh <username>@<server.name.ca>
ssh <server.name.ca> -l <username>
○ (Accept the certificate) Enter the password● Connect to a different port number:
ssh <username>@<server.name.ca> -p <num>
● From PuTTy (Windows, Linux):○ Host Name: <server.name.ca>○ Click on “Open” (Accept the certificate)○ Login as: <username>, Enter the password
59
Using SCP● Secure copy with SSH protocol● From a terminal (Mac OS X, Linux)
scp filename <user>@<server>:/full/path/
scp filename <user>@<server>:directory/
scp filename <user>@<server>:newname
scp -pr dirname <user>@<server>:
scp * <user>@<server>:directory/
● From WinSCP (Windows)○ Server name, credentials, drag & drop○ Beware of the text copy mode (\n vs \r\n)
60
Using SSH Keys● Having access with or without a passphrase:
ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (~/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
● Copy the public key to a remote site (id_rsa.pub):
scp .ssh/id_rsa.pub <user>@<server>:
ssh <user>@<server> \
"cat id_rsa.pub >> .ssh/authorized_keys"
61
Graphical User Interfaces in Linux
62
Graphical User Interfaces in Linux● Desktop Environments and Shell Interfaces● The X Server
63
Desktop Environments andShell Interfaces● Gnome
○ http://www.gnome.org/gnome-3/○ https://unity.ubuntu.com/about/
● KDE○ http://userbase.kde.org/An_introduction_to_KDE
● Linux Mint Cinnamon○ http://cinnamon.linuxmint.com/
● LXDE○ http://lxde.org/image_galleries/screenshots
● MATE (Gnome 2 fork)○ http://mate-desktop.org/index.html
● Xfce○ http://www.xfce.org/about/screenshots
64
The X Server● Decoupling the application and the rendering
engine○ X Client: OS, application, remote server○ X Server: keyboard, mouse, screen
● Using SSH: -X or -Yssh -X <user>@<server.name.ca>
● X Server for Windows : Xming, MobaXtermhttp://sourceforge.net/projects/xming/http://mobaxterm.mobatek.net/
● Open remote applicationcommand args... &
65
Conclusion
66
Conclusion● Try Linux in a virtual machine (Virtual Box)● Many distributions
○ http://en.wikipedia.org/wiki/List_of_Linux_distributions
○ Arch Linux, CentOS, Debian, Fedora, Gentoo, Linux Mint, Ubuntu, etc.
● Web pages are friendlier than Man pages
● Questions?guillimin@calculquebec.casupport@calculquebec.ca
67
top related