awk: generating reports - supinfo.com1lin-14]awk-generating-reports.pdf · awk: generating reports...
TRANSCRIPT
Text processing Campus-Booster ID : **XXXXX
www.supinfo.com Copyright © SUPINFO. All rights reserved
Awk: Generating reports
Your trainer…
Title: **Enter title or job role. Accomplishments: **What makes the presenter qualified to present this course.
Education: **List degrees if important. Publications: **Writings by the presenter on the subject of the course or presentation.
Contact: **Campus-Booster ID: [email protected]
Presenter’s Name
Awk: Generating reports
Course objectives
n Generate formatted reports. From any data source.
n Apply transformation functions. On data sources.
n Use selectors. Filter data
By completing this course, you will:
Awk: Generating reports
Course topics
n Introduction. How it works.
n Variables an output. Working with data.
n Functions. Use Builtin functions and write your owns.
n Control structures. If’s and loops.
n Patterns. Filtering data
Course’s plan:
Awk: Generating reports
About Awk
n Created in 1977 by AT&T and Bell
n Created by Aho, Weinberger and Kernighan
n Used to
n Manipulate files
n Create CGI scripts
n Generate reports
n Create data test and validation
Presentation:
Introduction
Synopsis Syntax of a command:
Introduction
Example:
awk [-F] [-v var=value] ‘code’ file
awk ‘{print $0}’ /etc/passwd
How it works
n Awk uses a group of instructions called ‘Pattern‘.
n Patterns are applied to records.
n Records
n Blocks of data separated by a new line or any character defined in the RS variable.
n Fields
n Strings in a record separated by a space or any character defined in the FS variable.
Globally:
Introduction
Fields and records
abcdef ghijklmnopqus tuvwxyz abcdef ghijklmnopqus tuvwxyz
Field Field
Introduction
Field
Record
Record
Stop-and-think Introduction
-FS ‘:’
-F ‘:’
-R ‘:’
-RS ‘:’
To work with /etc/passwd fields, which option and value will you use ?
Stop-and-think Introduction
-FS ‘:’
-F ‘:’
-R ‘:’
-RS ‘:’
To work with /etc/passwd fields, which option and value will you use ?
About variables Variables
n Variables
n Pre-defined
n User-defined
n Types
n Scalar
n Tables
n Fields
n Records
Store and retreive values.
Pre-defined Variables and output
Variable Value
FILENAME
RS
FS
NF
FNR
Target file
Record separator
Field separator
Number of fields in a record
Number of records
Some predefined variables:
Pre-defined Variables and output
Fields and records:
Variable Value
$0
$1, $2, etc …
Whole record
Different fields
User-defined Variables example:
Variables and output
var1 = NR var2 = my_value var3 = $0 var4 = $2 var5[1]= my_value
Using variables
n print
n printf
Display value with print and printf:
Variables and output
awk ‘{print $0}’ file awk ‘{printf “%s - %d”, $1, $2}’ file
Builtins
n Numerical functions
n cos(x)
n sin(x)
n log(x)
n String functions
n length(s)
n sub(r, s, t)
n gsub(r, s, t)
Some existing functions:
Functions
User-defined Syntax:
Functions
Example:
function functionname(param1, param2) { code code code
}
function hello(firstname) { print “Hello “ firstname
}
Stop-and-think Functions
Replace every occurrence of r by s in the t string
Replace the first occurrence of r by s in the t string
Replace every occurrence of s by t in the r string
Replace every occurrence of t by s in the r string
gsub(r,s,t) does:
Stop-and-think Functions
Replace every occurrence of r by s in the t string
Replace the first occurrence of r by s in the t string
Replace every occurrence of s by t in the r string
Replace every occurrence of t by s in the r string
gsub(r,s,t) does:
Control code flow
n Control structures
n if (condition) instruction [else instruction]
n while (condition) instruction
n for (expression; condition; expression) instruction
n Control commands
n break : Loop exit
n next : Go to the next record
n continue : Continue loop iteration
Common control structures:
Control Structures
Examples if-then-else example:
Control Structures
while & if example:
if (var1 == 3) print “var1 = 3”
else print “var1 != 3”
i = 0 while (i < 10) {
print i i++ if (i == 5)
break }
Event driven
n You can use as a pattern
n Regular expressions
n BEGIN and END expressions
n Comparison expressions
n The three combined
Associate patterns with actions
Pattern
Syntax:
PATTERN { code code
}
BEGIN & END
n BEGIN{action} executes the specified action before any other.
n END{action} executes the specified action after any other.
Header and footers
Patterns
Example : awk –F':' -f script /etc/passwd BEGIN {
print “Names:\n” } {print $1 } END {
print “Total number of users: “ NR }
Other patterns
Regular expression:
Patterns
Comparison expression:
Combination:
$1 [!]~ /^[Ss]upinfo$/ { code }
$1 == 4 { code }
$1 == 2 && $4 <= 6 { code }
Stop-and-think Patterns
Regexp
Boolean
BEGIN
END
Which pattern will you use to produce a nice header ?
For more Awk: Generating reports
Courses Publications
Web sites
www.labo-linux.com
www.blackbeltfactory.com
Linux Technologies: Edge Computing
Conferences
FOSDEM
RMLL
Solutions Linux
If you want to go into these subjects more deeply, …
www.supinfo.com
Sed & Awk
Congratulations
You have successfully completed the SUPINFO course module n°15
Awk: Generating reports