linuxcbt scripting edition
Post on 22-Dec-2015
93 Views
Preview:
DESCRIPTION
TRANSCRIPT
LinuxCBT Scripting Edition is a unique approach to teaching GNU/Linux Scripting skills. Serious Linux IT-professionals are expected to have a command of various scripting environments for administrative purposes. LinuxCBT Scripting Edition consists of the following 7 Modules:
Bourne Again Shell (BASH) Perl Python PHP Regular Expressions (RegEx) Awk & Sed field (Column) and streams (Automated Text Editing) KornShell
LinuxCBT Scripting Edition, is unparalleled in content, depth and expertise. LinuxCBT Scripting Edition prepares you or your organization for successfully developing and implementing business-critical GNU/Linux-based scripting solutions. Linux automation requires a solid scripting foundation.
Recommended Prerequisites:
Open mind & determination to master Linux Scripting Solutions - BASH/Perl/Python/PHP Access to a GNU/Linux-based PC to perform exercises in LinuxCBT Scripting Edition
Bourne Again Shell (BASH) Scripting - Module I
Introduction to BASH Command Line Interface (CLI) Identify BASH installation & key configuration files on RedHat & Debian GNU/Linux
distros .profile/.bash_profile/.bashrc/.bash_history
Identify & use common built-in BASH commands (pwd,cd,set,unset,export,source,etc.) BASH Command-line Expansion
Brace Expansion Tilde Expansion Parameter & variable Expansion Command Substitution
BASH Reserved words BASH History identification & configuration Aliases (ls,du,df,rm,cp) Backticks - command expansion & execution BASH Prompt configuration Command chaining Error level/Return code (Command Exit Status) identification Conditional execution of subsequent commands (&&,||)
Redirection | Error Handling Standard Input
Direct input from STDIN (Standard Input) Redirect input from a file
Standard Output Direct output to STDOUT (Standard Output ) Redirect output to a file Append Standard Output to an existing file
Standard Error Explore STDERR attributes
Redirect Standard Error Input/Output (I/O) Redirection
Pipes Sequential execution via pipes
Quoting & Escaping BASH Scripting
Basic Script definition (hello world), permissions modification & execution Internal BASH variables Global Variable definition Function definition Local Variable definition - function scope BASH Arrays Loops Menu creation using Select Job control - Foreground/Background processing Conditionals Case - Concise Conditional Branching Positional Parameters
BASH Script Definition Create BASH script to move multiple files to new names defined by positional
parameters Create script to monitor directory for file changes and report the differences via E-mail
to Administrator Create script to monitor directory size and report threshold encroachment to
Administrator via E-mail Define script to parse logs for keywords and notify the Administrator via E-mail Create script to monitor network connectivity between hosts and report outages to
Admin via E-mail Define script to monitor process, restart if fails, and E-mail Administrator at defineable
limits Create script to backup sub-directories using Tar & Gzip, define threshold, and SSH file
to remote host Log the output of BASH scripts to a Syslog-like format
BASH 3.x Updates Explore BASH 3.x on multiple platforms: Linux|Solaris|MacOSX|FreeBSD Compare and contrast to BASH 2.x Execute BASH 2.x commands|scripts within BASH 3.x on multiple platforms Evaluate results
top
Practical Extraction & Reporting Language (Perl) - Module II
Perl Basics Discuss Perl's capabilities Identify available Perl binaries on the system Discuss Perldoc - documentation program Discuss Perl interpreter Define first Perl program - Hello World Explain the application of the SHEBANG header within Perl scripts Explain warnings options
Explain strict options Discuss compilation of Perl scripts and compare and contrast with shell scripts Comments Print function Escape sequences including (\t, \n, etc.) Explain quoting rules and variable interpolation Variable definition Operators
Perl Variables Variable scopes (Global/Local) Scalars - single values
Discuss the properties of scalar variables Evaluate the definition of scalars Variable concatenation
Arrays - lists & lists of lists Discuss the properties of array variables Evaluate the definition of arrays Multi-dimensional arrays Array slices Array functions Ranges
Hashes - Key/Value pairs Discuss the properties of hashes Evaluate the definition of hashes
Control Structures Discuss various types of Subroutines and features Define non-returning sub-routines Define returning sub-routine Conditionals If - integer and string comparisons Else/elsif testing Unless For Loops Foreach Loops - Array iteration While Loops Until Loops
General Perl Information I Block Definition Use the X Functions for file system testing Positional parameters - ARGV Determine number of command line arguments Record Separators Internal Perl Variables Modules Explore the module path and applicable variables Discuss various ways to install Perl modules Install modules File I/O File Handles
Open Function Discuss rules regarding file handles and use of the open function Assign file handles to Perl arrays for iteration Process text files using applicable file I/O functions and foreach loops Generate data files using file I/O functions Die function Error handing with Perl's built-in variables Count records from input file Use the Split function to separate fields based on various delimiters Use Join to assemble lists
General Perl Information II References Chomp - strip superfluous characters Command substitution Use execute function to launch external programs Use system function to execute external commands and evaluate return status Regular Expressions discussion Demonstrate the many rules of Regular Expressions Process arrays using push/pop functions Generated data files and process them using split and field determination Mail integration using Sendmail Mail integration using downloaded Perl mail module Define and execute Perl one-liners Common Gateway Interface (CGI) - Apache 2.x integration Process GET & POST variables MySQL Database integration Identify required modules for MySQL integration Define connection object Execute SELECT queries Execute INSERT queries Execute UPDATE queries Execute DELETE queries
Perl LFTP Integration - Error Handling Integrate LFTP client with Perl Automate LFTP process with Perl Implement error-handling Evaluate results
top
Python - Object Oriented Linux Scripting Environment - Module III
Basics Discuss Python's capabilities & features Identify location of pre-installed Python Discuss Python's Interactive interface Use the interactive interface to ouput basic expressions Hello World Identify reserved keywords Escape sequences & escape character exploration Define Python script using Linux SHEBANG header
Execute Python scripts using the Python interpreter directly Discuss data types including strings, integers, lists, tuples, dictionaries, etc. Define standard variable types Use the print function to examine ouput of variables and expressions Use the type function to resolve data type Use the id function to expose the memory location of the stored variable Explain and define references and integrate with the id function Comments definition Perform basic arithmetic to illustrate the PEMDAS order of operations and examine how
to alter it General Python Information I
Explain and illustrate STDIN usage Examine the input function Examine the raw_input function Compare and contrast between input and raw_input functions Define small script to make use of input/raw_input and basic math functions Python Function concepts, applications and definition
String Manipulation Define and ascertain the length of strings String extraction using list elements and list slicing Discuss immutable and mutable data types Loop through string slices using for loops Compare strings using string operators Discuss Python's object-oriented nature of various modules including classes and
methods Import strings module Convert strings to upper and lower case using strings module methods Split strings into lists Join lists into strings
List Manipulation Define lists Discuss and Define tuples Compare and contrast lists and tuples Reverse lists Append lists Extend lists Nest lists Define multi-dimensional lists Stack Lists using Last In First Out (LIFO) using the pop function Define First In First Out (FIFO) lists to emulate queueing using the pop function Insert values into various positions of the list Use ranges to generate integers Lists with Integers Lists with Strings Emulate standard UNIX Syslog logfile format using Lists Parse emulated logfile format using the Split function Recap list manipulation using integers, strings and various list methods Explain and define Dictionaries - key/value pairs similar to Hashes in Perl
Control Structures
IF - Defintion and usage Discuss the importance of indentation Else/Elif - comparisons against integers and strings Comparison negation Write script to perform conditional testing Import sys module to evaluate positional parameters using ARGV Define script which requires minimum number of command-line arguments Nest multiple if/elif statements to match logic Explain how for loops work in Python for loops definition and usage Iterate through lists of strings using for loops Integrate for loops with string manipulation functions and log file parsing Define and use While loops Define and examine the consequences of incorrect logic with an infinite while loop Integrate while loop with raw_input function to obtain data from the end-user Discuss file handle concepts and applications Use control structures to generate output for use in File I/O exercises Use the open function to read a file into a file handle Iterate through the lines within the opened file using readline functions, etc. Open file handles for writing Open multiple file handles Write strings to files using string formatting options and variable substitution Parse source file by extracting columns to be stored in destination file Regular Expressions Discuss Regular Expressions Import Regular Expressions module (re) for usage Implement Regular Expressions (REs) Regular Expressions Matching of strings and variables REGEX Repeaters Integrate Regular Expressions with text file processing Exception Handling Try & Except Modules Identification of default Python modules Import modules (sys, os, etc.) for usage Use the dir function to ascertain the classes and methods available within modules Use the SHUTIL module for advanced, high-level File I/O (Copy, Move, etc.) Python - Syslog integration Define script using the Syslog module to log to syslog Common Gateway Interface CGI Discussion and applications Implement Python CGI with Apache 2.x Evaluate results of CGI integration Globbing
Dictionaries Further exploration of dictionaries Parse log files into dictionary data structure Organize results Analyse data
Time Review properties of various time functions Export time with various functions Use time functions to create log entries in process akin to Perl Evaluate results
top
PHP - Hyper Text Pre-Processor - Dynamic Web Applications & CLI - Module IV
Installation Identify Online PHP resources Download PHP Identify Apache prerequisite packages and programs - Modular support (mod.so) Extract, compile and install PHP with Apache modular support Compile and install PHP with Command Line Interface (CLI) support Identify key PHP configuration files & directories Configure Apache httpd.conf with PHP integration to provide PHP support Configure Apache with PHP binary and source support
PHP Basics Explain PHP-supported long/short/ASP parse-tags & identify configuration changes in
php.ini Hello World script output using echo to the web browser Describe the various comments supported by PHP Examine the ouput of PHP-maintained variables using phpinfo() Explain supported data types via variables (integers, strings, floating points, here
documents, arrays, etc.) Test instantiation of supported variables with default types Explain Quoting rules and variable interpolation Demonstrate escape sequences such as tab, new lines, spaces, etc. Exercise the echo command Define basic associated arrays (i.e. hashes/dictionaries) Define nested associative arrays
General Information Examine key CGI PHP-server variables
HTTP_USER_AGENT PHP_SELF REQUEST_METHOD DOCUMENT_ROOT REMOTE_ADDR HTTP_REFERRER
Explore HTTP GET Request Methods via the URLString and Forms Explore HTTP POST Request Methods via the URLString and Forms NULL Variables Basic Math - PEMDAS Integer and String Comparison Operators Examine execution operators for executing Linux Shell commands and obtaining the
ouput Variable assignment via execution operations Illustrate the usage of nested arrays
Control Structures
IF - comparisons against integers and strings Else/Elseif - comparisons against integers and strings Test whether variables are of the same type Comparison negation C-stlye for loops definition and usage Generate HTML option boxes using C-style for loops foreach list iterator loop definition and usage Generate HTML based on the contents of the list iterated by foreach While loops definition and usage Examine filesystem listings using execution operators and while loops do While definition and usage Break Continue Switch - for concise if/elseif conditional examination
General Information II Discuss the applications of Server Side Includes (SSI) Server Side Includes using require Server Side Includes using include Explain and examine the differences between require and include Demonstrate distributed include applications by including content using HTTP Header, leftnav, body, and footer definition and integration with require/include HTML Form defintion using include and control structures PHP Functions definition and sample usage ISSET & UNSET for examining and controlling variables Discuss the way PHP handles uploaded files Identify key configuration files which affect uploaded files HTTP File I/O - uploads & processing via HTML forms Define back-end action pages to handle HTTP file uploads Examine available file attributes of HTTP-uploaded files PHP SMTP E-mail integration discussion Define scripts to process forms and generate SMTP-based e-mail Examine PHP Mail function options Use mutt on the Linux system to examine PHP-generated SMTP-based e-mail
PHP - Dynamic Web Programming PHP cookie concepts & usage discussion Cookie definition and examination on server and client system (Mozilla Firefox) Set and unset cookies on client system Make decisions based on the existence of cookies on the client system Session Variables concepts & usage discussion Identify pertinent configuration options for session variables Define session variables Integrate cookies with session variables Download MySQL RDBMS packages for usage with PHP Install MySQL RDBMS for usage with PHP Download the latest phpMyAdmin for installation Install phpMyAdmin with Apache supportfor web-based administration of MySQL
RDBMS Recompile PHP for MySQL RDBMS support Update PHP configuration information to support MySQL
Define basic database structure for usage Structured Query Language (SQL) integration with PHP Explain steps for connecting PHP scripts to MySQL Define and execute SELECT queries Define and execute INSERT queries Define and execute UPDATE queries Define and execute DELETE queries Forms Processing with MySQL integration Forms with Cookies Forms with Session Variables
PHP - Command Line Interface (CLI) Discuss concepts and applications Identify CLI binary Discuss the possible ways of executing PHP scripts from the CLI Define and execute PHP one-liners Execute scripts using PHP binary as a prefix Explore executing PHP via the Interpreter mode PHP-Linux SHEBANG definition and usage Execute basic commands CLI - File I/O usage - Stat files Examine file attributes Determine access and modification times and make decisions using control structures
PHP - Input Validation Define form and action script to process form fields Submit form-fields for processing Examine results Define input validation masks with regular expressions Evaluate results Extend validation masks to command line interface (CLI) Supply PHP CLI with input Confirm validation masks
top
LinuxCBT Scripting Editon - RegEx - Module V
Introduction - Features Discuss course outline Explore system configuration Identify key systems to be used Install the Bluefish graphical editor - Supports POSIX and PERL-compatible RegExes List key Regular Expressions applications Enumerate and discuss key Regular Expressions features
Metacharacters Enumerate important metacharacters Provide examples of asterisk '*' usage Apply simple character classes to searches Perform searches in the Bluefish editor using literals and metacharacters Explain and apply anchors: '^', '$'
Character Classes Discuss features and applications
Perform searches in the Bluefish editor using character classes Search for case-insensitive characters Negate character class values using the circumflex character: '^' Specify ranges and inclusions Search for string boundaries Alternate between POSIX and Perl-compatible Regular Expressions in Bluefish Editor
RegExes with GREP & EGREP Discuss features and applications Perform simple GREP searches Use EGREP to find strings based on RegExes Parse installed applications using EGREP Extract 'su' attempts from the system log file: /var/log/messages Parse log file entries based on specific dates
Parenthetical Expressions Discuss features and applications Compare and contrast with character classes Search for strings using the Bluefish Editor Combine anchors with parenthesis and evaluate results Search for multiple groups of strings using alternation Use parenthetical expressions with EGREP Nest expressions using parenthesis
Quantifiers Discuss features and benefits Define RegEx tokens Enumerate and discuss RegEx quantifiers Test '?' using the Bluefish Editor and EGREP Test '*' using the Bluefish Editor and EGREP Test '+' using the Bluefish Editor and EGREP Test '*' using the Bluefish Editor and EGREP Test '{ }' - arbitrary quantifiers with the Bluefish Editor and EGREP Combine anchors, parenthesis and various quantifiers
Shortcut Sequences - POSIX & Perl Discuss features and benefits Enumerate and discuss key shortcut sequences Test shortcut sequences using the Bluefish Editor
Awk - Field Parser Discuss features and applications Use awk to extract fields from the process listing Apply RegEx literal strings to awk search and evaluate results Apply RegEx character classes to awk search and evaluate results Apply RegEx parenthetical expressions to awk search and evaluate results Apply RegEx quantifiers to awk search and evaluate results Incorporate conditional RegEx matching with awk search and evaluate results Extract fields, based on RegExes, from system log file: /var/log/messages
SED - Stream Editor Discuss features and applications Perform text transformation of text file using string literals Perform text transformation of text file using string literals and character classes Perform text transformation of text file using string literals and anchors
Perform text transformation of text file using string literals and character classes Evaluate results
GNU Utilities (GREP|Awk|SED) for Windows Discuss features and benefits Identify and connect to Windows Server target system Download and install GNU utilities for Windows: GREP, Awk, SED Perform EGREP searches and evaluate results Perform gawk searches and evaluate results Perform SED substitutions and evaluate results
Perl Regular Expressions Discuss features and applications Write simple Perl script to compare command line argument to RegEx and evaluate
results Alter Perl script to read text file and perform RegEx comparisons and evaluate results Incorporate character classes, parenthetical expressions, case-insensitivity and
quantifiers Evaluate results Substitute matched text using Perl RegExes Substitute matched text using Perl RegExes from the command line Update system log using text substitution from the command line
PHP Regular Expressions Discuss features and applications Perform POSIX and Perl-compatible RegExes from the command line Substitute text using RegExes Evaluate results Design HTML form to collect fields via the browser Design back-end, action script to process the POST fields using RegExes Halt processing if POST fields do NOT meed criteria based on RegExes Evaluate results
Apache Regular Expressions Discuss features and benefits Identify key configuration files Explore key RegEx-supporting Apache directives Match a range of directories using RegExes with <DirectoryMatch> directive Restrict access to files using RegExes with <FilesMatch> directive Match URL directory values using RegExes with <AliasMatch> directive Redirect users to alternate content using RegExes with <RedirectMatch> directive
Date | Time Extractions | Regular Expressions | Logs Identify log entries to filter Filter date | time ranges using regular expressions Substitute extracted columns with replacement data Evaluate results
HTML Updates Select HTML | PHP files to update Identify key strings Replace key strings with substitutes Perform widespread changes across HTML | PHP documents Evaluate results
top
LinuxCBT Scripting Edition - Awk & Sed - Module VI
Introduction - Features Discuss course outline Explore system configuration Identify key systems to be used Install the Bluefish graphical editor - Supports POSIX and PERL-compatible RegExes Discuss features common and distinct to: Awk and Sed
Regular Expressions Review Enumerate important metacharacters Provide examples of quatinfier usage Apply simple character classes to searches Perform searches in the Bluefish editor using literals and metacharacters Explain and apply anchors: '^', '$'
Intro to Sed - Stream Editor Discuss features and applications Explain general usage syntax Print specific lines from input Delete specific lines from input Match streams using RegExes Search for ranges of text using line numbers and text ranges Save transformed output
Sed - Search & Replace Discuss features and applications Discuss Left-hand and Righ-hand side (LHS | RHS) sections in search & replace actions Acquire input from: STDIN, PIPES and Files and transform according to criteria Backup and overwrite source files using Sed one-liners Perform substitutions based on matched addresses Reuse matched strings in replacement output Perform text substitutions across multiple files Perform multiple replacements in one command Apply substitutions globally across all matches
Sed - Scripts Discuss features and applications Explain general usage syntax Create script to remove blank lines from input stream Create script to make general substitutions Perform case-insensitive replacements via Sed script Insert strings into desried location in the input stream via Sed script Strip trailing numeric values from input stream via Sed script Apply changes to multiple files Save scripts for reuse
Intro to Awk - Field Processor & Reporter Discuss features and benefits Explain general usage syntax Print full lines from input Print desired fields from input Match patterns using RegExes Control the input field | column delimiters
Save output Awk - Scripts
Discuss features and benefits Explain structure of Awk scripts Implement BEGIN and END blocks Parse system files using Awk script
Awk - Variables Discuss features and applications Enumerate and discuss key Awk system variables Increment counters based on matched patterns using Awk Explain variable concatenation Explore arrays (lists of values) Create arrays using the split function
Awk - Operators Discuss features and applications Explore Boolean and Relational operators Discuss looping strategies Evaluate conditional testing using system variables Evaluate results
Awk - Process Records Discuss features and benefits Define and evaluate multiple input field separators with files with mixed delimiters Normalize output by controlling the Output Field Separator (OFS) Extract records from system file and evaluated Account for number of processed and matched records in END block
Awk - Print Formatting Discuss features and applications Compare and contrast 'print' and 'printf' output formatting Discuss 'printf' supported data types Evaluate 'printf' application in reporting
Awk & Sed - Various Examples Discuss features and applications Parse and cleanup HTML and PHP files with Sed & Awk Replace values in system file with Awk Evaluate results
Awk & Sed - Win32 Discuss features and benefits Download and install Awk & Sed for Win32 Transfer Linux files to Windows Test various parsing and substitution on Linux files Evaluate results
Sed - File Name Transformations - Batch Enumerate poorly-named files for transformation Identify key replacement strings for regular expressions match Define and apply regular expressions for transformations Transform prefixes and suffixes Evaluate results
Awk - E-Mail Address Parsing Generate sample data set
Define regular expressions patterns for transformation Transform sample data Evaluate results
top
LinuxCBT feat. KornSHell Edition - Module VII
Introduction - Features Discuss KornShell history Explore system configuration Identify key systems to be used Discuss key features
Shell Basics Create KornShell users on various systems Identify key KornShell binaries and configuration files Explore key KornShell variables Discuss shell redirection and piping
Command Chaining | Substitution Discuss features and applications Explain general usage of 'test' command Evaluate sequential execution of processes Invoke commands contingently using logical AND and OR Implement command substitution Evaluate defined variables using command substitution
Variables Discuss features and applications Discuss and reference positional parameters Explore standard command exit status variable Learn how to define variables Define persistent variables Unset variables and evaluate
The Prompt - $PS{1..4} Discuss various prompt shell variables Setup vi-style command history access Customize primary prompt with useful KornShell variables Define a persistent prompt Evaluate results
Functions Discuss features and benefits Explore POSIX and KornShell functions Define useful functions in a script Reference functions from a script Discuss how positional parameters are supported with funcitons Evaluate results
Loops Discuss features and benefits Identify the supported iteration mechanisms Implement for | while | until loops Create finite and infinite loops Evaluate results
Conditions Discuss features and applications Explore comparison testing Perform numeric comparisons Perform string comparisons Branch out using if | elif | else Evaluate results
Error Handling Discuss features and applications Create script which requires a positional parameter Error handle as necessary Confirm the existence of the file referenced by the positional parameter Compartmentalize error handling into a function Evaluate results
Backup Script Discuss features and benefits Write a script to backup various files and directories Ensure that script is executed with appropriate number of positional parameters Auto-create backup file name using a unique date identifier Tar and compress (using gzip) backup items Incorporate error handling Evaluate results
Arrays Discuss features and applications Explain supported arrays Define indexed array with various items Reference individual and total array elements Evaluate results Build indexed array using command substitution Loop through array elements Define associative array with various items Expose total array elements
Regular Expressions Discuss features and applications Highlight key metacharacters Use character-classes Implement RegExes with shell commands: ls|grep|awk|sed Evaluate results
Kornshell on Solaris Compare & contrast Linux and Solaris Korn implementation Create Kornshell user on Solaris Identify key files Explore Solaris Kornshell as the user Customize Solaris Kornshell environment Run scripts created on Linux Evaluate results
Case Discuss features and benefits Explain usage
Create cases based on simple RegExes Write script to identify file types using case Explore case usage in system scripts Evaluate results
Job Control Discuss features and benefits Background and foreground jobs Interact with running jobs Isolate jobs from TTY Evaluate results
Typeset Discuss features and benefits Define variables of disparate types Justfiy variables Transform positional parameters and variables Enumerate system variables Scope variables in and out of functions Evaluate results
Input Validation Discuss features and benefits Ensure the requisite number of positional parameters Confirm input prior to execution Evaluate results
top related