shell.scripting.ver.0.0.1b.pdf

8/16/2019 shell.scripting.ver.0.0.1b.pdf

1/172

02.12.2010

1

Shell Scriptingfor System Administrators

Binnur Kurt, Ph.D.Omega Training and Consultancy www.omegaegitim.com

[email protected]

Shell Scripting 2

Course Overview

1. UNIX Shells and Shell Scripts

2. Writing and Debugging Scripts

3. The Shell Environment

4. Regular Expressions and the grep Command

5. Using the sed Editor

6. The nawk Programming

Language

7. Conditionals

8. Interactive Scripts

9. Loops

10. The getopts Statement

11. Advanced Variables, Parameters, and Argument Lists

12. Functions

13. Traps


2/172

02.12.2010

2

Shell Scripting 3

Course Information

> Duration

– 5 Days

> Course Hours

– 9:30‐12:30

– 13:30‐16:30

Shell Scripting 4

Background

1995, B.Sc., ITU Computer Engineering

1997, M.Sc., ITU Computer Engineering

2007, Ph.D., ITU Computer Engineering

1995‐2003, RA, ITU CE

2004‐2008,

Lecturer, ITU

CE

2008‐to date, Trainer, Omega Training & Consultancy

2008‐to date, Visiting Faculty Member, ITU CE

2004‐to date, Ahmet Yesevi Üniversitesi, Uzaktan E ğitim


3/172

02.12.2010

3

Shell Scripting 5

> Java Desktop and Enterprise Technologies

> Composite Application Development

> Software Engineering

> OO Analysis and Design

> MySQL, Oracle

> Sun Java Directory Server, Application Server

> Digital Image & Video Analysis and Processing

> Real‐Time Computer Vision Systems

Areas of

Interest

Shell Scripting 6

Trainings Offered

C/C++

Fortran 2003

OS

Programming

Database

ASWebLogic Server


4/172

02.12.2010

4

Shell Scripting 7

Java EEFrameworks

Trainings Offered

Shell Scripting 8

Lectures

•Internet Programming•Object Oriented Analysis and Design•Web Application Services

•Internet Web Servisleri


5/172

02.12.2010

5

Shell Scripting 9

My Certifications

redhat CERTIFIED INSTRUCTOR

Shell Scripting 10

My Recent Books

•B.Kurt, Z.Çataltepe, “Platforms for Enterprise Servers,” pp. 209‐223, Chapter 9, CRC Press, 2010.


6/172

02.12.2010

6

Shell Scripting 11

My Recent

Books

•F.Kahraman, B.Kurt, M.Gökmen, “Robust Face Alignment for Illumination and Pose Invariant Face Recognition,” InTech, April 2010.

Shell Scripting 12

Contact

[email protected]

http://www3.itu.edu.tr/~bkurt

http://www.omegaegitim.com


7/172

02.12.2010

7

Module 1UNIX Shells and Shell Scripts

Shell Scripting UNIX Shells and Shell Scripts 141

Objectives

> Describe the role of shells in the UNIX environment

> Describe the standard shells

> Define shell script components

> Write a simple shell script


8/172

02.12.2010

8


What is

an

Operating

System?

> An elephant is a mouse with an operating system.


What is an Operating System?

> A piece of software that provides a convenient, efficient environment for the execution of user programs.

Hardware

Operating systems

Kernel

Applications software

Users


9/172

02.12.2010

9


Resource Abstraction

and

Sharing

> Hides the details of how the hardware operates

> Provides an abstract model of the operation of hardware components—generalize hardware behavior

—limit the flexibility in which hardware can be manipulated

> Computation model: processes, threads

>

Resources:

memory,

disk,

files,

cpu,

etc.


Why do we need an operating system?

> User viewpoint ─ provide user interface, command interpreter,

directory structure, utility programs (compilers, editors, filters)

> Program environment viewpoint ─ enhance the bare machine

higher‐level I/O, structure files, notion of independent processes,

improved store

(size,

protection)

> Efficiency viewpoint ─ replace a human operator scheduling jobs,

storing I/O (files), invoking necessary programs such as compiler

> Economic viewpoint ─ allow concurrent uses and good scheduling of

resources

So, the goals are to make the system convenient to use (via system

calls) and to manage resources efficiently.


10/172

02.12.2010

10


UNIX


Kernel

> The kernel is the core of the OS

> Manages all the physical resources of the computer

> After the shell passes the commands, the kernel executes the commands.


11/172

02.12.2010

11


What Is

a Shell?

> An interface between the user and the operating system (kernel)

> A command‐line interpreter

> A utility program

> A program started for each user when they log in or open a command or tool window

> A way for the user to execute utilities and other

programs


What Are a Shell’s Functions?

> Command‐line interpreter

> Programming language

> User environment

Shell Shell Script

command statement

interprets e x e c u t e s

c o n t a i n s


12/172

02.12.2010

12


Available Shells

> Bourne Shell sh

> Korn Shell ksh

> C Shell csh

> TC Shell tcsh

> B(ourne)A(gain) Shell bash

> Desktop Korn Shell dtksh

>

Job

Control

Shell

jsh> Restricted Shell rsh

> Z Shell zsh


Bourne Shell (sh)

> Written by Steve R. Bourne at AT&T Bell Labs

> Preferred shell for system shell programming

—Compactness

—Speed

> Default shell

in

Solaris

> Lacks several features

—Ability to recall previous commands

—Built‐in arithmetic and logical expression

> Default prompt for a non‐root user: $

> Default prompt for a root user: #


13/172

02.12.2010

13


C Shell

(csh)

> Written by Bill Joy at University of California, Berkeley

> Interactive features

—Ability to recall previous commands

—Built‐in arithmetic and c‐like expression

—Aliases

> Slower than Bourne Shell

> Default prompt for a non‐root user: %

> Default prompt

for

a root

user:

#


Korn Shell (ksh)

> Written by David Korn at AT&T Bell Labs

> Superset of the Bourne Shell

> Comparable to C Shell

> Programming features supported

—Ability to

recall

previous

commands

—Built‐in arithmetic, String manipulation

—C‐like Arrays

—Functions

> Faster than C Shell

> Runs scripts written for the Bourne Shell

> Default prompt for a non‐root user: $

> Default prompt for a root user: #


14/172

02.12.2010

14


Subshells – Child

Processes

> Compiled executable script

—Creates a child process

—The parent waits

—The child executes

> Shell script

—Invoke with a script name

—The child process is a shell

—The parent waits

—The child

executes

sh

ksh

csh

ksh

csh

exit

exit


Subshells – Child Processes (Contd .)

> Invoke with dot command

—The script runs in current shell

> Invoke with source command

> The script runs in current shell


15/172

02.12.2010

15


What Is

a Shell

Script?

> Contains shell and UNIX commands

—Specific purpose

—Reusable

—Executed like any command


What Is a Shell Script?

Shell scripts fall into three basic categories

> Automation Scripts

—repeated series of commands

> Encapsulation Scripts

—storing complex

commands

under

a simple

name

> Computation Scripts

—deriving information from data


16/172

02.12.2010

16


Programming Terminology

> Logic flow

> Loop

> User input

> Conditional branch

> Command control


Add a User

> useradd –d /export/home/kurt –m –s /bin/bash kurt

> passwd kurt


17/172

02.12.2010

17


Practice Session

#1

> How to learn which shell you are using

$ ps

or

$ pgrep –l –u user –n 'sh$'


Practice Session #2

> How to learn the default shell

$grep kurt /etc/passwd

or

$echo $SHELL


18/172

02.12.2010

18


Practice Session #3

> How to switch to another shell (e.g. ksh)

$ksh

or

$exec ksh

or

$exec /bin/ksh


Practice Session #4

> How to write number of files and number of directories in the current directory

first.sh:

#!/bin/shset -- *

files=$#

set -- */

dirs=$#

echo “Number of files in $PWD: $filesincluding $dirs directories”


19/172

02.12.2010

19

Module 2Writing and Debugging Scripts

Shell Scripting Writing and Debugging Scripts 382

Objectives

> Start a script with #!

> Put comments in a script

> Change permissions on a script

> Execute a script

> Debug a script


20/172

02.12.2010

20


Creating Shell

Scripts

> Create a file using any text editor

> Put UNIX commands, user programs, or the names of

other scripts as commands in the file

> Save the file, and exit from the editor


Creating Shell Scripts$ vi firstscript.sh

#!/bin/sh

clear

echo "SCRIPT BEGINS"

echo "Hello $LOGNAME!"

echo

echo "Todays

date

and

time:

\c"

date

echo

mynum=21

myday="Monday"

echo "The value of mynum is $mynum"

echo "The value of myday is $myday"

echo

echo "SCRIPT FINISHED!!"

echo


21/172

02.12.2010

21


Executing a Shell

Script

> Give the script execute permission

chmod u+x firstscript.sh

> Execute the script as a command

./firstscript.sh

echo $mynum

echo ${mynum}

>

Create

a

subshell

in

which

to

execute

the

scriptsh firstscript.sh

echo $mynum


Executing a Shell Script (Contd .)

> Execute the script using dot command

. firstscript.sh

echo $mynum

> Execute the script using source command

source firstscript.sh

echo $mynum


22/172

02.12.2010

22


Starting a Script

With

the

#!

Characters

> For a system boot script, use Bourne shell (/sbin/sh)

—Only the root file system may be mounted

> For a typical script use the shell that is most comfortable

to you or the one that supports all the features you desire

#!/bin/sh

#!/bin/csh

#!/bin/ksh


Putting Comments in a Script

> It is a good practice to put comments into programs

and shell scripts

> The comments should explain any specific lines that

might be confusing

> Comments does not affect the execution of the script unless a syntactical error is introduced when the

comments are added


23/172

02.12.2010

23


Putting Comments

in

a Script

$ cat scriptwithcomments.sh#!/bin/sh# This script clears the window, greets the user,# and displays the current date and time.clear # Clear the windowecho "SCRIPT BEGINS"echo "Hello $LOGNAME!" # Greet the userechoecho "Todays date and time: \c"date # Display current date and timeecho

mynum=21 # Set

a local

shell

variable

myday="Monday" # Set a local shell variableecho "The value of mynum is $mynum"echo "The value of myday is $myday"echoecho "SCRIPT FINISHED!!"echo


Putting Comments in a Script

$ ./scriptwithcomments

SCRIPT BEGINS

Hello root!

Todays date

and

time:

Tue

Nov

24

15:01:22

IST

2009

The value of mynum is 21

The value of myday is Monday

SCRIPT FINISHED!!


24/172

02.12.2010

24


Adding the

Debugging

Statement

> To run an entire script in debug mode, add -x after the

#!/bin/ksh on the first line:

#!/bin/ksh -x

> To run an entire script in debug mode from the command

line, add a -x to the ksh command used to execute the

script:

$ ksh -x script_name

> To run debug with options, use ‐x, ‐v, or ‐f


Debug Mode Controls

> -x displays the line after interpreting metacharacters and

variables

> -v displays the line before interpreting metacharacters

and variables

> -f disables file

‐name

substitutions

> set - option turns on the option

> set + option turns off the option


25/172

02.12.2010

25


Example: Debug

Mode

Specified

on

#!

Line

$ cat debug1.sh

#!/bin/sh ‐x

echo "Your terminal type is set to: $TERM"

echo

echo "Your login name is: $LOGNAME"

echo

echo

"Now

we

will

list

the

contents

of

the

/etc/security

directory"

ls /etc/security

echo


Example: Debug Mode Specified on #! Line

$ ./debug1.sh+ echo Your terminal type is set to: ansiYour terminal type is set to: ansi+ echo

+ echo Your login name is: root

Your login

name

is:

root+ echo

+ echo Now we will list the contents of the /etc/security directoryNow we will list the contents of the /etc/security directory+ ls /etc/securityaudit audit_user dev policy.conf

audit_class audit_warn device_policy priv_names

audit_control auth_attr exec_attr prof_attr

audit_event bsmconv extra_privs spool

audit_record_attr bsmunconv kmfpolicy.xml tsol

audit_startup crypt.conf lib


26/172

02.12.2010

26


Example: Debug

Mode

with

set –x Option

$ cat debug2.sh

#!/bin/sh

set ‐x


echo

set +x


echo

echo "Now

we

will

list

the

contents

of

the

/etc/security

directory"

ls /etc/security

echo


Example: Debug Mode with set –x Option

$ ./debug2.sh

+ echo Your terminal type is set to: ansi

Your terminal type is set to: ansi

+ echo

+ set

+x

Your login name is: root

Now we will list the contents of the /etc/security directoryaudit audit_user dev policy.conf







27/172

02.12.2010

27


Example: Debug

Mode

with

set –v Option

$ cat debug3.ksh

#!/bin/ksh

set ‐v


echo


echo

echo "Now

we

will

list

the

contents

of

the

/etc/security

directory"

ls /etc/security

echo


Example: Debug Mode with set –v Option$ ./debug3.kshecho "Your terminal type is set to: $TERM"Your terminal type is set to: ansiecho


Your

login

name

is:

rootecho

echo "Now we will list the contents of the /etc/security directory"Now we will list the contents of the /etc/security directoryls /etc/securityaudit audit_user dev policy.conf







28/172

02.12.2010

28


Born Shell

and

Korn

Shell

Debug

Modes

Born and Korn Options Korn Shell Specific Options

set –x set –o xtrace

set +x set +o xtrace

set –v set –o verbose

set +v set +o verbose

set –f set

–o

noglobset +f set +o noglob


Example: Debug Mode with set –o noglob

$ cat debug4.ksh

#!/bin/ksh


echo


echoset ‐o noglob

echo "Now we will list the contents of the /etc/security directory"

ls /etc/secur*

echo

set +o noglob

echo "Now we will list the contents of the /etc/security directory"

ls /etc/secur*

echo


29/172

02.12.2010

29


Example: Debug

Mode

with

set –o noglob

$./debug4.ksh

Your terminal type is set to: ansi

Your login name is: root

Now we will list the contents of the /etc/security directory

/etc/secur*: No such file or directory

Now we will list the contents of the /etc/security directory

audit

audit_user dev policy.conf audit_class audit_warn device_policy priv_names






set –o noglob

set –o noglob

ls *.avi

find . –name "*.avi" –print

set

+o

noglob


30/172

02.12.2010

30

Module 3The Shell Environment

Shell Scripting The Shell Environment 603

Objectives

> Use Bourne and Korn shell variables

> Assign values to shell variables

> Display the value of shell variables

> Make variables available to subprocesses using the export

command

> Display the value of environment variables

> Unset shell and environment variables

> Customize the user environment using the .profile file


31/172

02.12.2010

31


Objectives> Perform arithmetic operations

> Create and use aliases

> Display aliases and the values assigned to them

> Define the built‐in aliases

> Customize the Bourne and Korn shell environments

> Use the tilde expansion and command substitution

features of the Korn shell


User Startup Scripts

> / et c/ pr of i l e script

—runs first when a user logs in.

> $HOME/ . pr of i l e script

—runs second when a user logs in.

> $HOME/ . kshrc script

—runs third if the ENV variable is set.


32/172

02.12.2010

32


Working With

User

Initialization

Files

> Based on the shell, there might be either one or two

default initialization files in your home directory that

enable you to customize your working environment.


User Startup Scripts

> Example . pr of i l e script

$ cat /.profile

# This file initially did NOT exist for rootMANPATH=$MANPATH:/usr/share/man:/usr/dt/share/man:/usr/java1.2/man

ENV=$HOME/.kshrc

EDITOR=vi

export MANPATH ENV EDITOR


33/172

02.12.2010

33


User Startup

Scripts

> To change to the Korn shell, add these lines:

SHELL=/usr/bin/ksh #The variable determines default shell

# for subshells windows

export SHELL

/usr/bin/ksh #Invokes a Korn shell as a child of login shell

> Example .kshrc script$ cat .kshrc

PS1=”‘hostname‘ ! $”

set ‐

o trackall

alias l=’ls ‐laF’

alias ls=’ls ‐aF’

alias hi=’fc ‐l’

alias c=clear


Shell Variables

> Shell variables are capitalized by convention

> The shell maintains two list of variables

—Local to the current shell

—Global to all shells

> Use the set and env commands to display local and environmental variables


34/172

02.12.2010

34


$ set

AB2_DEFAULTSERVER=http://docs.sun.com/CUE_HOSTNAME=sunray10DISPLAY=:46.0DOMAIN=renegades.Central.Sun.COMDOMAIN_COUNT=1DTSOURCEPROFILE=trueDTUSERSESSION=milner‐sunray10‐46EDITOR=viERRNO=25FCEDIT=/bin/edLANG=CLOGNAME=milnerMAIL=/var/mail/milnerMAILCHECK=600MANPATH=/usr/dt/man:/usr/man:/usr/openwin/share/man:/usr/man:/usr/openwin/share/man:/usr/dt/man:/usr/dist/local/man/5.7PRINTER=hutchencePS1=’$

’

PS2=’> ’PS3=’#? ’PS4=’+ ’PWD=/home/milner/K‐Shell‐Course/InstructorGuide/ExamplesSHELL=/bin/kshTZ=US/Mountain...


$ envDTSOURCEPROFILE=trueDOMAIN=renegades.Central.Sun.COMDTUSERSESSION=milner‐sunray10‐46EDITOR=viLOGNAME=milnerMAIL=/var/mail/milnerCUE_HOSTNAME=sunray10PRINTER=hutchenceDISPLAY=:46.0TERM=dttermTZ=US/MountainLPDEST=hutchenceDOMAIN_COUNT=1...


35/172

02.12.2010

35


Introducing Korn

Shell

Variables

> A variable refers to a temporary storage area in memory.

Variables contain:

—Information needed for customizing the shell

—Information needed by other processes to function

properly

> The shell enables you to store values in variables.

> Korn shell programming uses two types of variables:

—Variables that are exported to sub‐shells

—Variables that are not exported to sub‐shells


Korn Shell Commands for Variables

> Korn shell commands used to set, unset, and view

variables:


36/172

02.12.2010

36


Creating Variables

in

the

Shell

> Create the variable, and give it a value with var=value

> Display the value of a variable with echo $var

$x=108

$name=“John Locke”

$echo $name $x

$export name

$ksh

$print $x

$print $name

John Locke


Creating Variables in the Shell

Make the variable known to subshells with export

> var=value

export var

> var=value; export var

> export var=value


37/172

02.12.2010

37


Creating Variables

in

the

Shell

> Unset the value, and release the variable with unset

$name=“John Locke”

echo $name

unset name

echo $name


Creating Variables in the Shell$var=x

$echo ${var:-y}

x

$var=

$echo ${var:-y}

y

$unset var

$echo ${var:=q }

q

$echo ${var:=z}

q


38/172

02.12.2010

38


Creating Variables

in

the

Shell

$for f in 4 8 15 16 23 42

do

list=${list:+$list,}$f

done

echo $list


Creating Variables in the Shell

$list= "a b c d e f“

$echo $list

$echo "$list"

$echo "$list\n" $list

$printf “%d %x %o\n" 15 15 15


39/172

02.12.2010

39


Exporting Variables

to

Subshells

> Created variables are local unless exported

> Subshells of a shell do not automatically inherit the

variables of their parent shell

> Only the variables that are exported are passed onto the

subshells

> Environment variables are passed to subprocesses

> var=value

export var

> var=value; export var

> export var=value


Reserved Variables

> Be careful about changing the values of these variables.

> The shell uses these variables.

> For a complete list of reserved variables, read the man

page for sh or ksh.


40/172

02.12.2010

40


Default Korn

Shell

Variables


Customizing Korn Shell Variables

> The PS1 prompt variable

> The PATH variable

> Extending the PATH variable


41/172

02.12.2010

41


The PS1

Prompt

Variable

> The shell prompt string is stored in the shell variable PS1,

and you can customize it according to your preferences.

$ PS1=“$LOGNAME@

uname ‐n

\$PWD $”

student@host1: $


The PS1 Prompt Variable

$ ILU=“I like UNIX”

$ PS1=“$ILU >”

I Like UNIX > echo $ILU

I like UNIX

I like UNIX >


42/172

02.12.2010

42


The PATH

Variable

> The PATH variable contains a list of directory path names,

separated by colons.

> When you perform a command on the command line, the

shell searches these directories from left to right to locate

that command.

> The shell executes the first command that it finds.

> If the shell does not find the command in any of the listed

directories, it displays

the

following

error

message:

ksh:command_name: not found


Extending the PATHVariable

$ echo $PATH

/usr/dt/bin:/usr/openwin/bin:/usr/bin

$

$ PATH=$PATH:~

$

$ echo $PATH

/usr/dt/bin:/usr/openwin/bin:/usr/bin:/export/home/student


43/172

02.12.2010

43


Special Shell

Variables

> $ contains the process identification number of the

current process

echo $$

> ? contains the exit status of the most recent foreground

process

ls /nodir

echo $?

> !contains the process ID of the last background job started

$xcalc &

$echo $!


Special Shell Variables

$true; $echo $?

0

$false; $echo $?

1


44/172

02.12.2010

44


Special Shell

Variables

> # tells the string length of the variable$var=LENGTH

$echo ${#var}

6

> % Remove the Shortest Match from the End$var=/usr/local/bin/crafty

$echo ${var%/*}

/usr/local/bin> %% Remove the Longest Match from the End

$var=usr/local/bin/crafty

$echo ${var%%/*}

usr


Special Shell Variables

> # Remove the Shortest Match from the Beginning


$echo ${var#*/}

local/bin/crafty

> ## Remove the Longest Match from the Beginning


$echo ${var##*/}

crafty


45/172


46/172

02.12.2010

46


Using the

Quoting

Characters

> Quoting is a process that instructs the shell to mask, or

ignore, the special meaning of meta characters.

> The quoting characters are

—single forward quotation marks (’ ’),

—double quotation marks (" "),

—backslash (\),

—parentheses ($ (command)).


Using the Quoting Characters

> Quotation marks around a string of meta characters

prevent the shell from interpreting the special meaning of

the meta characters.

> There are two types of quotation marks that mask the

special meaning

of

meta

characters:

—single forward quotation marks (’ ’): instruct the shell to

ignore all enclosed meta characters.

—double quotation marks (" ").


47/172

02.12.2010

47


Using the

Quoting

Characters

> Double quotation marks instruct the shell to ignore all

enclosed metacharacters, except for the following three

characters:

—The single backward quotation marks (‘) – Single

backward quotation marks instruct the shell to execute

and display the output for a UNIX system command.

—The backslash (\) – A backslash (\) character in front of

a meta

character

prevents

the

shell

from

interpreting

the next character as a meta character.


Using the Quoting Characters

> Double quotation marks instruct the shell to ignore all

enclosed metacharacters, except for the following three

characters:

—The dollar sign ($) – You can also use parentheses () to

perform command

substitution.

Parentheses

($

(command)) instructs the shell to execute and display

the output for the command enclosed within.


48/172

02.12.2010

48


Using the

Quoting

Characters

$ echo ’$SHELL’

$SHELL

$ echo "$SHELL"

/bin/ksh

$ echo "\$SHELL"

$SHELL

$

echo

“Today’s

date

is

‘date‘”Today’s date is Wed Jan 20 21:26:33 IST 2008

$ echo “The user is currently in the $(pwd) directory.”


Shell Command Substitution

> The Bourne shell uses ‘‘ (backquotes).

> The Korn shell supports the older Bourne shell syntax.

> The Korn shell uses the $(command ) syntax.


49/172


50/172

02.12.2010

50


Arithmetic

Operations

on

Korn

Shell

Variables

> Arithmetic evaluation is invoked by placing an integer

expression within two pairs of parentheses:

(( ... ))

> All calculations are performed using integer arithmetic.

Operator Operation Example Result

+ Addition ((x = 24 + 25)) 49

- Subtraction ((x = 100 - 25)) 75

* Multiplication ((x = 4 * 5)) 20

/ Division ((x = 10 / 3)) 3

% Integer Remainder ((x = 10 % 3)) 1


Arithmetic Precedence

1. Expressions within parentheses are evaluated first.

2. *, %, and / have greater precedence than + and ‐.

3. Everything else is evaluated from left to right.


51/172

02.12.2010

51


Bit‐wise

Operations

> The > operator performs a binary shift right

> The & operator ANDs two binary numbers together

> The | operator ORs two binary numbers together

> The ^ operator exclusively ORs two binary numbers

together


Example: Bit‐wise Operations

$x=3

$y=5

$((x = x ^ y))

$((y = x ^ y))

$((x = x ^ y))

$echo $x

5

$echo $y

3


52/172

02.12.2010

52


The Korn

Shell

let Statement

> The let statement is an alternative to the ((...))

statement.

> Type the arithmetic formula with no spaces unless the

formula is enclosed in double‐quote (") characters.

> It is more common to use the Korn shell’s ((...)) syntax

instead of the let statement.


The Korn Shell let Statement

$let a=1

$let b=3

$let c=4

$echo $a

$let "a = a + 1"

$let a=b+c

$echo $a

$((a=b+c))

$echo $a

$a=b+c

$echo $a


53/172

02.12.2010

53


bc‐ Arbitrary

precision

calculator

language

> bc is a language that supports arbitrary precision numbers

with interactive execution of statements.

> There are some similarities in the syntax to the C

programming language.

> A standard math library is available by command line

option.

> Example:

$echo "scale=32;4*a(1)" | bc –l

$pi=$(echo "scale=10;4*a(1)" | bc -l)


Korn Shell Aliases

> An alias is a way of assigning a simple name to what might

be a long or complicated command or series of

commands.

> Variables hold data; aliases hold commands.

> Aliases can

specify

a version

of

a command

when

more

than one version of the command is on the system.


54/172

02.12.2010

54


Built‐in

Aliases

> functions=’typeset ‐f’

> history=’fc ‐l’

> integer=’typeset ‐i’

> nohup=’nohup’

> r=’fc ‐e ‐’

> suspend=’kill ‐STOP $$’


Built‐in Aliases: Example

$integer k

$k=abc

ksh: abc: bad number


55/172

02.12.2010

55


Shell Parse

Order

1. Read the command.

2. Evaluate the keyword.

3. Evaluate the alias.

4. Evaluate the built‐in commands.

5. Evaluate functions.

6. Evaluate tilde expansion.

7. Perform command substitution.

8. Perform arithmetic expression substitution.

9. Evaluate metacharacters for expansion of file names.

10. Loop up the Command or script.

11. Execute the command.

Module 4

Regular Expressions

and

the

grep Command


56/172

02.12.2010

56

Shell Scripting Regular Expressions and grep Command 1114

Objectives> Use and describe regular expressions

> Describe the grep command

> Use the grep command to find patterns in a file

> Use regular expression characters with the grep

command


The grep Command

> Globally searches for a regular expression, and prints the

results

> Searches text files for a specific pattern

> When pattern is found, the entire line is printed

> Regular expression

characters

are

permitted

in

the

search

pattern


57/172


58/172

02.12.2010

58


Regular Expression

Metacharacters


Regular Expression Metacharacters


59/172

02.12.2010

59


Regular Expressions


Using the grep Command

$cat /etc/passwd | grep root

$grep root /etc/passwd

$xclock &

$ps –fe | grep xclock


60/172

02.12.2010

60


Escaping a Regular

Expression

> A \ (backslash) character escapes the RE characters.

> It interprets the next character literally, not as a

metacharacter.

> A backslash gives special meaning to the next character,

such as \, $, and $.


Escaping a Regular Expression


61/172

02.12.2010

61


Escaping a Regular

Expression


Line Anchors

> Use ^ for the beginning of the line.

> Use $ for the end of the line.


62/172

02.12.2010

62


Line Anchors


Word Anchors

> Use \ Use \> for the end of the word.


63/172


64/172

02.12.2010

64


Character Classes


Single Character Match

> The . regular expression matches any one character except

the newline character.


65/172


66/172

02.12.2010

66


Using the

egrep Command

> The egr ep command searches the contents of one or

more files for a pattern using extended regular expression

meta characters.

> Extended regular expression meta characters include the

regular expression meta characters that the gr ep

command uses along with some additional

metacharacters.


Extended Regular Expression Metacharacters


67/172

02.12.2010

67


Using the

find Command

> The pat hname, expr essi on, and act i on arguments for the f i nd command are shown in the table.


Expression for the find Command

> Expr essi ons that you can use with the find command.


68/172

02.12.2010

68


Actions for

the

find

Command

> Act i on arguments for the find command.


Actions for the find Command

> Additional action arguments for the f i nd command:


69/172

02.12.2010

69


Using the

find Command

find /etc –name “pas*” –print

find / -user kurt –mtime -7 –size 100 –print

find / -name core –exec rm {} \;

Module 5

Using the

sed Editor


70/172

02.12.2010

70

Shell Scripting Using the sed Editor 1395

Objectives> Use the sed editor to perform non‐interactive editing tasks

> Use regular expression characters with the sed command


Introduction to the sed Editor

The sed editor:

> Is a stream editor

> Is nondestructive

> Is non‐interactive

> Uses regular expressions


71/172

02.12.2010

71


The sed Editor


Command Format

sed [options] '[addresses] action [args]' files [ > outfile]


72/172

02.12.2010

72


Editing Commands

> The sed Commands:

>

The

sed Options:


Addressing

> Specifies a single line number or range of line numbers

> Uses $ for last line of file

> Searches for a regular expression

> Delimits a regular expression with a forward slash


73/172

02.12.2010

73


Using sed

to

Print

Text


Using sed to Print Text


74/172

02.12.2010

74


Using sed

to

Substitute

Text

> Substitute a new string for an old string:

sed 's/oldstring/newstring/' file

> Use g to substitute globally

> Use & to include the oldstring in the newstring


Using sed to Substitute Text


75/172

02.12.2010

75


Using sed

to

Substitute

Text


Reading From a File of New Text

> Read in a file after the line containing the search

expression.

> The r (read) command is followed by the path name to the

file.


76/172

02.12.2010

76


Reading From

a File

of

New

Text


Using sed to Delete Text

> Delete lines containing the search expression

> Delete lines in the address range

> Do not delete lines using !


77/172

02.12.2010

77


Using sed

to

Delete

Text


Using sed to Delete Text


78/172

02.12.2010

78


Reading sed Commands

From

a File

> Place the commands in a file

> Use -f to tell sed to read the file


Using sed to Write Output Files

> The w (write) command writes the specified records to the

named file.

> The w command is followed by the path name of the file.


79/172

02.12.2010

79


Using sed to

Write

Output

Files

Module 6

The nawk Programming

Language


80/172

02.12.2010

80

Shell Scripting The nawk Programming Language 1596

Objectives> Use nawk commands from the command line

> Write simple nawk programs to generate data reports

from text files

> Write simple nawk programs to generate numeric and

text reports from text files


Introduction to nawk Programming Language

> Looks at data by records and fields

> Uses regular expressions

> Uses numeric and text variables and functions

> Uses command‐line arguments


81/172

02.12.2010

81


Format of

the

nawk Command

> Commands have the form:

nawk 'statement' input.file

> Scripts are executed with:

nawk -f scriptfile input.file


Using nawk to Print Selected Fields

> Command conventions:

—Enclose the command in single quotes

—Enclose the command in braces {}

> Specify individual records with $0

> Specify individual fields with $1, $2, $3, and so on


82/172

02.12.2010

82


Using nawk to

Print

Selected

Fields


Formatting With the print Statement

\t Tab

\n Newline

\007 Bell

\011 Tab

\012 Newline

\042 "

\044 $

\045 %


83/172

02.12.2010

83


Formatting With

the

print Statement


Using Regular Expressions

> Regular expression metacharacters can be used in the

pattern.

> There are two special patterns: BEGIN and END.


84/172

02.12.2010

84


Using Regular

Expressions


Using Regular Expressions


85/172

02.12.2010

85


The BEGIN and

END Special

Patterns

> BEGIN – An action to take before reading any lines

> END – An action to take after all lines are read and

processed


The BEGIN and END Special Patterns


86/172

02.12.2010

86


The BEGIN and

END Special

Patterns


Using nawk Scripts


87/172

02.12.2010

87


Using nawk Scripts


Using Built‐in Variables


88/172

02.12.2010

88


Working With

Variables

> Input field separator:

nawk -F:

nawk ’BEGIN {FS=":"}

> Output field separator:

nawk ’{print $3, $4 "\t" $2}’ data.file

nawk ’BEGIN {OFS="\t"}


User‐Defined Variables

> Variable names should not conflict with the nawk variable

or function names.

> Variables are automatically initialized to a null string.

> Variables used in arithmetic statements or functions are

initialized to

0.


89/172

02.12.2010

89


User‐Defined

Variables


User‐Defined Variables


90/172

02.12.2010

90


User‐Defined

Variables


Variable Examples


91/172

02.12.2010

91


Variable

Examples


Variable Examples


92/172


93/172

02.12.2010

93

Module 7Conditionals

Shell Scripting Conditionals 1867

Objectives

> Use the command exit status as conditional control

> Use the if statement to test a condition

> Pass values using command‐line arguments (positional

parameters) into a script

> Create USAGE messages

> Place parameters on the command line

> Use conditional if, then, elif, else, and fi

constructs

> Use exit, let and test statements


94/172

02.12.2010

94


Objectives> Apply the &&, ||, and ! Boolean logic operators

> Use the case statement


The if Statement

i f commandt hen

block_of_statements

f i

$ cat snoopy.sh#!/bin/sh

# Script name: snoopy.sh

name=snoopy

if [ "$name" = "snoopy" ]

then

echo "It was a dark and stormy night."

fi


95/172

02.12.2010

95


Parts of

the

if Statement

> Command

> Block of statements

> End of the if statement


Exit Status

> Every command, program, and shell statement executed

has an exit status associated with it.

> The exit status is an integer variable and is saved in the ?

shell reserved variable.

> A value

of

0 for

the

exit

status

indicates

the

command

ran

successfully (no errors occurred).

> A nonzero exit status indicates failure (one or more errors

occurred or the command could not accomplish what was

asked).


96/172


97/172

02.12.2010

97


Numeric and

String

Comparison

> Number comparison:

—Bourne and Korn: Use [] and spaces

—Korn: Use (( )) and spaces are optional

> String comparison:

—Bourne and Korn: Use [ ] and spaces

—Korn: Use [[ ]] and spaces


Syntax for the if/then/else Statements

if commandthen

bl ock of st at ement selse

bl ock of st at ement sfi

$ cat snoopynap.ksh#!/bin/ksh# Script name: snoopynap.kshname=snoopyif [[ "$name" == "snoopy" ]]thenecho "It was a dark and stormy night."

elseecho "Snoopy is napping."

fi


98/172

02.12.2010

98


Syntax

for

the

if/then/else/elifStatementsif command1then

bl ock of st at ement selif command2then

bl ock of st at ement selse

bl ock of st at ement sfi


Syntax for the if/then/else/elifStatements


99/172


100/172

02.12.2010

100


Using if to

Check

Leap

Years


Nested if Statements

> Any command within an if statement can be another if

statement.

> Each if statement requires its own fi statement.


101/172

02.12.2010

101






102/172

02.12.2010

102


Testing File

Objects


Testing File Objects


103/172

02.12.2010

103


Boolean AND,

OR,

and

NOT

Operators

> AND operator is &&

> OR operator is ||

> NOT operator is !


The case Statement

> Use case to test compare a value against multiple

patterns.


104/172

02.12.2010

104


Example of

Using

the

case Statement


Replacing Complex if Statements with thecase Statement


105/172

02.12.2010

105


Replacing

Complex

if Statements

with

thecase Statement


Replacing Complex if Statements with thecase Statement


106/172

02.12.2010

106


The exit Statement

> Terminates the execution of the entire script

> Can be used when:

—The requested input is incorrect

—The command ran unsuccessfully

—Some other error occurred

Module 8

Interactive Scripts


107/172

02.12.2010

107

Shell Scripting Interactive Scripts 2138

Objectives> Use the print and echo statements to display text

> Use the read statement to interactively assign data to a

shell variable

> Read user input into one or more variables using one

read statement

> Use special characters, with print and echo, to make

the displayed text more user friendly

> Use file

descriptors

to

read

from

and

write

to

multiple

files

> Create a here document


Input and Output in a Script

> Read command‐line arguments

> Print prompts

> Read input

> Test input

> Print error messages

> Output to a file

> Input from a file


108/172

02.12.2010

108


The Korn

Shell

print Statement

> The print statement’s options:

Option Meaning

‐n Suppresses the newline after printing the message. This

usually is used when printing a prompt for user input.

‐r Turns off the special meaning of the \ character.

‐R

Does not interpret the ‐ that follows as an option to the

print statement, except if ‐R is followed by n; that is, if

the ‐

n option

follows

‐R,

‐n is still

taken

as

an

option.

The

‐R option is useful if you need to print negative numbers.

‐‐

Same as ‐R, except that a following ‐n option is taken

literally.


The Korn Shell print Statement

> The print statement’s special characters:

Character Meaning

\nPrints a newline character, which enables you to

print a message on several lines using one print

command

\t Prints a tab character, which is useful when creating

tables or a report

\a Ring the bell on the terminal, which draws the

attention of the user

\b Specifies a backspace character, which overwrites

the previous character


109/172

02.12.2010

109


Examples of

Using

the

print Statement


Examples of Using the print Statement


110/172

02.12.2010

110


Examples of

Using

the

echo Statement


Examples of Using the echo Statement


111/172

02.12.2010

111


The read Statement

> The read statement reads input from standard input.

> The input is broken into tokens that are consecutive

characters that do not contain white space.

> The contents of the IFS variable are used as token

delimiters.

> The default value of IFS is a space, a tab, and a newline.

> The first token is saved as the value of the first variable.

> If there are more tokens than variables, the last variable

holds all remaining tokens.


The read Statement

> If there are more variables than tokens, then the extra

variables are assigned a null value.

> If no variable names are supplied to the read command,

then:

—The Korn

shell

puts

the

values

read

in

the

REPLY

variable

—The Bourne shell gives an error message, read:

missing arguments


112/172

02.12.2010

112


Examples of

Using

the

read Statement


Examples of Using the read Statement


113/172


114/172

02.12.2010

114


Printing a Prompt


Prompting for Input – Korn Shell Shortcut


115/172


116/172

02.12.2010

116


User‐Defined

File

Descriptors

exec fd > filename

exec fd & fd

command


117/172

02.12.2010

117


File Descriptors

in

the

Bourne

Shell


File Descriptors in the Korn Shell


118/172

02.12.2010

118


File Descriptors

in

the

Korn

Shell


The here Document


119/172

02.12.2010

119


The here Document

Module 9

Loops


120/172

02.12.2010

120

Shell Scripting Loops 2399

Objectives> Write scripts that use for, while, and until loops

> Write a script using the select statement

> Describe when to use loops within a script

> Generate argument lists using command, variable, and

filename substitution


Shell Loops

> Shell loops repeatedly execute a block of statements.

> The for loop executes for each value in a list.

> The while loop executes while a condition is true.

> The until loop executes until a condition is true.


121/172

02.12.2010

121


The for Loop

Syntax

for var in argument_list ...

do

statement1

...

statementN

done


The for Loop Argument List

> Use an explicit list

> Use the contents of a variable

> Use command‐line arguments

> Use command substitution

> Use file names in command substitution

> Use file‐name substitution


122/172

02.12.2010

122


Using an

Explicit

List

to

Specify

Arguments

for var in arg1 arg2 arg3 arg4 ... argn

for fruit in apple orange banana kiwi

do

print "Value of fruit is: $fruit"

done

> The output for this for loop is:

Value of fruit is: apple Value of fruit is: orange

Value of fruit is: banana

Value of fruit is: kiwi


Using the Contents of a Variable to SpecifyArguments


123/172


124/172

02.12.2010

124


Using Command Substitution to SpecifyArguments


Using Command Substitution to SpecifyArguments


125/172

02.12.2010

125


Using File Names in Command Substitutionto

Specify

Arguments


Using File Names in Command Substitutionto Specify Arguments


126/172

02.12.2010

126


Using File‐Name Substitution to SpecifyArguments


The while Loop

> As long as the command_control succeeds, the loop body

continues to execute.

while control_command

do

statement1

...

statementN

done


127/172

02.12.2010

127


The while Loop

Syntax

> While the contents of $var are equal to “value,” the

loop continues.

while [ "$var" = "value" ]

while [[ "$var" == "value" ]]

> While the value of $num is less than or equal to 10, the

loop continues.


The while Loop Syntax


128/172

02.12.2010

128


The while Loop

Syntax


Example of Using a while Loop


129/172

02.12.2010

129


Example of

Using

a while Loop


Keyboard Input


130/172

02.12.2010

130


Keyboard Input


Redirecting Input for a while Loop


131/172


132/172


133/172


134/172

02.12.2010

134


Example of

Using

the

continue Statement


Example of Using the continue Statement


135/172


136/172

02.12.2010

136


Example of

Using

a select Loop


Example of Using a select Loop


137/172

02.12.2010

137


Example of

Using

a select Loop


Exiting the select Loop


138/172

02.12.2010

138


Exiting the

select Loop


Exiting the select Loop


139/172

02.12.2010

139


Submenus> The select loop can contain another select loop.

> Include the break statement in the action to exit the

submenu loop.

> Reset the PS3 variable to the prompt needed when

moving from one menu to another.


Example Using Submenus


140/172


141/172

02.12.2010

141


The for and

select Statements

Revisited


The shift Statement

> Process the arguments to the script in a while loop.

> Use shift to process the next argument.


142/172


143/172

02.12.2010

143

Module 10The getopts Statement

Shell Scripting The getopts Statement 28610

Objectives

> Process script options using the getopts statement


144/172


145/172

02.12.2010

145


Handling Invalid

Options

> Precede the list of single‐character options with a colon;

for example:

while getopts :xy opt_char

> The beginning colon:

—Sets the value of the opt_char variable to ?

—Sets the value of the OPTARG reserved variable to the

name of the invalid option


Specifying Arguments to Options

> Place a colon (:) immediately after the option character in

the getopts statement.

while getopts :x:y opt_char


146/172


147/172

02.12.2010

147


Forgetting an

Argument

to

an

Option

Module 11

Advanced Variables,

Parameters, and Argument Lists


148/172


149/172


150/172


151/172

02.12.2010

151

Shell Scripting Advanced Vars, Params, and Arg. Lists 30111

Example of

Using

String

Manipulation


Declaring an Integer Variable

> Example 1:$ typeset -i num $ num=5$ print $num

5> Example 2:$ typeset -i num $ num=25.34$ print $num 25


152/172

02.12.2010

152


Declaring an

Integer

Variable

> Example 3:$ typeset -i num # base 10 integer$ num=27$ print $num 27


Declaring an Integer Variable

$ typeset -i8 num # change to base 8

$ print $num

8#33

$ num=two

/ usr / bi n/ ksh: t wo: bad number

$ print $num

8#33


153/172


154/172

02.12.2010

154


Creating Bourne

Shell

Constants

readonly var[=value]

$ sh

$ var=constant

$ readonly var

$ unset var

var : i s r ead onl y

$ var=new_valuevar : i s r ead onl y


Creating Korn Shell Constants

typeset -r var [=value]

readonly var [=value]

$ ksh

$ typeset -r cvar=constant

$ unset cvar

ksh: cvar : i s r ead onl y

$ cvar=new_value

ksh: cvar : i s r ead onl y


155/172

02.12.2010

155


Removing Portions

of

a String


Examples of Removing Portions of a String

stringx=/usr/bin/local/bin

print ${stringx%/bin}

/ usr / bi n/ l ocal

print ${stringx%/bin*}


print ${stringx%%/bin}


print ${stringx%%/bin*}

/ usr


156/172

02.12.2010

156


Examples of

Removing

Portions

of

a String

print ${stringx#/usr/bin}

/ l ocal / bi n

print ${stringx#*/bin}

/ l ocal / bi n

print ${stringx##/usr/bin}

/ l ocal / bi n

print ${stringx##*/bin} print ${stringx##*/}

bi n


Korn Shell Arrays

> Korn shell arrays contain more than one value.

> Korn shell arrays are created when you use them.

> By default, each value is a string.


157/172

02.12.2010

157


Examples of

Using

Arrays

> To create an array of three strings:

arr[0]=big

arr[1]=small

arr[2]="medium sized"

> To create an array of three strings using the set command:set -A arr big small "medium sized"

> To create an array of five integers:integer num

num[0]=0num[1]=100

num[2]=200

num[3]=300

num[4]=400


Examples of Using Arrays

> To print the number of array elements in array num:

$ print ${#num[*]}

5

> To print the values of all array elements in array arr :

$ print ${arr[*]}bi g smal l medi um si zed

> To unset the array arr :

unset arr


158/172

02.12.2010

158


Using

shift Statement

with

Positional

Parameters


Using shift Statement with Positional Parameters


159/172

02.12.2010

159


Using

shift Statement

with

Positional

Parameters


Assigning Positional Parameter ValuesUsing the set Statement


160/172

02.12.2010

160


Example of

Using

the

set Statement


Example of Using the set Statement


161/172

02.12.2010

161


The

Values

of

the"$@"

and

"$*"

PositionalParameters

> The values of $@ and $* are identical, but the values of

"$@" and "$*" are different.

> "$@" expands to "$1" "$2" "$3" ... "$n"; that is, nseparate strings.

> "$*" expands to "$1x$2x$3x...$n", where x is the

first character in the set of delimiters for the IFS variable,

which

means

that

"$*"

is

one

long

string.

Module 12

Functions

shell.scripting.ver.0.0.1b.pdf

Documents

Transcript of shell.scripting.ver.0.0.1b.pdf