Tricks and Tips of UNIX Shell Programming

1. Getting timestamp of a file creation:

#!/bin/sh FILE_TO_CHECK="/logs/mylog.log" FILE_DATE=`ls -lrt ${FILE_TO_CHECK} | tr -s " " | cut -d" " -f6-8` CONVERTED_FILE_DATE=$(date -d "${FILE_DATE}" '+%Y%m%d') echo ${CONVERTED_FILE_DATE}

Returns date in format: 20101209

2. Comparing timestamps in format HHMM (like: 1201 or 0901, etc.) with NOW

We can get time (HHMM) part of the date and compare it to another timestamp as integer values. We just need to make sure that leading 0s are processed properly (for timestamps like 0000-0959).
This can be achieved by the following code: echo 0020 | sed 's/^0*//'.
Returned result will be: 20

For those, not familiar with regular expressions, the expression provided for sed means - replace one or more 0s at the beginning of the line with empty string - i.e. delete leading 0s.

#!/bin/sh NEEDED_TIME_HHMM=0001 CURR_DATETIME=`date '+%Y%m%d%H%M%S'` CURR_TIME_HHMM=`date '+%H%M'` CURR_DATE=`date '+%Y%m%d'` if [[ $(echo ${NEEDED_TIME_HHMM} | sed 's/^0*//') -le \ $(echo ${CURR_TIME_HHMM} | sed 's/^0*//') ]] then echo “${NEEDED_TIME_HHMM} le ${CURR_TIME_HHMM}” else echo “NOT ${NEEDED_TIME_HHMM} le ${CURR_TIME_HHMM}” fi echo “Compared $(echo ${NEEDED_TIME_HHMM} | sed 's/^0*//') \ and $(echo ${CURR_TIME_HHMM} | sed 's/^0*//')”

3. Starting a script in the background with output going to a log file

A script can be started in the background as follows:

./run_mycode.sh $1 > /var/tmp/logs/run_mycode.log 2>&1 &

where
- "$1" is a parameter (can be more: $2, $3, etc.)
- "> /var/tmp/logs/run_mycode.log" redirects output into the log file
- "2>&1" redirects STDERR to STDOUT, so that error messages will be visible in the same log with other messages. This is especially important when you start SYBASE isql in your script, as print command sends output to STDERR while select sends output to STDOUT. Further explanations regarding redirection in UNIX can be found here.
- final "&" means "run in the background"

4. To clean all the files with name like *.log older than 7 days, run:

VCS_LOG_DIR=/mydirectory
    find ${VCS_LOG_DIR} -name "*.log" -mtime +7 -exec rm -f {} \;

if you want to delete all the files up-to 7 days old, replace +7 for -7

5. Working with a list of values in a shell script

Here is a simple example that illustrates how to use a list of values saved in a file.
First create a file config_list.txt and populate it with values as follows

myproc_a
myproc_b
myproc_c
myproc_d

Now create the script below and execute it

#!/bin/sh SCRIPT_DIR=. CONFIG_DIR=. SERVERS=`cat ${CONFIG_DIR}/config_list.txt` for SERVER in $SERVERS do count=`expr $count + 1` echo "Monitoring Server ${count} Named ${SERVER}..." done

The output will look like

Monitoring Server 1 Named myproc_a...
Monitoring Server 2 Named myproc_b...
Monitoring Server 3 Named myproc_c...
Monitoring Server 4 Named myproc_d...

NOTE: Variable $count in your script counts the items in your list.

6. Forcing symbolic link

If you need to force an existing symbolic link to a different directory you can do it as follows

ln -s myOldDir myLink

$ ls -lrt
drwxr-xr-x 9 user1 users     4096 May  9 18:24 myOldDir
drwxr-xr-x 9 user1 users     4096 May  9 18:24 myNewDir
lrwxr-xr-x 9 user1 users       22 May  9 18:24 myLink -> myOldDir

Now force the relinking
ln -sfn myNewDir myLink

$ ls -lrt
drwxr-xr-x 9 user1 users     4096 May  9 18:24 myOldDir
drwxr-xr-x 9 user1 users     4096 May  9 18:24 myNewDir
lrwxr-xr-x 9 user1 users       22 May  9 18:24 myLink -> myNewDir

7. Using awk to calculate a sum of values in a file

Let's say we have a file with the data like below

testdata 1 testdata000000000000001.11 sfd
testdata 1 testdata000000000000002.22 sfd
testdata 1 testdata000000000000003.33 sfd
testdata 1 testdata000000000000001.01-sfd
testdata 1 testdata000000000000001.01-sfd

A numeric value is written in positions 20-37, a sign is in position 38, space means positive number
The command below will calculate the sum of the numbers. I had to split it into 2 lines to fit into this area, hence the \ in the end of the first line

awk '{ s=substr($0,38,1); $n=substr($0,20,18)+0; var=sprintf("%.2f", s $n); \
sum=sum+var;} END { printf("%.2f", sum);}' testdata.txt

Returns 4.64

8. Dynamic Variables (Variable Variable) in bash and ksh

Below are two scripts that demonstrate usage of dynamic variables in bash and ksh.

#!/bin/bash VAR_AAA=text_for_aaa VAR_BBB=text_for_bbb TYPE=AAA VAR_DYNO=VAR_${TYPE} echo ${VAR_DYNO}=${!VAR_DYNO}

#!/bin/ksh VAR_AAA=text_for_aaa VAR_BBB=text_for_bbb TYPE=AAA typeset -n VAR_DYNO="VAR_${TYPE}" echo VAR_${TYPE}=${VAR_DYNO}

Both scripts return VAR_AAA=text_for_aaa

9. Using an array to verify connectivity to multiple hosts

If you need to check if you have connectivity to a list of hosts you can use the following script

#!/bin/sh    funtln () { telnet ${SRV} << EOO quit EOO } SERVERS=( '192.168.1.105:443' '192.168.1.105:80' '192.168.1.105:443') echo ${SERVERS[*]} for SERVER in ${SERVERS[@]} do count=`expr $count + 1` echo "Testing Server ${count} Named ${SERVER}..." SRV=`echo ${SERVER} | sed 's/:/ /'` funtln done

10. Using Dynamic Array Variables in bash

If you need to check a log file for patterns used by some applications. Each of the applications can leave different patterns in a log file. You can use dynamic array variables in bash for this. Here is how.

Create a config file /tmp/test.conf with the following contents

APPS=("APP1" "APP2" "APP3") APP1=("A1_123" "A1_456") APP2=("A2_987") APP3=("A3_123" "A3_456")

APPS is a list of applications, APP1, APP2, APP3 are the lists of patterns that we want to check for each application.

The code will look like following

#!/bin/bash . /tmp/test.conf for APP in ${APPS[@]} do VAR_APP=${APP} echo "----Checking for ${VAR_APP}" eval TAGS=( '"${'${APP}'[@]}"' ) for TAG in "${TAGS[@]}" do echo "TAG=${TAG} " #Put grep ${TAG} mylogfile | wc -l here to get the counts done done

We source the config file, then go through the APPS array to get the names of the arrays that contain the patterns.

If you run this program you will get the following output

----Checking for APP1
TAG=A1_123
TAG=A1_456
----Checking for APP2
TAG=A2_987
----Checking for APP3
TAG=A3_123
TAG=A3_456