lunedì 27 gennaio 2014

Unix, check folder consistency

In my work the application I manage move using MFT solution.
Sometimes the received files need a local post processing (de/compression, de/cypher ...) and then software have to route managed file.
Always received file are moved by post processing in other server or area.
Each day I  have to be sure all the received file are routed to the target system.
Follow my solution to this issue, using Korn Shell.

All the received file "landing" in subfolder of a WorkArea. Each day the scheduler run my script:
#!/bin/ksh
# Shell CheckDir.ksh
# Check file in the work area
#- DEFINITION GLOBAL VARIABLES ---------------------------------------------------------------------------------
WORKDIR=/cft_work_prod         # Work Directory
LOG=/batch/log/ControllaDir.log     # LogFile
LOGA=/batch/log/ControllaDir.loga     # Temp  LogFile
LOGB=/batch/log/ControllaDir.logB     # Temp LogFile

EXCEPTIONS=/batch/bin/ControllaDirEccezioni.ini        #File containing exceprtions

#- PROCESSING  --------------------------------------------------------------------------------------------------
# Create file containing finded files.
find $WORKDIR/* -type f > $LOGB

#Clean output File from exceptions
cat $
EXCEPTIONS while read EXCEPTION
    do
        grep -v $
EXCEPTION     $LOGB > $LOGA
        mv $LOGA $LOGB
    done

echo "." >>$LOG
echo "##########################################################################################" >>$LOG
date >>$LOG  

#Check log file consistency
if [ -s $LOGB ]
    then
    more $LOGB >>$LOG
    else
    echo "In path $WORKDIR of system $HOSTNAME there aren't file!" >>$LOG
fi

#Send log file using SMTP
sendmail -v user@domainname < $LOG
At the beginning register variable, it is useful to use script in other environment. In the EXCEPTIONS file I store name of folder or files (absolute path) I know it is possible the script find, but their existence is not a problem for me.

The first step check for each file is present in WORKDIR and its subfolder. The parameter -type f help me to find only file.

The second step is useful to avoid false positive results. I know some files is not a problem if exist. Inserted them, or the folder name where i can find it, the step clean the log file from them. The EXCEPTIONS file have to report for each line the absolute path of the file.

I prefer have a logfile each day, even if it doesn't find occorrency. This is my check to know the script run correctly, a check on the check. To do that I rite on the log file a separator, the data running, and the name of checked dir and checked system.

Last step send file via SMTP using the send mail function of the AIX system.

This script run each morning few minutes before I start work. When I arrive check the e-mail, watch the log file ... and I spare a lot of time checking all the directory manually and I still take the time to quietly coffee, though I did not detect any anomalies.