Differences

This shows you the differences between two versions of the page.

--- purging_script [2013/10/10 09:37] – created dodger
+++ linux:purging_script [2018/04/17 08:51] – ↷ Page moved from lamp:purging_script to linux:purging_script dodger
@@ Line 3: / Line 3: @@
 This is script is a very simple method to drop unnecessary files from your filesystems (logs?) and rotate them so they don't growth forever :-P
+This script write 2 logs each time :
+  * ''$(dirname purge_files.sh)/logs/purge_files_$(date +%Y%m%d%H%M).log''
+  * ''$(dirname purge_files.sh)/logs/purge_files_$(date +%Y%m%d%H%M).err''
+So you will have detailed the script actions+errors there
-====== Script Code ======
+====== Instructions ======
+===== Usage =====
+<code bash>/home/scripts/purge_files/purge_files.sh</code>
+===== Configuration =====
+It's a file-based configuration, the variable inside the script which sets the file is:
+<code>
+CONFIGFILE=/home/scripts/purge_files/purge_files.config
+</code>
+==== Config file sytax ====
+Each line in the config file is set of file to be analysed and purged/rotated, the syntax is:
+<code>#path,mask,action,param1,param2,param3</code>
+Where:
+  * Path is the folder from which the script begin searching (serching with find)
+  * Mask: file mask for matching files, the script uses ''find -name'' (Ex:  *gz, *log, *txt)
+  * Action: Any of the avalaible actions detailed here down.
+Y las opciones disponibles son (explicadas como parámetros de find)
+=== Actions ===
+^ path ^ mask ^ action ^ param1 ^ param2 ^ param3 ^
+| /path | ? | bydate | mtime | maxdepth | N/A |
+| /path | ? | bymin | mmin | maxdepth | N/A |
+| /path | ? | rotate | size | maxdepth | N/A |
+| /path | ? | rotatezip | size | maxdepth | gzip parameters |
+| /path | ? | zip | mtime | maxdepth | gzip parameters |
+| /path | ? | dirbydate | mtime | maxdepth | N/A |
+| /path | ? | dirbymin | mmin | maxdepth | N/A |
+Description:
+  * bydate : Delete files based on modification time (''find -mtime +X -type f'')
+  * bymin : Drop files based on modification minute (''find -mmin +X -type f'')
+  * dirbydate : Drop folders based on modification time. (''find -mtime +X -type d'')
+  * dirbymin : Drop folders based on modification minute (''find -mmin +X -type d'')
+  * rotate : Rotate the file (''cat file > file.date && > file'')
+  * rotatezip : Rorate and compress the file (''gzip -c file > file.date.gz && > file'')
+  * zip : Compress the file (''gzip file'')
+==== Sample Config ====
+<code>#path,mask,action,param1,param2,param3
+# Allways purge the script logs
+/home/scripts/purge_files/logs,*.log,bydate,+4,
+/home/scripts/purge_files/logs,*.err,bydate,+4,
+# Target: nginx logs
+/var/log/nginx,*.gz,bydate,+20,,
+# Objetivo : Oracle listerner trace files
+/u01/app/oracle/diag/tnslsnr,*trc,bydate,+15,4,
+/u01/app/oracle/diag/tnslsnr,*trm,bydate,+15,4,
+</code>
+===== Script Code =====
 <file bash purge_files.sh>
 #!/bin/bash
@@ Line 10: / Line 71: @@
 # (C) dodger@ciberterminal.net
 #
-#
 # Exit codes:
@@ Line 48: / Line 109: @@
 MYDATE=$(date +%Y%m%d%H%M)
-CONFIGFILE=$(dirname $0)/$(basename $0 .sh).config
+CONFIGFILE=/home/scripts/purge_files/purge_files.config
-LOGDIR=CONFIG=$(dirname $0)/logs/
+LOGDIR=$(dirname $0)/logs/
 ########################################################################
@@ Line 93: / Line 153: @@
     [[ "${PARAM2}" ]] && AUX="-maxdepth ${PARAM2}" && MESSAGE="%s${MESSAGE} and maxdepth ${PARAM2}"
     printf "%s${MESSAGE}\n"
-    # workaround for eval "bug"
-#    local AUXDIR="/tmp/$(date +%s)"
-#    mkdir ${AUXDIR}
-#    cd ${AUXDIR}
     eval find ${FOLDER} ${AUX} -name "${MASK}" ${TYPEOF} ${OPT} ${PARAM1} ${FINDCMD}
-#    cd ${OLDPWD}
-#    rm -fr ${AUXDIR}
 }
@@ Line 149: / Line 203: @@
+[ ! -d ${LOGDIR} ] && mkdir -p ${LOGDIR}
+exec 1>> ${LOGDIR}/$(basename $0 .sh)_${MYDATE}.log
+exec 2>> ${LOGDIR}/$(basename $0 .sh)_${MYDATE}.err
-# exec 1>> ${LOGDIR}/$(basename $0 .sh).log
-# exec 2>> ${LOGDIR}/$(basename $0 .sh).err
@@ Line 172: / Line 225: @@
-#     if [[ ! "${LINE,,}" =~ ^\/[a-z0-9]{1,}/[\/\_\-\ a-z0-9]{1,},[\*\.\-\_\ \#a-z0-9]{3,20},(bydate|bymin|rotate|rotatezip|zip),\+[0-9]{1,3},(|\-[a-z0-9]{1,10}),$ ]] ; then
     if [[ ! "${LINE,,}" =~ ^\/[a-z0-9]{1,}\/[\.\/\_\ a-z0-9\-]{1,},[\*\.\_\ \#a-z0-9\-]{3,20},(dirbydate|dirbymin|bydate|bymin|rotate|rotatezip|zip),\+[0-9]{1,3}(|[kmg]),(|[0-9]{1,3}),(|\-[a-z0-9]{1,10})$ ]] ; then
         printf "%sskipping ${LINE}\nnot matching the config pattern, read instructions\n"
@@ Line 208: / Line 260: @@
             [[ "${PARAM2}" ]] && AUX="-maxdepth ${PARAM2}" && printf "%s and maxdepth ${PARAM2}\n" || printf "%s\n"
             [[ ! "${PARAM3}" ]] && PARAM3="-9fv"
-            find ${FOLDER} ${AUX} -name "${MASK}" -type f -mtime ${PARAM1} -exec gzip -9fv {} \;
+            find ${FOLDER} ${AUX} -name "${MASK}" -type f -mtime ${PARAM1} -exec gzip ${PARAM3} {} \;
         ;;
         * )