Ciberterminal Wiki

User Tools

Site Tools

Trace:
linux:purging_script

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
purging_script [2013/10/10 09:42] – dodgerlinux:purging_script [2022/02/11 11:36] (current) – external edit 127.0.0.1
Line 1: Line 1:
 +====== [SCRIPT] Purge script ======
 +
 ====== Description ====== ====== Description ======
 Have you ever fight with logrotate configuration files? Did you survived? Cool, this document is not for you xD Have you ever fight with logrotate configuration files? Did you survived? Cool, this document is not for you xD
 This is script is a very simple method to drop unnecessary files from your filesystems (logs?) and rotate them so they don't growth forever :-P This is script is a very simple method to drop unnecessary files from your filesystems (logs?) and rotate them so they don't growth forever :-P
 +
 +This script write 2 logs each time :
 +  * ''$(dirname purge_files.sh)/logs/purge_files_$(date +%Y%m%d%H%M).log''
 +  * ''$(dirname purge_files.sh)/logs/purge_files_$(date +%Y%m%d%H%M).err''
 +So you will have detailed the script actions+errors there
  
 ====== Instructions ====== ====== Instructions ======
  
 ===== Usage ===== ===== Usage =====
-<code bash>/u02/admin/BAVELPRO/scripts/purge_files.sh</code>+<code bash>/home/scripts/purge_files/purge_files.sh</code>
 ===== Configuration ===== ===== Configuration =====
 +It's a file-based configuration, the variable inside the script which sets the file is:
 <code> <code>
-/u02/admin/BAVELPRO/scripts/purge_files.config+CONFIGFILE=/home/scripts/purge_files/purge_files.config
 </code> </code>
 +
 ==== Config file sytax ==== ==== Config file sytax ====
 Each line in the config file is set of file to be analysed and purged/rotated, the syntax is: Each line in the config file is set of file to be analysed and purged/rotated, the syntax is:
Line 22: Line 31:
  
 Y las opciones disponibles son (explicadas como parámetros de find) Y las opciones disponibles son (explicadas como parámetros de find)
 +=== Actions ===
 +^ path ^ mask ^ action ^ param1 ^ param2 ^ param3 ^
 +| /path | ? | bydate | mtime | maxdepth | N/A |
 +| /path | ? | bymin | mmin | maxdepth | N/A |
 +| /path | ? | rotate | size | maxdepth | N/A |
 +| /path | ? | rotatezip | size | maxdepth | gzip parameters |
 +| /path | ? | zip | mtime | maxdepth | gzip parameters |
 +| /path | ? | dirbydate | mtime | maxdepth | N/A |
 +| /path | ? | dirbymin | mmin | maxdepth | N/A |
  
- path mask action +Description:
- param1 param2 +
- param3 +
-  /path +
- ? bydate +
- mtime maxdepth N/A +
-  /path ? bymin +
- mmin maxdepth N/A +
-  /path ? rotate +
- size maxdepth N/A +
-  /path ? rotatezip size +
- maxdepth gzip parameters +
-  /path ? zip +
- mtime maxdepth gzip parameters +
-  /path ? dirbydate +
- mtime maxdepth N/A +
-  /path ? dirbymin +
- mmin maxdepth N/A+
  
 +  * bydate : Delete files based on modification time (''find -mtime +X -type f'')
 +  * bymin : Drop files based on modification minute (''find -mmin +X -type f'')
 +  * dirbydate : Drop folders based on modification time. (''find -mtime +X -type d'')
 +  * dirbymin : Drop folders based on modification minute (''find -mmin +X -type d'')
 +  * rotate : Rotate the file (''cat file > file.date && > file'')
 +  * rotatezip : Rorate and compress the file (''gzip -c file > file.date.gz && > file'')
 +  * zip : Compress the file (''gzip file'')
  
-Actions +==== Sample Config ==== 
- +<code>#path,mask,action,param1,param2,param3 
-    bydate : elimina ficheros basándose en la fecha de modificación. (find -mtime X -type f) +# Allways purge the script logs 
-    bymin : elimina ficheros basándose en el minuto de modificación. (find -mmin X -type f) +/home/scripts/purge_files/logs,*.log,bydate,+4, 
-    dirbydate : elimina directorios basándose en la fecha de modificación. (find -mtime X -type d) +/home/scripts/purge_files/logs,*.err,bydate,+4,
-    dirbymin : elimina directorios basándose en el minuto de modificación. (find -mmin X -type d) +
-    rotate : rota el fichero (cat file > file.date && > file) +
-    rotatezip : rota el fichero y lo comprime al mismo tiempo (gzip -c file > file.date.gz && > file) +
-    zip : comprime el fichero (gzip file)+
  
 +# Target: nginx logs
 +/var/log/nginx,*.gz,bydate,+20,,
  
 +# Objetivo : Oracle listerner trace files
 +/u01/app/oracle/diag/tnslsnr,*trc,bydate,+15,4,
 +/u01/app/oracle/diag/tnslsnr,*trm,bydate,+15,4,
 +</code>
  
  
Line 63: Line 73:
 # (C) dodger@ciberterminal.net # (C) dodger@ciberterminal.net
 #  # 
-# +
  
 # Exit codes: # Exit codes:
Line 101: Line 111:
  
 MYDATE=$(date +%Y%m%d%H%M) MYDATE=$(date +%Y%m%d%H%M)
-CONFIGFILE=$(dirname $0)/$(basename $0 .sh).config +CONFIGFILE=/home/scripts/purge_files/purge_files.config 
-LOGDIR=CONFIG=$(dirname $0)/logs/ +LOGDIR=$(dirname $0)/logs/
  
 ######################################################################## ########################################################################
Line 125: Line 134:
          
     PLEASE READ     PLEASE READ
-    https://sites.google.com/a/voxelgroup.net/sistemas/oracle/documentacion-tecnica/rotado-de-logs-y-trazas\n"+    https://sites.google.com/a/ciberterminal.net/sistemas/oracle/documentacion-tecnica/rotado-de-logs-y-trazas\n"
     # VERY INITIAL CHECKS     # VERY INITIAL CHECKS
 } }
Line 146: Line 155:
     [[ "${PARAM2}" ]] && AUX="-maxdepth ${PARAM2}" && MESSAGE="%s${MESSAGE} and maxdepth ${PARAM2}"     [[ "${PARAM2}" ]] && AUX="-maxdepth ${PARAM2}" && MESSAGE="%s${MESSAGE} and maxdepth ${PARAM2}"
     printf "%s${MESSAGE}\n"     printf "%s${MESSAGE}\n"
-    # workaround for eval "bug" 
-#    local AUXDIR="/tmp/$(date +%s)" 
-#    mkdir ${AUXDIR} 
-#    cd ${AUXDIR} 
     eval find ${FOLDER} ${AUX} -name "${MASK}" ${TYPEOF} ${OPT} ${PARAM1} ${FINDCMD}     eval find ${FOLDER} ${AUX} -name "${MASK}" ${TYPEOF} ${OPT} ${PARAM1} ${FINDCMD}
-#    cd ${OLDPWD} 
-#    rm -fr ${AUXDIR} 
 } }
  
Line 202: Line 205:
  
  
 +[ ! -d ${LOGDIR} ] && mkdir -p ${LOGDIR}
  
- +exec 1>> ${LOGDIR}/$(basename $0 .sh)_${MYDATE}.log 
- +exec 2>> ${LOGDIR}/$(basename $0 .sh)_${MYDATE}.err
-# exec 1>> ${LOGDIR}/$(basename $0 .sh).log +
-# exec 2>> ${LOGDIR}/$(basename $0 .sh).err+
  
  
Line 225: Line 227:
  
  
-#     if [[ ! "${LINE,,}" =~ ^\/[a-z0-9]{1,}/[\/\_\-\ a-z0-9]{1,},[\*\.\-\_\ \#a-z0-9]{3,20},(bydate|bymin|rotate|rotatezip|zip),\+[0-9]{1,3},(|\-[a-z0-9]{1,10}),$ ]] ; then 
     if [[ ! "${LINE,,}" =~ ^\/[a-z0-9]{1,}\/[\.\/\_\ a-z0-9\-]{1,},[\*\.\_\ \#a-z0-9\-]{3,20},(dirbydate|dirbymin|bydate|bymin|rotate|rotatezip|zip),\+[0-9]{1,3}(|[kmg]),(|[0-9]{1,3}),(|\-[a-z0-9]{1,10})$ ]] ; then     if [[ ! "${LINE,,}" =~ ^\/[a-z0-9]{1,}\/[\.\/\_\ a-z0-9\-]{1,},[\*\.\_\ \#a-z0-9\-]{3,20},(dirbydate|dirbymin|bydate|bymin|rotate|rotatezip|zip),\+[0-9]{1,3}(|[kmg]),(|[0-9]{1,3}),(|\-[a-z0-9]{1,10})$ ]] ; then
         printf "%sskipping ${LINE}\nnot matching the config pattern, read instructions\n"         printf "%sskipping ${LINE}\nnot matching the config pattern, read instructions\n"
Line 261: Line 262:
             [[ "${PARAM2}" ]] && AUX="-maxdepth ${PARAM2}" && printf "%s and maxdepth ${PARAM2}\n" || printf "%s\n"             [[ "${PARAM2}" ]] && AUX="-maxdepth ${PARAM2}" && printf "%s and maxdepth ${PARAM2}\n" || printf "%s\n"
             [[ ! "${PARAM3}" ]] && PARAM3="-9fv"             [[ ! "${PARAM3}" ]] && PARAM3="-9fv"
-            find ${FOLDER} ${AUX} -name "${MASK}" -type f -mtime ${PARAM1} -exec gzip -9fv {} \;+            find ${FOLDER} ${AUX} -name "${MASK}" -type f -mtime ${PARAM1} -exec gzip ${PARAM3} {} \;
         ;;         ;;
         * )         * )
linux/purging_script.1381398151.txt.gz · Last modified: 2013/10/10 09:42 by dodger