運維記錄 – 業務日誌清理功能

  • 2019 年 10 月 8 日
  • 筆記

線上某些系統業務跑一段時間後，日誌就會越來越多，考慮到業務機器磁碟有限，需要添加業務日誌清理功能。根據日誌所在分區磁碟使用情況來判斷是否清理日誌，比如當日誌分區磁碟空間使用超過90%時，將一周前的日誌打包轉移到別處 (別的分區下或遠程存儲設備上)。腳本 (/opt/script/log_clear.sh) 如下:

#!/bin/bash    #定義日誌所在分區當前空間所佔比例數(去掉%)。grep -w表示精準匹配，只匹配"/"這個分區  LOG_PARTITION=$(`which df` -h|awk '{print $5,$6}'|grep -w "/"|cut -d" " -f1|awk -F"%" '{print $1}')  #定義一周前的日期，用於日誌分區空間超過設定的閾值後保留的份數(這裡保留一周的日誌)  KEEP_DATE=`date -d '-7 days' +%Y%m%d`  #定義日誌路徑  LOG_DIR=/opt/log/kevin  #定義日誌備份路徑(即當日誌分區空間超過閾值後，將日誌轉移到該處。這裡就轉移到了本地home分區下，也可以轉移到別的機器或遠程存儲設備上)  LOG_BACK_DIR=/home/log/kevin    #確保日誌備份路徑存在，如果不存在，則創建該路徑。  if [ ! -d ${LOG_BACK_DIR} ];then     `which mkdir` -p ${LOG_BACK_DIR}  else     echo "${LOG_BACK_DIR} is exist" >/dev/null 2>&1  fi    #當日誌分區當前空間超過90%時執行的動作  if [ ${LOG_PARTITION} -ge 90 ];then       #切換到日誌路徑下     cd ${LOG_DIR}     #截取日誌文件中的日期，比如stepweb_20190915.log日誌文件，則截取20190915     for LOG_DATE in $(ls -l ${LOG_DIR}|awk '{print $9}'|cut -d"_" -f2|awk -F"." '{print $1}')     do       #當日誌文件中截取的日期是一周之前的日誌時執行的動作       if [ ${LOG_DATE} -lt ${KEEP_DATE} ];then         #打包，轉移，刪除         #注意這裡tar使用-P參數，因為tar默認為相對路徑，使用絕對路徑的話會報錯"Removing leading `/』 from member names",使用-P參數就解決了該問題         `which tar` -zvPcf ${LOG_DIR}/stepweb_${LOG_DATE}.log.tar.gz ${LOG_DIR}/stepweb_${LOG_DATE}.log         mv ${LOG_DIR}/stepweb_${LOG_DATE}.log.tar.gz ${LOG_BACK_DIR}/         rm -rf ${LOG_DIR}/stepweb_${LOG_DATE}.log       else         echo "keep stepweb_${LOG_DATE}.log" >/dev/null 2>&1       fi     done  else     echo "${LOG_PARTITION} is available" >/dev/null 2>&1  fi

根據上面腳本驗證下：

查看本機磁碟使用情況，日誌所在分區磁碟當前使用率達到92%了  [root@yyweb kevin]# df -h  Filesystem               Size  Used Avail Use% Mounted on  /dev/mapper/centos-root   50G  46G   4G     92% /  devtmpfs                 3.9G     0  3.9G   0% /dev  tmpfs                    3.9G     0  3.9G   0% /dev/shm  tmpfs                    3.9G   34M  3.8G   1% /run  tmpfs                    3.9G     0  3.9G   0% /sys/fs/cgroup  /dev/sda1               1014M  183M  832M  19% /boot  /dev/mapper/centos-home  142G   33M  142G   1% /home    日誌所在路徑  [root@yyweb kevin]# pwd  /opt/log/kevin    日誌情況  [root@yyweb kevin]# ls  stepweb_20190810.log  stepweb_20190817.log  stepweb_20190824.log  stepweb_20190901.log  stepweb_20190908.log  stepweb_20190915.log  stepweb_20190811.log  stepweb_20190818.log  stepweb_20190825.log  stepweb_20190902.log  stepweb_20190909.log  stepweb_20190916.log  stepweb_20190812.log  stepweb_20190819.log  stepweb_20190826.log  stepweb_20190903.log  stepweb_20190910.log  stepweb_20190917.log  stepweb_20190813.log  stepweb_20190820.log  stepweb_20190827.log  stepweb_20190904.log  stepweb_20190911.log  stepweb_20190918.log  stepweb_20190814.log  stepweb_20190821.log  stepweb_20190828.log  stepweb_20190905.log  stepweb_20190912.log  stepweb_20190919.log  stepweb_20190815.log  stepweb_20190822.log  stepweb_20190829.log  stepweb_20190906.log  stepweb_20190913.log  stepweb_20190816.log  stepweb_20190823.log  stepweb_20190830.log  stepweb_20190907.log  stepweb_20190914.log    [root@yyweb kevin]# ls /home/  [root@yyweb kevin]#    執行日誌清理腳本  [root@yyweb kevin]# sh /opt/script/log_clear.sh  /opt/log/kevin/stepweb_20190810.log  /opt/log/kevin/stepweb_20190811.log  /opt/log/kevin/stepweb_20190812.log  /opt/log/kevin/stepweb_20190813.log  /opt/log/kevin/stepweb_20190814.log  /opt/log/kevin/stepweb_20190815.log  /opt/log/kevin/stepweb_20190816.log  /opt/log/kevin/stepweb_20190817.log  /opt/log/kevin/stepweb_20190818.log  /opt/log/kevin/stepweb_20190819.log  /opt/log/kevin/stepweb_20190820.log  /opt/log/kevin/stepweb_20190821.log  /opt/log/kevin/stepweb_20190822.log  /opt/log/kevin/stepweb_20190823.log  /opt/log/kevin/stepweb_20190824.log  /opt/log/kevin/stepweb_20190825.log  /opt/log/kevin/stepweb_20190826.log  /opt/log/kevin/stepweb_20190827.log  /opt/log/kevin/stepweb_20190828.log  /opt/log/kevin/stepweb_20190829.log  /opt/log/kevin/stepweb_20190830.log  /opt/log/kevin/stepweb_20190901.log  /opt/log/kevin/stepweb_20190902.log  /opt/log/kevin/stepweb_20190903.log  /opt/log/kevin/stepweb_20190904.log  /opt/log/kevin/stepweb_20190905.log  /opt/log/kevin/stepweb_20190906.log  /opt/log/kevin/stepweb_20190907.log  /opt/log/kevin/stepweb_20190908.log  /opt/log/kevin/stepweb_20190909.log  /opt/log/kevin/stepweb_20190910.log  /opt/log/kevin/stepweb_20190911.log  /opt/log/kevin/stepweb_20190912.log    日誌清理後，日誌路徑下只保留了最近一周的日誌  [root@yyweb kevin]# ls  stepweb_20190913.log  stepweb_20190915.log  stepweb_20190917.log  stepweb_20190919.log  stepweb_20190914.log  stepweb_20190916.log  stepweb_20190918.log    一周之前的日誌被打包轉移到/home/log/kevin下了  [root@yyweb kevin]# ls /home/log/kevin/  stepweb_20190810.log.tar.gz  stepweb_20190817.log.tar.gz  stepweb_20190824.log.tar.gz  stepweb_20190901.log.tar.gz  stepweb_20190908.log.tar.gz  stepweb_20190811.log.tar.gz  stepweb_20190818.log.tar.gz  stepweb_20190825.log.tar.gz  stepweb_20190902.log.tar.gz  stepweb_20190909.log.tar.gz  stepweb_20190812.log.tar.gz  stepweb_20190819.log.tar.gz  stepweb_20190826.log.tar.gz  stepweb_20190903.log.tar.gz  stepweb_20190910.log.tar.gz  stepweb_20190813.log.tar.gz  stepweb_20190820.log.tar.gz  stepweb_20190827.log.tar.gz  stepweb_20190904.log.tar.gz  stepweb_20190911.log.tar.gz  stepweb_20190814.log.tar.gz  stepweb_20190821.log.tar.gz  stepweb_20190828.log.tar.gz  stepweb_20190905.log.tar.gz  stepweb_20190912.log.tar.gz  stepweb_20190815.log.tar.gz  stepweb_20190822.log.tar.gz  stepweb_20190829.log.tar.gz  stepweb_20190906.log.tar.gz  stepweb_20190816.log.tar.gz  stepweb_20190823.log.tar.gz  stepweb_20190830.log.tar.gz  stepweb_20190907.log.tar.gz

再貼一個簡單的日誌處理腳本

#!/usr/bin/sh  #根據系統/服務/日誌保留天數三個參數壓縮日誌  #usage: sh clearlog.sh sysname appname keepdays  sysName=$1  appName=$2  keepDay=$3  logDir=/var/log/${sysName}/${appName}  logFile=${appName}.log  cd ${logDir}  find ./ -name "${logFile}.*[0-9][0-9]" -mtime +${keepDay} -exec gzip {} ;

VirMach 便宜 VPS

QNews

QNews