网站接入百度云加速后,nginx的访问日志里面多了很多云加速节点来探测网站是否存活的记录:
42.236.7.68 - - [30/May/2017:08:36:59 +0800] "HEAD / HTTP/1.1" 200 0 "-" "Baidu-YunJiaSu-Bot(su.baidu.com)" 42.236.7.68 - - [30/May/2017:08:36:59 +0800] "HEAD / HTTP/1.1" 200 0 "-" "Baidu-YunJiaSu-Bot(su.baidu.com)" 122.246.5.11 - - [30/May/2017:08:37:52 +0800] "HEAD / HTTP/1.1" 200 0 "-" "Baidu-YunJiaSu-Bot(su.baidu.com)" 117.34.28.13 - - [30/May/2017:08:37:52 +0800] "HEAD / HTTP/1.1" 200 0 "-" "Baidu-YunJiaSu-Bot(su.baidu.com)" 119.188.132.10 - - [30/May/2017:08:37:52 +0800] "HEAD / HTTP/1.1" 200 0 "-" "Baidu-YunJiaSu-Bot(su.baidu.com)" 124.95.168.140 - - [30/May/2017:08:37:52 +0800] "HEAD / HTTP/1.1" 200 0 "-" "Baidu-YunJiaSu-Bot(su.baidu.com)" 61.182.137.6 - - [30/May/2017:08:37:52 +0800] "HEAD / HTTP/1.1" 200 0 "-" "Baidu-YunJiaSu-Bot(su.baidu.com)" 122.246.5.11 - - [30/May/2017:08:37:53 +0800] "HEAD / HTTP/1.1" 200 0 "-" "Baidu-YunJiaSu-Bot(su.baidu.com)" 119.188.132.10 - - [30/May/2017:08:37:53 +0800] "HEAD / HTTP/1.1" 200 0 "-" "Baidu-YunJiaSu-Bot(su.baidu.com)" 117.34.28.13 - - [30/May/2017:08:37:53 +0800] "HEAD / HTTP/1.1" 200 0 "-" "Baidu-YunJiaSu-Bot(su.baidu.com)" 61.182.137.6 - - [30/May/2017:08:37:53 +0800] "HEAD / HTTP/1.1" 200 0 "-" "Baidu-YunJiaSu-Bot(su.baidu.com)" 124.95.168.140 - - [30/May/2017:08:37:53 +0800] "HEAD / HTTP/1.1" 200 0 "-" "Baidu-YunJiaSu-Bot(su.baidu.com)" [root@dish log]# grep -c YunJiaSu 20170529_wordpress.log 4131 [root@dish log]# grep -c YunJiaSu 20170528_wordpress.log 4068 [root@dish log]# grep -c YunJiaSu 20170527_wordpress.log 3744
显然这些记录是无意义的,我们要让nginx写日志的时候,自动排除这些云加速的HEAD请求。
网上搜到的方法经过测试后无效,不知道是不是我姿势不对:
if ($request_method = "HEAD" ) { access_log off; }
ngx_log_if模块
ngx_log_if模块是一个nginx的第三方模块,这个模块能很好的实现日志过滤功能。
下载 ngx_log_if 模块
cd /root/download wget https://github.com/cfsego/ngx_log_if/archive/master.zip unzip master.zip
nginx添加第三方模块
1、查看ngixn版本极其编译参数
/usr/local/nginx/sbin/nginx -V
2、进入nginx源码目录
cd nginx-1.4.4
3、以下是重新编译的代码和模块
./configure --user=www --group=www --prefix=/usr/local/nginx --without-http-cache --with-http_ssl_module --with-http_gzip_static_module --with-http_realip_module --add-module=/root/download/ngx_log_if-master
make 千万别make install,否则就覆盖安装了
make完之后在objs目录下就多了个nginx,这个就是新版本的程序了
4、备份旧的nginx程序
/usr/local/nginx/sbin/nginx -s stop cp /usr/local/nginx/sbin/nginx{,.bak}
5、把新的nginx程序覆盖旧的
cp objs/nginx /usr/local/nginx/sbin/nginx
通过ngx_log_if进行nginx日志过滤
server { access_log_bypass_if ($http_user_agent ~* 'Baidu-YunJiaSu-Bot' ); ## Your website name goes here. server_name hqidi.com www.hqidi.com;
若这个access_log_bypass_if语句加在location标签下无效的话,就把该语句加在server标签下,我就刚好遇到此问题。
ngx_log_if常用例子
server { # 不记录204响应状态的access日志 access_log_bypass_if ($status = 204); # 不记录200响应状态且uri为 status.nginx 的访问日志 access_log_bypass_if ($uri = 'status.nginx') and; access_log_bypass_if ($status = 200); # 不记录HEAD请求的日志 access_log_bypass_if ($request = 'HEAD'); }
原创文章,转载请注明: 转载自笛声
本文链接地址: nginx不记录指定日志
2 条评论
if ($request_method = "HEAD" ) {
access_log off;
}
因为access_log不能放在if里面 要放在location if里面才行
这个对日志文件大小好像有影响!
不错,可以考虑给我的Nginx也加上,呵呵!
话说这个百度云加速的爬虫真的好烦呀!