【原创】LOG监控和数据管理系统
Lewise.liu
1
目录
一、NETSCREEN防火墙监控........................................................................................................3
二、REDHAT syslog服务器端配置: ............................................................................................4
三、REDHAT客户端配置 ...............................................................................................................5
四、CISCO ASA5505 配置 .............................................................................................................6
五、CISCO SWITCH配置...............................................................................................................7
六、CISCO ROUTER......................................................................................................................8
七、WINDOWS SYSLOG转换 ......................................................................................................9
八、LOADBALANCE配置.............................................................................................................9
九、中间件/DB监控 ........................................................................................................................9
十、导入数据库.............................................................................................................................10
2
化了近一周多时间,编写和配置的 LOG 监控和数据管理系统终于完全上线了。
框架如下:
1)、整合:生产机房所有设备、OS、数据库、中间件的 LOG 日志整合到日志服务器;
2)、筛选:日志服务器上对所有日志进行分类过滤和筛选,频率为每五分钟;
3)、报警:筛选结果被发到两个地方:MAIL/SMS 报警和 MYSQL 数据库保存;
4)、保存:MYSQL 数据保存后,用于后期数据挖掘和
总结
初级经济法重点总结下载党员个人总结TXt高中句型全总结.doc高中句型全总结.doc理论力学知识点总结pdf
分析;
一、NETSCREEN 防火墙监控
启用多个系统日志服务器
在此例中,将安全设备配置为: 通过 TCP 将事件和流量日志发送到拥有下列 IP 地
址/ 端口号的三个系统日志服务器: 1.1.1.1/1514、2.2.2.1/2514 和 3.3.3.1/3514。
将安全级别和设备级别都设置为 Local0。
WebUI
Configuration > Report Settings > Syslog: 输入以下内容,然后单击 Apply:
Enable syslog messages: 选择此选项将日志发送到指定的系统日志服务器。
No.: 选择 1、2 和 3 以表示正在添加 3 个系统日志服务器。
IP/ Hostname: 1.1.1.1, 2.2.2.1, 3.3.3.1
Port: 1514, 2514, 3514
Security Facility: Local0, Local0, Local0
Facility: Local0, Local0, Local0
Event Log: ( 选择)
Traffic Log: ( 选择)
TCP: ( 选择)
SYSLOG 检查关键字:
Failure
Attack
threshold
Spoof
UDP flood
icmp flood
port scan
SYN flood
3
二、REDHAT syslog 服务器端配置:
1、/etc/syslog.conf 配置
把配置中的这一句
*.info;mail.none;news.none;authpriv.none;cron.none /var/log/messages
修改成
*.info;local4.none;mail.none;news.none;authpriv.none;cron.none /var/l
og/messages
后,pix 的日志就不会
记录
混凝土 养护记录下载土方回填监理旁站记录免费下载集备记录下载集备记录下载集备记录下载
到 messages 中去了。*.info 是所有对象一般信息全部记录到
messages
现有配置:
local7.* /var/log/boot.log
local0.* /var/log/netscreen.log
local1.* /var/log/router.log
local2.* /var/log/switch.log
local3.* /var/log/loadbalance.log
local4.* /var/log/asa5500.log
local5.* /var/log/db.log
*.info;local0.none;local1.none;local2.none;local3.none;local4.none;local5.none;
mail.none;news.none;authpriv.none;cron.none /var/log/os.log
2、/etc/sysconfig/syslog 配置
To enable remote logging, you will first need to configure the machine that will
receive the logs. syslogd uses configuration settings defined in the
/etc/sysconfig/syslog and /etc/syslog.conf files. To instruct syslogd to receive logs
from remote machines, open /etc/sysconfig/syslog in your preferred text editor and
locate the SYSLOGD_OPTIONS= line.
# Options to syslogd
# -m 0 disables 'MARK' messages.
# -r enables logging from remote machines
# -x disables DNS lookups on messages recieved with -r
# See syslogd(8) for more details
SYSLOGD_OPTIONS="-m 0"
...[output truncated]...
Append the -r parameter to the options line:
4
SYSLOGD_OPTIONS="-m 0 -r"
Once remote logging support is enabled on the remote logging server, each system
that will send logs to it must be configured to send its syslog output to the server,
rather than writing those logs to the local filesystem. To do this, edit the
/etc/syslog.conf file on each client system. For each of the various logging rules
defined in that file, you can replace the local log file with the address of the remote
logging server.
三、REDHAT 客户端配置
Hp-unix:
*.info;mail.none @192.168.154.24
Linux:
*.info;mail.none;news.none;authpriv.none;cron.none @192.168.154.24
因 hp-unix syslog 本身不提供客户端统一 facility,需要在服务器端通过脚本进行识别,如下:
#!/bin/sh
read stuff
SERVER=`echo $stuff |awk ‘{print $4}’`
If [ ${SERVER} = “” ]
echo $stuff >> /var/log/login_log/os.log
服务器端设置:
authpriv.* |/var/log/filter_log.sh
IBM AIX:
# stopsrc -s syslogd
# startsrc -s syslogd
# Log all kernel messages to remote
logging host.
kern.* @my.remote.logging.server
The example above will cause the client system to log all kernel messages to the
remote machine at @my.remote.logging.server.
It is also possible to configure syslogd to log all locally generated system messages,
by adding a wildcard line to the /etc/syslog.conf file:
5
# Log all messages to a remote logging server:
*.* @my.remote.logging.server
四、CISCO ASA5505 配置
logging enable
logging timestamp
logging list syslog level warnings
logging trap notifications
logging host inside 192.168.154.24
clock timezone pst +15 40
---pst:Pacific standard time;
logging list my_critical_messages level 2
logging list my_critical_messages message 611101−611323
claimserver(config)# show logging
Syslog logging: enabled
Facility: 20
Timestamp logging: enabled
Standby logging: disabled
Deny Conn when Queue Full: disabled
Console logging: disabled
Monitor logging: disabled
Buffer logging: disabled
Trap logging: level alerts, facility 20, 1631 messages logged
Logging to inside 192.168.154.24 (EMBLEM format)
History logging: disabled
Device ID: disabled
Mail logging: disabled
ASDM logging: level informational, 251 messages logged
6
【注意】:只有小于或等于 SS Value(如下:0、1、2)的信息被记录:
logging list my_critical_messages level 2
即:Sets the maximum level for system log messages.
"RSA authentication available in SSH clients is not supported in the SSH
server for Cisco IOS software."
//KEY WORD:
Failure
Exceed
Threshold
UPDOWN
五、CISCO SWITCH 配置
logging host 192.168.154.24
logging trap 6
logging facility local2
---service timestamps log uptime
---logging source-interface vlan 1 (自动已经找到)
//KEY WORD:
Loop-back
7
Threshold
Shutting down
Inactive
Error
Invalid
Limit
MAC
Fail
Storm
UPDOWN
Message Logging Level Keywords
六、CISCO ROUTER
手工配置:
logging facility local1
logging source-interface FastEthernet0/0
logging 192.168.154.24
----- logging host 192.168.154.24 transport udp 514
隐含已经实现以下配置:
logging buffered 51200 warnings
logging console debugging
logging monitor debugging
logging trap 6
logging on
//KEY WORD:
UPDOWN
8
七、WINDOWS SYSLOG 转换
1)、Copy the files evtsys.exe and evtsys.dll into the System32 directory (usually
%systemroot%\system32).
2)、配置
evtsys -i -h 192.168.154.24
在services中启动该服务;
维护:
1)、重新配置
net stop evtsys
evtsys -u
evtsys -i -h newhostname
net start evtsys
2)、默认evtsys facility为DAEMON
八、LOADBALANCE 配置
1)、RADWARE SYSLOG配置
设置local3;
2)、F5配置
因F5不提供类似SYSLOG接口,只有syslog-ng,暂不监控;
九、中间件/DB 监控
1、DB配置步骤:
首先:本地部署monitor_filter.sh脚本;
其次:在本地syslog.conf配置接收filter的facility;
local5.info @192.168.154.24
2、MiddleWare配置步骤:
MiddleWare因日志量大,需要筛选后,再传到远程SYSLOG服务器,并进行统一报警和保存。
步骤:
首先:本地部署monitor_filter.sh脚本;
其次:在本地syslog.conf配置接收filter的facility;
配置如下:
Local6.info @192.168.154.24
9
十、导入数据库
【注意】:在10M rows以下数据量时,不考虑使用ORACLE;
1、 导入MYSQL
采用perl的insert方式,详见脚本load_db.pl;
--未采用load方式;
mysql> LOAD DATA INFILE 'data.txt' INTO TABLE tbl_name
-> FIELDS TERMINATED BY ',' ENCLOSED BY '"'
-> LINES TERMINATED BY '\n';
2、导入ORACLE
10
一、NETSCREEN防火墙监控
二、REDHAT syslog服务器端配置:
三、REDHAT客户端配置
四、CISCO ASA5505配置
五、CISCO SWITCH配置
六、CISCO ROUTER
七、WINDOWS SYSLOG转换
八、LOADBALANCE配置
九、中间件/DB监控
十、导入数据库