diff options
| author | Yaroslav Halchenko <debian@onerussian.com> | 2016-03-07 21:50:47 -0500 |
|---|---|---|
| committer | Yaroslav Halchenko <debian@onerussian.com> | 2016-03-07 21:50:47 -0500 |
| commit | 45dce3cab0f86eb66bd6e7fcb46d1b0df5076836 (patch) | |
| tree | b6744470ade77492c3b8c8c4b771b2a7689b8445 | |
| parent | d67f502ff5dde5aa1a365d18610177cbb2d2650b (diff) | |
| parent | 0298ba2c1b0057857789f61be926bd39493a1678 (diff) | |
| download | fail2ban-45dce3cab0f86eb66bd6e7fcb46d1b0df5076836.tar.gz | |
Merge tag '0.9.4' into debian
ver. 0.9.4 (2016/03/08) - for-you-ladies
-----------
- Fixes:
* roundcube-auth jail typo for logpath
* Fix dnsToIp resolver for fqdn with large list of IPs (gh-1164)
* filter.d/apache-badbots.conf
- Updated useragent string regex adding escape for `+`
* filter.d/mysqld-auth.conf
- Updated "Access denied ..." regex for MySQL 5.6 and later (gh-1211, gh-1332)
* filter.d/sshd.conf
- Updated "Auth fail" regex for OpenSSH 5.9 and later
* Treat failed and killed execution of commands identically (only
different log messages), which addresses different behavior on different
exit codes of dash and bash (gh-1155)
* Fix jail.conf.5 man's section (gh-1226)
* Fixed default banaction for allports jails like pam-generic, recidive, etc
with new default variable `banaction_allports` (gh-1216)
* Fixed `fail2ban-regex` stops working on invalid (wrong encoded) character
for python version < 3.x (gh-1248)
* Use postfix_log logpath for postfix-rbl jail
* filters.d/postfix.conf - add 'Sender address rejected: Domain not found' failregex
* use `fail2ban_agent` as user-agent in actions badips, blocklist_de, etc (gh-1271)
* Fix ignoring the sender option by action_mw, action_mwl and action_c_mwl
* Changed filter.d/asterisk regex for "Call from ..." (few vulnerable now)
* Removed compression and rotation count from logrotate (inherit them from
the global logrotate config)
- New Features:
* New interpolation feature for definition config readers - `<known/parameter>`
(means last known init definition of filters or actions with name `parameter`).
This interpolation makes possible to extend a parameters of stock filter or
action directly in jail inside jail.local file, without creating a separately
filter.d/*.local file.
As extension to interpolation `%(known/parameter)s`, that does not works for
filter and action init parameters
* New actions:
- nftables-multiport and nftables-allports - filtering using nftables
framework. Note: it requires a pre-existing chain for the filtering rule.
* New filters:
- openhab - domotic software authentication failure with the
rest api and web interface (gh-1223)
- nginx-limit-req - ban hosts, that were failed through nginx by limit
request processing rate (ngx_http_limit_req_module)
- murmur - ban hosts that repeatedly attempt to connect to
murmur/mumble-server with an invalid server password or certificate.
- haproxy-http-auth - filter to match failed HTTP Authentications against a
HAProxy server
* New jails:
- murmur - bans TCP and UDP from the bad host on the default murmur port.
* sshd filter got new failregex to match "maximum authentication
attempts exceeded" (introduced in openssh 6.8)
* Added filter for Mac OS screen sharing (VNC) daemon
- Enhancements:
* Do not rotate empty log files
* Added new date pattern with year after day (e.g. Sun Jan 23 2005 21:59:59)
http://bugs.debian.org/798923
* Added openSUSE path configuration (Thanks Johannes Weberhofer)
* Allow to split ignoreip entries by ',' as well as by ' ' (gh-1197)
* Added a timeout (3 sec) to urlopen within badips.py action
(Thanks M. Maraun)
* Added check against atacker's Googlebot PTR fake records
(Thanks Pablo Rodriguez Fernandez)
* Enhance filter against atacker's Googlebot PTR fake records
(gh-1226)
* Nginx log paths extended (prefixed with "*" wildcard) (gh-1237)
* Added filter for openhab domotic software authentication failure with the
rest api and web interface (gh-1223)
* Add *_backend options for services to allow distros to set the default
backend per service, set default to systemd for Fedora as appropriate
* Performance improvements while monitoring large number of files (gh-1265).
Use associative array (dict) for monitored log files to speed up lookup
operations. Thanks @kshetragia
* Specified that fail2ban is PartOf iptables.service firewalld.service in
.service file -- would reload fail2ban if those services are restarted
* Provides new default `fail2ban_version` and interpolation variable
`fail2ban_agent` in jail.conf
* Enhance filter 'postfix' to ban incoming SMTP client with no fqdn hostname,
and to support multiple instances of postfix having varying suffix (gh-1331)
(Thanks Tom Hendrikx)
* files/gentoo-initd to use start-stop-daemon to robustify restarting the service
* tag '0.9.4': (138 commits)
MANIFEST RELEASE and man pages updates
Changes for the 0.9.4 release
datedetector: epoch time expression fix (now 10-11 chars, only whole number - anchored ^...\b or by special case within [], audit()) + test cases extended (positive/negative)
changelog about gentoo initd
added wp-admin
ENH(TST): a hypothetical example to show/test needing trailing anchoring
ENH: revert back to having detailed suffix anchored at the end for mysqld-auto.conf
Changelog for the recent PR and added Tom to THANKS
mysqld: failregex fixed (accepts different log level, more secure expression now); closes #1332
Add support for matching postfix multi-instance daemon names by default
DOC: removed Nick from listed as FreeBSD maintainer
DOC: adjusted ISSUE_TEMPLATE.md picking on @sebres's version
ENH: github templates for issues and PRs
ENH: add codecov support to travis.yml and bandge to README.md
gentoo-initd: Use start-stop-daemon in order to handle crashes better
regexp rewritten (few vulnerable as previous) + test case added
Update asterisk filter: changed regex for "Call from ...". Sometimes extension can have a plus symbol (+) because they can be phone number. Closes #1309
Add new regex into postfix filter. The new regexp is able to detect bad formatted SMTP EHLO command
Remove compression and count from logrotate
gentoo-initd: do not hide useful output
...
87 files changed, 2129 insertions, 947 deletions
diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md new file mode 100644 index 00000000..cb4b4bc6 --- /dev/null +++ b/.github/ISSUE_TEMPLATE.md @@ -0,0 +1,49 @@ +_We will be very grateful, if your problem was described as completely as possible, +enclosing excerpts from logs (if possible within DEBUG mode, if no errors evident +within INFO mode), and configuration in particular of effected relevant settings +(e.g., with ` fail2ban-client -d | grep 'affected-jail-name' ` for a particular +jail troubleshooting). +Thank you in advance for the details, because such issues like "It does not work" +alone could not help to resolve anything! +Thanks! (remove this paragraph and other comments upon reading)_ + +### Environment: + +_Fill out and check (`[x]`) the boxes which apply. If your Fail2Ban version is outdated, +and you can't verify that the issue persists in the recent release, better seek support +from the distribution you obtained Fail2Ban from_ + +- Fail2Ban version (including any possible distribution suffixes): +- OS, including release name/version: +- [ ] Fail2Ban installed via OS/distribution mechanisms +- [ ] You have not applied any additional foreign patches to the codebase +- [ ] Some customizations were done to the configuration (provide details below is so) + +### The issue: + +_Summary here_ + +#### Steps to reproduce + +#### Expected behavior + +#### Observed behavior + +#### Any additional information + +### Configuration, dump and another helpful excerpts + +#### Any customizations done to /etc/fail2ban/ configuration +``` +``` + +#### Relevant parts of /var/log/fail2ban.log file: +_preferably obtained while running fail2ban with `loglevel = 4`_ + +``` +``` + +#### Relevant lines from monitored log files in question: + +``` +```
\ No newline at end of file diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 00000000..a9fff350 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,9 @@ +Before submitting your PR, please review the following checklist: + +- [ ] **CONSIDER adding a unit test** if your PR resolves an issue +- [ ] **LIST ISSUES** this PR resolves +- [ ] **MAKE SURE** this PR doesn't break existing tests +- [ ] **KEEP PR small** so it could be easily reviewed. +- [ ] **AVOID** making unnecessary stylistic changes in unrelated code +- [ ] **ACCOMPANY** each new `failregex` for filter `X` with sample log lines + within `fail2ban/tests/files/logs/X` file
\ No newline at end of file diff --git a/.mailmap b/.mailmap new file mode 100644 index 00000000..9b7cce3a --- /dev/null +++ b/.mailmap @@ -0,0 +1,5 @@ +Lee Clemens <java@leeclemens.net> +Serg G. Brester <info@sebres.de> +Serg G. Brester <serg.brester@sebres.de> +Serg G. Brester <sergey.brester@W7-DEHBG0189.wincor-nixdorf.com> +Viktor Szépe <viktor@szepe.net> diff --git a/.pydevproject b/.pydevproject deleted file mode 100644 index 3c3130d9..00000000 --- a/.pydevproject +++ /dev/null @@ -1,12 +0,0 @@ -<?xml version="1.0" encoding="UTF-8" standalone="no"?> -<?eclipse-pydev version="1.0"?> - -<pydev_project> -<pydev_property name="org.python.pydev.PYTHON_PROJECT_VERSION">python 2.3</pydev_property> -<pydev_pathproperty name="org.python.pydev.PROJECT_SOURCE_PATH"> -<path>/fail2ban-0.8/client</path> -<path>/fail2ban-0.8/server</path> -<path>/fail2ban-0.8/testcases</path> -<path>/fail2ban-0.8</path> -</pydev_pathproperty> -</pydev_project> diff --git a/.travis.yml b/.travis.yml index f65e4896..50894a94 100644 --- a/.travis.yml +++ b/.travis.yml @@ -6,7 +6,8 @@ python: - 2.6 - 2.7 - pypy - - 3.2 + # disabled until coverage module fixes up compatibility issue + # - 3.2 - 3.3 - 3.4 - pypy3 @@ -22,7 +23,7 @@ install: # coverage - travis_retry pip install coverage # coveralls - - travis_retry pip install coveralls + - travis_retry pip install coveralls codecov # dnspython or dnspython3 - if [[ "$F2B_PY_2" ]]; then travis_retry pip install dnspython; fi - if [[ "$F2B_PY_3" ]]; then travis_retry pip install dnspython3; fi @@ -42,6 +43,7 @@ script: - sudo $VENV_BIN/pip install . after_success: - coveralls + - codecov matrix: fast_finish: true # Might be worth looking into @@ -6,6 +6,90 @@ Fail2Ban: Changelog =================== +ver. 0.9.4 (2016/03/08) - for-you-ladies +----------- + +- Fixes: + * roundcube-auth jail typo for logpath + * Fix dnsToIp resolver for fqdn with large list of IPs (gh-1164) + * filter.d/apache-badbots.conf + - Updated useragent string regex adding escape for `+` + * filter.d/mysqld-auth.conf + - Updated "Access denied ..." regex for MySQL 5.6 and later (gh-1211, gh-1332) + * filter.d/sshd.conf + - Updated "Auth fail" regex for OpenSSH 5.9 and later + * Treat failed and killed execution of commands identically (only + different log messages), which addresses different behavior on different + exit codes of dash and bash (gh-1155) + * Fix jail.conf.5 man's section (gh-1226) + * Fixed default banaction for allports jails like pam-generic, recidive, etc + with new default variable `banaction_allports` (gh-1216) + * Fixed `fail2ban-regex` stops working on invalid (wrong encoded) character + for python version < 3.x (gh-1248) + * Use postfix_log logpath for postfix-rbl jail + * filters.d/postfix.conf - add 'Sender address rejected: Domain not found' failregex + * use `fail2ban_agent` as user-agent in actions badips, blocklist_de, etc (gh-1271) + * Fix ignoring the sender option by action_mw, action_mwl and action_c_mwl + * Changed filter.d/asterisk regex for "Call from ..." (few vulnerable now) + * Removed compression and rotation count from logrotate (inherit them from + the global logrotate config) + +- New Features: + * New interpolation feature for definition config readers - `<known/parameter>` + (means last known init definition of filters or actions with name `parameter`). + This interpolation makes possible to extend a parameters of stock filter or + action directly in jail inside jail.local file, without creating a separately + filter.d/*.local file. + As extension to interpolation `%(known/parameter)s`, that does not works for + filter and action init parameters + * New actions: + - nftables-multiport and nftables-allports - filtering using nftables + framework. Note: it requires a pre-existing chain for the filtering rule. + * New filters: + - openhab - domotic software authentication failure with the + rest api and web interface (gh-1223) + - nginx-limit-req - ban hosts, that were failed through nginx by limit + request processing rate (ngx_http_limit_req_module) + - murmur - ban hosts that repeatedly attempt to connect to + murmur/mumble-server with an invalid server password or certificate. + - haproxy-http-auth - filter to match failed HTTP Authentications against a + HAProxy server + * New jails: + - murmur - bans TCP and UDP from the bad host on the default murmur port. + * sshd filter got new failregex to match "maximum authentication + attempts exceeded" (introduced in openssh 6.8) + * Added filter for Mac OS screen sharing (VNC) daemon + +- Enhancements: + * Do not rotate empty log files + * Added new date pattern with year after day (e.g. Sun Jan 23 2005 21:59:59) + http://bugs.debian.org/798923 + * Added openSUSE path configuration (Thanks Johannes Weberhofer) + * Allow to split ignoreip entries by ',' as well as by ' ' (gh-1197) + * Added a timeout (3 sec) to urlopen within badips.py action + (Thanks M. Maraun) + * Added check against atacker's Googlebot PTR fake records + (Thanks Pablo Rodriguez Fernandez) + * Enhance filter against atacker's Googlebot PTR fake records + (gh-1226) + * Nginx log paths extended (prefixed with "*" wildcard) (gh-1237) + * Added filter for openhab domotic software authentication failure with the + rest api and web interface (gh-1223) + * Add *_backend options for services to allow distros to set the default + backend per service, set default to systemd for Fedora as appropriate + * Performance improvements while monitoring large number of files (gh-1265). + Use associative array (dict) for monitored log files to speed up lookup + operations. Thanks @kshetragia + * Specified that fail2ban is PartOf iptables.service firewalld.service in + .service file -- would reload fail2ban if those services are restarted + * Provides new default `fail2ban_version` and interpolation variable + `fail2ban_agent` in jail.conf + * Enhance filter 'postfix' to ban incoming SMTP client with no fqdn hostname, + and to support multiple instances of postfix having varying suffix (gh-1331) + (Thanks Tom Hendrikx) + * files/gentoo-initd to use start-stop-daemon to robustify restarting the service + + ver. 0.9.3 (2015/08/01) - lets-all-stay-friends ---------- @@ -1,18 +1,8 @@ -CONTRIBUTING.md -COPYING -ChangeLog -DEVELOP -FILTERS -README.Solaris -README.md -RELEASE -THANKS -TODO -Vagrantfile bin/fail2ban-client bin/fail2ban-regex bin/fail2ban-server bin/fail2ban-testcases +ChangeLog config/action.d/apf.conf config/action.d/badips.conf config/action.d/badips.py @@ -31,20 +21,22 @@ config/action.d/ipfilter.conf config/action.d/ipfw.conf config/action.d/iptables-allports.conf config/action.d/iptables-common.conf +config/action.d/iptables.conf config/action.d/iptables-ipset-proto4.conf config/action.d/iptables-ipset-proto6-allports.conf config/action.d/iptables-ipset-proto6.conf -config/action.d/iptables-multiport-log.conf config/action.d/iptables-multiport.conf +config/action.d/iptables-multiport-log.conf config/action.d/iptables-new.conf config/action.d/iptables-xt_recent-echo.conf -config/action.d/iptables.conf config/action.d/mail-buffered.conf -config/action.d/mail-whois-lines.conf -config/action.d/mail-whois.conf config/action.d/mail.conf +config/action.d/mail-whois.conf +config/action.d/mail-whois-lines.conf config/action.d/mynetwatchman.conf -config/action.d/nsupdate.conf +config/action.d/nftables-allports.conf +config/action.d/nftables-common.conf +config/action.d/nftables-multiport.conf config/action.d/nsupdate.conf config/action.d/osx-afctl.conf config/action.d/osx-ipfw.conf @@ -52,13 +44,13 @@ config/action.d/pf.conf config/action.d/route.conf config/action.d/sendmail-buffered.conf config/action.d/sendmail-common.conf +config/action.d/sendmail.conf config/action.d/sendmail-geoip-lines.conf +config/action.d/sendmail-whois.conf config/action.d/sendmail-whois-ipjailmatches.conf config/action.d/sendmail-whois-ipmatches.conf config/action.d/sendmail-whois-lines.conf config/action.d/sendmail-whois-matches.conf -config/action.d/sendmail-whois.conf -config/action.d/sendmail.conf config/action.d/shorewall.conf config/action.d/smtp.py config/action.d/symbiosis-blacklist-allports.conf @@ -89,44 +81,43 @@ config/filter.d/dovecot.conf config/filter.d/dropbear.conf config/filter.d/ejabberd-auth.conf config/filter.d/exim-common.conf -config/filter.d/exim-spam.conf config/filter.d/exim.conf +config/filter.d/exim-spam.conf config/filter.d/freeswitch.conf config/filter.d/groupoffice.conf config/filter.d/gssftpd.conf config/filter.d/guacamole.conf +config/filter.d/haproxy-http-auth.conf config/filter.d/horde.conf config/filter.d/ignorecommands config/filter.d/ignorecommands/apache-fakegooglebot config/filter.d/kerio.conf config/filter.d/lighttpd-auth.conf config/filter.d/monit.conf +config/filter.d/murmur.conf config/filter.d/mysqld-auth.conf config/filter.d/nagios.conf config/filter.d/named-refused.conf config/filter.d/nginx-botsearch.conf config/filter.d/nginx-http-auth.conf +config/filter.d/nginx-limit-req.conf config/filter.d/nsd.conf +config/filter.d/openhab.conf config/filter.d/openwebmail.conf config/filter.d/oracleims.conf config/filter.d/pam-generic.conf -config/filter.d/pam-generic.conf -config/filter.d/pam-generic.conf config/filter.d/perdition.conf config/filter.d/php-url-fopen.conf -config/filter.d/php-url-fopen.conf -config/filter.d/php-url-fopen.conf config/filter.d/portsentry.conf +config/filter.d/postfix.conf config/filter.d/postfix-rbl.conf config/filter.d/postfix-sasl.conf -config/filter.d/postfix-sasl.conf -config/filter.d/postfix-sasl.conf -config/filter.d/postfix.conf config/filter.d/proftpd.conf config/filter.d/pure-ftpd.conf config/filter.d/qmail.conf config/filter.d/recidive.conf config/filter.d/roundcube-auth.conf +config/filter.d/screensharingd.conf config/filter.d/selinux-common.conf config/filter.d/selinux-ssh.conf config/filter.d/sendmail-auth.conf @@ -137,8 +128,8 @@ config/filter.d/sogo-auth.conf config/filter.d/solid-pop3d.conf config/filter.d/squid.conf config/filter.d/squirrelmail.conf -config/filter.d/sshd-ddos.conf config/filter.d/sshd.conf +config/filter.d/sshd-ddos.conf config/filter.d/stunnel.conf config/filter.d/suhosin.conf config/filter.d/tine20.conf @@ -152,13 +143,13 @@ config/paths-common.conf config/paths-debian.conf config/paths-fedora.conf config/paths-freebsd.conf +config/paths-opensuse.conf config/paths-osx.conf +CONTRIBUTING.md +COPYING +DEVELOP doc/run-rootless.txt fail2ban-2to3 -fail2ban-testcases-all -fail2ban-testcases-all-python3 -fail2ban/__init__.py -fail2ban/client/__init__.py fail2ban/client/actionreader.py fail2ban/client/beautifier.py fail2ban/client/configparserinc.py @@ -166,13 +157,15 @@ fail2ban/client/configreader.py fail2ban/client/configurator.py fail2ban/client/csocket.py fail2ban/client/fail2banreader.py +fail2ban/client/fail2banregex.py fail2ban/client/filterreader.py +fail2ban/client/__init__.py fail2ban/client/jailreader.py fail2ban/client/jailsreader.py fail2ban/exceptions.py fail2ban/helpers.py +fail2ban/__init__.py fail2ban/protocol.py -fail2ban/server/__init__.py fail2ban/server/action.py fail2ban/server/actions.py fail2ban/server/asyncserver.py @@ -183,11 +176,12 @@ fail2ban/server/datetemplate.py fail2ban/server/faildata.py fail2ban/server/failmanager.py fail2ban/server/failregex.py -fail2ban/server/filter.py fail2ban/server/filtergamin.py fail2ban/server/filterpoll.py +fail2ban/server/filter.py fail2ban/server/filterpyinotify.py fail2ban/server/filtersystemd.py +fail2ban/server/__init__.py fail2ban/server/iso8601.py fail2ban/server/jail.py fail2ban/server/jails.py @@ -197,7 +191,8 @@ fail2ban/server/server.py fail2ban/server/strptime.py fail2ban/server/ticket.py fail2ban/server/transmitter.py -fail2ban/tests/__init__.py +fail2ban-testcases-all +fail2ban-testcases-all-python3 fail2ban/tests/action_d/__init__.py fail2ban/tests/action_d/test_badips.py fail2ban/tests/action_d/test_smtp.py @@ -218,33 +213,34 @@ fail2ban/tests/config/paths-osx.conf fail2ban/tests/databasetestcase.py fail2ban/tests/datedetectortestcase.py fail2ban/tests/dummyjail.py +fail2ban/tests/fail2banregextestcase.py fail2ban/tests/failmanagertestcase.py -fail2ban/tests/files/action.d/action.py fail2ban/tests/files/action.d/action_checkainfo.py fail2ban/tests/files/action.d/action_errors.py fail2ban/tests/files/action.d/action_modifyainfo.py fail2ban/tests/files/action.d/action_noAction.py fail2ban/tests/files/action.d/action_nomethod.py -fail2ban/tests/files/config/apache-auth/README +fail2ban/tests/files/action.d/action.py +fail2ban/tests/files/config/apache-auth/basic/authz_owner/cant_get_me.html fail2ban/tests/files/config/apache-auth/basic/authz_owner/.htaccess fail2ban/tests/files/config/apache-auth/basic/authz_owner/.htpasswd -fail2ban/tests/files/config/apache-auth/basic/authz_owner/cant_get_me.html fail2ban/tests/files/config/apache-auth/basic/file/.htaccess fail2ban/tests/files/config/apache-auth/basic/file/.htpasswd -fail2ban/tests/files/config/apache-auth/digest.py -fail2ban/tests/files/config/apache-auth/digest/.htaccess -fail2ban/tests/files/config/apache-auth/digest/.htpasswd fail2ban/tests/files/config/apache-auth/digest_anon/.htaccess fail2ban/tests/files/config/apache-auth/digest_anon/.htpasswd +fail2ban/tests/files/config/apache-auth/digest/.htaccess +fail2ban/tests/files/config/apache-auth/digest/.htpasswd +fail2ban/tests/files/config/apache-auth/digest.py fail2ban/tests/files/config/apache-auth/digest_time/.htaccess fail2ban/tests/files/config/apache-auth/digest_time/.htpasswd fail2ban/tests/files/config/apache-auth/digest_wrongrelm/.htaccess fail2ban/tests/files/config/apache-auth/digest_wrongrelm/.htpasswd fail2ban/tests/files/config/apache-auth/noentry/.htaccess +fail2ban/tests/files/config/apache-auth/README fail2ban/tests/files/database_v1.db fail2ban/tests/files/filter.d/substition.conf -fail2ban/tests/files/filter.d/testcase-common.conf fail2ban/tests/files/filter.d/testcase01.conf +fail2ban/tests/files/filter.d/testcase-common.conf fail2ban/tests/files/ignorecommand.py fail2ban/tests/files/logs/3proxy fail2ban/tests/files/logs/apache-auth @@ -276,16 +272,20 @@ fail2ban/tests/files/logs/freeswitch fail2ban/tests/files/logs/groupoffice fail2ban/tests/files/logs/gssftpd fail2ban/tests/files/logs/guacamole +fail2ban/tests/files/logs/haproxy-http-auth fail2ban/tests/files/logs/horde fail2ban/tests/files/logs/kerio fail2ban/tests/files/logs/lighttpd-auth fail2ban/tests/files/logs/monit +fail2ban/tests/files/logs/murmur fail2ban/tests/files/logs/mysqld-auth fail2ban/tests/files/logs/nagios fail2ban/tests/files/logs/named-refused fail2ban/tests/files/logs/nginx-botsearch fail2ban/tests/files/logs/nginx-http-auth +fail2ban/tests/files/logs/nginx-limit-req fail2ban/tests/files/logs/nsd +fail2ban/tests/files/logs/openhab fail2ban/tests/files/logs/openwebmail fail2ban/tests/files/logs/oracleims fail2ban/tests/files/logs/pam-generic @@ -300,6 +300,7 @@ fail2ban/tests/files/logs/pure-ftpd fail2ban/tests/files/logs/qmail fail2ban/tests/files/logs/recidive fail2ban/tests/files/logs/roundcube-auth +fail2ban/tests/files/logs/screensharingd fail2ban/tests/files/logs/selinux-ssh fail2ban/tests/files/logs/sendmail-auth fail2ban/tests/files/logs/sendmail-reject @@ -319,14 +320,16 @@ fail2ban/tests/files/logs/vsftpd fail2ban/tests/files/logs/webmin-auth fail2ban/tests/files/logs/wuftpd fail2ban/tests/files/logs/xinetd-fail -fail2ban/tests/files/testcase-journal.log -fail2ban/tests/files/testcase-multiline.log -fail2ban/tests/files/testcase-usedns.log fail2ban/tests/files/testcase01.log fail2ban/tests/files/testcase02.log fail2ban/tests/files/testcase03.log fail2ban/tests/files/testcase04.log +fail2ban/tests/files/testcase-journal.log +fail2ban/tests/files/testcase-multiline.log +fail2ban/tests/files/testcase-usedns.log +fail2ban/tests/files/testcase-wrong-char.log fail2ban/tests/filtertestcase.py +fail2ban/tests/__init__.py fail2ban/tests/misctestcase.py fail2ban/tests/samplestestcase.py fail2ban/tests/servertestcase.py @@ -334,13 +337,13 @@ fail2ban/tests/sockettestcase.py fail2ban/tests/utils.py fail2ban/version.py files/bash-completion -files/cacti/README files/cacti/cacti_host_template_fail2ban.xml files/cacti/fail2ban_stats.sh +files/cacti/README files/debian-initd files/fail2ban-logrotate -files/fail2ban-tmpfiles.conf files/fail2ban.service +files/fail2ban-tmpfiles.conf files/fail2ban.upstart files/gen_badbots files/gentoo-confd @@ -349,21 +352,28 @@ files/ipmasq-ZZZzzz_fail2ban.rul files/logwatch/fail2ban files/macosx-initd files/monit/fail2ban -files/nagios/README files/nagios/check_fail2ban +files/nagios/README files/redhat-initd files/solaris-fail2ban.xml files/solaris-svc-fail2ban files/suse-initd +FILTERS kill-server +man/fail2ban.1 man/fail2ban-client.1 man/fail2ban-client.h2m man/fail2ban-regex.1 man/fail2ban-regex.h2m man/fail2ban-server.1 man/fail2ban-server.h2m -man/fail2ban.1 man/generate-man man/jail.conf.5 +README.md +README.Solaris +RELEASE setup.cfg setup.py +THANKS +TODO +Vagrantfile @@ -2,17 +2,19 @@ / _|__ _(_) |_ ) |__ __ _ _ _ | _/ _` | | |/ /| '_ \/ _` | ' \ |_| \__,_|_|_/___|_.__/\__,_|_||_| - v0.9.3 2015/08/01 + v0.9.4 2015/03/08 ## Fail2Ban: ban hosts that cause multiple authentication errors -Fail2Ban scans log files like /var/log/pwdfail and bans IP that makes too many -password failures. It updates firewall rules to reject the IP address. These -rules can be defined by the user. Fail2Ban can read multiple log files such as -sshd or Apache web server ones. +Fail2Ban scans log files like `/var/log/auth.log` and bans IP addresses having +too many failed login attempts. It does this by updating system firewall rules +to reject new connections from those IP addresses, for a configurable amount +of time. Fail2Ban comes out-of-the-box ready to read many standard log files, +such as those for sshd and Apache, and is easy to configure to read any log +file you choose, for any error you choose. -Fail2Ban is able to reduce the rate of incorrect authentications attempts -however it cannot eliminate the risk that weak authentication presents. +Though Fail2Ban is able to reduce the rate of incorrect authentications +attempts, it cannot eliminate the risk that weak authentication presents. Configure services to use only two factor or public/private authentication mechanisms if you really want to protect services. @@ -37,12 +39,12 @@ Optional: To install, just do: - tar xvfj fail2ban-0.9.3.tar.bz2 - cd fail2ban-0.9.3 + tar xvfj fail2ban-0.9.4.tar.bz2 + cd fail2ban-0.9.4 python setup.py install This will install Fail2Ban into the python library directory. The executable -scripts are placed into /usr/bin, and configuration under /etc/fail2ban. +scripts are placed into `/usr/bin`, and configuration under `/etc/fail2ban`. Fail2Ban should be correctly installed now. Just type: @@ -51,11 +53,20 @@ Fail2Ban should be correctly installed now. Just type: to see if everything is alright. You should always use fail2ban-client and never call fail2ban-server directly. +Please note that the system init/service script is not automatically installed. +To enable fail2ban as an automatic service, simply copy the script for your +distro from the `files` directory to `/etc/init.d`. Example (on a Debian-based +system): + + cp files/debian-initd /etc/init.d/fail2ban + update-rc.d fail2ban defaults + service fail2ban start + Configuration: -------------- -You can configure Fail2Ban using the files in /etc/fail2ban. It is possible to -configure the server using commands sent to it by fail2ban-client. The +You can configure Fail2Ban using the files in `/etc/fail2ban`. It is possible to +configure the server using commands sent to it by `fail2ban-client`. The available commands are described in the fail2ban-client(1) manpage. Also see fail2ban(1) and jail.conf(5) manpages for further references. @@ -66,6 +77,8 @@ Code status: * [](https://coveralls.io/r/fail2ban/fail2ban) +* [](https://codecov.io/github/fail2ban/fail2ban?branch=master) + Contact: -------- @@ -51,6 +51,11 @@ Preparation find -type f | grep -v -e '\.git' -e '/doc/' -e '\.travis' -e MANIFEST | sed -e 's,^\./,,g' | while read f; do grep -ne "^$f\$" MANIFEST >/dev/null || echo "$f" ; done + or an alternative for comparison with previous release + + git diff 0.9.4 | grep -B2 'index 0000000..' | grep -B1 'new file mode' | sed -n -e '/^diff /s,.* b/,,gp' >> MANIFEST + sort MANIFEST | uniq | sponge MANIFEST + * Run:: python setup.py sdist @@ -61,24 +66,24 @@ Preparation * Which indicates that testcases/files/logs/mysqld.log has been moved or is a directory:: - tar -C /tmp -jxf dist/fail2ban-0.9.3.tar.bz2 + tar -C /tmp -jxf dist/fail2ban-0.9.4.tar.bz2 * clean up current direcory:: - diff -rul --exclude \*.pyc . /tmp/fail2ban-0.9.3/ + diff -rul --exclude \*.pyc . /tmp/fail2ban-0.9.4/ * Only differences should be files that you don't want distributed. * Ensure the tests work from the tarball:: - cd /tmp/fail2ban-0.9.3/ && bin/fail2ban-testcases + cd /tmp/fail2ban-0.9.4/ && bin/fail2ban-testcases * Add/finalize the corresponding entry in the ChangeLog * To generate a list of committers use e.g.:: - git shortlog -sn 0.9.3.. | sed -e 's,^[ 0-9\t]*,,g' | tr '\n' '\|' | sed -e 's:|:, :g' + git shortlog -sn 0.9.4.. | sed -e 's,^[ 0-9\t]*,,g' | tr '\n' '\|' | sed -e 's:|:, :g' * Ensure the top of the ChangeLog has the right version and current date. * Ensure the top entry of the ChangeLog has the right version and current date. @@ -101,7 +106,7 @@ Preparation * Tag the release by using a signed (and annotated) tag. Cut/paste release ChangeLog entry as tag annotation:: - git tag -s 0.9.3 + git tag -s 0.9.4 Pre Release =========== @@ -116,7 +121,7 @@ Pre Release * http://packages.qa.debian.org/f/fail2ban.html - * FreeBSD: Christoph Theis theis@gmx.at>, Nick Hilliard <nick@foobar.org> + * FreeBSD: Christoph Theis theis@gmx.at> * http://svnweb.freebsd.org/ports/head/security/py-fail2ban/Makefile?view=markup * http://www.freebsd.org/cgi/query-pr-summary.cgi?text=fail2ban @@ -185,7 +190,7 @@ Post Release Add the following to the top of the ChangeLog:: - ver. 0.9.4 (2014/XX/XXX) - wanna-be-released + ver. 0.9.5 (2015/XX/XXX) - wanna-be-released ----------- - Fixes: @@ -40,6 +40,7 @@ Eric Gerbier Enrico Labedzki Eugene Hopkinson (SlowRiot) ftoppi +Florian Robert (1technophile) François Boulogne Frantisek Sumsal Frédéric @@ -65,12 +66,14 @@ Joël Bertrand JP Espinosa jserrachinha Justin Shore +Kevin Locke Kévin Drapel kjohnsonecl kojiro Lars Kneschke Lee Clemens leftyfb (Mike Rushton) +M. Maraun Manuel Arostegui Ramirez Marcel Dopita Mark Edgington @@ -87,6 +90,7 @@ Mika (mkl) Nick Munger onorua Orion Poplawski +Pablo Rodriguez Fernandez Paul Marrapese Paul Traina Noel Butler @@ -111,6 +115,7 @@ Steven Hiscocks TESTOVIK Thomas Mayer Tom Pike +Tom Hendrikx Tomas Pihl Tony Lawrence Tomasz Ciolek diff --git a/bin/fail2ban-regex b/bin/fail2ban-regex index b7b0579f..584c1ea7 100755 --- a/bin/fail2ban-regex +++ b/bin/fail2ban-regex @@ -29,563 +29,6 @@ __author__ = "Fail2Ban Developers" __copyright__ = "Copyright (c) 2004-2008 Cyril Jaquier, 2012-2014 Yaroslav Halchenko" __license__ = "GPL" -import getopt -import locale -import logging -import os -import shlex -import sys -import time -import time -import urllib -from optparse import OptionParser, Option +from fail2ban.client.fail2banregex import exec_command_line -from ConfigParser import NoOptionError, NoSectionError, MissingSectionHeaderError - -try: - from systemd import journal - from fail2ban.server.filtersystemd import FilterSystemd -except ImportError: - journal = None - -from fail2ban.version import version -from fail2ban.client.filterreader import FilterReader -from fail2ban.server.filter import Filter -from fail2ban.server.failregex import RegexException - -from fail2ban.helpers import FormatterWithTraceBack, getLogger -# Gets the instance of the logger. -logSys = getLogger("fail2ban") - -def debuggexURL(sample, regex): - q = urllib.urlencode({ 're': regex.replace('<HOST>', '(?&.ipv4)'), - 'str': sample, - 'flavor': 'python' }) - return 'http://www.debuggex.com/?' + q - -def shortstr(s, l=53): - """Return shortened string - """ - if len(s) > l: - return s[:l-3] + '...' - return s - -def pprint_list(l, header=None): - if not len(l): - return - if header: - s = "|- %s\n" % header - else: - s = '' - print s + "| " + "\n| ".join(l) + '\n`-' - -def file_lines_gen(hdlr): - for line in hdlr: - try: - line = line.decode(fail2banRegex.encoding, 'strict') - except UnicodeDecodeError: - if sys.version_info >= (3,): # Python 3 must be decoded - line = line.decode(fail2banRegex.encoding, 'ignore') - yield line - -def journal_lines_gen(myjournal): - while True: - try: - entry = myjournal.get_next() - except OSError: - continue - if not entry: - break - yield FilterSystemd.formatJournalEntry(entry) - -def get_opt_parser(): - # use module docstring for help output - p = OptionParser( - usage="%s [OPTIONS] <LOG> <REGEX> [IGNOREREGEX]\n" % sys.argv[0] + __doc__ - + """ -LOG: - string a string representing a log line - filename path to a log file (/var/log/auth.log) - "systemd-journal" search systemd journal (systemd-python required) - -REGEX: - string a string representing a 'failregex' - filename path to a filter file (filter.d/sshd.conf) - -IGNOREREGEX: - string a string representing an 'ignoreregex' - filename path to a filter file (filter.d/sshd.conf) - -Copyright (c) 2004-2008 Cyril Jaquier, 2008- Fail2Ban Contributors -Copyright of modifications held by their respective authors. -Licensed under the GNU General Public License v2 (GPL). - -Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>. -Many contributions by Yaroslav O. Halchenko and Steven Hiscocks. - -Report bugs to https://github.com/fail2ban/fail2ban/issues -""", - version="%prog " + version) - - p.add_options([ - Option("-d", "--datepattern", - help="set custom pattern used to match date/times"), - Option("-e", "--encoding", - help="File encoding. Default: system locale"), - Option("-L", "--maxlines", type=int, default=0, - help="maxlines for multi-line regex"), - Option("-m", "--journalmatch", - help="journalctl style matches overriding filter file. " - "\"systemd-journal\" only"), - Option('-l', "--log-level", type="choice", - dest="log_level", - choices=('heavydebug', 'debug', 'info', 'notice', 'warning', 'error', 'critical'), - default=None, - help="Log level for the Fail2Ban logger to use"), - Option("-v", "--verbose", action='store_true', - help="Be verbose in output"), - Option("-D", "--debuggex", action='store_true', - help="Produce debuggex.com urls for debugging there"), - Option("--print-no-missed", action='store_true', - help="Do not print any missed lines"), - Option("--print-no-ignored", action='store_true', - help="Do not print any ignored lines"), - Option("--print-all-matched", action='store_true', - help="Print all matched lines"), - Option("--print-all-missed", action='store_true', - help="Print all missed lines, no matter how many"), - Option("--print-all-ignored", action='store_true', - help="Print all ignored lines, no matter how many"), - Option("-t", "--log-traceback", action='store_true', - help="Enrich log-messages with compressed tracebacks"), - Option("--full-traceback", action='store_true', - help="Either to make the tracebacks full, not compressed (as by default)"), - ]) - - return p - - -class RegexStat(object): - - def __init__(self, failregex): - self._stats = 0 - self._failregex = failregex - self._ipList = list() - - def __str__(self): - return "%s(%r) %d failed: %s" \ - % (self.__class__, self._failregex, self._stats, self._ipList) - - def inc(self): - self._stats += 1 - - def getStats(self): - return self._stats - - def getFailRegex(self): - return self._failregex - - def appendIP(self, value): - self._ipList.append(value) - - def getIPList(self): - return self._ipList - - -class LineStats(object): - """Just a convenience container for stats - """ - def __init__(self): - self.tested = self.matched = 0 - self.matched_lines = [] - self.missed = 0 - self.missed_lines = [] - self.missed_lines_timeextracted = [] - self.ignored = 0 - self.ignored_lines = [] - self.ignored_lines_timeextracted = [] - - def __str__(self): - return "%(tested)d lines, %(ignored)d ignored, %(matched)d matched, %(missed)d missed" % self - - # just for convenient str - def __getitem__(self, key): - return getattr(self, key) - - -class Fail2banRegex(object): - - def __init__(self, opts): - self._verbose = opts.verbose - self._debuggex = opts.debuggex - self._maxlines = 20 - self._print_no_missed = opts.print_no_missed - self._print_no_ignored = opts.print_no_ignored - self._print_all_matched = opts.print_all_matched - self._print_all_missed = opts.print_all_missed - self._print_all_ignored = opts.print_all_ignored - self._maxlines_set = False # so we allow to override maxlines in cmdline - self._datepattern_set = False - self._journalmatch = None - - self.share_config=dict() - self._filter = Filter(None) - self._ignoreregex = list() - self._failregex = list() - self._time_elapsed = None - self._line_stats = LineStats() - - if opts.maxlines: - self.setMaxLines(opts.maxlines) - if opts.journalmatch is not None: - self.setJournalMatch(opts.journalmatch.split()) - if opts.datepattern: - self.setDatePattern(opts.datepattern) - if opts.encoding: - self.encoding = opts.encoding - else: - self.encoding = locale.getpreferredencoding() - - - - def setDatePattern(self, pattern): - if not self._datepattern_set: - self._filter.setDatePattern(pattern) - self._datepattern_set = True - if pattern is not None: - print "Use datepattern : %s" % ( - self._filter.getDatePattern()[1], ) - - def setMaxLines(self, v): - if not self._maxlines_set: - self._filter.setMaxLines(int(v)) - self._maxlines_set = True - print "Use maxlines : %d" % self._filter.getMaxLines() - - def setJournalMatch(self, v): - if self._journalmatch is None: - self._journalmatch = v - - def readRegex(self, value, regextype): - assert(regextype in ('fail', 'ignore')) - regex = regextype + 'regex' - if os.path.isfile(value) or os.path.isfile(value + '.conf'): - if os.path.basename(os.path.dirname(value)) == 'filter.d': - ## within filter.d folder - use standard loading algorithm to load filter completely (with .local etc.): - basedir = os.path.dirname(os.path.dirname(value)) - value = os.path.splitext(os.path.basename(value))[0] - print "Use %11s filter file : %s, basedir: %s" % (regex, value, basedir) - reader = FilterReader(value, 'fail2ban-regex-jail', {}, share_config=self.share_config, basedir=basedir) - if not reader.read(): - print "ERROR: failed to load filter %s" % value - return False - else: - ## foreign file - readexplicit this file and includes if possible: - print "Use %11s file : %s" % (regex, value) - reader = FilterReader(value, 'fail2ban-regex-jail', {}, share_config=self.share_config) - reader.setBaseDir(None) - if not reader.readexplicit(): - print "ERROR: failed to read %s" % value - return False - reader.getOptions(None) - readercommands = reader.convert() - regex_values = [ - RegexStat(m[3]) - for m in filter( - lambda x: x[0] == 'set' and x[2] == "add%sregex" % regextype, - readercommands)] - # Read out and set possible value of maxlines - for command in readercommands: - if command[2] == "maxlines": - maxlines = int(command[3]) - try: - self.setMaxLines(maxlines) - except ValueError: - print "ERROR: Invalid value for maxlines (%(maxlines)r) " \ - "read from %(value)s" % locals() - return False - elif command[2] == 'addjournalmatch': - journalmatch = command[3:] - self.setJournalMatch(journalmatch) - elif command[2] == 'datepattern': - datepattern = command[3] - self.setDatePattern(datepattern) - else: - print "Use %11s line : %s" % (regex, shortstr(value)) - regex_values = [RegexStat(value)] - - setattr(self, "_" + regex, regex_values) - for regex in regex_values: - getattr( - self._filter, - 'add%sRegex' % regextype.title())(regex.getFailRegex()) - return True - - def testIgnoreRegex(self, line): - found = False - try: - ret = self._filter.ignoreLine([(line, "", "")]) - if ret is not None: - found = True - regex = self._ignoreregex[ret].inc() - except RegexException, e: - print e - return False - return found - - def testRegex(self, line, date=None): - orgLineBuffer = self._filter._Filter__lineBuffer - fullBuffer = len(orgLineBuffer) >= self._filter.getMaxLines() - try: - line, ret = self._filter.processLine(line, date, checkAllRegex=True) - for match in ret: - # Append True/False flag depending if line was matched by - # more than one regex - match.append(len(ret)>1) - regex = self._failregex[match[0]] - regex.inc() - regex.appendIP(match) - except RegexException, e: - print e - return False - except IndexError: - print "Sorry, but no <HOST> found in regex" - return False - for bufLine in orgLineBuffer[int(fullBuffer):]: - if bufLine not in self._filter._Filter__lineBuffer: - try: - self._line_stats.missed_lines.pop( - self._line_stats.missed_lines.index("".join(bufLine))) - self._line_stats.missed_lines_timeextracted.pop( - self._line_stats.missed_lines_timeextracted.index( - "".join(bufLine[::2]))) - except ValueError: - pass - else: - self._line_stats.matched += 1 - self._line_stats.missed -= 1 - return line, ret - - def process(self, test_lines): - t0 = time.time() - for line_no, line in enumerate(test_lines): - if isinstance(line, tuple): - line_datetimestripped, ret = fail2banRegex.testRegex( - line[0], line[1]) - line = "".join(line[0]) - else: - line = line.rstrip('\r\n') - if line.startswith('#') or not line: - # skip comment and empty lines - continue - line_datetimestripped, ret = fail2banRegex.testRegex(line) - is_ignored = fail2banRegex.testIgnoreRegex(line_datetimestripped) - - if is_ignored: - self._line_stats.ignored += 1 - if not self._print_no_ignored and (self._print_all_ignored or self._line_stats.ignored <= self._maxlines + 1): - self._line_stats.ignored_lines.append(line) - self._line_stats.ignored_lines_timeextracted.append(line_datetimestripped) - - if len(ret) > 0: - assert(not is_ignored) - self._line_stats.matched += 1 - if self._print_all_matched: - self._line_stats.matched_lines.append(line) - else: - if not is_ignored: - self._line_stats.missed += 1 - if not self._print_no_missed and (self._print_all_missed or self._line_stats.missed <= self._maxlines + 1): - self._line_stats.missed_lines.append(line) - self._line_stats.missed_lines_timeextracted.append(line_datetimestripped) - self._line_stats.tested += 1 - - if line_no % 10 == 0 and self._filter.dateDetector is not None: - self._filter.dateDetector.sortTemplate() - self._time_elapsed = time.time() - t0 - - - - def printLines(self, ltype): - lstats = self._line_stats - assert(self._line_stats.missed == lstats.tested - (lstats.matched + lstats.ignored)) - lines = lstats[ltype] - l = lstats[ltype + '_lines'] - if lines: - header = "%s line(s):" % (ltype.capitalize(),) - if self._debuggex: - if ltype == 'missed' or ltype == 'matched': - regexlist = self._failregex - else: - regexlist = self._ignoreregex - l = lstats[ltype + '_lines_timeextracted'] - if lines < self._maxlines or getattr(self, '_print_all_' + ltype): - ans = [[]] - for arg in [l, regexlist]: - ans = [ x + [y] for x in ans for y in arg ] - b = map(lambda a: a[0] + ' | ' + a[1].getFailRegex() + ' | ' + debuggexURL(a[0], a[1].getFailRegex()), ans) - pprint_list([x.rstrip() for x in b], header) - else: - print "%s too many to print. Use --print-all-%s " \ - "to print all %d lines" % (header, ltype, lines) - elif lines < self._maxlines or getattr(self, '_print_all_' + ltype): - pprint_list([x.rstrip() for x in l], header) - else: - print "%s too many to print. Use --print-all-%s " \ - "to print all %d lines" % (header, ltype, lines) - - def printStats(self): - print - print "Results" - print "=======" - - def print_failregexes(title, failregexes): - # Print title - total, out = 0, [] - for cnt, failregex in enumerate(failregexes): - match = failregex.getStats() - total += match - if (match or self._verbose): - out.append("%2d) [%d] %s" % (cnt+1, match, failregex.getFailRegex())) - - if self._verbose and len(failregex.getIPList()): - for ip in failregex.getIPList(): - timeTuple = time.localtime(ip[2]) - timeString = time.strftime("%a %b %d %H:%M:%S %Y", timeTuple) - out.append( - " %s %s%s" % ( - ip[1], - timeString, - ip[-1] and " (multiple regex matched)" or "")) - - print "\n%s: %d total" % (title, total) - pprint_list(out, " #) [# of hits] regular expression") - return total - - # Print title - total = print_failregexes("Failregex", self._failregex) - _ = print_failregexes("Ignoreregex", self._ignoreregex) - - - if self._filter.dateDetector is not None: - print "\nDate template hits:" - out = [] - for template in self._filter.dateDetector.templates: - if self._verbose or template.hits: - out.append("[%d] %s" % ( - template.hits, template.name)) - pprint_list(out, "[# of hits] date format") - - print "\nLines: %s" % self._line_stats, - if self._time_elapsed is not None: - print "[processed in %.2f sec]" % self._time_elapsed, - print - - if self._print_all_matched: - self.printLines('matched') - if not self._print_no_ignored: - self.printLines('ignored') - if not self._print_no_missed: - self.printLines('missed') - - return True - - -if __name__ == "__main__": - - parser = get_opt_parser() - (opts, args) = parser.parse_args() - if opts.print_no_missed and opts.print_all_missed: - sys.stderr.write("ERROR: --print-no-missed and --print-all-missed are mutually exclusive.\n\n") - parser.print_help() - sys.exit(-1) - if opts.print_no_ignored and opts.print_all_ignored: - sys.stderr.write("ERROR: --print-no-ignored and --print-all-ignored are mutually exclusive.\n\n") - parser.print_help() - sys.exit(-1) - - print - print "Running tests" - print "=============" - print - - fail2banRegex = Fail2banRegex(opts) - - # We need 2 or 3 parameters - if not len(args) in (2, 3): - sys.stderr.write("ERROR: provide both <LOG> and <REGEX>.\n\n") - parser.print_help() - sys.exit(-1) - - # TODO: taken from -testcases -- move common functionality somewhere - if opts.log_level is not None: # pragma: no cover - # so we had explicit settings - logSys.setLevel(getattr(logging, opts.log_level.upper())) - else: # pragma: no cover - # suppress the logging but it would leave unittests' progress dots - # ticking, unless like with '-l critical' which would be silent - # unless error occurs - logSys.setLevel(getattr(logging, 'CRITICAL')) - - # Add the default logging handler - stdout = logging.StreamHandler(sys.stdout) - - fmt = 'D: %(message)s' - - if opts.log_traceback: - Formatter = FormatterWithTraceBack - fmt = (opts.full_traceback and ' %(tb)s' or ' %(tbc)s') + fmt - else: - Formatter = logging.Formatter - - # Custom log format for the verbose tests runs - if opts.verbose: # pragma: no cover - stdout.setFormatter(Formatter(' %(asctime)-15s %(thread)s' + fmt)) - else: # pragma: no cover - # just prefix with the space - stdout.setFormatter(Formatter(fmt)) - logSys.addHandler(stdout) - - cmd_log, cmd_regex = args[:2] - - fail2banRegex.readRegex(cmd_regex, 'fail') or sys.exit(-1) - - if len(args) == 3: - fail2banRegex.readRegex(args[2], 'ignore') or sys.exit(-1) - - if os.path.isfile(cmd_log): - try: - hdlr = open(cmd_log, 'rb') - print "Use log file : %s" % cmd_log - print "Use encoding : %s" % fail2banRegex.encoding - test_lines = file_lines_gen(hdlr) - except IOError, e: - print e - sys.exit(-1) - elif cmd_log == "systemd-journal": - if not journal: - print "Error: systemd library not found. Exiting..." - sys.exit(-1) - myjournal = journal.Reader(converters={'__CURSOR': lambda x: x}) - journalmatch = fail2banRegex._journalmatch - fail2banRegex.setDatePattern(None) - if journalmatch: - try: - for element in journalmatch: - if element == "+": - myjournal.add_disjunction() - else: - myjournal.add_match(element) - except ValueError: - print "Error: Invalid journalmatch: %s" % shortstr(" ".join(journalmatch)) - sys.exit(-1) - print "Use journal match : %s" % " ".join(journalmatch) - test_lines = journal_lines_gen(myjournal) - else: - print "Use single line : %s" % shortstr(cmd_log) - test_lines = [ cmd_log ] - print - - fail2banRegex.process(test_lines) - - fail2banRegex.printStats() or sys.exit(-1) +exec_command_line() diff --git a/config/action.d/badips.conf b/config/action.d/badips.conf index 4a5c0f97..70b46546 100644 --- a/config/action.d/badips.conf +++ b/config/action.d/badips.conf @@ -10,7 +10,7 @@ [Definition] -actionban = curl --fail --user-agent "fail2ban v0.8.12" http://www.badips.com/add/<category>/<ip> +actionban = curl --fail --user-agent "<agent>" http://www.badips.com/add/<category>/<ip> [Init] diff --git a/config/action.d/badips.py b/config/action.d/badips.py index a1df00a3..025289ca 100644 --- a/config/action.d/badips.py +++ b/config/action.d/badips.py @@ -21,7 +21,6 @@ import sys if sys.version_info < (2, 7): raise ImportError("badips.py action requires Python >= 2.7") import json -from functools import partial import threading import logging if sys.version_info >= (3, ): @@ -33,7 +32,6 @@ else: from urllib import urlencode from fail2ban.server.actions import ActionBase -from fail2ban.version import version as f2bVersion class BadIPsAction(ActionBase): @@ -72,6 +70,9 @@ class BadIPsAction(ActionBase): updateperiod : int, optional Time in seconds between updating bad IPs blacklist. Default 900 (15 minutes) + agent : str, optional + User agent transmitted to server. + Default `Fail2Ban/ver.` Raises ------ @@ -80,13 +81,14 @@ class BadIPsAction(ActionBase): """ _badips = "http://www.badips.com" - _Request = partial( - Request, headers={'User-Agent': "Fail2Ban %s" % f2bVersion}) + def _Request(self, url, **argv): + return Request(url, headers={'User-Agent': self.agent}, **argv) def __init__(self, jail, name, category, score=3, age="24h", key=None, - banaction=None, bancategory=None, bankey=None, updateperiod=900): + banaction=None, bancategory=None, bankey=None, updateperiod=900, agent="Fail2Ban"): super(BadIPsAction, self).__init__(jail, name) + self.agent = agent self.category = category self.score = score self.age = age @@ -117,7 +119,7 @@ class BadIPsAction(ActionBase): """ try: response = urlopen( - self._Request("/".join([self._badips, "get", "categories"]))) + self._Request("/".join([self._badips, "get", "categories"])), None, 3) except HTTPError as response: messages = json.loads(response.read().decode('utf-8')) self._logSys.error( diff --git a/config/action.d/blocklist_de.conf b/config/action.d/blocklist_de.conf index 6d520694..2f31d8b9 100644 --- a/config/action.d/blocklist_de.conf +++ b/config/action.d/blocklist_de.conf @@ -54,7 +54,7 @@ actioncheck = # Tags: See jail.conf(5) man page # Values: CMD # -actionban = curl --fail --data-urlencode 'server=<email>' --data 'apikey=<apikey>' --data 'service=<service>' --data 'ip=<ip>' --data-urlencode 'logs=<matches>' --data 'format=text' --user-agent "fail2ban v0.8.12" "https://www.blocklist.de/en/httpreports.html" +actionban = curl --fail --data-urlencode 'server=<email>' --data 'apikey=<apikey>' --data 'service=<service>' --data 'ip=<ip>' --data-urlencode 'logs=<matches>' --data 'format=text' --user-agent "<agent>" "https://www.blocklist.de/en/httpreports.html" # Option: actionunban # Notes.: command executed when unbanning an IP. Take care that the diff --git a/config/action.d/cloudflare.conf b/config/action.d/cloudflare.conf index 4bc90c97..aa87163c 100644 --- a/config/action.d/cloudflare.conf +++ b/config/action.d/cloudflare.conf @@ -9,6 +9,8 @@ # Referenced from http://www.normyee.net/blog/2012/02/02/adding-cloudflare-support-to-fail2ban by NORM YEE # # To get your CloudFlare API Key: https://www.cloudflare.com/a/account/my-account +# +# CloudFlare API error codes: https://www.cloudflare.com/docs/host-api.html#s4.2 [Definition] diff --git a/config/action.d/mynetwatchman.conf b/config/action.d/mynetwatchman.conf index 5245a4e3..8f3edf9e 100644 --- a/config/action.d/mynetwatchman.conf +++ b/config/action.d/mynetwatchman.conf @@ -111,13 +111,17 @@ myip = `ip -4 addr show dev eth0 | grep inet | head -n 1 | sed -r 's/.*inet ([0- # protocol = tcp +# Option: agent +# Default: Fail2ban +agent = Fail2ban + # Option: getcmd # Notes.: A command to fetch a URL. Should output page to STDOUT # Values: CMD Default: wget # -getcmd = wget --no-verbose --tries=3 --waitretry=10 --connect-timeout=10 --read-timeout=60 --retry-connrefused --output-document=- --user-agent=Fail2Ban +getcmd = wget --no-verbose --tries=3 --waitretry=10 --connect-timeout=10 --read-timeout=60 --retry-connrefused --output-document=- --user-agent=<agent> # Alternative value: -# getcmd = curl --silent --show-error --retry 3 --connect-timeout 10 --max-time 60 --user-agent Fail2Ban +# getcmd = curl --silent --show-error --retry 3 --connect-timeout 10 --max-time 60 --user-agent <agent> # Option: srcport # Notes.: The source port of the attack. You're unlikely to have this info, so diff --git a/config/action.d/nftables-allports.conf b/config/action.d/nftables-allports.conf new file mode 100644 index 00000000..afd0ca84 --- /dev/null +++ b/config/action.d/nftables-allports.conf @@ -0,0 +1,22 @@ +# Fail2Ban configuration file +# +# Author: Cyril Jaquier +# Modified: Yaroslav O. Halchenko <debian@onerussian.com> +# made active on all ports from original iptables.conf +# Modified: Alexander Belykh <albel727@ngs.ru> +# adapted for nftables +# + +[INCLUDES] + +before = nftables-common.conf + +[Definition] + +# Option: nftables_mode +# Notes.: additional expressions for nftables filter rule +# Values: nftables expressions +# +nftables_mode = ip protocol <protocol> + +[Init] diff --git a/config/action.d/nftables-common.conf b/config/action.d/nftables-common.conf new file mode 100644 index 00000000..80657c5c --- /dev/null +++ b/config/action.d/nftables-common.conf @@ -0,0 +1,119 @@ +# Fail2Ban configuration file +# +# Author: Daniel Black +# Author: Cyril Jaquier +# Modified: Yaroslav O. Halchenko <debian@onerussian.com> +# made active on all ports from original iptables.conf +# Modified: Alexander Belykh <albel727@ngs.ru> +# adapted for nftables +# +# This is a included configuration file and includes the definitions for the nftables +# used in all nftables based actions by default. +# +# The user can override the defaults in nftables-common.local + +[INCLUDES] + +after = nftables-common.local + +[Definition] + +# Option: nftables_mode +# Notes.: additional expressions for nftables filter rule +# Values: nftables expressions +# +nftables_mode = <protocol> dport \{ <port> \} + +# Option: actionstart +# Notes.: command executed once at the start of Fail2Ban. +# Values: CMD +# +actionstart = <nftables> add set <nftables_family> <nftables_table> f2b-<name> \{ type <nftables_type>\; \} + <nftables> insert rule <nftables_family> <nftables_table> <chain> %(nftables_mode)s ip saddr @f2b-<name> <blocktype> + +_nft_list = <nftables> --handle --numeric list chain <nftables_family> <nftables_table> <chain> +_nft_get_handle_id = grep -m1 'ip saddr @f2b-<name> <blocktype> # handle' | grep -oe ' handle [0-9]*' + +# Option: actionstop +# Notes.: command executed once at the end of Fail2Ban +# Values: CMD +# +actionstop = HANDLE_ID=$(%(_nft_list)s | %(_nft_get_handle_id)s) + <nftables> delete rule <nftables_family> <nftables_table> <chain> $HANDLE_ID + <nftables> delete set <nftables_family> <nftables_table> f2b-<name> + +# Option: actioncheck +# Notes.: command executed once before each actionban command +# Values: CMD +# +actioncheck = <nftables> list chain <nftables_family> <nftables_table> <chain> | grep -q '@f2b-<name>[ \t]' + +# Option: actionban +# Notes.: command executed when banning an IP. Take care that the +# command is executed with Fail2Ban user rights. +# Tags: See jail.conf(5) man page +# Values: CMD +# +actionban = <nftables> add element <nftables_family> <nftables_table> f2b-<name> \{ <ip> \} + +# Option: actionunban +# Notes.: command executed when unbanning an IP. Take care that the +# command is executed with Fail2Ban user rights. +# Tags: See jail.conf(5) man page +# Values: CMD +# +actionunban = <nftables> delete element <nftables_family> <nftables_table> f2b-<name> \{ <ip> \} + +[Init] + +# Option: nftables_type +# Notes.: address type to work with +# Values: [ipv4_addr | ipv6_addr] Default: ipv4_addr +# +nftables_type = ipv4_addr + +# Option: nftables_family +# Notes.: address family to work in +# Values: [ip | ip6 | inet] Default: inet +# +nftables_family = inet + +# Option: nftables_table +# Notes.: table in the address family to work in +# Values: STRING Default: filter +# +nftables_table = filter + +# Option: chain +# Notes specifies the nftables chain to which the Fail2Ban rules should be +# added +# Values: STRING Default: input +chain = input + +# Default name of the filtering set +# +name = default + +# Option: port +# Notes.: specifies port to monitor +# Values: [ NUM | STRING ] Default: +# +port = ssh + +# Option: protocol +# Notes.: internally used by config reader for interpolations. +# Values: [ tcp | udp ] Default: tcp +# +protocol = tcp + +# Option: blocktype +# Note: This is what the action does with rules. This can be any jump target +# as per the nftables man page (section 8). Common values are drop +# reject, reject with icmp type host-unreachable +# Values: STRING +blocktype = reject + +# Option: nftables +# Notes.: Actual command to be executed, including common to all calls options +# Values: STRING +nftables = nft diff --git a/config/action.d/nftables-multiport.conf b/config/action.d/nftables-multiport.conf new file mode 100644 index 00000000..d1afafb3 --- /dev/null +++ b/config/action.d/nftables-multiport.conf @@ -0,0 +1,22 @@ +# Fail2Ban configuration file +# +# Author: Cyril Jaquier +# Modified: Yaroslav O. Halchenko <debian@onerussian.com> +# made active on all ports from original iptables.conf +# Modified: Alexander Belykh <albel727@ngs.ru> +# adapted for nftables +# + +[INCLUDES] + +before = nftables-common.conf + +[Definition] + +# Option: nftables_mode +# Notes.: additional expressions for nftables filter rule +# Values: nftables expressions +# +nftables_mode = <protocol> dport \{ <port> \} + +[Init] diff --git a/config/action.d/route.conf b/config/action.d/route.conf index 123245e5..9b96a7b5 100644 --- a/config/action.d/route.conf +++ b/config/action.d/route.conf @@ -17,6 +17,9 @@ [Definition] actionban = ip route add <blocktype> <ip> actionunban = ip route del <blocktype> <ip> +actioncheck = +actionstart = +actionstop = [Init] diff --git a/config/filter.d/apache-badbots.conf b/config/filter.d/apache-badbots.conf index 75c0de03..48b30666 100644 --- a/config/filter.d/apache-badbots.conf +++ b/config/filter.d/apache-badbots.conf @@ -8,7 +8,7 @@ [Definition] badbotscustom = EmailCollector|WebEMailExtrac|TrackBack/1\.02|sogou music spider -badbots = Atomic_Email_Hunter/4\.0|atSpider/1\.0|autoemailspider|bwh3_user_agent|China Local Browse 2\.6|ContactBot/0\.2|ContentSmartz|DataCha0s/2\.0|DBrowse 1\.4b|DBrowse 1\.4d|Demo Bot DOT 16b|Demo Bot Z 16b|DSurf15a 01|DSurf15a 71|DSurf15a 81|DSurf15a VA|EBrowse 1\.4b|Educate Search VxB|EmailSiphon|EmailSpider|EmailWolf 1\.00|ESurf15a 15|ExtractorPro|Franklin Locator 1\.8|FSurf15a 01|Full Web Bot 0416B|Full Web Bot 0516B|Full Web Bot 2816B|Guestbook Auto Submitter|Industry Program 1\.0\.x|ISC Systems iRc Search 2\.1|IUPUI Research Bot v 1\.9a|LARBIN-EXPERIMENTAL \(efp@gmx\.net\)|LetsCrawl\.com/1\.0 +http\://letscrawl\.com/|Lincoln State Web Browser|LMQueueBot/0\.2|LWP\:\:Simple/5\.803|Mac Finder 1\.0\.xx|MFC Foundation Class Library 4\.0|Microsoft URL Control - 6\.00\.8xxx|Missauga Locate 1\.0\.0|Missigua Locator 1\.9|Missouri College Browse|Mizzu Labs 2\.2|Mo College 1\.9|MVAClient|Mozilla/2\.0 \(compatible; NEWT ActiveX; Win32\)|Mozilla/3\.0 \(compatible; Indy Library\)|Mozilla/3\.0 \(compatible; scan4mail \(advanced version\) http\://www\.peterspages\.net/?scan4mail\)|Mozilla/4\.0 \(compatible; Advanced Email Extractor v2\.xx\)|Mozilla/4\.0 \(compatible; Iplexx Spider/1\.0 http\://www\.iplexx\.at\)|Mozilla/4\.0 \(compatible; MSIE 5\.0; Windows NT; DigExt; DTS Agent|Mozilla/4\.0 efp@gmx\.net|Mozilla/5\.0 \(Version\: xxxx Type\:xx\)|NameOfAgent \(CMS Spider\)|NASA Search 1\.0|Nsauditor/1\.x|PBrowse 1\.4b|PEval 1\.4b|Poirot|Port Huron Labs|Production Bot 0116B|Production Bot 2016B|Production Bot DOT 3016B|Program Shareware 1\.0\.2|PSurf15a 11|PSurf15a 51|PSurf15a VA|psycheclone|RSurf15a 41|RSurf15a 51|RSurf15a 81|searchbot admin@google\.com|ShablastBot 1\.0|snap\.com beta crawler v0|Snapbot/1\.0|Snapbot/1\.0 \(Snap Shots, +http\://www\.snap\.com\)|sogou develop spider|Sogou Orion spider/3\.0\(+http\://www\.sogou\.com/docs/help/webmasters\.htm#07\)|sogou spider|Sogou web spider/3\.0\(+http\://www\.sogou\.com/docs/help/webmasters\.htm#07\)|sohu agent|SSurf15a 11 |TSurf15a 11|Under the Rainbow 2\.2|User-Agent\: Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1\)|VadixBot|WebVulnCrawl\.unknown/1\.0 libwww-perl/5\.803|Wells Search II|WEP Search 00 +badbots = Atomic_Email_Hunter/4\.0|atSpider/1\.0|autoemailspider|bwh3_user_agent|China Local Browse 2\.6|ContactBot/0\.2|ContentSmartz|DataCha0s/2\.0|DBrowse 1\.4b|DBrowse 1\.4d|Demo Bot DOT 16b|Demo Bot Z 16b|DSurf15a 01|DSurf15a 71|DSurf15a 81|DSurf15a VA|EBrowse 1\.4b|Educate Search VxB|EmailSiphon|EmailSpider|EmailWolf 1\.00|ESurf15a 15|ExtractorPro|Franklin Locator 1\.8|FSurf15a 01|Full Web Bot 0416B|Full Web Bot 0516B|Full Web Bot 2816B|Guestbook Auto Submitter|Industry Program 1\.0\.x|ISC Systems iRc Search 2\.1|IUPUI Research Bot v 1\.9a|LARBIN-EXPERIMENTAL \(efp@gmx\.net\)|LetsCrawl\.com/1\.0 \+http\://letscrawl\.com/|Lincoln State Web Browser|LMQueueBot/0\.2|LWP\:\:Simple/5\.803|Mac Finder 1\.0\.xx|MFC Foundation Class Library 4\.0|Microsoft URL Control - 6\.00\.8xxx|Missauga Locate 1\.0\.0|Missigua Locator 1\.9|Missouri College Browse|Mizzu Labs 2\.2|Mo College 1\.9|MVAClient|Mozilla/2\.0 \(compatible; NEWT ActiveX; Win32\)|Mozilla/3\.0 \(compatible; Indy Library\)|Mozilla/3\.0 \(compatible; scan4mail \(advanced version\) http\://www\.peterspages\.net/?scan4mail\)|Mozilla/4\.0 \(compatible; Advanced Email Extractor v2\.xx\)|Mozilla/4\.0 \(compatible; Iplexx Spider/1\.0 http\://www\.iplexx\.at\)|Mozilla/4\.0 \(compatible; MSIE 5\.0; Windows NT; DigExt; DTS Agent|Mozilla/4\.0 efp@gmx\.net|Mozilla/5\.0 \(Version\: xxxx Type\:xx\)|NameOfAgent \(CMS Spider\)|NASA Search 1\.0|Nsauditor/1\.x|PBrowse 1\.4b|PEval 1\.4b|Poirot|Port Huron Labs|Production Bot 0116B|Production Bot 2016B|Production Bot DOT 3016B|Program Shareware 1\.0\.2|PSurf15a 11|PSurf15a 51|PSurf15a VA|psycheclone|RSurf15a 41|RSurf15a 51|RSurf15a 81|searchbot admin@google\.com|ShablastBot 1\.0|snap\.com beta crawler v0|Snapbot/1\.0|Snapbot/1\.0 \(Snap Shots, \+http\://www\.snap\.com\)|sogou develop spider|Sogou Orion spider/3\.0\(\+http\://www\.sogou\.com/docs/help/webmasters\.htm#07\)|sogou spider|Sogou web spider/3\.0\(\+http\://www\.sogou\.com/docs/help/webmasters\.htm#07\)|sohu agent|SSurf15a 11 |TSurf15a 11|Under the Rainbow 2\.2|User-Agent\: Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1\)|VadixBot|WebVulnCrawl\.unknown/1\.0 libwww-perl/5\.803|Wells Search II|WEP Search 00 failregex = ^<HOST> -.*"(GET|POST|HEAD).*HTTP.*"(?:%(badbots)s|%(badbotscustom)s)"$ diff --git a/config/filter.d/asterisk.conf b/config/filter.d/asterisk.conf index b446c44e..3975fb29 100644 --- a/config/filter.d/asterisk.conf +++ b/config/filter.d/asterisk.conf @@ -19,7 +19,7 @@ iso8601 = \d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d+[+-]\d{4} log_prefix= (?:NOTICE|SECURITY)%(__pid_re)s:?(?:\[C-[\da-f]*\])? \S+:\d*( in \w+:)? failregex = ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?' - (Wrong password|Username/auth name mismatch|No matching peer found|Not a local domain|Device does not match ACL|Peer is not supposed to register|ACL error \(permit/deny\)|Not a local domain)$ - ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Call from '[^']*' \(<HOST>:\d+\) to extension '\d+' rejected because extension not found in context 'default'\.$ + ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Call from '[^']*' \(<HOST>:\d+\) to extension '[^']*' rejected because extension not found in context ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Host <HOST> failed to authenticate as '[^']*'$ ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s No registration for peer '[^']*' \(from <HOST>\)$ ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Host <HOST> failed MD5 authentication for '[^']*' \([^)]+\)$ diff --git a/config/filter.d/botsearch-common.conf b/config/filter.d/botsearch-common.conf index 65f9da2f..a673a310 100644 --- a/config/filter.d/botsearch-common.conf +++ b/config/filter.d/botsearch-common.conf @@ -11,9 +11,9 @@ webmail = roundcube|(ext)?mail|horde|(v-?)?webmail phpmyadmin = (typo3/|xampp/|admin/|)(pma|(php)?[Mm]y[Aa]dmin) -wordpress = wp-(login|signup)\.php +wordpress = wp-(login|signup|admin)\.php # DEV Notes: # Taken from apache-botsearch filter # -# Author: Frantisek Sumsal
\ No newline at end of file +# Author: Frantisek Sumsal diff --git a/config/filter.d/haproxy-http-auth.conf b/config/filter.d/haproxy-http-auth.conf new file mode 100644 index 00000000..298ca292 --- /dev/null +++ b/config/filter.d/haproxy-http-auth.conf @@ -0,0 +1,37 @@ +# Fail2Ban filter configuration file to match failed login attempts to +# HAProxy HTTP Authentication protected servers. +# +# PLEASE NOTE - When a user first hits the HTTP Auth a 401 is returned by the server +# which prompts their browser to ask for login details. +# This initial 401 is logged by HAProxy. +# In other words, even successful logins will have at least 1 fail regex match. +# Please keep this in mind when setting findtime and maxretry for jails. +# +# Author: Jordan Moeser +# + +[INCLUDES] + +# Read common prefixes. If any customizations available -- read them from +# common.local +before = common.conf + + +[Definition] + +_daemon = haproxy + +# Option: failregex +# Notes.: regex to match the password failures messages in the logfile. The +# host must be matched by a group named "host". The tag "<HOST>" can +# be used for standard IP/hostname matching and is only an alias for +# (?:::f{4,6}:)?(?P<host>[\w\-.^_]+) +# Values: TEXT +# +failregex = ^%(__prefix_line)s<HOST>.*<NOSRV> -1/-1/-1/-1/\+*\d* 401 + +# Option: ignoreregex +# Notes.: regex to ignore. If this regex matches, the line is ignored. +# Values: TEXT +# +ignoreregex = diff --git a/config/filter.d/ignorecommands/apache-fakegooglebot b/config/filter.d/ignorecommands/apache-fakegooglebot index 47ef51f6..19fb5107 100755 --- a/config/filter.d/ignorecommands/apache-fakegooglebot +++ b/config/filter.d/ignorecommands/apache-fakegooglebot @@ -26,7 +26,10 @@ def is_googlebot(ip): from fail2ban.server.filter import DNSUtils host = DNSUtils.ipToName(ip) - sys.exit(0 if (host and re.match('crawl-.*\.googlebot\.com', host)) else 1) + if not host or not re.match('.*\.google(bot)?\.com$', host): + sys.exit(1) + host_ips = DNSUtils.dnsToIp(host) + sys.exit(0 if ip in host_ips else 1) if __name__ == '__main__': is_googlebot(process_args(sys.argv)) diff --git a/config/filter.d/murmur.conf b/config/filter.d/murmur.conf new file mode 100644 index 00000000..3775a9d2 --- /dev/null +++ b/config/filter.d/murmur.conf @@ -0,0 +1,28 @@ +# Fail2Ban filter for murmur/mumble-server +# + +[INCLUDES] + +before = common.conf + + +[Definition] + +_daemon = murmurd + +# N.B. If you allow users to have usernames that include the '>' character you +# should change this to match the regex assigned to the 'username' +# variable in your server config file (murmur.ini / mumble-server.ini). +_usernameregex = [^>]+ + +_prefix = <W>[\n\s]*(\.\d{3})?\s+\d+ => <\d+:%(_usernameregex)s\(-1\)> Rejected connection from <HOST>:\d+: + +failregex = ^%(_prefix)s Invalid server password$ + ^%(_prefix)s Wrong certificate or password for existing user$ + +ignoreregex = + + +# DEV Notes: +# +# Author: Ross Brown diff --git a/config/filter.d/mysqld-auth.conf b/config/filter.d/mysqld-auth.conf index 92dc9a99..3ad70cb7 100644 --- a/config/filter.d/mysqld-auth.conf +++ b/config/filter.d/mysqld-auth.conf @@ -17,7 +17,7 @@ before = common.conf _daemon = mysqld -failregex = ^%(__prefix_line)s(\d{6} \s?\d{1,2}:\d{2}:\d{2} )?\[Warning\] Access denied for user '\w+'@'<HOST>' (to database '[^']*'|\(using password: (YES|NO)\))*\s*$ +failregex = ^%(__prefix_line)s(?:\d+ |\d{6} \s?\d{1,2}:\d{2}:\d{2} )?\[\w+\] Access denied for user '[^']+'@'<HOST>' (to database '[^']*'|\(using password: (YES|NO)\))*\s*$ ignoreregex = diff --git a/config/filter.d/nginx-limit-req.conf b/config/filter.d/nginx-limit-req.conf new file mode 100644 index 00000000..589d3d78 --- /dev/null +++ b/config/filter.d/nginx-limit-req.conf @@ -0,0 +1,45 @@ +# Fail2ban filter configuration for nginx :: limit_req +# used to ban hosts, that were failed through nginx by limit request processing rate +# +# Author: Serg G. Brester (sebres) +# +# To use 'nginx-limit-req' filter you should have `ngx_http_limit_req_module` +# and define `limit_req` and `limit_req_zone` as described in nginx documentation +# http://nginx.org/en/docs/http/ngx_http_limit_req_module.html +# +# Example: +# +# http { +# ... +# limit_req_zone $binary_remote_addr zone=lr_zone:10m rate=1r/s; +# ... +# # http, server, or location: +# location ... { +# limit_req zone=lr_zone burst=1 nodelay; +# ... +# } +# ... +# } +# ... +# + +[Definition] + +# Specify following expression to define exact zones, if you want to ban IPs limited +# from specified zones only. +# Example: +# +# ngx_limit_req_zones = lr_zone|lr_zone2 +# +ngx_limit_req_zones = [^"]+ + +# Use following full expression if you should range limit request to specified +# servers, requests, referrers etc. only : +# +# failregex = ^\s*\[error\] \d+#\d+: \*\d+ limiting requests, excess: [\d\.]+ by zone "(?:%(ngx_limit_req_zones)s)", client: <HOST>, server: \S*, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"(, referrer: "\S+")?\s*$ + +# Shortly, much faster and stable version of regexp: +failregex = ^\s*\[error\] \d+#\d+: \*\d+ limiting requests, excess: [\d\.]+ by zone "(?:%(ngx_limit_req_zones)s)", client: <HOST> + +ignoreregex = + diff --git a/config/filter.d/openhab.conf b/config/filter.d/openhab.conf new file mode 100644 index 00000000..83857c7a --- /dev/null +++ b/config/filter.d/openhab.conf @@ -0,0 +1,16 @@ +# Openhab brute force auth filter: /etc/fail2ban/filter.d/openhab.conf: +# +# Block IPs trying to auth openhab by web or rest api +# +# Matches e.g. +# 12.34.33.22 - - [26/sept./2015:18:04:43 +0200] "GET /openhab.app HTTP/1.1" 401 1382 +# 175.18.15.10 - - [02/sept./2015:00:11:31 +0200] "GET /rest/bindings HTTP/1.1" 401 1384 + +[Definition] +failregex = ^<HOST>\s+-\s+-\s+\[\]\s+"[A-Z]+ .*" 401 \d+\s*$ + +[Init] +datepattern = %%d/%%b[^/]*/%%Y:%%H:%%M:%%S %%z + + + diff --git a/config/filter.d/postfix-rbl.conf b/config/filter.d/postfix-rbl.conf index 05a8bbc7..c3f8c332 100644 --- a/config/filter.d/postfix-rbl.conf +++ b/config/filter.d/postfix-rbl.conf @@ -10,7 +10,7 @@ before = common.conf [Definition] -_daemon = postfix/smtpd +_daemon = postfix(-\w+)?/smtpd failregex = ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 454 4\.7\.1 Service unavailable; Client host \[\S+\] blocked using .* from=<\S*> to=<\S+> proto=ESMTP helo=<\S*>$ diff --git a/config/filter.d/postfix-sasl.conf b/config/filter.d/postfix-sasl.conf index 199e29bf..7ff995c9 100644 --- a/config/filter.d/postfix-sasl.conf +++ b/config/filter.d/postfix-sasl.conf @@ -7,7 +7,7 @@ before = common.conf [Definition] -_daemon = postfix/(submission/)?smtp(d|s) +_daemon = postfix(-\w+)?/(submission/)?smtp(d|s) failregex = ^%(__prefix_line)swarning: [-._\w]+\[<HOST>\]: SASL ((?i)LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed(: [ A-Za-z0-9+/:]*={0,2})?\s*$ diff --git a/config/filter.d/postfix.conf b/config/filter.d/postfix.conf index a994d772..002b02b2 100644 --- a/config/filter.d/postfix.conf +++ b/config/filter.d/postfix.conf @@ -10,12 +10,14 @@ before = common.conf [Definition] -_daemon = postfix/(submission/)?smtp(d|s) +_daemon = postfix(-\w+)?/(submission/)?smtp(d|s) failregex = ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 554 5\.7\.1 .*$ ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 450 4\.7\.1 Client host rejected: cannot find your hostname, (\[\S*\]); from=<\S*> to=<\S+> proto=ESMTP helo=<\S*>$ ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 450 4\.7\.1 : Helo command rejected: Host not found; from=<> to=<> proto=ESMTP helo= *$ + ^%(__prefix_line)sNOQUEUE: reject: EHLO from \S+\[<HOST>\]: 504 5\.5\.2 <\S+>: Helo command rejected: need fully-qualified hostname; ^%(__prefix_line)sNOQUEUE: reject: VRFY from \S+\[<HOST>\]: 550 5\.1\.1 .*$ + ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 450 4\.1\.8 <\S*>: Sender address rejected: Domain not found; from=<\S*> to=<\S+> proto=ESMTP helo=<\S*>$ ^%(__prefix_line)simproper command pipelining after \S+ from [^[]*\[<HOST>\]:?$ ignoreregex = diff --git a/config/filter.d/screensharingd.conf b/config/filter.d/screensharingd.conf new file mode 100644 index 00000000..4cd76465 --- /dev/null +++ b/config/filter.d/screensharingd.conf @@ -0,0 +1,31 @@ +# Fail2Ban configuration file +# +# Author: Simon Brown +# +# Filter for Mac OS X Screen Sharing service + +[INCLUDES] + +# Read common prefixes. If any customizations available -- read them from +# common.local +before = common.conf + + +[Definition] + +_daemon = screensharingd + +# Option: failregex +# Notes.: regex to match the password failures messages in the logfile. The +# host must be matched by a group named "host". The tag "<HOST>" can +# be used for standard IP/hostname matching and is only an alias for +# (?:::f{4,6}:)?(?P<host>[\w\-.^_]+) +# Values: TEXT +# +failregex = ^%(__prefix_line)sAuthentication: FAILED :: User Name: .+ :: Viewer Address: <HOST> :: Type: DH$ + +# Option: ignoreregex +# Notes.: regex to ignore. If this regex matches, the line is ignored. +# Values: TEXT +# +ignoreregex = diff --git a/config/filter.d/sshd.conf b/config/filter.d/sshd.conf index b000cd49..180ac52a 100644 --- a/config/filter.d/sshd.conf +++ b/config/filter.d/sshd.conf @@ -27,12 +27,13 @@ failregex = ^%(__prefix_line)s(?:error: PAM: )?[aA]uthentication (?:failure|erro ^%(__prefix_line)sUser .+ from <HOST> not allowed because listed in DenyUsers\s*$ ^%(__prefix_line)sUser .+ from <HOST> not allowed because not in any group\s*$ ^%(__prefix_line)srefused connect from \S+ \(<HOST>\)\s*$ - ^%(__prefix_line)sReceived disconnect from <HOST>: 3: \S+: Auth fail$ + ^%(__prefix_line)s(?:error: )?Received disconnect from <HOST>: 3: .*: Auth fail(?: \[preauth\])?$ ^%(__prefix_line)sUser .+ from <HOST> not allowed because a group is listed in DenyGroups\s*$ ^%(__prefix_line)sUser .+ from <HOST> not allowed because none of user's groups are listed in AllowGroups\s*$ ^(?P<__prefix>%(__prefix_line)s)User .+ not allowed because account is locked<SKIPLINES>(?P=__prefix)(?:error: )?Received disconnect from <HOST>: 11: .+ \[preauth\]$ ^(?P<__prefix>%(__prefix_line)s)Disconnecting: Too many authentication failures for .+? \[preauth\]<SKIPLINES>(?P=__prefix)(?:error: )?Connection closed by <HOST> \[preauth\]$ ^(?P<__prefix>%(__prefix_line)s)Connection from <HOST> port \d+(?: on \S+ port \d+)?<SKIPLINES>(?P=__prefix)Disconnecting: Too many authentication failures for .+? \[preauth\]$ + ^%(__prefix_line)s(error: )?maximum authentication attempts exceeded for .* from <HOST>(?: port \d*)?(?: ssh\d*)? \[preauth\]$ ^%(__prefix_line)spam_unix\(sshd:auth\):\s+authentication failure;\s*logname=\S*\s*uid=\d*\s*euid=\d*\s*tty=\S*\s*ruser=\S*\s*rhost=<HOST>\s.*$ ignoreregex = diff --git a/config/jail.conf b/config/jail.conf index f545ff13..cea9a06e 100644 --- a/config/jail.conf +++ b/config/jail.conf @@ -46,7 +46,7 @@ before = paths-debian.conf # "ignoreip" can be an IP address, a CIDR mask or a DNS host. Fail2ban will not # ban a host which matches an address in this list. Several addresses can be -# defined using space separator. +# defined using space (and/or comma) separator. ignoreip = 127.0.0.1/8 # External command that will take an tagged arguments to ignore, e.g. <ip>, @@ -80,7 +80,7 @@ maxretry = 5 # auto: will try to use the following backends, in order: # pyinotify, gamin, polling. # -# Note: if systemd backend is choses as the default but you enable a jail +# Note: if systemd backend is chosen as the default but you enable a jail # for which logs are present only in its own log files, specify some other # backend for that jail (e.g. polling) and provide empty value for # journalmatch. See https://github.com/fail2ban/fail2ban/issues/959#issuecomment-74901200 @@ -146,6 +146,9 @@ chain = INPUT # Usually should be overridden in a particular jail port = 0:65535 +# Format of user-agent https://tools.ietf.org/html/rfc7231#section-5.5.3 +fail2ban_agent = Fail2Ban/%(fail2ban_version)s + # # Action shortcuts. To be used to define action parameter @@ -154,18 +157,19 @@ port = 0:65535 # action_* variables. Can be overridden globally or per # section within jail.local file banaction = iptables-multiport +banaction_allports = iptables-allports # The simplest action to take: ban only action_ = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"] # ban & send an e-mail with whois report to the destemail. action_mw = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"] - %(mta)s-whois[name=%(__name__)s, dest="%(destemail)s", protocol="%(protocol)s", chain="%(chain)s"] + %(mta)s-whois[name=%(__name__)s, sender="%(sender)s", dest="%(destemail)s", protocol="%(protocol)s", chain="%(chain)s"] # ban & send an e-mail with whois report and relevant log lines # to the destemail. action_mwl = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"] - %(mta)s-whois-lines[name=%(__name__)s, dest="%(destemail)s", logpath=%(logpath)s, chain="%(chain)s"] + %(mta)s-whois-lines[name=%(__name__)s, sender="%(sender)s", dest="%(destemail)s", logpath=%(logpath)s, chain="%(chain)s"] # See the IMPORTANT note in action.d/xarf-login-attack for when to use this action # @@ -177,7 +181,7 @@ action_xarf = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(po # ban IP on CloudFlare & send an e-mail with whois report and relevant log lines # to the destemail. action_cf_mwl = cloudflare[cfuser="%(cfemail)s", cftoken="%(cfapikey)s"] - %(mta)s-whois-lines[name=%(__name__)s, dest="%(destemail)s", logpath=%(logpath)s, chain="%(chain)s"] + %(mta)s-whois-lines[name=%(__name__)s, sender="%(sender)s", dest="%(destemail)s", logpath=%(logpath)s, chain="%(chain)s"] # Report block via blocklist.de fail2ban reporting service API # @@ -186,7 +190,7 @@ action_cf_mwl = cloudflare[cfuser="%(cfemail)s", cftoken="%(cfapikey)s"] # [Init] # blocklist_de_apikey = {api key from registration] # -action_blocklist_de = blocklist_de[email="%(sender)s", service=%(filter)s, apikey="%(blocklist_de_apikey)s"] +action_blocklist_de = blocklist_de[email="%(sender)s", service=%(filter)s, apikey="%(blocklist_de_apikey)s", agent="%(fail2ban_agent)s"] # Report ban via badips.com, and use as blacklist # @@ -196,7 +200,11 @@ action_blocklist_de = blocklist_de[email="%(sender)s", service=%(filter)s, apik # NOTE: This action relies on banaction being present on start and therefore # should be last action defined for a jail. # -action_badips = badips.py[category="%(name)s", banaction="%(banaction)s"] +action_badips = badips.py[category="%(__name__)s", banaction="%(banaction)s", agent="%(fail2ban_agent)s"] +# +# Report ban via badips.com (uses action.d/badips.conf for reporting only) +# +action_badips_report = badips[category="%(__name__)s", agent="%(fail2ban_agent)s"] # Choose default action. To change, just override value of 'action' with the # interpolation to the chosen action shortcut (e.g. action_mw, action_mwl, etc) in jail.local @@ -216,6 +224,7 @@ action = %(action_)s port = ssh logpath = %(sshd_log)s +backend = %(sshd_backend)s [sshd-ddos] @@ -224,19 +233,20 @@ logpath = %(sshd_log)s # in the body. port = ssh logpath = %(sshd_log)s +backend = %(sshd_backend)s [dropbear] port = ssh logpath = %(dropbear_log)s +backend = %(dropbear_backend)s [selinux-ssh] port = ssh logpath = %(auditd_log)s -maxretry = 5 # @@ -262,7 +272,6 @@ maxretry = 1 port = http,https logpath = %(apache_error_log)s -maxretry = 6 [apache-overflows] @@ -300,23 +309,41 @@ port = http,https logpath = %(apache_error_log)s maxretry = 2 + [apache-shellshock] port = http,https logpath = %(apache_error_log)s maxretry = 1 + +[openhab-auth] + +filter = openhab +action = iptables-allports[name=NoAuthFailures] +logpath = /opt/openhab/logs/request.log + + [nginx-http-auth] port = http,https logpath = %(nginx_error_log)s +# To use 'nginx-limit-req' jail you should have `ngx_http_limit_req_module` +# and define `limit_req` and `limit_req_zone` as described in nginx documentation +# http://nginx.org/en/docs/http/ngx_http_limit_req_module.html +# or for example see in 'config/filter.d/nginx-limit-req.conf' +[nginx-limit-req] +port = http,https +logpath = %(nginx_error_log)s + [nginx-botsearch] port = http,https logpath = %(nginx_error_log)s maxretry = 2 + # Ban attackers that try to use PHP's URL-fopen() functionality # through GET/POST variables. - Experimental, with more than a year # of usage in production environments. @@ -348,7 +375,7 @@ logpath = %(lighttpd_error_log)s [roundcube-auth] port = http,https -logpath = logpath = %(roundcube_errors_log)s +logpath = %(roundcube_errors_log)s [openwebmail] @@ -381,7 +408,6 @@ logpath = /var/log/sogo/sogo.log logpath = /var/log/tine20/tine20.log port = http,https -maxretry = 5 # @@ -393,6 +419,7 @@ maxretry = 5 port = http,https logpath = %(syslog_daemon)s +backend = %(syslog_backend)s [guacamole] @@ -401,7 +428,6 @@ logpath = /var/log/tomcat*/catalina.out [monit] #Ban clients brute-forcing the monit gui login -filter = monit port = 2812 logpath = /var/log/monit @@ -410,12 +436,14 @@ logpath = /var/log/monit port = 10000 logpath = %(syslog_authpriv)s +backend = %(syslog_backend)s [froxlor-auth] port = http,https logpath = %(syslog_authpriv)s +backend = %(syslog_backend)s # @@ -444,27 +472,28 @@ logpath = /var/log/3proxy.log port = ftp,ftp-data,ftps,ftps-data logpath = %(proftpd_log)s +backend = %(proftpd_backend)s [pure-ftpd] port = ftp,ftp-data,ftps,ftps-data logpath = %(pureftpd_log)s -maxretry = 6 +backend = %(pureftpd_backend)s [gssftpd] port = ftp,ftp-data,ftps,ftps-data logpath = %(syslog_daemon)s -maxretry = 6 +backend = %(syslog_backend)s [wuftpd] port = ftp,ftp-data,ftps,ftps-data logpath = %(wuftpd_log)s -maxretry = 6 +backend = %(wuftpd_backend)s [vsftpd] @@ -491,18 +520,21 @@ logpath = /root/path/to/assp/logs/maillog.txt port = smtp,465,submission logpath = %(syslog_mail)s +backend = %(syslog_backend)s [postfix] port = smtp,465,submission logpath = %(postfix_log)s +backend = %(postfix_backend)s [postfix-rbl] port = smtp,465,submission -logpath = %(syslog_mail)s +logpath = %(postfix_log)s +backend = %(postfix_backend)s maxretry = 1 @@ -510,12 +542,14 @@ maxretry = 1 port = submission,465,smtp logpath = %(syslog_mail)s +backend = %(syslog_backend)s [sendmail-reject] port = smtp,465,submission logpath = %(syslog_mail)s +backend = %(syslog_backend)s [qmail-rbl] @@ -531,12 +565,14 @@ logpath = /service/qmail/log/main/current port = pop3,pop3s,imap,imaps,submission,465,sieve logpath = %(dovecot_log)s +backend = %(dovecot_backend)s [sieve] port = smtp,465,submission logpath = %(dovecot_log)s +backend = %(dovecot_backend)s [solid-pop3d] @@ -572,6 +608,7 @@ logpath = /opt/kerio/mailserver/store/logs/security.log port = smtp,465,submission,imap3,imaps,pop3,pop3s logpath = %(syslog_mail)s +backend = %(syslog_backend)s [postfix-sasl] @@ -581,12 +618,14 @@ port = smtp,465,submission,imap3,imaps,pop3,pop3s # running postfix since it would provide the same log lines at the # "warn" level but overall at the smaller filesize. logpath = %(postfix_log)s +backend = %(postfix_backend)s [perdition] port = imap3,imaps,pop3,pop3s logpath = %(syslog_mail)s +backend = %(syslog_backend)s [squirrelmail] @@ -599,12 +638,14 @@ logpath = /var/lib/squirrelmail/prefs/squirrelmail_access_log port = imap3,imaps logpath = %(syslog_mail)s +backend = %(syslog_backend)s [uwimap-auth] port = imap3,imaps logpath = %(syslog_mail)s +backend = %(syslog_backend)s # @@ -686,7 +727,7 @@ maxretry = 10 port = 3306 logpath = %(mysql_log)s -maxretry = 5 +backend = %(mysql_backend)s # Jail for more extended banning of persistent abusers @@ -699,10 +740,9 @@ maxretry = 5 [recidive] logpath = /var/log/fail2ban.log -banaction = iptables-allports +banaction = %(banaction_allports)s bantime = 604800 ; 1 week findtime = 86400 ; 1 day -maxretry = 5 # Generic filter for PAM. Has to be used with action which bans all @@ -710,14 +750,16 @@ maxretry = 5 [pam-generic] # pam-generic filter can be customized to monitor specific subset of 'tty's -banaction = iptables-allports +banaction = %(banaction_allports)s logpath = %(syslog_authpriv)s +backend = %(syslog_backend)s [xinetd-fail] banaction = iptables-multiport-log logpath = %(syslog_daemon)s +backend = %(syslog_backend)s maxretry = 2 @@ -746,25 +788,21 @@ action = %(banaction)s[name=%(__name__)s-tcp, port="%(tcpport)s", protocol="tcp # nobody except your own Nagios server should ever probe nrpe [nagios] -enabled = false logpath = %(syslog_daemon)s ; nrpe.cfg may define a different log_facility +backend = %(syslog_backend)s maxretry = 1 [oracleims] # see "oracleims" filter file for configuration requirement for Oracle IMS v6 and above -enabled = false logpath = /opt/sun/comms/messaging64/log/mail.log_current -maxretry = 6 -banaction = iptables-allports +banaction = %(banaction_allports)s [directadmin] -enabled = false logpath = /var/log/directadmin/login.log port = 2222 [portsentry] -enabled = false logpath = /var/lib/portsentry/portsentry.history maxretry = 1 @@ -780,3 +818,24 @@ returntype = DROP bantime = 3600 maxretry = 1 findtime = 1 + + +[murmur] +# AKA mumble-server +port = 64738 +action = %(banaction)s[name=%(__name__)s-tcp, port="%(port)s", protocol=tcp, chain="%(chain)s", actname=%(banaction)s-tcp] + %(banaction)s[name=%(__name__)s-udp, port="%(port)s", protocol=udp, chain="%(chain)s", actname=%(banaction)s-udp] +logpath = /var/log/mumble-server/mumble-server.log + + +[screensharingd] +# For Mac OS Screen Sharing Service (VNC) +logpath = /var/log/system.log +logencoding = utf-8 + +[haproxy-http-auth] +# HAProxy by default doesn't log to file you'll need to set it up to forward +# logs to a syslog server which would then write them to disk. +# See "haproxy-http-auth" filter for a brief cautionary note when setting +# maxretry and findtime. +logpath = /var/log/haproxy.log diff --git a/config/paths-common.conf b/config/paths-common.conf index bf3cfb6a..e2f08325 100644 --- a/config/paths-common.conf +++ b/config/paths-common.conf @@ -7,9 +7,13 @@ after = paths-overrides.local [DEFAULT] +default_backend = auto + sshd_log = %(syslog_authpriv)s +sshd_backend = %(default_backend)s dropbear_log = %(syslog_authpriv)s +dropbear_backend = %(default_backend)s # There is no sensible generic defaults for syslog log targets, thus # leaving them empty here so that no errors while parsing/interpolating configs @@ -18,15 +22,17 @@ syslog_ftp = syslog_local0 = syslog_mail_warn = syslog_user = +# Set the default syslog backend target to default_backend +syslog_backend = %(default_backend)s # from /etc/audit/auditd.conf auditd_log = /var/log/audit/audit.log exim_main_log = /var/log/exim/mainlog -nginx_error_log = /var/log/nginx/error.log +nginx_error_log = /var/log/nginx/*error.log -nginx_access_log = /var/log/nginx/access.log +nginx_access_log = /var/log/nginx/*access.log lighttpd_error_log = /var/log/lighttpd/error.log @@ -38,14 +44,17 @@ suhosin_log = %(syslog_user)s %(lighttpd_error_log)s # defaults to ftp or local2 if ftp doesn't exist proftpd_log = %(syslog_ftp)s +proftpd_backend = %(default_backend)s # http://svnweb.freebsd.org/ports/head/ftp/proftpd/files/patch-src_proftpd.8.in?view=markup # defaults to ftp but can be overwritten. pureftpd_log = %(syslog_ftp)s +pureftpd_backend = %(default_backend)s # ftp, daemon and then local7 are tried at configure time however it is overwriteable at configure time # wuftpd_log = %(syslog_ftp)s +wuftpd_backend = %(default_backend)s # syslog_enable defaults to no. so it defaults to vsftpd_log_file setting of /var/log/vsftpd.log # No distro seems to set it to syslog by default @@ -54,13 +63,16 @@ vsftpd_log = /var/log/vsftpd.log # Technically syslog_facility in main.cf can overwrite but no-one sane does this. postfix_log = %(syslog_mail_warn)s +postfix_backend = %(default_backend)s dovecot_log = %(syslog_mail_warn)s +dovecot_backend = %(default_backend)s # Seems to be set at compile time only to LOG_LOCAL0 (src/const.h) at Notice level solidpop3d_log = %(syslog_local0)s mysql_log = %(syslog_daemon)s +mysql_backend = %(default_backend)s roundcube_errors_log = /var/log/roundcube/errors diff --git a/config/paths-fedora.conf b/config/paths-fedora.conf index c5601d3c..b3c978ca 100644 --- a/config/paths-fedora.conf +++ b/config/paths-fedora.conf @@ -37,3 +37,15 @@ exim_main_log = /var/log/exim/main.log mysql_log = /var/lib/mysql/mysqld.log roundcube_errors_log = /var/log/roundcubemail/errors + +# These services will log to the journal via syslog, so use the journal by +# default. +syslog_backend = systemd +sshd_backend = systemd +dropbear_backend = systemd +proftpd_backend = systemd +pureftpd_backend = systemd +wuftpd_backend = systemd +postfix_backend = systemd +dovecot_backend = systemd +mysql_backend = systemd diff --git a/config/paths-opensuse.conf b/config/paths-opensuse.conf new file mode 100644 index 00000000..0d6ad522 --- /dev/null +++ b/config/paths-opensuse.conf @@ -0,0 +1,38 @@ +# openSUSE log-file locations + +[INCLUDES] + +before = paths-common.conf + +after = paths-overrides.local + + +[DEFAULT] + +syslog_local0 = /var/log/messages + +syslog_mail = /var/log/mail + +syslog_mail_warn = %(syslog_mail)s + +syslog_authpriv = %(syslog_local0)s + +syslog_user = %(syslog_local0)s + +syslog_ftp = %(syslog_local0)s + +syslog_daemon = %(syslog_local0)s + +apache_error_log = /var/log/apache2/*error_log + +apache_access_log = /var/log/apache2/*access_log + +pureftpd_log = %(syslog_local0)s + +exim_main_log = /var/log/exim/main.log + +mysql_log = /var/log/mysql/mysqld.log + +roundcube_errors_log = /srv/www/roundcubemail/logs/errors + +solidpop3d_log = %(syslog_mail)s diff --git a/fail2ban/client/configreader.py b/fail2ban/client/configreader.py index d5675cc8..c6dd1b60 100644 --- a/fail2ban/client/configreader.py +++ b/fail2ban/client/configreader.py @@ -285,8 +285,10 @@ class DefinitionInitConfigReader(ConfigReader): if self.has_section("Init"): for opt in self.options("Init"): + v = self.get("Init", opt) + self._initOpts['known/'+opt] = v if not opt in self._initOpts: - self._initOpts[opt] = self.get("Init", opt) + self._initOpts[opt] = v def convert(self): raise NotImplementedError diff --git a/fail2ban/client/fail2banregex.py b/fail2ban/client/fail2banregex.py new file mode 100755 index 00000000..d0cdab84 --- /dev/null +++ b/fail2ban/client/fail2banregex.py @@ -0,0 +1,599 @@ +#!/usr/bin/python +# emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*- +# vi: set ft=python sts=4 ts=4 sw=4 noet : +# +# This file is part of Fail2Ban. +# +# Fail2Ban is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# Fail2Ban is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with Fail2Ban; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. +""" +Fail2Ban reads log file that contains password failure report +and bans the corresponding IP addresses using firewall rules. + +This tools can test regular expressions for "fail2ban". + +""" + +__author__ = "Fail2Ban Developers" +__copyright__ = "Copyright (c) 2004-2008 Cyril Jaquier, 2012-2014 Yaroslav Halchenko" +__license__ = "GPL" + +import getopt +import locale +import logging +import os +import shlex +import sys +import time +import time +import urllib +from optparse import OptionParser, Option + +from ConfigParser import NoOptionError, NoSectionError, MissingSectionHeaderError + +try: + from systemd import journal + from ..server.filtersystemd import FilterSystemd +except ImportError: + journal = None + +from ..version import version +from .filterreader import FilterReader +from ..server.filter import Filter, FileContainer +from ..server.failregex import RegexException + +from ..helpers import FormatterWithTraceBack, getLogger +# Gets the instance of the logger. +logSys = getLogger("fail2ban") + +def debuggexURL(sample, regex): + q = urllib.urlencode({ 're': regex.replace('<HOST>', '(?&.ipv4)'), + 'str': sample, + 'flavor': 'python' }) + return 'http://www.debuggex.com/?' + q + +def output(args): + print(args) + +def shortstr(s, l=53): + """Return shortened string + """ + if len(s) > l: + return s[:l-3] + '...' + return s + +def pprint_list(l, header=None): + if not len(l): + return + if header: + s = "|- %s\n" % header + else: + s = '' + output( s + "| " + "\n| ".join(l) + '\n`-' ) + +def journal_lines_gen(myjournal): + while True: + try: + entry = myjournal.get_next() + except OSError: + continue + if not entry: + break + yield FilterSystemd.formatJournalEntry(entry) + +def get_opt_parser(): + # use module docstring for help output + p = OptionParser( + usage="%s [OPTIONS] <LOG> <REGEX> [IGNOREREGEX]\n" % sys.argv[0] + __doc__ + + """ +LOG: + string a string representing a log line + filename path to a log file (/var/log/auth.log) + "systemd-journal" search systemd journal (systemd-python required) + +REGEX: + string a string representing a 'failregex' + filename path to a filter file (filter.d/sshd.conf) + +IGNOREREGEX: + string a string representing an 'ignoreregex' + filename path to a filter file (filter.d/sshd.conf) + +Copyright (c) 2004-2008 Cyril Jaquier, 2008- Fail2Ban Contributors +Copyright of modifications held by their respective authors. +Licensed under the GNU General Public License v2 (GPL). + +Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>. +Many contributions by Yaroslav O. Halchenko and Steven Hiscocks. + +Report bugs to https://github.com/fail2ban/fail2ban/issues +""", + version="%prog " + version) + + p.add_options([ + Option("-d", "--datepattern", + help="set custom pattern used to match date/times"), + Option("-e", "--encoding", + help="File encoding. Default: system locale"), + Option("-L", "--maxlines", type=int, default=0, + help="maxlines for multi-line regex"), + Option("-m", "--journalmatch", + help="journalctl style matches overriding filter file. " + "\"systemd-journal\" only"), + Option('-l', "--log-level", type="choice", + dest="log_level", + choices=('heavydebug', 'debug', 'info', 'notice', 'warning', 'error', 'critical'), + default=None, + help="Log level for the Fail2Ban logger to use"), + Option("-v", "--verbose", action='store_true', + help="Be verbose in output"), + Option("-D", "--debuggex", action='store_true', + help="Produce debuggex.com urls for debugging there"), + Option("--print-no-missed", action='store_true', + help="Do not print any missed lines"), + Option("--print-no-ignored", action='store_true', + help="Do not print any ignored lines"), + Option("--print-all-matched", action='store_true', + help="Print all matched lines"), + Option("--print-all-missed", action='store_true', + help="Print all missed lines, no matter how many"), + Option("--print-all-ignored", action='store_true', + help="Print all ignored lines, no matter how many"), + Option("-t", "--log-traceback", action='store_true', + help="Enrich log-messages with compressed tracebacks"), + Option("--full-traceback", action='store_true', + help="Either to make the tracebacks full, not compressed (as by default)"), + ]) + + return p + + +class RegexStat(object): + + def __init__(self, failregex): + self._stats = 0 + self._failregex = failregex + self._ipList = list() + + def __str__(self): + return "%s(%r) %d failed: %s" \ + % (self.__class__, self._failregex, self._stats, self._ipList) + + def inc(self): + self._stats += 1 + + def getStats(self): + return self._stats + + def getFailRegex(self): + return self._failregex + + def appendIP(self, value): + self._ipList.append(value) + + def getIPList(self): + return self._ipList + + +class LineStats(object): + """Just a convenience container for stats + """ + def __init__(self): + self.tested = self.matched = 0 + self.matched_lines = [] + self.missed = 0 + self.missed_lines = [] + self.missed_lines_timeextracted = [] + self.ignored = 0 + self.ignored_lines = [] + self.ignored_lines_timeextracted = [] + + def __str__(self): + return "%(tested)d lines, %(ignored)d ignored, %(matched)d matched, %(missed)d missed" % self + + # just for convenient str + def __getitem__(self, key): + return getattr(self, key) if hasattr(self, key) else '' + + +class Fail2banRegex(object): + + def __init__(self, opts): + self._verbose = opts.verbose + self._debuggex = opts.debuggex + self._maxlines = 20 + self._print_no_missed = opts.print_no_missed + self._print_no_ignored = opts.print_no_ignored + self._print_all_matched = opts.print_all_matched + self._print_all_missed = opts.print_all_missed + self._print_all_ignored = opts.print_all_ignored + self._maxlines_set = False # so we allow to override maxlines in cmdline + self._datepattern_set = False + self._journalmatch = None + + self.share_config=dict() + self._filter = Filter(None) + self._ignoreregex = list() + self._failregex = list() + self._time_elapsed = None + self._line_stats = LineStats() + + if opts.maxlines: + self.setMaxLines(opts.maxlines) + if opts.journalmatch is not None: + self.setJournalMatch(opts.journalmatch.split()) + if opts.datepattern: + self.setDatePattern(opts.datepattern) + if opts.encoding: + self.encoding = opts.encoding + else: + self.encoding = locale.getpreferredencoding() + + def decode_line(self, line): + return FileContainer.decode_line('<LOG>', self.encoding, line) + + def encode_line(self, line): + return line.encode(self.encoding, 'ignore') + + def setDatePattern(self, pattern): + if not self._datepattern_set: + self._filter.setDatePattern(pattern) + self._datepattern_set = True + if pattern is not None: + output( "Use datepattern : %s" % ( + self._filter.getDatePattern()[1], ) ) + + def setMaxLines(self, v): + if not self._maxlines_set: + self._filter.setMaxLines(int(v)) + self._maxlines_set = True + output( "Use maxlines : %d" % self._filter.getMaxLines() ) + + def setJournalMatch(self, v): + if self._journalmatch is None: + self._journalmatch = v + + def readRegex(self, value, regextype): + assert(regextype in ('fail', 'ignore')) + regex = regextype + 'regex' + if os.path.isfile(value) or os.path.isfile(value + '.conf'): + if os.path.basename(os.path.dirname(value)) == 'filter.d': + ## within filter.d folder - use standard loading algorithm to load filter completely (with .local etc.): + basedir = os.path.dirname(os.path.dirname(value)) + value = os.path.splitext(os.path.basename(value))[0] + output( "Use %11s filter file : %s, basedir: %s" % (regex, value, basedir) ) + reader = FilterReader(value, 'fail2ban-regex-jail', {}, share_config=self.share_config, basedir=basedir) + if not reader.read(): + output( "ERROR: failed to load filter %s" % value ) + return False + else: + ## foreign file - readexplicit this file and includes if possible: + output( "Use %11s file : %s" % (regex, value) ) + reader = FilterReader(value, 'fail2ban-regex-jail', {}, share_config=self.share_config) + reader.setBaseDir(None) + if not reader.readexplicit(): + output( "ERROR: failed to read %s" % value ) + return False + reader.getOptions(None) + readercommands = reader.convert() + regex_values = [ + RegexStat(m[3]) + for m in filter( + lambda x: x[0] == 'set' and x[2] == "add%sregex" % regextype, + readercommands)] + # Read out and set possible value of maxlines + for command in readercommands: + if command[2] == "maxlines": + maxlines = int(command[3]) + try: + self.setMaxLines(maxlines) + except ValueError: + output( "ERROR: Invalid value for maxlines (%(maxlines)r) " \ + "read from %(value)s" % locals() ) + return False + elif command[2] == 'addjournalmatch': + journalmatch = command[3:] + self.setJournalMatch(journalmatch) + elif command[2] == 'datepattern': + datepattern = command[3] + self.setDatePattern(datepattern) + else: + output( "Use %11s line : %s" % (regex, shortstr(value)) ) + regex_values = [RegexStat(value)] + + setattr(self, "_" + regex, regex_values) + for regex in regex_values: + getattr( + self._filter, + 'add%sRegex' % regextype.title())(regex.getFailRegex()) + return True + + def testIgnoreRegex(self, line): + found = False + try: + ret = self._filter.ignoreLine([(line, "", "")]) + if ret is not None: + found = True + regex = self._ignoreregex[ret].inc() + except RegexException, e: + output( e ) + return False + return found + + def testRegex(self, line, date=None): + orgLineBuffer = self._filter._Filter__lineBuffer + fullBuffer = len(orgLineBuffer) >= self._filter.getMaxLines() + try: + line, ret = self._filter.processLine(line, date, checkAllRegex=True) + for match in ret: + # Append True/False flag depending if line was matched by + # more than one regex + match.append(len(ret)>1) + regex = self._failregex[match[0]] + regex.inc() + regex.appendIP(match) + except RegexException, e: + output( e ) + return False + except IndexError: + output( "Sorry, but no <HOST> found in regex" ) + return False + for bufLine in orgLineBuffer[int(fullBuffer):]: + if bufLine not in self._filter._Filter__lineBuffer: + try: + self._line_stats.missed_lines.pop( + self._line_stats.missed_lines.index("".join(bufLine))) + self._line_stats.missed_lines_timeextracted.pop( + self._line_stats.missed_lines_timeextracted.index( + "".join(bufLine[::2]))) + except ValueError: + pass + else: + self._line_stats.matched += 1 + self._line_stats.missed -= 1 + return line, ret + + def process(self, test_lines): + t0 = time.time() + for line_no, line in enumerate(test_lines): + if isinstance(line, tuple): + line_datetimestripped, ret = self.testRegex( + line[0], line[1]) + line = "".join(line[0]) + else: + line = line.rstrip('\r\n') + if line.startswith('#') or not line: + # skip comment and empty lines + continue + line_datetimestripped, ret = self.testRegex(line) + is_ignored = self.testIgnoreRegex(line_datetimestripped) + + if is_ignored: + self._line_stats.ignored += 1 + if not self._print_no_ignored and (self._print_all_ignored or self._line_stats.ignored <= self._maxlines + 1): + self._line_stats.ignored_lines.append(line) + self._line_stats.ignored_lines_timeextracted.append(line_datetimestripped) + + if len(ret) > 0: + assert(not is_ignored) + self._line_stats.matched += 1 + if self._print_all_matched: + self._line_stats.matched_lines.append(line) + else: + if not is_ignored: + self._line_stats.missed += 1 + if not self._print_no_missed and (self._print_all_missed or self._line_stats.missed <= self._maxlines + 1): + self._line_stats.missed_lines.append(line) + self._line_stats.missed_lines_timeextracted.append(line_datetimestripped) + self._line_stats.tested += 1 + + if line_no % 10 == 0 and self._filter.dateDetector is not None: + self._filter.dateDetector.sortTemplate() + self._time_elapsed = time.time() - t0 + + def printLines(self, ltype): + lstats = self._line_stats + assert(self._line_stats.missed == lstats.tested - (lstats.matched + lstats.ignored)) + lines = lstats[ltype] + l = lstats[ltype + '_lines'] + if lines: + header = "%s line(s):" % (ltype.capitalize(),) + if self._debuggex: + if ltype == 'missed' or ltype == 'matched': + regexlist = self._failregex + else: + regexlist = self._ignoreregex + l = lstats[ltype + '_lines_timeextracted'] + if lines < self._maxlines or getattr(self, '_print_all_' + ltype): + ans = [[]] + for arg in [l, regexlist]: + ans = [ x + [y] for x in ans for y in arg ] + b = map(lambda a: a[0] + ' | ' + a[1].getFailRegex() + ' | ' + + debuggexURL(self.encode_line(a[0]), a[1].getFailRegex()), ans) + pprint_list([x.rstrip() for x in b], header) + else: + output( "%s too many to print. Use --print-all-%s " \ + "to print all %d lines" % (header, ltype, lines) ) + elif lines < self._maxlines or getattr(self, '_print_all_' + ltype): + pprint_list([x.rstrip() for x in l], header) + else: + output( "%s too many to print. Use --print-all-%s " \ + "to print all %d lines" % (header, ltype, lines) ) + + def printStats(self): + output( "" ) + output( "Results" ) + output( "=======" ) + + def print_failregexes(title, failregexes): + # Print title + total, out = 0, [] + for cnt, failregex in enumerate(failregexes): + match = failregex.getStats() + total += match + if (match or self._verbose): + out.append("%2d) [%d] %s" % (cnt+1, match, failregex.getFailRegex())) + + if self._verbose and len(failregex.getIPList()): + for ip in failregex.getIPList(): + timeTuple = time.localtime(ip[2]) + timeString = time.strftime("%a %b %d %H:%M:%S %Y", timeTuple) + out.append( + " %s %s%s" % ( + ip[1], + timeString, + ip[-1] and " (multiple regex matched)" or "")) + + output( "\n%s: %d total" % (title, total) ) + pprint_list(out, " #) [# of hits] regular expression") + return total + + # Print title + total = print_failregexes("Failregex", self._failregex) + _ = print_failregexes("Ignoreregex", self._ignoreregex) + + + if self._filter.dateDetector is not None: + output( "\nDate template hits:" ) + out = [] + for template in self._filter.dateDetector.templates: + if self._verbose or template.hits: + out.append("[%d] %s" % ( + template.hits, template.name)) + pprint_list(out, "[# of hits] date format") + + output( "\nLines: %s" % self._line_stats, ) + if self._time_elapsed is not None: + output( "[processed in %.2f sec]" % self._time_elapsed, ) + output( "" ) + + if self._print_all_matched: + self.printLines('matched') + if not self._print_no_ignored: + self.printLines('ignored') + if not self._print_no_missed: + self.printLines('missed') + + return True + + def file_lines_gen(self, hdlr): + for line in hdlr: + yield self.decode_line(line) + + def start(self, opts, args): + + cmd_log, cmd_regex = args[:2] + + if not self.readRegex(cmd_regex, 'fail'): + return False + + if len(args) == 3 and not self.readRegex(args[2], 'ignore'): + return False + + if os.path.isfile(cmd_log): + try: + hdlr = open(cmd_log, 'rb') + output( "Use log file : %s" % cmd_log ) + output( "Use encoding : %s" % self.encoding ) + test_lines = self.file_lines_gen(hdlr) + except IOError, e: + output( e ) + return False + elif cmd_log == "systemd-journal": # pragma: no cover + if not journal: + output( "Error: systemd library not found. Exiting..." ) + return False + myjournal = journal.Reader(converters={'__CURSOR': lambda x: x}) + journalmatch = self._journalmatch + self.setDatePattern(None) + if journalmatch: + try: + for element in journalmatch: + if element == "+": + myjournal.add_disjunction() + else: + myjournal.add_match(element) + except ValueError: + output( "Error: Invalid journalmatch: %s" % shortstr(" ".join(journalmatch)) ) + return False + output( "Use journal match : %s" % " ".join(journalmatch) ) + test_lines = journal_lines_gen(myjournal) + else: + output( "Use single line : %s" % shortstr(cmd_log) ) + test_lines = [ cmd_log ] + output( "" ) + + self.process(test_lines) + + if not self.printStats(): + return False + + return True + + +def exec_command_line(): + parser = get_opt_parser() + (opts, args) = parser.parse_args() + if opts.print_no_missed and opts.print_all_missed: + sys.stderr.write("ERROR: --print-no-missed and --print-all-missed are mutually exclusive.\n\n") + parser.print_help() + sys.exit(-1) + if opts.print_no_ignored and opts.print_all_ignored: + sys.stderr.write("ERROR: --print-no-ignored and --print-all-ignored are mutually exclusive.\n\n") + parser.print_help() + sys.exit(-1) + + # We need 2 or 3 parameters + if not len(args) in (2, 3): + sys.stderr.write("ERROR: provide both <LOG> and <REGEX>.\n\n") + parser.print_help() + return False + + output( "" ) + output( "Running tests" ) + output( "=============" ) + output( "" ) + + # TODO: taken from -testcases -- move common functionality somewhere + if opts.log_level is not None: + # so we had explicit settings + logSys.setLevel(getattr(logging, opts.log_level.upper())) + else: + # suppress the logging but it would leave unittests' progress dots + # ticking, unless like with '-l critical' which would be silent + # unless error occurs + logSys.setLevel(getattr(logging, 'CRITICAL')) + + # Add the default logging handler + stdout = logging.StreamHandler(sys.stdout) + + fmt = 'D: %(message)s' + + if opts.log_traceback: + Formatter = FormatterWithTraceBack + fmt = (opts.full_traceback and ' %(tb)s' or ' %(tbc)s') + fmt + else: + Formatter = logging.Formatter + + # Custom log format for the verbose tests runs + if opts.verbose: + stdout.setFormatter(Formatter(' %(asctime)-15s %(thread)s' + fmt)) + else: + # just prefix with the space + stdout.setFormatter(Formatter(fmt)) + logSys.addHandler(stdout) + + fail2banRegex = Fail2banRegex(opts) + if not fail2banRegex.start(opts, args): + sys.exit(-1) diff --git a/fail2ban/client/jailreader.py b/fail2ban/client/jailreader.py index 6d0fddfa..56b8889c 100644 --- a/fail2ban/client/jailreader.py +++ b/fail2ban/client/jailreader.py @@ -32,7 +32,9 @@ import re from .configreader import ConfigReaderUnshared, ConfigReader from .filterreader import FilterReader from .actionreader import ActionReader +from ..version import version from ..helpers import getLogger +from ..helpers import splitcommaspace # Gets the instance of the logger. logSys = getLogger(__name__) @@ -107,6 +109,10 @@ class JailReader(ConfigReader): ["string", "filter", ""], ["string", "action", ""]] + # Before interpolation (substitution) add static options always available as default: + defsec = self._cfg.get_defaults() + defsec["fail2ban_version"] = version + # Read first options only needed for merge defaults ('known/...' from filter): self.__opts = ConfigReader.getOptions(self, self.__name, opts1st) if not self.__opts: @@ -208,10 +214,8 @@ class JailReader(ConfigReader): elif opt == "maxretry": stream.append(["set", self.__name, "maxretry", self.__opts[opt]]) elif opt == "ignoreip": - for ip in self.__opts[opt].split(): - # Do not send a command if the rule is empty. - if ip != '': - stream.append(["set", self.__name, "addignoreip", ip]) + for ip in splitcommaspace(self.__opts[opt]): + stream.append(["set", self.__name, "addignoreip", ip]) elif opt == "findtime": stream.append(["set", self.__name, "findtime", self.__opts[opt]]) elif opt == "bantime": diff --git a/fail2ban/helpers.py b/fail2ban/helpers.py index f5c3163a..8e1b0e32 100644 --- a/fail2ban/helpers.py +++ b/fail2ban/helpers.py @@ -127,3 +127,13 @@ def excepthook(exctype, value, traceback): getLogger("fail2ban").critical( "Unhandled exception in Fail2Ban:", exc_info=True) return sys.__excepthook__(exctype, value, traceback) + +def splitcommaspace(s): + """Helper to split on any comma or space + + Returns empty list if input is empty (or None) and filters + out empty entries + """ + if not s: + return [] + return filter(bool, re.split('[ ,]', s)) diff --git a/fail2ban/protocol.py b/fail2ban/protocol.py index 2cace91f..5d9fdd65 100644 --- a/fail2ban/protocol.py +++ b/fail2ban/protocol.py @@ -89,7 +89,7 @@ protocol = [ ["set <JAIL> unbanip <IP>", "manually Unban <IP> in <JAIL>"], ["set <JAIL> maxretry <RETRY>", "sets the number of failures <RETRY> before banning the host for <JAIL>"], ["set <JAIL> maxlines <LINES>", "sets the number of <LINES> to buffer for regex search for <JAIL>"], -["set <JAIL> addaction <ACT>[ <PYTHONFILE> <JSONKWARGS>]", "adds a new action named <NAME> for <JAIL>. Optionally for a Python based action, a <PYTHONFILE> and <JSONKWARGS> can be specified, else will be a Command Action"], +["set <JAIL> addaction <ACT>[ <PYTHONFILE> <JSONKWARGS>]", "adds a new action named <ACT> for <JAIL>. Optionally for a Python based action, a <PYTHONFILE> and <JSONKWARGS> can be specified, else will be a Command Action"], ["set <JAIL> delaction <ACT>", "removes the action <ACT> from <JAIL>"], ["", "COMMAND ACTION CONFIGURATION", ""], ["set <JAIL> action <ACT> actionstart <CMD>", "sets the start command <CMD> of the action <ACT> for <JAIL>"], diff --git a/fail2ban/server/action.py b/fail2ban/server/action.py index c58fde2c..de0c8efc 100644 --- a/fail2ban/server/action.py +++ b/fail2ban/server/action.py @@ -560,32 +560,33 @@ class CommandAction(ActionBase): return True _cmd_lock.acquire() - try: # Try wrapped within another try needed for python version < 2.5 + try: + retcode = None # to guarantee being defined upon early except stdout = tempfile.TemporaryFile(suffix=".stdout", prefix="fai2ban_") stderr = tempfile.TemporaryFile(suffix=".stderr", prefix="fai2ban_") - try: - popen = subprocess.Popen( - realCmd, stdout=stdout, stderr=stderr, shell=True, - preexec_fn=os.setsid # so that killpg does not kill our process - ) - stime = time.time() + + popen = subprocess.Popen( + realCmd, stdout=stdout, stderr=stderr, shell=True, + preexec_fn=os.setsid # so that killpg does not kill our process + ) + stime = time.time() + retcode = popen.poll() + while time.time() - stime <= timeout and retcode is None: + time.sleep(0.1) retcode = popen.poll() - while time.time() - stime <= timeout and retcode is None: - time.sleep(0.1) - retcode = popen.poll() - if retcode is None: - logSys.error("%s -- timed out after %i seconds." % - (realCmd, timeout)) - pgid = os.getpgid(popen.pid) - os.killpg(pgid, signal.SIGTERM) # Terminate the process + if retcode is None: + logSys.error("%s -- timed out after %i seconds." % + (realCmd, timeout)) + pgid = os.getpgid(popen.pid) + os.killpg(pgid, signal.SIGTERM) # Terminate the process + time.sleep(0.1) + retcode = popen.poll() + if retcode is None: # Still going... + os.killpg(pgid, signal.SIGKILL) # Kill the process time.sleep(0.1) retcode = popen.poll() - if retcode is None: # Still going... - os.killpg(pgid, signal.SIGKILL) # Kill the process - time.sleep(0.1) - retcode = popen.poll() - except OSError, e: - logSys.error("%s -- failed with %s" % (realCmd, e)) + except OSError as e: + logSys.error("%s -- failed with %s" % (realCmd, e)) finally: _cmd_lock.release() @@ -603,15 +604,16 @@ class CommandAction(ActionBase): return True elif retcode is None: logSys.error("%s -- unable to kill PID %i" % (realCmd, popen.pid)) - elif retcode < 0: - logSys.error("%s -- killed with %s" % - (realCmd, signame.get(-retcode, "signal %i" % -retcode))) + elif retcode < 0 or retcode > 128: + # dash would return negative while bash 128 + n + sigcode = -retcode if retcode < 0 else retcode - 128 + logSys.error("%s -- killed with %s (return code: %s)" % + (realCmd, signame.get(sigcode, "signal %i" % sigcode), retcode)) else: msg = _RETCODE_HINTS.get(retcode, None) logSys.error("%s -- returned %i" % (realCmd, retcode)) if msg: logSys.info("HINT on %i: %s" % (retcode, msg % locals())) - return False - raise RuntimeError("Command execution failed: %s" % realCmd) + return False diff --git a/fail2ban/server/datedetector.py b/fail2ban/server/datedetector.py index 95d368b5..a4839f5c 100644 --- a/fail2ban/server/datedetector.py +++ b/fail2ban/server/datedetector.py @@ -78,6 +78,10 @@ class DateDetector(object): # asctime with optional day, subsecond and/or year: # Sun Jan 23 21:59:59.011 2005 self.appendTemplate("(?:%a )?%b %d %H:%M:%S(?:\.%f)?(?: %Y)?") + # asctime with optional day, subsecond and/or year coming after day + # http://bugs.debian.org/798923 + # Sun Jan 23 2005 21:59:59.011 + self.appendTemplate("(?:%a )?%b %d %Y %H:%M:%S(?:\.%f)?") # simple date, optional subsecond (proftpd): # 2005-01-23 21:59:59 # simple date: 2005/01/23 21:59:59 diff --git a/fail2ban/server/datetemplate.py b/fail2ban/server/datetemplate.py index bcd17ec1..8210dad4 100644 --- a/fail2ban/server/datetemplate.py +++ b/fail2ban/server/datetemplate.py @@ -132,7 +132,7 @@ class DateEpoch(DateTemplate): def __init__(self): DateTemplate.__init__(self) - self.regex = "(?:^|(?P<square>(?<=^\[))|(?P<selinux>(?<=audit\()))\d{10}(?:\.\d{3,6})?(?(selinux)(?=:\d+\))(?(square)(?=\])))" + self.regex = r"(?:^|(?P<square>(?<=^\[))|(?P<selinux>(?<=audit\()))\d{10,11}\b(?:\.\d{3,6})?(?:(?(selinux)(?=:\d+\)))|(?(square)(?=\])))" def getDate(self, line): """Method to return the date for a log line. diff --git a/fail2ban/server/filter.py b/fail2ban/server/filter.py index 18afb135..d561b507 100644 --- a/fail2ban/server/filter.py +++ b/fail2ban/server/filter.py @@ -491,6 +491,7 @@ class Filter(JailThread): self.__lineBuffer = ( self.__lineBuffer + [tupleLine])[-self.__lineBufferSize:] + logSys.log(5, "Looking for failregex match of %r" % self.__lineBuffer) # Iterates over all the regular expressions. for failRegexIndex, failRegex in enumerate(self.__failRegex): @@ -552,7 +553,7 @@ class FileFilter(Filter): def __init__(self, jail, **kwargs): Filter.__init__(self, jail, **kwargs) ## The log file path. - self.__logPath = [] + self.__logs = dict() self.setLogEncoding("auto") ## @@ -560,17 +561,17 @@ class FileFilter(Filter): # # @param path log file path - def addLogPath(self, path, tail = False): - if self.containsLogPath(path): + def addLogPath(self, path, tail=False): + if path in self.__logs: logSys.error(path + " already exists") else: - container = FileContainer(path, self.getLogEncoding(), tail) + log = FileContainer(path, self.getLogEncoding(), tail) db = self.jail.database if db is not None: - lastpos = db.addLog(self.jail, container) + lastpos = db.addLog(self.jail, log) if lastpos and not tail: - container.setPos(lastpos) - self.__logPath.append(container) + log.setPos(lastpos) + self.__logs[path] = log logSys.info("Added logfile = %s" % path) self._addLogPath(path) # backend specific @@ -585,15 +586,16 @@ class FileFilter(Filter): # @param path the log file to delete def delLogPath(self, path): - for log in self.__logPath: - if log.getFileName() == path: - self.__logPath.remove(log) - db = self.jail.database - if db is not None: - db.updateLog(self.jail, log) - logSys.info("Removed logfile = %s" % path) - self._delLogPath(path) - return + try: + log = self.__logs.pop(path) + except KeyError: + return + db = self.jail.database + if db is not None: + db.updateLog(self.jail, log) + logSys.info("Removed logfile = %s" % path) + self._delLogPath(path) + return def _delLogPath(self, path): # pragma: no cover - overwritten function # nothing to do by default @@ -601,12 +603,12 @@ class FileFilter(Filter): pass ## - # Get the log file path + # Get the log containers # - # @return log file path + # @return log containers - def getLogPath(self): - return self.__logPath + def getLogs(self): + return self.__logs.values() ## # Check whether path is already monitored. @@ -615,10 +617,7 @@ class FileFilter(Filter): # @return True if the path is already monitored else False def containsLogPath(self, path): - for log in self.__logPath: - if log.getFileName() == path: - return True - return False + return path in self.__logs ## # Set the log file encoding @@ -629,7 +628,7 @@ class FileFilter(Filter): if encoding.lower() == "auto": encoding = locale.getpreferredencoding() codecs.lookup(encoding) # Raise LookupError if invalid codec - for log in self.getLogPath(): + for log in self.__logs.itervalues(): log.setEncoding(encoding) self.__encoding = encoding logSys.info("Set jail log file encoding to %s" % encoding) @@ -642,11 +641,8 @@ class FileFilter(Filter): def getLogEncoding(self): return self.__encoding - def getFileContainer(self, path): - for log in self.__logPath: - if log.getFileName() == path: - return log - return None + def getLog(self, path): + return self.__logs.get(path, None) ## # Gets all the failure in the log file. @@ -656,13 +652,13 @@ class FileFilter(Filter): # is created and is added to the FailManager. def getFailures(self, filename): - container = self.getFileContainer(filename) - if container is None: + log = self.getLog(filename) + if log is None: logSys.error("Unable to get failures in " + filename) return False # Try to open log file. try: - has_content = container.open() + has_content = log.open() # see http://python.org/dev/peps/pep-3151/ except IOError, e: logSys.error("Unable to open %s" % filename) @@ -672,7 +668,7 @@ class FileFilter(Filter): logSys.error("Error opening %s" % filename) logSys.exception(e) return False - except OSError, e: # pragma: no cover - Requires implemention error in FileContainer to generate + except Exception, e: # pragma: no cover - Requires implemention error in FileContainer to generate logSys.error("Internal errror in FileContainer open method - please report as a bug to https://github.com/fail2ban/fail2ban/issues") logSys.exception(e) return False @@ -683,22 +679,22 @@ class FileFilter(Filter): # start reading tested to be empty container -- race condition # might occur leading at least to tests failures. while has_content: - line = container.readline() + line = log.readline() if not line or not self.active: # The jail reached the bottom or has been stopped break self.processLineAndAdd(line) - container.close() + log.close() db = self.jail.database if db is not None: - db.updateLog(self.jail, container) + db.updateLog(self.jail, log) return True def status(self, flavor="basic"): """Status of Filter plus files being monitored. """ ret = super(FileFilter, self).status(flavor=flavor) - path = [m.getFileName() for m in self.getLogPath()] + path = self.__logs.keys() ret.append(("File list", path)) return ret @@ -792,23 +788,27 @@ class FileContainer: self.__handler.seek(self.__pos) return True - def readline(self): - if self.__handler is None: - return "" - line = self.__handler.readline() + @staticmethod + def decode_line(filename, enc, line): try: - line = line.decode(self.getEncoding(), 'strict') + line = line.decode(enc, 'strict') except UnicodeDecodeError: logSys.warning( "Error decoding line from '%s' with '%s'." " Consider setting logencoding=utf-8 (or another appropriate" " encoding) for this jail. Continuing" " to process line ignoring invalid characters: %r" % - (self.getFileName(), self.getEncoding(), line)) + (filename, enc, line)) # decode with replacing error chars: - line = line.decode(self.getEncoding(), 'replace') + line = line.decode(enc, 'replace') return line + def readline(self): + if self.__handler is None: + return "" + return FileContainer.decode_line( + self.getFileName(), self.getEncoding(), self.__handler.readline()) + def close(self): if not self.__handler is None: # Saves the last position. @@ -855,8 +855,9 @@ class DNSUtils: """ Convert a DNS into an IP address using the Python socket module. Thanks to Kevin Drapel. """ + # retrieve ip (todo: use AF_INET6 for IPv6) try: - return set(socket.gethostbyname_ex(dns)[2]) + return set([i[4][0] for i in socket.getaddrinfo(dns, None, socket.AF_INET, 0, socket.IPPROTO_TCP)]) except socket.error, e: logSys.warning("Unable to find a corresponding IP address for %s: %s" % (dns, e)) diff --git a/fail2ban/server/filtergamin.py b/fail2ban/server/filtergamin.py index 1f51744b..e731a8e9 100644 --- a/fail2ban/server/filtergamin.py +++ b/fail2ban/server/filtergamin.py @@ -129,6 +129,6 @@ class FilterGamin(FileFilter): # Desallocates the resources used by Gamin. def __cleanup(self): - for path in self.getLogPath(): - self.monitor.stop_watch(path.getFileName()) + for log in self.getLogs(): + self.monitor.stop_watch(log.getFileName()) del self.monitor diff --git a/fail2ban/server/filterpoll.py b/fail2ban/server/filterpoll.py index 25c3e119..d0b37775 100644 --- a/fail2ban/server/filterpoll.py +++ b/fail2ban/server/filterpoll.py @@ -88,10 +88,10 @@ class FilterPoll(FileFilter): while self.active: if logSys.getEffectiveLevel() <= 6: logSys.log(6, "Woke up idle=%s with %d files monitored", - self.idle, len(self.getLogPath())) + self.idle, len(self.getLogs())) if not self.idle: # Get file modification - for container in self.getLogPath(): + for container in self.getLogs(): filename = container.getFileName() if self.isModified(filename): self.getFailures(filename) diff --git a/fail2ban/server/server.py b/fail2ban/server/server.py index 6d19544d..3e371945 100644 --- a/fail2ban/server/server.py +++ b/fail2ban/server/server.py @@ -212,7 +212,7 @@ class Server: filter_ = self.__jails[name].filter if isinstance(filter_, FileFilter): return [m.getFileName() - for m in filter_.getLogPath()] + for m in filter_.getLogs()] else: # pragma: systemd no cover logSys.info("Jail %s is not a FileFilter instance" % name) return [] diff --git a/fail2ban/tests/actionstestcase.py b/fail2ban/tests/actionstestcase.py index bb295967..0ceb35d5 100644 --- a/fail2ban/tests/actionstestcase.py +++ b/fail2ban/tests/actionstestcase.py @@ -94,15 +94,15 @@ class ExecuteActions(LogCaptureTestCase): "Action", os.path.join(TEST_FILES_DIR, "action.d/action.py"), {'opt1': 'value'}) - self.assertTrue(self._is_logged("TestAction initialised")) + self.assertLogged("TestAction initialised") self.__actions.start() time.sleep(3) - self.assertTrue(self._is_logged("TestAction action start")) + self.assertLogged("TestAction action start") self.__actions.stop() self.__actions.join() - self.assertTrue(self._is_logged("TestAction action stop")) + self.assertLogged("TestAction action stop") self.assertRaises(IOError, self.__actions.add, "Action3", "/does/not/exist.py", {}) @@ -136,10 +136,10 @@ class ExecuteActions(LogCaptureTestCase): {}) self.__actions.start() time.sleep(3) - self.assertTrue(self._is_logged("Failed to start")) + self.assertLogged("Failed to start") self.__actions.stop() self.__actions.join() - self.assertTrue(self._is_logged("Failed to stop")) + self.assertLogged("Failed to stop") def testBanActionsAInfo(self): # Action which deletes IP address from aInfo @@ -155,13 +155,13 @@ class ExecuteActions(LogCaptureTestCase): self.__actions._Actions__checkBan() # Will fail if modification of aInfo from first action propagates # to second action, as both delete same key - self.assertFalse(self._is_logged("Failed to execute ban")) - self.assertTrue(self._is_logged("action1 ban deleted aInfo IP")) - self.assertTrue(self._is_logged("action2 ban deleted aInfo IP")) + self.assertNotLogged("Failed to execute ban") + self.assertLogged("action1 ban deleted aInfo IP") + self.assertLogged("action2 ban deleted aInfo IP") self.__actions._Actions__flushBan() # Will fail if modification of aInfo from first action propagates # to second action, as both delete same key - self.assertFalse(self._is_logged("Failed to execute unban")) - self.assertTrue(self._is_logged("action1 unban deleted aInfo IP")) - self.assertTrue(self._is_logged("action2 unban deleted aInfo IP")) + self.assertNotLogged("Failed to execute unban") + self.assertLogged("action1 unban deleted aInfo IP") + self.assertLogged("action2 unban deleted aInfo IP") diff --git a/fail2ban/tests/actiontestcase.py b/fail2ban/tests/actiontestcase.py index febbc619..289d8896 100644 --- a/fail2ban/tests/actiontestcase.py +++ b/fail2ban/tests/actiontestcase.py @@ -143,17 +143,17 @@ class CommandActionTest(LogCaptureTestCase): self.__action.actionunban = "true" self.assertEqual(self.__action.actionunban, 'true') - self.assertFalse(self._is_logged('returned')) + self.assertNotLogged('returned') # no action was actually executed yet self.__action.ban({'ip': None}) - self.assertTrue(self._is_logged('Invariant check failed')) - self.assertTrue(self._is_logged('returned successfully')) + self.assertLogged('Invariant check failed') + self.assertLogged('returned successfully') def testExecuteActionEmptyUnban(self): self.__action.actionunban = "" self.__action.unban({}) - self.assertTrue(self._is_logged('Nothing to do')) + self.assertLogged('Nothing to do') def testExecuteActionStartCtags(self): self.__action.HOST = "192.0.2.0" @@ -168,7 +168,7 @@ class CommandActionTest(LogCaptureTestCase): self.__action.actionban = "rm /tmp/fail2ban.test" self.__action.actioncheck = "[ -e /tmp/fail2ban.test ]" self.assertRaises(RuntimeError, self.__action.ban, {'ip': None}) - self.assertTrue(self._is_logged('Unable to restore environment')) + self.assertLogged('Unable to restore environment') def testExecuteActionChangeCtags(self): self.assertRaises(AttributeError, getattr, self.__action, "ROST") @@ -187,22 +187,23 @@ class CommandActionTest(LogCaptureTestCase): def testExecuteActionStartEmpty(self): self.__action.actionstart = "" self.__action.start() - self.assertTrue(self._is_logged('Nothing to do')) + self.assertLogged('Nothing to do') def testExecuteIncorrectCmd(self): CommandAction.executeCmd('/bin/ls >/dev/null\nbogusXXX now 2>/dev/null') - self.assertTrue(self._is_logged('HINT on 127: "Command not found"')) + self.assertLogged('HINT on 127: "Command not found"') def testExecuteTimeout(self): stime = time.time() # Should take a minute - self.assertRaises( - RuntimeError, CommandAction.executeCmd, 'sleep 60', timeout=2) + self.assertFalse(CommandAction.executeCmd('sleep 60', timeout=2)) # give a test still 1 second, because system could be too busy self.assertTrue(time.time() >= stime + 2 and time.time() <= stime + 3) - self.assertTrue(self._is_logged('sleep 60 -- timed out after 2 seconds') - or self._is_logged('sleep 60 -- timed out after 3 seconds')) - self.assertTrue(self._is_logged('sleep 60 -- killed with SIGTERM')) + self.assertLogged( + 'sleep 60 -- timed out after 2 seconds', + 'sleep 60 -- timed out after 3 seconds' + ) + self.assertLogged('sleep 60 -- killed with SIGTERM') def testExecuteTimeoutWithNastyChildren(self): # temporary file for a nasty kid shell script @@ -222,21 +223,20 @@ class CommandActionTest(LogCaptureTestCase): return int(f.read()) # First test if can kill the bastard - self.assertRaises( - RuntimeError, CommandAction.executeCmd, 'bash %s' % tmpFilename, timeout=.1) - # Verify that the proccess itself got killed + self.assertFalse(CommandAction.executeCmd( + 'bash %s' % tmpFilename, timeout=.1)) + # Verify that the process itself got killed self.assertFalse(pid_exists(getnastypid())) # process should have been killed - self.assertTrue(self._is_logged('timed out')) - self.assertTrue(self._is_logged('killed with SIGTERM')) + self.assertLogged('timed out') + self.assertLogged('killed with SIGTERM') # A bit evolved case even though, previous test already tests killing children processes - self.assertRaises( - RuntimeError, CommandAction.executeCmd, 'out=`bash %s`; echo ALRIGHT' % tmpFilename, - timeout=.2) - # Verify that the proccess itself got killed + self.assertFalse(CommandAction.executeCmd( + 'out=`bash %s`; echo ALRIGHT' % tmpFilename, timeout=.2)) + # Verify that the process itself got killed self.assertFalse(pid_exists(getnastypid())) - self.assertTrue(self._is_logged('timed out')) - self.assertTrue(self._is_logged('killed with SIGTERM')) + self.assertLogged('timed out') + self.assertLogged('killed with SIGTERM') os.unlink(tmpFilename) os.unlink(tmpFilename + '.pid') @@ -244,11 +244,11 @@ class CommandActionTest(LogCaptureTestCase): def testCaptureStdOutErr(self): CommandAction.executeCmd('echo "How now brown cow"') - self.assertTrue(self._is_logged("'How now brown cow\\n'")) + self.assertLogged("'How now brown cow\\n'") CommandAction.executeCmd( 'echo "The rain in Spain stays mainly in the plain" 1>&2') - self.assertTrue(self._is_logged( - "'The rain in Spain stays mainly in the plain\\n'")) + self.assertLogged( + "'The rain in Spain stays mainly in the plain\\n'") def testCallingMap(self): mymap = CallingMap(callme=lambda: str(10), error=lambda: int('a'), diff --git a/fail2ban/tests/clientreadertestcase.py b/fail2ban/tests/clientreadertestcase.py index 00128c33..0a3734e5 100644 --- a/fail2ban/tests/clientreadertestcase.py +++ b/fail2ban/tests/clientreadertestcase.py @@ -28,18 +28,20 @@ import re import shutil import tempfile import unittest -from ..client.configreader import ConfigReaderUnshared +from ..client.configreader import ConfigReader, ConfigReaderUnshared from ..client import configparserinc from ..client.jailreader import JailReader from ..client.filterreader import FilterReader from ..client.jailsreader import JailsReader from ..client.actionreader import ActionReader from ..client.configurator import Configurator +from ..version import version from .utils import LogCaptureTestCase TEST_FILES_DIR = os.path.join(os.path.dirname(__file__), "files") from .utils import CONFIG_DIR +CONFIG_DIR_TESTSHARE_CFG = {} STOCK = os.path.exists(os.path.join('config','fail2ban.conf')) @@ -165,24 +167,24 @@ class JailReaderTest(LogCaptureTestCase): self.__share_cfg = {} def testIncorrectJail(self): - jail = JailReader('XXXABSENTXXX', basedir=CONFIG_DIR, share_config = self.__share_cfg) + jail = JailReader('XXXABSENTXXX', basedir=CONFIG_DIR, share_config=self.__share_cfg) self.assertRaises(ValueError, jail.read) def testJailActionEmpty(self): - jail = JailReader('emptyaction', basedir=IMPERFECT_CONFIG, share_config = self.__share_cfg) + jail = JailReader('emptyaction', basedir=IMPERFECT_CONFIG, share_config=self.__share_cfg) self.assertTrue(jail.read()) self.assertTrue(jail.getOptions()) self.assertTrue(jail.isEnabled()) - self.assertTrue(self._is_logged('No filter set for jail emptyaction')) - self.assertTrue(self._is_logged('No actions were defined for emptyaction')) + self.assertLogged('No filter set for jail emptyaction') + self.assertLogged('No actions were defined for emptyaction') def testJailActionFilterMissing(self): - jail = JailReader('missingbitsjail', basedir=IMPERFECT_CONFIG, share_config = self.__share_cfg) + jail = JailReader('missingbitsjail', basedir=IMPERFECT_CONFIG, share_config=self.__share_cfg) self.assertTrue(jail.read()) self.assertFalse(jail.getOptions()) self.assertTrue(jail.isEnabled()) - self.assertTrue(self._is_logged("Found no accessible config files for 'filter.d/catchallthebadies' under %s" % IMPERFECT_CONFIG)) - self.assertTrue(self._is_logged('Unable to read the filter')) + self.assertLogged("Found no accessible config files for 'filter.d/catchallthebadies' under %s" % IMPERFECT_CONFIG) + self.assertLogged('Unable to read the filter') def testJailActionBrokenDef(self): jail = JailReader('brokenactiondef', basedir=IMPERFECT_CONFIG, @@ -190,17 +192,17 @@ class JailReaderTest(LogCaptureTestCase): self.assertTrue(jail.read()) self.assertFalse(jail.getOptions()) self.assertTrue(jail.isEnabled()) - self.assertTrue(self._is_logged('Error in action definition joho[foo')) + self.assertLogged('Error in action definition joho[foo') # This unittest has been deactivated for some time... - # self.assertTrue(self._is_logged( - # 'Caught exception: While reading action joho[foo we should have got 1 or 2 groups. Got: 0')) + # self.assertLogged( + # 'Caught exception: While reading action joho[foo we should have got 1 or 2 groups. Got: 0') # let's test for what is actually logged and handle changes in the future - self.assertTrue(self._is_logged( - "Caught exception: 'NoneType' object has no attribute 'endswith'")) + self.assertLogged( + "Caught exception: 'NoneType' object has no attribute 'endswith'") if STOCK: def testStockSSHJail(self): - jail = JailReader('sshd', basedir=CONFIG_DIR, share_config = self.__share_cfg) # we are running tests from root project dir atm + jail = JailReader('sshd', basedir=CONFIG_DIR, share_config=self.__share_cfg) # we are running tests from root project dir atm self.assertTrue(jail.read()) self.assertTrue(jail.getOptions()) self.assertFalse(jail.isEnabled()) @@ -221,7 +223,7 @@ class JailReaderTest(LogCaptureTestCase): self.assertEqual(('mail--ho_is', {}), JailReader.extractOptions("mail--ho_is['s']")) #self.printLog() - #self.assertTrue(self._is_logged("Invalid argument ['s'] in ''s''")) + #self.assertLogged("Invalid argument ['s'] in ''s''") self.assertEqual(('mail', {'a': ','}), JailReader.extractOptions("mail[a=',']")) @@ -251,6 +253,34 @@ class JailReaderTest(LogCaptureTestCase): result = JailReader.extractOptions(option) self.assertEqual(expected, result) + def testVersionAgent(self): + jail = JailReader('blocklisttest', force_enable=True, basedir=CONFIG_DIR) + # emulate jail.read(), because such jail not exists: + ConfigReader.read(jail, "jail"); + sections = jail._cfg.get_sections() + sections['blocklisttest'] = dict((('__name__', 'blocklisttest'), + ('filter', ''), ('failregex', '^test <HOST>$'), + ('sender', 'f2b-test@example.com'), ('blocklist_de_apikey', 'test-key'), + ('action', + '%(action_blocklist_de)s\n' + '%(action_badips_report)s\n' + '%(action_badips)s\n' + 'mynetwatchman[port=1234,protocol=udp,agent="%(fail2ban_agent)s"]' + ), + )) + # get options: + self.assertTrue(jail.getOptions()) + # convert and get stream + stream = jail.convert() + # get action and retrieve agent from it, compare with agent saved in version: + act = [o for o in stream if len(o) > 4 and (o[4] == 'agent' or o[4].endswith('badips.py'))] + useragent = 'Fail2Ban/%s' % version + self.assertEqual(len(act), 4) + self.assertEqual(act[0], ['set', 'blocklisttest', 'action', 'blocklist_de', 'agent', useragent]) + self.assertEqual(act[1], ['set', 'blocklisttest', 'action', 'badips', 'agent', useragent]) + self.assertEqual(eval(act[2][5]).get('agent', '<wrong>'), useragent) + self.assertEqual(act[3], ['set', 'blocklisttest', 'action', 'mynetwatchman', 'agent', useragent]) + def testGlob(self): d = tempfile.mkdtemp(prefix="f2b-temp") # Generate few files @@ -265,7 +295,7 @@ class JailReaderTest(LogCaptureTestCase): self.assertEqual(JailReader._glob(os.path.join(d, '*')), [f1]) # since f2 is dangling -- empty list self.assertEqual(JailReader._glob(f2), []) - self.assertTrue(self._is_logged('File %s is a dangling link, thus cannot be monitored' % f2)) + self.assertLogged('File %s is a dangling link, thus cannot be monitored' % f2) self.assertEqual(JailReader._glob(os.path.join(d, 'nonexisting')), []) os.remove(f1) os.remove(f2) @@ -274,6 +304,10 @@ class JailReaderTest(LogCaptureTestCase): class FilterReaderTest(unittest.TestCase): + def __init__(self, *args, **kwargs): + super(FilterReaderTest, self).__init__(*args, **kwargs) + self.__share_cfg = {} + def testConvert(self): output = [['set', 'testcase01', 'addfailregex', "^\\s*(?:\\S+ )?(?:kernel: \\[\\d+\\.\\d+\\] )?(?:@vserver_\\S+ )" @@ -311,9 +345,8 @@ class FilterReaderTest(unittest.TestCase): # is unreliable self.assertEqual(sorted(filterReader.convert()), sorted(output)) - filterReader = FilterReader( - "testcase01", "testcase01", {'maxlines': "5"}) - filterReader.setBaseDir(TEST_FILES_DIR) + filterReader = FilterReader("testcase01", "testcase01", {'maxlines': "5"}, + share_config=self.__share_cfg, basedir=TEST_FILES_DIR) filterReader.read() #filterReader.getOptions(["failregex", "ignoreregex"]) filterReader.getOptions(None) @@ -322,8 +355,8 @@ class FilterReaderTest(unittest.TestCase): def testFilterReaderSubstitionDefault(self): output = [['set', 'jailname', 'addfailregex', 'to=sweet@example.com fromip=<IP>']] - filterReader = FilterReader('substition', "jailname", {}) - filterReader.setBaseDir(TEST_FILES_DIR) + filterReader = FilterReader('substition', "jailname", {}, + share_config=self.__share_cfg, basedir=TEST_FILES_DIR) filterReader.read() filterReader.getOptions(None) c = filterReader.convert() @@ -331,16 +364,34 @@ class FilterReaderTest(unittest.TestCase): def testFilterReaderSubstitionSet(self): output = [['set', 'jailname', 'addfailregex', 'to=sour@example.com fromip=<IP>']] - filterReader = FilterReader('substition', "jailname", {'honeypot': 'sour@example.com'}) - filterReader.setBaseDir(TEST_FILES_DIR) + filterReader = FilterReader('substition', "jailname", {'honeypot': 'sour@example.com'}, + share_config=self.__share_cfg, basedir=TEST_FILES_DIR) + filterReader.read() + filterReader.getOptions(None) + c = filterReader.convert() + self.assertEqual(sorted(c), sorted(output)) + + def testFilterReaderSubstitionKnown(self): + output = [['set', 'jailname', 'addfailregex', 'to=test,sweet@example.com,test2,sweet@example.com fromip=<IP>']] + filterName, filterOpt = JailReader.extractOptions( + 'substition[honeypot="<sweet>,<known/honeypot>", sweet="test,<known/honeypot>,test2"]') + filterReader = FilterReader('substition', "jailname", filterOpt, + share_config=self.__share_cfg, basedir=TEST_FILES_DIR) filterReader.read() filterReader.getOptions(None) c = filterReader.convert() self.assertEqual(sorted(c), sorted(output)) def testFilterReaderSubstitionFail(self): - filterReader = FilterReader('substition', "jailname", {'honeypot': '<sweet>', 'sweet': '<honeypot>'}) - filterReader.setBaseDir(TEST_FILES_DIR) + # directly subst the same var : + filterReader = FilterReader('substition', "jailname", {'honeypot': '<honeypot>'}, + share_config=self.__share_cfg, basedir=TEST_FILES_DIR) + filterReader.read() + filterReader.getOptions(None) + self.assertRaises(ValueError, FilterReader.convert, filterReader) + # cross subst the same var : + filterReader = FilterReader('substition', "jailname", {'honeypot': '<sweet>', 'sweet': '<honeypot>'}, + share_config=self.__share_cfg, basedir=TEST_FILES_DIR) filterReader.read() filterReader.getOptions(None) self.assertRaises(ValueError, FilterReader.convert, filterReader) @@ -463,8 +514,8 @@ class JailsReaderTest(LogCaptureTestCase): ['start', 'missinglogfiles'], ['start', 'brokenaction'], ['start', 'parse_to_end_of_jail.conf'],])) - self.assertTrue(self._is_logged("Errors in jail 'missingbitsjail'. Skipping...")) - self.assertTrue(self._is_logged("No file(s) found for glob /weapons/of/mass/destruction")) + self.assertLogged("Errors in jail 'missingbitsjail'. Skipping...") + self.assertLogged("No file(s) found for glob /weapons/of/mass/destruction") if STOCK: def testReadStockActionConf(self): @@ -496,7 +547,7 @@ class JailsReaderTest(LogCaptureTestCase): #old_comm_commands = comm_commands[:] # make a copy #self.assertRaises(ValueError, jails.getOptions, "BOGUS") #self.printLog() - #self.assertTrue(self._is_logged("No section: 'BOGUS'")) + #self.assertLogged("No section: 'BOGUS'") ## and there should be no side-effects #self.assertEqual(jails.convert(), old_comm_commands) @@ -508,12 +559,13 @@ class JailsReaderTest(LogCaptureTestCase): if jail == 'INCLUDES': continue filterName = jails.get(jail, 'filter') + filterName, filterOpt = JailReader.extractOptions(filterName) allFilters.add(filterName) self.assertTrue(len(filterName)) # moreover we must have a file for it # and it must be readable as a Filter - filterReader = FilterReader(filterName, jail, {}) - filterReader.setBaseDir(CONFIG_DIR) + filterReader = FilterReader(filterName, jail, filterOpt, + share_config=self.__share_cfg, basedir=CONFIG_DIR) self.assertTrue(filterReader.read(),"Failed to read filter:" + filterName) # opens fine filterReader.getOptions({}) # reads fine @@ -551,7 +603,10 @@ class JailsReaderTest(LogCaptureTestCase): filters = set(os.path.splitext(os.path.split(a)[1])[0] for a in glob.glob(os.path.join('config', 'filter.d', '*.conf')) if not a.endswith('common.conf')) - filters_jail = set(jail.options['filter'] for jail in jails.jails) + # get filters of all jails (filter names without options inside filter[...]) + filters_jail = set( + JailReader.extractOptions(jail.options['filter'])[0] for jail in jails.jails + ) self.maxDiff = None self.assertTrue(filters.issubset(filters_jail), "More filters exists than are referenced in stock jail.conf %r" % filters.difference(filters_jail)) @@ -569,6 +624,12 @@ class JailsReaderTest(LogCaptureTestCase): # by default we have lots of jails ;) self.assertTrue(len(comm_commands)) + # some common sanity checks for commands + for command in comm_commands: + if len(command) >= 3 and [command[0], command[2]] == ['set', 'bantime']: + self.assertTrue(isinstance(command[3], int)) + self.assertTrue(command[3] > 0) + # and we know even some of them by heart for j in ['sshd', 'recidive']: # by default we have 'auto' backend ATM diff --git a/fail2ban/tests/databasetestcase.py b/fail2ban/tests/databasetestcase.py index dd813ee6..3d156eda 100644 --- a/fail2ban/tests/databasetestcase.py +++ b/fail2ban/tests/databasetestcase.py @@ -316,7 +316,7 @@ class DatabaseTest(LogCaptureTestCase): ticket.setAttempt(5) self.jail.putFailTicket(ticket) actions._Actions__checkBan() - self.assertTrue(self._is_logged("ban ainfo %s, %s, %s, %s" % (True, True, True, True))) + self.assertLogged("ban ainfo %s, %s, %s, %s" % (True, True, True, True)) def testPurge(self): if Fail2BanDb is None: # pragma: no cover diff --git a/fail2ban/tests/datedetectortestcase.py b/fail2ban/tests/datedetectortestcase.py index 0d758640..df1733d3 100644 --- a/fail2ban/tests/datedetectortestcase.py +++ b/fail2ban/tests/datedetectortestcase.py @@ -46,13 +46,23 @@ class DateDetectorTest(unittest.TestCase): tearDownMyTime() def testGetEpochTime(self): - log = "1138049999 [sshd] error: PAM: Authentication failure" - #date = [2006, 1, 23, 21, 59, 59, 0, 23, 0] - dateUnix = 1138049999.0 - - ( datelog, matchlog ) = self.__datedetector.getTime(log) - self.assertEqual(datelog, dateUnix) - self.assertEqual(matchlog.group(), '1138049999') + # correct epoch time, using all variants: + for dateUnix in (1138049999, 32535244799): + for date in ("%s", "[%s]", "[%s.555]", "audit(%s.555:101)"): + date = date % dateUnix + log = date + " [sshd] error: PAM: Authentication failure" + datelog = self.__datedetector.getTime(log) + self.assertTrue(datelog, "Parse epoch time for %s failed" % (date,)) + ( datelog, matchlog ) = datelog + self.assertEqual(int(datelog), dateUnix) + self.assertIn(matchlog.group(), (str(dateUnix), str(dateUnix)+'.555')) + # wrong, no epoch time (< 10 digits, more as 11 digits, begin/end of word) : + for dateUnix in ('123456789', '9999999999999999', '1138049999A', 'A1138049999'): + for date in ("%s", "[%s]", "[%s.555]", "audit(%s.555:101)"): + date = date % dateUnix + log = date + " [sshd] error: PAM: Authentication failure" + datelog = self.__datedetector.getTime(log) + self.assertFalse(datelog) def testGetTime(self): log = "Jan 23 21:59:59 [sshd] error: PAM: Authentication failure" @@ -74,6 +84,7 @@ class DateDetectorTest(unittest.TestCase): (False, "Jan 23 21:59:59"), (False, "Sun Jan 23 21:59:59 2005"), (False, "Sun Jan 23 21:59:59"), + (False, "Sun Jan 23 2005 21:59:59"), (False, "2005/01/23 21:59:59"), (False, "2005.01.23 21:59:59"), (False, "23/01/2005 21:59:59"), diff --git a/fail2ban/tests/fail2banregextestcase.py b/fail2ban/tests/fail2banregextestcase.py new file mode 100644 index 00000000..2fd362c7 --- /dev/null +++ b/fail2ban/tests/fail2banregextestcase.py @@ -0,0 +1,181 @@ +# emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*- +# vi: set ft=python sts=4 ts=4 sw=4 noet : + +# This file is part of Fail2Ban. +# +# Fail2Ban is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# Fail2Ban is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with Fail2Ban; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. + +# Fail2Ban developers + +__author__ = "Serg Brester" +__copyright__ = "Copyright (c) 2015 Serg G. Brester (sebres), 2008- Fail2Ban Contributors" +__license__ = "GPL" + +from __builtin__ import open as fopen +import unittest +import getpass +import os +import sys +import time +import tempfile +import uuid + +try: + from systemd import journal +except ImportError: + journal = None + +from ..client import fail2banregex +from ..client.fail2banregex import Fail2banRegex, get_opt_parser, output +from .utils import LogCaptureTestCase, logSys + + +fail2banregex.logSys = logSys +def _test_output(*args): + logSys.info(args[0]) + +fail2banregex.output = _test_output + +CONF_FILES_DIR = os.path.abspath( + os.path.join(os.path.dirname(__file__),"..", "..", "config")) +TEST_FILES_DIR = os.path.join(os.path.dirname(__file__), "files") + + +def _Fail2banRegex(*args): + parser = get_opt_parser() + (opts, args) = parser.parse_args(list(args)) + return (opts, args, Fail2banRegex(opts)) + +class Fail2banRegexTest(LogCaptureTestCase): + + RE_00 = r"(?:(?:Authentication failure|Failed [-/\w+]+) for(?: [iI](?:llegal|nvalid) user)?|[Ii](?:llegal|nvalid) user|ROOT LOGIN REFUSED) .*(?: from|FROM) <HOST>" + + FILENAME_01 = os.path.join(TEST_FILES_DIR, "testcase01.log") + FILENAME_02 = os.path.join(TEST_FILES_DIR, "testcase02.log") + FILENAME_WRONGCHAR = os.path.join(TEST_FILES_DIR, "testcase-wrong-char.log") + + FILTER_SSHD = os.path.join(CONF_FILES_DIR, 'filter.d', 'sshd.conf') + + def setUp(self): + """Call before every test case.""" + LogCaptureTestCase.setUp(self) + + def tearDown(self): + """Call after every test case.""" + LogCaptureTestCase.tearDown(self) + + def testWrongRE(self): + (opts, args, fail2banRegex) = _Fail2banRegex( + "test", r".** from <HOST>$" + ) + self.assertRaises(Exception, lambda: fail2banRegex.start(opts, args)) + self.assertLogged("Unable to compile regular expression") + + def testWrongIngnoreRE(self): + (opts, args, fail2banRegex) = _Fail2banRegex( + "test", r".*? from <HOST>$", r".**" + ) + self.assertRaises(Exception, lambda: fail2banRegex.start(opts, args)) + self.assertLogged("Unable to compile regular expression") + + def testDirectFound(self): + (opts, args, fail2banRegex) = _Fail2banRegex( + "--print-all-matched", "--print-no-missed", + "Dec 31 11:59:59 [sshd] error: PAM: Authentication failure for kevin from 192.0.2.0", + r"Authentication failure for .*? from <HOST>$" + ) + self.assertTrue(fail2banRegex.start(opts, args)) + self.assertLogged('Lines: 1 lines, 0 ignored, 1 matched, 0 missed') + + def testDirectNotFound(self): + (opts, args, fail2banRegex) = _Fail2banRegex( + "--print-all-missed", + "Dec 31 11:59:59 [sshd] error: PAM: Authentication failure for kevin from 192.0.2.0", + r"XYZ from <HOST>$" + ) + self.assertTrue(fail2banRegex.start(opts, args)) + self.assertLogged('Lines: 1 lines, 0 ignored, 0 matched, 1 missed') + + def testDirectIgnored(self): + (opts, args, fail2banRegex) = _Fail2banRegex( + "--print-all-ignored", + "Dec 31 11:59:59 [sshd] error: PAM: Authentication failure for kevin from 192.0.2.0", + r"Authentication failure for .*? from <HOST>$", + r"kevin from 192.0.2.0$" + ) + self.assertTrue(fail2banRegex.start(opts, args)) + self.assertLogged('Lines: 1 lines, 1 ignored, 0 matched, 0 missed') + + def testDirectRE_1(self): + (opts, args, fail2banRegex) = _Fail2banRegex( + "--print-all-matched", + Fail2banRegexTest.FILENAME_01, + Fail2banRegexTest.RE_00 + ) + self.assertTrue(fail2banRegex.start(opts, args)) + self.assertLogged('Lines: 19 lines, 0 ignored, 13 matched, 6 missed') + + self.assertLogged('Error decoding line'); + self.assertLogged('Continuing to process line ignoring invalid characters') + + self.assertLogged('Dez 31 11:59:59 [sshd] error: PAM: Authentication failure for kevin from 193.168.0.128') + self.assertLogged('Dec 31 11:59:59 [sshd] error: PAM: Authentication failure for kevin from 87.142.124.10') + + def testDirectRE_2(self): + (opts, args, fail2banRegex) = _Fail2banRegex( + "--print-all-matched", + Fail2banRegexTest.FILENAME_02, + Fail2banRegexTest.RE_00 + ) + self.assertTrue(fail2banRegex.start(opts, args)) + self.assertLogged('Lines: 13 lines, 0 ignored, 5 matched, 8 missed') + + def testVerbose(self): + (opts, args, fail2banRegex) = _Fail2banRegex( + "--verbose", "--print-no-missed", + Fail2banRegexTest.FILENAME_02, + Fail2banRegexTest.RE_00 + ) + self.assertTrue(fail2banRegex.start(opts, args)) + self.assertLogged('Lines: 13 lines, 0 ignored, 5 matched, 8 missed') + + self.assertLogged('141.3.81.106 Fri Aug 14 11:53:59 2015') + self.assertLogged('141.3.81.106 Fri Aug 14 11:54:59 2015') + + def testWronChar(self): + (opts, args, fail2banRegex) = _Fail2banRegex( + Fail2banRegexTest.FILENAME_WRONGCHAR, Fail2banRegexTest.FILTER_SSHD + ) + self.assertTrue(fail2banRegex.start(opts, args)) + self.assertLogged('Lines: 4 lines, 0 ignored, 2 matched, 2 missed') + + self.assertLogged('Error decoding line'); + self.assertLogged('Continuing to process line ignoring invalid characters:', '2015-01-14 20:00:58 user '); + self.assertLogged('Continuing to process line ignoring invalid characters:', '2015-01-14 20:00:59 user '); + + self.assertLogged('Nov 8 00:16:12 main sshd[32548]: input_userauth_request: invalid user llinco') + self.assertLogged('Nov 8 00:16:12 main sshd[32547]: pam_succeed_if(sshd:auth): error retrieving information about user llinco') + + def testWronCharDebuggex(self): + (opts, args, fail2banRegex) = _Fail2banRegex( + "--debuggex", "--print-all-matched", + Fail2banRegexTest.FILENAME_WRONGCHAR, Fail2banRegexTest.FILTER_SSHD + ) + self.assertTrue(fail2banRegex.start(opts, args)) + self.assertLogged('Lines: 4 lines, 0 ignored, 2 matched, 2 missed') + + self.assertLogged('http://') + + diff --git a/fail2ban/tests/files/logs/asterisk b/fail2ban/tests/files/logs/asterisk index ab018ba9..aa32a290 100644 --- a/fail2ban/tests/files/logs/asterisk +++ b/fail2ban/tests/files/logs/asterisk @@ -59,3 +59,11 @@ Nov 4 18:30:40 localhost asterisk[32229]: NOTICE[32257]: chan_sip.c:23417 in han # match UTF-8 in SessionID # failJSON: { "time": "2015-05-25T07:52:36", "match": true, "host": "10.250.251.252" } [2015-05-25 07:52:36] SECURITY[6988] res_security_log.c: SecurityEvent="InvalidAccountID",EventTV="2015-05-25T07:52:36.888+0300",Severity="Error",Service="PJSIP",EventVersion="1",AccountID="70000180",SessionID="Негодяй",LocalAddress="IPV4/UDP/1.2.3.4/5060",RemoteAddress="IPV4/UDP/10.250.251.252/5061" + +# match phone numbers with + symbol (and without number, or other context) +# failJSON: { "time": "2016-01-28T10:22:27", "match": true , "host": "1.2.3.4" } +[2016-01-28 10:22:27] NOTICE[3477][C-000003bb] chan_sip.c: Call from '' (1.2.3.4:10836) to extension '++441772285411' rejected because extension not found in context 'default'. +# failJSON: { "time": "2016-01-28T10:34:31", "match": true , "host": "1.2.3.4" } +[2016-01-28 10:34:31] NOTICE[3477][C-000003c3] chan_sip.c: Call from '' (1.2.3.4:10836) to extension '0+441772285407' rejected because extension not found in context 'default'. +# failJSON: { "time": "2016-01-28T10:34:33", "match": true , "host": "1.2.3.4" } +[2016-01-28 10:34:33] NOTICE[3477][C-000003c3] chan_sip.c: Call from '' (1.2.3.4:10836) to extension '' rejected because extension not found in context 'my-context'. diff --git a/fail2ban/tests/files/logs/haproxy-http-auth b/fail2ban/tests/files/logs/haproxy-http-auth new file mode 100644 index 00000000..298f1972 --- /dev/null +++ b/fail2ban/tests/files/logs/haproxy-http-auth @@ -0,0 +1,4 @@ +# failJSON: { "match": false } +Nov 14 22:45:27 test haproxy[760]: 192.168.33.1:58444 [14/Nov/2015:22:45:25.439] main app/app1 1939/0/1/0/1940 403 5168 - - ---- 3/3/0/0/0 0/0 "GET / HTTP/1.1" +# failJSON: { "time": "2004-11-14T22:45:11", "match": true , "host": "192.168.33.1" } +Nov 14 22:45:11 test haproxy[760]: 192.168.33.1:58430 [14/Nov/2015:22:45:11.608] main main/<NOSRV> -1/-1/-1/-1/0 401 248 - - PR-- 0/0/0/0/0 0/0 "GET / HTTP/1.1" diff --git a/fail2ban/tests/files/logs/murmur b/fail2ban/tests/files/logs/murmur new file mode 100644 index 00000000..bc18b7ea --- /dev/null +++ b/fail2ban/tests/files/logs/murmur @@ -0,0 +1,5 @@ +# failJSON: { "time": "2015-11-29T16:38:01", "match": true , "host": "192.168.0.1" } +<W>2015-11-29 16:38:01.818 1 => <4:testUsernameOne(-1)> Rejected connection from 192.168.0.1:29530: Invalid server password + +# failJSON: { "time": "2015-11-29T17:18:20", "match": true , "host": "192.168.1.2" } +<W>2015-11-29 17:18:20.962 1 => <8:testUsernameTwo(-1)> Rejected connection from 192.168.1.2:29761: Wrong certificate or password for existing user diff --git a/fail2ban/tests/files/logs/mysqld-auth b/fail2ban/tests/files/logs/mysqld-auth index aa684266..c77d214d 100644 --- a/fail2ban/tests/files/logs/mysqld-auth +++ b/fail2ban/tests/files/logs/mysqld-auth @@ -15,3 +15,10 @@ Sep 16 21:30:26 catinthehat mysqld: 130916 21:30:26 [Warning] Access denied for # failJSON: { "time": "2004-09-16T21:30:32", "match": true , "host": "74.207.241.159" } Sep 16 21:30:32 catinthehat mysqld: 130916 21:30:32 [Warning] Access denied for user 'hacker'@'74.207.241.159' (using password: NO) +# failJSON: { "time": "2015-10-07T06:09:42", "match": true , "host": "127.0.0.1", "desc": "mysql 5.6 log format" } +2015-10-07 06:09:42 5907 [Warning] Access denied for user 'root'@'127.0.0.1' (using password: YES) +# failJSON: { "time": "2016-02-24T15:26:18", "match": true , "host": "localhost", "desc": "mysql 5.6 log format, Note instead of Warning" } +2016-02-24T15:26:18.237955 6 [Note] Access denied for user 'root'@'localhost' (using password: YES) + +# failJSON: { "time": "2016-02-24T15:26:18", "match": false , "host": "localhost", "desc": "A hypothetical example of injection having full log line first (for paranoid yoh)" } +2016-02-24T15:26:18.237955 6 [Note] Access denied for user 'root'@'localhost' (using password: YES) condition lead to a hypothetical failure diff --git a/fail2ban/tests/files/logs/nginx-limit-req b/fail2ban/tests/files/logs/nginx-limit-req new file mode 100644 index 00000000..68f1b239 --- /dev/null +++ b/fail2ban/tests/files/logs/nginx-limit-req @@ -0,0 +1,6 @@ + +# failJSON: { "time": "2015-10-29T20:01:02", "match": true , "host": "1.2.3.4" } +2015/10/29 20:01:02 [error] 256554#0: *99927 limiting requests, excess: 1.852 by zone "one", client: 1.2.3.4, server: example.com, request: "POST /index.htm HTTP/1.0", host: "exmaple.com" + +# failJSON: { "time": "2015-10-29T19:24:05", "match": true , "host": "192.0.2.0" } +2015/10/29 19:24:05 [error] 12684#12684: *22174 limiting requests, excess: 1.495 by zone "one", client: 192.0.2.0, server: example.com, request: "GET /index.php HTTP/1.1", host: "example.com", referrer: "https://example.com" diff --git a/fail2ban/tests/files/logs/openhab b/fail2ban/tests/files/logs/openhab new file mode 100644 index 00000000..983989a9 --- /dev/null +++ b/fail2ban/tests/files/logs/openhab @@ -0,0 +1,11 @@ +# should match +# failJSON: { "time": "2015-09-02T00:11:31", "match": true , "host": "175.18.15.10" } +175.18.15.10 - - [02/sept./2015:00:11:31 +0200] "GET /openhab.app HTTP/1.1" 401 1382 +# failJSON: { "time": "2015-09-02T00:11:31", "match": true , "host": "175.18.15.10" } +175.18.15.10 - - [02/sept./2015:00:11:31 +0200] "GET /rest/bindings HTTP/1.1" 401 1384 + +# Should not match +# failJSON: { "match": false } +175.18.15.11 - - [17/oct./2015:00:35:12 +0200] "GET /openhab.app?sitemap=default&poll=true&__async=true&__source=waHome HTTP/1.1" 200 92 +# failJSON: { "match": false } +175.18.15.11 - - [16/oct./2015:20:29:38 +0200] "GET /rest/sitemaps/default/maison HTTP/1.1" 200 2837 diff --git a/fail2ban/tests/files/logs/postfix b/fail2ban/tests/files/logs/postfix index ee8720f8..3ec2886a 100644 --- a/fail2ban/tests/files/logs/postfix +++ b/fail2ban/tests/files/logs/postfix @@ -23,3 +23,12 @@ Dec 18 02:05:46 platypus postfix/smtpd[16349]: improper command pipelining after # failJSON: { "time": "2004-12-21T21:17:29", "match": true , "host": "93.184.216.34" } Dec 21 21:17:29 xxx postfix/smtpd[7150]: NOQUEUE: reject: RCPT from badserver.example.com[93.184.216.34]: 450 4.7.1 Client host rejected: cannot find your hostname, [93.184.216.34]; from=<badactor@example.com> to=<goodguy@example.com> proto=ESMTP helo=<badserver.example.com> + +# failJSON: { "time": "2004-11-22T22:33:44", "match": true , "host": "1.2.3.4" } +Nov 22 22:33:44 xxx postfix/smtpd[11111]: NOQUEUE: reject: RCPT from 1-2-3-4.example.com[1.2.3.4]: 450 4.1.8 <some@nonexistant.tld>: Sender address rejected: Domain not found; from=<some@nonexistant.tld> to=<goodguy@example.com> proto=ESMTP helo=<1-2-3-4.example.com> + +# failJSON: { "time": "2005-01-31T13:55:24", "match": true , "host": "78.107.251.238" } +Jan 31 13:55:24 xxx postfix/smtpd[3462]: NOQUEUE: reject: EHLO from s271272.static.corbina.ru[78.107.251.238]: 504 5.5.2 <User>: Helo command rejected: need fully-qualified hostname; proto=SMTP helo=<User> + +# failJSON: { "time": "2005-01-31T13:55:24", "match": true , "host": "78.107.251.238" } +Jan 31 13:55:24 xxx postfix-incoming/smtpd[3462]: NOQUEUE: reject: EHLO from s271272.static.corbina.ru[78.107.251.238]: 504 5.5.2 <User>: Helo command rejected: need fully-qualified hostname; proto=SMTP helo=<User> diff --git a/fail2ban/tests/files/logs/postfix-rbl b/fail2ban/tests/files/logs/postfix-rbl index fd420fa7..eff01bf9 100644 --- a/fail2ban/tests/files/logs/postfix-rbl +++ b/fail2ban/tests/files/logs/postfix-rbl @@ -1,2 +1,5 @@ # failJSON: { "time": "2004-12-30T18:19:15", "match": true , "host": "93.184.216.34" } Dec 30 18:19:15 xxx postfix/smtpd[1574]: NOQUEUE: reject: RCPT from badguy.example.com[93.184.216.34]: 454 4.7.1 Service unavailable; Client host [93.184.216.34] blocked using rbl.example.com; http://www.example.com/query?ip=93.184.216.34; from=<spammer@example.com> to=<goodguy@example.com> proto=ESMTP helo=<badguy.example.com> + +# failJSON: { "time": "2004-12-30T18:19:15", "match": true , "host": "93.184.216.34" } +Dec 30 18:19:15 xxx postfix-incoming/smtpd[1574]: NOQUEUE: reject: RCPT from badguy.example.com[93.184.216.34]: 454 4.7.1 Service unavailable; Client host [93.184.216.34] blocked using rbl.example.com; http://www.example.com/query?ip=93.184.216.34; from=<spammer@example.com> to=<goodguy@example.com> proto=ESMTP helo=<badguy.example.com> diff --git a/fail2ban/tests/files/logs/postfix-sasl b/fail2ban/tests/files/logs/postfix-sasl index beb8b11d..586f9584 100644 --- a/fail2ban/tests/files/logs/postfix-sasl +++ b/fail2ban/tests/files/logs/postfix-sasl @@ -21,3 +21,5 @@ Jan 29 08:11:45 mail postfix/smtpd[10752]: warning: unknown[1.1.1.1]: SASL LOGIN # failJSON: { "time": "2005-02-03T08:29:28", "match": false , "host": "1.1.1.1" } Feb 3 08:29:28 mail postfix/smtpd[21022]: warning: unknown[1.1.1.1]: SASL LOGIN authentication failed: Connection lost to authentication server +# failJSON: { "time": "2005-01-29T08:11:45", "match": true , "host": "1.1.1.1" } +Jan 29 08:11:45 mail postfix-incoming/smtpd[10752]: warning: unknown[1.1.1.1]: SASL LOGIN authentication failed: Password: diff --git a/fail2ban/tests/files/logs/screensharingd b/fail2ban/tests/files/logs/screensharingd new file mode 100644 index 00000000..0ec0ebd6 --- /dev/null +++ b/fail2ban/tests/files/logs/screensharingd @@ -0,0 +1,12 @@ +# NOTE: dates here include years -- this is not the typical configuration for the system.log +# file on Mac OS. However, without it the test routines will use 2004 as the year and matches will not pass. +# +# failJSON: { "match": false } +Oct 27 2015 09:24:46 test1.beezwax.net screensharingd[1170]: Authentication: SUCCEEDED :: User Name: simon :: Viewer Address: 192.168.5.247 :: Type: DH +# +# failJSON: { "time": "2015-10-27T12:35:40", "match": true , "host": "192.168.5.247" } +Oct 27 2015 12:35:40 test1.beezwax.net screensharingd[1170]: Authentication: FAILED :: User Name: sdfsdfs () mro :: Viewer Address: 192.168.5.247 :: Type: DH +# failJSON: { "time": "2015-10-27T12:35:50", "match": true , "host": "192.168.5.247" } +Oct 27 2015 12:35:50 test1.beezwax.net screensharingd[1170]: Authentication: FAILED :: User Name: brown_s :: :: Viewer Address: 192.168.5.247 :: Type: DH +# failJSON: { "time": "2015-10-27T12:26:01", "match": true , "host": "192.168.5.247" } +Oct 27 2015 12:26:01 test1.beezwax.net screensharingd[1170]: Authentication: FAILED :: User Name: brown @! s:: :: Viewer Address: 192.168.5.247 :: Type: DH diff --git a/fail2ban/tests/files/logs/sshd b/fail2ban/tests/files/logs/sshd index 2f4400bb..7baf4be7 100644 --- a/fail2ban/tests/files/logs/sshd +++ b/fail2ban/tests/files/logs/sshd @@ -132,6 +132,12 @@ Nov 23 21:50:37 sshd[7148]: Connection closed by 61.0.0.1 [preauth] # failJSON: { "time": "2005-07-13T18:44:28", "match": true , "host": "89.24.13.192", "desc": "from gh-289" } Jul 13 18:44:28 mdop sshd[4931]: Received disconnect from 89.24.13.192: 3: com.jcraft.jsch.JSchException: Auth fail +# failJSON: { "time": "2004-10-01T17:27:44", "match": true , "host": "94.249.236.6", "desc": "newer format per commit 36919d9f" } +Oct 1 17:27:44 localhost sshd[24077]: error: Received disconnect from 94.249.236.6: 3: com.jcraft.jsch.JSchException: Auth fail [preauth] + +# failJSON: { "time": "2004-10-01T17:27:44", "match": true , "host": "94.249.236.6", "desc": "space in disconnect description per commit 36919d9f" } +Oct 1 17:27:44 localhost sshd[24077]: error: Received disconnect from 94.249.236.6: 3: Ha ha, suckers!: Auth fail [preauth] + # failJSON: { "match": false } Feb 12 04:09:18 localhost sshd[26713]: Connection from 115.249.163.77 port 51353 # failJSON: { "time": "2005-02-12T04:09:21", "match": true , "host": "115.249.163.77", "desc": "from gh-457" } @@ -142,6 +148,9 @@ Feb 12 04:09:18 localhost sshd[26713]: Connection from 115.249.163.77 port 51353 # failJSON: { "time": "2005-02-12T04:09:21", "match": true , "host": "115.249.163.77", "desc": "Multiline match with interface address" } Feb 12 04:09:21 localhost sshd[26713]: Disconnecting: Too many authentication failures for root [preauth] +# failJSON: { "time": "2004-11-23T21:50:37", "match": true , "host": "61.0.0.1", "desc": "New logline format as openssh 6.8 to replace prev multiline version" } +Nov 23 21:50:37 myhost sshd[21810]: error: maximum authentication attempts exceeded for root from 61.0.0.1 port 49940 ssh2 [preauth] + # failJSON: { "match": false } Apr 27 13:02:04 host sshd[29116]: User root not allowed because account is locked # failJSON: { "match": false } diff --git a/fail2ban/tests/files/testcase-wrong-char.log b/fail2ban/tests/files/testcase-wrong-char.log new file mode 100644 index 00000000..9736020e --- /dev/null +++ b/fail2ban/tests/files/testcase-wrong-char.log @@ -0,0 +1,4 @@ +Nov 8 00:16:12 main sshd[32547]: Invalid user llinco\361ir from 192.0.2.0 +Nov 8 00:16:12 main sshd[32548]: input_userauth_request: invalid user llinco\361ir +Nov 8 00:16:12 main sshd[32547]: pam_succeed_if(sshd:auth): error retrieving information about user llincoir +Nov 8 00:16:14 main sshd[32547]: Failed password for invalid user llinco\361ir from 192.0.2.0 port 57025 ssh2 diff --git a/fail2ban/tests/filtertestcase.py b/fail2ban/tests/filtertestcase.py index 891b55a6..40879b66 100644 --- a/fail2ban/tests/filtertestcase.py +++ b/fail2ban/tests/filtertestcase.py @@ -90,7 +90,11 @@ def _assert_equal_entries(utest, found, output, count=None): found_time, output_time = \ MyTime.localtime(found[2]),\ MyTime.localtime(output[2]) - utest.assertEqual(found_time, output_time) + try: + utest.assertEqual(found_time, output_time) + except AssertionError as e: + # assert more structured: + utest.assertEqual((float(found[2]), found_time), (float(output[2]), output_time)) if len(output) > 3 and count is None: # match matches # do not check if custom count (e.g. going through them twice) if os.linesep != '\n' or sys.platform.startswith('cygwin'): @@ -216,6 +220,14 @@ class BasicFilter(unittest.TestCase): ("^%Y-%m-%d-%H%M%S.%f %z", "^Year-Month-Day-24hourMinuteSecond.Microseconds Zone offset")) + def testAssertWrongTime(self): + self.assertRaises(AssertionError, + lambda: _assert_equal_entries(self, + ('1.1.1.1', 1, 1421262060.0), + ('1.1.1.1', 1, 1421262059.0), + 1) + ) + class IgnoreIP(LogCaptureTestCase): @@ -260,14 +272,14 @@ class IgnoreIP(LogCaptureTestCase): self.filter.addIgnoreIP('192.168.1.0/25') self.filter.addFailRegex('<HOST>') self.filter.processLineAndAdd('1387203300.222 192.168.1.32') - self.assertTrue(self._is_logged('Ignore 192.168.1.32')) + self.assertLogged('Ignore 192.168.1.32') tearDownMyTime() def testIgnoreAddBannedIP(self): self.filter.addIgnoreIP('192.168.1.0/25') self.filter.addBannedIP('192.168.1.32') - self.assertFalse(self._is_logged('Ignore 192.168.1.32')) - self.assertTrue(self._is_logged('Requested to manually ban an ignored IP 192.168.1.32. User knows best. Proceeding to ban it.')) + self.assertNotLogged('Ignore 192.168.1.32') + self.assertLogged('Requested to manually ban an ignored IP 192.168.1.32. User knows best. Proceeding to ban it.') def testIgnoreCommand(self): self.filter.setIgnoreCommand(sys.executable + ' ' + os.path.join(TEST_FILES_DIR, "ignorecommand.py <ip>")) @@ -278,11 +290,11 @@ class IgnoreIP(LogCaptureTestCase): ip = "93.184.216.34" for ignore_source in ["dns", "ip", "command"]: self.filter.logIgnoreIp(ip, True, ignore_source=ignore_source) - self.assertTrue(self._is_logged("[%s] Ignore %s by %s" % (self.jail.name, ip, ignore_source))) + self.assertLogged("[%s] Ignore %s by %s" % (self.jail.name, ip, ignore_source)) def testIgnoreCauseNOK(self): self.filter.logIgnoreIp("example.com", False, ignore_source="NOT_LOGGED") - self.assertFalse(self._is_logged("[%s] Ignore %s by %s" % (self.jail.name, "example.com", "NOT_LOGGED"))) + self.assertNotLogged("[%s] Ignore %s by %s" % (self.jail.name, "example.com", "NOT_LOGGED")) class IgnoreIPDNS(IgnoreIP): @@ -382,18 +394,17 @@ class LogFileMonitor(LogCaptureTestCase): def testNoLogFile(self): _killfile(self.file, self.name) self.filter.getFailures(self.name) - failure_was_logged = self._is_logged('Unable to open %s' % self.name) - self.assertTrue(failure_was_logged) + self.assertLogged('Unable to open %s' % self.name) def testRemovingFailRegex(self): self.filter.delFailRegex(0) - self.assertFalse(self._is_logged('Cannot remove regular expression. Index 0 is not valid')) + self.assertNotLogged('Cannot remove regular expression. Index 0 is not valid') self.filter.delFailRegex(0) - self.assertTrue(self._is_logged('Cannot remove regular expression. Index 0 is not valid')) + self.assertLogged('Cannot remove regular expression. Index 0 is not valid') def testRemovingIgnoreRegex(self): self.filter.delIgnoreRegex(0) - self.assertTrue(self._is_logged('Cannot remove regular expression. Index 0 is not valid')) + self.assertLogged('Cannot remove regular expression. Index 0 is not valid') def testNewChangeViaIsModified(self): # it is a brand new one -- so first we think it is modified @@ -811,7 +822,7 @@ def get_monitor_failures_journal_testcase(Filter_): # pragma: systemd no cover return MonitorJournalFailures -class GetFailures(unittest.TestCase): +class GetFailures(LogCaptureTestCase): FILENAME_01 = os.path.join(TEST_FILES_DIR, "testcase01.log") FILENAME_02 = os.path.join(TEST_FILES_DIR, "testcase02.log") @@ -826,6 +837,7 @@ class GetFailures(unittest.TestCase): def setUp(self): """Call before every test case.""" + LogCaptureTestCase.setUp(self) setUpMyTime() self.jail = DummyJail() self.filter = FileFilter(self.jail) @@ -837,14 +849,27 @@ class GetFailures(unittest.TestCase): def tearDown(self): """Call after every test case.""" tearDownMyTime() + LogCaptureTestCase.tearDown(self) def testTail(self): + # There must be no containters registered, otherwise [-1] indexing would be wrong + self.assertEqual(self.filter.getLogs(), []) + self.filter.addLogPath(GetFailures.FILENAME_01, tail=True) + self.assertEqual(self.filter.getLogs()[-1].getPos(), 1653) + self.filter.getLogs()[-1].close() + self.assertEqual(self.filter.getLogs()[-1].readline(), "") + self.filter.delLogPath(GetFailures.FILENAME_01) + self.assertEqual(self.filter.getLogs(), []) + + def testNoLogAdded(self): self.filter.addLogPath(GetFailures.FILENAME_01, tail=True) - self.assertEqual(self.filter.getLogPath()[-1].getPos(), 1653) - self.filter.getLogPath()[-1].close() - self.assertEqual(self.filter.getLogPath()[-1].readline(), "") + self.assertTrue(self.filter.containsLogPath(GetFailures.FILENAME_01)) self.filter.delLogPath(GetFailures.FILENAME_01) - self.assertEqual(self.filter.getLogPath(),[]) + self.assertFalse(self.filter.containsLogPath(GetFailures.FILENAME_01)) + # and unknown (safety and cover) + self.assertFalse(self.filter.containsLogPath('unknown.log')) + self.filter.delLogPath('unknown.log') + def testGetFailures01(self, filename=None, failures=None): filename = filename or GetFailures.FILENAME_01 @@ -901,6 +926,41 @@ class GetFailures(unittest.TestCase): except FailManagerEmpty: pass + def testGetFailuresWrongChar(self): + # write wrong utf-8 char: + fname = tempfile.mktemp(prefix='tmp_fail2ban', suffix='crlf') + fout = fopen(fname, 'wb') + try: + # write: + for l in ( + b'2015-01-14 20:00:58 user \"test\xf1ing\" from \"192.0.2.0\"\n', # wrong utf-8 char + b'2015-01-14 20:00:59 user \"\xd1\xe2\xe5\xf2\xe0\" from \"192.0.2.0\"\n', # wrong utf-8 chars + b'2015-01-14 20:01:00 user \"testing\" from \"192.0.2.0\"\n' # correct utf-8 chars + ): + fout.write(l) + fout.close() + # + output = ('192.0.2.0', 3, 1421262060.0) + failregex = "^\s*user \"[^\"]*\" from \"<HOST>\"\s*$" + + # test encoding auto or direct set of encoding: + for enc in (None, 'utf-8', 'ascii'): + if enc is not None: + self.tearDown();self.setUp(); + self.filter.setLogEncoding(enc); + self.assertNotLogged('Error decoding line'); + self.filter.addLogPath(fname) + self.filter.addFailRegex(failregex) + self.filter.getFailures(fname) + _assert_correct_last_attempt(self, self.filter, output) + + self.assertLogged('Error decoding line'); + self.assertLogged('Continuing to process line ignoring invalid characters:', '2015-01-14 20:00:58 user '); + self.assertLogged('Continuing to process line ignoring invalid characters:', '2015-01-14 20:00:59 user '); + + finally: + _killfile(fout, fname) + def testGetFailuresUseDNS(self): # We should still catch failures with usedns = no ;-) output_yes = ('93.184.216.34', 2, 1124013539.0, @@ -1026,8 +1086,8 @@ class DNSUtilsTests(unittest.TestCase): self.assertEqual(res, []) def testIpToName(self): - res = DNSUtils.ipToName('66.249.66.1') - self.assertEqual(res, 'crawl-66-249-66-1.googlebot.com') + res = DNSUtils.ipToName('8.8.4.4') + self.assertEqual(res, 'google-public-dns-b.google.com') # invalid ip (TEST-NET-1 according to RFC 5737) res = DNSUtils.ipToName('192.0.2.0') self.assertEqual(res, None) diff --git a/fail2ban/tests/misctestcase.py b/fail2ban/tests/misctestcase.py index c95efa43..e28ce422 100644 --- a/fail2ban/tests/misctestcase.py +++ b/fail2ban/tests/misctestcase.py @@ -33,6 +33,7 @@ from glob import glob from StringIO import StringIO from ..helpers import formatExceptionInfo, mbasename, TraceBack, FormatterWithTraceBack, getLogger +from ..helpers import splitcommaspace from ..server.datetemplate import DatePatternRegex @@ -55,6 +56,14 @@ class HelpersTest(unittest.TestCase): # might be fragile due to ' vs " self.assertEqual(args, "('Very bad', None)") + def testsplitcommaspace(self): + self.assertEqual(splitcommaspace(None), []) + self.assertEqual(splitcommaspace(''), []) + self.assertEqual(splitcommaspace(' '), []) + self.assertEqual(splitcommaspace('1'), ['1']) + self.assertEqual(splitcommaspace(' 1 2 '), ['1', '2']) + self.assertEqual(splitcommaspace(' 1, 2 , '), ['1', '2']) + class SetupTest(unittest.TestCase): diff --git a/fail2ban/tests/servertestcase.py b/fail2ban/tests/servertestcase.py index 57453269..07e10c7d 100644 --- a/fail2ban/tests/servertestcase.py +++ b/fail2ban/tests/servertestcase.py @@ -113,19 +113,15 @@ class TransmitterBase(unittest.TestCase): self.assertEqual( self.transm.proceed(["get", jail, cmd]), (0, [])) for n, value in enumerate(values): - self.assertEqual( - self.transm.proceed(["set", jail, cmdAdd, value]), - (0, values[:n+1])) - self.assertEqual( - self.transm.proceed(["get", jail, cmd]), - (0, values[:n+1])) + ret = self.transm.proceed(["set", jail, cmdAdd, value]) + self.assertEqual((ret[0], sorted(ret[1])), (0, sorted(values[:n+1]))) + ret = self.transm.proceed(["get", jail, cmd]) + self.assertEqual((ret[0], sorted(ret[1])), (0, sorted(values[:n+1]))) for n, value in enumerate(values): - self.assertEqual( - self.transm.proceed(["set", jail, cmdDel, value]), - (0, values[n+1:])) - self.assertEqual( - self.transm.proceed(["get", jail, cmd]), - (0, values[n+1:])) + ret = self.transm.proceed(["set", jail, cmdDel, value]) + self.assertEqual((ret[0], sorted(ret[1])), (0, sorted(values[n+1:]))) + ret = self.transm.proceed(["get", jail, cmd]) + self.assertEqual((ret[0], sorted(ret[1])), (0, sorted(values[n+1:]))) def jailAddDelRegexTest(self, cmd, inValues, outValues, jail): cmdAdd = "add" + cmd @@ -168,8 +164,9 @@ class Transmitter(TransmitterBase): t0 = time.time() self.assertEqual(self.transm.proceed(["sleep", "1"]), (0, None)) t1 = time.time() - # Approx 1 second delay - self.assertAlmostEqual(t1 - t0, 1, places=1) + # Approx 1 second delay but not faster + dt = t1 - t0 + self.assertTrue(0.99 < dt < 1.1, msg="Sleep was %g sec" % dt) def testDatabase(self): tmp, tmpFilename = tempfile.mkstemp(".db", "fail2ban_") @@ -934,7 +931,7 @@ class LoggingTests(LogCaptureTestCase): badThread = _BadThread() badThread.start() badThread.join() - self.assertTrue(self._is_logged("Unhandled exception")) + self.assertLogged("Unhandled exception") finally: sys.__excepthook__ = prev_exchook self.assertEqual(len(x), 1) diff --git a/fail2ban/tests/utils.py b/fail2ban/tests/utils.py index e7993ead..0c29e638 100644 --- a/fail2ban/tests/utils.py +++ b/fail2ban/tests/utils.py @@ -85,6 +85,7 @@ def gatherTests(regexps=None, no_network=False): from . import misctestcase from . import databasetestcase from . import samplestestcase + from . import fail2banregextestcase if not regexps: # pragma: no cover tests = unittest.TestSuite() @@ -152,6 +153,9 @@ def gatherTests(regexps=None, no_network=False): # Filter Regex tests with sample logs tests.addTest(unittest.makeSuite(samplestestcase.FilterSamplesRegex)) + # bin/fail2ban-regex + tests.addTest(unittest.makeSuite(fail2banregextestcase.Fail2banRegexTest)) + # # Python action testcases # @@ -204,6 +208,18 @@ def gatherTests(regexps=None, no_network=False): return tests +# forwards compatibility of unittest.TestCase for some early python versions +if not hasattr(unittest.TestCase, 'assertIn'): + def __assertIn(self, a, b, msg=None): + if a not in b: # pragma: no cover + self.fail(msg or "%r was not found in %r" % (a, b)) + unittest.TestCase.assertIn = __assertIn + def __assertNotIn(self, a, b, msg=None): + if a in b: # pragma: no cover + self.fail(msg or "%r was found in %r" % (a, b)) + unittest.TestCase.assertNotIn = __assertNotIn + + class LogCaptureTestCase(unittest.TestCase): def setUp(self): @@ -232,6 +248,39 @@ class LogCaptureTestCase(unittest.TestCase): def _is_logged(self, s): return s in self._log.getvalue() + def assertLogged(self, *s): + """Assert that one of the strings was logged + + Preferable to assertTrue(self._is_logged(..))) + since provides message with the actual log. + + Parameters + ---------- + s : string or list/set/tuple of strings + Test should succeed if string (or any of the listed) is present in the log + """ + logged = self._log.getvalue() + for s_ in s: + if s_ in logged: + return + raise AssertionError("None among %r was found in the log: %r" % (s, logged)) + + def assertNotLogged(self, *s): + """Assert that strings were not logged + + Parameters + ---------- + s : string or list/set/tuple of strings + Test should succeed if the string (or at least one of the listed) is not + present in the log + """ + logged = self._log.getvalue() + for s_ in s: + if s_ not in logged: + return + raise AssertionError("All of the %r were found present in the log: %r" % (s, logged)) + + def getLog(self): return self._log.getvalue() diff --git a/fail2ban/version.py b/fail2ban/version.py index 4a0f220d..9e4ef0f8 100644 --- a/fail2ban/version.py +++ b/fail2ban/version.py @@ -21,7 +21,7 @@ # __author__ = "Cyril Jaquier, Yaroslav Halchenko, Steven Hiscocks, Daniel Black" -__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2005-2015 Yaroslav Halchenko, 2013-2014 Steven Hiscocks, Daniel Black" +__copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2005-2016 Yaroslav Halchenko, 2013-2014 Steven Hiscocks, Daniel Black" __license__ = "GPL-v2+" -version = "0.9.3" +version = "0.9.4" diff --git a/files/bash-completion b/files/bash-completion index 57ec15b3..36e0cbba 100644 --- a/files/bash-completion +++ b/files/bash-completion @@ -31,7 +31,7 @@ __fail2ban_jail_action_methods () { _fail2ban () { local cur prev words cword - _init_completion || return + _init_completion || return case $prev in -V|--version|-h|--help) diff --git a/files/fail2ban-logrotate b/files/fail2ban-logrotate index a09870af..13a94537 100644 --- a/files/fail2ban-logrotate +++ b/files/fail2ban-logrotate @@ -4,15 +4,11 @@ # # Debian: # https://github.com/fail2ban/fail2ban/blob/debian/debian/fail2ban.logrotate -# -# Fedora view: -# http://pkgs.fedoraproject.org/cgit/fail2ban.git/tree/fail2ban-logrotate /var/log/fail2ban.log { - rotate 7 missingok - compress + notifempty postrotate - /usr/bin/fail2ban-client flushlogs 1>/dev/null || true + /usr/bin/fail2ban-client flushlogs >/dev/null || true endscript } diff --git a/files/fail2ban.service b/files/fail2ban.service index 6ebbacc0..e79faed1 100644 --- a/files/fail2ban.service +++ b/files/fail2ban.service @@ -2,6 +2,7 @@ Description=Fail2Ban Service Documentation=man:fail2ban(1) After=network.target iptables.service firewalld.service +PartOf=iptables.service firewalld.service [Service] Type=forking diff --git a/files/gen_badbots b/files/gen_badbots index 278058f7..75a6a0d5 100755 --- a/files/gen_badbots +++ b/files/gen_badbots @@ -26,7 +26,7 @@ # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License -# along with this program; if not, write to the +# along with this program; if not, write to the # Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, # MA 02110-1301, USA. # @@ -42,7 +42,7 @@ done \ | grep -h -B4 '<td class="smallcell" nowrap>S </td>'\ | sed -e 's/ //g' \ | awk '/^--/{getline; gsub(" ",""); print $0}' \ -| sed -e 's/\([.\:|()]\)/\\\1/g' \ +| sed -e 's/\([.\:|()+]\)/\\\1/g' \ | uniq \ | tr '\n' '|' \ | sed -e 's/|$//g' diff --git a/files/gentoo-initd b/files/gentoo-initd index b56d4bdb..b7e1067b 100755 --- a/files/gentoo-initd +++ b/files/gentoo-initd @@ -30,27 +30,25 @@ depend() { start() { ebegin "Starting fail2ban" - if [ ! -d /var/run/fail2ban ]; then - mkdir /var/run/fail2ban || return 1 - fi - if [ -e /var/run/fail2ban/fail2ban.sock ]; then - # remove stalled sock file after system crash - # bug 347477 - rm -rf /var/run/fail2ban/fail2ban.sock || return 1 - fi - ${FAIL2BAN} start &> /dev/null + mkdir -p /var/run/fail2ban || return 1 + # remove stalled sock file after system crash + # bug 347477 + rm -f /var/run/fail2ban/fail2ban.sock || return 1 + start-stop-daemon --start --exec ${FAIL2BAN} start \ + --pidfile /var/run/fail2ban/fail2ban.pid eend $? "Failed to start fail2ban" } stop() { ebegin "Stopping fail2ban" - ${FAIL2BAN} stop &> /dev/null + start-stop-daemon --stop --exec ${FAIL2BAN} stop \ + --pidfile /var/run/fail2ban/fail2ban.pid eend $? "Failed to stop fail2ban" } reload() { ebegin "Reloading fail2ban" - ${FAIL2BAN} reload > /dev/null + ${FAIL2BAN} reload eend $? "Failed to reload fail2ban" } diff --git a/files/nagios/README b/files/nagios/README index 2b855d83..324260ff 100644 --- a/files/nagios/README +++ b/files/nagios/README @@ -8,8 +8,8 @@ How to use ---------- Just have to run the following command: $ ./check_fail2ban --help - -If you need to use this script with NRPE you just have to do the + +If you need to use this script with NRPE you just have to do the following steps: 1 allow your user to run the script with the sudo rights. Just add @@ -20,7 +20,7 @@ following steps: command[check_fail2ban]=/usr/bin/sudo /<path-to>/check_fail2ban 3 don't forget to restart your NRPE daemon - + /!\ be careful to let no one able to update the check_fail2ban ;) ------------------------------------------------------------------------------ @@ -37,7 +37,7 @@ HELP: 2.) delete the socket if available rm /var/run/fail2ban/fail2ban.sock -3.) start the Service +3.) start the Service /etc/init.d/fail2ban start 4.) check if fail2ban is working @@ -58,7 +58,7 @@ Options: -V, --version Print version information -D, --display=STRING - To modify the output display + To modify the output display default is "CHECK FAIL2BAN ACTIVITY" -P, --path-fail2ban_client=STRING Specify the path to the tw_cli binary diff --git a/files/nagios/check_fail2ban b/files/nagios/check_fail2ban index fc88e654..bbfa63aa 100755 --- a/files/nagios/check_fail2ban +++ b/files/nagios/check_fail2ban @@ -4,12 +4,12 @@ # -=- <check_fail2ban> -=- # ------------------------------------------------------- # -# Description : This plugin checks if the fail2ban server is running +# Description : This plugin checks if the fail2ban server is running # and how many IPs are currently banned. -# +# # # inspired by the work of Sebastian Mueller - http://www.elchtest.eu -# +# # # Version : 0.1 # ------------------------------------------------------- @@ -17,7 +17,7 @@ # - see the How to use section # # Out : -# - only print on the standard output +# - only print on the standard output # # Features : # - perfdata output @@ -51,8 +51,8 @@ # # Just have to run the following command: # $ ./check_fail2ban --help -# -# If you need to use this script with NRPE you just have to do the +# +# If you need to use this script with NRPE you just have to do the # following steps: # # 1 allow your user to run the script with the sudo rights. Just add @@ -64,7 +64,7 @@ # # 3 don't forget to restart your NRPE daemon # -# +# # /!\ be careful to let no one able to update the check_fail2ban ;) # ------------------------------------------------------------------------------ # @@ -251,7 +251,7 @@ Options: -V, --version Print version information -D, --display=STRING - To modify the output display + To modify the output display default is "CHECK FAIL2BAN ACTIVITY" -P, --path-fail2ban_client=STRING Specify the path to the tw_cli binary @@ -269,7 +269,7 @@ Options: If you want to activate the perfdata output -v, --verbose Show details for command-line debugging (Nagios may truncate the output) - + Send email to $a_mail if you have questions regarding use of this software. To submit patches or suggest improvements, send email to $a_mail @@ -315,7 +315,7 @@ sub obtain_jail_list { if ($return_code) { return -1; } - + my @jail_list; foreach (@command_output) { if ($_=~/^.*Jail list:\t+(.*)/) { @@ -323,7 +323,7 @@ sub obtain_jail_list { @jail_list = split(/,/, $1); } } - + return @jail_list; } diff --git a/files/solaris-svc-fail2ban b/files/solaris-svc-fail2ban index e397474b..96415323 100755 --- a/files/solaris-svc-fail2ban +++ b/files/solaris-svc-fail2ban @@ -2,7 +2,7 @@ # # fail2ban This init.d script is used to start fail2ban. # (C) by Hanno Wagner <wagner@rince.de>, License is GPL - + #set -x . /lib/svc/share/smf_include.sh diff --git a/files/suse-initd b/files/suse-initd index 09c25687..ddd26ec5 100644 --- a/files/suse-initd +++ b/files/suse-initd @@ -60,12 +60,12 @@ case "$1" in if [ -f $FAIL2BAN_SOCKET ] then - echo "$FAIL2BAN_SOCKET not removed .. removing .." + echo "$FAIL2BAN_SOCKET not removed .. removing .." rm $FAIL2BAN_SOCKET fi if [ -f $FAIL2BAN_PID ] then - echo "$FAIL2BAN_PID not removed .. removing .." + echo "$FAIL2BAN_PID not removed .. removing .." rm $FAIL2BAN_PID fi diff --git a/man/fail2ban-client.1 b/man/fail2ban-client.1 index a43123da..ec5db2d2 100644 --- a/man/fail2ban-client.1 +++ b/man/fail2ban-client.1 @@ -1,12 +1,12 @@ -.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.1. -.TH FAIL2BAN-CLIENT "1" "July 2015" "fail2ban-client v0.9.3" "User Commands" +.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.2. +.TH FAIL2BAN-CLIENT "1" "March 2016" "fail2ban-client v0.9.4" "User Commands" .SH NAME fail2ban-client \- configure and control the server .SH SYNOPSIS .B fail2ban-client [\fI\,OPTIONS\/\fR] \fI\,<COMMAND>\/\fR .SH DESCRIPTION -Fail2Ban v0.9.3 reads log file that contains password failure report +Fail2Ban v0.9.4 reads log file that contains password failure report and bans the corresponding IP addresses using firewall rules. .SH OPTIONS .TP @@ -229,7 +229,7 @@ sets the number of <LINES> to buffer for regex search for <JAIL> .TP \fBset <JAIL> addaction <ACT>[ <PYTHONFILE> <JSONKWARGS>]\fR -adds a new action named <NAME> for +adds a new action named <ACT> for <JAIL>. Optionally for a Python based action, a <PYTHONFILE> and <JSONKWARGS> can be specified, diff --git a/man/fail2ban-regex.1 b/man/fail2ban-regex.1 index c1ae40dc..740b461c 100644 --- a/man/fail2ban-regex.1 +++ b/man/fail2ban-regex.1 @@ -1,5 +1,5 @@ -.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.1. -.TH FAIL2BAN-REGEX "1" "July 2015" "fail2ban-regex 0.9.3" "User Commands" +.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.2. +.TH FAIL2BAN-REGEX "1" "March 2016" "fail2ban-regex 0.9.4" "User Commands" .SH NAME fail2ban-regex \- test Fail2ban "failregex" option .SH SYNOPSIS diff --git a/man/fail2ban-server.1 b/man/fail2ban-server.1 index 4260e748..90ec1d80 100644 --- a/man/fail2ban-server.1 +++ b/man/fail2ban-server.1 @@ -1,12 +1,12 @@ -.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.1. -.TH FAIL2BAN-SERVER "1" "July 2015" "fail2ban-server v0.9.3" "User Commands" +.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.2. +.TH FAIL2BAN-SERVER "1" "March 2016" "fail2ban-server v0.9.4" "User Commands" .SH NAME fail2ban-server \- start the server .SH SYNOPSIS .B fail2ban-server [\fI\,OPTIONS\/\fR] .SH DESCRIPTION -Fail2Ban v0.9.3 reads log file that contains password failure report +Fail2Ban v0.9.4 reads log file that contains password failure report and bans the corresponding IP addresses using firewall rules. .PP Only use this command for debugging purpose. Start the server with diff --git a/man/fail2ban-testcases.1 b/man/fail2ban-testcases.1 index 55eedd50..dc0fee32 100644 --- a/man/fail2ban-testcases.1 +++ b/man/fail2ban-testcases.1 @@ -1,5 +1,5 @@ -.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.1. -.TH FAIL2BAN-TESTCASES "1" "July 2015" "fail2ban-testcases 0.9.3" "User Commands" +.\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.2. +.TH FAIL2BAN-TESTCASES "1" "March 2016" "fail2ban-testcases 0.9.4" "User Commands" .SH NAME fail2ban-testcases \- run Fail2Ban unit-tests .SH SYNOPSIS diff --git a/man/jail.conf.5 b/man/jail.conf.5 index 45eea040..865c689e 100644 --- a/man/jail.conf.5 +++ b/man/jail.conf.5 @@ -1,4 +1,4 @@ -.TH JAIL.CONF "10" "October 2013" "Fail2Ban" "Fail2Ban Configuration" +.TH JAIL.CONF "5" "November 2015" "Fail2Ban" "Fail2Ban Configuration" .SH NAME jail.conf \- configuration for the fail2ban server .SH SYNOPSIS @@ -89,16 +89,36 @@ indicates that the specified file is to be parsed before the current file. indicates that the specified file is to be parsed after the current file. .RE -Using Python "string interpolation" mechanisms, other definitions are allowed and can later be used within other definitions as %(name)s. For example. +Using Python "string interpolation" mechanisms, other definitions are allowed and can later be used within other definitions as %(name)s. +Additionally fail2ban has an extended interpolation feature named \fB%(known/parameter)s\fR (means last known option with name \fBparameter\fR). This interpolation makes possible to extend a stock filter or jail regexp in .local file (opposite to simply set failregex/ignoreregex that overwrites it), e.g. .RS +.nf baduseragents = IE|wget +failregex = %(known/failregex)s + useragent=%(baduseragents)s +.fi .RE + +Additionally to interpolation \fB%(known/parameter)s\fR, that does not works for filter/action init parameters, an interpolation tag \fB<known/parameter>\fR can be used (means last known init definition of filters or actions with name \fBparameter\fR). This interpolation makes possible to extend a parameters of stock filter or action directly in jail inside \fIjail.conf/jail.local\fR file without creating a separately filter.d/*.local file, e.g. + .RS -failregex = useragent=%(baduseragents)s +# filter.d/test.conf: +.nf +[Init] +test.method = GET +baduseragents = IE|wget +[Definition] +failregex = ^%(__prefix_line)\\s+"<test.method>"\\s+test\\s+regexp\\s+-\\s+useragent=(?:<baduseragents>) + +# jail.local: +[test] +# use filter "test", overwrite method to "POST" and extend known bad agents with "badagent": +filter = test[test.method=POST, baduseragents="badagent|<known/baduseragents>"] +.fi .RE -Comments: use '#' for comment lines and '; ' (space is important) for inline comments. When using Python2.X '; ' can only be used on the first line due to an Python library bug. +Comments: use '#' for comment lines and '; ' (space is important) for inline comments. When using Python2.X, '; ' can only be used on the first line due to an Python library bug. .SH "FAIL2BAN CONFIGURATION FILE(S) (\fIfail2ban.conf\fB)" @@ -110,34 +130,44 @@ The items that can be set are: verbosity level of log output: CRITICAL, ERROR, WARNING, NOTICE, INFO, DEBUG. Default: ERROR .TP .B logtarget -log target: filename, SYSLOG, STDERR or STDOUT. Default: STDERR . Only a single log target can be specified. +log target: filename, SYSLOG, STDERR or STDOUT. Default: STDERR +.br +Only a single log target can be specified. If you change logtarget from the default value and you are using logrotate -- also adjust or disable rotation in the corresponding configuration file (e.g. /etc/logrotate.d/fail2ban on Debian systems). .TP .B socket -socket filename. Default: /var/run/fail2ban/fail2ban.sock . +socket filename. Default: /var/run/fail2ban/fail2ban.sock +.br This is used for communication with the fail2ban server daemon. Do not remove this file when Fail2ban is running. It will not be possible to communicate with the server afterwards. .TP .B pidfile -PID filename. Default: /var/run/fail2ban/fail2ban.pid. +PID filename. Default: /var/run/fail2ban/fail2ban.pid +.br This is used to store the process ID of the fail2ban server. .TP .B dbfile Database filename. Default: /var/lib/fail2ban/fail2ban.sqlite3 +.br This defines where the persistent data for fail2ban is stored. This persistent data allows bans to be reinstated and continue reading log files from the last read position when fail2ban is restarted. A value of \fINone\fR disables this feature. .TP .B dbpurgeage Database purge age in seconds. Default: 86400 (24hours) +.br This sets the age at which bans should be purged from the database. .SH "JAIL CONFIGURATION FILE(S) (\fIjail.conf\fB)" The following options are applicable to any jail. They appear in a section specifying the jail name or in the \fI[DEFAULT]\fR section which defines default values to be used if not specified in the individual section. .TP .B filter -name of the filter -- filename of the filter in /etc/fail2ban/filter.d/ without the .conf/.local extension. Only one filter can be specified. +name of the filter -- filename of the filter in /etc/fail2ban/filter.d/ without the .conf/.local extension. +.br +Only one filter can be specified. .TP .B logpath -filename(s) of the log files to be monitored, separated by new lines. Globs -- paths containing * and ? or [0-9] -- can be used however only the files that exist at start up matching this glob pattern will be considered. +filename(s) of the log files to be monitored, separated by new lines. +.br +Globs -- paths containing * and ? or [0-9] -- can be used however only the files that exist at start up matching this glob pattern will be considered. Optional space separated option 'tail' can be added to the end of the path to cause the log file to be read from the end, else default 'head' option reads file from the beginning @@ -146,8 +176,18 @@ Ensure syslog or the program that generates the log file isn't configured to com .B logencoding encoding of log files used for decoding. Default value of "auto" uses current system locale. .TP +.B banaction +banning action (default iptables-multiport) typically specified in the \fI[DEFAULT]\fR section for all jails. +.br +This parameter will be used by the standard substitution of \fIaction\fR and can be redefined central in the \fI[DEFAULT]\fR section inside \fIjail.local\fR (to apply it to all jails at once) or separately in each jail, where this substitution will be used. +.TP +.B banaction_allports +the same as \fIbanaction\fR but for some "allports" jails like "pam-generic" or "recidive" (default iptables-allports). +.TP .B action -action(s) from \fI/etc/fail2ban/action.d/\fR without the \fI.conf\fR/\fI.local\fR extension. Arguments can be passed to actions to override the default values from the [Init] section in the action file. Arguments are specified by: +action(s) from \fI/etc/fail2ban/action.d/\fR without the \fI.conf\fR/\fI.local\fR extension. +.br +Arguments can be passed to actions to override the default values from the [Init] section in the action file. Arguments are specified by: .RS .RS @@ -161,7 +201,9 @@ Values can also be quoted (required when value includes a ","). More that one ac list of IPs not to ban. They can include a CIDR mask too. .TP .B ignorecommand -command that is executed to determine if the current candidate IP for banning should not be banned. IP will not be banned if command returns successfully (exit code 0). +command that is executed to determine if the current candidate IP for banning should not be banned. +.br +IP will not be banned if command returns successfully (exit code 0). Like ACTION FILES, tags like <ip> are can be included in the ignorecommand value and will be substituted before execution. Currently only <ip> is supported however more will be added later. .TP .B bantime @@ -174,7 +216,9 @@ time interval (in seconds) before the current time where failures will count tow number of failures that have to occur in the last \fBfindtime\fR seconds to ban then IP. .TP .B backend -backend to be used to detect changes in the logpath. It defaults to "auto" which will try "pyinotify", "gamin", "systemd" before "polling". Any of these can be specified. "pyinotify" is only valid on Linux systems with the "pyinotify" Python libraries. "gamin" requires the "gamin" libraries. +backend to be used to detect changes in the logpath. +.br +It defaults to "auto" which will try "pyinotify", "gamin", "systemd" before "polling". Any of these can be specified. "pyinotify" is only valid on Linux systems with the "pyinotify" Python libraries. "gamin" requires the "gamin" libraries. .TP .B usedns use DNS to resolve HOST names that appear in the logs. By default it is "warn" which will resolve hostnames to IPs however it will also log a warning. If you are using DNS here you could be blocking the wrong IPs due to the asymmetric nature of reverse DNS (that the application used to write the domain name to log) compared to forward DNS that fail2ban uses to resolve this back to an IP (but not necessarily the same one). Ideally you should configure your applications to log a real IP. This can be set to "yes" to prevent warnings in the log or "no" to disable DNS resolution altogether (thus ignoring entries where hostname, not an IP is logged).. @@ -245,7 +289,7 @@ The maximum period of time in seconds that a command can executed, before being .RE Commands specified in the [Definition] section are executed through a system shell so shell redirection and process control is allowed. The commands should -return 0, otherwise error would be logged. Moreover if \fBactioncheck\fR exits with non-0 status, it is taken as indication that firewall status has changed and fail2ban needs to reinitialize itself (i.e. issue \fBactionstop\fR and \fBactionstart\fR commands). +return 0, otherwise error would be logged. Moreover if \fBactioncheck\fR exits with non-0 status, it is taken as indication that firewall status has changed and fail2ban needs to reinitialize itself (i.e. issue \fBactionstop\fR and \fBactionstart\fR commands). Tags are enclosed in <>. All the elements of [Init] are tags that are replaced in all action commands. Tags can be added by the \fBfail2ban-client\fR using the "set <JAIL> action <ACT>" command. \fB<br>\fR is a tag that is always a new line (\\n). @@ -306,7 +350,7 @@ is the regex to identify log entries that should be ignored by Fail2Ban, even if .PP -Similar to actions, filters have an [Init] section which can be overridden in \fIjail.conf/jail.local\fR. The filter [Init] section is limited to the following options: +Similar to actions, filters have an [Init] section which can be overridden in \fIjail.conf/jail.local\fR. Besides the filter-specific settings, the filter [Init] section can be used to set following standard options: .TP \fBmaxlines\fR specifies the maximum number of lines to buffer to match multi-line regexs. For some log formats this will not required to be changed. Other logs may require to increase this value if a particular log file is frequently written to. @@ -321,6 +365,8 @@ Also, special values of \fIEpoch\fR (UNIX Timestamp), \fITAI64N\fR and \fIISO860 \fBjournalmatch\fR specifies the systemd journal match used to filter the journal entries. See \fBjournalctl(1)\fR and \fBsystemd.journal-fields(7)\fR for matches syntax and more details on special journal fields. This option is only valid for the \fIsystemd\fR backend. .PP +Similar to actions [Init] section enables filter-specific settings. All parameters specified in [Init] section can be redefined or extended in \fIjail.conf/jail.local\fR. + Filters can also have a section called [INCLUDES]. This is used to read other configuration files. .TP |