diff options
| author | Yaroslav Halchenko <debian@onerussian.com> | 2017-05-10 21:39:51 -0400 |
|---|---|---|
| committer | Yaroslav Halchenko <debian@onerussian.com> | 2017-05-10 21:39:51 -0400 |
| commit | 96323b1da016cfbfd4cd6b865b6d7f3e6044434a (patch) | |
| tree | 8ebe198493dd0b585ab7b6a297c5f4fee86d97af | |
| parent | 7e0e9cda50b1b513c1dc21a15fd5009355ea956b (diff) | |
| parent | 35280044ffdb6a8166fc6ad5497cfae7032c71a6 (diff) | |
| download | fail2ban-96323b1da016cfbfd4cd6b865b6d7f3e6044434a.tar.gz | |
Merge tag '0.9.7' into debian
ver. 0.9.7 (2017/05/11) - awaiting-victory
-----------
0.9.x line is no longer heavily developed. If you are interested in
new features (e.g. IPv6 support), please consider 0.10 branch and its
releases.
* Fixed a systemd-journal handling in fail2ban-regex (gh-1657)
* filter.d/sshd.conf
- Fixed non-anchored part of failregex (misleading match of colon inside
IPv6 address instead of `: ` in the reason-part by missing space, gh-1658)
(0.10th resp. IPv6 relevant only, amend for gh-1479)
* config/pathes-freebsd.conf
- Fixed filenames for apache and nginx log files (gh-1667)
* filter.d/exim.conf
- optional part `(...)` after host-name before `[IP]` (gh-1751)
- new reason "Unrouteable address" for "rejected RCPT" regex (gh-1762)
- match of complex time like `D=2m42s` in regex "no MAIL in SMTP connection" (gh-1766)
* filter.d/sshd.conf
- new aggressive rules (gh-864):
- Connection reset by peer (multi-line rule during authorization process)
- No supported authentication methods available
- single line and multi-line expression optimized, added optional prefixes
and suffix (logged from several ssh versions), according to gh-1206;
- fixed expression received disconnect auth fail (optional space after port
part, gh-1652)
and suffix (logged from several ssh versions), according to gh-1206;
* filter.d/suhosin.conf
- greedy catch-all before `<HOST>` fixed (potential vulnerability)
* filter.d/cyrus-imap.conf
- accept entries without login-info resp. hostname before IP address (gh-1707)
* Filter tests extended with check of all config-regexp, that contains greedy catch-all
before `<HOST>`, that is hard-anchored at end or precise sub expression after `<HOST>`
* New Actions:
- action.d/netscaler: Block IPs on a Citrix Netscaler ADC (gh-1663)
* New Filters:
- filter.d/domino-smtp: IBM Domino SMTP task (gh-1603)
* Introduced new log-level `MSG` (as INFO-2, equivalent to 18)
* tag '0.9.7': (33 commits)
Preparing for 0.9.7 release
Added newly added files to MANIFEST
update ChangeLog
filter.d/exim.conf: added new reason for "rejected RCPT" regex: Unrouteable address
try to fix travis integration of pypy3: setuptools recently dropped support for Python 3.0 - 3.2, but old pypy3 based on Python 3.2.5
filter.d/exim.conf: cherry-picked from 0.10, match complex time like `D=2m42s` (closes gh-1766)
Update ChangeLog #1757
filter.d/exim.conf: optional part `(...)` after host-name before `[IP]`, normalized over whole config file.
BF: specify explicit time offset not a time zone name to avoid needing tzdata during testing
Update ChangeLog
amend resp. restore of change from 59c35bc44a175a672e084bc30511dfa3436ff052 (gh-129): - logging of "Log rotation detected" with new MSG level - introduces new log-level MSG (as INFO-2, 18)
Update mysqld-auth.conf
Update ChangeLog
filter.d/cyrus-imap.conf: fixed `failregex` - accept entries without login-info resp. hostname before IP address
evil symlink removed: does not supported by some file systems (e. g. development over net share)
sshd-amend: optional space after port part
suhosin.conf: removed greedy match
sshd.conf: fixed expression "received disconnect ... auth fail" - optional space after port part (gh-1652)
change log update after rebase
sshd: additionally aggressive filter rules - no matching cipher resp. no matching key exchange method (gh-1545, gh-1117)
...
41 files changed, 402 insertions, 134 deletions
diff --git a/.travis.yml b/.travis.yml index 9ef607da..40376075 100644 --- a/.travis.yml +++ b/.travis.yml @@ -10,7 +10,9 @@ python: # - 3.2 - 3.3 - 3.4 - - pypy3 + # disabled since setuptools dropped support for Python 3.0 - 3.2 + # - pypy3 + - pypy3.3-5.2-alpha1 before_install: - if [[ $TRAVIS_PYTHON_VERSION == 2* || $TRAVIS_PYTHON_VERSION == 'pypy' ]]; then export F2B_PY_2=true && echo "Set F2B_PY_2"; fi - if [[ $TRAVIS_PYTHON_VERSION == 3* || $TRAVIS_PYTHON_VERSION == 'pypy3' ]]; then export F2B_PY_3=true && echo "Set F2B_PY_3"; fi @@ -6,7 +6,8 @@ Fail2Ban: Changelog =================== -ver. 0.9.6 (2016/12/10) - stretch-is-coming + +ver. 0.9.7 (2017/05/11) - awaiting-victory ----------- 0.9.x line is no longer heavily developed. If you are interested in @@ -14,6 +15,48 @@ new features (e.g. IPv6 support), please consider 0.10 branch and its releases. ### Fixes +* Fixed a systemd-journal handling in fail2ban-regex (gh-1657) +* filter.d/sshd.conf + - Fixed non-anchored part of failregex (misleading match of colon inside + IPv6 address instead of `: ` in the reason-part by missing space, gh-1658) + (0.10th resp. IPv6 relevant only, amend for gh-1479) +* config/pathes-freebsd.conf + - Fixed filenames for apache and nginx log files (gh-1667) +* filter.d/exim.conf + - optional part `(...)` after host-name before `[IP]` (gh-1751) + - new reason "Unrouteable address" for "rejected RCPT" regex (gh-1762) + - match of complex time like `D=2m42s` in regex "no MAIL in SMTP connection" (gh-1766) +* filter.d/sshd.conf + - new aggressive rules (gh-864): + - Connection reset by peer (multi-line rule during authorization process) + - No supported authentication methods available + - single line and multi-line expression optimized, added optional prefixes + and suffix (logged from several ssh versions), according to gh-1206; + - fixed expression received disconnect auth fail (optional space after port + part, gh-1652) + and suffix (logged from several ssh versions), according to gh-1206; +* filter.d/suhosin.conf + - greedy catch-all before `<HOST>` fixed (potential vulnerability) +* filter.d/cyrus-imap.conf + - accept entries without login-info resp. hostname before IP address (gh-1707) +* Filter tests extended with check of all config-regexp, that contains greedy catch-all + before `<HOST>`, that is hard-anchored at end or precise sub expression after `<HOST>` + +### New Features +* New Actions: + - action.d/netscaler: Block IPs on a Citrix Netscaler ADC (gh-1663) + +* New Filters: + - filter.d/domino-smtp: IBM Domino SMTP task (gh-1603) + +### Enhancements +* Introduced new log-level `MSG` (as INFO-2, equivalent to 18) + + +ver. 0.9.6 (2016/12/10) - stretch-is-coming +----------- + +### Fixes * Misleading add resp. enable of (already available) jail in database, that induced a subsequent error: last position of log file will be never retrieved (gh-795) * Fixed a distribution related bug within testReadStockJailConfForceEnabled @@ -37,6 +37,7 @@ config/action.d/mail-whois-common.conf config/action.d/mail-whois.conf config/action.d/mail-whois-lines.conf config/action.d/mynetwatchman.conf +config/action.d/netscaler.conf config/action.d/nftables-allports.conf config/action.d/nftables-common.conf config/action.d/nftables-multiport.conf @@ -83,6 +84,7 @@ config/filter.d/courier-auth.conf config/filter.d/courier-smtp.conf config/filter.d/cyrus-imap.conf config/filter.d/directadmin.conf +config/filter.d/domino-smtp.conf config/filter.d/dovecot.conf config/filter.d/dropbear.conf config/filter.d/drupal-auth.conf @@ -138,6 +140,7 @@ config/filter.d/sogo-auth.conf config/filter.d/solid-pop3d.conf config/filter.d/squid.conf config/filter.d/squirrelmail.conf +config/filter.d/sshd-aggressive.conf config/filter.d/sshd.conf config/filter.d/sshd-ddos.conf config/filter.d/stunnel.conf @@ -278,6 +281,7 @@ fail2ban/tests/files/logs/courier-auth fail2ban/tests/files/logs/courier-smtp fail2ban/tests/files/logs/cyrus-imap fail2ban/tests/files/logs/directadmin +fail2ban/tests/files/logs/domino-smtp fail2ban/tests/files/logs/dovecot fail2ban/tests/files/logs/dropbear fail2ban/tests/files/logs/drupal-auth @@ -330,6 +334,7 @@ fail2ban/tests/files/logs/solid-pop3d fail2ban/tests/files/logs/squid fail2ban/tests/files/logs/squirrelmail fail2ban/tests/files/logs/sshd +fail2ban/tests/files/logs/sshd-aggressive fail2ban/tests/files/logs/sshd-ddos fail2ban/tests/files/logs/stunnel fail2ban/tests/files/logs/suhosin @@ -2,7 +2,7 @@ / _|__ _(_) |_ ) |__ __ _ _ _ | _/ _` | | |/ /| '_ \/ _` | ' \ |_| \__,_|_|_/___|_.__/\__,_|_||_| - v0.9.6 2016/12/10 + v0.9.7 2017/05/11 ## Fail2Ban: ban hosts that cause multiple authentication errors @@ -190,7 +190,7 @@ Post Release Add the following to the top of the ChangeLog:: - ver. 0.9.7 (2016/XX/XXX) - wanna-be-released + ver. 0.9.8 (2016/XX/XXX) - wanna-be-released ----------- ### Fixes diff --git a/config/action.d/firewallcmd-rich-logging.conf b/config/action.d/firewallcmd-rich-logging.conf index 1b88c2d9..c4a8b6f7 100644 --- a/config/action.d/firewallcmd-rich-logging.conf +++ b/config/action.d/firewallcmd-rich-logging.conf @@ -29,7 +29,7 @@ actioncheck = # firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' port port='<port>' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <blocktype>" # service name example: # firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' service name='<service>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <blocktype>" -# Because rich rules can only handle single or a range of ports we must split ports and execute the command for each port. Ports can be single and ranges seperated by a comma or space for an example: http, https, 22-60, 18 smtp +# Because rich rules can only handle single or a range of ports we must split ports and execute the command for each port. Ports can be single and ranges separated by a comma or space for an example: http, https, 22-60, 18 smtp actionban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --add-rich-rule="rule family='ipv4' source address='<ip>' port port='$p' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <blocktype>"; done diff --git a/config/action.d/firewallcmd-rich-rules.conf b/config/action.d/firewallcmd-rich-rules.conf index 4e39df54..5bf10b03 100644 --- a/config/action.d/firewallcmd-rich-rules.conf +++ b/config/action.d/firewallcmd-rich-rules.conf @@ -27,7 +27,7 @@ actioncheck = # firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' port port='<port>' protocol='<protocol>' <blocktype>" # service name example: # firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' service name='<service>' <blocktype>" -# Because rich rules can only handle single or a range of ports we must split ports and execute the command for each port. Ports can be single and ranges seperated by a comma or space for an example: http, https, 22-60, 18 smtp +# Because rich rules can only handle single or a range of ports we must split ports and execute the command for each port. Ports can be single and ranges separated by a comma or space for an example: http, https, 22-60, 18 smtp actionban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --add-rich-rule="rule family='ipv4' source address='<ip>' port port='$p' protocol='<protocol>' <blocktype>"; done diff --git a/config/action.d/netscaler.conf b/config/action.d/netscaler.conf new file mode 100644 index 00000000..87f7e7bf --- /dev/null +++ b/config/action.d/netscaler.conf @@ -0,0 +1,33 @@ +# Fail2ban Citrix Netscaler Action +# by Juliano Jeziorny +# juliano@jeziorny.eu +# +# The script will add offender IPs to a dataset on netscaler, the dataset can then be used to block the IPs at a cs/vserver or global level +# This dataset is then used to block IPs using responder policies on the netscaler. +# +# The script assumes using HTTPS with unsecure certificate to access the netscaler, +# if you have a valid certificate installed remove the -k from the curl lines, or if you want http change it accordingly (and remove the -k) +# +# This action depends on curl +# +# You need to populate the 3 options inside Init +# +# ns_host: IP or hostname of netslcaer appliance +# ns_auth: username:password, suggest base64 encoded for a little added security (echo -n "username:password" | base64) +# ns_dataset: Name of the netscaler dataset holding the IPs to be blocked. +# +# For further details on how to use it please check http://blog.ckzone.eu/2017/01/fail2ban-action-for-citrix-netscaler.html + +[Init] +ns_host = +ns_auth = +ns_dataset = + +[Definition] +actionstart = curl -kH 'Authorization: Basic <ns_auth>' https://<ns_host>/nitro/v1/config + +actioncheck = + +actionban = curl -k -H 'Authorization: Basic <ns_auth>' -X PUT -d '{"policydataset_value_binding":{"name":"<ns_dataset>","value":"<ip>"}}' https://<ns_host>/nitro/v1/config/ + +actionunban = curl -H 'Authorization: Basic <ns_auth>' -X DELETE -k "https://<ns_host>/nitro/v1/config/policydataset_value_binding/<ns_dataset>?args=value:<ip>" diff --git a/config/filter.d/assp.conf b/config/filter.d/assp.conf index 278e25cb..ddf18f33 100644 --- a/config/filter.d/assp.conf +++ b/config/filter.d/assp.conf @@ -8,7 +8,7 @@ # [Definition] -# Note: First three failregex matches below are for ASSP V1 with the remaining being designed for V2. Deleting the V1 regex is recommended but I left it in for compatibilty reasons. +# Note: First three failregex matches below are for ASSP V1 with the remaining being designed for V2. Deleting the V1 regex is recommended but I left it in for compatibility reasons. __assp_actions = (?:dropping|refusing) diff --git a/config/filter.d/cyrus-imap.conf b/config/filter.d/cyrus-imap.conf index 73764d9d..31dfda60 100644 --- a/config/filter.d/cyrus-imap.conf +++ b/config/filter.d/cyrus-imap.conf @@ -13,7 +13,7 @@ before = common.conf _daemon = (?:cyrus/)?(?:imap(d|s)?|pop3(d|s)?) -failregex = ^%(__prefix_line)sbadlogin: \S+ ?\[<HOST>\] \S+ .*?\[?SASL\(-13\): (authentication failure|user not found): .*\]?$ +failregex = ^%(__prefix_line)sbadlogin: [^\[]*\[<HOST>\] \S+ .*?\[?SASL\(-13\): (authentication failure|user not found): .*\]?$ ignoreregex = diff --git a/config/filter.d/domino-smtp.conf b/config/filter.d/domino-smtp.conf new file mode 100644 index 00000000..cdc17736 --- /dev/null +++ b/config/filter.d/domino-smtp.conf @@ -0,0 +1,47 @@ +# Fail2Ban configuration file for IBM Domino SMTP Server TASK to detect failed login attempts +# +# Author: Christian Brandlehner +# +# $Revision: 003 $ +# +# Configuration: +# Set the following Domino Server parameters in notes.ini: +# console_log_enabled=1 +# log_sessions=2 +# You also have to use a date and time format supported by fail2ban. Recommended notes.ini configuration is: +# DateOrder=DMY +# DateSeparator=- +# ClockType=24_Hour +# TimeSeparator=: +# +# Depending on your locale you might have to tweak the date and time format so fail2ban can read the log + +#[INCLUDES] +# Read common prefixes. If any customizations available -- read them from +# common.local +#before = common.conf + +[Definition] +# Option: failregex +# Notes.: regex to match the password failure messages in the logfile. The +# host must be matched by a group named "host". The tag "<HOST>" can +# be used for standard IP/hostname matching and is only an alias for +# (?:::f{4,6}:)?(?P<host>\S+) +# Values: TEXT +# +# Sample log entries (used different time formats and an extra sample with process info in front of date) +# 01-23-2009 19:54:51 SMTP Server: Authentication failed for user postmaster ; connecting host 1.2.3.4 +# [28325:00010-3735542592] 22-06-2014 09:56:12 smtp: postmaster [1.2.3.4] authentication failure using internet password +# 08-09-2014 06:14:27 smtp: postmaster [1.2.3.4] authentication failure using internet password +# 08-09-2014 06:14:27 SMTP Server: Authentication failed for user postmaster ; connecting host 1.2.3.4 + +__prefix = (?:\[[^\]]+\])?\s+ +failregex = ^%(__prefix)sSMTP Server: Authentication failed for user .*? \; connecting host <HOST>$ + ^%(__prefix)ssmtp: (?:[^\[]+ )*\[<HOST>\] authentication failure using internet password\s*$ +# Option: ignoreregex +# Notes.: regex to ignore. If this regex matches, the line is ignored. +# Values: TEXT +# + +ignoreregex = + diff --git a/config/filter.d/exim-common.conf b/config/filter.d/exim-common.conf index 0e1b74fa..b3b25750 100644 --- a/config/filter.d/exim-common.conf +++ b/config/filter.d/exim-common.conf @@ -9,7 +9,9 @@ after = exim-common.local [Definition] -host_info = (?:H=([\w.-]+ )?(?:\(\S+\) )?)?\[<HOST>\](?::\d+)? (?:I=\[\S+\](:\d+)? )?(?:U=\S+ )?(?:P=e?smtp )? +host_info_pre = (?:H=([\w.-]+ )?(?:\(\S+\) )?)? +host_info_suf = (?::\d+)?(?: I=\[\S+\](:\d+)?)?(?: U=\S+)?(?: P=e?smtp)?(?: F=(?:<>|[^@]+@\S+))?\s +host_info = %(host_info_pre)s\[<HOST>\]%(host_info_suf)s pid = (?: \[\d+\])? # DEV Notes: diff --git a/config/filter.d/exim.conf b/config/filter.d/exim.conf index a1d699c0..7848fe00 100644 --- a/config/filter.d/exim.conf +++ b/config/filter.d/exim.conf @@ -14,13 +14,13 @@ before = exim-common.conf [Definition] failregex = ^%(pid)s %(host_info)ssender verify fail for <\S+>: (?:Unknown user|Unrouteable address|all relevant MX records point to non-existent hosts)\s*$ - ^%(pid)s \w+ authenticator failed for (\S+ )?\(\S+\) \[<HOST>\](?::\d+)?(?: I=\[\S+\](:\d+)?)?: 535 Incorrect authentication data( \(set_id=.*\)|: \d+ Time\(s\))?\s*$ - ^%(pid)s %(host_info)sF=(?:<>|[^@]+@\S+) rejected RCPT [^@]+@\S+: (?:relay not permitted|Sender verify failed|Unknown user)\s*$ + ^%(pid)s \w+ authenticator failed for (?:[^\[\( ]* )?(?:\(\S*\) )?\[<HOST>\](?::\d+)?(?: I=\[\S+\](:\d+)?)?: 535 Incorrect authentication data( \(set_id=.*\)|: \d+ Time\(s\))?\s*$ + ^%(pid)s %(host_info)srejected RCPT [^@]+@\S+: (?:relay not permitted|Sender verify failed|Unknown user|Unrouteable address)\s*$ ^%(pid)s SMTP protocol synchronization error \([^)]*\): rejected (?:connection from|"\S+") %(host_info)s(?:next )?input=".*"\s*$ ^%(pid)s SMTP call from \S+ %(host_info)sdropped: too many nonmail commands \(last was "\S+"\)\s*$ ^%(pid)s SMTP protocol error in "AUTH \S*(?: \S*)?" %(host_info)sAUTH command used when not advertised\s*$ - ^%(pid)s no MAIL in SMTP connection from (?:\S* )?(?:\(\S*\) )?%(host_info)sD=\d+s(?: C=\S*)?\s*$ - ^%(pid)s \S+ SMTP connection from (?:\S* )?(?:\(\S*\) )?%(host_info)sclosed by DROP in ACL\s*$ + ^%(pid)s no MAIL in SMTP connection from (?:[^\[\( ]* )?(?:\(\S*\) )?%(host_info)sD=\d\S+s(?: C=\S*)?\s*$ + ^%(pid)s (?:[\w\-]+ )?SMTP connection from (?:[^\[\( ]* )?(?:\(\S*\) )?%(host_info)sclosed by DROP in ACL\s*$ ignoreregex = diff --git a/config/filter.d/mysqld-auth.conf b/config/filter.d/mysqld-auth.conf index 3ad70cb7..31bd2056 100644 --- a/config/filter.d/mysqld-auth.conf +++ b/config/filter.d/mysqld-auth.conf @@ -1,4 +1,4 @@ -# Fail2Ban filter for unsuccesfull MySQL authentication attempts +# Fail2Ban filter for unsuccesful MySQL authentication attempts # # # To log wrong MySQL access attempts add to /etc/my.cnf in [mysqld]: diff --git a/config/filter.d/sendmail-reject.conf b/config/filter.d/sendmail-reject.conf index 20d3648e..2f8fd882 100644 --- a/config/filter.d/sendmail-reject.conf +++ b/config/filter.d/sendmail-reject.conf @@ -25,7 +25,7 @@ failregex = ^%(__prefix_line)s\w{14}: ruleset=check_rcpt, arg1=(?P<email><\S+@\S ^%(__prefix_line)sruleset=check_relay, arg1=(?P<dom>\S+), arg2=<HOST>, relay=((?P=dom) )?\[(\d+\.){3}\d+\]( \(may be forged\))?, reject=421 4\.3\.2 (Connection rate limit exceeded\.|Too many open connections\.)$ ^%(__prefix_line)s\w{14}: rejecting commands from (\S* )?\[<HOST>\] due to pre-greeting traffic after \d+ seconds$ ^%(__prefix_line)s\w{14}: (\S+ )?\[<HOST>\]: ((?i)expn|vrfy) \S+ \[rejected\]$ - ^(?P<__prefix>%(__prefix_line)s\w+: )<[^@]+@[^>]+>\.\.\. No such user here<SKIPLINES>(?P=__prefix)from=<[^@]+@[^>]+>, size=\d+, class=\d+, nrcpts=\d+, bodytype=\w+, proto=E?SMTP, daemon=MTA, relay=\S+ \[<HOST>\]$ + ^(?P<__prefix>%(__prefix_line)s\w+: )<[^@]+@[^>]+>\.\.\. No such user here$<SKIPLINES>^(?P=__prefix)from=<[^@]+@[^>]+>, size=\d+, class=\d+, nrcpts=\d+, bodytype=\w+, proto=E?SMTP, daemon=MTA, relay=\S+ \[<HOST>\]$ ignoreregex = diff --git a/config/filter.d/sshd-aggressive.conf b/config/filter.d/sshd-aggressive.conf new file mode 100644 index 00000000..98175cbe --- /dev/null +++ b/config/filter.d/sshd-aggressive.conf @@ -0,0 +1,11 @@ +# Fail2Ban aggressive ssh filter for at attempted exploit +# +# Includes failregex of both sshd and sshd-ddos filters +# +[INCLUDES] + +before = sshd.conf + +[Definition] + +mode = %(aggressive)s diff --git a/config/filter.d/sshd-ddos.conf b/config/filter.d/sshd-ddos.conf index 4f71c7f3..69b42069 100644 --- a/config/filter.d/sshd-ddos.conf +++ b/config/filter.d/sshd-ddos.conf @@ -10,20 +10,8 @@ [INCLUDES] -# Read common prefixes. If any customizations available -- read them from -# common.local -before = common.conf +before = sshd.conf [Definition] -_daemon = sshd - -failregex = ^%(__prefix_line)sDid not receive identification string from <HOST>\s*$ - -ignoreregex = - -[Init] - -journalmatch = _SYSTEMD_UNIT=sshd.service + _COMM=sshd - -# Author: Yaroslav Halchenko +mode = %(ddos)s diff --git a/config/filter.d/sshd.conf b/config/filter.d/sshd.conf index 35cd8754..45566b36 100644 --- a/config/filter.d/sshd.conf +++ b/config/filter.d/sshd.conf @@ -14,27 +14,54 @@ # common.local before = common.conf -[Definition] +[DEFAULT] _daemon = sshd -failregex = ^%(__prefix_line)s(?:error: PAM: )?[aA]uthentication (?:failure|error|failed) for .* from <HOST>( via \S+)?\s*$ - ^%(__prefix_line)s(?:error: PAM: )?User not known to the underlying authentication module for .* from <HOST>\s*$ - ^%(__prefix_line)sFailed \S+ for (?P<cond_inv>invalid user )?(?P<user>(?P<cond_user>\S+)|(?(cond_inv)(?:(?! from ).)*?|[^:]+)) from <HOST>(?: port \d+)?(?: ssh\d*)?(?(cond_user):|(?:(?:(?! from ).)*)$) - ^%(__prefix_line)sROOT LOGIN REFUSED.* FROM <HOST>\s*$ - ^%(__prefix_line)s[iI](?:llegal|nvalid) user .*? from <HOST>(?: port \d+)?\s*$ - ^%(__prefix_line)sUser .+ from <HOST> not allowed because not listed in AllowUsers\s*$ - ^%(__prefix_line)sUser .+ from <HOST> not allowed because listed in DenyUsers\s*$ - ^%(__prefix_line)sUser .+ from <HOST> not allowed because not in any group\s*$ - ^%(__prefix_line)srefused connect from \S+ \(<HOST>\)\s*$ - ^%(__prefix_line)s(?:error: )?Received disconnect from <HOST>: 3: .*: Auth fail(?: \[preauth\])?$ - ^%(__prefix_line)sUser .+ from <HOST> not allowed because a group is listed in DenyGroups\s*$ - ^%(__prefix_line)sUser .+ from <HOST> not allowed because none of user's groups are listed in AllowGroups\s*$ - ^(?P<__prefix>%(__prefix_line)s)User .+ not allowed because account is locked<SKIPLINES>(?P=__prefix)(?:error: )?Received disconnect from <HOST>: 11: .+ \[preauth\]$ - ^(?P<__prefix>%(__prefix_line)s)Disconnecting: Too many authentication failures for .+? \[preauth\]<SKIPLINES>(?P=__prefix)(?:error: )?Connection closed by <HOST> \[preauth\]$ - ^(?P<__prefix>%(__prefix_line)s)Connection from <HOST> port \d+(?: on \S+ port \d+)?<SKIPLINES>(?P=__prefix)Disconnecting: Too many authentication failures for .+? \[preauth\]$ - ^%(__prefix_line)s(error: )?maximum authentication attempts exceeded for .* from <HOST>(?: port \d*)?(?: ssh\d*)? \[preauth\]$ - ^%(__prefix_line)spam_unix\(sshd:auth\):\s+authentication failure;\s*logname=\S*\s*uid=\d*\s*euid=\d*\s*tty=\S*\s*ruser=\S*\s*rhost=<HOST>\s.*$ +# optional prefix (logged from several ssh versions) like "error: ", "error: PAM: " or "fatal: " +__pref = (?:(?:error|fatal): (?:PAM: )?)? +# optional suffix (logged from several ssh versions) like " [preauth]" +__suff = (?: \[preauth\])?\s* +__on_port_opt = (?: port \d+)?(?: on \S+(?: port \d+)?)? + +# single line prefix: +__prefix_line_sl = %(__prefix_line)s%(__pref)s +# multi line prefixes (for first and second lines): +__prefix_line_ml1 = (?P<__prefix>%(__prefix_line)s)%(__pref)s +__prefix_line_ml2 = %(__suff)s$<SKIPLINES>^(?P=__prefix)%(__pref)s + +mode = %(normal)s + +normal = ^%(__prefix_line_sl)s[aA]uthentication (?:failure|error|failed) for .* from <HOST>( via \S+)?\s*%(__suff)s$ + ^%(__prefix_line_sl)sUser not known to the underlying authentication module for .* from <HOST>\s*%(__suff)s$ + ^%(__prefix_line_sl)sFailed \S+ for (?P<cond_inv>invalid user )?(?P<user>(?P<cond_user>\S+)|(?(cond_inv)(?:(?! from ).)*?|[^:]+)) from <HOST>%(__on_port_opt)s(?: ssh\d*)?(?(cond_user): |(?:(?:(?! from ).)*)$) + ^%(__prefix_line_sl)sROOT LOGIN REFUSED.* FROM <HOST>\s*%(__suff)s$ + ^%(__prefix_line_sl)s[iI](?:llegal|nvalid) user .*? from <HOST>%(__on_port_opt)s\s*$ + ^%(__prefix_line_sl)sUser .+ from <HOST> not allowed because not listed in AllowUsers\s*%(__suff)s$ + ^%(__prefix_line_sl)sUser .+ from <HOST> not allowed because listed in DenyUsers\s*%(__suff)s$ + ^%(__prefix_line_sl)sUser .+ from <HOST> not allowed because not in any group\s*%(__suff)s$ + ^%(__prefix_line_sl)srefused connect from \S+ \(<HOST>\)\s*%(__suff)s$ + ^%(__prefix_line_sl)sReceived disconnect from <HOST>%(__on_port_opt)s:\s*3: .*: Auth fail%(__suff)s$ + ^%(__prefix_line_sl)sUser .+ from <HOST> not allowed because a group is listed in DenyGroups\s*%(__suff)s$ + ^%(__prefix_line_sl)sUser .+ from <HOST> not allowed because none of user's groups are listed in AllowGroups\s*%(__suff)s$ + ^%(__prefix_line_sl)spam_unix\(sshd:auth\):\s+authentication failure;\s*logname=\S*\s*uid=\d*\s*euid=\d*\s*tty=\S*\s*ruser=\S*\s*rhost=<HOST>\s.*%(__suff)s$ + ^%(__prefix_line_sl)s(error: )?maximum authentication attempts exceeded for .* from <HOST>%(__on_port_opt)s(?: ssh\d*)? \[preauth\]$ + ^%(__prefix_line_ml1)sUser .+ not allowed because account is locked%(__prefix_line_ml2)sReceived disconnect from <HOST>: 11: .+%(__suff)s$ + ^%(__prefix_line_ml1)sDisconnecting: Too many authentication failures for .+?%(__prefix_line_ml2)sConnection closed by <HOST>%(__suff)s$ + ^%(__prefix_line_ml1)sConnection from <HOST>%(__on_port_opt)s%(__prefix_line_ml2)sDisconnecting: Too many authentication failures for .+%(__suff)s$ + +ddos = ^%(__prefix_line_sl)sDid not receive identification string from <HOST>%(__suff)s$ + ^%(__prefix_line_sl)sReceived disconnect from <HOST>%(__on_port_opt)s:\s*14: No supported authentication methods available%(__suff)s$ + ^%(__prefix_line_sl)sUnable to negotiate with <HOST>%(__on_port_opt)s: no matching (?:cipher|key exchange method) found. + ^%(__prefix_line_ml1)sConnection from <HOST>%(__on_port_opt)s%(__prefix_line_ml2)sUnable to negotiate a (?:cipher|key exchange method)%(__suff)s$ + ^%(__prefix_line_ml1)sSSH: Server;Ltype: (?:Authname|Version|Kex);Remote: <HOST>-\d+;[A-Z]\w+:.*%(__prefix_line_ml2)sRead from socket failed: Connection reset by peer%(__suff)s$ + +aggressive = %(normal)s + %(ddos)s + +[Definition] + +failregex = %(mode)s ignoreregex = diff --git a/config/filter.d/suhosin.conf b/config/filter.d/suhosin.conf index f125eadc..46fbe381 100644 --- a/config/filter.d/suhosin.conf +++ b/config/filter.d/suhosin.conf @@ -17,7 +17,7 @@ _daemon = (?:lighttpd|suhosin) _lighttpd_prefix = (?:\(mod_fastcgi\.c\.\d+\) FastCGI-stderr:\s) -failregex = ^%(__prefix_line)s%(_lighttpd_prefix)s?ALERT - .* \(attacker '<HOST>', file '.*'(?:, line \d+)?\)$ +failregex = ^%(__prefix_line)s%(_lighttpd_prefix)s?ALERT - .*? \(attacker '<HOST>', file '[^']*'(?:, line \d+)?\)$ ignoreregex = diff --git a/config/jail.conf b/config/jail.conf index d80e3d0a..9296b6af 100644 --- a/config/jail.conf +++ b/config/jail.conf @@ -223,6 +223,8 @@ action = %(action_)s [sshd] +# To use more aggressive sshd filter (inclusive sshd-ddos failregex): +#filter = sshd-aggressive port = ssh logpath = %(sshd_log)s backend = %(sshd_backend)s @@ -853,3 +855,8 @@ logpath = /var/log/haproxy.log port = ldap,ldaps filter = slapd logpath = /var/log/slapd.log + +[domino-smtp] +port = smtp,ssmtp +filter = domino-smtp +logpath = /home/domino01/data/IBM_TECHNICAL_SUPPORT/console.log diff --git a/config/paths-freebsd.conf b/config/paths-freebsd.conf index fd788ab4..91b23636 100644 --- a/config/paths-freebsd.conf +++ b/config/paths-freebsd.conf @@ -34,13 +34,13 @@ auditd_log = /dev/null # http://svnweb.freebsd.org/ports/head/www/apache24/files/patch-config.layout # http://svnweb.freebsd.org/ports/head/www/apache22/files/patch-config.layout -apache_error_log = /usr/local/www/logs/*error[_.]log +apache_error_log = /var/log/httpd-error.log -apache_access_log = /usr/local/www/logs/*access[_.]log +apache_access_log = /var/log/httpd-access.log # http://svnweb.freebsd.org/ports/head/www/nginx/Makefile?view=markup -nginx_error_log = /var/log/nginx-error.log +nginx_error_log = /var/log/nginx/error.log -nginx_access_log = /var/log/nginx-access.log +nginx_access_log = /var/log/nginx/access.log diff --git a/fail2ban/__init__.py b/fail2ban/__init__.py index cd92dbab..7c752e24 100644 --- a/fail2ban/__init__.py +++ b/fail2ban/__init__.py @@ -34,7 +34,9 @@ Below derived from: https://mail.python.org/pipermail/tutor/2007-August/056243.html """ +logging.MSG = logging.INFO - 2 logging.NOTICE = logging.INFO + 5 +logging.addLevelName(logging.MSG, 'MSG') logging.addLevelName(logging.NOTICE, 'NOTICE') diff --git a/fail2ban/client/fail2banregex.py b/fail2ban/client/fail2banregex.py index 71f50955..13fa35d9 100755 --- a/fail2ban/client/fail2banregex.py +++ b/fail2ban/client/fail2banregex.py @@ -43,12 +43,12 @@ from optparse import OptionParser, Option from ConfigParser import NoOptionError, NoSectionError, MissingSectionHeaderError try: - from systemd import journal from ..server.filtersystemd import FilterSystemd except ImportError: - journal = None + FilterSystemd = None from ..version import version +from .jailreader import JailReader from .filterreader import FilterReader from ..server.filter import Filter, FileContainer from ..server.failregex import RegexException @@ -82,7 +82,7 @@ def pprint_list(l, header=None): s = '' output( s + "| " + "\n| ".join(l) + '\n`-' ) -def journal_lines_gen(myjournal): +def journal_lines_gen(flt, myjournal): # pragma: no cover while True: try: entry = myjournal.get_next() @@ -90,7 +90,7 @@ def journal_lines_gen(myjournal): continue if not entry: break - yield FilterSystemd.formatJournalEntry(entry) + yield flt.formatJournalEntry(entry) def get_opt_parser(): # use module docstring for help output @@ -513,25 +513,22 @@ class Fail2banRegex(object): except IOError as e: output( e ) return False - elif cmd_log == "systemd-journal": # pragma: no cover - if not journal: + elif cmd_log.startswith("systemd-journal"): # pragma: no cover + if not FilterSystemd: output( "Error: systemd library not found. Exiting..." ) return False - myjournal = journal.Reader(converters={'__CURSOR': lambda x: x}) + output( "Use systemd journal" ) + output( "Use encoding : %s" % self.encoding ) + backend, beArgs = JailReader.extractOptions(cmd_log) + flt = FilterSystemd(None, **beArgs) + flt.setLogEncoding(self.encoding) + myjournal = flt.getJournalReader() journalmatch = self._journalmatch self.setDatePattern(None) if journalmatch: - try: - for element in journalmatch: - if element == "+": - myjournal.add_disjunction() - else: - myjournal.add_match(element) - except ValueError: - output( "Error: Invalid journalmatch: %s" % shortstr(" ".join(journalmatch)) ) - return False + flt.addJournalMatch(journalmatch) output( "Use journal match : %s" % " ".join(journalmatch) ) - test_lines = journal_lines_gen(myjournal) + test_lines = journal_lines_gen(flt, myjournal) else: output( "Use single line : %s" % shortstr(cmd_log) ) test_lines = [ cmd_log ] diff --git a/fail2ban/server/filter.py b/fail2ban/server/filter.py index 459a47d0..066ee68f 100644 --- a/fail2ban/server/filter.py +++ b/fail2ban/server/filter.py @@ -820,7 +820,7 @@ class FileContainer: ## sys.stdout.flush() # Compare hash and inode if self.__hash != myHash or self.__ino != stats.st_ino: - logSys.info("Log rotation detected for %s" % self.__filename) + logSys.log(logging.MSG, "Log rotation detected for %s" % self.__filename) self.__hash = myHash self.__ino = stats.st_ino self.__pos = 0 diff --git a/fail2ban/server/filtersystemd.py b/fail2ban/server/filtersystemd.py index 3023155c..908112a7 100644 --- a/fail2ban/server/filtersystemd.py +++ b/fail2ban/server/filtersystemd.py @@ -175,6 +175,14 @@ class FilterSystemd(JournalFilter): # pragma: systemd no cover return v ## + # Get journal reader + # + # @return journal reader + + def getJournalReader(self): + return self.__journal + + ## # Format journal log entry into syslog style # # @param entry systemd journal entry dict diff --git a/fail2ban/tests/clientreadertestcase.py b/fail2ban/tests/clientreadertestcase.py index 420d4b29..3f0f8481 100644 --- a/fail2ban/tests/clientreadertestcase.py +++ b/fail2ban/tests/clientreadertestcase.py @@ -597,7 +597,7 @@ class JailsReaderTest(LogCaptureTestCase): # grab all filter names filters = set(os.path.splitext(os.path.split(a)[1])[0] for a in glob.glob(os.path.join('config', 'filter.d', '*.conf')) - if not a.endswith('common.conf')) + if not (a.endswith('common.conf') or a.endswith('-aggressive.conf'))) # get filters of all jails (filter names without options inside filter[...]) filters_jail = set( JailReader.extractOptions(jail.options['filter'])[0] for jail in jails.jails diff --git a/fail2ban/tests/config/filter.d/common.conf b/fail2ban/tests/config/filter.d/common.conf deleted file mode 120000 index 83e92474..00000000 --- a/fail2ban/tests/config/filter.d/common.conf +++ /dev/null @@ -1 +0,0 @@ -../../../../config/filter.d/common.conf
\ No newline at end of file diff --git a/fail2ban/tests/config/filter.d/zzz-generic-example.conf b/fail2ban/tests/config/filter.d/zzz-generic-example.conf index e2ae91b0..2234b7b7 100644 --- a/fail2ban/tests/config/filter.d/zzz-generic-example.conf +++ b/fail2ban/tests/config/filter.d/zzz-generic-example.conf @@ -8,7 +8,7 @@ # Read common prefixes. If any customizations available -- read them from # common.local. common.conf is a symlink to the original common.conf and # should be copied (dereferenced) during installation -before = common.conf +before = ../../../../config/filter.d/common.conf [Definition] diff --git a/fail2ban/tests/files/logs/cyrus-imap b/fail2ban/tests/files/logs/cyrus-imap index f1edff06..c0ec54cb 100644 --- a/fail2ban/tests/files/logs/cyrus-imap +++ b/fail2ban/tests/files/logs/cyrus-imap @@ -16,3 +16,6 @@ Dec 30 16:03:27 somehost imapd[2517]: badlogin: local-somehost[1.2.3.4] OTP [SAS Jul 17 22:55:56 derry cyrus/imaps[7568]: badlogin: serafinat.xxxxxx [1.2.3.4] plain [SASL(-13): user not found: user: pressy@derry property: cmusaslsecretPLAIN not found in sasldb] # failJSON: { "time": "2005-07-18T16:46:42", "match": true , "host": "1.2.3.4" } Jul 18 16:46:42 derry cyrus/imaps[27449]: badlogin: serafinat.xxxxxx [1.2.3.4] PLAIN [SASL(-13): user not found: Password verification failed] + +# failJSON: { "time": "2005-03-08T05:25:21", "match": true , "host": "192.0.2.4", "desc": "entry without loginname/hostname before IP" } +Mar 8 05:25:21 host imap[22130]: badlogin: [192.0.2.4] plain [SASL(-13): authentication failure: Password verification failed]
\ No newline at end of file diff --git a/fail2ban/tests/files/logs/domino-smtp b/fail2ban/tests/files/logs/domino-smtp new file mode 100644 index 00000000..4987e7ea --- /dev/null +++ b/fail2ban/tests/files/logs/domino-smtp @@ -0,0 +1,8 @@ +# failJSON: { "time": "2005-07-03T23:07:20", "match": true , "host": "1.2.3.4" } +03-07-2005 23:07:20 SMTP Server: Authentication failed for user postmaster ; connecting host 1.2.3.4 +# failJSON: { "time": "2014-06-22T09:56:12", "match": true , "host": "1.2.3.4" } +[28325:00010-3735542592] 22-06-2014 09:56:12 smtp: postmaster [1.2.3.4] authentication failure using internet password +# failJSON: { "time": "2014-09-08T06:14:27", "match": true , "host": "1.2.3.4" } +08-09-2014 06:14:27 smtp: postmaster [1.2.3.4] authentication failure using internet password +# failJSON: { "time": "2016-11-07T22:21:20", "match": true , "host": "1.2.3.4" } +2016-11-07 22:21:20 smtp: postmaster [1.2.3.4] authentication failure using internet password diff --git a/fail2ban/tests/files/logs/exim b/fail2ban/tests/files/logs/exim index 9053bf8d..326cdf28 100644 --- a/fail2ban/tests/files/logs/exim +++ b/fail2ban/tests/files/logs/exim @@ -63,7 +63,14 @@ # failJSON: { "time": "2016-04-01T11:08:39", "match": true , "host": "192.0.2.1" } 2016-04-01 11:08:39 [18643] no MAIL in SMTP connection from host.example.com (SERVER) [192.0.2.1]:1418 I=[172.89.0.6]:25 D=34s C=EHLO,AUTH +# failJSON: { "time": "2016-04-01T11:08:40", "match": true , "host": "192.0.2.2" } +2016-04-01 11:08:40 [18643] no MAIL in SMTP connection from host.example.com (SERVER) [192.0.2.2]:1418 I=[172.89.0.6]:25 D=2m42s C=QUIT # failJSON: { "time": "2016-04-01T11:09:21", "match": true , "host": "192.0.2.1" } 2016-04-01 11:09:21 [18648] SMTP protocol error in "AUTH LOGIN" H=host.example.com (SERVER) [192.0.2.1]:4692 I=[172.89.0.6]:25 AUTH command used when not advertised # failJSON: { "time": "2016-03-27T16:48:48", "match": true , "host": "192.0.2.1" } 2016-03-27 16:48:48 [21478] 1akDqs-0005aQ-9b SMTP connection from host.example.com (SERVER) [192.0.2.1]:47714 I=[172.89.0.6]:25 closed by DROP in ACL + +# failJSON: { "time": "2017-04-23T22:45:59", "match": true , "host": "192.0.2.2", "desc": "optional part (...)" } +2017-04-23 22:45:59 fixed_login authenticator failed for bad.host.example.com [192.0.2.2]:54412 I=[172.89.0.6]:587: 535 Incorrect authentication data (set_id=user@example.com) +# failJSON: { "time": "2017-05-01T07:42:42", "match": true , "host": "192.0.2.3", "desc": "rejected RCPT - Unrouteable address" } +2017-05-01 07:42:42 H=some.rev.dns.if.found (the.connector.reports.this.name) [192.0.2.3] F=<some.name@some.domain> rejected RCPT <some.invalid.name@a.domain>: Unrouteable address diff --git a/fail2ban/tests/files/logs/sshd b/fail2ban/tests/files/logs/sshd index 0800f86b..5fbdbe81 100644 --- a/fail2ban/tests/files/logs/sshd +++ b/fail2ban/tests/files/logs/sshd @@ -139,6 +139,8 @@ Nov 23 21:50:37 sshd[7148]: Connection closed by 61.0.0.1 [preauth] # failJSON: { "time": "2005-07-13T18:44:28", "match": true , "host": "89.24.13.192", "desc": "from gh-289" } Jul 13 18:44:28 mdop sshd[4931]: Received disconnect from 89.24.13.192: 3: com.jcraft.jsch.JSchException: Auth fail +# failJSON: { "time": "2005-01-02T01:18:41", "match": true , "host": "10.0.0.1", "desc": "space after port is optional (gh-1652)" } +Jan 2 01:18:41 host sshd[11808]: error: Received disconnect from 10.0.0.1 port 7736:3: com.jcraft.jsch.JSchException: Auth fail [preauth] # failJSON: { "time": "2004-10-01T17:27:44", "match": true , "host": "94.249.236.6", "desc": "newer format per commit 36919d9f" } Oct 1 17:27:44 localhost sshd[24077]: error: Received disconnect from 94.249.236.6: 3: com.jcraft.jsch.JSchException: Auth fail [preauth] diff --git a/fail2ban/tests/files/logs/sshd-aggressive b/fail2ban/tests/files/logs/sshd-aggressive new file mode 100644 index 00000000..5b4d3a12 --- /dev/null +++ b/fail2ban/tests/files/logs/sshd-aggressive @@ -0,0 +1,3 @@ +# sshd-aggressive includes sshd and sshd-ddos failregex's: +# addFILE: "sshd" +# addFILE: "sshd-ddos"
\ No newline at end of file diff --git a/fail2ban/tests/files/logs/sshd-ddos b/fail2ban/tests/files/logs/sshd-ddos index 452abbde..f0a787a1 100644 --- a/fail2ban/tests/files/logs/sshd-ddos +++ b/fail2ban/tests/files/logs/sshd-ddos @@ -1,3 +1,41 @@ # http://forums.powervps.com/showthread.php?t=1667 # failJSON: { "time": "2005-06-07T01:10:56", "match": true , "host": "69.61.56.114" } Jun 7 01:10:56 host sshd[5937]: Did not receive identification string from 69.61.56.114 + +# gh-864(1): +# failJSON: { "match": false } +Nov 24 23:46:39 host sshd[32686]: SSH: Server;Ltype: Version;Remote: 127.0.0.1-1780;Protocol: 2.0;Client: libssh2_1.4.3 +# failJSON: { "time": "2004-11-24T23:46:43", "match": true , "host": "127.0.0.1", "desc": "Multiline for connection reset by peer (1)" } +Nov 24 23:46:43 host sshd[32686]: fatal: Read from socket failed: Connection reset by peer [preauth] + +# gh-864(2): +# failJSON: { "match": false } +Nov 24 23:46:40 host sshd[32686]: SSH: Server;Ltype: Kex;Remote: 127.0.0.1-1780;Enc: aes128-ctr;MAC: hmac-sha1;Comp: none [preauth] +# failJSON: { "time": "2004-11-24T23:46:43", "match": true , "host": "127.0.0.1", "desc": "Multiline for connection reset by peer (2)" } +Nov 24 23:46:43 host sshd[32686]: fatal: Read from socket failed: Connection reset by peer [preauth] + +# gh-864(3): +# failJSON: { "match": false } +Nov 24 23:46:41 host sshd[32686]: SSH: Server;Ltype: Authname;Remote: 127.0.0.1-1780;Name: root [preauth] +# failJSON: { "time": "2004-11-24T23:46:43", "match": true , "host": "127.0.0.1", "desc": "Multiline for connection reset by peer (3)" } +Nov 24 23:46:43 host sshd[32686]: fatal: Read from socket failed: Connection reset by peer [preauth] + +# several other cases from gh-864: +# failJSON: { "time": "2004-11-25T01:34:12", "match": true , "host": "127.0.0.1", "desc": "No supported authentication methods" } +Nov 25 01:34:12 srv sshd[123]: Received disconnect from 127.0.0.1: 14: No supported authentication methods available [preauth] +# failJSON: { "time": "2004-11-25T01:35:13", "match": true , "host": "127.0.0.1", "desc": "No supported authentication methods" } +Nov 25 01:35:13 srv sshd[123]: error: Received disconnect from 127.0.0.1: 14: No supported authentication methods available [preauth] +# failJSON: { "time": "2004-11-25T01:35:14", "match": true , "host": "192.168.2.92", "desc": "Optional space after port" } +Nov 25 01:35:14 srv sshd[3625]: error: Received disconnect from 192.168.2.92 port 1684:14: No supported authentication methods available [preauth] + +# gh-1545: +# failJSON: { "time": "2004-11-26T13:03:29", "match": true , "host": "192.0.2.1", "desc": "No matching cipher" } +Nov 26 13:03:29 srv sshd[45]: Unable to negotiate with 192.0.2.1 port 55419: no matching cipher found. Their offer: aes256-cbc,rijndael-cbc@lysator.liu.se,aes192-cbc,aes128-cbc,arcfour128,arcfour,3des-cbc,none [preauth] + +# gh-1117: +# failJSON: { "time": "2004-11-26T13:03:30", "match": true , "host": "192.0.2.2", "desc": "No matching key exchange method" } +Nov 26 13:03:30 srv sshd[45]: fatal: Unable to negotiate with 192.0.2.2 port 55419: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1 +# failJSON: { "match": false } +Nov 26 15:03:30 host sshd[22440]: Connection from 192.0.2.3 port 39678 on 192.168.1.9 port 22 +# failJSON: { "time": "2004-11-26T15:03:31", "match": true , "host": "192.0.2.3", "desc": "Multiline - no matching key exchange method" } +Nov 26 15:03:31 host sshd[22440]: fatal: Unable to negotiate a key exchange method [preauth]
\ No newline at end of file diff --git a/fail2ban/tests/samplestestcase.py b/fail2ban/tests/samplestestcase.py index 327410bc..0f368314 100644 --- a/fail2ban/tests/samplestestcase.py +++ b/fail2ban/tests/samplestestcase.py @@ -31,6 +31,7 @@ import re import sys import time import unittest +from ..server.failregex import Regex from ..server.filter import Filter from ..client.filterreader import FilterReader from .utils import setUpMyTime, tearDownMyTime, CONFIG_DIR @@ -38,6 +39,10 @@ from .utils import setUpMyTime, tearDownMyTime, CONFIG_DIR TEST_CONFIG_DIR = os.path.join(os.path.dirname(__file__), "config") TEST_FILES_DIR = os.path.join(os.path.dirname(__file__), "files") +# regexp to test greedy catch-all should be not-greedy: +RE_HOST = Regex('<HOST>').getRegex() +RE_WRONG_GREED = re.compile(r'\.[+\*](?!\?).*' + re.escape(RE_HOST) + r'.*(?:\.[+\*].*|[^\$])$') + class FilterSamplesRegex(unittest.TestCase): @@ -60,6 +65,19 @@ class FilterSamplesRegex(unittest.TestCase): >= 10, "Expected more FilterSampleRegexs tests") + def testReWrongGreedyCatchAll(self): + """Tests regexp RE_WRONG_GREED is intact (positive/negative)""" + self.assertTrue( + RE_WRONG_GREED.search('greedy .* test' + RE_HOST + ' test not hard-anchored')) + self.assertTrue( + RE_WRONG_GREED.search('greedy .+ test' + RE_HOST + ' test vary .* anchored$')) + self.assertFalse( + RE_WRONG_GREED.search('greedy .* test' + RE_HOST + ' test no catch-all, hard-anchored$')) + self.assertFalse( + RE_WRONG_GREED.search('non-greedy .*? test' + RE_HOST + ' test not hard-anchored')) + self.assertFalse( + RE_WRONG_GREED.search('non-greedy .+? test' + RE_HOST + ' test vary catch-all .* anchored$')) + def testSampleRegexsFactory(name, basedir): def testFilter(self): @@ -85,60 +103,75 @@ def testSampleRegexsFactory(name, basedir): os.path.isfile(os.path.join(TEST_FILES_DIR, "logs", name)), "No sample log file available for '%s' filter" % name) - logFile = fileinput.FileInput( - os.path.join(TEST_FILES_DIR, "logs", name)) - regexsUsed = set() - for line in logFile: - jsonREMatch = re.match("^# ?failJSON:(.+)$", line) - if jsonREMatch: - try: - faildata = json.loads(jsonREMatch.group(1)) - except ValueError as e: - raise ValueError("%s: %s:%i" % - (e, logFile.filename(), logFile.filelineno())) - line = next(logFile) - elif line.startswith("#") or not line.strip(): - continue - else: - faildata = {} - - ret = self.filter.processLine( - line, returnRawHost=True, checkAllRegex=True)[1] - if not ret: - # Check line is flagged as none match - self.assertFalse(faildata.get('match', True), - "Line not matched when should have: %s:%i %r" % - (logFile.filename(), logFile.filelineno(), line)) - elif ret: - # Check line is flagged to match - self.assertTrue(faildata.get('match', False), - "Line matched when shouldn't have: %s:%i %r" % - (logFile.filename(), logFile.filelineno(), line)) - self.assertEqual(len(ret), 1, "Multiple regexs matched %r - %s:%i" % - (map(lambda x: x[0], ret),logFile.filename(), logFile.filelineno())) - - # Verify timestamp and host as expected - failregex, host, fail2banTime, lines = ret[0] - self.assertEqual(host, faildata.get("host", None)) - - t = faildata.get("time", None) - try: - jsonTimeLocal = datetime.datetime.strptime(t, "%Y-%m-%dT%H:%M:%S") - except ValueError: - jsonTimeLocal = datetime.datetime.strptime(t, "%Y-%m-%dT%H:%M:%S.%f") - - jsonTime = time.mktime(jsonTimeLocal.timetuple()) - - jsonTime += jsonTimeLocal.microsecond / 1000000 - - self.assertEqual(fail2banTime, jsonTime, - "UTC Time mismatch fail2ban %s (%s) != failJson %s (%s) (diff %.3f seconds) on: %s:%i %r:" % - (fail2banTime, time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime(fail2banTime)), - jsonTime, time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime(jsonTime)), - fail2banTime - jsonTime, logFile.filename(), logFile.filelineno(), line ) ) - - regexsUsed.add(failregex) + filenames = [name] + i = 0 + while i < len(filenames): + filename = filenames[i]; i += 1; + logFile = fileinput.FileInput(os.path.join(TEST_FILES_DIR, "logs", + filename)) + + # test regexp contains greedy catch-all before <HOST>, that is + # not hard-anchored at end or has not precise sub expression after <HOST>: + for fr in self.filter.getFailRegex(): + if RE_WRONG_GREED.search(fr): #pragma: no cover + raise AssertionError("Following regexp of \"%s\" contains greedy catch-all before <HOST>, " + "that is not hard-anchored at end or has not precise sub expression after <HOST>:\n%s" % + (name, str(fr).replace(RE_HOST, '<HOST>'))) + + for line in logFile: + jsonREMatch = re.match("^# ?(failJSON|addFILE):(.+)$", line) + if jsonREMatch: + try: + faildata = json.loads(jsonREMatch.group(2)) + if jsonREMatch.group(1) == 'addFILE': + filenames.append(faildata) + continue + except ValueError as e: + raise ValueError("%s: %s:%i" % + (e, logFile.filename(), logFile.filelineno())) + line = next(logFile) + elif line.startswith("#") or not line.strip(): + continue + else: + faildata = {} + + ret = self.filter.processLine( + line, returnRawHost=True, checkAllRegex=True)[1] + if not ret: + # Check line is flagged as none match + self.assertFalse(faildata.get('match', True), + "Line not matched when should have: %s:%i %r" % + (logFile.filename(), logFile.filelineno(), line)) + elif ret: + # Check line is flagged to match + self.assertTrue(faildata.get('match', False), + "Line matched when shouldn't have: %s:%i %r" % + (logFile.filename(), logFile.filelineno(), line)) + self.assertEqual(len(ret), 1, "Multiple regexs matched %r - %s:%i" % + (map(lambda x: x[0], ret),logFile.filename(), logFile.filelineno())) + + # Verify timestamp and host as expected + failregex, host, fail2banTime, lines = ret[0] + self.assertEqual(host, faildata.get("host", None)) + + t = faildata.get("time", None) + try: + jsonTimeLocal = datetime.datetime.strptime(t, "%Y-%m-%dT%H:%M:%S") + except ValueError: + jsonTimeLocal = datetime.datetime.strptime(t, "%Y-%m-%dT%H:%M:%S.%f") + + jsonTime = time.mktime(jsonTimeLocal.timetuple()) + + jsonTime += jsonTimeLocal.microsecond / 1000000 + + self.assertEqual(fail2banTime, jsonTime, + "UTC Time mismatch fail2ban %s (%s) != failJson %s (%s) (diff %.3f seconds) on: %s:%i %r:" % + (fail2banTime, time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime(fail2banTime)), + jsonTime, time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime(jsonTime)), + fail2banTime - jsonTime, logFile.filename(), logFile.filelineno(), line ) ) + + regexsUsed.add(failregex) for failRegexIndex, failRegex in enumerate(self.filter.getFailRegex()): self.assertTrue( diff --git a/fail2ban/tests/utils.py b/fail2ban/tests/utils.py index 9155dc37..54861050 100644 --- a/fail2ban/tests/utils.py +++ b/fail2ban/tests/utils.py @@ -95,7 +95,10 @@ def setUpMyTime(): # Set the time to a fixed, known value # Sun Aug 14 12:00:00 CEST 2005 # yoh: we need to adjust TZ to match the one used by Cyril so all the timestamps match - os.environ['TZ'] = 'Europe/Zurich' + # This offset corresponds to Europe/Zurich timezone. Specifying it + # explicitly allows to avoid requiring tzdata package to be installed during + # testing. See https://bugs.debian.org/855920 for more information + os.environ['TZ'] = 'CET-01CEST-02,M3.5.0,M10.5.0' time.tzset() MyTime.setTime(1124013600) diff --git a/fail2ban/version.py b/fail2ban/version.py index 194918d7..e8344517 100644 --- a/fail2ban/version.py +++ b/fail2ban/version.py @@ -24,4 +24,4 @@ __author__ = "Cyril Jaquier, Yaroslav Halchenko, Steven Hiscocks, Daniel Black" __copyright__ = "Copyright (c) 2004 Cyril Jaquier, 2005-2016 Yaroslav Halchenko, 2013-2014 Steven Hiscocks, Daniel Black" __license__ = "GPL-v2+" -version = "0.9.6" +version = "0.9.7" diff --git a/man/fail2ban-client.1 b/man/fail2ban-client.1 index ff3e5f77..34a9d894 100644 --- a/man/fail2ban-client.1 +++ b/man/fail2ban-client.1 @@ -1,12 +1,12 @@ .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.4. -.TH FAIL2BAN-CLIENT "1" "December 2016" "fail2ban-client v0.9.6" "User Commands" +.TH FAIL2BAN-CLIENT "1" "May 2017" "fail2ban-client v0.9.7" "User Commands" .SH NAME fail2ban-client \- configure and control the server .SH SYNOPSIS .B fail2ban-client [\fI\,OPTIONS\/\fR] \fI\,<COMMAND>\/\fR .SH DESCRIPTION -Fail2Ban v0.9.6 reads log file that contains password failure report +Fail2Ban v0.9.7 reads log file that contains password failure report and bans the corresponding IP addresses using firewall rules. .SH OPTIONS .TP diff --git a/man/fail2ban-regex.1 b/man/fail2ban-regex.1 index 1ed2c327..d117ee38 100644 --- a/man/fail2ban-regex.1 +++ b/man/fail2ban-regex.1 @@ -1,5 +1,5 @@ .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.4. -.TH FAIL2BAN-REGEX "1" "December 2016" "fail2ban-regex 0.9.6" "User Commands" +.TH FAIL2BAN-REGEX "1" "May 2017" "fail2ban-regex 0.9.7" "User Commands" .SH NAME fail2ban-regex \- test Fail2ban "failregex" option .SH SYNOPSIS diff --git a/man/fail2ban-server.1 b/man/fail2ban-server.1 index 5278302c..b71f6b52 100644 --- a/man/fail2ban-server.1 +++ b/man/fail2ban-server.1 @@ -1,12 +1,12 @@ .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.4. -.TH FAIL2BAN-SERVER "1" "December 2016" "fail2ban-server v0.9.6" "User Commands" +.TH FAIL2BAN-SERVER "1" "May 2017" "fail2ban-server v0.9.7" "User Commands" .SH NAME fail2ban-server \- start the server .SH SYNOPSIS .B fail2ban-server [\fI\,OPTIONS\/\fR] .SH DESCRIPTION -Fail2Ban v0.9.6 reads log file that contains password failure report +Fail2Ban v0.9.7 reads log file that contains password failure report and bans the corresponding IP addresses using firewall rules. .PP Only use this command for debugging purpose. Start the server with diff --git a/man/fail2ban-testcases.1 b/man/fail2ban-testcases.1 index 658555ac..3d250f5c 100644 --- a/man/fail2ban-testcases.1 +++ b/man/fail2ban-testcases.1 @@ -1,5 +1,5 @@ .\" DO NOT MODIFY THIS FILE! It was generated by help2man 1.47.4. -.TH FAIL2BAN-TESTCASES "1" "December 2016" "fail2ban-testcases 0.9.6" "User Commands" +.TH FAIL2BAN-TESTCASES "1" "May 2017" "fail2ban-testcases 0.9.7" "User Commands" .SH NAME fail2ban-testcases \- run Fail2Ban unit-tests .SH SYNOPSIS |