Configuring Fail2Ban on Zimbra
by Barry De Graaff on August 24, 2022 in Product News, Community, Open Source, Security & Privacy
This article is a how-to guide on installing Fail2Ban to block attacking hosts using a null route or blackhole routes. This can help mitigate brute force attacks on Zimbra. Especially brute force attacks on SMTP are very common.
Prerequisite:
Fail2ban has been tested in combination with netfilter-persistent and iptables. If you use ufw or firewalld you may see errors when trying to ban/unban such as ERROR Failed to execute ban jail … action ‘route’. This article has been validated using a set-up installed using https://github.com/Zimbra/zinstaller which you can use to test fail2ban before applying to your production environment.
It is required the OIP configuration must be done before configuring
Fail2Ban service.
For a Single-Server Setup:
If you are running nginx on the same node as the mailstore, you will
need to add both 127.0.0.1 and the real IP address of that node:
sudo -u zimbra –
zmprov mcf +zimbraMailTrustedIP 127.0.0.1 +zimbraMailTrustedIP {IP of Server}
zmcontrol restart
For a Multi-Server Setup:
sudo -u zimbra –
zmprov mcf +zimbraHttpThrottleSafeIPs {IP of Mailbox-1}
zmprov mcf +zimbraHttpThrottleSafeIPs {IP of Mailbox-2}
zmprov mcf +zimbraMailTrustedIP {IP of Proxy-1}
zmprov mcf +zimbraMailTrustedIP {IP of Proxy-2}
zmcontrol restart
Installation and Configuration of Fail2Ban
1) Install Fail2Ban Package
On RHEL/CentOS 7/8:
yum install epel-release -y
yum install fail2ban -y
On Ubuntu 18/20:
apt-get clean all ; apt-get update
apt-get install fail2ban -y
2) Create a file /etc/fail2ban/jail.local and it will
override the default conf file /etc/fail2ban/jail.conf.
Add the local IP address of the Zimbra server in ignoreip =. You
can also add other IP addresses to ignore from Fail2Ban checking.
On a multi-server setup, add all server’s IP in ignoreip list.
nano /etc/fail2ban/jail.local
[DEFAULT]
"ignoreip" can be a list of IP addresses, CIDR masks or DNS hosts.
Fail2ban will not ban a host which matches an address in this list.
Several addresses can be defined using space (and/or comma) separator.
ignoreip = 127.0.0.1/8 ::1 10.137.26.29/32
ignoreip = 127.0.0.1/8 IP-ADDRESS-OF-ZIMBRA-SERVER/32
banaction = route
3) Create a jail file for Zimbra services.
nano /etc/fail2ban/jail.d/zimbra.local
[zimbra-smtp]
enabled = true
filter = zimbra-smtp
port = 25,465,587
logpath = /var/log/zimbra.log
maxretry = 3
findtime = 86400
bantime = 86400
action = route
[zimbra-web]
enabled = true
filter = zimbra-web
port = 80,443,7071,9071
logpath = /opt/zimbra/log/mailbox.log
maxretry = 5
findtime = 86400
bantime = 86400
action = route
Update: This article uses a regular expression that should work on most Zimbra deployments. To avoid double banning/unbanning which may lead to unpredictable results and errors this article combines the WebUI and Admin WebUI into a single jail called zimbra-web. This does mean that for most deployments a failed login will be counted double. So maxretry = 5 actually means you can try 3 times before being banned.
PROPERTY DESCRIPTION
ignoreip
This parameter identifies IP address that should be ignored by the banning system. By default, this is just set to ignore traffic coming from the machine itself, which is a pretty good setting to have.
banaction
This sets the action that will be used when the threshold is reached. There is actually the name of a file located in ’`/etc/fail2ban/action.d/’’ which calls the configured action using the .conf file. Here we configured route which calls route.conf to handle the routing table manipulation to ban an IP address.
findtime
This parameter sets the window that fail2ban will pay attention to when looking for repeated failed authentication attempts. The default is set to 600 seconds (10 minutes again), which means that the software will count the number of failed attempts in the last 10 minutes.
bantime
This parameter sets the length of a ban, in seconds.
maxretry
This sets the number of failed attempts that will be tolerated within the findtime window before a ban is instituted.
4) [Optional]
If you want to apply Fail2Ban for SSH then create jail file
sshd.local.
(No need to create filter rules for SSH, Fail2ban by default shipped
with filter rules for SSH)
On Ubuntu systems, SSH jail is by default enabled within the jail file
“/etc/fail2ban/jail.d/defaults-debian.conf”.
nano /etc/fail2ban/jail.d/sshd.local
[sshd]
enabled = true
port = 22
maxretry = 3
findtime = 600
bantime = 3600
5) Create filters for Zimbra services.
nano /etc/fail2ban/filter.d/zimbra-web.conf
[Definition]
failregex = .ip=;.authentication failed for .*$
ignoreregex =
nano /etc/fail2ban/filter.d/zimbra-smtp.conf
[Definition]
failregex = postfix\/submission\/smtpd[\d+]: warning: .[]: SASL \w+ authentication failed: authentication failure$ postfix\/smtps\/smtpd[\d+]: warning: .[]: SASL \w+ authentication failed: authentication failure$
ignoreregex =
6) Restart the Fail2ban service and enable it to start after system
reboot.
systemctl restart fail2ban
systemctl status fail2ban
systemctl enable fail2ban
7) Check the status of the Fail2Ban jails.
fail2ban-client status
The result should be similar to this:
[root@centos8 ~]# fail2ban-client status
Status
|- Number of jail: 3- Jail list: sshd, zimbra-smtp, zimbra-web [root@centos8 ~]# [root@centos8 ~]# fail2ban-client status sshd Status for the jail: sshd |- Filter | |- Currently failed: 0 | |- Total failed: 14 |– Journal matches: _SYSTEMD_UNIT=sshd.service + _COMM=sshd- Actions |- Currently banned: 1 |- Total banned: 2 – Banned IP list: 10.137.26.29
8) Check banned IP in routing table.
ip r
route -n
The result should be similar to this:
[root@centos8 ~]# ip r
default via 10.0.10.1 dev ens3
10.0.10.0/24 dev ens3 proto kernel scope link src 10.0.10.67
unreachable 10.137.26.29
[root@centos8 ~]#
[root@centos8 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.10.1 0.0.0.0 UG 0 0 0 ens3
10.0.10.0 0.0.0.0 255.255.255.0 U 0 0 0 ens3
10.137.26.29 – 255.255.255.255 !H 0 – 0 –
[root@centos8 ~]#
9) Ban and unban an IP manually.
Ban an IP address.
fail2ban-client set “Jail-Name” banip “IP-Address”
Example:
fail2ban-client set sshd banip 10.137.26.29
Unban an IP address.
fail2ban-client set “Jail-Name” unbanip “Banned IP-Address”
Example:
[root@centos8 ~]# fail2ban-client set sshd unbanip 10.137.26.29
Unban everyone.
Can be useful when something goes wrong with creating new RegEx filter:
fail2ban-client unban –all
Debugging of Fail2Ban:
The loglevel and target are configured in
/etc/fail2ban/fail2ban.conf you can also obtain the log level
and log target by running:
fail2ban-client get loglevel
fail2ban-client get logtarget
To watch the log for debugging purpose you can run:
tail -f $(fail2ban-client get logtarget | grep “`” | awk ‘{ print $2; }’)
Fail2ban works by parsing log files using regular expressions, you can test the regular expression by using fail2ban-regex like this:
fail2ban-regex /opt/zimbra/log/mailbox.log /etc/fail2ban/filter.d/zimbra-web.conf
Multi server and centralized syslog
Fail2ban is designed to work on the local server. So it does it’s ban actions on the same server where it reads the logs. This can be a problem if you run Zimbra in a multi server scenario, where you can read the logs on the mailbox server, but want to apply the ban on the proxy server.
In addition you may want to use a centralized logging server and if you decide to ban a bad actor, deny access to all servers in your environment.
To do this you would need to create a custom fail2ban action. And set up SSH public key authentication so the server where you run fail2ban can connect to the server where the ban action needs to be applied. This article is not meant to cover all possible scenarios, but to get you started here is a basic example:
Create a new action by copying the default route action:
cp /etc/fail2ban/action.d/route.conf /etc/fail2ban/action.d/remote-route.conf
Next replace the local ip route command with an SSH command to run remotely, from this:
[Definition]
actionban = ip route add
actionunban = ip route del
actioncheck =
actionstart =
actionstop =
[Init]
Option: blocktype
Note: Type can be blackhole, unreachable and prohibit. Unreachable and prohibit correspond to the ICMP reject messages.
Values: STRING
blocktype = unreachable
To this:
[Definition]
actionban = ssh root@remote-server -C ip route add
actionunban = ssh root@remote-server -C ip route del
actioncheck =
actionstart =
actionstop =
[Init]
blocktype = unreachable
Then configure fail2ban to use the new action, in /etc/fail2ban/jail.local and /etc/fail2ban/jail.d/zimbra.local change
banaction = route
…
action = route
to:
banaction = remote-route
…
action = remote-route
Please note that this is meant to be a simple example to get you started, it is probably best NOT to use the root account on the remote server. But for testing this is the easiest. Once you have an idea of how it works, you will probably want to wrap the remote banaction into a script and use sudo on an account with limited access.
Which would lead to something like:
[Definition]
actionban = /usr/local/sbin/my-banaction-script
actionunban = /usr/local/sbin/my-unbanaction-script
…
Then in /usr/local/sbin/my-banaction-script you could run the banaction to any number over servers over SSH, something like:
!/bin/bash
ssh banuser@remote-proxy1 -C sudo ip route del $1 $2 &
ssh banuser@remote-proxy2 -C sudo ip route del $1 $2 &
…etc