Simple SMTP Server

1. Intro

Web applications usually need to send email notifications. For example Moodle needs to notify students and teachers about various events. Without being able to send notifications, Moodle and many other web applications loose half of their usefulness. On many web applications you cannot even finish the registration process and cannot login, unless you verify your email address (the application sends you an email with a link that you have to click).

To send emails, an application needs to contact by SMTP a local or remote mail server. Installing a mail server locally is not so easy. If not done properly, mails that are sent will end up being classified as spam and will not reach the recipients.

We will install and setup a simple mail server. Its aim is not to be a full-fledged system, but just to support web applications (like Moodle) with sending notifications.

For security resons, it allows only some trusted hosts to send emails. This prevents any spammers from abusing it. It also does not have any local accounts and does not accept any emails, making it simpler (no need for antispam and antivirus software, no need for POP and IMAP, etc.)

It is just a simple and minimal SMTP server, which can be used by our web applications to send notification emails to their users.

note

For a very basic introduction to the email system, see: How Email Works

2. DNS configuration

To build a mail server for the domain user1.fs.al, we need to set up some DNS records:

cd /var/ds/nsd/
ls zones/
nano zones/user1.fs.al.db

Append these lines:

;;; SMTP
smtp.user1.fs.al.    IN    A           188.245.242.143
user1.fs.al.         IN    MX    10    smtp.user1.fs.al.
user1.fs.al.         IN    MX    20    smtp.user1.fs.al.
user1.fs.al.         IN    TXT         "v=spf1 mx -all"

zones/user1.fs.al.db

The file zones/user1.fs.al.db now should look like this:

$TTL    24h
$ORIGIN user1.fs.al.
@  1D  IN  SOA  ns1  admin (
    2025031001 ; serial
    3H         ; refresh
    15m        ; retry
    1w         ; expire
    2h         ; minimum
)

$INCLUDE /host/config/secondary.ns

ns1  IN  A  188.245.242.143

@    IN  A  188.245.242.143
*    IN  A  188.245.242.143

;;; SMTP
smtp.user1.fs.al.    IN    A           188.245.242.143
user1.fs.al.         IN    MX    10    smtp.user1.fs.al.
user1.fs.al.         IN    MX    20    smtp.user1.fs.al.
user1.fs.al.         IN    TXT         "v=spf1 mx -all"

suggerimento

Don't forget to also update the serial number!!

Then, notify the secondary nameservers to update the list of records for our domain:

ds @nsd restart

Some Explanations

• The A record is showing the IP address for the subdomain smtp.user1.fs.al.

• A record of type MX (Mail eXchange) allows a mail client to find out where is the mail server (MTA -- Mail Transport Agent) for an address like username@user1.fs.al.

  The client will try to contact first the mail server with the number 10 (the lowest number), and in case it fails, it will try the other mail server.

  note

  In our configuration both mail servers are the same. This is because we will configure our mail server to disconnect a new/unknown mail client when it tries to connect for the first, and will accept its connection subsequently. This is an attempt to avoid spambots.

• The record of type TXT shows to a mail receiver what are the legitimate mail servers that are allowed to tranfer emails for the mail domain @user1.fs.al. Our setting is that only the MX server can do it, and no one else (-all).

  note

  This is a standard measure against spammers and abusers of the email system. Without this DNS record, the emails that are sent by our mail server may be considered unsafe by the recipients, and may be classified as spam.

DNS changes may take from a few minutes to a couple of days to propagate. We can use dig to verify that these DNS records have been activated:

dig MX user1.fs.al +short
dig A smtp.user1.fs.al +short
dig TXT user1.fs.al +short

3. Build a postfix container

1. Get the scripts and initialize the container:

   ds pull postfix
   ds init postfix @smtp.user1.fs.al
   cd /var/ds/smtp.user1.fs.al/
   nano settings.sh

   Make sure to modify HOSTNAME, VIRTUAL_DOMAINS and FORWARD_ADDRESS:

   HOSTNAME='smtp.user1.fs.al'
   VIRTUAL_DOMAINS='
       user1.fs.al
   '
   
   FORWARD_ADDRESS='dashohoxha@gmail.com'

2. Build the container:

   ds make
   
   docker ps
   docker image ls

3. Check the installation:

   ls sslcert/
   ls /etc/cron.d/
   
   ls config/
   cat config/trusted_hosts
   cat config/virtual_alias_maps
   nano config/virtual_alias_maps.regexp
   ls config/dkim-keys/
   ls config/dkim-keys/user1.fs.al/
   
   ds shell
   ps ax
   pstree
   systemctl status postfix
   ls /etc/postfix/
   postconf -nf | less
   tail /var/log/mail.log
   exit

4. Make the mail server trustworthy

To increase the chances that the other mail servers take seriously the emails comming from our mailserver we have to do some extra DNS configurations. Otherwise the mails that are sent may end up being classified as spam and most probably will not reach the recipient.

4.1. Activate a DKIM key

DKIM keys are used by a mail server to sign the emails that it sends, so that the emails cannot be changed in transit, and so that the receiver can verify that this server is authorized to send emails for a domain. It is an important tool against spams and faked emails. If an smtp server signs the messages that it sends, it is less likely that they will be classified as spam.

The installation scripts have already generated a DKIM key, which is located at config/dkim-keys/user1.fs.al/. To activate it we need to add a record like this on the DNS configuration of the domain:

mail._domainkey.user1.fs.al.  IN  TXT  (
    "v=DKIM1; h=sha256; k=rsa; "
    "p=MIIBIjANBgkqhkiG9w0BAQE....kMJdAwIDAQAB"
)

The content of the public key is on the file: config/dkim-keys/user1.fs.al/mail.txt.

ls config/dkim-keys/user1.fs.al/mail.txt
cat config/dkim-keys/user1.fs.al/mail.txt
cp config/dkim-keys/user1.fs.al/mail.txt /var/ds/nsd/

cd /var/ds/nsd/
ls
ls zones/user1.fs.al.db
cat mail.txt >> zones/user1.fs.al.db
nano zones/user1.fs.al.db

ds check -z

Don't forget to update the serial number as well! And restart nsd:

ds restart

To check whether it has been activated or not, try with:

dig TXT mail._domainkey.user1.fs.al +short

4.2. Create a DMARC record

DMARC is a standard that allows you to set policies on who can send email for your domain based on DKIM and SPF.

You can add a DMARC Record on DNS that will allow you to get weekly reports from major ISPs about the usage of your email domain.

1. Go to http://dmarc.postmarkapp.com/ and add your email address where you want to receive reports, and the email domain name (user1.fs.al).

2. On the DNS configuration of the domain add a TXT record like this:

   _dmarc.user1.fs.al.  IN  TXT  (
       "v=DMARC1; p=none; pct=100; "
       "rua=mailto:re+x2i0yw1hoq7@dmarc.postmarkapp.com; "
       "sp=none; aspf=r;"
   )

   The value of this TXT record is the one generated by the website above.

3. Update the serial and restart the nsd container.

   ds check -z
   ds restart

4. Check that it has been activated:

   dig TXT _dmarc.user1.fs.al. +short

5. Test the SMTP server

For a quick automated test of the basic functionality of the server try: ds test1. The following tests can be done manually.

5.1 Send from inside

Let's send some test emails from inside the container.

1. Send to an existing account:

   cd /var/ds/smtp.user1.fs.al/
   ds shell
   
   swaks --from noreply@user1.fs.al --to info@user1.fs.al
   tail /var/log/mail.log
   
   cat /host/config/virtual_alias_maps
   cat /host/config/virtual_alias_maps.regexp

   This test succeeds because the recipient (info@user1.fs.al) matches the line /^info@/ on virtual_alias_maps.regexp. The sender can be anything (in this case noreply) and the email will still be sent, since we are sending from localhost, which is a trusted host.

   This email is actually forwarded to the address that is listed on virtual_alias_maps.regexp (check also the spam folder if you cannot find it on the inbox).

2. Send to a non-existing account:

   swaks --from info@user1.fs.al --to nobody@user1.fs.al
   tail /var/log/mail.log

   This test fails because the recipient (nobody@example.org) does not match any entry on virtual_alias_maps or virtual_alias_maps.regexp, so the mailserver does not know what to do with this email.

5.2 Send to gmail

1. Send to a gmail account (both from an existing and a non-existing alias):

   swaks --from info@user1.fs.al --to dashohoxha@gmail.com -tlso
   swaks --from noreply@user1.fs.al --to dashohoxha@gmail.com -tlso
   
   tail /var/log/mail.log

   Both of these tests will succeed (250 2.0.0 OK).

   The option -tlso (TLS Optional) enables the TLS encryption of the connection, if available on the server.

   note

   Notice that there are no lines about these messages on the mail log, because swaks is contacting directly the Google mail servers (it is not sending them through our smtp/postfix):

   === Trying gmail-smtp-in.l.google.com:25...
   === Connected to gmail-smtp-in.l.google.com.
   <-  220 mx.google.com ESMTP a640c23a62f3a-ac28589c433si356443866b.15 - gsmtp
   -> EHLO smtp.user1.fs.al
   <-  250-mx.google.com at your service, [188.245.242.143]

   Most probably these emails have been classified as "Spam" by Gmail.

   On gmail use "Show original" from the menu, to see the source of the received emails. Notice that the field DKIM is missing, since these messages are sent by swaks and not by our mailserver. Absense of a DKIM signature is usually a strong reason for classifying a message as spam.

2. Do the same test again, but now tell swaks to connect to the server smtp.user1.fs.al:

   swaks -tlso \
       --from info@user1.fs.al \
       --to dashohoxha@gmail.com \
       --server smtp.user1.fs.al
   swaks -tlso \
       --from noreply@user1.fs.al \
       --to dashohoxha@gmail.com \
       --server smtp.user1.fs.al
   
   tail /var/log/mail.log

   This time the message goes through our SMTP server, it is logged, and we can verify (by looking at the source of the received message) that it is signed with the DKIM key.

   Note: It may still be classified as spam by google, because its content looks suspicious.

3. Let's try the same test from outside the container:

   exit
   
   which swaks
   apt install swaks
   
   swaks -tlso --from nobody@user1.fs.al --to dashohoxha@gmail.com
   ds exec tail /var/log/mail.log

   This time swaks will not be able to contact the gmail servers directly (because it is missing some components), so it will try to contact localhost:25:

   *** MX Routing not available: requires Net::DNS.  Using localhost as mail server
   *** TLS not available: requires Net::SSLeay
   === Trying localhost:25...
   === Connected to localhost.
   <-  220 static.143.242.245.188.clients.your-server.de ESMTP Postfix (Debian/GNU)

4. You can also send a message from some external email address to info@user1.fs.al, and it will be forwarded to dashohoxha@gmail.com.

   But it is not possible to send to noreply@user1.fs.al, or to some other virtual alias that is not defined. Try it.

5.3 Send from the host

Let's try to send a test email from the host (outside the container):

swaks --from info@user1.fs.al --to admin@user1.fs.al -tlso
ds exec tail /var/log/mail.log

It is using the SMTP server localhost:25:

*** MX Routing not available: requires Net::DNS.  Using localhost as mail server
*** TLS not available: requires Net::SSLeay
=== Trying localhost:25...
=== Connected to localhost. 

swaks -tlso \
    --from info@user1.fs.al \
    --to admin@user1.fs.al \
    --server smtp.user1.fs.al
ds exec tail /var/log/mail.log

It may fail, because the IP of the host might not be on the list of the trusted hosts. Let's append it to config/trusted_hosts and run ds inject update.sh. Then try again.

echo `188.245.242.143` >> config/trusted_hosts
cat config/trusted_hosts
ds inject update.sh

swaks -tlso \
    --from info@user1.fs.al \
    --to admin@user1.fs.al \
    --server smtp.user1.fs.al
ds exec tail /var/log/mail.log

5.4 Add a new address

Let's try to send an email to xyz@user1.fs.al:

swaks --from info@user1.fs.al --to xyz@user1.fs.al -tlso
ds exec tail /var/log/mail.log

 -> RCPT TO:<xyz@user1.fs.al>
<** 550 5.1.1 <xyz@user1.fs.al>: Recipient address rejected:
                           User unknown in virtual alias table

smtp postfix/smtpd[864]: NOQUEUE: reject: RCPT from unknown[172.18.0.1]: 
550 5.1.1 <xyz@user1.fs.al>: Recipient address rejected: 
User unknown in virtual alias table; from=<info@user1.fs.al> 
to=<xyz@user1.fs.al> proto=ESMTP helo=<mycloud>

It fails because the recipient does not exist on the alias table. Let's add it at config/virtual_alias_maps, update the alias db, and try it again:

echo 'xyz@user1.fs.al  dashohoxha@gmail.com' \
    >> config/virtual_alias_maps
cat config/virtual_alias_maps
#ds inject update.sh
ds exec postmap /host/config/virtual_alias_maps

swaks --from info@user1.fs.al --to xyz@user1.fs.al -tlso
ds exec tail /var/log/mail.log

5.5 Test the ports 587/465

swaks --server smtp.user1.fs.al:587 \
      --from info@user1.fs.al --to admin@user1.fs.al

swaks --server smtp.user1.fs.al:465 \
      --from info@user1.fs.al --to admin@user1.fs.al

5.6 Sending from outside

important

Make sure that sending messages from outside the VPS fails:

swaks --server smtp.user1.fs.al:587 \
      --from info@user1.fs.al --to admin@user1.fs.al

If it does not fail, there is something wrong, because it means that everyone can use our SMTP to send emails, and it can be abused by the spammers.

suggerimento

To allow a server to send emails from our SMTP, we should append the public IP of the server to the list of trusted hosts:

echo '11.12.13.14' >> config/trusted_hosts
ds inject update.sh

6. Other topics

6.1 Check the health of the mail server

• Send an email to check-auth@verifier.port25.com:

  swaks --server smtp.user1.fs.al -tlso \
      --from info@user1.fs.al \
      --to check-auth@verifier.port25.com

  The automatic reply will give you important information about the status and health of your email server (for example whether the mails sent from it pass the SPF and DKIM checks, whether they are considered spam or not, etc.)

• Go to https://www.mail-tester.com/ and send a message to the email address displayed there, like this:

  swaks --server smtp.user1.fs.al -tlso \
      --from info@user1.fs.al \
      --to test-1p4f6@mail-tester.com

  Then click the button for checking the score.

• There are lots of other tools and websites that help to check the configuration of a mail server (DNS settings, configuration, security features, etc.) For example:

  • https://ns.tools
  • https://mxtoolbox.com/

6.2 Add another email domain

Let's say that we want to use this SMTP server for another domain, for example example.com.

1. Set DNS configurations like this:

   smtp.example.com.  IN  CNAME       smtp.user1.fs.al.
   
   example.com.       IN  MX     10   smtp.example.com.
   example.com.       IN  MX     20   smtp.example.com.
   example.com.       IN  TXT         "v=spf1 mx -all"

   Note that the first record (CNAME) redirects the domain smtp.example.com to the same IP address that smtp.user1.fs.al points to.

   Check these configurations:

   dig +short smtp.example.com.
   dig +short example.com. MX
   dig +short example.com. TXT

2. On the SMTP container, edit settings.sh and add the new domain at VIRTUAL_DOMAINS:

   cd /var/ds/smtp.user1.fs.al/
   nano settings.sh

   It should look like this:

   HOSTNAME='smtp.user1.fs.al'
   VIRTUAL_DOMAINS='
       user1.fs.al
       example.com
   '

3. Run ds make to rebuild the container. The rebuild is needed because we need to request a new SSL cert from letsencrypt, which covers all the domains (including the new one).

4. Go to http://dmarc.postmarkapp.com/ and generate a DMARC record for the new domain.

5. Update the DNS configuration with TXT records for DKIM and DMARK, which look like these:

   mail._domainkey.example.com.  IN  TXT  (
       "v=DKIM1; h=sha256; k=rsa; "
       "p=MIIBIjANBgkqhkiG9w0BAQE....kMJdAwIDAQAB"
   )
   
   _dmarc.example.com.           IN  TXT  (
       "v=DMARC1; p=none; pct=100; "
       "rua=mailto:re+x2i0yw1hoq7@dmarc.postmarkapp.com; "
       "sp=none; aspf=r;"
   )

   Note that:

   • The value of the key for the DKIM record can be found on the file: config/dkim-keys/example.com/mail.txt

   • The value of the DMARC record is the one obtained on the previous step.

   You can check them like this:

   dig +short mail._domainkey.example.com. TXT
   dig +short _dmarc.example.com. TXT

6.3 Relay domains

If another mail server is installed on the same machine where this SMTP server is installed, there is a problem, because they both need to use the port 25 for receiving mails. SMTP ports cannot be shared by more than one application, and it is not possible to proxy them (like HTTP or HTTPS ports, for example).

A workaround in this case would be to use this simple SMTP server as a relay for the mail domains that are served by the other mail server. It works like this:

1. When a connection comes on the port 25 (someone is trying to send an email), it is handled by the simple SMTP server, since this port is forwarded to this server.

2. If the domain of the receipient's address is one of the domains that is managed by the simple SMTP server, the mail is handled by this server itself.

3. Otherwise, if the domain is one of those that are managed by the other mail server, it is relayed instead to that mail server.

In other words, the simple SMTP server behaves like a kind of router for the emails that are received, based on the email domain of the receipent's address.

Let's see how to implement this SMTP relay. We will assume that the domains that are managed by the other mail server are example1.org, example2.org and example3.org. We can create the custom command ds relay-setup like this:

cd /var/ds/smtp.user1.fs.al/
mkdir -p cmd

cat <<'__EOF__' > cmd/relay-setup.sh
cmd_relay-setup() {
    # create a config file for relay_domains
    cat <<EOF > config/relay_domains
example1.org
example2.org
example3.org
EOF

    # create a config file for transport_maps
    local smtp_relay='smtp:10.214.77.201:25'
    cat <<EOF > config/transport_maps
example1.org    $smtp_relay
example2.org    $smtp_relay
example3.org    $smtp_relay
EOF

    # setup transport_maps
    ds exec postconf -e 'transport_maps=hash:/host/config/transport_maps'
    ds exec postmap /host/config/transport_maps

    # setup relay_domains
    ds exec postconf -e relay_domains=/host/config/relay_domains

    # reload postfix configuration
    ds exec postfix reload
}
__EOF__

cat cmd/relay-setup.sh

note

We are assuming that 10.214.77.201 is the internal (fixed) IP of the other mailserver.

In order to call this command automatically whenever we rebuild the container, we can override the command ds config like this:

cat <<'EOF' > cmd/config.sh
rename_function cmd_config standard_config
cmd_config() {
    standard_config
    ds relay-setup
}
EOF

cat cmd/config.sh