Setting Up a Mail Server with SMTP and POP3 in Kali Linux: A Comprehensive Guide

Setting Up a Mail Server with SMTP and POP3 in Kali Linux: A Comprehensive Guide

Kali Linux is a specialized distribution designed for cybersecurity professionals, penetration testers, and ethical hackers. Setting up a mail server with SMTP and POP3 protocols on Kali Linux enables users to send, receive, and test email functionalities in a controlled environment, which is particularly useful for security testing and application development. This professional article provides a detailed, step-by-step guide to installing and configuring a mail server using Postfix (for SMTP) and Dovecot (for POP3), including SQL integration with MySQL, practical examples, and ethical hacking considerations. The guide is tailored for beginners and intermediate users, reflecting best practices as of September 2025.

Introduction

A mail server facilitates email communication by handling sending (via SMTP), receiving, and retrieving (via POP3 or IMAP) emails. Postfix, a popular Mail Transfer Agent (MTA), and Dovecot, a secure IMAP and POP3 server, form a robust combination for managing email services on Kali Linux. This article covers the installation, configuration, and usage of Postfix and Dovecot with MySQL for user management, a practical example of a mail system, and techniques for ethical security testing. The focus is on creating a local mail server for testing purposes, with warnings about legal and ethical considerations.

Key Objectives

• Install and configure Postfix for SMTP and Dovecot for POP3.
• Integrate MySQL for managing email users and mailboxes.
• Use SMTP and POP3 to send and retrieve emails.
• Develop a practical PHP-based webmail interface.
• Provide ethical hacking techniques for testing mail server security.
• Demonstrate sending emails via command line and web interface.

Prerequisites

• Kali Linux 2025.x (or latest version) installed.
• Root or sudo privileges.
• Internet connection for package installation.
• Basic knowledge of Linux commands, SQL, and PHP.
• A controlled lab environment (e.g., local machine or VM) for testing.
• Explicit permission for any external testing (ethical hacking).
• A fully qualified domain name (FQDN) for testing (e.g., mail.example.com) or use localhost for local testing.

Installation and Setup

Step 1: Update Kali Linux

Ensure your system is up-to-date to avoid compatibility issues:

sudo apt update && sudo apt upgrade -y

This updates the package index and installs the latest patches.

Step 2: Install Postfix (SMTP Server)

Postfix is a reliable MTA for sending and receiving emails via SMTP. Install it with:

sudo apt install postfix postfix-mysql -y

During installation, select Internet Site when prompted and set the mail name to your FQDN (e.g., mail.example.com) or localhost for local testing. Verify Postfix is running:

sudo systemctl start postfix

sudo systemctl enable postfix

sudo systemctl status postfix

Check that Postfix is listening on port 25 (SMTP):

ss -ant | grep 25

Step 3: Install Dovecot (POP3 Server)

Dovecot provides POP3 and IMAP services for retrieving emails. Install it with:

sudo apt install dovecot-core dovecot-pop3d dovecot-mysql -y

Start and enable Dovecot:

sudo systemctl start dovecot

sudo systemctl enable dovecot

sudo systemctl status dovecot

Verify POP3 (port 110) and POP3S (port 995) are active:

ss -ant | grep -E '110|995'

Step 4: Install MySQL

MySQL will store user and mailbox information. Install it with:

sudo apt install mysql-server -y

Secure the MySQL installation:

sudo mysql_secure_installation

Follow prompts to set a root password, remove anonymous users, disallow remote root login, and remove the test database. Start and enable MySQL:

sudo systemctl start mysql

sudo systemctl enable mysql

Step 5: Install Apache2 and PHP

To create a webmail interface, install Apache2 and PHP:

sudo apt install apache2 php php-mysql libapache2-mod-php -y

Restart Apache2:

sudo systemctl restart apache2

Test PHP by creating /var/www/html/info.php:

sudo nano /var/www/html/info.php

Add:

<?php phpinfo(); ?>

Access http://localhost/info.php to confirm PHP is working.

Configuring the Mail Server

Step 6: Set Up MySQL Database and Users

Log in to MySQL:

sudo mysql -u root -p

Create a database and tables for mail users:

CREATE DATABASE mailserver;

USE mailserver;

CREATE TABLE virtual_users (

    id INT AUTO_INCREMENT PRIMARY KEY,

    domain_id INT NOT NULL,

    email VARCHAR(100) NOT NULL UNIQUE,

    password VARCHAR(106) NOT NULL

);

CREATE TABLE virtual_domains (

    id INT AUTO_INCREMENT PRIMARY KEY,

    name VARCHAR(50) NOT NULL

);

CREATE TABLE virtual_aliases (

    id INT AUTO_INCREMENT PRIMARY KEY,

    domain_id INT NOT NULL,

    source VARCHAR(100) NOT NULL,

    destination VARCHAR(100) NOT NULL

);

Insert a test domain and user:

INSERT INTO virtual_domains (id, name) VALUES (1, 'example.com');

INSERT INTO virtual_users (id, domain_id, email, password) VALUES (1, 1, 'test@example.com', ENCRYPT('secure_password', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))));

Create a MySQL user for Postfix and Dovecot:

CREATE USER 'mailuser'@'localhost' IDENTIFIED BY 'mailpassword';

GRANT SELECT ON mailserver.* TO 'mailuser'@'localhost';

FLUSH PRIVILEGES;

EXIT;

Step 7: Configure Postfix

Edit the main Postfix configuration file:

sudo nano /etc/postfix/main.cf

Add or modify:

myhostname = mail.example.com

mydestination = localhost

mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128

inet_interfaces = all

inet_protocols = all

virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf

virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf

smtpd_sasl_auth_enable = yes

smtpd_sasl_type = dovecot

smtpd_sasl_path = private/auth

smtpd_sasl_security_options = noanonymous

smtpd_tls_security_level = may

smtpd_tls_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem

smtpd_tls_key_file = /etc/ssl/private/ssl-cert-snakeoil.key

Create MySQL configuration files for Postfix. For /etc/postfix/mysql-virtual-mailbox-domains.cf:

sudo nano /etc/postfix/mysql-virtual-mailbox-domains.cf

Add:

user = mailuser

password = mailpassword

hosts = 127.0.0.1

dbname = mailserver

query = SELECT 1 FROM virtual_domains WHERE name='%s'

For /etc/postfix/mysql-virtual-mailbox-maps.cf:

sudo nano /etc/postfix/mysql-virtual-mailbox-maps.cf

Add:

user = mailuser

password = mailpassword

hosts = 127.0.0.1

dbname = mailserver

query = SELECT 1 FROM virtual_users WHERE email='%s'

For /etc/postfix/mysql-virtual-alias-maps.cf:

sudo nano /etc/postfix/mysql-virtual-alias-maps.cf

Add:

user = mailuser

password = mailpassword

hosts = 127.0.0.1

dbname = mailserver

query = SELECT destination FROM virtual_aliases WHERE source='%s'

Set permissions:

sudo chmod 640 /etc/postfix/mysql-*.cf

sudo chown root:postfix /etc/postfix/mysql-*.cf

Restart Postfix:

sudo systemctl restart postfix

Step 8: Configure Dovecot

Edit the main Dovecot configuration file:

sudo nano /etc/dovecot/dovecot.conf

Ensure:

protocols = pop3 pop3s

Edit /etc/dovecot/conf.d/10-auth.conf:

sudo nano /etc/dovecot/conf.d/10-auth.conf

Set:

disable_plaintext_auth = yes

auth_mechanisms = plain login

!include auth-sql.conf.ext

Edit /etc/dovecot/conf.d/10-mail.conf:

sudo nano /etc/dovecot/conf.d/10-mail.conf

Set:

mail_location = maildir:/var/mail/vhosts/%d/%n

Create the mail directory:

sudo mkdir -p /var/mail/vhosts/example.com/test

sudo chown -R vmail:vmail /var/mail/vhosts

Edit /etc/dovecot/conf.d/10-master.conf:

sudo nano /etc/dovecot/conf.d/10-master.conf

Ensure:

service auth {

  unix_listener /var/spool/postfix/private/auth {

    mode = 0660

    user = postfix

    group = postfix

  }

}

Create /etc/dovecot/dovecot-sql.conf.ext:

sudo nano /etc/dovecot/dovecot-sql.conf.ext

Add:

driver = mysql

connect = host=127.0.0.1 dbname=mailserver user=mailuser password=mailpassword

default_pass_scheme = SHA512-CRYPT

password_query = SELECT email as user, password FROM virtual_users WHERE email='%u';

Set permissions:

sudo chmod 640 /etc/dovecot/dovecot-sql.conf.ext

sudo chown root:dovecot /etc/dovecot/dovecot-sql.conf.ext

Restart Dovecot:

sudo systemctl restart dovecot

Sending and Retrieving Emails

Step 9: Test Sending Email via Command Line

Install mailutils for the mail command:

sudo apt install mailutils -y

Send a test email:

echo "This is a test email." | mail -s "Test Subject" test@example.com

Check the mail log:

sudo tail -f /var/log/mail.log

The email should be stored in /var/mail/vhosts/example.com/test.

Step 10: Test POP3 Retrieval

Use telnet to test POP3:

telnet localhost 110

Enter:

USER test@example.com

PASS secure_password

LIST

RETR 1

QUIT

You should see the test email. For secure POP3 (POP3S), use:

openssl s_client -connect localhost:995

Practical Example: Webmail Interface

Step 11: Create a PHP Webmail Interface

Create a PHP file to send emails:

sudo nano /var/www/html/webmail.php

Add:

<?php

if ($_SERVER["REQUEST_METHOD"] == "POST") {

    $to = $_POST['to'];

    $subject = $_POST['subject'];

    $message = $_POST['message'];

    $headers = "From: test@example.com";

    if (mail($to, $subject, $message, $headers)) {

        echo "Email sent successfully!";

    } else {

        echo "Failed to send email.";

    }

}

?>

 

<!DOCTYPE html>

<html>

<head>

    <title>Webmail</title>

    <style>

        form { width: 50%; margin: 20px auto; }

        label, input, textarea { display: block; margin: 10px 0; }

        input, textarea { width: 100%; padding: 8px; }

    </style>

</head>

<body>

    <h1>Send Email</h1>

    <form method="post">

        <label>To:</label>

        <input type="email" name="to" required>

        <label>Subject:</label>

        <input type="text" name="subject" required>

        <label>Message:</label>

        <textarea name="message" required></textarea>

        <input type="submit" value="Send Email">

    </form>

</body>

</html>

Access http://localhost/webmail.php, enter test@example.com as the recipient, and send a test email. Check /var/mail/vhosts/example.com/test for the email.

Ethical Hacking Techniques

Note: These techniques are for testing in controlled environments (e.g., localhost or DVWA) with explicit permission. Unauthorized testing violates laws like the U.S. Computer Fraud and Abuse Act or GDPR.

1. SMTP User Enumeration

Goal: Identify valid email accounts. Steps:

1. Use smtp-user-enum to test for valid users:

2.  sudo apt install smtp-user-enum

smtp-user-enum -M VRFY -u test -t localhost

3. Check for responses indicating test@example.com exists.
4. Mitigation: Disable VRFY/EXPN in /etc/postfix/main.cf:

smtpd_noop_commands = vrfy expn

Restart Postfix: sudo systemctl restart postfix.

2. Brute-Force POP3 Credentials

Goal: Test for weak passwords. Steps:

1. Use Hydra to attempt a brute-force attack:

hydra -l test@example.com -P /usr/share/wordlists/rockyou.txt localhost pop3

2. Monitor for successful logins.
3. Mitigation: Enforce strong passwords and enable disable_plaintext_auth = yes in Dovecot.

3. Sniffing Unencrypted Traffic

Goal: Capture SMTP or POP3 credentials. Steps:

1. Use Wireshark to monitor traffic:

sudo wireshark &

2. Filter for smtp or pop and attempt login via telnet localhost 110.
3. Mitigation: Enable TLS in Postfix (smtpd_tls_security_level = encrypt) and Dovecot (ssl = required).

Best Practices

• Secure Configurations: Use TLS for SMTP and POP3 (smtp_tls_security_level = encrypt, ssl = required).
• Strong Passwords: Enforce complex passwords in MySQL for virtual_users.
• Firewall Rules: Restrict ports 25, 110, and 995 to trusted networks:

sudo ufw allow proto tcp from 127.0.0.1 to any port 25,110,995

• Regular Updates: Keep Postfix, Dovecot, and MySQL updated: sudo apt update && sudo apt upgrade.
• Backup Data: Back up the MySQL database:

mysqldump -u mailuser -p mailserver > mailserver_backup.sql

• Ethical Testing: Test only in authorized environments and respect bug bounty scopes.
• Logging: Monitor /var/log/mail.log and /var/log/dovecot.log for issues.

Limitations

• Kali Linux Stability: Designed for testing, not production; consider Ubuntu for production servers.
• Resource Usage: Postfix and Dovecot can be resource-intensive; optimize configurations.
• Security Risks: Misconfigured servers can be exploited; always secure ports and credentials.
• External Delivery: Sending emails to external domains requires proper DNS (MX, SPF, DKIM) setup, which is complex and often blocked by VPS providers.

Conclusion

Setting up a mail server with Postfix and Dovecot on Kali Linux provides a powerful environment for testing email functionalities and security. This guide demonstrated how to configure SMTP and POP3, integrate MySQL for user management, create a webmail interface, and perform ethical security tests. Always ensure you have permission for testing and follow best practices to secure your server. For further learning, explore resources like postfix.org, dovecot.org, or practice with tools like Metasploit.

To convert this Markdown to .docx for your website, use Pandoc: pandoc mailserver-smtp-pop3-kali-linux-tutorial.md -o mailserver-smtp-pop3-kali-linux-tutorial.docx. Alternatively, paste into Microsoft Word or a CMS with Markdown support.