Dedicated Root Server

1. Intro

As we install more apps on our VPS, we need to add more resources to it (vCPU, RAM, Storage, etc), and its price starts to grow. If the price starts to approach 40-50 EUR, then it is better to get a Dedicated Root Server and to migrate all the apps there, because it provides more resources and better performance for a similar price.

A Dedicated Root Server is a real physical server that we manage ourselves, as opposed to a VPS, which is a virtual server. On a real server we can create virtual machines and containers, if needed.

2. Installation

First time that we access a dedicated root server, after purchasing it, we access it in a rescue mode, which is a temporary system running on our machine, so that we can install it or fix something.

We can use installimage to install Debian on it. This video shows how to do it.

The server has two disk drives: /dev/nvme0n1 and /dev/nvme1n1. By default the software RAID is enabled, with the option SWRAID 1 and SWRAIDLEVEL 1. With this configuration the disks work as a mirror of each-other. This is a robust configuration, because if one of the disks fails, the server will still continue to run. When the broken disk is replaced, it will be mirrored automatically.

However we are going to disable the RAID by setting SWRAID 0. As a result, the operating system will be installed only on the first disk, and the second one will be free. We will use it later as a storage for the Incus containers.

important

This choice allows us to have more available disk space, but it also makes our system more vulnerable to disk failures. If one of the disks fails, the whole system is corrupted and everything needs to be reinstalled from scratch. For this reason, we will make sure to have proper backups of everything, so that we can restore easily in case of a disaster. We will also use scripts for installing different apps, so that the installation is repeatable and reinstallation does not take a long time.

I prefer to disable IPv6, by setting the option IPV4_ONLY yes. It is also possible to set the HOSTNAME in the configuration file. After these modifications, the configuration settings should look like this (without the comment lines):

DRIVE1 /dev/nvme0n1
#DRIVE2 /dev/nvme1n1
SWRAID 0

HOSTNAME server1
IPV4_ONLY yes
USE_KERNEL_MODE_SETTING yes

PART /boot/efi esp 256M
PART swap swap 32G
PART /boot ext3 1024M
PART / ext4 all

IMAGE /root/.oldroot/nfs/install/../images/Debian-1201-bookworm-amd64-base.tar.gz

Once we save and close the configuration file (by pressing ESC), installimage will start the installation. After reboot, we can access the server with the same password that we accessed the rescue system.

tip

On the new system we can find the files /installimage.conf and /installimage.debug.

Download apps/part7/1-installation.cast

Loading asciinema cast...

3. Setup

3.1 Basic setup

1. Update and install some packages:

   ssh root@65.109.96.100
   
   apt update
   apt upgrade
   apt install \
       nano psmisc git tmux tmate asciinema \
       mosh glances unattended-upgrades

2. Edit /etc/vim/vimrc and uncomment set background=dark.

3. Install firewalld and fail2ban:

   apt install firewalld
   firewall-cmd --list-all
   firewall-cmd --zone=public --set-target=DROP --permanent
   firewall-cmd --reload
   firewall-cmd --list-all

   install fail2ban python3-systemd
   
   cat <<EOF > /etc/fail2ban/jail.local
   [DEFAULT]
   backend = systemd
   EOF
   
   systemctl restart fail2ban
   fail2ban-client status
   fail2ban-client status ssh

3.2 Use an SSH key

1. Generate an SSH key on the server:

   ssh-keygen -t ecdsa -f srv1
   
   ls -l
   mkdir -p ~/.ssh
   chmod 700 ~/.ssh
   touch ~/.ssh/authorized_keys
   chmod 600 ~/.ssh/authorized_keys
   
   cat srv1.pub >> ~/.ssh/authorized_keys
   cat ~/.ssh/authorized_keys
   exit

2. Transfer the private key to the local machine and use it to login:

   scp root@65.109.96.100:srv1 .
   ls -l srv1
   cat srv1
   ssh -i srv1 root@65.109.96.100    # should login without a password
   exit

3. On the local machine, create an SSH config for accessing srv1:

   mkdir -p ~/.ssh
   chmod 700 ~/.ssh
   touch ~/.ssh/config
   chmod 600 ~/.ssh/config
   
   cat << EOF >> /home/user1/.ssh/config
   Host srv1
       HostName 65.109.96.100
       User root
       Port 22
       IdentityFile ~/.ssh/srv1.key
   EOF
   
   mv srv1 ~/.ssh/srv1.key
   ls -al ~/.ssh/
   
   # test it
   ssh srv1

3.3 Disable password login.

Now that we can login with a private key, we can disable the password login on the server, to make it more secure.

1. Edit the file /etc/ssh/sshd_config on the server and make sure to set the option PasswordAuthentication to no, and te option PermitRootLogin to prohibit-password. Also make sure that KbdInteractiveAuthentication is no:

   #PermitRootLogin yes
   PermitRootLogin prohibit-password
   
   #PasswordAuthentication yes
   PasswordAuthentication no
   
   KbdInteractiveAuthentication no

2. Save the file and restart the sshd service:

   systemctl restart sshd
   exit

3. Make sure that you can still login with the private key. Test also that you cannot login with a password anymore.

   ssh srv1
   exit
   
   ssh root@65.109.96.100    # should fail

3.4 Change the SSH port

This is another step for making the server a bit more secure.

1. Edit /etc/ssh/sshd_config on the server and change the port from 22 to something else (for example with 4 or 5 digits):

   ssh srv1
   nano /etc/ssh/sshd_config

   #Port 22
   Port 2125

2. Open this port in the firewall:

   firewall-cmd --zone=public --add-port=2125/tcp
   firewall-cmd --list-all

3. Restart the SSH service:

   systemctl restart sshd
   exit

4. Change the port in ~/.ssh/config on the local machine and test that you can still login to the server:

   nano ~/.ssh/config
   ssh srv1

5. Make the firewall change permanent:

   firewall-cmd --zone=public --add-port=2125/tcp --permanent 
   firewall-cmd --zone=public --remove-service=ssh --permanent 
   firewall-cmd --reload
   firewall-cmd --list-all

This screencast shows some of the steps:

Download apps/part7/2-basic-setup.cast

Loading asciinema cast...

3. Migrate apps

3.1 docker-scripts

3.2 Incus