#+title: Shared System
#+date: 2026-03-24 Tue
#+author: W. Kosior
#+email: wkosior@agh.edu.pl
#+HTML_HEAD: <link rel="stylesheet" href="../org-dark-styles.css">

In these exercises we are going to configure some security aspects of a
multiuser system (a shared server).

To be graded, send
1. the required screenshots,
2. your =playbook.yaml=, and
3. other files used by the playbook
to @@html:<a href="email:wkosior@agh.edu.pl">wkosior@agh.edu.pl</a>@@ along with
your names.  You can work in teams of up to three people.

* Our Setup
We shall be using [[https://pluton.kt.agh.edu.pl/~wokosior/bso-vms/debian-13-nocloud-amd64.qcow2][the Debian VM image]] from [[../01-secure-booting.org][the first topic]].  Its credentials are
=root=​/​=security=.  It can be run with the following QEMU command.

#+begin_src shell-script
  qemu-system-x86_64 \
      -m 1G \
      -enable-kvm \
      -hda debian-13-nocloud-amd64.qcow2 \
      -net nic -net user,hostfwd=tcp::30022-:22 \
      -display none # Use `-vga virtio' / `-nographic' for a GUI / serial port.
#+end_src

If possible, choose one one your team's computers for running Ansible and one
for logging in to the VM interactively.  Which PC actually runs the VM matters
little.

* User creation
Copy SSH public key to your the VM and create an inventory file for Ansible.
Then, start with the following in your playbook.

#+begin_src yaml
  - name: Ansible Playbook for Shared Server
    hosts: tilde # Make the host names here and in the inventory match.
    tasks:
      - name: Add groups
        ansible.builtin.group:
          name: users

      - name: Set default user shell
        ansible.builtin.replace:
          path: /etc/default/useradd
          regexp: ^SHELL=.*
          replace: SHELL=/bin/bash

      - name: Add users
        ansible.builtin.user:
          name: "{{ item.name }}"
          group: users
          password: "{{ item.password }}"
          update_password: on_create
        loop:
          - name: theodore
            password: ? # Fill in.
          - name: eugenia
            password: ? # Fill in.
          - name: pancratius
            password: ? # Fill in.
#+end_src

The =loop= construct creates multiple user accounts.  Generate password hashes
for them using the =openssl passwd -5 -stdin= command.

*Prepare screenshot 1: output of a successful playbook execution that created
user accounts.*

* Quota configuration
Use the following task to install the =quota= package necessary for disk limits
configuration.

#+begin_src yaml
  - name: Install quota tools
    ansible.builtin.apt:
      name: quota
      cache_valid_time: 86400
#+end_src

Edit =/etc/fstab= to add =usrquota= and =grpquota= to the options of the root
filesystem.  They shall be used automatically upon boot, after initrafms is left
(the root filesystem gets *remounted* with the options from the =fstab=).

Instead of rebooting, remount the root filesystem manually.

#+begin_src shell-script
  mount -o remount /
#+end_src

Verify that the quota-related mount options are active.

#+begin_src shell-script
  grep -E '/\s.*usrquota,grpquota' /proc/mounts
#+end_src

Use the following to create quota files inside the root filesystem.  You can
ignore the warning that is likely to show up.

#+begin_src shell-script
  quotacheck --no-remount --user --group --create-files /
  ls -l /aquota* # See the created files.
#+end_src

Turn on the quota.  Note the statuses of =quotaon=.

#+begin_src shell-script
  quotaon -p /
  echo $?
  quotaon /
  quotaon -p /
  echo $?
#+end_src

Now, use Ansible to automate what you've just done.
1. For idempotent updates of =/etc/fstab= you can use the [[https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/script_module.html#ansible-collections-ansible-builtin-script-module][ansible.builtin.script]]
   module with the command below placed inside the script.
2. You can use a shell conditional to perform =mount -o remount /= only if
   =grep= fails to find the quota options in =/proc/mounts=.
3. You can use a shell conditional to only run =quotacheck= when an =aquota.=
   file is missing.
4. You can use a shell conditional to only perform =quotaon /= when =quotaon -p
   /= reports that the feature is currently off.

#+begin_src shell-script
  # The perl command to place inside the script.
  perl -pe '$_ =~ s/^((\S+\s+){3}\S+)/\1,usrquota,grpquota/
                unless /^(#.*)$|^\S+\s+\/\S|usrquota,grpquota/' \
       -i /etc/fstab
#+end_src

Now, set some limits as root.

#+begin_src shell-script
  repquota /
  export EDITOR=vi # optional
  edquota -u theodore / # set soft / hard to 10000 / 20000 blocks
#+end_src

As theodore, try to create a big file.
#+begin_src shell-script
  dd if=/dev/zero bs=1024 count=10000 of=my-file
  quota -u theodore
#+end_src

As root, view the block usage by users.
#+begin_src shell-script
  repquota /
#+end_src

As theodore, try to create another big file.
#+begin_src shell-script
  dd if=/dev/zero bs=1024 count=10000 of=my-other-file
#+end_src

This time theodore has exceeded not just the soft limit but also the hard one.

*Prepare screenshot 2: output of the error message when theodore exceeded hard
block limit.*

* Inode Limits
As theodore, create many small (empty) files.
#+begin_src shell-script
  rm my-other-file
  mkdir -p empty-files
  while touch empty-files/$RANDOM$RANDOM$RANDOM; do true; done
#+end_src

At some point further creation should fail, even though the filesystem seems to
have plenty of space left.

#+begin_src shell-script
  df /
  df -i /
#+end_src

Remove the problematic small files.  Then, set theodore's inode limits to 1000 /
2000 with edquota.  Then, as theodore, run the loop again.

#+begin_src shell-script
  mkdir -p empty-files
  while touch empty-files/$RANDOM$RANDOM$RANDOM; do true; done
#+end_src

Use =repquota= again to see current usage by theodore.  Then, change the grace
periods of soft limits to lower (=setquota='s =-t= option).

#+begin_src shell-script
  setquota -t $((60*60*24*2)) $((60*60*24*3)) /
#+end_src

Also change the remaining grace time of theodore (=setquota='s =-T= option).
Then, view the numbers using =repquota=.

Finally, use =setquota= instead of =edquota= to noninteractively set the quota
of the remaining users.  Note the behavior of the =-p= option of =setquota=.

#+begin_src shell-script
  setquota -u eugenia 15000 20000 1000 2000 /
  setquota -u -p eugenia pancratius / # Copy quotas of eugenia.
  repquota /
#+end_src

Now, put =setquota= commands in the playbook so that it can be used to replicate
your configuration.

*Prepare screenshot 3: output of the =ansible-playbook= command when running the
playbook that sets all users' quotas.*

* Automatically assigning quotas to users
Use Ansible to assign =/= quota to user nobody (who exists by default in the
system).

Now, cause each new users to use the quota of =nobody=.  To achieve this, use
Ansible to idempotently (use sed, perl or just install the entire file) change
the relevant quota option in =/etc/adduser.conf=.  You'll notice the option
immediately once you look at the comments in system's default =adduser.conf=.

You can use =-v= to report quota of all users, including those who have no files
on the filesystem (as in the case of user =nobody=).

#+begin_src shell-script
  repquota /
  repquota -v /
#+end_src

Once you've run the playbook, add a new user *interactively* and see that he's
had his quota copied from nobody.

#+begin_src shell-script
  adduser --ingroup users alexander
  repquota /
#+end_src

*Prepare screenshot 4: output of the two commands above.*

* Sudoers
We shall allow a single user to change disk quota grace time of all the others.
Create an executable shell script under =/usr/local/bin/=.  It should accept 3
arguments: username, block grace time and inode grace time.  It should call
=setquota=.

#+begin_src shell-script
  setquota -u "$1" -T "$2" "$3" /
#+end_src

Now, use =visudo -f /etc/sudoers.d/eugenia-setquota= and initially, populate the
file with garbage (hit some random keys on the keyboard).  When you try to save
the file, =visudo= should detect a syntax error and prevent you from deploying
configuration that could make =sudo= inoperable and render the system
inaccessible.  Type =h= to view =visudo='s help.

*Prepare screenshot 5: =visudo='s help displayed after trying to save an
incorrect =sudoers= configuration.*

Edit the file again and populate it with a line allowing eugenia to execute the
script as root.

#+begin_example
eugenia ALL = /usr/local/bin/YOUR-SCRIPT-NAME .*
#+end_example

Log in as eugenia and try running the script with =sudo= to change the grace
time of some other user with soft limit exceeded.  Then, configure Ansible to
install the script under =/etc/sudoers.d/= with permissions =440=.

* Doas
We shall configure =doas= to serve the same purpose that =sudo= has served.
Instal =doas= from the repository.  Use the manual (=man doas=) to find the path
of the config file used by =doas=.  Then, use Ansible to create it with the
following contents (with script path adapted) and with permissions 440.

#+begin_example
permit eugenia as root cmd /usr/local/bin/YOUR-SCRIPT-NAME
#+end_example

Log in as eugenia and try running the script with =doas= to change the grace
time of some other user with soft limit exceeded.  Additionally, run =doas= with
some other command to see that this is not allowed.

*Prepare screenshot 6: The aforementioned two =doas= invokations with the output
(when present) visible.*

* Resource limits
We shall now configure limits for other types of resources.  You can consult the
relevant manuals.

#+begin_src shell-script
  man getrlimit
  man limits.conf
#+end_src

Although these limits are per-process or per-session rather than per-user, we
can configure them to effectively limit how much resources each user can
consume.

Create a new file with the following contents (substitute the question marks to
limit maximum size of total available memory, use the aforementioned manuals to
find the relevant limit name).

#+begin_src example
@users      -     nofile  500
@users      -     ?       30000
pancratius  hard  ?       60000
#+end_src

Use Ansible to install this file under =/etc/security/limits.d/=, with filename
ending with =.conf=.

As pancratius, log in and run the following commands to view process' current
limits.

#+begin_src shell-script
  ulimit -Sa
  ulimit -Ha
#+end_src

Allocate 2 MiB of memory in bash.

#+begin_src shell-script
  MY_SHELL_VAR="$(yes | dd bs=1024 count=2048)"
  free -h # See how much memory is available in the system.
#+end_src

Now, try to allocate more.  Create a few more variables with 2 MiB of text each,
calling =free -h= every time.

*Prepare screenshot 7: output of a =free -h= commands showing a lot of available
momery, followed by a failing bash command that exceeded process' own limit.*

* Login sessions limit
Use Ansible to install another file under =limits.d=.  This time set the
=maxlogins= limit to 3 for all members of group =users=.  This is an exceptional
type of limit — it applies to user accounts, not to individual sessions.

In separate terminal tabs, try SSH'ing multiple times as pancratius.

*Prepare screenshot 8: an SSH connection failing due to account's logins limit.*

* Capabilities
Use Ansible to install =dumpcap= executable.

#+begin_src yaml
  - name: Install `dumpcap' program.
    ansible.builtin.apt:
      name: wireshark-common
      install_recommends: false
#+end_src

Verify that, as root, you can use dumpcap to capture packets from the command
line.

#+begin_src shell-script
  dumpcap -i ens3 -w my-network-packets.dump # Interrupt to stop.
#+end_src

Verify that you cannot do this as one of the other users.

Dumpcap should print a message explaining what to do to enable non-root users to
capture packets.  The official method uses Debian's =dpkg-reconfigure= to add
capabilites to dumpcap's binary and change its owner group to =wireshark=.  When
executed, dumpcap then checks if the executing process is a member of the
=wireshark= group.  If so, it starts dumping packets.

Marrying curses-based =dpkg-reconfigure= with Ansible is notoriously hard.
Instead, we can add the necessary capabilities to the binary ourselves.

Use the last command suggested by failed dumpcap invokation.  Then, perform a
successful dump as =pancratius=.

#+begin_src shell-script
  dumpcap -i ens3 -w pancratius-packets.dump
#+end_src

You have added =CAP_NET_RAW= and =CAP_NET_ADMIN= to the permitted set of the
executable.  You have also set the "Effective" bit.

Dumping now works, but we're not leveraging the fact that =dumpcap= binary is
smart.  The program knows how to enable the necessary capabilities in case they
are initially only present in the "Permitted" set.

Run =getcap /usr/bin/dumpcap= to view the capabilities associated with the file.

Now, modify the earlier command to put =CAP_NET_RAW= and =CAP_NET_ADMIN= in the
"Permitted" set of =dumpcap= but *not to* set the "Effective" bit.

As root, run the following.

#+begin_src shell-script
  getcap
  sudo -u pancratius dumpcap -i ens3 -w /home/pancratius/pancratius-packets.dump
#+end_src

*Prepare screenshot 9: Output of the two commands above.  The binary lacks the
"Effective" capability bit, but unprivileged dumping still works.*

/Note that =dumpcap= installed from other sources might not enable the
capabilitied by itself.  This has been witnessed with Ubuntu 24's version./

* Capabilities and Tracing
Make sure =strace= is installed in the VM.  Run the following command to see how
=strace= uses =ptrace()= to inspect other process' syscalls and print the
information.

#+begin_src shell-script
  strace ls # Run `ls' under `strace'.
#+end_src

Now, try executing the privileged =dumpcap= binary under =strace=.  Once again,
use pancratius' unprivileged account.  Notice that binary's capabilities are not
acquired by the process when it is being traced.

*Prepare screenshot 10: Final output lines of the =strace= command used to trace
=dumpcap=.  The output should indicate missing capabilities.*

* Cgroups
As pancratius, start a CPU-consuming process.

#+begin_src shell-script
  yes > /dev/null
#+end_src

As root, view the process list on the server and see that the =yes= command
consumes 100% of the CPU.

#+begin_src shell-script
  top # Press `f`, select `CPU' as the sorting value and press `q'.
#+end_src

Note its PID.

Now, look into the main cgroup.  Look for the =yes= process' PID there.

#+begin_src shell-script
  tail /sys/fs/cgroup/cgroup.procs
  wc -l /sys/fs/cgroup/cgroup.procs
  grep $YES_PID /sys/fs/cgroup/cgroup.procs # Not present here.
#+end_src

The process must be in one of the child / grandchild cgroups.  Find the PID of
=yes= in one of the =cgroup.procs= files under =/sys/fs/cgroup/user.slice/=.

Now, create a new cgroup.

#+begin_src shell-script
  mkdir /sys/fs/cgroup/naughty
  cat /sys/fs/cgroup/naughty/cgroup.procs
#+end_src

Write the pid of =yes= to =cgrop.procs= to tell the kernel to move it into this
cgroup.

Verify that the process is now indeed considered a member of the new cgroup.

#+begin_src shell-script
  cat /proc/142778/cgroup
#+end_src

Check that the =cpu= controller used to manage processor usage is supported by
the kernel and available to the cgroup.

#+begin_src shell-script
  cat /sys/fs/cgroup/naughty/cgroup.controllers
#+end_src

See the measured *total* CPU usage by =yes=.

#+begin_src shell-script
  cat /sys/fs/cgroup/naughty/cpu.stat
#+end_src

Check the current cgroup limit on CPU usage.  The first value designates the
limit while the second one — the time period considered when applying the limit.

#+begin_src shell-script
  cat /sys/fs/cgroup/naughty/cpu.max
#+end_src

Configure a 50% limit by writing to the file.  Replace value =max= with half of
the period value.

Use =top= again to see if CPU usage by =yes= dropped to 50%.

*Prepare screenshot 11: output of =top= with the CPU usage by =yes= reported as
about 50% or less.*

* System-wide Limits Using Cgroups
The init system (the initial process that starts (typically) all services in the
system) used by Debian is systemd.  It also does *MANY* (😉) other things and
can be configured to automatically apply limits to cgroups in which users'
processes are put.

#+begin_src shell-script
  # Documentation on setting limits with systemd + cgroups.
  man systemd.resource-control
#+end_src

Systemd associates groups of processes with units that it calls "slices".
Slices correspond to, among others, process groups of logged in users.  We can
configure limits by putting configuration files in the respective slices'
directories under =/etc/systemd=.

Use Ansible to create a file
=/etc/systemd/system/user-.slice.d/50-Stop-naughty-processes.conf= with the
following contents.

#+begin_src conf
  [Slice]
  CPUQuota=30%
  CPUQuotaPeriodSec=100ms
#+end_src

Notice that there is no user id after the minus sign in directory name
=user-.slice.d=.  The file shall be used by all user slices.

Make Ansible invoke =systemctl daemon-reload= each time it changes the file.

Once the file is there and =systemctl daemon-reload= has been invoked, log out
and log back in as pancratius.  View his cgroup's =cpu.max= file to see that
the limits are now applied automatically.

Using Ansible, add a similarly named file under =user-0.slice.d= but with 2x
higher limits.  Make =daemon-reload= happen afterwards.

Use the following command to verify that the configuration works as intended and
results in higher limits just for the root processes.

#+begin_src shell-script
grep . $(find /sys/fs/cgroup/user.slice -name "cpu.max")
#+end_src

*Prepare screenshot 12: output of the commands above.*

* The Fork Bomb and PID Limits
As pancratius, run another =yes > /dev/null= if you have already killed the
previous =yes= process.  Then, write 1 to the =cgroup.freeze= file of the =yes=
process' cgroup.  Verify in =top= that the =yes= process no longer consumes
*any* CPU.

As root, add pancratius' =bash= process to the =naughty= cgroup.  As pancratius,
stop the =yes= process and, in the command line, execute the fork bomb and watch
the system freeze as dozens of new processes get created by bash.

#+begin_src shell-script
:()(:|:);:
#+end_src

Terminate the VM and restart the system.  You'll now use a cgroup to terminate a
fork bomb process tree.

Log in as pancratius and, as root, check the maximum allowed number of
processes.  Decrease it to 20 by writing to a =pids.max= file of the cgroup.

#+begin_src shell-script
  cat /sys/fs/cgroup/user.slice/user-$PANCRATIUS_UID.slice/pids.max
#+end_src

Now, once again execute a fork bomb as pancratius.  This time do it in a loop.

#+begin_src shell-script
  :()(:|:);while true; do :;done
#+end_src

It should not break the system this time.  See all the bash process PIDs.

#+begin_src shell-script
  cat /sys/fs/cgroup/user.slice/user-$PANCRATIUS_UID.slice/session-*.scope/cgroup.procs
#+end_src

Now, kill them all at once.

#+begin_src shell-script
  echo 1 > /sys/fs/cgroup/user.slice/user-$PANCRATIUS_UID.slice/cgroup.kill
#+end_src

Use the earlier =cat= command and see that the empty cgroup got automatically
removed by systemd.

*Prepare screenshot 13: Output of the two =cat= commands and the =echo= between
them.*

* Extra (for Bonus Points!)
Configure Ansible Vault to store user password hashes outside of your playbook
file (which you could then share).

Follow the advice in the warning and change the system to use ext4 internal
quota instead of =aquota.user= and =aquota.group= files.