path: root/lectures/02-filesystems.org

#+title: Filesystems
#+date: 2026-03-09 Mon
#+author: W. Kosior
#+email: wkosior@agh.edu.pl

* Filesystem
- abstraction over memory
- storing
  - files
  - directories

* Fileystem, Cont.
- abstraction over memory
- storing
  - files
  - directories
  - metadata
    - permissions
    - attributes
    - filesystem label
    - filesystem UUID

* Fileystem, Cont…
- abstraction over memory
- storing
  - files
  - directories
  - metadata
    - permissions
    - attributes
- failure resistance
  - journaled filesystems…
  - …or CoW
- snapshots
- compression
- encryption
- …

* Popular Filesystems
- FAT (12-bit, 16-bit, 32-bit)
- Ext (2, 3, 4)
- NTFS

* Popular Filesystems, Cont.
- FAT (12-bit, 16-bit, 32-bit)
- Ext (2, 3, 4)
- NTFS
- XFS (…)
- UFS (…)
- HFS+
- APFS
- exFAT

* Popular Filesystems, Cont…
- FAT (12-bit, 16-bit, 32-bit)
- Ext (2, 3, 4)
- NTFS
- XFS (…)
- UFS (…)
- HFS+
- APFS
- CoW filesystems
  - ZFS
  - OpenZFS
  - BTRFS

* Popular Read-Only Filesystems
- ISO 9660
- SquashFS
- cpio archives (initramfs)

* Filesystem Placement Options
- directly on physical medium
- on partition
- on logical volume
- on encrypted volume
- on a RAID volume
- on file on another filesystem

* Filesystem Placement Options, Cont.
- directly on physical medium
- on partition
- on logical volume
- on encrypted volume
- on a RAID volume
- on file on another filesystem
- stacking, e.g.:
  - RAID + encryption + filesystem
  - partition + logical volume management + filesystem

* Partitioning
#+begin_example
                                                          backup
partition                                               partition
  table                                                   table
    |                                                       |
    |                                                       |
    V                                                       V
 +----+--------------------+------------+----------------+----+
 | PT |    part. 3 data    | free space |  part. 1 data  | PT |
 +----+--------------------+------------+----------------+----+
#+end_example

* Partitioning
- why?
  - multiple OSes
  - separate filesystems for different OS parts
    - logs, user directories & main system files
    - main system & recovery
    - swap partition
  - boot process / firmware requirements
- partition table formats
  - MBR
  - GPT
- resising /(worth realizing)/
  - resize partition
  - resize filesystem

* Logical Volume Management
#+begin_example
+--------+       +--------------+
| disk 1 | ----> |              |       +------------------+
+--------+       |              | ----> | logical volume 1 |
                 |              |       +------------------+
+--------+       |              |
| disk 2 | ----> | storage pool |
+--------+       |              |
                 |              |       +------------------+
+--------+       |              | ----> | logical volume 2 |
| disk 3 | ----> |              |       +------------------+
+--------+       +--------------+
#+end_example

* Popular Volume Encryption Tools
| encryption technology | platform                     |
|-----------------------+------------------------------|
| dm-crypt              | Linux kernel & DragonFly BSD |
| LUKS                  | Linux kernel & DragonFly BSD |
| softraid + CRYPTO     | OpenBSD                      |
| VeraCrypt             | /cross-platform/             |

* Popular Encrypted Filesystems
- APFS
- NTFS (BitLocker)
- ZFS

* Linux Unified Key Setup (LUKS), version 2
#+begin_example
+--------+--------------------------------------------+
| header |               encrypted data               |
+--------+--------------------------------------------+
#+end_example

- header:
  - up to 32 user keys
  - master key
    - encrypted with user keys
- passphrase / smart card / keyfile ----> PBKDF2 ----> user key
  - PBKDF2: PKCS #5, RFC 2898
- TRIM support
- support in GRUB2

* Disk Encryption Considerations
- encrypted boot partition?
  - Coreboot + GRUB2
- stacking
  - "filesystem on LVM on LUKS"
    - *encrypted swap* as separate volume or file
  - "filesystem on LUKS on LVM"
    - *encrypted swap* as file
- header erasure = data deletion
  - rogue SSD firmware saving LUKS headers?
- disk encryption on servers?
  - password on power-up?
  - easy data deletion :)

* RAID
| operation                       | name     | notes                    |
|---------------------------------+----------+--------------------------|
| striping                        | RAID 0   | speed                    |
| mirroring                       | RAID 1   | safety                   |
| mirroring+striping              | RAID 1+0 | speed+safety             |
| bit-level striping with parity  | RAID 2   | not popular              |
| byte-striping with parity       | RAID 3   | not popular              |
| striping with parity            | RAID 4   | dumber variant of RAID 5 |
| like RAID 4, distributed parity | RAID 5   | speed+safety             |
| like RAID 5, extra parity block | RAID 6   | speed+safety             |

* RAID 0
#+begin_example
             +-----+-----+-----+-----+-----+-----+
stored data: | A1  | A2  | A3  | A4  | A5  | A6  |
             +-----+-----+-----+-----+-----+-----+

             +-----+-----+-----+
on drive 1:  | A1  | A3  | A5  |
             +-----+-----+-----+

             +-----+-----+-----+
on drive 2:  | A2  | A4  | A6  |
             +-----+-----+-----+
#+end_example

* RAID 1
#+begin_example
             +-----+-----+-----+-----+-----+-----+
stored data: | A1  | A2  | A3  | A4  | A5  | A6  |
             +-----+-----+-----+-----+-----+-----+

             +-----+-----+-----+-----+-----+-----+
on drive 1:  | A1  | A2  | A3  | A4  | A5  | A6  |
             +-----+-----+-----+-----+-----+-----+

             +-----+-----+-----+-----+-----+-----+
on drive 2:  | A1  | A2  | A3  | A4  | A5  | A6  |
             +-----+-----+-----+-----+-----+-----+
#+end_example

* RAID 1+0
#+begin_example
             +-----+-----+-----+-----+-----+-----+
stored data: | A1  | A2  | A3  | A4  | A5  | A6  |
             +-----+-----+-----+-----+-----+-----+

             +-----+-----+-----+
on drive 1:  | A1  | A3  | A5  |
             +-----+-----+-----+

             +-----+-----+-----+
on drive 2:  | A1  | A3  | A5  |
             +-----+-----+-----+

             +-----+-----+-----+
on drive 3:  | A2  | A4  | A6  |
             +-----+-----+-----+

             +-----+-----+-----+
on drive 4:  | A2  | A4  | A6  |
             +-----+-----+-----+
#+end_example

* RAID 4
#+begin_example
             +-----+-----+-----+-----+-----+-----+
stored data: | A1  | A2  | B1  | B2  | C1  | C2  |
             +-----+-----+-----+-----+-----+-----+

             +-----+-----+-----+
on drive 1:  | A1  | B1  | C1  |
             +-----+-----+-----+

             +-----+-----+-----+
on drive 2:  | A2  | B2  | C2  |
             +-----+-----+-----+

             +-----+-----+-----+
on drive 3:  | Ap  | Bp  | Cp  |
             +-----+-----+-----+
#+end_example

* RAID 5
#+begin_example
             +-----+-----+-----+-----+-----+-----+
stored data: | A1  | A2  | B1  | B2  | C1  | C2  |
             +-----+-----+-----+-----+-----+-----+

             +-----+-----+-----+
on drive 1:  | A1  | B1  | Cp  |
             +-----+-----+-----+

             +-----+-----+-----+
on drive 2:  | A2  | Bp  | C2  |
             +-----+-----+-----+

             +-----+-----+-----+
on drive 3:  | Ap  | B2  | C1  |
             +-----+-----+-----+
#+end_example

* RAID 6
#+begin_example
             +-----+-----+-----+-----+-----+-----+-----+-----+
stored data: | A1  | A2  | B1  | B2  | C1  | C2  | D1  | D2  |
             +-----+-----+-----+-----+-----+-----+-----+-----+

             +-----+-----+-----+-----+
on drive 1:  | A1  | B1  | Cp  | Dq  |
             +-----+-----+-----+-----+

             +-----+-----+-----+-----+
on drive 2:  | A2  | Bp  | Cq  | D2  |
             +-----+-----+-----+-----+

             +-----+-----+-----+-----+
on drive 3:  | Ap  | Bq  | C1  | D1  |
             +-----+-----+-----+-----+

             +-----+-----+-----+-----+
on drive 4:  | Aq  | B2  | C2  | Dp  |
             +-----+-----+-----+-----+
#+end_example

* Many-in-One Filesystems
| fs            | compression* | encryption | logical volumes | snapshots | RAID |
|---------------+--------------+------------+-----------------+-----------+------|
| ZFS / OpenZFS | ✔            | ✔          | ✔               | ✔         | ✔    |
| BTRFS         | ✔            | x          | ✔               | ✔         | ✔    |
| NTFS          | ✔            | ✔          | x               | ✔         | x    |
| …                                                                              |

‍* /All *but* compression can be achieved by stacking LVM and other devices./

* Mounting Filesystems
- under drive letter
  - e.g., =C:\=, =E:\=, =F:\=
- under mount point
  - e.g. =/=, =/home=, =/var/log=
  - hiererchical filesystem (VFS)
    - =/= — root
  - bread and butter of UNIX
  - also achievable under Windows

* Special Filesystems / Mounts (Linux)
- tmpfs / ramfs
- devtmpfs / procfs / sysfs / efivarsfs

#+begin_src shell-script
  sudo mount -t procfs /mnt/proc/
#+end_src

* Special Filesystems / Mounts (Linux), cont.
- tmpfs / ramfs
- devtmpfs / procfs / sysfs / efivarsfs
- OverlayFS / Unionfs / aufs / …
- network filesystems
- bind mounts

* OverlayFS
#+begin_example
                                             create
                                               |
                                               V
merged directory:  a.txt  b.txt  dir/c.txt   n.txt
                                               |
                                               V
upper directory:          b.txt              n.txt


lower directory:   a.txt         dir/c.txt
#+end_example

* Bind Mounts
- =bind=
- =rbind=
  - propagate mounts underneath

#+begin_src shell-script
  sudo mount --bind ./my-var ./my-container/var
  cat /proc/mounts
  ls ./my-container/var/
#+end_src

* Special Filesystems / Mounts (Linux), cont…
- tmpfs / ramfs
- devtmpfs / procfs / sysfs / efivarsfs
- OverlayFS / Unionfs / aufs / …
- network filesystems
- bind mounts
- network filesystems

* Everything is a file
- devtmpfs / procfs / sysfs / efivarsfs
- Plan 9

* Popular Network Filesystems
- NFS
- CIFS
- SMB
- WebDAV
- Ceph
- arbitrary filesystems on network-shared block devices

* Special Filesystems / Mounts (Linux), cont…
- tmpfs / ramfs
- devtmpfs / procfs / sysfs / efivarsfs
- OverlayFS / Unionfs / aufs / …
- network filesystems
- bind mounts
- network filesystems
- FUSE

* FUSE
- +filesystem driver in the kernel+
- *filesystem driver in userspace*
  - unprivileged mounts (non-root)
  - third-party filesystems
  - overcoming driver licensing issues
  - lower driver code quality requirements
- since Linux 2.6.14 (September 2005)
- inspired by microkernel OS design
- also in BSDs
- third party implementations for macOS & Windows

* Mount options
- ro / rw
- nosuid, acl
- uid, gid (substitute ownership for, e.g., FAT filesystem files)
- more (often fs-specific)

* Root Filesystem Mounting
#+begin_example
            1. load              1. mount   2. execute
bootloader ---------> OS kernel ----------+------------+
  (GRUB)               (Linux)            |            |
                                          |            |
                                          V            |
                                   disk filesystem     |
                                                       V
                                                   /sbin/init
#+end_example

* Root Filesystem Mounting (Initramfs)
#+begin_example
              1. load
bootloader -+---------> OS kernel -------------------------+
  (GRUB)    |            (Linux)                           |
            |                          3. extract to ramfs |
            +-------------> initrd <-----------------------+
              2. load   (cpio archive)                     |
                                              4. execute   |
                               /sbin/init <----------------+
                                   |
    5. load fs / LVM / RAID driver |         6. mount
   --------------------------------+---------------------------+
                                   | (e.g., under `/new-root') |
            7. chroot / pivot_root |                           V
/new-root <------------------------+                    disk filesystem
                                   | 8. execute
                                   +------------> /usr/sbin/init
#+end_example

* Initramfs
- replacement for earlier initrd
  - still called "initrd" sometimes
- re-genarated upon kernel updates
  - manually: =mkinitramfs= script
- tools needed to boot, e.g.
  - kernel modules
  - shell
  - mdadm (for RAID / LVM)
  - =cryptsetup= tool (for LUKS)
- customization ---> scripts

* Initramfs Customization
1. create / edit =/etc/initramfs-tools/hooks/local-bottom/my-custom=
   - script to overlay =/new-root= with devtmpfs
2. =mkinitramfs=
3. reboot to your ephemeral OS

* Initramfs Customization Use-Cases
- ephemeral OS (changes not written to disk)
- diskless system
  - GRUB, kernel, initramfs & root filesystem on network
- authentication / authorization before mounting root filesystem

* =pivot_root=
- before:
  - =/= — ramfs
  - =/new-root= — ext4 filesystem from partition 3 on drive 1
- after:
  - =/= — ext4 filesystem from partition 3 on drive 1
  - =/new-root= — ramfs
- only affects the calling process & thread

* =chroot=
#+begin_src shell-script
  mount /dev/sdb1 /mnt # System under repair.
  mount -t proc my-proc /mnt/proc
  mount -t devtmpfs my-dev /mnt/dev
  mount -t sysfs my-sys /mnt/sys
  mount --bind /tmp /mnt/sys/tmp
  chroot /mnt
  echo 'Hello from the inside!'
#+end_src

* =chroot=
- > 40 years old
- precursor to modern container schemes
- *not* a (strong) security mechanism
  - escape method ≠ Linux kernel bug
- indispensible UNIX admin's tool
  - system repairs
  - VM-less OS image preparation
  - foreign binaries — userspace-only virtualization