Linux
From Wikipedia, the free encyclopedia
Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).
Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.
Rules
- Posts must be relevant to operating systems running the Linux kernel. GNU/Linux or otherwise.
- No misinformation
- No NSFW content
- No hate speech, bigotry, etc
Related Communities
Community icon by Alpár-Etele Méder, licensed under CC BY 3.0
There is no such thing as "distro agnostic" when you're describing the thing that makes a distribution what it is. That's like saying "I'm trying to make a package manager that integrates with all other package managers".
Why would you do that when they already have working package managers?
What you're actually describing in sharing layers and state changes is not possible the way you want, at least not with generic systems. In order to share a layer that is applicable to another system, you'd have to ensure that every single piece of that system is exactly the same, from the BIOS up. Think phones getting updates, as that's very similar.
Not really as you can have a separate boot partition with automatic bootloader updating. One user suggested using rsync to pull any changes. If I'm doing that already I could just have a post install script that gets the system ready.
All you really have to do for that is mount the partition readonly, and have a designated writable data partition for the rest. That can be as simple as setting it ro in your fstab.
How you ship updates can take many forms. If you don't need your distro atomic, you can temporarily remount readwrite, rsync the new version over and make it readonly again. If you want it atomic, there's the classic A/B scheme (Android, SteamOS), where you just download the image to the inactive partition and then just switch over when it's ready to boot into. You can also do btrfs/ZFS snapshots, where the current system is forked off a snapshot. On your builder you just make your changes, then take a snapshot, then zfs/btrfs send it as a snapshot to all your other machines and you just boot off that new snapshot (readonly). It's really not that magic: even Docker, if you dig deep enough, it's just essentially tarballs being downloaded then extracted each in their own folder, and the layering actually comes from stacking them with overlayfs. What rpm-ostree does, from a quick glance at the docs, is they leverage the immutability and just build a new version of the filesystem using hardlinks and you just switch root to it. If you've ever opened an rpm or deb file, it's just a regular tarball and the contents pretty much maps directly to the filesytem.
Here's an Arch package example, but rpm/deb are about the same:
max-p@desktop /v/c/p/aur> tar -tvf zfs-utils-2.2.6-3-x86_64.pkg.tar.zst
-rw-r--r-- root/root 114771 2024-10-13 01:43 .BUILDINFO
drwxr-xr-x root/root 0 2024-10-13 01:43 etc/
drwxr-xr-x root/root 0 2024-10-13 01:43 etc/bash_completion.d/
-rw-r--r-- root/root 15136 2024-10-13 01:43 etc/bash_completion.d/zfs
-rw-r--r-- root/root 15136 2024-10-13 01:43 etc/bash_completion.d/zpool
drwxr-xr-x root/root 0 2024-10-13 01:43 etc/default/
-rw-r--r-- root/root 4392 2024-10-13 01:43 etc/default/zfs
drwxr-xr-x root/root 0 2024-10-13 01:43 etc/zfs/
-rw-r--r-- root/root 165 2024-10-13 01:43 etc/zfs/vdev_id.conf.alias.example
-rw-r--r-- root/root 166 2024-10-13 01:43 etc/zfs/vdev_id.conf.multipath.example
-rw-r--r-- root/root 616 2024-10-13 01:43 etc/zfs/vdev_id.conf.sas_direct.example
-rw-r--r-- root/root 152 2024-10-13 01:43 etc/zfs/vdev_id.conf.sas_switch.example
-rw-r--r-- root/root 254 2024-10-13 01:43 etc/zfs/vdev_id.conf.scsi.example
drwxr-xr-x root/root 0 2024-10-13 01:43 etc/zfs/zed.d/
...
It's beautifully simple. You could for example install ArchLinux without pacman, by mostly just tar -x the individual package files directly to /. All the package manager does is track which file is owned by which package (so it's easier to remove), and dependency solving so it knows to go pull more stuff or it won't work, and mirror/download management.
How you get that set up is all up to you. Packer+Ansible can make you disk images and you can maybe just throw them on a web server and download them and dd them to the inactive partition of an A/B scheme, and that'd be quite distro-agnostic too. You could build the image as a Docker container and export it as a tarball. You can build a chroot. Or a systemd-nspawn instance. You can also just install a VM yourself and set it up to your liking and then just dd the disk image to your computers.
If you want some information on how SteamOS does it, https://iliana.fyi/blog/build-your-own-steamos-updates/
I like the idea of rsync as it is simple. I could combine rsync with btrfs to use snapshots are the working system and the current r/w system as the part that gets updated.
However, I think it would be good to use OStree as it has features like the ability to manage boot including auto rollbacks and easy switching between states.
auto rollbacks and easy switching between states.
That's the beauty of snapshots, you can boot them. So you just need GRUB to generate the correct menu and you can boot any arbitrary version of your system. On the ZFS side of things there's zfsbootmenu, but I'm pretty sure I've seen it for btrfs too. You don't even need rsync, you can use ssh $server btrfs send | btrfs recv and it should in theory be faster too (btrfs knows if you only modified one block of a big file).
and the current r/w system as the part that gets updated.
That kind of goes against the immutable thing. What I'd do is make a script that mounts a fork of the current snapshot readwrite into a temporary directory, chroot into it, install packages, exit chroot, unmount and then commit those changes as a snapshot. That's the closest I can think of that's easy to DIY that's basically what rpm-ostree install does. It does it differently (daemon that manages hardlinks), but filesystem snapshots basically do the same thing without the extra work.
However, I think it would be good to use OStree
I found this, maybe it'll help: https://ostreedev.github.io/ostree/adapting-existing/
It looks like the fundamental is the same, temporary directory you run the package manager into and then you commit the changes. So you can probably make it work with Debian if you want to spend the time.