Free LES Community Server from Hosteroid via MetalVPS!

Not_Oles · January 7

I heard the the filesystem code lives in the fs directory. So I took a look.

Hey! We can see ext4 in there! And xfs! And btrfs! If I understand right, the ZFS code is not present, but needs to be added via a kernel module.

root@manassas:/usr/local/src# cd linux
root@manassas:/usr/local/src/linux# ls
arch     CREDITS        fs        ipc      lib          mm      samples   tools
block    crypto         include   Kbuild   LICENSES     net     scripts   usr
certs    Documentation  init      Kconfig  MAINTAINERS  README  security  virt
COPYING  drivers        io_uring  kernel   Makefile     rust    sound
root@manassas:/usr/local/src/linux# cd fs; ls
9p                   dcache.c       fs_types.c          namei.c           remap_range.c
adfs                 debugfs        fs-writeback.c      namespace.c       romfs
affs                 devpts         fuse                netfs             select.c
afs                  direct-io.c    gfs2                nfs               seq_file.c
aio.c                dlm            hfs                 nfs_common        signalfd.c
anon_inodes.c        d_path.c       hfsplus             nfsd              smb
attr.c               drop_caches.c  hostfs              nilfs2            splice.c
autofs               ecryptfs       hpfs                nls               squashfs
backing-file.c       efivarfs       hugetlbfs           notify            stack.c
bad_inode.c          efs            init.c              nsfs.c            stat.c
bcachefs             erofs          inode.c             ntfs3             statfs.c
befs                 eventfd.c      internal.h          ocfs2             super.c
bfs                  eventpoll.c    ioctl.c             omfs              sync.c
binfmt_elf.c         exec.c         iomap               open.c            sysctls.c
binfmt_elf_fdpic.c   exfat          isofs               openpromfs        sysfs
binfmt_flat.c        exportfs       jbd2                orangefs          sysv
binfmt_misc.c        ext2           jffs2               overlayfs         tests
binfmt_script.c      ext4           jfs                 pidfs.c           timerfd.c
bpf_fs_kfuncs.c      f2fs           Kconfig             pipe.c            tracefs
btrfs                fat            Kconfig.binfmt      pnode.c           ubifs
buffer.c             fcntl.c        kernel_read_file.c  pnode.h           udf
cachefiles           fhandle.c      kernfs              posix_acl.c       ufs
ceph                 file.c         libfs.c             proc              unicode
char_dev.c           filesystems.c  lockd               proc_namespace.c  userfaultfd.c
coda                 file_table.c   locks.c             pstore            utimes.c
compat_binfmt_elf.c  freevxfs       Makefile            qnx4              vboxsf
configfs             fs_context.c   mbcache.c           qnx6              verity
coredump.c           fsopen.c       minix               quota             xattr.c
cramfs               fs_parser.c    mnt_idmapping.c     ramfs             xfs
crypto               fs_pin.c       mount.h             readdir.c         zonefs
dax.c                fs_struct.c    mpage.c             read_write.c
root@manassas:/usr/local/src/linux/fs#

Not_Oles · January 8

I hear from Google Gemini that the contents of the fs directory includes:

The Virtual File System Layer (VFS) -- abstracts common operations across various file systems to enable presenting a unified interface to user applications,
Specific File System Implementations -- btrfs, ext4, xfs, etc. -- each file systems implementation contains a "file_operations" structure with pointers to the filesystem-specific functions which implement user-level system call operations, such as read, write, open, and ioctl, and
Additional, related "helper" implementations -- e.g., read-write.c which handles certain generalized parts of read/write operations such as buffering and page-caching. For example, the functions in read-write.c might be used by multiple specific file system implementations.

I asked Google Gemini to organize the files within fs by listing the top level functions together with the files which implement each of the top level functions. Here below is Gemini's list. I commented a few places where Gemini's list didn't seem in accordance with Linus' HEAD.

Key Subsystems and Representative Files (fs/ directory):

Virtual Filesystem (VFS) Core: (Provides the common interface and infrastructure for all filesystems)
- fs/dcache.c: Directory entry cache (dentry).
- fs/inode.c: Inode management (core inode operations).
- fs/namei.c: Pathname lookup (resolving paths to inodes).
- fs/file.c: File object management (representing open files).
- fs/stat.c: File status information (stat() system call).
- fs/open.c: File opening (open() system call).
- fs/read_write.c: Core read/write operations and helpers (often used by specific filesystems). Not the primary entry point, but provides important utilities.
- fs/super.c: Superblock management (representing mounted filesystems).
- fs/vfs_kern.c: Miscellaneous VFS kernel functions. # No vfs_kern.c
- fs/libfs.c: Helper functions for various filesystem operations.
- fs/mount.c: Mounting and unmounting filesystems. # No mount.c. There is mount.h.
Specific Filesystem Implementations: (Provide the concrete implementation for each filesystem type)
- fs/ext4/: (Entire directory) Ext4 filesystem.
- fs/xfs/: (Entire directory) XFS filesystem.
- fs/btrfs/: (Entire directory) Btrfs filesystem.
- fs/fat/: (Entire directory) FAT filesystem.
- fs/ntfs/: (Entire directory) NTFS filesystem support.
- ... other filesystem directories ... (Each filesystem has its own subdirectory)
Buffer and Page Caching: (Manage caching of disk blocks and file data in memory)
- fs/buffer.c: Buffer cache management (for disk blocks).
- fs/pagemap.c: Page cache management (for file data). # No pagemap.c
- fs/bio.c: Block I/O layer. Handles submission and completion of block I/O requests. #No bio.c
Quota Management: (Limit disk usage by users and groups)
- fs/quota/: (Entire directory) Disk quota management.
File Locking: (Mechanisms for coordinating access to files)
- fs/locks.c: File locking (e.g., flock, fcntl locks).
Virtual Filesystems: (Provide filesystem-like interfaces to kernel resources)
- fs/proc/: (Entire directory) procfs (process information).
- fs/sysfs/: (Entire directory) sysfs (kernel parameters and system information).
- fs/debugfs.c: debugfs (kernel debugging).
- fs/tmpfs/: (Entire directory) tmpfs (RAM-based filesystem). # No tmpfs subdirectory
Namespaces: (Isolate filesystems and other resources)
- fs/nsfs.c: Namespace support (mount namespaces).
Other Utilities and Helpers: (Various supporting functions and modules)
- fs/ioctl.c: ioctl system calls (device-specific operations).
- fs/exec.c: Program execution (related to the execve() system call).
- fs/fs-writeback.c: Writeback mechanism (flushing dirty data to disk).
- fs/char_dev.c: Character device management (though some character device drivers might be elsewhere).
- fs/fs_struct.c: Per-process filesystem information.

I'm curious why the fs directory doesn't have subdirectories organized according to function. For example, there could be fs/vfs-core, fs/implementations, fs/caching, etc.

Apparently BSD kernels also have a Virtual Filesystem Layer ("VFS"). How early in Unix did the VFS layer appear? Was a VFS common in pre-Unix operating systems?

It's easy to imagine that ZFS conceivably might have its own implementation of some of the helper systems or even its own version of the VFS, and that these within-the-specific-filesystem implementations might have contributed to possible "difficulty" of including ZFS in the Linus kernel proper. I'm curious why ZFS wasn't included. Do we know?

Is anybody aware of a well recognized, introductory Linux kernel or BSD source code discussion which follows the "overview / top-down" style of analysis which I am trying to use here?

Is anybody up on the mistakes Google Gemini apparently made? Could these be as simple as, "Oh, that's version x.xx?"

Thanks for any help on any of these questions! Thanks for additional observations and comments!

Does anybody else want to join the server? If yes, please check How to Apply in the OP. Thanks @Hosteroid!

quangthang · January 8

Hello!
I'd like a slice for experimenting with NixOS and hosting some open-source projects (Vaultwarden, Whoogle search, Openwebui). If you need any additional infos about me, I'll happy to provide through PM.
My ssh public key is ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHZ7KokkDS4XU9M15R3htHbt4ZJ9NQeYxVbKWinbE3n5.
Thank you!

hobofl · January 8

@Not_Oles said: I'm curious why ZFS wasn't included. Do we know?

Not being that familiar with the history of the relevant discussions I don't know if there are also other (technical, political, social, whatever) reasons, but the reason that is usually given is the legal uncertainty of GPLv2 (the Linux license) and CDDL (the ZFS license) compatibility. Here's one summary, with links to other discussion (including opposing opinions):

https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/

Not_Oles · January 8

@quangthang said:
Hello!
I'd like a slice for experimenting with NixOS and hosting some open-source projects (Vaultwarden, Whoogle search, Openwebui). If you need any additional infos about me, I'll happy to provide through PM.
My ssh public key is ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHZ7KokkDS4XU9M15R3htHbt4ZJ9NQeYxVbKWinbE3n5.
Thank you!

Hi @quangthang!

Thanks for your request, which makes me happy! Of course, I definitely do remember you from previous servers, and so I am very delighted to have you joining hlcs!

I sent you login info. You are sharing root with me and with @cmeerw.

Please feel free to do whatever you want as long as it's White Hat. Please post here about what you are doing and about any questions you have.

Always best wishes!

Tom

Thanks to @Hosteroid for our very nice server!

quangthang · January 9

Hello everyone.
I just got around to do some basic setup (set up user account, change to fish shell).

BashVM is installed

It seems like it's not installed globally. I cloned the repo into my home dir, ran the install script again but it mostly skipping cause required pkgs and configs are already installed

...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

dnsmasq is already installed.

Default storage pool is already active.

Default network is already active.
...

I'm gonna go ahead and see if I can make a NixOS VM.

quangthang · January 9

VM created, now to the installation. Not sure if it's my part but i'm getting constant vnc timed out

quangthang · January 9

The installation process was successful but it's stuck on this screen. I've must have done something wrong. I think I'll grab the graphical iso to use their graphical installer.

quangthang · January 9

It's up!

quangthang · January 9

One of dependencies on my config keep failing build. Which is weird cause it built just fine in other instances and my local laptop. Anyway, I'm going to call it a day.

cmeerw · January 9

@quangthang said:

One of dependencies on my config keep failing build. Which is weird cause it built just fine in other instances and my local laptop. Anyway, I'm going to call it a day.

Isn't that just a timing thing, i.e. it took 6 ms too long?

Not_Oles · January 9

@hobofl said:

@Not_Oles said: I'm curious why ZFS wasn't included. Do we know?

Not being that familiar with the history of the relevant discussions I don't know if there are also other (technical, political, social, whatever) reasons, but the reason that is usually given is the legal uncertainty of GPLv2 (the Linux license) and CDDL (the ZFS license) compatibility. Here's one summary, with links to other discussion (including opposing opinions):

https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/

Hi @hobofl!

Sorry about my delayed response to your comment. I wanted to wait until I had a chance to look at the SFConservancy link you provided.

I really appreciate your comment for at least four reasons.

First, your comment concisely and directly answers the question I asked and not some other question.
Second, your comment provides excellent context: you sketched a typology of possible reasons why some software project might or might not be included in another software project: licensing, technical, social, political, other.
Third, your comment includes a link to relevant and comprehensive further discussion from multiple, inconsistent points-of-view.
Fourth, the tone of your comment is friendly and positive and helpful instead of negative and snarky.

Now that, thanks to you, I might understand a little better, I have to decide what to do.

It seems like Debian and the Conservancy and Oracle might have arrived, for the time being, at somewhat of a compromise regarding the use of ZFS on Linux. I guess that compromise might be good enough for me to continue glancing at ZFS on Debian on our Community Server. Nevertheless, I now imagine that I might want a really important reason before continuing with ZFS beyond an educational glance. I might want some specific context where ZFS clearly would be a better choice than any of the other available file systems.

Thanks again for your really helpful comment! Thanks to @Hosteroid for our beautiful Community Server!

Not_Oles · January 9

Hi @quangthang! I've seen your posts above but I haven't had a chance to read them carefully. Luckily @cmeerw was here and kindly responded! I'm glad you got in to the server! Yaaay!

Not_Oles · January 10

Hi again @quangthang!

I got a chance to read your posts and to look up a few things. I've never used NixOS, so I don't have anything to add to what you and @cmeerw already have said.

I hope you go ahead on the server and have a lot of fun! Best! <)

Thanks to @Hosteroid for the lovely server!

quangthang · January 10

@cmeerw said:

@quangthang said:

One of dependencies on my config keep failing build. Which is weird cause it built just fine in other instances and my local laptop. Anyway, I'm going to call it a day.

Isn't that just a timing thing, i.e. it took 6 ms too long?

Yes, It looks like the unit test is running performance evaluation. I don't expect it to fail by that small margin though 😭

terrorgen · January 10

Make sure you use the same NixOS channel checkout to eliminate the "it works in my other computers, just not this one" issue.

quangthang · January 10

@terrorgen said:
Make sure you use the same NixOS channel checkout to eliminate the "it works in my other computers, just not this one" issue.

Yes, I've basically copied my flake config over, with some boot loader config modification.

Not_Oles · January 10

If it's okay with @cmeerw, maybe @quangthang might like to try installing NixOS on the entire bare metal server? The server has an HTML5 IPMI available. We could then keep going with NixOS or change back to Debian or to something else later. Whatever people want is fine with me.

cmeerw · January 10

@Not_Oles said:
If it's okay with @cmeerw, maybe @quangthang might like to try installing NixOS on the entire bare metal server? The server has an HTML5 IPMI available. We could then keep going with NixOS or change back to Debian or to something else later. Whatever people want is fine with me.

I would prefer keeping the existing Debian installation on the server, and just installing NixOS onto its own logical volume to (dual-)boot into NixOS (so we could either relatively easily switch back to Debian or mount the existing data/home partition in NixOS), if @quangthang wants to run NixOS on bare metal.

Not_Oles · January 10

@quangthang said: BashVM is installed

It seems like it's not installed globally.

When I installed bashvm I wasn't sure where to put it. I wanted everybody to be able to read it, but I didn't want to change the root $PATH or the user $PATH.

What I did was put bashvm in /usr/local.

root@hlcs:~/.ssh# cd /usr/local
root@hlcs:/usr/local# ls
bashvm  bin  etc  games  include  lib  man  plan9  sbin  share  src
root@hlcs:/usr/local# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/pkg/bin:/usr/local/plan9/bin
root@hlcs:/usr/local# cd bashvm
root@hlcs:/usr/local/bashvm# ls
arm                            bashvm-dhcpv6-network-auto.sh     bashvm.sh
bashvm-add-port-forwarding.sh  bashvm-dhcpv6-network-manual.sh   bashvm-show-port-forwarding.sh
bashvm-create-auto-vm.sh       bashvm-installer.sh               LICENSE
bashvm-create-vm.sh            bashvm-monitor.sh                 README.md
bashvm-delete-auto-vm.sh       bashvm-remove-port-forwarding.sh
root@hlcs:/usr/local/bashvm#

Since everybody has root, maybe it might be okay to add bashvm to the $PATH?

root@hlcs:~# cp -p .bashrc .bashrc~
root@hlcs:~# ed .bashrc # This is GNU ed. Maybe I should use `9 ed`. /s 
769
a
PATH=$PATH:/usr/local/bashvm export PATH
.
w
810
q
root@hlcs:~# tail -n 5 .bashrc
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/pkg/bin export PATH
MANPATH=:/usr/pkg/man export MANPATH
PLAN9=/usr/local/plan9 export PLAN9
PATH=$PATH:$PLAN9/bin export PATH
PATH=$PATH:/usr/local/bashvm export PATH
root@hlcs:~# logout
Connection to xxx.xxx.xxx.xxx closed.
chronos@penguin:~/servers/hosteroid$ `head -n 1 login` # In bash, I type this as Ctrl-r ` /s
Linux hlcs.metalvps.com 6.1.0-28-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.119-1 (2024-11-22) x86_64
  [ . . . ]
Last login: Fri Jan 10 23:35:26 2025 from xxx.xxx.xxx.xxx
root@hlcs:~# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/pkg/bin:/usr/local/plan9/bin:/usr/local/bashvm
root@hlcs:~#

Not_Oles · January 10

@cmeerw said:

@Not_Oles said:
If it's okay with @cmeerw, maybe @quangthang might like to try installing NixOS on the entire bare metal server? The server has an HTML5 IPMI available. We could then keep going with NixOS or change back to Debian or to something else later. Whatever people want is fine with me.

I would prefer keeping the existing Debian installation on the server, and just installing NixOS onto its own logical volume to (dual-)boot into NixOS (so we could either relatively easily switch back to Debian or mount the existing data/home partition in NixOS), if @quangthang wants to run NixOS on bare metal.

@cmeerw Excellent! Thanks for letting us know!

cmeerw · January 11

Just did an apt-get dist-upgrade and a reboot to apply the Debian 12.9 updates.

Not_Oles · 3:17AM

Hello! I got another server temporarily so that I can mess with LVM without risk of causing disruption here. More on LVM in a little while.

If I understand right, @quangthang wants to try running Nix on ARM architecture, and our server is Intel.

@cmeerw Might you have plans to use our nice Community Server for another project any time before too long?

Does anyone else want to use our nice Community Server? Please check the OP section on How to Apply. Thanks!

Does anybody else (@cmeerw?) want to take over from me the administration of our nice Community Server?

If no LESbians want to use our server in the near future and if no one among us wants to take over its administration, maybe we should consider returning the server to Hosteroid?

Thanks @Hosteroid! Thanks LES!

cmeerw · 10:23AM

@Not_Oles said:
Hello! I got another server temporarily so that I can mess with LVM without risk of causing disruption here. More on LVM in a little while.

If I understand right, @quangthang wants to try running Nix on ARM architecture, and our server is Intel.

@cmeerw Might you have plans to use our nice Community Server for another project any time before too long?

Not sure... one thing that's on the back of my mind is the use of OpenStreetMap data - so maybe I could start looking at tilemaker. What's never clear with all of these OSM related projects is how much resources these projects need - so I guess you just have to try and see.

Does anyone else want to use our nice Community Server? Please check the OP section on How to Apply. Thanks!

Does anybody else (@cmeerw?) want to take over from me the administration of our nice Community Server?

Do you mean just the technical side or also the non-technical side (like user sign-ups)?

Free LES Community Server from Hosteroid via MetalVPS!

Comments