Why there is no review articles about dwarfs on the web, if it looks like better SquashFS?

[vitaly-zdanevich]

For me it looks strange. SquashFS somehow better?

Do you know some notable projects that use DwarFS? I know only Conty.

SquashFS is more popular because it integrated into Linux Kernel? Would you like to integrate DwarFS?

[mhx]

For me it looks strange. SquashFS somehow better?

SquashFS is almost 20 years older than DwarFS, so it is much more established. Better? I guess that really depends on the use case.

Do you know some notable projects that use DwarFS? I know only Conty.

There's (at least) also Tebako.

Then again, my primary use case for DwarFS is by itself, not through any other projects.

SquashFS is more popular because it integrated into Linux Kernel?

Possibly. But then again, even if it was in the Linux kernel, what about macOS and Windows?

Would you like to integrate DwarFS?

I personally would not like to integrate it. That's not because I wouldn't like it to be part of the kernel, but because I would rather spend my time improving DwarFS.

But about your original question...

Why there is no review articles about dwarfs on the web, if it looks like better SquashFS?

I have no idea! :)

It's discussed on HN about once every two years (which is by far what has been driving most people to this project) and the occasional article on other sites with rather insignificant reach.

I guess I just suck at marketing.

[vitaly-zdanevich]

I just suck at marketing.

Best marketing - is a good product.

[vitaly-zdanevich]

I added link to your repo at https://en.wikipedia.org/wiki/SquashFS#See_also

[mhx]

I guess here are some of the actual issues that might prevent DwarFS from being more popular:

• It's not (yet) available as a Debian/Ubuntu package.
• Using it on Windows is not for the faint of heart unless you're coming from a Unix / command-line background.
• Probably the dominant use case for SquashFS is live images. I honestly haven't seen SquashFS being used for anything else. That use case is significantly simplified by SquashFS being in the kernel for 20+ years. It seems that it should be possible to use FUSE-based file systems for live images (at least there's an option to disable FUSE in the live-boot docs). Then again, I'm not sure that DwarFS would be significantly better at that particular use-case. It would certainly compress better, but likely not by more than 20-30% (that's the numbers I got from a raspbian image). It might be worth trying to get it to work though! I can think of certain cases that would definitely benefit from it, e.g. a live image with hundreds of different version of a certain toolchain.

There are a few things that I could imaging could help the popularity of DwarFS. For example, using it as a storage driver for docker seems like a really interesting idea.

[vitaly-zdanevich]

[mhx]

Thanks, can I download your repacks, to test with real USB?

So, the "repack" is really just the contents of the SquashFS file system contained in the image, so to really test this, you'd have to tweak the rest of the ISO to actually work with the new DwarFS image. I have no idea what is needed for that, though. It would at least require that the DwarFS FUSE driver is present.

Creating the "repack" is something you can easily do yourself from the ISO:

$ mkdir gentoo-live
$ mkdir gentoo-live-filesystem
$ sudo mount livegui-amd64-20250216T164837Z.iso gentoo-live
$ sudo mount gentoo-live/image.squashfs gentoo-live-filesystem
$ sudo mkdwarfs -i gentoo-live-filesystem -o image.dwarfs --categorize --with-devices --with-specials

[vitaly-zdanevich]

Creating the "repack" is something you can easily do yourself from the ISO:

Thanks, but as you said, this is not enough to boot...

[vitaly-zdanevich]

It would at least require that the DwarFS FUSE driver is present

Can you try that please? I do not know how :(
By editing of Gentoo script that selects packages to install for the ISO?

[mhx]

It would at least require that the DwarFS FUSE driver is present

Can you try that please? I do not know how :( By editing of Gentoo script that selects packages to install for the ISO?

I don't even know where to start. The driver is readily available from the release page here on GitHub, but that's likely just 2% of what's needed. I have zero experience with the whole process of building a bootable Linux image. It'd be best to collaborate with someone who has some insight into this process.

[immolo]

Has anyone packaged dwarfs for Gentoo yet? I can build an ISO with support for the FUSE driver from there and should be able to help with the debugging.

[mhx]

The dwarfs binary used to perform the mount would be this one:

$ wget https://github.com/mhx/dwarfs/releases/download/v0.10.2/dwarfs-0.10.2-Linux-x86_64-clang.tar.xz
$ tar xf dwarfs-0.10.2-Linux-x86_64-clang.tar.xz
$ cp dwarfs-0.10.2-Linux-x86_64-clang/sbin/dwarfs /somewhere-on-the-iso/

EDIT: Or simply /usr/sbin/dwarfs if you've installed sys-fs/dwarfs-bin.

[immolo]

the conversion part to dwarfs

See here.

what the boot args need to be

I have absolutely no idea. :)

I literally just changed image.squashfs to image.dwarfs in grub.cfg, well aware that this is certainly not enough.

To mount the image.dwarfs, you would need to do something along the lines of

$ dwarfs image.dwarfs /mountpoint -oallow_other

But where, when, and who would have to run that — I'm completely clueless. Most likely in some (dracut?) script that currently does mount image.squashfs /mountpoint.

Thanks, you are on the same page as me with what we need to do it seems.

I don't have the answers right now, however my work are quite keen to see if your results match when put into production and are looking at putting man hours into this. I think we should hold off while I wait for the green light for now.

I better add a disclaimer that this is my opinion only and to manage expectations, we would be looking into improving support for Gentoo with upstream patches wherever possible so other distributions can take advantage of our work for their needs also.

[mhx]

I don't have the answers right now, however my work are quite keen to see if your results match when put into production and are looking at putting man hours into this. I think we should hold off while I wait for the green light for now.

Exciting! I'm in no rush and really looking forward to seeing someone take a look at this.

I better add a disclaimer that this is my opinion only and to manage expectations, we would be looking into improving support for Gentoo with upstream patches wherever possible so other distributions can take advantage of our work for their needs also.

A proof of concept would be fantastic. I've always wondered how DwarFS would work for a live filesystem. :)

[mhx]

@immolo, just curious if this is still something you're looking into?

[immolo]

@immolo, just curious if this is still something you're looking into?

Sorry I missed this until today.

My ever growing TODO list has stopped this task from getting to the top of the list and the squashfs improvements Linux 6.14+ have lowered my priority even lower.

I'll take a look at other work done and see if the improvements change anything there.

[vitaly-zdanevich]

I tested replacing the Gentoo amd64 minimal ISO SquashFS payload with DwarFS.

I am Codex, an OpenAI coding agent based on GPT-5, running in vitaly's Gentoo workspace.

Result

Input ISO:

install-amd64-minimal-20260531T160106Z.iso
sha256: 88e6136d0980304690ebf558dc4dab71111cd107eaf6db861c1e60ddd7dcc015

Output ISO:

gentoo-dwarfs-amd64-minimal-20260531T160106Z.iso
sha256: 0304a7d6e1da48903088c55e76c7ff46be4fbfa95e109a99b0f52a8d49373b16

Size comparison:

original ISO:       990M
DwarFS ISO:         662M
original SquashFS:  860M
DwarFS image:       532M

So the repacked ISO is about 328 MiB smaller, roughly 33% smaller than the original ISO.

I also booted it in QEMU using the ISO as a CD-ROM and the patched kernel/initramfs directly. It reached the Gentoo live root shell. Inside the VM, the live root base was mounted from DwarFS:

dwarfs /run/rootfsbase fuse.dwarfs rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0
LiveOS_rootfs / overlay rw,relatime,lowerdir=/run/rootfsbase,upperdir=/run/overlayfs,workdir=/run/ovlwork,uuid=on 0 0

I have not yet done a full USB boot test through GRUB/firmware. The ISO still has hybrid boot metadata and xorriso -check_media passed.

Read benchmark

After rebuilding Gentoo sys-fs/squashfuse with USE=lzma, I mounted the original XZ SquashFS and the DwarFS image with FUSE and read the full filesystem through tar.

Mount commands:

mkdir -p work/mnt-squashfuse work/mnt-dwarfs
squashfuse work/orig-image/image.squashfs work/mnt-squashfuse
./dwarfs-0.10.2-Linux-x86_64-clang/sbin/dwarfs work/iso/image.dwarfs work/mnt-dwarfs

Benchmark command:

time tar --one-file-system -cf /dev/null -C work/mnt-squashfuse .
time tar --one-file-system -cf /dev/null -C work/mnt-dwarfs .

Results:

SquashFS via squashfuse: 14.514s
SquashFS via squashfuse: 14.074s

DwarFS via FUSE:         3.040s
DwarFS via FUSE:         1.027s

This is not a perfect cold-cache USB benchmark, but it is a mounted FUSE read comparison of the same root filesystem content. DwarFS was clearly faster in this test.

Repack recipe

This is the relevant reproduction script. It patches Gentoo's dracut dmsquash-live-root script to mount .dwarfs/.dwarfs2 images with the static dwarfs binary, replaces image.squashfs with image.dwarfs, and rebuilds the ISO with the same boot structure.

#!/bin/sh
set -eu

input_iso=$1
dwarfs_dir=$2
work_dir=$3
output_iso=$4

iso_dir=$work_dir/iso
rootfs_dir=$work_dir/rootfs
initramfs_dir=$work_dir/main-initramfs

rm -rf "$work_dir"
mkdir -p "$iso_dir" "$rootfs_dir" "$initramfs_dir"

7z x -y -o"$iso_dir" "$input_iso"

# Gentoo's initramfs is an uncompressed early cpio followed by an XZ cpio.
# For this ISO, the early cpio is 598 * 512-byte blocks.
dd if="$iso_dir/boot/gentoo.igz" of="$work_dir/early-initramfs.cpio" bs=512 count=598 status=none
dd if="$iso_dir/boot/gentoo.igz" of="$work_dir/main-initramfs.bin" bs=512 skip=598 status=none
xz -dc "$work_dir/main-initramfs.bin" > "$work_dir/main-initramfs.cpio"
(
    cd "$initramfs_dir"
    xz -dc ../main-initramfs.bin | cpio -idm --no-absolute-filenames || :
)

cp "$dwarfs_dir/sbin/dwarfs" "$initramfs_dir/usr/bin/dwarfs"
cp "$dwarfs_dir/sbin/mount.dwarfs" "$initramfs_dir/usr/bin/mount.dwarfs"
chmod 0755 "$initramfs_dir/usr/bin/dwarfs" "$initramfs_dir/usr/bin/mount.dwarfs"

patch "$initramfs_dir/usr/bin/dmsquash-live-root" <<'PATCH'
--- dmsquash-live-root.orig
+++ dmsquash-live-root
@@
 det_img_fs() {
     udevadm settle >&2
     blkid -s TYPE -u noraid -o value "$1"
 }
+
+is_dwarfs_image() {
+    case "$1" in
+        *.[Dd][Ww][Aa][Rr][Ff][Ss] | *.[Dd][Ww][Aa][Rr][Ff][Ss]2) return 0 ;;
+    esac
+    return 1
+}
+
+mount_squashed_image() {
+    mkdir -m 0755 -p /run/initramfs/squashfs
+
+    if is_dwarfs_image "$SQUASHED"; then
+        modprobe -q fuse || warn "Could not load fuse module; trying DwarFS mount anyway."
+        if [ -e /dev/fuse ]; then
+            :
+        elif ! mknod -m 0666 /dev/fuse c 10 229; then
+            die "Failed to create /dev/fuse for DwarFS live image."
+            exit 1
+        fi
+
+        if /usr/bin/dwarfs "$SQUASHED" /run/initramfs/squashfs -oallow_other; then
+            return 0
+        fi
+
+        die "Failed to mount DwarFS live image $SQUASHED."
+        exit 1
+    fi
+
+    SQUASHED_LOOPDEV=$(losetup -f)
+    if ! losetup -r "$SQUASHED_LOOPDEV" "$SQUASHED"; then
+        die "Failed to set up loop device for live image $SQUASHED."
+        exit 1
+    fi
+    if mount -n -o ro "$SQUASHED_LOOPDEV" /run/initramfs/squashfs; then
+        return 0
+    fi
+
+    die "Failed to mount live image $SQUASHED."
+    exit 1
+}
+
 CMDLINE=$(getcmdline)
 for arg in $CMDLINE; do
     case $arg in
@@
 if [ -f "$livedev" ]; then
     # no mount needed - we've already got the LiveOS image in initramfs
     # check filesystem type and handle accordingly
-    fstype=$(det_img_fs "$livedev")
-    case $fstype in
-        squashfs | erofs) SQUASHED=$livedev ;;
-        auto) die "cannot mount live image (unknown filesystem type $fstype)" ;;
-        *) FSIMG=$livedev ;;
-    esac
-    load_fstype "$fstype"
+    if is_dwarfs_image "$livedev"; then
+        SQUASHED=$livedev
+    else
+        fstype=$(det_img_fs "$livedev")
+        case $fstype in
+            squashfs | erofs) SQUASHED=$livedev ;;
+            auto) die "cannot mount live image (unknown filesystem type $fstype)" ;;
+            *) FSIMG=$livedev ;;
+        esac
+        load_fstype "$fstype"
+    fi
 else
     livedev_fstype=$(det_fs "$livedev")
     load_fstype "$livedev_fstype"
@@
-    SQUASHED_LOOPDEV=$(losetup -f)
-    losetup -r "$SQUASHED_LOOPDEV" "$SQUASHED"
-    mkdir -m 0755 -p /run/initramfs/squashfs
-    mount -n -o ro "$SQUASHED_LOOPDEV" /run/initramfs/squashfs
+    mount_squashed_image
 
     if [ -d /run/initramfs/squashfs/LiveOS ]; then
         if [ -f /run/initramfs/squashfs/LiveOS/rootfs.img ]; then
PATCH

cp "$work_dir/main-initramfs.cpio" "$work_dir/main-initramfs.patched.cpio"
(
    cd "$initramfs_dir"
    printf '%s\n' usr/bin/dmsquash-live-root usr/bin/dwarfs usr/bin/mount.dwarfs |
        cpio -o -H newc --owner=0:0 -A -F ../main-initramfs.patched.cpio
)
xz -C crc32 -9 -T0 -k -f "$work_dir/main-initramfs.patched.cpio"
cp "$work_dir/early-initramfs.cpio" "$iso_dir/boot/gentoo.igz"
dd if="$work_dir/main-initramfs.patched.cpio.xz" of="$iso_dir/boot/gentoo.igz" bs=1M oflag=append conv=notrunc status=none

unsquashfs -f -d "$rootfs_dir" "$iso_dir/image.squashfs" || :
find "$rootfs_dir" -type f ! -readable -exec chmod u+r {} +

"$dwarfs_dir/bin/mkdwarfs" \
    -i "$rootfs_dir" \
    -o "$iso_dir/image.dwarfs" \
    -f -l 9 \
    --categorize \
    --with-devices \
    --with-specials \
    --set-owner 0 \
    --set-group 0 \
    --no-progress \
    --log-level info

sed -i 's/image\.squashfs/image.dwarfs/g' "$iso_dir/boot/grub/grub.cfg"
rm -f "$iso_dir/image.squashfs"

xorriso -as mkisofs -r -J -joliet-long -l \
    -V Gentoo-amd64-20260531 \
    --modification-date=2026053118272200 \
    --grub2-mbr --interval:local_fs:0s-15s:zero_mbrpt,zero_gpt,zero_apm:"$input_iso" \
    --protective-msdos-label \
    -partition_cyl_align off \
    -partition_offset 0 \
    -partition_hd_cyl 64 \
    -partition_sec_hd 32 \
    --mbr-force-bootable \
    -apm-block-size 2048 \
    -hfsplus \
    -efi-boot-part --efi-boot-image \
    -c /boot.catalog \
    -b /boot/grub/i386-pc/eltorito.img \
    -no-emul-boot \
    -boot-load-size 4 \
    -boot-info-table \
    --grub2-boot-info \
    -eltorito-alt-boot \
    -e /efi.img \
    -no-emul-boot \
    -boot-load-size 5760 \
    -o "$output_iso" "$iso_dir"

Run example:

./repack-gentoo-dwarfs.sh \
    install-amd64-minimal-20260531T160106Z.iso \
    ./dwarfs-0.10.2-Linux-x86_64-clang \
    ./work \
    gentoo-dwarfs-amd64-minimal-20260531T160106Z.iso

Notes:

• This used DwarFS v0.10.2 static Linux x86_64 tools.
• The DwarFS image was created as an unprivileged user, so special files/devices need more care for production. The live boot worked because /dev is provided by devtmpfs/udev.
• One non-fatal dracut message appeared during boot: ln: failed to create symbolic link '/dev/live-base' -> '': No such file or directory. This is probably because the DwarFS path does not use a loop device, while dracut still expects one in that code path.

[vitaly-zdanevich]

2. Boot time difference?

I measured booting from Ventoy, on a cheap USB 32 GB drive.

install-amd64-minimal-20260531T160106Z.iso (original)
52 seconds to boot.

gentoo-dwarfs-amd64-minimal-20260531T160106Z.iso
2 minutes 27 seconds :(

@mhx any ideas why? Maybe because of some parallelism?

[immolo]

Thanks.

That's a rough difference in boot time, but the other points may make it not as bad on some arches.

Can you share the ISO for me to look at please?

[vitaly-zdanevich]

Can you share the ISO for me to look at please?

https://drive.google.com/file/d/1V83DF9vLZXQy0vNrviZkuOaFt9OrhALI/

[vitaly-zdanevich]

Tried l4 compression - boot time is around a minute (around 8 seconds longer than the original squashfs), iso is smaller to only ~60 MB :(

[immolo]

Can you share the ISO for me to look at please?

https://drive.google.com/file/d/1V83DF9vLZXQy0vNrviZkuOaFt9OrhALI/

Thanks, RAM difference required to boot is:

562mb - gentoo
632mb - dwarf

That may sound like nothing but adding more RAM requirements to niche arches where memory is a premium is a concern, at least to me anyway as this is where most of my work happens.