Skip to content

openvmm: virtiofs: microVM readdirplus fails with EIO on btrfs subvolumes and snapshot placeholders #1605

Open

Description

@tangruize

Summary

In microVM mode, VirtioFsFile::read_dir_microvm compares the st_ino returned by lookup_helper (fuse_entry.attr.ino) against the directory snapshot's d_ino-derived entry.guest_inode_id and returns Err(lx::Error::EIO) whenever they differ. On btrfs, directory entries for child subvolume roots and for nested-subvolume placeholders inside snapshots permanently carry a parent-tree d_ino that differs from lstat's st_ino (256 for a subvolume root, 2 for an empty placeholder), causing FUSE_READDIRPLUS to fail with EIO on any directory containing a btrfs subvolume or snapshot placeholder. Found by testing.

Root Cause

When opening and first enumerating a directory in microVM mode, VirtioFsFile::build_microvm_directory_snapshot populates entry.guest_inode_id from getdents64's d_ino (self.inode.guest_ino(entry.inode_nr)). During FUSE_READDIRPLUS, VirtioFsFile::read_dir_microvm resolves the child via fs.lookup_helper(&self.inode, name) (which runs lstat on the child path) and rejects any mismatch between fuse_entry.attr.ino and entry.guest_inode_id with EIO:

                    let fuse_entry = fs.lookup_helper(&self.inode, name)?;
                    if fuse_entry.attr.ino != entry.guest_inode_id {
                        return Err(lx::Error::EIO);
                    }
                    if !buffer.dir_entry_plus(name, entry.next_cookie, fuse_entry) {
                        return Err(lx::Error::EIO);
                    }

On btrfs, a child subvolume entry in its parent directory's B-tree carries a ROOT_ITEM reference object ID as d_ino (for example 257 or 388), whereas lstat on the child path crosses into the child subvolume's fs-tree and reports st_ino = 256 (BTRFS_FIRST_FREE_OBJECTID). Likewise, when snapshotting a subvolume that contains a nested subvolume, btrfs materializes an empty directory placeholder inside the snapshot whose parent d_ino is the nested subvolume's object ID while lstat reports st_ino = 2 (BTRFS_EMPTY_SUBVOL_DIR_OBJECTID). Plain FUSE_READDIR and non-microVM FUSE_READDIRPLUS both succeed on these directories, whereas microVM FUSE_READDIRPLUS always hits fuse_entry.attr.ino != entry.guest_inode_id.

Consequence

Because the microVM profile negotiates FUSE_DO_READDIRPLUS and FUSE_READDIRPLUS_AUTO by default, standard guest directory listings (ls, find, stat-walking tools, language runtimes, and container rootfs walks) issue FUSE_READDIRPLUS. Any shared host directory on btrfs that contains a subvolume root (such as default Fedora/SUSE layouts, container/image storage, or btrfs-backed workspaces) or a snapshot containing a nested subvolume placeholder fails FUSE_READDIRPLUS with EIO (Input/output error), making the entire parent directory unlistable in the guest while leaking the lookup reference acquired by lookup_helper on that entry and any earlier entries in the aborted batch.

Reproduction

  1. On a Linux btrfs filesystem supporting unprivileged subvolumes, create a host directory root and run btrfs subvolume create root/sub (or snapshot a subvolume containing a nested subvolume sub/inner to root/snap).
  2. Construct a microVM VirtioFs share rooted at root, open the root directory (FUSE_OPENDIR), and issue FUSE_READDIRPLUS at offset = 0.
  3. Observe Fuse::read_dir_plus fail with Err(lx::Error::EIO) as soon as it reaches "sub" (or "inner" inside "snap"), whereas Fuse::lookup on the same name succeeds.

Related: #1604, #1619, #1611, #1634, #1622.

Activity

  1. tangruize commented on Nov 15, 2026

    @tangruize
    ContributorAuthor

    Suggested Fix

    Remove the if fuse_entry.attr.ino != entry.guest_inode_id { return Err(lx::Error::EIO); } check in VirtioFsFile::read_dir_microvm. fs.lookup_helper(&self.inode, name) already enforces path confinement and subtree access policy on the current child path, and buffer.dir_entry_plus encodes fuse_entry (including fuse_entry.attr.ino, which overrides dirent.ino for READDIRPLUS consumers) directly into the reply. Entries replaced between directory pages will likewise be listed with their current node and attributes rather than aborting the listing with EIO. When merging alongside the sibling read_dir_microvm changes for mid-listing lookup-reference rollback and open-directory self-aliasing, keep the node_id == dir_node_id || !buffer.dir_entry_plus(...) rollback guard without reintroducing fuse_entry.attr.ino != entry.guest_inode_id.

    Patch against OpenVMM 9d64daad3cbe14fb0591081b73c6c9648e18d0fb. Save the contents below as fix.patch and run git apply fix.patch.

    fix.patch
    diff --git a/vm/devices/virtio/virtiofs/src/microvm/file.rs b/vm/devices/virtio/virtiofs/src/microvm/file.rs
    index 4451a5f8a..1fc36dc79 100644
    --- a/vm/devices/virtio/virtiofs/src/microvm/file.rs
    +++ b/vm/devices/virtio/virtiofs/src/microvm/file.rs
    @@ -67,10 +67,11 @@ pub(crate) fn read_dir_microvm(
                         if !buffer.check_dir_entry_plus(name) {
                             break;
                         }
    +                    // Emit the entry using the current object returned by
    +                    // `lookup_helper`, even when its `st_ino` differs from the
    +                    // directory stream's `d_ino` (for example, a btrfs
    +                    // subvolume root or nested-subvolume placeholder).
                         let fuse_entry = fs.lookup_helper(&self.inode, name)?;
    -                    if fuse_entry.attr.ino != entry.guest_inode_id {
    -                        return Err(lx::Error::EIO);
    -                    }
                         if !buffer.dir_entry_plus(name, entry.next_cookie, fuse_entry) {
                             return Err(lx::Error::EIO);
                         }
    diff --git a/vm/devices/virtio/virtiofs/tests/microvm_subvolume_listing_tests.rs b/vm/devices/virtio/virtiofs/tests/microvm_subvolume_listing_tests.rs
    new file mode 100644
    index 000000000..235b98715
    --- /dev/null
    +++ b/vm/devices/virtio/virtiofs/tests/microvm_subvolume_listing_tests.rs
    @@ -0,0 +1,217 @@
    +// Copyright (c) Microsoft Corporation.
    +// Licensed under the MIT License.
    +
    +//! Integration tests verifying that microVM `READDIRPLUS` lists btrfs
    +//! subvolume roots and nested subvolume placeholders inside snapshots.
    +//!
    +//! On btrfs, a directory entry for a subvolume root or a snapshot's nested
    +//! subvolume placeholder carries a `d_ino` (the parent tree's object ID) that
    +//! differs from the `st_ino` reported by `lstat` on the child path (`256` for
    +//! a subvolume root, `2` for an empty placeholder). `READDIRPLUS` must return
    +//! the child with the node ID and attributes produced by `LOOKUP` rather than
    +//! failing with `EIO`.
    +//!
    +//! These tests require a btrfs filesystem where unprivileged subvolume
    +//! creation is permitted; on other filesystems they skip cleanly.
    +
    +#![cfg(target_os = "linux")]
    +
    +use fuse::Fuse;
    +use fuse::Request;
    +use fuse::protocol::*;
    +use std::path::Path;
    +use std::process::Command;
    +use tempfile::TempDir;
    +use virtiofs::VirtioFs;
    +use zerocopy::FromBytes;
    +use zerocopy::FromZeros;
    +use zerocopy::IntoBytes;
    +
    +const ROOT: u64 = FUSE_ROOT_ID;
    +
    +fn btrfs(args: &[&str]) -> bool {
    +    Command::new("btrfs")
    +        .args(args)
    +        .output()
    +        .is_ok_and(|output| output.status.success())
    +}
    +
    +struct Share {
    +    _dir: Option<TempDir>,
    +    fs: VirtioFs,
    +    unique: u64,
    +}
    +
    +impl Share {
    +    fn new(setup: impl FnOnce(&Path) -> bool) -> Option<Self> {
    +        let dir = tempfile::tempdir().expect("tempdir");
    +        if !setup(dir.path()) {
    +            eprintln!(
    +                "skipping: no unprivileged btrfs subvolume can be made in {:?}",
    +                dir.path()
    +            );
    +            return None;
    +        }
    +        let mut share = Self::from_path(dir.path());
    +        share._dir = Some(dir);
    +        Some(share)
    +    }
    +
    +    fn from_path(root: &Path) -> Self {
    +        let profile = virtiofs::profile::MicroVmVirtioFsProfile::from_attachment(
    +            virtiofs::profile::MICROVM_ATTACHMENT_ID.to_owned(),
    +            virtiofs::profile::microvm_root_identity(root).unwrap(),
    +            false,
    +            Vec::new(),
    +        )
    +        .unwrap();
    +        let fs = VirtioFs::new_microvm(root, profile).expect("VirtioFs::new_microvm");
    +        Self {
    +            _dir: None,
    +            fs,
    +            unique: 0,
    +        }
    +    }
    +
    +    fn request(&mut self, opcode: u32, node_id: u64, payload: &[u8]) -> Request {
    +        self.unique += 1;
    +        let mut header = fuse_in_header::new_zeroed();
    +        header.len = (size_of::<fuse_in_header>() + payload.len()) as u32;
    +        header.opcode = opcode;
    +        header.unique = self.unique;
    +        header.nodeid = node_id;
    +        header.uid = 1000;
    +        header.gid = 1000;
    +        header.pid = 42;
    +        let mut data = header.as_bytes().to_vec();
    +        data.extend_from_slice(payload);
    +        Request::new(data.as_slice()).expect("request parses")
    +    }
    +
    +    fn lookup(&mut self, parent: u64, name: &str) -> u64 {
    +        let mut payload = name.as_bytes().to_vec();
    +        payload.push(0);
    +        let request = self.request(FUSE_LOOKUP, parent, &payload);
    +        Fuse::lookup(&self.fs, &request, name.as_ref())
    +            .expect("lookup")
    +            .nodeid
    +    }
    +
    +    fn open_dir(&mut self, dir: u64) -> u64 {
    +        let request = self.request(FUSE_OPENDIR, dir, &[]);
    +        Fuse::open_dir(&self.fs, &request, lx::O_RDONLY as u32)
    +            .expect("opendir")
    +            .fh
    +    }
    +
    +    fn release_dir(&mut self, dir: u64, fh: u64) {
    +        let mut arg = fuse_release_in::new_zeroed();
    +        arg.fh = fh;
    +        let request = self.request(FUSE_RELEASEDIR, dir, arg.as_bytes());
    +        Fuse::release_dir(&self.fs, &request, &arg).expect("releasedir");
    +    }
    +
    +    fn readdirplus_at(&mut self, dir: u64, fh: u64, offset: u64) -> lx::Result<Vec<(String, u64)>> {
    +        let mut arg = fuse_read_in::new_zeroed();
    +        arg.fh = fh;
    +        arg.offset = offset;
    +        arg.size = 1 << 20;
    +        let request = self.request(FUSE_READDIRPLUS, dir, arg.as_bytes());
    +        let reply = Fuse::read_dir_plus(&self.fs, &request, &arg)?;
    +        let mut entries = Vec::new();
    +        let mut rest = reply.as_slice();
    +        while !rest.is_empty() {
    +            let (entry, _) = fuse_direntplus::read_from_prefix(rest).expect("entry decodes");
    +            let start = size_of::<fuse_direntplus>();
    +            let end = start + entry.dirent.namelen as usize;
    +            let name = String::from_utf8(rest[start..end].to_vec()).expect("UTF-8 name");
    +            entries.push((name, entry.entry_out.nodeid));
    +            rest = &rest[fuse_dirent_align(end)..];
    +        }
    +        Ok(entries)
    +    }
    +}
    +
    +#[test]
    +fn microvm_readdirplus_lists_a_subvolume_root() {
    +    let Some(mut share) =
    +        Share::new(|root| btrfs(&["subvolume", "create", &root.join("sub").to_string_lossy()]))
    +    else {
    +        return;
    +    };
    +    let sub = share.lookup(ROOT, "sub");
    +    assert_ne!(sub, ROOT);
    +
    +    let fh = share.open_dir(ROOT);
    +    let first_page = share.readdirplus_at(ROOT, fh, 0).expect("first page");
    +    let listed: Vec<&str> = first_page.iter().map(|(name, _)| name.as_str()).collect();
    +    assert_eq!(listed, vec![".", "..", "sub"]);
    +    let node = first_page.iter().find(|(name, _)| name == "sub").unwrap().1;
    +    assert_eq!(node, sub);
    +
    +    // Resuming pagination after ".." must also return "sub".
    +    let continuation = share.readdirplus_at(ROOT, fh, 2).expect("continuation");
    +    assert_eq!(continuation, vec![("sub".to_owned(), sub)]);
    +    share.release_dir(ROOT, fh);
    +}
    +
    +#[test]
    +fn microvm_readdirplus_lists_a_nested_subvolume_placeholder() {
    +    let dir = tempfile::tempdir().expect("tempdir");
    +    let root = dir.path();
    +    let created = btrfs(&["subvolume", "create", &root.join("sub").to_string_lossy()])
    +        && btrfs(&[
    +            "subvolume",
    +            "create",
    +            &root.join("sub/inner").to_string_lossy(),
    +        ])
    +        && std::fs::write(root.join("sub/inner/g"), b"NESTED").is_ok()
    +        && btrfs(&[
    +            "subvolume",
    +            "snapshot",
    +            &root.join("sub").to_string_lossy(),
    +            &root.join("snap").to_string_lossy(),
    +        ]);
    +    if !created {
    +        eprintln!(
    +            "skipping: no unprivileged btrfs subvolume can be made in {:?}",
    +            root
    +        );
    +        return;
    +    }
    +
    +    // Mount one share rooted at `dir` to inspect `snap` (`st_ino = 256`) and
    +    // its nested subvolume placeholder `snap/inner` (`st_ino = 2`).
    +    let mut snap_share = Share::from_path(root);
    +    let snap = snap_share.lookup(ROOT, "snap");
    +    let inner_in_snap = snap_share.lookup(snap, "inner");
    +    assert_ne!(snap, inner_in_snap);
    +
    +    let fh = snap_share.open_dir(snap);
    +    let entries = snap_share
    +        .readdirplus_at(snap, fh, 0)
    +        .expect("readdirplus of snap");
    +    let listed = entries
    +        .iter()
    +        .find(|(name, _)| name == "inner")
    +        .map(|(_, node)| *node)
    +        .unwrap_or_else(|| panic!("inner missing from {entries:?}"));
    +    assert_eq!(listed, inner_in_snap);
    +    snap_share.release_dir(snap, fh);
    +
    +    // Mount a separate share on `sub/inner` to confirm regular entries inside
    +    // the nested subvolume remain listable.
    +    let mut inner_share = Share::from_path(&root.join("sub/inner"));
    +    let g = inner_share.lookup(ROOT, "g");
    +    let fh = inner_share.open_dir(ROOT);
    +    let entries = inner_share
    +        .readdirplus_at(ROOT, fh, 0)
    +        .expect("readdirplus of inner");
    +    let listed = entries
    +        .iter()
    +        .find(|(name, _)| name == "g")
    +        .map(|(_, node)| *node)
    +        .unwrap_or_else(|| panic!("g missing from {entries:?}"));
    +    assert_eq!(listed, g);
    +    inner_share.release_dir(ROOT, fh);
    +}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: openvmmInitial title-keyword triage suggestionarea: snapshotInitial title-keyword triage suggestion

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions