Skip to content

openvmm: virtiofs: preflight_microvm_insert treats retyped dedup keys as alias additions, bypassing MAX_INODES #1626

Open

Description

@tangruize

Summary

In microVM virtio-fs, InodeMap::preflight_microvm_insert checks self.inodes_by_key.get(&inode.dedup_key()) without verifying that the existing entry has the same file_type() as inode. When a host inode number (DedupKey) is reused by an object of a different file type (such as a file unlinked while still tracked by the guest and replaced by a directory or symlink that reuses the freed host inode number), InodeMap::dedup_entry evicts the old inodes_by_key entry and allocates a brand-new node ID in inodes_by_node_id, yet preflight_microvm_insert ran preflight_microvm_alias_add instead of preflight_microvm_new_inode, bypassing the MAX_INODES (4096) cap and can_insert() check. Found during formal specification (blocks formal verification).

Root Cause

InodeMap::preflight_microvm_insert assumes any match in self.inodes_by_key will reuse the existing node and add an alias:

    pub(crate) fn preflight_microvm_insert(&self, inode: &VirtioFsInode) -> lx::Result<()> {
        let path = inode.clone_path();
        Self::validate_microvm_alias_path(&path)?;
        if let Some((existing, _)) = self.inodes_by_key.get(&inode.dedup_key()) {
            return self.preflight_microvm_alias_add(existing, &path);
        }
        self.preflight_microvm_new_inode(&path)
    }

However, InodeMap::insert and InodeMap::insert_reserved look up inodes_by_key through InodeMap::dedup_entry, which explicitly evicts the inodes_by_key mapping whenever existing.file_type() != inode.file_type() while leaving existing live inside self.inodes_by_node_id.values:

    fn dedup_entry<'a>(
        inodes_by_key: &'a mut HashMap<DedupKey, (Arc<VirtioFsInode>, u64)>,
        inode: &VirtioFsInode,
    ) -> Entry<'a, DedupKey, (Arc<VirtioFsInode>, u64)> {
        let key = inode.dedup_key();
        if inodes_by_key
            .get(&key)
            .is_some_and(|(existing, _)| existing.file_type() != inode.file_type())
        {
            inodes_by_key.remove(&key);
        }
        inodes_by_key.entry(key)
    }

Because dedup_entry returns Entry::Vacant, InodeMap::insert allocates a fresh node in self.inodes_by_node_id without ever having checked self.inodes_by_node_id.values.len() >= MAX_INODES || !self.inodes_by_node_id.can_insert().

Consequence

When self.inodes_by_node_id.values.len() has reached MAX_INODES (4096), looking up or inserting a host object whose (volume_id, device_nr, inode_nr) matches a still-tracked inode of a different file type bypasses MAX_INODES and inserts a 4,097th (and arbitrarily more) active inode into inodes_by_node_id.values. Once inodes_by_node_id.values.len() > MAX_INODES, subsequent microVM snapshot capture fails in save_microvm_state (anyhow::ensure!(inodes.inodes_by_node_id.values.len() <= MAX_INODES)). In addition, if inodes_by_node_id is exhausted (!can_insert()), InodeMap::insert reaches dedup_entry first and evicts existing from inodes_by_key before inodes_by_node_id.insert fails with ENOSPC; conversely, if existing already holds MAX_ALIASES_PER_INODE (256) aliases while inodes_by_node_id still has room, preflight_microvm_insert falsely rejects the retyped object with ENOSPC even though no alias is added to existing.

Reproduction

  1. Attach a writable microVM share, look up a regular file "file" so its (volume_id, device_nr, inode_nr) is recorded in inodes_by_key and inodes_by_node_id, and populate inodes_by_node_id up to MAX_INODES (4096).
  2. Replace that host inode number with an object of another file type (for example, unlink "file" while keeping its FUSE node referenced, create a directory or symlink that reuses the freed host inode number on tmpfs/ext4, or construct a VirtioFsInode sharing stat.device_nr and stat.inode_nr with S_IFDIR mode).
  3. Call inodes.insert_microvm(other) (via FUSE_LOOKUP or FUSE_READDIRPLUS): insert_microvm succeeds instead of returning Err(lx::Error::ENOSPC), growing inodes_by_node_id.values.len() to 4097 (MAX_INODES + 1), after which save_microvm_state fails.

Related: #1611, #1613.

Activity

  1. tangruize commented on Nov 15, 2026

    @tangruize
    ContributorAuthor

    Suggested Fix

    In InodeMap::preflight_microvm_insert (vm/devices/virtio/virtiofs/src/microvm/limits.rs), guard the inodes_by_key fast path with existing.file_type() == inode.file_type() so any dedup-key match whose file type differs falls through to self.preflight_microvm_new_inode(&path), matching InodeMap::dedup_entry.

    Patch against OpenVMM 9d64daad3cbe14fb0591081b73c6c9648e18d0fb. Save the contents below as fix.patch and run git apply fix.patch.

    fix.patch
    diff --git a/vm/devices/virtio/virtiofs/src/microvm/fs_tests.rs b/vm/devices/virtio/virtiofs/src/microvm/fs_tests.rs
    index 3f3e952a8..dfdbdba24 100644
    --- a/vm/devices/virtio/virtiofs/src/microvm/fs_tests.rs
    +++ b/vm/devices/virtio/virtiofs/src/microvm/fs_tests.rs
    @@ -10,6 +10,7 @@
     use super::profile::MicroVmVirtioFsProfile;
     use super::profile::microvm_root_identity;
     use super::saved_state::MAX_ALIASES_PER_INODE;
    +use super::saved_state::MAX_INODES;
     use super::saved_state::MAX_PATH_BYTES;
     use super::state::encode_relative_path;
     use super::state::fuse_negotiation_from_session;
    @@ -136,6 +137,55 @@ fn microvm_runtime_path_and_map_limits_are_enforced() {
         assert_eq!(fs.preflight_file_insert().unwrap_err(), lx::Error::ENOSPC);
     }
     
    +#[test]
    +fn a_dedup_key_of_another_file_type_takes_the_new_node_check() {
    +    // Inserting an object whose dedup key currently maps to a different file
    +    // type evicts the key and allocates a new node (`dedup_entry`), so
    +    // preflight must enforce new-node limits (`MAX_INODES` and `can_insert`)
    +    // rather than alias-addition limits on the evicted entry.
    +    let temporary_directory = tempdir().unwrap();
    +    std::fs::write(temporary_directory.path().join("file"), b"").unwrap();
    +    let profile = profile(temporary_directory.path());
    +    let fs = VirtioFs::new_microvm(temporary_directory.path(), profile).unwrap();
    +    let root = fs.get_inode(FUSE_ROOT_ID).unwrap();
    +    let (file, _) = root.lookup_child(lx::LxStr::from_bytes(b"file")).unwrap();
    +    let (existing, _) = fs.insert_inode(file).unwrap();
    +    let mut stat = root.volume.lstat(Path::new("file")).unwrap();
    +    stat.mode = lx::S_IFDIR | 0o755;
    +    let other = crate::inode::VirtioFsInode::with_attr(
    +        Arc::clone(&root.volume),
    +        PathBuf::from("other"),
    +        &stat,
    +    );
    +
    +    // Fill `inodes_by_node_id` up to `MAX_INODES`: a retyped object must be
    +    // rejected with `ENOSPC` without exceeding `MAX_INODES`.
    +    {
    +        let mut inodes = fs.inner.inodes.write();
    +        while inodes.inodes_by_node_id.values.len() < MAX_INODES {
    +            inodes.inodes_by_node_id.insert(Arc::clone(&existing));
    +        }
    +        assert_eq!(inodes.insert_microvm(other).err(), Some(lx::Error::ENOSPC));
    +        assert_eq!(inodes.inodes_by_node_id.values.len(), MAX_INODES);
    +    }
    +
    +    // Likewise when `next_handle` is exhausted.
    +    let other = crate::inode::VirtioFsInode::with_attr(
    +        Arc::clone(&root.volume),
    +        PathBuf::from("other"),
    +        &stat,
    +    );
    +    fs.inner.inodes.write().inodes_by_node_id.next_handle = 0;
    +    assert_eq!(
    +        fs.inner
    +            .inodes
    +            .read()
    +            .preflight_microvm_insert(&other)
    +            .unwrap_err(),
    +        lx::Error::ENOSPC
    +    );
    +}
    +
     #[test]
     fn microvm_denied_subtree_is_not_lookupable() {
         let temporary_directory = tempdir().unwrap();
    diff --git a/vm/devices/virtio/virtiofs/src/microvm/limits.rs b/vm/devices/virtio/virtiofs/src/microvm/limits.rs
    index b853735cb..0a9542a7e 100644
    --- a/vm/devices/virtio/virtiofs/src/microvm/limits.rs
    +++ b/vm/devices/virtio/virtiofs/src/microvm/limits.rs
    @@ -30,7 +30,12 @@ pub(crate) fn insert_microvm(
         pub(crate) fn preflight_microvm_insert(&self, inode: &VirtioFsInode) -> lx::Result<()> {
             let path = inode.clone_path();
             Self::validate_microvm_alias_path(&path)?;
    -        if let Some((existing, _)) = self.inodes_by_key.get(&inode.dedup_key()) {
    +        // A key whose node has another file type does not alias it: the
    +        // insert evicts the key and adds a new node (`dedup_entry`), so that
    +        // case takes the new node's check.
    +        if let Some((existing, _)) = self.inodes_by_key.get(&inode.dedup_key())
    +            && existing.file_type() == inode.file_type()
    +        {
                 return self.preflight_microvm_alias_add(existing, &path);
             }
             self.preflight_microvm_new_inode(&path)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: openvmmInitial title-keyword triage suggestion

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions