From b48cd4e876a17bd1f8d0144c0565f175d6079832 Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Fri, 25 Sep 2026 23:27:35 +0000 Subject: [PATCH 01/11] Add a node with one sudo command, uninstall it cleanly and resume downloads - Run as root, the node installer prepares the host: it creates the parsar-node system user when missing, adds it to the group that owns the Docker socket or /dev/kvm, and runs the node as a root-owned system service (User=parsar-node, enabled for boot). No lingering or login session is needed. Node work runs in a child with the service user's credentials; root never runs a file that user can write. - It never installs Docker, KVM or packages and never changes device permissions. Missing Docker or KVM, Docker without CPU and memory limits, too little capacity, SELinux enforcing, a foreign parsar-node account, or a node for the same installation already on the host stop it with one line before anything changes. Reruns change nothing. - The token can come on standard input (--enrollment-token-stdin), never in argv, the environment or sudo's log. The installer's local node still passes it in its variable until the installer rework. - --uninstall removes a node only after Core answers 401 to its credential (or with --force when Core is gone). It removes the service, node state and Docker network, and deletes the parsar-node account only when the installer created it and no node remains. It never removes sandboxes, volumes or images. - Without sudo, the installer finds the user's systemd manager from su or sudo -iu. A registered node that Core removed is told to uninstall. - Artifact downloads resume from a private partial file with HTTP Range; the complete file is still verified by size and SHA-256 before use. --- CONTRIBUTING.md | 21 +- deploy/install/distribution.py | 114 +++- deploy/install/node_install.py | 637 ++++++++++++++++-- deploy/install/node_spec.py | 4 + deploy/install/test_distribution.py | 25 +- deploy/install/test_node_install.py | 124 +++- docs/getting-started/install.md | 20 +- docs/getting-started/operations.md | 5 +- services/agents-api/HOSTED-SANDBOX-MANAGER.md | 41 +- 9 files changed, 868 insertions(+), 123 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 804abe4fd..1211bbaa5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -822,7 +822,9 @@ installation, generation, specification digest and release before writing node f registering or reconnecting. Reject drift rather than overwriting retained identity or using local resource defaults. Registration consumes a token only after these checks. The installer verifies downloaded files and starts the ordinary node process -as a user service; it performs no SSH installation, Session creation or model call. +as a user service, or, run as root, as a root-owned system service for the dedicated +`parsar-node` user it prepares; it performs no SSH installation, Session creation or +model call. The [Hosted Sandbox Manager](services/agents-api/HOSTED-SANDBOX-MANAGER.md) is a deployment-level admin surface, separate from Project credentials. The paired @@ -1841,11 +1843,18 @@ Docker socket or node identity mount in either mode. The ordinary standalone nod service owns its provider processes outside the Core container. Its `KillMode=process` preserves resident microVM/helper processes across a node-service restart. User KVM access and the Linux runtime libraries are prerequisites for microsandbox. The node -runs as a systemd user service and needs linger. The only other launcher allowed -is the root-run node installation (sudo mode) planned for phase 2b: one root-owned -system service per installation that runs the same node program as a dedicated -unprivileged service user. Do not add any other launcher, scheduler or recovery -path. Node services restart after failures without a start limit, so a node +installed by a normal user runs as a systemd user service and needs linger. Run as +root (sudo mode), the installer instead prepares the host: it creates the `parsar-node` +system user when missing, adds it to the Docker or KVM device group, and installs one +root-owned system service per installation that runs the same node program with +`User=parsar-node`. Root only prepares the account, group and unit; node work runs +with the service user's credentials, and root never runs a file that user can write. +Sudo mode never installs Docker, KVM or packages, never changes device permissions, +refuses SELinux-enforcing hosts and changes nothing when a prerequisite is missing. +`--uninstall` removes a node only after Core rejects its credential, never touches +sandboxes, volumes or images, and deletes the account only when the installer +created it and no node remains. Do not add any other launcher, scheduler or +recovery path. Node services restart after failures without a start limit, so a node outlasts a Core outage, and stop restarting when the node program exits 78 because Core answered 401 to its credential (a removed or retired node). The basic API image and binary builds remain independent artifacts. diff --git a/deploy/install/distribution.py b/deploy/install/distribution.py index 8c8103a31..d60aa64ee 100644 --- a/deploy/install/distribution.py +++ b/deploy/install/distribution.py @@ -147,44 +147,94 @@ def obtain_artifact(manifest, logical_path, destination, offline_root=None): if candidate.is_symlink() or not candidate.is_file(): raise DistributionError('Offline artifact must be a regular file: ' + logical_path) source = candidate + if source is not None: + return copy_artifact(source, target, entry, logical_path) base = manifest.get('artifact_base_url', '') - if source is None: - if not isinstance(base, str) or not base or urlsplit(base).query: - raise DistributionError('No downloadable artifact source; use the matching offline bundle') - url = safe_url(base.rstrip('/') + '/' + entry['filename']) + if not isinstance(base, str) or not base or urlsplit(base).query: + raise DistributionError('No downloadable artifact source; use the matching offline bundle') + url = safe_url(base.rstrip('/') + '/' + entry['filename']) + # A private partial file survives interruptions and reruns; the next attempt asks + # for the missing bytes only. The complete file is still verified as a whole. + partial = target.with_name('.' + target.name + '.partial') for attempt in range(3): - fd, temporary = tempfile.mkstemp(prefix='.artifact-', dir=target.parent) try: - with os.fdopen(fd, 'wb') as output: - stream = source.open('rb') if source else urllib.request.build_opener(NoRedirect()).open(url, timeout=30) - with stream: - count = 0 - while True: - block = stream.read(1024 * 1024) - if not block: - break - count += len(block) - if count > entry['size']: - raise DistributionError('Artifact exceeds published size: ' + logical_path) - output.write(block) - if count != entry['size']: - raise http.client.IncompleteRead(b'', entry['size'] - count) - if digest(temporary) != entry['sha256']: - raise DistributionError('Artifact checksum mismatch: ' + logical_path) - os.chmod(temporary, 0o700 if logical_path.startswith('native/') else 0o600) - os.replace(temporary, target) - return target + download_partial(url, partial, entry, logical_path) + break except urllib.error.HTTPError as error: - if error.code not in (408, 429, 500, 502, 503, 504) or attempt == 2: - raise DistributionError(f'Artifact download failed (HTTP {error.code}): {logical_path}. Check release access and retry.') from None + if error.code == 416: + partial.unlink(missing_ok=True) + elif error.code not in (408, 429, 500, 502, 503, 504) or attempt == 2: + raise DistributionError(f'Artifact download failed (HTTP {error.code}): {logical_path}. Check the console and retry.') from None except (urllib.error.URLError, socket.timeout, ConnectionError, http.client.HTTPException): - if attempt == 2 or source: - raise DistributionError('Artifact transfer interrupted: ' + logical_path + '. Check network access and rerun; installed state is retained.') from None - finally: - if os.path.exists(temporary): - os.unlink(temporary) + if attempt == 2: + raise DistributionError('Artifact transfer interrupted: ' + logical_path + '. Check network access and rerun; ' + 'the download resumes where it stopped.') from None time.sleep(attempt + 1) - raise DistributionError('Artifact download did not complete') + else: + raise DistributionError('Artifact download did not complete') + if digest(partial) != entry['sha256']: + partial.unlink() + raise DistributionError('Artifact checksum mismatch: ' + logical_path) + os.chmod(partial, 0o700 if logical_path.startswith('native/') else 0o600) + os.replace(partial, target) + return target + + +def copy_artifact(source, target, entry, logical_path): + fd, temporary = tempfile.mkstemp(prefix='.artifact-', dir=target.parent) + try: + with os.fdopen(fd, 'wb') as output, source.open('rb') as stream: + count = 0 + for block in iter(lambda: stream.read(1024 * 1024), b''): + count += len(block) + if count > entry['size']: + raise DistributionError('Artifact exceeds published size: ' + logical_path) + output.write(block) + if count != entry['size'] or digest(temporary) != entry['sha256']: + raise DistributionError('Artifact checksum mismatch: ' + logical_path) + os.chmod(temporary, 0o700 if logical_path.startswith('native/') else 0o600) + os.replace(temporary, target) + return target + finally: + if os.path.exists(temporary): + os.unlink(temporary) + + +def download_partial(url, partial, entry, logical_path): + """Complete the partial file, asking only for the bytes it is missing.""" + size = entry['size'] + offset = 0 + if partial.is_symlink() or (partial.exists() and not partial.is_file()): + raise DistributionError('Partial download must be a regular file: ' + logical_path) + if partial.exists(): + info = partial.stat() + if info.st_uid != os.getuid(): + raise DistributionError('Partial download must be owned by this user: ' + logical_path) + offset = info.st_size if info.st_size <= size else 0 + if offset == size: + return + request = urllib.request.Request(url, headers={'Range': f'bytes={offset}-'} if offset else {}) + with urllib.request.build_opener(NoRedirect()).open(request, timeout=30) as stream: + if offset and (stream.status != 206 or not stream.headers.get('Content-Range', '').startswith(f'bytes {offset}-')): + offset = 0 # The server sent the whole file; start over. + flags = os.O_WRONLY | os.O_CREAT | os.O_NOFOLLOW | (os.O_APPEND if offset else os.O_TRUNC) + with os.fdopen(os.open(partial, flags, 0o600), 'ab' if offset else 'wb') as output: + count, reported = offset, offset * 10 // size + while True: + block = stream.read(1024 * 1024) + if not block: + break + count += len(block) + if count > size: + output.close() + partial.unlink() + raise DistributionError('Artifact exceeds published size: ' + logical_path) + output.write(block) + if size >= 50 * 1024 * 1024 and count * 10 // size > reported: + reported = count * 10 // size + print(f'Downloading {logical_path}: {reported * 10}% of {size // (1024 * 1024)} MiB', flush=True) + if count != size: + raise http.client.IncompleteRead(b'', size - count) def runtime_archive(manifest, cache_root, offline_root=None): diff --git a/deploy/install/node_install.py b/deploy/install/node_install.py index d8b550eb1..2792112b9 100644 --- a/deploy/install/node_install.py +++ b/deploy/install/node_install.py @@ -1,7 +1,10 @@ #!/usr/bin/env python3 """Install and enroll one node from its Core console's matched distribution.""" import argparse +import contextlib import fcntl +import getpass +import grp import hashlib import http.client import io @@ -10,6 +13,7 @@ import os from pathlib import Path import platform +import pwd import re import shutil import socket @@ -34,6 +38,23 @@ class InstallError(Exception): COMMON = ("native/bin/parsar-sandbox-node", "runtime/seccomp.json") MICRO = ("native/bin/agents-api-microsandbox-provider", "native/microsandbox/msb", "native/microsandbox/libkrunfw.so.5.6.1") +DOCKER = ("docker", "--host", "unix:///var/run/docker.sock") +DOCKER_SOCKET = Path("/var/run/docker.sock") +KVM = Path("/dev/kvm") + +# Sudo mode: run as root, the installer prepares the host itself. The node runs as +# this dedicated service user under a root-owned system unit; root never runs a +# file the service user can write. +SERVICE_USER = "parsar-node" +SERVICE_HOME = Path("/var/lib/parsar-node") +SYSTEM_RECORDS = Path("/etc/parsar-node") +SYSTEM_UNITS = Path("/etc/systemd/system") +SYSTEM_LOCKS = Path("/run") +SYSTEMD_RUNNING = Path("/run/systemd/system") +SELINUX_ENFORCE = Path("/sys/fs/selinux/enforce") +USER_RUNTIME = Path("/run/user") +SAFE_PATH = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" +NOTHING_CHANGED = " Nothing was changed." def origin(value): @@ -101,18 +122,42 @@ def discard_unregistered(root): shutil.rmtree(root / "state/node") -def preflight(provider): +def preflight(provider, system=False): + """Host access for the node's own user; a system service needs no user session.""" if sys.version_info < (3, 9) or platform.system() != "Linux" or platform.machine() not in ("x86_64", "amd64") or os.getuid() == 0: raise InstallError("Run with Python 3.9+ as a non-root user on Linux amd64") - checked(["systemctl", "--user", "show", "--property=Version", "--value"], "A systemd user session is required") - if checked(["loginctl", "show-user", str(os.getuid()), "--property=Linger", "--value"], "Cannot check user lingering") != "yes": - raise InstallError("Ask the host administrator to enable user lingering before installing a node") + if not system: + user_bus() + checked(["systemctl", "--user", "show", "--property=Version", "--value"], "A systemd user session is required") + if checked(["loginctl", "show-user", str(os.getuid()), "--property=Linger", "--value"], "Cannot check user lingering") != "yes": + raise InstallError("Ask the host administrator to enable user lingering before installing a node, or run the command with sudo") if provider == "docker": - checked(["docker", "--host", "unix:///var/run/docker.sock", "info", "--format", "{{.ServerVersion}}"], "Docker access through /var/run/docker.sock is required") - elif not os.access("/dev/kvm", os.R_OK | os.W_OK): + checked(list(DOCKER) + ["info", "--format", "{{.ServerVersion}}"], "Docker access through /var/run/docker.sock is required") + elif not os.access(KVM, os.R_OK | os.W_OK): raise InstallError("microsandbox requires read/write access to /dev/kvm") +def user_bus(): + """Reach this user's systemd manager from a session that has no bus address. + + su or sudo -iu from a root shell leaves XDG_RUNTIME_DIR unset; lingering keeps the + user manager and its bus running under /run/user/.""" + if os.environ.get("XDG_RUNTIME_DIR"): + return + runtime = USER_RUNTIME / str(os.getuid()) + bus = runtime / "bus" + try: + info = bus.lstat() + except OSError: + info = None + if info is None or not stat.S_ISSOCK(info.st_mode) or info.st_uid != os.getuid(): + user = pwd.getpwuid(os.getuid()).pw_name + raise InstallError("No systemd user manager is running for " + user + ". Ask the host administrator to run " + "`sudo loginctl enable-linger " + user + "`, or run the command with sudo.") + os.environ["XDG_RUNTIME_DIR"] = str(runtime) + os.environ["DBUS_SESSION_BUS_ADDRESS"] = "unix:path=" + str(bus) + + class NoRedirect(urllib.request.HTTPRedirectHandler): def redirect_request(self, request, fp, code, message, headers, url): raise InstallError("Node bootstrap redirects are not supported") @@ -318,7 +363,12 @@ def quote(value): + "\n\n[Install]\nWantedBy=default.target\n") -def install(args, token): +def unit_name(installation_id): + return "parsar-node-" + installation_id + ".service" + + +def open_node(args, token, system=False): + """Read the Core specification and check the host; returns the node's state directory.""" root = Path.home() / ".parsar/nodes" / args.installation_id safe_directory(root) identity_file = root / "state/node/identity.json" @@ -327,7 +377,7 @@ def install(args, token): args.configuration = node_spec.fetch(args, token, retained, open_request, allow_enrollment=not (root / "registered.json").exists()) args.provider = args.configuration["provider"] print("Checking host requirements...", flush=True) - preflight(args.provider) + preflight(args.provider, system) if args.provider == "microsandbox": runtime_home = micro_home(args.installation_id) safe_directory(runtime_home) @@ -335,71 +385,89 @@ def install(args, token): if not owner.exists() and any(runtime_home.iterdir()): raise InstallError("Microsandbox home contains unowned state; refusing to adopt it") write_once(owner, json_text({"installation_id": args.installation_id})) - root = Path.home() / ".parsar/nodes" / args.installation_id - safe_directory(root) + return root + + +@contextlib.contextmanager +def install_lock(root): descriptor = os.open(root / "install.lock", os.O_CREAT | os.O_RDWR | os.O_NOFOLLOW, 0o600) with os.fdopen(descriptor, "w") as lock: try: fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) except BlockingIOError: raise InstallError("Another node installation is running") from None - print("Downloading and verifying node files...", flush=True) - manifest, sums = metadata(args.source_url, getattr(args, "bundle", None)) - node_spec.verify_release(args.configuration, manifest) - names = COMMON + (MICRO if args.provider == "microsandbox" else ()) - if "runtime/seccomp.json" not in sums: - raise InstallError("The distribution is missing required node checksums") - state = {"installation_id": args.installation_id, "provider": args.provider, "core_url": args.core_url, - "source_commit": manifest["source_commit"], "generation": args.configuration["generation"], - "specification_digest": args.configuration["specification_digest"]} - write_once(root / "installation.json", json_text(state)) - for name in names: - if name == "runtime/seccomp.json": - if getattr(args, "bundle", None) is None: - download(args.source_url, name, root, sums[name]) - else: - source = args.bundle / name - if source.is_symlink() or file_digest(source) != sums[name]: - raise InstallError("Local node payload checksum mismatch") - safe_directory((root / name).parent) - write_once(root / name, source.read_text()) + yield + + +def register_node(root, args, token): + """Download and verify the payload, prepare the Runtime and register; not the service.""" + print("Downloading and verifying node files...", flush=True) + manifest, sums = metadata(args.source_url, getattr(args, "bundle", None)) + node_spec.verify_release(args.configuration, manifest) + names = COMMON + (MICRO if args.provider == "microsandbox" else ()) + if "runtime/seccomp.json" not in sums: + raise InstallError("The distribution is missing required node checksums") + state = {"installation_id": args.installation_id, "provider": args.provider, "core_url": args.core_url, + "source_commit": manifest["source_commit"], "generation": args.configuration["generation"], + "specification_digest": args.configuration["specification_digest"]} + write_once(root / "installation.json", json_text(state)) + for name in names: + if name == "runtime/seccomp.json": + if getattr(args, "bundle", None) is None: + download(args.source_url, name, root, sums[name]) else: - target = root / name - safe_directory(target.parent) - existing_file(target) - distribution.obtain_artifact(manifest, name, target, getattr(args, "bundle", None)) - os.chmod(target, 0o700) - safe_directory(root / "state/node") - print("Checking the sandbox runtime...", flush=True) - runtime_image = prepare_runtime(root, args, manifest) - # Retain the original network policy when recovering a partial installation. - if not existing_file(root / "provider.json"): - write_once(root / "provider.json", json_text(provider_config(root, args, manifest, runtime_image))) + source = args.bundle / name + if source.is_symlink() or file_digest(source) != sums[name]: + raise InstallError("Local node payload checksum mismatch") + safe_directory((root / name).parent) + write_once(root / name, source.read_text()) else: - node_spec.verify_provider(json.loads((root / "provider.json").read_text()), args.configuration, runtime_image) - unit = root / ("parsar-node-" + args.installation_id + ".service") - write_once(unit, service_unit(root)) - marker = root / "registered.json" - if not existing_file(marker): - # The one-time credential is never passed through process arguments or service environments. - descriptor, secret_path = tempfile.mkstemp(prefix=".enrollment-", dir=root) + target = root / name + safe_directory(target.parent) + existing_file(target) + distribution.obtain_artifact(manifest, name, target, getattr(args, "bundle", None)) + os.chmod(target, 0o700) + safe_directory(root / "state/node") + print("Checking the sandbox runtime...", flush=True) + runtime_image = prepare_runtime(root, args, manifest) + # Retain the original network policy when recovering a partial installation. + if not existing_file(root / "provider.json"): + write_once(root / "provider.json", json_text(provider_config(root, args, manifest, runtime_image))) + else: + node_spec.verify_provider(json.loads((root / "provider.json").read_text()), args.configuration, runtime_image) + marker = root / "registered.json" + if not existing_file(marker): + # The one-time credential is never passed through process arguments or service environments. + descriptor, secret_path = tempfile.mkstemp(prefix=".enrollment-", dir=root) + try: + with os.fdopen(descriptor, "w") as secret: + secret.write(token) + print("Registering this node with Core...", flush=True) try: - with os.fdopen(descriptor, "w") as secret: - secret.write(token) - print("Registering this node with Core...", flush=True) - try: - checked([str(root / COMMON[0]), "register", "--config", str(root / "provider.json"), "--state-dir", str(root / "state/node"), - "--core-url", args.core_url, "--name", socket.gethostname(), - "--enrollment-token-file", secret_path], REGISTRATION_UNCONFIRMED, explain=registration_failure) - except AddressChanged: - discard_unregistered(root) - raise - write_once(marker, json_text(state)) - finally: - if os.path.exists(secret_path): - os.unlink(secret_path) - elif marker.read_text() != json_text(state): - raise InstallError("Registered node identity differs; refusing to replace it") + checked([str(root / COMMON[0]), "register", "--config", str(root / "provider.json"), "--state-dir", str(root / "state/node"), + "--core-url", args.core_url, "--name", socket.gethostname(), + "--enrollment-token-file", secret_path], REGISTRATION_UNCONFIRMED, explain=registration_failure) + except AddressChanged: + discard_unregistered(root) + raise + write_once(marker, json_text(state)) + finally: + if os.path.exists(secret_path): + os.unlink(secret_path) + elif marker.read_text() != json_text(state): + raise InstallError("Registered node identity differs; refusing to replace it") + + +def install(args, token): + """Non-root mode: the node runs as this user's systemd user service.""" + if (SYSTEM_RECORDS / (args.installation_id + ".json")).exists(): + raise InstallError("This host already runs a node for this installation as a system service. Rerun the command " + "with sudo, or uninstall that node with sudo first." + NOTHING_CHANGED) + root = open_node(args, token) + with install_lock(root): + register_node(root, args, token) + unit = root / unit_name(args.installation_id) + write_once(unit, service_unit(root)) print("Starting the node service...", flush=True) checked(["systemctl", "--user", "daemon-reload"], "Cannot reload the systemd user manager") checked(["systemctl", "--user", "enable", "--now", str(unit)], "Cannot start the node service; retained identity is unchanged") @@ -407,7 +475,405 @@ def install(args, token): print("Waiting for Core connection and provider readiness...", flush=True) wait_ready(root, args) print("Node connected to Core and provider ready. State: " + str(root)) + print("Logs: journalctl --user -u " + unit.name) + + +def prepare_service_node(args, token): + """Sudo mode, as the service user: everything but the root-owned system unit.""" + root = open_node(args, token, system=True) + with install_lock(root): + register_node(root, args, token) + + + +# Sudo mode ----------------------------------------------------------------- + +class ChildFailed(Exception): + """The service-user step failed and already printed why.""" + + +def as_service_user(account, function, *arguments): + """Run a step with the service user's credentials in a forked child. + + Every installer module is already imported, so the child never reads the root + caller's private copy of this program. The token stays in memory.""" + sys.stdout.flush() + sys.stderr.flush() + pid = os.fork() + if pid == 0: + code = 1 + try: + os.setgroups(os.getgrouplist(account.pw_name, account.pw_gid)) + os.setgid(account.pw_gid) + os.setuid(account.pw_uid) + os.umask(0o077) + os.environ.clear() + os.environ.update(HOME=account.pw_dir, USER=account.pw_name, LOGNAME=account.pw_name, PATH=SAFE_PATH, LANG="C.UTF-8") + os.chdir(account.pw_dir) + function(*arguments) + code = 0 + except (InstallError, node_spec.SpecificationError, distribution.DistributionError) as error: + print(str(error), file=sys.stderr) + except (OSError, ValueError, KeyError, subprocess.SubprocessError): + print("Node installation failed; check host prerequisites and retained private files", file=sys.stderr) + finally: + sys.stdout.flush() + sys.stderr.flush() + os._exit(code) + _, status = os.waitpid(pid, 0) + if not os.WIFEXITED(status) or os.WEXITSTATUS(status) != 0: + raise ChildFailed() + + +run_as = as_service_user + + +@contextlib.contextmanager +def system_lock(installation_id): + descriptor = os.open(SYSTEM_LOCKS / ("parsar-node-" + installation_id + ".lock"), os.O_CREAT | os.O_RDWR | os.O_NOFOLLOW, 0o600) + with os.fdopen(descriptor, "w") as lock: + try: + fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) + except BlockingIOError: + raise InstallError("Another node installation or uninstallation for this installation is running") from None + yield + + +def host_checks(): + if sys.version_info < (3, 9) or platform.system() != "Linux" or platform.machine() not in ("x86_64", "amd64"): + raise InstallError("Run with Python 3.9+ on Linux amd64." + NOTHING_CHANGED) + if not SYSTEMD_RUNNING.is_dir(): + raise InstallError("systemd is not the init system here, so this host cannot run the node service." + NOTHING_CHANGED) + for tool in ("systemctl", "useradd", "usermod", "userdel"): + if shutil.which(tool) is None: + raise InstallError(tool + " is required to prepare the node service user." + NOTHING_CHANGED) + try: + enforcing = SELINUX_ENFORCE.read_text().strip() == "1" + except OSError: + enforcing = False + if enforcing: + raise InstallError("SELinux is enforcing on this host, which sudo mode does not support yet. Run the no-sudo " + "command as a prepared user instead." + NOTHING_CHANGED) + + +def root_file(path, content): + """Write a root-owned 0644 file, or accept an identical existing one.""" + if path.is_symlink() or (path.exists() and not path.is_file()): + raise InstallError(str(path) + " must be a regular file") + if path.exists(): + if path.stat().st_uid != os.geteuid() or path.read_text() != content: + raise InstallError(str(path) + " differs from this installer's version; preserve it and inspect the node") + return + path.parent.mkdir(parents=True, exist_ok=True, mode=0o755) + descriptor, temporary = tempfile.mkstemp(prefix=".parsar-node-", dir=path.parent) + try: + with os.fdopen(descriptor, "w") as stream: + stream.write(content) + os.chmod(temporary, 0o644) + os.replace(temporary, path) + finally: + if os.path.exists(temporary): + os.unlink(temporary) + + +def read_root_json(path): + if not path.exists(): + return None + if path.is_symlink() or not path.is_file() or path.stat().st_uid != os.geteuid(): + raise InstallError(str(path) + " must be a regular file owned by root") + return json.loads(path.read_text()) + + +def service_account(): + try: + return pwd.getpwnam(SERVICE_USER) + except KeyError: + return None + +def host_capacity(provider, resources, docker_info): + if provider == "docker": + cpus, memory = docker_info.get("NCPU", 0), docker_info.get("MemTotal", 0) + else: + cpus, memory = os.cpu_count() or 0, 0 + with open("/proc/meminfo") as stream: + for line in stream: + if line.startswith("MemTotal:"): + memory = int(line.split()[1]) * 1024 + if cpus < resources["cpus"] or memory < resources["memory_mib"] * 1024 * 1024: + raise InstallError("This host has %d CPUs and %d MiB of memory; each sandbox needs %d CPUs and %d MiB." + % (cpus, memory // (1024 * 1024), resources["cpus"], resources["memory_mib"]) + NOTHING_CHANGED) + + +def provider_group(provider): + """Check Docker or KVM without installing either; return the group that grants access.""" + if provider == "docker": + if shutil.which("docker") is None: + raise InstallError("Docker Engine is not installed. Install it (https://docs.docker.com/engine/install/), " + "then rerun this command." + NOTHING_CHANGED) + try: + info = DOCKER_SOCKET.stat() + except OSError: + info = None + if info is None or not stat.S_ISSOCK(info.st_mode): + raise InstallError("Docker is not running: run `sudo systemctl enable --now docker`, then rerun this command." + NOTHING_CHANGED) + if info.st_gid == 0 or stat.S_IMODE(info.st_mode) & 0o060 != 0o060: + raise InstallError("Nodes reach Docker at /var/run/docker.sock through its group, but the socket is not " + "group-accessible (rootless Docker is not supported)." + NOTHING_CHANGED) + try: + details = json.loads(checked(list(DOCKER) + ["info", "--format", "{{json .}}"], "Docker is not running")) + except (InstallError, ValueError): + raise InstallError("Docker is not running: run `sudo systemctl enable --now docker`, then rerun this command." + NOTHING_CHANGED) from None + if details.get("MemoryLimit") is not True or details.get("CPUCfsQuota") is not True: + raise InstallError("Docker on this host does not enforce CPU and memory limits; use cgroup v2, then rerun this command." + NOTHING_CHANGED) + return group_name(info.st_gid), details + try: + info = KVM.stat() + except OSError: + info = None + if info is None or not stat.S_ISCHR(info.st_mode): + raise InstallError("KVM is unavailable: enable hardware virtualization (or nested virtualization for this VM) and " + "load kvm_intel or kvm_amd, then rerun this command." + NOTHING_CHANGED) + if stat.S_IMODE(info.st_mode) & 0o006 == 0o006: + return None, {} + if info.st_gid == 0 or stat.S_IMODE(info.st_mode) & 0o060 != 0o060: + raise InstallError("/dev/kvm must be group-accessible, for example root:kvm with mode 0660 (your distribution's " + "KVM package sets this), then rerun this command." + NOTHING_CHANGED) + return group_name(info.st_gid), {} + + +def group_name(gid): + try: + return grp.getgrgid(gid).gr_name + except KeyError: + raise InstallError("The device's group " + str(gid) + " has no name; give it a named group, then rerun." + NOTHING_CHANGED) from None + + +def other_node(args, provider): + """Refuse a second node for this installation on the same host or Docker engine.""" + sudo_user = os.environ.get("SUDO_USER", "") + if sudo_user and sudo_user != "root": + try: + home = Path(pwd.getpwnam(sudo_user).pw_dir) + except KeyError: + home = None + if home is not None and (home / ".parsar/nodes" / args.installation_id).exists(): + raise InstallError("This host already runs a node for this installation, installed without sudo by " + sudo_user + + ". Remove it on the Nodes page and uninstall it as that user first." + NOTHING_CHANGED) + if provider == "docker" and not (SERVICE_HOME / ".parsar/nodes" / args.installation_id).exists(): + networks = checked(list(DOCKER) + ["network", "ls", "--format", "{{.Name}}"], "Cannot inspect Docker networks").splitlines() + if "parsar-node-" + args.installation_id in networks: + raise InstallError("Another node for this installation already uses this Docker engine (network parsar-node-" + + args.installation_id + "). Remove it on the Nodes page and uninstall it first." + NOTHING_CHANGED) + + +def prepare_account(group): + """Create or adopt the service user and give it the provider's group.""" + account = service_account() + if account is None: + shell = next((path for path in ("/usr/sbin/nologin", "/sbin/nologin") if os.path.exists(path)), "/bin/false") + checked(["useradd", "--system", "--user-group", "--no-create-home", "--home-dir", str(SERVICE_HOME), + "--shell", shell, SERVICE_USER], "Cannot create the " + SERVICE_USER + " service user") + account = service_account() + root_file(SYSTEM_RECORDS / "user.json", json_text({"format": 1, "created": True, "uid": account.pw_uid})) + elif account.pw_dir != str(SERVICE_HOME) or not account.pw_shell.endswith(("nologin", "false")): + raise InstallError("An account named " + SERVICE_USER + " exists but was not created by this installer (home " + + account.pw_dir + ", shell " + account.pw_shell + "). Rename or remove it, then rerun." + NOTHING_CHANGED) + if SERVICE_HOME.is_symlink(): + raise InstallError(str(SERVICE_HOME) + " must not be a symlink") + if not SERVICE_HOME.exists(): + SERVICE_HOME.mkdir(mode=0o700, parents=True) + os.chown(SERVICE_HOME, account.pw_uid, account.pw_gid) + if SERVICE_HOME.stat().st_uid != account.pw_uid: + raise InstallError(str(SERVICE_HOME) + " is owned by another user; preserve it and inspect the host") + os.chmod(SERVICE_HOME, 0o700) + if group and SERVICE_USER not in grp.getgrnam(group).gr_mem: + checked(["usermod", "--append", "--groups", group, SERVICE_USER], "Cannot add " + SERVICE_USER + " to the " + group + " group") + return service_account() + + +def system_unit(root, provider): + def quote(value): + return '"' + str(value).replace("\\", "\\\\").replace('"', '\\"').replace("%", "%%") + '"' + after = "network-online.target docker.service" if provider == "docker" else "network-online.target" + # Root owns this file; the service user can change only its own node files. + return ("[Unit]\nDescription=Parsar sandbox node " + root.name + "\nWants=network-online.target\nAfter=" + after + + "\nStartLimitIntervalSec=0\n\n[Service]\nType=exec\nUser=" + SERVICE_USER + "\nGroup=" + SERVICE_USER + + "\nExecStart=:" + quote(root / COMMON[0]) + " run --config " + quote(root / "provider.json") + + " --state-dir " + quote(root / "state/node") + "\nWorkingDirectory=" + str(root).replace("%", "%%") + + "\nRestart=on-failure\nRestartSec=5s\nRestartPreventExitStatus=78\nKillMode=process\nUMask=0077" + + "\n\n[Install]\nWantedBy=multi-user.target\n") + + +def install_system(args, token): + """Sudo mode: prepare the host, then run the node as a root-owned system service.""" + host_checks() + with system_lock(args.installation_id): + record_path = SYSTEM_RECORDS / (args.installation_id + ".json") + record = read_root_json(record_path) + configuration = None + if record is None: + print("Reading the Core deployment specification...", flush=True) + configuration = node_spec.fetch(args, token, None, open_request, allow_enrollment=True) + provider = configuration["provider"] + else: + provider = record["provider"] + if record["core_url"] != args.core_url: + raise InstallError("This host's node uses " + record["core_url"] + ", but this command uses " + args.core_url + + ". Remove the node on the Nodes page, uninstall it, then run a new command." + NOTHING_CHANGED) + print("Checking host requirements...", flush=True) + group, details = provider_group(provider) + if record is None: + other_node(args, provider) + host_capacity(provider, configuration["specification"]["resources"], details) + account = prepare_account(group) + root = Path(account.pw_dir) / ".parsar/nodes" / args.installation_id + unit = SYSTEM_UNITS / unit_name(args.installation_id) + root_file(record_path, json_text({"format": 1, "installation_id": args.installation_id, "provider": provider, + "core_url": args.core_url, "node_root": str(root), "unit": str(unit)})) + args.system, args.provider = True, provider + run_as(account, prepare_service_node, args, token) + root_file(unit, system_unit(root, provider)) + print("Starting the node service...", flush=True) + checked(["systemctl", "daemon-reload"], "Cannot reload systemd") + checked(["systemctl", "enable", "--now", unit.name], "Cannot start the node service; retained identity is unchanged") + checked(["systemctl", "is-active", "--quiet", unit.name], "Node service is unavailable; inspect sudo journalctl -u " + unit.name) + print("Waiting for Core connection and provider readiness...", flush=True) + run_as(account, wait_ready, root, args) + print("Node connected to Core and provider ready. It runs as " + SERVICE_USER + " in the system service " + unit.name + ".") + print("Logs: sudo journalctl -u " + unit.name) + + +# Uninstall ------------------------------------------------------------------- + +def confirm_removed(root, owner, force): + """Continue only once Core rejects the node's credential, or when it never registered.""" + identity_file = root / "state/node/identity.json" + if not (root / "registered.json").exists() and not identity_file.exists(): + return + if force: + print("Skipping the Core check (--force).", flush=True) + return + if identity_file.is_symlink() or not identity_file.is_file() or identity_file.stat().st_uid != owner: + raise InstallError("Retained node identity is missing or unsafe; rerun with --force only if Core no longer exists." + NOTHING_CHANGED) + try: + stored = json.loads(identity_file.read_text()) + node_id, credential, core = stored["identity"]["node_id"], stored["credential"], stored["core_url"] + except (ValueError, KeyError, TypeError): + raise InstallError("Retained node identity is invalid; rerun with --force only if Core no longer exists." + NOTHING_CHANGED) from None + request = urllib.request.Request(core + "/api/v1/sandbox-node/identity?" + urlencode({"node_id": node_id}), + headers={"Authorization": "Bearer " + credential}) + try: + with open_request(request, timeout=15): + pass + except urllib.error.HTTPError as error: + if error.code == 401: + return + detail = "HTTP " + str(error.code) + except (urllib.error.URLError, TimeoutError, ConnectionError, http.client.HTTPException, InstallError): + detail = "Core unreachable" + else: + raise InstallError("Core still lists this node. Remove it on the Nodes page first; Core refuses while it keeps " + "sandboxes." + NOTHING_CHANGED) + raise InstallError("Cannot confirm with Core that this node was removed (" + detail + "). Rerun with --force only " + "if this Core no longer exists." + NOTHING_CHANGED) + + +def release_docker_network(root, installation_id): + """Remove the node's Docker network; never containers, volumes or images.""" + if shutil.which("docker") is None: + return + network = "parsar-node-" + installation_id + try: + if network not in checked(list(DOCKER) + ["network", "ls", "--format", "{{.Name}}"], "Docker unavailable").splitlines(): + return + remaining = checked(list(DOCKER) + ["ps", "--all", "--filter", "label=io.parsar.agents-api.installation=" + installation_id, + "--format", "{{.Names}}"], "Docker unavailable").split() + except InstallError: + print("Docker is unavailable, so network " + network + " was not removed.") + return + if remaining: + print("Kept Docker network " + network + ": these containers still exist: " + ", ".join(remaining)) + return + checked(list(DOCKER) + ["network", "rm", network], "Cannot remove Docker network " + network) + + +def remove_node_files(root, installation_id, home): + try: + image = json.loads((root / "provider.json").read_text()).get("docker", {}).get("image") + except (OSError, ValueError, AttributeError): + image = None + runtime_home = home / ".parsar/m" / hashlib.sha256(installation_id.encode()).hexdigest()[:12] + try: + owned = json.loads((runtime_home / "parsar-installation.json").read_text()).get("installation_id") == installation_id + except (OSError, ValueError, AttributeError): + owned = False + if owned and not runtime_home.is_symlink(): + shutil.rmtree(runtime_home) + if root.exists() and not root.is_symlink(): + shutil.rmtree(root) + if image: + print("Kept the Runtime image " + image + "; remove it with `docker image rm " + image + "` if no other node uses it.") + + +def uninstall_system(args): + """Sudo mode: remove the system service, the node files and, when unused, the service user.""" + if shutil.which("systemctl") is None: + raise InstallError("systemctl is required." + NOTHING_CHANGED) + with system_lock(args.installation_id): + record_path = SYSTEM_RECORDS / (args.installation_id + ".json") + read_root_json(record_path) + account = service_account() + home = Path(account.pw_dir) if account else SERVICE_HOME + root = home / ".parsar/nodes" / args.installation_id + unit = SYSTEM_UNITS / unit_name(args.installation_id) + if not record_path.exists() and not root.exists() and not unit.exists(): + print("No node for installation " + args.installation_id + " was installed with sudo on this host. If it was " + "installed without sudo, run the uninstall command as that user without sudo.") + else: + confirm_removed(root, account.pw_uid if account else -1, args.force) + if unit.exists(): + checked(["systemctl", "disable", "--now", unit.name], "Cannot stop the node service " + unit.name) + unit.unlink() + checked(["systemctl", "daemon-reload"], "Cannot reload systemd") + release_docker_network(root, args.installation_id) + remove_node_files(root, args.installation_id, home) + record_path.unlink(missing_ok=True) + print("Node for installation " + args.installation_id + " uninstalled from this host.") + remove_unused_account(account) + + +def remove_unused_account(account): + """Delete the service user only when this installer created it and no node uses it.""" + created = read_root_json(SYSTEM_RECORDS / "user.json") + if not account or not created or not created.get("created") or any(SYSTEM_RECORDS.glob("*-*-*-*-*.json")): + return + checked(["userdel", SERVICE_USER], "Cannot remove the " + SERVICE_USER + " user; stop its processes and rerun the uninstall command") + if SERVICE_HOME.exists() and not SERVICE_HOME.is_symlink(): + shutil.rmtree(SERVICE_HOME) + (SYSTEM_RECORDS / "user.json").unlink() + with contextlib.suppress(OSError): + SYSTEM_RECORDS.rmdir() + print("Removed the " + SERVICE_USER + " service user, which this installer created.") + + +def uninstall_user(args): + root = Path.home() / ".parsar/nodes" / args.installation_id + if not root.exists(): + print("No node for installation " + args.installation_id + " is installed for this user. If it was installed " + "with sudo, run the uninstall command with sudo.") + return + confirm_removed(root, os.getuid(), args.force) + unit = root / unit_name(args.installation_id) + service = unit.exists() + if service: + user_bus() + checked(["systemctl", "--user", "disable", "--now", unit.name], "Cannot stop the node service " + unit.name) + release_docker_network(root, args.installation_id) + remove_node_files(root, args.installation_id, Path.home()) + if service: + checked(["systemctl", "--user", "daemon-reload"], "Cannot reload the systemd user manager") + print("Node for installation " + args.installation_id + " uninstalled for this user.") def wait_ready(root, args, timeout=60): identity_file = root / "state/node/identity.json" @@ -451,32 +917,63 @@ def wait_ready(root, args, timeout=60): remaining = deadline - time.monotonic() if remaining > 0: time.sleep(min(2, remaining)) - raise InstallError(detail + "; state and service are retained. Inspect journalctl --user -u parsar-node-" - + args.installation_id + ".service, then rerun the installation command") + journal = ("sudo journalctl -u " if getattr(args, "system", False) else "journalctl --user -u ") + unit_name(args.installation_id) + raise InstallError(detail + "; state and service are retained. Inspect " + journal + ", then rerun the installation command") + + +def read_token(args, parser): + """The one-time token comes on standard input, never in argv or a sudo command line.""" + # The installer's own local node passes it in this variable to its child process. + environment = os.environ.pop("PARSAR_NODE_ENROLLMENT_TOKEN", None) + if args.enrollment_token_stdin and environment is not None: + parser.error("pass the enrollment token on standard input only") + if not args.enrollment_token_stdin: + return environment or "" + if sys.stdin.isatty(): + return getpass.getpass("Enrollment token: ").strip() + return sys.stdin.readline(4098).strip() def main(argv=None): - token = os.environ.pop("PARSAR_NODE_ENROLLMENT_TOKEN", "") parser = argparse.ArgumentParser(description=__doc__) - source = parser.add_mutually_exclusive_group(required=True) + source = parser.add_mutually_exclusive_group() source.add_argument("--source-url", type=origin) source.add_argument("--bundle", type=Path) - parser.add_argument("--core-url", type=origin, required=True) + parser.add_argument("--core-url", type=origin) parser.add_argument("--provider", choices=("docker", "microsandbox"), help="Optional assertion; Core owns provider selection") parser.add_argument("--installation-id", required=True) + parser.add_argument("--enrollment-token-stdin", action="store_true", help="Read the one-time enrollment token from standard input") + parser.add_argument("--uninstall", action="store_true", help="Remove this host's node after it was removed on the Nodes page") + parser.add_argument("--force", action="store_true", help="With --uninstall: skip the Core check, for a Core that no longer exists") args = parser.parse_args(argv) if str(uuid.UUID(args.installation_id)) != args.installation_id: raise InstallError("Installation ID must be a canonical UUID") + if args.uninstall: + if args.source_url or args.bundle or args.core_url or args.provider or args.enrollment_token_stdin: + parser.error("--uninstall takes only --installation-id and --force") + os.environ.pop("PARSAR_NODE_ENROLLMENT_TOKEN", None) + (uninstall_system if os.geteuid() == 0 else uninstall_user)(args) + return + if args.force: + parser.error("--force applies only to --uninstall") + if not (args.source_url or args.bundle) or not args.core_url: + parser.error("--source-url (or --bundle) and --core-url are required") if args.bundle is not None and (not args.bundle.is_absolute() or args.bundle.resolve() != args.bundle): raise InstallError("Local bundle must be an absolute directory without symlinks") + token = read_token(args, parser) if len(token) > 4096 or any(c.isspace() for c in token): raise InstallError("A valid one-time enrollment credential is required") - install(args, token) + if os.geteuid() == 0: + install_system(args, token) + else: + install(args, token) if __name__ == "__main__": try: main() + except ChildFailed: + sys.exit(1) except (InstallError, node_spec.SpecificationError, distribution.DistributionError, OSError, ValueError, KeyError, subprocess.SubprocessError) as error: print(str(error) if isinstance(error, (InstallError, node_spec.SpecificationError, distribution.DistributionError)) else "Node installation failed; check host prerequisites and retained private files", file=sys.stderr) sys.exit(1) diff --git a/deploy/install/node_spec.py b/deploy/install/node_spec.py index 3dff5f8ea..7017dcd29 100644 --- a/deploy/install/node_spec.py +++ b/deploy/install/node_spec.py @@ -110,6 +110,10 @@ def fetch(args, token, retained, open_request, allow_enrollment=False): raise ValueError() data = validate(json.loads(raw), args) except urllib.error.HTTPError as error: + if error.code == 401 and retained is not None and not allow_enrollment: + raise SpecificationError("Core no longer accepts this node: it was removed on the Nodes page, or a sandbox " + "deployment change retired it. Uninstall it with node-install.pyz --uninstall " + "--installation-id " + args.installation_id + ", then add the host with a new command.") from None if error.code == 404: raise SpecificationError("Core node configuration was not found (HTTP 404); route /api/v1 on the Core origin directly to Core, not to Web") from None if error.code == 409: diff --git a/deploy/install/test_distribution.py b/deploy/install/test_distribution.py index bdb9b7773..593531c35 100644 --- a/deploy/install/test_distribution.py +++ b/deploy/install/test_distribution.py @@ -23,11 +23,24 @@ def setUp(self): self.root = Path(self.temp.name).resolve() self.data = b'prebuilt artifact' * 1000 self.requests = [] + self.ranges = [] self.status = 200 + self.resumable = False test = self class Handler(http.server.BaseHTTPRequestHandler): def do_GET(self): test.requests.append(self.path) + test.ranges.append(self.headers.get('Range')) + if test.resumable: + # The first response breaks off halfway; a Range request gets the rest. + start = int(self.headers['Range'][6:-1]) if self.headers.get('Range') else 0 + self.send_response(206 if start else 200) + if start: + self.send_header('Content-Range', f'bytes {start}-{len(test.data) - 1}/{len(test.data)}') + self.send_header('Content-Length', str(len(test.data) - start)) + self.end_headers() + self.wfile.write(test.data[start:] if start else test.data[:len(test.data) // 2]) + return self.send_response(test.status) if test.status == 302: self.send_header('Location', 'https://elsewhere.example' + self.path) @@ -90,9 +103,19 @@ def test_truncated_transfer_can_be_rerun(self): self.data = self.data[:20] with patch.object(distribution.time, 'sleep'), self.assertRaisesRegex(distribution.DistributionError, 'interrupted'): distribution.obtain_artifact(self.manifest, 'native/bin/node', self.root / 'node') - self.assertEqual(list(self.root.iterdir()), []) + # Only the private partial file stays, for the rerun to continue. + self.assertEqual([path.name for path in self.root.iterdir()], ['.node.partial']) self.data = b'prebuilt artifact' * 1000 distribution.obtain_artifact(self.manifest, 'native/bin/node', self.root / 'node') + self.assertEqual([path.name for path in self.root.iterdir()], ['node']) + + def test_interrupted_download_resumes_with_the_missing_bytes(self): + self.resumable = True + with patch.object(distribution.time, 'sleep'): + distribution.obtain_artifact(self.manifest, 'native/bin/node', self.root / 'node') + self.assertEqual((self.root / 'node').read_bytes(), self.data) + self.assertEqual(self.ranges, [None, f'bytes={len(self.data) // 2}-']) + self.assertEqual([path.name for path in self.root.iterdir()], ['node']) def test_offline_and_runtime_expansion_are_verified(self): raw = b'synthetic tar contents' * 1000 diff --git a/deploy/install/test_node_install.py b/deploy/install/test_node_install.py index f653f5534..96c2f6777 100644 --- a/deploy/install/test_node_install.py +++ b/deploy/install/test_node_install.py @@ -6,11 +6,13 @@ import json import os from pathlib import Path +import socket import stat import subprocess import tempfile import unittest import urllib.error +from types import SimpleNamespace from unittest import mock import node_install as installer @@ -64,7 +66,8 @@ def configuration_response(self, request, **kwargs): "specification_digest": node_spec.digest(self.args.provider, spec)} return io.BytesIO(json.dumps(configuration).encode()) - def artifact_response(self, url, **kwargs): + def artifact_response(self, request, **kwargs): + url = getattr(request, "full_url", request) # The fixture manifest records a release URL; nodes still download from their console. self.assertTrue(url.startswith(self.args.source_url + "/node-install/artifacts/"), url) for name, item in self.manifest["artifacts"].items(): @@ -104,6 +107,13 @@ def checked(self, arguments, failure, **kwargs): raise installer.InstallError(failure) if "enable" in arguments and self.fail_service: raise installer.InstallError(failure) + if arguments[:1] == ["useradd"]: + self.account = SimpleNamespace(pw_name="parsar-node", pw_uid=os.getuid(), pw_gid=os.getgid(), + pw_dir=str(installer.SERVICE_HOME), pw_shell="/usr/sbin/nologin") + if arguments[:1] == ["usermod"]: + self.joined = True + if "{{json .}}" in arguments: + return json.dumps({"MemoryLimit": True, "CPUCfsQuota": True, "NCPU": 8, "MemTotal": 16 << 30}) if "load" in arguments: self.image_present = True if "inspect" in arguments: @@ -315,6 +325,118 @@ def test_main_removes_token_environment_before_any_subprocess(self): "--provider", "docker", "--installation-id", self.args.installation_id]) self.assertNotIn("PARSAR_NODE_ENROLLMENT_TOKEN", os.environ) + def sudo_host(self, enforcing=False): + """Sudo mode against temporary system paths; the service-user step runs in-process.""" + system = self.home / "system" + (system / "systemd").mkdir(parents=True) + (system / "run").mkdir() + (system / "selinux").write_text("1" if enforcing else "0") + docker_socket = socket.socket(socket.AF_UNIX) + docker_socket.bind(str(system / "docker.sock")) + self.addCleanup(docker_socket.close) + os.chmod(system / "docker.sock", 0o660) + self.account, self.joined, self.docker_installed = None, False, True + service_home = self.home / "service" + self.root = service_home / ".parsar/nodes" / self.args.installation_id + + def run_as(account, function, *arguments): + with mock.patch.object(installer.Path, "home", return_value=Path(account.pw_dir)): + function(*arguments) + for patch in (mock.patch.multiple(installer, SERVICE_HOME=service_home, SYSTEM_RECORDS=system / "etc", + SYSTEM_UNITS=system / "units", SYSTEM_LOCKS=system / "run", + SYSTEMD_RUNNING=system / "systemd", SELINUX_ENFORCE=system / "selinux", + DOCKER_SOCKET=system / "docker.sock", run_as=run_as, + service_account=lambda: self.account), + mock.patch.object(installer.shutil, "which", side_effect=lambda tool: None if tool == "docker" and not self.docker_installed else "/usr/bin/" + tool), + mock.patch.object(installer.grp, "getgrnam", side_effect=lambda name: SimpleNamespace(gr_mem=["parsar-node"] if self.joined else []))): + patch.start() + self.addCleanup(patch.stop) + return system + + def test_sudo_mode_prepares_the_host_and_reruns_without_changes(self): + system = self.sudo_host() + installer.install_system(self.args, "synthetic-once-token") + commands = [call for call, _ in self.calls] + self.assertIn(["useradd", "--system", "--user-group", "--no-create-home", "--home-dir", str(self.home / "service"), + "--shell", next(path for path in ("/usr/sbin/nologin", "/sbin/nologin", "/bin/false") if os.path.exists(path) or path == "/bin/false"), + "parsar-node"], commands) + self.assertTrue(any(call[:3] == ["usermod", "--append", "--groups"] for call in commands)) + self.assertIn(["systemctl", "enable", "--now", "parsar-node-" + self.args.installation_id + ".service"], commands) + unit = (system / "units" / ("parsar-node-" + self.args.installation_id + ".service")).read_text() + for line in ("User=parsar-node", "After=network-online.target docker.service", "WantedBy=multi-user.target", + "RestartPreventExitStatus=78", "StartLimitIntervalSec=0"): + self.assertIn(line, unit) + self.assertNotIn("synthetic-once-token", unit) + self.assertTrue((self.root / "registered.json").exists()) + self.assertFalse((self.root / ("parsar-node-" + self.args.installation_id + ".service")).exists()) + self.assertEqual(json.loads((system / "etc/user.json").read_text())["created"], True) + before = {path: path.read_bytes() for path in (system / "etc").iterdir()} + self.calls.clear() + installer.install_system(self.args, "") + commands = [call for call, _ in self.calls] + self.assertFalse([call for call in commands if call[:1] in (["useradd"], ["usermod"]) or "register" in call]) + self.assertEqual({path: path.read_bytes() for path in (system / "etc").iterdir()}, before) + + def test_sudo_mode_refusals_change_nothing(self): + foreign = SimpleNamespace(pw_name="parsar-node", pw_uid=4242, pw_gid=4242, pw_dir="/home/parsar-node", pw_shell="/bin/bash") + for case, message in (("selinux", "SELinux is enforcing"), ("docker", "Docker Engine is not installed"), + ("account", "not created by this installer")): + with self.subTest(case=case): + system = self.sudo_host(enforcing=case == "selinux") + self.docker_installed = case != "docker" + self.account = foreign if case == "account" else None + self.calls.clear() + with self.assertRaisesRegex(installer.InstallError, message + ".*Nothing was changed"): + installer.install_system(self.args, "synthetic-once-token") + self.assertFalse([call for call, _ in self.calls if call[:1] in (["useradd"], ["usermod"], ["systemctl"])]) + self.assertFalse((system / "etc").exists()) + self.assertFalse((system / "units").exists()) + for path in sorted(system.rglob("*"), reverse=True): + path.unlink() if not path.is_dir() else path.rmdir() + system.rmdir() + + def test_uninstall_waits_for_core_to_reject_the_node(self): + system = self.sudo_host() + installer.install_system(self.args, "synthetic-once-token") + uninstall = SimpleNamespace(installation_id=self.args.installation_id, force=False) + with mock.patch.object(installer, "open_request", return_value=io.BytesIO(b"{}")), \ + self.assertRaisesRegex(installer.InstallError, "still lists this node.*Nothing was changed"): + installer.uninstall_system(uninstall) + self.assertTrue((self.root / "registered.json").exists()) + rejected = urllib.error.HTTPError("https://core.example", 401, "", {}, None) + self.calls.clear() + with mock.patch.object(installer, "open_request", side_effect=rejected): + installer.uninstall_system(uninstall) + commands = [call for call, _ in self.calls] + self.assertIn(["systemctl", "disable", "--now", "parsar-node-" + self.args.installation_id + ".service"], commands) + self.assertIn(["userdel", "parsar-node"], commands) + self.assertFalse(any(call[-2:] == ["rm", "--force"] or "prune" in call or ("image" in call and "rm" in call) for call in commands)) + self.assertFalse(self.root.exists()) + self.assertFalse((system / "units" / ("parsar-node-" + self.args.installation_id + ".service")).exists()) + self.assertFalse((system / "etc").exists()) + + def test_token_comes_on_standard_input_only(self): + arguments = ["--source-url", self.args.source_url, "--core-url", self.args.core_url, "--installation-id", + self.args.installation_id, "--enrollment-token-stdin"] + with mock.patch.object(installer, "install") as install, mock.patch.object(installer.sys, "stdin", io.StringIO("synthetic-once-token\n")), \ + mock.patch.object(installer.os, "geteuid", return_value=1000): + installer.main(arguments) + self.assertEqual(install.call_args.args[1], "synthetic-once-token") + with mock.patch.dict(os.environ, {"PARSAR_NODE_ENROLLMENT_TOKEN": "other"}), self.assertRaises(SystemExit): + installer.main(arguments) + + def test_user_manager_bus_is_found_without_a_login_session(self): + runtime = self.home / "run-user" + (runtime / str(os.getuid())).mkdir(parents=True) + with mock.patch.object(installer, "USER_RUNTIME", runtime), mock.patch.dict(os.environ, {}, clear=True): + with self.assertRaisesRegex(installer.InstallError, "enable-linger"): + installer.user_bus() + bus = socket.socket(socket.AF_UNIX) + self.addCleanup(bus.close) + bus.bind(str(runtime / str(os.getuid()) / "bus")) + installer.user_bus() + self.assertEqual(os.environ["DBUS_SESSION_BUS_ADDRESS"], "unix:path=" + str(runtime / str(os.getuid()) / "bus")) + def test_offline_bundle_uses_same_bootstrap_and_verified_artifacts(self): bundle = self.home / "bundle" bundle.mkdir() diff --git a/docs/getting-started/install.md b/docs/getting-started/install.md index 15a77fec4..af41a39b4 100644 --- a/docs/getting-started/install.md +++ b/docs/getting-started/install.md @@ -59,8 +59,9 @@ no Go, Node, Rust or source checkout. The manifest identifies every asset by immutable revision, SHA-256 and byte size. Adding a node downloads only its selected provider's assets. Runtime images use compressed archives; verified local files and already imported images are reused. -Downloads use temporary files and bounded retries, so a truncated response is -never promoted into the cache. An optional `-offline.tar.gz` bundle contains the +Downloads use private partial files and bounded retries: an interrupted download, +or a rerun after one, asks the console only for the missing bytes, and a file is used +only after its size and SHA-256 match the manifest. An optional `-offline.tar.gz` bundle contains the same assets locally. Nodes download these assets only from the console (Web) that generated their @@ -142,21 +143,26 @@ Choose English or Chinese through the System language selector. specification; node files cannot override it. Later changes require global maintenance and completed cleanup. Microsandbox uses a five-minute idle timeout and one-day snapshot retention. -4. For own-machine hosting, click **Add node**, copy the command, and run it as a non-root user on the - target Linux amd64 host. It downloads the matched bootstrap and execution assets from your console +4. For own-machine hosting, click **Add node**, copy the command, and run it on the target Linux amd64 + host with sudo, or as root. It prepares the host itself: the `parsar-node` service user, its Docker or + KVM group and a system service. Hosts without sudo use the no-sudo command as a prepared user instead. + It downloads the matched bootstrap and execution assets from your console (never from a release URL), verifies checksums, reads the saved generation and specification without consuming enrollment, verifies the payload matches that Runtime, imports the image only when missing, and writes the matching provider configuration, - registers the node and starts a systemd user service. The service keeps retrying while Core + registers the node and starts its service. The service keeps retrying while Core is unreachable and stops for good once the node is removed. The installer waits for Core to confirm connection and provider readiness. Web refreshes node health automatically. Wait for the node to be online and its provider to be ready; registration alone does not mean it can accept work. A registered retry uses its retained node credential and refuses changed resource or Runtime settings. -The target host needs curl, sha256sum, Python 3.9+, a systemd user session with lingering enabled, -and either Docker socket access or microsandbox's KVM/native-library prerequisites. +The target host needs curl, sha256sum, Python 3.9+ and systemd, plus Docker Engine or +KVM with microsandbox's native libraries; the command never installs them and names +what is missing. The no-sudo command also needs a user with lingering and Docker or KVM +access. See [Register a host](https://github.com/MiniMax-AI/parsar-core/blob/main/services/agents-api/HOSTED-SANDBOX-MANAGER.md#register-a-host) +for both modes and for removing a node from its host. The command checks host access before downloading the Runtime and verifies microsandbox's shared libraries after downloading its native programs. The console serves only fixed, non-secret distribution files at `/node-install/`; diff --git a/docs/getting-started/operations.md b/docs/getting-started/operations.md index 91c0b8652..bbb733f32 100644 --- a/docs/getting-started/operations.md +++ b/docs/getting-started/operations.md @@ -193,8 +193,9 @@ Retain together: - `admin/`, including on zero-node installations, containing the [Core key](#core-key) and its digest file; - each separately installed node's private configuration and persistent identity - directory on its host (`~/.parsar/nodes//` for the Web-generated - installer), as described in the + directory on its host (`/var/lib/parsar-node/.parsar/nodes//` for a + node added with sudo, `~/.parsar/nodes//` for one added as a normal + user), as described in the [node guide](../../services/agents-api/HOSTED-SANDBOX-MANAGER.md#register-a-host); - the same `~/.parsar/nodes//` identity for a local node, including its credential, generation, specification digest and highest accepted owner epoch; diff --git a/services/agents-api/HOSTED-SANDBOX-MANAGER.md b/services/agents-api/HOSTED-SANDBOX-MANAGER.md index d0c5c42ed..b12b59fec 100644 --- a/services/agents-api/HOSTED-SANDBOX-MANAGER.md +++ b/services/agents-api/HOSTED-SANDBOX-MANAGER.md @@ -90,7 +90,8 @@ server configuration and cannot select a different Runtime or resource profile. Closing the dialog discards its one-time command. An expired command requires explicit regeneration; failed or uncertain writes are never retried automatically. The installer checks prerequisites, downloads the matched payload, checks its hashes, prepares provider configuration and starts the existing node -program as a systemd user service. Web polls readiness and capacity while waiting. +program as a service (see [Register a host](#register-a-host)). Web polls readiness +and capacity while waiting. It does not install software through SSH. Registration itself does not create a Session, sandbox or model request. Hosted Session admission fails until setup is complete and a ready node has capacity. E2B allocates directly without this node requirement. @@ -156,9 +157,33 @@ in the bundle's `artifacts/` directory first; `/console/config` lists the provid whose assets it holds (`node_artifacts`). It reuses verified cache entries and exact imported images, then waits for Core to confirm connection and provider readiness. The enrollment token -is transient and never a console/project credential. Python 3.9+, a systemd user -session with lingering, and Docker access or KVM/native-library prerequisites -must already exist on the target host. Rerunning the same command preserves the +is transient and never a console/project credential; the command passes it to the +installer on standard input (`--enrollment-token-stdin`), never in a process argument, +an environment variable or sudo's log. + +The installer chooses how the node runs from the user that runs it: + +- **With sudo or as root (the default command).** It prepares the host itself. It + creates the system user `parsar-node` (home `/var/lib/parsar-node`) if missing, + adds it to the group that owns `/var/run/docker.sock` or `/dev/kvm`, and runs the + node as the root-owned system service `/etc/systemd/system/parsar-node-.service` + with `User=parsar-node`, enabled for boot. No lingering or login session is needed; + logs are in `sudo journalctl -u parsar-node-.service`. Node state + lives in `/var/lib/parsar-node/.parsar/nodes//`, and what the + installer created is recorded in `/etc/parsar-node/`. All node work runs as + `parsar-node`; root only prepares the account and the service. It never installs + Docker, KVM or packages, never changes device permissions and refuses + SELinux-enforcing hosts; a missing prerequisite stops it with a one-line hint + before anything changes. A foreign account named `parsar-node`, or another node + for the same installation on the host, is refused. +- **As a normal user (for hosts without sudo).** The node runs as that user's + systemd user service with state under `~/.parsar/nodes//`. The + user needs lingering and Docker or KVM access, which an administrator grants + beforehand. The installer finds the user's systemd manager even from `su` or + `sudo -iu`, where no login session sets `XDG_RUNTIME_DIR`. + +Python 3.9+, curl and sha256sum, and Docker Engine or KVM with microsandbox's +native libraries must already exist on the target host. Rerunning the same command preserves the node's private identity. A registered retry reads configuration with its retained node credential and `X-Parsar-Node-ID`; it does not enroll again. Changes to the Core origin, installation, generation, specification or release are refused @@ -272,6 +297,14 @@ unknown operations or cleanup records cannot be removed. Resolve those resources through their normal lifecycle and then retry; the API reports the conflict. Offline resources remain owned and visible. Explicit removal permanently retires the node identity; adding that host again requires a fresh private state directory. +After removal, clean up the host with the installer's `--uninstall` (the same +checked download as the add command, then `sudo python3 node-install.pyz --uninstall +--installation-id `, or without sudo for a node installed as a +normal user). It proceeds only when Core answers 401 to the node's credential, or +with `--force` for a Core that no longer exists. It stops and removes the service, +the node state and the Docker network, and deletes the `parsar-node` account only +when the installer created it and no other node uses it. It never removes sandboxes, +volumes or images. The host can then be added again with a new command. The node program then exits with status 78 when Core answers 401 to its credential, and the installed service does not restart it. Other failures, including an unreachable Core or a 403 from a proxy in front of it, restart the service every From 39cc3940e8fc01ab3cf857bca34fa8eb354be458 Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Fri, 25 Sep 2026 23:38:38 +0000 Subject: [PATCH 02/11] Read Docker's CPU quota support under its Engine API name in sudo mode docker info's JSON reports CpuCfsQuota, not the Go field name CPUCfsQuota, so every Docker host was refused as lacking CPU limits. Found on mx2. --- deploy/install/node_install.py | 3 ++- deploy/install/test_node_install.py | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/deploy/install/node_install.py b/deploy/install/node_install.py index 2792112b9..447842ea0 100644 --- a/deploy/install/node_install.py +++ b/deploy/install/node_install.py @@ -624,7 +624,8 @@ def provider_group(provider): details = json.loads(checked(list(DOCKER) + ["info", "--format", "{{json .}}"], "Docker is not running")) except (InstallError, ValueError): raise InstallError("Docker is not running: run `sudo systemctl enable --now docker`, then rerun this command." + NOTHING_CHANGED) from None - if details.get("MemoryLimit") is not True or details.get("CPUCfsQuota") is not True: + # docker info's JSON uses the Engine API names (CpuCfsQuota), not the Go field names. + if details.get("MemoryLimit") is not True or details.get("CpuCfsQuota") is not True: raise InstallError("Docker on this host does not enforce CPU and memory limits; use cgroup v2, then rerun this command." + NOTHING_CHANGED) return group_name(info.st_gid), details try: diff --git a/deploy/install/test_node_install.py b/deploy/install/test_node_install.py index 96c2f6777..c922cba2a 100644 --- a/deploy/install/test_node_install.py +++ b/deploy/install/test_node_install.py @@ -113,7 +113,7 @@ def checked(self, arguments, failure, **kwargs): if arguments[:1] == ["usermod"]: self.joined = True if "{{json .}}" in arguments: - return json.dumps({"MemoryLimit": True, "CPUCfsQuota": True, "NCPU": 8, "MemTotal": 16 << 30}) + return json.dumps({"MemoryLimit": True, "CpuCfsQuota": True, "NCPU": 8, "MemTotal": 16 << 30}) if "load" in arguments: self.image_present = True if "inspect" in arguments: From 23c8232cc4304e152a3d4587cbaea96a74a6625a Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Fri, 25 Sep 2026 23:49:43 +0000 Subject: [PATCH 03/11] Keep sudo-mode records readable and leave no failed unit after uninstall Found on mx2: - /etc/parsar-node was created 0700 under the command's umask 077, so a no-sudo run could not see a sudo-mode node and failed with a generic error instead of refusing. The directory is now 0755 (records stay 0644), and an unreadable one counts as present. - A removed node's service ends failed (exit 78); uninstall now resets that state so systemd lists no leftover unit. - The service-user step no longer repeats root's first two progress lines. --- deploy/install/node_install.py | 22 ++++++++++++++++++---- deploy/install/test_node_install.py | 2 ++ 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/deploy/install/node_install.py b/deploy/install/node_install.py index 447842ea0..8d6268907 100644 --- a/deploy/install/node_install.py +++ b/deploy/install/node_install.py @@ -373,10 +373,12 @@ def open_node(args, token, system=False): safe_directory(root) identity_file = root / "state/node/identity.json" retained = json.loads(identity_file.read_text()) if existing_file(identity_file) else None - print("Reading the Core deployment specification...", flush=True) + if not system: # In sudo mode, root has already said so. + print("Reading the Core deployment specification...", flush=True) args.configuration = node_spec.fetch(args, token, retained, open_request, allow_enrollment=not (root / "registered.json").exists()) args.provider = args.configuration["provider"] - print("Checking host requirements...", flush=True) + if not system: + print("Checking host requirements...", flush=True) preflight(args.provider, system) if args.provider == "microsandbox": runtime_home = micro_home(args.installation_id) @@ -460,7 +462,11 @@ def register_node(root, args, token): def install(args, token): """Non-root mode: the node runs as this user's systemd user service.""" - if (SYSTEM_RECORDS / (args.installation_id + ".json")).exists(): + try: + system_node = (SYSTEM_RECORDS / (args.installation_id + ".json")).exists() + except PermissionError: + system_node = True + if system_node: raise InstallError("This host already runs a node for this installation as a system service. Rerun the command " "with sudo, or uninstall that node with sudo first." + NOTHING_CHANGED) root = open_node(args, token) @@ -564,7 +570,10 @@ def root_file(path, content): if path.stat().st_uid != os.geteuid() or path.read_text() != content: raise InstallError(str(path) + " differs from this installer's version; preserve it and inspect the node") return - path.parent.mkdir(parents=True, exist_ok=True, mode=0o755) + if not path.parent.exists(): + # Readable by every user, so a no-sudo run can see a sudo-mode node; the caller's umask is 077. + path.parent.mkdir(parents=True, mode=0o755) + os.chmod(path.parent, 0o755) descriptor, temporary = tempfile.mkstemp(prefix=".parsar-node-", dir=path.parent) try: with os.fdopen(descriptor, "w") as stream: @@ -837,6 +846,9 @@ def uninstall_system(args): checked(["systemctl", "disable", "--now", unit.name], "Cannot stop the node service " + unit.name) unit.unlink() checked(["systemctl", "daemon-reload"], "Cannot reload systemd") + # A removed node's service ends failed (exit 78); drop that state with the unit. + with contextlib.suppress(InstallError): + checked(["systemctl", "reset-failed", unit.name], "Cannot reset " + unit.name) release_docker_network(root, args.installation_id) remove_node_files(root, args.installation_id, home) record_path.unlink(missing_ok=True) @@ -874,6 +886,8 @@ def uninstall_user(args): remove_node_files(root, args.installation_id, Path.home()) if service: checked(["systemctl", "--user", "daemon-reload"], "Cannot reload the systemd user manager") + with contextlib.suppress(InstallError): + checked(["systemctl", "--user", "reset-failed", unit.name], "Cannot reset " + unit.name) print("Node for installation " + args.installation_id + " uninstalled for this user.") def wait_ready(root, args, timeout=60): diff --git a/deploy/install/test_node_install.py b/deploy/install/test_node_install.py index c922cba2a..30d6d97e7 100644 --- a/deploy/install/test_node_install.py +++ b/deploy/install/test_node_install.py @@ -370,6 +370,7 @@ def test_sudo_mode_prepares_the_host_and_reruns_without_changes(self): self.assertTrue((self.root / "registered.json").exists()) self.assertFalse((self.root / ("parsar-node-" + self.args.installation_id + ".service")).exists()) self.assertEqual(json.loads((system / "etc/user.json").read_text())["created"], True) + self.assertEqual(stat.S_IMODE((system / "etc").stat().st_mode), 0o755) before = {path: path.read_bytes() for path in (system / "etc").iterdir()} self.calls.clear() installer.install_system(self.args, "") @@ -410,6 +411,7 @@ def test_uninstall_waits_for_core_to_reject_the_node(self): commands = [call for call, _ in self.calls] self.assertIn(["systemctl", "disable", "--now", "parsar-node-" + self.args.installation_id + ".service"], commands) self.assertIn(["userdel", "parsar-node"], commands) + self.assertIn(["systemctl", "reset-failed", "parsar-node-" + self.args.installation_id + ".service"], commands) self.assertFalse(any(call[-2:] == ["rm", "--force"] or "prune" in call or ("image" in call and "rm" in call) for call in commands)) self.assertFalse(self.root.exists()) self.assertFalse((system / "units" / ("parsar-node-" + self.args.installation_id + ".service")).exists()) From a3fa5d1a3ca7b1b35a614562ed7c33203545c228 Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Sat, 26 Sep 2026 00:55:07 +0000 Subject: [PATCH 04/11] Keep root out of the service user's files when uninstalling a sudo-mode node Review round 1 of the sudo mode: - Uninstall no longer follows links the service user controls. The Core check and the node-file deletion run with parsar-node's own credentials and refuse any path reached through a symbolic link. Root keeps only systemctl, the Docker network, the account, and the home it removes after userdel succeeds. - The Core address for that check comes from the root-owned record, checked like a command-line origin. The retained identity is read without following links and with the owner and size checked on the open file. - The account is validated before any path is derived from it or it is deleted: its recorded uid, home and nologin shell must match. /etc/parsar-node/account.json records whether the installer created or adopted it and which groups it added. An adopted account is left as found, minus those groups. - A microsandbox store (images and sandbox state) is kept on uninstall, with instructions; a created account stays until it is gone. - The service user joins only the docker or kvm group; a device owned by any other group is refused. Every check, including the home directory and a stale account record, runs before useradd. One host-wide lock serializes installs and uninstalls. Root mode sets its own PATH and refuses a token in the environment (sudo would log it). A second node is detected in any account's home. The service-user step reports unexpected failures clearly. - Resumed downloads send If-Range, so a changed file restarts from zero. - Docs state that Docker mode makes the node root-equivalent, that sudo log_input records standard input, the checked uninstall command, and what uninstall keeps. --- CONTRIBUTING.md | 24 +- deploy/install/distribution.py | 35 +- deploy/install/node_install.py | 300 ++++++++++++------ deploy/install/test_distribution.py | 5 + deploy/install/test_node_install.py | 64 +++- docs/getting-started/install.md | 6 +- services/agents-api/HOSTED-SANDBOX-MANAGER.md | 58 +++- 7 files changed, 367 insertions(+), 125 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1211bbaa5..59b1fdeca 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1844,16 +1844,20 @@ service owns its provider processes outside the Core container. Its `KillMode=pr preserves resident microVM/helper processes across a node-service restart. User KVM access and the Linux runtime libraries are prerequisites for microsandbox. The node installed by a normal user runs as a systemd user service and needs linger. Run as -root (sudo mode), the installer instead prepares the host: it creates the `parsar-node` -system user when missing, adds it to the Docker or KVM device group, and installs one -root-owned system service per installation that runs the same node program with -`User=parsar-node`. Root only prepares the account, group and unit; node work runs -with the service user's credentials, and root never runs a file that user can write. -Sudo mode never installs Docker, KVM or packages, never changes device permissions, -refuses SELinux-enforcing hosts and changes nothing when a prerequisite is missing. -`--uninstall` removes a node only after Core rejects its credential, never touches -sandboxes, volumes or images, and deletes the account only when the installer -created it and no node remains. Do not add any other launcher, scheduler or +root (sudo mode), the installer instead prepares the host: it creates or adopts the +`parsar-node` system user, adds it to the `docker` or `kvm` device group (no other +group), and installs one root-owned system service per installation that runs the +same node program with `User=parsar-node`. Docker group membership makes that user, +and so the node, root-equivalent on the host; that is inherent to Docker sandboxes, +not a least-privilege boundary. Microsandbox needs only `kvm`. Files the service +user owns are read, written and deleted only with its credentials, never by root, +and root never runs a file that user can write or opens a URL it wrote. Sudo mode +never installs Docker, KVM or packages, never changes device permissions, refuses +SELinux-enforcing hosts and a token in the environment, and changes nothing when a +check fails. `--uninstall` removes a node only after Core rejects its credential, +never touches sandboxes, volumes or images (it keeps the Runtime image and the +microsandbox store), deletes the account only when the installer created it and no +node remains, and otherwise removes only the groups it added. Do not add any other launcher, scheduler or recovery path. Node services restart after failures without a start limit, so a node outlasts a Core outage, and stop restarting when the node program exits 78 because Core answered 401 to its credential (a removed or retired node). diff --git a/deploy/install/distribution.py b/deploy/install/distribution.py index d60aa64ee..e61031308 100644 --- a/deploy/install/distribution.py +++ b/deploy/install/distribution.py @@ -162,7 +162,7 @@ def obtain_artifact(manifest, logical_path, destination, offline_root=None): break except urllib.error.HTTPError as error: if error.code == 416: - partial.unlink(missing_ok=True) + discard_partial(partial) elif error.code not in (408, 429, 500, 502, 503, 504) or attempt == 2: raise DistributionError(f'Artifact download failed (HTTP {error.code}): {logical_path}. Check the console and retry.') from None except (urllib.error.URLError, socket.timeout, ConnectionError, http.client.HTTPException): @@ -173,13 +173,23 @@ def obtain_artifact(manifest, logical_path, destination, offline_root=None): else: raise DistributionError('Artifact download did not complete') if digest(partial) != entry['sha256']: - partial.unlink() + discard_partial(partial) raise DistributionError('Artifact checksum mismatch: ' + logical_path) os.chmod(partial, 0o700 if logical_path.startswith('native/') else 0o600) os.replace(partial, target) + validator_path(partial).unlink(missing_ok=True) return target +def validator_path(partial): + return partial.with_name(partial.name + '.validator') + + +def discard_partial(partial): + partial.unlink(missing_ok=True) + validator_path(partial).unlink(missing_ok=True) + + def copy_artifact(source, target, entry, logical_path): fd, temporary = tempfile.mkstemp(prefix='.artifact-', dir=target.parent) try: @@ -213,10 +223,27 @@ def download_partial(url, partial, entry, logical_path): offset = info.st_size if info.st_size <= size else 0 if offset == size: return - request = urllib.request.Request(url, headers={'Range': f'bytes={offset}-'} if offset else {}) + headers = {} + if offset: + headers['Range'] = f'bytes={offset}-' + # If-Range makes a server whose file changed since the partial began send it whole. + try: + validator = validator_path(partial).read_text().strip() + except OSError: + validator = '' + if validator: + headers['If-Range'] = validator + request = urllib.request.Request(url, headers=headers) with urllib.request.build_opener(NoRedirect()).open(request, timeout=30) as stream: if offset and (stream.status != 206 or not stream.headers.get('Content-Range', '').startswith(f'bytes {offset}-')): offset = 0 # The server sent the whole file; start over. + if not offset: + validator = stream.headers.get('ETag') or stream.headers.get('Last-Modified') or '' + if validator and all(32 <= ord(c) < 127 for c in validator) and len(validator) < 200: + with os.fdopen(os.open(validator_path(partial), os.O_WRONLY | os.O_CREAT | os.O_TRUNC | os.O_NOFOLLOW, 0o600), 'w') as note: + note.write(validator) + else: + validator_path(partial).unlink(missing_ok=True) flags = os.O_WRONLY | os.O_CREAT | os.O_NOFOLLOW | (os.O_APPEND if offset else os.O_TRUNC) with os.fdopen(os.open(partial, flags, 0o600), 'ab' if offset else 'wb') as output: count, reported = offset, offset * 10 // size @@ -227,7 +254,7 @@ def download_partial(url, partial, entry, logical_path): count += len(block) if count > size: output.close() - partial.unlink() + discard_partial(partial) raise DistributionError('Artifact exceeds published size: ' + logical_path) output.write(block) if size >= 50 * 1024 * 1024 and count * 10 // size > reported: diff --git a/deploy/install/node_install.py b/deploy/install/node_install.py index 8d6268907..eb5505fde 100644 --- a/deploy/install/node_install.py +++ b/deploy/install/node_install.py @@ -502,7 +502,8 @@ def as_service_user(account, function, *arguments): """Run a step with the service user's credentials in a forked child. Every installer module is already imported, so the child never reads the root - caller's private copy of this program. The token stays in memory.""" + caller's private copy of this program. The token stays in memory. Files the + service user owns are read, written and deleted only here, never by root.""" sys.stdout.flush() sys.stderr.flush() pid = os.fork() @@ -520,8 +521,9 @@ def as_service_user(account, function, *arguments): code = 0 except (InstallError, node_spec.SpecificationError, distribution.DistributionError) as error: print(str(error), file=sys.stderr) - except (OSError, ValueError, KeyError, subprocess.SubprocessError): - print("Node installation failed; check host prerequisites and retained private files", file=sys.stderr) + except Exception as error: # noqa: BLE001 - the child must always report and exit + print("The step running as " + SERVICE_USER + " failed unexpectedly (" + type(error).__name__ + + "). Inspect the host, then rerun the command.", file=sys.stderr) finally: sys.stdout.flush() sys.stderr.flush() @@ -535,13 +537,18 @@ def as_service_user(account, function, *arguments): @contextlib.contextmanager -def system_lock(installation_id): - descriptor = os.open(SYSTEM_LOCKS / ("parsar-node-" + installation_id + ".lock"), os.O_CREAT | os.O_RDWR | os.O_NOFOLLOW, 0o600) +def host_lock(): + """One sudo-mode installation or uninstallation at a time on this host. + + Account preparation and deletion are host-wide, so installing one Core's node can + never race removing another's. The lock lives in /run, which the next boot clears.""" + descriptor = os.open(SYSTEM_LOCKS / "parsar-node.lock", os.O_CREAT | os.O_RDWR | os.O_NOFOLLOW, 0o600) with os.fdopen(descriptor, "w") as lock: try: fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) except BlockingIOError: - raise InstallError("Another node installation or uninstallation for this installation is running") from None + print("Waiting for another node installation or uninstallation on this host...", flush=True) + fcntl.flock(lock, fcntl.LOCK_EX) yield @@ -550,7 +557,7 @@ def host_checks(): raise InstallError("Run with Python 3.9+ on Linux amd64." + NOTHING_CHANGED) if not SYSTEMD_RUNNING.is_dir(): raise InstallError("systemd is not the init system here, so this host cannot run the node service." + NOTHING_CHANGED) - for tool in ("systemctl", "useradd", "usermod", "userdel"): + for tool in ("systemctl", "useradd", "usermod", "userdel", "gpasswd"): if shutil.which(tool) is None: raise InstallError(tool + " is required to prepare the node service user." + NOTHING_CHANGED) try: @@ -562,14 +569,17 @@ def host_checks(): "command as a prepared user instead." + NOTHING_CHANGED) -def root_file(path, content): - """Write a root-owned 0644 file, or accept an identical existing one.""" +def root_file(path, content, replace=False): + """Write a root-owned 0644 file, or accept an identical existing one (or replace it).""" if path.is_symlink() or (path.exists() and not path.is_file()): raise InstallError(str(path) + " must be a regular file") if path.exists(): - if path.stat().st_uid != os.geteuid() or path.read_text() != content: + if path.stat().st_uid != os.geteuid(): + raise InstallError(str(path) + " must be owned by root; preserve it and inspect the node") + if path.read_text() == content: + return + if not replace: raise InstallError(str(path) + " differs from this installer's version; preserve it and inspect the node") - return if not path.parent.exists(): # Readable by every user, so a no-sudo run can see a sudo-mode node; the caller's umask is 077. path.parent.mkdir(parents=True, mode=0o755) @@ -586,7 +596,7 @@ def root_file(path, content): def read_root_json(path): - if not path.exists(): + if not path.exists() and not path.is_symlink(): return None if path.is_symlink() or not path.is_file() or path.stat().st_uid != os.geteuid(): raise InstallError(str(path) + " must be a regular file owned by root") @@ -600,6 +610,12 @@ def service_account(): return None +def ours(account, recorded_uid=None): + """The account the installer created or adopted: its home, a nologin shell and, when recorded, its uid.""" + return (account is not None and account.pw_dir == str(SERVICE_HOME) and account.pw_shell.endswith(("nologin", "false")) + and recorded_uid in (None, account.pw_uid)) + + def host_capacity(provider, resources, docker_info): if provider == "docker": cpus, memory = docker_info.get("NCPU", 0), docker_info.get("MemTotal", 0) @@ -614,6 +630,11 @@ def host_capacity(provider, resources, docker_info): % (cpus, memory // (1024 * 1024), resources["cpus"], resources["memory_mib"]) + NOTHING_CHANGED) +# The service user joins only the group that owns the provider's device, and only +# these names; a device owned by a privileged group (disk, sudo, wheel, adm) is refused. +DEVICE_GROUPS = {"docker": "docker", "microsandbox": "kvm"} + + def provider_group(provider): """Check Docker or KVM without installing either; return the group that grants access.""" if provider == "docker": @@ -636,7 +657,7 @@ def provider_group(provider): # docker info's JSON uses the Engine API names (CpuCfsQuota), not the Go field names. if details.get("MemoryLimit") is not True or details.get("CpuCfsQuota") is not True: raise InstallError("Docker on this host does not enforce CPU and memory limits; use cgroup v2, then rerun this command." + NOTHING_CHANGED) - return group_name(info.st_gid), details + return device_group(provider, DOCKER_SOCKET, info.st_gid), details try: info = KVM.stat() except OSError: @@ -649,26 +670,31 @@ def provider_group(provider): if info.st_gid == 0 or stat.S_IMODE(info.st_mode) & 0o060 != 0o060: raise InstallError("/dev/kvm must be group-accessible, for example root:kvm with mode 0660 (your distribution's " "KVM package sets this), then rerun this command." + NOTHING_CHANGED) - return group_name(info.st_gid), {} + return device_group(provider, KVM, info.st_gid), {} -def group_name(gid): +def device_group(provider, device, gid): try: - return grp.getgrgid(gid).gr_name + name = grp.getgrgid(gid).gr_name except KeyError: - raise InstallError("The device's group " + str(gid) + " has no name; give it a named group, then rerun." + NOTHING_CHANGED) from None + name = str(gid) + if name != DEVICE_GROUPS[provider]: + raise InstallError(str(device) + " belongs to the group " + name + "; the node's service user joins only the " + + DEVICE_GROUPS[provider] + " group. Give the device that group, then rerun this command." + NOTHING_CHANGED) + return name def other_node(args, provider): """Refuse a second node for this installation on the same host or Docker engine.""" - sudo_user = os.environ.get("SUDO_USER", "") - if sudo_user and sudo_user != "root": + for entry in pwd.getpwall(): + if entry.pw_name == SERVICE_USER or not entry.pw_dir.startswith("/"): + continue try: - home = Path(pwd.getpwnam(sudo_user).pw_dir) - except KeyError: - home = None - if home is not None and (home / ".parsar/nodes" / args.installation_id).exists(): - raise InstallError("This host already runs a node for this installation, installed without sudo by " + sudo_user + present = (Path(entry.pw_dir) / ".parsar/nodes" / args.installation_id).exists() + except OSError: + present = False + if present: + raise InstallError("This host already runs a node for this installation, installed without sudo by " + entry.pw_name + ". Remove it on the Nodes page and uninstall it as that user first." + NOTHING_CHANGED) if provider == "docker" and not (SERVICE_HOME / ".parsar/nodes" / args.installation_id).exists(): networks = checked(list(DOCKER) + ["network", "ls", "--format", "{{.Name}}"], "Cannot inspect Docker networks").splitlines() @@ -677,28 +703,43 @@ def other_node(args, provider): + args.installation_id + "). Remove it on the Nodes page and uninstall it first." + NOTHING_CHANGED) -def prepare_account(group): - """Create or adopt the service user and give it the provider's group.""" - account = service_account() +def account_plan(): + """Check the service account, its record and home before changing anything. + + Returns the existing account (or None when it must be created) and the account record.""" + account, record = service_account(), read_root_json(SYSTEM_RECORDS / "account.json") + if account is not None and not ours(account, (record or {}).get("uid")): + raise InstallError("An account named " + SERVICE_USER + " exists but was not created or adopted by this installer " + "(home " + account.pw_dir + ", shell " + account.pw_shell + ", uid " + str(account.pw_uid) + + "). Rename or remove it, then rerun." + NOTHING_CHANGED) + if SERVICE_HOME.is_symlink() or (SERVICE_HOME.exists() and not SERVICE_HOME.is_dir()): + raise InstallError(str(SERVICE_HOME) + " must be a directory, not a symbolic link." + NOTHING_CHANGED) + if SERVICE_HOME.exists() and (account is None or SERVICE_HOME.stat().st_uid != account.pw_uid): + raise InstallError(str(SERVICE_HOME) + " exists but does not belong to the " + SERVICE_USER + " account; preserve " + "it and inspect the host." + NOTHING_CHANGED) + return account, record + + +def prepare_account(account, record, group): + """Create or adopt the service user and give it the provider's group; record what changed.""" if account is None: + # A record of an account that no longer exists is stale; the new account replaces it. + (SYSTEM_RECORDS / "account.json").unlink(missing_ok=True) shell = next((path for path in ("/usr/sbin/nologin", "/sbin/nologin") if os.path.exists(path)), "/bin/false") checked(["useradd", "--system", "--user-group", "--no-create-home", "--home-dir", str(SERVICE_HOME), "--shell", shell, SERVICE_USER], "Cannot create the " + SERVICE_USER + " service user") account = service_account() - root_file(SYSTEM_RECORDS / "user.json", json_text({"format": 1, "created": True, "uid": account.pw_uid})) - elif account.pw_dir != str(SERVICE_HOME) or not account.pw_shell.endswith(("nologin", "false")): - raise InstallError("An account named " + SERVICE_USER + " exists but was not created by this installer (home " - + account.pw_dir + ", shell " + account.pw_shell + "). Rename or remove it, then rerun." + NOTHING_CHANGED) - if SERVICE_HOME.is_symlink(): - raise InstallError(str(SERVICE_HOME) + " must not be a symlink") + record = {"format": 1, "created": True, "uid": account.pw_uid, "groups_added": []} + elif record is None: + record = {"format": 1, "created": False, "uid": account.pw_uid, "groups_added": []} if not SERVICE_HOME.exists(): SERVICE_HOME.mkdir(mode=0o700, parents=True) os.chown(SERVICE_HOME, account.pw_uid, account.pw_gid) - if SERVICE_HOME.stat().st_uid != account.pw_uid: - raise InstallError(str(SERVICE_HOME) + " is owned by another user; preserve it and inspect the host") os.chmod(SERVICE_HOME, 0o700) if group and SERVICE_USER not in grp.getgrnam(group).gr_mem: checked(["usermod", "--append", "--groups", group, SERVICE_USER], "Cannot add " + SERVICE_USER + " to the " + group + " group") + record = dict(record, groups_added=sorted(set(record["groups_added"]) | {group})) + root_file(SYSTEM_RECORDS / "account.json", json_text(record), replace=True) return service_account() @@ -715,12 +756,33 @@ def quote(value): + "\n\n[Install]\nWantedBy=multi-user.target\n") +def recorded_origin(value, source): + """A Core address read from a file, checked with the same rule as the command line.""" + try: + return origin(value) + except (argparse.ArgumentTypeError, TypeError, AttributeError): + raise InstallError(source + " holds an invalid Core address; preserve it and inspect the host." + NOTHING_CHANGED) from None + + +def node_record(installation_id): + """The root-owned record of this installation's sudo-mode node, checked against the fixed paths.""" + record = read_root_json(SYSTEM_RECORDS / (installation_id + ".json")) + if record is not None: + expected = {"installation_id": installation_id, "node_root": str(SERVICE_HOME / ".parsar/nodes" / installation_id), + "unit": str(SYSTEM_UNITS / unit_name(installation_id))} + if any(record.get(key) != value for key, value in expected.items()) or record.get("provider") not in DEVICE_GROUPS: + raise InstallError(str(SYSTEM_RECORDS / (installation_id + ".json")) + " is not this installer's record; preserve " + "it and inspect the host." + NOTHING_CHANGED) + recorded_origin(record.get("core_url"), str(SYSTEM_RECORDS / (installation_id + ".json"))) + return record + + def install_system(args, token): """Sudo mode: prepare the host, then run the node as a root-owned system service.""" + os.environ["PATH"] = SAFE_PATH host_checks() - with system_lock(args.installation_id): - record_path = SYSTEM_RECORDS / (args.installation_id + ".json") - record = read_root_json(record_path) + with host_lock(): + record = node_record(args.installation_id) configuration = None if record is None: print("Reading the Core deployment specification...", flush=True) @@ -736,11 +798,14 @@ def install_system(args, token): if record is None: other_node(args, provider) host_capacity(provider, configuration["specification"]["resources"], details) - account = prepare_account(group) - root = Path(account.pw_dir) / ".parsar/nodes" / args.installation_id + account, account_record = account_plan() + # Every check has passed; from here on the host changes. + account = prepare_account(account, account_record, group) + root = SERVICE_HOME / ".parsar/nodes" / args.installation_id unit = SYSTEM_UNITS / unit_name(args.installation_id) - root_file(record_path, json_text({"format": 1, "installation_id": args.installation_id, "provider": provider, - "core_url": args.core_url, "node_root": str(root), "unit": str(unit)})) + root_file(SYSTEM_RECORDS / (args.installation_id + ".json"), + json_text({"format": 1, "installation_id": args.installation_id, "provider": provider, + "core_url": args.core_url, "node_root": str(root), "unit": str(unit)})) args.system, args.provider = True, provider run_as(account, prepare_service_node, args, token) root_file(unit, system_unit(root, provider)) @@ -756,22 +821,48 @@ def install_system(args, token): # Uninstall ------------------------------------------------------------------- -def confirm_removed(root, owner, force): - """Continue only once Core rejects the node's credential, or when it never registered.""" +def private_json(path): + """Read one of this user's private JSON files without following links or racing a swap.""" + try: + descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW) + except OSError: + return None + with os.fdopen(descriptor) as stream: + info = os.fstat(stream.fileno()) + if not stat.S_ISREG(info.st_mode) or info.st_uid != os.getuid() or info.st_size > 16384: + return None + try: + return json.loads(stream.read()) + except ValueError: + return None + + +def no_links(path): + """Refuse a path whose components include a symbolic link.""" + if os.path.realpath(path) != str(path): + raise InstallError(str(path) + " is reached through a symbolic link; inspect and remove it by hand." + NOTHING_CHANGED) + + +def confirm_removed(root, core_url, force): + """Continue only once Core rejects the node's credential, or when it never registered. + + Runs as the node's own user. The Core address comes from the caller: in sudo mode + the root-owned record, never a file the service user can write.""" + no_links(root) identity_file = root / "state/node/identity.json" if not (root / "registered.json").exists() and not identity_file.exists(): return if force: print("Skipping the Core check (--force).", flush=True) return - if identity_file.is_symlink() or not identity_file.is_file() or identity_file.stat().st_uid != owner: - raise InstallError("Retained node identity is missing or unsafe; rerun with --force only if Core no longer exists." + NOTHING_CHANGED) + stored = private_json(identity_file) try: - stored = json.loads(identity_file.read_text()) - node_id, credential, core = stored["identity"]["node_id"], stored["credential"], stored["core_url"] - except (ValueError, KeyError, TypeError): - raise InstallError("Retained node identity is invalid; rerun with --force only if Core no longer exists." + NOTHING_CHANGED) from None - request = urllib.request.Request(core + "/api/v1/sandbox-node/identity?" + urlencode({"node_id": node_id}), + node_id, credential = stored["identity"]["node_id"], stored["credential"] + if str(uuid.UUID(node_id)) != node_id or not re.fullmatch(r"[0-9a-f]{64}", credential): + raise ValueError() + except (TypeError, KeyError, ValueError): + raise InstallError("Retained node identity is missing or invalid; rerun with --force only if Core no longer exists." + NOTHING_CHANGED) from None + request = urllib.request.Request(core_url + "/api/v1/sandbox-node/identity?" + urlencode({"node_id": node_id}), headers={"Authorization": "Bearer " + credential}) try: with open_request(request, timeout=15): @@ -789,7 +880,7 @@ def confirm_removed(root, owner, force): "if this Core no longer exists." + NOTHING_CHANGED) -def release_docker_network(root, installation_id): +def release_docker_network(installation_id): """Remove the node's Docker network; never containers, volumes or images.""" if shutil.which("docker") is None: return @@ -808,40 +899,46 @@ def release_docker_network(root, installation_id): checked(list(DOCKER) + ["network", "rm", network], "Cannot remove Docker network " + network) -def remove_node_files(root, installation_id, home): - try: - image = json.loads((root / "provider.json").read_text()).get("docker", {}).get("image") - except (OSError, ValueError, AttributeError): - image = None - runtime_home = home / ".parsar/m" / hashlib.sha256(installation_id.encode()).hexdigest()[:12] - try: - owned = json.loads((runtime_home / "parsar-installation.json").read_text()).get("installation_id") == installation_id - except (OSError, ValueError, AttributeError): - owned = False - if owned and not runtime_home.is_symlink(): - shutil.rmtree(runtime_home) - if root.exists() and not root.is_symlink(): +def remove_node_files(root, installation_id): + """Delete the node's state directory; keep the Runtime image and the microsandbox store. + + Runs as the node's own user, so a link it planted can never reach another user's files.""" + no_links(root) + provider = private_json(root / "provider.json") or {} + image = (provider.get("docker") or {}).get("image") + runtime_home = micro_home(installation_id) + if root.exists(): shutil.rmtree(root) if image: print("Kept the Runtime image " + image + "; remove it with `docker image rm " + image + "` if no other node uses it.") + if runtime_home.exists(): + print("Kept the microsandbox store " + str(runtime_home) + " with its images and any sandbox state. When no " + "microVM runs (`pgrep -u " + os.environ.get("USER", "") + "` is empty), remove it with `rm -rf " + str(runtime_home) + + "` (sudo mode: `sudo rm -rf`), then rerun this uninstall command.") def uninstall_system(args): """Sudo mode: remove the system service, the node files and, when unused, the service user.""" + os.environ["PATH"] = SAFE_PATH if shutil.which("systemctl") is None: raise InstallError("systemctl is required." + NOTHING_CHANGED) - with system_lock(args.installation_id): - record_path = SYSTEM_RECORDS / (args.installation_id + ".json") - read_root_json(record_path) - account = service_account() - home = Path(account.pw_dir) if account else SERVICE_HOME - root = home / ".parsar/nodes" / args.installation_id + with host_lock(): + record = node_record(args.installation_id) + account, account_record = service_account(), read_root_json(SYSTEM_RECORDS / "account.json") + root = SERVICE_HOME / ".parsar/nodes" / args.installation_id unit = SYSTEM_UNITS / unit_name(args.installation_id) - if not record_path.exists() and not root.exists() and not unit.exists(): + if record is None and not root.exists() and not unit.exists(): print("No node for installation " + args.installation_id + " was installed with sudo on this host. If it was " "installed without sudo, run the uninstall command as that user without sudo.") else: - confirm_removed(root, account.pw_uid if account else -1, args.force) + if not ours(account, (account_record or {}).get("uid")): + raise InstallError("The " + SERVICE_USER + " account is missing or differs from the one this installer " + "recorded, so its files are not removed. Inspect the host." + NOTHING_CHANGED) + core_url = record["core_url"] if record else None + if core_url is None and not args.force: + raise InstallError("This node's record is missing, so Core cannot be asked; rerun with --force only if " + "Core no longer exists." + NOTHING_CHANGED) + run_as(account, confirm_removed, root, core_url, args.force) if unit.exists(): checked(["systemctl", "disable", "--now", unit.name], "Cannot stop the node service " + unit.name) unit.unlink() @@ -849,25 +946,38 @@ def uninstall_system(args): # A removed node's service ends failed (exit 78); drop that state with the unit. with contextlib.suppress(InstallError): checked(["systemctl", "reset-failed", unit.name], "Cannot reset " + unit.name) - release_docker_network(root, args.installation_id) - remove_node_files(root, args.installation_id, home) - record_path.unlink(missing_ok=True) + release_docker_network(args.installation_id) + run_as(account, remove_node_files, root, args.installation_id) + (SYSTEM_RECORDS / (args.installation_id + ".json")).unlink(missing_ok=True) print("Node for installation " + args.installation_id + " uninstalled from this host.") - remove_unused_account(account) + release_account(account, account_record) -def remove_unused_account(account): - """Delete the service user only when this installer created it and no node uses it.""" - created = read_root_json(SYSTEM_RECORDS / "user.json") - if not account or not created or not created.get("created") or any(SYSTEM_RECORDS.glob("*-*-*-*-*.json")): +def release_account(account, record): + """When no node remains, undo the account changes: delete a created account, or leave an adopted one as found.""" + if record is None or any(SYSTEM_RECORDS.glob("*-*-*-*-*.json")): return - checked(["userdel", SERVICE_USER], "Cannot remove the " + SERVICE_USER + " user; stop its processes and rerun the uninstall command") - if SERVICE_HOME.exists() and not SERVICE_HOME.is_symlink(): - shutil.rmtree(SERVICE_HOME) - (SYSTEM_RECORDS / "user.json").unlink() + if not ours(account, record.get("uid")): + print("The " + SERVICE_USER + " account differs from the one this installer recorded; it was left alone.") + return + if record.get("created"): + stores = sorted(str(path) for path in (SERVICE_HOME / ".parsar/m").glob("*")) if (SERVICE_HOME / ".parsar/m").is_dir() else [] + if stores: + print("Kept the " + SERVICE_USER + " user while microsandbox stores remain: " + ", ".join(stores) + + ". Remove them, then rerun this uninstall command.") + return + checked(["userdel", SERVICE_USER], "Cannot remove the " + SERVICE_USER + " user; stop its processes and rerun the uninstall command") + # No process can run as the deleted user, so nothing races this removal. + if SERVICE_HOME.is_dir() and not SERVICE_HOME.is_symlink(): + shutil.rmtree(SERVICE_HOME) + print("Removed the " + SERVICE_USER + " service user, which this installer created.") + else: + for group in record.get("groups_added", []): + checked(["gpasswd", "--delete", SERVICE_USER, group], "Cannot remove " + SERVICE_USER + " from the " + group + " group") + print("Removed " + SERVICE_USER + " from the " + group + " group, which this installer added.") + (SYSTEM_RECORDS / "account.json").unlink() with contextlib.suppress(OSError): SYSTEM_RECORDS.rmdir() - print("Removed the " + SERVICE_USER + " service user, which this installer created.") def uninstall_user(args): @@ -876,20 +986,27 @@ def uninstall_user(args): print("No node for installation " + args.installation_id + " is installed for this user. If it was installed " "with sudo, run the uninstall command with sudo.") return - confirm_removed(root, os.getuid(), args.force) + identity = private_json(root / "state/node/identity.json") or {} + core_url = identity.get("core_url") + if core_url is not None: + recorded_origin(core_url, str(root / "state/node/identity.json")) + elif (root / "registered.json").exists() and not args.force: + raise InstallError("Retained node identity is missing; rerun with --force only if Core no longer exists." + NOTHING_CHANGED) + confirm_removed(root, core_url, args.force) unit = root / unit_name(args.installation_id) service = unit.exists() if service: user_bus() checked(["systemctl", "--user", "disable", "--now", unit.name], "Cannot stop the node service " + unit.name) - release_docker_network(root, args.installation_id) - remove_node_files(root, args.installation_id, Path.home()) + release_docker_network(args.installation_id) + remove_node_files(root, args.installation_id) if service: checked(["systemctl", "--user", "daemon-reload"], "Cannot reload the systemd user manager") with contextlib.suppress(InstallError): checked(["systemctl", "--user", "reset-failed", unit.name], "Cannot reset " + unit.name) print("Node for installation " + args.installation_id + " uninstalled for this user.") + def wait_ready(root, args, timeout=60): identity_file = root / "state/node/identity.json" if not existing_file(identity_file) or stat.S_IMODE(identity_file.stat().st_mode) != 0o600: @@ -938,10 +1055,13 @@ def wait_ready(root, args, timeout=60): def read_token(args, parser): """The one-time token comes on standard input, never in argv or a sudo command line.""" - # The installer's own local node passes it in this variable to its child process. + # The installer's own local node, which never runs as root, passes it in this variable. environment = os.environ.pop("PARSAR_NODE_ENROLLMENT_TOKEN", None) if args.enrollment_token_stdin and environment is not None: parser.error("pass the enrollment token on standard input only") + if environment is not None and os.geteuid() == 0: + # `sudo VAR=... python3` would put the token in sudo's command line and log. + parser.error("with sudo or as root, pass the enrollment token on standard input with --enrollment-token-stdin") if not args.enrollment_token_stdin: return environment or "" if sys.stdin.isatty(): diff --git a/deploy/install/test_distribution.py b/deploy/install/test_distribution.py index 593531c35..141372d41 100644 --- a/deploy/install/test_distribution.py +++ b/deploy/install/test_distribution.py @@ -24,6 +24,7 @@ def setUp(self): self.data = b'prebuilt artifact' * 1000 self.requests = [] self.ranges = [] + self.if_ranges = [] self.status = 200 self.resumable = False test = self @@ -31,10 +32,12 @@ class Handler(http.server.BaseHTTPRequestHandler): def do_GET(self): test.requests.append(self.path) test.ranges.append(self.headers.get('Range')) + test.if_ranges.append(self.headers.get('If-Range')) if test.resumable: # The first response breaks off halfway; a Range request gets the rest. start = int(self.headers['Range'][6:-1]) if self.headers.get('Range') else 0 self.send_response(206 if start else 200) + self.send_header('Last-Modified', 'Sat, 26 Sep 2026 00:00:00 GMT') if start: self.send_header('Content-Range', f'bytes {start}-{len(test.data) - 1}/{len(test.data)}') self.send_header('Content-Length', str(len(test.data) - start)) @@ -115,6 +118,8 @@ def test_interrupted_download_resumes_with_the_missing_bytes(self): distribution.obtain_artifact(self.manifest, 'native/bin/node', self.root / 'node') self.assertEqual((self.root / 'node').read_bytes(), self.data) self.assertEqual(self.ranges, [None, f'bytes={len(self.data) // 2}-']) + # If-Range: a file that changed since the partial began comes back whole instead of mixed. + self.assertEqual(self.if_ranges, [None, 'Sat, 26 Sep 2026 00:00:00 GMT']) self.assertEqual([path.name for path in self.root.iterdir()], ['node']) def test_offline_and_runtime_expansion_are_verified(self): diff --git a/deploy/install/test_node_install.py b/deploy/install/test_node_install.py index 30d6d97e7..dc630f466 100644 --- a/deploy/install/test_node_install.py +++ b/deploy/install/test_node_install.py @@ -19,6 +19,11 @@ import node_spec +class Response(io.BytesIO): + """An HTTP response body with the status and headers the downloader reads.""" + status, headers = 200, {} + + class NodeInstallTests(unittest.TestCase): def setUp(self): base = Path.home() / ".parsar/tests/node-install" @@ -72,7 +77,7 @@ def artifact_response(self, request, **kwargs): self.assertTrue(url.startswith(self.args.source_url + "/node-install/artifacts/"), url) for name, item in self.manifest["artifacts"].items(): if url.endswith("/" + item["filename"]): - return io.BytesIO(self.payloads[name]) + return Response(self.payloads[name]) raise AssertionError("Unexpected artifact URL: " + url) def refresh_manifest(self): @@ -335,7 +340,7 @@ def sudo_host(self, enforcing=False): docker_socket.bind(str(system / "docker.sock")) self.addCleanup(docker_socket.close) os.chmod(system / "docker.sock", 0o660) - self.account, self.joined, self.docker_installed = None, False, True + self.account, self.joined, self.docker_installed, self.device_group = None, False, True, "docker" service_home = self.home / "service" self.root = service_home / ".parsar/nodes" / self.args.installation_id @@ -348,7 +353,8 @@ def run_as(account, function, *arguments): DOCKER_SOCKET=system / "docker.sock", run_as=run_as, service_account=lambda: self.account), mock.patch.object(installer.shutil, "which", side_effect=lambda tool: None if tool == "docker" and not self.docker_installed else "/usr/bin/" + tool), - mock.patch.object(installer.grp, "getgrnam", side_effect=lambda name: SimpleNamespace(gr_mem=["parsar-node"] if self.joined else []))): + mock.patch.object(installer.grp, "getgrnam", side_effect=lambda name: SimpleNamespace(gr_mem=["parsar-node"] if self.joined else [])), + mock.patch.object(installer.grp, "getgrgid", side_effect=lambda gid: SimpleNamespace(gr_name=self.device_group))): patch.start() self.addCleanup(patch.stop) return system @@ -369,7 +375,8 @@ def test_sudo_mode_prepares_the_host_and_reruns_without_changes(self): self.assertNotIn("synthetic-once-token", unit) self.assertTrue((self.root / "registered.json").exists()) self.assertFalse((self.root / ("parsar-node-" + self.args.installation_id + ".service")).exists()) - self.assertEqual(json.loads((system / "etc/user.json").read_text())["created"], True) + account = json.loads((system / "etc/account.json").read_text()) + self.assertEqual((account["created"], account["groups_added"]), (True, ["docker"])) self.assertEqual(stat.S_IMODE((system / "etc").stat().st_mode), 0o755) before = {path: path.read_bytes() for path in (system / "etc").iterdir()} self.calls.clear() @@ -377,15 +384,25 @@ def test_sudo_mode_prepares_the_host_and_reruns_without_changes(self): commands = [call for call, _ in self.calls] self.assertFalse([call for call in commands if call[:1] in (["useradd"], ["usermod"]) or "register" in call]) self.assertEqual({path: path.read_bytes() for path in (system / "etc").iterdir()}, before) + # A command naming another Core address is refused before anything changes. + self.args.core_url, self.calls[:] = "https://moved.example", [] + with self.assertRaisesRegex(installer.InstallError, "this command uses https://moved.example.*Nothing was changed"): + installer.install_system(self.args, "") + self.assertFalse([call for call, _ in self.calls if call[:1] in (["useradd"], ["usermod"], ["systemctl"])]) + self.assertEqual({path: path.read_bytes() for path in (system / "etc").iterdir()}, before) def test_sudo_mode_refusals_change_nothing(self): foreign = SimpleNamespace(pw_name="parsar-node", pw_uid=4242, pw_gid=4242, pw_dir="/home/parsar-node", pw_shell="/bin/bash") for case, message in (("selinux", "SELinux is enforcing"), ("docker", "Docker Engine is not installed"), - ("account", "not created by this installer")): + ("account", "not created or adopted by this installer"), ("group", "belongs to the group disk"), + ("home", "does not belong to the parsar-node account")): with self.subTest(case=case): system = self.sudo_host(enforcing=case == "selinux") self.docker_installed = case != "docker" self.account = foreign if case == "account" else None + self.device_group = "disk" if case == "group" else "docker" + if case == "home": # a home directory left without its account + (self.home / "service").mkdir() self.calls.clear() with self.assertRaisesRegex(installer.InstallError, message + ".*Nothing was changed"): installer.install_system(self.args, "synthetic-once-token") @@ -395,6 +412,8 @@ def test_sudo_mode_refusals_change_nothing(self): for path in sorted(system.rglob("*"), reverse=True): path.unlink() if not path.is_dir() else path.rmdir() system.rmdir() + if (self.home / "service").exists(): + (self.home / "service").rmdir() def test_uninstall_waits_for_core_to_reject_the_node(self): system = self.sudo_host() @@ -417,6 +436,36 @@ def test_uninstall_waits_for_core_to_reject_the_node(self): self.assertFalse((system / "units" / ("parsar-node-" + self.args.installation_id + ".service")).exists()) self.assertFalse((system / "etc").exists()) + def test_uninstall_leaves_an_adopted_account_as_found(self): + system = self.sudo_host() + (self.home / "service").mkdir(mode=0o700) + self.account = SimpleNamespace(pw_name="parsar-node", pw_uid=os.getuid(), pw_gid=os.getgid(), + pw_dir=str(self.home / "service"), pw_shell="/usr/sbin/nologin") + installer.install_system(self.args, "synthetic-once-token") + self.assertFalse([call for call, _ in self.calls if call[:1] == ["useradd"]]) + self.assertEqual(json.loads((system / "etc/account.json").read_text())["created"], False) + self.calls.clear() + with mock.patch.object(installer, "open_request", side_effect=urllib.error.HTTPError("https://core.example", 401, "", {}, None)): + installer.uninstall_system(SimpleNamespace(installation_id=self.args.installation_id, force=False)) + commands = [call for call, _ in self.calls] + self.assertNotIn(["userdel", "parsar-node"], commands) + self.assertIn(["gpasswd", "--delete", "parsar-node", "docker"], commands) + self.assertTrue((self.home / "service").is_dir()) + self.assertFalse((system / "etc").exists()) + + def test_uninstall_never_follows_a_link_in_the_service_home(self): + # The service user owns its home; a link it plants must not steer deletion elsewhere. + system = self.sudo_host() + installer.install_system(self.args, "synthetic-once-token") + victim = self.home / "victim" + (self.home / "service/.parsar").rename(victim) + (self.home / "service/.parsar").symlink_to(victim) + with mock.patch.object(installer, "open_request", side_effect=urllib.error.HTTPError("https://core.example", 401, "", {}, None)), \ + self.assertRaisesRegex(installer.InstallError, "symbolic link.*Nothing was changed"): + installer.uninstall_system(SimpleNamespace(installation_id=self.args.installation_id, force=True)) + self.assertTrue((victim / "nodes" / self.args.installation_id / "registered.json").exists()) + self.assertTrue((system / "units" / ("parsar-node-" + self.args.installation_id + ".service")).exists()) + def test_token_comes_on_standard_input_only(self): arguments = ["--source-url", self.args.source_url, "--core-url", self.args.core_url, "--installation-id", self.args.installation_id, "--enrollment-token-stdin"] @@ -426,6 +475,11 @@ def test_token_comes_on_standard_input_only(self): self.assertEqual(install.call_args.args[1], "synthetic-once-token") with mock.patch.dict(os.environ, {"PARSAR_NODE_ENROLLMENT_TOKEN": "other"}), self.assertRaises(SystemExit): installer.main(arguments) + # As root, a token in the environment would come from `sudo VAR=...`, which sudo logs. + with mock.patch.dict(os.environ, {"PARSAR_NODE_ENROLLMENT_TOKEN": "other"}), mock.patch.object(installer.os, "geteuid", return_value=0), \ + mock.patch.object(installer, "install_system") as install_system, self.assertRaises(SystemExit): + installer.main(arguments[:-1]) + install_system.assert_not_called() def test_user_manager_bus_is_found_without_a_login_session(self): runtime = self.home / "run-user" diff --git a/docs/getting-started/install.md b/docs/getting-started/install.md index af41a39b4..2708b45be 100644 --- a/docs/getting-started/install.md +++ b/docs/getting-started/install.md @@ -144,8 +144,10 @@ Choose English or Chinese through the System language selector. maintenance and completed cleanup. Microsandbox uses a five-minute idle timeout and one-day snapshot retention. 4. For own-machine hosting, click **Add node**, copy the command, and run it on the target Linux amd64 - host with sudo, or as root. It prepares the host itself: the `parsar-node` service user, its Docker or - KVM group and a system service. Hosts without sudo use the no-sudo command as a prepared user instead. + host with sudo, or as root. It prepares the host itself: the `parsar-node` service user, its `docker` + or `kvm` group and a system service. For Docker nodes, the `docker` group makes that user, and so the + node, root-equivalent on the host; use hosts dedicated to sandboxes. Hosts without sudo use the no-sudo + command as a prepared user instead. It downloads the matched bootstrap and execution assets from your console (never from a release URL), verifies checksums, reads the saved generation and specification without consuming enrollment, diff --git a/services/agents-api/HOSTED-SANDBOX-MANAGER.md b/services/agents-api/HOSTED-SANDBOX-MANAGER.md index b12b59fec..0cff72be9 100644 --- a/services/agents-api/HOSTED-SANDBOX-MANAGER.md +++ b/services/agents-api/HOSTED-SANDBOX-MANAGER.md @@ -165,17 +165,32 @@ The installer chooses how the node runs from the user that runs it: - **With sudo or as root (the default command).** It prepares the host itself. It creates the system user `parsar-node` (home `/var/lib/parsar-node`) if missing, - adds it to the group that owns `/var/run/docker.sock` or `/dev/kvm`, and runs the - node as the root-owned system service `/etc/systemd/system/parsar-node-.service` - with `User=parsar-node`, enabled for boot. No lingering or login session is needed; + or adopts an existing one with that home and a nologin shell. It adds the user to + the `docker` group (Docker) or the `kvm` group (microsandbox), whichever owns the + device; any other device group is refused. The node runs as the root-owned system + service `/etc/systemd/system/parsar-node-.service` with + `User=parsar-node`, enabled for boot. No lingering or login session is needed; logs are in `sudo journalctl -u parsar-node-.service`. Node state lives in `/var/lib/parsar-node/.parsar/nodes//`, and what the - installer created is recorded in `/etc/parsar-node/`. All node work runs as - `parsar-node`; root only prepares the account and the service. It never installs + installer created or changed is recorded in `/etc/parsar-node/`. Files the service + user owns are written and deleted only with that user's credentials; root handles + the account, the group, the unit and the Docker network. It never installs Docker, KVM or packages, never changes device permissions and refuses SELinux-enforcing hosts; a missing prerequisite stops it with a one-line hint before anything changes. A foreign account named `parsar-node`, or another node - for the same installation on the host, is refused. + for the same installation on the host (any user's `~/.parsar/nodes/` + or the same Docker engine), is refused. + + **Docker mode is root-equivalent.** Membership in the `docker` group lets + `parsar-node`, and so anything that controls the node, act as root on that host. + This is inherent to running sandboxes on Docker and equally true in the no-sudo + mode. Add Docker nodes only on hosts dedicated to running sandboxes. Microsandbox + nodes need only the `kvm` group. + + Pass the token on standard input, as the command does. In sudo mode the installer + refuses `PARSAR_NODE_ENROLLMENT_TOKEN`, because `sudo VAR=… python3` records the + variable in sudo's log. A sudoers policy with `log_input` records standard input + as well; the token is single-use and expires after ten minutes. - **As a normal user (for hosts without sudo).** The node runs as that user's systemd user service with state under `~/.parsar/nodes//`. The user needs lingering and Docker or KVM access, which an administrator grants @@ -297,14 +312,29 @@ unknown operations or cleanup records cannot be removed. Resolve those resources through their normal lifecycle and then retry; the API reports the conflict. Offline resources remain owned and visible. Explicit removal permanently retires the node identity; adding that host again requires a fresh private state directory. -After removal, clean up the host with the installer's `--uninstall` (the same -checked download as the add command, then `sudo python3 node-install.pyz --uninstall ---installation-id `, or without sudo for a node installed as a -normal user). It proceeds only when Core answers 401 to the node's credential, or -with `--force` for a Core that no longer exists. It stops and removes the service, -the node state and the Docker network, and deletes the `parsar-node` account only -when the installer created it and no other node uses it. It never removes sandboxes, -volumes or images. The host can then be added again with a new command. +After removal, clean up the host with the installer's `--uninstall`, using the same +checked download as the add command: + +```sh + (umask 077; d=$(mktemp -d) || exit; trap 'rm -rf "$d"' EXIT; s=; [ "$(id -u)" -eq 0 ] || s=sudo +curl -fsS --max-time 30 --max-filesize 1048576 'https://core.example/node-install/node-install.pyz' -o "$d/node-install.pyz" && +printf '%s %s\n' '' "$d/node-install.pyz" | sha256sum -c --status && +$s python3 "$d/node-install.pyz" --uninstall --installation-id '') +``` + +For a node installed as a normal user, run it as that user; it does not use sudo +then. The installer's checksum is `node_installer_sha256` in the console's +`/console/config`. Uninstall proceeds only when Core answers 401 to the node's +credential, or with `--force` for a Core that no longer exists. It stops and removes +the service, the node state and the Docker network. The `parsar-node` account is +deleted only when the installer created it and no node remains; an adopted account +is left as found, minus the groups the installer added. Uninstall never removes +sandboxes, volumes or images: it keeps the Runtime image and a microsandbox node's +store (`/var/lib/parsar-node/.parsar/m/`, its images and any sandbox state), +prints how to remove them, and keeps a created account until that store is gone. +With `--force`, microVMs that `KillMode=process` left running may still use the +store, so check `pgrep -u parsar-node` first. The host can then be added again with +a new command. The node program then exits with status 78 when Core answers 401 to its credential, and the installed service does not restart it. Other failures, including an unreachable Core or a 403 from a proxy in front of it, restart the service every From 0b2d21131278785e6a8f8a06e5dc384f2e8d5a2f Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Sat, 26 Sep 2026 01:06:19 +0000 Subject: [PATCH 05/11] Say what uninstall leaves for an adopted parsar-node account Found on mx2: the account and the home directory the installer created for it stay; only the groups the installer added are removed. --- services/agents-api/HOSTED-SANDBOX-MANAGER.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/services/agents-api/HOSTED-SANDBOX-MANAGER.md b/services/agents-api/HOSTED-SANDBOX-MANAGER.md index 0cff72be9..25ec7b77b 100644 --- a/services/agents-api/HOSTED-SANDBOX-MANAGER.md +++ b/services/agents-api/HOSTED-SANDBOX-MANAGER.md @@ -327,8 +327,8 @@ then. The installer's checksum is `node_installer_sha256` in the console's `/console/config`. Uninstall proceeds only when Core answers 401 to the node's credential, or with `--force` for a Core that no longer exists. It stops and removes the service, the node state and the Docker network. The `parsar-node` account is -deleted only when the installer created it and no node remains; an adopted account -is left as found, minus the groups the installer added. Uninstall never removes +deleted only when the installer created it and no node remains. An adopted account +and its home directory stay; uninstall removes only the groups the installer added. Uninstall never removes sandboxes, volumes or images: it keeps the Runtime image and a microsandbox node's store (`/var/lib/parsar-node/.parsar/m/`, its images and any sandbox state), prints how to remove them, and keeps a created account until that store is gone. From 1273d6d642f7199e2e432215f538678b13928c69 Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Sat, 26 Sep 2026 01:54:22 +0000 Subject: [PATCH 06/11] Detach the service-user step from the administrator's terminal Review round 2, P1: the child that runs as parsar-node kept root's controlling terminal. A program the service user can replace (ldd, msb, parsar-sandbox-node, a docker CLI plugin) could then open /dev/tty and inject commands into the administrator's root shell with TIOCSTI. The child now starts its own session (setsid) before dropping privileges, takes /dev/null as standard input, and writes to pipes the parent relays, so it holds no terminal and cannot open /dev/tty. The token was already read by the parent and stays in memory. Its own session gets no terminal signals, so the parent stops the child's process group on Ctrl-C or any error and reaps it. The service user's docker calls read a root-owned, empty DOCKER_CONFIG directory, so no CLI plugin or credential helper it controls runs. A test runs as_service_user and checks the new session, /dev/null input and DOCKER_CONFIG. --- deploy/install/node_install.py | 80 +++++++++++++++++++++++++++-- deploy/install/test_node_install.py | 15 ++++++ 2 files changed, 92 insertions(+), 3 deletions(-) diff --git a/deploy/install/node_install.py b/deploy/install/node_install.py index eb5505fde..f6b2cd8a5 100644 --- a/deploy/install/node_install.py +++ b/deploy/install/node_install.py @@ -15,7 +15,9 @@ import platform import pwd import re +import selectors import shutil +import signal import socket import stat import subprocess @@ -54,6 +56,9 @@ class InstallError(Exception): SELINUX_ENFORCE = Path("/sys/fs/selinux/enforce") USER_RUNTIME = Path("/run/user") SAFE_PATH = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" +# A root-owned, empty Docker configuration for the service user's docker calls, so no +# CLI plugin or credential helper the service user controls runs in the installer. +CHILD_DOCKER_CONFIG = Path("/run/parsar-node-docker") NOTHING_CHANGED = " Nothing was changed." @@ -498,24 +503,73 @@ class ChildFailed(Exception): """The service-user step failed and already printed why.""" +def relay(output, errors): + """Copy the child's stdout and stderr pipes to ours until both close.""" + streams = {output: sys.stdout, errors: sys.stderr} + with selectors.DefaultSelector() as selector: + for descriptor in streams: + selector.register(descriptor, selectors.EVENT_READ) + while streams: + for key, _ in selector.select(): + data = os.read(key.fd, 65536) + if not data: + selector.unregister(key.fd) + os.close(key.fd) + del streams[key.fd] + continue + streams[key.fd].write(data.decode(errors="replace")) + streams[key.fd].flush() + + +def stop_child(pid): + """End the child and everything it started; its own session gets no terminal signals.""" + with contextlib.suppress(ProcessLookupError): + os.killpg(pid, signal.SIGTERM) + deadline = time.monotonic() + 5 + while time.monotonic() < deadline: + if os.waitpid(pid, os.WNOHANG)[0]: + return + time.sleep(0.1) + with contextlib.suppress(ProcessLookupError): + os.killpg(pid, signal.SIGKILL) + os.waitpid(pid, 0) + + def as_service_user(account, function, *arguments): """Run a step with the service user's credentials in a forked child. + The child starts a new session with /dev/null as standard input and pipes as + output, so no program it runs holds the administrator's terminal (TIOCSTI). Every installer module is already imported, so the child never reads the root - caller's private copy of this program. The token stays in memory. Files the + caller's private copy of this program, and the token stays in memory. Files the service user owns are read, written and deleted only here, never by root.""" sys.stdout.flush() sys.stderr.flush() + output_read, output_write = os.pipe() + errors_read, errors_write = os.pipe() pid = os.fork() if pid == 0: code = 1 try: + os.close(output_read) + os.close(errors_read) + os.setsid() + null = os.open(os.devnull, os.O_RDWR) + os.dup2(null, 0) + os.dup2(output_write, 1) + os.dup2(errors_write, 2) + for descriptor in (null, output_write, errors_write): + if descriptor > 2: + os.close(descriptor) + sys.stdout = open(1, "w", buffering=1, closefd=False) + sys.stderr = open(2, "w", buffering=1, closefd=False) os.setgroups(os.getgrouplist(account.pw_name, account.pw_gid)) os.setgid(account.pw_gid) os.setuid(account.pw_uid) os.umask(0o077) os.environ.clear() - os.environ.update(HOME=account.pw_dir, USER=account.pw_name, LOGNAME=account.pw_name, PATH=SAFE_PATH, LANG="C.UTF-8") + os.environ.update(HOME=account.pw_dir, USER=account.pw_name, LOGNAME=account.pw_name, PATH=SAFE_PATH, + LANG="C.UTF-8", DOCKER_CONFIG=str(CHILD_DOCKER_CONFIG)) os.chdir(account.pw_dir) function(*arguments) code = 0 @@ -528,7 +582,14 @@ def as_service_user(account, function, *arguments): sys.stdout.flush() sys.stderr.flush() os._exit(code) - _, status = os.waitpid(pid, 0) + os.close(output_write) + os.close(errors_write) + try: + relay(output_read, errors_read) + _, status = os.waitpid(pid, 0) + except BaseException: + stop_child(pid) + raise if not os.WIFEXITED(status) or os.WEXITSTATUS(status) != 0: raise ChildFailed() @@ -536,6 +597,17 @@ def as_service_user(account, function, *arguments): run_as = as_service_user +def child_docker_config(): + """Create the root-owned empty directory the service user's docker calls read.""" + path = CHILD_DOCKER_CONFIG + if not path.exists() and not path.is_symlink(): + path.mkdir(mode=0o755) + os.chmod(path, 0o755) + info = path.lstat() + if not stat.S_ISDIR(info.st_mode) or info.st_uid != os.geteuid() or stat.S_IMODE(info.st_mode) & 0o022: + raise InstallError(str(path) + " must be a directory only root can write; inspect the host.") + + @contextlib.contextmanager def host_lock(): """One sudo-mode installation or uninstallation at a time on this host. @@ -801,6 +873,7 @@ def install_system(args, token): account, account_record = account_plan() # Every check has passed; from here on the host changes. account = prepare_account(account, account_record, group) + child_docker_config() root = SERVICE_HOME / ".parsar/nodes" / args.installation_id unit = SYSTEM_UNITS / unit_name(args.installation_id) root_file(SYSTEM_RECORDS / (args.installation_id + ".json"), @@ -938,6 +1011,7 @@ def uninstall_system(args): if core_url is None and not args.force: raise InstallError("This node's record is missing, so Core cannot be asked; rerun with --force only if " "Core no longer exists." + NOTHING_CHANGED) + child_docker_config() run_as(account, confirm_removed, root, core_url, args.force) if unit.exists(): checked(["systemctl", "disable", "--now", unit.name], "Cannot stop the node service " + unit.name) diff --git a/deploy/install/test_node_install.py b/deploy/install/test_node_install.py index dc630f466..efe262fa6 100644 --- a/deploy/install/test_node_install.py +++ b/deploy/install/test_node_install.py @@ -349,6 +349,7 @@ def run_as(account, function, *arguments): function(*arguments) for patch in (mock.patch.multiple(installer, SERVICE_HOME=service_home, SYSTEM_RECORDS=system / "etc", SYSTEM_UNITS=system / "units", SYSTEM_LOCKS=system / "run", + CHILD_DOCKER_CONFIG=system / "docker-config", SYSTEMD_RUNNING=system / "systemd", SELINUX_ENFORCE=system / "selinux", DOCKER_SOCKET=system / "docker.sock", run_as=run_as, service_account=lambda: self.account), @@ -466,6 +467,20 @@ def test_uninstall_never_follows_a_link_in_the_service_home(self): self.assertTrue((victim / "nodes" / self.args.installation_id / "registered.json").exists()) self.assertTrue((system / "units" / ("parsar-node-" + self.args.installation_id + ".service")).exists()) + def test_service_user_step_has_no_terminal_and_reads_nothing(self): + """The forked child starts its own session with /dev/null as input; its output is relayed.""" + account = SimpleNamespace(pw_name="parsar-node", pw_uid=os.getuid(), pw_gid=os.getgid(), + pw_dir=str(self.home), pw_shell="/usr/sbin/nologin") + + def probe(): + print("session-leader=%s stdin=%s docker-config=%s" % (os.getsid(0) == os.getpid(), + os.readlink("/proc/self/fd/0"), os.environ["DOCKER_CONFIG"])) + output = io.StringIO() + with mock.patch.object(installer.os, "setgroups"), mock.patch.object(installer.os, "setgid"), \ + mock.patch.object(installer.os, "setuid"), mock.patch.object(installer.sys, "stdout", output): + installer.as_service_user(account, probe) + self.assertIn("session-leader=True stdin=/dev/null docker-config=" + str(installer.CHILD_DOCKER_CONFIG), output.getvalue()) + def test_token_comes_on_standard_input_only(self): arguments = ["--source-url", self.args.source_url, "--core-url", self.args.core_url, "--installation-id", self.args.installation_id, "--enrollment-token-stdin"] From c6a8dcd5439b5861dd0aeb8b0463a5495ef85880 Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Sat, 26 Sep 2026 01:54:35 +0000 Subject: [PATCH 07/11] Serve one Core per host in sudo mode and keep root off the service home Review round 2, P3s: - account.json records a created account right after useradd, so a later failure cannot make it look adopted. Adoption is refused for uid or gid 0 and for groups beyond its own, docker and kvm. An adopted home's mode is restored on uninstall. - Root no longer follows the service user's links for decisions. Whether a node is installed comes from root-owned records and the unit. The microsandbox-store check opens each component without following links. The second-node check uses /etc/parsar-node, the invoking user's home and the Docker network instead of statting every home. - Sudo mode serves one Core per host, since its nodes share parsar-node; a second Core's node is refused. - The host-wide lock is created only after the refusal checks, re-checks the records it protects, has a bounded wait, and is shared by non-root installs and uninstalls. - Cleanup instructions use sudo -u parsar-node rm -rf. gpasswd is skipped when the user is no longer a member. The unit uses the account's own primary group (no Group=). uninstall_user tolerates an identity file without core_url. - Tests: node work goes through run_as, one Core per host, the account record written after useradd, adoption criteria, and the adopted home's mode. --- CONTRIBUTING.md | 7 +- deploy/install/node_install.py | 264 ++++++++++++------ deploy/install/test_node_install.py | 60 +++- services/agents-api/HOSTED-SANDBOX-MANAGER.md | 13 +- 4 files changed, 241 insertions(+), 103 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 59b1fdeca..b85ce1390 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1847,11 +1847,14 @@ installed by a normal user runs as a systemd user service and needs linger. Run root (sudo mode), the installer instead prepares the host: it creates or adopts the `parsar-node` system user, adds it to the `docker` or `kvm` device group (no other group), and installs one root-owned system service per installation that runs the -same node program with `User=parsar-node`. Docker group membership makes that user, +same node program with `User=parsar-node`. Sudo mode serves one Core per host, +because its nodes share that account. Docker group membership makes that user, and so the node, root-equivalent on the host; that is inherent to Docker sandboxes, not a least-privilege boundary. Microsandbox needs only `kvm`. Files the service user owns are read, written and deleted only with its credentials, never by root, -and root never runs a file that user can write or opens a URL it wrote. Sudo mode +in a child that starts its own session with /dev/null as input, so nothing it runs +can reach the administrator's terminal; root never runs a file that user can write, +opens a URL it wrote, or follows a link in its home. Sudo mode never installs Docker, KVM or packages, never changes device permissions, refuses SELinux-enforcing hosts and a token in the environment, and changes nothing when a check fails. `--uninstall` removes a node only after Core rejects its credential, diff --git a/deploy/install/node_install.py b/deploy/install/node_install.py index f6b2cd8a5..166ffc93e 100644 --- a/deploy/install/node_install.py +++ b/deploy/install/node_install.py @@ -59,6 +59,7 @@ class InstallError(Exception): # A root-owned, empty Docker configuration for the service user's docker calls, so no # CLI plugin or credential helper the service user controls runs in the installer. CHILD_DOCKER_CONFIG = Path("/run/parsar-node-docker") +LOCK_WAIT_SECONDS = 600 NOTHING_CHANGED = " Nothing was changed." @@ -475,7 +476,7 @@ def install(args, token): raise InstallError("This host already runs a node for this installation as a system service. Rerun the command " "with sudo, or uninstall that node with sudo first." + NOTHING_CHANGED) root = open_node(args, token) - with install_lock(root): + with host_lock(), install_lock(root): register_node(root, args, token) unit = root / unit_name(args.installation_id) write_once(unit, service_unit(root)) @@ -610,17 +611,41 @@ def child_docker_config(): @contextlib.contextmanager def host_lock(): - """One sudo-mode installation or uninstallation at a time on this host. - - Account preparation and deletion are host-wide, so installing one Core's node can - never race removing another's. The lock lives in /run, which the next boot clears.""" - descriptor = os.open(SYSTEM_LOCKS / "parsar-node.lock", os.O_CREAT | os.O_RDWR | os.O_NOFOLLOW, 0o600) - with os.fdopen(descriptor, "w") as lock: - try: - fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) - except BlockingIOError: - print("Waiting for another node installation or uninstallation on this host...", flush=True) - fcntl.flock(lock, fcntl.LOCK_EX) + """One node installation or uninstallation at a time on this host. + + Root takes it exclusively around every sudo-mode change, so installing one node + never races removing another's account. Normal users share it when it exists. + It lives in /run, which the next boot clears.""" + path = SYSTEM_LOCKS / "parsar-node.lock" + root = os.geteuid() == 0 + descriptor = None + try: + if root: + descriptor = os.open(path, os.O_CREAT | os.O_RDONLY | os.O_NOFOLLOW, 0o644) + else: + descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW) + except OSError: + if root: + raise + if descriptor is None: + yield # No sudo-mode run has happened on this host since boot. + return + with os.fdopen(descriptor) as lock: + operation = fcntl.LOCK_EX if root else fcntl.LOCK_SH + deadline = time.monotonic() + LOCK_WAIT_SECONDS + waiting = False + while True: + try: + fcntl.flock(lock, operation | fcntl.LOCK_NB) + break + except BlockingIOError: + if time.monotonic() > deadline: + raise InstallError("Another node installation or uninstallation on this host still holds " + + str(path) + "; find it with `sudo fuser " + str(path) + "`, then rerun.") from None + if not waiting: + print("Waiting for another node installation or uninstallation on this host...", flush=True) + waiting = True + time.sleep(1) yield @@ -675,6 +700,28 @@ def read_root_json(path): return json.loads(path.read_text()) +def node_records(): + """Installation IDs of the sudo-mode nodes recorded on this host.""" + if not SYSTEM_RECORDS.is_dir(): + return [] + return sorted(path.name[:-5] for path in SYSTEM_RECORDS.glob("*-*-*-*-*.json")) + + +def listdir_nofollow(base, *parts): + """Entries of base/parts, opening every component without following links; [] if absent or linked.""" + descriptors = [] + try: + descriptors.append(os.open(base, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)) + for part in parts: + descriptors.append(os.open(part, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW, dir_fd=descriptors[-1])) + return sorted(os.listdir(descriptors[-1])) + except OSError: + return [] + finally: + for descriptor in descriptors: + os.close(descriptor) + + def service_account(): try: return pwd.getpwnam(SERVICE_USER) @@ -682,10 +729,23 @@ def service_account(): return None +def group_id(name): + try: + return grp.getgrnam(name).gr_gid + except (KeyError, AttributeError): + return None + + def ours(account, recorded_uid=None): - """The account the installer created or adopted: its home, a nologin shell and, when recorded, its uid.""" - return (account is not None and account.pw_dir == str(SERVICE_HOME) and account.pw_shell.endswith(("nologin", "false")) - and recorded_uid in (None, account.pw_uid)) + """The account the installer created or adopted. + + It has the service home, a nologin shell, the recorded uid when there is one, + neither uid nor gid 0, and no groups besides its own, docker and kvm.""" + if (account is None or account.pw_dir != str(SERVICE_HOME) or not account.pw_shell.endswith(("nologin", "false")) + or recorded_uid not in (None, account.pw_uid) or account.pw_uid == 0 or account.pw_gid == 0): + return False + allowed = {account.pw_gid} | {gid for gid in map(group_id, DEVICE_GROUPS.values()) if gid is not None} + return set(os.getgrouplist(account.pw_name, account.pw_gid)) <= allowed def host_capacity(provider, resources, docker_info): @@ -757,18 +817,26 @@ def device_group(provider, device, gid): def other_node(args, provider): - """Refuse a second node for this installation on the same host or Docker engine.""" - for entry in pwd.getpwall(): - if entry.pw_name == SERVICE_USER or not entry.pw_dir.startswith("/"): - continue + """Refuse a second sudo-mode Core on this host, or a second node for this installation. + + Sudo-mode nodes share the parsar-node account, so one host serves one Core. Nodes + installed without sudo are found in the invoking user's home and, for Docker, by + their network on this engine; other users' homes are not searched.""" + for installation in node_records(): + if installation != args.installation_id: + raise InstallError("This host already runs a sudo-mode node for another Core (installation " + installation + + "). Sudo mode serves one Core per host: remove that node on its Nodes page and " + "uninstall it first." + NOTHING_CHANGED) + sudo_user = os.environ.get("SUDO_USER", "") + if sudo_user and sudo_user != "root": try: - present = (Path(entry.pw_dir) / ".parsar/nodes" / args.installation_id).exists() - except OSError: - present = False - if present: - raise InstallError("This host already runs a node for this installation, installed without sudo by " + entry.pw_name + home = pwd.getpwnam(sudo_user).pw_dir + except KeyError: + home = "" + if home.startswith("/") and listdir_nofollow(home, ".parsar", "nodes", args.installation_id): + raise InstallError("This host already runs a node for this installation, installed without sudo by " + sudo_user + ". Remove it on the Nodes page and uninstall it as that user first." + NOTHING_CHANGED) - if provider == "docker" and not (SERVICE_HOME / ".parsar/nodes" / args.installation_id).exists(): + if provider == "docker": networks = checked(list(DOCKER) + ["network", "ls", "--format", "{{.Name}}"], "Cannot inspect Docker networks").splitlines() if "parsar-node-" + args.installation_id in networks: raise InstallError("Another node for this installation already uses this Docker engine (network parsar-node-" @@ -783,7 +851,7 @@ def account_plan(): if account is not None and not ours(account, (record or {}).get("uid")): raise InstallError("An account named " + SERVICE_USER + " exists but was not created or adopted by this installer " "(home " + account.pw_dir + ", shell " + account.pw_shell + ", uid " + str(account.pw_uid) - + "). Rename or remove it, then rerun." + NOTHING_CHANGED) + + ", or groups beyond its own, docker and kvm). Rename or remove it, then rerun." + NOTHING_CHANGED) if SERVICE_HOME.is_symlink() or (SERVICE_HOME.exists() and not SERVICE_HOME.is_dir()): raise InstallError(str(SERVICE_HOME) + " must be a directory, not a symbolic link." + NOTHING_CHANGED) if SERVICE_HOME.exists() and (account is None or SERVICE_HOME.stat().st_uid != account.pw_uid): @@ -793,17 +861,21 @@ def account_plan(): def prepare_account(account, record, group): - """Create or adopt the service user and give it the provider's group; record what changed.""" + """Create or adopt the service user and give it the provider's group; record every change at once.""" + records = SYSTEM_RECORDS / "account.json" if account is None: # A record of an account that no longer exists is stale; the new account replaces it. - (SYSTEM_RECORDS / "account.json").unlink(missing_ok=True) + records.unlink(missing_ok=True) shell = next((path for path in ("/usr/sbin/nologin", "/sbin/nologin") if os.path.exists(path)), "/bin/false") checked(["useradd", "--system", "--user-group", "--no-create-home", "--home-dir", str(SERVICE_HOME), "--shell", shell, SERVICE_USER], "Cannot create the " + SERVICE_USER + " service user") account = service_account() record = {"format": 1, "created": True, "uid": account.pw_uid, "groups_added": []} + root_file(records, json_text(record), replace=True) elif record is None: - record = {"format": 1, "created": False, "uid": account.pw_uid, "groups_added": []} + record = {"format": 1, "created": False, "uid": account.pw_uid, "groups_added": [], + "home_mode": oct(stat.S_IMODE(SERVICE_HOME.stat().st_mode)) if SERVICE_HOME.exists() else None} + root_file(records, json_text(record), replace=True) if not SERVICE_HOME.exists(): SERVICE_HOME.mkdir(mode=0o700, parents=True) os.chown(SERVICE_HOME, account.pw_uid, account.pw_gid) @@ -811,7 +883,7 @@ def prepare_account(account, record, group): if group and SERVICE_USER not in grp.getgrnam(group).gr_mem: checked(["usermod", "--append", "--groups", group, SERVICE_USER], "Cannot add " + SERVICE_USER + " to the " + group + " group") record = dict(record, groups_added=sorted(set(record["groups_added"]) | {group})) - root_file(SYSTEM_RECORDS / "account.json", json_text(record), replace=True) + root_file(records, json_text(record), replace=True) return service_account() @@ -819,9 +891,10 @@ def system_unit(root, provider): def quote(value): return '"' + str(value).replace("\\", "\\\\").replace('"', '\\"').replace("%", "%%") + '"' after = "network-online.target docker.service" if provider == "docker" else "network-online.target" - # Root owns this file; the service user can change only its own node files. + # Root owns this file; the service user can change only its own node files. The + # service runs with the account's own primary group. return ("[Unit]\nDescription=Parsar sandbox node " + root.name + "\nWants=network-online.target\nAfter=" + after - + "\nStartLimitIntervalSec=0\n\n[Service]\nType=exec\nUser=" + SERVICE_USER + "\nGroup=" + SERVICE_USER + + "\nStartLimitIntervalSec=0\n\n[Service]\nType=exec\nUser=" + SERVICE_USER + "\nExecStart=:" + quote(root / COMMON[0]) + " run --config " + quote(root / "provider.json") + " --state-dir " + quote(root / "state/node") + "\nWorkingDirectory=" + str(root).replace("%", "%%") + "\nRestart=on-failure\nRestartSec=5s\nRestartPreventExitStatus=78\nKillMode=process\nUMask=0077" @@ -853,23 +926,26 @@ def install_system(args, token): """Sudo mode: prepare the host, then run the node as a root-owned system service.""" os.environ["PATH"] = SAFE_PATH host_checks() + # Checks that change nothing run first, so a refusal leaves no trace, not even a lock. + record = node_record(args.installation_id) + configuration = None + if record is None: + print("Reading the Core deployment specification...", flush=True) + configuration = node_spec.fetch(args, token, None, open_request, allow_enrollment=True) + provider = configuration["provider"] + else: + provider = record["provider"] + if record["core_url"] != args.core_url: + raise InstallError("This host's node uses " + record["core_url"] + ", but this command uses " + args.core_url + + ". Remove the node on the Nodes page, uninstall it, then run a new command." + NOTHING_CHANGED) + print("Checking host requirements...", flush=True) + group, details = provider_group(provider) + if record is None: + other_node(args, provider) + host_capacity(provider, configuration["specification"]["resources"], details) with host_lock(): - record = node_record(args.installation_id) - configuration = None - if record is None: - print("Reading the Core deployment specification...", flush=True) - configuration = node_spec.fetch(args, token, None, open_request, allow_enrollment=True) - provider = configuration["provider"] - else: - provider = record["provider"] - if record["core_url"] != args.core_url: - raise InstallError("This host's node uses " + record["core_url"] + ", but this command uses " + args.core_url - + ". Remove the node on the Nodes page, uninstall it, then run a new command." + NOTHING_CHANGED) - print("Checking host requirements...", flush=True) - group, details = provider_group(provider) - if record is None: - other_node(args, provider) - host_capacity(provider, configuration["specification"]["resources"], details) + if node_record(args.installation_id) != record or (record is None and [i for i in node_records() if i != args.installation_id]): + raise InstallError("Another node installation changed this host meanwhile; rerun the command." + NOTHING_CHANGED) account, account_record = account_plan() # Every check has passed; from here on the host changes. account = prepare_account(account, account_record, group) @@ -895,7 +971,7 @@ def install_system(args, token): # Uninstall ------------------------------------------------------------------- def private_json(path): - """Read one of this user's private JSON files without following links or racing a swap.""" + """Read one of this user's private JSON objects without following links or racing a swap.""" try: descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW) except OSError: @@ -905,9 +981,10 @@ def private_json(path): if not stat.S_ISREG(info.st_mode) or info.st_uid != os.getuid() or info.st_size > 16384: return None try: - return json.loads(stream.read()) + value = json.loads(stream.read()) except ValueError: return None + return value if isinstance(value, dict) else None def no_links(path): @@ -928,12 +1005,12 @@ def confirm_removed(root, core_url, force): if force: print("Skipping the Core check (--force).", flush=True) return - stored = private_json(identity_file) + stored = private_json(identity_file) or {} try: node_id, credential = stored["identity"]["node_id"], stored["credential"] if str(uuid.UUID(node_id)) != node_id or not re.fullmatch(r"[0-9a-f]{64}", credential): raise ValueError() - except (TypeError, KeyError, ValueError): + except (TypeError, KeyError, ValueError, AttributeError): raise InstallError("Retained node identity is missing or invalid; rerun with --force only if Core no longer exists." + NOTHING_CHANGED) from None request = urllib.request.Request(core_url + "/api/v1/sandbox-node/identity?" + urlencode({"node_id": node_id}), headers={"Authorization": "Bearer " + credential}) @@ -985,9 +1062,11 @@ def remove_node_files(root, installation_id): if image: print("Kept the Runtime image " + image + "; remove it with `docker image rm " + image + "` if no other node uses it.") if runtime_home.exists(): + sudo_mode = str(Path.home()) == str(SERVICE_HOME) + remove = ("sudo -u " + SERVICE_USER + " rm -rf " if sudo_mode else "rm -rf ") + str(runtime_home) print("Kept the microsandbox store " + str(runtime_home) + " with its images and any sandbox state. When no " - "microVM runs (`pgrep -u " + os.environ.get("USER", "") + "` is empty), remove it with `rm -rf " + str(runtime_home) - + "` (sudo mode: `sudo rm -rf`), then rerun this uninstall command.") + "microVM runs (`pgrep -u " + os.environ.get("USER", "") + "` is empty), remove it with `" + remove + + "`, then rerun this uninstall command.") def uninstall_system(args): @@ -995,14 +1074,19 @@ def uninstall_system(args): os.environ["PATH"] = SAFE_PATH if shutil.which("systemctl") is None: raise InstallError("systemctl is required." + NOTHING_CHANGED) + record = node_record(args.installation_id) + unit = SYSTEM_UNITS / unit_name(args.installation_id) + # Only root-owned files decide whether a node is installed; the service home is not read here. + if record is None and not unit.exists() and not (SYSTEM_RECORDS / "account.json").exists(): + print("No node for installation " + args.installation_id + " was installed with sudo on this host. If it was " + "installed without sudo, run the uninstall command as that user without sudo.") + return with host_lock(): record = node_record(args.installation_id) account, account_record = service_account(), read_root_json(SYSTEM_RECORDS / "account.json") root = SERVICE_HOME / ".parsar/nodes" / args.installation_id - unit = SYSTEM_UNITS / unit_name(args.installation_id) - if record is None and not root.exists() and not unit.exists(): - print("No node for installation " + args.installation_id + " was installed with sudo on this host. If it was " - "installed without sudo, run the uninstall command as that user without sudo.") + if record is None and not unit.exists(): + print("No node for installation " + args.installation_id + " was installed with sudo on this host.") else: if not ours(account, (account_record or {}).get("uid")): raise InstallError("The " + SERVICE_USER + " account is missing or differs from the one this installer " @@ -1029,16 +1113,16 @@ def uninstall_system(args): def release_account(account, record): """When no node remains, undo the account changes: delete a created account, or leave an adopted one as found.""" - if record is None or any(SYSTEM_RECORDS.glob("*-*-*-*-*.json")): + if record is None or node_records(): return if not ours(account, record.get("uid")): print("The " + SERVICE_USER + " account differs from the one this installer recorded; it was left alone.") return if record.get("created"): - stores = sorted(str(path) for path in (SERVICE_HOME / ".parsar/m").glob("*")) if (SERVICE_HOME / ".parsar/m").is_dir() else [] + stores = [str(SERVICE_HOME / ".parsar/m" / name) for name in listdir_nofollow(SERVICE_HOME, ".parsar", "m")] if stores: print("Kept the " + SERVICE_USER + " user while microsandbox stores remain: " + ", ".join(stores) - + ". Remove them, then rerun this uninstall command.") + + ". Remove them with `sudo -u " + SERVICE_USER + " rm -rf `, then rerun this uninstall command.") return checked(["userdel", SERVICE_USER], "Cannot remove the " + SERVICE_USER + " user; stop its processes and rerun the uninstall command") # No process can run as the deleted user, so nothing races this removal. @@ -1047,37 +1131,45 @@ def release_account(account, record): print("Removed the " + SERVICE_USER + " service user, which this installer created.") else: for group in record.get("groups_added", []): - checked(["gpasswd", "--delete", SERVICE_USER, group], "Cannot remove " + SERVICE_USER + " from the " + group + " group") - print("Removed " + SERVICE_USER + " from the " + group + " group, which this installer added.") + try: + member = SERVICE_USER in grp.getgrnam(group).gr_mem + except KeyError: + member = False + if member: + checked(["gpasswd", "--delete", SERVICE_USER, group], "Cannot remove " + SERVICE_USER + " from the " + group + " group") + print("Removed " + SERVICE_USER + " from the " + group + " group, which this installer added.") + if record.get("home_mode") and SERVICE_HOME.is_dir() and not SERVICE_HOME.is_symlink(): + os.chmod(SERVICE_HOME, int(record["home_mode"], 8)) (SYSTEM_RECORDS / "account.json").unlink() with contextlib.suppress(OSError): SYSTEM_RECORDS.rmdir() def uninstall_user(args): - root = Path.home() / ".parsar/nodes" / args.installation_id - if not root.exists(): - print("No node for installation " + args.installation_id + " is installed for this user. If it was installed " - "with sudo, run the uninstall command with sudo.") - return - identity = private_json(root / "state/node/identity.json") or {} - core_url = identity.get("core_url") - if core_url is not None: - recorded_origin(core_url, str(root / "state/node/identity.json")) - elif (root / "registered.json").exists() and not args.force: - raise InstallError("Retained node identity is missing; rerun with --force only if Core no longer exists." + NOTHING_CHANGED) - confirm_removed(root, core_url, args.force) - unit = root / unit_name(args.installation_id) - service = unit.exists() - if service: - user_bus() - checked(["systemctl", "--user", "disable", "--now", unit.name], "Cannot stop the node service " + unit.name) - release_docker_network(args.installation_id) - remove_node_files(root, args.installation_id) - if service: - checked(["systemctl", "--user", "daemon-reload"], "Cannot reload the systemd user manager") - with contextlib.suppress(InstallError): - checked(["systemctl", "--user", "reset-failed", unit.name], "Cannot reset " + unit.name) + with host_lock(): + root = Path.home() / ".parsar/nodes" / args.installation_id + if not root.exists(): + print("No node for installation " + args.installation_id + " is installed for this user. If it was installed " + "with sudo, run the uninstall command with sudo.") + return + identity = private_json(root / "state/node/identity.json") or {} + core_url = identity.get("core_url") + if core_url is not None: + recorded_origin(core_url, str(root / "state/node/identity.json")) + elif (root / "registered.json").exists() and not args.force: + raise InstallError("Retained node identity is missing or invalid; rerun with --force only if Core no longer exists." + NOTHING_CHANGED) + confirm_removed(root, core_url, args.force) + unit = root / unit_name(args.installation_id) + service = unit.exists() + if service: + user_bus() + checked(["systemctl", "--user", "disable", "--now", unit.name], "Cannot stop the node service " + unit.name) + release_docker_network(args.installation_id) + remove_node_files(root, args.installation_id) + if service: + checked(["systemctl", "--user", "daemon-reload"], "Cannot reload the systemd user manager") + with contextlib.suppress(InstallError): + checked(["systemctl", "--user", "reset-failed", unit.name], "Cannot reset " + unit.name) print("Node for installation " + args.installation_id + " uninstalled for this user.") diff --git a/deploy/install/test_node_install.py b/deploy/install/test_node_install.py index efe262fa6..b6969af6b 100644 --- a/deploy/install/test_node_install.py +++ b/deploy/install/test_node_install.py @@ -116,6 +116,8 @@ def checked(self, arguments, failure, **kwargs): self.account = SimpleNamespace(pw_name="parsar-node", pw_uid=os.getuid(), pw_gid=os.getgid(), pw_dir=str(installer.SERVICE_HOME), pw_shell="/usr/sbin/nologin") if arguments[:1] == ["usermod"]: + if getattr(self, "fail_usermod", False): + raise installer.InstallError(failure) self.joined = True if "{{json .}}" in arguments: return json.dumps({"MemoryLimit": True, "CpuCfsQuota": True, "NCPU": 8, "MemTotal": 16 << 30}) @@ -344,7 +346,10 @@ def sudo_host(self, enforcing=False): service_home = self.home / "service" self.root = service_home / ".parsar/nodes" / self.args.installation_id + self.service_steps = [] + def run_as(account, function, *arguments): + self.service_steps.append("wait_ready" if function is installer.wait_ready else function.__name__) with mock.patch.object(installer.Path, "home", return_value=Path(account.pw_dir)): function(*arguments) for patch in (mock.patch.multiple(installer, SERVICE_HOME=service_home, SYSTEM_RECORDS=system / "etc", @@ -374,6 +379,9 @@ def test_sudo_mode_prepares_the_host_and_reruns_without_changes(self): "RestartPreventExitStatus=78", "StartLimitIntervalSec=0"): self.assertIn(line, unit) self.assertNotIn("synthetic-once-token", unit) + self.assertNotIn("Group=", unit) + # Every step that touches the service user's files runs as that user. + self.assertEqual(self.service_steps, ["prepare_service_node", "wait_ready"]) self.assertTrue((self.root / "registered.json").exists()) self.assertFalse((self.root / ("parsar-node-" + self.args.installation_id + ".service")).exists()) account = json.loads((system / "etc/account.json").read_text()) @@ -429,6 +437,7 @@ def test_uninstall_waits_for_core_to_reject_the_node(self): with mock.patch.object(installer, "open_request", side_effect=rejected): installer.uninstall_system(uninstall) commands = [call for call, _ in self.calls] + self.assertEqual(self.service_steps[-2:], ["confirm_removed", "remove_node_files"]) self.assertIn(["systemctl", "disable", "--now", "parsar-node-" + self.args.installation_id + ".service"], commands) self.assertIn(["userdel", "parsar-node"], commands) self.assertIn(["systemctl", "reset-failed", "parsar-node-" + self.args.installation_id + ".service"], commands) @@ -439,7 +448,8 @@ def test_uninstall_waits_for_core_to_reject_the_node(self): def test_uninstall_leaves_an_adopted_account_as_found(self): system = self.sudo_host() - (self.home / "service").mkdir(mode=0o700) + (self.home / "service").mkdir() + os.chmod(self.home / "service", 0o755) self.account = SimpleNamespace(pw_name="parsar-node", pw_uid=os.getuid(), pw_gid=os.getgid(), pw_dir=str(self.home / "service"), pw_shell="/usr/sbin/nologin") installer.install_system(self.args, "synthetic-once-token") @@ -452,20 +462,34 @@ def test_uninstall_leaves_an_adopted_account_as_found(self): self.assertNotIn(["userdel", "parsar-node"], commands) self.assertIn(["gpasswd", "--delete", "parsar-node", "docker"], commands) self.assertTrue((self.home / "service").is_dir()) + self.assertEqual(stat.S_IMODE((self.home / "service").stat().st_mode), 0o755) self.assertFalse((system / "etc").exists()) - def test_uninstall_never_follows_a_link_in_the_service_home(self): - # The service user owns its home; a link it plants must not steer deletion elsewhere. + def test_sudo_mode_serves_one_core_per_host(self): system = self.sudo_host() installer.install_system(self.args, "synthetic-once-token") - victim = self.home / "victim" - (self.home / "service/.parsar").rename(victim) - (self.home / "service/.parsar").symlink_to(victim) - with mock.patch.object(installer, "open_request", side_effect=urllib.error.HTTPError("https://core.example", 401, "", {}, None)), \ - self.assertRaisesRegex(installer.InstallError, "symbolic link.*Nothing was changed"): - installer.uninstall_system(SimpleNamespace(installation_id=self.args.installation_id, force=True)) - self.assertTrue((victim / "nodes" / self.args.installation_id / "registered.json").exists()) - self.assertTrue((system / "units" / ("parsar-node-" + self.args.installation_id + ".service")).exists()) + self.args.installation_id, self.calls[:] = "0c6f35d2-6d7c-4a53-8d5c-3a3b1d3a0f11", [] + with self.assertRaisesRegex(installer.InstallError, "one Core per host.*Nothing was changed"): + installer.install_system(self.args, "synthetic-once-token") + self.assertFalse([call for call, _ in self.calls if call[:1] in (["useradd"], ["usermod"], ["systemctl"])]) + self.assertEqual(installer.node_records(), ["94be54a1-138c-4f30-bc87-b13686272dbe"]) + + def test_created_account_is_recorded_before_any_later_step(self): + system = self.sudo_host() + self.fail_usermod = True + with self.assertRaises(installer.InstallError): + installer.install_system(self.args, "synthetic-once-token") + self.assertEqual(json.loads((system / "etc/account.json").read_text())["created"], True) + + def test_adoption_refuses_root_ids_and_extra_groups(self): + service = str(installer.SERVICE_HOME) + account = SimpleNamespace(pw_name="parsar-node", pw_uid=990, pw_gid=990, pw_dir=service, pw_shell="/usr/sbin/nologin") + with mock.patch.object(installer.os, "getgrouplist", return_value=[990]): + self.assertTrue(installer.ours(account)) + self.assertFalse(installer.ours(SimpleNamespace(**dict(vars(account), pw_uid=0)))) + self.assertFalse(installer.ours(SimpleNamespace(**dict(vars(account), pw_gid=0)))) + with mock.patch.object(installer.os, "getgrouplist", return_value=[990, 27]): + self.assertFalse(installer.ours(account)) def test_service_user_step_has_no_terminal_and_reads_nothing(self): """The forked child starts its own session with /dev/null as input; its output is relayed.""" @@ -481,6 +505,20 @@ def probe(): installer.as_service_user(account, probe) self.assertIn("session-leader=True stdin=/dev/null docker-config=" + str(installer.CHILD_DOCKER_CONFIG), output.getvalue()) + + def test_uninstall_never_follows_a_link_in_the_service_home(self): + # The service user owns its home; a link it plants must not steer deletion elsewhere. + system = self.sudo_host() + installer.install_system(self.args, "synthetic-once-token") + victim = self.home / "victim" + (self.home / "service/.parsar").rename(victim) + (self.home / "service/.parsar").symlink_to(victim) + with mock.patch.object(installer, "open_request", side_effect=urllib.error.HTTPError("https://core.example", 401, "", {}, None)), \ + self.assertRaisesRegex(installer.InstallError, "symbolic link.*Nothing was changed"): + installer.uninstall_system(SimpleNamespace(installation_id=self.args.installation_id, force=True)) + self.assertTrue((victim / "nodes" / self.args.installation_id / "registered.json").exists()) + self.assertTrue((system / "units" / ("parsar-node-" + self.args.installation_id + ".service")).exists()) + def test_token_comes_on_standard_input_only(self): arguments = ["--source-url", self.args.source_url, "--core-url", self.args.core_url, "--installation-id", self.args.installation_id, "--enrollment-token-stdin"] diff --git a/services/agents-api/HOSTED-SANDBOX-MANAGER.md b/services/agents-api/HOSTED-SANDBOX-MANAGER.md index 25ec7b77b..24409d822 100644 --- a/services/agents-api/HOSTED-SANDBOX-MANAGER.md +++ b/services/agents-api/HOSTED-SANDBOX-MANAGER.md @@ -177,9 +177,13 @@ The installer chooses how the node runs from the user that runs it: the account, the group, the unit and the Docker network. It never installs Docker, KVM or packages, never changes device permissions and refuses SELinux-enforcing hosts; a missing prerequisite stops it with a one-line hint - before anything changes. A foreign account named `parsar-node`, or another node - for the same installation on the host (any user's `~/.parsar/nodes/` - or the same Docker engine), is refused. + before anything changes. A foreign account named `parsar-node` is refused. Sudo + mode serves **one Core per host**: every sudo-mode node shares `parsar-node`, so a + node for a second Core is refused. A node for the same installation installed + without sudo is found in the invoking user's home or, for Docker, by its network + on the same engine; other users' homes are not searched. The steps that run as + `parsar-node` start in their own session with no terminal, so nothing they run + can reach the administrator's terminal. **Docker mode is root-equivalent.** Membership in the `docker` group lets `parsar-node`, and so anything that controls the node, act as root on that host. @@ -331,7 +335,8 @@ deleted only when the installer created it and no node remains. An adopted accou and its home directory stay; uninstall removes only the groups the installer added. Uninstall never removes sandboxes, volumes or images: it keeps the Runtime image and a microsandbox node's store (`/var/lib/parsar-node/.parsar/m/`, its images and any sandbox state), -prints how to remove them, and keeps a created account until that store is gone. +prints how to remove them (`sudo -u parsar-node rm -rf `), and keeps a +created account until that store is gone. With `--force`, microVMs that `KillMode=process` left running may still use the store, so check `pgrep -u parsar-node` first. The host can then be added again with a new command. From 220f0675874ca971aaba4ee334fce3510148841b Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Sat, 26 Sep 2026 02:37:55 +0000 Subject: [PATCH 08/11] Stop the service-user step when the installer is interrupted or its terminal closes The child runs in its own session, so a closed terminal's SIGHUP reached only the root parent. The parent died and left the child running as parsar-node, still downloading and holding the host lock; a broken output pipe even counted as a network error and was retried. While the child runs, the parent now turns SIGINT, SIGHUP and SIGTERM (unless ignored, as under nohup) into an error that stops and reaps the child. The child always reaches its exit, even when its output pipe is closed. --- CONTRIBUTING.md | 4 +- deploy/install/node_install.py | 113 +++++++++++------- deploy/install/test_node_install.py | 21 ++++ services/agents-api/HOSTED-SANDBOX-MANAGER.md | 3 +- 4 files changed, 97 insertions(+), 44 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ba78b6bf1..cc0d2e9b2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1867,7 +1867,9 @@ and so the node, root-equivalent on the host; that is inherent to Docker sandbox not a least-privilege boundary. Microsandbox needs only `kvm`. Files the service user owns are read, written and deleted only with its credentials, never by root, in a child that starts its own session with /dev/null as input, so nothing it runs -can reach the administrator's terminal; root never runs a file that user can write, +can reach the administrator's terminal. The installer turns SIGINT, SIGHUP and +SIGTERM into stopping that child, which would otherwise outlive a closed terminal. +Root never runs a file that user can write, opens a URL it wrote, or follows a link in its home. Sudo mode never installs Docker, KVM or packages, never changes device permissions, refuses SELinux-enforcing hosts and a token in the environment, and changes nothing when a diff --git a/deploy/install/node_install.py b/deploy/install/node_install.py index 166ffc93e..e5e10508b 100644 --- a/deploy/install/node_install.py +++ b/deploy/install/node_install.py @@ -536,11 +536,21 @@ def stop_child(pid): os.waitpid(pid, 0) +def interrupted(number, frame): + raise InstallError("The installation was interrupted; run the command again to continue.") + + +# The child's own session gets none of the terminal's signals, so while it runs the +# parent turns these into an error that stops the child before the parent exits. +STOP_SIGNALS = (signal.SIGINT, signal.SIGHUP, signal.SIGTERM) + + def as_service_user(account, function, *arguments): """Run a step with the service user's credentials in a forked child. The child starts a new session with /dev/null as standard input and pipes as output, so no program it runs holds the administrator's terminal (TIOCSTI). + Interrupting the installer or closing its terminal stops the child too. Every installer module is already imported, so the child never reads the root caller's private copy of this program, and the token stays in memory. Files the service user owns are read, written and deleted only here, never by root.""" @@ -548,53 +558,72 @@ def as_service_user(account, function, *arguments): sys.stderr.flush() output_read, output_write = os.pipe() errors_read, errors_write = os.pipe() - pid = os.fork() - if pid == 0: - code = 1 - try: - os.close(output_read) - os.close(errors_read) - os.setsid() - null = os.open(os.devnull, os.O_RDWR) - os.dup2(null, 0) - os.dup2(output_write, 1) - os.dup2(errors_write, 2) - for descriptor in (null, output_write, errors_write): - if descriptor > 2: - os.close(descriptor) - sys.stdout = open(1, "w", buffering=1, closefd=False) - sys.stderr = open(2, "w", buffering=1, closefd=False) - os.setgroups(os.getgrouplist(account.pw_name, account.pw_gid)) - os.setgid(account.pw_gid) - os.setuid(account.pw_uid) - os.umask(0o077) - os.environ.clear() - os.environ.update(HOME=account.pw_dir, USER=account.pw_name, LOGNAME=account.pw_name, PATH=SAFE_PATH, - LANG="C.UTF-8", DOCKER_CONFIG=str(CHILD_DOCKER_CONFIG)) - os.chdir(account.pw_dir) - function(*arguments) - code = 0 - except (InstallError, node_spec.SpecificationError, distribution.DistributionError) as error: - print(str(error), file=sys.stderr) - except Exception as error: # noqa: BLE001 - the child must always report and exit - print("The step running as " + SERVICE_USER + " failed unexpectedly (" + type(error).__name__ - + "). Inspect the host, then rerun the command.", file=sys.stderr) - finally: - sys.stdout.flush() - sys.stderr.flush() - os._exit(code) - os.close(output_write) - os.close(errors_write) + previous = {number: signal.getsignal(number) for number in STOP_SIGNALS} + for number, handler in previous.items(): + if handler is not signal.SIG_IGN: # As under nohup, an ignored signal stays ignored. + signal.signal(number, interrupted) try: - relay(output_read, errors_read) - _, status = os.waitpid(pid, 0) - except BaseException: - stop_child(pid) - raise + pid = os.fork() + if pid == 0: + service_child(account, function, arguments, (output_read, errors_read), output_write, errors_write) + os.close(output_write) + os.close(errors_write) + try: + relay(output_read, errors_read) + _, status = os.waitpid(pid, 0) + except BaseException: + for number in STOP_SIGNALS: # A second signal must not cut the stop short. + signal.signal(number, signal.SIG_IGN) + stop_child(pid) + raise + finally: + for number, handler in previous.items(): + signal.signal(number, handler) if not os.WIFEXITED(status) or os.WEXITSTATUS(status) != 0: raise ChildFailed() +def service_child(account, function, arguments, read_ends, output_write, errors_write): + """The forked child of as_service_user; it never returns into the root caller's code.""" + code = 1 + try: + for number in STOP_SIGNALS: + signal.signal(number, signal.SIG_DFL) + for descriptor in read_ends: + os.close(descriptor) + os.setsid() + null = os.open(os.devnull, os.O_RDWR) + os.dup2(null, 0) + os.dup2(output_write, 1) + os.dup2(errors_write, 2) + for descriptor in (null, output_write, errors_write): + if descriptor > 2: + os.close(descriptor) + sys.stdout = open(1, "w", buffering=1, closefd=False) + sys.stderr = open(2, "w", buffering=1, closefd=False) + os.setgroups(os.getgrouplist(account.pw_name, account.pw_gid)) + os.setgid(account.pw_gid) + os.setuid(account.pw_uid) + os.umask(0o077) + os.environ.clear() + os.environ.update(HOME=account.pw_dir, USER=account.pw_name, LOGNAME=account.pw_name, PATH=SAFE_PATH, + LANG="C.UTF-8", DOCKER_CONFIG=str(CHILD_DOCKER_CONFIG)) + os.chdir(account.pw_dir) + function(*arguments) + code = 0 + except (InstallError, node_spec.SpecificationError, distribution.DistributionError) as error: + print(str(error), file=sys.stderr) + except Exception as error: # noqa: BLE001 - the child must always report and exit + print("The step running as " + SERVICE_USER + " failed unexpectedly (" + type(error).__name__ + + "). Inspect the host, then rerun the command.", file=sys.stderr) + finally: + # A closed output pipe must not skip the exit below. + for stream in (sys.stdout, sys.stderr): + with contextlib.suppress(BaseException): + stream.flush() + os._exit(code) + + run_as = as_service_user diff --git a/deploy/install/test_node_install.py b/deploy/install/test_node_install.py index b6969af6b..2bbb19790 100644 --- a/deploy/install/test_node_install.py +++ b/deploy/install/test_node_install.py @@ -6,10 +6,12 @@ import json import os from pathlib import Path +import signal import socket import stat import subprocess import tempfile +import time import unittest import urllib.error from types import SimpleNamespace @@ -505,6 +507,25 @@ def probe(): installer.as_service_user(account, probe) self.assertIn("session-leader=True stdin=/dev/null docker-config=" + str(installer.CHILD_DOCKER_CONFIG), output.getvalue()) + def test_closing_the_terminal_stops_the_service_user_step(self): + """The child's own session misses SIGHUP; the installer must stop and reap it before it exits.""" + account = SimpleNamespace(pw_name="parsar-node", pw_uid=os.getuid(), pw_gid=os.getgid(), + pw_dir=str(self.home), pw_shell="/usr/sbin/nologin") + record = self.home / "child.pid" + + def long_step(): + record.write_text(str(os.getpid())) + os.kill(os.getppid(), signal.SIGHUP) # The administrator's terminal closes. + time.sleep(60) + started = time.monotonic() + with mock.patch.object(installer.os, "setgroups"), mock.patch.object(installer.os, "setgid"), \ + mock.patch.object(installer.os, "setuid"), mock.patch.object(installer.sys, "stdout", io.StringIO()), \ + self.assertRaisesRegex(installer.InstallError, "interrupted"): + installer.as_service_user(account, long_step) + self.assertLess(time.monotonic() - started, 30) + with self.assertRaises(ProcessLookupError): # Stopped and reaped, not left running. + os.kill(int(record.read_text()), 0) + def test_uninstall_never_follows_a_link_in_the_service_home(self): # The service user owns its home; a link it plants must not steer deletion elsewhere. diff --git a/services/agents-api/HOSTED-SANDBOX-MANAGER.md b/services/agents-api/HOSTED-SANDBOX-MANAGER.md index 5833ffd6c..0912148e2 100644 --- a/services/agents-api/HOSTED-SANDBOX-MANAGER.md +++ b/services/agents-api/HOSTED-SANDBOX-MANAGER.md @@ -183,7 +183,8 @@ The installer chooses how the node runs from the user that runs it: without sudo is found in the invoking user's home or, for Docker, by its network on the same engine; other users' homes are not searched. The steps that run as `parsar-node` start in their own session with no terminal, so nothing they run - can reach the administrator's terminal. + can reach the administrator's terminal. Interrupting the installer or closing + its terminal stops those steps as well. **Docker mode is root-equivalent.** Membership in the `docker` group lets `parsar-node`, and so anything that controls the node, act as root on that host. From 53c09c4c089f3e35457f194f31ce71f9ad8dec06 Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Sat, 26 Sep 2026 02:45:39 +0000 Subject: [PATCH 09/11] Run the hangup test with SIGHUP at its default, as in a terminal session Under nohup, as in the gate, SIGHUP is ignored and the installer rightly leaves it ignored, so the test must restore the terminal default itself. --- deploy/install/test_node_install.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/deploy/install/test_node_install.py b/deploy/install/test_node_install.py index 2bbb19790..c49a91c1f 100644 --- a/deploy/install/test_node_install.py +++ b/deploy/install/test_node_install.py @@ -512,17 +512,19 @@ def test_closing_the_terminal_stops_the_service_user_step(self): account = SimpleNamespace(pw_name="parsar-node", pw_uid=os.getuid(), pw_gid=os.getgid(), pw_dir=str(self.home), pw_shell="/usr/sbin/nologin") record = self.home / "child.pid" + # As in a terminal session; a runner under nohup would otherwise ignore SIGHUP. + self.addCleanup(signal.signal, signal.SIGHUP, signal.signal(signal.SIGHUP, signal.SIG_DFL)) def long_step(): record.write_text(str(os.getpid())) os.kill(os.getppid(), signal.SIGHUP) # The administrator's terminal closes. - time.sleep(60) + time.sleep(30) started = time.monotonic() with mock.patch.object(installer.os, "setgroups"), mock.patch.object(installer.os, "setgid"), \ mock.patch.object(installer.os, "setuid"), mock.patch.object(installer.sys, "stdout", io.StringIO()), \ self.assertRaisesRegex(installer.InstallError, "interrupted"): installer.as_service_user(account, long_step) - self.assertLess(time.monotonic() - started, 30) + self.assertLess(time.monotonic() - started, 15) with self.assertRaises(ProcessLookupError): # Stopped and reaped, not left running. os.kill(int(record.read_text()), 0) From 0dea1280572a94fe3e4b7aa60150d60663acb919 Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Sat, 26 Sep 2026 03:18:59 +0000 Subject: [PATCH 10/11] Keep the host lock readable by no-sudo installs under the command's umask The Web command runs under umask 077, so root created /run/parsar-node.lock as 0600 and a no-sudo install on the same host could not open it to share the lock; it then went ahead without waiting. Root now sets the lock file to 0644. --- deploy/install/node_install.py | 1 + deploy/install/test_node_install.py | 10 ++++++++++ 2 files changed, 11 insertions(+) diff --git a/deploy/install/node_install.py b/deploy/install/node_install.py index 7ede17ba9..f783da57d 100644 --- a/deploy/install/node_install.py +++ b/deploy/install/node_install.py @@ -651,6 +651,7 @@ def host_lock(): try: if root: descriptor = os.open(path, os.O_CREAT | os.O_RDONLY | os.O_NOFOLLOW, 0o644) + os.fchmod(descriptor, 0o644) # Readable by normal users despite the command's umask 077. else: descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW) except OSError: diff --git a/deploy/install/test_node_install.py b/deploy/install/test_node_install.py index c49a91c1f..c00adcaf2 100644 --- a/deploy/install/test_node_install.py +++ b/deploy/install/test_node_install.py @@ -402,6 +402,16 @@ def test_sudo_mode_prepares_the_host_and_reruns_without_changes(self): self.assertFalse([call for call, _ in self.calls if call[:1] in (["useradd"], ["usermod"], ["systemctl"])]) self.assertEqual({path: path.read_bytes() for path in (system / "etc").iterdir()}, before) + def test_no_sudo_installs_can_share_the_host_lock_root_created(self): + locks = self.home / "run" + locks.mkdir() + previous = os.umask(0o077) # As the Web command sets it. + self.addCleanup(os.umask, previous) + with mock.patch.object(installer, "SYSTEM_LOCKS", locks), mock.patch.object(installer.os, "geteuid", return_value=0): + with installer.host_lock(): + pass + self.assertEqual(stat.S_IMODE((locks / "parsar-node.lock").stat().st_mode), 0o644) + def test_sudo_mode_refusals_change_nothing(self): foreign = SimpleNamespace(pw_name="parsar-node", pw_uid=4242, pw_gid=4242, pw_dir="/home/parsar-node", pw_shell="/bin/bash") for case, message in (("selinux", "SELinux is enforcing"), ("docker", "Docker Engine is not installed"), From 0ef0d4bdbd34c0ada6f2651ce25ec2186a1bf06f Mon Sep 17 00:00:00 2001 From: SaladDay <1203511142@qq.com> Date: Sat, 26 Sep 2026 04:03:03 +0000 Subject: [PATCH 11/11] Show service-user output as plain text and isolate the service-user step further Review round 3 of the sudo-mode node installer: - Relay the child's output through an incremental UTF-8 decoder and replace C0 controls other than tab and newline, DEL and C1 with "?", so nothing the service user controls can write OSC 52, retitle, erase or forge lines on root's terminal. It also writes in the terminal's own encoding and keeps characters split across reads intact. Uninstall prints the kept Runtime image only when provider.json holds an image ID, and store names only as plain text. - After setuid the child joins a new, empty session keyring, so it cannot use the administrator's keys (keyrings(7)). - Block SIGINT, SIGHUP and SIGTERM across the fork; each side installs its own handlers before unblocking. - The child dies with the parent (PR_SET_PDEATHSIG) and exits at once if the parent is already gone. A closed output pipe stops a download instead of being retried as a network error. - stop_child keeps the child a zombie while it waits, then SIGKILLs its whole process group, so programs that ignore SIGTERM end too. - Ctrl-C outside the service-user steps, as while waiting for the host lock, prints the interrupted message instead of a traceback; a child ended by a signal is reported as such. - A download that brings less than 64 KiB in 60 s stops, keeping its part for the next run to resume. --- CONTRIBUTING.md | 8 +- deploy/install/distribution.py | 21 ++- deploy/install/node_install.py | 141 ++++++++++++++---- deploy/install/test_distribution.py | 7 + deploy/install/test_node_install.py | 119 ++++++++++++--- services/agents-api/HOSTED-SANDBOX-MANAGER.md | 8 +- 6 files changed, 244 insertions(+), 60 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index bfbc996db..62e94d8a5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1868,9 +1868,11 @@ and so the node, root-equivalent on the host; that is inherent to Docker sandbox not a least-privilege boundary. Microsandbox needs only `kvm`. Files the service user owns are read, written and deleted only with its credentials, never by root, in a child that starts its own session with /dev/null as input, so nothing it runs -can reach the administrator's terminal. The installer turns SIGINT, SIGHUP and -SIGTERM into stopping that child, which would otherwise outlive a closed terminal. -Root never runs a file that user can write, +can reach the administrator's terminal. That child also joins a new session +keyring and dies with its parent, and root shows its output only as plain text +(terminal controls become `?`). The installer turns SIGINT, SIGHUP and SIGTERM +into stopping that child and what it started, which would otherwise outlive a +closed terminal. Root never runs a file that user can write, opens a URL it wrote, or follows a link in its home. Sudo mode never installs Docker, KVM or packages, never changes device permissions, refuses SELinux-enforcing hosts and a token in the environment, and changes nothing when a diff --git a/deploy/install/distribution.py b/deploy/install/distribution.py index e61031308..1a0f77fe9 100644 --- a/deploy/install/distribution.py +++ b/deploy/install/distribution.py @@ -210,6 +210,10 @@ def copy_artifact(source, target, entry, logical_path): os.unlink(temporary) +# A download that brings fewer bytes than this in a window stops; a rerun resumes it. +SLOW_SECONDS, SLOW_BYTES = 60, 64 * 1024 + + def download_partial(url, partial, entry, logical_path): """Complete the partial file, asking only for the bytes it is missing.""" size = entry['size'] @@ -247,8 +251,9 @@ def download_partial(url, partial, entry, logical_path): flags = os.O_WRONLY | os.O_CREAT | os.O_NOFOLLOW | (os.O_APPEND if offset else os.O_TRUNC) with os.fdopen(os.open(partial, flags, 0o600), 'ab' if offset else 'wb') as output: count, reported = offset, offset * 10 // size + window, window_count = time.monotonic(), 0 while True: - block = stream.read(1024 * 1024) + block = stream.read1(1024 * 1024) if not block: break count += len(block) @@ -257,9 +262,21 @@ def download_partial(url, partial, entry, logical_path): discard_partial(partial) raise DistributionError('Artifact exceeds published size: ' + logical_path) output.write(block) + window_count += len(block) + if time.monotonic() - window >= SLOW_SECONDS: + if window_count < SLOW_BYTES: + raise DistributionError(f'Artifact download stalled (under {SLOW_BYTES // 1024} KiB in {SLOW_SECONDS} s): ' + f'{logical_path}. The downloaded part is kept; check the network, then rerun ' + 'the command to resume.') + window, window_count = time.monotonic(), 0 if size >= 50 * 1024 * 1024 and count * 10 // size > reported: reported = count * 10 // size - print(f'Downloading {logical_path}: {reported * 10}% of {size // (1024 * 1024)} MiB', flush=True) + try: + print(f'Downloading {logical_path}: {reported * 10}% of {size // (1024 * 1024)} MiB', flush=True) + except BrokenPipeError: + # Nobody reads the output any more (the installer's terminal is gone); + # stop instead of retrying it as a network error. + raise DistributionError('Output closed; stopped downloading ' + logical_path) from None if count != size: raise http.client.IncompleteRead(b'', size - count) diff --git a/deploy/install/node_install.py b/deploy/install/node_install.py index f783da57d..72ebe01c5 100644 --- a/deploy/install/node_install.py +++ b/deploy/install/node_install.py @@ -1,7 +1,10 @@ #!/usr/bin/env python3 """Install and enroll one node from its Core console's matched distribution.""" import argparse +import codecs import contextlib +import ctypes +import errno import fcntl import getpass import grp @@ -504,53 +507,97 @@ class ChildFailed(Exception): """The service-user step failed and already printed why.""" +# What a service-user step must not send to the administrator's terminal: C0 controls +# other than tab and newline, DEL and C1. ESC starts OSC 52 clipboard writes, title +# changes and cursor moves; carriage returns and erases could hide or forge lines. +TERMINAL_CONTROLS = re.compile(r"[\x00-\x08\x0b-\x1f\x7f-\x9f]") + + +def plain(text, encoding=None): + """Text the terminal shows as written, in an encoding the stream can write.""" + encoding = encoding or "utf-8" + return TERMINAL_CONTROLS.sub("?", text).encode(encoding, "replace").decode(encoding) + + def relay(output, errors): - """Copy the child's stdout and stderr pipes to ours until both close.""" - streams = {output: sys.stdout, errors: sys.stderr} + """Copy the child's stdout and stderr pipes to ours as plain text until both close.""" + streams = {output: (sys.stdout, codecs.getincrementaldecoder("utf-8")("replace")), + errors: (sys.stderr, codecs.getincrementaldecoder("utf-8")("replace"))} with selectors.DefaultSelector() as selector: for descriptor in streams: selector.register(descriptor, selectors.EVENT_READ) while streams: for key, _ in selector.select(): data = os.read(key.fd, 65536) + stream, decoder = streams[key.fd] + # The decoder keeps a character split across reads until its last byte arrives. + text = plain(decoder.decode(data, final=not data), getattr(stream, "encoding", None)) + if text: + stream.write(text) + stream.flush() if not data: selector.unregister(key.fd) os.close(key.fd) del streams[key.fd] - continue - streams[key.fd].write(data.decode(errors="replace")) - streams[key.fd].flush() + + +STOP_WAIT_SECONDS = 5 + + +def signal_child(pid, number): + # The child's own process group once it has started its session, and the child + # itself before then. It is never reaped before this, so pid is still its own. + for send in (os.killpg, os.kill): + with contextlib.suppress(ProcessLookupError): + send(pid, number) def stop_child(pid): """End the child and everything it started; its own session gets no terminal signals.""" - with contextlib.suppress(ProcessLookupError): - os.killpg(pid, signal.SIGTERM) - deadline = time.monotonic() + 5 - while time.monotonic() < deadline: - if os.waitpid(pid, os.WNOHANG)[0]: - return + def exited(): + # WNOWAIT leaves the child a zombie, so its process group ID cannot be reused + # before the SIGKILL below reaches what the child left running. + return os.waitid(os.P_PID, pid, os.WEXITED | os.WNOHANG | os.WNOWAIT) is not None + try: + exited() + except ChildProcessError: + return # Already reaped. + signal_child(pid, signal.SIGTERM) + deadline = time.monotonic() + STOP_WAIT_SECONDS + while not exited() and time.monotonic() < deadline: time.sleep(0.1) - with contextlib.suppress(ProcessLookupError): - os.killpg(pid, signal.SIGKILL) + signal_child(pid, signal.SIGKILL) os.waitpid(pid, 0) def interrupted(number, frame): - raise InstallError("The installation was interrupted; run the command again to continue.") + raise InstallError(INTERRUPTED) +INTERRUPTED = "The command was interrupted; run it again to continue." # The child's own session gets none of the terminal's signals, so while it runs the # parent turns these into an error that stops the child before the parent exits. STOP_SIGNALS = (signal.SIGINT, signal.SIGHUP, signal.SIGTERM) +KEYCTL_SYSCALL = 250 # x86_64, the only architecture this installer supports. +KEYCTL_JOIN_SESSION_KEYRING = 1 +PR_SET_PDEATHSIG = 1 + + +def libc_call(name, *arguments): + function = getattr(ctypes.CDLL(None, use_errno=True), name) + function.restype = ctypes.c_long + if function(*(ctypes.c_long(argument) for argument in arguments)) == -1: + number = ctypes.get_errno() + raise OSError(number, os.strerror(number)) def as_service_user(account, function, *arguments): """Run a step with the service user's credentials in a forked child. The child starts a new session with /dev/null as standard input and pipes as - output, so no program it runs holds the administrator's terminal (TIOCSTI). - Interrupting the installer or closing its terminal stops the child too. + output, so no program it runs holds the administrator's terminal (TIOCSTI); the + parent shows that output only as plain text. Interrupting the installer or + closing its terminal stops the child too, and the child dies with the parent. Every installer module is already imported, so the child never reads the root caller's private copy of this program, and the token stays in memory. Files the service user owns are read, written and deleted only here, never by root.""" @@ -558,37 +605,49 @@ def as_service_user(account, function, *arguments): sys.stderr.flush() output_read, output_write = os.pipe() errors_read, errors_write = os.pipe() + parent = os.getpid() previous = {number: signal.getsignal(number) for number in STOP_SIGNALS} - for number, handler in previous.items(): - if handler is not signal.SIG_IGN: # As under nohup, an ignored signal stays ignored. - signal.signal(number, interrupted) + # Held from before the fork until each side has its handlers, so neither a signal + # nor the resulting error can reach the wrong process. + mask = signal.pthread_sigmask(signal.SIG_BLOCK, STOP_SIGNALS) + pid = None try: + for number, handler in previous.items(): + if handler is not signal.SIG_IGN: # As under nohup, an ignored signal stays ignored. + signal.signal(number, interrupted) pid = os.fork() if pid == 0: - service_child(account, function, arguments, (output_read, errors_read), output_write, errors_write) + service_child(account, function, arguments, parent, mask, (output_read, errors_read), output_write, errors_write) + signal.pthread_sigmask(signal.SIG_SETMASK, mask) os.close(output_write) os.close(errors_write) - try: - relay(output_read, errors_read) - _, status = os.waitpid(pid, 0) - except BaseException: - for number in STOP_SIGNALS: # A second signal must not cut the stop short. - signal.signal(number, signal.SIG_IGN) + relay(output_read, errors_read) + _, status = os.waitpid(pid, 0) + except BaseException: + for number in STOP_SIGNALS: # A second signal must not cut the stop short. + signal.signal(number, signal.SIG_IGN) + if pid: stop_child(pid) - raise + raise finally: + signal.pthread_sigmask(signal.SIG_SETMASK, mask) for number, handler in previous.items(): signal.signal(number, handler) + if os.WIFSIGNALED(status): + number = os.WTERMSIG(status) + raise ChildFailed("The step running as " + SERVICE_USER + " was stopped by signal " + str(number) + + " (" + (signal.strsignal(number) or "unknown") + "). Run the command again to continue.") if not os.WIFEXITED(status) or os.WEXITSTATUS(status) != 0: raise ChildFailed() -def service_child(account, function, arguments, read_ends, output_write, errors_write): +def service_child(account, function, arguments, parent, mask, read_ends, output_write, errors_write): """The forked child of as_service_user; it never returns into the root caller's code.""" code = 1 try: for number in STOP_SIGNALS: signal.signal(number, signal.SIG_DFL) + signal.pthread_sigmask(signal.SIG_SETMASK, mask) for descriptor in read_ends: os.close(descriptor) os.setsid() @@ -604,6 +663,19 @@ def service_child(account, function, arguments, read_ends, output_write, errors_ os.setgroups(os.getgrouplist(account.pw_name, account.pw_gid)) os.setgid(account.pw_gid) os.setuid(account.pw_uid) + try: + # The session keyring survives fork, setuid and exec and gives its possessor + # the administrator's keys (keyrings(7)); join a new, empty one instead. + libc_call("syscall", KEYCTL_SYSCALL, KEYCTL_JOIN_SESSION_KEYRING, 0) + except OSError as error: + if error.errno != errno.ENOSYS: # Without kernel keyrings there is none to leave. + raise InstallError("Cannot give the " + SERVICE_USER + " step its own session keyring: " + + error.strerror + ". Inspect the host, then rerun the command.") from None + # Set after setuid, which clears it: the child dies with the parent, and a + # parent that already died is noticed here. + libc_call("prctl", PR_SET_PDEATHSIG, signal.SIGKILL) + if os.getppid() != parent: + os._exit(1) os.umask(0o077) os.environ.clear() os.environ.update(HOME=account.pw_dir, USER=account.pw_name, LOGNAME=account.pw_name, PATH=SAFE_PATH, @@ -1089,7 +1161,8 @@ def remove_node_files(root, installation_id): runtime_home = micro_home(installation_id) if root.exists(): shutil.rmtree(root) - if image: + # The service user can write provider.json; print only what an image ID can be. + if isinstance(image, str) and re.fullmatch(r"sha256:[0-9a-f]{64}", image): print("Kept the Runtime image " + image + "; remove it with `docker image rm " + image + "` if no other node uses it.") if runtime_home.exists(): sudo_mode = str(Path.home()) == str(SERVICE_HOME) @@ -1151,7 +1224,8 @@ def release_account(account, record): if record.get("created"): stores = [str(SERVICE_HOME / ".parsar/m" / name) for name in listdir_nofollow(SERVICE_HOME, ".parsar", "m")] if stores: - print("Kept the " + SERVICE_USER + " user while microsandbox stores remain: " + ", ".join(stores) + # The names come from the service user's home; show them only as plain text. + print("Kept the " + SERVICE_USER + " user while microsandbox stores remain: " + plain(", ".join(stores), sys.stdout.encoding) + ". Remove them with `sudo -u " + SERVICE_USER + " rm -rf `, then rerun this uninstall command.") return checked(["userdel", SERVICE_USER], "Cannot remove the " + SERVICE_USER + " user; stop its processes and rerun the uninstall command") @@ -1303,8 +1377,13 @@ def main(argv=None): if __name__ == "__main__": try: main() - except ChildFailed: + except ChildFailed as failure: + if str(failure): + print(str(failure), file=sys.stderr) sys.exit(1) + except KeyboardInterrupt: # Ctrl-C outside the service-user steps, as while waiting for the host lock. + print(INTERRUPTED, file=sys.stderr) + sys.exit(130) except (InstallError, node_spec.SpecificationError, distribution.DistributionError, OSError, ValueError, KeyError, subprocess.SubprocessError) as error: print(str(error) if isinstance(error, (InstallError, node_spec.SpecificationError, distribution.DistributionError)) else "Node installation failed; check host prerequisites and retained private files", file=sys.stderr) sys.exit(1) diff --git a/deploy/install/test_distribution.py b/deploy/install/test_distribution.py index 141372d41..4b098e90b 100644 --- a/deploy/install/test_distribution.py +++ b/deploy/install/test_distribution.py @@ -122,6 +122,13 @@ def test_interrupted_download_resumes_with_the_missing_bytes(self): self.assertEqual(self.if_ranges, [None, 'Sat, 26 Sep 2026 00:00:00 GMT']) self.assertEqual([path.name for path in self.root.iterdir()], ['node']) + def test_a_stalled_download_stops_and_keeps_its_part(self): + with patch.object(distribution, 'SLOW_SECONDS', 0), \ + self.assertRaisesRegex(distribution.DistributionError, 'stalled.*rerun the command to resume'): + distribution.obtain_artifact(self.manifest, 'native/bin/node', self.root / 'node') + self.assertEqual(len(self.requests), 1) # Not retried as a network error. + self.assertEqual([path.name for path in self.root.iterdir()], ['.node.partial']) + def test_offline_and_runtime_expansion_are_verified(self): raw = b'synthetic tar contents' * 1000 self.data = gzip.compress(raw) diff --git a/deploy/install/test_node_install.py b/deploy/install/test_node_install.py index c00adcaf2..950205811 100644 --- a/deploy/install/test_node_install.py +++ b/deploy/install/test_node_install.py @@ -1,5 +1,6 @@ """Exercise node installation without running providers or changing user services.""" import argparse +import fcntl import hashlib import gzip import io @@ -10,6 +11,7 @@ import socket import stat import subprocess +import sys import tempfile import time import unittest @@ -21,6 +23,19 @@ import node_spec + +def ended(pid, wait=5): + """The process is gone or a zombie waiting for its new parent to reap it.""" + deadline = time.monotonic() + wait + while time.monotonic() < deadline: + try: + if Path("/proc/%d/stat" % pid).read_text().rsplit(")", 1)[1].split()[0] == "Z": + return True + except (FileNotFoundError, ProcessLookupError): + return True + time.sleep(0.05) + return False + class Response(io.BytesIO): """An HTTP response body with the status and headers the downloader reads.""" status, headers = 200, {} @@ -503,41 +518,103 @@ def test_adoption_refuses_root_ids_and_extra_groups(self): with mock.patch.object(installer.os, "getgrouplist", return_value=[990, 27]): self.assertFalse(installer.ours(account)) - def test_service_user_step_has_no_terminal_and_reads_nothing(self): - """The forked child starts its own session with /dev/null as input; its output is relayed.""" + def service_step(self, function, output=None, errors=None): + """Run function through the real fork of as_service_user, as this test's user.""" account = SimpleNamespace(pw_name="parsar-node", pw_uid=os.getuid(), pw_gid=os.getgid(), pw_dir=str(self.home), pw_shell="/usr/sbin/nologin") + with mock.patch.object(installer.os, "setgroups"), mock.patch.object(installer.os, "setgid"), \ + mock.patch.object(installer.os, "setuid"), mock.patch.object(installer.sys, "stdout", output or io.StringIO()), \ + mock.patch.object(installer.sys, "stderr", errors or io.StringIO()): + installer.as_service_user(account, function) + def test_service_user_step_has_no_terminal_and_reads_nothing(self): + """The forked child starts its own session with /dev/null as input; its output is relayed.""" def probe(): - print("session-leader=%s stdin=%s docker-config=%s" % (os.getsid(0) == os.getpid(), - os.readlink("/proc/self/fd/0"), os.environ["DOCKER_CONFIG"])) + try: + open("/dev/tty").close() + tty = "opened" + except OSError: + tty = "unavailable" + print("session-leader=%s stdin=%s docker-config=%s tty=%s" % (os.getsid(0) == os.getpid(), + os.readlink("/proc/self/fd/0"), os.environ["DOCKER_CONFIG"], tty)) output = io.StringIO() - with mock.patch.object(installer.os, "setgroups"), mock.patch.object(installer.os, "setgid"), \ - mock.patch.object(installer.os, "setuid"), mock.patch.object(installer.sys, "stdout", output): - installer.as_service_user(account, probe) - self.assertIn("session-leader=True stdin=/dev/null docker-config=" + str(installer.CHILD_DOCKER_CONFIG), output.getvalue()) - - def test_closing_the_terminal_stops_the_service_user_step(self): - """The child's own session misses SIGHUP; the installer must stop and reap it before it exits.""" - account = SimpleNamespace(pw_name="parsar-node", pw_uid=os.getuid(), pw_gid=os.getgid(), - pw_dir=str(self.home), pw_shell="/usr/sbin/nologin") - record = self.home / "child.pid" + self.service_step(probe, output) + self.assertIn("session-leader=True stdin=/dev/null docker-config=" + str(installer.CHILD_DOCKER_CONFIG) + " tty=unavailable", + output.getvalue()) + + def test_service_user_output_reaches_the_terminal_as_plain_text(self): + def forge(): + os.write(1, b"\x1b]52;c;ZWNobyBoaQ==\x07copied\r\x1b[2KFinish with: sudo sh\n") + os.write(1, "caf\u00e9".encode()[:-1]) # A character split across two reads. + time.sleep(0.2) + os.write(1, "caf\u00e9".encode()[-1:] + b"\n") + os.write(2, b"\x1b]0;title\x07\xc2\x9bwarning\n") + output, errors = io.StringIO(), io.StringIO() + self.service_step(forge, output, errors) + self.assertEqual(output.getvalue(), "?]52;c;ZWNobyBoaQ==?copied??[2KFinish with: sudo sh\ncaf\u00e9\n") + self.assertEqual(errors.getvalue(), "?]0;title??warning\n") + + def test_uninstall_prints_only_an_image_id_from_the_service_home(self): + root = self.home / "node" + root.mkdir() + (root / "provider.json").write_text(json.dumps({"docker": {"image": "x\nFinish with: sudo sh"}})) + output = io.StringIO() + with mock.patch.object(installer.sys, "stdout", output): + installer.remove_node_files(root, self.args.installation_id) + self.assertNotIn("Finish", output.getvalue()) + + def test_service_user_step_gets_its_own_session_keyring(self): + libc = installer.ctypes.CDLL(None, use_errno=True) + libc.syscall.restype = installer.ctypes.c_long + + def session_keyring(): # KEYCTL_GET_KEYRING_ID of KEY_SPEC_SESSION_KEYRING + return libc.syscall(*(installer.ctypes.c_long(value) for value in (installer.KEYCTL_SYSCALL, 0, -3, 0))) + before = session_keyring() + if before == -1: + self.skipTest("keyctl is unavailable here") + output = io.StringIO() + self.service_step(lambda: print("keyring=%d" % session_keyring()), output) + inside = int(output.getvalue().split("keyring=")[1]) + self.assertGreater(inside, 0) + self.assertNotEqual(inside, before) + + def test_closing_the_terminal_stops_the_step_and_everything_it_started(self): + """The child's own session misses SIGHUP; the installer stops it and what it started, and frees the lock.""" + record = self.home / "pids" # As in a terminal session; a runner under nohup would otherwise ignore SIGHUP. self.addCleanup(signal.signal, signal.SIGHUP, signal.signal(signal.SIGHUP, signal.SIG_DFL)) def long_step(): - record.write_text(str(os.getpid())) + # A program that ignores SIGTERM: only the SIGKILL to the child's process group ends it. + stubborn = subprocess.Popen([sys.executable, "-c", "import signal, time; signal.signal(signal.SIGTERM, " + "signal.SIG_IGN); print(flush=True); time.sleep(60)"], stdout=subprocess.PIPE) + stubborn.stdout.readline() + record.write_text("%d %d" % (os.getpid(), stubborn.pid)) os.kill(os.getppid(), signal.SIGHUP) # The administrator's terminal closes. time.sleep(30) started = time.monotonic() - with mock.patch.object(installer.os, "setgroups"), mock.patch.object(installer.os, "setgid"), \ - mock.patch.object(installer.os, "setuid"), mock.patch.object(installer.sys, "stdout", io.StringIO()), \ + with mock.patch.object(installer, "SYSTEM_LOCKS", self.home), mock.patch.object(installer.os, "geteuid", return_value=0), \ self.assertRaisesRegex(installer.InstallError, "interrupted"): - installer.as_service_user(account, long_step) + with installer.host_lock(): + self.service_step(long_step) self.assertLess(time.monotonic() - started, 15) - with self.assertRaises(ProcessLookupError): # Stopped and reaped, not left running. - os.kill(int(record.read_text()), 0) - + for pid in map(int, record.read_text().split()): + self.assertTrue(ended(pid), pid) + with open(self.home / "parsar-node.lock") as lock: # The child held it too; nothing does now. + fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) + + def test_an_ignored_hangup_stays_ignored(self): + """Under nohup a closed terminal ends neither the installer nor the step.""" + self.addCleanup(signal.signal, signal.SIGHUP, signal.signal(signal.SIGHUP, signal.SIG_IGN)) + + def step(): + os.kill(os.getppid(), signal.SIGHUP) + time.sleep(0.2) + print("finished") + output = io.StringIO() + self.service_step(step, output) + self.assertEqual(output.getvalue(), "finished\n") + self.assertIs(signal.getsignal(signal.SIGHUP), signal.SIG_IGN) def test_uninstall_never_follows_a_link_in_the_service_home(self): # The service user owns its home; a link it plants must not steer deletion elsewhere. diff --git a/services/agents-api/HOSTED-SANDBOX-MANAGER.md b/services/agents-api/HOSTED-SANDBOX-MANAGER.md index 6d0426c2e..1c3901876 100644 --- a/services/agents-api/HOSTED-SANDBOX-MANAGER.md +++ b/services/agents-api/HOSTED-SANDBOX-MANAGER.md @@ -161,7 +161,8 @@ entries and exact imported images, then waits for Core to confirm connection and provider readiness. The enrollment token is transient and never a console/project credential; the command passes it to the installer on standard input (`--enrollment-token-stdin`), never in a process argument, -an environment variable or sudo's log. +an environment variable or sudo's log. A download that brings less than 64 KiB in a +minute stops; the downloaded part is kept, and running a new command resumes it. The installer chooses how the node runs from the user that runs it: @@ -185,8 +186,9 @@ The installer chooses how the node runs from the user that runs it: without sudo is found in the invoking user's home or, for Docker, by its network on the same engine; other users' homes are not searched. The steps that run as `parsar-node` start in their own session with no terminal, so nothing they run - can reach the administrator's terminal. Interrupting the installer or closing - its terminal stops those steps as well. + can reach the administrator's terminal, and their output appears only as plain + text. Interrupting the installer or closing its terminal stops those steps as + well. **Docker mode is root-equivalent.** Membership in the `docker` group lets `parsar-node`, and so anything that controls the node, act as root on that host.