diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 2cdd53c4c..62e94d8a5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -822,7 +822,9 @@ installation, generation, specification digest and release before writing node f registering or reconnecting. Reject drift rather than overwriting retained identity or using local resource defaults. Registration consumes a token only after these checks. The installer verifies downloaded files and starts the ordinary node process -as a user service; it performs no SSH installation, Session creation or model call. +as a user service, or, run as root, as a root-owned system service for the dedicated +`parsar-node` user it prepares; it performs no SSH installation, Session creation or +model call. The [Hosted Sandbox Manager](services/agents-api/HOSTED-SANDBOX-MANAGER.md) is a deployment-level admin surface, separate from Project credentials. The paired @@ -1856,11 +1858,29 @@ Docker socket or node identity mount in either mode. The ordinary standalone nod service owns its provider processes outside the Core container. Its `KillMode=process` preserves resident microVM/helper processes across a node-service restart. User KVM access and the Linux runtime libraries are prerequisites for microsandbox. The node -runs as a systemd user service and needs linger. The only other launcher allowed -is the root-run node installation (sudo mode) planned for phase 2b: one root-owned -system service per installation that runs the same node program as a dedicated -unprivileged service user. Do not add any other launcher, scheduler or recovery -path. Node services restart after failures without a start limit, so a node +installed by a normal user runs as a systemd user service and needs linger. Run as +root (sudo mode), the installer instead prepares the host: it creates or adopts the +`parsar-node` system user, adds it to the `docker` or `kvm` device group (no other +group), and installs one root-owned system service per installation that runs the +same node program with `User=parsar-node`. Sudo mode serves one Core per host, +because its nodes share that account. Docker group membership makes that user, +and so the node, root-equivalent on the host; that is inherent to Docker sandboxes, +not a least-privilege boundary. Microsandbox needs only `kvm`. Files the service +user owns are read, written and deleted only with its credentials, never by root, +in a child that starts its own session with /dev/null as input, so nothing it runs +can reach the administrator's terminal. That child also joins a new session +keyring and dies with its parent, and root shows its output only as plain text +(terminal controls become `?`). The installer turns SIGINT, SIGHUP and SIGTERM +into stopping that child and what it started, which would otherwise outlive a +closed terminal. Root never runs a file that user can write, +opens a URL it wrote, or follows a link in its home. Sudo mode +never installs Docker, KVM or packages, never changes device permissions, refuses +SELinux-enforcing hosts and a token in the environment, and changes nothing when a +check fails. `--uninstall` removes a node only after Core rejects its credential, +never touches sandboxes, volumes or images (it keeps the Runtime image and the +microsandbox store), deletes the account only when the installer created it and no +node remains, and otherwise removes only the groups it added. Do not add any other launcher, scheduler or +recovery path. Node services restart after failures without a start limit, so a node outlasts a Core outage, and stop restarting when the node program exits 78 because Core answered 401 to its credential (a removed or retired node). The basic API image and binary builds remain independent artifacts. @@ -1872,7 +1892,8 @@ microsandbox selections use Web's Standard size from `apps/web/src/features/sandbox/standard-sizes.json`, which the distribution build copies into the bundle; keep no second copy of those values. An existing database selection is never overwritten by installer defaults. Node configuration and identity live under -`~/.parsar/nodes//`; microsandbox uses its separate short private +`~/.parsar/nodes//` in the node account's home (`/var/lib/parsar-node` +in sudo mode); microsandbox uses its separate short private Runtime home. Zero-node installs create no node identity state but retain the paired Core key for first setup. diff --git a/deploy/install/distribution.py b/deploy/install/distribution.py index 8c8103a31..1a0f77fe9 100644 --- a/deploy/install/distribution.py +++ b/deploy/install/distribution.py @@ -147,44 +147,138 @@ def obtain_artifact(manifest, logical_path, destination, offline_root=None): if candidate.is_symlink() or not candidate.is_file(): raise DistributionError('Offline artifact must be a regular file: ' + logical_path) source = candidate + if source is not None: + return copy_artifact(source, target, entry, logical_path) base = manifest.get('artifact_base_url', '') - if source is None: - if not isinstance(base, str) or not base or urlsplit(base).query: - raise DistributionError('No downloadable artifact source; use the matching offline bundle') - url = safe_url(base.rstrip('/') + '/' + entry['filename']) + if not isinstance(base, str) or not base or urlsplit(base).query: + raise DistributionError('No downloadable artifact source; use the matching offline bundle') + url = safe_url(base.rstrip('/') + '/' + entry['filename']) + # A private partial file survives interruptions and reruns; the next attempt asks + # for the missing bytes only. The complete file is still verified as a whole. + partial = target.with_name('.' + target.name + '.partial') for attempt in range(3): - fd, temporary = tempfile.mkstemp(prefix='.artifact-', dir=target.parent) try: - with os.fdopen(fd, 'wb') as output: - stream = source.open('rb') if source else urllib.request.build_opener(NoRedirect()).open(url, timeout=30) - with stream: - count = 0 - while True: - block = stream.read(1024 * 1024) - if not block: - break - count += len(block) - if count > entry['size']: - raise DistributionError('Artifact exceeds published size: ' + logical_path) - output.write(block) - if count != entry['size']: - raise http.client.IncompleteRead(b'', entry['size'] - count) - if digest(temporary) != entry['sha256']: - raise DistributionError('Artifact checksum mismatch: ' + logical_path) - os.chmod(temporary, 0o700 if logical_path.startswith('native/') else 0o600) - os.replace(temporary, target) - return target + download_partial(url, partial, entry, logical_path) + break except urllib.error.HTTPError as error: - if error.code not in (408, 429, 500, 502, 503, 504) or attempt == 2: - raise DistributionError(f'Artifact download failed (HTTP {error.code}): {logical_path}. Check release access and retry.') from None + if error.code == 416: + discard_partial(partial) + elif error.code not in (408, 429, 500, 502, 503, 504) or attempt == 2: + raise DistributionError(f'Artifact download failed (HTTP {error.code}): {logical_path}. Check the console and retry.') from None except (urllib.error.URLError, socket.timeout, ConnectionError, http.client.HTTPException): - if attempt == 2 or source: - raise DistributionError('Artifact transfer interrupted: ' + logical_path + '. Check network access and rerun; installed state is retained.') from None - finally: - if os.path.exists(temporary): - os.unlink(temporary) + if attempt == 2: + raise DistributionError('Artifact transfer interrupted: ' + logical_path + '. Check network access and rerun; ' + 'the download resumes where it stopped.') from None time.sleep(attempt + 1) - raise DistributionError('Artifact download did not complete') + else: + raise DistributionError('Artifact download did not complete') + if digest(partial) != entry['sha256']: + discard_partial(partial) + raise DistributionError('Artifact checksum mismatch: ' + logical_path) + os.chmod(partial, 0o700 if logical_path.startswith('native/') else 0o600) + os.replace(partial, target) + validator_path(partial).unlink(missing_ok=True) + return target + + +def validator_path(partial): + return partial.with_name(partial.name + '.validator') + + +def discard_partial(partial): + partial.unlink(missing_ok=True) + validator_path(partial).unlink(missing_ok=True) + + +def copy_artifact(source, target, entry, logical_path): + fd, temporary = tempfile.mkstemp(prefix='.artifact-', dir=target.parent) + try: + with os.fdopen(fd, 'wb') as output, source.open('rb') as stream: + count = 0 + for block in iter(lambda: stream.read(1024 * 1024), b''): + count += len(block) + if count > entry['size']: + raise DistributionError('Artifact exceeds published size: ' + logical_path) + output.write(block) + if count != entry['size'] or digest(temporary) != entry['sha256']: + raise DistributionError('Artifact checksum mismatch: ' + logical_path) + os.chmod(temporary, 0o700 if logical_path.startswith('native/') else 0o600) + os.replace(temporary, target) + return target + finally: + if os.path.exists(temporary): + os.unlink(temporary) + + +# A download that brings fewer bytes than this in a window stops; a rerun resumes it. +SLOW_SECONDS, SLOW_BYTES = 60, 64 * 1024 + + +def download_partial(url, partial, entry, logical_path): + """Complete the partial file, asking only for the bytes it is missing.""" + size = entry['size'] + offset = 0 + if partial.is_symlink() or (partial.exists() and not partial.is_file()): + raise DistributionError('Partial download must be a regular file: ' + logical_path) + if partial.exists(): + info = partial.stat() + if info.st_uid != os.getuid(): + raise DistributionError('Partial download must be owned by this user: ' + logical_path) + offset = info.st_size if info.st_size <= size else 0 + if offset == size: + return + headers = {} + if offset: + headers['Range'] = f'bytes={offset}-' + # If-Range makes a server whose file changed since the partial began send it whole. + try: + validator = validator_path(partial).read_text().strip() + except OSError: + validator = '' + if validator: + headers['If-Range'] = validator + request = urllib.request.Request(url, headers=headers) + with urllib.request.build_opener(NoRedirect()).open(request, timeout=30) as stream: + if offset and (stream.status != 206 or not stream.headers.get('Content-Range', '').startswith(f'bytes {offset}-')): + offset = 0 # The server sent the whole file; start over. + if not offset: + validator = stream.headers.get('ETag') or stream.headers.get('Last-Modified') or '' + if validator and all(32 <= ord(c) < 127 for c in validator) and len(validator) < 200: + with os.fdopen(os.open(validator_path(partial), os.O_WRONLY | os.O_CREAT | os.O_TRUNC | os.O_NOFOLLOW, 0o600), 'w') as note: + note.write(validator) + else: + validator_path(partial).unlink(missing_ok=True) + flags = os.O_WRONLY | os.O_CREAT | os.O_NOFOLLOW | (os.O_APPEND if offset else os.O_TRUNC) + with os.fdopen(os.open(partial, flags, 0o600), 'ab' if offset else 'wb') as output: + count, reported = offset, offset * 10 // size + window, window_count = time.monotonic(), 0 + while True: + block = stream.read1(1024 * 1024) + if not block: + break + count += len(block) + if count > size: + output.close() + discard_partial(partial) + raise DistributionError('Artifact exceeds published size: ' + logical_path) + output.write(block) + window_count += len(block) + if time.monotonic() - window >= SLOW_SECONDS: + if window_count < SLOW_BYTES: + raise DistributionError(f'Artifact download stalled (under {SLOW_BYTES // 1024} KiB in {SLOW_SECONDS} s): ' + f'{logical_path}. The downloaded part is kept; check the network, then rerun ' + 'the command to resume.') + window, window_count = time.monotonic(), 0 + if size >= 50 * 1024 * 1024 and count * 10 // size > reported: + reported = count * 10 // size + try: + print(f'Downloading {logical_path}: {reported * 10}% of {size // (1024 * 1024)} MiB', flush=True) + except BrokenPipeError: + # Nobody reads the output any more (the installer's terminal is gone); + # stop instead of retrying it as a network error. + raise DistributionError('Output closed; stopped downloading ' + logical_path) from None + if count != size: + raise http.client.IncompleteRead(b'', size - count) def runtime_archive(manifest, cache_root, offline_root=None): diff --git a/deploy/install/node_install.py b/deploy/install/node_install.py index d8b550eb1..72ebe01c5 100644 --- a/deploy/install/node_install.py +++ b/deploy/install/node_install.py @@ -1,7 +1,13 @@ #!/usr/bin/env python3 """Install and enroll one node from its Core console's matched distribution.""" import argparse +import codecs +import contextlib +import ctypes +import errno import fcntl +import getpass +import grp import hashlib import http.client import io @@ -10,8 +16,11 @@ import os from pathlib import Path import platform +import pwd import re +import selectors import shutil +import signal import socket import stat import subprocess @@ -34,6 +43,27 @@ class InstallError(Exception): COMMON = ("native/bin/parsar-sandbox-node", "runtime/seccomp.json") MICRO = ("native/bin/agents-api-microsandbox-provider", "native/microsandbox/msb", "native/microsandbox/libkrunfw.so.5.6.1") +DOCKER = ("docker", "--host", "unix:///var/run/docker.sock") +DOCKER_SOCKET = Path("/var/run/docker.sock") +KVM = Path("/dev/kvm") + +# Sudo mode: run as root, the installer prepares the host itself. The node runs as +# this dedicated service user under a root-owned system unit; root never runs a +# file the service user can write. +SERVICE_USER = "parsar-node" +SERVICE_HOME = Path("/var/lib/parsar-node") +SYSTEM_RECORDS = Path("/etc/parsar-node") +SYSTEM_UNITS = Path("/etc/systemd/system") +SYSTEM_LOCKS = Path("/run") +SYSTEMD_RUNNING = Path("/run/systemd/system") +SELINUX_ENFORCE = Path("/sys/fs/selinux/enforce") +USER_RUNTIME = Path("/run/user") +SAFE_PATH = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" +# A root-owned, empty Docker configuration for the service user's docker calls, so no +# CLI plugin or credential helper the service user controls runs in the installer. +CHILD_DOCKER_CONFIG = Path("/run/parsar-node-docker") +LOCK_WAIT_SECONDS = 600 +NOTHING_CHANGED = " Nothing was changed." def origin(value): @@ -101,18 +131,42 @@ def discard_unregistered(root): shutil.rmtree(root / "state/node") -def preflight(provider): +def preflight(provider, system=False): + """Host access for the node's own user; a system service needs no user session.""" if sys.version_info < (3, 9) or platform.system() != "Linux" or platform.machine() not in ("x86_64", "amd64") or os.getuid() == 0: raise InstallError("Run with Python 3.9+ as a non-root user on Linux amd64") - checked(["systemctl", "--user", "show", "--property=Version", "--value"], "A systemd user session is required") - if checked(["loginctl", "show-user", str(os.getuid()), "--property=Linger", "--value"], "Cannot check user lingering") != "yes": - raise InstallError("Ask the host administrator to enable user lingering before installing a node") + if not system: + user_bus() + checked(["systemctl", "--user", "show", "--property=Version", "--value"], "A systemd user session is required") + if checked(["loginctl", "show-user", str(os.getuid()), "--property=Linger", "--value"], "Cannot check user lingering") != "yes": + raise InstallError("Ask the host administrator to enable user lingering before installing a node, or run the command with sudo") if provider == "docker": - checked(["docker", "--host", "unix:///var/run/docker.sock", "info", "--format", "{{.ServerVersion}}"], "Docker access through /var/run/docker.sock is required") - elif not os.access("/dev/kvm", os.R_OK | os.W_OK): + checked(list(DOCKER) + ["info", "--format", "{{.ServerVersion}}"], "Docker access through /var/run/docker.sock is required") + elif not os.access(KVM, os.R_OK | os.W_OK): raise InstallError("microsandbox requires read/write access to /dev/kvm") +def user_bus(): + """Reach this user's systemd manager from a session that has no bus address. + + su or sudo -iu from a root shell leaves XDG_RUNTIME_DIR unset; lingering keeps the + user manager and its bus running under /run/user/.""" + if os.environ.get("XDG_RUNTIME_DIR"): + return + runtime = USER_RUNTIME / str(os.getuid()) + bus = runtime / "bus" + try: + info = bus.lstat() + except OSError: + info = None + if info is None or not stat.S_ISSOCK(info.st_mode) or info.st_uid != os.getuid(): + user = pwd.getpwuid(os.getuid()).pw_name + raise InstallError("No systemd user manager is running for " + user + ". Ask the host administrator to run " + "`sudo loginctl enable-linger " + user + "`, or run the command with sudo.") + os.environ["XDG_RUNTIME_DIR"] = str(runtime) + os.environ["DBUS_SESSION_BUS_ADDRESS"] = "unix:path=" + str(bus) + + class NoRedirect(urllib.request.HTTPRedirectHandler): def redirect_request(self, request, fp, code, message, headers, url): raise InstallError("Node bootstrap redirects are not supported") @@ -318,16 +372,23 @@ def quote(value): + "\n\n[Install]\nWantedBy=default.target\n") -def install(args, token): +def unit_name(installation_id): + return "parsar-node-" + installation_id + ".service" + + +def open_node(args, token, system=False): + """Read the Core specification and check the host; returns the node's state directory.""" root = Path.home() / ".parsar/nodes" / args.installation_id safe_directory(root) identity_file = root / "state/node/identity.json" retained = json.loads(identity_file.read_text()) if existing_file(identity_file) else None - print("Reading the Core deployment specification...", flush=True) + if not system: # In sudo mode, root has already said so. + print("Reading the Core deployment specification...", flush=True) args.configuration = node_spec.fetch(args, token, retained, open_request, allow_enrollment=not (root / "registered.json").exists()) args.provider = args.configuration["provider"] - print("Checking host requirements...", flush=True) - preflight(args.provider) + if not system: + print("Checking host requirements...", flush=True) + preflight(args.provider, system) if args.provider == "microsandbox": runtime_home = micro_home(args.installation_id) safe_directory(runtime_home) @@ -335,71 +396,93 @@ def install(args, token): if not owner.exists() and any(runtime_home.iterdir()): raise InstallError("Microsandbox home contains unowned state; refusing to adopt it") write_once(owner, json_text({"installation_id": args.installation_id})) - root = Path.home() / ".parsar/nodes" / args.installation_id - safe_directory(root) + return root + + +@contextlib.contextmanager +def install_lock(root): descriptor = os.open(root / "install.lock", os.O_CREAT | os.O_RDWR | os.O_NOFOLLOW, 0o600) with os.fdopen(descriptor, "w") as lock: try: fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) except BlockingIOError: raise InstallError("Another node installation is running") from None - print("Downloading and verifying node files...", flush=True) - manifest, sums = metadata(args.source_url, getattr(args, "bundle", None)) - node_spec.verify_release(args.configuration, manifest) - names = COMMON + (MICRO if args.provider == "microsandbox" else ()) - if "runtime/seccomp.json" not in sums: - raise InstallError("The distribution is missing required node checksums") - state = {"installation_id": args.installation_id, "provider": args.provider, "core_url": args.core_url, - "source_commit": manifest["source_commit"], "generation": args.configuration["generation"], - "specification_digest": args.configuration["specification_digest"]} - write_once(root / "installation.json", json_text(state)) - for name in names: - if name == "runtime/seccomp.json": - if getattr(args, "bundle", None) is None: - download(args.source_url, name, root, sums[name]) - else: - source = args.bundle / name - if source.is_symlink() or file_digest(source) != sums[name]: - raise InstallError("Local node payload checksum mismatch") - safe_directory((root / name).parent) - write_once(root / name, source.read_text()) + yield + + +def register_node(root, args, token): + """Download and verify the payload, prepare the Runtime and register; not the service.""" + print("Downloading and verifying node files...", flush=True) + manifest, sums = metadata(args.source_url, getattr(args, "bundle", None)) + node_spec.verify_release(args.configuration, manifest) + names = COMMON + (MICRO if args.provider == "microsandbox" else ()) + if "runtime/seccomp.json" not in sums: + raise InstallError("The distribution is missing required node checksums") + state = {"installation_id": args.installation_id, "provider": args.provider, "core_url": args.core_url, + "source_commit": manifest["source_commit"], "generation": args.configuration["generation"], + "specification_digest": args.configuration["specification_digest"]} + write_once(root / "installation.json", json_text(state)) + for name in names: + if name == "runtime/seccomp.json": + if getattr(args, "bundle", None) is None: + download(args.source_url, name, root, sums[name]) else: - target = root / name - safe_directory(target.parent) - existing_file(target) - distribution.obtain_artifact(manifest, name, target, getattr(args, "bundle", None)) - os.chmod(target, 0o700) - safe_directory(root / "state/node") - print("Checking the sandbox runtime...", flush=True) - runtime_image = prepare_runtime(root, args, manifest) - # Retain the original network policy when recovering a partial installation. - if not existing_file(root / "provider.json"): - write_once(root / "provider.json", json_text(provider_config(root, args, manifest, runtime_image))) + source = args.bundle / name + if source.is_symlink() or file_digest(source) != sums[name]: + raise InstallError("Local node payload checksum mismatch") + safe_directory((root / name).parent) + write_once(root / name, source.read_text()) else: - node_spec.verify_provider(json.loads((root / "provider.json").read_text()), args.configuration, runtime_image) - unit = root / ("parsar-node-" + args.installation_id + ".service") - write_once(unit, service_unit(root)) - marker = root / "registered.json" - if not existing_file(marker): - # The one-time credential is never passed through process arguments or service environments. - descriptor, secret_path = tempfile.mkstemp(prefix=".enrollment-", dir=root) + target = root / name + safe_directory(target.parent) + existing_file(target) + distribution.obtain_artifact(manifest, name, target, getattr(args, "bundle", None)) + os.chmod(target, 0o700) + safe_directory(root / "state/node") + print("Checking the sandbox runtime...", flush=True) + runtime_image = prepare_runtime(root, args, manifest) + # Retain the original network policy when recovering a partial installation. + if not existing_file(root / "provider.json"): + write_once(root / "provider.json", json_text(provider_config(root, args, manifest, runtime_image))) + else: + node_spec.verify_provider(json.loads((root / "provider.json").read_text()), args.configuration, runtime_image) + marker = root / "registered.json" + if not existing_file(marker): + # The one-time credential is never passed through process arguments or service environments. + descriptor, secret_path = tempfile.mkstemp(prefix=".enrollment-", dir=root) + try: + with os.fdopen(descriptor, "w") as secret: + secret.write(token) + print("Registering this node with Core...", flush=True) try: - with os.fdopen(descriptor, "w") as secret: - secret.write(token) - print("Registering this node with Core...", flush=True) - try: - checked([str(root / COMMON[0]), "register", "--config", str(root / "provider.json"), "--state-dir", str(root / "state/node"), - "--core-url", args.core_url, "--name", socket.gethostname(), - "--enrollment-token-file", secret_path], REGISTRATION_UNCONFIRMED, explain=registration_failure) - except AddressChanged: - discard_unregistered(root) - raise - write_once(marker, json_text(state)) - finally: - if os.path.exists(secret_path): - os.unlink(secret_path) - elif marker.read_text() != json_text(state): - raise InstallError("Registered node identity differs; refusing to replace it") + checked([str(root / COMMON[0]), "register", "--config", str(root / "provider.json"), "--state-dir", str(root / "state/node"), + "--core-url", args.core_url, "--name", socket.gethostname(), + "--enrollment-token-file", secret_path], REGISTRATION_UNCONFIRMED, explain=registration_failure) + except AddressChanged: + discard_unregistered(root) + raise + write_once(marker, json_text(state)) + finally: + if os.path.exists(secret_path): + os.unlink(secret_path) + elif marker.read_text() != json_text(state): + raise InstallError("Registered node identity differs; refusing to replace it") + + +def install(args, token): + """Non-root mode: the node runs as this user's systemd user service.""" + try: + system_node = (SYSTEM_RECORDS / (args.installation_id + ".json")).exists() + except PermissionError: + system_node = True + if system_node: + raise InstallError("This host already runs a node for this installation as a system service. Rerun the command " + "with sudo, or uninstall that node with sudo first." + NOTHING_CHANGED) + root = open_node(args, token) + with host_lock(), install_lock(root): + register_node(root, args, token) + unit = root / unit_name(args.installation_id) + write_once(unit, service_unit(root)) print("Starting the node service...", flush=True) checked(["systemctl", "--user", "daemon-reload"], "Cannot reload the systemd user manager") checked(["systemctl", "--user", "enable", "--now", str(unit)], "Cannot start the node service; retained identity is unchanged") @@ -407,6 +490,791 @@ def install(args, token): print("Waiting for Core connection and provider readiness...", flush=True) wait_ready(root, args) print("Node connected to Core and provider ready. State: " + str(root)) + print("Logs: journalctl --user -u " + unit.name) + + +def prepare_service_node(args, token): + """Sudo mode, as the service user: everything but the root-owned system unit.""" + root = open_node(args, token, system=True) + with install_lock(root): + register_node(root, args, token) + + + +# Sudo mode ----------------------------------------------------------------- + +class ChildFailed(Exception): + """The service-user step failed and already printed why.""" + + +# What a service-user step must not send to the administrator's terminal: C0 controls +# other than tab and newline, DEL and C1. ESC starts OSC 52 clipboard writes, title +# changes and cursor moves; carriage returns and erases could hide or forge lines. +TERMINAL_CONTROLS = re.compile(r"[\x00-\x08\x0b-\x1f\x7f-\x9f]") + + +def plain(text, encoding=None): + """Text the terminal shows as written, in an encoding the stream can write.""" + encoding = encoding or "utf-8" + return TERMINAL_CONTROLS.sub("?", text).encode(encoding, "replace").decode(encoding) + + +def relay(output, errors): + """Copy the child's stdout and stderr pipes to ours as plain text until both close.""" + streams = {output: (sys.stdout, codecs.getincrementaldecoder("utf-8")("replace")), + errors: (sys.stderr, codecs.getincrementaldecoder("utf-8")("replace"))} + with selectors.DefaultSelector() as selector: + for descriptor in streams: + selector.register(descriptor, selectors.EVENT_READ) + while streams: + for key, _ in selector.select(): + data = os.read(key.fd, 65536) + stream, decoder = streams[key.fd] + # The decoder keeps a character split across reads until its last byte arrives. + text = plain(decoder.decode(data, final=not data), getattr(stream, "encoding", None)) + if text: + stream.write(text) + stream.flush() + if not data: + selector.unregister(key.fd) + os.close(key.fd) + del streams[key.fd] + + +STOP_WAIT_SECONDS = 5 + + +def signal_child(pid, number): + # The child's own process group once it has started its session, and the child + # itself before then. It is never reaped before this, so pid is still its own. + for send in (os.killpg, os.kill): + with contextlib.suppress(ProcessLookupError): + send(pid, number) + + +def stop_child(pid): + """End the child and everything it started; its own session gets no terminal signals.""" + def exited(): + # WNOWAIT leaves the child a zombie, so its process group ID cannot be reused + # before the SIGKILL below reaches what the child left running. + return os.waitid(os.P_PID, pid, os.WEXITED | os.WNOHANG | os.WNOWAIT) is not None + try: + exited() + except ChildProcessError: + return # Already reaped. + signal_child(pid, signal.SIGTERM) + deadline = time.monotonic() + STOP_WAIT_SECONDS + while not exited() and time.monotonic() < deadline: + time.sleep(0.1) + signal_child(pid, signal.SIGKILL) + os.waitpid(pid, 0) + + +def interrupted(number, frame): + raise InstallError(INTERRUPTED) + + +INTERRUPTED = "The command was interrupted; run it again to continue." +# The child's own session gets none of the terminal's signals, so while it runs the +# parent turns these into an error that stops the child before the parent exits. +STOP_SIGNALS = (signal.SIGINT, signal.SIGHUP, signal.SIGTERM) +KEYCTL_SYSCALL = 250 # x86_64, the only architecture this installer supports. +KEYCTL_JOIN_SESSION_KEYRING = 1 +PR_SET_PDEATHSIG = 1 + + +def libc_call(name, *arguments): + function = getattr(ctypes.CDLL(None, use_errno=True), name) + function.restype = ctypes.c_long + if function(*(ctypes.c_long(argument) for argument in arguments)) == -1: + number = ctypes.get_errno() + raise OSError(number, os.strerror(number)) + + +def as_service_user(account, function, *arguments): + """Run a step with the service user's credentials in a forked child. + + The child starts a new session with /dev/null as standard input and pipes as + output, so no program it runs holds the administrator's terminal (TIOCSTI); the + parent shows that output only as plain text. Interrupting the installer or + closing its terminal stops the child too, and the child dies with the parent. + Every installer module is already imported, so the child never reads the root + caller's private copy of this program, and the token stays in memory. Files the + service user owns are read, written and deleted only here, never by root.""" + sys.stdout.flush() + sys.stderr.flush() + output_read, output_write = os.pipe() + errors_read, errors_write = os.pipe() + parent = os.getpid() + previous = {number: signal.getsignal(number) for number in STOP_SIGNALS} + # Held from before the fork until each side has its handlers, so neither a signal + # nor the resulting error can reach the wrong process. + mask = signal.pthread_sigmask(signal.SIG_BLOCK, STOP_SIGNALS) + pid = None + try: + for number, handler in previous.items(): + if handler is not signal.SIG_IGN: # As under nohup, an ignored signal stays ignored. + signal.signal(number, interrupted) + pid = os.fork() + if pid == 0: + service_child(account, function, arguments, parent, mask, (output_read, errors_read), output_write, errors_write) + signal.pthread_sigmask(signal.SIG_SETMASK, mask) + os.close(output_write) + os.close(errors_write) + relay(output_read, errors_read) + _, status = os.waitpid(pid, 0) + except BaseException: + for number in STOP_SIGNALS: # A second signal must not cut the stop short. + signal.signal(number, signal.SIG_IGN) + if pid: + stop_child(pid) + raise + finally: + signal.pthread_sigmask(signal.SIG_SETMASK, mask) + for number, handler in previous.items(): + signal.signal(number, handler) + if os.WIFSIGNALED(status): + number = os.WTERMSIG(status) + raise ChildFailed("The step running as " + SERVICE_USER + " was stopped by signal " + str(number) + + " (" + (signal.strsignal(number) or "unknown") + "). Run the command again to continue.") + if not os.WIFEXITED(status) or os.WEXITSTATUS(status) != 0: + raise ChildFailed() + + +def service_child(account, function, arguments, parent, mask, read_ends, output_write, errors_write): + """The forked child of as_service_user; it never returns into the root caller's code.""" + code = 1 + try: + for number in STOP_SIGNALS: + signal.signal(number, signal.SIG_DFL) + signal.pthread_sigmask(signal.SIG_SETMASK, mask) + for descriptor in read_ends: + os.close(descriptor) + os.setsid() + null = os.open(os.devnull, os.O_RDWR) + os.dup2(null, 0) + os.dup2(output_write, 1) + os.dup2(errors_write, 2) + for descriptor in (null, output_write, errors_write): + if descriptor > 2: + os.close(descriptor) + sys.stdout = open(1, "w", buffering=1, closefd=False) + sys.stderr = open(2, "w", buffering=1, closefd=False) + os.setgroups(os.getgrouplist(account.pw_name, account.pw_gid)) + os.setgid(account.pw_gid) + os.setuid(account.pw_uid) + try: + # The session keyring survives fork, setuid and exec and gives its possessor + # the administrator's keys (keyrings(7)); join a new, empty one instead. + libc_call("syscall", KEYCTL_SYSCALL, KEYCTL_JOIN_SESSION_KEYRING, 0) + except OSError as error: + if error.errno != errno.ENOSYS: # Without kernel keyrings there is none to leave. + raise InstallError("Cannot give the " + SERVICE_USER + " step its own session keyring: " + + error.strerror + ". Inspect the host, then rerun the command.") from None + # Set after setuid, which clears it: the child dies with the parent, and a + # parent that already died is noticed here. + libc_call("prctl", PR_SET_PDEATHSIG, signal.SIGKILL) + if os.getppid() != parent: + os._exit(1) + os.umask(0o077) + os.environ.clear() + os.environ.update(HOME=account.pw_dir, USER=account.pw_name, LOGNAME=account.pw_name, PATH=SAFE_PATH, + LANG="C.UTF-8", DOCKER_CONFIG=str(CHILD_DOCKER_CONFIG)) + os.chdir(account.pw_dir) + function(*arguments) + code = 0 + except (InstallError, node_spec.SpecificationError, distribution.DistributionError) as error: + print(str(error), file=sys.stderr) + except Exception as error: # noqa: BLE001 - the child must always report and exit + print("The step running as " + SERVICE_USER + " failed unexpectedly (" + type(error).__name__ + + "). Inspect the host, then rerun the command.", file=sys.stderr) + finally: + # A closed output pipe must not skip the exit below. + for stream in (sys.stdout, sys.stderr): + with contextlib.suppress(BaseException): + stream.flush() + os._exit(code) + + +run_as = as_service_user + + +def child_docker_config(): + """Create the root-owned empty directory the service user's docker calls read.""" + path = CHILD_DOCKER_CONFIG + if not path.exists() and not path.is_symlink(): + path.mkdir(mode=0o755) + os.chmod(path, 0o755) + info = path.lstat() + if not stat.S_ISDIR(info.st_mode) or info.st_uid != os.geteuid() or stat.S_IMODE(info.st_mode) & 0o022: + raise InstallError(str(path) + " must be a directory only root can write; inspect the host.") + + +@contextlib.contextmanager +def host_lock(): + """One node installation or uninstallation at a time on this host. + + Root takes it exclusively around every sudo-mode change, so installing one node + never races removing another's account. Normal users share it when it exists. + It lives in /run, which the next boot clears.""" + path = SYSTEM_LOCKS / "parsar-node.lock" + root = os.geteuid() == 0 + descriptor = None + try: + if root: + descriptor = os.open(path, os.O_CREAT | os.O_RDONLY | os.O_NOFOLLOW, 0o644) + os.fchmod(descriptor, 0o644) # Readable by normal users despite the command's umask 077. + else: + descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW) + except OSError: + if root: + raise + if descriptor is None: + yield # No sudo-mode run has happened on this host since boot. + return + with os.fdopen(descriptor) as lock: + operation = fcntl.LOCK_EX if root else fcntl.LOCK_SH + deadline = time.monotonic() + LOCK_WAIT_SECONDS + waiting = False + while True: + try: + fcntl.flock(lock, operation | fcntl.LOCK_NB) + break + except BlockingIOError: + if time.monotonic() > deadline: + raise InstallError("Another node installation or uninstallation on this host still holds " + + str(path) + "; find it with `sudo fuser " + str(path) + "`, then rerun.") from None + if not waiting: + print("Waiting for another node installation or uninstallation on this host...", flush=True) + waiting = True + time.sleep(1) + yield + + +def host_checks(): + if sys.version_info < (3, 9) or platform.system() != "Linux" or platform.machine() not in ("x86_64", "amd64"): + raise InstallError("Run with Python 3.9+ on Linux amd64." + NOTHING_CHANGED) + if not SYSTEMD_RUNNING.is_dir(): + raise InstallError("systemd is not the init system here, so this host cannot run the node service." + NOTHING_CHANGED) + for tool in ("systemctl", "useradd", "usermod", "userdel", "gpasswd"): + if shutil.which(tool) is None: + raise InstallError(tool + " is required to prepare the node service user." + NOTHING_CHANGED) + try: + enforcing = SELINUX_ENFORCE.read_text().strip() == "1" + except OSError: + enforcing = False + if enforcing: + raise InstallError("SELinux is enforcing on this host, which sudo mode does not support yet. Run the no-sudo " + "command as a prepared user instead." + NOTHING_CHANGED) + + +def root_file(path, content, replace=False): + """Write a root-owned 0644 file, or accept an identical existing one (or replace it).""" + if path.is_symlink() or (path.exists() and not path.is_file()): + raise InstallError(str(path) + " must be a regular file") + if path.exists(): + if path.stat().st_uid != os.geteuid(): + raise InstallError(str(path) + " must be owned by root; preserve it and inspect the node") + if path.read_text() == content: + return + if not replace: + raise InstallError(str(path) + " differs from this installer's version; preserve it and inspect the node") + if not path.parent.exists(): + # Readable by every user, so a no-sudo run can see a sudo-mode node; the caller's umask is 077. + path.parent.mkdir(parents=True, mode=0o755) + os.chmod(path.parent, 0o755) + descriptor, temporary = tempfile.mkstemp(prefix=".parsar-node-", dir=path.parent) + try: + with os.fdopen(descriptor, "w") as stream: + stream.write(content) + os.chmod(temporary, 0o644) + os.replace(temporary, path) + finally: + if os.path.exists(temporary): + os.unlink(temporary) + + +def read_root_json(path): + if not path.exists() and not path.is_symlink(): + return None + if path.is_symlink() or not path.is_file() or path.stat().st_uid != os.geteuid(): + raise InstallError(str(path) + " must be a regular file owned by root") + return json.loads(path.read_text()) + + +def node_records(): + """Installation IDs of the sudo-mode nodes recorded on this host.""" + if not SYSTEM_RECORDS.is_dir(): + return [] + return sorted(path.name[:-5] for path in SYSTEM_RECORDS.glob("*-*-*-*-*.json")) + + +def listdir_nofollow(base, *parts): + """Entries of base/parts, opening every component without following links; [] if absent or linked.""" + descriptors = [] + try: + descriptors.append(os.open(base, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)) + for part in parts: + descriptors.append(os.open(part, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW, dir_fd=descriptors[-1])) + return sorted(os.listdir(descriptors[-1])) + except OSError: + return [] + finally: + for descriptor in descriptors: + os.close(descriptor) + + +def service_account(): + try: + return pwd.getpwnam(SERVICE_USER) + except KeyError: + return None + + +def group_id(name): + try: + return grp.getgrnam(name).gr_gid + except (KeyError, AttributeError): + return None + + +def ours(account, recorded_uid=None): + """The account the installer created or adopted. + + It has the service home, a nologin shell, the recorded uid when there is one, + neither uid nor gid 0, and no groups besides its own, docker and kvm.""" + if (account is None or account.pw_dir != str(SERVICE_HOME) or not account.pw_shell.endswith(("nologin", "false")) + or recorded_uid not in (None, account.pw_uid) or account.pw_uid == 0 or account.pw_gid == 0): + return False + allowed = {account.pw_gid} | {gid for gid in map(group_id, DEVICE_GROUPS.values()) if gid is not None} + return set(os.getgrouplist(account.pw_name, account.pw_gid)) <= allowed + + +def host_capacity(provider, resources, docker_info): + if provider == "docker": + cpus, memory = docker_info.get("NCPU", 0), docker_info.get("MemTotal", 0) + else: + cpus, memory = os.cpu_count() or 0, 0 + with open("/proc/meminfo") as stream: + for line in stream: + if line.startswith("MemTotal:"): + memory = int(line.split()[1]) * 1024 + if cpus < resources["cpus"] or memory < resources["memory_mib"] * 1024 * 1024: + raise InstallError("This host has %d CPUs and %d MiB of memory; each sandbox needs %d CPUs and %d MiB." + % (cpus, memory // (1024 * 1024), resources["cpus"], resources["memory_mib"]) + NOTHING_CHANGED) + + +# The service user joins only the group that owns the provider's device, and only +# these names; a device owned by a privileged group (disk, sudo, wheel, adm) is refused. +DEVICE_GROUPS = {"docker": "docker", "microsandbox": "kvm"} + + +def provider_group(provider): + """Check Docker or KVM without installing either; return the group that grants access.""" + if provider == "docker": + if shutil.which("docker") is None: + raise InstallError("Docker Engine is not installed. Install it (https://docs.docker.com/engine/install/), " + "then rerun this command." + NOTHING_CHANGED) + try: + info = DOCKER_SOCKET.stat() + except OSError: + info = None + if info is None or not stat.S_ISSOCK(info.st_mode): + raise InstallError("Docker is not running: run `sudo systemctl enable --now docker`, then rerun this command." + NOTHING_CHANGED) + if info.st_gid == 0 or stat.S_IMODE(info.st_mode) & 0o060 != 0o060: + raise InstallError("Nodes reach Docker at /var/run/docker.sock through its group, but the socket is not " + "group-accessible (rootless Docker is not supported)." + NOTHING_CHANGED) + try: + details = json.loads(checked(list(DOCKER) + ["info", "--format", "{{json .}}"], "Docker is not running")) + except (InstallError, ValueError): + raise InstallError("Docker is not running: run `sudo systemctl enable --now docker`, then rerun this command." + NOTHING_CHANGED) from None + # docker info's JSON uses the Engine API names (CpuCfsQuota), not the Go field names. + if details.get("MemoryLimit") is not True or details.get("CpuCfsQuota") is not True: + raise InstallError("Docker on this host does not enforce CPU and memory limits; use cgroup v2, then rerun this command." + NOTHING_CHANGED) + return device_group(provider, DOCKER_SOCKET, info.st_gid), details + try: + info = KVM.stat() + except OSError: + info = None + if info is None or not stat.S_ISCHR(info.st_mode): + raise InstallError("KVM is unavailable: enable hardware virtualization (or nested virtualization for this VM) and " + "load kvm_intel or kvm_amd, then rerun this command." + NOTHING_CHANGED) + if stat.S_IMODE(info.st_mode) & 0o006 == 0o006: + return None, {} + if info.st_gid == 0 or stat.S_IMODE(info.st_mode) & 0o060 != 0o060: + raise InstallError("/dev/kvm must be group-accessible, for example root:kvm with mode 0660 (your distribution's " + "KVM package sets this), then rerun this command." + NOTHING_CHANGED) + return device_group(provider, KVM, info.st_gid), {} + + +def device_group(provider, device, gid): + try: + name = grp.getgrgid(gid).gr_name + except KeyError: + name = str(gid) + if name != DEVICE_GROUPS[provider]: + raise InstallError(str(device) + " belongs to the group " + name + "; the node's service user joins only the " + + DEVICE_GROUPS[provider] + " group. Give the device that group, then rerun this command." + NOTHING_CHANGED) + return name + + +def other_node(args, provider): + """Refuse a second sudo-mode Core on this host, or a second node for this installation. + + Sudo-mode nodes share the parsar-node account, so one host serves one Core. Nodes + installed without sudo are found in the invoking user's home and, for Docker, by + their network on this engine; other users' homes are not searched.""" + for installation in node_records(): + if installation != args.installation_id: + raise InstallError("This host already runs a sudo-mode node for another Core (installation " + installation + + "). Sudo mode serves one Core per host: remove that node on its Nodes page and " + "uninstall it first." + NOTHING_CHANGED) + sudo_user = os.environ.get("SUDO_USER", "") + if sudo_user and sudo_user != "root": + try: + home = pwd.getpwnam(sudo_user).pw_dir + except KeyError: + home = "" + if home.startswith("/") and listdir_nofollow(home, ".parsar", "nodes", args.installation_id): + raise InstallError("This host already runs a node for this installation, installed without sudo by " + sudo_user + + ". Remove it on the Nodes page and uninstall it as that user first." + NOTHING_CHANGED) + if provider == "docker": + networks = checked(list(DOCKER) + ["network", "ls", "--format", "{{.Name}}"], "Cannot inspect Docker networks").splitlines() + if "parsar-node-" + args.installation_id in networks: + raise InstallError("Another node for this installation already uses this Docker engine (network parsar-node-" + + args.installation_id + "). Remove it on the Nodes page and uninstall it first." + NOTHING_CHANGED) + + +def account_plan(): + """Check the service account, its record and home before changing anything. + + Returns the existing account (or None when it must be created) and the account record.""" + account, record = service_account(), read_root_json(SYSTEM_RECORDS / "account.json") + if account is not None and not ours(account, (record or {}).get("uid")): + raise InstallError("An account named " + SERVICE_USER + " exists but was not created or adopted by this installer " + "(home " + account.pw_dir + ", shell " + account.pw_shell + ", uid " + str(account.pw_uid) + + ", or groups beyond its own, docker and kvm). Rename or remove it, then rerun." + NOTHING_CHANGED) + if SERVICE_HOME.is_symlink() or (SERVICE_HOME.exists() and not SERVICE_HOME.is_dir()): + raise InstallError(str(SERVICE_HOME) + " must be a directory, not a symbolic link." + NOTHING_CHANGED) + if SERVICE_HOME.exists() and (account is None or SERVICE_HOME.stat().st_uid != account.pw_uid): + raise InstallError(str(SERVICE_HOME) + " exists but does not belong to the " + SERVICE_USER + " account; preserve " + "it and inspect the host." + NOTHING_CHANGED) + return account, record + + +def prepare_account(account, record, group): + """Create or adopt the service user and give it the provider's group; record every change at once.""" + records = SYSTEM_RECORDS / "account.json" + if account is None: + # A record of an account that no longer exists is stale; the new account replaces it. + records.unlink(missing_ok=True) + shell = next((path for path in ("/usr/sbin/nologin", "/sbin/nologin") if os.path.exists(path)), "/bin/false") + checked(["useradd", "--system", "--user-group", "--no-create-home", "--home-dir", str(SERVICE_HOME), + "--shell", shell, SERVICE_USER], "Cannot create the " + SERVICE_USER + " service user") + account = service_account() + record = {"format": 1, "created": True, "uid": account.pw_uid, "groups_added": []} + root_file(records, json_text(record), replace=True) + elif record is None: + record = {"format": 1, "created": False, "uid": account.pw_uid, "groups_added": [], + "home_mode": oct(stat.S_IMODE(SERVICE_HOME.stat().st_mode)) if SERVICE_HOME.exists() else None} + root_file(records, json_text(record), replace=True) + if not SERVICE_HOME.exists(): + SERVICE_HOME.mkdir(mode=0o700, parents=True) + os.chown(SERVICE_HOME, account.pw_uid, account.pw_gid) + os.chmod(SERVICE_HOME, 0o700) + if group and SERVICE_USER not in grp.getgrnam(group).gr_mem: + checked(["usermod", "--append", "--groups", group, SERVICE_USER], "Cannot add " + SERVICE_USER + " to the " + group + " group") + record = dict(record, groups_added=sorted(set(record["groups_added"]) | {group})) + root_file(records, json_text(record), replace=True) + return service_account() + + +def system_unit(root, provider): + def quote(value): + return '"' + str(value).replace("\\", "\\\\").replace('"', '\\"').replace("%", "%%") + '"' + after = "network-online.target docker.service" if provider == "docker" else "network-online.target" + # Root owns this file; the service user can change only its own node files. The + # service runs with the account's own primary group. + return ("[Unit]\nDescription=Parsar sandbox node " + root.name + "\nWants=network-online.target\nAfter=" + after + + "\nStartLimitIntervalSec=0\n\n[Service]\nType=exec\nUser=" + SERVICE_USER + + "\nExecStart=:" + quote(root / COMMON[0]) + " run --config " + quote(root / "provider.json") + + " --state-dir " + quote(root / "state/node") + "\nWorkingDirectory=" + str(root).replace("%", "%%") + + "\nRestart=on-failure\nRestartSec=5s\nRestartPreventExitStatus=78\nKillMode=process\nUMask=0077" + + "\n\n[Install]\nWantedBy=multi-user.target\n") + + +def recorded_origin(value, source): + """A Core address read from a file, checked with the same rule as the command line.""" + try: + return origin(value) + except (argparse.ArgumentTypeError, TypeError, AttributeError): + raise InstallError(source + " holds an invalid Core address; preserve it and inspect the host." + NOTHING_CHANGED) from None + + +def node_record(installation_id): + """The root-owned record of this installation's sudo-mode node, checked against the fixed paths.""" + record = read_root_json(SYSTEM_RECORDS / (installation_id + ".json")) + if record is not None: + expected = {"installation_id": installation_id, "node_root": str(SERVICE_HOME / ".parsar/nodes" / installation_id), + "unit": str(SYSTEM_UNITS / unit_name(installation_id))} + if any(record.get(key) != value for key, value in expected.items()) or record.get("provider") not in DEVICE_GROUPS: + raise InstallError(str(SYSTEM_RECORDS / (installation_id + ".json")) + " is not this installer's record; preserve " + "it and inspect the host." + NOTHING_CHANGED) + recorded_origin(record.get("core_url"), str(SYSTEM_RECORDS / (installation_id + ".json"))) + return record + + +def install_system(args, token): + """Sudo mode: prepare the host, then run the node as a root-owned system service.""" + os.environ["PATH"] = SAFE_PATH + host_checks() + # Checks that change nothing run first, so a refusal leaves no trace, not even a lock. + record = node_record(args.installation_id) + configuration = None + if record is None: + print("Reading the Core deployment specification...", flush=True) + configuration = node_spec.fetch(args, token, None, open_request, allow_enrollment=True) + provider = configuration["provider"] + else: + provider = record["provider"] + if record["core_url"] != args.core_url: + raise InstallError("This host's node uses " + record["core_url"] + ", but this command uses " + args.core_url + + ". Remove the node on the Nodes page, uninstall it, then run a new command." + NOTHING_CHANGED) + print("Checking host requirements...", flush=True) + group, details = provider_group(provider) + if record is None: + other_node(args, provider) + host_capacity(provider, configuration["specification"]["resources"], details) + with host_lock(): + if node_record(args.installation_id) != record or (record is None and [i for i in node_records() if i != args.installation_id]): + raise InstallError("Another node installation changed this host meanwhile; rerun the command." + NOTHING_CHANGED) + account, account_record = account_plan() + # Every check has passed; from here on the host changes. + account = prepare_account(account, account_record, group) + child_docker_config() + root = SERVICE_HOME / ".parsar/nodes" / args.installation_id + unit = SYSTEM_UNITS / unit_name(args.installation_id) + root_file(SYSTEM_RECORDS / (args.installation_id + ".json"), + json_text({"format": 1, "installation_id": args.installation_id, "provider": provider, + "core_url": args.core_url, "node_root": str(root), "unit": str(unit)})) + args.system, args.provider = True, provider + run_as(account, prepare_service_node, args, token) + root_file(unit, system_unit(root, provider)) + print("Starting the node service...", flush=True) + checked(["systemctl", "daemon-reload"], "Cannot reload systemd") + checked(["systemctl", "enable", "--now", unit.name], "Cannot start the node service; retained identity is unchanged") + checked(["systemctl", "is-active", "--quiet", unit.name], "Node service is unavailable; inspect sudo journalctl -u " + unit.name) + print("Waiting for Core connection and provider readiness...", flush=True) + run_as(account, wait_ready, root, args) + print("Node connected to Core and provider ready. It runs as " + SERVICE_USER + " in the system service " + unit.name + ".") + print("Logs: sudo journalctl -u " + unit.name) + + +# Uninstall ------------------------------------------------------------------- + +def private_json(path): + """Read one of this user's private JSON objects without following links or racing a swap.""" + try: + descriptor = os.open(path, os.O_RDONLY | os.O_NOFOLLOW) + except OSError: + return None + with os.fdopen(descriptor) as stream: + info = os.fstat(stream.fileno()) + if not stat.S_ISREG(info.st_mode) or info.st_uid != os.getuid() or info.st_size > 16384: + return None + try: + value = json.loads(stream.read()) + except ValueError: + return None + return value if isinstance(value, dict) else None + + +def no_links(path): + """Refuse a path whose components include a symbolic link.""" + if os.path.realpath(path) != str(path): + raise InstallError(str(path) + " is reached through a symbolic link; inspect and remove it by hand." + NOTHING_CHANGED) + + +def confirm_removed(root, core_url, force): + """Continue only once Core rejects the node's credential, or when it never registered. + + Runs as the node's own user. The Core address comes from the caller: in sudo mode + the root-owned record, never a file the service user can write.""" + no_links(root) + identity_file = root / "state/node/identity.json" + if not (root / "registered.json").exists() and not identity_file.exists(): + return + if force: + print("Skipping the Core check (--force).", flush=True) + return + stored = private_json(identity_file) or {} + try: + node_id, credential = stored["identity"]["node_id"], stored["credential"] + if str(uuid.UUID(node_id)) != node_id or not re.fullmatch(r"[0-9a-f]{64}", credential): + raise ValueError() + except (TypeError, KeyError, ValueError, AttributeError): + raise InstallError("Retained node identity is missing or invalid; rerun with --force only if Core no longer exists." + NOTHING_CHANGED) from None + request = urllib.request.Request(core_url + "/api/v1/sandbox-node/identity?" + urlencode({"node_id": node_id}), + headers={"Authorization": "Bearer " + credential}) + try: + with open_request(request, timeout=15): + pass + except urllib.error.HTTPError as error: + if error.code == 401: + return + detail = "HTTP " + str(error.code) + except (urllib.error.URLError, TimeoutError, ConnectionError, http.client.HTTPException, InstallError): + detail = "Core unreachable" + else: + raise InstallError("Core still lists this node. Remove it on the Nodes page first; Core refuses while it keeps " + "sandboxes." + NOTHING_CHANGED) + raise InstallError("Cannot confirm with Core that this node was removed (" + detail + "). Rerun with --force only " + "if this Core no longer exists." + NOTHING_CHANGED) + + +def release_docker_network(installation_id): + """Remove the node's Docker network; never containers, volumes or images.""" + if shutil.which("docker") is None: + return + network = "parsar-node-" + installation_id + try: + if network not in checked(list(DOCKER) + ["network", "ls", "--format", "{{.Name}}"], "Docker unavailable").splitlines(): + return + remaining = checked(list(DOCKER) + ["ps", "--all", "--filter", "label=io.parsar.agents-api.installation=" + installation_id, + "--format", "{{.Names}}"], "Docker unavailable").split() + except InstallError: + print("Docker is unavailable, so network " + network + " was not removed.") + return + if remaining: + print("Kept Docker network " + network + ": these containers still exist: " + ", ".join(remaining)) + return + checked(list(DOCKER) + ["network", "rm", network], "Cannot remove Docker network " + network) + + +def remove_node_files(root, installation_id): + """Delete the node's state directory; keep the Runtime image and the microsandbox store. + + Runs as the node's own user, so a link it planted can never reach another user's files.""" + no_links(root) + provider = private_json(root / "provider.json") or {} + image = (provider.get("docker") or {}).get("image") + runtime_home = micro_home(installation_id) + if root.exists(): + shutil.rmtree(root) + # The service user can write provider.json; print only what an image ID can be. + if isinstance(image, str) and re.fullmatch(r"sha256:[0-9a-f]{64}", image): + print("Kept the Runtime image " + image + "; remove it with `docker image rm " + image + "` if no other node uses it.") + if runtime_home.exists(): + sudo_mode = str(Path.home()) == str(SERVICE_HOME) + remove = ("sudo -u " + SERVICE_USER + " rm -rf " if sudo_mode else "rm -rf ") + str(runtime_home) + print("Kept the microsandbox store " + str(runtime_home) + " with its images and any sandbox state. When no " + "microVM runs (`pgrep -u " + os.environ.get("USER", "") + "` is empty), remove it with `" + remove + + "`, then rerun this uninstall command.") + + +def uninstall_system(args): + """Sudo mode: remove the system service, the node files and, when unused, the service user.""" + os.environ["PATH"] = SAFE_PATH + if shutil.which("systemctl") is None: + raise InstallError("systemctl is required." + NOTHING_CHANGED) + record = node_record(args.installation_id) + unit = SYSTEM_UNITS / unit_name(args.installation_id) + # Only root-owned files decide whether a node is installed; the service home is not read here. + if record is None and not unit.exists() and not (SYSTEM_RECORDS / "account.json").exists(): + print("No node for installation " + args.installation_id + " was installed with sudo on this host. If it was " + "installed without sudo, run the uninstall command as that user without sudo.") + return + with host_lock(): + record = node_record(args.installation_id) + account, account_record = service_account(), read_root_json(SYSTEM_RECORDS / "account.json") + root = SERVICE_HOME / ".parsar/nodes" / args.installation_id + if record is None and not unit.exists(): + print("No node for installation " + args.installation_id + " was installed with sudo on this host.") + else: + if not ours(account, (account_record or {}).get("uid")): + raise InstallError("The " + SERVICE_USER + " account is missing or differs from the one this installer " + "recorded, so its files are not removed. Inspect the host." + NOTHING_CHANGED) + core_url = record["core_url"] if record else None + if core_url is None and not args.force: + raise InstallError("This node's record is missing, so Core cannot be asked; rerun with --force only if " + "Core no longer exists." + NOTHING_CHANGED) + child_docker_config() + run_as(account, confirm_removed, root, core_url, args.force) + if unit.exists(): + checked(["systemctl", "disable", "--now", unit.name], "Cannot stop the node service " + unit.name) + unit.unlink() + checked(["systemctl", "daemon-reload"], "Cannot reload systemd") + # A removed node's service ends failed (exit 78); drop that state with the unit. + with contextlib.suppress(InstallError): + checked(["systemctl", "reset-failed", unit.name], "Cannot reset " + unit.name) + release_docker_network(args.installation_id) + run_as(account, remove_node_files, root, args.installation_id) + (SYSTEM_RECORDS / (args.installation_id + ".json")).unlink(missing_ok=True) + print("Node for installation " + args.installation_id + " uninstalled from this host.") + release_account(account, account_record) + + +def release_account(account, record): + """When no node remains, undo the account changes: delete a created account, or leave an adopted one as found.""" + if record is None or node_records(): + return + if not ours(account, record.get("uid")): + print("The " + SERVICE_USER + " account differs from the one this installer recorded; it was left alone.") + return + if record.get("created"): + stores = [str(SERVICE_HOME / ".parsar/m" / name) for name in listdir_nofollow(SERVICE_HOME, ".parsar", "m")] + if stores: + # The names come from the service user's home; show them only as plain text. + print("Kept the " + SERVICE_USER + " user while microsandbox stores remain: " + plain(", ".join(stores), sys.stdout.encoding) + + ". Remove them with `sudo -u " + SERVICE_USER + " rm -rf `, then rerun this uninstall command.") + return + checked(["userdel", SERVICE_USER], "Cannot remove the " + SERVICE_USER + " user; stop its processes and rerun the uninstall command") + # No process can run as the deleted user, so nothing races this removal. + if SERVICE_HOME.is_dir() and not SERVICE_HOME.is_symlink(): + shutil.rmtree(SERVICE_HOME) + print("Removed the " + SERVICE_USER + " service user, which this installer created.") + else: + for group in record.get("groups_added", []): + try: + member = SERVICE_USER in grp.getgrnam(group).gr_mem + except KeyError: + member = False + if member: + checked(["gpasswd", "--delete", SERVICE_USER, group], "Cannot remove " + SERVICE_USER + " from the " + group + " group") + print("Removed " + SERVICE_USER + " from the " + group + " group, which this installer added.") + if record.get("home_mode") and SERVICE_HOME.is_dir() and not SERVICE_HOME.is_symlink(): + os.chmod(SERVICE_HOME, int(record["home_mode"], 8)) + (SYSTEM_RECORDS / "account.json").unlink() + with contextlib.suppress(OSError): + SYSTEM_RECORDS.rmdir() + + +def uninstall_user(args): + with host_lock(): + root = Path.home() / ".parsar/nodes" / args.installation_id + if not root.exists(): + print("No node for installation " + args.installation_id + " is installed for this user. If it was installed " + "with sudo, run the uninstall command with sudo.") + return + identity = private_json(root / "state/node/identity.json") or {} + core_url = identity.get("core_url") + if core_url is not None: + recorded_origin(core_url, str(root / "state/node/identity.json")) + elif (root / "registered.json").exists() and not args.force: + raise InstallError("Retained node identity is missing or invalid; rerun with --force only if Core no longer exists." + NOTHING_CHANGED) + confirm_removed(root, core_url, args.force) + unit = root / unit_name(args.installation_id) + service = unit.exists() + if service: + user_bus() + checked(["systemctl", "--user", "disable", "--now", unit.name], "Cannot stop the node service " + unit.name) + release_docker_network(args.installation_id) + remove_node_files(root, args.installation_id) + if service: + checked(["systemctl", "--user", "daemon-reload"], "Cannot reload the systemd user manager") + with contextlib.suppress(InstallError): + checked(["systemctl", "--user", "reset-failed", unit.name], "Cannot reset " + unit.name) + print("Node for installation " + args.installation_id + " uninstalled for this user.") def wait_ready(root, args, timeout=60): @@ -451,32 +1319,71 @@ def wait_ready(root, args, timeout=60): remaining = deadline - time.monotonic() if remaining > 0: time.sleep(min(2, remaining)) - raise InstallError(detail + "; state and service are retained. Inspect journalctl --user -u parsar-node-" - + args.installation_id + ".service, then rerun the installation command") + journal = ("sudo journalctl -u " if getattr(args, "system", False) else "journalctl --user -u ") + unit_name(args.installation_id) + raise InstallError(detail + "; state and service are retained. Inspect " + journal + ", then rerun the installation command") + + +def read_token(args, parser): + """The one-time token comes on standard input, never in argv or a sudo command line.""" + # Until the Web console's command sends it on standard input, its no-sudo form passes it here. + environment = os.environ.pop("PARSAR_NODE_ENROLLMENT_TOKEN", None) + if args.enrollment_token_stdin and environment is not None: + parser.error("pass the enrollment token on standard input only") + if environment is not None and os.geteuid() == 0: + # `sudo VAR=... python3` would put the token in sudo's command line and log. + parser.error("with sudo or as root, pass the enrollment token on standard input with --enrollment-token-stdin") + if not args.enrollment_token_stdin: + return environment or "" + if sys.stdin.isatty(): + return getpass.getpass("Enrollment token: ").strip() + return sys.stdin.readline(4098).strip() def main(argv=None): - token = os.environ.pop("PARSAR_NODE_ENROLLMENT_TOKEN", "") parser = argparse.ArgumentParser(description=__doc__) - source = parser.add_mutually_exclusive_group(required=True) + source = parser.add_mutually_exclusive_group() source.add_argument("--source-url", type=origin) source.add_argument("--bundle", type=Path) - parser.add_argument("--core-url", type=origin, required=True) + parser.add_argument("--core-url", type=origin) parser.add_argument("--provider", choices=("docker", "microsandbox"), help="Optional assertion; Core owns provider selection") parser.add_argument("--installation-id", required=True) + parser.add_argument("--enrollment-token-stdin", action="store_true", help="Read the one-time enrollment token from standard input") + parser.add_argument("--uninstall", action="store_true", help="Remove this host's node after it was removed on the Nodes page") + parser.add_argument("--force", action="store_true", help="With --uninstall: skip the Core check, for a Core that no longer exists") args = parser.parse_args(argv) if str(uuid.UUID(args.installation_id)) != args.installation_id: raise InstallError("Installation ID must be a canonical UUID") + if args.uninstall: + if args.source_url or args.bundle or args.core_url or args.provider or args.enrollment_token_stdin: + parser.error("--uninstall takes only --installation-id and --force") + os.environ.pop("PARSAR_NODE_ENROLLMENT_TOKEN", None) + (uninstall_system if os.geteuid() == 0 else uninstall_user)(args) + return + if args.force: + parser.error("--force applies only to --uninstall") + if not (args.source_url or args.bundle) or not args.core_url: + parser.error("--source-url (or --bundle) and --core-url are required") if args.bundle is not None and (not args.bundle.is_absolute() or args.bundle.resolve() != args.bundle): raise InstallError("Local bundle must be an absolute directory without symlinks") + token = read_token(args, parser) if len(token) > 4096 or any(c.isspace() for c in token): raise InstallError("A valid one-time enrollment credential is required") - install(args, token) + if os.geteuid() == 0: + install_system(args, token) + else: + install(args, token) if __name__ == "__main__": try: main() + except ChildFailed as failure: + if str(failure): + print(str(failure), file=sys.stderr) + sys.exit(1) + except KeyboardInterrupt: # Ctrl-C outside the service-user steps, as while waiting for the host lock. + print(INTERRUPTED, file=sys.stderr) + sys.exit(130) except (InstallError, node_spec.SpecificationError, distribution.DistributionError, OSError, ValueError, KeyError, subprocess.SubprocessError) as error: print(str(error) if isinstance(error, (InstallError, node_spec.SpecificationError, distribution.DistributionError)) else "Node installation failed; check host prerequisites and retained private files", file=sys.stderr) sys.exit(1) diff --git a/deploy/install/node_spec.py b/deploy/install/node_spec.py index 3dff5f8ea..7017dcd29 100644 --- a/deploy/install/node_spec.py +++ b/deploy/install/node_spec.py @@ -110,6 +110,10 @@ def fetch(args, token, retained, open_request, allow_enrollment=False): raise ValueError() data = validate(json.loads(raw), args) except urllib.error.HTTPError as error: + if error.code == 401 and retained is not None and not allow_enrollment: + raise SpecificationError("Core no longer accepts this node: it was removed on the Nodes page, or a sandbox " + "deployment change retired it. Uninstall it with node-install.pyz --uninstall " + "--installation-id " + args.installation_id + ", then add the host with a new command.") from None if error.code == 404: raise SpecificationError("Core node configuration was not found (HTTP 404); route /api/v1 on the Core origin directly to Core, not to Web") from None if error.code == 409: diff --git a/deploy/install/test_distribution.py b/deploy/install/test_distribution.py index bdb9b7773..4b098e90b 100644 --- a/deploy/install/test_distribution.py +++ b/deploy/install/test_distribution.py @@ -23,11 +23,27 @@ def setUp(self): self.root = Path(self.temp.name).resolve() self.data = b'prebuilt artifact' * 1000 self.requests = [] + self.ranges = [] + self.if_ranges = [] self.status = 200 + self.resumable = False test = self class Handler(http.server.BaseHTTPRequestHandler): def do_GET(self): test.requests.append(self.path) + test.ranges.append(self.headers.get('Range')) + test.if_ranges.append(self.headers.get('If-Range')) + if test.resumable: + # The first response breaks off halfway; a Range request gets the rest. + start = int(self.headers['Range'][6:-1]) if self.headers.get('Range') else 0 + self.send_response(206 if start else 200) + self.send_header('Last-Modified', 'Sat, 26 Sep 2026 00:00:00 GMT') + if start: + self.send_header('Content-Range', f'bytes {start}-{len(test.data) - 1}/{len(test.data)}') + self.send_header('Content-Length', str(len(test.data) - start)) + self.end_headers() + self.wfile.write(test.data[start:] if start else test.data[:len(test.data) // 2]) + return self.send_response(test.status) if test.status == 302: self.send_header('Location', 'https://elsewhere.example' + self.path) @@ -90,9 +106,28 @@ def test_truncated_transfer_can_be_rerun(self): self.data = self.data[:20] with patch.object(distribution.time, 'sleep'), self.assertRaisesRegex(distribution.DistributionError, 'interrupted'): distribution.obtain_artifact(self.manifest, 'native/bin/node', self.root / 'node') - self.assertEqual(list(self.root.iterdir()), []) + # Only the private partial file stays, for the rerun to continue. + self.assertEqual([path.name for path in self.root.iterdir()], ['.node.partial']) self.data = b'prebuilt artifact' * 1000 distribution.obtain_artifact(self.manifest, 'native/bin/node', self.root / 'node') + self.assertEqual([path.name for path in self.root.iterdir()], ['node']) + + def test_interrupted_download_resumes_with_the_missing_bytes(self): + self.resumable = True + with patch.object(distribution.time, 'sleep'): + distribution.obtain_artifact(self.manifest, 'native/bin/node', self.root / 'node') + self.assertEqual((self.root / 'node').read_bytes(), self.data) + self.assertEqual(self.ranges, [None, f'bytes={len(self.data) // 2}-']) + # If-Range: a file that changed since the partial began comes back whole instead of mixed. + self.assertEqual(self.if_ranges, [None, 'Sat, 26 Sep 2026 00:00:00 GMT']) + self.assertEqual([path.name for path in self.root.iterdir()], ['node']) + + def test_a_stalled_download_stops_and_keeps_its_part(self): + with patch.object(distribution, 'SLOW_SECONDS', 0), \ + self.assertRaisesRegex(distribution.DistributionError, 'stalled.*rerun the command to resume'): + distribution.obtain_artifact(self.manifest, 'native/bin/node', self.root / 'node') + self.assertEqual(len(self.requests), 1) # Not retried as a network error. + self.assertEqual([path.name for path in self.root.iterdir()], ['.node.partial']) def test_offline_and_runtime_expansion_are_verified(self): raw = b'synthetic tar contents' * 1000 diff --git a/deploy/install/test_node_install.py b/deploy/install/test_node_install.py index f653f5534..950205811 100644 --- a/deploy/install/test_node_install.py +++ b/deploy/install/test_node_install.py @@ -1,22 +1,46 @@ """Exercise node installation without running providers or changing user services.""" import argparse +import fcntl import hashlib import gzip import io import json import os from pathlib import Path +import signal +import socket import stat import subprocess +import sys import tempfile +import time import unittest import urllib.error +from types import SimpleNamespace from unittest import mock import node_install as installer import node_spec + +def ended(pid, wait=5): + """The process is gone or a zombie waiting for its new parent to reap it.""" + deadline = time.monotonic() + wait + while time.monotonic() < deadline: + try: + if Path("/proc/%d/stat" % pid).read_text().rsplit(")", 1)[1].split()[0] == "Z": + return True + except (FileNotFoundError, ProcessLookupError): + return True + time.sleep(0.05) + return False + +class Response(io.BytesIO): + """An HTTP response body with the status and headers the downloader reads.""" + status, headers = 200, {} + + class NodeInstallTests(unittest.TestCase): def setUp(self): base = Path.home() / ".parsar/tests/node-install" @@ -64,12 +88,13 @@ def configuration_response(self, request, **kwargs): "specification_digest": node_spec.digest(self.args.provider, spec)} return io.BytesIO(json.dumps(configuration).encode()) - def artifact_response(self, url, **kwargs): + def artifact_response(self, request, **kwargs): + url = getattr(request, "full_url", request) # The fixture manifest records a release URL; nodes still download from their console. self.assertTrue(url.startswith(self.args.source_url + "/node-install/artifacts/"), url) for name, item in self.manifest["artifacts"].items(): if url.endswith("/" + item["filename"]): - return io.BytesIO(self.payloads[name]) + return Response(self.payloads[name]) raise AssertionError("Unexpected artifact URL: " + url) def refresh_manifest(self): @@ -104,6 +129,15 @@ def checked(self, arguments, failure, **kwargs): raise installer.InstallError(failure) if "enable" in arguments and self.fail_service: raise installer.InstallError(failure) + if arguments[:1] == ["useradd"]: + self.account = SimpleNamespace(pw_name="parsar-node", pw_uid=os.getuid(), pw_gid=os.getgid(), + pw_dir=str(installer.SERVICE_HOME), pw_shell="/usr/sbin/nologin") + if arguments[:1] == ["usermod"]: + if getattr(self, "fail_usermod", False): + raise installer.InstallError(failure) + self.joined = True + if "{{json .}}" in arguments: + return json.dumps({"MemoryLimit": True, "CpuCfsQuota": True, "NCPU": 8, "MemTotal": 16 << 30}) if "load" in arguments: self.image_present = True if "inspect" in arguments: @@ -315,6 +349,313 @@ def test_main_removes_token_environment_before_any_subprocess(self): "--provider", "docker", "--installation-id", self.args.installation_id]) self.assertNotIn("PARSAR_NODE_ENROLLMENT_TOKEN", os.environ) + def sudo_host(self, enforcing=False): + """Sudo mode against temporary system paths; the service-user step runs in-process.""" + system = self.home / "system" + (system / "systemd").mkdir(parents=True) + (system / "run").mkdir() + (system / "selinux").write_text("1" if enforcing else "0") + docker_socket = socket.socket(socket.AF_UNIX) + docker_socket.bind(str(system / "docker.sock")) + self.addCleanup(docker_socket.close) + os.chmod(system / "docker.sock", 0o660) + self.account, self.joined, self.docker_installed, self.device_group = None, False, True, "docker" + service_home = self.home / "service" + self.root = service_home / ".parsar/nodes" / self.args.installation_id + + self.service_steps = [] + + def run_as(account, function, *arguments): + self.service_steps.append("wait_ready" if function is installer.wait_ready else function.__name__) + with mock.patch.object(installer.Path, "home", return_value=Path(account.pw_dir)): + function(*arguments) + for patch in (mock.patch.multiple(installer, SERVICE_HOME=service_home, SYSTEM_RECORDS=system / "etc", + SYSTEM_UNITS=system / "units", SYSTEM_LOCKS=system / "run", + CHILD_DOCKER_CONFIG=system / "docker-config", + SYSTEMD_RUNNING=system / "systemd", SELINUX_ENFORCE=system / "selinux", + DOCKER_SOCKET=system / "docker.sock", run_as=run_as, + service_account=lambda: self.account), + mock.patch.object(installer.shutil, "which", side_effect=lambda tool: None if tool == "docker" and not self.docker_installed else "/usr/bin/" + tool), + mock.patch.object(installer.grp, "getgrnam", side_effect=lambda name: SimpleNamespace(gr_mem=["parsar-node"] if self.joined else [])), + mock.patch.object(installer.grp, "getgrgid", side_effect=lambda gid: SimpleNamespace(gr_name=self.device_group))): + patch.start() + self.addCleanup(patch.stop) + return system + + def test_sudo_mode_prepares_the_host_and_reruns_without_changes(self): + system = self.sudo_host() + installer.install_system(self.args, "synthetic-once-token") + commands = [call for call, _ in self.calls] + self.assertIn(["useradd", "--system", "--user-group", "--no-create-home", "--home-dir", str(self.home / "service"), + "--shell", next(path for path in ("/usr/sbin/nologin", "/sbin/nologin", "/bin/false") if os.path.exists(path) or path == "/bin/false"), + "parsar-node"], commands) + self.assertTrue(any(call[:3] == ["usermod", "--append", "--groups"] for call in commands)) + self.assertIn(["systemctl", "enable", "--now", "parsar-node-" + self.args.installation_id + ".service"], commands) + unit = (system / "units" / ("parsar-node-" + self.args.installation_id + ".service")).read_text() + for line in ("User=parsar-node", "After=network-online.target docker.service", "WantedBy=multi-user.target", + "RestartPreventExitStatus=78", "StartLimitIntervalSec=0"): + self.assertIn(line, unit) + self.assertNotIn("synthetic-once-token", unit) + self.assertNotIn("Group=", unit) + # Every step that touches the service user's files runs as that user. + self.assertEqual(self.service_steps, ["prepare_service_node", "wait_ready"]) + self.assertTrue((self.root / "registered.json").exists()) + self.assertFalse((self.root / ("parsar-node-" + self.args.installation_id + ".service")).exists()) + account = json.loads((system / "etc/account.json").read_text()) + self.assertEqual((account["created"], account["groups_added"]), (True, ["docker"])) + self.assertEqual(stat.S_IMODE((system / "etc").stat().st_mode), 0o755) + before = {path: path.read_bytes() for path in (system / "etc").iterdir()} + self.calls.clear() + installer.install_system(self.args, "") + commands = [call for call, _ in self.calls] + self.assertFalse([call for call in commands if call[:1] in (["useradd"], ["usermod"]) or "register" in call]) + self.assertEqual({path: path.read_bytes() for path in (system / "etc").iterdir()}, before) + # A command naming another Core address is refused before anything changes. + self.args.core_url, self.calls[:] = "https://moved.example", [] + with self.assertRaisesRegex(installer.InstallError, "this command uses https://moved.example.*Nothing was changed"): + installer.install_system(self.args, "") + self.assertFalse([call for call, _ in self.calls if call[:1] in (["useradd"], ["usermod"], ["systemctl"])]) + self.assertEqual({path: path.read_bytes() for path in (system / "etc").iterdir()}, before) + + def test_no_sudo_installs_can_share_the_host_lock_root_created(self): + locks = self.home / "run" + locks.mkdir() + previous = os.umask(0o077) # As the Web command sets it. + self.addCleanup(os.umask, previous) + with mock.patch.object(installer, "SYSTEM_LOCKS", locks), mock.patch.object(installer.os, "geteuid", return_value=0): + with installer.host_lock(): + pass + self.assertEqual(stat.S_IMODE((locks / "parsar-node.lock").stat().st_mode), 0o644) + + def test_sudo_mode_refusals_change_nothing(self): + foreign = SimpleNamespace(pw_name="parsar-node", pw_uid=4242, pw_gid=4242, pw_dir="/home/parsar-node", pw_shell="/bin/bash") + for case, message in (("selinux", "SELinux is enforcing"), ("docker", "Docker Engine is not installed"), + ("account", "not created or adopted by this installer"), ("group", "belongs to the group disk"), + ("home", "does not belong to the parsar-node account")): + with self.subTest(case=case): + system = self.sudo_host(enforcing=case == "selinux") + self.docker_installed = case != "docker" + self.account = foreign if case == "account" else None + self.device_group = "disk" if case == "group" else "docker" + if case == "home": # a home directory left without its account + (self.home / "service").mkdir() + self.calls.clear() + with self.assertRaisesRegex(installer.InstallError, message + ".*Nothing was changed"): + installer.install_system(self.args, "synthetic-once-token") + self.assertFalse([call for call, _ in self.calls if call[:1] in (["useradd"], ["usermod"], ["systemctl"])]) + self.assertFalse((system / "etc").exists()) + self.assertFalse((system / "units").exists()) + for path in sorted(system.rglob("*"), reverse=True): + path.unlink() if not path.is_dir() else path.rmdir() + system.rmdir() + if (self.home / "service").exists(): + (self.home / "service").rmdir() + + def test_uninstall_waits_for_core_to_reject_the_node(self): + system = self.sudo_host() + installer.install_system(self.args, "synthetic-once-token") + uninstall = SimpleNamespace(installation_id=self.args.installation_id, force=False) + with mock.patch.object(installer, "open_request", return_value=io.BytesIO(b"{}")), \ + self.assertRaisesRegex(installer.InstallError, "still lists this node.*Nothing was changed"): + installer.uninstall_system(uninstall) + self.assertTrue((self.root / "registered.json").exists()) + rejected = urllib.error.HTTPError("https://core.example", 401, "", {}, None) + self.calls.clear() + with mock.patch.object(installer, "open_request", side_effect=rejected): + installer.uninstall_system(uninstall) + commands = [call for call, _ in self.calls] + self.assertEqual(self.service_steps[-2:], ["confirm_removed", "remove_node_files"]) + self.assertIn(["systemctl", "disable", "--now", "parsar-node-" + self.args.installation_id + ".service"], commands) + self.assertIn(["userdel", "parsar-node"], commands) + self.assertIn(["systemctl", "reset-failed", "parsar-node-" + self.args.installation_id + ".service"], commands) + self.assertFalse(any(call[-2:] == ["rm", "--force"] or "prune" in call or ("image" in call and "rm" in call) for call in commands)) + self.assertFalse(self.root.exists()) + self.assertFalse((system / "units" / ("parsar-node-" + self.args.installation_id + ".service")).exists()) + self.assertFalse((system / "etc").exists()) + + def test_uninstall_leaves_an_adopted_account_as_found(self): + system = self.sudo_host() + (self.home / "service").mkdir() + os.chmod(self.home / "service", 0o755) + self.account = SimpleNamespace(pw_name="parsar-node", pw_uid=os.getuid(), pw_gid=os.getgid(), + pw_dir=str(self.home / "service"), pw_shell="/usr/sbin/nologin") + installer.install_system(self.args, "synthetic-once-token") + self.assertFalse([call for call, _ in self.calls if call[:1] == ["useradd"]]) + self.assertEqual(json.loads((system / "etc/account.json").read_text())["created"], False) + self.calls.clear() + with mock.patch.object(installer, "open_request", side_effect=urllib.error.HTTPError("https://core.example", 401, "", {}, None)): + installer.uninstall_system(SimpleNamespace(installation_id=self.args.installation_id, force=False)) + commands = [call for call, _ in self.calls] + self.assertNotIn(["userdel", "parsar-node"], commands) + self.assertIn(["gpasswd", "--delete", "parsar-node", "docker"], commands) + self.assertTrue((self.home / "service").is_dir()) + self.assertEqual(stat.S_IMODE((self.home / "service").stat().st_mode), 0o755) + self.assertFalse((system / "etc").exists()) + + def test_sudo_mode_serves_one_core_per_host(self): + system = self.sudo_host() + installer.install_system(self.args, "synthetic-once-token") + self.args.installation_id, self.calls[:] = "0c6f35d2-6d7c-4a53-8d5c-3a3b1d3a0f11", [] + with self.assertRaisesRegex(installer.InstallError, "one Core per host.*Nothing was changed"): + installer.install_system(self.args, "synthetic-once-token") + self.assertFalse([call for call, _ in self.calls if call[:1] in (["useradd"], ["usermod"], ["systemctl"])]) + self.assertEqual(installer.node_records(), ["94be54a1-138c-4f30-bc87-b13686272dbe"]) + + def test_created_account_is_recorded_before_any_later_step(self): + system = self.sudo_host() + self.fail_usermod = True + with self.assertRaises(installer.InstallError): + installer.install_system(self.args, "synthetic-once-token") + self.assertEqual(json.loads((system / "etc/account.json").read_text())["created"], True) + + def test_adoption_refuses_root_ids_and_extra_groups(self): + service = str(installer.SERVICE_HOME) + account = SimpleNamespace(pw_name="parsar-node", pw_uid=990, pw_gid=990, pw_dir=service, pw_shell="/usr/sbin/nologin") + with mock.patch.object(installer.os, "getgrouplist", return_value=[990]): + self.assertTrue(installer.ours(account)) + self.assertFalse(installer.ours(SimpleNamespace(**dict(vars(account), pw_uid=0)))) + self.assertFalse(installer.ours(SimpleNamespace(**dict(vars(account), pw_gid=0)))) + with mock.patch.object(installer.os, "getgrouplist", return_value=[990, 27]): + self.assertFalse(installer.ours(account)) + + def service_step(self, function, output=None, errors=None): + """Run function through the real fork of as_service_user, as this test's user.""" + account = SimpleNamespace(pw_name="parsar-node", pw_uid=os.getuid(), pw_gid=os.getgid(), + pw_dir=str(self.home), pw_shell="/usr/sbin/nologin") + with mock.patch.object(installer.os, "setgroups"), mock.patch.object(installer.os, "setgid"), \ + mock.patch.object(installer.os, "setuid"), mock.patch.object(installer.sys, "stdout", output or io.StringIO()), \ + mock.patch.object(installer.sys, "stderr", errors or io.StringIO()): + installer.as_service_user(account, function) + + def test_service_user_step_has_no_terminal_and_reads_nothing(self): + """The forked child starts its own session with /dev/null as input; its output is relayed.""" + def probe(): + try: + open("/dev/tty").close() + tty = "opened" + except OSError: + tty = "unavailable" + print("session-leader=%s stdin=%s docker-config=%s tty=%s" % (os.getsid(0) == os.getpid(), + os.readlink("/proc/self/fd/0"), os.environ["DOCKER_CONFIG"], tty)) + output = io.StringIO() + self.service_step(probe, output) + self.assertIn("session-leader=True stdin=/dev/null docker-config=" + str(installer.CHILD_DOCKER_CONFIG) + " tty=unavailable", + output.getvalue()) + + def test_service_user_output_reaches_the_terminal_as_plain_text(self): + def forge(): + os.write(1, b"\x1b]52;c;ZWNobyBoaQ==\x07copied\r\x1b[2KFinish with: sudo sh\n") + os.write(1, "caf\u00e9".encode()[:-1]) # A character split across two reads. + time.sleep(0.2) + os.write(1, "caf\u00e9".encode()[-1:] + b"\n") + os.write(2, b"\x1b]0;title\x07\xc2\x9bwarning\n") + output, errors = io.StringIO(), io.StringIO() + self.service_step(forge, output, errors) + self.assertEqual(output.getvalue(), "?]52;c;ZWNobyBoaQ==?copied??[2KFinish with: sudo sh\ncaf\u00e9\n") + self.assertEqual(errors.getvalue(), "?]0;title??warning\n") + + def test_uninstall_prints_only_an_image_id_from_the_service_home(self): + root = self.home / "node" + root.mkdir() + (root / "provider.json").write_text(json.dumps({"docker": {"image": "x\nFinish with: sudo sh"}})) + output = io.StringIO() + with mock.patch.object(installer.sys, "stdout", output): + installer.remove_node_files(root, self.args.installation_id) + self.assertNotIn("Finish", output.getvalue()) + + def test_service_user_step_gets_its_own_session_keyring(self): + libc = installer.ctypes.CDLL(None, use_errno=True) + libc.syscall.restype = installer.ctypes.c_long + + def session_keyring(): # KEYCTL_GET_KEYRING_ID of KEY_SPEC_SESSION_KEYRING + return libc.syscall(*(installer.ctypes.c_long(value) for value in (installer.KEYCTL_SYSCALL, 0, -3, 0))) + before = session_keyring() + if before == -1: + self.skipTest("keyctl is unavailable here") + output = io.StringIO() + self.service_step(lambda: print("keyring=%d" % session_keyring()), output) + inside = int(output.getvalue().split("keyring=")[1]) + self.assertGreater(inside, 0) + self.assertNotEqual(inside, before) + + def test_closing_the_terminal_stops_the_step_and_everything_it_started(self): + """The child's own session misses SIGHUP; the installer stops it and what it started, and frees the lock.""" + record = self.home / "pids" + # As in a terminal session; a runner under nohup would otherwise ignore SIGHUP. + self.addCleanup(signal.signal, signal.SIGHUP, signal.signal(signal.SIGHUP, signal.SIG_DFL)) + + def long_step(): + # A program that ignores SIGTERM: only the SIGKILL to the child's process group ends it. + stubborn = subprocess.Popen([sys.executable, "-c", "import signal, time; signal.signal(signal.SIGTERM, " + "signal.SIG_IGN); print(flush=True); time.sleep(60)"], stdout=subprocess.PIPE) + stubborn.stdout.readline() + record.write_text("%d %d" % (os.getpid(), stubborn.pid)) + os.kill(os.getppid(), signal.SIGHUP) # The administrator's terminal closes. + time.sleep(30) + started = time.monotonic() + with mock.patch.object(installer, "SYSTEM_LOCKS", self.home), mock.patch.object(installer.os, "geteuid", return_value=0), \ + self.assertRaisesRegex(installer.InstallError, "interrupted"): + with installer.host_lock(): + self.service_step(long_step) + self.assertLess(time.monotonic() - started, 15) + for pid in map(int, record.read_text().split()): + self.assertTrue(ended(pid), pid) + with open(self.home / "parsar-node.lock") as lock: # The child held it too; nothing does now. + fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB) + + def test_an_ignored_hangup_stays_ignored(self): + """Under nohup a closed terminal ends neither the installer nor the step.""" + self.addCleanup(signal.signal, signal.SIGHUP, signal.signal(signal.SIGHUP, signal.SIG_IGN)) + + def step(): + os.kill(os.getppid(), signal.SIGHUP) + time.sleep(0.2) + print("finished") + output = io.StringIO() + self.service_step(step, output) + self.assertEqual(output.getvalue(), "finished\n") + self.assertIs(signal.getsignal(signal.SIGHUP), signal.SIG_IGN) + + def test_uninstall_never_follows_a_link_in_the_service_home(self): + # The service user owns its home; a link it plants must not steer deletion elsewhere. + system = self.sudo_host() + installer.install_system(self.args, "synthetic-once-token") + victim = self.home / "victim" + (self.home / "service/.parsar").rename(victim) + (self.home / "service/.parsar").symlink_to(victim) + with mock.patch.object(installer, "open_request", side_effect=urllib.error.HTTPError("https://core.example", 401, "", {}, None)), \ + self.assertRaisesRegex(installer.InstallError, "symbolic link.*Nothing was changed"): + installer.uninstall_system(SimpleNamespace(installation_id=self.args.installation_id, force=True)) + self.assertTrue((victim / "nodes" / self.args.installation_id / "registered.json").exists()) + self.assertTrue((system / "units" / ("parsar-node-" + self.args.installation_id + ".service")).exists()) + + def test_token_comes_on_standard_input_only(self): + arguments = ["--source-url", self.args.source_url, "--core-url", self.args.core_url, "--installation-id", + self.args.installation_id, "--enrollment-token-stdin"] + with mock.patch.object(installer, "install") as install, mock.patch.object(installer.sys, "stdin", io.StringIO("synthetic-once-token\n")), \ + mock.patch.object(installer.os, "geteuid", return_value=1000): + installer.main(arguments) + self.assertEqual(install.call_args.args[1], "synthetic-once-token") + with mock.patch.dict(os.environ, {"PARSAR_NODE_ENROLLMENT_TOKEN": "other"}), self.assertRaises(SystemExit): + installer.main(arguments) + # As root, a token in the environment would come from `sudo VAR=...`, which sudo logs. + with mock.patch.dict(os.environ, {"PARSAR_NODE_ENROLLMENT_TOKEN": "other"}), mock.patch.object(installer.os, "geteuid", return_value=0), \ + mock.patch.object(installer, "install_system") as install_system, self.assertRaises(SystemExit): + installer.main(arguments[:-1]) + install_system.assert_not_called() + + def test_user_manager_bus_is_found_without_a_login_session(self): + runtime = self.home / "run-user" + (runtime / str(os.getuid())).mkdir(parents=True) + with mock.patch.object(installer, "USER_RUNTIME", runtime), mock.patch.dict(os.environ, {}, clear=True): + with self.assertRaisesRegex(installer.InstallError, "enable-linger"): + installer.user_bus() + bus = socket.socket(socket.AF_UNIX) + self.addCleanup(bus.close) + bus.bind(str(runtime / str(os.getuid()) / "bus")) + installer.user_bus() + self.assertEqual(os.environ["DBUS_SESSION_BUS_ADDRESS"], "unix:path=" + str(runtime / str(os.getuid()) / "bus")) + def test_offline_bundle_uses_same_bootstrap_and_verified_artifacts(self): bundle = self.home / "bundle" bundle.mkdir() diff --git a/docs/getting-started/install.md b/docs/getting-started/install.md index d890f303c..9bd0763b1 100644 --- a/docs/getting-started/install.md +++ b/docs/getting-started/install.md @@ -60,8 +60,9 @@ no Go, Node, Rust or source checkout. The manifest identifies every asset by immutable revision, SHA-256 and byte size. Adding a node downloads only its selected provider's assets. Runtime images use compressed archives; verified local files and already imported images are reused. -Downloads use temporary files and bounded retries, so a truncated response is -never promoted into the cache. An optional `-offline.tar.gz` bundle contains the +Downloads use private partial files and bounded retries: an interrupted download, +or a rerun after one, asks the console only for the missing bytes, and a file is used +only after its size and SHA-256 match the manifest. An optional `-offline.tar.gz` bundle contains the same assets locally. Nodes download these assets only from the console (Web) that generated their @@ -149,21 +150,28 @@ Choose English or Chinese through the System language selector. specification; node files cannot override it. Later changes require global maintenance and completed cleanup. Microsandbox uses a five-minute idle timeout and one-day snapshot retention. -4. For own-machine hosting, click **Add node**, copy the command, and run it as a non-root user on the - target Linux amd64 host. It downloads the matched bootstrap and execution assets from your console +4. For own-machine hosting, click **Add node**, copy the command, and run it on the target Linux amd64 + host with sudo, or as root. It prepares the host itself: the `parsar-node` service user, its `docker` + or `kvm` group and a system service. For Docker nodes, the `docker` group makes that user, and so the + node, root-equivalent on the host; use hosts dedicated to sandboxes. Hosts without sudo use the no-sudo + command as a prepared user instead. + It downloads the matched bootstrap and execution assets from your console (never from a release URL), verifies checksums, reads the saved generation and specification without consuming enrollment, verifies the payload matches that Runtime, imports the image only when missing, and writes the matching provider configuration, - registers the node and starts a systemd user service. The service keeps retrying while Core + registers the node and starts its service. The service keeps retrying while Core is unreachable and stops for good once the node is removed. The installer waits for Core to confirm connection and provider readiness. Web refreshes node health automatically. Wait for the node to be online and its provider to be ready; registration alone does not mean it can accept work. A registered retry uses its retained node credential and refuses changed resource or Runtime settings. -The target host needs curl, sha256sum, Python 3.9+, a systemd user session with lingering enabled, -and either Docker socket access or microsandbox's KVM/native-library prerequisites. +The target host needs curl, sha256sum, Python 3.9+ and systemd, plus Docker Engine or +KVM with microsandbox's native libraries; the command never installs them and names +what is missing. The no-sudo command also needs a user with lingering and Docker or KVM +access. See [Register a host](https://github.com/MiniMax-AI/parsar-core/blob/main/services/agents-api/HOSTED-SANDBOX-MANAGER.md#register-a-host) +for both modes and for removing a node from its host. The command checks host access before downloading the Runtime and verifies microsandbox's shared libraries after downloading its native programs. The console serves only fixed, non-secret distribution files at `/node-install/`; @@ -304,8 +312,9 @@ paths belong to the separate node service; the Core container receives no Docker socket or node state mount. Web receives neither provider authority nor node secrets. Nodes keep configuration and identity under -`~/.parsar/nodes//`. Microsandbox stores its private Runtime home -under `~/.parsar/m//`. Preserve these directories and +`~/.parsar/nodes//` in the home of the account that runs them +(`/var/lib/parsar-node` for a node added with sudo). Microsandbox stores its private +Runtime home under `~/.parsar/m//` in the same home. Preserve these directories and backend storage across restarts. A missing identity is a recovery incident, not permission to register over existing resources. diff --git a/docs/getting-started/operations.md b/docs/getting-started/operations.md index d4b814718..deba79b67 100644 --- a/docs/getting-started/operations.md +++ b/docs/getting-started/operations.md @@ -195,8 +195,9 @@ Retain together: `core.key` is the [Core key](#core-key), including on zero-node installations; - the database-owned deployment specification; - each separately installed node's private configuration and persistent identity - directory on its host (`~/.parsar/nodes//` for the Web-generated - installer), as described in the + directory on its host (`/var/lib/parsar-node/.parsar/nodes//` for a + node added with sudo, `~/.parsar/nodes//` for one added as a normal + user), as described in the [node guide](../../services/agents-api/HOSTED-SANDBOX-MANAGER.md#register-a-host); - E2B's private `state/e2b` receipts and SDK connection materials when selected; - microsandbox's private state/cache/disks/snapshots, or Docker-owned Runtime diff --git a/services/agents-api/HOSTED-SANDBOX-MANAGER.md b/services/agents-api/HOSTED-SANDBOX-MANAGER.md index 3af790337..1c3901876 100644 --- a/services/agents-api/HOSTED-SANDBOX-MANAGER.md +++ b/services/agents-api/HOSTED-SANDBOX-MANAGER.md @@ -92,7 +92,8 @@ server configuration and cannot select a different Runtime or resource profile. Closing the dialog discards its one-time command. An expired command requires explicit regeneration; failed or uncertain writes are never retried automatically. The installer checks prerequisites, downloads the matched payload, checks its hashes, prepares provider configuration and starts the existing node -program as a systemd user service. Web polls readiness and capacity while waiting. +program as a service (see [Register a host](#register-a-host)). Web polls readiness +and capacity while waiting. It does not install software through SSH. Registration itself does not create a Session, sandbox or model request. Hosted Session admission fails until setup is complete and a ready node has capacity. E2B allocates directly without this node requirement. @@ -128,8 +129,9 @@ contains limits; its top-level `resources` contains cleanup counts. The Core installer never enrolls its own host. The Core host joins like any other host, through **Add node**, with the same database selection and registration checks. Node identity and configuration live under -`~/.parsar/nodes//`; preserve that private directory and its -backend storage together. Core has no embedded local provider configuration or node +`~/.parsar/nodes//` in the home of the account that runs the node +(`/var/lib/parsar-node` for a node added with sudo); preserve that private directory +and its backend storage together. Core has no embedded local provider configuration or node identity mount. ## Older file-managed installations @@ -157,9 +159,55 @@ in the bundle's `artifacts/` directory first; `/console/config` lists the provid whose assets it holds (`node_artifacts`). It reuses verified cache entries and exact imported images, then waits for Core to confirm connection and provider readiness. The enrollment token -is transient and never a console/project credential. Python 3.9+, a systemd user -session with lingering, and Docker access or KVM/native-library prerequisites -must already exist on the target host. Rerunning the same command preserves the +is transient and never a console/project credential; the command passes it to the +installer on standard input (`--enrollment-token-stdin`), never in a process argument, +an environment variable or sudo's log. A download that brings less than 64 KiB in a +minute stops; the downloaded part is kept, and running a new command resumes it. + +The installer chooses how the node runs from the user that runs it: + +- **With sudo or as root (the default command).** It prepares the host itself. It + creates the system user `parsar-node` (home `/var/lib/parsar-node`) if missing, + or adopts an existing one with that home and a nologin shell. It adds the user to + the `docker` group (Docker) or the `kvm` group (microsandbox), whichever owns the + device; any other device group is refused. The node runs as the root-owned system + service `/etc/systemd/system/parsar-node-.service` with + `User=parsar-node`, enabled for boot. No lingering or login session is needed; + logs are in `sudo journalctl -u parsar-node-.service`. Node state + lives in `/var/lib/parsar-node/.parsar/nodes//`, and what the + installer created or changed is recorded in `/etc/parsar-node/`. Files the service + user owns are written and deleted only with that user's credentials; root handles + the account, the group, the unit and the Docker network. It never installs + Docker, KVM or packages, never changes device permissions and refuses + SELinux-enforcing hosts; a missing prerequisite stops it with a one-line hint + before anything changes. A foreign account named `parsar-node` is refused. Sudo + mode serves **one Core per host**: every sudo-mode node shares `parsar-node`, so a + node for a second Core is refused. A node for the same installation installed + without sudo is found in the invoking user's home or, for Docker, by its network + on the same engine; other users' homes are not searched. The steps that run as + `parsar-node` start in their own session with no terminal, so nothing they run + can reach the administrator's terminal, and their output appears only as plain + text. Interrupting the installer or closing its terminal stops those steps as + well. + + **Docker mode is root-equivalent.** Membership in the `docker` group lets + `parsar-node`, and so anything that controls the node, act as root on that host. + This is inherent to running sandboxes on Docker and equally true in the no-sudo + mode. Add Docker nodes only on hosts dedicated to running sandboxes. Microsandbox + nodes need only the `kvm` group. + + Pass the token on standard input, as the command does. In sudo mode the installer + refuses `PARSAR_NODE_ENROLLMENT_TOKEN`, because `sudo VAR=… python3` records the + variable in sudo's log. A sudoers policy with `log_input` records standard input + as well; the token is single-use and expires after ten minutes. +- **As a normal user (for hosts without sudo).** The node runs as that user's + systemd user service with state under `~/.parsar/nodes//`. The + user needs lingering and Docker or KVM access, which an administrator grants + beforehand. The installer finds the user's systemd manager even from `su` or + `sudo -iu`, where no login session sets `XDG_RUNTIME_DIR`. + +Python 3.9+, curl and sha256sum, and Docker Engine or KVM with microsandbox's +native libraries must already exist on the target host. Rerunning the same command preserves the node's private identity. A registered retry reads configuration with its retained node credential and `X-Parsar-Node-ID`; it does not enroll again. Changes to the Core origin, installation, generation, specification or release are refused @@ -273,6 +321,30 @@ unknown operations or cleanup records cannot be removed. Resolve those resources through their normal lifecycle and then retry; the API reports the conflict. Offline resources remain owned and visible. Explicit removal permanently retires the node identity; adding that host again requires a fresh private state directory. +After removal, clean up the host with the installer's `--uninstall`, using the same +checked download as the add command: + +```sh + (umask 077; d=$(mktemp -d) || exit; trap 'rm -rf "$d"' EXIT; s=; [ "$(id -u)" -eq 0 ] || s=sudo +curl -fsS --max-time 30 --max-filesize 1048576 'https://core.example/node-install/node-install.pyz' -o "$d/node-install.pyz" && +printf '%s %s\n' '' "$d/node-install.pyz" | sha256sum -c --status && +$s python3 "$d/node-install.pyz" --uninstall --installation-id '') +``` + +For a node installed as a normal user, run it as that user; it does not use sudo +then. The installer's checksum is `node_installer_sha256` in the console's +`/console/config`. Uninstall proceeds only when Core answers 401 to the node's +credential, or with `--force` for a Core that no longer exists. It stops and removes +the service, the node state and the Docker network. The `parsar-node` account is +deleted only when the installer created it and no node remains. An adopted account +and its home directory stay; uninstall removes only the groups the installer added. Uninstall never removes +sandboxes, volumes or images: it keeps the Runtime image and a microsandbox node's +store (`/var/lib/parsar-node/.parsar/m/`, its images and any sandbox state), +prints how to remove them (`sudo -u parsar-node rm -rf `), and keeps a +created account until that store is gone. +With `--force`, microVMs that `KillMode=process` left running may still use the +store, so check `pgrep -u parsar-node` first. The host can then be added again with +a new command. The node program then exits with status 78 when Core answers 401 to its credential, and the installed service does not restart it. Other failures, including an unreachable Core or a 403 from a proxy in front of it, restart the service every diff --git a/services/agents-api/deploy/microsandbox/README.md b/services/agents-api/deploy/microsandbox/README.md index d12cf58eb..3418015e3 100644 --- a/services/agents-api/deploy/microsandbox/README.md +++ b/services/agents-api/deploy/microsandbox/README.md @@ -81,7 +81,8 @@ it does not install or upgrade them. For a manual node installation, create a private, short runtime state path, for example `~/.parsar/msb`, with mode 0700. The ordinary installer instead selects `~/.parsar/m//` and stores node identity/configuration -under `~/.parsar/nodes//`. Keep these on persistent local storage +under `~/.parsar/nodes//`, both in the home of the account that runs +the node (`/var/lib/parsar-node` for a node added with sudo). Keep these on persistent local storage reserved for this installation. Unix socket path limits apply. Runtime storage contains confidential disks, memory snapshots and SDK state; preserve it with the node identity and Core database when recovering the host.