diff --git a/contracts/agents-api/node-generation-protocol.md b/contracts/agents-api/node-generation-protocol.md index a7b293f4..1e9cf97f 100644 --- a/contracts/agents-api/node-generation-protocol.md +++ b/contracts/agents-api/node-generation-protocol.md @@ -10,7 +10,7 @@ Every frame is one JSON text message whose `version` equals `node.ProtocolVersio ## Connection -1. The node dials `/api/v1/sandbox-node/connect?node_id=` on its stored Core origin (`wss` for `https`) with its node credential as a Bearer header. Core answers 401 to a rejected credential, which the node treats as permanent; any other failure, including a 403 from a proxy, is retried with bounded backoff. Core refuses a second connection for a node identity while one is opening, live or closing, with 409. +1. The node dials `/api/v1/sandbox-node/connect?node_id=` on its stored Core origin (`wss` for `https`, or `ws` for a non-loopback `http` origin admitted by the retained `allow_insecure_origin` policy) with its node credential as a Bearer header. Core answers 401 to a rejected credential, which the node treats as permanent; any other failure, including a 403 from a proxy, is retried with bounded backoff. Core refuses a second connection for a node identity while one is opening, live or closing, with 409. 2. Within 15 seconds the node sends `hello` with its identity (`node_id`, `installation_id`, `provider`, `backend_fingerprint`, the enrolled `deployment_generation` and `specification_digest`, `max_active`, `max_retained`), its first health report and, when it can prepare and retain several deployment generations, `generation_management: true`. Core closes the connection unless the identity matches the authenticated node. 3. Core records the node's presence, then replies `welcome` with a new `connection_id`, the current `owner_epoch` and, for a generation-managing node, `deployment`: the target `generation`, its `specification_digest` and a nullable `serving_generation`. The node stores a higher owner epoch and refuses a lower one. 4. Every 10 seconds the node sends `heartbeat` with `connection_id`, `owner_epoch` and health. On each heartbeat Core authenticates the node credential again and checks the owner epoch, records the health and replies `heartbeat_ack`, with `deployment` for a generation-managing node. Either peer closes the connection after 35 seconds without a frame. @@ -91,7 +91,7 @@ A dropped generation can never be prepared or used again. A helper's exit is evi ## Matched fresh installation -The host program release and Core's selected Runtime release are independent. A fresh node gets its executable and private preparer from the console's current release, and reads the exact Runtime source, image identities and native runtime and firmware digests from its authenticated configuration. When that Runtime is older, the console still serves its immutable `releases//` manifest, checksums and allowlisted artifacts: the Runtime helper, firmware, seccomp profile and image bytes come from the selected release, and artifact URLs stay pinned to their verified manifest even if the console's current release changes during a download. +The host program release and Core's selected Runtime release are independent. A fresh node gets its executable and private preparer from the console's current release, and reads the exact Runtime source, image identities and native runtime and firmware digests from its authenticated configuration. Artifact transfers use the policy in the node's retained identity: HTTPS, or plaintext HTTP only from the enrolled console origin when that identity recorded `allow_insecure_origin`. When that Runtime is older, the console still serves its immutable `releases//` manifest, checksums and allowlisted artifacts: the Runtime helper, firmware, seccomp profile and image bytes come from the selected release, and artifact URLs stay pinned to their verified manifest even if the console's current release changes during a download. A missing retained release refuses the installation rather than substituting the current Runtime, and so does a local bundle that holds only a different Runtime. These refusals happen before the installer writes the node identity, imports the Runtime, registers the node or starts its service. diff --git a/contracts/agents-api/zh/node-generation-protocol.md b/contracts/agents-api/zh/node-generation-protocol.md index e6e5de8f..99987be1 100644 --- a/contracts/agents-api/zh/node-generation-protocol.md +++ b/contracts/agents-api/zh/node-generation-protocol.md @@ -1,7 +1,7 @@ --- title: "沙箱节点协议" source: contracts/agents-api/node-generation-protocol.md -source_hash: 1ee43dfcdd0eec0806ea3bc8a4c1227e10bd8ac5e69486505cb113a98e3f548a +source_hash: b67076a5ea3b9e13c18bc444d21f645e7af6c86c8e104ae0b6653433081c4b7b --- 沙箱节点在其主机上运行 Docker 或 microsandbox Provider,并通过一个 WebSocket 与 Core 相连。Core 通过该连接发送 Provider 操作;节点针对本地 Provider 执行这些操作,并报告就绪状态、主机测量值及其持有的部署代次。Core 始终是唯一的生命周期所有者:节点绝不重试变更操作或调度工作。帧和校验器位于 [`services/core/internal/sandbox/node`](https://github.com/MiniMax-AI/OpenAgentCore/tree/main/services/core/internal/sandbox/node)(`wire.go`、`generation_wire.go`);节点用于注册和读取配置的 HTTP 路由位于[机器连接 API](machine-api.md#node-routes)。 @@ -12,7 +12,7 @@ source_hash: 1ee43dfcdd0eec0806ea3bc8a4c1227e10bd8ac5e69486505cb113a98e3f548a ## 连接 {#connection} -1. 节点在其存储的 Core 源地址上发起对 `/api/v1/sandbox-node/connect?node_id=` 的连接(对于 `https` 使用 `wss`),并以 Bearer 请求头发送节点凭据。Core 对被拒绝的凭据返回 401,节点将其视为永久性拒绝;其他任何失败(包括代理返回的 403)都会使用有界退避进行重试。当某个节点身份已有一个连接正在建立、存活或关闭时,Core 会以 409 拒绝第二个连接。 +1. 节点在其存储的 Core 源地址上发起对 `/api/v1/sandbox-node/connect?node_id=` 的连接(对于 `https` 使用 `wss`;保留的 `allow_insecure_origin` 策略允许时,非回环 `http` 源地址使用 `ws`),并以 Bearer 请求头发送节点凭据。Core 对被拒绝的凭据返回 401,节点将其视为永久性拒绝;其他任何失败(包括代理返回的 403)都会使用有界退避进行重试。当某个节点身份已有一个连接正在建立、存活或关闭时,Core 会以 409 拒绝第二个连接。 2. 节点须在 15 秒内发送 `hello`,其中包含节点身份(`node_id`、`installation_id`、`provider`、`backend_fingerprint`、已登记的 `deployment_generation` 和 `specification_digest`、`max_active`、`max_retained`)、首次健康报告;如果节点能够准备并保留多个部署代次,还包含 `generation_management: true`。除非身份与已认证节点匹配,否则 Core 会关闭连接。 3. Core 记录节点的存在状态,然后回复 `welcome`,其中包含新的 `connection_id` 和当前 `owner_epoch`;对于支持代次管理的节点,还包含 `deployment`:目标 `generation`、其 `specification_digest` 和可空的 `serving_generation`。节点保存更高的所有者 epoch,并拒绝更低的值。 4. 节点每 10 秒发送一次 `heartbeat`,其中包含 `connection_id`、`owner_epoch` 和健康状态。对于每次心跳,Core 都会再次认证节点凭据并检查所有者 epoch,记录健康状态并回复 `heartbeat_ack`;对于支持代次管理的节点,回复中还包含 `deployment`。任一端连续 35 秒未收到任何帧时都会关闭连接。 @@ -93,7 +93,7 @@ Core 的丢弃授权是回收的必要条件,但并非充分条件。排队和 ## 完全匹配的新安装 {#matched-fresh-installation} -主机程序版本与 Core 选择的 Runtime 版本彼此独立。全新节点从控制台当前版本获取其可执行文件和私有准备器,并从经认证的配置中读取确切的 Runtime 源、镜像身份以及原生运行时和固件摘要。当该 Runtime 较旧时,控制台仍会提供其不可变的 `releases//` 清单、校验和以及允许列表中的制品:Runtime 辅助程序、固件、seccomp 配置文件和镜像字节都来自所选版本;即使下载期间控制台的当前版本发生变化,制品 URL 仍固定到已验证的清单。 +主机程序版本与 Core 选择的 Runtime 版本彼此独立。全新节点从控制台当前版本获取其可执行文件和私有准备器,并从经认证的配置中读取确切的 Runtime 源、镜像身份以及原生运行时和固件摘要。制品传输使用节点保留身份中的策略:HTTPS,或在该身份记录了 `allow_insecure_origin` 时仅从已登记的明文控制台源地址使用明文 HTTP。当该 Runtime 较旧时,控制台仍会提供其不可变的 `releases//` 清单、校验和以及允许列表中的制品:Runtime 辅助程序、固件、seccomp 配置文件和镜像字节都来自所选版本;即使下载期间控制台的当前版本发生变化,制品 URL 仍固定到已验证的清单。 所需的保留版本缺失时,安装会被拒绝,而不会替换为当前 Runtime;仅包含另一个 Runtime 的本地捆绑包也会导致拒绝。所有这些拒绝都发生在安装器写入节点身份、导入 Runtime、注册节点或启动服务之前。 diff --git a/deploy/node/README.md b/deploy/node/README.md index 0cd68fb9..30955717 100644 --- a/deploy/node/README.md +++ b/deploy/node/README.md @@ -32,6 +32,7 @@ The distribution manifest is the one download contract for nodes: flat versioned - A Compose installation keeps only the node metadata from its release archive; Core's image never acquires execution-only payloads. - A node obtains bootstrap metadata from the console that generated its command. Web serves artifacts it has locally and redirects missing declared execution artifacts to the versioned HTTPS release base in the verified manifest. Web never downloads or caches those bytes. - Only artifact requests may follow HTTPS redirects, and only without credentials or cookies. Metadata and enrollment requests stay on the configured console. The console publishes only fixed non-secret files and declared artifact names. +- A node whose retained identity recorded `allow_insecure_origin` may transfer artifacts over plaintext HTTP, but only from the console origin it enrolled with: an artifact redirect may stay on that same plaintext origin or upgrade to HTTPS, never downgrade an HTTPS transfer and never move to another plaintext host. Metadata and enrollment requests still stay on the configured console, and TLS certificate verification is never relaxed. - Download into private temporary files, verify size and SHA-256 before an atomic rename, resume interrupted transfers, and reuse only verified cache entries or exact image identities. Never select a release other than the pinned one. - Release downloads are anonymous. Never add repository credentials to installed node or Runtime configuration. - Manual builds use the `build-` release tag and tag builds the `v*` tag. The manifest's download base must match the release tag; artifact file names and source provenance keep the full source SHA. diff --git a/deploy/node/distribution.py b/deploy/node/distribution.py index c31fcbe9..36fecdc5 100644 --- a/deploy/node/distribution.py +++ b/deploy/node/distribution.py @@ -96,7 +96,21 @@ def artifact(manifest, name): return entry -def safe_url(value): +def origin_of(value): + """Return the normalized (scheme, host, port) origin, or None without a usable host.""" + parsed = urlsplit(value) + try: + port = parsed.port + except ValueError: + return None + if not parsed.hostname: + return None + if port is None: + port = {'http': 80, 'https': 443}.get(parsed.scheme) + return (parsed.scheme, parsed.hostname, port) + + +def safe_url(value, allow_insecure_origin=False, source_origin=None): try: parsed = urlsplit(value) parsed.port @@ -108,7 +122,13 @@ def safe_url(value): pass if (not parsed.hostname or parsed.username is not None or parsed.password is not None or parsed.fragment or any(c.isspace() for c in value) - or '\\' in value or parsed.scheme != 'https' and not (parsed.scheme == 'http' and loopback)): + or '\\' in value): + raise ValueError() + # Plaintext is admitted only for loopback testing or, behind the opt-in + # switch, for the configured console origin itself; every other scheme + # and every cross-host plaintext URL keeps the original refusal. + if parsed.scheme != 'https' and not (parsed.scheme == 'http' and ( + loopback or (allow_insecure_origin and source_origin is not None and origin_of(value) == source_origin))): raise ValueError() except ValueError: raise ArtifactError('Artifact downloads require HTTPS; loopback HTTP is only for local testing') from None @@ -117,9 +137,19 @@ def safe_url(value): class ArtifactRedirect(urllib.request.HTTPRedirectHandler): """Only artifact bytes may follow HTTPS redirects; metadata stays on Core.""" + def __init__(self, allow_insecure_origin=False, source_origin=None): + super().__init__() + self.allow_insecure_origin = allow_insecure_origin + self.source_origin = source_origin + def redirect_request(self, request, fp, code, msg, headers, newurl): - safe_url(newurl) - if urlsplit(newurl).scheme != 'https' or request.get_method() not in ('GET', 'HEAD'): + safe_url(newurl, self.allow_insecure_origin, self.source_origin) + # A plaintext hop is only a same-origin resume of an http source; an HTTPS + # transfer never downgrades, and a plaintext hop never changes host. + if (urlsplit(newurl).scheme != 'https' and not (self.allow_insecure_origin + and urlsplit(newurl).scheme == 'http' and self.source_origin is not None + and origin_of(newurl) == self.source_origin and urlsplit(request.full_url).scheme != 'https') + or request.get_method() not in ('GET', 'HEAD')): raise ArtifactError('Artifact redirects require HTTPS') # Carry resume headers, never credentials or cookies, to a release/CDN host. forwarded = {name: value for name, value in request.header_items() @@ -149,7 +179,7 @@ def matches(path, entry): return path.stat().st_size == entry['size'] and digest(path) == entry['sha256'] -def obtain_artifact(manifest, logical_path, destination, offline_root=None): +def obtain_artifact(manifest, logical_path, destination, offline_root=None, allow_insecure_origin=False, source_url=None): entry = artifact(manifest, logical_path) target = checked_path(destination) if target.exists(): @@ -168,13 +198,14 @@ def obtain_artifact(manifest, logical_path, destination, offline_root=None): base = manifest.get('artifact_base_url', '') if not isinstance(base, str) or not base or urlsplit(base).query: raise ArtifactError('No downloadable artifact source; use the matching offline bundle') - url = safe_url(base.rstrip('/') + '/' + entry['filename']) + source_origin = origin_of(source_url) if (allow_insecure_origin and source_url) else None + url = safe_url(base.rstrip('/') + '/' + entry['filename'], allow_insecure_origin, source_origin) # A private partial file survives interruptions and reruns; the next attempt asks # for the missing bytes only. The complete file is still verified as a whole. partial = target.with_name('.' + target.name + '.partial') for attempt in range(3): try: - download_partial(url, partial, entry, logical_path) + download_partial(url, partial, entry, logical_path, allow_insecure_origin, source_origin) break except urllib.error.HTTPError as error: if error.code == 416: @@ -230,7 +261,7 @@ def copy_artifact(source, target, entry, logical_path): SLOW_SECONDS, SLOW_BYTES = 60, 64 * 1024 -def download_partial(url, partial, entry, logical_path): +def download_partial(url, partial, entry, logical_path, allow_insecure_origin=False, source_origin=None): """Complete the partial file, asking only for the bytes it is missing.""" size = entry['size'] offset = 0 @@ -254,7 +285,7 @@ def download_partial(url, partial, entry, logical_path): if validator: headers['If-Range'] = validator request = urllib.request.Request(url, headers=headers) - with urllib.request.build_opener(ArtifactRedirect()).open(request, timeout=30) as stream: + with urllib.request.build_opener(ArtifactRedirect(allow_insecure_origin, source_origin)).open(request, timeout=30) as stream: if offset and (stream.status != 206 or not stream.headers.get('Content-Range', '').startswith(f'bytes {offset}-')): offset = 0 # The server sent the whole file; start over. if not offset: @@ -297,7 +328,7 @@ def download_partial(url, partial, entry, logical_path): raise http.client.IncompleteRead(b'', size - count) -def runtime_archive(manifest, cache_root, offline_root=None): +def runtime_archive(manifest, cache_root, offline_root=None, allow_insecure_origin=False, source_url=None): entry = artifact(manifest, 'images/runtime.tar.gz') expanded = {'sha256': entry.get('unpacked_sha256'), 'size': entry.get('unpacked_size')} if (not re.fullmatch(r'[0-9a-f]{64}', str(expanded['sha256'])) @@ -309,7 +340,8 @@ def runtime_archive(manifest, cache_root, offline_root=None): if not matches(target, expanded): raise ArtifactError('Cached Runtime archive differs; preserve state and inspect it') return target - archive = obtain_artifact(manifest, 'images/runtime.tar.gz', root / 'images/runtime.tar.gz', offline_root) + archive = obtain_artifact(manifest, 'images/runtime.tar.gz', root / 'images/runtime.tar.gz', offline_root, + allow_insecure_origin, source_url) fd, temporary = tempfile.mkstemp(prefix='.runtime-', dir=target.parent) try: with os.fdopen(fd, 'wb') as output, gzip.open(archive, 'rb') as stream: @@ -330,8 +362,10 @@ def runtime_archive(manifest, cache_root, offline_root=None): return target -def load_manifest(source_url=None, offline_root=None): +def load_manifest(source_url=None, offline_root=None, allow_insecure_origin=False): """Read the matched public manifest without transmitting installation credentials.""" + source_origin = origin_of(source_url) if (allow_insecure_origin and source_url) else None + def read(name): if offline_root is not None: with (Path(offline_root) / name).open('rb') as stream: @@ -339,7 +373,7 @@ def read(name): else: if not source_url: raise DistributionError('A Core source URL or offline bundle is required') - url = safe_url(source_url.rstrip('/') + '/node-install/' + name) + url = safe_url(source_url.rstrip('/') + '/node-install/' + name, allow_insecure_origin, source_origin) for attempt in range(3): try: with urllib.request.build_opener(NoRedirect()).open(url, timeout=30) as stream: diff --git a/deploy/node/node_generations.py b/deploy/node/node_generations.py index 294de99d..52ca2997 100644 --- a/deploy/node/node_generations.py +++ b/deploy/node/node_generations.py @@ -353,7 +353,8 @@ def runtime_files(root, value, args, manifest, sums, installer): if name == "runtime/seccomp.json": installer.download(args.source_url, name, release, sums[name], prefix="releases/" + source + "/") else: - installer.distribution.obtain_artifact(manifest, name, release / name) + installer.distribution.obtain_artifact(manifest, name, release / name, None, + getattr(args, "allow_insecure_origin", False), args.source_url) os.chmod(release / name, 0o700) atomic_json(release / "manifest.json", manifest) return release @@ -361,6 +362,9 @@ def runtime_files(root, value, args, manifest, sums, installer): def prepare(args, installer): root, identity = owned_root(args, installer) + # The retained identity is the single home of the enrollment policy; project it + # onto this helper invocation so every download below uses the same decision. + args.allow_insecure_origin = bool(identity.get("allow_insecure_origin", False)) with installer.install_lock(root), collection_lease( root, args.generation, installer, marker_identity(args), initialize=args.generation not in retained_configs(root, installer) @@ -400,7 +404,7 @@ def prepare(args, installer): settings = installer.private_json(root / "preparation.json") # The retained identity records the enrollment policy; a node enrolled # with allow_insecure_origin may keep an http source_url in preparation.json. - args.source_url = installer.origin(settings["source_url"], bool(identity.get("allow_insecure_origin", False))) + args.source_url = installer.origin(settings["source_url"], args.allow_insecure_origin) args.bundle = None manifest, sums = installer.metadata(args.source_url, prefix="releases/" + runtime["source_commit"] + "/") try: diff --git a/deploy/node/node_install.py b/deploy/node/node_install.py index ead70534..8afc4e52 100644 --- a/deploy/node/node_install.py +++ b/deploy/node/node_install.py @@ -69,6 +69,8 @@ class RuntimeDownloadError(InstallError): CHILD_DOCKER_CONFIG = Path("/run/oac-node-docker") LOCK_WAIT_SECONDS = 600 NOTHING_CHANGED = " Nothing was changed." +INSECURE_ORIGIN_WARNING = ("Warning: allow_insecure_origin is enabled; this node may download node artifacts and connect " + "to Core over plaintext HTTP. Use it only on a trusted development network.") def origin(value, allow_insecure_origin=False): @@ -315,7 +317,9 @@ def prepare_runtime(root, args, manifest): if args.provider == "docker": docker = ["docker", "--host", "unix:///var/run/docker.sock"] image = distribution.ensure_docker_image( - manifest, "runtime", lambda: distribution.runtime_archive(manifest, root, getattr(args, "bundle", None)), docker) + manifest, "runtime", lambda: distribution.runtime_archive( + manifest, root, getattr(args, "bundle", None), + getattr(args, "allow_insecure_origin", False), args.source_url), docker) network = "oac-node-" + args.installation_id networks = checked(docker + ["network", "ls", "--format", "{{.Name}}"], "Cannot inspect Docker networks").splitlines() if network not in networks: @@ -338,7 +342,8 @@ def matches(): except (InstallError, ValueError, AttributeError): return False if not matches(): - archive = distribution.runtime_archive(manifest, root, getattr(args, "bundle", None)) + archive = distribution.runtime_archive(manifest, root, getattr(args, "bundle", None), + getattr(args, "allow_insecure_origin", False), args.source_url) checked([str(root / MICRO[1]), "image", "load", "--input", str(archive), "--tag", manifest["runtime_ref"], "--quiet"], "Cannot import the microsandbox runtime image; check free disk space and host libraries", timeout=1800, env=env) if not matches(): @@ -417,7 +422,8 @@ def register_node(root, args, token, helper_archive=None): target = root / name safe_directory(target.parent) existing_file(target) - distribution.obtain_artifact(program_manifest if name in provider_assets.artifacts(args.provider, ("node",)) else manifest, name, target, getattr(args, "bundle", None)) + distribution.obtain_artifact(program_manifest if name in provider_assets.artifacts(args.provider, ("node",)) else manifest, name, target, + getattr(args, "bundle", None), args.allow_insecure_origin, args.source_url) os.chmod(target, 0o700) node_generations.install_helper(root, args, sys.modules[__name__], helper_archive) safe_directory(root / "state/node") @@ -1301,6 +1307,8 @@ def main(argv=None): setattr(args, name, origin(value, args.allow_insecure_origin)) except argparse.ArgumentTypeError as error: parser.error(str(error)) + if args.allow_insecure_origin: + print(INSECURE_ORIGIN_WARNING, file=sys.stderr) if str(uuid.UUID(args.installation_id)) != args.installation_id: raise InstallError("Installation ID must be a canonical UUID") if args.update: diff --git a/deploy/node/test_distribution.py b/deploy/node/test_distribution.py index 32a7eb70..e52b028c 100644 --- a/deploy/node/test_distribution.py +++ b/deploy/node/test_distribution.py @@ -160,6 +160,61 @@ def test_url_and_path_boundaries(self): with self.assertRaises(distribution.DistributionError): distribution.obtain_artifact(self.manifest, 'native/bin/node', self.root / 'link') + def test_origin_normalization(self): + self.assertEqual(distribution.origin_of('http://10.20.30.40'), ('http', '10.20.30.40', 80)) + self.assertEqual(distribution.origin_of('https://10.20.30.40:8443/x'), ('https', '10.20.30.40', 8443)) + self.assertIsNone(distribution.origin_of('file:///tmp/x')) + + def test_insecure_origin_admits_only_the_configured_plaintext_origin(self): + source = distribution.origin_of('http://10.20.30.40') + for value in ('http://10.20.30.40/artifact', 'https://release.example/artifact'): + self.assertEqual(distribution.safe_url(value, True, source), value) + # Cross-host plaintext, a plaintext downgrade and every default-off call stay refused. + for value in ('http://10.20.30.41/artifact', 'http://release.example/artifact'): + with self.assertRaises(distribution.DistributionError): + distribution.safe_url(value, True, source) + with self.assertRaises(distribution.DistributionError): + distribution.safe_url('http://10.20.30.40/artifact', False, source) + with self.assertRaises(distribution.DistributionError): + distribution.safe_url('http://10.20.30.40/artifact', True, distribution.origin_of('https://10.20.30.40')) + + def test_insecure_origin_is_projected_to_the_artifact_transfer(self): + self.manifest['artifact_base_url'] = 'http://10.20.30.40/node-install/artifacts' + captured = [] + + def download(url, partial, entry, logical_path, allow_insecure_origin=False, source_origin=None): + captured.append((url, allow_insecure_origin, source_origin)) + partial.write_bytes(self.data) + + with patch.object(distribution, 'download_partial', side_effect=download): + target = self.root / 'node' + distribution.obtain_artifact(self.manifest, 'native/bin/node', target, + allow_insecure_origin=True, source_url='http://10.20.30.40') + self.assertEqual(target.read_bytes(), self.data) + self.assertEqual(captured, [( + 'http://10.20.30.40/node-install/artifacts/' + self.manifest['artifacts']['native/bin/node']['filename'], + True, ('http', '10.20.30.40', 80))]) + with patch.object(distribution, 'download_partial') as download: + with self.assertRaisesRegex(distribution.DistributionError, 'HTTPS'): + distribution.obtain_artifact(self.manifest, 'native/bin/node', self.root / 'other') + download.assert_not_called() + + def test_insecure_origin_redirect_stays_same_origin_and_never_downgrades(self): + source = distribution.origin_of('http://10.20.30.40') + redirect = distribution.ArtifactRedirect(True, source) + request = distribution.urllib.request.Request('http://10.20.30.40/node', + headers={'Range': 'bytes=5-', 'If-Range': 'etag'}) + allowed = redirect.redirect_request(request, None, 307, '', {}, 'http://10.20.30.40/release/file') + self.assertEqual(allowed.full_url, 'http://10.20.30.40/release/file') + self.assertEqual(dict((k.lower(), v) for k, v in allowed.header_items()), + {'range': 'bytes=5-', 'if-range': 'etag'}) + for target in ('http://10.20.30.41/file', 'http://127.0.0.1/file'): + with self.subTest(target=target), self.assertRaises(distribution.ArtifactError): + redirect.redirect_request(request, None, 302, '', {}, target) + secure = distribution.urllib.request.Request('https://10.20.30.40/node') + with self.assertRaises(distribution.ArtifactError): + redirect.redirect_request(secure, None, 302, '', {}, 'http://10.20.30.40/file') + def test_artifact_downgrades_and_metadata_redirects_are_refused(self): self.status = 302 @@ -314,6 +369,17 @@ def test_console_is_the_only_artifact_source(self): loaded = distribution.load_manifest(source_url='https://console.example') self.assertEqual(loaded['artifact_base_url'], 'https://console.example/node-install/artifacts') + def test_insecure_origin_metadata_requires_the_switch(self): + manifest = json.dumps({'source_commit': 'a' * 40, 'platform': 'linux/amd64', 'artifact_base_url': ''}).encode() + sums = (hashlib.sha256(manifest).hexdigest() + ' manifest.json\n').encode() + files = {'SHA256SUMS': sums, 'manifest.json': manifest} + opener = Mock(open=lambda url, timeout: io.BytesIO(files[url.rsplit('/', 1)[1]])) + with patch.object(distribution.urllib.request, 'build_opener', return_value=opener): + loaded = distribution.load_manifest(source_url='http://10.20.30.40', allow_insecure_origin=True) + self.assertEqual(loaded['artifact_base_url'], 'http://10.20.30.40/node-install/artifacts') + with self.assertRaisesRegex(distribution.DistributionError, 'HTTPS'): + distribution.load_manifest(source_url='http://10.20.30.40') + if __name__ == '__main__': unittest.main() diff --git a/deploy/node/test_node_generations.py b/deploy/node/test_node_generations.py index ca64cb5a..fc20dfea 100644 --- a/deploy/node/test_node_generations.py +++ b/deploy/node/test_node_generations.py @@ -245,5 +245,35 @@ def test_release_path_conflict_is_refused_before_native_mutation(self): self.assertTrue((foreign / "artifact").exists()) +class RuntimeFilesPolicyTests(unittest.TestCase): + def test_runtime_files_projects_the_enrollment_policy_to_the_download(self): + source = "b" * 40 + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + release = root / "releases" / source + (release / "runtime").mkdir(parents=True) + value = {"docker": {"seccomp_file": str(release / "runtime/seccomp.json")}} + args = SimpleNamespace(source_url="http://10.20.30.40", provider="docker", allow_insecure_origin=True, + configuration={"specification": {"runtime": {"source_commit": source}}}) + entry = {"filename": "oac-" + source + "-linux-amd64-native-bin-node", "sha256": "a" * 64, "size": 1} + manifest = {"source_commit": source, "artifact_base_url": "http://10.20.30.40/node-install/releases/" + source + "/artifacts", + "artifacts": {"native/bin/node": entry}} + fake = mock.Mock() + fake.provider_assets.artifacts.return_value = ["native/bin/node"] + fake.private_json.return_value = None + fake.existing_file.return_value = False + fake.distribution.artifact.return_value = entry + captured = [] + + def obtain(manifest_arg, name, path, offline_root=None, allow_insecure_origin=False, source_url=None): + captured.append((name, allow_insecure_origin, source_url)) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(b"x") + + fake.distribution.obtain_artifact.side_effect = obtain + self.assertEqual(node_generations.runtime_files(root, value, args, manifest, {}, fake), release) + self.assertEqual(captured, [("native/bin/node", True, "http://10.20.30.40")]) + + if __name__ == "__main__": unittest.main() diff --git a/deploy/node/test_node_install.py b/deploy/node/test_node_install.py index 0e4a244f..6f67f48f 100644 --- a/deploy/node/test_node_install.py +++ b/deploy/node/test_node_install.py @@ -365,10 +365,10 @@ def test_interrupted_new_generation_recovers_partial_download_at_same_identity(s self.args.generation = 2 self.args.specification_digest = config["specification_digest"] real_obtain = installer.distribution.obtain_artifact - def interrupted(manifest, name, path): + def interrupted(manifest, name, path, offline_root=None, allow_insecure_origin=False, source_url=None): if name == installer.MICRO[1]: raise OSError("interrupted partial download") - return real_obtain(manifest, name, path) + return real_obtain(manifest, name, path, offline_root, allow_insecure_origin, source_url) with mock.patch.object(installer.node_spec, "fetch", return_value=config): with mock.patch.object(installer.distribution, "obtain_artifact", side_effect=interrupted): with self.assertRaises(OSError): diff --git a/docs/configuration.md b/docs/configuration.md index f10f9153..8f1fdee5 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -31,7 +31,7 @@ Installer flags in [installation options](./getting-started/install-options.md) `OAC_PUBLIC_URL` is the one origin that applications, nodes, sandboxes and self-hosted executors use. Core derives the daemon WebSocket URL, the self-hosted `remote_url` and each sandbox's connection address from it. The installation serves Web over HTTP on `OAC_WEB_PORT`; your reverse proxy or hosting platform terminates HTTPS and routes to that port. -A `http://` origin is accepted only for a loopback host. A development or test installation can set `OAC_ALLOW_INSECURE_ORIGIN=1` to accept a non-loopback one; TLS certificate verification stays on. +A `http://` origin is accepted only for a loopback host. A development or test installation can set `OAC_ALLOW_INSECURE_ORIGIN=1` to accept a non-loopback one, and nodes may then also download their artifacts from that plaintext origin; TLS certificate verification stays on. To change it, point the reverse proxy at the new address first, then edit `OAC_PUBLIC_URL` and run `oac apply`. Afterwards: @@ -46,7 +46,7 @@ To change it, point the reverse proxy at the new address first, then edit `OAC_P | Variable | Default | Meaning | | --- | --- | --- | | `OAC_PUBLIC_URL` | `http://localhost:8080` | Origin applications, nodes, sandboxes and self-hosted executors use. Managed domain setup writes the HTTPS origin and recreates Core and Web | -| `OAC_ALLOW_INSECURE_ORIGIN` | unset | `1` permits a non-loopback plain-HTTP `OAC_PUBLIC_URL` for development and testing. TLS certificate verification stays on | +| `OAC_ALLOW_INSECURE_ORIGIN` | unset | `1` permits a non-loopback plain-HTTP `OAC_PUBLIC_URL` for development and testing, including node artifact downloads from that origin. TLS certificate verification stays on | | `OAC_HOST` | `127.0.0.1` | Address published by `ports.yaml`. `install.sh` sets `0.0.0.0` | | `OAC_WEB_PORT` | `8080` | Host port of Web | | `COMPOSE_FILE` | `compose.yaml:ports.yaml` | The Compose files. `ports.yaml` publishes Web and Core's loopback admin API; hosting platforms omit it | diff --git a/docs/getting-started/nodes.md b/docs/getting-started/nodes.md index 366a8e39..f76d933e 100644 --- a/docs/getting-started/nodes.md +++ b/docs/getting-started/nodes.md @@ -8,7 +8,7 @@ You add a node by generating a command in Web and running it on the host. The [s ## Before you add a node -- **Core has an HTTPS public URL** that the host and its sandboxes can reach. Nodes download from Core's console and connect to Core at `public_url`. Until it is set, Add node says *Set a public HTTPS address before adding nodes*; see [Configure the public address](./install.md#configure-the-domain-and-https). +- **Core has an HTTPS public URL** that the host and its sandboxes can reach. Nodes download from Core's console and connect to Core at `public_url`. Until it is set, Add node says *Set a public HTTPS address before adding nodes*; see [Configure the public address](./install.md#configure-the-domain-and-https). A development installation with `OAC_ALLOW_INSECURE_ORIGIN=1` may use a non-loopback `http://` URL instead: the generated command then carries `--allow-insecure-origin`, and both the artifact download and the node connection are plaintext. - **The sandbox configuration is saved.** Open **System** → **Manage sandbox configuration**, choose **Own machines**, the backend and a sandbox size, and **Save configuration**. To change a saved configuration, choose **Reset deployment** first. Every node of an installation uses that backend. - **The console can serve the node files.** Nodes download their Runtime and provider files from the console, which redirects to the release for files it does not hold, and check each file's size and SHA-256 against the release manifest. Node hosts therefore need access to the release. Without the files, Add node says *This console has no node files for …*. @@ -151,6 +151,8 @@ Use manual registration when you manage the node's files and service yourself in `oac-node run` exits with status 78 once Core rejects its credential, after the node is removed; configure the supervisor not to restart it then (systemd: `RestartPreventExitStatus=78`). +To register over a non-loopback `http://` origin for development or testing, add `--allow-insecure-origin` to `oac-node register`. The node then downloads artifacts and keeps a `ws://` connection over plaintext HTTP, and `oac-node run` takes no such flag: it uses the policy recorded at registration. Plaintext HTTP carries no transport encryption and is not for production. + The node connects out to Core; Core needs no SSH or Docker TCP access to the host. Registration writes the node's identity to the state directory before contacting Core, so a lost response can be retried under the same identity. Keep the state directory on persistent storage, private to the node's account and used by one process at a time. Never copy it to another directory or host: Core refuses a second connection for a node while the first is open. The provider file can't change the node's capacity or sandbox configuration. Core compares its digest at registration and on every connection, and a node whose file differs takes no work until the approved configuration is restored. ## When a node host fails diff --git a/docs/getting-started/self-hosted.md b/docs/getting-started/self-hosted.md index 4dc25ed0..5fe48b44 100644 --- a/docs/getting-started/self-hosted.md +++ b/docs/getting-started/self-hosted.md @@ -20,7 +20,7 @@ The installer brings its own pinned Node.js and Harness versions (listed in [`sc The machine needs: -- HTTPS access to Core (plain HTTP only on loopback), and to the release download host unless Core carries an offline copy of the installers; +- HTTPS access to Core (plain HTTP only on loopback, or a non-loopback HTTP URL with `OAC_ALLOW_INSECURE_ORIGIN=1`), and to the release download host unless Core carries an offline copy of the installers; - Bash for environment setup and MiniMax Code tools; on Windows, Git Bash, which Claude Code also requires; - Python and pip when the Session's packages need them; - any system packages your setup needs. The daemon never runs apt, sudo or another elevation command, so install them through the host's normal administration. diff --git a/docs/zh/configuration.md b/docs/zh/configuration.md index ee3091cc..e645d345 100644 --- a/docs/zh/configuration.md +++ b/docs/zh/configuration.md @@ -1,7 +1,7 @@ --- title: "配置参考" source: docs/configuration.md -source_hash: 1dbb0c3af7312e925639e9e0a3c868697ff7cbbd321d7aa387ca6732927a38a3 +source_hash: fab6322e73b6b8ca7e55dd7c6c67fd53b31426fa527e13210384c5e36176fc7b --- Core 安装的每项设置都恰好只有一个归属位置。共有两类: @@ -33,7 +33,7 @@ Web 的 **System** 页面显示该安装的地址、默认模型和沙箱配置 `OAC_PUBLIC_URL` 是应用、节点、沙箱和自托管执行器使用的唯一源地址。Core 从中派生守护进程 WebSocket URL、自托管 `remote_url` 和每个沙箱的连接地址。安装通过 `OAC_WEB_PORT` 以 HTTP 提供 Web;反向代理或托管平台终止 HTTPS 并把流量转到该端口。 -`http://` 源地址仅对回环主机被接受。开发或测试安装可以设置 `OAC_ALLOW_INSECURE_ORIGIN=1` 来接受非回环源地址;TLS 证书校验保持不变。 +`http://` 源地址仅对回环主机被接受。开发或测试安装可以设置 `OAC_ALLOW_INSECURE_ORIGIN=1` 来接受非回环源地址,节点随后也可以从该明文源地址下载制品;TLS 证书校验保持不变。 要更改它,先把反向代理指向新地址,然后编辑 `OAC_PUBLIC_URL` 并运行 `oac apply`。之后: @@ -50,7 +50,7 @@ Web 的 **System** 页面显示该安装的地址、默认模型和沙箱配置 | Variable | Default | Meaning | | --- | --- | --- | | `OAC_PUBLIC_URL` | `http://localhost:8080` | Origin applications, nodes, sandboxes and self-hosted executors use. Managed domain setup writes the HTTPS origin and recreates Core and Web | -| `OAC_ALLOW_INSECURE_ORIGIN` | unset | `1` permits a non-loopback plain-HTTP `OAC_PUBLIC_URL` for development and testing. TLS certificate verification stays on | +| `OAC_ALLOW_INSECURE_ORIGIN` | unset | `1` permits a non-loopback plain-HTTP `OAC_PUBLIC_URL` for development and testing, including node artifact downloads from that origin. TLS certificate verification stays on | | `OAC_HOST` | `127.0.0.1` | Address published by `ports.yaml`. `install.sh` sets `0.0.0.0` | | `OAC_WEB_PORT` | `8080` | Host port of Web | | `COMPOSE_FILE` | `compose.yaml:ports.yaml` | The Compose files. `ports.yaml` publishes Web and Core's loopback admin API; hosting platforms omit it | diff --git a/docs/zh/getting-started/nodes.md b/docs/zh/getting-started/nodes.md index ab996dca..3da45840 100644 --- a/docs/zh/getting-started/nodes.md +++ b/docs/zh/getting-started/nodes.md @@ -1,7 +1,7 @@ --- title: "添加和管理节点" source: docs/getting-started/nodes.md -source_hash: f7bfd1ac05cbed8a1f31badee219164c5e1a16aa897e89551abeb8b8d0010215 +source_hash: 029e420d682563c7b948c14172fe3e463b7dfccecc2c972fb2537c8e372cb85f --- 节点是一台 Linux 主机,在沙箱后端为 Docker 或 microsandbox 时,为 Core 托管 Session 运行沙箱。Core 将新 Session 分配给有空余容量的节点;节点创建沙箱,沙箱回连 Core。E2B 不需要节点。应用为自己的 Session 连接的机器是[自托管执行器](self-hosted.md),而不是节点。 @@ -10,7 +10,7 @@ source_hash: f7bfd1ac05cbed8a1f31badee219164c5e1a16aa897e89551abeb8b8d0010215 ## 添加节点前 {#before-you-add-a-node} -- **Core 已有主机及沙箱可访问的 HTTPS 公开 URL。** 节点从 Core 控制台下载文件,并通过 `public_url` 连接 Core。设置前,Add node 显示 *Set a public HTTPS address before adding nodes*;参阅[配置公开地址](install.md#configure-the-domain-and-https)。 +- **Core 已有主机及沙箱可访问的 HTTPS 公开 URL。** 节点从 Core 控制台下载文件,并通过 `public_url` 连接 Core。设置前,Add node 显示 *Set a public HTTPS address before adding nodes*;参阅[配置公开地址](install.md#configure-the-domain-and-https)。使用 `OAC_ALLOW_INSECURE_ORIGIN=1` 的开发安装可以改用非回环的 `http://` URL:生成的命令会携带 `--allow-insecure-origin`,制品下载与节点连接均为明文。 - **沙箱配置已保存。** 打开 **System** → **Manage sandbox configuration**,选择 **Own machines**、后端和沙箱规格,最后选择 **Save configuration**。要更改已保存的配置,先选择 **Reset deployment**。同一安装的所有节点使用同一后端。 - **控制台能提供节点文件。** 节点从控制台下载 Runtime 和提供商文件;控制台缺少文件时重定向到发行下载地址。节点依据发行清单检查各文件的大小和 SHA-256。因此节点主机需要能访问发行下载地址。缺少文件时,Add node 显示 *This console has no node files for …*。 @@ -153,6 +153,8 @@ root 只准备账号、组和服务单元;其他操作(包括 Docker 网络 节点移除后,Core 拒绝其凭据时,`oac-node run` 以状态 78 退出;配置管理器不要在该情况下重启(systemd:`RestartPreventExitStatus=78`)。 +为开发或测试而在非回环 `http://` 源地址上注册时,在 `oac-node register` 上添加 `--allow-insecure-origin`。节点随后以明文 HTTP 下载制品并保持 `ws://` 连接;`oac-node run` 不接受该 flag,而使用注册时记录的策略。明文 HTTP 不提供传输加密,不适用于生产环境。 + 节点向外连接 Core;Core 不需要通过 SSH 或 Docker TCP 访问主机。注册在联系 Core 前先将节点身份写入状态目录,因此响应丢失时可以复用同一身份重试。状态目录放在持久存储上,仅节点账号可访问,每次只由一个进程使用。不要复制到其他目录或主机:节点已有连接打开时,Core 拒绝第二个连接。提供商文件不能修改节点容量或沙箱配置。Core 在注册及每次连接时比较摘要;文件不匹配的节点在恢复批准配置前不接收工作。 ## 节点主机故障时 {#when-a-node-host-fails} diff --git a/docs/zh/getting-started/self-hosted.md b/docs/zh/getting-started/self-hosted.md index 524f13b8..9cc471ba 100644 --- a/docs/zh/getting-started/self-hosted.md +++ b/docs/zh/getting-started/self-hosted.md @@ -1,7 +1,7 @@ --- title: "自托管执行器" source: docs/getting-started/self-hosted.md -source_hash: c560a575e51c02ccb474a629f5655e59779643696826c7677d0b5a949be2e5fc +source_hash: 56ed086e4a7487b8112488b7c13794170a1727d0fb6b20ca9fa3b68b926a784f --- `self_hosted` Session 在应用拥有的机器上运行:工作站、虚拟机或你管理的沙箱。应用通过 `/v1` 创建 Session,并获得安装 `oac-daemon`、启动它并连接 Core 的命令。Web 在 Session 页面展示同一命令;Web 是可选的。Core 不创建、停止或回收这台机器。 @@ -22,7 +22,7 @@ Session 自带模型提供商;安装默认模型不适用([原因](../../../ 机器需要: -- 通过 HTTPS 访问 Core(仅回环地址允许明文 HTTP),以及访问发布下载主机;如果 Core 已有安装程序的离线副本,则无需后者; +- 通过 HTTPS 访问 Core(仅回环地址允许明文 HTTP;启用 `OAC_ALLOW_INSECURE_ORIGIN=1` 时可使用非回环 HTTP URL),以及访问发布下载主机;如果 Core 已有安装程序的离线副本,则无需后者; - 用于环境设置和 MiniMax Code 工具的 Bash;Windows 上需要 Git Bash,Claude Code 也要求它; - Session 的软件包需要时,安装 Python 和 pip; - 环境设置所需的系统软件包。守护进程不运行 apt、sudo 或其他提权命令,请通过主机的常规管理方式安装。 diff --git a/services/core/cmd/sandbox-node/generations.go b/services/core/cmd/sandbox-node/generations.go index 3cd17d93..96e03694 100644 --- a/services/core/cmd/sandbox-node/generations.go +++ b/services/core/cmd/sandbox-node/generations.go @@ -19,6 +19,18 @@ import ( providerconfig "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox/providers" ) +// helperArguments is the single command line for the generation helper. The +// retained insecure-origin policy travels as an explicit flag so the helper +// never re-derives it from the command's own Core URL. +func helperArguments(helper, installationID, action string, generation uint64, digest string, allowInsecureOrigin bool) []string { + arguments := []string{"python3", helper, "--installation-id", installationID, "--generation-action", action, + "--generation", strconv.FormatUint(generation, 10), "--specification-digest", digest} + if allowInsecureOrigin { + arguments = append(arguments, "--allow-insecure-origin") + } + return arguments +} + func runGenerations(ctx context.Context, registry *providerconfig.Registry, configFile, stateDir string) error { root := filepath.Dir(configFile) if stateDir != filepath.Join(root, "state", "node") { @@ -118,7 +130,8 @@ func runGenerations(ctx context.Context, registry *providerconfig.Registry, conf } helper := filepath.Join(root, "generation-preparer.pyz") runHelper := func(ctx context.Context, action string, generation uint64, digest string) error { - command := exec.CommandContext(ctx, "python3", helper, "--installation-id", base.InstallationID, "--generation-action", action, "--generation", strconv.FormatUint(generation, 10), "--specification-digest", digest) + arguments := helperArguments(helper, base.InstallationID, action, generation, digest, stored.AllowInsecureOrigin) + command := exec.CommandContext(ctx, arguments[0], arguments[1:]...) command.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} // Only the preparation/collection process group is canceled. Native sandbox // helpers retain their independent allocation-lock completion semantics. diff --git a/services/core/cmd/sandbox-node/generations_test.go b/services/core/cmd/sandbox-node/generations_test.go index 56a01527..b5d471e9 100644 --- a/services/core/cmd/sandbox-node/generations_test.go +++ b/services/core/cmd/sandbox-node/generations_test.go @@ -7,6 +7,7 @@ import ( "os" "os/exec" "path/filepath" + "reflect" "syscall" "testing" @@ -14,6 +15,19 @@ import ( providerconfig "github.com/MiniMax-AI/OpenAgentCore/services/core/internal/sandbox/providers" ) +func TestGenerationHelperArgumentsCarryInsecureOrigin(t *testing.T) { + defaults := helperArguments("/state/generation-preparer.pyz", "installation", "prepare", 7, "digest", false) + want := []string{"python3", "/state/generation-preparer.pyz", "--installation-id", "installation", + "--generation-action", "prepare", "--generation", "7", "--specification-digest", "digest"} + if !reflect.DeepEqual(defaults, want) { + t.Fatalf("default helper arguments changed: %v", defaults) + } + allowed := helperArguments("/state/generation-preparer.pyz", "installation", "collect", 7, "digest", true) + if allowed[len(allowed)-1] != "--allow-insecure-origin" { + t.Fatalf("insecure-origin policy was not projected: %v", allowed) + } +} + func TestGenerationJournalRestartIdentity(t *testing.T) { config := providerconfig.Config{InstallationID: "installation", Generation: 9, Provider: "docker"} stateDir := t.TempDir()