Repository navigation
Observe published local retry delays and committed replay - #275
Merged
Merged
Conversation
Member
Author
|
General scenario semantics review: PASS for exact The selected cases exercise five callbacks through a normally registered published Worker: The replacement is accurately described as a fresh Worker instance in the same process. It registers before polling, replays the committed result after the signal without another callback, requires the same final result, and checks exact preservation of the earlier history prefix and retry reports. The synthetic controls distinguish zero-tail timing/history and remote-event substitutions. The two 35-second case bounds and finite replacement poll count remain explicit. This review supplies scenario source semantics only. Exact current CI, normal-registry consumer identity, complete caller/profile and cleanup review, and the named actual observation remain owner gates. It supplies no installed timing receipt, new SDK release, cold-process guarantee or clock/redrive/SQLite qualification, and does not resume the set-aside #274 path. |
|
Member
Author
|
Source acceptance for exact A fresh normal-registry4.17 consumer compiles successfully with Rust1.86, exact archive6de774879fe8411776b60736391b2c6b426bb1bc125268718004656efd52f5c3 and all118 members equal. Manifest118eb66a/lock9e065e60; testSHAf95e947b2b3c199bb929e0f7176fb712d3b0f235612c97cbbae859df6cdaa862; compiled executableSHA5cd20d42dc317f6b49637595799af611bcd756a53899e81d1dccb37e9649590a (22,490,816bytes); build messages59285d52. Build exit0/OOMfalse. Both synthetic timing/history guards pass, with the single installed entrypoint ignored; their consumer exits0/OOMfalse. Owned compiler/control IDs and nonce inventories are absent. This is source/build preparation. It provides no actual timing result, new package release or clock/redrive/SQLite continuation. Normal accepted-main verification, the complete ordinary caller/profile review and one named actual observation remain separate gates. |
Member
Author
|
The ordinary local-retry qualifier is merged at Separate one-run caller proposal for shared #219: use the normal crates.io 4.17.0 consumer and this ignored ordinary Worker test against the qualified published native 3.10.0 producer. The short list must yield committed reports [1,2,2,2] and independently measured callback gaps at least those durations; the empty list must yield four explicit zeros. A replacement Worker instance replays the committed result without additional callbacks. This proposal has no installed result yet. The fresh build and two pure source validators pass with exit0/OOMfalse; both exact build/control IDs and empty ID filters have now been independently confirmed absent. The receipt below binds the unmodified registry archive, manifest, lock, source and compiled executable. Earlier source formatting failures remain historical NONPASS. This is a new, separately reviewed caller proposal. Prior logging runner review is background evidence, not authority to run this item. The packet does not depend on the set-aside clock/SQLite items and contains no SQL or database capture. The only Server state is private container tmpfs. Effects are one cached immutable Server at 1 CPU/1 GiB/256 PIDs with read-only root, nonroot user, dropped capabilities and explicit loopback publication; one host consumer has two-CPU affinity, 2 GiB address-space and 60-second CPU limits, plus a 90-second wall bound. The local bridge disables masquerade and inter-container communication and refuses foreign attachments. Workload is bounded at 180 seconds, followed by a 120-second cleanup reserve; the outer timeout is 340 seconds with a 20-second final kill interval. No image pull/build, external service, package publication or production credential is selected. Full IDs and exact owner/task/nonce labels bind cleanup, including uncertain create replies. Config/image identity is checked before and after effects, and runtime/consumer stdout+stderr, OOM and independent absence must be retained. The literal bootstrap is supplied directly to One exact bounded observation requires the complete caller/profile review before execution. Current source/main CI does not qualify the installed result. Frozen packet: {
"controller_sha256": "40a5d60ed02706e427260ff6c662ed4d42854de7d5db4a16ef08069a0325db7b",
"controls_sha256": "2bcc1b6afc1e111f9b4117187596678d4642290dfa3a60bc1961f30f38805bee",
"literal_bootstrap_sha256": "a5150333964c0e2d9fc69ddd95cdbcc7fe042b783b550de6190963b1bdb142ac",
"build_receipt_sha256": "9fff98616c7d87b937ea0d843c8c8ba9cef3271049a884abd49a2aff793bf5b7",
"test_sha256": "f95e947b2b3c199bb929e0f7176fb712d3b0f235612c97cbbae859df6cdaa862",
"binary_sha256": "5cd20d42dc317f6b49637595799af611bcd756a53899e81d1dccb37e9649590a",
"qualified_server": "3.10.0",
"server_source": "b6e04ce7e679464e91f6ceae83310d3bb3204092",
"server_child": "sha256:9d64e9aeee84b46d21715826a3e7e5a96eb8d4ce7fccb126b88b15596ab14973",
"scope": "one ordinary219 local-retry timing observation; no clock/redrive/database operation",
"source_controls": 14,
"workload_seconds": 180,
"parent_seconds": 90,
"cleanup_seconds": 120,
"parent_wait_seconds": 10,
"outer_seconds": 340,
"outer_kill_seconds": 20
}Registry build receipt: {
"version": "4.17.0",
"installation": "crates.io registry without source override",
"package_commit": "52208134296009bbae1af316c8efac89d3465990",
"crate_sha256": "6de774879fe8411776b60736391b2c6b426bb1bc125268718004656efd52f5c3",
"crate_file_count": 118,
"crate_members_equal": true,
"test_source_commit": "9c25dbc17af8d99f4be5633aa06d009526847976",
"accepted_main": "d0a8de84833f78fc31749d4e817afa03ee73f42a",
"test_sha256": "f95e947b2b3c199bb929e0f7176fb712d3b0f235612c97cbbae859df6cdaa862",
"manifest_sha256": "118eb66abb44a42bb3aef686379ba087ab12ee02592a05c75e934a8f2cc34ef9",
"lock_sha256": "9e065e60f19c97eda6d87f2be065bacb0f78c1c96c99bc83daa147b97c058466",
"build_messages_relative": "fixture/build.jsonl",
"build_messages_sha256": "59285d5275f199750611c2ed6827a79a2bdee1d824ce6351f719c6270107fe50",
"executable_relative": "fixture/target/debug/deps/local_retry_timing_server-38ab37d823ed3661",
"executable_sha256": "5cd20d42dc317f6b49637595799af611bcd756a53899e81d1dccb37e9649590a",
"executable_bytes": 22490816,
"rust_image": "rust@sha256:300ec56abce8cc9448ddea2172747d048ed902a3090e6b57babb2bf19f754081",
"cargo_version": "1.86.0",
"build_network": "none",
"cargo_source_override": false,
"ordinary_controls": {
"pass": 2,
"ignored_installed": 1,
"sdk_server_executed": false
}
}Exact controller source: """One fixed ordinary local-retry observation for shared219; separate exact review required."""
import base64
import fcntl
import hashlib
import json
import os
import pathlib
import resource
import re
import secrets
import signal
import stat
import subprocess
import sys
import time
import urllib.error
import urllib.request
if sys.flags.optimize != 0 or sys.flags.no_site != 1 or sys.dont_write_bytecode is not True:
raise RuntimeError("unsafe caller Python startup")
assert len(sys.argv) == 3
class BudgetExpired(RuntimeError):
pass
def workload_deadline(signum, frame):
raise BudgetExpired("ordinary retry observation deadline")
def cleanup_signals():
previous = signal.pthread_sigmask(signal.SIG_BLOCK, {signal.SIGALRM, signal.SIGTERM})
signal.alarm(0)
signal.signal(signal.SIGALRM, signal.SIG_IGN)
signal.signal(signal.SIGTERM, signal.SIG_IGN)
signal.pthread_sigmask(signal.SIG_SETMASK, previous)
ROOT = pathlib.Path(sys.argv[1]).absolute()
SOURCE = pathlib.Path(sys.argv[2]).absolute()
IMAGE = 'ghcr.io/durable-workflow/server@sha256:9d64e9aeee84b46d21715826a3e7e5a96eb8d4ce7fccb126b88b15596ab14973'
REVISION = 'b6e04ce7e679464e91f6ceae83310d3bb3204092'
RECEIPT = '9fff98616c7d87b937ea0d843c8c8ba9cef3271049a884abd49a2aff793bf5b7'
BINARY = '5cd20d42dc317f6b49637595799af611bcd756a53899e81d1dccb37e9649590a'
TEST = 'f95e947b2b3c199bb929e0f7176fb712d3b0f235612c97cbbae859df6cdaa862'
def digest(data):
return hashlib.sha256(data).hexdigest()
def directory(path):
assert path.is_absolute() and '..' not in path.parts
current = os.open('/', os.O_RDONLY | os.O_DIRECTORY)
try:
for part in path.parts[1:]:
following = os.open(part, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW, dir_fd=current)
os.close(current)
current = following
return current
except BaseException:
os.close(current)
raise
def read_at(root, relative):
parts = pathlib.PurePosixPath(relative).parts
assert parts and not relative.startswith('/') and '..' not in parts
parent = os.dup(root)
try:
for part in parts[:-1]:
following = os.open(part, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW, dir_fd=parent)
os.close(parent)
parent = following
fd = os.open(parts[-1], os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK, dir_fd=parent)
try:
info = os.fstat(fd)
assert stat.S_ISREG(info.st_mode) and info.st_nlink == 1 and info.st_uid == os.getuid()
assert info.st_size <= 64 * 1024**2
value = os.pread(fd, info.st_size + 1, 0)
after = os.fstat(fd)
fields = ('st_dev', 'st_ino', 'st_size', 'st_uid', 'st_gid', 'st_mode',
'st_nlink', 'st_mtime_ns', 'st_ctime_ns')
assert len(value) == info.st_size
assert all(getattr(info, field) == getattr(after, field) for field in fields)
return value
finally:
os.close(fd)
finally:
os.close(parent)
def exclusive(root, name):
assert '/' not in name and name not in ('', '.', '..')
return os.open(name, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW,
0o600, dir_fd=root)
def output(root, fd, name, value):
before = os.fstat(fd)
named = os.stat(name, dir_fd=root, follow_symlinks=False)
assert (named.st_dev, named.st_ino) == (before.st_dev, before.st_ino)
assert stat.S_ISREG(named.st_mode) and stat.S_IMODE(named.st_mode) == 0o600
assert named.st_uid == os.getuid() and named.st_nlink == 1
with os.fdopen(os.dup(fd), 'wb') as handle:
handle.write(value)
handle.flush()
after = os.stat(name, dir_fd=root, follow_symlinks=False)
assert (after.st_dev, after.st_ino, after.st_nlink) == (before.st_dev, before.st_ino, 1)
root_fd = directory(ROOT)
root_info = os.fstat(root_fd)
assert root_info.st_uid == os.getuid() and stat.S_IMODE(root_info.st_mode) == 0o700
receipt_bytes = read_at(root_fd, 'build-receipt.json')
assert digest(receipt_bytes) == RECEIPT
receipt = json.loads(receipt_bytes)
assert receipt['executable_sha256'] == BINARY
assert digest(read_at(root_fd, 'fixture/tests/local_retry_timing_server.rs')) == TEST
assert digest(read_at(root_fd, 'fixture/Cargo.lock')) == receipt['lock_sha256']
assert digest(read_at(root_fd, 'fixture/Cargo.toml')) == receipt['manifest_sha256']
assert digest(read_at(root_fd, 'fixture/build.jsonl')) == receipt['build_messages_sha256']
binary_bytes = read_at(root_fd, receipt['executable_relative'])
assert len(binary_bytes) == receipt['executable_bytes']
assert digest(binary_bytes) == BINARY
fd = os.memfd_create('reviewed-local-retry', os.MFD_ALLOW_SEALING)
with os.fdopen(os.dup(fd), 'wb') as image_file:
image_file.write(binary_bytes)
os.fchmod(fd, 0o500)
seals = fcntl.F_SEAL_WRITE | fcntl.F_SEAL_GROW | fcntl.F_SEAL_SHRINK | fcntl.F_SEAL_SEAL
fcntl.fcntl(fd, fcntl.F_ADD_SEALS, seals)
assert fcntl.fcntl(fd, fcntl.F_GET_SEALS) == seals
assert digest(os.pread(fd, len(binary_bytes) + 1, 0)) == BINARY
supervisor = SOURCE / 'scripts/acceptance/installed-native/cell-supervisor.sh'
source_fd = directory(SOURCE)
supervisor_bytes = read_at(source_fd, 'scripts/acceptance/installed-native/cell-supervisor.sh')
os.close(source_fd)
assert digest(supervisor_bytes) == '25a591f514b3823093a37b66c47c588cba8e983594c8da2d7940ac4f565b922b'
def runtime_command(script, deadline):
script = script.decode()
original = ' /bin/sh "$0" --command "$@"'
assert script.count(original) == 1
wrapper = 'supervisor=$1\nshift\n' + script.replace(original,
' /bin/sh -c "$supervisor" supervisor --command "$@"')
return ['-c', wrapper, 'supervisor', script, str(deadline), '10', 'both',
'/usr/local/bin/durable-workflow-server']
# procfs fdinfo admits only kernel-created decimal descriptor names. A same-UID
# writer cannot create config.json, contexts or plugin paths in this namespace.
config_path = '/proc/' + str(os.getpid()) + '/fdinfo'
config_fd = os.open(config_path, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
config_info = os.fstat(config_fd)
evidence = {name: exclusive(root_fd, name) for name in
('retry.stdout', 'retry.stderr', 'runtime.stdout', 'runtime.stderr', 'retry-result.json',
'container-created.json', 'container-started.json', 'container-returned.json',
'network-created.json', 'network-returned.json')}
CLOSED_ENV = {'PATH': '/usr/bin:/bin', 'HOME': '/nonexistent', 'LANG': 'C', 'LC_ALL': 'C'}
def docker_command(args, config):
assert args[0] == 'docker'
return ['/usr/bin/docker', '--host', 'unix:///run/docker.sock', '--config',
'/proc/self/fd/' + str(config)] + args[1:]
def call(args, wall=30, **kwargs):
tool = os.stat('/usr/bin/docker', follow_symlinks=False)
assert stat.S_ISREG(tool.st_mode) and tool.st_uid == 0 and tool.st_nlink == 1
assert stat.S_IMODE(tool.st_mode) == 0o755
assert digest(pathlib.Path('/usr/bin/docker').read_bytes()) == '40cdaf7fd0f21089dd9e15b0c3a7dd7f2399027f010e366dac6304ae0615954a'
assert all(name.isdecimal() for name in os.listdir(config_fd))
config_named = os.stat(config_path, follow_symlinks=False)
config_open = os.fstat(config_fd)
assert (config_named.st_dev, config_named.st_ino) == (config_info.st_dev, config_info.st_ino)
assert (config_open.st_dev, config_open.st_ino) == (config_info.st_dev, config_info.st_ino)
kwargs.setdefault('check', True)
return subprocess.run(docker_command(args, config_fd), capture_output=True, timeout=wall,
cwd='/', env=CLOSED_ENV, pass_fds=(config_fd,), **kwargs)
socket_info = os.stat('/run/docker.sock', follow_symlinks=False)
assert stat.S_ISSOCK(socket_info.st_mode) and socket_info.st_uid == 0
def engine():
current = os.stat('/run/docker.sock', follow_symlinks=False)
assert (current.st_dev, current.st_ino, current.st_uid, current.st_gid, current.st_mode) == (
socket_info.st_dev, socket_info.st_ino, socket_info.st_uid, socket_info.st_gid, socket_info.st_mode)
identity = call(['docker', 'info', '--format', '{{.ID}}']).stdout.strip()
assert digest(identity) == '0a179ff32531e41cfc2d465b1d46afc1df91c0975e9627969f75c67e00b0e892'
def image_identity():
image = json.loads(call(['docker', 'image', 'inspect', IMAGE]).stdout)[0]
assert image['Id'] == IMAGE.split('@')[1]
assert image['RepoDigests'] == [IMAGE] and image['Descriptor']['digest'] == IMAGE.split('@')[1]
assert image['Os'] == 'linux' and image['Architecture'] == 'amd64'
assert digest(json.dumps(image['Config'], sort_keys=True, separators=(',', ':')).encode()) == 'c88782b528aee2a1b7804cd62cb95afac9df8c5b2f89c471924c6c1085c63b62'
return image
engine()
image = image_identity()
assert image['Os'] == 'linux' and image['Architecture'] == 'amd64'
assert image['Config']['Labels']['org.opencontainers.image.revision'] == REVISION
assert image['Config']['Labels']['dev.durable-workflow.release.tag'] == '3.10.0'
assert image['Config']['Entrypoint'] == ['/usr/local/bin/durable-workflow-server']
nonce = secrets.token_hex(16)
name = 'sdk-rust-219-timing-' + nonce[:12]
labels = {'owner': 'sdk-rust', 'task': 'retry219', 'qualification_nonce': nonce}
label_args = [argument for key, value in labels.items() for argument in ('--label', key + '=' + value)]
token = 'fixture-' + secrets.token_hex(24)
key = 'base64:' + base64.b64encode(secrets.token_bytes(32)).decode()
container_id = network_id = None
process = None
pending_create = None
result = {'outcome': 'fail', 'build_receipt_sha256': RECEIPT,
'binary_sha256': BINARY, 'server_manifest': IMAGE, 'server_source': REVISION,
'started_at': time.time(), 'limits': {'consumer_cpus': 2, 'consumer_as': 2147483648,
'consumer_cpu_seconds': 60, 'consumer_wall_seconds': 90}}
def owned(kind, identity):
value = json.loads(call(['docker', kind, 'inspect', identity]).stdout)[0]
actual = value['Config']['Labels'] if kind == 'container' else value['Labels']
assert value['Id'] == identity and all(actual.get(k) == v for k, v in labels.items())
return value
def full_id(raw):
identity = raw.decode('ascii').strip()
assert re.fullmatch('[0-9a-f]{64}', identity), 'uncertain create reply'
return identity
def reconcile_created(kind, identity):
if identity is not None:
assert re.fullmatch('[0-9a-f]{64}', identity)
return identity
filters = [arg for key, val in labels.items() for arg in ('--filter', 'label=' + key + '=' + val)]
query = ['docker', 'ps', '-aq', '--no-trunc'] if kind == 'container' else ['docker', 'network', 'ls', '-q', '--no-trunc']
candidates = call(query + filters).stdout.split()
assert len(candidates) <= 1
if candidates:
recovered = full_id(candidates[0])
value = owned(kind, recovered)
assert value['Name'].lstrip('/') == name
return recovered
return None
def remove_resource(kind, identity):
value = owned(kind, identity)
call(['docker', 'rm', '-f', '-v', identity] if kind == 'container'
else ['docker', 'network', 'rm', identity])
observed = call(['docker', kind, 'inspect', identity], check=False)
missing = ('no such container: ' + identity if kind == 'container'
else 'network ' + identity + ' not found').encode()
assert observed.returncode != 0 and json.loads(observed.stdout) == []
assert missing in observed.stderr.lower()
query = ['docker', 'ps', '-aq', '--no-trunc'] if kind == 'container' else ['docker', 'network', 'ls', '-q', '--no-trunc']
assert call(query + ['--filter', 'id=' + identity]).stdout.strip() == b''
return value
class NoRedirect(urllib.request.HTTPRedirectHandler):
def redirect_request(self, *args):
raise AssertionError('redirect refused')
def consumer_limits():
os.sched_setaffinity(0, sorted(os.sched_getaffinity(0))[:2])
resource.setrlimit(resource.RLIMIT_AS, (2 * 1024**3, 2 * 1024**3))
resource.setrlimit(resource.RLIMIT_CPU, (60, 60))
resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
def validate_network(value, runtime=None):
assert value['Driver'] == 'bridge' and value['Scope'] == 'local'
assert value['Internal'] is False and value['Ingress'] is False and value['Attachable'] is False
assert value['EnableIPv6'] is False
assert value['Options'] == {
'com.docker.network.bridge.enable_ip_masquerade': 'false',
'com.docker.network.bridge.enable_icc': 'false',
'com.docker.network.bridge.host_binding_ipv4': '127.0.0.1'}
assert set(value.get('Containers') or {}) <= ({runtime} if runtime else set())
def validate_container(value, image, network, name, command, labels, tmpfs):
assert value['Image'] == image['Id']
config, host = value['Config'], value['HostConfig']
expected_env = dict(entry.split('=', 1) for entry in image['Config']['Env'])
expected_env.update(DB_CONNECTION='sqlite', DB_DATABASE='/app/database/runtime.sqlite',
DW_RUST_MAINTENANCE_LOOP='1')
assert len(config['Env']) == len(expected_env)
assert dict(entry.split('=', 1) for entry in config['Env']) == expected_env
assert config['Image'] == IMAGE and config['Entrypoint'] == ['/bin/sh'] and config['Cmd'] == command
assert config['Hostname'] == name and config['User'] == '1000:1000'
assert config['WorkingDir'] == '/app' and config['StopSignal'] == 'SIGTERM'
assert config['Volumes'] == image['Config']['Volumes']
assert config['ExposedPorts'] == {'8080/tcp': {}}
assert config['Labels'] == dict(image['Config']['Labels'], **labels)
assert not config['Tty'] and not config['OpenStdin']
expected = {'ReadonlyRootfs': True, 'Init': True, 'Privileged': False,
'NanoCpus': 10**9, 'Memory': 1024**3, 'MemorySwap': 1024**3, 'PidsLimit': 256,
'CapAdd': None, 'CapDrop': ['ALL'], 'SecurityOpt': ['no-new-privileges'],
'NetworkMode': network, 'Binds': None, 'Tmpfs': tmpfs,
'PortBindings': {'8080/tcp': [{'HostIp': '127.0.0.1', 'HostPort': ''}]},
'PidMode': '', 'UTSMode': '', 'UsernsMode': '', 'IpcMode': 'private',
'Devices': [], 'DeviceRequests': None, 'VolumesFrom': None, 'ExtraHosts': None,
'AutoRemove': False, 'RestartPolicy': {'Name': 'no', 'MaximumRetryCount': 0}}
assert all(host.get(key) == val for key, val in expected.items())
assert 'OomKillDisable' in host and host['OomKillDisable'] in (False, None)
assert not host.get('Mounts')
assert all(m['Type'] == 'tmpfs' and m['Destination'] in tmpfs for m in value['Mounts'])
packet = json.loads(json.dumps({k: value[k] for k in ('Image', 'Config', 'HostConfig', 'Mounts')}))
# Docker29 changes this unset/false default during start. Both retain the
# kernel OOM killer; true is refused. Every other field remains exact.
packet['HostConfig']['OomKillDisable'] = False
return digest(json.dumps(packet, sort_keys=True, separators=(',', ':')).encode())
def validate_records(records):
assert type(records) is list and len(records) == 2
workflows, runs = set(), set()
for record, (case, expected) in zip(records, [('short', [1, 2, 2, 2]), ('empty', [0, 0, 0, 0])], strict=True):
assert set(record) == {'case','workflow_id','run_id','expected_backoff_seconds','callback_gap_ms',
'retry_reports','callback_attempts','callbacks_after_replay','history'}
assert record['case'] == case and record['expected_backoff_seconds'] == expected
workflow, run = record['workflow_id'], record['run_id']
assert isinstance(workflow,str) and re.fullmatch('rust-retry-'+case+'-[a-f0-9-]{36}', workflow)
assert isinstance(run,str) and re.fullmatch('[A-Za-z0-9_-]{1,128}',run)
assert workflow not in workflows and run not in runs
workflows.add(workflow);runs.add(run)
assert type(record['callback_attempts']) is int and record['callback_attempts'] == 5
assert type(record['callbacks_after_replay']) is int and record['callbacks_after_replay'] == 0
gaps = record['callback_gap_ms']
assert type(gaps) is list and len(gaps) == 4
assert all(type(gap) is int and delay*1000 <= gap < 35000 for gap,delay in zip(gaps,expected,strict=True))
history = record['history']
assert history['history_complete'] is True and history['workflow']['details_state'] == 'retained'
assert history['workflow']['instance_id'] == workflow and history['workflow']['run_id'] == run
retries = [event['payload'] for event in history['history_events'] if event['type']=='ActivityRetryScheduled']
assert len(retries) == 4 and retries == record['retry_reports']
execution = retries[0]['activity_execution_id']
assert isinstance(execution,str) and execution
for attempt,(retry,delay) in enumerate(zip(retries,expected,strict=True),1):
assert retry['execution_mode']=='local' and retry['local_activity'] is True
assert retry['activity_execution_id']==execution and retry['sequence']==1
assert type(retry['retry_after_attempt']) is int and retry['retry_after_attempt']==attempt
assert type(retry['retry_backoff_seconds']) is int and retry['retry_backoff_seconds']==delay
def cleanup_resources(container_id, network_id, result, pending_create, seconds=120, collect_logs=False):
global call
original = call
stop = time.monotonic()+seconds
def limited(args, wall=30, **kwargs):
remaining=stop-time.monotonic()
if remaining<=0:raise BudgetExpired('cleanup reserve exhausted')
return original(args,wall=min(wall,remaining),**kwargs)
call=limited
errors=[]
try:
for kind,identity in [('container',container_id),('network',network_id)]:
try:
identity=reconcile_created(kind,None if pending_create==kind else identity)
if identity is None and pending_create==kind:errors.append(kind+':uncertain-create')
if identity:
result[kind+'_id']=identity
if kind=='container' and collect_logs:
try:
logs=call(['docker','logs','--tail','100',identity],check=False,wall=10)
assert len(logs.stdout)+len(logs.stderr)<=128*1024
out,err=logs.stdout,logs.stderr
for secret in (token.encode(),key.encode()):
out=out.replace(secret,b'[redacted]');err=err.replace(secret,b'[redacted]')
output(root_fd,evidence['runtime.stdout'],'runtime.stdout',out)
output(root_fd,evidence['runtime.stderr'],'runtime.stderr',err)
except Exception as error:errors.append('runtime-logs:'+type(error).__name__)
value=remove_resource(kind,identity)
if kind=='container':
result['runtime_oom']=value['State']['OOMKilled']
if result['runtime_oom']:result['outcome']='fail'
except Exception as error:errors.append(kind+':'+type(error).__name__)
finally:call=original
return errors
tmpfs = {'/tmp': 'mode=1777,size=64m',
'/app/database': 'uid=1000,gid=1000,mode=0700,size=64m',
'/app/storage': 'uid=1000,gid=1000,mode=0700,size=64m',
'/credentials': 'uid=1000,gid=1000,mode=0700,size=64k,noexec,nosuid,nodev'}
command = None
created_config = None
signal.signal(signal.SIGALRM,workload_deadline)
signal.signal(signal.SIGTERM,workload_deadline)
signal.pthread_sigmask(signal.SIG_UNBLOCK,{signal.SIGALRM,signal.SIGTERM})
signal.alarm(180)
try:
engine()
image_identity()
pending_create='network'
network_id = full_id(call(['docker', 'network', 'create', '--driver', 'bridge',
'--opt', 'com.docker.network.bridge.enable_ip_masquerade=false',
'--opt', 'com.docker.network.bridge.enable_icc=false',
'--opt', 'com.docker.network.bridge.host_binding_ipv4=127.0.0.1'] + label_args + [name]).stdout)
result['network_id'] = network_id
created_network = owned('network', network_id)
pending_create=None
output(root_fd, evidence['network-created.json'], 'network-created.json',
(json.dumps(created_network, indent=2) + '\n').encode())
validate_network(created_network)
deadline = str(int(time.time()) + 240)
command = runtime_command(supervisor_bytes, deadline)
engine()
image_identity()
pending_create='container'
container_id = full_id(call(['docker', 'create', '--pull=never', '--init', '--name', name,
'--hostname', name] + label_args + [
'--cpus', '1', '--memory', '1g', '--memory-swap', '1g', '--pids-limit', '256', '--read-only', '--user', '1000:1000',
'--cap-drop', 'ALL', '--security-opt', 'no-new-privileges', '--network', network_id,
'--publish', '127.0.0.1::8080', '--tmpfs', '/tmp:mode=1777,size=64m',
'--tmpfs', '/app/database:uid=1000,gid=1000,mode=0700,size=64m',
'--tmpfs', '/app/storage:uid=1000,gid=1000,mode=0700,size=64m',
'--tmpfs', '/credentials:uid=1000,gid=1000,mode=0700,size=64k,noexec,nosuid,nodev',
'--env', 'DB_CONNECTION=sqlite', '--env', 'DB_DATABASE=/app/database/runtime.sqlite',
'--env', 'DW_RUST_MAINTENANCE_LOOP=1', '--entrypoint', '/bin/sh', IMAGE] + command).stdout)
result['container_id'] = container_id
before = owned('container', container_id)
pending_create=None
output(root_fd, evidence['container-created.json'], 'container-created.json',
(json.dumps(before, indent=2) + '\n').encode())
result['container_id'], result['network_id'] = container_id, network_id
created_config = validate_container(before, image, network_id, name, command, labels, tmpfs)
validate_network(owned('network', network_id), container_id)
result['created_config_sha256'] = created_config
engine()
image_identity()
assert validate_container(owned('container', container_id), image, network_id,
name, command, labels, tmpfs) == created_config
call(['docker', 'start', container_id])
started = owned('container', container_id)
output(root_fd, evidence['container-started.json'], 'container-started.json',
(json.dumps(started, indent=2) + '\n').encode())
assert validate_container(started, image, network_id,
name, command, labels, tmpfs) == created_config
validate_network(owned('network', network_id), container_id)
call(['docker', 'exec', '-i', container_id, '/bin/sh', '-c',
'umask 077; IFS= read -r token && IFS= read -r key && '
'printf %s "$token" > /credentials/token && printf %s "$key" > /credentials/server-key && '
': > /credentials/ready.tmp && mv /credentials/ready.tmp /credentials/ready'],
input=(token + '\n' + key + '\n').encode())
live = owned('container', container_id)
ports = live['NetworkSettings']['Ports']['8080/tcp']
assert len(ports) == 1 and ports[0]['HostIp'] == '127.0.0.1'
port = int(ports[0]['HostPort'])
assert 0 < port < 65536
origin = 'http://127.0.0.1:' + str(port)
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}), NoRedirect())
for attempt in range(60):
try:
request = urllib.request.Request(origin + '/api/cluster/info',
headers={'Authorization': 'Bearer ' + token, 'X-Namespace': 'default'})
with opener.open(request, timeout=1) as response:
cluster = json.load(response)
assert cluster['version'] == '3.10.0'
result['cluster_info'] = cluster
break
except (urllib.error.URLError, ConnectionError):
time.sleep(.25)
else:
raise AssertionError('runtime readiness bound expired')
args = ['/proc/self/fd/' + str(fd), '--ignored', '--exact',
'published_local_retries_repeat_last_and_replay_committed_results',
'--nocapture', '--test-threads=1']
env = {'DURABLE_WORKFLOW_LOCAL_ISOLATED': '1', 'DURABLE_WORKFLOW_SERVER_URL': origin,
'DURABLE_WORKFLOW_AUTH_TOKEN': token, 'DURABLE_WORKFLOW_EXPECT_SERVER_VERSION': '3.10.0'}
process = subprocess.Popen(args, env=env, pass_fds=(fd,), start_new_session=True,
preexec_fn=consumer_limits, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
try:
stdout, stderr = process.communicate(timeout=90)
except subprocess.TimeoutExpired:
if process.poll() is None:
try:os.killpg(process.pid,signal.SIGKILL)
except ProcessLookupError:pass
stdout, stderr = process.communicate(timeout=10)
result['observer_timeout'] = True
assert len(stdout) + len(stderr) <= 1024**2
assert all(secret.encode() not in stdout + stderr for secret in (token, key))
output(root_fd, evidence['retry.stdout'], 'retry.stdout', stdout)
output(root_fd, evidence['retry.stderr'], 'retry.stderr', stderr)
result['parent_pid'], result['exit_code'] = process.pid, process.returncode
assert process.returncode == 0, 'local retry observer failed'
marker='LOCAL_RETRY_TIMING '
records=[json.loads(line[line.index(marker)+len(marker):]) for line in stdout.decode().splitlines() if marker in line]
validate_records(records)
result['observations']=records
assert digest(os.pread(fd, len(binary_bytes) + 1, 0)) == BINARY
engine()
image_identity()
returned = owned('container', container_id)
output(root_fd, evidence['container-returned.json'], 'container-returned.json',
(json.dumps(returned, indent=2) + '\n').encode())
assert validate_container(returned, image, network_id,
name, command, labels, tmpfs) == created_config
final_network = owned('network', network_id)
output(root_fd, evidence['network-returned.json'], 'network-returned.json',
(json.dumps(final_network, indent=2) + '\n').encode())
validate_network(final_network, container_id)
result['outcome'] = 'pass'
except BaseException as error:
result['error'] = type(error).__name__
raise
finally:
cleanup_signals()
parent_errors=[]
if process is not None:
try:
if process.poll() is None:
os.killpg(process.pid,signal.SIGKILL)
process.wait(timeout=10)
except ProcessLookupError:pass
except Exception as error:parent_errors.append('observer:'+type(error).__name__)
try:os.close(fd)
except OSError as error:parent_errors.append('binary-fd:'+type(error).__name__)
errors=parent_errors+cleanup_resources(container_id,network_id,result,pending_create,seconds=120,collect_logs=True)
result['cleanup'] = {'pass': not errors, 'errors': errors}
if errors:
result['outcome'] = 'fail'
result['finished_at'] = time.time()
named_root = ROOT.stat(follow_symlinks=False)
assert (named_root.st_dev, named_root.st_ino) == (root_info.st_dev, root_info.st_ino)
output(root_fd, evidence['retry-result.json'], 'retry-result.json',
(json.dumps(result, indent=2) + '\n').encode())
for descriptor in evidence.values():
os.close(descriptor)
os.close(config_fd)
os.close(root_fd)
assert not errors, errors
print(json.dumps(result))Exact inert controls: import ast
import copy
import hashlib
import os
import pathlib
import re
import stat
import subprocess
import sys
import tempfile
import time
import signal
import unittest
from unittest import mock
SOURCE = pathlib.Path(sys.argv[1])
SUPERVISOR = pathlib.Path(sys.argv[2])
sys.argv = sys.argv[:1]
module = ast.parse(SOURCE.read_text())
keep = {'digest', 'directory', 'read_at', 'exclusive', 'output', 'runtime_command', 'docker_command', 'call', 'validate_network', 'validate_container', 'full_id', 'reconcile_created', 'remove_resource', 'owned', 'validate_records', 'cleanup_resources', 'BudgetExpired', 'workload_deadline'}
ns = {'pathlib': pathlib, 'os': os, 'stat': stat, 'hashlib': hashlib, 'subprocess': subprocess,
'json': __import__('json'), 're': re, 'CLOSED_ENV': {'PATH': '/usr/bin:/bin', 'HOME': '/nonexistent', 'LANG': 'C', 'LC_ALL': 'C'},
'IMAGE': 'synthetic-image', 'name':'owned', 'time':time, 'signal':signal}
exec(compile(ast.Module(body=[node for node in module.body if isinstance(node, (ast.FunctionDef, ast.ClassDef)) and node.name in keep], type_ignores=[]), str(SOURCE), 'exec'), ns)
class Controls(unittest.TestCase):
def test_supervisor_replacement_discriminates_predecessor(self):
script = SUPERVISOR.read_bytes()
self.assertEqual(hashlib.sha256(script).hexdigest(), '25a591f514b3823093a37b66c47c588cba8e983594c8da2d7940ac4f565b922b')
with tempfile.TemporaryDirectory() as tmp:
path = pathlib.Path(tmp) / 'supervisor'
path.write_bytes(script)
captured = path.read_bytes()
command = ns['runtime_command'](captured, int(time.time()) + 10)
command[-2:] = ['none', '/bin/echo', 'ORIGINAL']
path.write_text('printf SUBSTITUTED')
predecessor = subprocess.check_output(['/bin/sh', str(path), '10', '10', 'none', '/bin/echo', 'ORIGINAL'])
self.assertEqual(predecessor, b'SUBSTITUTED')
self.assertEqual(subprocess.check_output(['/bin/sh'] + command), b'ORIGINAL\n')
self.assertNotIn(str(path), command)
def test_exclusive_output_refuses_symlink(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp)
marker = root / 'marker'; marker.write_bytes(b'unchanged')
(root / 'logging.stdout').symlink_to(marker)
(root / 'logging.stdout').write_bytes(b'PREDECESSOR')
self.assertEqual(marker.read_bytes(), b'PREDECESSOR')
marker.write_bytes(b'unchanged')
fd = ns['directory'](root)
try:
with self.assertRaises(FileExistsError): ns['exclusive'](fd, 'logging.stdout')
self.assertEqual(marker.read_bytes(), b'unchanged')
finally: os.close(fd)
def test_output_replacement_refuses_before_foreign_write(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp); fd = ns['directory'](root)
out = ns['exclusive'](fd, 'logging.stdout')
try:
self.assertEqual(stat.S_IMODE(os.fstat(out).st_mode), 0o600)
(root / 'logging.stdout').rename(root / 'held')
marker = root / 'marker'; marker.write_bytes(b'unchanged')
(root / 'logging.stdout').symlink_to(marker)
with self.assertRaises(AssertionError): ns['output'](fd, out, 'logging.stdout', b'bad')
self.assertEqual(marker.read_bytes(), b'unchanged')
finally: os.close(out); os.close(fd)
def test_input_and_root_links_refused(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp); (root / 'alias').symlink_to(root, target_is_directory=True)
with self.assertRaises(OSError): ns['directory'](root / 'alias')
fd = ns['directory'](root)
try:
(root / 'value').write_bytes(b'expected'); (root / 'link').symlink_to(root / 'value')
with self.assertRaises(OSError): ns['read_at'](fd, 'link')
self.assertEqual(ns['read_at'](fd, 'value'), b'expected')
finally: os.close(fd)
def test_effectful_tool_and_environment_are_closed(self):
configpath = '/proc/' + str(os.getpid()) + '/fdinfo'
configfd = os.open(configpath, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
ns.update(config_fd=configfd, config_path=configpath, config_info=os.fstat(configfd))
captured = []
def fake(*args, **kwargs):
# Runs after call validation, just before the consumer would open config.
with self.assertRaises(OSError):
opened = os.open('config.json', os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600, dir_fd=configfd)
os.close(opened)
self.assertFalse(os.path.lexists(pathlib.Path(configpath) / 'config.json'))
captured.append((args, kwargs)); return object()
try:
with mock.patch.dict(os.environ, {'PATH': '/synthetic-untrusted', 'DOCKER_HOST': 'tcp://wrong', 'DOCKER_CONFIG': '/wrong', 'DOCKER_CONTEXT': 'wrong'}):
with mock.patch.object(subprocess, 'run', side_effect=fake): ns['call'](['docker', 'create', '--pull=never'])
args, kwargs = captured[0]
self.assertEqual(args[0][:3], ['/usr/bin/docker', '--host', 'unix:///run/docker.sock'])
self.assertEqual(kwargs['env'], ns['CLOSED_ENV'])
self.assertEqual(kwargs['pass_fds'], (configfd,))
self.assertEqual(kwargs['cwd'], '/')
self.assertNotIn('DOCKER_HOST', kwargs['env'])
finally: os.close(configfd)
def test_predecessor_config_changes_between_check_and_open(self):
with tempfile.TemporaryDirectory() as tmp:
path = pathlib.Path(tmp) / 'config.json'; path.write_text('{}\n')
before = path.stat()
self.assertEqual(path.read_text(), '{}\n')
# The old validation has completed; a writer changes the same inode.
path.write_text('{"credsStore":"synthetic"}')
self.assertEqual(path.stat().st_ino, before.st_ino)
self.assertEqual(path.read_text(), '{"credsStore":"synthetic"}')
def test_uncertain_network_and_container_replies_recover_full_id(self):
from types import SimpleNamespace
for kind in ('network', 'container'):
for reply in (b'', b'not-an-id', b'A' * 64, b'\xff'):
with self.subTest(kind=kind, reply=reply):
identity = 'a' * 64
labels = {'owner':'sdk-rust','task':'logger227','qualification_nonce':'synthetic-nonce'}
present = {identity}; operations = []
def fake(args, **kwargs):
operations.append(args)
if 'inspect' in args:
selected = args[-1]
if selected not in present:
message = ('No such container: '+selected if kind=='container' else 'network '+selected+' not found').encode()
return SimpleNamespace(returncode=1, stderr=message, stdout=b'[]\n')
value={'Id':selected,'Name':'owned','Config':{'Labels':labels},'Labels':labels,'State':{'OOMKilled':False}}
return SimpleNamespace(returncode=0, stderr=b'', stdout=ns['json'].dumps([value]).encode())
if 'rm' in args:
self.assertEqual(args[-1], identity); present.remove(identity)
return SimpleNamespace(returncode=0, stderr=b'', stdout=identity.encode())
if 'id='+identity in args:
return SimpleNamespace(returncode=0, stderr=b'', stdout=(identity.encode() if present else b''))
for key,val in labels.items():self.assertIn('label='+key+'='+val, args)
self.assertIn('--no-trunc', args)
return SimpleNamespace(returncode=0, stderr=b'', stdout=(identity+'\n').encode())
old_identity = reply.decode('ascii', errors='replace').strip()
self.assertIsNotNone(old_identity) # Old is-None recovery skips every reply.
granted = None
with self.assertRaises((AssertionError,UnicodeError)):
granted = ns['full_id'](reply)
self.assertIsNone(granted)
with mock.patch.dict(ns, {'labels':labels,'call':fake}):
recovered=ns['reconcile_created'](kind,granted)
self.assertEqual(recovered,identity)
ns['remove_resource'](kind,recovered)
self.assertEqual(present,set())
self.assertIn('id='+identity,operations[-1])
def test_configuration_mutations_are_refused(self):
image = {'Id': 'synthetic-id', 'Config': {'Env': ['PATH=/usr/bin'], 'Labels': {}, 'Volumes': {'/app/database': {}, '/app/storage': {}}}}
command = ['-c', 'captured']; tmpfs = {'/tmp': 'size=64m'}
config = {'Env': ['PATH=/usr/bin', 'DB_CONNECTION=sqlite', 'DB_DATABASE=/app/database/runtime.sqlite', 'DW_RUST_MAINTENANCE_LOOP=1'],
'Image':'synthetic-image','Entrypoint':['/bin/sh'],'Cmd':command,'Hostname':'owned','User':'1000:1000','WorkingDir':'/app','StopSignal':'SIGTERM',
'Volumes': image['Config']['Volumes'],'ExposedPorts':{'8080/tcp':{}},'Labels':{'owner':'sdk-rust'},'Tty':False,'OpenStdin':False}
host = {'ReadonlyRootfs':True,'Init':True,'Privileged':False,'NanoCpus':10**9,'Memory':1024**3,'MemorySwap':1024**3,'PidsLimit':256,
'CapAdd':None,'CapDrop':['ALL'],'SecurityOpt':['no-new-privileges'],'NetworkMode':'network-id','Binds':None,'Tmpfs':tmpfs,
'PortBindings':{'8080/tcp':[{'HostIp':'127.0.0.1','HostPort':''}]},'PidMode':'','UTSMode':'','UsernsMode':'','IpcMode':'private',
'Devices':[],'DeviceRequests':None,'VolumesFrom':None,'ExtraHosts':None,'AutoRemove':False,'OomKillDisable':False,'RestartPolicy':{'Name':'no','MaximumRetryCount':0}}
value={'Image':'synthetic-id','Config':config,'HostConfig':host,'Mounts':[]}
check=lambda v: ns['validate_container'](v,image,'network-id','owned',command,{'owner':'sdk-rust'},tmpfs)
check(value)
started=copy.deepcopy(value);started['HostConfig']['OomKillDisable']=None
self.assertEqual(check(value),check(started))
mutations=[('Image',None,'wrong'),('Config','Cmd',['wrong']),('Config','Env',config['Env']+['INJECTED=yes']),('Config','Labels',{}),
('HostConfig','Privileged',True),('HostConfig','NanoCpus',0),('HostConfig','Memory',0),('HostConfig','ReadonlyRootfs',False),
('HostConfig','Binds',['synthetic:/app']),('HostConfig','NetworkMode','host'),('HostConfig','SecurityOpt',[]),('HostConfig','OomKillDisable',True),('Mounts',None,[{'Type':'bind','Destination':'/app'}])]
for group,key,new in mutations:
with self.subTest(field=(group,key)):
changed=copy.deepcopy(value)
if key is None:changed[group]=new
else:changed[group][key]=new
with self.assertRaises(AssertionError):check(changed)
def test_network_profile_refuses_wrong_routing_or_attachment(self):
value={'Driver':'bridge','Scope':'local','Internal':False,'Ingress':False,'Attachable':False,'EnableIPv6':False,
'Options':{'com.docker.network.bridge.enable_ip_masquerade':'false','com.docker.network.bridge.enable_icc':'false','com.docker.network.bridge.host_binding_ipv4':'127.0.0.1'},'Containers':{}}
ns['validate_network'](value)
value['Containers']={'owned-runtime':{}}
ns['validate_network'](value,'owned-runtime')
for key,bad in [('Internal',True),('Driver','host'),('Attachable',True),('Containers',{'foreign':{}}),('Options',{})]:
with self.subTest(field=key):
changed=copy.deepcopy(value);changed[key]=bad
with self.assertRaises(AssertionError):ns['validate_network'](changed,'owned-runtime')
def records(self):
records=[]
for index,(case,delays) in enumerate([('short',[1,2,2,2]),('empty',[0,0,0,0])],1):
workflow='rust-retry-'+case+'-00000000-0000-0000-0000-'+str(index).zfill(12)
run='run-'+case
retries=[{'sequence':1,'activity_execution_id':'execution-'+case,'execution_mode':'local',
'local_activity':True,'retry_after_attempt':attempt,'retry_backoff_seconds':delay}
for attempt,delay in enumerate(delays,1)]
records.append({'case':case,'workflow_id':workflow,'run_id':run,'expected_backoff_seconds':delays,
'callback_gap_ms':[delay*1000 for delay in delays],'retry_reports':retries,'callback_attempts':5,
'callbacks_after_replay':0,'history':{'history_complete':True,'workflow':{
'instance_id':workflow,'run_id':run,'details_state':'retained'},
'history_events':[{'type':'ActivityRetryScheduled','payload':row} for row in retries]}})
return records
def test_observation_accepts_only_both_bound_cases(self):
ns['validate_records'](self.records())
for change in ('early','zero-tail','remote','incomplete','extra-callback','duplicate-run','missing-zero'):
with self.subTest(change=change):
records=self.records()
if change=='early':records[0]['callback_gap_ms'][2]=0
if change=='zero-tail':records[0]['retry_reports'][3]['retry_backoff_seconds']=0
if change=='remote':records[0]['retry_reports'][0]['execution_mode']='remote'
if change=='incomplete':records[0]['history']['history_complete']=False
if change=='extra-callback':records[0]['callbacks_after_replay']=1
if change=='duplicate-run':records[1]['run_id']=records[0]['run_id']
if change=='missing-zero':records[1]['retry_reports'][0]['retry_backoff_seconds']=None
with self.assertRaises((AssertionError,KeyError)):ns['validate_records'](records)
def test_startup_guard_is_before_root_access(self):
guard=next(n for n in module.body if isinstance(n,ast.If) and 'unsafe caller' in ast.unparse(n))
root=next(n for n in module.body if isinstance(n,ast.Assign) and any(isinstance(t,ast.Name) and t.id=='ROOT' for t in n.targets))
self.assertLess(guard.lineno,root.lineno)
from types import SimpleNamespace
for opt,site,bytecode in [(1,1,True),(0,0,True),(0,1,False)]:
fake=SimpleNamespace(flags=SimpleNamespace(optimize=opt,no_site=site),dont_write_bytecode=bytecode)
with self.assertRaises(RuntimeError):exec(compile(ast.Module(body=[guard],type_ignores=[]),'guard','exec'),{'sys':fake})
def test_deadline_recovers_committed_unknown_create(self):
for pending in ('container','network'):
present={pending:'a'*64};removed=[];result={'outcome':'fail'}
def recover(kind,identity):return identity if identity is not None else present.get(kind)
def remove(kind,identity):
self.assertEqual(present.pop(kind),identity);removed.append(identity);return {'State':{'OOMKilled':False}}
with mock.patch.dict(ns,{'reconcile_created':recover,'remove_resource':remove}):
try:ns['workload_deadline'](signal.SIGTERM,None)
except ns['BudgetExpired']:errors=ns['cleanup_resources'](None,None,result,pending,seconds=1)
self.assertEqual(errors,[]);self.assertEqual(present,{});self.assertEqual(removed,['a'*64])
def test_unresolved_create_is_nonpass_and_cleanup_calls_are_budgeted(self):
seen=[]
def call(args,wall=30,**kwargs):seen.append(wall)
def recover(kind,identity):ns['call'](['inert-query']);return None
with mock.patch.dict(ns,{'call':call,'reconcile_created':recover}):
errors=ns['cleanup_resources'](None,None,{},'container',seconds=.5)
self.assertIs(ns['call'],call)
self.assertEqual(errors,['container:uncertain-create'])
self.assertTrue(seen);self.assertTrue(all(0<wall<=.5 for wall in seen))
def test_literal_bootstrap_refuses_substitution_before_execution(self):
bootstrap = ast.parse(SOURCE.with_name('219-caller-bootstrap.py').read_bytes())
function = next(node for node in bootstrap.body if isinstance(node, ast.FunctionDef) and node.name == 'capture')
scope = {'os': os, 'pathlib': pathlib, 'stat': stat, 'hashlib': hashlib}
exec(compile(ast.Module(body=[function], type_ignores=[]), '<capture-control>', 'exec'), scope)
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp)
source = root / 'caller.py'
marker = []
original = b'marker.append("original")'
source.write_bytes(original)
expected = hashlib.sha256(original).hexdigest()
captured = scope['capture'](str(source), expected)
source.write_bytes(b'marker.append("replacement")')
exec(compile(source.read_bytes(), '<pathname-predecessor>', 'exec'), {'marker': marker})
self.assertEqual(marker, ['replacement'])
marker.clear()
with self.assertRaisesRegex(RuntimeError, 'source hash refused'):
scope['capture'](str(source), expected)
self.assertEqual(marker, [])
exec(compile(captured, '<captured-candidate>', 'exec'), {'marker': marker})
self.assertEqual(marker, ['original'])
source.unlink()
target = root / 'target.py'; target.write_bytes(original)
source.symlink_to(target)
with self.assertRaises(OSError):
scope['capture'](str(source), expected)
unittest.main() |
Member
Author
|
Security static caller/profile review for the frozen shared-#219 packet ( The registry/package/build receipt, sealed executable, captured supervisor and literal-bootstrap bindings are coherent. The retry observation itself correctly distinguishes the committed [1,2,2,2] reports from callback gaps, checks the explicit-zero case, and requires replacement-Worker replay without callbacks. Exact labels/full-ID reconciliation, closed Docker config, synthetic credentials and the cleanup reserve are also appropriately scoped. Two caller boundaries remain open:
No Server, SDK, SQL, credential, publication, or other set-aside item was run in this review. The merged #275 source and its main checks remain accepted in their existing source scope; this HOLD applies only to the proposed private caller/profile and authorizes no observation. Return one frozen successor packet and inert countercontrols before any effect. |
Member
Author
|
Frozen shared #219 caller successor for Security HOLD6871331364. No timing observation or new Docker object has run. The complete Config/HostConfig/Mounts projection now equals one admitted profile: unknown, omitted and changed fields refuse. The fixed Docker29/runc profile comes from the retained accepted 4.17 registry-logging config on this same engine and published 3.10 image. Private cgroup namespace, runc runtime and bounded local logging are explicit create options. Only the documented create/start OomKillDisable false/null default is normalized. Stable CgroupnsMode, GroupAdd, DNS, Sysctls, Ulimits, masked/readonly paths, syslog, PublishAllPorts, OomScoreAdj and CgroupParent mutations accepted by the predecessor now refuse. Subprocess stdout/stderr share a byte budget while selector reads occur. Docker and consumer output have a 1 MiB combined limit; runtime logs have 128 KiB; cluster HTTP bodies have 64 KiB. A read never requests more than the remaining budget plus one overflow byte. Overflow or timeout kills/reaps the owned child within a three-second wait, closes its pipes and propagates into the existing cleanup reserve. Log overflow is recorded but does not skip resource removal. The original 180-second workload, 90-second consumer, 120-second cleanup and 340+20-second outer bounds remain. Package/build/test/binary/producer and scenario are unchanged. All 19 inert controls pass. They include the complete predecessor/candidate isolation-field comparison, byte-bound stdout/stderr/combined streams, timeout kill/reap, bounded HTTP-body reads, and cleanup continuation on overflow, alongside the original ownership/config/bootstrap controls. Test subprocesses are literal language-only Python controls; no Docker call, SDK/Server entrypoint, SQL or installed timing case ran. The separate controls comment carries the exact source and hash in this packet. The old HOLD and original packet remain preserved. The bootstrap is still supplied as exact literal Frozen successor: {
"controller_sha256": "5b762bda9f8e369ea9cacc43ad890affd9b33046ecc11a1e7bdff12744a5e8e8",
"controls_sha256": "017cc34b46ad6332f314d6018a4290be0c9255c87475d005f695548ec0ea033a",
"literal_bootstrap_sha256": "1f1b15405b50245a612c4a4f310ab25dd69bea18e5ce3957994a288831e3d393",
"build_receipt_sha256": "9fff98616c7d87b937ea0d843c8c8ba9cef3271049a884abd49a2aff793bf5b7",
"test_sha256": "f95e947b2b3c199bb929e0f7176fb712d3b0f235612c97cbbae859df6cdaa862",
"binary_sha256": "5cd20d42dc317f6b49637595799af611bcd756a53899e81d1dccb37e9649590a",
"qualified_server": "3.10.0",
"server_source": "b6e04ce7e679464e91f6ceae83310d3bb3204092",
"server_child": "sha256:9d64e9aeee84b46d21715826a3e7e5a96eb8d4ce7fccb126b88b15596ab14973",
"scope": "one ordinary219 local-retry timing observation; no clock/redrive/database operation",
"source_controls": 19,
"workload_seconds": 180,
"parent_seconds": 90,
"cleanup_seconds": 120,
"parent_wait_seconds": 10,
"outer_seconds": 340,
"outer_kill_seconds": 20,
"container_profile": "complete Config/HostConfig/Mounts equality; OomKillDisable false/null only normalized",
"stream_byte_limits": {
"docker_stdout_stderr_combined": 1048576,
"runtime_logs_combined": 131072,
"consumer_stdout_stderr_combined": 1048576,
"cluster_http_body": 65536
}
}Exact controller (effectful top-level: static review only; never import/run during review): """One fixed ordinary local-retry observation for shared219; separate exact review required."""
import base64
import fcntl
import hashlib
import json
import os
import pathlib
import resource
import re
import secrets
import selectors
import signal
import stat
import subprocess
import sys
import time
import urllib.error
import urllib.request
if sys.flags.optimize != 0 or sys.flags.no_site != 1 or sys.dont_write_bytecode is not True:
raise RuntimeError("unsafe caller Python startup")
assert len(sys.argv) == 3
class BudgetExpired(RuntimeError):
pass
def workload_deadline(signum, frame):
raise BudgetExpired("ordinary retry observation deadline")
def cleanup_signals():
previous = signal.pthread_sigmask(signal.SIG_BLOCK, {signal.SIGALRM, signal.SIGTERM})
signal.alarm(0)
signal.signal(signal.SIGALRM, signal.SIG_IGN)
signal.signal(signal.SIGTERM, signal.SIG_IGN)
signal.pthread_sigmask(signal.SIG_SETMASK, previous)
ROOT = pathlib.Path(sys.argv[1]).absolute()
SOURCE = pathlib.Path(sys.argv[2]).absolute()
IMAGE = 'ghcr.io/durable-workflow/server@sha256:9d64e9aeee84b46d21715826a3e7e5a96eb8d4ce7fccb126b88b15596ab14973'
REVISION = 'b6e04ce7e679464e91f6ceae83310d3bb3204092'
RECEIPT = '9fff98616c7d87b937ea0d843c8c8ba9cef3271049a884abd49a2aff793bf5b7'
BINARY = '5cd20d42dc317f6b49637595799af611bcd756a53899e81d1dccb37e9649590a'
TEST = 'f95e947b2b3c199bb929e0f7176fb712d3b0f235612c97cbbae859df6cdaa862'
def digest(data):
return hashlib.sha256(data).hexdigest()
def directory(path):
assert path.is_absolute() and '..' not in path.parts
current = os.open('/', os.O_RDONLY | os.O_DIRECTORY)
try:
for part in path.parts[1:]:
following = os.open(part, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW, dir_fd=current)
os.close(current)
current = following
return current
except BaseException:
os.close(current)
raise
def read_at(root, relative):
parts = pathlib.PurePosixPath(relative).parts
assert parts and not relative.startswith('/') and '..' not in parts
parent = os.dup(root)
try:
for part in parts[:-1]:
following = os.open(part, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW, dir_fd=parent)
os.close(parent)
parent = following
fd = os.open(parts[-1], os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK, dir_fd=parent)
try:
info = os.fstat(fd)
assert stat.S_ISREG(info.st_mode) and info.st_nlink == 1 and info.st_uid == os.getuid()
assert info.st_size <= 64 * 1024**2
value = os.pread(fd, info.st_size + 1, 0)
after = os.fstat(fd)
fields = ('st_dev', 'st_ino', 'st_size', 'st_uid', 'st_gid', 'st_mode',
'st_nlink', 'st_mtime_ns', 'st_ctime_ns')
assert len(value) == info.st_size
assert all(getattr(info, field) == getattr(after, field) for field in fields)
return value
finally:
os.close(fd)
finally:
os.close(parent)
def exclusive(root, name):
assert '/' not in name and name not in ('', '.', '..')
return os.open(name, os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW,
0o600, dir_fd=root)
def output(root, fd, name, value):
before = os.fstat(fd)
named = os.stat(name, dir_fd=root, follow_symlinks=False)
assert (named.st_dev, named.st_ino) == (before.st_dev, before.st_ino)
assert stat.S_ISREG(named.st_mode) and stat.S_IMODE(named.st_mode) == 0o600
assert named.st_uid == os.getuid() and named.st_nlink == 1
with os.fdopen(os.dup(fd), 'wb') as handle:
handle.write(value)
handle.flush()
after = os.stat(name, dir_fd=root, follow_symlinks=False)
assert (after.st_dev, after.st_ino, after.st_nlink) == (before.st_dev, before.st_ino, 1)
root_fd = directory(ROOT)
root_info = os.fstat(root_fd)
assert root_info.st_uid == os.getuid() and stat.S_IMODE(root_info.st_mode) == 0o700
receipt_bytes = read_at(root_fd, 'build-receipt.json')
assert digest(receipt_bytes) == RECEIPT
receipt = json.loads(receipt_bytes)
assert receipt['executable_sha256'] == BINARY
assert digest(read_at(root_fd, 'fixture/tests/local_retry_timing_server.rs')) == TEST
assert digest(read_at(root_fd, 'fixture/Cargo.lock')) == receipt['lock_sha256']
assert digest(read_at(root_fd, 'fixture/Cargo.toml')) == receipt['manifest_sha256']
assert digest(read_at(root_fd, 'fixture/build.jsonl')) == receipt['build_messages_sha256']
binary_bytes = read_at(root_fd, receipt['executable_relative'])
assert len(binary_bytes) == receipt['executable_bytes']
assert digest(binary_bytes) == BINARY
fd = os.memfd_create('reviewed-local-retry', os.MFD_ALLOW_SEALING)
with os.fdopen(os.dup(fd), 'wb') as image_file:
image_file.write(binary_bytes)
os.fchmod(fd, 0o500)
seals = fcntl.F_SEAL_WRITE | fcntl.F_SEAL_GROW | fcntl.F_SEAL_SHRINK | fcntl.F_SEAL_SEAL
fcntl.fcntl(fd, fcntl.F_ADD_SEALS, seals)
assert fcntl.fcntl(fd, fcntl.F_GET_SEALS) == seals
assert digest(os.pread(fd, len(binary_bytes) + 1, 0)) == BINARY
supervisor = SOURCE / 'scripts/acceptance/installed-native/cell-supervisor.sh'
source_fd = directory(SOURCE)
supervisor_bytes = read_at(source_fd, 'scripts/acceptance/installed-native/cell-supervisor.sh')
os.close(source_fd)
assert digest(supervisor_bytes) == '25a591f514b3823093a37b66c47c588cba8e983594c8da2d7940ac4f565b922b'
def runtime_command(script, deadline):
script = script.decode()
original = ' /bin/sh "$0" --command "$@"'
assert script.count(original) == 1
wrapper = 'supervisor=$1\nshift\n' + script.replace(original,
' /bin/sh -c "$supervisor" supervisor --command "$@"')
return ['-c', wrapper, 'supervisor', script, str(deadline), '10', 'both',
'/usr/local/bin/durable-workflow-server']
# procfs fdinfo admits only kernel-created decimal descriptor names. A same-UID
# writer cannot create config.json, contexts or plugin paths in this namespace.
config_path = '/proc/' + str(os.getpid()) + '/fdinfo'
config_fd = os.open(config_path, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
config_info = os.fstat(config_fd)
evidence = {name: exclusive(root_fd, name) for name in
('retry.stdout', 'retry.stderr', 'runtime.stdout', 'runtime.stderr', 'retry-result.json',
'container-created.json', 'container-started.json', 'container-returned.json',
'network-created.json', 'network-returned.json')}
CLOSED_ENV = {'PATH': '/usr/bin:/bin', 'HOME': '/nonexistent', 'LANG': 'C', 'LC_ALL': 'C'}
def docker_command(args, config):
assert args[0] == 'docker'
return ['/usr/bin/docker', '--host', 'unix:///run/docker.sock', '--config',
'/proc/self/fd/' + str(config)] + args[1:]
class OutputOverflow(RuntimeError):
pass
def stop_process(process):
if process.poll() is None:
try: os.killpg(process.pid, signal.SIGKILL)
except ProcessLookupError: pass
process.wait(timeout=3)
def read_process(process, wall, byte_limit):
assert 0 < byte_limit <= 1024**2 and wall > 0
stop = time.monotonic() + wall
selector = selectors.DefaultSelector()
output_bytes = [bytearray(), bytearray()]
complete = False
try:
for index, stream in enumerate((process.stdout, process.stderr)):
os.set_blocking(stream.fileno(), False)
selector.register(stream, selectors.EVENT_READ, index)
total = 0
while selector.get_map():
remaining = stop - time.monotonic()
if remaining <= 0:
raise subprocess.TimeoutExpired(process.args, wall)
for selected, _ in selector.select(min(remaining, .2)):
chunk = os.read(selected.fileobj.fileno(), min(65536, byte_limit + 1 - total))
if not chunk:
selector.unregister(selected.fileobj)
continue
total += len(chunk)
if total > byte_limit:
raise OutputOverflow('subprocess output byte limit')
output_bytes[selected.data].extend(chunk)
remaining = stop - time.monotonic()
if remaining <= 0:
raise subprocess.TimeoutExpired(process.args, wall)
process.wait(timeout=remaining)
complete = True
return bytes(output_bytes[0]), bytes(output_bytes[1])
finally:
selector.close()
for stream in (process.stdout, process.stderr):
stream.close()
if not complete:
stop_process(process)
def bounded_run(args, *, wall, byte_limit=1024**2, check=True, input=None, **kwargs):
assert input is None or isinstance(input, bytes) and len(input) <= 4096
process = subprocess.Popen(args, stdin=subprocess.PIPE if input is not None else subprocess.DEVNULL,
stdout=subprocess.PIPE, stderr=subprocess.PIPE, start_new_session=True, **kwargs)
try:
if input is not None:
os.set_blocking(process.stdin.fileno(), False)
assert os.write(process.stdin.fileno(), input) == len(input)
process.stdin.close()
stdout, stderr = read_process(process, wall, byte_limit)
if check and process.returncode:
raise subprocess.CalledProcessError(process.returncode, args, stdout, stderr)
return subprocess.CompletedProcess(args, process.returncode, stdout, stderr)
finally:
if process.stdin is not None and not process.stdin.closed:
process.stdin.close()
if process.poll() is None:
stop_process(process)
for stream in (process.stdout, process.stderr):
if not stream.closed:
stream.close()
def bounded_http_json(response, byte_limit=65536):
assert 0 < byte_limit <= 65536
result = bytearray()
while True:
chunk = response.read(min(8192, byte_limit + 1 - len(result)))
if not chunk:
return json.loads(result)
result.extend(chunk)
if len(result) > byte_limit:
raise OutputOverflow('HTTP body byte limit')
def call(args, wall=30, byte_limit=1024**2, **kwargs):
tool = os.stat('/usr/bin/docker', follow_symlinks=False)
assert stat.S_ISREG(tool.st_mode) and tool.st_uid == 0 and tool.st_nlink == 1
assert stat.S_IMODE(tool.st_mode) == 0o755
assert digest(pathlib.Path('/usr/bin/docker').read_bytes()) == '40cdaf7fd0f21089dd9e15b0c3a7dd7f2399027f010e366dac6304ae0615954a'
assert all(name.isdecimal() for name in os.listdir(config_fd))
config_named = os.stat(config_path, follow_symlinks=False)
config_open = os.fstat(config_fd)
assert (config_named.st_dev, config_named.st_ino) == (config_info.st_dev, config_info.st_ino)
assert (config_open.st_dev, config_open.st_ino) == (config_info.st_dev, config_info.st_ino)
kwargs.setdefault('check', True)
return bounded_run(docker_command(args, config_fd), wall=wall, byte_limit=byte_limit,
cwd='/', env=CLOSED_ENV, pass_fds=(config_fd,), **kwargs)
socket_info = os.stat('/run/docker.sock', follow_symlinks=False)
assert stat.S_ISSOCK(socket_info.st_mode) and socket_info.st_uid == 0
def engine():
current = os.stat('/run/docker.sock', follow_symlinks=False)
assert (current.st_dev, current.st_ino, current.st_uid, current.st_gid, current.st_mode) == (
socket_info.st_dev, socket_info.st_ino, socket_info.st_uid, socket_info.st_gid, socket_info.st_mode)
identity = call(['docker', 'info', '--format', '{{.ID}}']).stdout.strip()
assert digest(identity) == '0a179ff32531e41cfc2d465b1d46afc1df91c0975e9627969f75c67e00b0e892'
def image_identity():
image = json.loads(call(['docker', 'image', 'inspect', IMAGE]).stdout)[0]
assert image['Id'] == IMAGE.split('@')[1]
assert image['RepoDigests'] == [IMAGE] and image['Descriptor']['digest'] == IMAGE.split('@')[1]
assert image['Os'] == 'linux' and image['Architecture'] == 'amd64'
assert digest(json.dumps(image['Config'], sort_keys=True, separators=(',', ':')).encode()) == 'c88782b528aee2a1b7804cd62cb95afac9df8c5b2f89c471924c6c1085c63b62'
return image
engine()
image = image_identity()
assert image['Os'] == 'linux' and image['Architecture'] == 'amd64'
assert image['Config']['Labels']['org.opencontainers.image.revision'] == REVISION
assert image['Config']['Labels']['dev.durable-workflow.release.tag'] == '3.10.0'
assert image['Config']['Entrypoint'] == ['/usr/local/bin/durable-workflow-server']
nonce = secrets.token_hex(16)
name = 'sdk-rust-219-timing-' + nonce[:12]
labels = {'owner': 'sdk-rust', 'task': 'retry219', 'qualification_nonce': nonce}
label_args = [argument for key, value in labels.items() for argument in ('--label', key + '=' + value)]
token = 'fixture-' + secrets.token_hex(24)
key = 'base64:' + base64.b64encode(secrets.token_bytes(32)).decode()
container_id = network_id = None
process = None
pending_create = None
result = {'outcome': 'fail', 'build_receipt_sha256': RECEIPT,
'binary_sha256': BINARY, 'server_manifest': IMAGE, 'server_source': REVISION,
'started_at': time.time(), 'limits': {'consumer_cpus': 2, 'consumer_as': 2147483648,
'consumer_cpu_seconds': 60, 'consumer_wall_seconds': 90}}
def owned(kind, identity):
value = json.loads(call(['docker', kind, 'inspect', identity]).stdout)[0]
actual = value['Config']['Labels'] if kind == 'container' else value['Labels']
assert value['Id'] == identity and all(actual.get(k) == v for k, v in labels.items())
return value
def full_id(raw):
identity = raw.decode('ascii').strip()
assert re.fullmatch('[0-9a-f]{64}', identity), 'uncertain create reply'
return identity
def reconcile_created(kind, identity):
if identity is not None:
assert re.fullmatch('[0-9a-f]{64}', identity)
return identity
filters = [arg for key, val in labels.items() for arg in ('--filter', 'label=' + key + '=' + val)]
query = ['docker', 'ps', '-aq', '--no-trunc'] if kind == 'container' else ['docker', 'network', 'ls', '-q', '--no-trunc']
candidates = call(query + filters).stdout.split()
assert len(candidates) <= 1
if candidates:
recovered = full_id(candidates[0])
value = owned(kind, recovered)
assert value['Name'].lstrip('/') == name
return recovered
return None
def remove_resource(kind, identity):
value = owned(kind, identity)
call(['docker', 'rm', '-f', '-v', identity] if kind == 'container'
else ['docker', 'network', 'rm', identity])
observed = call(['docker', kind, 'inspect', identity], check=False)
missing = ('no such container: ' + identity if kind == 'container'
else 'network ' + identity + ' not found').encode()
assert observed.returncode != 0 and json.loads(observed.stdout) == []
assert missing in observed.stderr.lower()
query = ['docker', 'ps', '-aq', '--no-trunc'] if kind == 'container' else ['docker', 'network', 'ls', '-q', '--no-trunc']
assert call(query + ['--filter', 'id=' + identity]).stdout.strip() == b''
return value
class NoRedirect(urllib.request.HTTPRedirectHandler):
def redirect_request(self, *args):
raise AssertionError('redirect refused')
def consumer_limits():
os.sched_setaffinity(0, sorted(os.sched_getaffinity(0))[:2])
resource.setrlimit(resource.RLIMIT_AS, (2 * 1024**3, 2 * 1024**3))
resource.setrlimit(resource.RLIMIT_CPU, (60, 60))
resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
def validate_network(value, runtime=None):
assert value['Driver'] == 'bridge' and value['Scope'] == 'local'
assert value['Internal'] is False and value['Ingress'] is False and value['Attachable'] is False
assert value['EnableIPv6'] is False
assert value['Options'] == {
'com.docker.network.bridge.enable_ip_masquerade': 'false',
'com.docker.network.bridge.enable_icc': 'false',
'com.docker.network.bridge.host_binding_ipv4': '127.0.0.1'}
assert set(value.get('Containers') or {}) <= ({runtime} if runtime else set())
def admitted_profile(image, network, name, command, labels, tmpfs):
config = {
'Hostname': name, 'Domainname': '', 'User': '1000:1000',
'AttachStdin': False, 'AttachStdout': True, 'AttachStderr': True,
'ExposedPorts': {'8080/tcp': {}}, 'Tty': False, 'OpenStdin': False,
'StdinOnce': False, 'Env': image['Config']['Env'] + [
'DB_CONNECTION=sqlite', 'DB_DATABASE=/app/database/runtime.sqlite',
'DW_RUST_MAINTENANCE_LOOP=1'],
'Cmd': command, 'Image': IMAGE, 'Volumes': image['Config']['Volumes'],
'WorkingDir': '/app', 'Entrypoint': ['/bin/sh'],
'Labels': dict(image['Config']['Labels'], **labels), 'StopSignal': 'SIGTERM'}
# Complete Docker29/runc profile, frozen from the accepted same-engine
# 3.10 logging observation. Unknown, omitted and changed fields all refuse.
host = {'Binds': None,
'ContainerIDFile': '',
'LogConfig': {'Type': 'local', 'Config': {'max-file': '3', 'max-size': '20m'}},
'PortBindings': {'8080/tcp': [{'HostIp': '127.0.0.1', 'HostPort': ''}]},
'RestartPolicy': {'Name': 'no', 'MaximumRetryCount': 0},
'AutoRemove': False,
'VolumeDriver': '',
'VolumesFrom': None,
'ConsoleSize': [0, 0],
'CapAdd': None,
'CapDrop': ['ALL'],
'CgroupnsMode': 'private',
'Dns': None,
'DnsOptions': [],
'DnsSearch': [],
'ExtraHosts': None,
'GroupAdd': None,
'IpcMode': 'private',
'Cgroup': '',
'Links': None,
'OomScoreAdj': 0,
'PidMode': '',
'Privileged': False,
'PublishAllPorts': False,
'ReadonlyRootfs': True,
'SecurityOpt': ['no-new-privileges'],
'UTSMode': '',
'UsernsMode': '',
'ShmSize': 67108864,
'Runtime': 'runc',
'Isolation': '',
'CpuShares': 0,
'Memory': 1073741824,
'NanoCpus': 1000000000,
'CgroupParent': '',
'BlkioWeight': 0,
'BlkioWeightDevice': [],
'BlkioDeviceReadBps': [],
'BlkioDeviceWriteBps': [],
'BlkioDeviceReadIOps': [],
'BlkioDeviceWriteIOps': [],
'CpuPeriod': 0,
'CpuQuota': 0,
'CpuRealtimePeriod': 0,
'CpuRealtimeRuntime': 0,
'CpusetCpus': '',
'CpusetMems': '',
'Devices': [],
'DeviceCgroupRules': None,
'DeviceRequests': None,
'MemoryReservation': 0,
'MemorySwap': 1073741824,
'MemorySwappiness': None,
'OomKillDisable': False,
'PidsLimit': 256,
'Ulimits': [],
'CpuCount': 0,
'CpuPercent': 0,
'IOMaximumIOps': 0,
'IOMaximumBandwidth': 0,
'MaskedPaths': ['/proc/acpi',
'/proc/asound',
'/proc/interrupts',
'/proc/kcore',
'/proc/keys',
'/proc/latency_stats',
'/proc/sched_debug',
'/proc/scsi',
'/proc/timer_list',
'/proc/timer_stats',
'/sys/devices/virtual/powercap',
'/sys/firmware',
'/sys/devices/system/cpu/cpu0/thermal_throttle',
'/sys/devices/system/cpu/cpu1/thermal_throttle',
'/sys/devices/system/cpu/cpu2/thermal_throttle',
'/sys/devices/system/cpu/cpu3/thermal_throttle',
'/sys/devices/system/cpu/cpu4/thermal_throttle',
'/sys/devices/system/cpu/cpu5/thermal_throttle',
'/sys/devices/system/cpu/cpu6/thermal_throttle',
'/sys/devices/system/cpu/cpu7/thermal_throttle',
'/sys/devices/system/cpu/cpu8/thermal_throttle',
'/sys/devices/system/cpu/cpu9/thermal_throttle',
'/sys/devices/system/cpu/cpu10/thermal_throttle',
'/sys/devices/system/cpu/cpu11/thermal_throttle',
'/sys/devices/system/cpu/cpu12/thermal_throttle',
'/sys/devices/system/cpu/cpu13/thermal_throttle',
'/sys/devices/system/cpu/cpu14/thermal_throttle',
'/sys/devices/system/cpu/cpu15/thermal_throttle',
'/sys/devices/system/cpu/cpu16/thermal_throttle',
'/sys/devices/system/cpu/cpu17/thermal_throttle',
'/sys/devices/system/cpu/cpu18/thermal_throttle',
'/sys/devices/system/cpu/cpu19/thermal_throttle'],
'ReadonlyPaths': ['/proc/bus', '/proc/fs', '/proc/irq', '/proc/sys', '/proc/sysrq-trigger'],
'Init': True}
host.update(NetworkMode=network, Tmpfs=tmpfs)
return {'Image': image['Id'], 'Config': config, 'HostConfig': host, 'Mounts': []}
def validate_container(value, image, network, name, command, labels, tmpfs):
packet = json.loads(json.dumps({k: value[k] for k in ('Image', 'Config', 'HostConfig', 'Mounts')}))
# Docker29 changes this single unset/false default during start. Both retain
# the kernel OOM killer; true and all other profile differences refuse.
assert 'OomKillDisable' in packet['HostConfig']
assert packet['HostConfig']['OomKillDisable'] in (False, None)
packet['HostConfig']['OomKillDisable'] = False
assert packet == admitted_profile(image, network, name, command, labels, tmpfs)
return digest(json.dumps(packet, sort_keys=True, separators=(',', ':')).encode())
def validate_records(records):
assert type(records) is list and len(records) == 2
workflows, runs = set(), set()
for record, (case, expected) in zip(records, [('short', [1, 2, 2, 2]), ('empty', [0, 0, 0, 0])], strict=True):
assert set(record) == {'case','workflow_id','run_id','expected_backoff_seconds','callback_gap_ms',
'retry_reports','callback_attempts','callbacks_after_replay','history'}
assert record['case'] == case and record['expected_backoff_seconds'] == expected
workflow, run = record['workflow_id'], record['run_id']
assert isinstance(workflow,str) and re.fullmatch('rust-retry-'+case+'-[a-f0-9-]{36}', workflow)
assert isinstance(run,str) and re.fullmatch('[A-Za-z0-9_-]{1,128}',run)
assert workflow not in workflows and run not in runs
workflows.add(workflow);runs.add(run)
assert type(record['callback_attempts']) is int and record['callback_attempts'] == 5
assert type(record['callbacks_after_replay']) is int and record['callbacks_after_replay'] == 0
gaps = record['callback_gap_ms']
assert type(gaps) is list and len(gaps) == 4
assert all(type(gap) is int and delay*1000 <= gap < 35000 for gap,delay in zip(gaps,expected,strict=True))
history = record['history']
assert history['history_complete'] is True and history['workflow']['details_state'] == 'retained'
assert history['workflow']['instance_id'] == workflow and history['workflow']['run_id'] == run
retries = [event['payload'] for event in history['history_events'] if event['type']=='ActivityRetryScheduled']
assert len(retries) == 4 and retries == record['retry_reports']
execution = retries[0]['activity_execution_id']
assert isinstance(execution,str) and execution
for attempt,(retry,delay) in enumerate(zip(retries,expected,strict=True),1):
assert retry['execution_mode']=='local' and retry['local_activity'] is True
assert retry['activity_execution_id']==execution and retry['sequence']==1
assert type(retry['retry_after_attempt']) is int and retry['retry_after_attempt']==attempt
assert type(retry['retry_backoff_seconds']) is int and retry['retry_backoff_seconds']==delay
def cleanup_resources(container_id, network_id, result, pending_create, seconds=120, collect_logs=False):
global call
original = call
stop = time.monotonic()+seconds
def limited(args, wall=30, **kwargs):
remaining=stop-time.monotonic()
if remaining<=0:raise BudgetExpired('cleanup reserve exhausted')
return original(args,wall=min(wall,remaining),**kwargs)
call=limited
errors=[]
try:
for kind,identity in [('container',container_id),('network',network_id)]:
try:
identity=reconcile_created(kind,None if pending_create==kind else identity)
if identity is None and pending_create==kind:errors.append(kind+':uncertain-create')
if identity:
result[kind+'_id']=identity
if kind=='container' and collect_logs:
try:
logs=call(['docker','logs','--tail','100',identity],check=False,wall=10,byte_limit=128*1024)
assert len(logs.stdout)+len(logs.stderr)<=128*1024
out,err=logs.stdout,logs.stderr
for secret in (token.encode(),key.encode()):
out=out.replace(secret,b'[redacted]');err=err.replace(secret,b'[redacted]')
output(root_fd,evidence['runtime.stdout'],'runtime.stdout',out)
output(root_fd,evidence['runtime.stderr'],'runtime.stderr',err)
except Exception as error:errors.append('runtime-logs:'+type(error).__name__)
value=remove_resource(kind,identity)
if kind=='container':
result['runtime_oom']=value['State']['OOMKilled']
if result['runtime_oom']:result['outcome']='fail'
except Exception as error:errors.append(kind+':'+type(error).__name__)
finally:call=original
return errors
tmpfs = {'/tmp': 'mode=1777,size=64m',
'/app/database': 'uid=1000,gid=1000,mode=0700,size=64m',
'/app/storage': 'uid=1000,gid=1000,mode=0700,size=64m',
'/credentials': 'uid=1000,gid=1000,mode=0700,size=64k,noexec,nosuid,nodev'}
command = None
created_config = None
signal.signal(signal.SIGALRM,workload_deadline)
signal.signal(signal.SIGTERM,workload_deadline)
signal.pthread_sigmask(signal.SIG_UNBLOCK,{signal.SIGALRM,signal.SIGTERM})
signal.alarm(180)
try:
engine()
image_identity()
pending_create='network'
network_id = full_id(call(['docker', 'network', 'create', '--driver', 'bridge',
'--opt', 'com.docker.network.bridge.enable_ip_masquerade=false',
'--opt', 'com.docker.network.bridge.enable_icc=false',
'--opt', 'com.docker.network.bridge.host_binding_ipv4=127.0.0.1'] + label_args + [name]).stdout)
result['network_id'] = network_id
created_network = owned('network', network_id)
pending_create=None
output(root_fd, evidence['network-created.json'], 'network-created.json',
(json.dumps(created_network, indent=2) + '\n').encode())
validate_network(created_network)
deadline = str(int(time.time()) + 240)
command = runtime_command(supervisor_bytes, deadline)
engine()
image_identity()
pending_create='container'
container_id = full_id(call(['docker', 'create', '--pull=never', '--init', '--name', name,
'--hostname', name] + label_args + [
'--cpus', '1', '--memory', '1g', '--memory-swap', '1g', '--pids-limit', '256', '--read-only', '--user', '1000:1000',
'--cap-drop', 'ALL', '--security-opt', 'no-new-privileges',
'--runtime', 'runc', '--cgroupns', 'private', '--log-driver', 'local',
'--log-opt', 'max-file=3', '--log-opt', 'max-size=20m', '--network', network_id,
'--publish', '127.0.0.1::8080', '--tmpfs', '/tmp:mode=1777,size=64m',
'--tmpfs', '/app/database:uid=1000,gid=1000,mode=0700,size=64m',
'--tmpfs', '/app/storage:uid=1000,gid=1000,mode=0700,size=64m',
'--tmpfs', '/credentials:uid=1000,gid=1000,mode=0700,size=64k,noexec,nosuid,nodev',
'--env', 'DB_CONNECTION=sqlite', '--env', 'DB_DATABASE=/app/database/runtime.sqlite',
'--env', 'DW_RUST_MAINTENANCE_LOOP=1', '--entrypoint', '/bin/sh', IMAGE] + command).stdout)
result['container_id'] = container_id
before = owned('container', container_id)
pending_create=None
output(root_fd, evidence['container-created.json'], 'container-created.json',
(json.dumps(before, indent=2) + '\n').encode())
result['container_id'], result['network_id'] = container_id, network_id
created_config = validate_container(before, image, network_id, name, command, labels, tmpfs)
validate_network(owned('network', network_id), container_id)
result['created_config_sha256'] = created_config
engine()
image_identity()
assert validate_container(owned('container', container_id), image, network_id,
name, command, labels, tmpfs) == created_config
call(['docker', 'start', container_id])
started = owned('container', container_id)
output(root_fd, evidence['container-started.json'], 'container-started.json',
(json.dumps(started, indent=2) + '\n').encode())
assert validate_container(started, image, network_id,
name, command, labels, tmpfs) == created_config
validate_network(owned('network', network_id), container_id)
call(['docker', 'exec', '-i', container_id, '/bin/sh', '-c',
'umask 077; IFS= read -r token && IFS= read -r key && '
'printf %s "$token" > /credentials/token && printf %s "$key" > /credentials/server-key && '
': > /credentials/ready.tmp && mv /credentials/ready.tmp /credentials/ready'],
input=(token + '\n' + key + '\n').encode())
live = owned('container', container_id)
ports = live['NetworkSettings']['Ports']['8080/tcp']
assert len(ports) == 1 and ports[0]['HostIp'] == '127.0.0.1'
port = int(ports[0]['HostPort'])
assert 0 < port < 65536
origin = 'http://127.0.0.1:' + str(port)
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}), NoRedirect())
for attempt in range(60):
try:
request = urllib.request.Request(origin + '/api/cluster/info',
headers={'Authorization': 'Bearer ' + token, 'X-Namespace': 'default'})
with opener.open(request, timeout=1) as response:
cluster = bounded_http_json(response)
assert cluster['version'] == '3.10.0'
result['cluster_info'] = cluster
break
except (urllib.error.URLError, ConnectionError):
time.sleep(.25)
else:
raise AssertionError('runtime readiness bound expired')
args = ['/proc/self/fd/' + str(fd), '--ignored', '--exact',
'published_local_retries_repeat_last_and_replay_committed_results',
'--nocapture', '--test-threads=1']
env = {'DURABLE_WORKFLOW_LOCAL_ISOLATED': '1', 'DURABLE_WORKFLOW_SERVER_URL': origin,
'DURABLE_WORKFLOW_AUTH_TOKEN': token, 'DURABLE_WORKFLOW_EXPECT_SERVER_VERSION': '3.10.0'}
process = subprocess.Popen(args, env=env, pass_fds=(fd,), start_new_session=True,
preexec_fn=consumer_limits, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
try:
stdout, stderr = read_process(process, 90, 1024**2)
except subprocess.TimeoutExpired:
result['observer_timeout'] = True
raise
except OutputOverflow:
result['observer_output_overflow'] = True
raise
assert len(stdout) + len(stderr) <= 1024**2
assert all(secret.encode() not in stdout + stderr for secret in (token, key))
output(root_fd, evidence['retry.stdout'], 'retry.stdout', stdout)
output(root_fd, evidence['retry.stderr'], 'retry.stderr', stderr)
result['parent_pid'], result['exit_code'] = process.pid, process.returncode
assert process.returncode == 0, 'local retry observer failed'
marker='LOCAL_RETRY_TIMING '
records=[json.loads(line[line.index(marker)+len(marker):]) for line in stdout.decode().splitlines() if marker in line]
validate_records(records)
result['observations']=records
assert digest(os.pread(fd, len(binary_bytes) + 1, 0)) == BINARY
engine()
image_identity()
returned = owned('container', container_id)
output(root_fd, evidence['container-returned.json'], 'container-returned.json',
(json.dumps(returned, indent=2) + '\n').encode())
assert validate_container(returned, image, network_id,
name, command, labels, tmpfs) == created_config
final_network = owned('network', network_id)
output(root_fd, evidence['network-returned.json'], 'network-returned.json',
(json.dumps(final_network, indent=2) + '\n').encode())
validate_network(final_network, container_id)
result['outcome'] = 'pass'
except BaseException as error:
result['error'] = type(error).__name__
raise
finally:
cleanup_signals()
parent_errors=[]
if process is not None:
try:
if process.poll() is None:
os.killpg(process.pid,signal.SIGKILL)
process.wait(timeout=10)
except ProcessLookupError:pass
except Exception as error:parent_errors.append('observer:'+type(error).__name__)
try:os.close(fd)
except OSError as error:parent_errors.append('binary-fd:'+type(error).__name__)
errors=parent_errors+cleanup_resources(container_id,network_id,result,pending_create,seconds=120,collect_logs=True)
result['cleanup'] = {'pass': not errors, 'errors': errors}
if errors:
result['outcome'] = 'fail'
result['finished_at'] = time.time()
named_root = ROOT.stat(follow_symlinks=False)
assert (named_root.st_dev, named_root.st_ino) == (root_info.st_dev, root_info.st_ino)
output(root_fd, evidence['retry-result.json'], 'retry-result.json',
(json.dumps(result, indent=2) + '\n').encode())
for descriptor in evidence.values():
os.close(descriptor)
os.close(config_fd)
os.close(root_fd)
assert not errors, errors
print(json.dumps(result)) |
Member
Author
|
Exact inert countercontrols for the frozen shared #219 successor. The controller is extracted by AST declarations only; no controller/bootstrap top-level execution occurs. Public predecessor source40a5d60e is retained under the Controls SHA256: import ast
import copy
import hashlib
import io
import os
import pathlib
import re
import selectors
import stat
import subprocess
import sys
import tempfile
import time
import signal
import unittest
from unittest import mock
SOURCE = pathlib.Path(sys.argv[1])
SUPERVISOR = pathlib.Path(sys.argv[2])
sys.argv = sys.argv[:1]
module = ast.parse(SOURCE.read_text())
keep = {'digest', 'directory', 'read_at', 'exclusive', 'output', 'runtime_command', 'docker_command', 'call', 'validate_network', 'validate_container', 'full_id', 'reconcile_created', 'remove_resource', 'owned', 'validate_records', 'cleanup_resources', 'BudgetExpired', 'workload_deadline', 'admitted_profile', 'OutputOverflow', 'stop_process', 'read_process', 'bounded_run', 'bounded_http_json'}
ns = {'pathlib': pathlib, 'os': os, 'stat': stat, 'hashlib': hashlib, 'subprocess': subprocess,
'json': __import__('json'), 're': re, 'CLOSED_ENV': {'PATH': '/usr/bin:/bin', 'HOME': '/nonexistent', 'LANG': 'C', 'LC_ALL': 'C'},
'IMAGE': 'synthetic-image', 'name':'owned', 'time':time, 'signal':signal, 'selectors':selectors}
exec(compile(ast.Module(body=[node for node in module.body if isinstance(node, (ast.FunctionDef, ast.ClassDef)) and node.name in keep], type_ignores=[]), str(SOURCE), 'exec'), ns)
class Controls(unittest.TestCase):
def test_supervisor_replacement_discriminates_predecessor(self):
script = SUPERVISOR.read_bytes()
self.assertEqual(hashlib.sha256(script).hexdigest(), '25a591f514b3823093a37b66c47c588cba8e983594c8da2d7940ac4f565b922b')
with tempfile.TemporaryDirectory() as tmp:
path = pathlib.Path(tmp) / 'supervisor'
path.write_bytes(script)
captured = path.read_bytes()
command = ns['runtime_command'](captured, int(time.time()) + 10)
command[-2:] = ['none', '/bin/echo', 'ORIGINAL']
path.write_text('printf SUBSTITUTED')
predecessor = subprocess.check_output(['/bin/sh', str(path), '10', '10', 'none', '/bin/echo', 'ORIGINAL'])
self.assertEqual(predecessor, b'SUBSTITUTED')
self.assertEqual(subprocess.check_output(['/bin/sh'] + command), b'ORIGINAL\n')
self.assertNotIn(str(path), command)
def test_exclusive_output_refuses_symlink(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp)
marker = root / 'marker'; marker.write_bytes(b'unchanged')
(root / 'logging.stdout').symlink_to(marker)
(root / 'logging.stdout').write_bytes(b'PREDECESSOR')
self.assertEqual(marker.read_bytes(), b'PREDECESSOR')
marker.write_bytes(b'unchanged')
fd = ns['directory'](root)
try:
with self.assertRaises(FileExistsError): ns['exclusive'](fd, 'logging.stdout')
self.assertEqual(marker.read_bytes(), b'unchanged')
finally: os.close(fd)
def test_output_replacement_refuses_before_foreign_write(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp); fd = ns['directory'](root)
out = ns['exclusive'](fd, 'logging.stdout')
try:
self.assertEqual(stat.S_IMODE(os.fstat(out).st_mode), 0o600)
(root / 'logging.stdout').rename(root / 'held')
marker = root / 'marker'; marker.write_bytes(b'unchanged')
(root / 'logging.stdout').symlink_to(marker)
with self.assertRaises(AssertionError): ns['output'](fd, out, 'logging.stdout', b'bad')
self.assertEqual(marker.read_bytes(), b'unchanged')
finally: os.close(out); os.close(fd)
def test_input_and_root_links_refused(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp); (root / 'alias').symlink_to(root, target_is_directory=True)
with self.assertRaises(OSError): ns['directory'](root / 'alias')
fd = ns['directory'](root)
try:
(root / 'value').write_bytes(b'expected'); (root / 'link').symlink_to(root / 'value')
with self.assertRaises(OSError): ns['read_at'](fd, 'link')
self.assertEqual(ns['read_at'](fd, 'value'), b'expected')
finally: os.close(fd)
def test_effectful_tool_and_environment_are_closed(self):
configpath = '/proc/' + str(os.getpid()) + '/fdinfo'
configfd = os.open(configpath, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
ns.update(config_fd=configfd, config_path=configpath, config_info=os.fstat(configfd))
captured = []
def fake(*args, **kwargs):
# Runs after call validation, just before the consumer would open config.
with self.assertRaises(OSError):
opened = os.open('config.json', os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600, dir_fd=configfd)
os.close(opened)
self.assertFalse(os.path.lexists(pathlib.Path(configpath) / 'config.json'))
captured.append((args, kwargs)); return object()
try:
with mock.patch.dict(os.environ, {'PATH': '/synthetic-untrusted', 'DOCKER_HOST': 'tcp://wrong', 'DOCKER_CONFIG': '/wrong', 'DOCKER_CONTEXT': 'wrong'}):
with mock.patch.dict(ns, {'bounded_run':fake}): ns['call'](['docker', 'create', '--pull=never'])
args, kwargs = captured[0]
self.assertEqual(args[0][:3], ['/usr/bin/docker', '--host', 'unix:///run/docker.sock'])
self.assertEqual(kwargs['env'], ns['CLOSED_ENV'])
self.assertEqual(kwargs['pass_fds'], (configfd,))
self.assertEqual(kwargs['cwd'], '/')
self.assertNotIn('DOCKER_HOST', kwargs['env'])
finally: os.close(configfd)
def test_predecessor_config_changes_between_check_and_open(self):
with tempfile.TemporaryDirectory() as tmp:
path = pathlib.Path(tmp) / 'config.json'; path.write_text('{}\n')
before = path.stat()
self.assertEqual(path.read_text(), '{}\n')
# The old validation has completed; a writer changes the same inode.
path.write_text('{"credsStore":"synthetic"}')
self.assertEqual(path.stat().st_ino, before.st_ino)
self.assertEqual(path.read_text(), '{"credsStore":"synthetic"}')
def test_uncertain_network_and_container_replies_recover_full_id(self):
from types import SimpleNamespace
for kind in ('network', 'container'):
for reply in (b'', b'not-an-id', b'A' * 64, b'\xff'):
with self.subTest(kind=kind, reply=reply):
identity = 'a' * 64
labels = {'owner':'sdk-rust','task':'logger227','qualification_nonce':'synthetic-nonce'}
present = {identity}; operations = []
def fake(args, **kwargs):
operations.append(args)
if 'inspect' in args:
selected = args[-1]
if selected not in present:
message = ('No such container: '+selected if kind=='container' else 'network '+selected+' not found').encode()
return SimpleNamespace(returncode=1, stderr=message, stdout=b'[]\n')
value={'Id':selected,'Name':'owned','Config':{'Labels':labels},'Labels':labels,'State':{'OOMKilled':False}}
return SimpleNamespace(returncode=0, stderr=b'', stdout=ns['json'].dumps([value]).encode())
if 'rm' in args:
self.assertEqual(args[-1], identity); present.remove(identity)
return SimpleNamespace(returncode=0, stderr=b'', stdout=identity.encode())
if 'id='+identity in args:
return SimpleNamespace(returncode=0, stderr=b'', stdout=(identity.encode() if present else b''))
for key,val in labels.items():self.assertIn('label='+key+'='+val, args)
self.assertIn('--no-trunc', args)
return SimpleNamespace(returncode=0, stderr=b'', stdout=(identity+'\n').encode())
old_identity = reply.decode('ascii', errors='replace').strip()
self.assertIsNotNone(old_identity) # Old is-None recovery skips every reply.
granted = None
with self.assertRaises((AssertionError,UnicodeError)):
granted = ns['full_id'](reply)
self.assertIsNone(granted)
with mock.patch.dict(ns, {'labels':labels,'call':fake}):
recovered=ns['reconcile_created'](kind,granted)
self.assertEqual(recovered,identity)
ns['remove_resource'](kind,recovered)
self.assertEqual(present,set())
self.assertIn('id='+identity,operations[-1])
def test_configuration_mutations_are_refused(self):
image = {'Id': 'synthetic-id', 'Config': {'Env': ['PATH=/usr/bin'], 'Labels': {}, 'Volumes': {'/app/database': {}, '/app/storage': {}}}}
command = ['-c', 'captured']; tmpfs = {'/tmp': 'size=64m'}
value=ns['admitted_profile'](image,'network-id','owned',command,{'owner':'sdk-rust'},tmpfs)
config=value['Config'];host=value['HostConfig']
check=lambda v: ns['validate_container'](v,image,'network-id','owned',command,{'owner':'sdk-rust'},tmpfs)
check(value)
started=copy.deepcopy(value);started['HostConfig']['OomKillDisable']=None
self.assertEqual(check(value),check(started))
mutations=[('Image',None,'wrong'),('Config','Cmd',['wrong']),('Config','Env',config['Env']+['INJECTED=yes']),('Config','Labels',{}),
('HostConfig','Privileged',True),('HostConfig','NanoCpus',0),('HostConfig','Memory',0),('HostConfig','ReadonlyRootfs',False),
('HostConfig','Binds',['synthetic:/app']),('HostConfig','NetworkMode','host'),('HostConfig','SecurityOpt',[]),('HostConfig','OomKillDisable',True),('Mounts',None,[{'Type':'bind','Destination':'/app'}])]
for group,key,new in mutations:
with self.subTest(field=(group,key)):
changed=copy.deepcopy(value)
if key is None:changed[group]=new
else:changed[group][key]=new
with self.assertRaises(AssertionError):check(changed)
def test_network_profile_refuses_wrong_routing_or_attachment(self):
value={'Driver':'bridge','Scope':'local','Internal':False,'Ingress':False,'Attachable':False,'EnableIPv6':False,
'Options':{'com.docker.network.bridge.enable_ip_masquerade':'false','com.docker.network.bridge.enable_icc':'false','com.docker.network.bridge.host_binding_ipv4':'127.0.0.1'},'Containers':{}}
ns['validate_network'](value)
value['Containers']={'owned-runtime':{}}
ns['validate_network'](value,'owned-runtime')
for key,bad in [('Internal',True),('Driver','host'),('Attachable',True),('Containers',{'foreign':{}}),('Options',{})]:
with self.subTest(field=key):
changed=copy.deepcopy(value);changed[key]=bad
with self.assertRaises(AssertionError):ns['validate_network'](changed,'owned-runtime')
def records(self):
records=[]
for index,(case,delays) in enumerate([('short',[1,2,2,2]),('empty',[0,0,0,0])],1):
workflow='rust-retry-'+case+'-00000000-0000-0000-0000-'+str(index).zfill(12)
run='run-'+case
retries=[{'sequence':1,'activity_execution_id':'execution-'+case,'execution_mode':'local',
'local_activity':True,'retry_after_attempt':attempt,'retry_backoff_seconds':delay}
for attempt,delay in enumerate(delays,1)]
records.append({'case':case,'workflow_id':workflow,'run_id':run,'expected_backoff_seconds':delays,
'callback_gap_ms':[delay*1000 for delay in delays],'retry_reports':retries,'callback_attempts':5,
'callbacks_after_replay':0,'history':{'history_complete':True,'workflow':{
'instance_id':workflow,'run_id':run,'details_state':'retained'},
'history_events':[{'type':'ActivityRetryScheduled','payload':row} for row in retries]}})
return records
def test_observation_accepts_only_both_bound_cases(self):
ns['validate_records'](self.records())
for change in ('early','zero-tail','remote','incomplete','extra-callback','duplicate-run','missing-zero'):
with self.subTest(change=change):
records=self.records()
if change=='early':records[0]['callback_gap_ms'][2]=0
if change=='zero-tail':records[0]['retry_reports'][3]['retry_backoff_seconds']=0
if change=='remote':records[0]['retry_reports'][0]['execution_mode']='remote'
if change=='incomplete':records[0]['history']['history_complete']=False
if change=='extra-callback':records[0]['callbacks_after_replay']=1
if change=='duplicate-run':records[1]['run_id']=records[0]['run_id']
if change=='missing-zero':records[1]['retry_reports'][0]['retry_backoff_seconds']=None
with self.assertRaises((AssertionError,KeyError)):ns['validate_records'](records)
def test_startup_guard_is_before_root_access(self):
guard=next(n for n in module.body if isinstance(n,ast.If) and 'unsafe caller' in ast.unparse(n))
root=next(n for n in module.body if isinstance(n,ast.Assign) and any(isinstance(t,ast.Name) and t.id=='ROOT' for t in n.targets))
self.assertLess(guard.lineno,root.lineno)
from types import SimpleNamespace
for opt,site,bytecode in [(1,1,True),(0,0,True),(0,1,False)]:
fake=SimpleNamespace(flags=SimpleNamespace(optimize=opt,no_site=site),dont_write_bytecode=bytecode)
with self.assertRaises(RuntimeError):exec(compile(ast.Module(body=[guard],type_ignores=[]),'guard','exec'),{'sys':fake})
def test_deadline_recovers_committed_unknown_create(self):
for pending in ('container','network'):
present={pending:'a'*64};removed=[];result={'outcome':'fail'}
def recover(kind,identity):return identity if identity is not None else present.get(kind)
def remove(kind,identity):
self.assertEqual(present.pop(kind),identity);removed.append(identity);return {'State':{'OOMKilled':False}}
with mock.patch.dict(ns,{'reconcile_created':recover,'remove_resource':remove}):
try:ns['workload_deadline'](signal.SIGTERM,None)
except ns['BudgetExpired']:errors=ns['cleanup_resources'](None,None,result,pending,seconds=1)
self.assertEqual(errors,[]);self.assertEqual(present,{});self.assertEqual(removed,['a'*64])
def test_unresolved_create_is_nonpass_and_cleanup_calls_are_budgeted(self):
seen=[]
def call(args,wall=30,**kwargs):seen.append(wall)
def recover(kind,identity):ns['call'](['inert-query']);return None
with mock.patch.dict(ns,{'call':call,'reconcile_created':recover}):
errors=ns['cleanup_resources'](None,None,{},'container',seconds=.5)
self.assertIs(ns['call'],call)
self.assertEqual(errors,['container:uncertain-create'])
self.assertTrue(seen);self.assertTrue(all(0<wall<=.5 for wall in seen))
def test_literal_bootstrap_refuses_substitution_before_execution(self):
bootstrap = ast.parse(SOURCE.with_name('219-caller-bootstrap.py').read_bytes())
function = next(node for node in bootstrap.body if isinstance(node, ast.FunctionDef) and node.name == 'capture')
scope = {'os': os, 'pathlib': pathlib, 'stat': stat, 'hashlib': hashlib}
exec(compile(ast.Module(body=[function], type_ignores=[]), '<capture-control>', 'exec'), scope)
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp)
source = root / 'caller.py'
marker = []
original = b'marker.append("original")'
source.write_bytes(original)
expected = hashlib.sha256(original).hexdigest()
captured = scope['capture'](str(source), expected)
source.write_bytes(b'marker.append("replacement")')
exec(compile(source.read_bytes(), '<pathname-predecessor>', 'exec'), {'marker': marker})
self.assertEqual(marker, ['replacement'])
marker.clear()
with self.assertRaisesRegex(RuntimeError, 'source hash refused'):
scope['capture'](str(source), expected)
self.assertEqual(marker, [])
exec(compile(captured, '<captured-candidate>', 'exec'), {'marker': marker})
self.assertEqual(marker, ['original'])
source.unlink()
target = root / 'target.py'; target.write_bytes(original)
source.symlink_to(target)
with self.assertRaises(OSError):
scope['capture'](str(source), expected)
def test_complete_profile_discriminates_stable_predecessor_gaps(self):
predecessor = ast.parse(SOURCE.with_name('hold1-manual-219-timing.py').read_bytes())
old = {'json':ns['json'],'digest':ns['digest'],'IMAGE':'synthetic-image'}
nodes=[node for node in predecessor.body if isinstance(node,ast.FunctionDef) and node.name=='validate_container']
exec(compile(ast.Module(body=nodes,type_ignores=[]),'<profile-predecessor>','exec'),old)
image={'Id':'synthetic-id','Config':{'Env':['PATH=/usr/bin'],'Volumes':{},'Labels':{}}}
args=(image,'network-id','owned',['-c','captured'],{'owner':'sdk-rust'},{'/tmp':'size=64m'})
value=ns['admitted_profile'](*args)
old['validate_container'](value,*args)
mutations={'CgroupnsMode':'host','GroupAdd':['0'],'Dns':['203.0.113.1'],
'Sysctls':{'kernel.shmmax':'999999'},'Ulimits':[{'Name':'nofile','Hard':1048576,'Soft':1048576}],
'ReadonlyPaths':[],'MaskedPaths':[],'LogConfig':{'Type':'syslog','Config':{}},
'PublishAllPorts':True,'OomScoreAdj':-1000,'CgroupParent':'/'}
for field,change in mutations.items():
with self.subTest(field=field):
changed=copy.deepcopy(value);changed['HostConfig'][field]=change
old['validate_container'](changed,*args)
with self.assertRaises(AssertionError):ns['validate_container'](changed,*args)
for group in ('Config','HostConfig'):
added=copy.deepcopy(value);added[group]['unknown_field']='unselected'
with self.assertRaises(AssertionError):ns['validate_container'](added,*args)
missing=copy.deepcopy(value);missing[group].pop(next(iter(missing[group])))
with self.assertRaises(AssertionError):ns['validate_container'](missing,*args)
mounted=copy.deepcopy(value);mounted['Mounts']=[{'Type':'tmpfs','Destination':'/tmp','RW':True}]
with self.assertRaises(AssertionError):ns['validate_container'](mounted,*args)
def test_streaming_process_bounds_kill_and_reap(self):
# Inert language-only subprocesses; no Docker, SDK or service invocation.
for target in ('stdout','stderr','combined'):
with self.subTest(target=target):
code={'stdout':'os.write(1,b"x"*4096)',
'stderr':'os.write(2,b"x"*4096)',
'combined':'os.write(1,b"x"*80);os.write(2,b"x"*80)'}[target]
process=subprocess.Popen([sys.executable,'-I','-S','-B','-c',
'import os,time;'+code+';time.sleep(10)'],stdout=subprocess.PIPE,
stderr=subprocess.PIPE,start_new_session=True)
with self.assertRaises(ns['OutputOverflow']):ns['read_process'](process,2,128)
self.assertIsNotNone(process.returncode)
self.assertTrue(process.stdout.closed and process.stderr.closed)
old=subprocess.run([sys.executable,'-I','-S','-B','-c',
'import os;os.write(1,b"x"*4096)'],capture_output=True,check=True)
self.assertGreater(len(old.stdout),128) # Prior post-capture check is already too late.
def test_bounded_run_and_timeout_return_no_live_child(self):
result=ns['bounded_run']([sys.executable,'-I','-S','-B','-c',
'import os,sys;os.write(1,sys.stdin.buffer.read());os.write(2,b"err")'],
wall=2,byte_limit=128,input=b'fixture')
self.assertEqual((result.stdout,result.stderr),(b'fixture',b'err'))
process=subprocess.Popen([sys.executable,'-I','-S','-B','-c',
'import time;time.sleep(10)'],stdout=subprocess.PIPE,stderr=subprocess.PIPE,start_new_session=True)
with self.assertRaises(subprocess.TimeoutExpired):ns['read_process'](process,.02,128)
self.assertIsNotNone(process.returncode)
def test_http_body_refuses_before_unbounded_read(self):
class BoundedBody:
def __init__(self,value):self.value=io.BytesIO(value);self.requests=[]
def read(self,size):
self.assert_size(size);self.requests.append(size);return self.value.read(size)
def assert_size(self,size):
if not 0<size<=8192:raise AssertionError('unbounded response read')
good=BoundedBody(b'{"version":"3.10.0"}')
self.assertEqual(ns['bounded_http_json'](good),{'version':'3.10.0'})
oversized=BoundedBody(b' '*4096)
with self.assertRaises(ns['OutputOverflow']):ns['bounded_http_json'](oversized,128)
self.assertEqual(oversized.value.tell(),129)
self.assertGreater(len(io.BytesIO(b' '*4096).read()),128) # Prior unbounded JSON read.
def test_overflow_still_reconciles_and_removes_owned_resources(self):
removed=[];result={'outcome':'fail'}
def remove(kind,identity):removed.append((kind,identity));return {'State':{'OOMKilled':False}}
with mock.patch.dict(ns,{'reconcile_created':lambda kind,identity:identity,'remove_resource':remove,
'call':lambda *a,**k:(_ for _ in ()).throw(ns['OutputOverflow']('log limit'))}):
errors=ns['cleanup_resources']('a'*64,'b'*64,result,None,seconds=1,collect_logs=True)
self.assertEqual(removed,[('container','a'*64),('network','b'*64)])
self.assertEqual(errors,['runtime-logs:OutputOverflow'])
self.assertEqual(result['outcome'],'fail')
unittest.main() |
Member
Author
|
Security static caller review of the frozen ordinary-219 successor: HOLD for one remaining network-attachment boundary. I did not import or execute the effectful controller/bootstrap. I read exact controller 5b762bda, controls 017cc34b and bootstrap 1f1b1540, verified their hashes, and ran only the pure AST/inert harness: 19/19 pass. The two prior blockers are otherwise closed. validate_container now compares the complete Config/HostConfig/Mounts projection and normalizes only documented OomKillDisable false/null. The read paths enforce combined subprocess, runtime-log and HTTP limits while reading; overflow/timeout kills and reaps the owned process, and cleanup remains reachable. The remaining gap is that network membership is not fail-closed. validate_network currently uses a subset test, so an expected runtime network with zero attached containers passes. validate_container deliberately excludes NetworkSettings, so the container may also carry an additional foreign-network attachment while Config/HostConfig/Mounts and the selected network all pass. That would bypass the intended no-masquerade/no-ICC routing boundary. Before one observation, require exact selected-network membership: empty before a runtime exists and exactly the owned runtime afterward. Also validate the container NetworkSettings.Networks projection has exactly the selected network with matching NetworkID and no extra attachment, at the existing created/started/returned checkpoints. Add inert missing-selected and extra-network controls, including predecessor acceptance if applicable. No Docker, SDK, Server, credential, timing or publication effect is authorized by this review. The accepted SDK #275 source remains separate; this HOLD applies only to the proposed ordinary-219 caller/profile. |
Member
Author
|
Frozen narrow network successor for Security HOLD6871426356. No timing observation or Docker effect ran. The selected network now requires exact membership: empty at the created/unstarted checkpoints, then exactly the owned runtime after start and on return. The container must have exactly the selected network name, with Docker29's empty NetworkID before start and the exact selected full NetworkID afterward. Missing, extra and wrong-ID attachments refuse. The name/ID lifecycle distinction was checked against all three retained created/started/returned config snapshots from the previously accepted same-engine 3.10 logging observation, without creating or starting anything. All 21 inert controls pass. New controls prove the predecessor's acceptance of a missing runtime and missing/extra/wrong-ID container attachments, candidate refusal at both lifecycle stages, and positive created/started profiles. Complete Config/HostConfig/Mounts admission, streaming byte bounds, kill/reap, cleanup, package/build/scenario and workload limits are unchanged. Prior packet and HOLD records are preserved. The public base controller is exact5b762bda in the preceding packet; the complete controller delta is below. Current inert controls are included in full. Bootstrap changed only its exact controller pin and remains a literal -c invocation with privately fixed paths. This remains a proposal for one separately reviewed ordinary shared219 timing observation. The effectful controller/bootstrap were not imported or run by the test harness; no set-aside item or installed result is selected here. Frozen packet: {
"controller_sha256": "db7afcf59ba216b9b36633da7e2cfa0d2088fab651ab639c45310775457192d7",
"controls_sha256": "73cf6d6e28877bf538cf6ca4396a4176e29a3de7d69d43a91bc66f52d6d695d8",
"literal_bootstrap_sha256": "bd05be4c40ee0140b9f609e6afa79d11d23612e62bc62911aff0ccc41187d07b",
"build_receipt_sha256": "9fff98616c7d87b937ea0d843c8c8ba9cef3271049a884abd49a2aff793bf5b7",
"test_sha256": "f95e947b2b3c199bb929e0f7176fb712d3b0f235612c97cbbae859df6cdaa862",
"binary_sha256": "5cd20d42dc317f6b49637595799af611bcd756a53899e81d1dccb37e9649590a",
"qualified_server": "3.10.0",
"server_source": "b6e04ce7e679464e91f6ceae83310d3bb3204092",
"server_child": "sha256:9d64e9aeee84b46d21715826a3e7e5a96eb8d4ce7fccb126b88b15596ab14973",
"scope": "one ordinary219 local-retry timing observation; no clock/redrive/database operation",
"source_controls": 21,
"workload_seconds": 180,
"parent_seconds": 90,
"cleanup_seconds": 120,
"parent_wait_seconds": 10,
"outer_seconds": 340,
"outer_kill_seconds": 20,
"container_profile": "complete Config/HostConfig/Mounts equality; OomKillDisable false/null only normalized",
"stream_byte_limits": {
"docker_stdout_stderr_combined": 1048576,
"runtime_logs_combined": 131072,
"consumer_stdout_stderr_combined": 1048576,
"cluster_http_body": 65536
},
"network_membership": "empty before start; exact runtime after start; container has only selected network name and stage-matching NetworkID"
}Complete controller delta from public5b762bda: --- previous5b762
+++ network-successor
@@ -343,7 +343,7 @@
'com.docker.network.bridge.enable_ip_masquerade': 'false',
'com.docker.network.bridge.enable_icc': 'false',
'com.docker.network.bridge.host_binding_ipv4': '127.0.0.1'}
- assert set(value.get('Containers') or {}) <= ({runtime} if runtime else set())
+ assert set(value.get('Containers') or {}) == ({runtime} if runtime else set())
def admitted_profile(image, network, name, command, labels, tmpfs):
config = {
@@ -455,7 +455,13 @@
host.update(NetworkMode=network, Tmpfs=tmpfs)
return {'Image': image['Id'], 'Config': config, 'HostConfig': host, 'Mounts': []}
-def validate_container(value, image, network, name, command, labels, tmpfs):
+def validate_container(value, image, network, name, command, labels, tmpfs, *, started=False):
+ networks = value['NetworkSettings']['Networks']
+ assert set(networks) == {name}
+ # Docker29 resolves the selected name to its full NetworkID on start.
+ # The created, unstarted endpoint has an empty ID; neither stage admits
+ # a missing selected network or an additional attachment.
+ assert networks[name]['NetworkID'] == (network if started else '')
packet = json.loads(json.dumps({k: value[k] for k in ('Image', 'Config', 'HostConfig', 'Mounts')}))
# Docker29 changes this single unset/false default during start. Both retain
# the kernel OOM killer; true and all other profile differences refuse.
@@ -579,7 +585,7 @@
(json.dumps(before, indent=2) + '\n').encode())
result['container_id'], result['network_id'] = container_id, network_id
created_config = validate_container(before, image, network_id, name, command, labels, tmpfs)
- validate_network(owned('network', network_id), container_id)
+ validate_network(owned('network', network_id))
result['created_config_sha256'] = created_config
engine()
image_identity()
@@ -590,7 +596,7 @@
output(root_fd, evidence['container-started.json'], 'container-started.json',
(json.dumps(started, indent=2) + '\n').encode())
assert validate_container(started, image, network_id,
- name, command, labels, tmpfs) == created_config
+ name, command, labels, tmpfs, started=True) == created_config
validate_network(owned('network', network_id), container_id)
call(['docker', 'exec', '-i', container_id, '/bin/sh', '-c',
'umask 077; IFS= read -r token && IFS= read -r key && '
@@ -649,7 +655,7 @@
output(root_fd, evidence['container-returned.json'], 'container-returned.json',
(json.dumps(returned, indent=2) + '\n').encode())
assert validate_container(returned, image, network_id,
- name, command, labels, tmpfs) == created_config
+ name, command, labels, tmpfs, started=True) == created_config
final_network = owned('network', network_id)
output(root_fd, evidence['network-returned.json'], 'network-returned.json',
(json.dumps(final_network, indent=2) + '\n').encode())Exact inert controls: import ast
import copy
import hashlib
import io
import os
import pathlib
import re
import selectors
import stat
import subprocess
import sys
import tempfile
import time
import signal
import unittest
from unittest import mock
SOURCE = pathlib.Path(sys.argv[1])
SUPERVISOR = pathlib.Path(sys.argv[2])
sys.argv = sys.argv[:1]
module = ast.parse(SOURCE.read_text())
keep = {'digest', 'directory', 'read_at', 'exclusive', 'output', 'runtime_command', 'docker_command', 'call', 'validate_network', 'validate_container', 'full_id', 'reconcile_created', 'remove_resource', 'owned', 'validate_records', 'cleanup_resources', 'BudgetExpired', 'workload_deadline', 'admitted_profile', 'OutputOverflow', 'stop_process', 'read_process', 'bounded_run', 'bounded_http_json'}
ns = {'pathlib': pathlib, 'os': os, 'stat': stat, 'hashlib': hashlib, 'subprocess': subprocess,
'json': __import__('json'), 're': re, 'CLOSED_ENV': {'PATH': '/usr/bin:/bin', 'HOME': '/nonexistent', 'LANG': 'C', 'LC_ALL': 'C'},
'IMAGE': 'synthetic-image', 'name':'owned', 'time':time, 'signal':signal, 'selectors':selectors}
exec(compile(ast.Module(body=[node for node in module.body if isinstance(node, (ast.FunctionDef, ast.ClassDef)) and node.name in keep], type_ignores=[]), str(SOURCE), 'exec'), ns)
class Controls(unittest.TestCase):
def test_supervisor_replacement_discriminates_predecessor(self):
script = SUPERVISOR.read_bytes()
self.assertEqual(hashlib.sha256(script).hexdigest(), '25a591f514b3823093a37b66c47c588cba8e983594c8da2d7940ac4f565b922b')
with tempfile.TemporaryDirectory() as tmp:
path = pathlib.Path(tmp) / 'supervisor'
path.write_bytes(script)
captured = path.read_bytes()
command = ns['runtime_command'](captured, int(time.time()) + 10)
command[-2:] = ['none', '/bin/echo', 'ORIGINAL']
path.write_text('printf SUBSTITUTED')
predecessor = subprocess.check_output(['/bin/sh', str(path), '10', '10', 'none', '/bin/echo', 'ORIGINAL'])
self.assertEqual(predecessor, b'SUBSTITUTED')
self.assertEqual(subprocess.check_output(['/bin/sh'] + command), b'ORIGINAL\n')
self.assertNotIn(str(path), command)
def test_exclusive_output_refuses_symlink(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp)
marker = root / 'marker'; marker.write_bytes(b'unchanged')
(root / 'logging.stdout').symlink_to(marker)
(root / 'logging.stdout').write_bytes(b'PREDECESSOR')
self.assertEqual(marker.read_bytes(), b'PREDECESSOR')
marker.write_bytes(b'unchanged')
fd = ns['directory'](root)
try:
with self.assertRaises(FileExistsError): ns['exclusive'](fd, 'logging.stdout')
self.assertEqual(marker.read_bytes(), b'unchanged')
finally: os.close(fd)
def test_output_replacement_refuses_before_foreign_write(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp); fd = ns['directory'](root)
out = ns['exclusive'](fd, 'logging.stdout')
try:
self.assertEqual(stat.S_IMODE(os.fstat(out).st_mode), 0o600)
(root / 'logging.stdout').rename(root / 'held')
marker = root / 'marker'; marker.write_bytes(b'unchanged')
(root / 'logging.stdout').symlink_to(marker)
with self.assertRaises(AssertionError): ns['output'](fd, out, 'logging.stdout', b'bad')
self.assertEqual(marker.read_bytes(), b'unchanged')
finally: os.close(out); os.close(fd)
def test_input_and_root_links_refused(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp); (root / 'alias').symlink_to(root, target_is_directory=True)
with self.assertRaises(OSError): ns['directory'](root / 'alias')
fd = ns['directory'](root)
try:
(root / 'value').write_bytes(b'expected'); (root / 'link').symlink_to(root / 'value')
with self.assertRaises(OSError): ns['read_at'](fd, 'link')
self.assertEqual(ns['read_at'](fd, 'value'), b'expected')
finally: os.close(fd)
def test_effectful_tool_and_environment_are_closed(self):
configpath = '/proc/' + str(os.getpid()) + '/fdinfo'
configfd = os.open(configpath, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
ns.update(config_fd=configfd, config_path=configpath, config_info=os.fstat(configfd))
captured = []
def fake(*args, **kwargs):
# Runs after call validation, just before the consumer would open config.
with self.assertRaises(OSError):
opened = os.open('config.json', os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600, dir_fd=configfd)
os.close(opened)
self.assertFalse(os.path.lexists(pathlib.Path(configpath) / 'config.json'))
captured.append((args, kwargs)); return object()
try:
with mock.patch.dict(os.environ, {'PATH': '/synthetic-untrusted', 'DOCKER_HOST': 'tcp://wrong', 'DOCKER_CONFIG': '/wrong', 'DOCKER_CONTEXT': 'wrong'}):
with mock.patch.dict(ns, {'bounded_run':fake}): ns['call'](['docker', 'create', '--pull=never'])
args, kwargs = captured[0]
self.assertEqual(args[0][:3], ['/usr/bin/docker', '--host', 'unix:///run/docker.sock'])
self.assertEqual(kwargs['env'], ns['CLOSED_ENV'])
self.assertEqual(kwargs['pass_fds'], (configfd,))
self.assertEqual(kwargs['cwd'], '/')
self.assertNotIn('DOCKER_HOST', kwargs['env'])
finally: os.close(configfd)
def test_predecessor_config_changes_between_check_and_open(self):
with tempfile.TemporaryDirectory() as tmp:
path = pathlib.Path(tmp) / 'config.json'; path.write_text('{}\n')
before = path.stat()
self.assertEqual(path.read_text(), '{}\n')
# The old validation has completed; a writer changes the same inode.
path.write_text('{"credsStore":"synthetic"}')
self.assertEqual(path.stat().st_ino, before.st_ino)
self.assertEqual(path.read_text(), '{"credsStore":"synthetic"}')
def test_uncertain_network_and_container_replies_recover_full_id(self):
from types import SimpleNamespace
for kind in ('network', 'container'):
for reply in (b'', b'not-an-id', b'A' * 64, b'\xff'):
with self.subTest(kind=kind, reply=reply):
identity = 'a' * 64
labels = {'owner':'sdk-rust','task':'logger227','qualification_nonce':'synthetic-nonce'}
present = {identity}; operations = []
def fake(args, **kwargs):
operations.append(args)
if 'inspect' in args:
selected = args[-1]
if selected not in present:
message = ('No such container: '+selected if kind=='container' else 'network '+selected+' not found').encode()
return SimpleNamespace(returncode=1, stderr=message, stdout=b'[]\n')
value={'Id':selected,'Name':'owned','Config':{'Labels':labels},'Labels':labels,'State':{'OOMKilled':False}}
return SimpleNamespace(returncode=0, stderr=b'', stdout=ns['json'].dumps([value]).encode())
if 'rm' in args:
self.assertEqual(args[-1], identity); present.remove(identity)
return SimpleNamespace(returncode=0, stderr=b'', stdout=identity.encode())
if 'id='+identity in args:
return SimpleNamespace(returncode=0, stderr=b'', stdout=(identity.encode() if present else b''))
for key,val in labels.items():self.assertIn('label='+key+'='+val, args)
self.assertIn('--no-trunc', args)
return SimpleNamespace(returncode=0, stderr=b'', stdout=(identity+'\n').encode())
old_identity = reply.decode('ascii', errors='replace').strip()
self.assertIsNotNone(old_identity) # Old is-None recovery skips every reply.
granted = None
with self.assertRaises((AssertionError,UnicodeError)):
granted = ns['full_id'](reply)
self.assertIsNone(granted)
with mock.patch.dict(ns, {'labels':labels,'call':fake}):
recovered=ns['reconcile_created'](kind,granted)
self.assertEqual(recovered,identity)
ns['remove_resource'](kind,recovered)
self.assertEqual(present,set())
self.assertIn('id='+identity,operations[-1])
def test_configuration_mutations_are_refused(self):
image = {'Id': 'synthetic-id', 'Config': {'Env': ['PATH=/usr/bin'], 'Labels': {}, 'Volumes': {'/app/database': {}, '/app/storage': {}}}}
command = ['-c', 'captured']; tmpfs = {'/tmp': 'size=64m'}
value=ns['admitted_profile'](image,'network-id','owned',command,{'owner':'sdk-rust'},tmpfs)
value['NetworkSettings']={'Networks':{'owned':{'NetworkID':''}}}
config=value['Config'];host=value['HostConfig']
check=lambda v: ns['validate_container'](v,image,'network-id','owned',command,{'owner':'sdk-rust'},tmpfs)
check(value)
started=copy.deepcopy(value);started['HostConfig']['OomKillDisable']=None
self.assertEqual(check(value),check(started))
mutations=[('Image',None,'wrong'),('Config','Cmd',['wrong']),('Config','Env',config['Env']+['INJECTED=yes']),('Config','Labels',{}),
('HostConfig','Privileged',True),('HostConfig','NanoCpus',0),('HostConfig','Memory',0),('HostConfig','ReadonlyRootfs',False),
('HostConfig','Binds',['synthetic:/app']),('HostConfig','NetworkMode','host'),('HostConfig','SecurityOpt',[]),('HostConfig','OomKillDisable',True),('Mounts',None,[{'Type':'bind','Destination':'/app'}])]
for group,key,new in mutations:
with self.subTest(field=(group,key)):
changed=copy.deepcopy(value)
if key is None:changed[group]=new
else:changed[group][key]=new
with self.assertRaises(AssertionError):check(changed)
def test_network_profile_refuses_wrong_routing_or_attachment(self):
value={'Driver':'bridge','Scope':'local','Internal':False,'Ingress':False,'Attachable':False,'EnableIPv6':False,
'Options':{'com.docker.network.bridge.enable_ip_masquerade':'false','com.docker.network.bridge.enable_icc':'false','com.docker.network.bridge.host_binding_ipv4':'127.0.0.1'},'Containers':{}}
ns['validate_network'](value)
value['Containers']={'owned-runtime':{}}
ns['validate_network'](value,'owned-runtime')
for key,bad in [('Internal',True),('Driver','host'),('Attachable',True),('Containers',{'foreign':{}}),('Options',{})]:
with self.subTest(field=key):
changed=copy.deepcopy(value);changed[key]=bad
with self.assertRaises(AssertionError):ns['validate_network'](changed,'owned-runtime')
def records(self):
records=[]
for index,(case,delays) in enumerate([('short',[1,2,2,2]),('empty',[0,0,0,0])],1):
workflow='rust-retry-'+case+'-00000000-0000-0000-0000-'+str(index).zfill(12)
run='run-'+case
retries=[{'sequence':1,'activity_execution_id':'execution-'+case,'execution_mode':'local',
'local_activity':True,'retry_after_attempt':attempt,'retry_backoff_seconds':delay}
for attempt,delay in enumerate(delays,1)]
records.append({'case':case,'workflow_id':workflow,'run_id':run,'expected_backoff_seconds':delays,
'callback_gap_ms':[delay*1000 for delay in delays],'retry_reports':retries,'callback_attempts':5,
'callbacks_after_replay':0,'history':{'history_complete':True,'workflow':{
'instance_id':workflow,'run_id':run,'details_state':'retained'},
'history_events':[{'type':'ActivityRetryScheduled','payload':row} for row in retries]}})
return records
def test_observation_accepts_only_both_bound_cases(self):
ns['validate_records'](self.records())
for change in ('early','zero-tail','remote','incomplete','extra-callback','duplicate-run','missing-zero'):
with self.subTest(change=change):
records=self.records()
if change=='early':records[0]['callback_gap_ms'][2]=0
if change=='zero-tail':records[0]['retry_reports'][3]['retry_backoff_seconds']=0
if change=='remote':records[0]['retry_reports'][0]['execution_mode']='remote'
if change=='incomplete':records[0]['history']['history_complete']=False
if change=='extra-callback':records[0]['callbacks_after_replay']=1
if change=='duplicate-run':records[1]['run_id']=records[0]['run_id']
if change=='missing-zero':records[1]['retry_reports'][0]['retry_backoff_seconds']=None
with self.assertRaises((AssertionError,KeyError)):ns['validate_records'](records)
def test_startup_guard_is_before_root_access(self):
guard=next(n for n in module.body if isinstance(n,ast.If) and 'unsafe caller' in ast.unparse(n))
root=next(n for n in module.body if isinstance(n,ast.Assign) and any(isinstance(t,ast.Name) and t.id=='ROOT' for t in n.targets))
self.assertLess(guard.lineno,root.lineno)
from types import SimpleNamespace
for opt,site,bytecode in [(1,1,True),(0,0,True),(0,1,False)]:
fake=SimpleNamespace(flags=SimpleNamespace(optimize=opt,no_site=site),dont_write_bytecode=bytecode)
with self.assertRaises(RuntimeError):exec(compile(ast.Module(body=[guard],type_ignores=[]),'guard','exec'),{'sys':fake})
def test_deadline_recovers_committed_unknown_create(self):
for pending in ('container','network'):
present={pending:'a'*64};removed=[];result={'outcome':'fail'}
def recover(kind,identity):return identity if identity is not None else present.get(kind)
def remove(kind,identity):
self.assertEqual(present.pop(kind),identity);removed.append(identity);return {'State':{'OOMKilled':False}}
with mock.patch.dict(ns,{'reconcile_created':recover,'remove_resource':remove}):
try:ns['workload_deadline'](signal.SIGTERM,None)
except ns['BudgetExpired']:errors=ns['cleanup_resources'](None,None,result,pending,seconds=1)
self.assertEqual(errors,[]);self.assertEqual(present,{});self.assertEqual(removed,['a'*64])
def test_unresolved_create_is_nonpass_and_cleanup_calls_are_budgeted(self):
seen=[]
def call(args,wall=30,**kwargs):seen.append(wall)
def recover(kind,identity):ns['call'](['inert-query']);return None
with mock.patch.dict(ns,{'call':call,'reconcile_created':recover}):
errors=ns['cleanup_resources'](None,None,{},'container',seconds=.5)
self.assertIs(ns['call'],call)
self.assertEqual(errors,['container:uncertain-create'])
self.assertTrue(seen);self.assertTrue(all(0<wall<=.5 for wall in seen))
def test_literal_bootstrap_refuses_substitution_before_execution(self):
bootstrap = ast.parse(SOURCE.with_name('219-caller-bootstrap.py').read_bytes())
function = next(node for node in bootstrap.body if isinstance(node, ast.FunctionDef) and node.name == 'capture')
scope = {'os': os, 'pathlib': pathlib, 'stat': stat, 'hashlib': hashlib}
exec(compile(ast.Module(body=[function], type_ignores=[]), '<capture-control>', 'exec'), scope)
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp)
source = root / 'caller.py'
marker = []
original = b'marker.append("original")'
source.write_bytes(original)
expected = hashlib.sha256(original).hexdigest()
captured = scope['capture'](str(source), expected)
source.write_bytes(b'marker.append("replacement")')
exec(compile(source.read_bytes(), '<pathname-predecessor>', 'exec'), {'marker': marker})
self.assertEqual(marker, ['replacement'])
marker.clear()
with self.assertRaisesRegex(RuntimeError, 'source hash refused'):
scope['capture'](str(source), expected)
self.assertEqual(marker, [])
exec(compile(captured, '<captured-candidate>', 'exec'), {'marker': marker})
self.assertEqual(marker, ['original'])
source.unlink()
target = root / 'target.py'; target.write_bytes(original)
source.symlink_to(target)
with self.assertRaises(OSError):
scope['capture'](str(source), expected)
def test_complete_profile_discriminates_stable_predecessor_gaps(self):
predecessor = ast.parse(SOURCE.with_name('hold1-manual-219-timing.py').read_bytes())
old = {'json':ns['json'],'digest':ns['digest'],'IMAGE':'synthetic-image'}
nodes=[node for node in predecessor.body if isinstance(node,ast.FunctionDef) and node.name=='validate_container']
exec(compile(ast.Module(body=nodes,type_ignores=[]),'<profile-predecessor>','exec'),old)
image={'Id':'synthetic-id','Config':{'Env':['PATH=/usr/bin'],'Volumes':{},'Labels':{}}}
args=(image,'network-id','owned',['-c','captured'],{'owner':'sdk-rust'},{'/tmp':'size=64m'})
value=ns['admitted_profile'](*args)
value['NetworkSettings']={'Networks':{'owned':{'NetworkID':''}}}
old['validate_container'](value,*args)
mutations={'CgroupnsMode':'host','GroupAdd':['0'],'Dns':['203.0.113.1'],
'Sysctls':{'kernel.shmmax':'999999'},'Ulimits':[{'Name':'nofile','Hard':1048576,'Soft':1048576}],
'ReadonlyPaths':[],'MaskedPaths':[],'LogConfig':{'Type':'syslog','Config':{}},
'PublishAllPorts':True,'OomScoreAdj':-1000,'CgroupParent':'/'}
for field,change in mutations.items():
with self.subTest(field=field):
changed=copy.deepcopy(value);changed['HostConfig'][field]=change
old['validate_container'](changed,*args)
with self.assertRaises(AssertionError):ns['validate_container'](changed,*args)
for group in ('Config','HostConfig'):
added=copy.deepcopy(value);added[group]['unknown_field']='unselected'
with self.assertRaises(AssertionError):ns['validate_container'](added,*args)
missing=copy.deepcopy(value);missing[group].pop(next(iter(missing[group])))
with self.assertRaises(AssertionError):ns['validate_container'](missing,*args)
mounted=copy.deepcopy(value);mounted['Mounts']=[{'Type':'tmpfs','Destination':'/tmp','RW':True}]
with self.assertRaises(AssertionError):ns['validate_container'](mounted,*args)
def test_streaming_process_bounds_kill_and_reap(self):
# Inert language-only subprocesses; no Docker, SDK or service invocation.
for target in ('stdout','stderr','combined'):
with self.subTest(target=target):
code={'stdout':'os.write(1,b"x"*4096)',
'stderr':'os.write(2,b"x"*4096)',
'combined':'os.write(1,b"x"*80);os.write(2,b"x"*80)'}[target]
process=subprocess.Popen([sys.executable,'-I','-S','-B','-c',
'import os,time;'+code+';time.sleep(10)'],stdout=subprocess.PIPE,
stderr=subprocess.PIPE,start_new_session=True)
with self.assertRaises(ns['OutputOverflow']):ns['read_process'](process,2,128)
self.assertIsNotNone(process.returncode)
self.assertTrue(process.stdout.closed and process.stderr.closed)
old=subprocess.run([sys.executable,'-I','-S','-B','-c',
'import os;os.write(1,b"x"*4096)'],capture_output=True,check=True)
self.assertGreater(len(old.stdout),128) # Prior post-capture check is already too late.
def test_bounded_run_and_timeout_return_no_live_child(self):
result=ns['bounded_run']([sys.executable,'-I','-S','-B','-c',
'import os,sys;os.write(1,sys.stdin.buffer.read());os.write(2,b"err")'],
wall=2,byte_limit=128,input=b'fixture')
self.assertEqual((result.stdout,result.stderr),(b'fixture',b'err'))
process=subprocess.Popen([sys.executable,'-I','-S','-B','-c',
'import time;time.sleep(10)'],stdout=subprocess.PIPE,stderr=subprocess.PIPE,start_new_session=True)
with self.assertRaises(subprocess.TimeoutExpired):ns['read_process'](process,.02,128)
self.assertIsNotNone(process.returncode)
def test_http_body_refuses_before_unbounded_read(self):
class BoundedBody:
def __init__(self,value):self.value=io.BytesIO(value);self.requests=[]
def read(self,size):
self.assert_size(size);self.requests.append(size);return self.value.read(size)
def assert_size(self,size):
if not 0<size<=8192:raise AssertionError('unbounded response read')
good=BoundedBody(b'{"version":"3.10.0"}')
self.assertEqual(ns['bounded_http_json'](good),{'version':'3.10.0'})
oversized=BoundedBody(b' '*4096)
with self.assertRaises(ns['OutputOverflow']):ns['bounded_http_json'](oversized,128)
self.assertEqual(oversized.value.tell(),129)
self.assertGreater(len(io.BytesIO(b' '*4096).read()),128) # Prior unbounded JSON read.
def test_overflow_still_reconciles_and_removes_owned_resources(self):
removed=[];result={'outcome':'fail'}
def remove(kind,identity):removed.append((kind,identity));return {'State':{'OOMKilled':False}}
with mock.patch.dict(ns,{'reconcile_created':lambda kind,identity:identity,'remove_resource':remove,
'call':lambda *a,**k:(_ for _ in ()).throw(ns['OutputOverflow']('log limit'))}):
errors=ns['cleanup_resources']('a'*64,'b'*64,result,None,seconds=1,collect_logs=True)
self.assertEqual(removed,[('container','a'*64),('network','b'*64)])
self.assertEqual(errors,['runtime-logs:OutputOverflow'])
self.assertEqual(result['outcome'],'fail')
def test_selected_network_membership_is_exact_by_stage(self):
predecessor=ast.parse(SOURCE.with_name('hold2-manual-219-timing.py').read_bytes())
old={}
node=next(n for n in predecessor.body if isinstance(n,ast.FunctionDef) and n.name=='validate_network')
exec(compile(ast.Module(body=[node],type_ignores=[]),'<network-predecessor>','exec'),old)
value={'Driver':'bridge','Scope':'local','Internal':False,'Ingress':False,'Attachable':False,'EnableIPv6':False,
'Options':{'com.docker.network.bridge.enable_ip_masquerade':'false',
'com.docker.network.bridge.enable_icc':'false',
'com.docker.network.bridge.host_binding_ipv4':'127.0.0.1'},'Containers':{}}
ns['validate_network'](value) # Before start, the network has no endpoints.
old['validate_network'](value,'owned-runtime') # The prior subset admitted a missing endpoint.
with self.assertRaises(AssertionError):ns['validate_network'](value,'owned-runtime')
value['Containers']={'owned-runtime':{}}
ns['validate_network'](value,'owned-runtime')
with self.assertRaises(AssertionError):ns['validate_network'](value)
value['Containers']['foreign-runtime']={}
with self.assertRaises(AssertionError):ns['validate_network'](value,'owned-runtime')
def test_container_refuses_missing_extra_or_wrong_selected_attachment(self):
predecessor=ast.parse(SOURCE.with_name('hold2-manual-219-timing.py').read_bytes())
old={key:ns[key] for key in ('json','digest','admitted_profile')}
node=next(n for n in predecessor.body if isinstance(n,ast.FunctionDef) and n.name=='validate_container')
exec(compile(ast.Module(body=[node],type_ignores=[]),'<attachment-predecessor>','exec'),old)
image={'Id':'synthetic-id','Config':{'Env':['PATH=/usr/bin'],'Volumes':{},'Labels':{}}}
args=(image,'network-id','owned',['-c','captured'],{'owner':'sdk-rust'},{'/tmp':'size=64m'})
value=ns['admitted_profile'](*args)
value['NetworkSettings']={'Networks':{'owned':{'NetworkID':''}}}
created=ns['validate_container'](value,*args)
active=copy.deepcopy(value);active['NetworkSettings']['Networks']['owned']['NetworkID']='network-id'
self.assertEqual(ns['validate_container'](active,*args,started=True),created)
for started,base in [(False,value),(True,active)]:
for case in ('missing','extra','wrong-id'):
with self.subTest(started=started,case=case):
changed=copy.deepcopy(base);networks=changed['NetworkSettings']['Networks']
if case=='missing':networks.clear()
if case=='extra':networks['foreign']={'NetworkID':'foreign-id'}
if case=='wrong-id':networks['owned']['NetworkID']='foreign-id'
old['validate_container'](changed,*args)
with self.assertRaises(AssertionError):ns['validate_container'](changed,*args,started=started)
with self.assertRaises(AssertionError):ns['validate_container'](value,*args,started=True)
with self.assertRaises(AssertionError):ns['validate_container'](active,*args)
unittest.main() |
Member
Author
|
Security exact caller/profile PASS for one bounded ordinary-219 observation using controller db7afcf59ba216b9b36633da7e2cfa0d2088fab651ab639c45310775457192d, controls 73cf6d6e28877bf538cf6ca4396a4176e29a3de7d69d43a91bc66f52d6d695d8 and literal bootstrap bd05be4c40ee0140b9f609e6afa79d11d23612e62bc62911aff0ccc41187d07b. I statically reviewed the complete r3 delta and did not import or execute the effectful controller/bootstrap. I ran only the pure AST/inert harness: 21/21 pass. The prior network HOLD is closed:
Together with the prior reviewed correction, Config/HostConfig/Mounts is exact except false/null OomKillDisable, all process/HTTP/log output is bounded while reading, overflow and timeout kill/reap, and cleanup remains reached. Build receipt, package/binary/test, qualified Server tuple, scenario semantics and resource deadlines are unchanged. This permits one observation under the exact frozen packet only. It is not SDK source/release qualification, publication authority, production access, or permission to touch the set-aside #264/#265/#274 work. The resulting runtime/config/output/cleanup evidence and independent absence still require terminal counter-review before any shared-219 conclusion. |
Member
Author
|
The one reviewed db7afcf5/bd05be4c invocation is terminal runner-blocked/NONPASS, exit1 after about one second, at the created-profile admission check. Neither the Server nor the SDK consumer started; no callback, timing or history observation ran. No unchanged retry is selected. The full admitted/created projection differs only at Config.Env. Docker applies CLI overrides first and removes the overridden inherited entries; my expected profile instead appended overrides to the inherited environment, preserving duplicate default keys. The created container retained the exact requested environment values, but the expected list representation was wrong. The complete profile guard refused before start. This is a caller expectation defect, not a product failure. Captured container The original packet/config/result and seven-day private evidence are preserved. A narrow successor must encode the actual Docker override/removal order, retain exact closed-profile comparisons, add the predecessor-failing positive control, use a fresh exclusive output root and return for separate exact caller review before another invocation. Source #275 and its main checks remain accepted; shared #219 installed timing remains open. |
Member
Author
|
Narrow caller successor after the first runner-blocked NONPASS, for separate review before any fresh invocation. The first container stayed created/unstarted; no Server or SDK consumer ran. Its exact container/network IDs, ID filters and owner/task/nonce inventories are independently absent, OOMfalse and cleanupPASS. The original packet/config/result and seven-day evidence remain preserved. The controller changes only its admitted Config.Env construction: CLI overrides appear first; inherited entries with those keys are removed; every other inherited entry retains its exact order. Full profile equality remains strict, including order, with no added normalization or accepted field. The complete delta from reviewed db7afcf5 is below. The retained actual first created-profile bytes fail the predecessor and pass this successor. A new inert control also proves the correct override/default behavior and refuses duplicate defaults or reversed order. All 22 controls pass; only pure AST and inert Python language controls ran. A fresh exclusive output root now contains six byte-identical copied inputs: receipt, manifest, lock, test, build messages and compiled executable. The original build/registry archive, test/binary identities and scenario are unchanged; no new build/installation or package release is selected. The literal bootstrap changes only its controller hash and that fixed output/input root. Its exact bytes and literal -c invocation are retained privately for complete review. No fresh Docker/Server/SDK invocation has run. The complete Config/HostConfig/Mounts, stage-exact network membership/attachment, streaming limits, deadlines, synthetic credentials, sealed executable and exact cleanup boundaries are unchanged. This remains one separately reviewed ordinary219 timing observation, with no set-aside item continuation or installed acceptance inferred from source checks. Frozen packet: {
"controller_sha256": "d30462f3a112ee87d8b1fd92a4e43b94509f0dce5e820f9c5351df6cead4183f",
"controls_sha256": "f05f07c7a770e858c81a866531a8439a5589564bed7347bd3f40c8d0268e3460",
"literal_bootstrap_sha256": "3fcdec285cba532023fde16c291dcc76069fc2c87eff28eab5807465cfd86632",
"build_receipt_sha256": "9fff98616c7d87b937ea0d843c8c8ba9cef3271049a884abd49a2aff793bf5b7",
"test_sha256": "f95e947b2b3c199bb929e0f7176fb712d3b0f235612c97cbbae859df6cdaa862",
"binary_sha256": "5cd20d42dc317f6b49637595799af611bcd756a53899e81d1dccb37e9649590a",
"qualified_server": "3.10.0",
"server_source": "b6e04ce7e679464e91f6ceae83310d3bb3204092",
"server_child": "sha256:9d64e9aeee84b46d21715826a3e7e5a96eb8d4ce7fccb126b88b15596ab14973",
"scope": "one ordinary219 local-retry timing observation; no clock/redrive/database operation",
"source_controls": 22,
"workload_seconds": 180,
"parent_seconds": 90,
"cleanup_seconds": 120,
"parent_wait_seconds": 10,
"outer_seconds": 340,
"outer_kill_seconds": 20,
"container_profile": "complete Config/HostConfig/Mounts equality; OomKillDisable false/null only normalized",
"stream_byte_limits": {
"docker_stdout_stderr_combined": 1048576,
"runtime_logs_combined": 131072,
"consumer_stdout_stderr_combined": 1048576,
"cluster_http_body": 65536
},
"network_membership": "empty before start; exact runtime after start; container has only selected network name and stage-matching NetworkID",
"prior_attempt": "runner-blocked/NONPASS before Server/consumer start; exact cleanup independently confirmed",
"environment_profile": "explicit CLI entries first; overridden inherited entries removed; exact remaining order retained"
}Complete controller delta: --- previous-db7afcf5
+++ environment-successor
@@ -346,13 +346,18 @@
assert set(value.get('Containers') or {}) == ({runtime} if runtime else set())
def admitted_profile(image, network, name, command, labels, tmpfs):
+ overrides = ['DB_CONNECTION=sqlite', 'DB_DATABASE=/app/database/runtime.sqlite',
+ 'DW_RUST_MAINTENANCE_LOOP=1']
+ override_keys = {entry.split('=', 1)[0] for entry in overrides}
+ # Docker puts explicit CLI entries first and removes inherited entries
+ # with those keys. Preserve exact order and reject duplicate defaults.
+ environment = overrides + [entry for entry in image['Config']['Env']
+ if entry.split('=', 1)[0] not in override_keys]
config = {
'Hostname': name, 'Domainname': '', 'User': '1000:1000',
'AttachStdin': False, 'AttachStdout': True, 'AttachStderr': True,
'ExposedPorts': {'8080/tcp': {}}, 'Tty': False, 'OpenStdin': False,
- 'StdinOnce': False, 'Env': image['Config']['Env'] + [
- 'DB_CONNECTION=sqlite', 'DB_DATABASE=/app/database/runtime.sqlite',
- 'DW_RUST_MAINTENANCE_LOOP=1'],
+ 'StdinOnce': False, 'Env': environment,
'Cmd': command, 'Image': IMAGE, 'Volumes': image['Config']['Volumes'],
'WorkingDir': '/app', 'Entrypoint': ['/bin/sh'],
'Labels': dict(image['Config']['Labels'], **labels), 'StopSignal': 'SIGTERM'}Exact inert controls: import ast
import copy
import hashlib
import io
import os
import pathlib
import re
import selectors
import stat
import subprocess
import sys
import tempfile
import time
import signal
import unittest
from unittest import mock
SOURCE = pathlib.Path(sys.argv[1])
SUPERVISOR = pathlib.Path(sys.argv[2])
sys.argv = sys.argv[:1]
module = ast.parse(SOURCE.read_text())
keep = {'digest', 'directory', 'read_at', 'exclusive', 'output', 'runtime_command', 'docker_command', 'call', 'validate_network', 'validate_container', 'full_id', 'reconcile_created', 'remove_resource', 'owned', 'validate_records', 'cleanup_resources', 'BudgetExpired', 'workload_deadline', 'admitted_profile', 'OutputOverflow', 'stop_process', 'read_process', 'bounded_run', 'bounded_http_json'}
ns = {'pathlib': pathlib, 'os': os, 'stat': stat, 'hashlib': hashlib, 'subprocess': subprocess,
'json': __import__('json'), 're': re, 'CLOSED_ENV': {'PATH': '/usr/bin:/bin', 'HOME': '/nonexistent', 'LANG': 'C', 'LC_ALL': 'C'},
'IMAGE': 'synthetic-image', 'name':'owned', 'time':time, 'signal':signal, 'selectors':selectors}
exec(compile(ast.Module(body=[node for node in module.body if isinstance(node, (ast.FunctionDef, ast.ClassDef)) and node.name in keep], type_ignores=[]), str(SOURCE), 'exec'), ns)
class Controls(unittest.TestCase):
def test_supervisor_replacement_discriminates_predecessor(self):
script = SUPERVISOR.read_bytes()
self.assertEqual(hashlib.sha256(script).hexdigest(), '25a591f514b3823093a37b66c47c588cba8e983594c8da2d7940ac4f565b922b')
with tempfile.TemporaryDirectory() as tmp:
path = pathlib.Path(tmp) / 'supervisor'
path.write_bytes(script)
captured = path.read_bytes()
command = ns['runtime_command'](captured, int(time.time()) + 10)
command[-2:] = ['none', '/bin/echo', 'ORIGINAL']
path.write_text('printf SUBSTITUTED')
predecessor = subprocess.check_output(['/bin/sh', str(path), '10', '10', 'none', '/bin/echo', 'ORIGINAL'])
self.assertEqual(predecessor, b'SUBSTITUTED')
self.assertEqual(subprocess.check_output(['/bin/sh'] + command), b'ORIGINAL\n')
self.assertNotIn(str(path), command)
def test_exclusive_output_refuses_symlink(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp)
marker = root / 'marker'; marker.write_bytes(b'unchanged')
(root / 'logging.stdout').symlink_to(marker)
(root / 'logging.stdout').write_bytes(b'PREDECESSOR')
self.assertEqual(marker.read_bytes(), b'PREDECESSOR')
marker.write_bytes(b'unchanged')
fd = ns['directory'](root)
try:
with self.assertRaises(FileExistsError): ns['exclusive'](fd, 'logging.stdout')
self.assertEqual(marker.read_bytes(), b'unchanged')
finally: os.close(fd)
def test_output_replacement_refuses_before_foreign_write(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp); fd = ns['directory'](root)
out = ns['exclusive'](fd, 'logging.stdout')
try:
self.assertEqual(stat.S_IMODE(os.fstat(out).st_mode), 0o600)
(root / 'logging.stdout').rename(root / 'held')
marker = root / 'marker'; marker.write_bytes(b'unchanged')
(root / 'logging.stdout').symlink_to(marker)
with self.assertRaises(AssertionError): ns['output'](fd, out, 'logging.stdout', b'bad')
self.assertEqual(marker.read_bytes(), b'unchanged')
finally: os.close(out); os.close(fd)
def test_input_and_root_links_refused(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp); (root / 'alias').symlink_to(root, target_is_directory=True)
with self.assertRaises(OSError): ns['directory'](root / 'alias')
fd = ns['directory'](root)
try:
(root / 'value').write_bytes(b'expected'); (root / 'link').symlink_to(root / 'value')
with self.assertRaises(OSError): ns['read_at'](fd, 'link')
self.assertEqual(ns['read_at'](fd, 'value'), b'expected')
finally: os.close(fd)
def test_effectful_tool_and_environment_are_closed(self):
configpath = '/proc/' + str(os.getpid()) + '/fdinfo'
configfd = os.open(configpath, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
ns.update(config_fd=configfd, config_path=configpath, config_info=os.fstat(configfd))
captured = []
def fake(*args, **kwargs):
# Runs after call validation, just before the consumer would open config.
with self.assertRaises(OSError):
opened = os.open('config.json', os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600, dir_fd=configfd)
os.close(opened)
self.assertFalse(os.path.lexists(pathlib.Path(configpath) / 'config.json'))
captured.append((args, kwargs)); return object()
try:
with mock.patch.dict(os.environ, {'PATH': '/synthetic-untrusted', 'DOCKER_HOST': 'tcp://wrong', 'DOCKER_CONFIG': '/wrong', 'DOCKER_CONTEXT': 'wrong'}):
with mock.patch.dict(ns, {'bounded_run':fake}): ns['call'](['docker', 'create', '--pull=never'])
args, kwargs = captured[0]
self.assertEqual(args[0][:3], ['/usr/bin/docker', '--host', 'unix:///run/docker.sock'])
self.assertEqual(kwargs['env'], ns['CLOSED_ENV'])
self.assertEqual(kwargs['pass_fds'], (configfd,))
self.assertEqual(kwargs['cwd'], '/')
self.assertNotIn('DOCKER_HOST', kwargs['env'])
finally: os.close(configfd)
def test_predecessor_config_changes_between_check_and_open(self):
with tempfile.TemporaryDirectory() as tmp:
path = pathlib.Path(tmp) / 'config.json'; path.write_text('{}\n')
before = path.stat()
self.assertEqual(path.read_text(), '{}\n')
# The old validation has completed; a writer changes the same inode.
path.write_text('{"credsStore":"synthetic"}')
self.assertEqual(path.stat().st_ino, before.st_ino)
self.assertEqual(path.read_text(), '{"credsStore":"synthetic"}')
def test_uncertain_network_and_container_replies_recover_full_id(self):
from types import SimpleNamespace
for kind in ('network', 'container'):
for reply in (b'', b'not-an-id', b'A' * 64, b'\xff'):
with self.subTest(kind=kind, reply=reply):
identity = 'a' * 64
labels = {'owner':'sdk-rust','task':'logger227','qualification_nonce':'synthetic-nonce'}
present = {identity}; operations = []
def fake(args, **kwargs):
operations.append(args)
if 'inspect' in args:
selected = args[-1]
if selected not in present:
message = ('No such container: '+selected if kind=='container' else 'network '+selected+' not found').encode()
return SimpleNamespace(returncode=1, stderr=message, stdout=b'[]\n')
value={'Id':selected,'Name':'owned','Config':{'Labels':labels},'Labels':labels,'State':{'OOMKilled':False}}
return SimpleNamespace(returncode=0, stderr=b'', stdout=ns['json'].dumps([value]).encode())
if 'rm' in args:
self.assertEqual(args[-1], identity); present.remove(identity)
return SimpleNamespace(returncode=0, stderr=b'', stdout=identity.encode())
if 'id='+identity in args:
return SimpleNamespace(returncode=0, stderr=b'', stdout=(identity.encode() if present else b''))
for key,val in labels.items():self.assertIn('label='+key+'='+val, args)
self.assertIn('--no-trunc', args)
return SimpleNamespace(returncode=0, stderr=b'', stdout=(identity+'\n').encode())
old_identity = reply.decode('ascii', errors='replace').strip()
self.assertIsNotNone(old_identity) # Old is-None recovery skips every reply.
granted = None
with self.assertRaises((AssertionError,UnicodeError)):
granted = ns['full_id'](reply)
self.assertIsNone(granted)
with mock.patch.dict(ns, {'labels':labels,'call':fake}):
recovered=ns['reconcile_created'](kind,granted)
self.assertEqual(recovered,identity)
ns['remove_resource'](kind,recovered)
self.assertEqual(present,set())
self.assertIn('id='+identity,operations[-1])
def test_configuration_mutations_are_refused(self):
image = {'Id': 'synthetic-id', 'Config': {'Env': ['PATH=/usr/bin'], 'Labels': {}, 'Volumes': {'/app/database': {}, '/app/storage': {}}}}
command = ['-c', 'captured']; tmpfs = {'/tmp': 'size=64m'}
value=ns['admitted_profile'](image,'network-id','owned',command,{'owner':'sdk-rust'},tmpfs)
value['NetworkSettings']={'Networks':{'owned':{'NetworkID':''}}}
config=value['Config'];host=value['HostConfig']
check=lambda v: ns['validate_container'](v,image,'network-id','owned',command,{'owner':'sdk-rust'},tmpfs)
check(value)
started=copy.deepcopy(value);started['HostConfig']['OomKillDisable']=None
self.assertEqual(check(value),check(started))
mutations=[('Image',None,'wrong'),('Config','Cmd',['wrong']),('Config','Env',config['Env']+['INJECTED=yes']),('Config','Labels',{}),
('HostConfig','Privileged',True),('HostConfig','NanoCpus',0),('HostConfig','Memory',0),('HostConfig','ReadonlyRootfs',False),
('HostConfig','Binds',['synthetic:/app']),('HostConfig','NetworkMode','host'),('HostConfig','SecurityOpt',[]),('HostConfig','OomKillDisable',True),('Mounts',None,[{'Type':'bind','Destination':'/app'}])]
for group,key,new in mutations:
with self.subTest(field=(group,key)):
changed=copy.deepcopy(value)
if key is None:changed[group]=new
else:changed[group][key]=new
with self.assertRaises(AssertionError):check(changed)
def test_network_profile_refuses_wrong_routing_or_attachment(self):
value={'Driver':'bridge','Scope':'local','Internal':False,'Ingress':False,'Attachable':False,'EnableIPv6':False,
'Options':{'com.docker.network.bridge.enable_ip_masquerade':'false','com.docker.network.bridge.enable_icc':'false','com.docker.network.bridge.host_binding_ipv4':'127.0.0.1'},'Containers':{}}
ns['validate_network'](value)
value['Containers']={'owned-runtime':{}}
ns['validate_network'](value,'owned-runtime')
for key,bad in [('Internal',True),('Driver','host'),('Attachable',True),('Containers',{'foreign':{}}),('Options',{})]:
with self.subTest(field=key):
changed=copy.deepcopy(value);changed[key]=bad
with self.assertRaises(AssertionError):ns['validate_network'](changed,'owned-runtime')
def records(self):
records=[]
for index,(case,delays) in enumerate([('short',[1,2,2,2]),('empty',[0,0,0,0])],1):
workflow='rust-retry-'+case+'-00000000-0000-0000-0000-'+str(index).zfill(12)
run='run-'+case
retries=[{'sequence':1,'activity_execution_id':'execution-'+case,'execution_mode':'local',
'local_activity':True,'retry_after_attempt':attempt,'retry_backoff_seconds':delay}
for attempt,delay in enumerate(delays,1)]
records.append({'case':case,'workflow_id':workflow,'run_id':run,'expected_backoff_seconds':delays,
'callback_gap_ms':[delay*1000 for delay in delays],'retry_reports':retries,'callback_attempts':5,
'callbacks_after_replay':0,'history':{'history_complete':True,'workflow':{
'instance_id':workflow,'run_id':run,'details_state':'retained'},
'history_events':[{'type':'ActivityRetryScheduled','payload':row} for row in retries]}})
return records
def test_observation_accepts_only_both_bound_cases(self):
ns['validate_records'](self.records())
for change in ('early','zero-tail','remote','incomplete','extra-callback','duplicate-run','missing-zero'):
with self.subTest(change=change):
records=self.records()
if change=='early':records[0]['callback_gap_ms'][2]=0
if change=='zero-tail':records[0]['retry_reports'][3]['retry_backoff_seconds']=0
if change=='remote':records[0]['retry_reports'][0]['execution_mode']='remote'
if change=='incomplete':records[0]['history']['history_complete']=False
if change=='extra-callback':records[0]['callbacks_after_replay']=1
if change=='duplicate-run':records[1]['run_id']=records[0]['run_id']
if change=='missing-zero':records[1]['retry_reports'][0]['retry_backoff_seconds']=None
with self.assertRaises((AssertionError,KeyError)):ns['validate_records'](records)
def test_startup_guard_is_before_root_access(self):
guard=next(n for n in module.body if isinstance(n,ast.If) and 'unsafe caller' in ast.unparse(n))
root=next(n for n in module.body if isinstance(n,ast.Assign) and any(isinstance(t,ast.Name) and t.id=='ROOT' for t in n.targets))
self.assertLess(guard.lineno,root.lineno)
from types import SimpleNamespace
for opt,site,bytecode in [(1,1,True),(0,0,True),(0,1,False)]:
fake=SimpleNamespace(flags=SimpleNamespace(optimize=opt,no_site=site),dont_write_bytecode=bytecode)
with self.assertRaises(RuntimeError):exec(compile(ast.Module(body=[guard],type_ignores=[]),'guard','exec'),{'sys':fake})
def test_deadline_recovers_committed_unknown_create(self):
for pending in ('container','network'):
present={pending:'a'*64};removed=[];result={'outcome':'fail'}
def recover(kind,identity):return identity if identity is not None else present.get(kind)
def remove(kind,identity):
self.assertEqual(present.pop(kind),identity);removed.append(identity);return {'State':{'OOMKilled':False}}
with mock.patch.dict(ns,{'reconcile_created':recover,'remove_resource':remove}):
try:ns['workload_deadline'](signal.SIGTERM,None)
except ns['BudgetExpired']:errors=ns['cleanup_resources'](None,None,result,pending,seconds=1)
self.assertEqual(errors,[]);self.assertEqual(present,{});self.assertEqual(removed,['a'*64])
def test_unresolved_create_is_nonpass_and_cleanup_calls_are_budgeted(self):
seen=[]
def call(args,wall=30,**kwargs):seen.append(wall)
def recover(kind,identity):ns['call'](['inert-query']);return None
with mock.patch.dict(ns,{'call':call,'reconcile_created':recover}):
errors=ns['cleanup_resources'](None,None,{},'container',seconds=.5)
self.assertIs(ns['call'],call)
self.assertEqual(errors,['container:uncertain-create'])
self.assertTrue(seen);self.assertTrue(all(0<wall<=.5 for wall in seen))
def test_literal_bootstrap_refuses_substitution_before_execution(self):
bootstrap = ast.parse(SOURCE.with_name('219-caller-bootstrap.py').read_bytes())
function = next(node for node in bootstrap.body if isinstance(node, ast.FunctionDef) and node.name == 'capture')
scope = {'os': os, 'pathlib': pathlib, 'stat': stat, 'hashlib': hashlib}
exec(compile(ast.Module(body=[function], type_ignores=[]), '<capture-control>', 'exec'), scope)
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp)
source = root / 'caller.py'
marker = []
original = b'marker.append("original")'
source.write_bytes(original)
expected = hashlib.sha256(original).hexdigest()
captured = scope['capture'](str(source), expected)
source.write_bytes(b'marker.append("replacement")')
exec(compile(source.read_bytes(), '<pathname-predecessor>', 'exec'), {'marker': marker})
self.assertEqual(marker, ['replacement'])
marker.clear()
with self.assertRaisesRegex(RuntimeError, 'source hash refused'):
scope['capture'](str(source), expected)
self.assertEqual(marker, [])
exec(compile(captured, '<captured-candidate>', 'exec'), {'marker': marker})
self.assertEqual(marker, ['original'])
source.unlink()
target = root / 'target.py'; target.write_bytes(original)
source.symlink_to(target)
with self.assertRaises(OSError):
scope['capture'](str(source), expected)
def test_complete_profile_discriminates_stable_predecessor_gaps(self):
predecessor = ast.parse(SOURCE.with_name('hold1-manual-219-timing.py').read_bytes())
old = {'json':ns['json'],'digest':ns['digest'],'IMAGE':'synthetic-image'}
nodes=[node for node in predecessor.body if isinstance(node,ast.FunctionDef) and node.name=='validate_container']
exec(compile(ast.Module(body=nodes,type_ignores=[]),'<profile-predecessor>','exec'),old)
image={'Id':'synthetic-id','Config':{'Env':['PATH=/usr/bin'],'Volumes':{},'Labels':{}}}
args=(image,'network-id','owned',['-c','captured'],{'owner':'sdk-rust'},{'/tmp':'size=64m'})
value=ns['admitted_profile'](*args)
value['NetworkSettings']={'Networks':{'owned':{'NetworkID':''}}}
old['validate_container'](value,*args)
mutations={'CgroupnsMode':'host','GroupAdd':['0'],'Dns':['203.0.113.1'],
'Sysctls':{'kernel.shmmax':'999999'},'Ulimits':[{'Name':'nofile','Hard':1048576,'Soft':1048576}],
'ReadonlyPaths':[],'MaskedPaths':[],'LogConfig':{'Type':'syslog','Config':{}},
'PublishAllPorts':True,'OomScoreAdj':-1000,'CgroupParent':'/'}
for field,change in mutations.items():
with self.subTest(field=field):
changed=copy.deepcopy(value);changed['HostConfig'][field]=change
old['validate_container'](changed,*args)
with self.assertRaises(AssertionError):ns['validate_container'](changed,*args)
for group in ('Config','HostConfig'):
added=copy.deepcopy(value);added[group]['unknown_field']='unselected'
with self.assertRaises(AssertionError):ns['validate_container'](added,*args)
missing=copy.deepcopy(value);missing[group].pop(next(iter(missing[group])))
with self.assertRaises(AssertionError):ns['validate_container'](missing,*args)
mounted=copy.deepcopy(value);mounted['Mounts']=[{'Type':'tmpfs','Destination':'/tmp','RW':True}]
with self.assertRaises(AssertionError):ns['validate_container'](mounted,*args)
def test_streaming_process_bounds_kill_and_reap(self):
# Inert language-only subprocesses; no Docker, SDK or service invocation.
for target in ('stdout','stderr','combined'):
with self.subTest(target=target):
code={'stdout':'os.write(1,b"x"*4096)',
'stderr':'os.write(2,b"x"*4096)',
'combined':'os.write(1,b"x"*80);os.write(2,b"x"*80)'}[target]
process=subprocess.Popen([sys.executable,'-I','-S','-B','-c',
'import os,time;'+code+';time.sleep(10)'],stdout=subprocess.PIPE,
stderr=subprocess.PIPE,start_new_session=True)
with self.assertRaises(ns['OutputOverflow']):ns['read_process'](process,2,128)
self.assertIsNotNone(process.returncode)
self.assertTrue(process.stdout.closed and process.stderr.closed)
old=subprocess.run([sys.executable,'-I','-S','-B','-c',
'import os;os.write(1,b"x"*4096)'],capture_output=True,check=True)
self.assertGreater(len(old.stdout),128) # Prior post-capture check is already too late.
def test_bounded_run_and_timeout_return_no_live_child(self):
result=ns['bounded_run']([sys.executable,'-I','-S','-B','-c',
'import os,sys;os.write(1,sys.stdin.buffer.read());os.write(2,b"err")'],
wall=2,byte_limit=128,input=b'fixture')
self.assertEqual((result.stdout,result.stderr),(b'fixture',b'err'))
process=subprocess.Popen([sys.executable,'-I','-S','-B','-c',
'import time;time.sleep(10)'],stdout=subprocess.PIPE,stderr=subprocess.PIPE,start_new_session=True)
with self.assertRaises(subprocess.TimeoutExpired):ns['read_process'](process,.02,128)
self.assertIsNotNone(process.returncode)
def test_http_body_refuses_before_unbounded_read(self):
class BoundedBody:
def __init__(self,value):self.value=io.BytesIO(value);self.requests=[]
def read(self,size):
self.assert_size(size);self.requests.append(size);return self.value.read(size)
def assert_size(self,size):
if not 0<size<=8192:raise AssertionError('unbounded response read')
good=BoundedBody(b'{"version":"3.10.0"}')
self.assertEqual(ns['bounded_http_json'](good),{'version':'3.10.0'})
oversized=BoundedBody(b' '*4096)
with self.assertRaises(ns['OutputOverflow']):ns['bounded_http_json'](oversized,128)
self.assertEqual(oversized.value.tell(),129)
self.assertGreater(len(io.BytesIO(b' '*4096).read()),128) # Prior unbounded JSON read.
def test_overflow_still_reconciles_and_removes_owned_resources(self):
removed=[];result={'outcome':'fail'}
def remove(kind,identity):removed.append((kind,identity));return {'State':{'OOMKilled':False}}
with mock.patch.dict(ns,{'reconcile_created':lambda kind,identity:identity,'remove_resource':remove,
'call':lambda *a,**k:(_ for _ in ()).throw(ns['OutputOverflow']('log limit'))}):
errors=ns['cleanup_resources']('a'*64,'b'*64,result,None,seconds=1,collect_logs=True)
self.assertEqual(removed,[('container','a'*64),('network','b'*64)])
self.assertEqual(errors,['runtime-logs:OutputOverflow'])
self.assertEqual(result['outcome'],'fail')
def test_selected_network_membership_is_exact_by_stage(self):
predecessor=ast.parse(SOURCE.with_name('hold2-manual-219-timing.py').read_bytes())
old={}
node=next(n for n in predecessor.body if isinstance(n,ast.FunctionDef) and n.name=='validate_network')
exec(compile(ast.Module(body=[node],type_ignores=[]),'<network-predecessor>','exec'),old)
value={'Driver':'bridge','Scope':'local','Internal':False,'Ingress':False,'Attachable':False,'EnableIPv6':False,
'Options':{'com.docker.network.bridge.enable_ip_masquerade':'false',
'com.docker.network.bridge.enable_icc':'false',
'com.docker.network.bridge.host_binding_ipv4':'127.0.0.1'},'Containers':{}}
ns['validate_network'](value) # Before start, the network has no endpoints.
old['validate_network'](value,'owned-runtime') # The prior subset admitted a missing endpoint.
with self.assertRaises(AssertionError):ns['validate_network'](value,'owned-runtime')
value['Containers']={'owned-runtime':{}}
ns['validate_network'](value,'owned-runtime')
with self.assertRaises(AssertionError):ns['validate_network'](value)
value['Containers']['foreign-runtime']={}
with self.assertRaises(AssertionError):ns['validate_network'](value,'owned-runtime')
def test_container_refuses_missing_extra_or_wrong_selected_attachment(self):
predecessor=ast.parse(SOURCE.with_name('hold2-manual-219-timing.py').read_bytes())
old={key:ns[key] for key in ('json','digest','admitted_profile')}
node=next(n for n in predecessor.body if isinstance(n,ast.FunctionDef) and n.name=='validate_container')
exec(compile(ast.Module(body=[node],type_ignores=[]),'<attachment-predecessor>','exec'),old)
image={'Id':'synthetic-id','Config':{'Env':['PATH=/usr/bin'],'Volumes':{},'Labels':{}}}
args=(image,'network-id','owned',['-c','captured'],{'owner':'sdk-rust'},{'/tmp':'size=64m'})
value=ns['admitted_profile'](*args)
value['NetworkSettings']={'Networks':{'owned':{'NetworkID':''}}}
created=ns['validate_container'](value,*args)
active=copy.deepcopy(value);active['NetworkSettings']['Networks']['owned']['NetworkID']='network-id'
self.assertEqual(ns['validate_container'](active,*args,started=True),created)
for started,base in [(False,value),(True,active)]:
for case in ('missing','extra','wrong-id'):
with self.subTest(started=started,case=case):
changed=copy.deepcopy(base);networks=changed['NetworkSettings']['Networks']
if case=='missing':networks.clear()
if case=='extra':networks['foreign']={'NetworkID':'foreign-id'}
if case=='wrong-id':networks['owned']['NetworkID']='foreign-id'
old['validate_container'](changed,*args)
with self.assertRaises(AssertionError):ns['validate_container'](changed,*args,started=started)
with self.assertRaises(AssertionError):ns['validate_container'](value,*args,started=True)
with self.assertRaises(AssertionError):ns['validate_container'](active,*args)
def test_docker_environment_overrides_remove_inherited_defaults(self):
predecessor=ast.parse(SOURCE.with_name('attempt1-manual-219-timing.py').read_bytes())
old={'json':ns['json'],'digest':ns['digest'],'IMAGE':'synthetic-image'}
nodes=[n for n in predecessor.body if isinstance(n,ast.FunctionDef) and n.name in ('admitted_profile','validate_container')]
exec(compile(ast.Module(body=nodes,type_ignores=[]),'<environment-predecessor>','exec'),old)
image={'Id':'synthetic-id','Config':{'Env':['PATH=/usr/bin','DW_BIND_ADDRESS=0.0.0.0:8080',
'DW_RUST_MAINTENANCE_LOOP=1','DB_DATABASE=/app/database/database.sqlite'],'Volumes':{},'Labels':{}}}
args=(image,'network-id','owned',['-c','captured'],{'owner':'sdk-rust'},{'/tmp':'size=64m'})
value=ns['admitted_profile'](*args)
value['NetworkSettings']={'Networks':{'owned':{'NetworkID':''}}}
self.assertEqual(value['Config']['Env'],['DB_CONNECTION=sqlite',
'DB_DATABASE=/app/database/runtime.sqlite','DW_RUST_MAINTENANCE_LOOP=1',
'PATH=/usr/bin','DW_BIND_ADDRESS=0.0.0.0:8080'])
ns['validate_container'](value,*args)
with self.assertRaises(AssertionError):old['validate_container'](value,*args)
changed=copy.deepcopy(value);changed['Config']['Env'].append('DB_DATABASE=/app/database/database.sqlite')
with self.assertRaises(AssertionError):ns['validate_container'](changed,*args)
changed=copy.deepcopy(value);changed['Config']['Env'].reverse()
with self.assertRaises(AssertionError):ns['validate_container'](changed,*args)
unittest.main() |
Member
Author
|
Security terminal review of ordinary-219 attempt 1 and narrow successor disposition. Attempt 1 is accepted as runner-blocked/NONPASS, not an SDK or Server observation. The retained result binds container 3c4a3b2f4ad111c2bf8bfed3a3118f1c6618289b57d920f8da8ea58a6663755a and network 75228e9e7e04f94f63ebf890dd4f987af013ade3c289859b305d5a92bcc69354. The container remained created/unstarted: Running false, PID 0, zero start/finish timestamps, OOM false. Retry/runtime stdout and stderr are all zero bytes. Cleanup reports PASS, and the independent readback shows full-ID inspect absence, empty ID filters and empty exact owner/task/nonce inventories for both resources. No SDK consumer or Server process started. The only profile delta is Config.Env ordering/replacement. Preserve this result as NONPASS. Narrow source/profile PASS for one fresh invocation at controller d30462f3a112ee87d8b1fd92a4e43b94509f0dce5e820f9c5351df6cead4183f, controls f05f07c7a770e858c81a866531a8439a5589564bed7347bd3f40c8d0268e3460 and bootstrap 3fcdec285cba532023fde16c291dcc76069fc2c87eff28eab5807465cfd86632. I statically reviewed the complete r4 delta and never imported or executed the effectful controller/bootstrap. The correction models Docker29 exactly: explicit CLI entries first, all inherited entries with overridden keys removed, every other inherited entry retained in exact order. Full Config equality remains strict; no normalization or additional accepted field was added. Against the retained actual created profile, the predecessor refuses and the candidate passes with byte-for-byte equal Env. Reversed order and duplicate restored defaults refuse. The pure AST/inert harness passes 22/22. The fresh r2 input root contains six byte-identical copies of the original receipt, manifest, lock, test, build messages and executable; I independently matched every byte count and SHA-256. No rebuild or reinstall occurred. One fresh bounded invocation is permitted under these exact hashes/root only. Terminal runtime/config/output/cleanup and independent absence evidence still require counter-review before any shared-219 conclusion. This grants no SDK source/release/publication authority and does not touch #264/#265/#274 or any other set-aside item. |
Member
Author
|
The one admitted r2 invocation is terminal runner-blocked/NONPASS, exit1 at the created-profile check, before any Server or consumer start. The prior expected-value correction removed overridden inherited defaults correctly, but I still assumed a stable CLI-entry order. Docker29 returned the exact same five unique key/value entries with its three CLI entries reordered. The full admitted/actual projection differs only in Config.Env ordering. Container Both original NONPASS packets remain preserved. No unchanged retry is selected. The narrow successor must normalize only ordering after proving every environment key unique, then compare the complete admitted key/value entries; duplicates, missing/extra keys and changed values must refuse. A fresh exclusive root and new exact caller review remain before any further invocation. No source/package/release or shared acceptance is inferred from these setup attempts. |
Member
Author
|
Narrow unique-environment successor for separate review after attempt2 runner-blocked NONPASS. Both attempts stopped at created-profile admission before any Server or SDK consumer start. Both full-ID and exact label inventories are independently absent; outputs empty, OOMfalse/cleanupPASS. Both original packets and seven-day evidence are preserved. The two retained creates contain the same five unique environment entries in different orders. My prior claim that Docker preserves explicit CLI order was too strong. This successor normalizes only Env ordering, after requiring a list of valid encoded entries with unique keys. It compares the complete sorted entries against the exact admitted values. Duplicate identical or conflicting keys, missing/extra/malformed entries and changed values refuse. The complete Config/HostConfig/Mounts and network profiles remain closed; no other field or normalization was added. The full narrow delta from reviewed d30462f3 is below. All 23 inert controls pass, including all 120 permutations of the five valid entries and refusal of every value/key defect above. The actual retained created profiles from both attempts pass the candidate; the ordered predecessor fails the second profile. These checks extract only declarations by AST. No new Docker object, SDK/Server start or timing observation ran. The fresh exclusive r3 root contains six byte-identical copies of the original receipt/manifest/lock/test/build messages/binary; no rebuild or reinstall occurred. Bootstrap changes only its controller pin and fixed root. Full exact private invocation remains literal This is a request for one new separately reviewed bounded ordinary219 invocation, with all original output/deadline/resource/cleanup limits retained. No set-aside item or package/release/shared acceptance is selected. Frozen packet: {
"controller_sha256": "972598460bdc11068ca4d88e00077117511f99cefdf6ac0d252551ae2f951dd8",
"controls_sha256": "84002d1f6833aaa65fa1c12f76f365bd3b9f547cab7c844b7840c91f9e6ca998",
"literal_bootstrap_sha256": "9bc692292fe1b1e0feb190d96b14c1db704ac91534f4014ed4d9effcf6b538d7",
"build_receipt_sha256": "9fff98616c7d87b937ea0d843c8c8ba9cef3271049a884abd49a2aff793bf5b7",
"test_sha256": "f95e947b2b3c199bb929e0f7176fb712d3b0f235612c97cbbae859df6cdaa862",
"binary_sha256": "5cd20d42dc317f6b49637595799af611bcd756a53899e81d1dccb37e9649590a",
"qualified_server": "3.10.0",
"server_source": "b6e04ce7e679464e91f6ceae83310d3bb3204092",
"server_child": "sha256:9d64e9aeee84b46d21715826a3e7e5a96eb8d4ce7fccb126b88b15596ab14973",
"scope": "one ordinary219 local-retry timing observation; no clock/redrive/database operation",
"source_controls": 23,
"workload_seconds": 180,
"parent_seconds": 90,
"cleanup_seconds": 120,
"parent_wait_seconds": 10,
"outer_seconds": 340,
"outer_kill_seconds": 20,
"container_profile": "complete Config/HostConfig/Mounts equality; only unique-key Env order and OomKillDisable false/null normalized",
"stream_byte_limits": {
"docker_stdout_stderr_combined": 1048576,
"runtime_logs_combined": 131072,
"consumer_stdout_stderr_combined": 1048576,
"cluster_http_body": 65536
},
"network_membership": "empty before start; exact runtime after start; container has only selected network name and stage-matching NetworkID",
"prior_attempt": "two runner-blocked/NONPASS invocations before Server/consumer start; exact cleanup independently confirmed",
"environment_profile": "unique keys required; exact complete entries compared after order canonicalization; duplicate/missing/extra/malformed/changed values refuse"
}Complete controller delta: --- previous-d30462f3
+++ unique-environment-successor
@@ -349,10 +349,10 @@
overrides = ['DB_CONNECTION=sqlite', 'DB_DATABASE=/app/database/runtime.sqlite',
'DW_RUST_MAINTENANCE_LOOP=1']
override_keys = {entry.split('=', 1)[0] for entry in overrides}
- # Docker puts explicit CLI entries first and removes inherited entries
- # with those keys. Preserve exact order and reject duplicate defaults.
- environment = overrides + [entry for entry in image['Config']['Env']
- if entry.split('=', 1)[0] not in override_keys]
+ # Docker removes overridden inherited entries; its explicit-entry order
+ # varied between the two retained creates. Canonicalize order only.
+ environment = sorted(overrides + [entry for entry in image['Config']['Env']
+ if entry.split('=', 1)[0] not in override_keys])
config = {
'Hostname': name, 'Domainname': '', 'User': '1000:1000',
'AttachStdin': False, 'AttachStdout': True, 'AttachStderr': True,
@@ -468,6 +468,12 @@
# a missing selected network or an additional attachment.
assert networks[name]['NetworkID'] == (network if started else '')
packet = json.loads(json.dumps({k: value[k] for k in ('Image', 'Config', 'HostConfig', 'Mounts')}))
+ environment = packet['Config']['Env']
+ assert isinstance(environment, list)
+ assert all(isinstance(entry, str) and '=' in entry for entry in environment)
+ keys = [entry.split('=', 1)[0] for entry in environment]
+ assert len(keys) == len(set(keys))
+ packet['Config']['Env'] = sorted(environment)
# Docker29 changes this single unset/false default during start. Both retain
# the kernel OOM killer; true and all other profile differences refuse.
assert 'OomKillDisable' in packet['HostConfig']Exact inert controls: import ast
import copy
import hashlib
import io
import os
import pathlib
import re
import selectors
import stat
import subprocess
import sys
import tempfile
import time
import signal
import unittest
from unittest import mock
SOURCE = pathlib.Path(sys.argv[1])
SUPERVISOR = pathlib.Path(sys.argv[2])
sys.argv = sys.argv[:1]
module = ast.parse(SOURCE.read_text())
keep = {'digest', 'directory', 'read_at', 'exclusive', 'output', 'runtime_command', 'docker_command', 'call', 'validate_network', 'validate_container', 'full_id', 'reconcile_created', 'remove_resource', 'owned', 'validate_records', 'cleanup_resources', 'BudgetExpired', 'workload_deadline', 'admitted_profile', 'OutputOverflow', 'stop_process', 'read_process', 'bounded_run', 'bounded_http_json'}
ns = {'pathlib': pathlib, 'os': os, 'stat': stat, 'hashlib': hashlib, 'subprocess': subprocess,
'json': __import__('json'), 're': re, 'CLOSED_ENV': {'PATH': '/usr/bin:/bin', 'HOME': '/nonexistent', 'LANG': 'C', 'LC_ALL': 'C'},
'IMAGE': 'synthetic-image', 'name':'owned', 'time':time, 'signal':signal, 'selectors':selectors}
exec(compile(ast.Module(body=[node for node in module.body if isinstance(node, (ast.FunctionDef, ast.ClassDef)) and node.name in keep], type_ignores=[]), str(SOURCE), 'exec'), ns)
class Controls(unittest.TestCase):
def test_supervisor_replacement_discriminates_predecessor(self):
script = SUPERVISOR.read_bytes()
self.assertEqual(hashlib.sha256(script).hexdigest(), '25a591f514b3823093a37b66c47c588cba8e983594c8da2d7940ac4f565b922b')
with tempfile.TemporaryDirectory() as tmp:
path = pathlib.Path(tmp) / 'supervisor'
path.write_bytes(script)
captured = path.read_bytes()
command = ns['runtime_command'](captured, int(time.time()) + 10)
command[-2:] = ['none', '/bin/echo', 'ORIGINAL']
path.write_text('printf SUBSTITUTED')
predecessor = subprocess.check_output(['/bin/sh', str(path), '10', '10', 'none', '/bin/echo', 'ORIGINAL'])
self.assertEqual(predecessor, b'SUBSTITUTED')
self.assertEqual(subprocess.check_output(['/bin/sh'] + command), b'ORIGINAL\n')
self.assertNotIn(str(path), command)
def test_exclusive_output_refuses_symlink(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp)
marker = root / 'marker'; marker.write_bytes(b'unchanged')
(root / 'logging.stdout').symlink_to(marker)
(root / 'logging.stdout').write_bytes(b'PREDECESSOR')
self.assertEqual(marker.read_bytes(), b'PREDECESSOR')
marker.write_bytes(b'unchanged')
fd = ns['directory'](root)
try:
with self.assertRaises(FileExistsError): ns['exclusive'](fd, 'logging.stdout')
self.assertEqual(marker.read_bytes(), b'unchanged')
finally: os.close(fd)
def test_output_replacement_refuses_before_foreign_write(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp); fd = ns['directory'](root)
out = ns['exclusive'](fd, 'logging.stdout')
try:
self.assertEqual(stat.S_IMODE(os.fstat(out).st_mode), 0o600)
(root / 'logging.stdout').rename(root / 'held')
marker = root / 'marker'; marker.write_bytes(b'unchanged')
(root / 'logging.stdout').symlink_to(marker)
with self.assertRaises(AssertionError): ns['output'](fd, out, 'logging.stdout', b'bad')
self.assertEqual(marker.read_bytes(), b'unchanged')
finally: os.close(out); os.close(fd)
def test_input_and_root_links_refused(self):
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp); (root / 'alias').symlink_to(root, target_is_directory=True)
with self.assertRaises(OSError): ns['directory'](root / 'alias')
fd = ns['directory'](root)
try:
(root / 'value').write_bytes(b'expected'); (root / 'link').symlink_to(root / 'value')
with self.assertRaises(OSError): ns['read_at'](fd, 'link')
self.assertEqual(ns['read_at'](fd, 'value'), b'expected')
finally: os.close(fd)
def test_effectful_tool_and_environment_are_closed(self):
configpath = '/proc/' + str(os.getpid()) + '/fdinfo'
configfd = os.open(configpath, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
ns.update(config_fd=configfd, config_path=configpath, config_info=os.fstat(configfd))
captured = []
def fake(*args, **kwargs):
# Runs after call validation, just before the consumer would open config.
with self.assertRaises(OSError):
opened = os.open('config.json', os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600, dir_fd=configfd)
os.close(opened)
self.assertFalse(os.path.lexists(pathlib.Path(configpath) / 'config.json'))
captured.append((args, kwargs)); return object()
try:
with mock.patch.dict(os.environ, {'PATH': '/synthetic-untrusted', 'DOCKER_HOST': 'tcp://wrong', 'DOCKER_CONFIG': '/wrong', 'DOCKER_CONTEXT': 'wrong'}):
with mock.patch.dict(ns, {'bounded_run':fake}): ns['call'](['docker', 'create', '--pull=never'])
args, kwargs = captured[0]
self.assertEqual(args[0][:3], ['/usr/bin/docker', '--host', 'unix:///run/docker.sock'])
self.assertEqual(kwargs['env'], ns['CLOSED_ENV'])
self.assertEqual(kwargs['pass_fds'], (configfd,))
self.assertEqual(kwargs['cwd'], '/')
self.assertNotIn('DOCKER_HOST', kwargs['env'])
finally: os.close(configfd)
def test_predecessor_config_changes_between_check_and_open(self):
with tempfile.TemporaryDirectory() as tmp:
path = pathlib.Path(tmp) / 'config.json'; path.write_text('{}\n')
before = path.stat()
self.assertEqual(path.read_text(), '{}\n')
# The old validation has completed; a writer changes the same inode.
path.write_text('{"credsStore":"synthetic"}')
self.assertEqual(path.stat().st_ino, before.st_ino)
self.assertEqual(path.read_text(), '{"credsStore":"synthetic"}')
def test_uncertain_network_and_container_replies_recover_full_id(self):
from types import SimpleNamespace
for kind in ('network', 'container'):
for reply in (b'', b'not-an-id', b'A' * 64, b'\xff'):
with self.subTest(kind=kind, reply=reply):
identity = 'a' * 64
labels = {'owner':'sdk-rust','task':'logger227','qualification_nonce':'synthetic-nonce'}
present = {identity}; operations = []
def fake(args, **kwargs):
operations.append(args)
if 'inspect' in args:
selected = args[-1]
if selected not in present:
message = ('No such container: '+selected if kind=='container' else 'network '+selected+' not found').encode()
return SimpleNamespace(returncode=1, stderr=message, stdout=b'[]\n')
value={'Id':selected,'Name':'owned','Config':{'Labels':labels},'Labels':labels,'State':{'OOMKilled':False}}
return SimpleNamespace(returncode=0, stderr=b'', stdout=ns['json'].dumps([value]).encode())
if 'rm' in args:
self.assertEqual(args[-1], identity); present.remove(identity)
return SimpleNamespace(returncode=0, stderr=b'', stdout=identity.encode())
if 'id='+identity in args:
return SimpleNamespace(returncode=0, stderr=b'', stdout=(identity.encode() if present else b''))
for key,val in labels.items():self.assertIn('label='+key+'='+val, args)
self.assertIn('--no-trunc', args)
return SimpleNamespace(returncode=0, stderr=b'', stdout=(identity+'\n').encode())
old_identity = reply.decode('ascii', errors='replace').strip()
self.assertIsNotNone(old_identity) # Old is-None recovery skips every reply.
granted = None
with self.assertRaises((AssertionError,UnicodeError)):
granted = ns['full_id'](reply)
self.assertIsNone(granted)
with mock.patch.dict(ns, {'labels':labels,'call':fake}):
recovered=ns['reconcile_created'](kind,granted)
self.assertEqual(recovered,identity)
ns['remove_resource'](kind,recovered)
self.assertEqual(present,set())
self.assertIn('id='+identity,operations[-1])
def test_configuration_mutations_are_refused(self):
image = {'Id': 'synthetic-id', 'Config': {'Env': ['PATH=/usr/bin'], 'Labels': {}, 'Volumes': {'/app/database': {}, '/app/storage': {}}}}
command = ['-c', 'captured']; tmpfs = {'/tmp': 'size=64m'}
value=ns['admitted_profile'](image,'network-id','owned',command,{'owner':'sdk-rust'},tmpfs)
value['NetworkSettings']={'Networks':{'owned':{'NetworkID':''}}}
config=value['Config'];host=value['HostConfig']
check=lambda v: ns['validate_container'](v,image,'network-id','owned',command,{'owner':'sdk-rust'},tmpfs)
check(value)
started=copy.deepcopy(value);started['HostConfig']['OomKillDisable']=None
self.assertEqual(check(value),check(started))
mutations=[('Image',None,'wrong'),('Config','Cmd',['wrong']),('Config','Env',config['Env']+['INJECTED=yes']),('Config','Labels',{}),
('HostConfig','Privileged',True),('HostConfig','NanoCpus',0),('HostConfig','Memory',0),('HostConfig','ReadonlyRootfs',False),
('HostConfig','Binds',['synthetic:/app']),('HostConfig','NetworkMode','host'),('HostConfig','SecurityOpt',[]),('HostConfig','OomKillDisable',True),('Mounts',None,[{'Type':'bind','Destination':'/app'}])]
for group,key,new in mutations:
with self.subTest(field=(group,key)):
changed=copy.deepcopy(value)
if key is None:changed[group]=new
else:changed[group][key]=new
with self.assertRaises(AssertionError):check(changed)
def test_network_profile_refuses_wrong_routing_or_attachment(self):
value={'Driver':'bridge','Scope':'local','Internal':False,'Ingress':False,'Attachable':False,'EnableIPv6':False,
'Options':{'com.docker.network.bridge.enable_ip_masquerade':'false','com.docker.network.bridge.enable_icc':'false','com.docker.network.bridge.host_binding_ipv4':'127.0.0.1'},'Containers':{}}
ns['validate_network'](value)
value['Containers']={'owned-runtime':{}}
ns['validate_network'](value,'owned-runtime')
for key,bad in [('Internal',True),('Driver','host'),('Attachable',True),('Containers',{'foreign':{}}),('Options',{})]:
with self.subTest(field=key):
changed=copy.deepcopy(value);changed[key]=bad
with self.assertRaises(AssertionError):ns['validate_network'](changed,'owned-runtime')
def records(self):
records=[]
for index,(case,delays) in enumerate([('short',[1,2,2,2]),('empty',[0,0,0,0])],1):
workflow='rust-retry-'+case+'-00000000-0000-0000-0000-'+str(index).zfill(12)
run='run-'+case
retries=[{'sequence':1,'activity_execution_id':'execution-'+case,'execution_mode':'local',
'local_activity':True,'retry_after_attempt':attempt,'retry_backoff_seconds':delay}
for attempt,delay in enumerate(delays,1)]
records.append({'case':case,'workflow_id':workflow,'run_id':run,'expected_backoff_seconds':delays,
'callback_gap_ms':[delay*1000 for delay in delays],'retry_reports':retries,'callback_attempts':5,
'callbacks_after_replay':0,'history':{'history_complete':True,'workflow':{
'instance_id':workflow,'run_id':run,'details_state':'retained'},
'history_events':[{'type':'ActivityRetryScheduled','payload':row} for row in retries]}})
return records
def test_observation_accepts_only_both_bound_cases(self):
ns['validate_records'](self.records())
for change in ('early','zero-tail','remote','incomplete','extra-callback','duplicate-run','missing-zero'):
with self.subTest(change=change):
records=self.records()
if change=='early':records[0]['callback_gap_ms'][2]=0
if change=='zero-tail':records[0]['retry_reports'][3]['retry_backoff_seconds']=0
if change=='remote':records[0]['retry_reports'][0]['execution_mode']='remote'
if change=='incomplete':records[0]['history']['history_complete']=False
if change=='extra-callback':records[0]['callbacks_after_replay']=1
if change=='duplicate-run':records[1]['run_id']=records[0]['run_id']
if change=='missing-zero':records[1]['retry_reports'][0]['retry_backoff_seconds']=None
with self.assertRaises((AssertionError,KeyError)):ns['validate_records'](records)
def test_startup_guard_is_before_root_access(self):
guard=next(n for n in module.body if isinstance(n,ast.If) and 'unsafe caller' in ast.unparse(n))
root=next(n for n in module.body if isinstance(n,ast.Assign) and any(isinstance(t,ast.Name) and t.id=='ROOT' for t in n.targets))
self.assertLess(guard.lineno,root.lineno)
from types import SimpleNamespace
for opt,site,bytecode in [(1,1,True),(0,0,True),(0,1,False)]:
fake=SimpleNamespace(flags=SimpleNamespace(optimize=opt,no_site=site),dont_write_bytecode=bytecode)
with self.assertRaises(RuntimeError):exec(compile(ast.Module(body=[guard],type_ignores=[]),'guard','exec'),{'sys':fake})
def test_deadline_recovers_committed_unknown_create(self):
for pending in ('container','network'):
present={pending:'a'*64};removed=[];result={'outcome':'fail'}
def recover(kind,identity):return identity if identity is not None else present.get(kind)
def remove(kind,identity):
self.assertEqual(present.pop(kind),identity);removed.append(identity);return {'State':{'OOMKilled':False}}
with mock.patch.dict(ns,{'reconcile_created':recover,'remove_resource':remove}):
try:ns['workload_deadline'](signal.SIGTERM,None)
except ns['BudgetExpired']:errors=ns['cleanup_resources'](None,None,result,pending,seconds=1)
self.assertEqual(errors,[]);self.assertEqual(present,{});self.assertEqual(removed,['a'*64])
def test_unresolved_create_is_nonpass_and_cleanup_calls_are_budgeted(self):
seen=[]
def call(args,wall=30,**kwargs):seen.append(wall)
def recover(kind,identity):ns['call'](['inert-query']);return None
with mock.patch.dict(ns,{'call':call,'reconcile_created':recover}):
errors=ns['cleanup_resources'](None,None,{},'container',seconds=.5)
self.assertIs(ns['call'],call)
self.assertEqual(errors,['container:uncertain-create'])
self.assertTrue(seen);self.assertTrue(all(0<wall<=.5 for wall in seen))
def test_literal_bootstrap_refuses_substitution_before_execution(self):
bootstrap = ast.parse(SOURCE.with_name('219-caller-bootstrap.py').read_bytes())
function = next(node for node in bootstrap.body if isinstance(node, ast.FunctionDef) and node.name == 'capture')
scope = {'os': os, 'pathlib': pathlib, 'stat': stat, 'hashlib': hashlib}
exec(compile(ast.Module(body=[function], type_ignores=[]), '<capture-control>', 'exec'), scope)
with tempfile.TemporaryDirectory() as tmp:
root = pathlib.Path(tmp)
source = root / 'caller.py'
marker = []
original = b'marker.append("original")'
source.write_bytes(original)
expected = hashlib.sha256(original).hexdigest()
captured = scope['capture'](str(source), expected)
source.write_bytes(b'marker.append("replacement")')
exec(compile(source.read_bytes(), '<pathname-predecessor>', 'exec'), {'marker': marker})
self.assertEqual(marker, ['replacement'])
marker.clear()
with self.assertRaisesRegex(RuntimeError, 'source hash refused'):
scope['capture'](str(source), expected)
self.assertEqual(marker, [])
exec(compile(captured, '<captured-candidate>', 'exec'), {'marker': marker})
self.assertEqual(marker, ['original'])
source.unlink()
target = root / 'target.py'; target.write_bytes(original)
source.symlink_to(target)
with self.assertRaises(OSError):
scope['capture'](str(source), expected)
def test_complete_profile_discriminates_stable_predecessor_gaps(self):
predecessor = ast.parse(SOURCE.with_name('hold1-manual-219-timing.py').read_bytes())
old = {'json':ns['json'],'digest':ns['digest'],'IMAGE':'synthetic-image'}
nodes=[node for node in predecessor.body if isinstance(node,ast.FunctionDef) and node.name=='validate_container']
exec(compile(ast.Module(body=nodes,type_ignores=[]),'<profile-predecessor>','exec'),old)
image={'Id':'synthetic-id','Config':{'Env':['PATH=/usr/bin'],'Volumes':{},'Labels':{}}}
args=(image,'network-id','owned',['-c','captured'],{'owner':'sdk-rust'},{'/tmp':'size=64m'})
value=ns['admitted_profile'](*args)
value['NetworkSettings']={'Networks':{'owned':{'NetworkID':''}}}
old['validate_container'](value,*args)
mutations={'CgroupnsMode':'host','GroupAdd':['0'],'Dns':['203.0.113.1'],
'Sysctls':{'kernel.shmmax':'999999'},'Ulimits':[{'Name':'nofile','Hard':1048576,'Soft':1048576}],
'ReadonlyPaths':[],'MaskedPaths':[],'LogConfig':{'Type':'syslog','Config':{}},
'PublishAllPorts':True,'OomScoreAdj':-1000,'CgroupParent':'/'}
for field,change in mutations.items():
with self.subTest(field=field):
changed=copy.deepcopy(value);changed['HostConfig'][field]=change
old['validate_container'](changed,*args)
with self.assertRaises(AssertionError):ns['validate_container'](changed,*args)
for group in ('Config','HostConfig'):
added=copy.deepcopy(value);added[group]['unknown_field']='unselected'
with self.assertRaises(AssertionError):ns['validate_container'](added,*args)
missing=copy.deepcopy(value);missing[group].pop(next(iter(missing[group])))
with self.assertRaises(AssertionError):ns['validate_container'](missing,*args)
mounted=copy.deepcopy(value);mounted['Mounts']=[{'Type':'tmpfs','Destination':'/tmp','RW':True}]
with self.assertRaises(AssertionError):ns['validate_container'](mounted,*args)
def test_streaming_process_bounds_kill_and_reap(self):
# Inert language-only subprocesses; no Docker, SDK or service invocation.
for target in ('stdout','stderr','combined'):
with self.subTest(target=target):
code={'stdout':'os.write(1,b"x"*4096)',
'stderr':'os.write(2,b"x"*4096)',
'combined':'os.write(1,b"x"*80);os.write(2,b"x"*80)'}[target]
process=subprocess.Popen([sys.executable,'-I','-S','-B','-c',
'import os,time;'+code+';time.sleep(10)'],stdout=subprocess.PIPE,
stderr=subprocess.PIPE,start_new_session=True)
with self.assertRaises(ns['OutputOverflow']):ns['read_process'](process,2,128)
self.assertIsNotNone(process.returncode)
self.assertTrue(process.stdout.closed and process.stderr.closed)
old=subprocess.run([sys.executable,'-I','-S','-B','-c',
'import os;os.write(1,b"x"*4096)'],capture_output=True,check=True)
self.assertGreater(len(old.stdout),128) # Prior post-capture check is already too late.
def test_bounded_run_and_timeout_return_no_live_child(self):
result=ns['bounded_run']([sys.executable,'-I','-S','-B','-c',
'import os,sys;os.write(1,sys.stdin.buffer.read());os.write(2,b"err")'],
wall=2,byte_limit=128,input=b'fixture')
self.assertEqual((result.stdout,result.stderr),(b'fixture',b'err'))
process=subprocess.Popen([sys.executable,'-I','-S','-B','-c',
'import time;time.sleep(10)'],stdout=subprocess.PIPE,stderr=subprocess.PIPE,start_new_session=True)
with self.assertRaises(subprocess.TimeoutExpired):ns['read_process'](process,.02,128)
self.assertIsNotNone(process.returncode)
def test_http_body_refuses_before_unbounded_read(self):
class BoundedBody:
def __init__(self,value):self.value=io.BytesIO(value);self.requests=[]
def read(self,size):
self.assert_size(size);self.requests.append(size);return self.value.read(size)
def assert_size(self,size):
if not 0<size<=8192:raise AssertionError('unbounded response read')
good=BoundedBody(b'{"version":"3.10.0"}')
self.assertEqual(ns['bounded_http_json'](good),{'version':'3.10.0'})
oversized=BoundedBody(b' '*4096)
with self.assertRaises(ns['OutputOverflow']):ns['bounded_http_json'](oversized,128)
self.assertEqual(oversized.value.tell(),129)
self.assertGreater(len(io.BytesIO(b' '*4096).read()),128) # Prior unbounded JSON read.
def test_overflow_still_reconciles_and_removes_owned_resources(self):
removed=[];result={'outcome':'fail'}
def remove(kind,identity):removed.append((kind,identity));return {'State':{'OOMKilled':False}}
with mock.patch.dict(ns,{'reconcile_created':lambda kind,identity:identity,'remove_resource':remove,
'call':lambda *a,**k:(_ for _ in ()).throw(ns['OutputOverflow']('log limit'))}):
errors=ns['cleanup_resources']('a'*64,'b'*64,result,None,seconds=1,collect_logs=True)
self.assertEqual(removed,[('container','a'*64),('network','b'*64)])
self.assertEqual(errors,['runtime-logs:OutputOverflow'])
self.assertEqual(result['outcome'],'fail')
def test_selected_network_membership_is_exact_by_stage(self):
predecessor=ast.parse(SOURCE.with_name('hold2-manual-219-timing.py').read_bytes())
old={}
node=next(n for n in predecessor.body if isinstance(n,ast.FunctionDef) and n.name=='validate_network')
exec(compile(ast.Module(body=[node],type_ignores=[]),'<network-predecessor>','exec'),old)
value={'Driver':'bridge','Scope':'local','Internal':False,'Ingress':False,'Attachable':False,'EnableIPv6':False,
'Options':{'com.docker.network.bridge.enable_ip_masquerade':'false',
'com.docker.network.bridge.enable_icc':'false',
'com.docker.network.bridge.host_binding_ipv4':'127.0.0.1'},'Containers':{}}
ns['validate_network'](value) # Before start, the network has no endpoints.
old['validate_network'](value,'owned-runtime') # The prior subset admitted a missing endpoint.
with self.assertRaises(AssertionError):ns['validate_network'](value,'owned-runtime')
value['Containers']={'owned-runtime':{}}
ns['validate_network'](value,'owned-runtime')
with self.assertRaises(AssertionError):ns['validate_network'](value)
value['Containers']['foreign-runtime']={}
with self.assertRaises(AssertionError):ns['validate_network'](value,'owned-runtime')
def test_container_refuses_missing_extra_or_wrong_selected_attachment(self):
predecessor=ast.parse(SOURCE.with_name('hold2-manual-219-timing.py').read_bytes())
old={key:ns[key] for key in ('json','digest','admitted_profile')}
node=next(n for n in predecessor.body if isinstance(n,ast.FunctionDef) and n.name=='validate_container')
exec(compile(ast.Module(body=[node],type_ignores=[]),'<attachment-predecessor>','exec'),old)
image={'Id':'synthetic-id','Config':{'Env':['PATH=/usr/bin'],'Volumes':{},'Labels':{}}}
args=(image,'network-id','owned',['-c','captured'],{'owner':'sdk-rust'},{'/tmp':'size=64m'})
value=ns['admitted_profile'](*args)
value['NetworkSettings']={'Networks':{'owned':{'NetworkID':''}}}
created=ns['validate_container'](value,*args)
active=copy.deepcopy(value);active['NetworkSettings']['Networks']['owned']['NetworkID']='network-id'
self.assertEqual(ns['validate_container'](active,*args,started=True),created)
for started,base in [(False,value),(True,active)]:
for case in ('missing','extra','wrong-id'):
with self.subTest(started=started,case=case):
changed=copy.deepcopy(base);networks=changed['NetworkSettings']['Networks']
if case=='missing':networks.clear()
if case=='extra':networks['foreign']={'NetworkID':'foreign-id'}
if case=='wrong-id':networks['owned']['NetworkID']='foreign-id'
old['validate_container'](changed,*args)
with self.assertRaises(AssertionError):ns['validate_container'](changed,*args,started=started)
with self.assertRaises(AssertionError):ns['validate_container'](value,*args,started=True)
with self.assertRaises(AssertionError):ns['validate_container'](active,*args)
def test_docker_environment_overrides_remove_inherited_defaults(self):
predecessor=ast.parse(SOURCE.with_name('attempt1-manual-219-timing.py').read_bytes())
old={'json':ns['json'],'digest':ns['digest'],'IMAGE':'synthetic-image'}
nodes=[n for n in predecessor.body if isinstance(n,ast.FunctionDef) and n.name in ('admitted_profile','validate_container')]
exec(compile(ast.Module(body=nodes,type_ignores=[]),'<environment-predecessor>','exec'),old)
image={'Id':'synthetic-id','Config':{'Env':['PATH=/usr/bin','DW_BIND_ADDRESS=0.0.0.0:8080',
'DW_RUST_MAINTENANCE_LOOP=1','DB_DATABASE=/app/database/database.sqlite'],'Volumes':{},'Labels':{}}}
args=(image,'network-id','owned',['-c','captured'],{'owner':'sdk-rust'},{'/tmp':'size=64m'})
value=ns['admitted_profile'](*args)
value['NetworkSettings']={'Networks':{'owned':{'NetworkID':''}}}
self.assertEqual(value['Config']['Env'],sorted(['DB_CONNECTION=sqlite',
'DB_DATABASE=/app/database/runtime.sqlite','DW_RUST_MAINTENANCE_LOOP=1',
'PATH=/usr/bin','DW_BIND_ADDRESS=0.0.0.0:8080']))
ns['validate_container'](value,*args)
with self.assertRaises(AssertionError):old['validate_container'](value,*args)
changed=copy.deepcopy(value);changed['Config']['Env'].append('DB_DATABASE=/app/database/database.sqlite')
with self.assertRaises(AssertionError):ns['validate_container'](changed,*args)
changed=copy.deepcopy(value);changed['Config']['Env'].reverse()
self.assertEqual(ns['validate_container'](changed,*args),ns['validate_container'](value,*args))
def test_environment_order_only_normalization_is_closed(self):
import itertools
predecessor=ast.parse(SOURCE.with_name('attempt2-manual-219-timing.py').read_bytes())
old={'json':ns['json'],'digest':ns['digest'],'IMAGE':'synthetic-image'}
nodes=[n for n in predecessor.body if isinstance(n,ast.FunctionDef) and n.name in ('admitted_profile','validate_container')]
exec(compile(ast.Module(body=nodes,type_ignores=[]),'<ordered-environment-predecessor>','exec'),old)
image={'Id':'synthetic-id','Config':{'Env':['PATH=/usr/bin','DW_BIND_ADDRESS=0.0.0.0:8080',
'DW_RUST_MAINTENANCE_LOOP=1','DB_DATABASE=/app/database/database.sqlite'],'Volumes':{},'Labels':{}}}
args=(image,'network-id','owned',['-c','captured'],{'owner':'sdk-rust'},{'/tmp':'size=64m'})
value=ns['admitted_profile'](*args);value['NetworkSettings']={'Networks':{'owned':{'NetworkID':''}}}
expected=ns['validate_container'](value,*args)
for permutation in itertools.permutations(value['Config']['Env']):
changed=copy.deepcopy(value);changed['Config']['Env']=list(permutation)
self.assertEqual(ns['validate_container'](changed,*args),expected)
second=copy.deepcopy(value);second['Config']['Env']=['DB_DATABASE=/app/database/runtime.sqlite',
'DW_RUST_MAINTENANCE_LOOP=1','DB_CONNECTION=sqlite','PATH=/usr/bin','DW_BIND_ADDRESS=0.0.0.0:8080']
with self.assertRaises(AssertionError):old['validate_container'](second,*args)
ns['validate_container'](second,*args)
for case in ('duplicate-identical','duplicate-conflict','changed-value','missing','extra','malformed'):
with self.subTest(case=case):
changed=copy.deepcopy(value);entries=changed['Config']['Env']
if case=='duplicate-identical':entries.append(entries[0])
if case=='duplicate-conflict':entries.append('DB_DATABASE=/foreign')
if case=='changed-value':entries[0]='DB_CONNECTION=mysql'
if case=='missing':entries.pop()
if case=='extra':entries.append('FOREIGN=value')
if case=='malformed':entries.append('NO_EQUALS')
with self.assertRaises(AssertionError):ns['validate_container'](changed,*args)
unittest.main() |
Member
Author
|
Security accepts the second invocation as a retained runner-blocked NONPASS and admits one fresh bounded invocation with the exact r5 successor only. For attempt 2 I independently matched For the successor I matched controller I ran the pure AST/inert harness without importing the effectful controller/bootstrap: all 23 controls pass. The fresh r3 root's six receipt/manifest/lock/test/build/binary inputs match the prior qualified bytes, including build receipt One fresh invocation is permitted under those exact hashes and the existing bounded profile. Preserve both prior NONPASS attempts. Terminal result/config/OOM/stream/cleanup and independent absence evidence remain required before any timing or shared #219 acceptance. This does not authorize a release or any set-aside item. |
Member
Author
|
The one reviewed r5 invocation is terminal NONPASS, consumer exit101 after 8.13 seconds. Unlike the two earlier setup refusals, the exact published native3.10 runtime and normal-registry4.17 consumer both started. The consumer reached the short-list case's replacement-Worker signal gate, where Server returned404 The qualifier calls Full-ID container Both earlier unstarted setup NONPASS attempts remain preserved. Shared219 installed timing stays open. No release, clock/redrive/SQL or set-aside item scope is introduced. |
Member
Author
|
Security terminal counter-review of the retained r5 observation:
This accepts only the terminal evidence and cleanup classification. It does not qualify timing and does not authorize an unchanged rerun. The source correction is reviewed separately on #276; any later observation still requires fresh package-test/build/parent/caller identities and its own disposition. |
Member
Author
|
Fresh normal-registry/caller packet for the accepted signal-declaration qualifier correction SDK276. Source head1724a530 merged as The fresh normal crates.io4.17 consumer uses unchanged manifest/lock/public-registry inputs and the exact6de archive with all118 source members equal. Offline/locked Rust1.86 compilation passes; two pure guards pass with the installed entrypoint ignored. Build/control exit0/OOMfalse, full-ID and exact nonce inventories independently absent. No SDK API/Worker/Server ran during this preparation. The original three NONPASS attempts remain preserved; Security's latest terminal counter-review is 6871805138. The controller differs from the fully reviewed97259846 packet only in the three receipt/binary/test constants below. Profile admission, unique-key Env order normalization, network membership/stages, bounded reads, sealed executable, synthetic credentials, process/resource/deadline/cleanup behavior and qualified native3.10 tuple are unchanged. All23 inert controls pass with the same exact controls source84002d1f. The literal bootstrap changes its controller pin and fixed fresh build/source roots, provided privately; it is still supplied directly as exact This is a proposal for one new separately reviewed bounded ordinary219 timing observation, not permission inferred from the source merge, a timing pass, or a new release. Runtime/config/output/OOM/cleanup and independent absence require terminal review afterward. No clock/redrive/SQL/collector or set-aside item is selected. Frozen packet: {
"controller_sha256": "620b86bc2163aa88eedc29e6a178bd0b82bec161f19a433c60481d5c24f4f115",
"controls_sha256": "84002d1f6833aaa65fa1c12f76f365bd3b9f547cab7c844b7840c91f9e6ca998",
"literal_bootstrap_sha256": "e927603238a51a1471b630f2aa0cd0a57011908f978e40d96366458dc5e318dd",
"build_receipt_sha256": "a927ffaa1eb8af32d1a510a8bf0cb62da47a83493d8578fc5a4c6f09c8326750",
"test_sha256": "b11e724e9d01fc3b5fe9dad8cfd7a8c22f724a270b19481784b91d9e7f751545",
"binary_sha256": "2c92f7f96a1c45a4d9a1c63be7f7ca42531c97d447e4c2c55fa9ce024152d1dd",
"qualified_server": "3.10.0",
"server_source": "b6e04ce7e679464e91f6ceae83310d3bb3204092",
"server_child": "sha256:9d64e9aeee84b46d21715826a3e7e5a96eb8d4ce7fccb126b88b15596ab14973",
"scope": "one ordinary219 local-retry timing observation; no clock/redrive/database operation",
"source_controls": 23,
"workload_seconds": 180,
"parent_seconds": 90,
"cleanup_seconds": 120,
"parent_wait_seconds": 10,
"outer_seconds": 340,
"outer_kill_seconds": 20,
"container_profile": "complete Config/HostConfig/Mounts equality; only unique-key Env order and OomKillDisable false/null normalized",
"stream_byte_limits": {
"docker_stdout_stderr_combined": 1048576,
"runtime_logs_combined": 131072,
"consumer_stdout_stderr_combined": 1048576,
"cluster_http_body": 65536
},
"network_membership": "empty before start; exact runtime after start; container has only selected network name and stage-matching NetworkID",
"prior_attempt": "three preserved NONPASS: two prestart profile refusals, one undeclared qualifier signal; all resources independently absent",
"environment_profile": "unique keys required; exact complete entries compared after order canonicalization; duplicate/missing/extra/malformed/changed values refuse",
"test_source_commit": "1724a530a1f3c33bee5e1eb9333caacf4ef4ea7a",
"accepted_main": "1129d797f266e518eedf642460c89b412dca9dad"
}Exact fresh build receipt: {
"version": "4.17.0",
"installation": "crates.io registry without source override",
"package_commit": "52208134296009bbae1af316c8efac89d3465990",
"crate_sha256": "6de774879fe8411776b60736391b2c6b426bb1bc125268718004656efd52f5c3",
"crate_file_count": 118,
"crate_members_equal": true,
"test_source_commit": "1724a530a1f3c33bee5e1eb9333caacf4ef4ea7a",
"accepted_main": "1129d797f266e518eedf642460c89b412dca9dad",
"test_sha256": "b11e724e9d01fc3b5fe9dad8cfd7a8c22f724a270b19481784b91d9e7f751545",
"manifest_sha256": "118eb66abb44a42bb3aef686379ba087ab12ee02592a05c75e934a8f2cc34ef9",
"lock_sha256": "9e065e60f19c97eda6d87f2be065bacb0f78c1c96c99bc83daa147b97c058466",
"build_messages_relative": "fixture/build.jsonl",
"build_messages_sha256": "fa31e7cd4a2a673947f8d6d5dafc09232ffaef1de0d2e1b137e2130177d5bce8",
"executable_relative": "fixture/target/debug/deps/local_retry_timing_server-38ab37d823ed3661",
"executable_sha256": "2c92f7f96a1c45a4d9a1c63be7f7ca42531c97d447e4c2c55fa9ce024152d1dd",
"executable_bytes": 22509496,
"rust_image": "rust@sha256:300ec56abce8cc9448ddea2172747d048ed902a3090e6b57babb2bf19f754081",
"cargo_version": "1.86.0",
"build_network": "none",
"cargo_source_override": false,
"ordinary_controls": {
"pass": 2,
"ignored_installed": 1,
"sdk_server_executed": false
}
}Complete controller delta from reviewed97259846: --- accepted97259846
+++ declared-signal-pins
@@ -38,9 +38,9 @@
SOURCE = pathlib.Path(sys.argv[2]).absolute()
IMAGE = 'ghcr.io/durable-workflow/server@sha256:9d64e9aeee84b46d21715826a3e7e5a96eb8d4ce7fccb126b88b15596ab14973'
REVISION = 'b6e04ce7e679464e91f6ceae83310d3bb3204092'
-RECEIPT = '9fff98616c7d87b937ea0d843c8c8ba9cef3271049a884abd49a2aff793bf5b7'
-BINARY = '5cd20d42dc317f6b49637595799af611bcd756a53899e81d1dccb37e9649590a'
-TEST = 'f95e947b2b3c199bb929e0f7176fb712d3b0f235612c97cbbae859df6cdaa862'
+RECEIPT = 'a927ffaa1eb8af32d1a510a8bf0cb62da47a83493d8578fc5a4c6f09c8326750'
+BINARY = '2c92f7f96a1c45a4d9a1c63be7f7ca42531c97d447e4c2c55fa9ce024152d1dd'
+TEST = 'b11e724e9d01fc3b5fe9dad8cfd7a8c22f724a270b19481784b91d9e7f751545'
def digest(data):
return hashlib.sha256(data).hexdigest() |
Member
Author
|
Security static data/caller review PASS for one fresh bounded ordinary #219 observation, tied to the exact declared-signal packet in the preceding owner comment. Accepted source/build binding:
Caller boundary:
Effect disposition: one invocation is permitted only with the exact literal bootstrap bytes supplied directly under the previously reviewed closed environment, using controller This does not itself qualify timing, release, or any set-aside #264/#265/#274 work. |
Member
Author
|
Named Rust published-package local-retry timing observation for shared219: both cases observed PASS, pending independent terminal counter-review. This is the one exact invocation admitted by Security6871928363; no repeated unchanged run is selected. The normal crates.io
The short-list gaps independently exceed every requested delay. Empty-list zero reports are explicit, while measured scheduling/execution latency remains nonzero. Both runs have complete retained histories with exactly one local scheduled event, five started attempts, four retry events and one completion. Reports bind each run/logical activity/attempt; replacement Worker instances complete Consumer exit0 (one ignored qualifier selected, two pure tests filtered), stderr0B; stdout224823B. Runtime stdout0B/stderr64B. Runtime OOMfalse/cleanupPASS. Created/started/returned complete profile projections independently recompute to the same SHA Exact container The seven-day private timing-evidence manifest SHA is Bound run identities: [
{
"case": "short",
"workflow_id": "rust-retry-short-3b0de1d5-bac0-4c20-93ab-1f99151e0903",
"run_id": "01M7C63FQPPFCZ9DM5FG05ZE2A"
},
{
"case": "empty",
"workflow_id": "rust-retry-empty-5c87c6ad-8ab2-42ec-befb-69405ba8c6bf",
"run_id": "01M7C63RNVCTYK780FV6S1H3ED"
}
] |
Member
Author
|
Security terminal counter-review PASS for the one exact declared-signal ordinary #219 observation, scoped to published Rust SDK 4.17.0 against the qualified native Server 3.10.0 tuple. Evidence integrity and identity:
Observed contract:
Lifecycle/cleanup:
This accepts the Rust 4.17.0 installed observation for shared #219 on the exact reviewed package/producer/profile. The three earlier NONPASS observations remain preserved and are not relabeled. No further run, new SDK/Server release, cross-binding acceptance, or set-aside #264/#265/#274 scope follows. |
Member
Author
|
Final Rust219 handoff: the exact declared-signal published4.17/native3.10 observation is independently accepted at Security6871989408, with the full owner receipt6871965908. Both retry/report/timing/replacement-replay cases, exact package/source/build/caller identities, config projections, OOMfalse and independent full-ID/label/PID cleanup are PASS. All three prior NONPASS attempts remain preserved. Rust's omission source/docs and this named installed timing meet its current219 criteria; shared closure remains General-owned. No new run/release or set-aside scope is selected. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Shared durable-workflow/.github#219 lacked a named Rust published-package observation of actual local retry waiting. This adds an ignored ordinary Worker/local-activity qualifier, two synthetic validation controls and its invocation guide.
The short-list case runs five callbacks with [1,2], binds four committed local ActivityRetryScheduled reports to the run/logical activity/attempts, and independently requires monotonic callback gaps of at least 1,2,2,2 seconds. The empty-list control requires four explicit zero reports without promising zero scheduling latency. A replacement Worker instance consumes a signal and replays the committed local result without extra callbacks or a changed history prefix.
The qualifier requires an exact-registry consumer and a qualified isolated numeric-loopback Server tuple. Its separately reviewed caller owns package/build/producer/resource/cleanup evidence. No production policy, API default or wire change, SDK release, clock/redrive execution, database read or dependency on set-aside items is introduced.
Validation: General's exact-head scenario review passes; all 19 head checks are terminal (17 success and two expected qualification skips). The normal merge tree equals the reviewed source exactly, all four actual-main workflows succeed, and the remote topic branch is absent. A fresh crates.io 4.17 consumer compiles with locked/offline Rust 1.86 inputs, and both synthetic guards pass with the installed entrypoint ignored. Both preparation containers are independently absent. The qualifier-only signal declaration correction is main-integrated in #276. A fresh exact-registry4.17/native3.10 observation passes both cases and independent terminal Security review6871989408, including timing/history/replay/config/cleanup evidence. All three prior NONPASS attempts remain preserved; this closes only Rust219 criteria, with shared closure retained by General.