Skip to content

Commit b110af1

Browse files
committed
feat: add safe actor destruction
Authorize and linearize reference destruction on the actor instance row. Cascade actor-owned durable work, fence stale activations, and prevent claimed reminders from resurrecting a deleted incarnation. Cover deletion and in-flight races with deterministic Minitest integration tests.
1 parent 6ac4f5f commit b110af1

24 files changed

Lines changed: 652 additions & 49 deletions

CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,4 +12,6 @@
1212
- Add JSON actor state, versioned migrations, retries, and dead letters.
1313
- Add transactional effects, actor-to-actor messages, and durable reminders.
1414
- Add observable Turbo replacements through a durable broadcast outbox.
15+
- Add authorized, fenced actor destruction with cascading mailbox, reminder,
16+
effect, broadcast, and dead-letter cleanup.
1517
- Support SQLite, PostgreSQL, and MySQL.

docs/architecture.md

Lines changed: 60 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,13 @@
22

33
## Purpose
44

5-
Solid Objects is a Rails engine that provides database-backed virtual actors for MySQL, PostgreSQL, and SQLite. A virtual actor is a logical object addressed by type and ID whose in-memory activation is created on demand, processes one mailbox turn at a time, persists JSON state, and can disappear when idle without losing its identity or state.
5+
Solid Objects ports the Cloudflare Durable Objects programming model to Rails.
6+
It is a database-backed virtual actor runtime for MySQL, PostgreSQL, and
7+
SQLite. A virtual actor is a logical object addressed by type and ID whose
8+
in-memory activation is created on demand, processes one mailbox turn at a
9+
time, persists JSON state, and can disappear when idle without losing its
10+
identity or state. This ports the programming model, not Cloudflare's
11+
serverless runtime, global placement, storage API, or platform guarantees.
612

713
The runtime contract is:
814

@@ -53,11 +59,12 @@ The registry maps a stable persisted actor type string to a Ruby actor class. Re
5359
A reference contains actor type and normalized actor ID. It is cheap,
5460
serializable as data, and does not imply an active Ruby object. Declared
5561
message methods delegate to `tell`; declared query and attribute methods
56-
delegate to `ask`. Both paths authorize and enqueue through the client.
62+
delegate to `ask`. `destroy` is a reserved synchronous reference operation.
63+
All three paths authorize through the client.
5764

5865
### Client and mailbox
5966

60-
The client finds or creates the actor instance and atomically allocates a sequence. It inserts one durable message-history row and one ready-membership row. It validates message names and JSON payloads before writing and enforces idempotency-key uniqueness, payload limits, and the per-actor mailbox cap. Distributed rate limiting and global admission control are not implemented.
67+
The client finds or creates the actor instance and atomically allocates a sequence. It inserts one durable message-history row and one ready-membership row. It validates message names and JSON payloads before writing and enforces idempotency-key uniqueness, payload limits, and the per-actor mailbox cap. It also authorizes and coordinates actor destruction. Distributed rate limiting and global admission control are not implemented.
6168

6269
Message execution state is table membership, not a status column. The durable message remains for results, retention, and diagnostics. Only live work occupies `ready_messages` or `claimed_messages`, so completed history cannot inflate the polling index.
6370

@@ -100,7 +107,11 @@ An effect worker claims due effect rows through the database coordination adapte
100107

101108
### Reminder scheduler
102109

103-
The scheduler claims due reminder definitions, enqueues ordinary actor messages, and advances recurring reminders or completes one-shot reminders. A unique occurrence key prevents two schedulers from producing two mailbox rows for the same reminder occurrence.
110+
The scheduler claims due reminder definitions, locks the source actor instance,
111+
then enqueues the ordinary actor message and advances or completes the reminder
112+
in one transaction. A unique occurrence key prevents two schedulers from
113+
producing two mailbox rows for the same reminder occurrence. Locking the source
114+
instance first prevents a claimed reminder from recreating a destroyed actor.
104115

105116
### Broadcast worker
106117

@@ -141,9 +152,9 @@ updates; it is not independently persisted.
141152
Lifecycle hooks are deterministic local hooks:
142153

143154
- `on_activate` runs after state load and migration. State changes made there are included with the next successful message commit, not persisted on activation alone.
144-
- `on_deactivate` runs only on graceful local deactivation. Its state changes are not persisted and it must not be used for durable work.
155+
- `on_deactivate` runs only on graceful local deactivation. Its state changes are not persisted and it must not be used for durable work. Explicit destruction does not run lifecycle hooks.
145156

146-
Durable cleanup belongs in messages, reminders, or effects.
157+
Durable application cleanup belongs in messages, reminders, or effects.
147158

148159
## Enqueue and sequence allocation
149160

@@ -164,6 +175,37 @@ The increment and insert roll back together. The unique index on `(actor_type, a
164175

165176
Committed concurrent enqueues have one database-defined sequence order. No order is promised between transactions that have not committed.
166177

178+
## Actor destruction
179+
180+
`ActorClass.ref(actor_id).destroy` is a synchronous, idempotent runtime
181+
operation. It is forbidden from actor context and has a separate
182+
`authorize_destroy` policy that runs before actor existence is revealed.
183+
184+
Destruction uses one transaction:
185+
186+
1. Resolve the actor type through the registry.
187+
2. Authorize the actor type and ID.
188+
3. Lock the actor instance by logical identity.
189+
4. Return `false` if it does not exist.
190+
5. Delete the instance.
191+
6. Let foreign-key cascades delete message history, ready and claimed
192+
memberships, dead letters, reminders, effects, and broadcasts.
193+
7. Commit, emit `solid_objects.actor.destroyed`, and wake local waiters.
194+
195+
The instance primary key is the actor-incarnation boundary. A worker holding an
196+
old lease can continue running Ruby code, but its fenced transaction cannot
197+
find the deleted instance and raises `LostActivation`. If the same logical
198+
identity is referenced later, enqueue creates a new instance with default
199+
state, state version, sequence 1, and a new primary key. An enqueue that loses
200+
the instance between lookup and locking retries against the new incarnation.
201+
202+
A claimed reminder locks the source instance before enqueueing its occurrence,
203+
so it either commits before destruction and is deleted by the cascade, or
204+
observes the missing instance and does nothing. An already-running external
205+
effect, actor-to-actor delivery, or broadcast may have crossed the database
206+
boundary before destruction; it cannot be recalled. Its completion sees the
207+
deleted outbox row and cannot enqueue a callback or recreate the source actor.
208+
167209
## Candidate selection and fairness
168210

169211
An actor is eligible when:
@@ -324,7 +366,10 @@ A reminder record contains actor identity, a reminder name, target message, JSON
324366
schedule :expire, at: 30.minutes.from_now, arguments: {}
325367
```
326368

327-
When due, the scheduler creates a normal mailbox row with an idempotency key derived from reminder ID and occurrence. The mailbox provides sequential processing and ordinary retry behavior.
369+
When due, the scheduler locks the source instance and creates a normal mailbox
370+
row with an idempotency key derived from reminder ID and occurrence. The
371+
mailbox insert and reminder advancement commit atomically. The mailbox provides
372+
sequential processing and ordinary retry behavior.
328373

329374
Solid Objects persists each occurrence by its mailbox row. Unlike Orleans reminders, an outage does not intentionally discard a due occurrence. Recurring catch-up is configurable:
330375

@@ -373,14 +418,18 @@ Each channel subscription transmits current observable replacements before strea
373418

374419
## Authorization
375420

376-
Configuration provides four explicit policies:
421+
Configuration provides five explicit policies:
377422

378423
- `authorize_message`
379424
- `authorize_query`
425+
- `authorize_destroy`
380426
- `authorize_subscription`
381427
- `authorize_administration`
382428

383-
Each receives a request context, registered actor class, actor ID, and operation details. A host can set request context using an isolated execution-state carrier. Internal runtime deliveries carry a system context that is separately recognizable.
429+
Each receives a request context, actor type, actor ID, and relevant operation
430+
details. A host can set request context using an isolated execution-state
431+
carrier. Internal runtime deliveries carry a system context that is separately
432+
recognizable.
384433

385434
No controller, channel, or administrative command treats an actor ID, message ID, request ID, or signed stream name as authorization.
386435

@@ -515,7 +564,8 @@ PostgreSQL transaction-level advisory locks may be used for optional singleton m
515564
| Successful message commit | Fenced state, durable message result, claimed-membership deletion, effects, reminders, actor outbox, broadcasts |
516565
| Failed message attempt | Conditional error, claimed deletion, ready reinsertion or dead letter |
517566
| Renew or release lease | Conditional instance update |
518-
| Deliver reminder occurrence | Mailbox enqueue is one transaction; reminder advance is a second claim-checked transaction, bridged by a stable occurrence idempotency key |
567+
| Destroy actor | Instance identity lock and cascading delete of state, mailbox, reminders, and outboxes |
568+
| Deliver reminder occurrence | Source instance lock, mailbox enqueue, and reminder advance, with a stable occurrence idempotency key |
519569
| Claim outbox batch | Backend claim transaction and delivery ownership |
520570
| Record outbox outcome | Success or retry/dead status |
521571

docs/correctness.md

Lines changed: 34 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,35 @@ finalization locks the instance and checks:
3232
A stale worker can continue running Ruby code, but it cannot commit state,
3333
completion, or outboxes.
3434

35+
## Destruction
36+
37+
`reference.destroy` locks and deletes the actor instance in one transaction.
38+
Cascading foreign keys delete its message history, ready and claimed
39+
memberships, dead letters, reminders, effects, and broadcasts. The operation
40+
returns `true` when it deletes an incarnation and `false` when no incarnation
41+
exists.
42+
43+
The deleted instance primary key is also the fencing boundary. An activation
44+
that was executing before destruction raises `LostActivation` when it attempts
45+
to commit because its instance no longer exists. Reusing the logical actor
46+
type and ID creates a fresh incarnation with a different primary key, default
47+
state, and sequence 1; an old lease cannot address or commit into it.
48+
49+
An enqueue racing with destruction is ordered by the instance lock. Work that
50+
commits first is deleted; work that observes the deletion retries against a new
51+
incarnation. A claimed reminder cannot resurrect an old incarnation because
52+
reminder delivery locks the source instance and atomically advances the
53+
reminder with its mailbox insert.
54+
55+
Destruction removes pending and claimed outbox records, but it cannot recall
56+
external I/O, actor-to-actor delivery, or a broadcast that began before the
57+
delete. A stale outbox completion cannot record success, enqueue an actor
58+
callback, or recreate the source actor. Applications must still make external
59+
effect handlers idempotent.
60+
61+
Destruction is synchronous, forbidden from actor context, authorized by
62+
`authorize_destroy`, and does not run `on_deactivate`.
63+
3564
## Crash matrix
3665

3766
| Failure point | Durable outcome |
@@ -71,7 +100,9 @@ The following are atomic:
71100
- state, state version, message result/completion, claimed deletion, effects,
72101
reminders, outbound actor messages, and observable broadcasts;
73102
- failed-attempt record plus ready reinsertion or dead letter;
74-
- effect completion plus its optional actor outcome message.
103+
- effect completion plus its optional actor outcome message;
104+
- reminder occurrence enqueue plus reminder advancement; and
105+
- actor destruction plus cascading removal of all actor-owned rows.
75106

76107
Actor Ruby code and external I/O are never inside the actor-state transaction.
77108

@@ -80,6 +111,8 @@ Actor Ruby code and external I/O are never inside the actor-state transaction.
80111
`ask` is a durable message followed by result polling and wake-up hints. Timeout
81112
does not cancel the message. The current cross-process fallback is polling;
82113
therefore polling-only ask is not recommended in latency-sensitive HTTP paths.
114+
Destroying the actor while an `ask` is waiting removes its message, wakes the
115+
caller, and raises `SolidObjects::ActorDestroyed`.
83116

84117
## Database dependencies
85118

docs/database-schema.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,11 @@ One row per `(actor_type, actor_id)`. Stores JSON state, state version,
1111
next-message sequence, activation owner/expiration/generation, pause state, and
1212
lifecycle timestamps.
1313

14+
Deleting an instance is the actor-incarnation boundary. Foreign keys cascade
15+
the delete through messages, ready and claimed memberships, reminders, effects,
16+
broadcasts, and dead letters. Reusing the logical identity creates a new
17+
instance primary key with fresh state and message sequence.
18+
1419
Indexes:
1520

1621
- unique identity: enforces one logical actor;
@@ -96,3 +101,11 @@ index. Moving a message between ready and claimed tables makes executable state
96101
physical table membership. The hot indexes stay proportional to live work,
97102
avoid backend-specific partial indexes, and are portable across all supported
98103
databases.
104+
105+
## Cascading destruction
106+
107+
The public `reference.destroy` operation locks the instance row before deleting
108+
it. Every actor-owned table has a cascading foreign key either directly to the
109+
instance or through its message row. No application-side bulk delete can leave
110+
an executable orphan. Process registry rows are not actor-owned and remain
111+
available for worker lifecycle accounting.

docs/operations.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,24 @@ is evidence that alarms are being lost.
7676

7777
Never bulk-update actor state. That bypasses lease ownership and fencing.
7878

79+
## Actor destruction
80+
81+
Delete an actor only through its authorized reference:
82+
83+
```ruby
84+
Counter.ref("global").destroy(authorization_context: Current.user)
85+
```
86+
87+
Do not delete `solid_objects_instances` directly. The public operation locks
88+
the identity, invalidates stale activations through the deleted incarnation
89+
key, cascades through all actor-owned rows, emits
90+
`solid_objects.actor.destroyed`, and wakes local waiters.
91+
92+
Destruction removes pending outboxes but cannot recall external I/O,
93+
actor-to-actor delivery, or a broadcast that already started. Confirm
94+
downstream idempotency and application retention requirements before deleting
95+
an actor. Reusing the same actor type and ID creates a fresh incarnation.
96+
7997
## Monitoring
8098

8199
Alert on:
@@ -86,6 +104,7 @@ Alert on:
86104
- actor turn duration and failures;
87105
- lost-activation rate;
88106
- dead-letter creation;
107+
- actor destruction rate;
89108
- stale process heartbeats;
90109
- effect and broadcast retry/dead counts;
91110
- due-reminder lag;

docs/security.md

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,14 +3,16 @@
33
## Policy hooks
44

55
Solid Objects has separate hooks for sending messages, querying state,
6-
subscribing to actor streams, and administration. The host application supplies
7-
the authenticated request or connection as `authorization_context`.
8-
All four hooks deny by default.
6+
destroying actors, subscribing to actor streams, and administration. The host
7+
application supplies the authenticated request or connection as
8+
`authorization_context`. All five hooks deny by default.
99

1010
Method-style reference calls do not bypass these hooks. Public instance methods
1111
declared on an actor are part of its remotely addressable message surface and
1212
delegate to the authorized `tell` path. Keep implementation helpers private or
1313
protected. Query and attribute methods delegate to the authorized `ask` path.
14+
`reference.destroy` delegates to `authorize_destroy` before checking whether
15+
the actor exists, so denial does not reveal actor existence.
1416

1517
Internal runtime delivery bypasses the public client only for rows already
1618
created by a committed actor turn. It never converts a database actor type into
@@ -47,6 +49,10 @@ host authentication and audit their use.
4749
Instrumentation excludes arguments, state, results, and effect payloads by
4850
default. Review custom logging and effect handlers for accidental disclosure.
4951

52+
Actor destruction is not an administrative shortcut. Authorize tenancy and
53+
ownership explicitly in `authorize_destroy`; knowledge of an actor ID is never
54+
permission to delete its state or queued work.
55+
5056
## Denial of service
5157

5258
Configure mailbox and byte limits. Add host rate limiting before public actor

lib/generators/solid_objects/templates/solid_objects.rb

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
configuration.reminder_scheduler_count = 1
88
configuration.authorize_message = ->(**) { false }
99
configuration.authorize_query = ->(**) { false }
10+
configuration.authorize_destroy = ->(**) { false }
1011
configuration.authorize_subscription = ->(**) { false }
1112
configuration.authorize_administration = ->(**) { false }
1213
end

lib/solid_objects/client.rb

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,35 @@ def ask(reference, message_name, arguments, timeout:, idempotency_key: nil, auth
5959
wait_for_result(message_reference, timeout:)
6060
end
6161

62+
# @rbs (Reference, ?authorization_context: untyped) -> bool
63+
def destroy(reference, authorization_context: nil)
64+
raise ActorCallCycle, "actors cannot synchronously destroy another actor" if Context.current_actor
65+
66+
SolidObjects.registry.fetch(reference.actor_type)
67+
authorize_destroy!(reference, authorization_context:)
68+
instance_id = SolidObjects.database_adapter.transaction do
69+
instance = Instance.lock.find_by(
70+
actor_type: reference.actor_type,
71+
actor_id: reference.actor_id
72+
)
73+
next unless instance
74+
75+
instance_id = instance.id
76+
instance.delete
77+
instance_id
78+
end
79+
return false unless instance_id
80+
81+
SolidObjects.instrument(
82+
:"actor.destroyed",
83+
instance_id:,
84+
actor_type: reference.actor_type,
85+
actor_id: reference.actor_id
86+
)
87+
SolidObjects.wake_up.signal
88+
true
89+
end
90+
6291
private
6392

6493
attr_reader :mailbox
@@ -77,6 +106,18 @@ def authorize!(hook, reference, message_name, arguments, authorization_context:)
77106
raise Unauthorized, "actor invocation is not authorized"
78107
end
79108

109+
# @rbs (Reference, authorization_context: untyped) -> void
110+
def authorize_destroy!(reference, authorization_context:)
111+
authorized = SolidObjects.configuration.authorize_destroy.call(
112+
actor_type: reference.actor_type,
113+
actor_id: reference.actor_id,
114+
authorization_context:
115+
)
116+
return if authorized
117+
118+
raise Unauthorized, "actor destruction is not authorized"
119+
end
120+
80121
# @rbs (MessageReference, timeout: Numeric) -> untyped
81122
def wait_for_result(message_reference, timeout:)
82123
deadline = monotonic_now + timeout.to_f
@@ -100,6 +141,8 @@ def wait_for_result(message_reference, timeout:)
100141
timeout: [ remaining, SolidObjects.configuration.ask_polling_interval ].min
101142
)
102143
end
144+
rescue ActiveRecord::RecordNotFound
145+
raise ActorDestroyed, "actor was destroyed while waiting for its result"
103146
end
104147

105148
# @rbs () -> Float

lib/solid_objects/configuration.rb

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ class Configuration
3131
# @rbs @wake_up_adapter: untyped
3232
# @rbs @authorize_message: Proc
3333
# @rbs @authorize_query: Proc
34+
# @rbs @authorize_destroy: Proc
3435
# @rbs @authorize_subscription: Proc
3536
# @rbs @authorize_administration: Proc
3637

@@ -63,6 +64,7 @@ class Configuration
6364
:wake_up_adapter,
6465
:authorize_message,
6566
:authorize_query,
67+
:authorize_destroy,
6668
:authorize_subscription,
6769
:authorize_administration
6870

@@ -101,6 +103,7 @@ def initialize
101103
end
102104
@authorize_message = ->(**) { false }
103105
@authorize_query = ->(**) { false }
106+
@authorize_destroy = ->(**) { false }
104107
@authorize_subscription = ->(**) { false }
105108
@authorize_administration = ->(**) { false }
106109
end

lib/solid_objects/errors.rb

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,9 @@ class InvalidStreamToken < Error
3737
class LostActivation < Error
3838
end
3939

40+
class ActorDestroyed < LostActivation
41+
end
42+
4043
class AskTimeout < Error
4144
end
4245

0 commit comments

Comments
 (0)