Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ The contract is `openapi.yaml` in the root of the repository. The job `OpenAPI l

## Deleting the account

`DELETE /v1/account` takes `{"password": "..."}` and deletes the business with everything under it, then clears the cookie and answers 204. The password is checked again so that a stolen session cannot do it alone, a wrong one answers 403 `invalid_credentials` and not 401, and a user has 5 wrong passwords an hour, shared with `DELETE /v1/gateways/{id}`. Only a wrong password uses up a try, so a call that ends in 503 `busy` or with the right password does not. After the fifth wrong one the route answers 429 `rate_limited` with `Retry-After` for every call, also with the right password. The budget is read again after the password check, so a burst of parallel calls cannot get a right password through once the budget is spent. The readings, the hourly sums and the anomalies go first in batches, and the business row goes last. A 503 `busy` can therefore come after a part of the readings was deleted, or after the whole delete when the time ran out just after the commit. The same call can be repeated until it ends with 204. A 401 on the repeat means that the session is gone, which a finished delete causes, and so does a session that ended in the meantime. The app cannot tell the two apart from the answer, so it sends the user to the sign in page. A sign in with the right password works only while the account still exists, and a failed sign in proves nothing, because a wrong password gives the same answer. It cannot be undone.
`DELETE /v1/account` takes `{"password": "..."}` and deletes the business with everything under it, then clears the cookie and answers 204. The password is checked again so that a stolen session cannot do it alone, a wrong one answers 403 `invalid_credentials` and not 401, and a user has 5 wrong passwords an hour, shared with `DELETE /v1/gateways/{id}`. Only a wrong password uses up a try, so a call that ends in 503 `busy` or with the right password does not. After the fifth wrong one the route answers 429 `rate_limited` with `Retry-After` for every call, also with the right password. The budget is read again after the password check, so a burst of parallel calls cannot get a right password through once the budget is spent. The hubs of the business are cut off first, so they stop sending and cannot add readings during the delete, and a call that stops half way leaves them cut. Then the readings, the hourly sums and the anomalies go in batches, and the business row goes last. A 503 `busy` can therefore come after a part of the readings was deleted, or after the whole delete when the time ran out just after the commit. The same call can be repeated until it ends with 204. A 401 on the repeat means that the session is gone, which a finished delete causes, and so does a session that ended in the meantime. The app cannot tell the two apart from the answer, so it sends the user to the sign in page. A sign in with the right password works only while the account still exists, and a failed sign in proves nothing, because a wrong password gives the same answer. It cannot be undone.

## Errors

Expand Down
2 changes: 1 addition & 1 deletion docs/privacy.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ There is no analytics script and no advertising. The service sets one cookie, th

## Deleting an account

The owner deletes the account in the app with `DELETE /v1/account`, which asks for the password again. The route first deletes the readings, the hourly sums and the anomalies of the business in batches that commit one by one. Then it deletes the business row, and every other table under the business goes with it in that one statement, because the keys cascade, see [Data model](data-model.md). That covers every table under the business, for example the user, the sessions, the gateways, the devices, the tips and the relay data. A request that stops half way leaves the account in place with part of its readings gone, and the next call goes on. It cannot be undone. The operator can still delete a business row by hand for an owner who cannot sign in. The logs, the backups, the reverse proxy of the host, Sentry, Grafana Cloud and the mail provider keep their own copies and follow their own retention.
The owner deletes the account in the app with `DELETE /v1/account`, which asks for the password again. The route first cuts off the hubs of the business, so they stop sending, and then deletes the readings, the hourly sums and the anomalies in batches that commit one by one. Then it deletes the business row, and every other table under the business goes with it in that one statement, because the keys cascade, see [Data model](data-model.md). That covers every table under the business, for example the user, the sessions, the gateways, the devices, the tips and the relay data. A request that stops half way leaves the account in place with part of its readings gone, and the next call goes on. It cannot be undone. The operator can still delete a business row by hand for an owner who cannot sign in. The logs, the backups, the reverse proxy of the host, Sentry, Grafana Cloud and the mail provider keep their own copies and follow their own retention.

The owner can also remove one gateway with `DELETE /v1/gateways/{id}`, which asks for the password again in the same way. It cuts the key first, deletes the readings, the hourly sums and the anomalies of that gateway in batches, and then deletes the gateway row, and its nodes, devices, tips, relay commands and schedules go with it because the keys cascade. A request that stops half way leaves a gateway with a cut key and fewer rows, and the next call goes on. It cannot be undone. The copies that the paragraph above names keep their own retention.

Expand Down
46 changes: 46 additions & 0 deletions internal/auth/auth_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1149,6 +1149,52 @@ func TestOnlyAWrongPasswordUsesUpATryToDeleteAnAccount(t *testing.T) {
}
}

func TestCuttingOffTheGatewaysOfABusinessStopsOnlyItsOwnHubs(t *testing.T) {
e := testutil.New(t)
budi := e.SeedDevice(t, "budi@example.com", "ESP32-AABBCC")
siti := e.SeedDevice(t, "siti@example.com", "ESP32-DDEEFF")

// a code that waits for the hardware of the first business
e.Exec(t, `INSERT INTO gateway_enrollments (poll_hash, code_hash, hardware_id, key_hash, expires_at)
VALUES (sha256('poll'), sha256('code'), 'ESP32-AABBCC', sha256('key'), now() + interval '1 hour')`)

if err := auth.NewStore(e.Pool).CutOffGateways(t.Context(), budi.Business); err != nil {
t.Fatal(err)
}

cut := func(business string) int {
var n int
if err := e.Pool.QueryRow(t.Context(), "SELECT count(*) FROM gateways WHERE business_id = $1 AND key_revoked_at IS NOT NULL", business).Scan(&n); err != nil {
t.Fatal(err)
}

return n
}

if cut(budi.Business) != 1 || cut(siti.Business) != 0 {
t.Fatalf("the cut off reached %d hubs of the first business and %d of the second, want 1 and 0", cut(budi.Business), cut(siti.Business))
}

var waiting int
if err := e.Pool.QueryRow(t.Context(), "SELECT count(*) FROM gateway_enrollments").Scan(&waiting); err != nil || waiting != 0 {
t.Fatalf("%d codes still wait, err %v", waiting, err)
}

// a second cut off keeps the time of the first
var first, second time.Time
if err := e.Pool.QueryRow(t.Context(), "SELECT key_revoked_at FROM gateways WHERE business_id = $1", budi.Business).Scan(&first); err != nil {
t.Fatal(err)
}

if err := auth.NewStore(e.Pool).CutOffGateways(t.Context(), budi.Business); err != nil {
t.Fatal(err)
}

if err := e.Pool.QueryRow(t.Context(), "SELECT key_revoked_at FROM gateways WHERE business_id = $1", budi.Business).Scan(&second); err != nil || !second.Equal(first) {
t.Fatalf("the second cut off moved the time from %v to %v, err %v", first, second, err)
}
}

func TestACheckOfTheLimiterCountsNothing(t *testing.T) {
e := testutil.New(t)
limiter := auth.NewLimiter(e.Pool)
Expand Down
41 changes: 38 additions & 3 deletions internal/auth/store.go
Original file line number Diff line number Diff line change
Expand Up @@ -457,19 +457,54 @@ var bigTablesOfABusiness = []string{
"DELETE FROM anomalies WHERE ctid IN (SELECT ctid FROM anomalies WHERE gateway_id IN (SELECT id FROM gateways WHERE business_id = $1) LIMIT $2)",
}

// CutOffGateways stops every hub of the business from sending, as the delete of one gateway does, so no hub adds readings while the
// batches of a delete run. A code that waits for the same hardware goes too, because a claim of it would bring a cut key back. The
// rows are locked in the order of the claim, the waiting codes first and the gateways after them.
func (s *Store) CutOffGateways(ctx context.Context, businessID string) error {
tx, err := s.pool.BeginTx(ctx, pgx.TxOptions{})
if err != nil {
return fmt.Errorf("start the cut off of the gateways %w", err)
}

defer func() { _ = tx.Rollback(ctx) }()

if _, err := tx.Exec(ctx,
"DELETE FROM gateway_enrollments WHERE gateway_id IS NULL AND hardware_id IN (SELECT hardware_id FROM gateways WHERE business_id = $1)", businessID); err != nil {
return fmt.Errorf("remove the waiting codes of the gateways %w", err)
}

// the key hash becomes a value that no key has, and the revoked time is what stops the hub
if _, err := tx.Exec(ctx,
"UPDATE gateways SET key_hash = sha256(convert_to(gen_random_uuid()::text, 'UTF8')), key_revoked_at = now() WHERE business_id = $1 AND key_revoked_at IS NULL", businessID); err != nil {
return fmt.Errorf("cut off the gateways %w", err)
}

if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("finish the cut off of the gateways %w", err)
}

return nil
}

// DeleteBusiness deletes the business, and every other table under it goes with it because the keys cascade. A business can hold millions
// of readings, and one statement for all of them could run past the deadline of the request every time, so the big tables go first in
// batches that each commit by themselves, as the janitor does. A call that stops half way has deleted a part of the readings and
// nothing else, and the next call goes on from there. The business row goes last, in one statement.
// batches that each commit by themselves, as the janitor does. The hubs are cut off first. A call that stops half way has cut the hubs and
// deleted a part of the readings and nothing else, and the next call goes on from there. The business row goes last, in one statement.
func (s *Store) DeleteBusiness(ctx context.Context, businessID string) error {
if err := s.CutOffGateways(ctx, businessID); err != nil {
Comment thread
SyafaHadyan marked this conversation as resolved.
Resolved
return err
}

for _, query := range bigTablesOfABusiness {
for {
tag, err := s.pool.Exec(ctx, query, businessID, deleteBatch)
if err != nil {
return fmt.Errorf("delete a batch of rows of the business %w", err)
}

if tag.RowsAffected() < deleteBatch {
// a batch can come back short when a concurrent writer took rows of it, so only an empty batch ends the loop
// and any row that is left goes with the cascade of the business row
if tag.RowsAffected() == 0 {
Comment thread
SyafaHadyan marked this conversation as resolved.
Resolved
break
}
}
Expand Down
2 changes: 1 addition & 1 deletion openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ paths:
tags: [Account]
operationId: deleteAccount
summary: Delete the account and the business with everything under it
description: "The password is checked again, so a stolen session cannot delete the account alone. The delete takes the business and everything under it. That covers the user, the sessions, the gateways with their keys, the nodes, the devices, the readings, the anomalies, the tips, the relay commands and schedules, the rooms and the production records. It cannot be undone. The answer clears the cookie like logout. A wrong password answers 403 `invalid_credentials`, and not 401, because the session is still valid. A user has 5 wrong passwords an hour, shared with `DELETE /v1/gateways/{id}`, and only a wrong password uses up a try, so a call that ends in 503 `busy` or with the right password does not. After the fifth wrong one the route answers 429 for every call, also with the right password. The big tables go first in batches, so a 503 `busy` can come after a part of the readings was deleted, or after the whole delete when the time ran out just after the commit. The same call can be repeated until it ends with 204. A 401 on the repeat means that the session is gone, which a finished delete causes, and so does a session that ended in the meantime. The app cannot tell the two apart from the answer, so it sends the user to the sign in page. A sign in with the right password works only while the account still exists, and a failed sign in proves nothing, because a wrong password gives the same answer."
description: "The password is checked again, so a stolen session cannot delete the account alone. The delete takes the business and everything under it. That covers the user, the sessions, the gateways with their keys, the nodes, the devices, the readings, the anomalies, the tips, the relay commands and schedules, the rooms and the production records. It cannot be undone. The answer clears the cookie like logout. A wrong password answers 403 `invalid_credentials`, and not 401, because the session is still valid. A user has 5 wrong passwords an hour, shared with `DELETE /v1/gateways/{id}`, and only a wrong password uses up a try, so a call that ends in 503 `busy` or with the right password does not. After the fifth wrong one the route answers 429 for every call, also with the right password. The hubs of the business are cut off first, so they stop sending and cannot add readings during the delete, and a call that stops half way leaves them cut. Then the big tables go in batches, so a 503 `busy` can come after a part of the readings was deleted, or after the whole delete when the time ran out just after the commit. The same call can be repeated until it ends with 204. A 401 on the repeat means that the session is gone, which a finished delete causes, and so does a session that ended in the meantime. The app cannot tell the two apart from the answer, so it sends the user to the sign in page. A sign in with the right password works only while the account still exists, and a failed sign in proves nothing, because a wrong password gives the same answer."
requestBody:
required: true
content:
Expand Down
Loading