diff --git a/docs/api.md b/docs/api.md index 2bd8a14..f4d595a 100644 --- a/docs/api.md +++ b/docs/api.md @@ -19,7 +19,7 @@ The contract is `openapi.yaml` in the root of the repository. The job `OpenAPI l ## Deleting the account -`DELETE /v1/account` takes `{"password": "..."}` and deletes the business with everything under it, then clears the cookie and answers 204. The password is checked again so that a stolen session cannot do it alone, a wrong one answers 403 `invalid_credentials` and not 401, and a user has 5 wrong passwords an hour. Only a wrong password uses up a try, so a call that ends in 503 `busy` or with the right password does not. After the fifth wrong one the route answers 429 `rate_limited` with `Retry-After` for every call, also with the right password. The budget is read again after the password check, so a burst of parallel calls cannot get a right password through once the budget is spent. The readings, the hourly sums and the anomalies go first in batches, and the business row goes last. A 503 `busy` can therefore come after a part of the readings was deleted, or after the whole delete when the time ran out just after the commit. The same call can be repeated until it ends with 204. A 401 on the repeat means that the session is gone, which a finished delete causes, and so does a session that ended in the meantime. The app cannot tell the two apart from the answer, so it sends the user to the sign in page. A sign in with the right password works only while the account still exists, and a failed sign in proves nothing, because a wrong password gives the same answer. It cannot be undone. +`DELETE /v1/account` takes `{"password": "..."}` and deletes the business with everything under it, then clears the cookie and answers 204. The password is checked again so that a stolen session cannot do it alone, a wrong one answers 403 `invalid_credentials` and not 401, and a user has 5 wrong passwords an hour, shared with `DELETE /v1/gateways/{id}`. Only a wrong password uses up a try, so a call that ends in 503 `busy` or with the right password does not. After the fifth wrong one the route answers 429 `rate_limited` with `Retry-After` for every call, also with the right password. The budget is read again after the password check, so a burst of parallel calls cannot get a right password through once the budget is spent. The readings, the hourly sums and the anomalies go first in batches, and the business row goes last. A 503 `busy` can therefore come after a part of the readings was deleted, or after the whole delete when the time ran out just after the commit. The same call can be repeated until it ends with 204. A 401 on the repeat means that the session is gone, which a finished delete causes, and so does a session that ended in the meantime. The app cannot tell the two apart from the answer, so it sends the user to the sign in page. A sign in with the right password works only while the account still exists, and a failed sign in proves nothing, because a wrong password gives the same answer. It cannot be undone. ## Errors @@ -43,7 +43,8 @@ An amount is a whole number of rupiah. A tip shows its saving rounded to the nea 2. The owner types the code in the app (`POST /v1/gateways`). The gateway polls and learns that it belongs to a business. 3. From then on the gateway sends its key as a bearer token. The service keeps only the hash of the key, so a copy of the database cannot act as a gateway. TLS protects the key in transit, and the sequence number of a batch makes a retry safe. A signature on every request would add nothing over TLS here, because a party that sees the plain request also sees the key. 4. The owner can cut off the key of a gateway with `POST /v1/gateways/{id}/revoke-key`, which answers 204. The old key gets 401 `invalid_gateway_key` from then on, and the devices and readings stay. The list shows `keyRevoked` as true until the hardware enrolls again with a new key and the owner claims the new code. The route returns no claim code, because the gateway makes its own, and it removes a claim code that waits for the same hardware, so the gateway has to enroll again with a new key. An id of another business answers 404. -5. The owner starts a pairing in the app. The next unclaimed node of that kind that the gateway reports becomes the device. `POST /v1/pairings` answers 409 `no_gateway` when the business has no gateway with a working key, which is the case when it has none and when the owner cut the key of every one, because a gateway with a cut key cannot report a node. +5. The owner can remove a gateway with `DELETE /v1/gateways/{id}`, which takes `{"password": "..."}`, answers 204 and cannot be undone. The password is checked again so that a stolen session cannot do it alone, a wrong one answers 403 `invalid_credentials`, and a user has 5 wrong passwords an hour for this route and `DELETE /v1/account` together. Only a wrong password uses up a try, and after the fifth wrong one the route answers 429 `rate_limited` for every call, also with the right password. A repeat of the call sends the password again. The gateway goes with its nodes, devices, readings, hourly sums, anomalies, tips, relay commands and schedules. The rooms stay. The service cuts the key first, so the hardware stops adding batches, apart from one that was already running. The readings, the hourly sums and the anomalies then go in batches, and the gateway row goes last. A 503 `busy` can therefore come after a part of the rows was deleted, and the same call can be repeated until it ends with 204. A 404 on the repeat means that the delete finished or that the id is not a gateway of the business, and the app cannot tell the two apart. The hardware gets 401 `invalid_gateway_key` from then on, and it can enroll again from the start. A claim code that waits for the same hardware is removed. A finished pairing that pointed at one of its devices then shows `deviceId` as null. An id of another business, an unknown id and an id that is not a plain lower case uuid all answer 404. +6. The owner starts a pairing in the app. The next unclaimed node of that kind that the gateway reports becomes the device. `POST /v1/pairings` answers 409 `no_gateway` when the business has no gateway with a working key, which is the case when it has none and when the owner cut the key of every one, because a gateway with a cut key cannot report a node. The list and the detail of a device carry `kind`, which is `plug` or `clamp`, so the app can show the relay switch only on a plug. `kindLabel` is the text for the screen and is not meant to be parsed. diff --git a/docs/faq.md b/docs/faq.md index 604eda5..8cabac6 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -37,6 +37,8 @@ The app is in Indonesian, and the API keeps a few of those words in its fields a **Can an owner delete an account?** Yes. `DELETE /v1/account` with the password deletes the business and everything under it, and it cannot be undone. See [Privacy](privacy.md#deleting-an-account). +**Can an owner remove a gateway?** Yes. `DELETE /v1/gateways/{id}` with the password removes the gateway with its devices and readings, and it cannot be undone. The hardware gets 401 and can enroll again from the start. See [API](api.md). + **Can a session be ended from the app?** Yes. The owner can list the signed in devices, end one of them, or end all the others, and signing out ends the session of that device. A session also ends after 30 days without use, after 90 days, or when the owner resets the password. See [Security](security.md). **How long is the data kept?** Raw readings 30 days, hourly sums 400 days and anomalies 90 days. See [Privacy](privacy.md#how-long-it-stays). diff --git a/docs/privacy.md b/docs/privacy.md index b0bf377..38613af 100644 --- a/docs/privacy.md +++ b/docs/privacy.md @@ -54,6 +54,8 @@ There is no analytics script and no advertising. The service sets one cookie, th The owner deletes the account in the app with `DELETE /v1/account`, which asks for the password again. The route first deletes the readings, the hourly sums and the anomalies of the business in batches that commit one by one. Then it deletes the business row, and every other table under the business goes with it in that one statement, because the keys cascade, see [Data model](data-model.md). That covers every table under the business, for example the user, the sessions, the gateways, the devices, the tips and the relay data. A request that stops half way leaves the account in place with part of its readings gone, and the next call goes on. It cannot be undone. The operator can still delete a business row by hand for an owner who cannot sign in. The logs, the backups, the reverse proxy of the host, Sentry, Grafana Cloud and the mail provider keep their own copies and follow their own retention. +The owner can also remove one gateway with `DELETE /v1/gateways/{id}`, which asks for the password again in the same way. It cuts the key first, deletes the readings, the hourly sums and the anomalies of that gateway in batches, and then deletes the gateway row, and its nodes, devices, tips, relay commands and schedules go with it because the keys cascade. A request that stops half way leaves a gateway with a cut key and fewer rows, and the next call goes on. It cannot be undone. The copies that the paragraph above names keep their own retention. + ## Consent A sign up counts as the consent to the terms and the privacy policy. The service stores the time and the version of the texts in `users`, so a later change of the texts can be tied to who accepted which version. diff --git a/docs/security.md b/docs/security.md index 1fdd5c1..5bcbab9 100644 --- a/docs/security.md +++ b/docs/security.md @@ -26,7 +26,7 @@ This page describes what the API protects, who it protects it from and what stop | Guessing a password | A password of about 1400 of the most common ones is refused when it is set. A password is hashed with argon2id, with a cost that follows the OWASP guidance. Only a few hashes run at once, and a request waits a few seconds for a free slot and answers 503 `busy` only when none opens in that time. A sign in with an unknown email does about the same work as a wrong password, and the answer is the same | | Flooding the account routes | Each public account route has a limit for the client address, and most also have one for the email, for example 10 sign ins for an email in 15 minutes. The counters live in PostgreSQL, so every copy of the service shares them. A flood with a new email each time is limited by the address only, and the proxy has to limit it too | | Reading another business | Every query is limited to the business of the signed in account. An id of another business answers 404 like an id that does not exist, so the answer does not tell whether it exists | -| A stolen session | The cookie is `HttpOnly` and `SameSite=Lax`, and it is `Secure` with the `__Host-` prefix in production. The service stores only the SHA-256 hash of a session. A session ends after 30 days without use or 90 days in all, an account keeps at most 20, and a password reset ends every session of the account, and the owner can end one or all the others in the list of devices | +| A stolen session | The cookie is `HttpOnly` and `SameSite=Lax`, and it is `Secure` with the `__Host-` prefix in production. The service stores only the SHA-256 hash of a session. A session ends after 30 days without use or 90 days in all, an account keeps at most 20, and a password reset ends every session of the account, and the owner can end one or all the others in the list of devices. An action that cannot be undone, the delete of the account and the delete of a gateway, asks for the password again, and both share a budget of 5 wrong passwords an hour | | A forged request from another site | The CORS rule allows only the origin of the app, and the browser refuses the preflight of any other origin. A request that changes data also needs the header `X-Requested-With`, and its `Origin` must be the origin of the app when the browser sends one. A page of another site cannot add the header without a preflight | | A stolen gateway key | The service stores only the SHA-256 hash of the key, so a copy of the database does not hold a key that works. It does allow offline guesses against the hash, so a key must come from a strong random source, such as 32 random bytes. A gateway can send 600 batches an hour, and a batch has a sequence number, so a replay is acknowledged again and does not write twice | | A mail link that leaks | A verification link lives 24 hours and a reset link one hour. The service stores only the hash of a token, a link works once, and the answer of the reset route is 202 for every email in a valid form, so that route does not show which emails have an account | diff --git a/internal/app/app.go b/internal/app/app.go index d71d594..3cb537b 100644 --- a/internal/app/app.go +++ b/internal/app/app.go @@ -71,7 +71,7 @@ func New(d Deps) *fiber.App { business.NewHandlers(business.NewStore(d.Pool)).Register(v1, require) production.NewHandlers(production.NewStore(d.Pool)).Register(v1, require) - gateways := gateway.NewHandlers(gateway.NewService(d.Pool, limiter)) + gateways := gateway.NewHandlers(gateway.NewService(d.Pool, limiter), accounts.ConfirmPassword) gateways.RegisterApp(v1, require) gateways.RegisterDevice(app.Group("/device/v1")) device.NewHandlers(device.NewPairings(d.Pool, limiter)).Register(v1, require) diff --git a/internal/auth/service.go b/internal/auth/service.go index 6e7953c..9a5fdfa 100644 --- a/internal/auth/service.go +++ b/internal/auth/service.go @@ -52,7 +52,7 @@ var ( ruleForgotIP = Rule{Bucket: "forgot-ip", Max: 20, Window: time.Hour} ruleChangeIP = Rule{Bucket: "change-ip", Max: 10, Window: time.Hour} ruleChangeEmail = Rule{Bucket: "change-email", Max: 5, Window: time.Hour} - ruleDeleteAccount = Rule{Bucket: "delete-account", Max: 5, Window: time.Hour} + ruleDeleteAccount = Rule{Bucket: "delete-account", Max: 5, Window: time.Hour} // shared by every action that asks for the password again ) // Service holds the rules of the accounts @@ -350,9 +350,20 @@ func (s *Service) Logout(ctx context.Context, token string) error { } // DeleteAccount deletes the business of the user, and with it everything under the business, after the password is checked again so a -// stolen session cannot do it alone. The tries are limited for each user, because the session is all that a guesser needs. Only a wrong -// password uses up a try, so a call that ends in 503 busy or with the right password does not. +// stolen session cannot do it alone, see ConfirmPassword. func (s *Service) DeleteAccount(ctx context.Context, user *User, password string) error { + if err := s.ConfirmPassword(ctx, user, password); err != nil { + return err + } + + return s.store.DeleteBusiness(ctx, user.BusinessID) +} + +// ConfirmPassword asks the user for the password again before an action that cannot be undone, so a stolen session cannot do it alone. +// It answers nil only for the right password. The tries are limited for each user and the budget is shared by every such action, +// because the session is all that a guesser needs and two routes must not give a guesser two budgets. Only a wrong password uses up a +// try, so a call that ends in 503 busy or with the right password does not. +func (s *Service) ConfirmPassword(ctx context.Context, user *User, password string) error { if password == "" { return api.Validation(map[string]string{"password": "required"}) } @@ -388,11 +399,7 @@ func (s *Service) DeleteAccount(ctx context.Context, user *User, password string } // the budget is read again, because parallel wrong tries may have spent it while this hash ran - if err := s.check(ctx, ruleDeleteAccount, user.ID); err != nil { - return err - } - - return s.store.DeleteBusiness(ctx, user.BusinessID) + return s.check(ctx, ruleDeleteAccount, user.ID) } // Sessions lists the open sessions of the user, and marks the one that the token belongs to diff --git a/internal/gateway/enroll.go b/internal/gateway/enroll.go index f9263f1..8082e02 100644 --- a/internal/gateway/enroll.go +++ b/internal/gateway/enroll.go @@ -426,3 +426,78 @@ func (s *Service) RevokeKey(ctx context.Context, businessID, id string) error { return nil } + +// deleteBatch is how many rows one statement removes from a big table +const deleteBatch = 5000 + +// bigTablesOfAGateway are the tables that can hold millions of rows for one gateway, with the statement that removes one batch of them +var bigTablesOfAGateway = []string{ + "DELETE FROM readings WHERE ctid IN (SELECT ctid FROM readings WHERE gateway_id = $1 LIMIT $2)", + "DELETE FROM readings_hourly WHERE ctid IN (SELECT ctid FROM readings_hourly WHERE gateway_id = $1 LIMIT $2)", + "DELETE FROM anomalies WHERE ctid IN (SELECT ctid FROM anomalies WHERE gateway_id = $1 LIMIT $2)", +} + +// Delete removes a gateway of the business, and its nodes, devices, tips, relay commands and schedules go with it because the keys +// cascade. The key is cut first, so the hardware stops adding rows while the big tables are emptied, and the checks of RevokeKey give +// the same not found answers. The readings, the hourly sums and the anomalies then go in batches that each commit by themselves, as in +// the delete of an account, and the gateway row goes last. A call that stops half way leaves a gateway with a cut key and fewer rows, and +// the same call goes on from there until it ends, after which the id answers not found. The hardware gets 401 invalid_gateway_key +// and can enroll again from the start. +func (s *Service) Delete(ctx context.Context, businessID, id string) error { + if err := s.RevokeKey(ctx, businessID, id); err != nil { + return err + } + + for _, query := range bigTablesOfAGateway { + for { + tag, err := s.pool.Exec(ctx, query, id, deleteBatch) + if err != nil { + return fmt.Errorf("delete a batch of rows of the gateway %w", err) + } + + // a batch can come back short when a concurrent writer took rows of it, so only an empty batch ends the loop + if tag.RowsAffected() == 0 { + break + } + } + } + + tx, err := s.pool.BeginTx(ctx, pgx.TxOptions{}) + if err != nil { + return fmt.Errorf("start the delete of the gateway %w", err) + } + + defer func() { _ = tx.Rollback(ctx) }() + + // the lock order of Claim and RevokeKey, the waiting codes of the hardware first and the gateway after them + var hardwareID string + + err = tx.QueryRow(ctx, "SELECT hardware_id FROM gateways WHERE id = $1 AND business_id = $2", id, businessID).Scan(&hardwareID) + if errors.Is(err, pgx.ErrNoRows) { + return api.NotFound() + } + + if err != nil { + return fmt.Errorf("read the gateway %w", err) + } + + // the hardware may have enrolled again while the rows went, and a claim of that code would bring a gateway back that the owner removed + if _, err := tx.Exec(ctx, "DELETE FROM gateway_enrollments WHERE gateway_id IS NULL AND hardware_id = $1", hardwareID); err != nil { + return fmt.Errorf("remove the waiting codes %w", err) + } + + tag, err := tx.Exec(ctx, "DELETE FROM gateways WHERE id = $1 AND business_id = $2", id, businessID) + if err != nil { + return fmt.Errorf("delete the gateway %w", err) + } + + if tag.RowsAffected() == 0 { + return api.NotFound() + } + + if err := tx.Commit(ctx); err != nil { + return fmt.Errorf("finish the delete of the gateway %w", err) + } + + return nil +} diff --git a/internal/gateway/gateway_test.go b/internal/gateway/gateway_test.go index 9588c6d..c963ba8 100644 --- a/internal/gateway/gateway_test.go +++ b/internal/gateway/gateway_test.go @@ -14,6 +14,9 @@ import ( "github.com/aethera-deeptech/backend/internal/testutil" ) +// ownerPassword is the body of a delete that asks for the password, the one that SignUp gives every account +const ownerPassword = `{"password":"kata sandi baik"}` + // key makes a gateway key of the right length and the hash that the gateway sends when it enrolls func key(seed string) (string, string) { k := strings.Repeat(seed, 43)[:43] @@ -717,3 +720,238 @@ func TestARetryOfTheLastBatchKeepsTheGatewayOnlineAndAWrongNumberDoesNot(t *test t.Fatalf("there are %d readings (%v), a retry must not store them again", readings, err) } } + +func TestDeletingAGatewayRemovesItsRowsAndStopsTheHardware(t *testing.T) { + g := enrolled(t, "budi@example.com", "ESP32-AABBCC", "a") + + if r := g.send(t, batch(1, "NODE-0001", "plug", sample(0, 310))); r.Status != http.StatusOK { + t.Fatalf("the first batch answered %d %v", r.Status, r.Body) + } + + room, _ := g.e.Call(t, http.MethodPost, "/v1/rooms", `{"name":"Gudang"}`, g.session).Body["id"].(string) + if _, err := g.e.Pool.Exec(t.Context(), + "INSERT INTO devices (business_id, gateway_id, node_id, kind, name, room_id) SELECT business_id, id, 'NODE-0001', 'plug', 'Mesin', $2 FROM gateways WHERE id = $1", + g.gateway, room); err != nil { + t.Fatal(err) + } + + r := g.e.Call(t, http.MethodDelete, "/v1/gateways/"+g.gateway, ownerPassword, g.session) + if r.Status != http.StatusNoContent || len(r.Raw) != 0 { + t.Fatalf("delete answered %d %v", r.Status, r.Body) + } + + var gateways, nodes, devices, readings, hourly int + if err := g.e.Pool.QueryRow(t.Context(), + "SELECT (SELECT count(*) FROM gateways), (SELECT count(*) FROM nodes), (SELECT count(*) FROM devices), (SELECT count(*) FROM readings), (SELECT count(*) FROM readings_hourly)"). + Scan(&gateways, &nodes, &devices, &readings, &hourly); err != nil { + t.Fatal(err) + } + + if gateways+nodes+devices+readings+hourly != 0 { + t.Fatalf("after the delete there are %d gateways, %d nodes, %d devices, %d readings and %d hourly rows", gateways, nodes, devices, readings, hourly) + } + + // the room belongs to the business and stays + if r := g.e.Call(t, http.MethodGet, "/v1/rooms", "", g.session); r.Status != http.StatusOK { + t.Fatalf("the rooms answered %d", r.Status) + } + + // the hardware gets the answer of a key that is not known, and the id is gone + if r := g.send(t, batch(2, "NODE-0001", "plug", sample(0, 5))); r.Status != http.StatusUnauthorized || r.Code() != "invalid_gateway_key" { + t.Fatalf("the hardware answered %d %v, want 401 invalid_gateway_key", r.Status, r.Body) + } + + if r := g.e.Call(t, http.MethodDelete, "/v1/gateways/"+g.gateway, ownerPassword, g.session); r.Status != http.StatusNotFound { + t.Fatalf("a second delete answered %d, want 404", r.Status) + } + + if list := g.e.Call(t, http.MethodGet, "/v1/gateways", "", g.session); list.Status != http.StatusOK || len(list.Body["items"].([]any)) != 0 { + t.Fatalf("the list after the delete is %d %v", list.Status, list.Body) + } +} + +func TestDeletingAGatewayLeavesTheGatewayOfAnotherBusinessAlone(t *testing.T) { + e := testutil.New(t) + mine := enrolledOn(t, e, "budi@example.com", "ESP32-AABBCC", "a") + other := enrolledOn(t, e, "ani@example.com", "ESP32-DDEEFF", "b") + + if r := mine.send(t, batch(1, "NODE-0001", "plug", sample(10, 120))); r.Status != http.StatusOK { + t.Fatalf("the batch of mine answered %d %v", r.Status, r.Body) + } + + // the two batches of the other gateway use different seconds, because a sample with the same time is stored once + if r := other.send(t, batch(1, "NODE-0001", "plug", sample(10, 310))); r.Status != http.StatusOK { + t.Fatalf("the batch of the other answered %d %v", r.Status, r.Body) + } + + if r := mine.e.Call(t, http.MethodDelete, "/v1/gateways/"+other.gateway, ownerPassword, mine.session); r.Status != http.StatusNotFound { + t.Fatalf("a delete of the gateway of another business answered %d, want 404", r.Status) + } + + // the refused delete did not cut the key either + if r := other.send(t, batch(2, "NODE-0001", "plug", sample(0, 5))); r.Status != http.StatusOK { + t.Fatalf("the other gateway answered %d %v after a refused delete", r.Status, r.Body) + } + + if r := mine.e.Call(t, http.MethodDelete, "/v1/gateways/"+mine.gateway, ownerPassword, mine.session); r.Status != http.StatusNoContent { + t.Fatalf("delete answered %d %v", r.Status, r.Body) + } + + var readings, mineLeft int + if err := e.Pool.QueryRow(t.Context(), "SELECT count(*), count(*) FILTER (WHERE gateway_id = $1) FROM readings", mine.gateway).Scan(&readings, &mineLeft); err != nil || readings != 2 || mineLeft != 0 { + t.Fatalf("after the delete there are %d readings and %d of the deleted gateway (%v), the two of the other gateway must stay and none of mine", readings, mineLeft, err) + } +} + +func TestDeletingAGatewayRemovesACodeThatWaitsForTheSameHardware(t *testing.T) { + g := enrolled(t, "budi@example.com", "ESP32-AABBCC", "a") + + _, oldHash := key("a") + + r := g.e.CallWith(t, http.MethodPost, "/device/v1/enroll", `{"hardwareId":"ESP32-AABBCC","keyHash":"`+oldHash+`"}`, nil) + code, _ := r.Body["claimCode"].(string) + + if r = g.e.Call(t, http.MethodDelete, "/v1/gateways/"+g.gateway, ownerPassword, g.session); r.Status != http.StatusNoContent { + t.Fatalf("delete answered %d", r.Status) + } + + if r = g.e.Call(t, http.MethodPost, "/v1/gateways", `{"claimCode":"`+code+`","name":"Hub"}`, g.session); r.Status != http.StatusNotFound || r.Code() != "claim_code_unknown" { + t.Fatalf("the waiting code answered %d %v, want 404 claim_code_unknown", r.Status, r.Body) + } + + // the hardware can start again from nothing + _, hash := key("n") + + enroll := g.e.CallWith(t, http.MethodPost, "/device/v1/enroll", `{"hardwareId":"ESP32-AABBCC","keyHash":"`+hash+`"}`, nil) + fresh, _ := enroll.Body["claimCode"].(string) + + if r = g.e.Call(t, http.MethodPost, "/v1/gateways", `{"claimCode":"`+fresh+`","name":"Hub baru"}`, g.session); r.Status != http.StatusCreated || r.Body["id"] == g.gateway { + t.Fatalf("a new claim answered %d %v, want 201 with a new id", r.Status, r.Body) + } +} + +func TestOnlyASignedInOwnerCanDeleteAGatewayAndABadIDIsNotFound(t *testing.T) { + g := enrolled(t, "budi@example.com", "ESP32-AABBCC", "a") + + if r := g.e.Call(t, http.MethodDelete, "/v1/gateways/"+g.gateway, ownerPassword, ""); r.Status != http.StatusUnauthorized { + t.Fatalf("a delete with no session answered %d, want 401", r.Status) + } + + for _, id := range []string{"nope", "00000000-0000-4000-8000-000000000000", "urn:uuid:" + g.gateway, strings.ToUpper(g.gateway)} { + if r := g.e.Call(t, http.MethodDelete, "/v1/gateways/"+id, ownerPassword, g.session); r.Status != http.StatusNotFound { + t.Errorf("a delete of %q answered %d, want 404", id, r.Status) + } + } + + // nothing of this cut the key + if r := g.send(t, batch(1, "NODE-0001", "plug", sample(0, 5))); r.Status != http.StatusOK { + t.Fatalf("the gateway answered %d %v after refused deletes", r.Status, r.Body) + } +} + +func TestDeletingAGatewayEmptiesMoreRowsThanOneBatch(t *testing.T) { + g := enrolled(t, "budi@example.com", "ESP32-AABBCC", "a") + + if r := g.send(t, batch(1, "NODE-0001", "plug", sample(0, 310))); r.Status != http.StatusOK { + t.Fatalf("the first batch answered %d %v", r.Status, r.Body) + } + + // more rows than the 5000 of one batch, so the loop has to run more than once + if _, err := g.e.Pool.Exec(t.Context(), + "INSERT INTO readings (gateway_id, node_id, t, watts) SELECT $1, 'NODE-0001', now() - make_interval(secs => 100 + n), 10 FROM generate_series(1, 12000) AS n", + g.gateway); err != nil { + t.Fatal(err) + } + + if r := g.e.Call(t, http.MethodDelete, "/v1/gateways/"+g.gateway, ownerPassword, g.session); r.Status != http.StatusNoContent { + t.Fatalf("delete answered %d %v", r.Status, r.Body) + } + + var readings int + if err := g.e.Pool.QueryRow(t.Context(), "SELECT count(*) FROM readings").Scan(&readings); err != nil || readings != 0 { + t.Fatalf("after the delete there are %d readings (%v)", readings, err) + } +} + +func TestDeletingAGatewayNeedsTheRightPassword(t *testing.T) { + g := enrolled(t, "budi@example.com", "ESP32-AABBCC", "a") + + for name, tc := range map[string]struct { + body string + status int + code string + }{ + "no body": {``, http.StatusBadRequest, "invalid_json"}, + "no password": {`{}`, http.StatusUnprocessableEntity, "validation_failed"}, + "wrong password": {`{"password":"bukan kata sandi"}`, http.StatusForbidden, "invalid_credentials"}, + } { + r := g.e.Call(t, http.MethodDelete, "/v1/gateways/"+g.gateway, tc.body, g.session) + if r.Status != tc.status || r.Code() != tc.code { + t.Errorf("%s answered %d %v, want %d %s", name, r.Status, r.Body, tc.status, tc.code) + } + } + + // nothing of this cut the key or removed the gateway or its rows + if r := g.send(t, batch(1, "NODE-0001", "plug", sample(0, 5))); r.Status != http.StatusOK { + t.Fatalf("the gateway answered %d %v after refused deletes", r.Status, r.Body) + } + + var gateways, readings int + if err := g.e.Pool.QueryRow(t.Context(), "SELECT (SELECT count(*) FROM gateways), (SELECT count(*) FROM readings)").Scan(&gateways, &readings); err != nil || gateways != 1 || readings != 1 { + t.Fatalf("after the refused deletes there are %d gateways and %d readings (%v), want 1 and 1", gateways, readings, err) + } + + // the session is still fine + if r := g.e.Call(t, http.MethodGet, "/v1/auth/me", "", g.session); r.Status != http.StatusOK { + t.Fatalf("the session answered %d after refused deletes", r.Status) + } +} + +func TestTheWrongPasswordsOfAGatewayDeleteSpendTheBudgetOfTheAccountDelete(t *testing.T) { + g := enrolled(t, "budi@example.com", "ESP32-AABBCC", "a") + + for i := range 5 { + if r := g.e.Call(t, http.MethodDelete, "/v1/gateways/"+g.gateway, `{"password":"salah"}`, g.session); r.Status != http.StatusForbidden { + t.Fatalf("try %d answered %d", i+1, r.Status) + } + } + + // the right password is refused once the budget is spent, and so is the delete of the account + r := g.e.Call(t, http.MethodDelete, "/v1/gateways/"+g.gateway, ownerPassword, g.session) + if r.Status != http.StatusTooManyRequests || r.Code() != "rate_limited" || r.Body["retryAfterSeconds"] == nil { + t.Fatalf("the right password with the budget spent answered %d %v, want 429", r.Status, r.Body) + } + + if r := g.e.Call(t, http.MethodDelete, "/v1/account", ownerPassword, g.session); r.Status != http.StatusTooManyRequests { + t.Fatalf("the delete of the account answered %d, want 429 from the shared budget", r.Status) + } + + if r := g.send(t, batch(1, "NODE-0001", "plug", sample(0, 5))); r.Status != http.StatusOK { + t.Fatalf("the gateway answered %d %v", r.Status, r.Body) + } +} + +func TestARightPasswordDoesNotUseUpATryAndTheAccountDeleteSpendsTheSameBudget(t *testing.T) { + g := enrolled(t, "budi@example.com", "ESP32-AABBCC", "a") + + // 4 wrong tries of the account delete, then a right gateway delete that works, then one more wrong try that is still a 403 + for i := range 4 { + if r := g.e.Call(t, http.MethodDelete, "/v1/account", `{"password":"salah"}`, g.session); r.Status != http.StatusForbidden { + t.Fatalf("try %d answered %d", i+1, r.Status) + } + } + + if r := g.e.Call(t, http.MethodDelete, "/v1/gateways/"+g.gateway, ownerPassword, g.session); r.Status != http.StatusNoContent { + t.Fatalf("the right password after 4 wrong ones answered %d %v, want 204", r.Status, r.Body) + } + + if r := g.e.Call(t, http.MethodDelete, "/v1/gateways/"+g.gateway, `{"password":"salah"}`, g.session); r.Status != http.StatusForbidden { + t.Fatalf("the fifth wrong try answered %d %v, want 403 because the right one cost nothing", r.Status, r.Body) + } + + // the budget is now spent by the account delete tries and the gateway route shows it + r := g.e.Call(t, http.MethodDelete, "/v1/gateways/"+g.gateway, ownerPassword, g.session) + if r.Status != http.StatusTooManyRequests { + t.Fatalf("the sixth call answered %d %v, want 429", r.Status, r.Body) + } +} diff --git a/internal/gateway/handlers.go b/internal/gateway/handlers.go index a0ae03f..0253a0e 100644 --- a/internal/gateway/handlers.go +++ b/internal/gateway/handlers.go @@ -1,6 +1,7 @@ package gateway import ( + "context" "net/http" "strings" "time" @@ -13,13 +14,17 @@ import ( // Handlers are the routes that the app calls to manage its gateways and the routes that a gateway calls type Handlers struct { - svc *Service - now func() time.Time + svc *Service + confirm PasswordCheck + now func() time.Time } -// NewHandlers returns the routes -func NewHandlers(svc *Service) *Handlers { - return &Handlers{svc: svc, now: time.Now} +// PasswordCheck asks a signed in user for the password again and answers nil only for the right one +type PasswordCheck func(ctx context.Context, user *auth.User, password string) error + +// NewHandlers returns the routes. The confirm function guards the routes that cannot be undone. +func NewHandlers(svc *Service, confirm PasswordCheck) *Handlers { + return &Handlers{svc: svc, confirm: confirm, now: time.Now} } // RegisterApp adds the routes of the app under /v1 behind the middleware that needs a signed in user @@ -27,6 +32,7 @@ func (h *Handlers) RegisterApp(r fiber.Router, require fiber.Handler) { r.Get("/gateways", require, h.list) r.Post("/gateways", require, h.claim) r.Post("/gateways/:id/revoke-key", require, h.revokeKey) + r.Delete("/gateways/:id", api.BodyLimit(maxConfirmBody), require, h.deleteGateway) } // RegisterDevice adds the routes of a gateway under /device/v1. They take no cookie and need no CSRF header, because a gateway is @@ -48,6 +54,9 @@ func refuseEncoding(c fiber.Ctx) error { return c.Next() } +// maxConfirmBody is the largest body of a request that asks for the password again +const maxConfirmBody = 2 << 10 + // maxEnrollBody is the largest body of an enroll request, which has a hardware id, a version and a hash const maxEnrollBody = 2 << 10 @@ -86,6 +95,29 @@ func (h *Handlers) revokeKey(c fiber.Ctx) error { return c.SendStatus(http.StatusNoContent) } +// deleteGateway removes a gateway after a check of the password, which is the same check and the same budget as the delete of an account +func (h *Handlers) deleteGateway(c fiber.Ctx) error { + var in struct { + Password string `json:"password"` + } + + if err := api.Decode(c, &in); err != nil { + return err + } + + user := auth.UserFrom(c) + + if err := h.confirm(c.Context(), user, in.Password); err != nil { + return err + } + + if err := h.svc.Delete(c.Context(), user.BusinessID, c.Params("id")); err != nil { + return err + } + + return c.SendStatus(http.StatusNoContent) +} + func (h *Handlers) enroll(c fiber.Ctx) error { var in struct { HardwareID string `json:"hardwareId"` diff --git a/openapi.yaml b/openapi.yaml index fd7a98e..973c1e7 100644 --- a/openapi.yaml +++ b/openapi.yaml @@ -226,7 +226,7 @@ paths: tags: [Account] operationId: deleteAccount summary: Delete the account and the business with everything under it - description: "The password is checked again, so a stolen session cannot delete the account alone. The delete takes the business and everything under it. That covers the user, the sessions, the gateways with their keys, the nodes, the devices, the readings, the anomalies, the tips, the relay commands and schedules, the rooms and the production records. It cannot be undone. The answer clears the cookie like logout. A wrong password answers 403 `invalid_credentials`, and not 401, because the session is still valid. A user has 5 wrong passwords an hour, and only a wrong password uses up a try, so a call that ends in 503 `busy` or with the right password does not. After the fifth wrong one the route answers 429 for every call, also with the right password. The big tables go first in batches, so a 503 `busy` can come after a part of the readings was deleted, or after the whole delete when the time ran out just after the commit. The same call can be repeated until it ends with 204. A 401 on the repeat means that the session is gone, which a finished delete causes, and so does a session that ended in the meantime. The app cannot tell the two apart from the answer, so it sends the user to the sign in page. A sign in with the right password works only while the account still exists, and a failed sign in proves nothing, because a wrong password gives the same answer." + description: "The password is checked again, so a stolen session cannot delete the account alone. The delete takes the business and everything under it. That covers the user, the sessions, the gateways with their keys, the nodes, the devices, the readings, the anomalies, the tips, the relay commands and schedules, the rooms and the production records. It cannot be undone. The answer clears the cookie like logout. A wrong password answers 403 `invalid_credentials`, and not 401, because the session is still valid. A user has 5 wrong passwords an hour, shared with `DELETE /v1/gateways/{id}`, and only a wrong password uses up a try, so a call that ends in 503 `busy` or with the right password does not. After the fifth wrong one the route answers 429 for every call, also with the right password. The big tables go first in batches, so a 503 `busy` can come after a part of the readings was deleted, or after the whole delete when the time ran out just after the commit. The same call can be repeated until it ends with 204. A 401 on the repeat means that the session is gone, which a finished delete causes, and so does a session that ended in the meantime. The app cannot tell the two apart from the answer, so it sends the user to the sign in page. A sign in with the right password works only while the account still exists, and a failed sign in proves nothing, because a wrong password gives the same answer." requestBody: required: true content: @@ -616,6 +616,41 @@ paths: "401": {$ref: "#/components/responses/Unauthorized"} "404": {$ref: "#/components/responses/NotFound"} "4XX": {$ref: "#/components/responses/Error"} + /v1/gateways/{id}: + delete: + tags: [Pasang alat] + operationId: deleteGateway + summary: Remove a gateway + description: "The password is checked again, so a stolen session cannot remove a gateway alone. It is the same check and the same budget as `DELETE /v1/account`, so a user has 5 wrong passwords an hour for both routes together, only a wrong password uses up a try, and after the fifth wrong one the route answers 429 for every call, also with the right password. The check comes first, so a repeat of the call sends the password again. The gateway goes with its nodes, devices, readings, hourly sums, anomalies, tips, relay commands and schedules. It cannot be undone. The rooms stay. The service cuts the key first, so the hardware stops adding batches, apart from one that was already running, and then deletes the big tables in batches and the gateway row last. A 503 `busy` can therefore come after a part of the rows was deleted, and the same call can be repeated until it ends with 204. A 404 on the repeat means that the delete finished or that the id is not a gateway of the business, and the app cannot tell the two apart. The hardware gets 401 `invalid_gateway_key` from then on and can enroll again from the start, and a claim code that waits for the same hardware is removed. An id of another business, an unknown id and an id that is not a plain lower case uuid all answer 404." + parameters: + - name: id + in: path + required: true + schema: {type: string, format: uuid} + requestBody: + required: true + content: + application/json: + schema: + type: object + required: [password] + additionalProperties: false + properties: + password: {type: string, minLength: 1} + responses: + "204": + description: The gateway and everything under it is deleted + "401": {$ref: "#/components/responses/Unauthorized"} + "403": + description: The password is wrong (code `invalid_credentials`), or the CSRF header or the origin is missing + content: + application/json: + schema: {$ref: "#/components/schemas/Error"} + "404": {$ref: "#/components/responses/NotFound"} + "422": {$ref: "#/components/responses/Validation"} + "429": {$ref: "#/components/responses/RateLimited"} + "503": {$ref: "#/components/responses/Busy"} + "4XX": {$ref: "#/components/responses/Error"} /v1/pairings: post: tags: [Pasang alat] @@ -659,7 +694,7 @@ paths: tags: [Pasang alat] operationId: getPairing summary: Read the state of a pairing - description: The app asks again every 3 seconds while the status is `searching`. `failed` covers a search that timed out after 3 minutes and a search that the owner stopped. `connected` has the id of the new device. The device exists as soon as the first reading of the node has arrived, so there is no state between `searching` and `connected`. + description: The app asks again every 3 seconds while the status is `searching`. `failed` covers a search that timed out after 3 minutes and a search that the owner stopped. `connected` has the id of the new device, or null when the device was deleted later, for example with its gateway. The device exists as soon as the first reading of the node has arrived, so there is no state between `searching` and `connected`. responses: "200": description: The pairing