Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ The contract is `openapi.yaml` in the root of the repository. The job `OpenAPI l

## Deleting the account

`DELETE /v1/account` takes `{"password": "..."}` and deletes the business with everything under it, then clears the cookie and answers 204. The password is checked again so that a stolen session cannot do it alone, a wrong one answers 403 `invalid_credentials` and not 401, and a user has 5 wrong passwords an hour. Only a wrong password uses up a try, so a call that ends in 503 `busy` or with the right password does not. After the fifth wrong one the route answers 429 `rate_limited` with `Retry-After` for every call, also with the right password. The budget is read again after the password check, so a burst of parallel calls cannot get a right password through once the budget is spent. The readings, the hourly sums and the anomalies go first in batches, and the business row goes last. A 503 `busy` can therefore come after a part of the readings was deleted, or after the whole delete when the time ran out just after the commit. The same call can be repeated until it ends with 204. A 401 on the repeat means that the session is gone, which a finished delete causes, and so does a session that ended in the meantime. The app cannot tell the two apart from the answer, so it sends the user to the sign in page. A sign in with the right password works only while the account still exists, and a failed sign in proves nothing, because a wrong password gives the same answer. It cannot be undone.
`DELETE /v1/account` takes `{"password": "..."}` and deletes the business with everything under it, then clears the cookie and answers 204. The password is checked again so that a stolen session cannot do it alone, a wrong one answers 403 `invalid_credentials` and not 401, and a user has 5 wrong passwords an hour, shared with `DELETE /v1/gateways/{id}`. Only a wrong password uses up a try, so a call that ends in 503 `busy` or with the right password does not. After the fifth wrong one the route answers 429 `rate_limited` with `Retry-After` for every call, also with the right password. The budget is read again after the password check, so a burst of parallel calls cannot get a right password through once the budget is spent. The readings, the hourly sums and the anomalies go first in batches, and the business row goes last. A 503 `busy` can therefore come after a part of the readings was deleted, or after the whole delete when the time ran out just after the commit. The same call can be repeated until it ends with 204. A 401 on the repeat means that the session is gone, which a finished delete causes, and so does a session that ended in the meantime. The app cannot tell the two apart from the answer, so it sends the user to the sign in page. A sign in with the right password works only while the account still exists, and a failed sign in proves nothing, because a wrong password gives the same answer. It cannot be undone.

## Errors

Expand All @@ -43,7 +43,8 @@ An amount is a whole number of rupiah. A tip shows its saving rounded to the nea
2. The owner types the code in the app (`POST /v1/gateways`). The gateway polls and learns that it belongs to a business.
3. From then on the gateway sends its key as a bearer token. The service keeps only the hash of the key, so a copy of the database cannot act as a gateway. TLS protects the key in transit, and the sequence number of a batch makes a retry safe. A signature on every request would add nothing over TLS here, because a party that sees the plain request also sees the key.
4. The owner can cut off the key of a gateway with `POST /v1/gateways/{id}/revoke-key`, which answers 204. The old key gets 401 `invalid_gateway_key` from then on, and the devices and readings stay. The list shows `keyRevoked` as true until the hardware enrolls again with a new key and the owner claims the new code. The route returns no claim code, because the gateway makes its own, and it removes a claim code that waits for the same hardware, so the gateway has to enroll again with a new key. An id of another business answers 404.
5. The owner starts a pairing in the app. The next unclaimed node of that kind that the gateway reports becomes the device. `POST /v1/pairings` answers 409 `no_gateway` when the business has no gateway with a working key, which is the case when it has none and when the owner cut the key of every one, because a gateway with a cut key cannot report a node.
5. The owner can remove a gateway with `DELETE /v1/gateways/{id}`, which takes `{"password": "..."}`, answers 204 and cannot be undone. The password is checked again so that a stolen session cannot do it alone, a wrong one answers 403 `invalid_credentials`, and a user has 5 wrong passwords an hour for this route and `DELETE /v1/account` together. Only a wrong password uses up a try, and after the fifth wrong one the route answers 429 `rate_limited` for every call, also with the right password. A repeat of the call sends the password again. The gateway goes with its nodes, devices, readings, hourly sums, anomalies, tips, relay commands and schedules. The rooms stay. The service cuts the key first, so the hardware stops adding batches, apart from one that was already running. The readings, the hourly sums and the anomalies then go in batches, and the gateway row goes last. A 503 `busy` can therefore come after a part of the rows was deleted, and the same call can be repeated until it ends with 204. A 404 on the repeat means that the delete finished or that the id is not a gateway of the business, and the app cannot tell the two apart. The hardware gets 401 `invalid_gateway_key` from then on, and it can enroll again from the start. A claim code that waits for the same hardware is removed. A finished pairing that pointed at one of its devices then shows `deviceId` as null. An id of another business, an unknown id and an id that is not a plain lower case uuid all answer 404.
6. The owner starts a pairing in the app. The next unclaimed node of that kind that the gateway reports becomes the device. `POST /v1/pairings` answers 409 `no_gateway` when the business has no gateway with a working key, which is the case when it has none and when the owner cut the key of every one, because a gateway with a cut key cannot report a node.

The list and the detail of a device carry `kind`, which is `plug` or `clamp`, so the app can show the relay switch only on a plug. `kindLabel` is the text for the screen and is not meant to be parsed.

Expand Down
2 changes: 2 additions & 0 deletions docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ The app is in Indonesian, and the API keeps a few of those words in its fields a

**Can an owner delete an account?** Yes. `DELETE /v1/account` with the password deletes the business and everything under it, and it cannot be undone. See [Privacy](privacy.md#deleting-an-account).

**Can an owner remove a gateway?** Yes. `DELETE /v1/gateways/{id}` with the password removes the gateway with its devices and readings, and it cannot be undone. The hardware gets 401 and can enroll again from the start. See [API](api.md).

**Can a session be ended from the app?** Yes. The owner can list the signed in devices, end one of them, or end all the others, and signing out ends the session of that device. A session also ends after 30 days without use, after 90 days, or when the owner resets the password. See [Security](security.md).

**How long is the data kept?** Raw readings 30 days, hourly sums 400 days and anomalies 90 days. See [Privacy](privacy.md#how-long-it-stays).
Expand Down
2 changes: 2 additions & 0 deletions docs/privacy.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ There is no analytics script and no advertising. The service sets one cookie, th

The owner deletes the account in the app with `DELETE /v1/account`, which asks for the password again. The route first deletes the readings, the hourly sums and the anomalies of the business in batches that commit one by one. Then it deletes the business row, and every other table under the business goes with it in that one statement, because the keys cascade, see [Data model](data-model.md). That covers every table under the business, for example the user, the sessions, the gateways, the devices, the tips and the relay data. A request that stops half way leaves the account in place with part of its readings gone, and the next call goes on. It cannot be undone. The operator can still delete a business row by hand for an owner who cannot sign in. The logs, the backups, the reverse proxy of the host, Sentry, Grafana Cloud and the mail provider keep their own copies and follow their own retention.

The owner can also remove one gateway with `DELETE /v1/gateways/{id}`, which asks for the password again in the same way. It cuts the key first, deletes the readings, the hourly sums and the anomalies of that gateway in batches, and then deletes the gateway row, and its nodes, devices, tips, relay commands and schedules go with it because the keys cascade. A request that stops half way leaves a gateway with a cut key and fewer rows, and the next call goes on. It cannot be undone. The copies that the paragraph above names keep their own retention.

## Consent

A sign up counts as the consent to the terms and the privacy policy. The service stores the time and the version of the texts in `users`, so a later change of the texts can be tied to who accepted which version.
2 changes: 1 addition & 1 deletion docs/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ This page describes what the API protects, who it protects it from and what stop
| Guessing a password | A password of about 1400 of the most common ones is refused when it is set. A password is hashed with argon2id, with a cost that follows the OWASP guidance. Only a few hashes run at once, and a request waits a few seconds for a free slot and answers 503 `busy` only when none opens in that time. A sign in with an unknown email does about the same work as a wrong password, and the answer is the same |
| Flooding the account routes | Each public account route has a limit for the client address, and most also have one for the email, for example 10 sign ins for an email in 15 minutes. The counters live in PostgreSQL, so every copy of the service shares them. A flood with a new email each time is limited by the address only, and the proxy has to limit it too |
| Reading another business | Every query is limited to the business of the signed in account. An id of another business answers 404 like an id that does not exist, so the answer does not tell whether it exists |
| A stolen session | The cookie is `HttpOnly` and `SameSite=Lax`, and it is `Secure` with the `__Host-` prefix in production. The service stores only the SHA-256 hash of a session. A session ends after 30 days without use or 90 days in all, an account keeps at most 20, and a password reset ends every session of the account, and the owner can end one or all the others in the list of devices |
| A stolen session | The cookie is `HttpOnly` and `SameSite=Lax`, and it is `Secure` with the `__Host-` prefix in production. The service stores only the SHA-256 hash of a session. A session ends after 30 days without use or 90 days in all, an account keeps at most 20, and a password reset ends every session of the account, and the owner can end one or all the others in the list of devices. An action that cannot be undone, the delete of the account and the delete of a gateway, asks for the password again, and both share a budget of 5 wrong passwords an hour |
| A forged request from another site | The CORS rule allows only the origin of the app, and the browser refuses the preflight of any other origin. A request that changes data also needs the header `X-Requested-With`, and its `Origin` must be the origin of the app when the browser sends one. A page of another site cannot add the header without a preflight |
| A stolen gateway key | The service stores only the SHA-256 hash of the key, so a copy of the database does not hold a key that works. It does allow offline guesses against the hash, so a key must come from a strong random source, such as 32 random bytes. A gateway can send 600 batches an hour, and a batch has a sequence number, so a replay is acknowledged again and does not write twice |
| A mail link that leaks | A verification link lives 24 hours and a reset link one hour. The service stores only the hash of a token, a link works once, and the answer of the reset route is 202 for every email in a valid form, so that route does not show which emails have an account |
Expand Down
2 changes: 1 addition & 1 deletion internal/app/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ func New(d Deps) *fiber.App {
business.NewHandlers(business.NewStore(d.Pool)).Register(v1, require)
production.NewHandlers(production.NewStore(d.Pool)).Register(v1, require)

gateways := gateway.NewHandlers(gateway.NewService(d.Pool, limiter))
gateways := gateway.NewHandlers(gateway.NewService(d.Pool, limiter), accounts.ConfirmPassword)
gateways.RegisterApp(v1, require)
gateways.RegisterDevice(app.Group("/device/v1"))
device.NewHandlers(device.NewPairings(d.Pool, limiter)).Register(v1, require)
Expand Down
23 changes: 15 additions & 8 deletions internal/auth/service.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ var (
ruleForgotIP = Rule{Bucket: "forgot-ip", Max: 20, Window: time.Hour}
ruleChangeIP = Rule{Bucket: "change-ip", Max: 10, Window: time.Hour}
ruleChangeEmail = Rule{Bucket: "change-email", Max: 5, Window: time.Hour}
ruleDeleteAccount = Rule{Bucket: "delete-account", Max: 5, Window: time.Hour}
ruleDeleteAccount = Rule{Bucket: "delete-account", Max: 5, Window: time.Hour} // shared by every action that asks for the password again
)

// Service holds the rules of the accounts
Expand Down Expand Up @@ -350,9 +350,20 @@ func (s *Service) Logout(ctx context.Context, token string) error {
}

// DeleteAccount deletes the business of the user, and with it everything under the business, after the password is checked again so a
// stolen session cannot do it alone. The tries are limited for each user, because the session is all that a guesser needs. Only a wrong
// password uses up a try, so a call that ends in 503 busy or with the right password does not.
// stolen session cannot do it alone, see ConfirmPassword.
func (s *Service) DeleteAccount(ctx context.Context, user *User, password string) error {
if err := s.ConfirmPassword(ctx, user, password); err != nil {
return err
}

return s.store.DeleteBusiness(ctx, user.BusinessID)
}

// ConfirmPassword asks the user for the password again before an action that cannot be undone, so a stolen session cannot do it alone.
// It answers nil only for the right password. The tries are limited for each user and the budget is shared by every such action,
// because the session is all that a guesser needs and two routes must not give a guesser two budgets. Only a wrong password uses up a
// try, so a call that ends in 503 busy or with the right password does not.
func (s *Service) ConfirmPassword(ctx context.Context, user *User, password string) error {
if password == "" {
return api.Validation(map[string]string{"password": "required"})
}
Expand Down Expand Up @@ -388,11 +399,7 @@ func (s *Service) DeleteAccount(ctx context.Context, user *User, password string
}

// the budget is read again, because parallel wrong tries may have spent it while this hash ran
if err := s.check(ctx, ruleDeleteAccount, user.ID); err != nil {
return err
}

return s.store.DeleteBusiness(ctx, user.BusinessID)
return s.check(ctx, ruleDeleteAccount, user.ID)
}

// Sessions lists the open sessions of the user, and marks the one that the token belongs to
Expand Down
75 changes: 75 additions & 0 deletions internal/gateway/enroll.go
Original file line number Diff line number Diff line change
Expand Up @@ -426,3 +426,78 @@ func (s *Service) RevokeKey(ctx context.Context, businessID, id string) error {

return nil
}

// deleteBatch is how many rows one statement removes from a big table
const deleteBatch = 5000

// bigTablesOfAGateway are the tables that can hold millions of rows for one gateway, with the statement that removes one batch of them
var bigTablesOfAGateway = []string{
"DELETE FROM readings WHERE ctid IN (SELECT ctid FROM readings WHERE gateway_id = $1 LIMIT $2)",
"DELETE FROM readings_hourly WHERE ctid IN (SELECT ctid FROM readings_hourly WHERE gateway_id = $1 LIMIT $2)",
"DELETE FROM anomalies WHERE ctid IN (SELECT ctid FROM anomalies WHERE gateway_id = $1 LIMIT $2)",
}
Comment thread
SyafaHadyan marked this conversation as resolved.
Resolved

// Delete removes a gateway of the business, and its nodes, devices, tips, relay commands and schedules go with it because the keys
// cascade. The key is cut first, so the hardware stops adding rows while the big tables are emptied, and the checks of RevokeKey give
// the same not found answers. The readings, the hourly sums and the anomalies then go in batches that each commit by themselves, as in
// the delete of an account, and the gateway row goes last. A call that stops half way leaves a gateway with a cut key and fewer rows, and
// the same call goes on from there until it ends, after which the id answers not found. The hardware gets 401 invalid_gateway_key
// and can enroll again from the start.
func (s *Service) Delete(ctx context.Context, businessID, id string) error {
if err := s.RevokeKey(ctx, businessID, id); err != nil {
return err
}

for _, query := range bigTablesOfAGateway {
for {
tag, err := s.pool.Exec(ctx, query, id, deleteBatch)
if err != nil {
return fmt.Errorf("delete a batch of rows of the gateway %w", err)
}

// a batch can come back short when a concurrent writer took rows of it, so only an empty batch ends the loop
if tag.RowsAffected() == 0 {
break
}
}
}

tx, err := s.pool.BeginTx(ctx, pgx.TxOptions{})
if err != nil {
return fmt.Errorf("start the delete of the gateway %w", err)
}

defer func() { _ = tx.Rollback(ctx) }()

// the lock order of Claim and RevokeKey, the waiting codes of the hardware first and the gateway after them
var hardwareID string

err = tx.QueryRow(ctx, "SELECT hardware_id FROM gateways WHERE id = $1 AND business_id = $2", id, businessID).Scan(&hardwareID)
if errors.Is(err, pgx.ErrNoRows) {
return api.NotFound()
}

if err != nil {
return fmt.Errorf("read the gateway %w", err)
}

// the hardware may have enrolled again while the rows went, and a claim of that code would bring a gateway back that the owner removed
if _, err := tx.Exec(ctx, "DELETE FROM gateway_enrollments WHERE gateway_id IS NULL AND hardware_id = $1", hardwareID); err != nil {
return fmt.Errorf("remove the waiting codes %w", err)
}

tag, err := tx.Exec(ctx, "DELETE FROM gateways WHERE id = $1 AND business_id = $2", id, businessID)
if err != nil {
return fmt.Errorf("delete the gateway %w", err)
}

if tag.RowsAffected() == 0 {
return api.NotFound()
}

if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("finish the delete of the gateway %w", err)
}

return nil
}
Loading
Loading