Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,36 @@ All notable changes to EigenScript are documented here.
`str of` over the hard doubles and the exact-integer band, validated
with a planted fault in each half (the precision escalation and the
integer bound are independently load-bearing).
- **DB results carry their SQL type, and DB failures raise (#887,
#888).** Two defects in one function, both silent. `db_query_json`
emitted every column as a JSON string, so SQL `false` arrived as
`"f"` — a non-empty string, therefore **truthy** — and
`if row.is_admin:` passed for a non-admin; NULL and `''` were both
`""` with no way to tell them apart; and `9 > 10` held because `'9' >
'1'`. And every failure — syntax error, missing table, revoked
permission, dead connection — returned `[]`, the same value a
successful query over an empty table returns, so a reporting script
kept printing "0 rows" forever after a schema change.
Now: `PQgetisnull` → `null` (checked before the type), `boolean` →
`true`/`false`, the exact-integer and float types → JSON numbers,
everything else → strings, through **one classifier shared with
`db_query_value`** so the two cannot drift. Failures raise a catchable
`io` error carrying libpq's own first line; a genuinely empty result
is still `[]`/`""` and only that. `db_connect` still reports by return
value, so probing for a database needs no `try`.
Two calls documented in `docs/BUILTINS.md`: **`numeric` stays a
string** (arbitrary-precision decimal cannot round-trip through a
binary double — silently rounding money is the defect class this
fixes; `::float8` is the opt-in), and a **`bigint` past 2^53 raises**
naming the column and the fix (`id::text`) rather than rounding a
primary key. `bigint` is a number rather than a string because
`count(*)` and `sum(integer)` return it — leaving it text would have
left the bug unfixed for most real numeric queries. The mapping is a
function of the column's SQL type alone, never of the row's value.
`tests/test_db.eigs` gained the no-connection raise checks (they run
wherever the extension is compiled in — previously the whole file was
effectively inert without a server) plus DB18–DB26 against the live
CI postgres service.

- **chart renders 1.5× faster at high point counts (#828).** The series
hot loop called `_chart_map` — a fresh 2-element list — per plotted
Expand Down
67 changes: 63 additions & 4 deletions docs/BUILTINS.md
Original file line number Diff line number Diff line change
Expand Up @@ -681,10 +681,69 @@ Requires full build with libpq. PostgreSQL client.

| Name | Signature | Description |
|------|-----------|-------------|
| `db_connect` | `db_connect of null` | Connect via DATABASE_URL env var |
| `db_query_value` | `db_query_value of sql` or `db_query_value of [sql, p1, p2]` | Execute query, return first value with optional params |
| `db_execute` | `db_execute of sql` or `db_execute of [sql, p1, p2]` | Execute command with optional params |
| `db_query_json` | `db_query_json of sql` or `db_query_json of [sql, p1, p2]` | Execute query, return all rows as JSON with optional params |
| `db_connect` | `db_connect of null` | Connect via DATABASE_URL env var; returns a status JSON, never raises |
| `db_query_value` | `db_query_value of sql` or `db_query_value of [sql, p1, p2]` | Execute query, return row 0 col 0 typed by its SQL type; `null` for SQL NULL, `""` for no rows |
| `db_execute` | `db_execute of sql` or `db_execute of [sql, p1, p2]` | Execute command with optional params; returns `"ok"` |
| `db_query_json` | `db_query_json of sql` or `db_query_json of [sql, p1, p2]` | Execute query, return all rows as a JSON array of objects, each value typed by its SQL type |

### Failures raise (#888)

`db_connect` is the only one that reports by return value — it hands back
`{"status": ...}` so a program can probe for a database without a `try`.
Every other db builtin **raises** a catchable `io` error when the statement
fails or there is no connection, carrying libpq's own first line
(`ERROR: relation "orders" does not exist`). A genuinely empty result is
still `[]` / `""`, and only that.

They used to return `[]` / `""` for a syntax error, a missing table, a
revoked permission *and* an empty table alike, so a reporting script kept
printing "0 rows" forever after a schema change and a migration that did
nothing looked healthy in CI.

```eigenscript
try:
rows is json_decode of (db_query_json of "SELECT * FROM orders")
catch e:
print of ("query failed: " + e.message) # e.kind is "io"
```

### SQL types survive the trip (#887)

Values carry their column's SQL type rather than arriving as strings:

| SQL type | Arrives as | Note |
|---|---|---|
| NULL (any column type) | `null` | Distinct from `""` — checked before the type |
| `boolean` | `true` / `false` → `1` / `0` | `if row.is_admin:` means what it reads as |
| `smallint`, `integer`, `bigint`, `oid` | number | `bigint` past 2^53 **raises** — see below |
| `real`, `double precision` | number | `NaN`/`Infinity` arrive as strings; JSON has no literal for them |
| `numeric` | **string** | Deliberate — see below |
| everything else | string | text, date, uuid, json, … unchanged |

The mapping is a function of the column's SQL type alone, never of the
row's value: a column that decoded as a number for row 1 and a string for
row 100 would break `row.n + 1` on data rather than on schema.

**`numeric` stays a string.** It is PostgreSQL's arbitrary-precision decimal
— the money type — and an EigenScript number is a binary double, which
cannot hold `numeric(38,10)` or even `0.1` exactly. Preserving the digits
is the safe default; `SELECT amount::float8` is the one-token opt-in to a
number when approximate is fine. Note `avg()` and `sum(numeric)` return
`numeric`, so those want the cast; `count(*)` and `sum(integer)` return
`bigint` and are already numbers.

**A `bigint` past 2^53 raises** instead of silently rounding, naming the
column and the fix:

```
Error line 3: db: column 'id' value 9007199254740993 exceeds the exact-integer
range of a number (2^53); select it as text (id::text) to keep the digits
```

Before this, every value was a string: SQL `false` arrived as `"f"`, which
is a non-empty string and therefore **truthy**, so `if row.is_admin:` passed
for a non-admin; NULL and `''` were both `""`; and `9 > 10` was true because
`'9' > '1'`.

## Optional: Model Extension

Expand Down
2 changes: 1 addition & 1 deletion docs/DIAGNOSTICS.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ below are the contract.)
| `value` | right type, unacceptable value | `index must be an integer, got 1.5`, `chr of 0`, invalid channel |
| `index_range` | index/slice outside bounds | `index 10 out of range (list length 3)` |
| `parse` | runtime-surfaced parse/compile failure | `eval: parse error in code string`, `import: parse errors in 'm'` |
| `io` | the outside world failed | `import: cannot read 'm'`, `store_open: cannot create`, thread-create failure |
| `io` | the outside world failed | `import: cannot read 'm'`, `store_open: cannot create`, thread-create failure, `db: query failed: ERROR: relation "orders" does not exist` |
| `limit` | engine resource cap hit | `call stack overflow`, `store_put: record too large`, route table full |
| `sandbox` | sandbox policy denial or budget | `blocked in sandbox`, `sandbox memory budget exceeded` |
| `interrupt` | host-requested abort (`eigs_abort`) | `aborted` |
Expand Down
Loading
Loading