perspective-correction-js

Dependency-free perspective (keystone) correction for photographed documents, in about 60 lines of JavaScript: solve a homography from the four corners of the page, then warp with an inverse map and bilinear sampling. It works on anything shaped like ImageData, in the browser or in Node.

It also covers the part that surprised us: four corners alone don't tell you the page's aspect ratio, and guessing it from side lengths can be off by more than 40 %.

This is a simplified, standalone version of the perspective step in LensUp, a browser-based document scanner we build. A Japanese write-up of the same material is on Qiita.

File	What it is
warp.mjs	solveHomography, outputSize, warpPerspective
aspect-ratio.mjs	aspectFromFocal: the page's real aspect ratio when the focal length is known (Zhang & He)
test.mjs	Warps a synthetic photo back and measures the error
simulate-aspect.mjs	Pinhole-camera simulation of the aspect ratio problem

Usage

import { warpPerspective } from './warp.mjs';

const ctx = canvas.getContext('2d', { willReadFrequently: true });
const src = ctx.getImageData(0, 0, canvas.width, canvas.height);
const out = warpPerspective(src, quad); // quad: page corners [TL, TR, BR, BL] in pixels

const result = document.createElement('canvas');
result.width = out.width; result.height = out.height;
result.getContext('2d').putImageData(new ImageData(out.data, out.width, out.height), 0, 0);

• solveHomography(src, dst) returns the 8 coefficients [a, b, c, d, e, f, g, h] mapping src[i] to dst[i], or null when there is no solution (for example, three collinear corners).
• outputSize(quad, cap = 1800) returns { width, height }: the longer of each pair of opposite sides, with the long edge capped.
• warpPerspective(img, quad, cap = 1800) returns { width, height, data } with RGBA data. Pixels that fall outside the source are white.

Run the tests

node test.mjs
node simulate-aspect.mjs

Output on Node 22:

output size 615 x 487 | outputSize(): {"width":615,"height":487}
mean abs error vs document (0-255): 1.09 | naive bbox-crop baseline: 87.21
corners map back to quad: [["150.000","100.000"],["620.000","160.000"],["700.000","640.000"],["90.000","560.000"]]
singular quad (3 collinear points) -> null
phone-like f=2200px, tilt 20°: true w/h=0.707 | side-length estimate=0.840 (18.7%) | Zhang&He with known focal=0.707 (-0.0%) | ...
phone-like f=2200px, tilt 35°: true w/h=0.707 | side-length estimate=1.023 (44.7%) | Zhang&He with known focal=0.707 (0.0%) | ...
phone-like f=2200px, tilt 50°: true w/h=0.707 | side-length estimate=1.340 (89.5%) | Zhang&He with known focal=0.707 (0.0%) | ...

How it works

The homography

The four corners of the skewed page in the photo and the four corners of the output rectangle are related by a projective transform:

x = (a·u + b·v + c) / (g·u + h·v + 1)
y = (d·u + e·v + f) / (g·u + h·v + 1)

There are eight unknowns, a to h. Each corner pair gives two equations, so four corners give eight. solveHomography builds that 8×8 system and solves it by Gauss-Jordan elimination with partial pivoting. In a real app it's also worth checking, before solving, that the quad is convex, not self-intersecting, and has no tiny sides.

Draw with the inverse map

The obvious approach is to push every source pixel to its new position (a forward map). Where the image gets stretched, neighbouring source pixels land more than one output pixel apart and you get holes. So warpPerspective goes the other way: for every output pixel it works out where to read from in the source.

The small trick: solving solveHomography in the direction output rectangle → source quad gives you the inverse map's coefficients directly, so there is no 3×3 matrix to invert.

Mind the half pixel

The corner coordinates describe the edges of the image, but pixel values sit at pixel centres. That's why the code maps u + 0.5 (the centre of the output pixel) and subtracts 0.5 to get back to array indices. Leave it out and the whole result shifts by half a pixel, which shows up as a one-pixel smear or gap along the borders.

The read position is almost never an integer, so the four neighbouring pixels are blended by distance (bilinear interpolation). That keeps the edges of letters from going jagged.

Does it work?

test.mjs draws a 300×420 checkerboard "document" into an 800×700 "photo" at known corners [[150,100],[620,160],[700,640],[90,560]], corrects it with warpPerspective, and compares the result with the document rescaled to the same size.

Method	Mean absolute error vs the document (0–255)
warpPerspective	1.09
Crop the quad's bounding box and stretch it	87.21

The remaining error comes from interpolating twice: once to make the photo, once to correct it. Mapping the four output corners back through the inverse map lands exactly on the corners we started from.

The trap: four corners don't fix the aspect ratio

That same test shows a problem. outputSize takes the longer of each pair of opposite sides, and here that gives 615×487, a landscape image. The document was 300×420, portrait.

That isn't a bug in outputSize. A homography can map a rectangle of any proportions onto the same quad, so a tall page tilted one way and a wider page tilted another way can produce the same four corners. What breaks the tie is the camera, in particular its focal length.

simulate-aspect.mjs shows how far off the side-length guess gets. It places an A4 page (width/height 0.707) in front of a pinhole camera (3000×4000 photo, focal length 2200 px) with its top edge tilted toward the camera:

Tilt	Ratio from side lengths	Error	Zhang & He with the focal length
20°	0.840	+18.7 %	0.707
35°	1.023	+44.7 %	0.707
50°	1.340	+89.5 %	0.707

At 35° an A4 page comes out almost square. With the focal length known, the method in Zhang & He, Whiteboard scanning and image enhancement (Microsoft Research, MSR-TR-2003-39, section 4.1) recovers the ratio; aspectFromFocal implements it, and in this idealized simulation it gives exactly 0.707. But you can't count on a reliable focal length for a photo handed to a web page.

Two things help in practice:

• If you know the paper, use its ratio. Document scanning usually involves a known size: A4, US Letter, an ID card (ID-1). LensUp's perspective tool has an "Original page shape" setting (A4, Letter, ID-1 or a custom ratio), and only when it's left on Unknown do the side lengths decide.
• Shoot as straight-on as you can. The smaller the tilt, the smaller the side-length error.

Notes from production

What the version in LensUp does on top of this:

• Shrinks the input first. Photos are scaled to a 2200 px long edge when they load; looping over every pixel of a full-size phone photo is too slow.
• Caps the output. That's the cap parameter, 1800 px.
• Keeps the UI responsive. For outputs taller than 256 rows, the loop yields to the main thread every 64 rows (something like if (v % 64 === 0) await new Promise((r) => setTimeout(r));) and checks an AbortSignal.
• Reads only what it needs. For each 64-row strip of output, it computes which part of the source the strip can touch and calls getImageData on just that region. On strongly diagonal quads those regions overlap a lot, so if they would add up to more than twice the source, it reads the source once instead.

The hard parts turned out to be elsewhere: deciding where the corners are, and deciding what shape the result should be. LensUp's perspective page used to detect page edges automatically; we removed that, because on real photos (a white form on a white desk, a card on a larger sheet) it would sometimes lock onto the wrong rectangle and crop straight through content. Now you drag four handles to the corners. You can try it on LensUp's perspective correction page; the warp runs in the browser and the photo isn't uploaded.

License

MIT