Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,12 +131,23 @@ Credit purchases are created through `/api/credits/create-invoice`; successful w

### Free audit

The homepage form queues a free rule-based audit. Email is optional: users get the on-page report immediately via `/r/<share_token>`, and receive a PDF only when they provide an email. Anonymous free scans are unlisted by default; `/recent` and sitemap deep links include only scans where the submitter explicitly opted into public listing. Common tracking parameters such as `utm_*`, `fbclid`, and `gclid` are stripped before new URLs are saved.
The homepage form queues a free scan — either the rule-based **AEO audit** or the **Slop Score** (see below). Email is optional: users get the on-page report immediately via `/r/<share_token>`, and receive a PDF only when they provide an email. Anonymous free scans are unlisted by default; `/recent` and sitemap deep links include only scans where the submitter explicitly opted into public listing. Common tracking parameters such as `utm_*`, `fbclid`, and `gclid` are stripped before new URLs are saved.

### Project scan

Signed-in users can save sites as projects, choose one or more engines, and run scans from the project page. A multi-engine scan creates one `audits` row per engine and ties them together with `scan_run_id` for side-by-side reports, consolidated Markdown/PDF, and project score history.

### Slop Score

`lib/audit/slop-engine.ts` sweeps up to 50 same-origin pages (sitemap.xml first, then breadth-first from the entry page) plus a handful of stylesheets, and scores how careless the site looks: **0 is pristine, 100 is maximum slop**. Analyzers live in `lib/audit/checks/slop.ts` across three dimensions — **content** (filler phrasing, no first-party evidence, thin/near-duplicate/boilerplate pages, placeholders, stale copyright, high-confidence misspellings), **code** (leaked template variables, dev/staging hosts, `console.*` and TODO leftovers, duplicate metadata, dead links, deprecated tags) and **design** (missing viewport, unsized images, placeholder alt text, stock-only imagery, inline-style density, palette/typography/`!important` sprawl). Output is the headline score plus a per-page fix list and systemic rollups for defects that live in a shared template.

Two design rules matter when extending it:

- **It reports observable defects, never "this was written by AI."** An AI-probability score is unfalsifiable, the classifiers are unreliable, and it would accuse paying customers. Every finding must be something the owner can verify in ten seconds and fix.
- **Ambiguous markers only count in unambiguous positions.** Dogfooding on our own blog showed `coming soon`, `your brand name`, `[product]` and `[your site]` all appear in legitimate prose, and a `.netlify.app` hostname appears as scan-result *text* on `/recent` — so standalone-only matching and attribute-only host scanning are load-bearing, not stylistic. `tests/slop.test.ts` guards each case.

It is free (`cost: 0`), runs no LLM, and is therefore immune to the shared-provider-quota outages that stall Autoblog.

### Credits and engines

Rule-based scans cost 0 credits. Paid AI-model scans cost 1 credit per engine. Credit packs are defined in `lib/credits.ts`; failures and user-aborted paid scans refund credits.
Expand Down
15 changes: 13 additions & 2 deletions app/actions/runAudit.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,21 @@ async function notifyWorker(auditId: string, pdfEmail?: string) {
}
}

const ALL_ENGINES: Engine[] = ["rule", "spec", "dns", "links", "vu1nz", "claude", "openai", "gemini", "qwen", "kimi", "deepseek", "zai", "perplexity", "fugu"];
const ALL_ENGINES: Engine[] = ["rule", "spec", "dns", "links", "slop", "vu1nz", "claude", "openai", "gemini", "qwen", "kimi", "deepseek", "zai", "perplexity", "fugu"];

// Free engines an anonymous visitor may pick from the hero form. Both are
// deterministic, self-hosted, and page-budgeted, so opening them up costs us
// nothing per run and can't be turned into an LLM-spend or partner-API faucet.
// The per-IP anonymous daily limit still applies to both.
const ANON_ENGINES: Engine[] = ["rule", "slop"];

function normalizeEngines(input: unknown, signedIn: boolean): Engine[] {
if (!signedIn) return ["rule"];
if (!signedIn) {
const picked = Array.isArray(input)
? input.find((e): e is Engine => typeof e === "string" && (ANON_ENGINES as string[]).includes(e))
: undefined;
return [picked ?? "rule"];
}
if (!Array.isArray(input) || input.length === 0) return DEFAULT_PROJECT_ENGINES;
const cleaned = dedupeEngines(
input.filter((e): e is Engine =>
Expand Down
1 change: 1 addition & 0 deletions components/charts/engine-trend.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ export type EngineTrendPoint = {

const ENGINE_COLORS: Record<Engine, string> = {
rule: "#9aa3b2",
slop: "#d946ef",
spec: "#64748b",
dns: "#84cc16",
links: "#14b8a6",
Expand Down
2 changes: 1 addition & 1 deletion components/engines-panel.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import { DEFAULT_PROJECT_ENGINES, ENGINES, selectionCost, type Engine } from "@/
import { runScanForProject } from "@/app/actions/runAudit";
import { updateProjectEngines } from "@/app/actions/projects";

const ENGINE_ORDER: Engine[] = ["rule", "spec", "dns", "links", "vu1nz", "claude", "openai", "gemini", "perplexity", "qwen", "kimi", "deepseek", "zai", "fugu"];
const ENGINE_ORDER: Engine[] = ["rule", "slop", "spec", "dns", "links", "vu1nz", "claude", "openai", "gemini", "perplexity", "qwen", "kimi", "deepseek", "zai", "fugu"];

export function EnginesPanel({
projectId,
Expand Down
51 changes: 48 additions & 3 deletions components/hero-audit-form.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,28 @@ declare global {
}
}

export function HeroAuditForm() {
type HeroScan = "rule" | "slop";

// Only the two free, self-hosted engines are offered anonymously — kept in
// sync with ANON_ENGINES in app/actions/runAudit.ts.
const SCAN_OPTIONS: Array<{ id: HeroScan; label: string; blurb: string }> = [
{
id: "rule",
label: "AEO audit",
blurb: "How AI crawlers and answer engines see your site, with fixes.",
},
{
id: "slop",
label: "Slop Score",
blurb: "Sweeps up to 50 pages for careless content, code, and design.",
},
];

export function HeroAuditForm({ defaultScan = "rule" }: { defaultScan?: HeroScan }) {
const router = useRouter();
const [pending, startTransition] = useTransition();
const [url, setUrl] = useState("");
const [scan, setScan] = useState<HeroScan>(defaultScan);
const [listPublic, setListPublic] = useState(false);
const [error, setError] = useState<string | null>(null);
const [urlError, setUrlError] = useState<string | null>(null);
Expand All @@ -41,7 +59,7 @@ export function HeroAuditForm() {
// URL-first: the report is generated and shown on-page for free.
// Email / phone / monthly-sales are collected AFTER the report
// (on /r/<token>) only if the user wants the PDF emailed.
const res = await startAuditFromForm({ url, listPublic });
const res = await startAuditFromForm({ url, listPublic, engines: [scan] });
if (!res.ok) {
setError(res.error ?? "Could not start audit.");
// Slugify the error message so we don't blow past Datafa.st's
Expand All @@ -56,13 +74,40 @@ export function HeroAuditForm() {
}
window.datafast?.("audit_submitted", {
listed_public: listPublic ? "yes" : "no",
scan,
});
router.push(`/r/${res.token}`);
});
}

return (
<form onSubmit={submit} noValidate className="card flex flex-col gap-3 p-4 text-left">
<div>
<span className="mb-1 block text-xs font-semibold uppercase tracking-wider text-[var(--color-muted)]">
Scan type
</span>
<div className="flex flex-col gap-2 sm:flex-row" role="radiogroup" aria-label="Scan type">
{SCAN_OPTIONS.map((opt) => (
<button
key={opt.id}
type="button"
role="radio"
aria-checked={scan === opt.id}
onClick={() => setScan(opt.id)}
className="flex-1 rounded-lg border p-3 text-left transition-colors"
style={{
borderColor: scan === opt.id ? "var(--color-accent)" : "var(--color-border)",
background: scan === opt.id ? "color-mix(in oklab, var(--color-accent) 8%, transparent)" : "transparent",
}}
>
<span className="block text-sm font-semibold">{opt.label}</span>
<span className="mt-0.5 block text-xs leading-snug text-[var(--color-muted)]">
{opt.blurb}
</span>
</button>
))}
</div>
</div>
<Field label="Website URL" error={urlError}>
<div className="flex flex-col gap-2 sm:flex-row">
<input
Expand All @@ -83,7 +128,7 @@ export function HeroAuditForm() {
className="btn btn-primary whitespace-nowrap"
disabled={pending}
>
{pending ? "Starting…" : "Run free audit"}
{pending ? "Starting…" : scan === "slop" ? "Get free Slop Score" : "Run free audit"}
</button>
</div>
</Field>
Expand Down
7 changes: 6 additions & 1 deletion components/report/report-view.tsx
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
import { SECTIONS } from "@/lib/audit/prompt";
import { SectionFindings, type FixRun } from "./section";
import { DataFoundTable } from "./data-found";
import { SlopMeter } from "./slop-meter";
import { SLOP_SECTION } from "@/lib/audit/checks/slop";
import type { Finding } from "@/lib/audit/types";
import { ENGINES, type Engine } from "@/lib/credits";

Expand Down Expand Up @@ -98,6 +100,7 @@ export function ReportView({
}
return (
<SectionShell key={s} index={i + 1} title={s}>
{s === SLOP_SECTION && <SlopMeter findings={sectionFindings} />}
<SectionFindings
findings={sectionFindings}
fixContext={fixContext}
Expand Down Expand Up @@ -278,6 +281,7 @@ function EngineReport({
}
return (
<SectionShell key={`${audit.id}-${s}`} index={i + 1} title={s}>
{s === SLOP_SECTION && <SlopMeter findings={sectionFindings} />}
<SectionFindings
findings={sectionFindings}
fixContext={fixContext}
Expand All @@ -294,7 +298,8 @@ export function reportSections(findings: Finding[], engine?: string | null): str
const findingSections = Array.from(
new Set(findings.map((f) => f.section).filter(Boolean)),
);
const compactEngine = engine === "dns" || engine === "links" || engine === "spec";
const compactEngine =
engine === "dns" || engine === "links" || engine === "spec" || engine === "slop";
if (compactEngine) {
return findingSections.length > 0 ? findingSections : [...SECTIONS];
}
Expand Down
120 changes: 120 additions & 0 deletions components/report/slop-meter.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
import type { Finding } from "@/lib/audit/types";

// The shareable headline of a Slop Score scan.
//
// Form choice: the score is a single number, so it's a hero stat — not a chart.
// The per-dimension breakdown is three magnitudes of the SAME measure (slop
// points), so it gets one neutral hue with identity carried by the text labels;
// the status tokens stay reserved for the score's own state, paired with the
// grade word so state is never conveyed by color alone.

type SlopEvidence = {
score?: number;
grade?: string;
byDimension?: { content?: number; code?: number; design?: number };
totals?: { pages?: number; issues?: number; words?: number };
};

const DIMENSIONS = [
{ key: "content", label: "Content", hint: "filler, thin pages, duplicates, stale dates" },
{ key: "code", label: "Code", hint: "leaked template vars, dev URLs, dead links" },
{ key: "design", label: "Design", hint: "viewport, alt text, layout shift, style drift" },
] as const;

export function SlopMeter({ findings }: { findings: Finding[] }) {
const headline = findings.find((f) => f.check_key === "slop.score");
const ev = (headline?.evidence ?? {}) as SlopEvidence;
const score = typeof ev.score === "number" ? ev.score : null;
if (score === null) return null;

const grade = ev.grade ?? "";
const dims = ev.byDimension ?? {};
const totals = ev.totals ?? {};
// Lower is better here, which inverts the usual AEO dial.
const tone =
score <= 25 ? "var(--color-pass)" : score <= 50 ? "var(--color-warn)" : "var(--color-fail)";
const max = Math.max(1, ...DIMENSIONS.map((d) => dims[d.key] ?? 0));

return (
<div className="card space-y-5 p-4 sm:p-6">
<div className="flex flex-col gap-5 sm:flex-row sm:items-center sm:gap-8">
<div className="sm:flex-shrink-0">
<p className="text-xs uppercase tracking-wider text-[var(--color-muted)]">Slop Score</p>
<p className="mt-1 flex items-baseline gap-1.5">
<span className="text-5xl font-extrabold leading-none" style={{ color: tone }}>
{score}
</span>
<span className="text-sm text-[var(--color-muted)]">/ 100</span>
</p>
<p className="mt-1.5 text-sm font-semibold" style={{ color: tone }}>
{grade}
</p>
</div>

<div className="min-w-0 flex-1 space-y-3">
{/* Track runs low→high slop so the filled portion reads as "how much
slop", matching the number above it. */}
<div
className="h-2 w-full overflow-hidden rounded-full"
style={{ background: "var(--color-border)" }}
role="img"
aria-label={`Slop score ${score} out of 100 — ${grade}`}
>
<div
className="h-full rounded-full"
style={{ width: `${Math.max(2, score)}%`, background: tone }}
/>
</div>
<div className="flex justify-between text-[10px] uppercase tracking-wider text-[var(--color-muted)]">
<span>0 — pristine</span>
<span>100 — maximum slop</span>
</div>
<p className="text-sm text-[var(--color-muted)]">
{[
typeof totals.pages === "number"
? `${totals.pages} page${totals.pages === 1 ? "" : "s"} swept`
: null,
typeof totals.words === "number" ? `${totals.words.toLocaleString()} words` : null,
typeof totals.issues === "number"
? `${totals.issues} issue${totals.issues === 1 ? "" : "s"}`
: null,
]
.filter(Boolean)
.join(" · ")}
</p>
</div>
</div>

<div className="space-y-2.5 border-t border-[var(--color-border)] pt-4">
<p className="text-xs uppercase tracking-wider text-[var(--color-muted)]">
Slop points by dimension
</p>
{DIMENSIONS.map((d) => {
const v = dims[d.key] ?? 0;
return (
<div key={d.key} className="grid grid-cols-[5rem_1fr_2.5rem] items-center gap-3 text-sm">
<span className="font-medium">{d.label}</span>
<span className="h-1.5 w-full rounded-full" style={{ background: "var(--color-border)" }}>
<span
className="block h-full rounded-full"
style={{
width: `${v === 0 ? 0 : Math.max(3, (v / max) * 100)}%`,
background: "var(--color-accent)",
}}
/>
</span>
<span className="text-right tabular-nums text-[var(--color-muted)]">{v}</span>
<span className="col-span-3 -mt-1.5 text-xs text-[var(--color-muted)]">{d.hint}</span>
</div>
);
})}
</div>

<p className="border-t border-[var(--color-border)] pt-4 text-xs text-[var(--color-muted)]">
This measures <strong>observable defects</strong> — placeholder copy, near-duplicate pages,
leaked template variables, missing first-party evidence, stale dates, design drift. It does
not estimate whether anything was written by AI.
</p>
</div>
);
}
Loading
Loading