Skip to content

Refactor/canvas - #45

Open
cherriae wants to merge 24 commits into
mainfrom
refactor/canvas
Open

cherriae wants to merge 24 commits into
mainfrom
refactor/canvas

Conversation

@cherriae

Copy link
Copy Markdown
Member

Summary

Checklist

  • If code changes were made then they have been tested.
    • I have updated the documentation to reflect the changes.
  • This PR fixes an issue.
  • This PR adds something new (e.g. subsystem).
  • This PR is not a code change (e.g. README, typehinting, examples, refactoring, ...)

cherriae and others added 24 commits October 18, 2026 10:51
app/scout/routes.py (1451 lines) becomes a package on the same
"scouting" blueprint: records, pit, lighthouse, leaderboards, events,
plus common (blueprint init and shared helpers). URLs, endpoint names
and methods are unchanged.

- SCOUTER_LOOKUP / team_access_match replace eight copies of the
  scouter join and team access filter
- actor_name replaces the repeated username-or-Anonymous expression
- fuel_total / _alliance_fuel replace hand-written fuel sums
- drop unused tba global, format_team_stats, commented check_team

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All data access already went through the MongoDB singleton via
DatabaseManager; remove what bypassed or duplicated it:

- drop flask_pymongo (two unused PyMongo clients/pools in app.py and
  the auth blueprint) and its requirement
- drop legacy get_database_connection / release_connection /
  force_close_connection / mark_db_accessed and the unused db_accessed
  flag; DBManager alias becomes the DatabaseManager class
- drop app.db_managers and app.user_manager (written, never read)
- remove the create_app collection bootstrap; each manager already
  ensures its own collections (TeamManager now owns "assignments").
  Fresh databases now get the team_data/pit_scouting indexes the
  bootstrap used to pre-empt
- remove teardown hook that closed the global connection when an
  exception escaped the error handlers
- auth: remove dead save/send_profile_picture and local copies of
  allowed_file / async_route; the stricter is_safe_url moves to utils
  (auth was its only caller) with a test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The auth, team and notification managers declared async methods but
only ran blocking pymongo calls, and every route paid an asyncio.run
per request to drive them. Everything is now plain synchronous code:

- async def/await removed from managers and routes; async_route and
  the flask[async] extra are gone
- with_mongodb_retry now actually retries: wrapping a coroutine
  function returned the coroutine immediately, so ConnectionFailure /
  ServerSelectionTimeoutError were never caught (27 methods)
- /api/search uses requests (10s timeout) instead of aiohttp, which
  is dropped from requirements
- remove the fire-and-forget instant assignment notification in
  create_or_update_assignment: it called attributes that do not exist
  (self.mongo_uri, self.get_connection, existing_connection=) and the
  task was cancelled when asyncio.run returned, so it never sent
  anything. A working version is tracked as a separate fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Route modules no longer build MongoDB pipelines; ScoutingManager owns
every query and exposes them by meaning (get_team_leaderboard,
get_compare_stats, get_team_records, get_auto_paths,
get_match_alliances, get_visible_fuel, get_scouter_leaderboard, ...).

- one access_match/_visible_records replaces the per-method copies of
  the scouter join and team visibility filter
- _record_fields is shared by add_scouting_data and update_team_data
  (update still leaves auto_climb untouched, as before)
- PIT_PROJECTION shared by the pit readers
- drop unused has_team_data, get_team_stats, get_team_matches,
  get_auto_paths(team_number), get_pit_scouting, commented-out code,
  and a debug count_documents that ran on every pit list load

Pipelines sent by every scouting route were captured before and after
this change and are identical apart from $or clause order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add and edit each parsed the pit form by hand and had drifted: edit
used int() directly, so a blank driver_years or auto_routes raised and
the edit failed, and its error log referenced `data` before it was
assigned. _pit_form() now builds the shared fields with add's lenient
parsing (blank or missing numbers are 0); add/edit only add their own
metadata. Behaviour change: blank numeric fields on edit now save as 0
instead of failing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- _with_object_ids replaces the per-class ObjectId coercion; User,
  Team and AssignmentSubscription.create_from_db become one-line
  classmethods
- User no longer overrides UserMixin's is_authenticated/is_active/
  is_anonymous properties with methods (a bound method is always
  truthy, so is_anonymous was "true" for every user); callers only
  ever tested truthiness of the first two
- remove unused PitScouting class, TeamData.formatted_date (read an
  attribute that was never set), User.update_team_number, and
  Team.add_user/remove_user

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Delete 25 "# @limiter.limit(...)" lines rather than enable them: the
limiter keys on client IP, and scouters at an event share the venue's
address, so per-IP limits sized for one person would throttle a whole
team. Also remove the commented-out /notifications/test route and
NotificationManager.test_notification.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- base.html's 278-line service worker/offline script moves to
  static/js/base.js; settings.html's 355-line script moves to
  static/js/auth/settings.js. The two template values they used are
  passed as data- attributes read via document.currentScript (which
  also stops HTML-escaping the username inside a JS string)
- add.js and edit.js shared ~430 identical lines of form, canvas,
  toolbar and keyboard-shortcut setup; that is now
  initScoutFormCanvas() in static/js/scout/form-canvas.js, with the
  two real differences as options (edit loads the saved path; add's
  Home button also resizes). Inlining the function reproduces both
  original files line for line.
- remove pit-scouting/edit.js, which no template loads
- service worker precaches base.js and form-canvas.js (the add page
  works offline); cache names bumped to v11 so clients refetch
- offline.html keeps its inline script so it stays self-contained

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Security fixes (behaviour changes):

- CSRFProtect is enabled. Every POST form carries a csrf_token field,
  and a fetch wrapper at the top of base.html adds X-CSRFToken to
  same-origin writes. Tokens live for the session (no one-hour limit)
  because scouting pages stay open all event and queue offline.
  update_assignment_status is exempt: the service worker's "mark
  complete" action cannot read the token, and the route only accepts
  JSON, which browsers will not send cross-site without a preflight.
  Rejected requests get "session expired" (JSON 400 or flash+redirect).
- SECRET_KEY no longer falls back to the public "team334"; the app
  refuses to start without it, since anyone knowing the key can forge
  a session for any user.
- Session and remember-me cookies are SameSite=Lax.
- Flask-CORS with origins "*" is removed; nothing calls the app
  cross-origin (the standalone app is a WebView on the site itself).
- Record delete, pit delete and team leave are POST-only; deletes were
  GET links that any cross-site link could trigger.
- The catch-all exception handler now passes HTTP errors (400/405/415)
  through instead of turning them into a 500 page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… check

- The notifications blueprint registered teardown_appcontext to stop
  the worker, which runs after every request, so reminders stopped
  after the first page load. Stop it at process exit instead.
- Creating an assignment never pushed to assignees (the old
  fire-and-forget task called attributes that did not exist and was
  cancelled immediately). create_or_update_assignment now returns the
  new assignment and the route hands it to
  NotificationManager.notify_assignment_created, which sends on a
  background thread so the request does not wait on push services.
- /auth/check_username raised UnboundLocalError (500) on a non-JSON
  body; it now treats it as an empty username.
- /leaderboard gets @login_required like every other scouting page
  (the app-wide before_request already redirected anonymous users).

Tests share an app harness (tests/app_harness.py) that builds the real
app with MongoDB mocked; the new regression tests fail on the old code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test harness built the app with .env loaded, and Flask-Limiter
reads RATELIMIT_ENABLED only at init_app, so rate-limit counters went
to whatever MONGO_URI pointed at. The harness now skips .env, points
MONGO_URI at an unresolvable host, uses memory:// rate-limit storage,
and fails on any real MongoClient. RATELIMIT_STORAGE_URI can be set
separately from MONGO_URI (it still defaults to it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seven read routes logged their entire result on every request (the
full leaderboard, every match, every pit record, every compared team's
raw match documents), and write routes logged each submitted form
twice. That cost string formatting and disk on the busiest pages and
grew log files quickly during events.

Log lines now carry counts and identity instead: team/event/match for
records, entry ids, team numbers, result counts. Also stop logging
push subscription JSON (it contains the browser's encryption keys) and
team join codes. test_route_logging drives each changed route through
its success path and asserts the summary is logged and no payload
content is.
bindEvents() was a 740-line function holding every handler inline, and
mouse and touch each had their own ~300-line copy of the tool logic
(select, resize handles, move, pen, shapes). The copies had drifted, so
fixes reached desktop but not phones.

The tool logic now lives once in pointerDown/pointerMove/pointerUp
(with selectAt, finishSelection, finishStroke); the mouse and touch
handlers only translate events into a position, plus their own panning
(shift/middle drag; two-finger pan and pinch). bindEvents() just wires
handlers, and keyboard shortcuts are onKeyDown().

Touch now behaves like mouse, which fixes three touch bugs:
- dragging a resize handle jumped the shape to nearly twice its size
  on the first move (the drag start was never recorded for touch)
- drawing after Undo kept the redo history, so Redo replayed the
  undone stroke on top of the new one
- a selection box dragged over empty space stayed on screen

Smaller changes:
- a stroke finished by the mouse leaving the canvas also clears redo
- the cursor stays "grabbing" while panning with the select tool
- drop the per-point pressure value and its velocity tracking, and the
  unused lastEvent, streamline, minWidth/maxWidth fields: rendering
  computes pressure itself and never read them (old saves still load)

tests/test_canvas.js drives Canvas.js with mouse and touch events in a
fake DOM (drawing, shapes, select, move, resize, undo/redo, pan, pinch,
wheel zoom, shortcuts, save/load, flip/rotate, readonly); it runs under
npm test. Against the old file only the four fixed behaviours fail.
Randomised sessions replayed through old and new code matched step for
step: mouse (old code with the two mouse fixes applied), single-finger
touch against the old mouse path, and two-finger pan/zoom.

Service worker cache bumped to v12 so clients fetch the new file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Escape while dragging a selection box cleared the box but left the
  board "selecting", so every later mouse move and release threw a
  TypeError until the page was reloaded. Escape now ends the box.
- form-canvas.js called CanvasField.startPanning/stopPanning on
  middle-click; Canvas has no such methods, so every middle-click on
  the add/edit pages threw. Canvas already pans on the middle button;
  the page now only stops the browser's autoscroll.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shared mouse handler converted every mousemove to canvas
coordinates (a getBoundingClientRect call) even when hovering with the
pen or a shape tool, where the old handler did nothing. Return early
when no gesture is active and the select tool is not showing resize
cursors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every pointer move during a drag repainted the whole board: background,
field image and every stroke, each with its curve recomputed. The cost
grew with the amount of ink, so fast drawing on a busy board stuttered
(about 7 ms per move with 200 strokes in Edge on a desktop, several
times that on phones).

During a drag only the selection, the stroke being drawn and shape
previews change. redrawCanvas() is split into drawScene() (background,
field, unselected strokes) and drawOverlay(); pointer moves call
redrawGesture(), which renders the scene once per gesture into an
offscreen canvas and blits it on later moves. Every full redrawCanvas()
(undo, pan, zoom, flip, load, any caller outside the drag) and every
new gesture discards the copy.

The scene is drawn into the offscreen canvas rather than copied from
the board: the board's desynchronized context is read when the copy is
flushed, which already included the overlay. Line caps and joins are
carried between the two contexts because shapes inherit them.

Per-move cost in Edge is now 0.1-0.5 ms regardless of board size (was
0.4 ms with 8 strokes, 2.5 ms with 60, 7 ms with 200). Canvas pixels
were compared after every input event between the previous and new
code in Edge for pen, shape previews on a shapes-only board, box
select + move + resize, undo mid-drag, flipped/rotated view, and a
touch stroke with a pinch in the middle: identical in all 278 frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Static files are served cache-first, so a browser that already installed
the v12 worker would keep the previous Canvas.js.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bindEvents() was a 740-line function holding every handler inline, and
mouse and touch each had their own ~300-line copy of the tool logic
(select, resize handles, move, pen, shapes). The copies had drifted, so
fixes reached desktop but not phones.

The tool logic now lives once in pointerDown/pointerMove/pointerUp
(with selectAt, finishSelection, finishStroke); the mouse and touch
handlers only translate events into a position, plus their own panning
(shift/middle drag; two-finger pan and pinch). bindEvents() just wires
handlers, and keyboard shortcuts are onKeyDown().

Touch now behaves like mouse, which fixes three touch bugs:
- dragging a resize handle jumped the shape to nearly twice its size
  on the first move (the drag start was never recorded for touch)
- drawing after Undo kept the redo history, so Redo replayed the
  undone stroke on top of the new one
- a selection box dragged over empty space stayed on screen

Smaller changes:
- a stroke finished by the mouse leaving the canvas also clears redo
- the cursor stays "grabbing" while panning with the select tool
- drop the per-point pressure value and its velocity tracking, and the
  unused lastEvent, streamline, minWidth/maxWidth fields: rendering
  computes pressure itself and never read them (old saves still load)

tests/test_canvas.js drives Canvas.js with mouse and touch events in a
fake DOM (drawing, shapes, select, move, resize, undo/redo, pan, pinch,
wheel zoom, shortcuts, save/load, flip/rotate, readonly); it runs under
npm test. Against the old file only the four fixed behaviours fail.
Randomised sessions replayed through old and new code matched step for
step: mouse (old code with the two mouse fixes applied), single-finger
touch against the old mouse path, and two-finger pan/zoom.

Service worker cache bumped to v12 so clients fetch the new file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Escape while dragging a selection box cleared the box but left the
  board "selecting", so every later mouse move and release threw a
  TypeError until the page was reloaded. Escape now ends the box.
- form-canvas.js called CanvasField.startPanning/stopPanning on
  middle-click; Canvas has no such methods, so every middle-click on
  the add/edit pages threw. Canvas already pans on the middle button;
  the page now only stops the browser's autoscroll.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shared mouse handler converted every mousemove to canvas
coordinates (a getBoundingClientRect call) even when hovering with the
pen or a shape tool, where the old handler did nothing. Return early
when no gesture is active and the select tool is not showing resize
cursors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every pointer move during a drag repainted the whole board: background,
field image and every stroke, each with its curve recomputed. The cost
grew with the amount of ink, so fast drawing on a busy board stuttered
(about 7 ms per move with 200 strokes in Edge on a desktop, several
times that on phones).

During a drag only the selection, the stroke being drawn and shape
previews change. redrawCanvas() is split into drawScene() (background,
field, unselected strokes) and drawOverlay(); pointer moves call
redrawGesture(), which renders the scene once per gesture into an
offscreen canvas and blits it on later moves. Every full redrawCanvas()
(undo, pan, zoom, flip, load, any caller outside the drag) and every
new gesture discards the copy.

The scene is drawn into the offscreen canvas rather than copied from
the board: the board's desynchronized context is read when the copy is
flushed, which already included the overlay. Line caps and joins are
carried between the two contexts because shapes inherit them.

Per-move cost in Edge is now 0.1-0.5 ms regardless of board size (was
0.4 ms with 8 strokes, 2.5 ms with 60, 7 ms with 200). Canvas pixels
were compared after every input event between the previous and new
code in Edge for pen, shape previews on a shapes-only board, box
select + move + resize, undo mid-drag, flipped/rotated view, and a
touch stroke with a pinch in the middle: identical in all 278 frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Static files are served cache-first, so a browser that already installed
the v12 worker would keep the previous Canvas.js.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merging the pre-rebase refactor/canvas history kept both sides of
app/app.py, repeating imports, the SECRET_KEY check, cookie settings,
csrf.init_app and the exception handler. The repeated SECRET_KEY
keyword in app.config.update() is a SyntaxError, so the app could not
start. This branch never changes app.py, so take main's version and
drop the urlparse and is_safe_url imports the CodeQL redirect fix
left unused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant