Dashboard - #241
Open
SimonSadler wants to merge 20 commits into
Open
Dashboard#241SimonSadler wants to merge 20 commits into
SimonSadler wants to merge 20 commits into
Conversation
SimonSadler
force-pushed
the
dashboard
branch
from
September 23, 2026 13:47
7cc84e9 to
27211ca
Compare
The beginnings of a web-based dashboard for viewing and comparing cyclone trajectories. The datasette-geojson-map plugin has specific requirements for the GeoJSON geometry column; the "geometry" field of a GeoJSON structure must be the root. A tweak has been made to the construction of the track GeoJSON in order to support this. The change is temporary; the plugin will be replaced or improved in later versions. Tests have not been updated so some will fail.
To replace the existing Leaflet map. MapLibre is more modern, fully featured and faster than Leaflet. Extensive tests show it can easily handle 1,00,000 points and tracks. The layer system also allows simpler overlays. MapLibre also correctly renders 0–360 longitudes so there is no longer any need to wrap longitude values to the -180–180 range. This solves the problem where lines cross the antimeridian.
Add a [dashboard] extra to the main pyproject file for installing Datasette and the custom plugin.
Loading GeoJSON into the map was slow on very large datasets (around 2s). Reducing data size and side-stepping some data conversion on the server-side has reduced this to 0.5s. There are other options for optimisation that can be taken based on eventual usage. Two of these are: storing GeoJSON in one blob rather than row by row, and using a more optimal format such as MapLibre Vector Tiles.
Tracks and points are added as map layers via a magic `layer_` column in the database view. Each layer is coloured accessibly for visual distinction.
Also removes the (undesired) background layer without further configuration.
Previously GeoJSON was built and stored in the database at the time of creation. This big shift generates the required GeoJSON at map rendering time. Performance is very similar to the previous approach as that also required some data wrangling to merge it into one dataset. Dynamic generation gives the following benefits: - Flexibility of structuring GeoJSON data in any way required. - Property names come from the query, not baked in or abbreviated. - Layering does not require the slight hack of hooking up the column name to a fixed property name. - Filtering is more flexible and powerful because views are now based on observation points rather than tracks. - The database is much smaller without the GeoJSON data (which was the same data as stored in the tables, just in text format). - No need to retrieve or hide GeoJSON columns from the table view. This initial change is for observation points only.
Track lines are formed from observations by grouping on the trajectory_id. This is done via a generalised group_by setting and system that allows for future expansion.
Pre-built GeoJSON is no longer required in the database now that it is generated just-in-time on the client-side. The benefits of dynamic creation (repeated from a previous commit): - Flexibility of structuring GeoJSON data in any way required. - Property names come from the query, not baked in or abbreviated. - Layering does not require the slight hack of hooking up the column name to a fixed property name. - Filtering is more flexible and powerful because views are now based on observation points rather than tracks. - The database is much smaller without the GeoJSON data (which was the same data as stored in the tables, just in text format). - No need to retrieve or hide GeoJSON columns from the table view.
- Reduce map height so query controls are more present. - Disable map scroll-zoom so page is easier to scroll. - Disable drag-rotate to avoid confusion. - Add compass control. - Only add a line layer if lines were created. - Change pointer when hovering over points and lines.
Colours can now be specified for single and multiple layers. Style improvements for layer control and facets. The best map for dark mode so far is the MapTiler `dataviz-v4-dark`. An API key is required so this option is not included in the config.
SimonSadler
force-pushed
the
dashboard
branch
from
September 23, 2026 20:52
27211ca to
168ef0d
Compare
Datasette facets added for swifter searching.
Point radius can now be linked to property value.
Opacity can now be linked to a property value. This allows for point and line visual variation without adding multiple layers. Layers can now be used solely for comparing distinct outputs/collections. The palette configuration has been adapted for distinction in readiness for this. The layer control opacity slider doesn't support opacity expressions so it has been disabled to accommodate this.
Datasette v1.0 is still in alpha, but it's very mature at this point and contains fixes that are beneficial (fixed position of facets being one). I've seen no issues and the changes were minimal.
SimonSadler
marked this pull request as ready for review
October 2, 2026 07:54
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A web-based dashboard for viewing and comparing cyclone trajectories. Tracks and observations are viewed on a map.
See dashboard/README.md for full instructions.
In short, once you have your
venvsetup:I've sent a prebuilt database file via Slack that can get you started quickly.
Copy the file to the root of your checkout and then run this command (this will be simplified to one statement in later versions):
This takes you to the Datasette homepage, where you can select your database and explore tables and views. The views have the most useful data structures and also trigger the map view.
These direct links will take you to the data views:
http://127.0.0.1:8001/6hr_track_mm_oceans_1980on/points
http://127.0.0.1:8001/6hr_track_mm_oceans_1980on/tracks_layer_year
Note that the ocean and landfall detection are in a separate branch, so the filters are not as useful as the version I've been showing recently. The database I've attached does have the ocean and landfall data within so if you link the metadata to your database as described in the README, you can then add the
ocean_idandlandfallcolumns to thefacetskey, under thepointsview, for example.The line count is mostly due to the two style sheets. Changes to existing code are fairly minimal.