Tree Canopy Detection

My experiments for Solafune's Tree Canopy Detection competition (August 2025 – January 2026): instance segmentation of individual trees and groups of trees in RGB aerial and satellite imagery.

This repo is a record of exploration rather than a polished solution. Over the course of the competition I tried seven setups across MMSegmentation, MMDetection, Hugging Face Transformers, Ultralytics and plain PyTorch. None of them got close to the top of the leaderboard. The write-up below is mostly about what went wrong and why.

The challenge

Each evaluation image is a 1024 × 1024 RGB tile at one of five ground resolutions (10, 20, 40, 60 or 80 cm) and one of five scene types (agriculture/plantation, urban, rural, industrial, open field). The imagery comes from several public sources (SWISSIMAGE, NAIP, SkySat, OAM-TCD and others). Every canopy has to be predicted as a polygon with one of two classes:

• individual_tree
• group_of_trees

Scoring is mAP at IoU 0.75, computed per image and then averaged with weights. The weights favour harder resolutions (80 cm counts 3× as much as 10 cm) and the scenes the organisers care about (agriculture/plantation 2×, urban 1.5×). Full details are on the competition page.

Results

Approach	Weighted mAP@0.75
YOLO26x-seg (06)	0.233
Mask2Former, Swin-S via Hugging Face (04)	0.158
DINOv2 + Mask2Former decoder (07)	0.000

My best result came from the quickest baseline, put together in a day while I was debugging the custom DINO models. The model I spent the most effort on scored zero.

What I tried

The folders in approaches/ are numbered in the order I worked on them. Each one is self-contained, with its own preprocessing, training and prediction scripts.

01 · Semantic segmentation, Mask2Former Swin-L on MMSegmentation (Aug 2025) My first submission treated the task as semantic segmentation with three classes (background, individual tree, group of trees) on 512 × 512 crops. Predicted masks were split into instances with cv2.findContours. This had two flaws. Touching crowns merge into a single blob, and every polygon got a hardcoded confidence of 1.0. Both are fatal for an instance AP metric, which needs both separate objects and a meaningful ranking.

02 · Instance segmentation, Mask2Former Swin-S on MMDetection (Aug – Dec 2025) Two days later I switched to proper instance segmentation. This setup has:

• Solafune polygons converted to COCO annotations, with a train/val split stratified by scene type
• an offline augmentation tool built on Albumentations (flips, rotations, affine transforms, colour shifts, blur, noise) that keeps masks and boxes aligned
• online random resize (0.5–2×) plus 1024 crops
• 40k iterations with AMP, keeping the best checkpoint by segm_mAP_75
• inference with a 0.3 score threshold, morphological mask cleanup and polygon conversion

Most of the time here went into augmentation bugs, a Hungarian-matcher error and getting MMCV's CUDA ops to build. The Dockerfile pins a combination that works.

03 · HTC, Swin-L on MMDetection (Nov – Dec 2025) This was an attempt to move to a heavier two-stage model: Hybrid Task Cascade without the semantic branch, with a Swin-L backbone pretrained on ImageNet-22k. It never got past MMDetection 3.x config inheritance and MMCV's nms_impl ... not found errors (see fix_mmcv.sh), and I didn't make a submission with it.

04 · Mask2Former via Hugging Face Transformers (Dec 2025) This approach moves off the OpenMMLab stack. It fine-tunes facebook/mask2former-swin-small-coco-instance with the HF Trainer. To get around a resize bug in the image processor, images are resized to 512 × 512 by hand, which throws away half the resolution on the 10 cm tiles.

05 · DINOv3-SAT backbone + custom Mask2Former decoder (Dec 2025 – Jan 2026) This was a from-scratch PyTorch model:

• Backbone: the satellite-pretrained DINOv3 ViT-L/16 (dinov3-vitl16-pretrain-sat493m), frozen, with features from layers 6, 12, 18 and 24 projected into a four-level pyramid.
• Head: the Mask2Former transformer decoder, Hungarian matcher and criterion, ported from the official implementation, with 200 queries and 9 decoder layers.
• Training: an EMA of the weights, and sampling weighted by the competition's scene and resolution weights.
• Speed: precompute_features.py caches backbone outputs so that only the decoder has to be trained.

The submitted model had collapsed. All 150 test images got a single mask covering the whole tile.

06 · YOLO26x-seg (Ultralytics) (Jan 2026) A fast baseline built while the DINO models were being debugged. It converts COCO annotations to YOLO segmentation format and trains at 1024 px with full 180° rotation, flips, mosaic and mixup, validating at IoU 0.75. It produced plenty of instances (about 18.7k), but every prediction came out as individual_tree. The model never predicted the second class.

07 · DINOv2 backbone + custom Mask2Former decoder (Jan 2026) This uses the same decoder as 05 on DINOv2 with registers. I moved to DINOv2 partly for licensing reasons. The competition only allows weights under CC, MIT, BSD, public-domain or Apache-2.0 licenses, and DINOv3 weights are released under Meta's own DINOv3 License with gated access, so approach 05 didn't meet that rule. DINOv2 weights are Apache-2.0. It also adds:

• a faster training loop (train_fast.py)
• flip TTA
• partial backbone unfreezing with a lower learning rate
• diagnose_loss.py, diagnose_masks.py and visualize_predictions.py, written to work out why the masks kept collapsing

I tried a learnable mask bias, cosine-similarity mask logits, output scaling and removing feature normalisation. The submission still came out as one prediction per image, and 64 of the 150 cover the whole tile.

Lessons

• Sanity-check the submission file, not just the training loss. A few lines counting predictions per image, the share that cover the whole tile and the class histogram would have caught the collapse in 05 and 07 and the missing class in 06 before I spent submissions on them.
• Get a strong, boring baseline working before writing a model yourself. An off-the-shelf YOLO26 run beat everything else, even though it never predicted one of the two classes. Porting the Mask2Former decoder, matcher and loss onto a frozen ViT has many subtle failure modes, and debugging them took most of January for a score of zero. That time would have been better spent fixing the YOLO class issue and tuning it.
• Resolution matters for small objects. Approaches 04 and 07 trained at 512–560 px on 1024 px tiles. Crowns at 10 cm resolution are small, and the metric demands IoU ≥ 0.75.
• Build the competition metric locally from day one. The weighted mAP@0.75 evaluator in metrics.py only arrived in January. Until then I was choosing checkpoints with plain COCO mAP, which ignores the scene and resolution weighting.
• Go deep rather than wide. Seven setups, none of them tuned, is the real reason for the result.

Repository layout

approaches/
  01-semantic-mask2former-mmseg/   MMSegmentation config + contour-based instance extraction
  02-mask2former-mmdet/            MMDetection config, preprocessing, offline augmentation, Dockerfile
  03-htc-swinl-mmdet/              HTC + Swin-L config, MMCV fix script, Dockerfile
  04-mask2former-hf/               Hugging Face dataset, Trainer script, inference, Dockerfile
  05-dinov3-mask2former/           DINOv3 adapter, Mask2Former decoder/matcher/criterion, feature caching
  06-yolo26-seg/                   COCO→YOLO conversion, training, inference, local evaluation
  07-dinov2-mask2former/           DINOv2 adapter, fast training loop, TTA, debugging scripts
metrics.py                         IoU / PQ / F-score helpers and the competition's weighted mAP@0.75
notebooks/                         Augmentation and prediction visualisation (outputs cleared)

Running the code

The competition data is not included. Under the competition terms it is distributed only by Solafune and must be deleted after the competition. As the competition rules require, the following statement applies to the data and content provided by Solafune:

cf. @solafune(https://solafune.com) Use for any purpose other than participation in the competition or commercial use is prohibited. If you would like to use them for any of the above purposes, please contact us.

If you have access to the data, place it under data/ inside the approach folder you want to run:

data/
  train_images.zip
  train_annotations.json
  evaluation_images/

Every approach follows the competition's preprocess → train → predict structure. For example, for approach 02:

cd approaches/02-mask2former-mmdet
docker build -t tcd-mmdet .
docker run --gpus all --ipc=host --rm -it -v "$PWD/data:/app/data" tcd-mmdet ./run_pipeline.sh

• Approaches 02–04 have a Dockerfile, because the OpenMMLab stack (MMCV CUDA ops in particular) is fragile to install by hand.
• Approaches 05–07 only need the packages in their requirements.txt.
• Approach 01 expects a sibling mmsegmentation checkout, as noted in its config.
• Approaches 05–07 import the shared metrics.py through a symlink.

Everything was developed on a single RTX 3090 (24 GB).

License

The code is released under the Apache License 2.0 (see LICENSE and NOTICE). The Mask2Former decoder, matcher and criterion in approaches 05 and 07 are adapted from facebookresearch/Mask2Former (MIT). The IoU metric helpers in metrics.py come from Solafune's solafune-tools (MIT). The competition data belongs to Solafune and its original providers and isn't redistributed here.