Description:
I'm working on implementing the CONCEPTBANK framework described in the paper "Taming SAM3 in the Wild: A Concept Bank for Open-Vocabulary Segmentation". I need clarification on the exact data requirements, code execution steps, and parameter configurations for building the concept bank. Below are my specific questions:
🔍 Key Questions
Datasets Required
Which datasets' training sets are needed to build CONCEPTBANK?
(Confirmed from paper: VOC21, PC60, COCO-O, ADE20K, Cityscapes, LoveDA, Potsdam, Vaihingen, iSAID)
Specifically:
- For
ADE20K, do I need the full training split?
- For
LoveDA, is the train split sufficient?
- Are ground-truth masks required for all datasets?
Code Execution
I found the script sam3_concept_bank.py in the repository. What are the exact command-line instructions to build a concept bank?
(Example format I expect):
python sam3_concept_bank.py --dataset ade20k --output ./conceptbank_ade20k --k 5 --temperature 0.1
Critical Parameters
Which parameters are essential to tune?
--k (number of representative supports per class) --temperature (for soft fusion in Stage III) --llm (e.g., gpt-3.5-turbo for text augmentation)
Implementation Gaps
I've downloaded:
- ADE20K training set (15k images + masks)
- COCO-2017 train (118k images + masks)
But I'm stuck on:
- How to generate text prompts for Stage III (paper mentions LLM)
- How to compute
Dice scores for candidate embeddings (Stage III)
📌 Expected Output Format
A working example for ADE20K:
# Build CONCEPTBANK for ADE20K (150 classes)
python sam3_concept_bank.py \
--dataset ade20k \
--data_root ./ADEChallengeData2016/ \
--output ./conceptbank_ade20k \
--k 5 \
--temperature 0.1 \
--llm gpt-3.5-turbo
📂 Dataset Requirements Summary
| Dataset |
Required Split |
Masks Needed? |
Classes |
| ADE20K |
training |
✅ Yes |
150 |
| COCO-O |
train2017 |
✅ Yes |
80 |
| LoveDA |
train |
✅ Yes |
7 |
| Vaihingen |
train |
✅ Yes |
6 |
| Potsdam |
train |
✅ Yes |
6 |
Note: Paper states "support set = training set of target dataset" (Section 3.1)
❓ Why This Matters
The CONCEPTBANK is parameter-free and offline (built once per dataset), but the paper lacks concrete code examples. Without clear instructions, I cannot replicate the results (e.g., 5.2% mIoU gain on ADE20K).
✅ What I've Tried
- Ran SAM3 on ADE20K training images → extracted embeddings
- Attempted manual implementation of Stage I (prototype estimation)
- Failed at Stage II (top-K support mining) and Stage III (LLM fusion) due to missing pipeline details.
Thank you for your time! If you have a working example or documentation, I’d greatly appreciate a link to the relevant code section. This will help the open-source community advance open-vocabulary segmentation research.
Description:
I'm working on implementing the CONCEPTBANK framework described in the paper "Taming SAM3 in the Wild: A Concept Bank for Open-Vocabulary Segmentation". I need clarification on the exact data requirements, code execution steps, and parameter configurations for building the concept bank. Below are my specific questions:
🔍 Key Questions
Datasets Required
Which datasets' training sets are needed to build CONCEPTBANK?
(Confirmed from paper: VOC21, PC60, COCO-O, ADE20K, Cityscapes, LoveDA, Potsdam, Vaihingen, iSAID)
Specifically:
ADE20K, do I need the fulltrainingsplit?LoveDA, is thetrainsplit sufficient?Code Execution
I found the script
sam3_concept_bank.pyin the repository. What are the exact command-line instructions to build a concept bank?(Example format I expect):
Critical Parameters
Which parameters are essential to tune?
--k(number of representative supports per class)--temperature(for soft fusion in Stage III)--llm(e.g.,gpt-3.5-turbofor text augmentation)Implementation Gaps
I've downloaded:
But I'm stuck on:
Dicescores for candidate embeddings (Stage III)📌 Expected Output Format
A working example for ADE20K:
📂 Dataset Requirements Summary
❓ Why This Matters
The CONCEPTBANK is parameter-free and offline (built once per dataset), but the paper lacks concrete code examples. Without clear instructions, I cannot replicate the results (e.g., 5.2% mIoU gain on ADE20K).
✅ What I've Tried
Thank you for your time! If you have a working example or documentation, I’d greatly appreciate a link to the relevant code section. This will help the open-source community advance open-vocabulary segmentation research.