A Python tool that classifies markdown guides using sentence embeddings to detect duplicates, variants, amendments, and new content, exporting structured results for analysis.
This repo is to test the classification process. It is not part of The B.L.U.E. System but instead a prototype proposal for implementating a classification system.
- Create and activate environment
python3 -m venv env
source env/bin/activate- Install dependencies
pip install -r requirements.txt-
Add guides locally to test
- create
test_datadirectory in repo root directory - create
existingandnewdirectories intest_datadirectory - add markdown files with content to both
existingandnewdirectories
- create
-
Run script
python3 src/app.py- Results in
output/classification_results.json
Currently only classifying based on embeddings is complete.
