Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions .github/workflows/run_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -80,10 +80,16 @@ jobs:
PYTHONIOENCODING: utf-8
if: runner.os == 'Windows'

- name: Test datamint-config CLI
- name: Test datamint-config CLI (deprecated alias)
run: datamint-config --api-key testapikey
timeout-minutes: 1
env:
env:
PYTHONIOENCODING: utf-8

- name: Test datamint config CLI (unified)
run: datamint config --api-key testapikey
timeout-minutes: 1
env:
PYTHONIOENCODING: utf-8

- name: Upload pytest test results
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,15 +69,15 @@ For instance, create the enviroment once with `python3 -m venv datamint-env` and
**2. Configure your API key**

```bash
datamint-config
datamint config
```

Follow the prompts (ask your administrator if you don't have a key yet). Environment variable and programmatic options are in the [Setup API Key guide](https://sonanceai.github.io/datamint-python-api/getting_started.html#setup-api-key).

**3. Scaffold a project — the fastest way to start**

```bash
datamint-init
datamint init
```

This is the recommended on-ramp: it asks for a project name and task type (**segmentation**, **classification**, or **detection**), then generates a ready-to-run, numbered set of scripts (`01_upload_data.py` → `06_deploy.py`) — upload data, train, and deploy by running them in order.
Expand All @@ -103,7 +103,7 @@ results = trainer.fit()
| [📖 API Reference](https://sonanceai.github.io/datamint-python-api/client_api.html) | Complete API documentation |
| [🔥 PyTorch Integration](https://sonanceai.github.io/datamint-python-api/pytorch_integration.html) | ML workflow integration |
| [🧠 Trainer Guide](https://sonanceai.github.io/datamint-python-api/trainer_api.html) | Built-in trainers, trainer lifecycle, and custom model integration |
| [🛠️ Command Line Tools](https://sonanceai.github.io/datamint-python-api/command_line_tools.html) | Full reference for `datamint-upload`, `datamint-init`, and `datamint-config` |
| [🛠️ Command Line Tools](https://sonanceai.github.io/datamint-python-api/command_line_tools.html) | Full reference for `datamint upload`, `datamint init`, and `datamint config` |
| [🔒 SSL Troubleshooting](https://sonanceai.github.io/datamint-python-api/ssl_troubleshooting.html) | Fixing `SSLCertVerificationError` |
| [📓 Notebooks](notebooks/) | Numbered, runnable tutorials. Start at `01_getting_started` and work through annotations, datasets, experiment tracking, deployment, and a full end-to-end example |

Expand Down
24 changes: 19 additions & 5 deletions datamint/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ def _resolve_version() -> str:

def _build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="python -m datamint",
prog="datamint",
description="Datamint command-line interface.",
epilog="Available commands: config, upload",
epilog=f"Available commands: {', '.join(_COMMANDS)}.",
)
parser.add_argument(
"command",
Expand All @@ -43,11 +43,25 @@ def _build_parser() -> argparse.ArgumentParser:

def main() -> None:
parser = _build_parser()
# Parse only the first positional argument; leave the rest for the subcommand.
args, remaining = parser.parse_known_args()
argv = sys.argv[1:]

if not argv:
# Bare "datamint" with no subcommand: show help instead of an argparse error
parser.print_help()
sys.exit(0)

# Parse only the first token (the command name) so that flags meant for the subcommand
# (e.g. "datamint upload --help") are forwarded untouched instead of being swallowed by
# this top-level parser's own -h/--help/--version handling, which would otherwise trigger
# as soon as it sees them anywhere in the argument list.
args = parser.parse_args(argv[:1])
remaining = argv[1:]

# Replace argv so the subcommand sees only its own arguments.
sys.argv = [f"datamint-{args.command}", *remaining]
# Note: a space (not a hyphen) so nested argparse usage lines read "datamint <command>"
# (argparse's default prog is os.path.basename(sys.argv[0]), which is the string as-is
# when it has no path separator).
sys.argv = [f"datamint {args.command}", *remaining]

module_path = _COMMANDS[args.command]
try:
Expand Down
24 changes: 16 additions & 8 deletions datamint/client_cmd_tools/datamint_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
from typing_extensions import NotRequired

from datamint import configs
from datamint.utils.env import is_legacy_cli_invocation
from datamint.utils.logging_utils import ConsoleWrapperHandler, load_cmdline_logging_config

_LOGGER = logging.getLogger(__name__)
Expand Down Expand Up @@ -751,22 +752,29 @@ def main():
console_handlers = [h for h in _USER_LOGGER.handlers if isinstance(h, ConsoleWrapperHandler)]
if console_handlers:
console = console_handlers[0].console

if is_legacy_cli_invocation('config'):
console.print(
"[warning]'datamint-config' is deprecated and will be removed in a future "
"release. Use 'datamint config' instead.[/warning]"
)

parser = argparse.ArgumentParser(
description='🔧 Datamint API Configuration Tool',
epilog="""
Examples:
datamint-config # Interactive mode
datamint-config --api-key YOUR_KEY # Set API key
datamint-config --list-local-data # Show local cache/data groups and filter selectors
datamint-config --clean-local-data resources
datamint config # Interactive mode
datamint config --api-key YOUR_KEY # Set API key
datamint config --list-local-data # Show local cache/data groups and filter selectors
datamint config --clean-local-data resources
# Clean a cache namespace
datamint-config --clean-local-data channel:training-data
datamint config --clean-local-data channel:training-data
# Clean cached resources for one upload channel
datamint-config --clean-local-data tag:tutorial
datamint config --clean-local-data tag:tutorial
# Clean cached resources matching one tag
datamint-config --clean-local-data "Example Project"
datamint config --clean-local-data "Example Project"
# Clean a legacy dataset folder
datamint-config --clean-all-local-data # Clean all local data groups
datamint config --clean-all-local-data # Clean all local data groups

More Documentation: https://sonanceai.github.io/datamint-python-api/command_line_tools.html
""",
Expand Down
15 changes: 11 additions & 4 deletions datamint/client_cmd_tools/datamint_inference.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""datamint-inference command-line tool.
"""datamint inference command-line tool.

Run local inference with a registered Datamint model against a local file, without
writing any Python. Loads the model via MLflow (`models:/<name>/latest`), builds a
Expand All @@ -16,6 +16,7 @@

from datamint.client_cmd_tools.datamint_upload import _is_valid_path_argparse, handle_api_key
from datamint.exceptions import DatamintException, ItemNotFoundError
from datamint.utils.env import is_legacy_cli_invocation
from datamint.utils.logging_utils import ConsoleWrapperHandler, load_cmdline_logging_config

_LOGGER = logging.getLogger(__name__)
Expand Down Expand Up @@ -158,11 +159,11 @@ def _parse_args() -> argparse.Namespace:
description='Run local inference with a registered Datamint model against a local file.',
epilog="""
Examples:
datamint-inference file.png --model-name MyModel
datamint inference file.png --model-name MyModel
# Predict using the model registered as 'MyModel'
datamint-inference file.png --model-name my-model-alias --project MyProject
datamint inference file.png --model-name my-model-alias --project MyProject
# Model registered under a different name than its project
datamint-inference file.png --model-name MyModel --output result.png
datamint inference file.png --model-name MyModel --output result.png
# Also save a visualization of the predictions

More Documentation: https://sonanceai.github.io/datamint-python-api/command_line_tools.html
Expand All @@ -189,6 +190,12 @@ def main() -> None:
load_cmdline_logging_config()
CONSOLE = [h for h in _USER_LOGGER.handlers if isinstance(h, ConsoleWrapperHandler)][0].console

if is_legacy_cli_invocation('inference'):
CONSOLE.print(
"[warning]'datamint-inference' is deprecated and will be removed in a future "
"release. Use 'datamint inference' instead.[/warning]"
)

args = _parse_args()

if args.verbose:
Expand Down
16 changes: 12 additions & 4 deletions datamint/client_cmd_tools/datamint_init.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
from rich.prompt import Prompt, Confirm
from rich.rule import Rule

from datamint.utils.env import is_legacy_cli_invocation

console = Console()

# ---------------------------------------------------------------------------
Expand All @@ -31,7 +33,7 @@

```bash
pip install datamint
datamint-config --api-key YOUR_API_KEY
datamint config --api-key YOUR_API_KEY
```

"""
Expand Down Expand Up @@ -483,7 +485,7 @@

```bash
pip install datamint
datamint-config --api-key YOUR_API_KEY
datamint config --api-key YOUR_API_KEY
```

"""
Expand Down Expand Up @@ -923,7 +925,7 @@

```bash
pip install datamint
datamint-config --api-key YOUR_API_KEY
datamint config --api-key YOUR_API_KEY
```

"""
Expand Down Expand Up @@ -1443,7 +1445,7 @@ def _generate_segmentation_files(project_name: str) -> dict[str, str]:

def _print_header() -> None:
console.print()
console.rule("[bold]datamint-init[/bold]")
console.rule("[bold]datamint init[/bold]")
console.print()
console.print(
" This command generates a set of Python scripts that walk you\n"
Expand All @@ -1465,6 +1467,12 @@ def _print_header() -> None:


def main() -> None:
if is_legacy_cli_invocation('init'):
console.print(
"[yellow]'datamint-init' is deprecated and will be removed in a future "
"release. Use 'datamint init' instead.[/yellow]"
)

_print_header()

try:
Expand Down
23 changes: 15 additions & 8 deletions datamint/client_cmd_tools/datamint_train.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""datamint-train command-line tool.
"""datamint train command-line tool.

Train a model on a Datamint project using a built-in one-line trainer, without writing
any Python. Auto-detects the task (segmentation/classification/detection) and data format
Expand All @@ -23,6 +23,7 @@
from datamint import Api, configs
from datamint.client_cmd_tools.datamint_upload import handle_api_key
from datamint.exceptions import DatamintException
from datamint.utils.env import is_legacy_cli_invocation
from datamint.utils.logging_utils import ConsoleWrapperHandler, load_cmdline_logging_config

if TYPE_CHECKING:
Expand Down Expand Up @@ -94,7 +95,7 @@ def _detect(console: Console, project_name: str) -> tuple['DatamintBaseDataset',

if isinstance(dataset, VideoDataset):
raise DatamintTrainCliError(
"Video projects aren't supported by datamint-train yet. "
"Video projects aren't supported by datamint train yet. "
"Use the Python SDK (datamint.lightning) to build a custom training loop."
)

Expand Down Expand Up @@ -255,9 +256,9 @@ def _valid_aliases_for(task: str, fmt: str) -> list[str]:

def _print_header(console: Console) -> None:
console.print()
console.rule("[bold]datamint-train[/bold]")
console.rule("[bold]datamint train[/bold]")
console.print()
console.print(" Answer a few questions and datamint-train will pick a model for you.")
console.print(" Answer a few questions and datamint train will pick a model for you.")
console.print()


Expand Down Expand Up @@ -355,12 +356,12 @@ def _parse_args() -> argparse.Namespace:
description='Train a model on a Datamint project using a built-in one-line trainer.',
epilog="""
Examples:
datamint-train --project MyProject --model yolox --max-epochs 20
datamint train --project MyProject --model yolox --max-epochs 20
# Train a specific model
datamint-train --project MyProject # Auto-detect task, data format, and model
datamint-train --project MyProject --dry-run
datamint train --project MyProject # Auto-detect task, data format, and model
datamint train --project MyProject --dry-run
# Preview the detected plan without training
datamint-train --interactive # Guided wizard, no flags needed
datamint train --interactive # Guided wizard, no flags needed

More Documentation: https://sonanceai.github.io/datamint-python-api/command_line_tools.html#training-a-model
""",
Expand Down Expand Up @@ -399,6 +400,12 @@ def main() -> None:
load_cmdline_logging_config()
CONSOLE = [h for h in _USER_LOGGER.handlers if isinstance(h, ConsoleWrapperHandler)][0].console

if is_legacy_cli_invocation('train'):
CONSOLE.print(
"[warning]'datamint-train' is deprecated and will be removed in a future "
"release. Use 'datamint train' instead.[/warning]"
)

args = _parse_args()

if args.verbose:
Expand Down
7 changes: 7 additions & 0 deletions datamint/client_cmd_tools/datamint_upload.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
from datamint import __version__ as datamint_version
from datamint import configs
from datamint.utils.logging_utils import load_cmdline_logging_config, ConsoleWrapperHandler
from datamint.utils.env import is_legacy_cli_invocation
from rich.console import Console
import yaml
from collections.abc import Iterable
Expand Down Expand Up @@ -770,6 +771,12 @@ def main():
load_cmdline_logging_config()
CONSOLE = [h for h in _USER_LOGGER.handlers if isinstance(h, ConsoleWrapperHandler)][0].console

if is_legacy_cli_invocation('upload'):
CONSOLE.print(
"[warning]'datamint-upload' is deprecated and will be removed in a future "
"release. Use 'datamint upload' instead.[/warning]"
)

try:
args, files_path, segfiles, metadata_files = _parse_args()
except Exception as e:
Expand Down
4 changes: 2 additions & 2 deletions datamint/mlflow/env_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,14 +118,14 @@ def ensure_mlflow_configured() -> None:
if not os.getenv('MLFLOW_TRACKING_URI'):
raise ValueError(
"MLflow environment not configured. Please either:\n"
"1. Run 'datamint-config --default-url <url>', or\n"
"1. Run 'datamint config --default-url <url>', or\n"
"2. Set DATAMINT_API_URL environment variable, or\n"
"3. Manually set MLFLOW_TRACKING_URI environment variable"
)
if not os.getenv('MLFLOW_TRACKING_TOKEN'):
raise ValueError(
"MLflow environment not configured. Please either:\n"
"1. Run 'datamint-config', or\n"
"1. Run 'datamint config', or\n"
"2. Set DATAMINT_API_KEY environment variable, or\n"
"3. Manually set MLFLOW_TRACKING_TOKEN environment variable"
)
7 changes: 7 additions & 0 deletions datamint/utils/env.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,10 @@ def ensure_asyncio_loop():
import nest_asyncio
nest_asyncio.apply()
_ASYNCIO_LOOP_PATCHED = True


def is_legacy_cli_invocation(command: str) -> bool:
"""Check if the current process was launched via the deprecated `datamint-<command>` script."""
import os
import sys
return os.path.basename(sys.argv[0]) == f"datamint-{command}"
4 changes: 2 additions & 2 deletions dev/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Quick reference for developers working on the `datamint-python-api` SDK.

## Configuration

The SDK uses a local configuration file for authentication and settings. When the user runs [`datamint-config`](./datamint/client_cmd_tools/datamint_config.py), configuration is stored in `~/.config/datamintapi/datamintapi.yaml` (via [PlatformDirs](https://pypi.org/project/platformdirs/)).
The SDK uses a local configuration file for authentication and settings. When the user runs [`datamint config`](./datamint/client_cmd_tools/datamint_config.py), configuration is stored in `~/.config/datamintapi/datamintapi.yaml` (via [PlatformDirs](https://pypi.org/project/platformdirs/)).

## Module Structure

Expand All @@ -39,7 +39,7 @@ datamint/
│ ├── flavors/ # Model flavors and prediction routing
│ ├── tracking/ # Custom DatamintStore
│ └── artifact/ # DatamintArtifactsRepository plugin
├── client_cmd_tools/ # CLI tools (datamint-upload, datamint-config)
├── client_cmd_tools/ # CLI tools (datamint upload, datamint config)
└── utils/ # General Utilities
```

Expand Down
Loading
Loading