diff --git a/ANNOUNCE.md b/ANNOUNCE.md index 25283667..3294159e 100644 --- a/ANNOUNCE.md +++ b/ANNOUNCE.md @@ -6,11 +6,11 @@ Blosc2 and HDF5 data repositories. This is a major release that includes a big refactoring for getting rid of the PubSub code (which, to be frank, was not bringing too much benefit). -We have added a new `cat2agent` to watch a directory and sync changes to a +We have added a new `cat2-agent` to watch a directory and sync changes to a Caterva2 server. Finally, fixed a few bugs and added some new features, such as the -`cat2agent` command line client, which allows you to watch a directory and +`cat2-agent` command line client, which allows you to watch a directory and sync changes to a Caterva2 server. This is particularly useful for automatically uploading new datasets to a Caterva2 server, or for keeping a local copy of a remote Caterva2 server. diff --git a/README.md b/README.md index 94c4d6d5..4e220039 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ client.get("@public/examples/tomo-guess-test.b2nd") ``` - Via the [command line client](https://ironarray.io/caterva2-doc/tutorials/cli.html) ```sh -cat2cli info @public/kevlar/entry/data/data.b2nd +cat2-client info @public/kevlar/entry/data/data.b2nd ``` - Via the [REST API](https://ironarray.io/caterva2-doc/tutorials/RESTAPI.html) using a REST client like [Postman](https://www.postman.com/) or [curl](https://curl.se/) (see [here](https://cat2.cloud/demo/docs)). @@ -72,7 +72,7 @@ accesses the relevant datasets stored either locally or remotely. The server ser ``` In general, if you intend to run Caterva2 services, client programs, or the test suite, you need to enable the proper extra features by appending `[feature1,feature2...]` to the last argument of `pip` commands above. The following extras are supported: -- `subscriber` for running the Caterva2 server service +- `server` for running the Caterva2 server service - `clients` to use Caterva2 client programs (command-line or terminal) - `blosc2-plugins` to enable extra Blosc2 features like Btune or JPEG 2000 support - `plugins` to enable web GUI features like the tomography display @@ -114,7 +114,7 @@ dir1/ ds-1d-b.b2nd ds-1d.b2nd ds-hello Now: -- create a virtual environment and install Caterva2 with the `[subscriber,clients]` extras (see above). +- create a virtual environment and install Caterva2 with the `[server,clients]` extras (see above). - copy the configuration file `caterva2.sample.toml` to `caterva2.toml`. Servers (and clients, to a limited extent) may get their configuration from a `caterva2.toml` file at the current directory (or an alternative file given with the `--conf` option). @@ -123,17 +123,17 @@ See also [configuration.md](configuration.md) in Caterva2 tutorials. Then run the server: ```sh -CATERVA2_SECRET=c2sikrit cat2sub & # server +CATERVA2_SECRET=c2sikrit cat2-server & # server ``` The `CATERVA2_SECRET` environment variable is obligatory and is explained below in the following section. ### User authentication The Caterva2 server includes some support for authenticating users. To enable it, run the server with the environment variable `CATERVA2_SECRET` set to some non-empty, secure string that will be used for various user management operations. Note that new accounts may be registered, but their addresses are not verified. Password recovery does not work either. -To create a user, you can use the `cat2adduser` command line client. For example: +To create a user, you can use the `cat2-admin adduser` command. For example: ```sh -cat2adduser user@example.com foobar11 +cat2-admin adduser user@example.com foobar11 ``` Client queries then require the same user credentials: @@ -145,11 +145,11 @@ client = cat2.Client("https://cat2.cloud/demo", ('user@example.com', 'foobar11') - The command line client can be authenticated with the `--user` and `--pass` options ### The command line client -Now that the services are running, we can use the `cat2cli` client to talk +Now that the services are running, we can use the `cat2-client` client to talk to the server. In another shell, let's list all the available roots in the system: ```sh -cat2cli --user "user@example.com" --pass "foobar11" roots +cat2-client --user "user@example.com" --pass "foobar11" roots ``` ``` @@ -160,20 +160,20 @@ cat2cli --user "user@example.com" --pass "foobar11" roots First let's upload a file from the `root-example`folder to the `@personal` root: ```sh -cat2cli --username user@example.com --password foobar11 upload root-example/ds-1d.b2nd @personal/ds-1d.b2nd +cat2-client --username user@example.com --password foobar11 upload root-example/ds-1d.b2nd @personal/ds-1d.b2nd ``` Now, one can list the datasets in the `@personal` root and see that the uploaded file appears ```sh -cat2cli --username user@example.com --password foobar11 list @personal +cat2-client --username user@example.com --password foobar11 list @personal >> ds-1d.b2nd ``` Let's ask the server for more info about the dataset: ```sh -cat2cli --username user@example.com --password foobar11 info @personal/ds-1d.b2nd +cat2-client --username user@example.com --password foobar11 info @personal/ds-1d.b2nd ``` ``` @@ -191,7 +191,7 @@ Getting info for @personal/ds-1d.b2nd 'cparams': {'codec': 5, 'codec_meta': 0, 'clevel': 1, 'filters': [0, 0, 0, 0, 0, 1], 'filters_meta': [0, 0, 0, 0, 0, 0], 'typesize': 8, 'blocksize': 80, 'nthreads': 1, 'splitmode': 1, 'tuner': 0, 'use_dict': False, 'filters, meta': [[1, 0]]}, 'cratio': 1.5929908403026682, 'nbytes': 8000, - 'urlpath': '/home/lshaw/Caterva2/_caterva2/sub/personal/2fa87091-84c6-44f9-a57e-7f04290630b1/ds-1d.b2nd', + 'urlpath': '/home/lshaw/Caterva2/_caterva2/state/personal/2fa87091-84c6-44f9-a57e-7f04290630b1/ds-1d.b2nd', 'vlmeta': {}, 'nchunks': 10, 'mtime': None @@ -202,10 +202,10 @@ Getting info for @personal/ds-1d.b2nd This command returns a JSON object with the dataset's metadata, including its shape, chunks, blocks, data type, and compression parameters. The `schunk` field contains information about the underlying Blosc2 super-chunk that stores the dataset's data. -There are more commands available in the `cat2cli` client; ask for help with: +There are more commands available in the `cat2-client` client; ask for help with: ```sh -cat2cli --help +cat2-client --help ``` ### Docs diff --git a/RELEASING.rst b/RELEASING.rst index 6df9682c..82cb19a9 100644 --- a/RELEASING.rst +++ b/RELEASING.rst @@ -36,7 +36,7 @@ the content of ``caterva2.sample.toml`` to ``caterva2.toml`` and run the following commands:: $ rm -r _caterva2/ - $ cat2adduser superuser@example.com foobarfoo -S + $ cat2-admin adduser superuser@example.com foobarfoo -S $ CATERVA2_SECRET="123" cat2sub And experiment a bit with uploading, browsing and downloading files. diff --git a/SPECS.md b/SPECS.md index 1b0c3c16..80f7c70f 100644 --- a/SPECS.md +++ b/SPECS.md @@ -43,7 +43,7 @@ There should be a configuration file (by default $CWD/caterva2.toml) where the c [server] http = "localhost:8000" urlbase = "https://cat2.example.com" # e.g. served by reverse proxy -statedir = "_caterva2/sub" +statedir = "_caterva2/state" loglevel = "warning" ``` diff --git a/caterva2.sample.toml b/caterva2.sample.toml index 3e0f5914..cca8512d 100644 --- a/caterva2.sample.toml +++ b/caterva2.sample.toml @@ -5,7 +5,7 @@ # The server section must define: # -# - statedir: the directory where the server's data will be stored (default: _caterva2/sub) +# - statedir: the directory where the server's data will be stored (default: _caterva2/state) # - http: where the server listens to (a unix socket or a host/port) (default: localhost:8000) # - urlbase: the base url users will use to reach the server (default: http://localhost:8000) # - quota: if defined, it will limit the disk usage (default: 0, no limit) @@ -13,9 +13,9 @@ # - login: if true, users will need to authenticate (default: true) # - register: if true, users will be able to register (default: false) # -[subscriber] -statedir = "_caterva2/sub" -#http = "_caterva2/sub/uvicorn.socket" +[server] +statedir = "_caterva2/state" +#http = "_caterva2/state/uvicorn.socket" http = "localhost:8000" urlbase = "http://localhost:8000" quota = "10G" diff --git a/caterva2/api_utils.py b/caterva2/api_utils.py index 5238dbab..8b3416bb 100644 --- a/caterva2/api_utils.py +++ b/caterva2/api_utils.py @@ -39,14 +39,14 @@ def slice_to_string(slice_): def get_auth_cookie(urlbase, user_auth, timeout=5): """ - Authenticate to a subscriber as a user and get an authorization cookie. + Authenticate to a server as a user and get an authorization cookie. Authentication fields will usually be ``username`` and ``password``. Parameters ---------- urlbase : str - The base of URLs of the subscriber to query. + The base of URLs of the server to query. user_auth : dict A mapping of fields and values used as data to be posted for authenticating the user. @@ -55,7 +55,7 @@ def get_auth_cookie(urlbase, user_auth, timeout=5): ------- str An authentication token that may be used as a cookie in further - requests to the subscriber. + requests to the server. Examples -------- diff --git a/caterva2/clients/cli.py b/caterva2/clients/cli.py index dcc34087..1cb8c780 100644 --- a/caterva2/clients/cli.py +++ b/caterva2/clients/cli.py @@ -443,10 +443,10 @@ def main(): conf = utils.get_conf() parser = utils.get_parser() parser.add_argument( - "--subscriber", + "--server", dest="urlbase", type=utils.urlbase_type, - default=conf.get("subscriber.url", cat2.sub_urlbase_default), + default=conf.get("server.url", cat2.sub_urlbase_default), ) parser.add_argument("--username", default=conf.get("client.username")) parser.add_argument("--password", default=conf.get("client.password")) diff --git a/caterva2/clients/tbrowser.py b/caterva2/clients/tbrowser.py index 7a4ec0da..61388dcd 100644 --- a/caterva2/clients/tbrowser.py +++ b/caterva2/clients/tbrowser.py @@ -51,10 +51,10 @@ def main(): conf = utils.get_conf() parser = utils.get_parser() parser.add_argument( - "--subscriber", + "--server", dest="urlbase", type=utils.urlbase_type, - default=conf.get("subscriber.url", api.sub_urlbase_default), + default=conf.get("server.url", api.sub_urlbase_default), ) parser.add_argument("--username", default=conf.get("client.username")) parser.add_argument("--password", default=conf.get("client.password")) diff --git a/caterva2/services/server.py b/caterva2/services/server.py index 226461c1..a8284001 100644 --- a/caterva2/services/server.py +++ b/caterva2/services/server.py @@ -15,7 +15,6 @@ import io import itertools import json -import logging import mimetypes import os import pathlib @@ -53,8 +52,6 @@ # Set CATERVA2_SECRET=XXX in .env file in working directory dotenv.load_dotenv() -# Logging -logger = logging.getLogger("sub") # State locks = {} @@ -2579,11 +2576,11 @@ def guess_dset_ctype(path: pathlib.Path, meta) -> str | None: def main(): # Load configuration (args) - conf = utils.get_conf("subscriber") + conf = utils.get_conf("server") parser = utils.get_parser( http=conf.get(".http", "localhost:8000"), loglevel=conf.get(".loglevel", "warning"), - statedir=conf.get(".statedir", "_caterva2/sub"), + statedir=conf.get(".statedir", "_caterva2/state"), ) args = utils.run_parser(parser) diff --git a/caterva2/services/settings.py b/caterva2/services/settings.py index b6e98950..8213d7e3 100644 --- a/caterva2/services/settings.py +++ b/caterva2/services/settings.py @@ -27,7 +27,7 @@ def parse_size(size): return int(number * units[unit]) -conf = utils.get_conf("subscriber") +conf = utils.get_conf("server") urlbase = conf.get(".urlbase", "http://localhost:8000") login = conf.get(".login", True) diff --git a/caterva2/tests/caterva2-login.toml b/caterva2/tests/caterva2-login.toml index 885e5ae0..6428fe89 100644 --- a/caterva2/tests/caterva2-login.toml +++ b/caterva2/tests/caterva2-login.toml @@ -1,4 +1,4 @@ -[subscriber] +[server] login = true register = true maxusers = 5 diff --git a/caterva2/tests/caterva2-nologin.toml b/caterva2/tests/caterva2-nologin.toml index c466cd70..071b3e28 100644 --- a/caterva2/tests/caterva2-nologin.toml +++ b/caterva2/tests/caterva2-nologin.toml @@ -1,4 +1,4 @@ -[subscriber] +[server] login = false register = false maxusers = 5 diff --git a/caterva2/tests/conftest.py b/caterva2/tests/conftest.py index e61ab4fc..0db504cf 100644 --- a/caterva2/tests/conftest.py +++ b/caterva2/tests/conftest.py @@ -27,7 +27,7 @@ def pytest_configure(config): @pytest.fixture(scope="session") def client(services): # noqa: F811 - urlbase = services.get_urlbase("subscriber") + urlbase = services.get_urlbase("server") return cat2.Client(urlbase) @@ -36,5 +36,5 @@ def auth_client(services, sub_user): # noqa: F811 if not sub_user: return None - urlbase = services.get_urlbase("subscriber") + urlbase = services.get_urlbase("server") return cat2.Client(urlbase, sub_user) diff --git a/caterva2/tests/services.py b/caterva2/tests/services.py index 92dad4b6..04c70b53 100644 --- a/caterva2/tests/services.py +++ b/caterva2/tests/services.py @@ -90,7 +90,7 @@ def http_service_check(conf, conf_sect, def_host, path): def sub_check(conf): - return http_service_check(conf, "subscriber", get_sub_ep(), "/api/roots") + return http_service_check(conf, "server", get_sub_ep(), "/api/roots") TestRoot = collections.namedtuple("TestRoot", ["name", "source"]) @@ -110,7 +110,7 @@ def __init__(self, state_dir, reuse_state=True, roots=None, configuration=None): self._setup_done = False def _start_server(self, *args, check=None): - name = "subscriber" + name = "server" if check is not None and check(): raise RuntimeError( f'check for service "{name}" succeeded before start' @@ -222,7 +222,7 @@ def make_sub_user(services): if not os.environ.get("CATERVA2_SECRET"): return None - state_dir = services.state_dir / "subscriber" + state_dir = services.state_dir / "server" return srv_utils.add_user( "user@example.com", password="foobar11", is_superuser=True, state_dir=state_dir ) diff --git a/caterva2/tests/test_api.py b/caterva2/tests/test_api.py index 94f7197e..2df44d55 100644 --- a/caterva2/tests/test_api.py +++ b/caterva2/tests/test_api.py @@ -24,7 +24,7 @@ @pytest.fixture def fill_public(client, examples_dir): # Manually copy some files to the public area (TEST_STATE_DIR) - dest_dir = pathlib.Path(TEST_STATE_DIR) / "subscriber/public" + dest_dir = pathlib.Path(TEST_STATE_DIR) / "server/public" fnames = [str(fname.relative_to(examples_dir)) for fname in examples_dir.rglob("*") if fname.is_file()] for fname in fnames: orig = examples_dir / fname @@ -937,8 +937,8 @@ def test_adduser_maxexceeded(auth_client, configuration): pytest.skip("authentication support needed") # TODO: make this to work; currently this returns None - # maxusers = configuration.get("subscriber.maxusers") - # For now, keep in sync with subscriber.maxusers in caterva2/tests/caterva2-login.toml + # maxusers = configuration.get("server.maxusers") + # For now, keep in sync with server.maxusers in caterva2/tests/caterva2-login.toml maxusers = 5 # Add maxusers users; we already have one user, so the next loop should fail # when reaching the creation of last user diff --git a/caterva2/tests/test_cli.py b/caterva2/tests/test_cli.py index 23922da9..062fdb30 100644 --- a/caterva2/tests/test_cli.py +++ b/caterva2/tests/test_cli.py @@ -19,7 +19,7 @@ @pytest.fixture def sub_urlbase(services): - return services.get_urlbase("subscriber") + return services.get_urlbase("server") def cli(cargs, binary=False, sub_user=None) -> str or dict: diff --git a/caterva2/tools/adduser.py b/caterva2/tools/adduser.py index 887f263c..8a2ace6b 100644 --- a/caterva2/tools/adduser.py +++ b/caterva2/tools/adduser.py @@ -8,9 +8,10 @@ ############################################################################### """ -Add a user to the subscriber database. +Add a user to the server database. -Contrarily to `cat2cli adduser`, this script does not require a running Caterva2 subscriber. +Contrarily to `cat2-client adduser`, this script does not require a running Caterva2 server. +Note: This script is deprecated. Use `cat2-admin adduser` instead. """ from caterva2 import utils @@ -19,9 +20,9 @@ def main(): # Load configuration (args) - conf = utils.get_conf("subscriber") + conf = utils.get_conf("server") parser = utils.get_parser( - statedir=conf.get(".statedir", "_caterva2/sub"), + statedir=conf.get(".statedir", "_caterva2/state"), ) parser.add_argument("username") parser.add_argument("password", nargs="?") diff --git a/caterva2/tools/admin.py b/caterva2/tools/admin.py new file mode 100644 index 00000000..95d7ef6e --- /dev/null +++ b/caterva2/tools/admin.py @@ -0,0 +1,73 @@ +############################################################################### +# Caterva2 - On demand access to remote Blosc2 data repositories +# +# Copyright (c) 2023 ironArray SLU +# https://www.blosc.org +# License: GNU Affero General Public License v3.0 +# See LICENSE.txt for details about copyright and rights to use. +############################################################################### + +""" +Administration commands for Caterva2 server. + +This module provides commands for server administration tasks such as user management. +These commands are meant to be used on the same machine as the server. +""" + +import argparse +import sys + +from caterva2 import utils +from caterva2.services import srv_utils + + +def adduser_command(args): + """Add a user to the server database.""" + # Load configuration + # conf = utils.get_conf("server") + + # Add user + statedir = args.statedir.resolve() + user = srv_utils.add_user(args.username, args.password, args.superuser, state_dir=statedir) + print("Password:", user.password) + + +def main(): + """Main entry point for cat2-admin command.""" + parser = argparse.ArgumentParser( + prog="cat2-admin", description="Administration commands for Caterva2 server" + ) + + # Global options + parser.add_argument( + "--statedir", + type=utils.get_path_type(), + default="_caterva2/state", + help="State directory for the server", + ) + + # Subcommands + subparsers = parser.add_subparsers(dest="command", help="Available commands") + + # adduser subcommand + adduser_parser = subparsers.add_parser("adduser", help="Add a user to the server database") + adduser_parser.add_argument("username", help="Username for the new user") + adduser_parser.add_argument("password", nargs="?", help="Password for the new user (optional)") + adduser_parser.add_argument( + "--superuser", "-S", action="store_true", default=False, help="Make user a superuser" + ) + adduser_parser.set_defaults(func=adduser_command) + + # Parse arguments + args = parser.parse_args() + + if not hasattr(args, "func"): + parser.print_help() + sys.exit(1) + + # Execute the command + args.func(args) + + +if __name__ == "__main__": + main() diff --git a/doc/tutorials/cli.md b/doc/tutorials/cli.md index 811e0450..e1b6c1a6 100644 --- a/doc/tutorials/cli.md +++ b/doc/tutorials/cli.md @@ -1,23 +1,23 @@ (Using-the-command-line-client)= # Using the command-line client -For quick queries to a server or for use in shell scripts, Caterva2 ships the `cat2cli` program. To use it, you need to install Caterva2 with the `clients` extra, as well as `subscriber` in order to be able to query something. +For quick queries to a server or for use in shell scripts, Caterva2 ships the `cat2-client` program. To use it, you need to install Caterva2 with the `clients` extra, as well as `server` in order to be able to query something. ```sh -python -m pip install caterva2[clients,subscriber] +python -m pip install caterva2[clients,server] ``` -To create a user, you can use the `cat2adduser` command line client. For example: +To create a user, you can use the `cat2-admin adduser` command. For example: ```sh -cat2adduser user@example.com foobar11 +cat2-admin adduser user@example.com foobar11 ``` -Now that the services are running, we can use the `cat2cli` client to talk +Now that the services are running, we can use the `cat2-client` client to talk to the server. In another shell, let's list all the available roots in the system: ```sh -cat2cli --user "user@example.com" --pass "foobar11" roots +cat2-client --user "user@example.com" --pass "foobar11" roots ``` ``` @@ -28,20 +28,20 @@ cat2cli --user "user@example.com" --pass "foobar11" roots First let's upload a file from the `root-example`folder to the `@personal` root: ```sh -cat2cli --username user@example.com --password foobar11 upload root-example/ds-1d.b2nd @personal/ds-1d.b2nd +cat2-client --username user@example.com --password foobar11 upload root-example/ds-1d.b2nd @personal/ds-1d.b2nd ``` Now, one can list the datasets in the `@personal` root and see that the uploaded file appears ```sh -cat2cli --username user@example.com --password foobar11 list @personal +cat2-client --username user@example.com --password foobar11 list @personal >> ds-1d.b2nd ``` Let's ask the server for more info about the dataset: ```sh -cat2cli --username user@example.com --password foobar11 info @personal/ds-1d.b2nd +cat2-client --username user@example.com --password foobar11 info @personal/ds-1d.b2nd ``` ``` @@ -70,8 +70,8 @@ Getting info for @personal/ds-1d.b2nd This command returns a JSON object with the dataset's metadata, including its shape, chunks, blocks, data type, and compression parameters. The `schunk` field contains information about the underlying Blosc2 super-chunk that stores the dataset's data. -There are more commands available in the `cat2cli` client; ask for help with: +There are more commands available in the `cat2-client` client; ask for help with: ```sh -cat2cli --help +cat2-client --help ``` diff --git a/doc/tutorials/configuration.md b/doc/tutorials/configuration.md index 9cd5f9cb..8389e9ba 100644 --- a/doc/tutorials/configuration.md +++ b/doc/tutorials/configuration.md @@ -1,7 +1,7 @@ (caterva2.toml)= # The `caterva2.toml` configuration file -We've seen that the `cat2cli` program accepts some command-line options to tune its operation (check the `--help` option). This is even more important for services as we shall see in following sections. Thus, Caterva2 programs support getting some settings from a TOML configuration file, by default `caterva2.toml` in the current directory (though you may override it with the `--conf` option). +We've seen that the `cat2-client` program accepts some command-line options to tune its operation (check the `--help` option). This is even more important for services as we shall see in following sections. Thus, Caterva2 programs support getting some settings from a TOML configuration file, by default `caterva2.toml` in the current directory (though you may override it with the `--conf` option). The configuration file may hold settings for different programs, with a separate section for each program. Thus, a program may check the file for its own settings, but also for those of other programs which may be of use to itself. This allows compact configurations in a single file. For instance, below is a sample configuration file for the server program and some client app: @@ -21,7 +21,7 @@ The configuration file may hold settings for different programs, with a separate # - login: if true, users will need to authenticate (default: true) # - register: if true, users will be able to register (default: false) # -[subscriber] +[server] statedir = "_caterva2/sub" #http = "_caterva2/sub/uvicorn.socket" http = "localhost:8002" @@ -31,7 +31,7 @@ maxusers = 5 register = true # allow users to register # The client section defines the credentials for the client to authenticate -# against the subscriber. +# against the server. [client] username = "" password = "" diff --git a/doc/tutorials/independent-services.md b/doc/tutorials/independent-services.md index c5f7fc46..3a74f691 100644 --- a/doc/tutorials/independent-services.md +++ b/doc/tutorials/independent-services.md @@ -15,7 +15,7 @@ The client and server hosts need a Caterva2 installation with the `services` ext python -m pip install caterva2[services] ``` -The workstation should be fine with a plain installation, but we'll also install the `clients` extra to perform quick tests with `cat2cli`: +The workstation should be fine with a plain installation, but we'll also install the `clients` extra to perform quick tests with `cat2-client`: ```sh python -m pip install caterva2[clients] @@ -88,7 +88,7 @@ By now, everything should look familiar to you (including the custom port and st To start the subscriber, just run: ```sh -cat2sub +cat2-server ``` ### User authentication @@ -96,7 +96,7 @@ cat2sub If the subscriber is to support user authentication (to restrict access, allow computing expressions or uploading files), it will need a `CATERVA2_SECRET` environment variable to be defined with its own secret token. That token should be persisted somewhere so as to use the same one every time the subscriber runs. You may start the subscriber like this: ```sh -env CATERVA2_SECRET=c2sikrit cat2sub +env CATERVA2_SECRET=c2sikrit cat2-server ``` Then users will need to register via the [Web client](Using-the-Web-client). @@ -113,15 +113,15 @@ urlbase = "https://sub.edu.example.org:3126" # reverse proxy address Clients at the example workstation need to know the address of the subscriber that they will use. -The command-line client `cat2cli` provides the `--subscriber` option for that. Running this at the workstation: +The command-line client `cat2-client` provides the `--server` option for that. Running this at the workstation: ```sh -cat2cli --subscriber http://sub.edu.example.org:3126 roots +cat2-client --server http://sub.edu.example.org:3126 roots ``` -Will retrieve the list of known roots from the subscriber that we set up above. Should authentication be needed, `--username` and `--password` options may also be used. +Will retrieve the list of known roots from the server that we set up above. Should authentication be needed, `--username` and `--password` options may also be used. -Since `cat2cli` also supports `caterva2.toml`, this configuration in the current directory: +Since `cat2-client` also supports `caterva2.toml`, this configuration in the current directory: ```toml [subscriber] @@ -135,7 +135,7 @@ urlbase = "http://sub.edu.example.org:3126" # "https://..." if needed Should allow you to run the previous command just like this: ```sh -cat2cli roots +cat2-client roots ``` When using the programmatic API, you need to provide the subscriber address explicitly: diff --git a/doc/utilities/cat2-admin.md b/doc/utilities/cat2-admin.md new file mode 100644 index 00000000..4c632bb9 --- /dev/null +++ b/doc/utilities/cat2-admin.md @@ -0,0 +1,58 @@ +(cat2-admin)= +# `cat2-admin` -- Caterva2 server administration + +This program provides administration commands for managing a Caterva2 server. These commands are meant to be used on the same machine as the server and typically require access to the server's state directory. + +## Installation + +To use `cat2-admin`, you need to install Caterva2 with the server extra: + +```sh +python -m pip install caterva2[server] +``` + +## Usage + +``` +cat2-admin [GLOBAL_OPTIONS...] COMMAND [COMMAND_OPTIONS...] COMMAND_ARGUMENTS... +``` + +### Global Options + +- `--statedir PATH`: Specify the state directory for the server (default: `_caterva2/sub`) +- `--help`: Show help message and exit + +### Commands + +#### `adduser` - Add a user to the server database + +Add a new user to the server database. + +``` +cat2-admin adduser [OPTIONS] USERNAME [PASSWORD] +``` + +**Arguments:** +- `USERNAME`: Username for the new user +- `PASSWORD`: Password for the new user (optional, will be generated if not provided) + +**Options:** +- `--superuser`, `-S`: Make the user a superuser +- `--help`: Show help for this command + +**Examples:** + +```sh +# Add a regular user with auto-generated password +cat2-admin adduser alice + +# Add a superuser with a specific password +cat2-admin adduser bob mypassword --superuser + +# Add a user to a custom state directory +cat2-admin --statedir /custom/path adduser charlie +``` + +## Configuration + +`cat2-admin` uses the same configuration system as other Caterva2 tools. It can read settings from a TOML configuration file (`caterva2.toml` in the current directory unless overridden). diff --git a/doc/utilities/cat2-client.md b/doc/utilities/cat2-client.md new file mode 100644 index 00000000..b1b2de6b --- /dev/null +++ b/doc/utilities/cat2-client.md @@ -0,0 +1,24 @@ +(cat2-client)= +# `cat2-client` -- Command-line Caterva2 client + +This program allows interacting with a Caterva2 server from the command line, in interactive shell sessions or invoked by other programs. To use it, the `clients` extra needs to be installed: + +```sh +python -m pip install caterva2[clients] +``` + +Running `cat2-client --help` should provide a list of supported commands that may be invoked like this: + +``` +cat2-client [GENERIC_OPTION...] COMMAND [COMMAND_OPTION...] COMMAND_ARGUMENTS... +``` + +Another relevant generic option besides `--help` is `--server`, which overrides the base of server URLs used by default. It should be a HTTP(S) URL, for example `http://sub.edu.example.org:3126`. Finally, the generic options `--username` and `--password` may be used in case your server requires user authentication. + +`--help` is also available as a command option which shows the options and arguments accepted by that command (e.g. `cat2-client roots --help`). Another command option is `--json`, which forces the output of commands that accept it to be in JSON format, as that may be more amenable for parsing by other programs. + +`cat2-client` may use a TOML configuration file (`caterva2.toml` in the current directory unless overridden with the generic `--conf` option). It may get the server address from there (`urlbase` or `http` settings in `[server]` section), as well as user authentication options (in the `[client]` section). Command-line options override settings read from the configuration file. + +For a short tutorial on `cat2-client`, see [](Using-the-command-line-client). + +**Note**: This is the primary command-line client for Caterva2. diff --git a/doc/utilities/cat2cli.md b/doc/utilities/cat2cli.md deleted file mode 100644 index c324aebb..00000000 --- a/doc/utilities/cat2cli.md +++ /dev/null @@ -1,22 +0,0 @@ -(cat2cli)= -# `cat2cli` -- Command-line Caterva2 client - -This program allows interacting with a Caterva2 subscriber from the command line, in interactive shell sessions or invoked by other programs. To use it, the `clients` extra needs to be installed: - -```sh -python -m pip install caterva2[clients] -``` - -Running `cat2cli --help` should provide a list of supported commands that may be invoked like this: - -``` -cat2cli [GENERIC_OPTION...] COMMAND [COMMAND_OPTION...] COMMAND_ARGUMENTS... -``` - -Another relevant generic option besides `--help` is `--subscriber`, which overrides the base of subscriber URLs used by default. It should be a HTTP(S) URL, for example `http://sub.edu.example.org:3126`. Finally, the generic options `--username` and `--password` may be used in case your subscriber requires user authentication. - -`--help` is also available as a command option which shows the options and arguments accepted by that command (e.g. `cat2cli roots --help`). Another command option is `--json`, which forces the output of commands that accept it to be in JSON format, as that may be more amenable for parsing by other programs. - -`cat2cli` may use a TOML configuration file (`caterva2.toml` in the current directory unless overridden with the generic `--conf` option). It may get the subscriber address from there (`urlbase` or `http` settings in `[subscriber]` section), as well as user authentication options (in the `[client]` section). Command-line options override settings read from the configuration file. - -For a short tutorial on `cat2cli`, see [](Using-the-command-line-client). diff --git a/doc/utilities/index.md b/doc/utilities/index.md index 780b2dfd..011691bf 100644 --- a/doc/utilities/index.md +++ b/doc/utilities/index.md @@ -1,13 +1,32 @@ # Utilities -Although the main role of the Caterva2 package is to provide a Python library for coding Caterva2 clients in Python, it also includes additional utilities to help with particular tasks. The sections below cover some of them. Please mind that they may be part of extra Caterva2 features with additional dependencies and requirements. +Caterva2 comes with several command-line utilities for different tasks. + +## Main Commands + +The main utilities follow a new unified naming scheme: + +- **cat2-client**: Query a server from terminal +- **cat2-agent**: Watch a directory and sync changes to a Caterva2 server +- **cat2-server**: Launch the server with fine-tuned behavior +- **cat2-admin**: Server administration commands (new) + +## Additional Utilities + +- **cat2import**: Import data from HDF5 to Caterva2 +- **cat2export**: Export data from Caterva2 to HDF5 +- **cat2tbrowser**: Terminal-based browser for datasets ```{toctree} --- maxdepth: 2 --- -cat2cli +cat2-client +cat2-agent +cat2-admin +cat2-server +cat2tbrowser cat2import cat2export ``` diff --git a/pyproject.toml b/pyproject.toml index 89abe98c..2b28759c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -50,7 +50,7 @@ base-services = [ "uvicorn", "watchfiles", ] -subscriber = [ +server = [ "aiosqlite", "caterva2[base-services]", "fastapi-mail", @@ -69,7 +69,7 @@ subscriber = [ services = [ "caterva2[base-services]", "caterva2[hdf5]", - "caterva2[subscriber]", + "caterva2[server]", ] dev = [ "pre-commit", @@ -107,13 +107,18 @@ only-include = ["caterva2", "root-example"] Home = "https://github.com/ironArray/Caterva2" [project.scripts] -cat2sub = "caterva2.services.server:main" -cat2agent = "caterva2.clients.agent:main" -cat2cli = "caterva2.clients.cli:main" -cat2tbrowser = "caterva2.clients.tbrowser:main" +# Main commands with unified naming scheme +cat2-server = "caterva2.services.server:main" +cat2-agent = "caterva2.clients.agent:main" +cat2-client = "caterva2.clients.cli:main" +cat2-admin = "caterva2.tools.admin:main" + +# Utilities for import/export and browsing cat2import = "caterva2.tools.hdf5_to_cat2:main" cat2export = "caterva2.tools.cat2_to_hdf5:main" -cat2adduser = "caterva2.tools.adduser:main" +cat2tbrowser = "caterva2.clients.tbrowser:main" + + [tool.ruff] line-length = 109