Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 3 additions & 15 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: install assets lite-build lite-dev lite-test bro pub-dir pub-color pub-gris sub
.PHONY: install assets lite-build lite-dev lite-test run

VENV = ./venv
BIN = $(VENV)/bin
Expand Down Expand Up @@ -38,18 +38,6 @@ lite-test:
$(MAKE) lite-build


# To run the different services, for convenience
bro:
${BIN}/python3 -m caterva2.services.bro

pub-dir:
${BIN}/python3 -m caterva2.services.pub --id dir

pub-color:
${BIN}/python3 -m caterva2.services.pub --id color

pub-gris:
${BIN}/python3 -m caterva2.services.pub --id gris

sub:
# To run the server, for convenience
run:
BLOSC_TRACE=1 ${BIN}/python3 -m caterva2.services.sub
36 changes: 2 additions & 34 deletions README-DEVELOPERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ As the config files are already there, this essentially boils down to:
pre-commit install
```

Also, for running the tests, one needs to run manually the broker, publisher and subscriber.
Also, for running the tests, one needs to run manually the subscriber.
There is a `caterva2.tests.services` script that does this.

## Build CSS and JS
Expand Down Expand Up @@ -67,33 +67,6 @@ USE_REQUESTS=1 CATERVA2_SECRET=c2sikrit pytest
```
```

### With external daemons

To have daemons running across several test runs (for faster testing), start the daemons:

```shell
python -m caterva2.tests.services &
```

or, if you prefer:

```shell
cat2bro &
cat2pub foo root-example &
cat2sub &
```

State files will be stored in dir `_caterva2/`.

Finally, in another shell (unless you like to hear the daemons chatting), run the tests:

```shell
env CATERVA2_USE_EXTERNAL=1 python -m pytest -s
```

For stopping the daemons, you will have to kill the `caterva2.tests.services` process.
If you started them manually, you will have to kill them manually too (sorry!).

## Build wheels

We are using [hatch](https://hatch.pypa.io) as the build system, so for building wheels and
Expand All @@ -118,12 +91,7 @@ cd .. # to avoid using the source code
python -m caterva2.tests -v
```

Please note that the services should be not running at this point. In case you want to check against
the current services, you can do:

```shell
env CATERVA2_USE_EXTERNAL=1 python -m caterva2.tests -v
```
Please note that the services should be not running at this point.

## Create docs

Expand Down
148 changes: 0 additions & 148 deletions README-PUBSUB.md

This file was deleted.

13 changes: 3 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,14 +96,6 @@ python -m pytest -v

Tests will use a copy of Caterva2's `root-example` directory. After they finish, state files will be left under the `_caterva2_tests` directory for inspection (it will be re-created when tests are run again).

In case you want to run the tests with your own running daemons, you can do:

```shell
env CATERVA2_USE_EXTERNAL=1 python -m caterva2.tests -v
```

Neither `root-example` nor `_caterva2_tests` will be used in this case.

## Quick start

(Find more detailed step-by-step [tutorials](Tutorials) in Caterva2 documentation.)
Expand All @@ -123,9 +115,10 @@ dir1/ ds-1d-b.b2nd ds-1d.b2nd ds-hello
Now:

- create a virtual environment and install Caterva2 with the `[subscriber,clients]` extras (see above).
- copy the configuration file `caterva2-standalone.sample.toml` to `caterva2.toml`.
- copy the configuration file `caterva2.sample.toml` to `caterva2.toml`.

For more advanced configuration options, see the fully documented `caterva2.sample.toml` file (see also [caterva2.toml](caterva2.toml) in Caterva2 tutorials). Subscribers (and clients, to a limited extent) may get their configuration from a `caterva2.toml` file at the current directory (or an alternative file given with the `--conf` option).
Subscribers (and clients, to a limited extent) may get their configuration from a `caterva2.toml` file at the current directory (or an alternative file given with the `--conf` option).
See also [configuration.md](configuration.md) in Caterva2 tutorials.

Then run the subscriber:

Expand Down
2 changes: 1 addition & 1 deletion RELEASING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ installing the wheel, then test it::
Check that the examples in docstrings are up to date. You will
need to register a user in https://cat2.cloud/demo/ with
username 'user@example.com' and password 'foo'. Then, copy
the content of ``caterva2-standalone.sample.toml`` to ``caterva2.toml``
the content of ``caterva2.sample.toml`` to ``caterva2.toml``
and run the following commands::

$ rm -r _caterva2/
Expand Down
29 changes: 0 additions & 29 deletions caterva2-standalone.sample.toml

This file was deleted.

71 changes: 23 additions & 48 deletions caterva2.sample.toml
Original file line number Diff line number Diff line change
@@ -1,54 +1,29 @@
# Example configuration file for Caterva2 components.
# Example configuration for a standalone subscriber
#
# This may be parsed by different programs, and each program may look up settings in its own section, or in other programs' sections, if present. For instance, there is no setting in the ``subscriber`` section for the broker endpoint; instead, the subscriber program will look ``broker.http`` up. For instance, in a subscriber configuration file::
#
# [broker]
# http = ... # Broker HTTP endpoint, to be used by subscriber.
# # No need for more broker settings unless the broker is to use this file.
#
# [subscriber]
# ...
# It's possible to run only the subscriber. Then the configuration has only a
# section for the subscriber. And maybe another one for the client.

# The subscriber section must define:
#
# Some sections may appear multiple times, each with a different ID (see below). However, if you are to use a single program of each category, you should be file with ID-less sections.
# - statedir: the directory where the subcriber's data will be stored (default: _caterva2/sub)
# - http: where the subscriber listens to (a unix socket or a host/port) (default: localhost:8002)
# - urlbase: the base url users will use to reach the subscriber (default: http://localhost:8002)
# - quota: if defined, it will limit the disk usage (default: 0, no limit)
# - maxusers: if defined, it will limit the number of users (default: 0, no limit)
# - login: if true, users will need to authenticate (default: true)
# - register: if true, users will be able to register (default: false)
#
# All sections and settings are optional.

# The configuration of the broker service.
# Only one of these is allowed for the moment.
[broker]
http = "localhost:8000" # The ``host:port`` endpoint where the service listens for HTTP requests. Other programs may look ``broker.http`` up to find how to connect to a broker. Use ``*`` as a host to listen on all addresses.
statedir = "_caterva2/bro" # The directory where the service will place state files.
loglevel = "warning" # All service messages having this severity or worse will be logged.

# The configuration of the publisher service.
# Several of these are allowed, each with a different ID (the string after the dot). A publisher invoked with ``--id=something`` will look its configuration up in the ``publisher.something`` section.
[publisher.1]
http = "localhost:8001" # The ``host:port`` endpoint where the service listens for HTTP requests. Use ``*`` as a host to listen on all addresses.
statedir = "_caterva2/pub" # The directory where the service will place state files.
loglevel = "warning" # All service messages having this severity or worse will be logged.
name = "foo" # The name given to the root to be registered at the broker. This setting has no default, if it is not defined here, you need to give it to the publisher as an argument.
root = "root-examples" # The location (directory, HDF5 file...) containing the datasets for the registered root.

# Only one of these is allowed. It will be used by a publisher invoked with no ID, or it may be used by other programs to find how to connect to a publisher (``publisher.http``).
[publisher]
http = "localhost:8001"
# ... other settings as above ...

# The configuration of the subscriber service.
# Several of these are allowed, each with a different ID (the string after the dot). A subscriber invoked with ``--id=something`` will look its configuration up in the ``subscriber.something`` section.
[subscriber.1]
http = "localhost:8002" # The ``host:port`` endpoint where the service listens for HTTP requests. Use ``*`` as a host to listen on all addresses.
urlbase = "https://cat2.example.com" # The base of URLs, if different from ``http://<subscriber.http>``.
statedir = "_caterva2/sub" # The directory where the service will place state files.
loglevel = "warning" # All service messages having this severity or worse will be logged.

# Only one of these is allowed. It will be used by a subscriber invoked with no ID, or it may be used by other programs to find how to connect to a subscriber (``subscriber.url``).
[subscriber]
urlbase = "https://cat2.example.com"
# ... other settings as above ...
statedir = "_caterva2/sub"
#http = "_caterva2/sub/uvicorn.socket"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that still working? If so, maybe it is worth a simple line what is that.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, they are described above:

# - statedir: the directory where the subcriber's data will be stored (default: _caterva2/sub)
# - http: where the subscriber listens to (a unix socket or a host/port) (default: localhost:8002)
# - urlbase: the base url users will use to reach the subscriber (default: http://localhost:8002)

http = "localhost:8002"
urlbase = "http://localhost:8002"
quota = "10G"
maxusers = 5
register = true # allow users to register

# Common configuration of client programs.
# Only one of these is allowed for the moment.
# The client section defines the credentials for the client to authenticate
# against the subscriber.
[client]
username = "" # If present and not empty, a name to be used to authenticate the user to the subscriber and get an authorization token.
password = "" # If present and not empty, a password for the previous user.
username = ""
password = ""
Loading