diff --git a/docs/API.md b/docs/API.md new file mode 100644 index 0000000..7467a25 --- /dev/null +++ b/docs/API.md @@ -0,0 +1,121 @@ +# Public API Reference + +Reference of the public interfaces exposed by `alloc`. Signatures are +verified against the source. Internal helpers (leading underscore) are +omitted unless noted. + +## alloc.models.networks + +### class ActorCriticNetworks + +DDPG actor-critic pair for portfolio allocation. + + ActorCriticNetworks( + input_dim: int, + num_assets: int, + actor_lr: float = 1e-4, + critic_lr: float = 1e-3, + dropout: float = 0.1, + gamma: float = 0.99, + tau: float = 0.005, + min_cash_allocation: float = 0.0, + buffer_capacity: int = 1_000_000, + seed: int | None = None, + ) + +Attributes: `input_dim`, `num_assets`, `gamma`, `tau`, +`min_cash_allocation`, `dropout`, `actor`, `critic`, `actor_target`, +`critic_target`, `actor_optimizer`, `critic_optimizer`, `replay_buffer`. + +Methods: + +- `get_allocation(state, add_noise=False, noise_scale=0.1) -> np.ndarray` + — public inference entry point. Returns a length-`num_assets` allocation + vector summing to 1.0, last element = cash (>= `min_cash_allocation`). +- `update_critic(...)` / `update_actor(...)` — DDPG training steps. +- `_soft_update_targets()` — soft-update target networks (internal). + +> **Gap (TICKET-052):** no `save_model(directory)` / `load_model(directory)` +> round-trip yet. Only raw `.h5` weights are saved by `alloc.core.main`. + +### class ReplayBuffer + +Fixed-capacity experience replay. `add(...)`, `sample(n)`, `__len__`. + +### class CashLayer / CashLambda + +Keras layers enforcing a minimum cash fraction on the last allocation +element. `get_config()` returns `{"min_cash": ...}`. + +## alloc.models.data + +- `get_multi_asset_data(tickers, client, end_date=None, hourly_days=7, + daily_days=365, weekly_weeks=52) -> dict[str, dict[str, list[float]]]` + — fetch hourly/daily/weekly closes. +- `build_state_vector(multi_freq_data, current_allocation, tickers, + n_hourly, n_daily, n_weekly) -> np.ndarray` — fixed-dim state vector + (normalised price windows + current allocation appended). +- `fetch_latest_prices(tickers, client) -> dict[str, float]` — latest trade + price per ticker (0.0 on error). + +## alloc.models.portfolio + +### class Portfolio + + Portfolio(tickers: list[str], initial_cash: float = 100_000.0, + transaction_cost: float = 0.001) + +Attributes: `tickers`, `cash`, `transaction_cost`, `shares_held` +(zeroed at init), `portfolio_values`. + +Methods: + +- `get_portfolio_value(prices) -> float` +- `get_allocation(prices) -> dict[str, float]` — current weights incl. `'cash'`. +- `execute_trades(target_allocation, prices) -> dict` — rebalance with + shortfall scaling + transaction costs; returns metadata incl. + `scale_factor`, `total_transaction_costs`. + +## alloc.core + +- `SimulationRunner(tickers, initial_value, networks, data_pipeline, client, + transaction_cost, risk_aversion, gamma, tau, diversification_weight, + concentration_penalty, min_cash, batch_size, verbose)` — `.run(trading_days)`. +- `parse_args(argv=None) -> argparse.Namespace` +- `main(argv=None) -> None` — backtest/predict CLI entry point. +- `create_trainer(conservative=False) -> Callable` — trainer factory used by + the workflow CLI. +- `save_results(results, path, mode="backtest")`, + `load_results(path, mode="backtest")`, `serialize_results(results)`. + +## alloc.cli + +Multi-trial training workflow CLI. + +- `build_parser() -> argparse.ArgumentParser` +- `parse_args(argv=None) -> argparse.Namespace` +- `main(argv=None) -> int` — exit codes: 0 success, 1 user error, 2 workflow + failure, 3 unexpected. + +## alloc.utils.workflow + +- `TrainingConfig` — dataclass of training hyperparameters. +- `TrainingTrial` / `WorkflowResult` — result containers. +- `WorkflowRunner(config, trainer)` — `.run() -> WorkflowResult`. + +## alloc.lib + +- `PolygonClient(api_key, cache)` — Polygon.io wrapper (cached). +- `DiskCache(cache_dir, enabled)` — disk cache with per-type TTL. +- `crawl_package(...)` (dashboard) / `generate_html(...)`, `publish(...)` + (publish_dashboard). +- `alloc.lib.utils` — scalar coercion, allocation formatting, price-index + helpers. + +## Planned (TICKET-053) + +- `alloc.lib.rebalance.rebalance_portfolio(model_path, tickers, positions, + client, n_hourly=5, n_daily=5, n_weekly=5, transaction_cost=0.0, + initial_value=None) -> dict` — load trained model, fetch latest prices, + build live state, `get_allocation`, seed `Portfolio` from positions, + `execute_trades`, return recommended orders + post-execution value. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..de4acbe --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,79 @@ +# Architecture + +How `alloc` is structured and how data flows through it. + +## Operating model + +`alloc` is a reinforcement-learning portfolio allocation engine. Each +training run produces a **short-lived model snapshot** tuned to current market +conditions. The workflow is cyclical: ingest fresh data -> spawn candidate +models -> rank by Sharpe / outperformance -> deploy -> repeat. + +There are two distinct entry points: + +1. **Single-run backtest/predict** — `alloc.core.main` (and + `python -m alloc.core`). Runs one `SimulationRunner` over a historical + window and serialises results. +2. **Multi-trial workflow** — `alloc.cli.main` (and `python -m alloc`). + Orchestrates many training trials via `WorkflowRunner`, scoring and ranking + candidates. + +## Data flow (single-run path) + + PolygonClient (alloc.lib.client) + | (DiskCache, alloc.lib.cache) + v + alloc.models.data + get_multi_asset_data() -> hourly/daily/weekly closes + build_state_vector() -> fixed-dim state vector + fetch_latest_prices() -> latest trade prices + | + v + alloc.core.SimulationRunner.run() + | per trading day: + | state = build_state_vector(...) + | action = networks.get_allocation(state) + | portfolio.execute_trades(action, prices) + | reward = composite (return, risk, cost, diversification, concentration) + | networks.update_critic / update_actor + v + results dict -> save_results() -> {path}/results.json | prediction_results.json + (backtest only) actor/critic .h5 weights saved + +## Key design decisions + +- **Injected client, no singleton.** `alloc.models.data` and + `SimulationRunner` receive the `PolygonClient` as an argument, keeping the + data layer testable with a stub client. +- **Fixed-dimension state.** `build_state_vector` normalises each frequency + window by its most-recent price (last element = 1.0) and pads short windows + with zeros, so the state vector has a constant size regardless of history + length. The current allocation is appended. +- **Cash as residual.** The actor emits `num_assets` sigmoid outputs; the + `CashLambda` layer enforces a minimum cash fraction on the last element and + re-normalises to sum 1.0. +- **Soft targets.** DDPG target networks are hard-copied at init and + soft-updated each step with coefficient `tau`. +- **Realistic execution.** `Portfolio.execute_trades` sells first, then scales + buys down proportionally if cash is insufficient (shortfall scaling), and + applies transaction costs on total traded value. + +## Model persistence (current state) + +`alloc.core.main` (backtest mode) writes `actor_weights.h5` and +`critic_weights.h5` to the model directory. **There is no load path and no +config file** — a saved model cannot be re-instantiated. This is the parity +gap being closed by TICKET-052 (add `save_model`/`load_model` that persist +weights + config) and TICKET-053 (a live-rebalance entry point that loads a +trained model and emits recommended orders). + +## Testing + +- `tests/` mirrors the package: `test_actor_critic.py`, `test_data.py`, + `test_portfolio.py`, `test_core.py`, `test_cli.py`, `test_workflow.py`, + `test_cache.py`, `test_client.py`, `test_dashboard*.py`, + `test_cycle_signals.py`, `test_replay_buffer.py`, `test_state_builder.py`, + `test_utils.py`, `test_settings.py`, `test_packages.py`, plus integration + tests (`test_ddpg_integration.py`, `test_cache_settings_integration.py`, + `test_dashboard_integration.py`). +- `tests/conftest.py` provides shared fixtures. diff --git a/docs/MODULES.md b/docs/MODULES.md new file mode 100644 index 0000000..11d980b --- /dev/null +++ b/docs/MODULES.md @@ -0,0 +1,73 @@ +# Module Catalog + +Catalog of every module in the `alloc` package and its relationships. +Line counts and public symbols are verified against the source. + +## Package layout + + alloc/ + ├── __init__.py # package marker, __version__ + ├── __main__.py # `python -m alloc` -> alloc.cli.main + ├── cli.py # multi-trial training workflow CLI + ├── core.py # SimulationRunner + backtest/predict CLI + results I/O + ├── models/ + │ ├── __init__.py + │ ├── data.py # multi-frequency data fetch + state vector build + │ ├── networks.py # DDPG actor-critic + replay buffer + CashLambda + │ └── portfolio.py # Portfolio tracking, trade execution, reward + ├── lib/ + │ ├── __init__.py + │ ├── cache.py # DiskCache + TTL decorator helpers + │ ├── client.py # Polygon.io StocksClient wrapper (cached) + │ ├── cycle_signals.py # terminal tree-view of health signals + │ ├── dashboard.py # crawls package -> JSON health metadata + │ ├── publish_dashboard.py # JSON metadata -> HTML dashboard + │ └── utils.py # scalar coercion, formatting, price-index helpers + └── utils/ + ├── __init__.py + └── workflow.py # TrainingConfig / WorkflowRunner multi-trial orchestration + +## Module responsibilities + +| Module | Responsibility | Key public symbols | +|---|---|---| +| `alloc.core` | DDPG simulation loop; backtest/predict CLI; results serialisation | `SimulationRunner`, `parse_args`, `main`, `create_trainer`, `save_results`, `load_results`, `serialize_results` | +| `alloc.cli` | Multi-trial training workflow CLI (argparse, typed converters, exit codes) | `build_parser`, `parse_args`, `main`, `EXIT_*` | +| `alloc.models.data` | Fetch hourly/daily/weekly prices; build fixed-dim state vectors | `get_multi_asset_data`, `build_state_vector`, `fetch_latest_prices` | +| `alloc.models.networks` | DDPG actor-critic pair, soft targets, replay buffer, cash constraint | `ActorCriticNetworks`, `ReplayBuffer`, `CashLayer`, `CashLambda`, `_calculate_cash` | +| `alloc.models.portfolio` | Holdings tracking, trade execution (shortfall scaling + costs), reward | `Portfolio` | +| `alloc.lib.client` | Polygon.io API wrapper with disk caching | `PolygonClient` | +| `alloc.lib.cache` | Disk-backed cache with per-type TTL | `DiskCache` (+ decorator helpers) | +| `alloc.lib.cycle_signals` | Terminal tree-view of dashboard health signals | (viewer entry points) | +| `alloc.lib.dashboard` | Crawl `alloc/` -> per-module JSON health metadata | `crawl_package` | +| `alloc.lib.publish_dashboard` | Render JSON metadata -> HTML dashboard | `generate_html`, `publish` | +| `alloc.lib.utils` | Scalar-price coercion, allocation formatting, timestamp/price-index helpers | (helper functions) | +| `alloc.utils.workflow` | Multi-trial training orchestration and scoring | `TrainingConfig`, `TrainingTrial`, `WorkflowResult`, `WorkflowRunner` | + +## Dependency graph (intra-package) + + alloc.cli -> alloc.utils.workflow -> alloc.core.create_trainer + alloc.__main__ -> alloc.cli + alloc.core -> alloc.models.networks + -> alloc.models.data + -> alloc.models.portfolio + -> alloc.lib.client -> alloc.lib.cache + alloc.models.networks -> (tensorflow / keras only) + alloc.models.data -> (client injected, no hard dep) + alloc.lib.dashboard -> alloc.lib.publish_dashboard (optional, CLI-driven) + +Notes: +- `alloc.core` is the central orchestrator for the single-run backtest/predict + path; `alloc.cli` is the separate multi-trial workflow entry point. +- `alloc.models.data` receives the client as an injected argument (no + module-level singleton), keeping it testable. +- `alloc.lib.dashboard` / `publish_dashboard` are optional, invoked only when + the CLI `--publish-dashboard` flag is set. + +## Known gaps (tracked as tickets) + +- **TICKET-052** — `ActorCriticNetworks` has no `save_model`/`load_model` + round-trip; only raw `.h5` weights are saved by `alloc.core.main`. +- **TICKET-053** — No live-rebalance entry point; `--predict` runs a fresh + forward simulation rather than loading a trained model and emitting + recommended orders. diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..f98684f --- /dev/null +++ b/docs/README.md @@ -0,0 +1,43 @@ +# alloc — Documentation + +Welcome. This directory documents the `alloc` package for newcomers landing +at the repo. + +## Start here + +1. **README.md** (repo root) — what `alloc` is, install, usage, philosophy. +2. **docs/MODULES.md** — catalog of every module and its relationships. +3. **docs/ARCHITECTURE.md** — how the system is structured and how data flows. +4. **docs/API.md** — reference of the public interfaces. + +## Quick orientation + +- **Two entry points:** + - `python -m alloc.core --backtest --tickers AAPL,META` — single-run + backtest/predict. + - `python -m alloc --tickers AAPL,META --iterations 5` — multi-trial + training workflow. +- **Core loop:** fetch multi-frequency data -> build a fixed-dim state vector + -> DDPG actor emits an allocation -> portfolio executes trades -> composite + reward -> update actor/critic. +- **Models are short-lived snapshots** tuned to the current regime; retrain + when the regime shifts. + +## Open work (parity gap) + +The seed's predict mode is a true live-rebalance (load trained model -> +recommended orders). `alloc` does not yet have that. Tracked as: + +- **TICKET-052** — model persistence round-trip + (`ActorCriticNetworks.save_model` / `load_model`). +- **TICKET-053** — live-rebalance entry point + (`alloc.lib.rebalance.rebalance_portfolio` + CLI wiring). + +See `tickets/` for full detail. + +## Conventions + +- The client is **injected**, never a module-level singleton. +- Public symbols are documented in `docs/API.md`; internal helpers use a + leading underscore. +- Every module has a module-level docstring; public functions are documented. diff --git a/tickets/TICKET-052.md b/tickets/TICKET-052.md new file mode 100644 index 0000000..176d0c3 --- /dev/null +++ b/tickets/TICKET-052.md @@ -0,0 +1,70 @@ +# TICKET-052: Model persistence round-trip for ActorCriticNetworks + +**Status:** OPEN +**Cycle:** 43 +**Priority:** High +**Target module:** `alloc/models/networks.py` — `ActorCriticNetworks` + +## Summary + +Backtest mode (`alloc/core.py` `main`) saves `actor_weights.h5` and +`critic_weights.h5` after training, but there is **no load path**: a saved +model cannot be re-instantiated. The seed's predict mode loads a previously +trained model before producing a live allocation. To close that parity gap +(see TICKET-053), `ActorCriticNetworks` needs a save/load round-trip that +persists both the weights **and** the config needed to reconstruct the +network (input_dim, num_assets, min_cash_allocation, and the hyperparameters +that affect architecture: dropout, gamma, tau, actor_lr, critic_lr). + +## Evidence + +- `alloc/core.py` lines 734-735: `networks.actor.save_weights(...)` / + `networks.critic.save_weights(...)` — save only, never load. No config file + is written, so even the dimensions are not persisted. +- `alloc/models/networks.py` `ActorCriticNetworks.__init__` (lines 285-340) + builds actor/critic/targets/optimizers from `input_dim`, `num_assets`, + `min_cash_allocation`, `dropout`. There is no `save_model`/`load_model` + method and no config file is written. +- Config is stored as instance attributes at lines 304-309: + `self.input_dim`, `self.num_assets`, `self.gamma`, `self.tau`, + `self.min_cash_allocation`, `self.dropout`. +- **Gap (refined):** `actor_lr` and `critic_lr` are **not** stored as + instance attributes. They are only used to construct the optimizers at + lines 322-323 (`keras.optimizers.Adam(learning_rate=actor_lr)` / + `...critic_lr`). So `save_model` cannot read them back from the instance + without either (a) adding `self.actor_lr` / `self.critic_lr` in `__init__`, + or (b) reading `self.actor_optimizer.learning_rate`. Option (a) is cleaner + and is required for the round-trip to be lossless. +- The actor architecture depends on `input_dim` and `num_assets` (per-asset + branch widths scale with the asset index, lines 381-388: `w1 = 32 + i*4`, + `w2 = 16 + i*2`), and the cash constraint depends on + `min_cash_allocation` (line 408). Loading weights into a freshly-built + network with the wrong dimensions will fail or silently mismatch. The + config must be persisted alongside the weights. + +## Implementation plan + +1. **`__init__` change:** store `self.actor_lr = actor_lr` and + `self.critic_lr = critic_lr` (lines ~304-309) so the learning rates are + part of the persisted config. +2. **`ActorCriticNetworks.save_model(directory)`** — write: + - `actor_weights.h5`, `critic_weights.h5` (via `keras` `save_weights`). + - `model_config.json` with `input_dim`, `num_assets`, + `min_cash_allocation`, `dropout`, `gamma`, `tau`, `actor_lr`, + `critic_lr`. +3. **`ActorCriticNetworks.load_model(directory)`** (classmethod) — read + `model_config.json`, construct an `ActorCriticNetworks` with those + parameters, then `actor.load_weights` / `critic.load_weights`, and + re-sync the target networks (`actor_target.set_weights`, + `critic_target.set_weights`). Return the instance. +4. Raise `FileNotFoundError` with a clear message if `model_config.json` or + either weights file is missing. + +## Verification + +- `pytest tests/test_actor_critic.py -x -q` — new round-trip test passes: + build a small network, save to a tmp dir, load, assert + `get_allocation(state)` is identical (or near-identical) before/after and + that all config fields (incl. `actor_lr`/`critic_lr`) round-trip. +- `ruff check alloc/models/networks.py` — clean. +- `mypy alloc/models/networks.py --ignore-missing-imports` — clean. diff --git a/tickets/TICKET-053.md b/tickets/TICKET-053.md new file mode 100644 index 0000000..3d91bb9 --- /dev/null +++ b/tickets/TICKET-053.md @@ -0,0 +1,104 @@ +# TICKET-053: Live-rebalance entry point (load trained model → recommended orders) + +**Status:** OPEN +**Cycle:** 43 +**Priority:** High +**Depends on:** TICKET-052 (model persistence round-trip) +**Target module:** new `alloc/lib/rebalance.py` + `alloc/core.py` CLI wiring + +## Summary + +The seed's predict mode is a **true live-rebalance**: it loads a previously +trained actor-critic model, fetches the latest market prices, constructs the +current market state, calls the model's public `get_allocation` to produce a +recommended allocation, executes the trades against the current portfolio +(transaction costs + shortfall scaling), and emits recommended orders plus the +post-execution portfolio value. + +alloc/ has **no equivalent**. Its `--predict` CLI flag (`alloc/core.py` +`main`) just runs a fresh `SimulationRunner.run()` and saves +`prediction_results.json`; it never loads a trained model, never fetches +latest prices to build a live state, and never emits recommended orders from a +trained model. + +## Evidence + +- `alloc/core.py` `main` (lines 635-745): `--predict` sets `mode_str` + (line 655) but the code path is identical to backtest except it skips saving + weights (lines 733-735 only run under `if args.backtest`). No model load, no + latest-price fetch, no `get_allocation` on a live state. +- `alloc/models/data.py` already provides the pieces: + - `get_multi_asset_data(tickers, client, ...)` (line 161) + - `build_state_vector(multi_freq_data, current_allocation, tickers, + n_hourly, n_daily, n_weekly)` (line 255) + - `fetch_latest_prices(tickers, client)` (line 334) +- `alloc/models/portfolio.py`: + - `Portfolio.__init__(tickers, initial_cash=100_000.0, + transaction_cost=0.001)` (line 49) — note `shares_held` is zeroed at init + (line 64), so seeding from live positions requires mutating + `portfolio.shares_held` and `portfolio.cash` after construction. + - `Portfolio.get_allocation(prices)` (line 72) — current weights incl. cash. + - `Portfolio.execute_trades(target_allocation, prices)` (line 93) — rebalance + execution with shortfall scaling and transaction costs; returns a dict with + `scale_factor` and `total_transaction_costs`. +- `alloc/models/networks.py` `ActorCriticNetworks.get_allocation(state)` + (line 488) is the public inference entry point (returns a length- + `num_assets` vector, last element = cash). +- The only missing glue is: load model → fetch data → build state → + `get_allocation` → seed a `Portfolio` from current positions → + `execute_trades` → return recommended orders + post-execution value. + +## Implementation plan + +1. **New module `alloc/lib/rebalance.py`** with a + `rebalance_portfolio(...)` function (and a small dataclass or dict result): + - Inputs: `model_path`, `tickers`, `positions` (dict ticker→dollar value), + `client`, `n_hourly=5, n_daily=5, n_weekly=5`, `transaction_cost=0.0`, + `initial_value` (optional; derived from positions + cash if omitted). + - Steps: + a. `networks = ActorCriticNetworks.load_model(model_path)` (TICKET-052). + b. `prices = fetch_latest_prices(tickers, client)`. + c. `multi_freq = get_multi_asset_data(tickers, client)`. + d. Build `current_allocation` (non-cash weights) from `positions` + + `prices` (value / total, cash = remainder). + e. `state = build_state_vector(multi_freq, current_allocation, tickers, + n_hourly, n_daily, n_weekly)`. + f. `allocation = networks.get_allocation(state)`. + g. Seed a `Portfolio(tickers, initial_cash=initial_value, + transaction_cost=...)`; then set `portfolio.shares_held[t] = + positions[t] / prices[t]` and `portfolio.cash = initial_value - + sum(positions.values())` (Portfolio zeroes shares at init, line 64). + h. `target = {t: allocation[i] for i, t in enumerate(tickers)}; + target['cash'] = allocation[-1]`. + i. `execution = portfolio.execute_trades(target, prices)`. + - Return a dict with: `recommended_allocation` (ticker→weight incl. cash), + `recommended_orders` (list of {ticker, action, shares, price, value}), + `portfolio_value_before`, `portfolio_value_after`, + `total_transaction_costs`, `scale_factor`. +2. **CLI wiring in `alloc/core.py`**: add a `--rebalance` mode (or a + `--positions` + `--model-path` predict path) that calls + `rebalance_portfolio` and logs the recommended orders, mirroring the seed's + predict output. Keep the existing `--predict` (forward simulation) intact. +3. **Tests** (`tests/test_rebalance.py`): use a fake client (patch + `fetch_latest_prices`/`get_multi_asset_data` at the module level via + `patch.object` on the rebalance module's imported references, or inject a + stub client) and a small saved model (via TICKET-052 `save_model`) to + assert: allocation sums to 1.0, cash >= min_cash, orders are produced, + post-execution value is finite, and transaction costs are applied. + +## Verification + +- `pytest tests/test_rebalance.py -x -q` — all pass. +- `pytest tests/ -x -q` — full suite green. +- `ruff check alloc/` — clean. +- `mypy alloc/ --ignore-missing-imports` — clean. + +## Notes + +- **Semantics reference:** read `~/Research/new-trader/trader/core.py` + (predict mode, ~lines 863-935) and + `~/Research/new-trader/utils/trader_tools/multi_rebalance_helper.py` + (`rebalance_portfolio`, `positions_to_allocation`) for understanding only. + Nothing is copied; the implementation targets alloc's existing public API + (`get_allocation`, `build_state_vector`, `fetch_latest_prices`, + `execute_trades`).