copilot-instructions.md

Copilot instructions (Cacti develop)

Big Picture

Cacti is a robust, extensible network monitoring and graphing tool built on the LAMP stack.

• Core Purpose: Polls devices (SNMP, scripts), stores time-series data in RRDtool, and renders historical graphs for network operations.

• Architecture:

  • Frontend: PHP 8.3+ web UI for configuration and visualization.
  • Backend: MySQL/MariaDB for metadata/state/configuration/statistics; RRD files for metrics.
  • Polling: Scalable data collection via cmd.php (PHP) or spine (C), supporting remote pollers.

• Key Components:

  • Templates: Abstraction for Graphs, Data Sources, and Devices to simplify management.
  • Plugins: Extension system for adding features (e.g., Thold, Weathermap).
  • Automation: Rules for discovering and adding devices automatically.

Entry points & Architecture

• Web: Pages start with include/auth.php (loads include/global.php, enforces auth).
• CLI: Scripts load include/cli_check.php (enforces CLI context).
• Polling & Data Flow (Principles of Operation):
  • Hierarchy: Sites > Data Collectors > Devices > Graphs.
  • Collection: poller.php (cron) or cactid.php (daemon) fetch data via SNMP, script_server.php (resident PHP pool), or external scripts.
  • Storage: Metadata in MySQL/MariaDB; Metrics in RRDfiles (.rrd).
  • Visualization: RRDtool renders graphs (see graph_image.php).
• API: REST endpoints in api/ (Slim framework).
• CLI Scripts: Administrative scripts located in cli/
• SNMP Query Templates: XML templates with snmp OID's to poll located in resource\snmp_queries

Project conventions (follow these)

• Coding Style:
  • Indentation: Tabs (4 spaces width).
  • Braces: Start on the same line (if (...) {).
  • Quotes: Single quotes ' preferred over double " unless interpolating.
  • Constants: true, false, null should be lowercase.
  • Arrays: Use foreach over while(list() = each()).
• Database (MariaDB/Mysql):
  • Use helpers in lib/database.php; prefer prepared calls like db_fetch_row_prepared() / db_fetch_assoc_prepared() / db_execute_prepared().
  • Do NOT quote numeric values in SQL. Use single quotes for strings.
  • Always use AS for aliases.
  • Use explicit INNER JOIN, LEFT JOIN, etc. (no implicit joins).
• Request handling (web):
  • Avoid direct $_GET/$_POST/$_REQUEST.
  • Use get_request_var() / get_filter_request_var() and form_input_validate().
  • Validation: Use validate_store_request_vars() where possible.
  • Request variable caching (aliases in lib/html_utility.php):
    • gfrv($name) / get_filter_request_var() — validates input (default: integer via filter_var), stores the sanitized value in $_CACTI_REQUEST, and returns it.
    • grv($name) / get_request_var() — retrieves from $_CACTI_REQUEST cache. Returns pre-validated data if gfrv() was called first for that variable.
    • gnrv($name) / get_nfilter_request_var() — retrieves from cache without filtering. Safe if gfrv() was called first.
    • srv($name, $value) / set_request_var() — stores a value in $_CACTI_REQUEST, $_REQUEST, $_POST, and $_GET.
    • Do NOT flag grv()/gnrv() calls as unsanitized when gfrv() was already called for that variable in the same request. The validation happens once at the gfrv() call site; subsequent retrievals return cached, pre-validated data.
    • Adding redundant (int) casts or parameterization after gfrv() has validated is unnecessary.
• Common page flow: set_default_action(); switch (get_request_var('action')) { ... } and render via top_header()/bottom_footer().
• CSRF: AJAX posts include __csrf_magic: csrfMagicToken.
• Logging: use cacti_log(...) and cacti_log_file().
• i18n: wrap UI strings with __('...').
  • Translatable strings are managed with GNU gettext. locales/po/cacti.pot is the source template; Weblate owns syncing the per-language .po/.mo files.
  • Any PR that adds, changes, or removes a __('...') (or __n()/__esc() etc.) string MUST update locales/po/cacti.pot in the same PR. This is mandatory — a PR that touches translatable strings without updating cacti.pot is incomplete. Removing a string counts: it must be dropped from the template too.
  • Never commit the per-language .po or compiled .mo files — locales/po/cacti.pot is the only translation artifact a PR may add or modify. Stage only that file.
  • Regenerate with locales/update-pot.sh (it runs xgettext), or, to keep the diff focused, add/remove just the affected msgid entries. Validate with msgfmt --check-format -o /dev/null locales/po/cacti.pot.
  • If update-pot.sh rewrites .po/.mo side effects, revert them (git checkout -- locales/po/*.po locales/LC_MESSAGES) so the PR touches cacti.pot only.
• Plugins:
  • Hooks via api_plugin_hook(...) in lib/plugins.php.
  • Must have an INFO file (INI format).
  • Use top_header()/bottom_footer() (no direct includes).
  • Add &header=false to URLs for AJAX requests.
• CHANGELOG:
  • Within each release section, keep -issue entries first and -feature entries after them.
  • Append new issue entries at the end of the issue block and new feature entries at the end of the feature block.
  • Keep numbered entries in their existing numeric order; every new entry must include an issue or pull-request number (#1234).
  • Preserve the existing entry prefixes, release headings, and blank-line spacing.
• URLs and redirects:
  • Use cacti_url($path, $params) for RFC 3986 query-string construction, including URLs with fragments.
  • Use cacti_redirect($path, $params) for local redirects; it rejects absolute and protocol-relative destinations (checking both raw and percent-decoded forms) and terminates the request.
  • Escape output for HTML attribute contexts with html_escape_attr(), not htmle()/html_escape(); only the attribute helper double-encodes to stop pre-encoded entities from breaking out of the attribute.

Maintenance fix scope

Keep maintenance fixes focused on the reported defect and preserve branch compatibility.

• Before adding persistent caches, staging tables, locks, shutdown handlers, or invalidation hooks across management APIs, show why the existing mechanism cannot support the fix. Use a concrete failure case from the target branch's implementation and its callers.
• Identify the authoritative data, the derived result, and the exact operations that can change that result. Add invalidation hooks only for those operations. A component that displays or combines existing data does not automatically own its lifecycle.
• If a cache design requires hooks across unrelated subsystems, reconsider the cache design before expanding the patch. Do not use dependencies introduced by the proposed design as proof that the original bug requires that scope.
• Keep architectural changes in a separate proposal unless required for correctness. Before expanding a focused fix across unrelated subsystems, explain the dependency with a concrete example and the smallest viable alternative. Continue work already within scope.
• Tests must validate intended product behavior and realistic failures. Passing tests or tests that require every proposed hook do not establish that the design or expanded scope is necessary.
• For maintenance branches such as 1.2.x, prefer the smallest compatible fix. Avoid unrelated refactors, new dependencies, and expanded CI requirements.

Pull request pre-flight (MANDATORY)

• Before creating any pull request you MUST run composer run-script php-cs-fixit to apply the repo's php-cs-fixer formatting rules, then commit any resulting changes. This script fixes in place (unlike the dry-run composer phpcsfixer), so creating a PR without running it is what trips the coding-standards job in CI. Run it from WSL (see Windows/WSL note below). Do not open the PR until the working tree is clean after the fixer runs.
• Every commit must also pass PHPStan static analysis. Run composer run-script phpstan -- --memory-limit=2G from WSL and confirm it reports No errors before opening the PR. The default 128M limit crashes a parallel worker, so always pass --memory-limit=2G (or higher). The repo config (.phpstan.neon) analyses at level 8, which is cumulative and therefore satisfies the Level 6 floor; tests/* is excluded from analysis, so test-only files will not surface here but production code (e.g. lib/) will.

Workflows you’ll actually use

• Windows/WSL: When working in VS Code from a Windows machine, run all fixes, builds, linters, tests, and git operations inside WSL (a Linux distro) rather than native Windows. The toolchain (PHP 8.3, Composer, php-cs-fixer, Pest) and the repo's tab indentation / \n line-ending conventions are Linux-first; running them on native Windows produces spurious diffs and failures. Edit the WSL-mounted checkout (e.g. under /mnt/c/... or a native WSL path) and invoke composer scripts such as composer php-cs-fixit from the WSL shell.
• Install deps: composer install (CI validates via .github/workflows/syntax.yml).
• Install/upgrade DB: php -q cli/install_cacti.php --accept-eula --install --force and php -q cli/upgrade_database.php --forcever=$(cat include/cacti_version).
• Run poller: php poller.php --poller=1 --force --debug (daemon debug: ./cactid.php --foreground --debug).
• Repo checks: composer lint, composer phpstan, composer phpcsfixer (dry-run).
• Refresh security baselines when the sink or hotspot guard reports drift: tests/security/build_sink_inventory.sh | tr -d '\r' | LC_ALL=C sort > tests/security/baselines/sink_inventory.baseline.tsv and the matching build_architectural_helper_report.sh --hotspots for architectural_hotspots.baseline.tsv. Use the command the verifier prints rather than editing the file. The verifiers strip :LINE before comparing, so most of the resulting diff is renumbering they ignore; say in the pull request how many entries are substantive.

Dependency management: never vendor packages

• Do not add, copy, or update third-party package source under include/vendor/ or elsewhere in the repository. Existing vendored dependencies may be removed by an in-scope change, but must not be expanded or refreshed.
• Add PHP dependencies through Composer: update composer.json, update and commit composer.lock, and let composer install install the locked packages in development, CI, and release builds.
• Do not commit Composer-installed package trees or hand-edit Composer-generated autoload metadata. Validate dependency changes with composer validate and a clean composer install.
• If a third-party asset cannot be managed through Composer, stop and document the concrete packaging requirement before adding files. Do not silently create a vendored exception.

Polling and script-server performance

Treat polling performance as an architectural constraint when changing poller.php, cmd.php, script_server.php, polling scripts, SNMP helpers, subprocess wrappers, lib/poller.php or RRD update paths. Cacti installations can have millions of data sources. Trace the actual call chain before adding work: script-server launch, script include, function call, SNMP query, process execution and individual SNMP requests. Preserve existing process reuse and avoid introducing repeated includes, launches or network requests per sample.

• Resolve shared metadata at template scope and cache it once per process or explicitly defined polling cycle. Cache empty results with a separate loaded flag or equivalent sentinel; an empty array must not cause another query on every pass. Define invalidation for long-lived workers and configuration changes so the cache does not serve stale metadata indefinitely.
• Do not query or scan individual data sources inside every loop to answer a template-level question. Do not preload millions of source records as a substitute. Source-specific state, ownership, graph bindings, orphan handling and permissions still require correct source-specific checks; sharing a template does not make those values interchangeable. Batch necessary reads and keep cache cardinality and memory bounded.
• Do not add fixed sleeps to each subprocess or SNMP call. Prefer readiness waiting and monotonic deadlines where supported, while preserving active output handling, silent-child timeouts, exit status and resource cleanup. Measure overhead relative to direct execution. An extra 50 ms across 20,000 serialized calls adds approximately 1,000 seconds before useful work.
• Confirm the implementation on the target branch. cacti_exec() and transport implementations differ across release branches; a fix or test for one branch must not be assumed applicable to another.
• Treat credential caching as a separate optimization. Measure hit rates, avoided lookups, time and memory before claiming a benefit. Preserve device, poller and credential identity, invalidation and secret-handling boundaries; never log secrets as performance evidence. Credential caching does not fix repeated template queries or per-call process delays.
• For changes to these paths, run applicable polling performance regressions. Instrument query and call counts over repeated passes, including empty-cache results. Compare subprocess wrapper overhead with direct execution and cover streaming output, silent-child timeouts and exit codes. Exercise representative synthetic scales, including 150k, 1M and 2.5M sources where the branch's harness supports them, and record runtime, elapsed time and peak memory. Keep timings from hosted runners diagnostic; use reproducible count-based regressions and tests that detect reintroduced repeated queries or fixed delays.
• Label synthetic PHP processing results as synthetic. They do not establish real RTM throughput, database query plans, SNMP/Spine performance, RRD I/O, remote replication or full Boost behavior. Production-scale claims require a representative end-to-end benchmark, including those costs.

Database Optimization (DBA Mode)

• Context: The full schema (DDL/DML) is in cacti.sql. Check it for table structures and indexes.
• Optimize: Proactively look for slow query patterns (e.g., missing indexes, non-sargable WHERE clauses).
• Best Practices:
  • Prefer JOIN over subqueries where efficient.
  • Ensure columns in WHERE/JOIN clauses are indexed (check cacti.sql).
  • Avoid SELECT * in production code; list columns explicitly.

Security Advisor (AppSec Mode)

• Context: Cacti Takes Security Seriously consider the security implications when reviewing current and new code base
• Vulnerability Checks:
  • SQL Injection: STRICTLY enforce prepared statements (db_*_prepared). Flag any variable interpolation in SQL strings.
  • XSS: Ensure user input is sanitized/escaped before output (use htmlspecialchars or Cacti's wrappers).
  • CSRF: Verify forms/AJAX include __csrf_magic.
  • Command Injection: Scrutinize shell_exec, exec, passthru. Ensure arguments are escaped (escapeshellarg).
  • Auth: Verify include/auth.php is loaded and permissions are checked (api_user_realm_auth).
• Security advisories and CVE policy:
  • develop/1.3.x is unreleased. A vulnerability that exists only on develop does not get a CVE requested, since no released version is affected — document that determination in the GHSA description (a dated "Triage Note") instead.
  • 1.2.x is the released/LTS branch. If the same vulnerable code is also reachable on 1.2.x (or was historically, for an unpatched released version), a CVE is requested.
  • Before deciding CVE vs. no-CVE, verify the claim against both branches' actual current source rather than trusting the reporter's stated affected-version range — it can be wrong (e.g. describing a bug already fixed on 1.2.x that only regressed on develop).
  • If a develop-only advisory is later found to also affect a released 1.2.x version, request a CVE at that point.

Clean-as-you-code

• While implementing a change, do small, behavior-preserving refactors in the touched area (dedupe logic, extract helpers, simplify conditions).
• Formatting: Enforce tabs for indentation (no spaces for indentation).
• Legacy: Avoid preg_* if simple string functions work. Use preg_* over ereg_*.
• Avoid drive-by rewrites: no repo-wide formatting, no unrelated cleanup, and preserve existing file conventions (keep the GPL header blocks).

Unit test organization (Pest)

• Unit tests live in tests/Unit/ and use Pest (test() / it() functions, not PHPUnit classes). The Pest bootstrap is tests/Pest.php; a lightweight source-scan bootstrap is tests/bootstrap-unit.php.

• Tests are organized into category subfolders by domain. New tests must follow this structure — do not add flat files to tests/Unit/ root:

  • Security/ — security-related tests, subfoldered by vulnerability class (SqlInjection/, Xss/, Shell/, Auth/, Csrf/, Csp/, Redirect/, PathTraversal/, Ssrf/, Crypto/, InputValidation/, Session/, Ldap/, Type/, ShellInjection/, Misc/).
  • DataCollection/ — poller/SNMP/cmd/data-query tests (Poller/, Snmp/, Cmd/, DataQuery/, DataInput/).
  • Ui/ — front-end/graph rendering tests (Aggregate/, Graph/, Html/, Theme/, Colour/, Tree/, Spikekill/, Utility/).
  • Core/ — core library tests, with subfolders for distinct subsystems (Rrd/, RemoteAgent/, Automation/, Cli/, Mailer/, Boost/, Availability_Tests/).
  • Database/, Installer/, Scripts/, Plugin/ — domain-specific folders at the top level.

• When adding a new test, place it in the matching category folder. If a new domain is needed, create a folder for it rather than leaving the file flat in tests/Unit/.

• File naming: do not encode GitHub issue or GHSA numbers in filenames. Name the file after the behavior it tests (e.g. PercentileContractTest.php, not Issue7070PercentileContractTest.php). The GHSA/issue ID belongs in the test's test('...') description string or a file-level docblock, not the filename.

• __DIR__ path depth: the depth depends on both the folder's nesting level and on what the path points at. Paths that reach the repository root need one more level than paths that stop at tests/, so a single file often needs two different depths.

  test file location	to reach repo root (lib/, include/)	to reach tests/ (Helpers/, fixtures/, tools/)
  tests/Unit/	dirname(__DIR__, 2)	dirname(__DIR__)
  tests/Unit/<Category>/	dirname(__DIR__, 3)	dirname(__DIR__, 2)
  tests/Unit/<Category>/<Subcategory>/	dirname(__DIR__, 4)	dirname(__DIR__, 3)

  • After moving a test between folders, update every path in it, and check each one individually. Setting them all to the repo-root depth silently breaks the tests/-internal ones. A require that resolves to a missing file is fatal during bootstrap and stops the whole suite rather than failing one test.
  • Verify a move by running the suite, not by reading the diff.

Pending upstream dependency: select2 multi-select dropdown search

• include/layout.js (select2-multi-count init, both this branch and feature/select2-select-conversion-1.2.x) works around select2 4.1.0 not natively supporting "search from the dropdown panel instead of an inline box" for multi-selects. It manually builds custom dropdownAdapter/selectionAdapter combinations via $.fn.select2.amd.require('select2/...') (synchronous string-form require - see the comment at that call site for why array-form must not be used).
• TheWitness has an upstream pull request open against select2/select2 proposing this behavior as a first-class option: https://github.com/select2/select2/pull/6462
• Action item for any future Copilot session working in this repo: when starting work here, check whether a select2 release has shipped that includes that PR (check select2/select2 releases/CHANGELOG for a first-class dropdown-search-for-multiple-selects option). If it has:
  1. Bump the vendored include/js/select2.js to that release.
  2. Replace the manual adapter-composition workaround in include/layout.js with the new native option on both feature/select2-select-conversion (develop) and feature/select2-select-conversion-1.2.x (1.2.x), or their successor branches/develop/1.2.x if already merged.
  3. Remove the now-unnecessary multiCountDropdownAdapter/ multiCountSelectionAdapter wiring and related comments once the native option covers it.
  4. Open a Cacti PR for this cleanup as a matter of course - don't wait to be asked.