Skip to content

Latest commit

 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

influxdb-cloud-alternative

InfluxDB line protocol Flux Telegraf MIT

Export a time range as line protocol, replay it somewhere else with backpressure, and then check window by window that the two sides hold the same points. Plus an honest account of what InfluxDB Cloud does that a single InfluxDB 2.7 instance does not — starting with the query language, because that is where most of these migrations actually fail.

Decide three things before you export anything

1. Which InfluxDB are you leaving? This matters more than anything else in this repo. tools/lp_export.py speaks Flux, so it reads InfluxDB 1.8+ and 2.x sources — OSS 2.x and InfluxDB Cloud (TSM). If your source is InfluxDB 3 — Cloud Serverless, Cloud Dedicated, Core or Enterprise — Flux is not available there at all, and this exporter will not connect. Export those with InfluxQL over the v1 /query endpoint or SQL over Flight, produce line protocol, and hand the files to lp_replay.py, which does not care where they came from.

2. What query language does your dashboard speak? Moving from InfluxDB 3 to a hosted 2.7 bucket gets you Flux back and takes SQL away. Moving the other way takes Flux away. Neither direction is free, and no data migration tool fixes a dashboard.

3. What produces the data? If the answer is Telegraf, the migration is a config change and a backfill. If the answer is a Flux task writing into a downsampled bucket, you have a scheduled job to recreate as well — tasks are not data and do not travel with an export.


Flux, InfluxQL, SQL: which one survives

InfluxData's own position, from The future of Flux: Flux is in maintenance mode — supported, security-patched, not developed — and it could not be carried into InfluxDB 3 because that is a ground-up rewrite in Rust. The documentation recommends InfluxQL or SQL for anything you want to future-proof.

InfluxDB 1.x InfluxDB 2.x (and freebase.cloud's 2.7.4) InfluxDB 3
InfluxQL yes yes, including the v1 /query compatibility endpoint yes
Flux 1.8+ yes no
SQL no no yes

Last verified: 2026-08-18.

Practical consequences, in the order you will hit them:

  • InfluxQL is the portable choice. It runs on every version in that table. examples/queries.influxql writes six real queries in it, with a closing section on what does not translate from Flux.
  • Flux's to() has no InfluxQL equivalent. InfluxQL cannot write query results back into a bucket, so downsampling-by-task has to stay in Flux on 2.x, or be rebuilt on InfluxDB 3's processing engine.
  • join() across measurements is Flux-only. In InfluxQL you denormalise at write time or join in the client.
  • A long re-ingested as a double breaks mean() silently. Not an error — just no rows. This is the single most common consequence of a hand-rolled line-protocol replay, and it is why lp_export.py reads types from the #datatype annotation row rather than inferring them.

Export

pip install requests

export SOURCE_INFLUX_URL='https://us-east-1-1.aws.cloud2.influxdata.com'
export SOURCE_INFLUX_TOKEN='...'
export SOURCE_INFLUX_ORG='my-org'
export SOURCE_INFLUX_BUCKET='telemetry'

python3 tools/lp_export.py \
  --start 2026-07-01T00:00:00Z --stop 2026-08-01T00:00:00Z \
  --window 1h --out export/

One file per window, named for its start instant:

export/telemetry-20260701T000000Z.lp
export/telemetry-20260701T010000Z.lp
...

Windows are the unit of work everywhere in this repo — export, replay and verification all use the same boundaries, which is what makes the counts comparable afterwards. A re-run skips windows whose file already exists, so an interrupted export resumes rather than restarts.

Two details worth knowing:

  • Timestamps are parsed to nanoseconds by hand. Flux emits nine fractional digits; datetime.fromisoformat would truncate to microseconds and quietly merge points that were distinct. In a time-series database the timestamp is part of the key.
  • Empty windows are reported, not ignored. A gap in the export is a gap you will not notice again until someone queries that day next quarter.

Replay

export TARGET_INFLUX_URL='https://HOST:8086'
export TARGET_INFLUX_TOKEN='...'
export TARGET_INFLUX_ORG='my-org'
export TARGET_INFLUX_BUCKET='telemetry'

python3 tools/lp_replay.py --in export/ --dry-run
python3 tools/lp_replay.py --in export/

The target is anything that speaks the v2 write API, a bucket on a free instance included. Backpressure is the whole design:

Response What the tool does
429 sleeps for exactly Retry-After, halves the batch size
413 halves the batch and retries the same lines
500504 exponential backoff with jitter, batch size unchanged
other 4xx stops, prints the body, saves the checkpoint

After five clean batches the size doubles again, up to --max-batch. The effect on a rate-limited plan is a run that slows down to whatever the endpoint will accept and then stays there, instead of oscillating between hammering and erroring.

Progress goes to replay-checkpoint.json per file. Re-running skips confirmed files and replays a partial one from its beginning — which is safe, because line protocol writes are idempotent: the same measurement, tag set, field key and timestamp overwrites, it does not append. That single property is what makes retries in this tool safe and is worth confirming holds for your schema before you rely on it.

Verify

python3 tools/ts_verify.py \
  --start 2026-07-01T00:00:00Z --stop 2026-08-01T00:00:00Z --window 1h

Per window, per measurement and field:

window                measurement.field                     src       dst       Δ  sums
------------------------------------------------------------------------------------------
2026-07-01T00:00:00Z  chamber.temperature_c                 180       180       0  -3240.5 / -3240.5
2026-07-01T00:00:00Z  chamber.door_open                     180       180       0  non-numeric
2026-07-01T00:00:00Z  compressor.starts                     180       180       0  1620 / 1620
2026-07-01T01:00:00Z  chamber.temperature_c                 180       174      -6  -3132.1 / -3021.4   <-- differs

Counts alone are not enough. A replay that mangled an escape, lost precision, or changed a field's type can land exactly the right number of points with the wrong values in them, so the tool also sums every numeric field. Summing is cheap, order-independent, and catches all three. String and boolean fields are counted but not summed, and the output says so rather than implying a check it did not perform.

Non-zero exit on any window that differs. Because files are named by window, fixing a bad window is: delete that file, re-export it, replay again.

Rollback

python3 tools/lp_replay.py --in export/ --rollback \
  --predicate '_measurement="chamber"' --yes

This issues POST /api/v2/delete for the time range covered by the exported files. Read the warning it prints: InfluxDB deletes by time range and predicate, not by "rows this tool wrote". If the target bucket holds anything else in that window, a predicate-less delete takes it too. Narrow it, or replay into a dedicated bucket in the first place — which is the safer pattern and costs nothing.


What InfluxDB Cloud does that a single 2.7 instance does not

Last verified: 2026-08-18 against InfluxData's documentation.

InfluxDB Cloud Free InfluxDB 2.7.4 instance
Storage engine InfluxDB 3 / IOx — columnar, Parquet-backed, with SQL 2.x TSM
Query languages SQL and InfluxQL on Serverless; Flux on Cloud TSM Flux and InfluxQL
Free plan Documented limits: 30-day retention, 2 buckets, data-in 5 MB per 5 minutes, read payload 300 MB per 5 minutes n/a
Telegraf Same agent, plus managed configuration in the UI Same agent, you keep the config
Tasks Managed scheduler Available on 2.x, you own the schedule
Operations Backups, scaling, upgrades handled Yours

Three things that deserve saying properly:

  • The IOx engine is a real advance. Columnar storage over Parquet with SQL on top changes what is practical at high cardinality — the failure mode that used to end 1.x deployments. If cardinality is your problem, InfluxDB 3 is the answer to it, and a 2.7 instance is not.
  • Telegraf is the best part of the ecosystem and it is free. Hundreds of input plugins, one binary, works against any of these endpoints. examples/telegraf.conf points at whatever you set $INFLUX_URL to, a free 2.7.4 endpoint among them. Nobody should migrate away from Telegraf; the question is only where it writes.
  • Managed tasks are underrated. A downsampling schedule that someone else keeps running is worth money, and recreating it is the part of the migration people forget until a dashboard goes flat.

Stay on InfluxDB Cloud if…

  • Your queries are SQL. InfluxDB 2.7 does not speak SQL. Rewriting a dashboard's worth of SQL into Flux or InfluxQL to save a hosting bill is usually a bad trade, and it is a one-way door.
  • Cardinality is why you moved to Cloud in the first place. If you went to Serverless because series cardinality was killing a 1.x or 2.x deployment, going back to a TSM engine reintroduces exactly that problem.
  • You are ingesting continuously at volume. Sustained ingest with retention measured in years is what a managed time-series service is for. A free instance is for development, prototyping and small production workloads.
  • Retention beyond your comfort is a compliance requirement. Do not migrate a system with an audit obligation onto anything without a backup story you have personally tested.

Time-series data has a property that makes migrations less scary than most: it is append-mostly and idempotent on rewrite. You can replay the same range twice with no harm. That means you can migrate in slices, verify each slice, and keep both sides written to during the overlap — which is the approach this tooling is shaped for.

InfluxDB on freebase.cloud

Create a session at freebase.cloud and choose the InfluxDB engine; see the free InfluxDB instance page.

InfluxDB 2.7.4: line protocol ingestion, Flux, InfluxQL including the v1 compatibility endpoints, buckets with retention durations, and tasks. Telegraf writes to it through outputs.influxdb_v2 with no special handling, and Grafana takes it as a data source with a token, org and bucket.

influx write --host https://HOST:8086 \
  --token "$INFLUX_TOKEN" --org myorg --bucket telemetry \
  "chamber,unit=fr-101,site=depot-north temperature_c=-18.35 $(date +%s%N)"

Free tier: development, prototyping and small production workloads. No credit card.

Reaching a bucket from an AI assistant

Generate a token under Settings → MCP → New Token and copy the URL it gives you for that connection. In ChatGPT: Settings → AppsAdvanced settings → developer mode → Apps → Create → paste the endpoint → Auth NoneScan ToolsCreate. (Settings → Connectors is the other documented path; OpenAI's two pages disagree.) Developer mode is documented for Pro, Plus, Business, Enterprise and Edu; full write access is currently rolling out to Business, Enterprise and Edu workspaces, so treat writes as unavailable until you have confirmed them on your own plan.

From LangChain:

from langchain_mcp_adapters.client import MultiServerMCPClient

client = MultiServerMCPClient({
    "metrics": {"transport": "http", "url": "https://freebase.cloud/api/mcp/YOUR_TOKEN"}
})
tools = await client.get_tools()

The connection exposes metrics_query, metrics_store, metrics_list_tables (measurements, here) and metrics_annotate_table — the last of which is worth using: telling a model that chamber is a refrigeration unit in a depot and that door_open is a boolean saves it from inventing a Flux query around a field that does not exist. Claude setup: how to connect Claude to InfluxDB.


tools/lp_export.py    Flux → line protocol, one file per window
tools/lp_replay.py    write with 429/413/5xx backpressure, checkpointed, rollback
tools/ts_verify.py    per-window count and numeric sum comparison
examples/seed_metrics.sh     six hours of cold-chain telemetry via curl
examples/downsample.flux     a 15-minute task, and why tasks are not data
examples/queries.influxql    the same questions in the portable language
examples/telegraf.conf       agent config with credentials from the environment

MIT licensed.

freebase.cloud is an independent service and is not affiliated with InfluxData, Inc., Grafana Labs, OpenAI or Anthropic.

About

InfluxDB Cloud alternative — free time-series options with Flux, InfluxQL and Prometheus remote_write

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages