Skip to content

cp.md

Latest commit

 

History

History
131 lines (92 loc) · 8.3 KB

cp.md

File metadata and controls

131 lines (92 loc) · 8.3 KB
Raw

rc cp

Purpose

rc cp copies files and objects between local paths and S3-compatible remote paths. It is a legacy-compatible command; prefer rc object copy for new scripts.

Syntax

rc [GLOBAL OPTIONS] cp [OPTIONS] <SOURCE>... <TARGET>

Parameters

Parameter Description
SOURCE One or more local files/directories or remote objects/prefixes.
TARGET Local directory or remote prefix for multiple sources; local or remote path for one source.
-r, --recursive Recursively copy a directory or prefix.
--overwrite Overwrite destination data where supported.
--dry-run Show planned copies without copying data.
--preserve Preserve applicable metadata.
--content-type Set object content type for uploads.
--storage-class Set destination storage class for uploads where supported.
--enc-s3 <TARGET> Apply SSE-S3 to the named remote destination write.
--enc-kms <TARGET>=<KMS_KEY_ID> Apply SSE-KMS to the named remote destination write.
--include <GLOB> Include source-relative paths matching the glob. Repeatable.
--exclude <GLOB> Exclude matching paths after includes are evaluated. Repeatable.
--newer-than <AGE> Select sources strictly newer than an age such as 1h or 7d.
--older-than <AGE> Select sources strictly older than an age such as 1h or 7d.
--rewind <TIME> Select source metadata at or before a UTC timestamp or age.
--concurrency <COUNT> Bound all in-flight leaf transfers across the command. Defaults to 4.
--rate-limit <RATE> Pace aggregate transfer starts by expected bytes, for example 10MiB/s.
--retry-attempts <COUNT> Maximum attempts for classified transient failures. Defaults to 3.
--retry-initial-backoff-ms <MS> Initial retry backoff. Defaults to 100.
--retry-max-backoff-ms <MS> Maximum retry backoff. Defaults to 10000.
--continue-on-error Continue eligible work after an item fails; the final exit code remains non-zero.
--fail-empty Return the not-found exit code when no source passes selection.
--summary Print deterministic aggregate counters in human output; bulk and recursive copies summarize automatically.
--portable-names When downloading to a local filesystem, reject keys that cannot be created on Windows. Unix destinations accept characters such as : by default.

Examples

Upload a file:

rc cp ./report.json local/reports/report.json

Upload a directory recursively:

rc object copy ./reports/ local/reports/ --recursive

Copy between buckets on the same alias:

rc cp local/reports/summary.json local/archive/summary.json

Copy between aliases:

rc cp --overwrite stage/data/report.json prod/archive/report.json
rc cp --recursive --overwrite stage/data/ prod/archive/

Copy multiple files with command-wide controls:

rc cp ./january.csv ./february.csv local/reports/ --concurrency 8 --rate-limit 10MiB/s --summary

Filter a recursive upload using source-relative paths and UTC metadata:

rc cp ./reports/ local/archive/ --recursive --include '*.csv' --exclude 'private-*' --newer-than 7d

Upload with explicit destination encryption:

rc cp ./report.json local/archive/report.json --enc-s3 local/archive/report.json

Recursively upload a directory and apply one KMS key to the remote target prefix:

rc cp ./reports/ local/archive/ --recursive --enc-kms local/archive/=alias/archive-key

Behavior

The last path is always the target. Multiple sources require a local directory or remote prefix target, and ambiguous targets fail before any transfer starts. Sources can mix local and remote paths only where the command can infer a valid copy direction. Same-alias S3-to-S3 copies use server-side CopyObject, including recursive prefix copies. Cross-alias copies download through a temporary file and upload with the destination alias credentials. Use trailing slashes consistently when copying directory-like prefixes.

Include rules restrict the candidate set when present. Exclude rules are evaluated afterwards and always win, regardless of flag order. --newer-than and --older-than use strict UTC comparisons; --rewind includes its boundary. Candidates without required source timestamps are skipped. Empty selections succeed unless --fail-empty is passed.

Concurrency, rate pacing, and retry budgets are global to the command rather than per source. Only classified transient failures are retried. --continue-on-error preserves individual failures and continues remaining work, but the aggregate exit code is still non-zero. Summaries count planned, skipped, successful, failed, cancelled, and transferred bytes. Planned bulk JSON output is intentionally unavailable until the versioned output contract supports it; combining --json with multi-source, recursive, filtered, rate-limited, or summary planning fails explicitly instead of silently changing JSON output. Legacy single-object JSON output remains unchanged.

Destination encryption flags apply only to remote writes. On rc cp, the selector in --enc-s3 or --enc-kms must match the command destination exactly:

  • For a single-object write, use the full remote object path.
  • For a recursive upload or remote-to-remote copy, use the same remote prefix passed as TARGET.

The current implementation supports SSE-S3 and SSE-KMS. It does not support SSE-C, repeated encryption selectors, or MinIO mc-style prefix fan-out matching beyond the exact destination argument for the current command. For shared encryption rules across commands, see Encryption workflows.

When the server returns a source or destination object version ID, JSON copy output uses the output v3 versioned_objects envelope with data.operation set to copy. data.source_version_id identifies the copied source version and data.version_id identifies the created destination version. Copies for which the backend reports no version information retain the legacy JSON shape.

Recursive downloads map object keys onto the local filesystem using / separators. Traversal, absolute keys, backslashes, and control characters are always rejected. Characters such as : are accepted on Unix destinations unless --portable-names is set.

BREAKING object-key portability contract migration

--portable-names is additive. Unix destinations no longer apply Windows filename rules by default, so keys containing : are accepted. Remote-to-remote copies keep the original key. This PR must be marked BREAKING because docs/reference/rc/cp.md is a protected CLI behavior contract. No JSON schema or config schema_version bump applies.

Cross-alias copies record the source ETag in the destination user metadata x-amz-meta-rc-source-etag, the same key rc mirror writes. The destination computes its own ETag during the upload, so without this record a later rc mirror --compare auto or rc diff --compare auto could not tell a faithful copy from a changed object and would copy the tree a second time. This entry is rc bookkeeping rather than user data, so it is written even with --metadata-directive replace. Same-alias copies use server-side CopyObject, which preserves the ETag, and do not need it.

BREAKING cross-alias copy contract migration

Cross-alias rc cp is additive and does not change same-alias CopyObject behavior. Destinations on a different alias are copied by download then upload instead of failing as unsupported_feature. Source content type and user metadata are preserved unless --metadata-directive replace is set. Cross-alias uploads additionally record x-amz-meta-rc-source-etag so incremental rc mirror and rc diff runs recognize the copy. This PR must be marked BREAKING because docs/reference/rc/cp.md is a protected CLI behavior contract. No JSON schema or config schema_version bump applies.

Global options shown in command syntax use the same meaning everywhere:

Option Description
--format auto|human|json Select automatic, human-readable, or JSON output.
--json Emit JSON output where the command supports structured output.
--no-color Disable terminal colors.
--no-progress Disable progress bars.
-q, --quiet Suppress non-error output.
--debug Enable debug logging.