feat: sanitize ECS snapshot tag keys and add per-BSL/VSL credential support

[nomanuddin]

Fixes #64

Problem

1. InvalidTagKey.Malformed on volume backups

Velero passes tag keys such as velero.io/backup and velero.io/pv when
calling CreateSnapshot. ECS disks may also carry CSI-assigned tags like
kubernetes.io/created-for/pvc/name. All of these contain / which is
rejected by the Alibaba Cloud ECS tag key validation, causing every volume
backup to fail with InvalidTagKey.Malformed.

2. No per-location credential support

BSL and VSL share the same process-wide env vars, making it impossible to
use different credentials for object store (BSL) and volume snapshots (VSL)
on the same Velero deployment — e.g. a static access key for BSL
alongside an ECS instance RAM role for VSL.

Solution

1. Tag key sanitization (volume_snapshotter.go)

Add sanitizeTagKey() which:

• Replaces any character not in [a-zA-Z0-9_\-\.] with _
• Truncates to 128 characters
• Skips keys starting with forbidden prefixes (aliyun, acs:, http://, https://)

Applied in getTags() for both Velero-supplied and volume-copied tags.
Also fixes originalVolumeAZTagKey constant which itself contained a /.

2. Per-BSL/VSL credential support (common.go)

Add three optional config keys for BackupStorageLocation and VolumeSnapshotLocation:

Key	Description
accessKeyId	Alibaba Cloud Access Key ID
accessKeySecret	Alibaba Cloud Access Key Secret
stsToken	STS token (optional)

Two credential patterns are now supported:

1. Shared credential (existing): a single Kubernetes Secret used for both BSL and VSL,
   passed via --secret-file or spec.credential — works for most cases.

2. Per-location credential (new): set spec.credential on each BSL/VSL to reference
   a separate Kubernetes Secret (same approach as the AWS plugin). Velero v1.10+ mounts
   each secret and injects credentialsFile into the plugin config per location.

Alternatively, set accessKeyId + accessKeySecret directly in the BSL/VSL config
when a Kubernetes Secret is not available.

All existing auth methods (env vars, credentialsFile, RAM role) are fully preserved
as fallback: fully backward compatible.

Testing

• 11 unit tests for sanitizeTagKey covering all edge cases
• 5 unit tests for per-location credential priority and fallback behavior

Local kind cluster validation (Velero 1.11.0, linux/amd64)

• Configured separate Kubernetes secrets per BSL and VSL using spec.credential on each location
• BSL validated as Available with OSS credentials ✅
• VSL configured with ECS snapshot credentials ✅
• Volume backup completed and uploaded to Alibaba OSS bucket successfully ✅
• Full cluster object backup: credentials authenticated correctly, upload timed out due to local network constraints (not a plugin issue)

Note on tag sanitization validation: The sanitizeTagKey() logic is covered by 11 unit
tests. End-to-end validation on real ECS snapshots is pending deployment to a live cluster
— the fix addresses the InvalidTagKey.Malformed errors observed in production logs where
328+ errors were recorded per backup run.

---

I noticed PR #65 addresses a similar issue by filtering out tags with forbidden prefixes (acs:, aliyun, http://, https://). This PR takes a different and more complete approach:

• PR [Patch] 跳过非法标签 #65 drops any tag whose key starts with a forbidden prefix — tags from Velero itself (e.g. velero.io/backup, velero.io/pv) would also be silently discarded since they contain / which is invalid.
• This PR sanitizes the key by replacing invalid characters with _ rather than dropping the tag entirely. This means Velero's own metadata tags are preserved as velero.io_backup, velero.io_pv etc., keeping the snapshot traceable back to the originating backup.

The two fixes are complementary: happy to coordinate or consolidate if the maintainers prefer a single PR.

[nomanuddin]

@AlbeeSo, mowangdk could you please review this PR when you get a chance? It fixes InvalidTagKey.Malformed errors on volume backups (active issue on clusters using ACK) and adds optional per-BSL/VSL inline credentials

Happy to adjust anything based on your feedback.

[nomanuddin]

cc @AlbeeSo @DahuK — would appreciate a review when you get a chance. This addresses the same InvalidTagKey.Malformed error reported in #64.

[AlbeeSo]

Thanks for the PR! I authored #67 which has already been merged into master. However, that fix intentionally only filters out the acs: prefix — these are system-reserved tags that the ECS backend silently adds to volumes/snapshots (e.g. acs:ecs:createdBy) and are known to be rejected by Create* APIs.
According to the Alibaba Cloud ECS API documentation (https://www.alibabacloud.com/help/en/ecs/developer-reference/api-ecs-2014-05-26-createsnapshot), the documented tag key restrictions are:

• Cannot start with aliyun or acs:
• Cannot contain http:// or https://
  There is no explicit mention of / being an invalid character in the official docs. The acs: filter aligns with what [Patch] 跳过非法标签 #65 also addressed. I chose not to apply broader sanitization (like replacing / with _) because unconditionally transforming tag keys could introduce unexpected behavior — for example, key collisions when two different keys map to the same sanitized form, or making it harder to correlate snapshots back to their source.
  Could you try building from the current master branch (which includes fix: filter out acs: system-reserved tags to avoid InvalidTagKey #67) and confirm whether you still see InvalidTagKey.Malformed errors?

[AlbeeSo]

Also, I've submitted #68 which takes a different approach to the per-location credential isolation problem. Would appreciate your thoughts on it.

The key difference is in how credentialsFile is consumed:

The root cause of the isolation issue is that godotenv.Overload calls os.Setenv, permanently mutating process-wide environment variables. This means:

• If BSL loads credentials via credentialsFile (from spec.credential), and VSL has no credentialsFile (intending to fall back to ECS RAM Role), VSL will incorrectly pick up BSL's stale env vars instead of going through the RAM Role path.

	Current master	This PR (#66)	#68
Credential file loading	godotenv.Overload (pollutes process env)	godotenv.Overload (unchanged) + inline config keys checked first	godotenv.Read (local map only, no env pollution)
BSL=AK + VSL=RAM Role	Bug: VSL gets BSL's AK	Fixed only if BSL uses inline config keys; still broken via credentialsFile	Fixed for all paths
Credential storage	Secret	Plaintext in CRD config (accessKeyId/accessKeySecret)	Secret

My concern with the inline config keys approach is that it adds a parallel credential path while the underlying godotenv.Overload issue remains for anyone using spec.credential. #68 fixes it at the root by switching to godotenv.Read, which makes spec.credential work correctly for per-location isolation without needing any new config keys.

For your second issue (per-BSL/VSL credentials): could you check if #68 addresses your scenario? It uses Velero's standard spec.credential on BSL/VSL to reference separate Kubernetes Secrets (dotenv format with ALIBABA_CLOUD_ACCESS_KEY_ID/ALIBABA_CLOUD_ACCESS_KEY_SECRET).