`DayClampingStrategy` ceiling skips a full day for already-midnight values (`clamping: DAY` loses one day per window)

[bazarnov]

Summary

DayClampingStrategy.clamp with is_ceiling=True (the default wired for clamping: {target: DAY} on a DatetimeBasedCursor) unconditionally adds one day after flooring the value to midnight. A value that is ALREADY midnight-aligned therefore jumps a full day forward. When the concurrent cursor uses the clamped value as the next slice's lower boundary, every stepped window loses one day of data.

Root cause

airbyte_cdk/sources/streams/concurrent/clamping.py L34-L42 (v7.23.8):

class DayClampingStrategy(ClampingStrategy):
    def clamp(self, value: datetime) -> datetime:
        return_value = value.replace(hour=0, minute=0, second=0, microsecond=0)
        if self._is_ceiling:
            return return_value + timedelta(days=1)
        return return_value

There is no already-aligned guard. Compare with MonthClampingStrategy in the same file (L45-L56), which checks needs_to_round = value.day != 1 and returns the value untouched when it is already on the boundary. WeekClampingStrategy has the equivalent guard as well. Only the DAY strategy is missing it.

Reproduction

from datetime import datetime, timezone
from airbyte_cdk.sources.streams.concurrent.clamping import DayClampingStrategy

s = DayClampingStrategy()  # is_ceiling=True, as built for clamping: {target: DAY}
print(s.clamp(datetime(2024, 1, 1, tzinfo=timezone.utc)))
# 2024-01-02 00:00:00+00:00 - a midnight value is pushed a full day forward

Slice-level effect, measured with the real concurrent cursor (start 2024-01-01, end 2024-06-15, step P30D, granularity PT1S, clamping DAY):

2024-01-02 .. 2024-01-31T23:59:59      <- 2024-01-01 lost
2024-02-02 .. 2024-03-02T23:59:59      <- gap: 2024-02-01 never queried
2024-03-04 .. 2024-04-02T23:59:59      <- gap: 2024-03-03 never queried
2024-04-04 .. 2024-05-03T23:59:59      <- repeats every window

The slice loop sets the next lower boundary from the clamped upper and re-ceils it, so the skipped day recurs at every window boundary.

Impact

Any DatetimeBasedCursor configured with clamping: {target: DAY} and a stepped range silently drops one calendar day of data per window. WEEK/MONTH targets are unaffected thanks to their guards.

Suggested fix

Mirror the MonthClampingStrategy guard: return the value unchanged when it is already midnight-aligned, e.g. needs_to_round = (value.hour, value.minute, value.second, value.microsecond) != (0, 0, 0, 0).

Precedent

Found while reviewing airbytehq/airbyte#75495 (source-klaviyo reporting streams): day-aligned report windows were the natural fix there, clamping: {target: DAY} was the obvious tool, and pre-merge verification caught the dropped day - the connector shipped with a midnight-floored start date instead.

[varunbiluri]

I reproduced this on current main and prepared a minimal tested fix, but this repository currently restricts PR creation to collaborators.

Patch branch: https://github.com/varunbiluri/airbyte-python-cdk/tree/fix/day-clamping-boundary
Commit: bea21697

The change preserves midnight-aligned day boundaries (matching the existing week/month behavior) while continuing to ceil non-aligned timestamps. The regression test failed before the fix and passes afterward.

Validation completed:

• focused clamping and cursor suites: 56 passed
• Ruff check and format: passed
• mypy: 456 source files clean
• poetry check: passed

The broader local check reached Docker-backed integration tests and stopped because the Docker daemon is unavailable; the affected suites are clean. If a collaborator would like to sponsor this change, the commit is ready to cherry-pick.