🐛 Bug
Paraformer.inference() passes the two timestamp-related outputs from the CIF predictor to ts_prediction_lfr6_standard() in reverse order.
calc_predictor() returns:
(acoustic_embeds, token_num, alphas, cif_peak)
but the current caller effectively does:
ts_prediction_lfr6_standard(
timestamp_pre_peak_index, # cif_peak
timestamp_alphas, # alphas
copy.copy(token),
vad_offset=kwargs.get("begin_time", 0),
upsample_rate=1,
)
while the helper contract is:
ts_prediction_lfr6_standard(us_alphas, us_peaks, char_list, ...)
Therefore CIF peaks are treated as alpha weights, and alpha weights are treated as peaks.
This normally does not raise an exception. The helper can enter its normalization fallback and still return the expected number of timestamp segments, so the result looks structurally valid while its boundaries are incorrect.
A fix with regression tests is already available in #3716.
To Reproduce
This deterministic example uses the real CIF implementation and the real timestamp helper:
import torch
from funasr.models.paraformer.cif_predictor import cif
from funasr.utils.timestamp_tools import ts_prediction_lfr6_standard
weights = [0.1, 0.1, 0.4, 0.6] + [0.0] * 4 + [0.1] * 10
alphas = torch.tensor([weights])
hidden = torch.zeros(1, len(weights), 2)
_, peaks = cif(hidden, alphas, threshold=1.0)
tokens = ["你", "你"]
# Current Paraformer argument order
_, current = ts_prediction_lfr6_standard(
peaks[0],
alphas[0],
tokens,
upsample_rate=1,
)
# Order required by the helper API
_, expected = ts_prediction_lfr6_standard(
alphas[0],
peaks[0],
tokens,
upsample_rate=1,
)
print("current:", current)
print("expected:", expected)
Observed output:
current: [[329, 690], [690, 1080]]
expected: [[90, 510], [510, 1080]]
Both calls return two segments. Consequently, tests that verify only output shape or segment count do not detect the defect.
Expected behavior
The caller should use explicit keyword arguments:
timestamp_str, timestamp = ts_prediction_lfr6_standard(
us_alphas=timestamp_alphas,
us_peaks=timestamp_pre_peak_index,
char_list=copy.copy(token),
vad_offset=kwargs.get("begin_time", 0),
upsample_rate=1,
)
Per-sample padding trimming, VAD offsets, and the helper's legitimate alpha-normalization fallback should remain unchanged.
Regression tests should validate tensor identity/content and actual timestamp boundaries rather than only tensor shapes.
Impact
A real-audio A/B test used the same:
- checkpoint and model checkout;
- input audio;
- captured alpha tensors;
- captured peak tensors;
- token inputs;
- 84-token transcript.
Only the argument order supplied to the timestamp helper was changed.
| Configuration |
Largest absolute start change |
Largest absolute end change |
| Whole audio, no VAD |
8.100 s |
8.100 s |
FSMN-VAD v2.0.4, batch_size_s=300 |
6.300 s |
6.179 s |
These are measured timestamp-output changes caused by the argument order, not an alignment-accuracy claim against human labels.
Full reproduction evidence, environment details, audio, and per-token comparisons are available in:
Scope
This issue is limited to the vanilla Paraformer inference timestamp path.
E-Paraformer has a superficially similar reversed call, but its PifPredictor currently returns cif_peak=None; timestamp support for that path requires separate handling.
The transcription path with timestamps disabled is unaffected.
Environment
- OS: macOS 26.5.1 arm64
- Python: 3.9.6
- FunASR package metadata: 1.4.15, editable source checkout
- PyTorch / torchaudio: 2.8.0 / 2.8.0
- ModelScope: 1.37.1
- Device: CPU
- Validated source baseline:
615ef059c989fca12a546505abb22c5c5b6f329b
- Fix commit:
cbce0a2ca1bf8813fa41e3edde7e05fc2b7bbc90
Related but not duplicate: #1226 reported inaccurate Paraformer timestamps as a user-visible symptom; this issue identifies a specific argument-order defect in the current vanilla Paraformer inference path.
🐛 Bug
Paraformer.inference()passes the two timestamp-related outputs from the CIF predictor tots_prediction_lfr6_standard()in reverse order.calc_predictor()returns:but the current caller effectively does:
while the helper contract is:
Therefore CIF peaks are treated as alpha weights, and alpha weights are treated as peaks.
This normally does not raise an exception. The helper can enter its normalization fallback and still return the expected number of timestamp segments, so the result looks structurally valid while its boundaries are incorrect.
A fix with regression tests is already available in #3716.
To Reproduce
This deterministic example uses the real CIF implementation and the real timestamp helper:
Observed output:
Both calls return two segments. Consequently, tests that verify only output shape or segment count do not detect the defect.
Expected behavior
The caller should use explicit keyword arguments:
Per-sample padding trimming, VAD offsets, and the helper's legitimate alpha-normalization fallback should remain unchanged.
Regression tests should validate tensor identity/content and actual timestamp boundaries rather than only tensor shapes.
Impact
A real-audio A/B test used the same:
Only the argument order supplied to the timestamp helper was changed.
batch_size_s=300These are measured timestamp-output changes caused by the argument order, not an alignment-accuracy claim against human labels.
Full reproduction evidence, environment details, audio, and per-token comparisons are available in:
Scope
This issue is limited to the vanilla Paraformer inference timestamp path.
E-Paraformer has a superficially similar reversed call, but its
PifPredictorcurrently returnscif_peak=None; timestamp support for that path requires separate handling.The transcription path with timestamps disabled is unaffected.
Environment
615ef059c989fca12a546505abb22c5c5b6f329bcbce0a2ca1bf8813fa41e3edde7e05fc2b7bbc90Related but not duplicate: #1226 reported inaccurate Paraformer timestamps as a user-visible symptom; this issue identifies a specific argument-order defect in the current vanilla Paraformer inference path.