Updating tensorwave fork of axolotl - #2
Open
nikhil-tensorwave wants to merge 1461 commits into
Open
Conversation
* reduce autotune search space * consistent docstrings
* fix token state json and mistral tokenizer issue * centralize constants * forgot to commit constants file * Fix weakref in pickling relora state dict * make curl a bit quieter so it doesn't log 2K lines * fix path traversal for olmoe test * more test fixes that weren't flagged previously * chore: lint * skip tests that fail b/c of OutOfResources * scattermoe as slow tests * update fbgemm-genai for torch 2.10
* support offloading layers to CPU * chore: lint * revert change * update docs
* synthetic datasets for benchmarking and testing * fix synthetic dataset parse from config and add tests * use type=_synthetic
* liger support for qwen 3.5 and fused rmsnorm+gated * support for qwen 3.5 moe * fix version ref * fixups for PR code review
… [skip ci] * feat: LoRA kernel support for bias, dropout, dora, embeddings * chore: lint * chore: lint * address PR feedback, add regression tests, add fsdp2 tests for lora kernels * update tests for new sigs * update tests now that bias and dropout are supported
… w curl (#3534) [skip ci]
* post merge lora fixes for CI * handle lora kernel auto-enable for moe without grouped_mm * prefer not to import torch in schema validation
* roundup_power2_divisions not needed with newer pytorch versions * remove typo * update qwen3.5 moe 35b-a3b yaml for 5090 * more bug fixes * fix tests to match updated trainer * don't use fa2 for hooks test * reset plugins on the instance * retry download * fix references to renamed axolotl_cfg property on trainer * Fix ref to trainer cfg
…uage Models (#3527) [skip ci] * EBFT wip * fixes * more fixeS * add missing strided module * ebft fixes for multi-turn * make ebft work with async * add example for ebft w qwen3.5 * fix for split thinking and update yaml for lora over linear attention only * enforce_eager for vllm arg in schema * fix sync weights * fix multi-gpu * handle updated sig for mm * ddp fixes * improve multi-gpu handling, don't calculate logits, adaptive completion length * chore: lint * chore: lint * support completion_mean * Address corereview feedback * clamp min IS ratio * Address PR code review * more fixes identified * address code review * Fix property from rebase conflict
This reverts commit 1f1ebb8.
* nemo gym integration with grpo wip * mostly working * cleanup * simplify * update docs * nemo gym support wip * cleanup * chore: lint * address PR review and add more tests * chore: lint * post merge lora fixes for CI (#3536) [skip ci] * post merge lora fixes for CI * handle lora kernel auto-enable for moe without grouped_mm * prefer not to import torch in schema validation * address pr comments, add timeout, add tests * roundup_power2_divisions not needed with newer pytorch versions (#3540) * roundup_power2_divisions not needed with newer pytorch versions * remove typo * update qwen3.5 moe 35b-a3b yaml for 5090 * more bug fixes * fix tests to match updated trainer * don't use fa2 for hooks test * reset plugins on the instance * retry download * fix references to renamed axolotl_cfg property on trainer * Fix ref to trainer cfg * fix: robust handling of race condition on patching check (#3543) [skip ci] * EBFT: Matching Features, Not Tokens: Energy-Based Fine-Tuning of Language Models (#3527) [skip ci] * EBFT wip * fixes * more fixeS * add missing strided module * ebft fixes for multi-turn * make ebft work with async * add example for ebft w qwen3.5 * fix for split thinking and update yaml for lora over linear attention only * enforce_eager for vllm arg in schema * fix sync weights * fix multi-gpu * handle updated sig for mm * ddp fixes * improve multi-gpu handling, don't calculate logits, adaptive completion length * chore: lint * chore: lint * support completion_mean * Address corereview feedback * clamp min IS ratio * Address PR code review * more fixes identified * address code review * Fix property from rebase conflict * fix for ebft sync and update docs * make trainer loss patch check a solo test --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…#3526) * feat: add Ernie 4.5 and subsequently custom routing support * Update routing.py * chore: lint * fix minor nits * removed deepseek v2 * remove unneeded change --------- Co-authored-by: Wing Lian <wing@axolotl.ai>
* merge_method added * merge_efficient core implement * Update src/axolotl/cli/merge_lora.py Co-authored-by: Wing Lian <wing.lian@gmail.com> * Update src/axolotl/utils/lora_merge_efficient.py Co-authored-by: Wing Lian <wing.lian@gmail.com> * standard to leagcy + rstrip + try/except for do_merge_lora_efficient(cfg=cfg) * fix: 'dict' object has no attribute 'lora_alpha' * into -> debug * lint * lint2 * moved everythign to cpu + peformance improvments * lint * Update src/axolotl/cli/merge_lora.py Co-authored-by: Dan Saunders <danjsaund@gmail.com> * Update src/axolotl/cli/merge_lora.py Co-authored-by: Dan Saunders <danjsaund@gmail.com> * string handeling + try except remove * merge_method -> merge_lora_methods * remove duplicate cal + safetensor + move to lora_merge.py * lint * handle quant-dequant, handle experts * fix parameter merging and prefer peft's native merge logic per module --------- Co-authored-by: Wing Lian <wing.lian@gmail.com> Co-authored-by: Dan Saunders <danjsaund@gmail.com>
…, better handle errors with vllm /reset_prefix_cache (#3549) * dispatch scored rollouts to plugins, extend path for external plugins, better handle errors with vllm /reset_prefix_cache * address PR comments, lint
* fix: handle get_open_port import across TRL versions TRL 0.29+ removed get_open_port from exports; fall back to importing directly from vllm.utils or vllm.utils.network_utils. * support DP with vllm and make generation_batch_size confifurable
* deprecate torch 2.8.0 support * shell lint * odd naming of manylinux wheels for x86
* support flattening/packing for GRPO * more flattening * fix tests * improve dead vllm handling * refactor out process handling for vllm serve and move bench flattening tests to gpu tests * add validation for flattening with liger * isolate batch flattening test * flaky test
* nemo support * config * rename , config * nemotron packing * config fix * read me + configs * gc compat bug * config chnages for qwen and pad token nemo * patch nemotron_h weight renaming so it doesn't get reversed to embedding (singular noun) on checkpoint save * lint * revert qwen3.5 config changes, not needed in this pr * lint * Update examples/nemotron-h/120b-a12b-qlora.yaml Co-authored-by: NanoCode012 <kevinvong@rocketmail.com> * Update examples/nemotron-h/nano-30b-a3b-qlora.yaml Co-authored-by: NanoCode012 <kevinvong@rocketmail.com> * readme + validation * lazy load comment * Update examples/nemotron-h/120b-a12b-qlora.yaml Co-authored-by: NanoCode012 <kevinvong@rocketmail.com> * val fix * add nemo to multi packing --------- Co-authored-by: Wing Lian <wing@axolotl.ai> Co-authored-by: NanoCode012 <kevinvong@rocketmail.com>
* Deperecate dpo_norm_loss * Rename chosen/rejected_input_ids to chosen/rejected_ids to match TRL huggingface/trl#5179 * Remove deprecated rpo_alpha * Remove dead_code tokenize_row * Add _tokenize override to prevent double bos token on Llama DPO * Fix DPO loss type now list not string * Linting fix * PR fixes * update _tokenize override for DPO for multimodal
* bug-fix: only apply patches when CUDA is available This will otherwise crash when performing operations with CUDA_VISIBLE_DEVICES=, such as LoRA merging on CPU. This patch only patches the Qwen 3.5 model, since that's the only one I've tested. This patch should most likely check torch.cuda for all other models as well. One limitation here is that I'm assuming the user runs CUDA, but that assumption is not restricted to this patch so it is probably fine. * include patch_qwen3_next_modeling_packing, patch_qwen3_5_moe_modeling_packing, and patch_qwen3_5_vlm_flash_attention in cuda guard
* upgrade transformers to 5.4.0 * allow fail for tests requiring phi3 tokenizer * ring-flash-attn skips * skip tests for now
* qwen3.5 configs * update shared experts readme
* allow bf16 flag but warn Reason: when doing e.g. LoRA merges with CUDA_VISIBLE_DEVICES=, this will unnecessarily crash, even though the LoRA merge operation would have finished successfully. This seems to warrant changing it to a warning instead, as the code will most likely crash later if bf16 is unavailable and training begins anyway. * don't use deprecated LOG.warn * update tests to reflect validation change
…kernels (#3865) Expert weight stacks over 2^31 elements (e.g. 512x5120x2048 = 5.4e9 at Nemotron-3-Ultra scale, 896x2048x2048 = 3.8e9 at Kimi-K3 scale) overflowed the i32 E_idx*stride pointer products: an illegal memory access in the grouped dW kernel and, worse, silent out-of-bounds dW writes that corrupt neighboring allocations. Same class of overflow in the sonicmoe NVFP4 triton codecs (row*K products in dequant/quant/fake-quant kernels). Promote the expert index / row id to i64 at every site that multiplies it by a per-expert stride. Adds a >2^31-element regression test (fails pre-fix on the dW kernel; the forward sites are covered prophylactically since their index dtype currently arrives as int64).
…kip ci] Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.6.0 to 6.3.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@a26af69...ece7cb0) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: 6.3.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… [skip ci] Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 5.10.0 to 6.2.0. - [Release notes](https://github.com/docker/metadata-action/releases) - [Commits](docker/metadata-action@c299e40...dc80280) --- updated-dependencies: - dependency-name: docker/metadata-action dependency-version: 6.2.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… [skip ci] Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 5.4.0 to 7.3.0. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](docker/build-push-action@ca052bb...53b7df9) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-version: 7.3.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…p ci] Bumps [marocchino/sticky-pull-request-comment](https://github.com/marocchino/sticky-pull-request-comment) from 2.9.4 to 3.0.5. - [Release notes](https://github.com/marocchino/sticky-pull-request-comment/releases) - [Commits](marocchino/sticky-pull-request-comment@7737449...5770ad5) --- updated-dependencies: - dependency-name: marocchino/sticky-pull-request-comment dependency-version: 3.0.5 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…#3854) [skip ci] Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6.1.0 to 8.1.1. - [Release notes](https://github.com/peter-evans/create-pull-request/releases) - [Commits](peter-evans/create-pull-request@c5a7806...5f6978f) --- updated-dependencies: - dependency-name: peter-evans/create-pull-request dependency-version: 8.1.1 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Read dataclass field help from the "help" metadata key
add_options_from_dataclass built each Click option with
help=field.metadata.get("description"), but every CLI dataclass documents
its fields with metadata={"help": ...}, which is the dataclasses
convention (and the key HfArgumentParser reads). The lookup therefore
returned None for every field, so all option descriptions were dropped
from --help.
The "description" key belongs to the Pydantic path in the same module,
where _get_field_description reads Field(description=...); it looks like
it was carried over to the dataclass path by mistake. No dataclass field
in the repo uses a "description" metadata key.
The most visible case is `axolotl vllm-serve --help`, where all nine
documented options rendered with no description at all.
* Move dataclass option tests into test_nested_options.py
* fix: ensure trust_remote_code supersedes * fix: also redact data_files
…#3859) `cache_dir` defaults to `None`, and `str(None)` is `"None"`, so `download_tokenizer_from_hf_hub` received a literal `"None"` path whenever a caller omitted the argument. That created a `None/` hub cache in the working directory and bypassed the real Hugging Face cache, re-downloading on every fresh working directory. `download_tokenizer_from_hf_hub` accepts `str | Path | None` and applies the default cache when given `None`. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* rfc: add staged model support profiles * rfc: harden model support profile contracts * rfc: address model support review feedback * rfc: memoize profile resolution and document matcher scope Address review feedback on the staged model-support profiles: - Memoize declarative profile resolution per descriptor class in a WeakKeyDictionary. `_ProfileProjection.__get__`, `check_capability`, and matcher dispatch previously rebuilt the full family/profile merge (dict merges, MappingProxy construction) on every access. Profile-only descriptors now also short-circuit `resolve_model_support` to the cached declarative result; legacy descriptors keep the exact per-call overlay. - Document that config/processor matchers are global predicates evaluated for every run, the shadow/ambiguity consequences of a loose matcher, and that overriding a built-in for the whole run requires same-`model_type` registration rather than a matcher. - Add regression tests for the memoization identity and the legacy fast-path bypass. * Fix mypy overload error: make resolve_model_support overloads contiguous with impl The two @overload stubs were separated from the implementation by _LEGACY_DECLARATION_NAMES and _class_declares_legacy, so mypy reported no-overload-impl on the stubs and no-redef on the implementation. Move the stubs to sit directly above the implementation. Pure reordering; no behavior change. * rfc: reject non-ModelSupport classes at registration A duck-typed class with model_types slipped through registration, then poisoned registry-wide matcher scans and failed later resolution with an opaque tuple.index ValueError. Fail fast with a TypeError instead. * rfc: fix legacy-hook visibility and nested-dispatch suppression Two dispatch defects found in adversarial review: - _class_declares_legacy cached its answer per class, so a legacy method assigned onto a descriptor class after its first resolution (test fixture, plugin) was permanently invisible to hook dispatch while a direct method call still ran it. Drop the cache; the MRO scan is cheap. - The legacy re-entrancy guard keyed on (support, phase) alone, so a nested dispatch for the same support and phase with a *different* cfg (e.g. a hook loading a second model of the same architecture) was silently skipped, dropping all family/profile/legacy hooks for the inner load. Include the cfg identity in the guard token; same-cfg super()/redispatch suppression is unchanged. * test: paddleocr capability set gained lora_kernels from upstream * rfc: address model support review round 2 - registry: import builtins outside the registry lock — holding it across imports inverts lock order against a thread mid-import of a builtin module (regression test deadlocks on the old code); drop the no-op except/raise - registry: accept a list for model_types and normalize to the documented tuple, so pre-refactor descriptors keep registering - registry: make the ambiguous-matcher error actionable (narrow matchers, or register under an exact model_type) - profile: validate the memoized declarative resolution against the current profile/model_types, so runtime profile reassignment recomputes instead of serving a stale result - profile/docs: spell out that None on ModelMatchers always means inherit, unlike ModelStrategyOverrides where omission inherits and None removes - loaders: a non-multimodal profile's auto-model class no longer shadows the multimodal mapping when the run resolved as multimodal (warn + fall back); cover the string-valued transformers names mapping with a test - schemas: read lora_kernels through resolve_model_support() — a legacy class-level capabilities attribute shadows the projection, so the attribute path missed profile-declared capabilities on hybrid descriptors; document the projection contract on ModelSupport - tests: capture the Experimental warning via capture_axolotl_warnings (an earlier configure_logging() leaves caplog blind); pass capabilities to validate_config so the auto-enable validator actually runs (it lives on AxolotlConfigWCapabilities — bare validate_config never exercised it) - cosmetic: cfg attribute access instead of getattr, plain dashes in the docs phase table
…registrations (#3884) * fix(model_support): anchor the kimi dynamic-module redirect to exact module stems The substring match also captured sibling remote-code modules such as modeling_kimi_vl (Kimi-VL), redirecting them to the in-tree kimi-linear copies and crashing class resolution. * feat(model_support): declarative weight-conversion and patch-mapping registrations ModelRegistrations gives profiles a typed seam for the two transformers registries that model integration needs as a pair: checkpoint conversion mappings (register_checkpoint_conversion_mapping) and module class swaps (register_patch_mapping). Both are applied idempotently at the BEFORE_MODEL_BUILD boundary. save_pretrained(save_original_format=True) reverses registered conversions, so registration warns when a transform carries a quantization operation or an operation without reverse_op instead of failing at save time. * fix(model_support): use capture_axolotl_warnings in registrations warn test * docs(model_support): fix registrations example imports and reversal wording * refactor(model_support): make _warn_irreversible_weight_transforms a PatchManager staticmethod --------- Co-authored-by: NanoCode012 <nano@axolotl.ai>
…ernel (#3885) * fix(attention): don't route fp32/CPU QKV into the sdpa varlen flash kernel The sdpa_varlen fast path guarded on mask/dropout/head_dim/scaling but not on dtype or device, so sdpa + sample_packing with fp32 (or CPU) tensors fed torch.nn.attention.varlen.varlen_attn, whose backing flash kernel only supports CUDA fp16/bf16 — crashing with 'FlashAttention only support fp16 and bf16 data type' on torch 2.12.1. Such rows now fall back to stock SDPA with the rebuilt block-diagonal mask (documents stay isolated). * test(sdpa_varlen): run the fallback tests on CPU and cover the device guard * fix(sdpa_varlen): skip the patch entirely when the run isn't CUDA fp16/bf16 * increase max steps for flaky e2e test --------- Co-authored-by: NanoCode012 <nano@axolotl.ai>
* fix(processing): declare Mistral3 role boundaries for chat_template masking * feat(liger): support mistral3 and ministral3 * fix(mistral): load mistral-common tokenizer from a local directory * docs(ministral3): drop the stale transformers-v5 branch step * fix(merge): skip mistral consolidated weights when copying non-model files * fix(merge): compose scoped weight renamings so multimodal adapters merge * docs(examples): add shieldstral fine-tuning example * chore: concise * docs(readme): note shieldstral model support under latest updates * docs: publish shieldstral model guide and link it from the readme * docs: publish model guides for gemma4, mistral4, qwen3.5 and glm models * docs(shieldstral): link the mistral blog post * docs(shieldstral): drop the ministral3 cross-link * docs(shieldstral): make the vision section standalone * docs(shieldstral): clarify the getting started steps * docs(shieldstral): link the multimodal dataset format docs
…kip ci] Bumps [actions/setup-python](https://github.com/actions/setup-python) from 6.3.0 to 7.0.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@ece7cb0...5fda3b9) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: 7.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…3913) [skip ci] Bumps [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) from 1.14.0 to 1.14.1. - [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases) - [Commits](pypa/gh-action-pypi-publish@cef2210...ba38be9) --- updated-dependencies: - dependency-name: pypa/gh-action-pypi-publish dependency-version: 1.14.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…3912) [skip ci] The delinearize_llama4 command defines model and output as click.argument (positional), not options. The documented '--model'/'--output' flags make Click error with 'No such option: --model'. Use positional arguments.
* chore(deps): bump peft to 0.20.0 peft 0.20.0 ships the fix for kwargs-only forward through ModulesToSaveWrapper / AuxiliaryTrainingWrapper (huggingface/peft#3199), which previously crashed lora_modules_to_save entries invoked with keyword arguments only (e.g. a VLM vision tower). Adds a regression test covering that path. Closes #3801 * test: drop WHAT-only helper docstrings per repo comment convention * test: trim module docstring to the essential constraint
* fix(ebft): access EBFTConfig fields by attribute in validators * test(ebft): add validation and e2e coverage, drop conflicting torch_compile from example * fix(ebft): move ebft_pretrain strategy into the prompt_strategies package * chore(ebft): drop duplicated example strategies, add type resolution test
* fix(kto): keep multi-turn history in argilla_chat prompt * fix(test): correct tokenizer and pin kto-mix-15k data_files in test_kto_lora
…et (#3882) [skip ci] * perf(quantize): pass raw ptrs/ints to bnb dequant when argtypes are set * fix: bump bnb * fix(qgalore): adapt shim to bnb 0.50 removal of non-blockwise 8-bit API bnb 0.50 (bitsandbytes-foundation/bitsandbytes#1871) removed F.optimizer_update_8bit and F.percentile_clipping, dropped the percentile_clipping/block_wise keys from get_config(), and removed both params from Optimizer2State.__init__ -- so q-galore's positional super().__init__ call silently assigned percentile_clipping=100 to max_unorm and block_wise=True to skip_zeros. Mirror the legacy F attrs only when present, default the missing config keys, and re-map the constructor call. --------- Co-authored-by: Wing Lian <wing@axolotl.ai>
…skip ci] Bumps [actions/github-script](https://github.com/actions/github-script) from 7.1.0 to 9.0.0. - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@f28e40c...3a2844b) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ip ci] * build(deps): bump docker/login-action from 3.7.0 to 4.4.0 Bumps [docker/login-action](https://github.com/docker/login-action) from 3.7.0 to 4.4.0. - [Release notes](https://github.com/docker/login-action/releases) - [Commits](docker/login-action@c94ce9f...af1e73f) --- updated-dependencies: - dependency-name: docker/login-action dependency-version: 4.4.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * build(deps): bump docker/login-action from 4.4.0 to 4.5.1 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Bumps [actions/checkout](https://github.com/actions/checkout) from 7.0.0 to 7.0.1. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@9c091bb...3d3c42e) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 7.0.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…3848) (#3849) [skip ci] * fix(samplers): correct packing stats when drop_last discards a batch generate_batches() re-read batches[-1] after reslicing it away, so the total_slots adjustment measured the wrong batch and collapsed to a no-op, and it raised IndexError when the only batch was the incomplete one. Capture the dropped batch before reslicing, and discount both its slots and its tokens so the efficiency ratio stays consistent. Fixes #3848. * style: ruff format wrap in multipack drop_last stats * fix(samplers): guard empty batches and discount sequential drop-last stats * test(samplers): pin drop_last packing stats to retained bins * test(samplers): parametrize drop_last stats test over RandomSampler RandomSampler is what the training path uses; the retained-bin/batch-count and aggregate token assertions are invariant to sampling order for these fixtures, so both samplers are exercised. * fix: restore trailing newline in test_packed_batch_sampler.py * style: trim drop_last stats test comment to one line --------- Co-authored-by: Anai Guo <antai12232931@anaiguo.com> Co-authored-by: Wing Lian <wing@axolotl.ai> Co-authored-by: NanoCode012 <nano@axolotl.ai>
* fix: route FA4 through native flash_attention_4 and require quack>=0.6.0 * chore: clarify wording * feat: add test * fix: detect FA4 Blackwell backward hang on cutlass-dsl>=4.6.0 * chore: drop FA4 README bullet * feat: accept flash_attention_torch and keep capability flags on pinned hub kernels * fix: pin the deep-gemm hub kernel to version 1
* fix token inf time * tets attr lint * lint + rmv checks * beter sweep * patch sentinels * testssss * chore: tighten comments in chat_template turn locator --------- Co-authored-by: Your Name <you@example.com> Co-authored-by: NanoCode012 <nano@axolotl.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Motivation and Context
How has this been tested?
Screenshots (if appropriate)
Types of changes
Social Handles (Optional)