Skip to content

Fix CI workflows, compiler flags, and flaky tests#1415

Merged
bmehta001 merged 4 commits intomicrosoft:mainfrom
bmehta001:bhamehta/ci-fixes
Apr 29, 2026
Merged

Fix CI workflows, compiler flags, and flaky tests#1415
bmehta001 merged 4 commits intomicrosoft:mainfrom
bmehta001:bhamehta/ci-fixes

Conversation

@bmehta001
Copy link
Copy Markdown
Contributor

@bmehta001 bmehta001 commented Mar 18, 2026

CI/build/test fixes only — no runtime behavior changes. Runtime fixes (data races, memory leak, shutdown safety) are in #1429.

Modernize CI workflows (11 files)

  • Update all GitHub Actions to latest versions (checkout@v4, setup-java@v5, setup-android@v3, etc.)
  • Add concurrency groups with cancel-in-progress scoped to pull_request events only — push builds on main always run to completion so a rapid second push can't hide a broken commit
  • Add 30-minute timeout-minutes to iOS CI to prevent deadlock hangs (the CancelAllRequests spin loop can hang indefinitely if an NSURLSession completion handler never fires on the simulator)
  • Fix iOS simulator destination: resolve simulator UUID via xcrun simctl list to avoid ambiguous -destination matching
  • Fix Android SDK path detection in CodeQL workflow

Fix compiler flags (2 files)

  • Split GCC and Clang warning flags — -Wno-unknown-warning-option is Clang-only and was breaking GCC builds
  • Add -fno-finite-math-only to all non-MSVC release flags — restores IEEE 754 compliance (INFINITY macro) needed by sqlite3 and tests when -ffast-math is enabled
  • Remove global -Wno-reorder, scope to Sanitizer.cpp only via set_source_files_properties — the only file that triggers the warning is in the lib/modules submodule where Sanitizer.hpp declares members in a different order than the constructor init list

Improve iOS test script and documentation (3 files)

  • Rewrite build-tests-ios.sh to dynamically resolve simulator UUID for unambiguous xcodebuild destination
  • Document python3 requirement in README
  • Update docs/cpp-start-ios.md with current build instructions

Fix flaky functional tests (6 files)

  • Unique phase2 DB paths (APITest.cpp): replace PAL::sleep(500) with unique-per-invocation DB paths using an atomic counter. The SDK closes SQLite via sqlite3_close_v2() which defers file-descriptor cleanup — reusing a fixed path and std::remove()-ing files with deferred fds still open caused iOS vnode unlinked while in use errors
  • SQLite journal cleanup (APITest.cpp, AISendTests.cpp): remove -wal, -shm, -journal files in CleanStorage() to prevent cross-test contamination
  • Config contamination (BasicFuncTests.cpp): reset CFG_INT_CACHE_FILE_SIZE and CFG_INT_STORAGE_FULL_PCT in test Initialize()LogManager::GetLogConfiguration() returns a global singleton, so tests that modify config fields contaminate subsequent tests
  • yield() → sleep(10) (BasicFuncTests.cpp): reduces CPU contention on single-core iOS simulator runners and gives the network stack time to deliver
  • Multi-batch upload flakiness (BasicFuncTests.cpp): add SetTransmitProfile(TransmitProfile_RealTime) before UploadNow() to set all timer delays to 0 — prevents events from being held back in lower-priority batches
  • Use 127.0.0.1 (BasicFuncTests.cpp): replace localhost with 127.0.0.1 for the local test HTTP server — avoids DNS resolution failures on iOS simulator
  • Increased timeouts: widen waitForEvents timeouts across tests to accommodate slow CI runners
  • Atomic counter (MultipleLogManagersTests.cpp): fix data race on shared event counter

@bmehta001 bmehta001 requested a review from a team as a code owner March 18, 2026 05:39
@bmehta001 bmehta001 force-pushed the bhamehta/ci-fixes branch 7 times, most recently from f4b3bd9 to 1872937 Compare March 19, 2026 04:35
@bmehta001 bmehta001 self-assigned this Mar 19, 2026
Comment thread lib/http/HttpClient_Apple.hpp Outdated
Comment thread lib/http/HttpClient_Apple.mm Outdated
Comment thread lib/http/HttpClient_Apple.mm Outdated
Copy link
Copy Markdown
Contributor

@lalitb lalitb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for the comments.

Comment thread lib/tpm/TransmissionPolicyManager.cpp Outdated
@bmehta001 bmehta001 force-pushed the bhamehta/ci-fixes branch from 7f4e119 to 79342fe Compare April 1, 2026 08:41
@bmehta001 bmehta001 requested a review from ThomsonTan April 1, 2026 21:55
Comment thread .github/workflows/build-android.yml Outdated
Comment thread lib/pal/WorkerThread.cpp Outdated
Comment thread CMakeLists.txt Outdated
@bmehta001 bmehta001 force-pushed the bhamehta/ci-fixes branch 4 times, most recently from ebdf617 to c917965 Compare April 20, 2026 18:52
Comment thread lib/http/HttpClient_Apple.mm Outdated
Comment thread build-tests-ios.sh Outdated
Comment thread .github/workflows/build-ios-mac.yml Outdated
@bmehta001 bmehta001 requested a review from lalitb April 24, 2026 13:49
@lalitb
Copy link
Copy Markdown
Contributor

lalitb commented Apr 27, 2026

@bmehta001 - The CI fixes are important and also more safe, but this PR also changes some runtime behavior in the HTTP client, WorkerThread, TPM, Logger, and HttpResponseDecoder.

Can we keep this PR scoped to the CI/build/test fixes, and move the runtime race/shutdown fixes to a separate PR? If any of these runtime changes are needed to unblock CI, can you call that out with the specific failure they address and how it was validated?

Comment thread lib/pal/WorkerThread.cpp
bmehta001 and others added 4 commits April 28, 2026 11:48
- Update Actions to latest versions (checkout@v4, setup-java@v5, etc.)
- Add concurrency groups scoped to PR events only (push builds on main
  always run to completion)
- Add 30-minute timeout to iOS CI to prevent deadlock hangs
- Fix iOS simulator UUID resolution for unambiguous destination
- Fix Android SDK path detection in CodeQL workflow
- Document python3 requirement for iOS tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Split GCC/Clang warning flags (Clang-only flags were breaking GCC)
- Add -fno-finite-math-only to restore IEEE 754 compliance needed by
  sqlite3 when -ffast-math is enabled
- Remove global -Wno-reorder, scope to Sanitizer.cpp only (submodule
  declares members in wrong order)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Resolve simulator UUID for unambiguous xcodebuild destination
- Add python3 dependency documentation
- Update iOS getting-started docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Use unique phase2 DB paths instead of sleep to avoid SQLite vnode
  collisions from deferred sqlite3_close_v2 cleanup
- Clean SQLite journal files (-wal, -shm, -journal) in CleanStorage
- Fix cross-test config contamination (reset storage settings)
- Use sleep(10) instead of yield() in waitForEvents to reduce CPU
  contention on single-core simulator runners
- Increase timeouts for iOS simulator environment
- Fix multi-batch upload flakiness with SetTransmitProfile(RealTime)
- Use 127.0.0.1 instead of localhost for local test server

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bmehta001
Copy link
Copy Markdown
Contributor Author

@lalitb - Yes, will remove all functional changes to put in another PR, even if a few tests fail in the CI on this branch, just to keep things simple.

@bmehta001 bmehta001 changed the title Fix pipelines and tests Fix CI workflows, compiler flags, and flaky tests Apr 29, 2026
Copy link
Copy Markdown
Contributor

@lalitb lalitb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bmehta001 - Nice work, Appreciate making the CI green!

@bmehta001 bmehta001 merged commit efc4ac9 into microsoft:main Apr 29, 2026
26 of 27 checks passed
@bmehta001 bmehta001 deleted the bhamehta/ci-fixes branch April 29, 2026 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants