Skip to content

[🚀 Feature]: Selenium GHA Testing Plan #17539

@titusfortner

Description

@titusfortner

Description

This supersedes #16809 - a lot of underlying work has been done on build process across the bindings since that issue was created to support a slightly better vision.

Problem

  • Github Actions runs are taking too long, especially when multiple small PRs are being made.
  • ci.yml workflow has significant overlap with ci-rbe.yml workflow
  • There are things that can't be tested in RBE that aren't being tested on a GHA Runner
  • Tests are inconsistent across bindings

Proposal

RBE remains the primary per-PR test gate. GHA Runners should only run tests that provide sufficient additional signal.

GHA runners should be scoped by affected Bazel targets (this is wired up but not used yet).

GHA Tests

Run every time there are affected targets:

  1. Windows: Affected targets list filtered for Edge + se_manager + os_sensitive -safari -ie (large & small tests)
  2. Mac: Affected targets list filtered for Safari + se_manager + os_sensitive -ie (large & small tests)
  3. Linux: Affected targets list filtered for se_manager + rbe_skipped -safari -ie

Run when tagged with [language]
4. Unit Test Matrix: first/last version and alternate interpreters as applicable

Run manually and/or scheduled
Run all the above where affected targets are everything

Additional/Future Considerations

  1. consider running periodically all targets on windows not filtering to just edge as a stopgap
  2. Ideally we can replace .skipped-tests with bazel tags or test runner guards/annotations
  3. fix Java to use target_compatible_with instead of generating targets for everything and filtering
  4. consider splitting skip-rbe tag into a reason for skipping

Have you considered any alternatives or workarounds?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions