Add openstack-tobiko container built from source - #82
Conversation
ee31bef to
f452228
Compare
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Central YAML (base), Organization UI (inherited) Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Merge Failed. This change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
a0832aa to
26a0e65
Compare
Build and install the pinned tobiko wheel like the other s2i services, and run pytest from that install instead of tox or a git checkout, so the image stays hermetic and disconnected clusters. Map test-operator TOBIKO_TESTENV to pytest paths, leave clouds.yaml and tobiko.conf on their mounts. Assisted-By: Cursor-Grok 4.6
| # | ||
| # Default path is hermetic and disconnected: tobiko is already installed as a | ||
| # wheel, pytest runs against that package, and no git/tox/venv/chown is used. | ||
| # clouds.yaml and tobiko.conf must be mounted at the usual client paths. | ||
| # | ||
| # test-operator parameters this entrypoint SKIPS (do not use them to pick code | ||
| # or to copy credentials from the logs PVC): | ||
| # | ||
| # spec.version / workflow[].version -> TOBIKO_VERSION | ||
| # Ignored. Used to git checkout at runtime. The pin is containerImage | ||
| # (sources.txt at image build). A non-empty value only logs a warning. | ||
| # | ||
| # TOBIKO_RUN_TESTS_TIMEOUT | ||
| # Ignored. Old TCIB mapped this to tox. Per-test timeout comes from | ||
| # pytest --timeout (by testenv). test-operator does not send this. | ||
| # | ||
| # clouds.yaml / tobiko.conf under external_files (the logs PVC) | ||
| # Not copied. test-operator already mounts: | ||
| # /etc/openstack/clouds.yaml | ||
| # ~/.config/openstack/clouds.yaml | ||
| # /etc/openstack/secure.yaml | ||
| # /etc/tobiko/tobiko.conf | ||
| # | ||
| # Honored: containerImage, testenv (TOBIKO_TESTENV), pytestAddopts / | ||
| # skipRegexList (TOBIKO_PYTEST_ADDOPTS), preventCreate, numProcesses | ||
| # (TOX_NUM_PROCESSES), debug, patch (optional overlay), privateKey/publicKey | ||
| # (copied from TOBIKO_KEYS_FOLDER to ~/.ssh). |
There was a problem hiding this comment.
while some of this info is useful this reads liek note an llm wrote about the change it made betwen the revsiosn rather then comment that should be commited and maintaiend
i.e. and no git/tox/venv/chown is used.
or
# Honored: containerImage, testenv (TOBIKO_TESTENV), pytestAddopts /
# skipRegexList (TOBIKO_PYTEST_ADDOPTS), preventCreate, numProcesses
# (TOX_NUM_PROCESSES), debug, patch (optional overlay), privateKey/publicKey
# (copied from TOBIKO_KEYS_FOLDER to ~/.ssh).
| TOBIKO_KEYS_FOLDER="${TOBIKO_KEYS_FOLDER:-${TOBIKO_DIR}/external_files}" | ||
| TOBIKO_LOGS_DIR_NAME="${TOBIKO_LOGS_DIR_NAME:-tobiko}" | ||
| TOBIKO_REPORT_NAME="${TOBIKO_REPORT_NAME:-tobiko_results}" | ||
| TOBIKO_PATCH_REPOSITORY="${TOBIKO_PATCH_REPOSITORY:-https://opendev.org/x/tobiko.git}" |
There was a problem hiding this comment.
im not asking you to change this but i think this is fien but you could also set these as ENV vars in the contaienr so that they will be defeind if you ever oc rsh into it or run a debug pod.
| # on the writable EmptyDirs (/var/lib/tobiko, /tmp). | ||
| export HOME="${TOBIKO_DIR}" | ||
| export TMPDIR="${TMPDIR:-/tmp}" | ||
| export PYTHONDONTWRITEBYTECODE=1 |
There was a problem hiding this comment.
do we really want to confirue forced bytecode compiliation
that really only help if you exect the script multiple times in teh same contaienr which woudl not be the typcial usage pattern here
| chmod 700 "${TOBIKO_DIR}/.ssh" | ||
| cp "${TOBIKO_KEYS_FOLDER}/${TOBIKO_PRIVATE_KEY_FILE}"* "${TOBIKO_DIR}/.ssh/" | ||
| chmod 600 "${TOBIKO_DIR}/.ssh/${TOBIKO_PRIVATE_KEY_FILE}" 2>/dev/null || true | ||
| fi |
There was a problem hiding this comment.
this shoudl proably have a todo to remove this in the future as the as the test operator should mount the ssh key into the correct localy with the correct permissions
|
|
||
| # Locate the installed package without importing tobiko (import runs oslo | ||
| # logging and needs ${TOBIKO_DIR}/tobiko to already exist). | ||
| INSTALLED_TOBIKO="$(python3 -c "import importlib.util, os; s = importlib.util.find_spec('tobiko'); print(os.path.dirname(os.path.abspath(s.origin)))")" |
There was a problem hiding this comment.
this feels like you are workign around a bug in tobiko
you shoudl not need to know the location where tis insteadd to execute it
| # Optional CI overlay: checkout a patch and prefer it on PYTHONPATH (no pip/venv). | ||
| if [[ -n "${TOBIKO_PATCH_REFSPEC}" ]]; then | ||
| PATCH_DIR="${TOBIKO_DIR}/src/tobiko" | ||
| mkdir -p "$(dirname "${PATCH_DIR}")" | ||
| git clone "${TOBIKO_PATCH_REPOSITORY}" "${PATCH_DIR}" | ||
| git -C "${PATCH_DIR}" fetch origin "${TOBIKO_PATCH_REFSPEC}" | ||
| git -C "${PATCH_DIR}" checkout FETCH_HEAD | ||
| export PYTHONPATH="${PATCH_DIR}${PYTHONPATH:+:${PYTHONPATH}}" | ||
| INSTALLED_TOBIKO="${PATCH_DIR}/tobiko" | ||
| fi | ||
|
|
||
| # Keep ${TOBIKO_DIR}/tobiko as a real directory (writable for tobiko.log). | ||
| # Symlink package contents so pytest paths like | ||
| # tobiko/tests/functional/podified/test_topology.py still resolve. | ||
| for _entry in "${INSTALLED_TOBIKO}"/*; do | ||
| [ -e "${_entry}" ] || continue | ||
| ln -sfn "${_entry}" "${TOBIKO_DIR}/tobiko/$(basename "${_entry}")" | ||
| done | ||
| unset _entry |
There was a problem hiding this comment.
it would be a lot clena if you wrap all the logic reslated to doing a soruce install into a function so its not inlien with the default flow
ideally you would have 2 fucntion one for the clean default flow that invoke the preinstalled code
this woudl be the default code path in ci.
and a secodn for the case wehre you doing a srouce overried dreictly
note the correct way to actuly use a diffent tobko version woudl be for the s2i content provider to build a new iamge based on a depend-on or similar refence in the job.
the git path woudl only ever be used locally even then its quitionable if we shoudl supprot that sicne you can just build the contianer you want to use by locally checkign out the relevent souce
| TEST_PATH="" | ||
| PYTEST_EXTRA=() | ||
| PYTEST_TIMEOUT="${PYTEST_TIMEOUT:-300}" | ||
| case "${TESTENV_NAME}" in | ||
| py3) TEST_PATH="tobiko/tests/unit" ;; | ||
| cover) | ||
| TEST_PATH="tobiko/tests/unit" | ||
| PYTEST_EXTRA+=(--cov=tobiko) | ||
| ;; | ||
| functional|manila|scenario|neutron|octavia|designate|nova) | ||
| PYTEST_TIMEOUT=2400 | ||
| case "${TESTENV_NAME}" in | ||
| functional) TEST_PATH="tobiko/tests/functional" ;; | ||
| manila) TEST_PATH="tobiko/tests/scenario/manila" ;; | ||
| scenario) TEST_PATH="tobiko/tests/scenario" ;; | ||
| neutron) TEST_PATH="tobiko/tests/scenario/neutron" ;; | ||
| octavia) TEST_PATH="tobiko/tests/scenario/octavia" ;; | ||
| designate) TEST_PATH="tobiko/tests/scenario/designate" ;; | ||
| nova) TEST_PATH="tobiko/tests/scenario/nova" ;; | ||
| esac | ||
| ;; | ||
| ovn_migration) | ||
| PYTEST_TIMEOUT=2400 | ||
| TEST_PATH="tobiko/tests/scenario" | ||
| PYTEST_EXTRA+=(-m "not skip_during_ovn_migration and not background and not flaky") | ||
| ;; | ||
| sanity|neutron_sanity|shiftstack_sanity) | ||
| PYTEST_TIMEOUT=1800 | ||
| case "${TESTENV_NAME}" in | ||
| sanity) TEST_PATH="tobiko/tests/sanity" ;; | ||
| neutron_sanity) TEST_PATH="tobiko/tests/sanity/neutron" ;; | ||
| shiftstack_sanity) TEST_PATH="tobiko/tests/sanity/shiftstack" ;; | ||
| esac | ||
| ;; | ||
| ovn_migration_sanity) | ||
| PYTEST_TIMEOUT=1800 | ||
| TEST_PATH="tobiko/tests/sanity" | ||
| PYTEST_EXTRA+=(-m "ovn_migration and not skip_during_ovn_migration and not background and not flaky") | ||
| ;; | ||
| faults|octavia_faults|neutron_faults|ha_faults|instanceha|podified_faults|podified_ha_faults|update_podified_post) | ||
| PYTEST_TIMEOUT=3600 | ||
| if [[ "${NUM_PROCESSES}" == auto ]]; then | ||
| NUM_PROCESSES=1 | ||
| fi | ||
| case "${TESTENV_NAME}" in | ||
| faults) TEST_PATH="tobiko/tests/faults" ;; | ||
| octavia_faults) TEST_PATH="tobiko/tests/faults/octavia" ;; | ||
| neutron_faults) TEST_PATH="tobiko/tests/faults/neutron" ;; | ||
| ha_faults) TEST_PATH="tobiko/tests/faults/ha" ;; | ||
| instanceha) TEST_PATH="tobiko/tests/faults/iha" ;; | ||
| podified_faults) TEST_PATH="tobiko/tests/faults/podified" ;; | ||
| podified_ha_faults) TEST_PATH="tobiko/tests/faults/podified/ha" ;; | ||
| update_podified_post) TEST_PATH="tobiko/tests/update/podified/post" ;; | ||
| esac | ||
| ;; | ||
| *) |
There was a problem hiding this comment.
this does not feel like it shoudl be in the image
it either shoudl be directly supprot by tibiko or via a script in its tools directory that we copy form the souce assuming its not jsut exposed via the test operatr cr
the s2i contianer reallly shoudl not know or ditcat how the test are executed like this.
why ware we creating this script at all when tobiko provide a help to do this nativly
https://opendev.org/x/tobiko/src/branch/master/tools/run_tests.py
all dix did was set teh envionmt vaiable
https://opendev.org/x/tobiko/src/branch/master/tox.ini#L167-L173
and then invoke run_tests.py
https://opendev.org/x/tobiko/src/branch/master/tox.ini#L43
so maintianing this wapper in this repo does not look like a maintance burden we should take on.
i woudl suggest removing all of the supprot for git cloning and make this script do only the following
export the env ard needed to invoke /tools/run_tests.py form the tobico source you can copy that to a well knwo location such as /usr/bin/run_tests.py
and then copy the resulte if reqruied to the final location need by the test operator althogh i think you can configure that via
TOX_REPORT_DIR and perhasp TOX_COVER_DIR
4 participants
Build and install the pinned tobiko wheel like the other s2i services,
and run pytest from that install instead of tox or a git checkout, so
the image stays hermetic and disconnected clusters.
Map test-operator TOBIKO_TESTENV to pytest paths, leave clouds.yaml
and tobiko.conf on their mounts.
Assisted-By: Cursor-Grok 4.6