Skip to content

Apply the units before the common removelists, and let a build select them #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
marcos-mendez merged 1 commit into from
Sep 27, 2026
Merged

Apply the units before the common removelists, and let a build select them #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions COVERAGE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,25 @@ Pull request #1 merged on 2026-09-26 (merge commit e79be43) and brought
down, and is only ever raised. The sections that follow record the state
before the merge.

## 2026-09-27: the unit loop and where it runs

`tests/coverage.sh` now runs two suites and counts the checks of both:

| Suite | Checks | What it measures |
|-------|--------|------------------|
| tests/source-date-epoch.sh | 19 of 19 | the `SOURCE_DATE_EPOCH` handling |
| tests/units.sh | 38 of 38 | the unit loop, `UNITS` and the position of the units in `root.patched` |

Total 57 of 57, 100 percent. The gate stays at 100.

`tests/units.sh` does not dry run. It builds a fake product with three
units against this checkout's `product.mk` with the four fab tools replaced
by stubs that log their arguments, and asserts on the log: what was
applied, in which order, and what a failing tool does. That is why it can
tell that a unit whose `conf` is not executable is skipped, which a
`make -n` of the same recipe cannot, since the `[ -x ]` guard is resolved
by the shell and not by make.

## Baseline before the merge: 0 percent, nothing measured

`tests/` holds `regtest.sh` (77 lines), `override.sh`, `parseopts.py` and
Expand Down
39 changes: 30 additions & 9 deletions share/product.mk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,14 @@ ifeq ($(wildcard $(REMOVELIST)),)
endif

UNIT_DIRS ?= unit.d

# The units this build applies, as directory paths. Every unit under
# UNIT_DIRS by default. A build layered on a parent image overrides it with
# the units the parent has not applied yet, because a unit's conf script
# runs once and is not written to run twice; the plans of every unit are
# still resolved (root.spec below), since the plan only names packages.
UNITS ?= $(sort $(wildcard $(UNIT_DIRS)/*))

CONF_SCRIPTS ?= conf.d
PATCHES_PATH ?= patches.d

Expand Down Expand Up @@ -191,6 +199,7 @@ define help/body
@echo ' PLAN $(value PLAN)'
@echo ' REMOVELIST $(value REMOVELIST)'
@echo ' UNIT_DIRS $(value UNIT_DIRS)/'
@echo ' UNITS $(value UNITS)'
@echo ' ROOT_OVERLAY $(value ROOT_OVERLAY)/'
@echo ' CONF_SCRIPTS $(value CONF_SCRIPTS)/'
@echo ' PATCHES_PATH $(value PATCHES_PATH)/'
Expand Down Expand Up @@ -282,6 +291,10 @@ endef

# target: root.spec
root.spec/deps ?= $(STAMPS_DIR)/bootstrap $(wildcard plan/*)
# every unit's plan is resolved, including the plans of the units UNITS
# leaves out: a plan names packages, and a package the parent image already
# carries resolves to the version that is there rather than being installed
# a second time
define root.spec/body
unit_plans="$(wildcard $(UNIT_DIRS)/*/plan)"; \
fab-plan-resolve $(PLAN) $$unit_plans $(EXTRA_PLAN) --bootstrap=$(BOOTSTRAP) --output=$O/root.spec $(foreach var,$(_CONF_VARS_BUILTIN),-D '$(var)=$($(var))')
Expand Down Expand Up @@ -375,23 +388,31 @@ define root.patched/body
fi
)

# apply the common removelists
$(foreach removelist,$(_COMMON_REMOVELISTS),
fab-apply-removelist $(removelist) $O/root.patched; \
)

# apply the product-local units
$(foreach unit,$(wildcard $(UNIT_DIRS)/*), \
# apply the unit overlays, in unit name order and all of them before any
# unit conf script, the order the common overlays and the common conf
# scripts above are applied in: a unit is a component of the same kind,
# and a recipe that composes several must not depend on which one of
# them fab happens to reach first
$(foreach unit,$(UNITS), \
if [ -d $(unit)/overlay ]; then \
echo fab-apply-overlay $(unit)/overlay $O/root.patched; \
fab-apply-overlay $(unit)/overlay $O/root.patched; \
fab-apply-overlay $(unit)/overlay $O/root.patched || exit; \
fi; \
)

# run the unit configuration scripts
$(foreach unit,$(UNITS), \
if [ -x $(unit)/conf ]; then \
echo fab-chroot $O/root.patched --script $(unit)/conf; \
fab-chroot $O/root.patched --script $(unit)/conf; \
fab-chroot $O/root.patched --script $(unit)/conf || exit; \
fi; \
)

# apply the common removelists, after the units, so that a removelist
# can remove a file a unit brought in
$(foreach removelist,$(_COMMON_REMOVELISTS),
fab-apply-removelist $(removelist) $O/root.patched; \
)

# apply the product-local root overlay
if [ -d $(ROOT_OVERLAY) ]; then \
Expand Down
30 changes: 21 additions & 9 deletions tests/coverage.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
#!/bin/bash
# Coverage of the SOURCE_DATE_EPOCH code in share/product.mk.
# Coverage of the project-authored code in share/product.mk:
# tests/source-date-epoch.sh the SOURCE_DATE_EPOCH handling
# tests/units.sh the unit loop and UNITS
#
# make has no line coverage tool, so the number here is the share of the
# checks in tests/source-date-epoch.sh that pass: one check per effect of
# the touched code (both arms of the ifneq, both xorriso recipes, isohybrid,
# FAB_CHROOT_ENV, fab-plan-resolve and the untouched mksquashfs options).
# checks in those scripts that pass: one check per effect of the touched
# code (both arms of the ifneq, both xorriso recipes, isohybrid,
# FAB_CHROOT_ENV, fab-plan-resolve and the untouched mksquashfs options for
# the first; each unit input, each position in root.patched, each value of
# UNITS and each failure path for the second).
# Exits 1 when that share is below the threshold (default 95).
#
# tests/coverage.sh [THRESHOLD] (or COVERAGE_THRESHOLD in the environment)
Expand All @@ -13,13 +17,21 @@ set -euo pipefail
here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
threshold="${1:-${COVERAGE_THRESHOLD:-95}}"

tap="$("$here/source-date-epoch.sh" || true)"
printf '%s\n' "$tap"
total="$(grep -cE '^(not )?ok [0-9]+ ' <<<"$tap")"
passed="$(grep -cE '^ok [0-9]+ ' <<<"$tap")"
suites=(source-date-epoch units)

total=0
passed=0
for suite in "${suites[@]}"; do
echo "# $suite"
tap="$("$here/$suite.sh" || true)"
printf '%s\n' "$tap"
total=$((total + $(grep -cE '^(not )?ok [0-9]+ ' <<<"$tap")))
passed=$((passed + $(grep -cE '^ok [0-9]+ ' <<<"$tap")))
done

percent="$(awk -v p="$passed" -v t="$total" 'BEGIN { printf "%.2f", (t ? 100 * p / t : 0) }')"

echo "product.mk SOURCE_DATE_EPOCH: $percent percent ($passed of $total checks) covered, threshold $threshold"
echo "product.mk: $percent percent ($passed of $total checks) covered, threshold $threshold"
if ! awk -v p="$percent" -v t="$threshold" 'BEGIN { exit !(p + 0 >= t + 0) }'; then
echo "coverage below threshold" >&2
exit 1
Expand Down
Loading
Loading