Introduce a plain foreman flavor

[ekohl]

Why are you introducing these changes? (Problem description, related links)

Today Foreman can be deployed without Katello on both RPM and Debian installations and this allows users to mirror that.

What are the changes introduced in this pull request?

The flavor itself is rather trivial, but there are some changes made to only deploy certain roles depending on the features enabled. That is likely incomplete. In particular, Apache is not yet modified to only deploy the required services. In the future when a plain foreman-proxy role is introduced then that will need adjustment.

There are also heavy testing changes, heavily based on #509 which in turn is based on #508. I'm wondering how much of those commits already belong there.

Right now the test suite fails because the foreman feature is only set in the flavor and not in the parameters.

It's also important to look at how this will play into #431.

How to test this pull request

Steps to reproduce:

• Run foremanctl deploy --flavor foreman
• Observe only Foreman it deployed without Pulp and Candlepin
• Test suite passes

Checklist

• Tests added/updated (if applicable)
• Documentation updated (if applicable)

[arvind4501]

Our CI is currently not designed to run flavor specific tests https://github.com/theforeman/foremanctl/blob/master/.github/workflows/test.yml#L54, it adds all features but surely that won't make sense, as we want to only add relevent features for a flavor(ex: add bmc to plain foreman does not make any sense),

Thinking out loud:-
Should we make test workflow per flavor, I am still trying to get all data points which can help in multi-flavor testing

[ekohl]

Should we make test workflow per flavor, I am still trying to get all data points which can help in multi-flavor testing

If you're talking about a GitHub Actions workflow: I think that does make sense. We probably can't test all combinations, but finding a good set is exactly what I think we should do.

[arvind4501]

If you're talking about a GitHub Actions workflow

yes,and i think without flavor specific workflows we might end up with a single tests workflow which adds all available features(as of today) and will keep CI red

[ekohl]

I'd still like to see this. Some PRs that are opened because of this, but can be merged already:

• Introduce feature-based test directory structure #509
• Mark slow tests to allow for quicker iteration #655
• Only test for azure-rm and google features when enabled #656
• Mark tests with the appropriate feature #657

After that I'll rebase.

[ekohl]

The exception is that import_paths and export_paths are unbound in this code:

foremanctl/tests/pulp_test.py

Lines 11 to 22 in 4d2d756

	def load_pulp_paths_from_parameters():
	test_dir = os.path.dirname(os.path.abspath(__file__))
	foremanctl_dir = os.path.dirname(test_dir)
	params_file = os.path.join(foremanctl_dir, '.var', 'lib', 'foremanctl', 'parameters.yaml')
	if os.path.exists(params_file):
	with open(params_file, 'r') as f:
	params = yaml.safe_load(f)
	import_paths = params.get('pulp_import_paths', [])
	export_paths = params.get('pulp_export_paths', [])
	return import_paths, export_paths

That means the parameters.yaml file doesn't exist.

[ekohl]

That means the parameters.yaml file doesn't exist.

Ah, I moved the file around and it tries to derive the root from the file location.

[ekohl]

I wasn't sure about this, but I see tests/feature/foreman-proxy/base_test.py also imports FOREMAN_PROXY_PORT.

Then there's also this:

foremanctl/tests/migration_test.py

Lines 7 to 9 in 9ffe360

	@pytest.fixture(scope="module")
	def obsah_state_path():
	return os.environ.get("OBSAH_STATE", "/var/lib/foremanctl")

Should it be a global fixture?