Fix mirrors hang when /status endpoint is down

[h8d13]

Aims to fix #4029 or start a discussion about it (I've had this when the endpoint is down).

By assigning a bool to the def load_remote_mirrors in the parent (which was naïve about the state).

This can be tested on the branch above and using something similar to block URL (temporarily):

echo "127.0.0.1 archlinux.org" | sudo tee -a /etc/hosts or when it's down like currently (15:00 UTC+1).

Second thing I'm wondering about:

  def get_status_by_region(self, region: str, speed_sort: bool) -> list[MirrorStatusEntryV3]:
      mappings = self._mappings()
      region_list = mappings[region]

      # Only sort if we have remote mirror data with score/speed info
      # Local mirrors lack this data and can be modified manually before-hand
      # Or reflector potentially ran already
      if self._fetched_remote and speed_sort:
          # original return
          return sorted(region_list, key=lambda mirror: (mirror.score, mirror.speed))
      # just return as-is without sorting?
      return region_list

Another day another DDoS ? I'll ping thsi for the funny lore too #3974

[h8d13]

Also had fun on another branch where we can use mirror endpoint from archlinux.de (or other models?)

master...h8d13:archinstall-patch:refs/heads/dot-mirrors-de

[Torxed]

We could also fall back to /etc/pacman.d/mirrorlist as it should contain a new enough mirror list when we can't use the online lookup.

And the regions are "there" in the shape of comments, so we'd just need to parse "which region are we currently on" and all subsequent mirror entries (even if commented out) belong to that region.

[h8d13]

It is already the fallback behavior problem was that timeout didn't work and the parent logic didn't know about the child status

The extra options are totally optional and can be cherry picked if you don't want them (today's commits)

[Torxed]

Ah, I think I'm slowly waking up and getting the hang of this.

So would the --offline flag be sufficient instead of the following, or do we need the fine tuning of arguments and code logic:

archinstall/archinstall/lib/args.py

Lines 397 to 402 in 421892b

	parser.add_argument(
	'--local-mirr',
	action='store_true',
	default=False,
	help='Disabled mirrors fetch and use local existing file instead.',
	)

I'm not against it, just wondering if --offline could serve well here, as it would help with other online issues too.

[Torxed]

Ah, I think I'm slowly waking up and getting the hang of this.

So would the --offline flag be sufficient instead of the following, or do we need the fine tuning of arguments and code logic:

archinstall/archinstall/lib/args.py

Lines 397 to 402 in 421892b

	parser.add_argument(
	'--local-mirr',
	action='store_true',
	default=False,
	help='Disabled mirrors fetch and use local existing file instead.',
	)

I'm not against it, just wondering if --offline could serve well here, as it would help with other online issues too.

Also regarding this, the ISO should already be running reflector.service as a pre-cursor that archinstall waits for, so we shouldn't need to run it again?

But we could perhaps re-name the --local-mirr to --skip-reflector/--skip-mirrors? And keep it, but make it so this:

archinstall/archinstall/lib/installer.py

Lines 197 to 199 in 5fcea37

	info('Waiting for automatic mirror selection (reflector) to complete.')
	while self._service_state('reflector') not in ('dead', 'failed', 'exited'):
	time.sleep(1)

Behaves like this:

archinstall/archinstall/lib/installer.py

Lines 180 to 181 in 5fcea37

	if not arch_config_handler.args.skip_ntp:
	info(tr('Waiting for time sync (timedatectl show) to complete.'))

[h8d13]

Imma test an install just for sanity check :d

Both tests pass with and without the new arg, and expected behaviors (one hanging a bit then going through, and the other skipping instantly both issues). 🕺

[svartkanin]

I'm somewhat against introducing a new flag to specifically handle the mirror fetching failures, if this continues we'll have a flag for every menu entry which isn't sustainable and very confusing for users.

I'd rather recommend to handle this gracefully in the menu and provide the users with options if possible or just fall back to the local config.

[Torxed]

That works too. As long as we have a way to deal with these DDoS issues, otherwise archinstall is "useless" as it won't be able to pass the mirror stage.

So currently these --skip-??? are needed for environments where NTP is blocked, GPG is blocked or mirror generators are unaccessible (reflector + our /status logic, both relies on the same DDoS'ed endpoint).

After that I don't think we have more --skip-??? flags needed as those are the online resources currently blocking progress.

And the menu would need to internally handle and det a skip feature to circumvent the issues.

[svartkanin]

Then instead of having the flag would it make sense to have a "failed to fetch mirrors, fallback to locale?" Yes/no?

[h8d13]

@svartkanin i removed it... see files changed based on both of your feedbacks

[Torxed]

The code looks okay so I'll approve it.
We should try this out for a bit before merging, but maybe it would be nice to get into the Jan ISO? Considering the ongoing DDoS?

[h8d13]

Its been happening one or twice per month now, so I'd consider fixing this more priority, you can sleep better knowing the installer works regardless of initial fetch. (along with the NVIDIA changes). If there is still anything with the code that needs change lmk

[svartkanin]

Seems to work fine