diff --git a/docs/user/intro.rst b/docs/user/intro.rst index a9c9092d4..24b5095df 100644 --- a/docs/user/intro.rst +++ b/docs/user/intro.rst @@ -70,9 +70,9 @@ e.g.: - Sending configuration updates. - :doc:`Executing shell commands `. -- :doc:`Executing mass commands `: Run a command on - multiple devices at once, see the :ref:`batch command API - ` for details. +- :ref:`Executing mass commands `: Run a command on + multiple devices at once, from the admin or with the :ref:`batch command + API `. - Perform firmware upgrades via the additional :doc:`firmware upgrade module `. diff --git a/docs/user/shell-commands.rst b/docs/user/shell-commands.rst index c63374daf..4a29b5870 100644 --- a/docs/user/shell-commands.rst +++ b/docs/user/shell-commands.rst @@ -194,24 +194,83 @@ useful for rebooting all devices in a group, changing passwords across multiple devices, or running diagnostics on all devices in an organization. -**Targeting options:** +Sending a Mass Command +~~~~~~~~~~~~~~~~~~~~~~ -- ``organization``: All devices in an organization. -- ``devices``: Explicit list of device UUIDs. -- ``group``: Device group UUID. -- ``location``: Location UUID. +Open *Network Operations* > *Mass command execute* from the menu. The +first step asks for: -If ``devices`` is provided, ``group`` and ``location`` are ignored. -Otherwise, ``group`` and ``location`` can be used together to narrow the -target set within the organization. +- the **command type** and its inputs, which change with the type + selected; +- a **label** to identify the mass command later, and optional **notes**; +- the **targets**: organization, device group and location. -If no targeting options are provided, the command targets all devices in -the organization. Superusers can omit ``organization`` to target all -devices across organizations. +The targets decide which devices are matched. Using more than one narrows +the selection: a group and a location together match only the devices +which are in that group *and* at that location. -For superusers, ``organization`` is set automatically when ``group`` or -``location`` is provided. +Superusers can leave every target empty to run the command on all the +devices of the system. Other users must choose at least one target, and +only see the command types enabled for their organizations (see +:ref:`openwisp_controller_organization_enabled_commands`). -Refer to the :ref:`Batch Command API ` -documentation for the available endpoints, request parameters, and -examples. +Reviewing the Devices +~~~~~~~~~~~~~~~~~~~~~ + +The second step shows a summary of the command and the list of the devices +it matched. + +Devices can be left out by unchecking them: the counter and the *Execute +on N devices* button follow the selection, which is kept while paging +through the list. *Back* returns to the first step with the form still +filled in. + +The mass command starts when the *Execute* button is clicked. + +Following the Results +~~~~~~~~~~~~~~~~~~~~~ + +After executing, the mass command page opens. It shows the status of the +mass command, how many devices are affected, the devices which were +skipped, and one row per device with its status and output. + +The rows are updated in real time, so the page does not need to be +reloaded to follow the progress. The table can be searched by device name +and filtered by status, device group and location (and by organization for +superusers). + +.. note:: + + Commands are executed in the background, one device at a time, so a + mass command sent to many devices keeps updating for a while after the + page is opened. + +Finding Past Mass Commands +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +*Network Operations* > *Mass command admin* lists the mass commands which +were sent, most recent first. + +The list can be searched by label, notes, organization, device, location +and group name, and filtered by organization, status, type, group and +location. Clicking a mass command opens the page described above. + +Skipped Devices +~~~~~~~~~~~~~~~ + +A device is skipped when the command cannot be created for it, for example +when the device has no access credentials, or when the command type is not +enabled for its organization. + +Skipped devices are not executed, but they are shown: the **Skipped +devices** field summarizes how many there are and why, and each one is +listed in the results table with the *skipped* status and the reason as +its output. They can be found with the status filter. + +Using the API +~~~~~~~~~~~~~ + +The same operations are available over the REST API, which also accepts an +explicit list of devices instead of the targets described above. Refer to +the :ref:`Batch Command API ` documentation +for the available endpoints, request parameters and examples. diff --git a/docs/user/websocket-api.rst b/docs/user/websocket-api.rst index 781b3d59a..2ae54bbed 100644 --- a/docs/user/websocket-api.rst +++ b/docs/user/websocket-api.rst @@ -17,6 +17,8 @@ All endpoints: shown in JavaScript-style notation with inline comments for readability. - Push real-time updates after the connection is established. - Do not accept client messages: any data sent from the client is ignored. + The only exception is the mass command endpoint, which accepts the + single request documented below. Authentication and Authorization -------------------------------- @@ -161,3 +163,92 @@ After the connection is established, the server pushes a message every time the geometry of any mobile location in a subscribed organization is updated. The payload is identical to the one documented for the `2. Single Location Updates`_ endpoint. + +4. Mass Command Updates +~~~~~~~~~~~~~~~~~~~~~~~ + +Connection URL: + +:: + + wss:///ws/controller/batch-command/ + +Scope ++++++ + +Progress of a single mass command: its status and the result of every +device it runs on. See :ref:`mass_commands`. + +Authorization ++++++++++++++ + +A user is authorized if: + +- The user is a superuser, OR +- The user is marked as staff AND has the ``connection.view_batchcommand`` + or ``connection.change_batchcommand`` permission AND manages the + organization of the mass command. + +Real-time Updates ++++++++++++++++++ + +The server pushes a message every time the mass command or one of its +commands changes. The ``type`` field tells the two apart. + +When the mass command itself changes, for example when it moves from +``idle`` to ``in-progress``: + +.. code-block:: javascript + + { + "type": "batch_status", + "id": "", // Mass command identifier + "label": "", // Label given when it was sent + "status": "", // "idle", "in-progress", "success" or "failed" + "status_display": "", // Status as shown in the user interface + "affected_devices": , // Number of devices the command runs on + "skipped_count": , // Number of devices which were skipped + "skipped_preview": [ /* ... */], // First and last skipped devices, with the reason + "total_rows": // Affected plus skipped devices + } + +When the command of one device changes: + +.. code-block:: javascript + + { + "type": "command_update", + "id": "", // Command identifier + "device": "", // Device identifier + "device_name": "", // Device name + "status": "", // "in-progress", "success" or "failed" + "status_display": "", // Status as shown in the user interface + "output": "", // Output collected so far + "modified": "", // Last modification, formatted for display + "index": , // Position of the row, sent only for new commands + "total_rows": // Affected plus skipped devices, sent with "index" + } + +Requesting the Current State +++++++++++++++++++++++++++++ + +A client which connects while the mass command is already running can ask +for the results it missed: + +.. code-block:: javascript + + { + "type": "request_current_state", + "page": 1 // Page of results, 20 rows per page + } + +The server replies with one message holding that page: + +.. code-block:: javascript + + { + "type": "batch_state", + "batch_status": { /* ... */ }, // Same fields as the "batch_status" message + "commands": [ /* ... */ ], // Rows of the requested page + "total_rows": // Affected plus skipped devices + } diff --git a/openwisp_controller/connection/admin.py b/openwisp_controller/connection/admin.py index f15c98d58..c321f92e1 100644 --- a/openwisp_controller/connection/admin.py +++ b/openwisp_controller/connection/admin.py @@ -1,26 +1,49 @@ +import hashlib +import logging from datetime import timedelta +from types import SimpleNamespace +from uuid import UUID, uuid4 import reversion import swapper from django import forms -from django.contrib import admin +from django.contrib import admin, messages +from django.core.exceptions import ObjectDoesNotExist, PermissionDenied, ValidationError +from django.core.paginator import EmptyPage, PageNotAnInteger, Paginator +from django.db.models import Count, Q from django.http import HttpResponseForbidden, JsonResponse +from django.shortcuts import redirect +from django.template.response import TemplateResponse from django.urls import path, resolve -from django.utils.html import format_html +from django.utils.html import format_html, format_html_join +from django.utils.safestring import mark_safe from django.utils.timezone import localtime from django.utils.translation import gettext_lazy as _ from openwisp_users.multitenancy import MultitenantOrgFilter -from openwisp_utils.admin import TimeReadonlyAdminMixin +from openwisp_utils.admin import ReadOnlyAdmin, TimeReadonlyAdminMixin from ..admin import MultitenantAdminMixin from ..config.admin import DeactivatedDeviceReadOnlyMixin, DeviceAdmin +from .filters import GroupFilter, LocationFilter, TypeFilter from .schema import schema -from .widgets import CommandSchemaWidget, CredentialsSchemaWidget +from .widgets import ( + BatchCommandSchemaWidget, + CommandSchemaWidget, + CredentialsSchemaWidget, + OrganizationScopedSelect, +) + +logger = logging.getLogger(__name__) Credentials = swapper.load_model("connection", "Credentials") DeviceConnection = swapper.load_model("connection", "DeviceConnection") Command = swapper.load_model("connection", "Command") +BatchCommand = swapper.load_model("connection", "BatchCommand") +Device = swapper.load_model("config", "Device") +DeviceGroup = swapper.load_model("config", "DeviceGroup") +Location = swapper.load_model("geo", "Location") +Organization = swapper.load_model("openwisp_users", "Organization") class CredentialsForm(forms.ModelForm): @@ -35,6 +58,124 @@ class Meta: widgets = {"input": CommandSchemaWidget} +class BatchCommandExecutionForm(forms.ModelForm): + required_css_class = "required" + + class Meta: + model = BatchCommand + fields = [ + "organization", + "label", + "notes", + "type", + "input", + "group", + "location", + ] + widgets = { + "label": forms.TextInput(attrs={"class": "vTextField"}), + "notes": forms.Textarea(attrs={"rows": 3}), + "input": BatchCommandSchemaWidget, + "group": OrganizationScopedSelect, + "location": OrganizationScopedSelect, + } + + class Media: + js = [ + "admin/js/vendor/jquery/jquery.min.js", + "admin/js/vendor/select2/select2.full.min.js", + "admin/js/jquery.init.js", + "connection/js/execute-command.js", + ] + css = { + "screen": [ + "admin/css/vendor/select2/select2.min.css", + "admin/css/autocomplete.css", + ] + } + + def __init__(self, *args, request=None, **kwargs): + super().__init__(*args, **kwargs) + self.request = request + if request is None or request.user.is_superuser: + return + organization_ids = request.user.organizations_managed + self.fields["organization"].queryset = self.fields[ + "organization" + ].queryset.filter(id__in=organization_ids) + for field_name in ("group", "location"): + self.fields[field_name].queryset = self.fields[field_name].queryset.filter( + organization_id__in=organization_ids + ) + allowed_commands = {} + for organization_id in organization_ids: + allowed_commands.update( + dict(Command.get_org_allowed_commands(organization_id=organization_id)) + ) + empty_choices = [ + choice for choice in self.fields["type"].choices if not choice[0] + ] + self.fields["type"].choices = empty_choices + list(allowed_commands.items()) + + def clean(self): + cleaned_data = super().clean() + if self.request is None or self.request.user.is_superuser: + return cleaned_data + organization = cleaned_data.get("organization") + group = cleaned_data.get("group") + location = cleaned_data.get("location") + # a batch without any target would run on every device of the + # deployment, which only superusers are allowed to do + if not any([organization, group, location]): + raise ValidationError( + _( + "Please select at least one of: organization, device group," + " or location." + ) + ) + # "organizations_managed" is a list of organization UUIDs as strings + organization_ids = self.request.user.organizations_managed + related_organizations = ( + ("organization", organization.pk if organization else None), + ("group", group.organization_id if group else None), + ("location", location.organization_id if location else None), + ) + for field_name, organization_id in related_organizations: + if organization_id is None: + continue + if str(organization_id) not in organization_ids: + self.add_error(field_name, _("Select a valid choice.")) + return cleaned_data + + def to_session(self): + """Returns the cleaned values as JSON serializable primitives. + + The session uses the JSON serializer, so model instances and UUIDs + cannot be stored as they are. + """ + + def _pk(value): + return str(value.pk) if value else None + + return { + # Namespaces the device selection the confirm page keeps in + # sessionStorage, which lives as long as the browser tab: without + # it a wizard would inherit the devices unselected by a previous + # one. It has to be issued here rather than by the browser, + # because the session is shared between tabs and sessionStorage + # is not. It is also submitted back on execution, which runs + # only the wizard the confirm page was rendered with. + "token": uuid4().hex, + "type": self.cleaned_data["type"], + "label": self.cleaned_data["label"], + "notes": self.cleaned_data.get("notes") or "", + "input": self.cleaned_data.get("input"), + "organization_id": _pk(self.cleaned_data.get("organization")), + "group_id": _pk(self.cleaned_data.get("group")), + "location_id": _pk(self.cleaned_data.get("location")), + } + + @admin.register(Credentials) class CredentialsAdmin(MultitenantAdminMixin, TimeReadonlyAdminMixin, admin.ModelAdmin): list_display = ( @@ -215,3 +356,689 @@ def schema_view(self, request): CommandInline, ] DeviceAdmin.add_reversion_following(follow=["deviceconnection_set"]) + + +class BatchCommandDeviceAdminMixin: + """Applied on top of the ModelAdmin registered for Device, for + openwisp-monitoring which replaces that registration and + its extra columns must appear. + + Filters and search are emptied because the devices are already chosen on + the execute page, this table only excludes some of them. + """ + + list_display_links = ["name"] + list_filter = [] + search_fields = [] + actions = None + list_per_page = 20 + ordering = ["name"] + change_list_template = "admin/connection/batch_command/confirm_command.html" + import_export_change_list_template = None + + def __init__(self, model, admin_site, devices=None): + super().__init__(model, admin_site) + self.devices = devices + + def get_list_display(self, request): + return ["select_device"] + list(super().get_list_display(request)) + + def get_queryset(self, request): + return super().get_queryset(request).filter(pk__in=self.devices) + + @admin.display(description="") + def select_device(self, obj): + return format_html( + '', + obj.pk, + _("Include {}").format(obj.name), + ) + + +class BatchCommandAdmin(MultitenantAdminMixin, ReadOnlyAdmin): + execute_command_template = "admin/connection/batch_command/execute_command.html" + confirm_command_template = "admin/connection/batch_command/confirm_command.html" + session_key = "batch_command_wizard" + list_display = [ + "label", + "organization_display", + "colored_status", + "type", + "affected_devices", + "created", + ] + ordering = ("-created",) + list_filter = [ + MultitenantOrgFilter, + "status", + TypeFilter, + GroupFilter, + LocationFilter, + ] + list_select_related = ("organization",) + search_fields = [ + "label", + "notes", + "organization__name", + "devices__name", + "location__name", + "group__name", + ] + change_form_template = ( + "admin/connection/batch_command/batch_command_change_form.html" + ) + device_commands_per_page = 20 + exclude = ("devices",) + fields = [ + "organization_display", + "label", + "notes", + "colored_status", + "type", + "formatted_input", + "affected_devices", + "display_skipped_devices", + "group", + "location", + "created", + "modified", + ] + readonly_fields = [ + "organization_display", + "colored_status", + "formatted_input", + "affected_devices", + "display_skipped_devices", + ] + + class Media: + css = { + "all": [ + "admin/css/changelists.css", + "admin/css/ow-filters.css", + "connection/css/batch-command.css", + ] + } + + def get_urls(self): + options = self.model._meta + return [ + path( + "execute/", + self.admin_site.admin_view(self.execute_command_view), + name=f"{options.app_label}_{options.model_name}_execute", + ), + path( + "confirm/", + self.admin_site.admin_view(self.confirm_command_view), + name=f"{options.app_label}_{options.model_name}_confirm", + ), + path( + "ui/schema.json", + self.admin_site.admin_view(self.schema_view), + name=f"{options.app_label}_{options.model_name}_schema", + ), + ] + super().get_urls() + + def _check_add_permission(self, request): + permission = f"{self.opts.app_label}.add_{self.opts.model_name}" + if not request.user.has_perm(permission): + raise PermissionDenied + + def schema_view(self, request): + """Returns the schemas of the command types the wizard offers. + + The type choices are the union over the organizations the user + manages, so the schemas are too: the editor fetches them once on + page load, before an organization has been picked. + """ + self._check_add_permission(request) + if request.user.is_superuser: + return JsonResponse(Command.get_org_schema()) + schemas = {} + for organization_id in request.user.organizations_managed: + schemas.update( + Command.get_org_schema(organization_id=organization_id) or {} + ) + return JsonResponse(schemas) + + def execute_command_view(self, request): + """First step of the mass command workflow: collect the details. + A valid submission goes to the session and redirects to the confirm + page, so that its device table can be paginated with plain GETs. + """ + self._check_add_permission(request) + if request.method == "POST": + form = BatchCommandExecutionForm(request.POST, request=request) + if form.is_valid(): + request.session[self.session_key] = form.to_session() + return redirect( + f"admin:{self.opts.app_label}_{self.opts.model_name}_confirm" + ) + else: + request.session.pop(self.session_key, None) + form = BatchCommandExecutionForm(request=request) + context = { + **self.admin_site.each_context(request), + "title": _("Execute mass command"), + "opts": self.opts, + "form": form, + # not combined with self.media: ModelAdmin.media loads + # jquery.init.js before select2, the opposite of what select2 + # needs (see BatchCommandExecutionForm.Media) + "media": form.media, + "has_view_permission": self.has_view_permission(request), + } + return TemplateResponse(request, self.execute_command_template, context) + + def confirm_command_view(self, request): + """Second step: review the targeted devices and dispatch the command. + Dispatching is decided by the HTTP method alone. + """ + self._check_add_permission(request) + if request.method == "POST": + return self._execute_batch_command(request) + wizard = request.session.get(self.session_key) + if not wizard: + return self._restart(request) + devices = self._resolve_target_queryset(request, wizard) + wizard["devices_digest"] = self._devices_digest(devices) + request.session[self.session_key] = wizard + device_admin = self.get_device_admin(devices) + # changelist_view() assembles the whole changelist context (cl, + # media, pagination) and renders the change_list_template of the + # mixin, which is the confirm page extending the stock changelist + # template + return device_admin.changelist_view( + request, extra_context=self._confirm_context(request, wizard, devices) + ) + + def get_device_admin(self, devices): + """Builds the ModelAdmin rendering the device table of the confirm page. + Composed with whichever ModelAdmin is registered for Device, so the + table shows the columns of the device changelist as it actually is: + openwisp-monitoring replaces that registration to add its own. + Resolved per request, when the registration is final. + """ + # TODO: replace _registry with get_model_admin once Django 4.2 is dropped + registered = self.admin_site._registry[Device].__class__ + # the mixin comes first so that its attributes win over the + # registered admin's + device_admin_class = type( + "BatchCommandDeviceAdmin", + (BatchCommandDeviceAdminMixin, registered), + {"readonly_fields": list(registered.readonly_fields)}, + ) + return device_admin_class(Device, self.admin_site, devices=devices) + + def get_device_changelist_template(self): + """The template the registered Device admin renders its changelist with. + The confirm page extends it rather than the stock one, because that + is where other modules load the assets their columns need. Read from + the class: django-import-export rewrites it on the instance. + """ + # TODO: replace _registry with get_model_admin once Django 4.2 is dropped + registered = self.admin_site._registry[Device].__class__ + return getattr(registered, "change_list_template", None) or ( + "admin/change_list.html" + ) + + def _restart(self, request): + """Sends the user back to step one when there is no wizard to show.""" + self.message_user( + request, + _("Please fill in the mass command details to continue."), + messages.WARNING, + ) + return redirect(f"admin:{self.opts.app_label}_{self.opts.model_name}_execute") + + def _resolve_target_queryset(self, request, wizard): + """Devices matched by the organization, group and location chosen. + The targeting rule lives on the model so this page and the execution + cannot drift apart; the multitenancy scope and the ordering the + pagination needs are admin concerns, applied on top. + """ + try: + devices = BatchCommand.dry_run( + organization_id=wizard.get("organization_id"), + group_id=wizard.get("group_id"), + location_id=wizard.get("location_id"), + )["devices"] + except (ObjectDoesNotExist, ValidationError) as error: + logger.warning( + "Failed to resolve devices for mass command wizard" + " (organization_id=%s, group_id=%s, location_id=%s): %s", + wizard.get("organization_id"), + wizard.get("group_id"), + wizard.get("location_id"), + error, + ) + return Device.objects.none() + if not request.user.is_superuser: + devices = devices.filter( + organization_id__in=request.user.organizations_managed + ) + return devices.distinct().order_by("name") + + def _devices_digest(self, devices): + """Identifies the set of devices a confirm page was rendered with, + so that only the reviewed set is executed. + """ + pks = sorted(str(pk) for pk in devices.values_list("pk", flat=True)) + return hashlib.sha256(",".join(pks).encode()).hexdigest() + + def _confirm_context(self, request, wizard, devices): + targets = [] + for model, key in ( + (Organization, "organization_id"), + (DeviceGroup, "group_id"), + (Location, "location_id"), + ): + if not wizard.get(key): + continue + target = model.objects.filter(pk=wizard[key]).first() + if target: + targets.append(str(target)) + command_types = dict(BatchCommand._meta.get_field("type").choices) + return { + "title": _("Review mass command"), + "batch_opts": self.opts, + # the template this page extends, see get_device_changelist_template() + "device_changelist_template": self.get_device_changelist_template(), + "wizard": wizard, + "command_type_display": command_types.get(wizard["type"], wizard["type"]), + "command_description": self._describe_input(wizard.get("input")), + "targets_display": ", ".join(targets) if targets else _("All devices"), + "device_count": devices.count(), + "has_view_permission": self.has_view_permission(request), + } + + def _describe_input(self, command_input): + """Renders the submitted input for the review step, so that every + registered command type is shown and not only custom ones. + """ + if not isinstance(command_input, dict): + return "" + if "command" in command_input: + return command_input["command"] + return ", ".join( + f"{key}: {value}" + for key, value in command_input.items() + if "password" not in key + ) + + def _execute_batch_command(self, request): + """Applies the device selection and dispatches the mass command. + Only the wizard the confirm page was rendered with is executed, and + it is removed before the batch is created, so a double submit finds + nothing and restarts. + """ + wizard = request.session.get(self.session_key) + if not wizard or request.POST.get("token") != wizard.get("token"): + return self._restart(request) + del request.session[self.session_key] + devices = self._resolve_target_queryset(request, wizard) + if self._devices_digest(devices) != wizard.get("devices_digest"): + request.session[self.session_key] = wizard + self.message_user( + request, + _("The targeted devices changed, please review them again."), + messages.WARNING, + ) + return redirect( + f"admin:{self.opts.app_label}_{self.opts.model_name}_confirm" + ) + # The confirm page only lists the devices matched on the execute + # page, so the selection can only ever remove from that set: the + # browser never supplies a device to add. + excluded = self._get_pk_list(request.POST, "excluded") + selection = devices.exclude(pk__in=excluded) + kwargs = { + "type": wizard["type"], + "label": wizard["label"], + "input": wizard.get("input"), + "notes": wizard.get("notes") or "", + "organization_id": wizard.get("organization_id"), + "group_id": wizard.get("group_id"), + "location_id": wizard.get("location_id"), + "devices": list(selection.distinct()), + } + try: + batch = BatchCommand.execute(**kwargs) + except ObjectDoesNotExist: + return self._restart(request) + except ValidationError as error: + # put the wizard back so the user can correct the selection + request.session[self.session_key] = wizard + self.message_user(request, error.messages[0], messages.ERROR) + return redirect( + f"admin:{self.opts.app_label}_{self.opts.model_name}_confirm" + ) + self.message_user( + request, _("Mass command executed successfully."), messages.SUCCESS + ) + return redirect( + f"admin:{self.opts.app_label}_{self.opts.model_name}_change", batch.pk + ) + + @staticmethod + def _get_uuid(value): + try: + return str(UUID(str(value))) + except (AttributeError, TypeError, ValueError): + return "" + + @staticmethod + def _get_pk_list(source, name): + pks = ( + BatchCommandAdmin._get_uuid(pk) for pk in source.get(name, "").split(",") + ) + return [pk for pk in pks if pk] + + def get_readonly_fields(self, request, obj=None): + fields = super().get_readonly_fields(request, obj) + return fields + list(self.__class__.readonly_fields) + + def get_queryset(self, request): + return ( + super() + .get_queryset(request) + .annotate(_affected_devices=Count("batch_commands", distinct=True)) + ) + + def get_object(self, request, object_id, from_field=None): + """Avoids duplicating queries in change_view custom logic""" + cache_attr = f"_cached_object_{object_id}_{from_field}" + if not hasattr(request, cache_attr): + setattr( + request, cache_attr, super().get_object(request, object_id, from_field) + ) + return getattr(request, cache_attr) + + def _get_commands(self, request, obj): + qs = Command.objects.filter(batch_command=obj).select_related("device") + if not request.user.is_superuser: + qs = qs.filter( + device__organization_id__in=request.user.organizations_managed + ) + return qs + + def organization_display(self, obj): + if obj.organization: + return obj.organization.name + # Will return Shared systemwide (no organization) after + # https://github.com/openwisp/openwisp-users/issues/238 + return _("All") + + organization_display.short_description = _("organization") + organization_display.admin_order_field = "organization" + + def colored_status(self, obj): + css_class = f"command-status {obj.status}" + return format_html( + '{1}', + css_class, + obj.get_status_display(), + ) + + colored_status.short_description = _("status") + + def formatted_input(self, obj): + if not obj.input: + return "-" + if obj.type == "change_password": + return "********" + return self._describe_input(obj.input) or "-" + + formatted_input.short_description = _("input") + + def affected_devices(self, obj): + count = getattr(obj, "_affected_devices", None) + if count is None: + count = obj.affected_devices + return count + + affected_devices.short_description = _("affected devices") + affected_devices.admin_order_field = "_affected_devices" + + def display_skipped_devices(self, obj): + if not obj.skipped_devices: + return "-" + rows = obj.get_skipped_preview() + lines = [str(len(obj.skipped_devices))] + lines += [ + format_html("{}: {}", row["device_name"], row["output"]) for row in rows + ] + if len(rows) < len(obj.skipped_devices): + lines.insert(-1, "\u2026") + return format_html( + '
{}
', + format_html_join(mark_safe("
"), "{}", ((line,) for line in lines)), + ) + + display_skipped_devices.short_description = _("skipped devices") + + def _build_filter_specs( + self, + request, + obj, + current_status, + current_location=None, + current_group=None, + current_org=None, + ): + filter_specs = [] + params = request.GET.copy() + params.pop("page", None) + + def _make_choice(current_value, display, param_name, value): + q = params.copy() + q.pop(param_name, None) + if value: + q[param_name] = value + qs = q.urlencode() + query_string = f"?{qs}" if qs else "" + return { + "display": display, + "selected": current_value == value, + "query_string": query_string, + } + + status_choices = [] + for status_value, display_name in ( + (("", _("All")),) + Command.STATUS_CHOICES + (("skipped", _("skipped")),) + ): + status_choices.append( + _make_choice(current_status, display_name, "status", status_value) + ) + + filter_specs.append(SimpleNamespace(title=_("status"), choices=status_choices)) + + batch_devices = Device.objects.filter( + Q(command__batch_command=obj) | Q(pk__in=obj.skipped_devices.keys()) + ) + + # Location filter + location_spec = self._build_related_filter( + _("location"), + "location_id", + current_location or "", + batch_devices.exclude(devicelocation__location__isnull=True) + .values_list( + "devicelocation__location__id", + "devicelocation__location__name", + ) + .distinct(), + _make_choice, + ) + if location_spec: + filter_specs.append(location_spec) + + # Group filter + group_spec = self._build_related_filter( + _("device group"), + "group_id", + current_group or "", + batch_devices.filter(group__isnull=False) + .values_list("group__id", "group__name") + .distinct(), + _make_choice, + ) + if group_spec: + filter_specs.append(group_spec) + + # Organization filter (superusers only) + if request.user.is_superuser: + org_spec = self._build_related_filter( + _("organization"), + "organization_id", + current_org or "", + batch_devices.values_list( + "organization__id", "organization__name" + ).distinct(), + _make_choice, + ) + if org_spec: + filter_specs.append(org_spec) + + return filter_specs + + def _build_related_filter(self, title, param_name, current_value, qs, make_choice): + choices = [make_choice(current_value, _("All"), param_name, "")] + for obj_id, obj_name in qs: + if obj_id: + choices.append( + make_choice(current_value, obj_name, param_name, str(obj_id)) + ) + if len(choices) <= 1: + return None + return SimpleNamespace(title=title, choices=choices) + + @staticmethod + def _command_row(command): + return { + "device_name": command.device.name, + "device": command.device.pk, + "status": command.status, + "status_display": command.get_status_display(), + "output": command.output_preview, + "modified": command.modified, + "is_skipped": False, + } + + def _paginate_commands(self, commands_qs, skipped_items, page_param, per_page=None): + """Returns one page of rows without loading the whole batch in memory. + Commands keep the ordering of ``AbstractCommand.Meta`` ("created"), + so the newest is always last and the change page can append results + live. Skipped devices are not Command rows, they come as a list and + follow the commands. + """ + per_page = per_page or self.device_commands_per_page + commands_count = commands_qs.count() + total = commands_count + len(skipped_items) + paginator = Paginator(range(total), per_page) + try: + page_obj = paginator.page(page_param or 1) + except (PageNotAnInteger, EmptyPage): + page_obj = paginator.page(1) + start = (page_obj.number - 1) * per_page + end = start + per_page + commands_end = min(end, commands_count) + rows = [ + self._command_row(command) for command in commands_qs[start:commands_end] + ] + skipped_start = max(0, start - commands_count) + skipped_end = max(0, end - commands_count) + rows += [ + BatchCommand.build_skipped_row(pk, skipped) + for pk, skipped in skipped_items[skipped_start:skipped_end] + ] + return page_obj, paginator, rows + + def _get_active_filters(self, request): + return { + "q": request.GET.get("q", ""), + "status": request.GET.get("status", ""), + "location_id": self._get_uuid(request.GET.get("location_id", "")), + "group_id": self._get_uuid(request.GET.get("group_id", "")), + "organization_id": self._get_uuid(request.GET.get("organization_id", "")), + } + + def _apply_command_filters(self, qs, filters): + status = filters["status"] + if status == "skipped": + return qs.none() + if filters["q"]: + qs = qs.filter(device__name__icontains=filters["q"]) + if status: + qs = qs.filter(status=status) + if filters["location_id"]: + qs = qs.filter(device__devicelocation__location_id=filters["location_id"]) + if filters["group_id"]: + qs = qs.filter(device__group_id=filters["group_id"]) + if filters["organization_id"]: + qs = qs.filter(device__organization_id=filters["organization_id"]) + return qs + + def _get_matching_skipped_devices(self, obj, filters): + related = ( + filters["organization_id"], + filters["group_id"], + filters["location_id"], + ) + matching = None + if any(related): + organization_id, group_id, location_id = related + devices = Device.objects.filter(pk__in=obj.skipped_devices.keys()) + if organization_id: + devices = devices.filter(organization_id=organization_id) + if group_id: + devices = devices.filter(group_id=group_id) + if location_id: + devices = devices.filter(devicelocation__location_id=location_id) + matching = {str(pk) for pk in devices.values_list("pk", flat=True)} + query = filters["q"].lower() if filters["q"] else "" + return [ + (pk, skipped) + for pk, skipped in obj.skipped_devices.items() + if (matching is None or pk in matching) + and (not query or query in skipped["name"].lower()) + ] + + def change_view(self, request, object_id, form_url="", extra_context=None): + extra_context = extra_context or {} + obj = self.get_object(request, object_id) + if obj: + commands_qs = self._get_commands(request, obj) + filters = self._get_active_filters(request) + commands_qs = self._apply_command_filters(commands_qs, filters) + skipped_items = [] + if obj.skipped_devices and filters["status"] in ("", "skipped"): + skipped_items = self._get_matching_skipped_devices(obj, filters) + page_obj, paginator, commands = self._paginate_commands( + commands_qs, skipped_items, request.GET.get("page", 1) + ) + filter_specs = self._build_filter_specs( + request, + obj, + filters["status"], + current_location=filters["location_id"], + current_group=filters["group_id"], + current_org=filters["organization_id"], + ) + extra_context.update( + { + "commands": commands, + "page_obj": page_obj, + "paginator": paginator, + "filter_specs": filter_specs, + "has_active_filters": any( + value for key, value in filters.items() if key != "q" + ), + } + ) + return super().change_view(request, object_id, extra_context=extra_context) + + +admin.site.register(BatchCommand, BatchCommandAdmin) diff --git a/openwisp_controller/connection/apps.py b/openwisp_controller/connection/apps.py index f67e326d0..7c2485527 100644 --- a/openwisp_controller/connection/apps.py +++ b/openwisp_controller/connection/apps.py @@ -1,18 +1,24 @@ +import logging + from asgiref.sync import async_to_sync from channels import layers from django.apps import AppConfig from django.db import transaction from django.db.models.signals import post_save +from django.utils.formats import date_format +from django.utils.timezone import localtime from django.utils.translation import gettext_lazy as _ from openwisp_notifications.signals import notify from openwisp_notifications.types import register_notification_type from swapper import get_model_name, load_model -from openwisp_utils.admin_theme.menu import register_menu_subitem +from openwisp_utils.admin_theme.menu import register_menu_group, register_menu_subitem from ..config.signals import config_deactivating, config_modified from .signals import is_working_changed +logger = logging.getLogger(__name__) + class ConnectionConfig(AppConfig): name = "openwisp_controller.connection" @@ -37,6 +43,7 @@ def ready(self): Config = load_model("config", "Config") Credentials = load_model("connection", "Credentials") Command = load_model("connection", "Command") + BatchCommand = load_model("connection", "BatchCommand") config_modified.connect( self.config_modified_receiver, dispatch_uid="connection.update_config" @@ -61,24 +68,79 @@ def ready(self): sender=Command, dispatch_uid="command_save_handler", ) + post_save.connect( + self.batch_command_save_receiver, + sender=BatchCommand, + dispatch_uid="batch_command_save_handler", + ) @classmethod def config_modified_receiver(cls, **kwargs): transaction.on_commit(lambda: cls._launch_update_config(kwargs["device"])) + @classmethod + def _send_batch_update(cls, group, data): + def send(): + try: + async_to_sync(layers.get_channel_layer().group_send)( + group, {"type": "send.update", "data": data} + ) + except Exception: + logger.exception("Failed to send update to %s", group) + + transaction.on_commit(send) + @classmethod def command_save_receiver(cls, sender, created, instance, **kwargs): from .api.serializers import CommandSerializer - channel_layer = layers.get_channel_layer() - if created: - # Trigger websocket message only when command status is updated + if created and not instance.batch_command_id: return serialized_data = CommandSerializer(instance).data - async_to_sync(channel_layer.group_send)( - f"config.device-{instance.device_id}", - {"type": "send.update", "model": "Command", "data": serialized_data}, - ) + if not created: + async_to_sync(layers.get_channel_layer().group_send)( + f"config.device-{instance.device_id}", + {"type": "send.update", "model": "Command", "data": serialized_data}, + ) + if instance.batch_command_id: + batch_data = dict(serialized_data) + batch_data.pop("input", None) + batch_data["device_name"] = instance.device.name + batch_data["status_display"] = instance.get_status_display() + batch_data["output"] = instance.output_preview + batch_data["modified"] = date_format( + localtime(instance.modified), "DATETIME_FORMAT" + ) + batch_data["type"] = "command_update" + if created: + batch = instance.batch_command + index = getattr(instance, "_batch_index", None) + if index is None: + index = batch.affected_devices - 1 + affected_devices = index + 1 + batch_data["index"] = index + batch_data["affected_devices"] = affected_devices + batch_data["total_rows"] = affected_devices + len( + batch.skipped_devices or {} + ) + cls._send_batch_update( + f"config.batchcommand-{instance.batch_command_id}", batch_data + ) + + @classmethod + def batch_command_save_receiver(cls, sender, instance, **kwargs): + from .api.serializers import BatchCommandSerializer + + batch_data = BatchCommandSerializer(instance).data + batch_data["status_display"] = instance.get_status_display() + batch_data["type"] = "batch_status" + affected_devices = instance.affected_devices + skipped_count = len(batch_data.pop("skipped_devices", None) or {}) + batch_data["affected_devices"] = affected_devices + batch_data["total_rows"] = affected_devices + skipped_count + batch_data["skipped_count"] = skipped_count + batch_data["skipped_preview"] = instance.get_skipped_preview() + cls._send_batch_update(f"config.batchcommand-{instance.pk}", batch_data) @classmethod def _launch_update_config(cls, device): @@ -188,3 +250,24 @@ def register_menu_groups(self): "icon": "ow-access-credential", }, ) + register_menu_group( + position=35, + config={ + "label": _("Network Operations"), + "icon": "ow-build", + "items": { + 1: { + "label": _("Mass command admin"), + "model": get_model_name("connection", "BatchCommand"), + "name": "changelist", + "icon": "ow-mass-upgrade", + }, + 2: { + "label": _("Mass command execute"), + "model": get_model_name("connection", "BatchCommand"), + "name": "execute", + "icon": "ow-mass-upgrade", + }, + }, + }, + ) diff --git a/openwisp_controller/connection/base/models.py b/openwisp_controller/connection/base/models.py index 75290a2d7..67d195936 100644 --- a/openwisp_controller/connection/base/models.py +++ b/openwisp_controller/connection/base/models.py @@ -516,7 +516,7 @@ def __str__(self): def clean(self): if self.device.is_fully_deactivated(): - raise ValidationError({"device": _("Device is deactivated.")}) + raise ValidationError({"device": _("Device is deactivated")}) self._verify_command_type_allowed() self._verify_connection() try: @@ -527,7 +527,7 @@ def clean(self): def _verify_connection(self): """Raises validation error if device has no connection and credentials.""" if self.device and not self.device.deviceconnection_set.exists(): - raise ValidationError({"device": _("Device has no credentials assigned.")}) + raise ValidationError({"device": _("Device has no credentials assigned")}) def _verify_command_type_allowed(self): """Raises validation error if command type is not allowed.""" @@ -550,6 +550,20 @@ def _verify_command_type_allowed(self): } ) + @property + def output_preview(self): + """Last line of the output, for tables which list many commands.""" + lines = (self.output or "").strip().splitlines() + if not lines: + return "" + max_length = 100 + line = lines[-1] + truncated = len(lines) > 1 + if len(line) > max_length: + line = line[-max_length:] + truncated = True + return f"… {line}" if truncated else line + @property def is_custom(self): return self.type == "custom" @@ -792,10 +806,10 @@ class AbstractBatchCommand(ValidateOrgMixin, TimeStampedEditableModel): blank=True, null=True, default=dict, - verbose_name=_("Skipped devices"), + verbose_name=_("skipped devices"), help_text=_( - "Maps device UUIDs to validation error messages for devices " - "that were skipped during command creation." + "Maps device UUIDs to the name of the device and the validation " + "error that caused it to be skipped during command creation." ), ) @@ -807,8 +821,33 @@ class Meta: def __str__(self): return self.label - @cached_property + @property def total_devices(self): + return self.affected_devices + len(self.skipped_devices or {}) + + @staticmethod + def build_skipped_row(device_pk, skipped): + return { + "device": str(device_pk), + "device_name": skipped["name"], + "status": "skipped", + "status_display": gettext("skipped"), + "output": skipped["error"], + "modified": None, + "is_skipped": True, + } + + def get_skipped_rows(self, start=0, end=None): + items = list((self.skipped_devices or {}).items())[start:end] + return [self.build_skipped_row(pk, skipped) for pk, skipped in items] + + def get_skipped_preview(self, limit=10): + if len(self.skipped_devices or {}) <= limit: + return self.get_skipped_rows() + return self.get_skipped_rows(end=2) + self.get_skipped_rows(start=-1) + + @property + def affected_devices(self): return self.batch_commands.count() @property @@ -843,7 +882,7 @@ def _validate_org_relations(self): return self._validate_org_relation("group", field_error="group") self._validate_org_relation("location", field_error="location") - if self.pk and self.devices.exists(): + if not self._state.adding and self.devices.exists(): org_mismatch = self.devices.exclude(organization=self.organization).exists() if org_mismatch: raise ValidationError( @@ -884,12 +923,13 @@ def clean(self): def resolve_devices(self): """ - Returns an iterator of devices targeted by this batch command, + Returns a queryset of devices targeted by this batch command, resolved from explicit M2M devices or filtered by organization, - group, and location. Returns an empty iterator if no devices match. + group, and location. Callers which walk the whole result should + consume it with iterator(). """ if self.pk and self.devices.exists(): - return self.devices.select_related("config").iterator() + return self.devices.select_related("config") Device = load_model("config", "Device") qs = Device.objects.select_related("config") if self.organization_id: @@ -898,7 +938,7 @@ def resolve_devices(self): qs = qs.filter(group=self.group) if self.location: qs = qs.filter(devicelocation__location=self.location) - return qs.iterator() + return qs @classmethod def execute(cls, **kwargs): @@ -912,12 +952,13 @@ def execute(cls, **kwargs): with transaction.atomic(): batch.full_clean() batch.save() - if devices_list is not None: + if devices_list is None: + devices_list = list(batch.resolve_devices()) batch.devices.set(devices_list) - batch._validate_org_relations() else: - batch.devices.set(list(batch.resolve_devices())) - if not batch.devices.exists(): + batch.devices.set(devices_list) + batch._validate_org_relations() + if not devices_list: raise ValidationError( _("No devices match the specified criteria."), ) @@ -946,7 +987,7 @@ def dry_run(cls, **kwargs): cls._validate_devices_org(devices_list, batch.organization_id) if devices_list is not None: return {"devices": list(devices_list)} - return {"devices": list(batch.resolve_devices())} + return {"devices": batch.resolve_devices()} def _clean_sensitive_info(self): if self.type == "change_password": @@ -964,12 +1005,14 @@ def create_commands(self): ) if not updated: return - self.refresh_from_db(fields=["status"]) + self.status = "in-progress" + self.save(update_fields=["status"]) Command = load_model("connection", "Command") Device = load_model("config", "Device") self.skipped_devices = {} device_pks = [] - for device in self.resolve_devices(): + created_count = 0 + for device in self.resolve_devices().iterator(): device_pks.append(device.pk) command = Command( device=device, @@ -978,15 +1021,17 @@ def create_commands(self): batch_command=self, ) try: - # Validate before the atomic block so errors like - # ValidationError don't create/rollback command.full_clean() - with transaction.atomic(): - command.save() + command._batch_index = created_count + command.save() + created_count += 1 except ValidationError as e: - self.skipped_devices[str(device.pk)] = ( - e.messages if hasattr(e, "messages") else [str(e)] - ) + self.skipped_devices[str(device.pk)] = { + "name": device.name, + "error": ( + ", ".join(e.messages) if hasattr(e, "messages") else str(e) + ), + } logger.warning( "Skipping device %s for batch %s: %s", device.pk, @@ -1012,52 +1057,50 @@ def calculate_and_update_status(self): - All commands completed successfully: status set to "success". - Status unchanged: no database write performed. """ - with transaction.atomic(): - batch = self.__class__.objects.select_for_update().get(pk=self.pk) - stats = batch.batch_commands.aggregate( - total_operations=models.Count("id"), - in_progress=models.Count( - models.Case( - models.When(status="in-progress", then=1), - output_field=models.IntegerField(), - ) - ), - completed=models.Count( - models.Case( - models.When(~models.Q(status="in-progress"), then=1), - output_field=models.IntegerField(), - ) - ), - successful=models.Count( - models.Case( - models.When(status="success", then=1), - output_field=models.IntegerField(), - ) - ), - failed=models.Count( - models.Case( - models.When(status="failed", then=1), - output_field=models.IntegerField(), - ) - ), - ) - if stats["total_operations"] == 0: - if batch.skipped_devices: - new_status = "failed" - else: - new_status = "idle" - elif stats["in_progress"] > 0: - new_status = "in-progress" - elif stats["failed"] > 0: + batch = self.__class__.objects.get(pk=self.pk) + stats = batch.batch_commands.aggregate( + total_operations=models.Count("id"), + in_progress=models.Count( + models.Case( + models.When(status="in-progress", then=1), + output_field=models.IntegerField(), + ) + ), + completed=models.Count( + models.Case( + models.When(~models.Q(status="in-progress"), then=1), + output_field=models.IntegerField(), + ) + ), + successful=models.Count( + models.Case( + models.When(status="success", then=1), + output_field=models.IntegerField(), + ) + ), + failed=models.Count( + models.Case( + models.When(status="failed", then=1), + output_field=models.IntegerField(), + ) + ), + ) + if stats["total_operations"] == 0: + if batch.skipped_devices: new_status = "failed" - elif ( - stats["successful"] > 0 - and stats["completed"] == stats["total_operations"] - ): - if batch.skipped_devices: - new_status = "failed" - else: - new_status = "success" - if batch.status != new_status: - batch.status = new_status - batch.save(update_fields=["status"]) + else: + new_status = "idle" + elif stats["in_progress"] > 0: + new_status = "in-progress" + elif stats["failed"] > 0: + new_status = "failed" + elif ( + stats["successful"] > 0 and stats["completed"] == stats["total_operations"] + ): + if batch.skipped_devices: + new_status = "failed" + else: + new_status = "success" + if batch.status != new_status: + batch.status = new_status + batch.save(update_fields=["status"]) diff --git a/openwisp_controller/connection/channels/consumers.py b/openwisp_controller/connection/channels/consumers.py index 7b4955c47..8fc6b34d4 100644 --- a/openwisp_controller/connection/channels/consumers.py +++ b/openwisp_controller/connection/channels/consumers.py @@ -1,11 +1,18 @@ import json +import logging from copy import deepcopy +from django.utils.formats import date_format +from django.utils.timezone import localtime from swapper import load_model from ...config.base.channels_consumer import BaseDeviceConsumer +from ..api.serializers import BatchCommandSerializer, CommandSerializer + +logger = logging.getLogger(__name__) Device = load_model("config", "Device") +BatchCommand = load_model("connection", "BatchCommand") class CommandConsumer(BaseDeviceConsumer): @@ -13,3 +20,98 @@ def send_update(self, event): data = deepcopy(event) data.pop("type") self.send(json.dumps(data)) + + +class BatchCommandConsumer(BaseDeviceConsumer): + model = BatchCommand + channel_layer_group = "config.batchcommand" + per_page = 20 + current_state_message = "request_current_state" + + def send_update(self, event): + self.send(json.dumps(event["data"])) + + def is_user_authorized(self): + user = self.scope["user"] + if user.is_superuser: + return True + opts = self.model._meta + if not user.is_staff or not any( + user.has_perm(f"{opts.app_label}.{action}_{opts.model_name}") + for action in ("view", "change") + ): + return False + organization_id = ( + self.model.objects.filter(pk=self.scope["url_route"]["kwargs"]["pk"]) + .values_list("organization_id", flat=True) + .first() + ) + return bool(organization_id) and user.is_manager(str(organization_id)) + + def receive(self, text_data): + try: + content = json.loads(text_data) + except ValueError: + content = None + if not isinstance(content, dict): + logger.warning( + "Received a websocket message which is not a valid JSON object" + ) + return + message_type = content.get("type") + if message_type == self.current_state_message: + self._handle_current_state_request(content.get("page")) + else: + logger.warning(f"Unknown websocket message type received: {message_type}") + + def _handle_current_state_request(self, page=None): + """Handle request for current state of the operation""" + + batch = BatchCommand.objects.filter( + pk=self.scope["url_route"]["kwargs"]["pk"] + ).first() + if not batch: + # deleted after the connection was accepted + return + batch_status = BatchCommandSerializer(batch).data + batch_status["status_display"] = batch.get_status_display() + commands_count = batch.batch_commands.count() + batch_status["affected_devices"] = commands_count + batch_status["skipped_count"] = len( + batch_status.pop("skipped_devices", None) or {} + ) + batch_status["skipped_preview"] = batch.get_skipped_preview() + try: + page = max(int(page), 1) + except (TypeError, ValueError): + page = 1 + start = (page - 1) * self.per_page + end = start + self.per_page + commands_end = min(end, commands_count) + page_commands = batch.batch_commands.select_related("device")[ + start:commands_end + ] + commands = [] + for command in page_commands: + row = CommandSerializer(command).data + row.pop("input", None) + row["device_name"] = command.device.name + row["status_display"] = command.get_status_display() + row["output"] = command.output_preview + row["modified"] = date_format( + localtime(command.modified), "DATETIME_FORMAT" + ) + commands.append(row) + commands += batch.get_skipped_rows( + max(0, start - commands_count), max(0, end - commands_count) + ) + self.send( + json.dumps( + { + "type": "batch_state", + "batch_status": batch_status, + "commands": commands, + "total_rows": commands_count + batch_status["skipped_count"], + } + ) + ) diff --git a/openwisp_controller/connection/channels/routing.py b/openwisp_controller/connection/channels/routing.py index 2012b86b2..7b8afad04 100644 --- a/openwisp_controller/connection/channels/routing.py +++ b/openwisp_controller/connection/channels/routing.py @@ -8,5 +8,9 @@ def get_routes(consumer=ow_consumer): path( "ws/controller/device//command", consumer.CommandConsumer.as_asgi(), - ) + ), + path( + "ws/controller/batch-command/", + consumer.BatchCommandConsumer.as_asgi(), + ), ] diff --git a/openwisp_controller/connection/filters.py b/openwisp_controller/connection/filters.py new file mode 100644 index 000000000..7d03b9be0 --- /dev/null +++ b/openwisp_controller/connection/filters.py @@ -0,0 +1,36 @@ +from django.contrib import admin +from django.utils.translation import gettext_lazy as _ +from swapper import load_model + +from openwisp_users.multitenancy import MultitenantRelatedOrgFilter + + +class GroupFilter(MultitenantRelatedOrgFilter): + field_name = "group" + parameter_name = "group_id" + title = _("group") + + +class LocationFilter(MultitenantRelatedOrgFilter): + field_name = "location" + parameter_name = "location_id" + title = _("location") + + +class TypeFilter(admin.SimpleListFilter): + title = _("type") + parameter_name = "type" + + def lookups(self, request, model_admin): + BatchCommand = load_model("connection", "BatchCommand") + qs = BatchCommand.objects.all() + if not request.user.is_superuser: + qs = qs.filter(organization_id__in=request.user.organizations_managed) + types = qs.values_list("type", flat=True).distinct() + choices = dict(BatchCommand._meta.get_field("type").choices) + return [(t, choices.get(t, t)) for t in types] + + def queryset(self, request, queryset): + if self.value(): + return queryset.filter(type=self.value()) + return queryset diff --git a/openwisp_controller/connection/migrations/0011_batchcommand_command_batch_command.py b/openwisp_controller/connection/migrations/0011_batchcommand_command_batch_command.py index 8059c39c4..e63a48708 100644 --- a/openwisp_controller/connection/migrations/0011_batchcommand_command_batch_command.py +++ b/openwisp_controller/connection/migrations/0011_batchcommand_command_batch_command.py @@ -94,11 +94,12 @@ class Migration(migrations.Migration): blank=True, default=dict, help_text=( - "Maps device UUIDs to validation error messages for " - "devices that were skipped during command creation." + "Maps device UUIDs to the name of the device and the " + "validation error that caused it to be skipped during " + "command creation." ), null=True, - verbose_name="Skipped devices", + verbose_name="skipped devices", ), ), ( diff --git a/openwisp_controller/connection/static/connection/css/batch-command.css b/openwisp_controller/connection/static/connection/css/batch-command.css new file mode 100644 index 000000000..bf5307b85 --- /dev/null +++ b/openwisp_controller/connection/static/connection/css/batch-command.css @@ -0,0 +1,308 @@ +/* ==== Mass Command Change Page CSS ==== */ +#batchcommand_form .submit-row { + display: none; +} +.commands-title { + font-size: 22px; + font-weight: 300; + margin: 0; + padding: 0; +} +.search-section { + padding: 20px; +} +.search-form { + display: flex; + align-items: center; +} +#main #content .search-icon { + width: 20px; + height: 20px; + margin-right: 15px; + font-size: 16px; + margin-top: -3px; +} +#main #content .search-input { + padding: 10px 15px; +} +#main #content .search-button { + padding: 10px 20px; + margin-left: 15px; +} +.filter-clear-link:hover { + color: var(--ow-color-fg-darker); +} +.results-table { + width: 100%; + border-collapse: collapse; + /* the output column takes whatever these three leave over */ + --device-column: 23%; + --status-column: 15%; + --modified-column: 15%; +} +.results-table th:nth-child(1), +.results-table td:nth-child(1):not(.empty-results) { + width: var(--device-column); +} +.results-table th:nth-child(2), +.results-table td:nth-child(2) { + width: var(--status-column); + white-space: nowrap; +} +.results-table th:nth-child(4), +.results-table td:nth-child(4) { + width: var(--modified-column); + white-space: nowrap; +} +#main #content .device-link { + color: var(--ow-color-primary); + font-weight: bold; +} +.device-name-disabled { + color: var(--body-quiet-color); + font-style: italic; +} +.empty-results { + padding: 40px; + text-align: center; + color: var(--body-quiet-color); + font-style: italic; +} +.pagination { + padding: 15px 20px; + text-align: right; + border-top: 2px solid var(--hairline-color); + background: var(--darkened-bg); +} +.pagination a { + color: var(--body-fg); + text-decoration: none; + margin: 0 5px; +} +.pagination .current-page { + margin: 0 10px; + color: var(--body-quiet-color); +} +.paginator { + color: var(--body-quiet-color); + padding: 10px 20px; + border-bottom: 1px solid var(--hairline-color); + margin: 0; +} +.command-status { + font-weight: bold; +} +.command-status.success { + color: var(--ow-color-success); +} +.command-status.failed { + color: var(--error-fg); +} +.command-status.in-progress { + color: var(--body-quiet-color); +} +.command-status.skipped { + color: var(--body-quiet-color); + opacity: 0.7; +} +.command-output pre { + white-space: pre-wrap; + word-wrap: break-word; + margin: 0; + padding: 0; + font: inherit; + color: inherit; + background: transparent; +} +.skipped-devices-list { + line-height: 1.7; +} +.field-display_skipped_devices .readonly.readonly { + padding: 0; +} + +/* List Filters */ +#main #content .left-arrow { + left: -1.125rem; +} +#main #content .right-arrow { + right: -1.125rem; +} +#main #content #ow-changelist-filter { + padding: 1.25rem 0rem; +} +#main #content .filters-top { + margin-bottom: 0.5rem; +} + +/* ==== Stepper CSS ==== */ +.stepper { + --step-active-bg: var(--ow-color-primary); + --step-active-text: var(--ow-color-white); + --step-inactive-bg: var(--ow-color-fg-light); + --step-inactive-text: var(--ow-color-fg-dark); + --arrow-color: var(--ow-color-fg-dark); + display: inline-flex; + align-items: stretch; + overflow: hidden; + margin-bottom: 1.75rem; +} +.stepper-step { + align-items: center; + cursor: pointer; + display: flex; + gap: 0.75rem; + padding: 0.875rem 1.5rem 0.875rem 0; + position: relative; +} +.stepper-badge { + align-items: center; + border-radius: 50%; + display: flex; + flex-shrink: 0; + font-size: 0.8125rem; + font-weight: 600; + height: 2rem; + justify-content: center; + position: relative; + width: 2rem; + z-index: 1; +} +.stepper-step.active .stepper-badge { + background-color: var(--step-active-bg); + color: var(--step-active-text); +} +.stepper-step.inactive .stepper-badge { + background-color: var(--step-inactive-bg); + color: var(--step-inactive-text); +} +.stepper-label { + display: flex; + flex-direction: column; + gap: 0.2rem; + min-width: 0; +} +.stepper-label-text { + font-size: 0.875rem; + font-weight: 500; + line-height: 1.2; + white-space: nowrap; +} +.stepper-step.active .stepper-label-text { + color: var(--step-active-bg); + font-weight: 600; +} +.stepper-step.inactive .stepper-label-text { + color: var(--step-inactive-text); + font-weight: 500; +} +.stepper-divider { + align-items: center; + display: flex; + flex-shrink: 0; + justify-content: center; + padding: 0.5rem 1rem 0.5rem 0; +} +.stepper-arrow { + color: var(--arrow-color); + display: block; + flex-shrink: 0; + height: 1rem; + width: 1rem; +} + +/* ==== Confirm Page CSS ==== */ +.confirm-command .stepper { + margin-bottom: 1.5rem; +} +.command-summary { + margin-bottom: 1.5rem; +} +.command-summary .form-row { + padding: 8px 12px; +} +.devices-heading { + margin-bottom: 1rem; +} +.confirm-command #changelist { + margin-top: 0; +} +/* no admin actions on this changelist, so the row would be empty */ +.confirm-command #changelist .actions { + display: none; +} +/* Django styles its own rows with "tr:has(.action-select:checked)", which cannot match here */ +.confirm-command #changelist tbody tr:has(.device-checkbox:checked) { + background-color: var(--selected-row); +} +@media (forced-colors: active) { + .confirm-command #changelist tbody tr:has(.device-checkbox:checked) { + background-color: SelectedItem; + } +} +.confirm-command #result_list th.column-select_device, +.confirm-command #result_list td.field-select_device { + text-align: center; + width: 2rem; +} +.execute-form .submit-row { + display: flex; + gap: 0.5rem; + justify-content: flex-end; +} +.execute-batch-command .jsoneditor-wrapper > fieldset.module { + background: none; + border: none; + margin: 0; + padding: 0; +} +.execute-batch-command .jsoneditor-wrapper > fieldset > h2, +.execute-batch-command .jsoneditor h3 { + display: none !important; +} +.execute-batch-command.no-command-type .jsoneditor-wrapper { + display: none; +} +.execute-batch-command #main .jsoneditor-wrapper div.jsoneditor .form-row { + display: flex; + flex-wrap: wrap; + padding: 15px; +} +.execute-batch-command .jsoneditor-wrapper div.jsoneditor label { + font-weight: bold; + margin-left: 0; +} +.execute-batch-command .jsoneditor .form-row > .help { + flex-basis: 100%; +} +.execute-batch-command .jsoneditor .errorlist { + display: none; +} +.execute-batch-command .jsoneditor.command-errors .errorlist { + display: block; +} +.execute-batch-command .jsoneditor .form-row.errors input, +.execute-batch-command .jsoneditor .form-row.errors select, +.execute-batch-command .jsoneditor .form-row.errors textarea { + border-color: var(--border-color); +} +.execute-batch-command .jsoneditor.command-errors .form-row.errors input, +.execute-batch-command .jsoneditor.command-errors .form-row.errors select, +.execute-batch-command .jsoneditor.command-errors .form-row.errors textarea { + border-color: var(--ow-color-danger); +} +.execute-batch-command .jsoneditor .form-row > ul.errorlist { + order: -1; + flex-basis: calc(100% - 160px); + margin-left: 160px; + padding-left: 10px; +} +.execute-batch-command .form-row.field-input { + display: none; +} +.execute-batch-command .form-row.field-input.errors { + display: block !important; +} +.execute-batch-command .form-row.field-input.errors .flex-container { + display: none; +} diff --git a/openwisp_controller/connection/static/connection/js/batch-command.js b/openwisp_controller/connection/static/connection/js/batch-command.js new file mode 100644 index 000000000..e63da7d31 --- /dev/null +++ b/openwisp_controller/connection/static/connection/js/batch-command.js @@ -0,0 +1,308 @@ +"use strict"; + +const DEFAULT_PER_PAGE = 20; +const DEVICE_URL_PLACEHOLDER = "00000000-0000-0000-0000-000000000000"; + +django.jQuery(function ($) { + const batchCommandWebSocket = new ReconnectingWebSocket(getWebSocketUrl(), null, { + debug: false, + automaticOpen: false, + timeoutInterval: 7000, + }); + batchCommandWebSocket.addEventListener("open", function () { + requestCurrentState($, batchCommandWebSocket); + }); + batchCommandWebSocket.addEventListener("message", function (e) { + const data = JSON.parse(e.data); + if (data.type === "command_update") { + handleCommandMessage($, data); + } else if (data.type === "batch_status") { + handleBatchStatusMessage($, data, batchCommandWebSocket); + } else if (data.type === "batch_state") { + handleBatchStateMessage($, data); + } + }); + batchCommandWebSocket.open(); +}); + +function getWebSocketUrl() { + return `${getWebSocketProtocol()}${owControllerApiHost.host}/ws/controller/batch-command/${batchCommandId}`; +} + +function getWebSocketProtocol() { + let protocol = "ws://"; + if (window.location.protocol === "https:") { + protocol = "wss://"; + } + return protocol; +} + +function requestCurrentState($, websocket) { + if (websocket.readyState !== WebSocket.OPEN) { + return; + } + try { + websocket.send( + JSON.stringify({ + type: "request_current_state", + batch_id: batchCommandId, + page: getCurrentPage($), + }), + ); + } catch (error) { + console.error("Error requesting current batch state:", error); + } +} + +function handleCommandMessage($, data) { + updateTotals($, data.affected_devices, data.total_rows); + renderCommand($, data); +} + +function handleBatchStatusMessage($, data, websocket) { + const $status = $(".field-colored_status .readonly .command-status"); + if ($status.length && data.status && data.status_display) { + $status + .removeClass() + .addClass("command-status " + data.status) + .text(data.status_display); + } + updateSkippedDevices($, data); + updateTotals($, data.affected_devices, data.total_rows); + const $table = $("#result_list"); + if ( + websocket && + data.skipped_count && + data.skipped_count !== $table.data("skippedCount") + ) { + $table.data("skippedCount", data.skipped_count); + requestCurrentState($, websocket); + } +} + +function updateSkippedDevices($, data) { + if (!data.skipped_count) { + return; + } + let $list = $(".field-display_skipped_devices .skipped-devices-list"); + if (!$list.length) { + const $readonly = $(".field-display_skipped_devices .readonly"); + if (!$readonly.length) { + return; + } + $list = $("
").addClass("skipped-devices-list"); + $readonly.empty().append($list); + } + $list.empty().append(document.createTextNode(String(data.skipped_count))); + const rows = data.skipped_preview || []; + rows.forEach(function (row, index) { + if (index === rows.length - 1 && rows.length < data.skipped_count) { + $list.append($("
")).append(document.createTextNode("\u2026")); + } + $list + .append($("
")) + .append(document.createTextNode(row.device_name + ": " + row.output)); + }); +} + +function handleBatchStateMessage($, data) { + if (data.batch_status) { + handleBatchStatusMessage($, data.batch_status); + } + updateTotals( + $, + data.batch_status ? data.batch_status.affected_devices : null, + data.total_rows, + ); + if (!data.commands || !Array.isArray(data.commands)) { + return; + } + data.commands.forEach(function (command) { + const $row = $("#batch-command-row-" + command.device); + if ($row.length) { + updateRow($, $row, command); + } else if (!hasActiveFilters()) { + insertRow($, command); + } + }); +} + +function renderCommand($, data) { + const $row = $("#batch-command-row-" + data.device); + if ($row.length) { + updateRow($, $row, data); + } else if (belongsOnCurrentPage($, data)) { + insertRow($, data); + } + // otherwise the row is on another page, the server renders it there +} + +// The server sends the position of newly created results only, the page is +// worked out here from the size the table was rendered with: this keeps the +// first page at "per page" rows while the paginator keeps growing. +function belongsOnCurrentPage($, data) { + // with a filter on, the pushed totals are unfiltered and page boundaries + // cannot be worked out + if (hasActiveFilters()) { + return false; + } + if (data.index == null) { + return false; + } + const renderedRows = $("#result_list tbody tr").not(":has(td.empty-results)").length; + if (renderedRows >= getPerPage($)) { + return false; + } + // the paginator is 1-based, so position 0 is on page 1 + return Math.floor(data.index / getPerPage($)) + 1 === getCurrentPage($); +} + +function updateRow($, $row, data) { + const activeFilter = getActiveStatusFilter($); + if (activeFilter && activeFilter !== data.status) { + $row.remove(); + return; + } + $row + .find(".command-status") + .removeClass() + .addClass("command-status " + data.status) + .text(data.status_display); + $row.find(".command-output pre").text(data.output || "-"); + $row.find("td:last-child").text(data.modified || "-"); +} + +function insertRow($, data) { + $("#result_list td.empty-results").closest("tr").remove(); + const $tableBody = $("#result_list tbody"); + const rowClass = $tableBody.find("tr").length % 2 === 0 ? "row1" : "row2"; + const $row = $("").attr({ + id: "batch-command-row-" + data.device, + "data-device-pk": data.device, + class: rowClass, + }); + if (data.is_skipped) { + $row.append( + $("").append( + $("").addClass("device-name-disabled").text(data.device_name), + ), + ); + } else { + $row.append( + $("").append( + $("") + .attr({ + href: getDeviceChangeUrl($, data.device), + class: "device-link", + }) + .text(data.device_name), + ), + ); + } + $row.append( + $("").append( + $("") + .addClass("command-status " + data.status) + .text(data.status_display), + ), + ); + $row.append( + $("") + .addClass("command-output") + .append($("
").text(data.output || "-")),
+  );
+  $row.append($("").text(data.modified || "-"));
+  $tableBody.append($row);
+}
+
+function updateTotals($, affectedDevices, totalRows) {
+  if (affectedDevices != null) {
+    const $affected = $(".field-affected_devices .readonly");
+    if ($affected.length) {
+      $affected.text(String(affectedDevices));
+    }
+  }
+  // counts are filtered server side, the totals pushed here are not
+  if (totalRows == null || hasActiveFilters()) {
+    return;
+  }
+  const $paginator = $(".results-container .paginator");
+  if ($paginator.length) {
+    $paginator.text(
+      interpolate(ngettext("%s command", "%s commands", totalRows), [totalRows]),
+    );
+  }
+  renderPagination($, totalRows);
+}
+
+function renderPagination($, totalRows) {
+  const currentPage = getCurrentPage($);
+  const perPage = getPerPage($);
+  const totalPages = Math.max(1, Math.ceil(totalRows / perPage));
+  $(".results-container .pagination").remove();
+  if (totalPages <= 1) {
+    return;
+  }
+  const params = new URLSearchParams(window.location.search);
+  params.delete("page");
+  const baseQuery = params.toString();
+  const buildHref = function (page) {
+    return "?" + (baseQuery ? baseQuery + "&page=" + page : "page=" + page);
+  };
+  const $stepLinks = $("").addClass("step-links");
+  if (currentPage > 1) {
+    $stepLinks.append(
+      $("")
+        .attr("href", buildHref(currentPage - 1))
+        .text(gettext("Previous")),
+    );
+  }
+  $stepLinks.append(
+    $("")
+      .addClass("current-page")
+      .text(
+        interpolate(
+          gettext("Page %(current)s of %(total)s"),
+          { current: currentPage, total: totalPages },
+          true,
+        ),
+      ),
+  );
+  if (currentPage < totalPages) {
+    $stepLinks.append(
+      $("")
+        .attr("href", buildHref(currentPage + 1))
+        .text(gettext("Next")),
+    );
+  }
+  $("
").addClass("pagination").append($stepLinks).appendTo(".results-container"); +} + +function getDeviceChangeUrl($, devicePk) { + const template = $("#result_list").attr("data-device-url"); + if (!template) { + return "#"; + } + return template.replace(DEVICE_URL_PLACEHOLDER, devicePk) + "#command_set-2-group"; +} + +function getActiveStatusFilter($) { + return $("#result_list").attr("data-active-status") || ""; +} + +function hasActiveFilters() { + const params = new URLSearchParams(window.location.search); + return ["q", "status", "location_id", "group_id", "organization_id"].some( + function (name) { + return !!params.get(name); + }, + ); +} + +function getCurrentPage($) { + return parseInt($("#result_list").attr("data-current-page"), 10) || 1; +} + +function getPerPage($) { + return parseInt($("#result_list").attr("data-per-page"), 10) || DEFAULT_PER_PAGE; +} diff --git a/openwisp_controller/connection/static/connection/js/execute-command.js b/openwisp_controller/connection/static/connection/js/execute-command.js new file mode 100644 index 000000000..b52577c2c --- /dev/null +++ b/openwisp_controller/connection/static/connection/js/execute-command.js @@ -0,0 +1,263 @@ +"use strict"; + +const EXCLUDED_STORAGE_PREFIX = "ow-batch-command-excluded:"; +const WIZARD_SELECTS = "#id_type, #id_organization, #id_group, #id_location"; +const COMMAND_EDITOR_ID = "id_input_jsoneditor"; + +django.jQuery(function ($) { + initExecuteCommandForm($); + initDeviceSelection($); +}); + +function initExecuteCommandForm($) { + const $typeSelect = $("#id_type"); + if (!$typeSelect.length) { + return; + } + const $form = $typeSelect.closest("form"); + + clearAbandonedExclusions(); + + $(WIZARD_SELECTS).select2({ + theme: "default", + placeholder: gettext("Select an option"), + allowClear: true, + width: "resolve", + }); + + initOrganizationScope($); + initCommandInput($, $typeSelect); + + // admin pages are served no-store, so a back navigation restores the field + // values after select2 has already rendered its labels + $(window).on("pageshow", function () { + $(WIZARD_SELECTS).each(function () { + const $field = $(this); + if ($field.data("select2")) { + $field.trigger("change.select2"); + } + }); + }); + + $form.on("submit", function (event) { + clearFieldErrors($); + const type = $typeSelect.val(); + let hasError = false; + if (!type) { + showFieldError( + $typeSelect.closest(".form-row"), + gettext("This field is required."), + ); + hasError = true; + } + if (!$.trim($("#id_label").val() || "")) { + showFieldError( + $("#id_label").closest(".form-row"), + gettext("This field is required."), + ); + hasError = true; + } + if (showCommandErrors($)) { + hasError = true; + } + if (hasError) { + event.preventDefault(); + event.stopImmediatePropagation(); + } + }); +} + +// mirrors checkInputIsValid() in commands.js: the editor renders its errors as +// soon as it is built, so they are kept hidden until the form is submitted +function showCommandErrors($) { + const editor = (django._jsonEditors || {})[COMMAND_EDITOR_ID]; + if (!editor) { + return false; + } + const errors = editor.validate(); + // a field is redisplayed only when it was edited or when "show_errors" + // changed since the last call, "always" skips both checks + editor.options.show_errors = "always"; + editor.root.showValidationErrors(errors); + $("#" + COMMAND_EDITOR_ID).addClass("command-errors"); + return errors.length > 0; +} + +// with no type the editor is handed the whole schema map and renders it as a +// generic root object +function initCommandInput($, $typeSelect) { + function toggle() { + $(document.body).toggleClass("no-command-type", !$typeSelect.val()); + // the container is reused, its errors belong to the previous type + $("#" + COMMAND_EDITOR_ID).removeClass("command-errors"); + } + + $typeSelect.on("change", toggle); + toggle(); +} + +function initOrganizationScope($) { + const $organization = $("#id_organization"); + const fields = ["#id_group", "#id_location"]; + fields.forEach(function (selector) { + $(selector).data("allOptions", $(selector).find("option").clone()); + }); + + function applyScope() { + const organizationId = $organization.val() || ""; + fields.forEach(function (selector) { + const $field = $(selector); + const current = $field.val(); + const $options = $field.data("allOptions").filter(function () { + const value = $(this).attr("value"); + return ( + !value || + !organizationId || + $(this).attr("data-organization-id") === organizationId + ); + }); + $field.empty().append($options.clone()); + $field.val($options.filter('[value="' + current + '"]').length ? current : ""); + $field.trigger("change.select2"); + }); + } + + $organization.on("change", applyScope); + applyScope(); +} + +function initDeviceSelection($) { + const $form = $("#execute-form"); + if (!$form.length) { + return; + } + // sessionStorage outlives the wizard, so the key is namespaced by its token + const storageKey = EXCLUDED_STORAGE_PREFIX + ($form.data("wizard-token") || ""); + const $table = $("#result_list"); + const $excludedField = $("#id_excluded"); + const $count = $("#selected-count"); + const $button = $("#execute-button"); + const totalDevices = parseInt($form.data("total-devices"), 10) || 0; + const excluded = getStoredExclusions($, storageKey); + + function updateSelectionSummary() { + const pks = Object.keys(excluded); + const selected = Math.max(totalDevices - pks.length, 0); + $excludedField.val(pks.join(",")); + setStoredExclusions(storageKey, pks); + $count.text(selected); + $button.text( + interpolate(ngettext("Execute on %s device", "Execute on %s devices", selected), [ + selected, + ]), + ); + $button.prop("disabled", selected === 0); + updateSelectAllCheckbox(); + } + + function updateSelectAllCheckbox() { + const $checkboxes = $table.find(".device-checkbox"); + $("#select-all-devices").prop( + "checked", + $checkboxes.length > 0 && + $checkboxes.filter(":checked").length === $checkboxes.length, + ); + } + + $table.on("change", ".device-checkbox", function () { + const pk = $(this).val(); + if (this.checked) { + delete excluded[pk]; + } else { + excluded[pk] = true; + } + updateSelectionSummary(); + }); + + // devices the user cannot see are never toggled implicitly + $table.on("change", "#select-all-devices", function () { + const checked = this.checked; + $table.find(".device-checkbox").each(function () { + const $checkbox = $(this); + if ($checkbox.prop("checked") !== checked) { + $checkbox.prop("checked", checked).trigger("change"); + } + }); + }); + + $form.on("submit", function () { + // guards against a double click creating two mass commands + $button.prop("disabled", true); + }); + + renderSelectAllCheckbox($, $table); + restoreDeviceCheckboxes($, $table, excluded); + updateSelectionSummary(); +} + +function renderSelectAllCheckbox($, $table) { + const $header = $table.find("thead th").first(); + if (!$header.length || $header.find("#select-all-devices").length) { + return; + } + $header.append( + $("").attr({ + type: "checkbox", + id: "select-all-devices", + title: gettext("Select all devices on this page"), + }), + ); +} + +function restoreDeviceCheckboxes($, $table, excluded) { + $table.find(".device-checkbox").each(function () { + const $checkbox = $(this); + $checkbox.prop("checked", !excluded[$checkbox.val()]); + }); +} + +// paging the device table is an ordinary page load, which would forget them +function getStoredExclusions($, storageKey) { + const stored = {}; + try { + const raw = window.sessionStorage.getItem(storageKey); + $.each(raw ? JSON.parse(raw) : [], function (index, pk) { + stored[pk] = true; + }); + } catch (error) { + // private browsing modes can make sessionStorage unavailable + } + return stored; +} + +function setStoredExclusions(storageKey, pks) { + try { + window.sessionStorage.setItem(storageKey, JSON.stringify(pks)); + } catch (error) { + // see getStoredExclusions() + } +} + +function clearAbandonedExclusions() { + try { + const storage = window.sessionStorage; + for (let i = storage.length - 1; i >= 0; i--) { + const key = storage.key(i); + if (key && key.indexOf(EXCLUDED_STORAGE_PREFIX) === 0) { + storage.removeItem(key); + } + } + } catch (error) { + // see getStoredExclusions() + } +} + +function clearFieldErrors($) { + $(".form-row.errors").removeClass("errors"); + $(".form-row .errorlist").not(".jsoneditor .errorlist").remove(); +} + +function showFieldError($row, message) { + $row.addClass("errors"); + $row.prepend('
  • ' + message + "
"); +} diff --git a/openwisp_controller/connection/templates/admin/connection/batch_command/batch_command_change_form.html b/openwisp_controller/connection/templates/admin/connection/batch_command/batch_command_change_form.html new file mode 100644 index 000000000..a07f2fba4 --- /dev/null +++ b/openwisp_controller/connection/templates/admin/connection/batch_command/batch_command_change_form.html @@ -0,0 +1,183 @@ +{% extends "admin/change_form.html" %} +{% load i18n admin_urls static admin_list ow_tags %} + +{% block extrahead %} +{{ block.super }} + + + + +{% endblock %} + +{% block content %} +{{ block.super }} + +

{% trans "Commands" %}

+ +{% if filter_specs %} +
+{% endif %} + +
+
+ + + + + + + {% for param, value in request.GET.items %} + {% if param != 'q' and param != 'page' %} + + {% endif %} + {% endfor %} +
+
+ +
+ + + + + + + + + + + {% for command in commands %} + + + + + + + {% empty %} + + + + {% endfor %} + +
{% trans "Device" %}{% trans "Status" %}{% trans "Output" %}{% trans "Modified" %}
+ {% if command.is_skipped %} + {{ command.device_name }} + {% else %} + + {{ command.device_name }} + + {% endif %} + + {{ command.status_display }} + +
{{ command.output|default:"-" }}
+
{{ command.modified|date:"DATETIME_FORMAT"|default:"-" }}
{% trans "No commands found." %}
+ + {% if paginator %} +

+ {% blocktrans count counter=paginator.count %} + {{ counter }} command + {% plural %}{{ counter }} commands + {% endblocktrans %} +

+ {% endif %} + + {% if page_obj.has_other_pages %} + + {% endif %} +
+{% endblock %} + +{% block footer %} +{{ block.super }} + + + +{% endblock %} diff --git a/openwisp_controller/connection/templates/admin/connection/batch_command/confirm_command.html b/openwisp_controller/connection/templates/admin/connection/batch_command/confirm_command.html new file mode 100644 index 000000000..7401ec413 --- /dev/null +++ b/openwisp_controller/connection/templates/admin/connection/batch_command/confirm_command.html @@ -0,0 +1,125 @@ +{% extends device_changelist_template|default:"admin/change_list.html" %} +{% load i18n admin_urls static %} + +{# step two: extends the changelist template of the registered Device admin, #} +{# so the assets of columns added by other modules load too #} +{# see BatchCommandAdmin.get_device_changelist_template() #} + +{% block extrastyle %} +{{ block.super }} + + +{% endblock %} + +{% block extrahead %} +{{ block.super }} + + +{% endblock %} + +{% block bodyclass %}{{ block.super }} confirm-command{% endblock %} + +{# hides the changelist's "Add device" button #} +{% block object-tools %}{% endblock %} + +{% block breadcrumbs %} + +{% endblock %} + +{% block content %} + + +
+

{% trans 'Summary' %}

+
+
+ +
{{ command_type_display }}
+
+
+ {% if command_description %} +
+
+ +
{{ command_description }}
+
+
+ {% endif %} +
+
+ +
{{ wizard.label }}
+
+
+
+
+ +
{{ targets_display }}
+
+
+
+
+ +
+ {{ device_count }} {% trans 'devices' %} +
+
+
+
+
+ +
{{ request.user }} — {% now "DATETIME_FORMAT" %}
+
+
+
+ +
+

{% trans 'Affected devices' %}

+
+ +{# renders the changelist: the device table, its pagination, and the
wrapping both #} +{# HTML does not allow nested forms, so the execute form below is a sibling of that one: #} +{# nesting it made the browser drop the opening tag, leaving the button outside any form #} +{{ block.super }} + + + {% csrf_token %} + + +
+ {% trans 'Back' %} + +
+
+{% endblock %} diff --git a/openwisp_controller/connection/templates/admin/connection/batch_command/execute_command.html b/openwisp_controller/connection/templates/admin/connection/batch_command/execute_command.html new file mode 100644 index 000000000..bcd564b44 --- /dev/null +++ b/openwisp_controller/connection/templates/admin/connection/batch_command/execute_command.html @@ -0,0 +1,80 @@ +{% extends "admin/base_site.html" %} +{% load i18n admin_urls static %} + +{# step one: collects the command details and targets, saves them in the session, #} +{# then redirects to the confirm page #} + +{% block extrastyle %} +{{ block.super }} + + +{{ media.css }} +{% endblock %} + +{% block extrahead %} +{{ block.super }} + +{{ media.js }} +{% endblock %} + +{% block bodyclass %}{{ block.super }} app-{{ opts.app_label }} model-{{ opts.model_name }} execute-batch-command{% endblock %} + +{% block breadcrumbs %} + +{% endblock %} + +{% block content %} +
+
+ {% csrf_token %} + + + {% if form.non_field_errors %} +

{{ form.non_field_errors|join:" " }}

+ {% endif %} + +
+

{% trans "Command" %}

+ {% include "admin/connection/batch_command/form_row.html" with field=form.type %} + {% include "admin/connection/batch_command/form_row.html" with field=form.input %} + {% include "admin/connection/batch_command/form_row.html" with field=form.label %} + {% include "admin/connection/batch_command/form_row.html" with field=form.notes %} +
+ +
+

{% trans "Targets" %}

+ {% include "admin/connection/batch_command/form_row.html" with field=form.organization %} + {% include "admin/connection/batch_command/form_row.html" with field=form.location %} + {% include "admin/connection/batch_command/form_row.html" with field=form.group %} +
+ +
+ {% trans "Cancel" %} + +
+
+
+{% endblock %} diff --git a/openwisp_controller/connection/templates/admin/connection/batch_command/form_row.html b/openwisp_controller/connection/templates/admin/connection/batch_command/form_row.html new file mode 100644 index 000000000..bec22c341 --- /dev/null +++ b/openwisp_controller/connection/templates/admin/connection/batch_command/form_row.html @@ -0,0 +1,12 @@ +
+ {{ field.errors }} +
+ {{ field.label_tag }} + {{ field }} +
+ {% if field.help_text %} +
+
{{ field.help_text }}
+
+ {% endif %} +
diff --git a/openwisp_controller/connection/tests/pytest.py b/openwisp_controller/connection/tests/pytest.py index d7f7f5659..12bba889f 100644 --- a/openwisp_controller/connection/tests/pytest.py +++ b/openwisp_controller/connection/tests/pytest.py @@ -1,18 +1,26 @@ from unittest import mock +from uuid import uuid4 import pytest from channels.db import database_sync_to_async from channels.testing import WebsocketCommunicator from django.conf import settings +from django.contrib.auth import get_user_model +from django.contrib.auth.models import AnonymousUser, Permission from django.utils import timezone from django.utils.module_loading import import_string from swapper import load_model from openwisp_controller.connection.tests.utils import CreateCommandMixin +from .. import apps +from ..channels.consumers import BatchCommandConsumer from .test_models import BaseTestModels +User = get_user_model() Command = load_model("connection", "Command") +BatchCommand = load_model("connection", "BatchCommand") +OrganizationUser = load_model("openwisp_users", "OrganizationUser") @pytest.mark.asyncio @@ -107,3 +115,267 @@ async def test_multiple_connections_receive_updates_with_redis( assert response2 == expected_response await communicator1.disconnect() await communicator2.disconnect() + + +@pytest.mark.asyncio +@pytest.mark.django_db(transaction=True) +class TestBatchCommandConsumer(BaseTestModels, CreateCommandMixin): + application = import_string(getattr(settings, "ASGI_APPLICATION")) + path = "ws/controller/batch-command" + + async def _connect(self, pk, user=None): + communicator = WebsocketCommunicator( + BatchCommandConsumer.as_asgi(), f"{self.path}/{pk}" + ) + communicator.scope["url_route"] = {"kwargs": {"pk": str(pk)}} + if user is not None: + communicator.scope["user"] = user + connected, _ = await communicator.connect() + return communicator, connected + + async def _connect_through_route(self, admin_client, batch): + session_id = admin_client.cookies["sessionid"].value + communicator = WebsocketCommunicator( + self.application, + path=f"{self.path}/{batch.pk}", + headers=[(b"cookie", f"sessionid={session_id}".encode("ascii"))], + ) + connected, _ = await communicator.connect() + return communicator, connected + + async def _drain(self, communicator): + while not await communicator.receive_nothing(): + await communicator.receive_json_from() + + async def _receive_until(self, communicator, message_type, limit=4): + for _ in range(limit): + message = await communicator.receive_json_from() + if message.get("type") == message_type: + return message + raise AssertionError(f"{message_type} was never received") + + @database_sync_to_async + def _create_batch(self, organization=None, **kwargs): + return self._create_batch_command(organization=organization, **kwargs) + + @database_sync_to_async + def _create_staff(self, username, org=None, codenames=(), is_staff=True): + user = self._create_user( + username=username, email=f"{username}@test.com", is_staff=is_staff + ) + user.user_permissions.set(Permission.objects.filter(codename__in=codenames)) + if org is not None: + OrganizationUser.objects.create(user=user, organization=org, is_admin=True) + return User.objects.get(pk=user.pk) + + @database_sync_to_async + def _set_skipped(self, batch, count): + batch.skipped_devices = { + str(uuid4()): {"name": f"skipped{index}", "error": f"error {index}"} + for index in range(count) + } + batch.save(update_fields=["skipped_devices"]) + return list(batch.skipped_devices) + + async def test_batch_command_consumer_authorization(self, admin_user, admin_client): + org = await database_sync_to_async(self._get_org)() + org2 = await database_sync_to_async(self._create_org)(name="org2", slug="org2") + batch = await self._create_batch(organization=org) + communicator, connected = await self._connect_through_route(admin_client, batch) + assert connected is True + await communicator.disconnect() + communicator, connected = await self._connect(batch.pk) + assert connected is False + communicator, connected = await self._connect(batch.pk, AnonymousUser()) + assert connected is False + communicator, connected = await self._connect(batch.pk, admin_user) + assert connected is True + await communicator.disconnect() + manager = await self._create_staff( + "manager", org=org, codenames=["view_batchcommand", "add_batchcommand"] + ) + communicator, connected = await self._connect(batch.pk, manager) + assert connected is True + await communicator.disconnect() + add_only = await self._create_staff( + "add-only", org=org, codenames=["add_batchcommand"] + ) + communicator, connected = await self._connect(batch.pk, add_only) + assert connected is False + outsider = await self._create_staff( + "outsider", org=org2, codenames=["view_batchcommand"] + ) + communicator, connected = await self._connect(batch.pk, outsider) + assert connected is False + non_staff = await self._create_staff( + "non-staff", org=org, codenames=["view_batchcommand"], is_staff=False + ) + communicator, connected = await self._connect(batch.pk, non_staff) + assert connected is False + # a batch without an organization spans every organization + shared_batch = await self._create_batch(organization=None) + communicator, connected = await self._connect(shared_batch.pk, manager) + assert connected is False + communicator, connected = await self._connect(shared_batch.pk, admin_user) + assert connected is True + await communicator.disconnect() + communicator, connected = await self._connect(uuid4(), manager) + assert connected is False + # a rejected connection was never added to a group + await communicator.disconnect() + + @mock.patch("paramiko.SSHClient.connect") + async def test_batch_command_consumer_current_state( + self, mocked_connect, admin_user + ): + org = await database_sync_to_async(self._get_org)() + device_conn = await database_sync_to_async(self._create_device_connection)() + batch = await self._create_batch(organization=org) + with mock.patch.object(Command, "_schedule_command"): + command = await database_sync_to_async(Command.objects.create)( + batch_command=batch, + device=device_conn.device, + connection=device_conn, + type="custom", + input={"command": "echo test"}, + status="success", + output="line one\nline two", + ) + skipped_pks = await self._set_skipped(batch, 3) + await database_sync_to_async(batch.refresh_from_db)() + communicator, connected = await self._connect(batch.pk, admin_user) + assert connected is True + with mock.patch.object(BatchCommandConsumer, "per_page", 2): + await communicator.send_json_to( + {"type": "request_current_state", "page": 1} + ) + page1 = await communicator.receive_json_from() + assert page1["type"] == "batch_state" + assert page1["total_rows"] == 4 + batch_status = page1["batch_status"] + assert batch_status["status_display"] == batch.get_status_display() + assert batch_status["affected_devices"] == 1 + assert batch_status["skipped_count"] == 3 + assert [row["device"] for row in batch_status["skipped_preview"]] == ( + skipped_pks + ) + assert "skipped_devices" not in batch_status + assert [row["device"] for row in page1["commands"]] == [ + str(command.device_id), + skipped_pks[0], + ] + command_row = page1["commands"][0] + assert command_row["device_name"] == device_conn.device.name + assert command_row["status_display"] == command.get_status_display() + assert command_row["output"] == "… line two" + assert command_row["modified"] + assert "input" not in command_row + await communicator.send_json_to( + {"type": "request_current_state", "page": 2} + ) + page2 = await communicator.receive_json_from() + assert [row["device"] for row in page2["commands"]] == skipped_pks[1:] + assert all(row["is_skipped"] for row in page2["commands"]) + for page in (0, -1, "abc", None): + await communicator.send_json_to( + {"type": "request_current_state", "page": page} + ) + response = await communicator.receive_json_from() + assert response["commands"] == page1["commands"] + await communicator.disconnect() + communicator, connected = await self._connect(batch.pk, admin_user) + assert connected is True + await communicator.send_json_to({"type": "request_current_state"}) + response = await communicator.receive_json_from() + assert response["total_rows"] == 4 + assert [row["device"] for row in response["commands"]] == [ + str(command.device_id) + ] + skipped_pks + # a batch deleted after the connection was accepted is not answered + await database_sync_to_async(BatchCommand.objects.filter(pk=batch.pk).delete)() + await communicator.send_json_to({"type": "request_current_state"}) + assert await communicator.receive_nothing() is True + await communicator.disconnect() + + async def test_batch_command_consumer_invalid_messages(self, admin_user): + org = await database_sync_to_async(self._get_org)() + batch = await self._create_batch(organization=org) + communicator, connected = await self._connect(batch.pk, admin_user) + assert connected is True + with mock.patch( + "openwisp_controller.connection.channels.consumers.logger" + ) as logger: + for message in ("not json", "[]", '"string"', "null"): + await communicator.send_to(text_data=message) + assert await communicator.receive_nothing() is True + for message in ({"type": "unknown"}, {}): + await communicator.send_json_to(message) + assert await communicator.receive_nothing() is True + assert logger.warning.call_count == 6 + await communicator.disconnect() + + @mock.patch("paramiko.SSHClient.connect") + async def test_batch_command_consumer_updates(self, mocked_connect, admin_user): + org = await database_sync_to_async(self._get_org)() + device_conn = await database_sync_to_async(self._create_device_connection)() + batch = await self._create_batch(organization=org) + other_batch = await self._create_batch(organization=org, label="other") + communicator, connected = await self._connect(batch.pk, admin_user) + assert connected is True + other, other_connected = await self._connect(other_batch.pk, admin_user) + assert other_connected is True + watcher, watcher_connected = await self._connect(batch.pk, admin_user) + assert watcher_connected is True + with mock.patch.object(Command, "_schedule_command"): + command = await database_sync_to_async(Command.objects.create)( + batch_command=batch, + device=device_conn.device, + connection=device_conn, + type="custom", + input={"command": "echo test"}, + ) + created = await self._receive_until(communicator, "command_update") + assert created["id"] == str(command.pk) + assert created["index"] == 0 + assert created["affected_devices"] == 1 + assert created["total_rows"] == 1 + assert created["device_name"] == device_conn.device.name + assert "input" not in created + assert await self._receive_until(watcher, "command_update") == created + assert await communicator.receive_nothing() is True + command.status = "success" + command.output = "done" + await database_sync_to_async(command.save)() + updated = await self._receive_until(communicator, "command_update") + assert updated["status"] == "success" + assert updated["output"] == "done" + assert "index" not in updated + await self._drain(communicator) + await self._set_skipped(batch, 2) + status = await self._receive_until(communicator, "batch_status") + assert status["skipped_count"] == 2 + assert [row["device_name"] for row in status["skipped_preview"]] == [ + "skipped0", + "skipped1", + ] + assert status["affected_devices"] == 1 + assert status["total_rows"] == 3 + # updates are namespaced per batch + assert await other.receive_nothing() is True + with mock.patch.object( + apps.layers, "get_channel_layer", side_effect=RuntimeError("no layer") + ), mock.patch.object(apps, "logger") as logger, mock.patch.object( + Command, "_schedule_command" + ): + await database_sync_to_async(Command.objects.create)( + batch_command=batch, + device=device_conn.device, + connection=device_conn, + type="custom", + input={"command": "echo test"}, + ) + logger.exception.assert_called_once() + assert await database_sync_to_async(batch.batch_commands.count)() == 2 + await communicator.disconnect() + await watcher.disconnect() + await other.disconnect() diff --git a/openwisp_controller/connection/tests/test_admin.py b/openwisp_controller/connection/tests/test_admin.py index 379346bc2..b9a086319 100644 --- a/openwisp_controller/connection/tests/test_admin.py +++ b/openwisp_controller/connection/tests/test_admin.py @@ -1,19 +1,28 @@ import json from unittest.mock import patch +from uuid import uuid4 +from django.contrib import admin from django.contrib.auth.models import Permission +from django.core.exceptions import ValidationError from django.test import TestCase, override_settings from django.urls import reverse from swapper import load_model -from openwisp_controller.connection.commands import ORGANIZATION_ENABLED_COMMANDS +from openwisp_controller.connection.commands import ( + COMMANDS, + ORGANIZATION_COMMAND_SCHEMA, + ORGANIZATION_ENABLED_COMMANDS, +) from ... import settings as module_settings from ...tests import _get_updated_templates_settings from ...tests.utils import TestAdminMixin +from ..admin import BatchCommandAdmin, BatchCommandExecutionForm from ..connectors.ssh import Ssh +from ..filters import GroupFilter, LocationFilter, TypeFilter from ..widgets import CredentialsSchemaWidget -from .utils import CreateConnectionsMixin +from .utils import BatchCommandMixin, CreateConnectionsMixin Template = load_model("config", "Template") Config = load_model("config", "Config") @@ -22,6 +31,10 @@ DeviceConnection = load_model("connection", "DeviceConnection") Command = load_model("connection", "Command") Group = load_model("openwisp_users", "Group") +DeviceGroup = load_model("config", "DeviceGroup") +Location = load_model("geo", "Location") +DeviceLocation = load_model("geo", "DeviceLocation") +BatchCommand = load_model("connection", "BatchCommand") class TestConnectionAdmin(TestAdminMixin, CreateConnectionsMixin, TestCase): @@ -280,3 +293,669 @@ def test_notification_host_setting(self, ctx_processors=[]): response = self.client.get(url) self.assertContains(response, "https://example.com") self.assertNotContains(response, "owControllerApiHost = window.location") + + +class TestBatchCommandAdmin(BatchCommandMixin, TestCase): + app_label = "connection" + + def setUp(self): + self._create_admin() + self.execute_url = reverse("admin:connection_batchcommand_execute") + self.confirm_url = reverse("admin:connection_batchcommand_confirm") + self.changelist_url = reverse("admin:connection_batchcommand_changelist") + + def test_wizard_permissions_and_tenant_isolation(self): + org = self._get_org() + org2 = self._create_org(name="org2", slug="org2") + self._create_device(organization=org) + device2 = self._create_device( + name="device2", mac_address="00:11:22:33:44:02", organization=org2 + ) + with self.subTest("view permission is not enough"): + viewer = self._create_operator( + organizations=[org], username="viewer", email="viewer@test.com" + ) + viewer.groups.clear() + viewer.user_permissions.set( + Permission.objects.filter(codename="view_batchcommand") + ) + self.client.force_login(viewer) + self.assertEqual(self.client.get(self.execute_url).status_code, 403) + self.assertEqual(self.client.get(self.confirm_url).status_code, 403) + with self.subTest("the operator group can reach the wizard"): + operator = self._create_operator(organizations=[org]) + self.client.force_login(operator) + self.assertFalse( + BatchCommandAdmin(BatchCommand, admin.site).has_add_permission(None) + ) + self.assertEqual(self.client.get(self.execute_url).status_code, 200) + with self.subTest("a target is required for non superusers"): + response = self._post_execute() + self.assertContains(response, "Please select at least one of") + with self.subTest("devices of unmanaged organizations are not reachable"): + wizard = self._start_wizard(organization=str(org.pk)) + wizard["organization_id"] = str(org2.pk) + session = self.client.session + session[BatchCommandAdmin.session_key] = wizard + session.save() + self.client.get(self.confirm_url) + response = self._post_confirm(wizard["token"]) + self.assertIn( + "No devices match the specified criteria.", self._messages(response) + ) + self.assertFalse(BatchCommand.objects.exists()) + with self.subTest("superusers may target every device"): + self._login() + self._start_wizard() + response = self.client.get(self.confirm_url) + self.assertEqual(response.status_code, 200) + self.assertEqual(response.context["device_count"], 2) + self.assertIn(device2, response.context["cl"].queryset) + + def test_wizard_incompatible_scopes(self): + org = self._get_org() + org2 = self._create_org(name="org2", slug="org2") + group2 = DeviceGroup.objects.create(name="group2", organization=org2) + location2 = Location.objects.create( + name="location2", type="indoor", organization=org2 + ) + group = DeviceGroup.objects.create(name="group1", organization=org) + location = Location.objects.create( + name="location1", type="indoor", organization=org + ) + device = self._create_device(organization=org, group=group) + operator = self._create_operator(organizations=[org, org2]) + self.client.force_login(operator) + with self.subTest("group of another organization"): + response = self._post_execute( + organization=str(org.pk), group=str(group2.pk) + ) + self.assertIn("group", response.context["form"].errors) + with self.subTest("location of another organization"): + response = self._post_execute( + organization=str(org.pk), location=str(location2.pk) + ) + self.assertIn("location", response.context["form"].errors) + with self.subTest("the organization is derived from the group"): + wizard = self._start_wizard(group=str(group.pk)) + self.assertEqual(wizard["group_id"], str(group.pk)) + response = self.client.get(self.confirm_url) + self.assertEqual(response.context["device_count"], 1) + with self.subTest("scopes which share no devices"): + wizard = self._start_wizard( + organization=str(org.pk), + group=str(group.pk), + location=str(location.pk), + ) + response = self.client.get(self.confirm_url) + self.assertEqual(response.context["device_count"], 0) + response = self._post_confirm(wizard["token"]) + self.assertIn( + "No devices match the specified criteria.", self._messages(response) + ) + self.assertFalse(BatchCommand.objects.exists()) + self.assertIn(device, Device.objects.all()) + + def test_wizard_form_fields(self): + org = self._get_org() + org2 = self._create_org(name="org2", slug="org2") + group = DeviceGroup.objects.create(name="group1", organization=org) + group2 = DeviceGroup.objects.create(name="group2", organization=org2) + location = Location.objects.create( + name="location1", type="indoor", organization=org + ) + location2 = Location.objects.create( + name="location2", type="indoor", organization=org2 + ) + operator = self._create_operator(organizations=[org]) + self.client.force_login(operator) + with self.subTest("choices are limited to the managed organizations"): + form = self.client.get(self.execute_url).context["form"] + self.assertEqual(list(form.fields["organization"].queryset), [org]) + self.assertEqual(list(form.fields["group"].queryset), [group]) + self.assertEqual(list(form.fields["location"].queryset), [location]) + with self.subTest("types are limited to the enabled commands"): + with patch.dict(ORGANIZATION_ENABLED_COMMANDS, {str(org.pk): ("reboot",)}): + form = self.client.get(self.execute_url).context["form"] + choices = form.fields["type"].choices + self.assertEqual([value for value, _ in choices], ["", "reboot"]) + with self.subTest("superusers are not restricted"): + self._login() + form = self.client.get(self.execute_url).context["form"] + self.assertIn(group2, form.fields["group"].queryset) + self.assertIn(location2, form.fields["location"].queryset) + self.assertIn(org2, form.fields["organization"].queryset) + + def test_wizard_schema_view(self): + org = self._get_org() + url = reverse("admin:connection_batchcommand_schema") + with self.subTest("superusers get every enabled type"): + self._login() + schemas = self.client.get(url).json() + form = self.client.get(self.execute_url).context["form"] + choices = {value for value, _ in form.fields["type"].choices if value} + self.assertEqual(set(schemas), choices) + with self.subTest("managers get the union of their organizations"): + operator = self._create_operator(organizations=[org]) + self.client.force_login(operator) + with patch.dict( + ORGANIZATION_COMMAND_SCHEMA, + {str(org.pk): {"reboot": COMMANDS["reboot"]["schema"]}}, + ): + self.assertEqual(list(self.client.get(url).json()), ["reboot"]) + with self.subTest("the add permission is required"): + viewer = self._create_operator( + organizations=[org], username="viewer", email="viewer@test.com" + ) + viewer.groups.clear() + viewer.user_permissions.set( + Permission.objects.filter(codename="view_batchcommand") + ) + self.client.force_login(viewer) + self.assertEqual(self.client.get(url).status_code, 403) + + def test_wizard_organization_guard_survives_a_wider_queryset(self): + org = self._get_org() + org2 = self._create_org(name="org2", slug="org2") + group2 = DeviceGroup.objects.create(name="group2", organization=org2) + operator = self._create_operator(organizations=[org]) + self.client.force_login(operator) + request = self.client.get(self.execute_url).wsgi_request + form = BatchCommandExecutionForm( + data={ + "type": "custom", + "input": '{"command": "echo test"}', + "label": "test-label", + "notes": "", + "organization": "", + "group": str(group2.pk), + "location": "", + }, + request=request, + ) + form.fields["group"].queryset = DeviceGroup.objects.all() + self.assertFalse(form.is_valid()) + self.assertEqual(form.errors["group"], ["Select a valid choice."]) + + def test_wizard_review_step_input(self): + model_admin = BatchCommandAdmin(BatchCommand, admin.site) + cases = ( + (None, ""), + ("not-a-mapping", ""), + ({"command": "uptime"}, "uptime"), + ({"config": "network"}, "config: network"), + ( + {"service": "firewall", "action": "restart"}, + "service: firewall, action: restart", + ), + ({"password": "tester123", "confirm_password": "tester123"}, ""), + ) + for command_input, expected in cases: + with self.subTest(str(command_input)): + self.assertEqual(model_admin._describe_input(command_input), expected) + + def test_wizard_recovers_from_unresolvable_targets(self): + org = self._get_org() + self._create_device(organization=org) + self._login() + with self.subTest("a confirm page without a wizard restarts"): + response = self.client.get(self.confirm_url) + self.assertRedirects(response, self.execute_url) + with self.subTest("targets which cannot be resolved list no devices"): + self._start_wizard(organization=str(org.pk)) + with patch.object( + BatchCommand, "dry_run", side_effect=ValidationError("broken") + ): + response = self.client.get(self.confirm_url) + self.assertEqual(response.context["device_count"], 0) + with self.subTest("a batch which disappears restarts"): + wizard = self._start_wizard(organization=str(org.pk)) + self.client.get(self.confirm_url) + with patch.object(BatchCommand, "execute", side_effect=Device.DoesNotExist): + response = self._post_confirm(wizard["token"]) + self.assertRedirects(response, self.execute_url) + self.assertFalse(BatchCommand.objects.exists()) + + def test_wizard_stale_and_parallel_sessions(self): + org = self._get_org() + self._create_device(organization=org) + self._login() + restart_message = "Please fill in the mass command details to continue." + with self.subTest("no wizard in the session"): + response = self._post_confirm("any-token") + self.assertRedirects(response, self.execute_url) + self.assertIn(restart_message, self._messages(response)) + with self.subTest("a token from another tab"): + wizard = self._start_wizard(organization=str(org.pk)) + self.client.get(self.confirm_url) + response = self._post_confirm("stale-token") + self.assertRedirects(response, self.execute_url) + self.assertIn(restart_message, self._messages(response)) + self.assertFalse(BatchCommand.objects.exists()) + with self.subTest("double submit"): + wizard = self._start_wizard(organization=str(org.pk)) + self.client.get(self.confirm_url) + self._post_confirm(wizard["token"]) + batch = BatchCommand.objects.get() + response = self._post_confirm(wizard["token"]) + self.assertRedirects(response, self.execute_url) + self.assertIn(restart_message, self._messages(response)) + self.assertEqual(list(BatchCommand.objects.all()), [batch]) + BatchCommand.objects.all().delete() + with self.subTest("the targeted devices changed"): + wizard = self._start_wizard(organization=str(org.pk)) + self.client.get(self.confirm_url) + self._create_device( + name="late-device", + mac_address="00:11:22:33:44:99", + organization=org, + ) + response = self._post_confirm(wizard["token"]) + self.assertRedirects(response, self.confirm_url) + self.assertIn( + "The targeted devices changed, please review them again.", + self._messages(response), + ) + self.assertFalse(BatchCommand.objects.exists()) + self.assertIn(BatchCommandAdmin.session_key, self.client.session) + + def test_wizard_device_selection(self): + org = self._get_org() + devices = [self._create_device(organization=org)] + devices += [ + self._create_device( + name=f"device{index}", + mac_address=f"00:11:22:33:44:0{index}", + organization=org, + ) + for index in range(1, 4) + ] + self._login() + with self.subTest("excluded devices are left out"): + wizard = self._start_wizard(organization=str(org.pk)) + self.client.get(self.confirm_url) + self._post_confirm(wizard["token"], excluded=str(devices[-1].pk)) + batch = BatchCommand.objects.get() + self.assertEqual(set(batch.devices.all()), set(devices[:-1])) + BatchCommand.objects.all().delete() + with self.subTest("malformed entries are ignored"): + wizard = self._start_wizard(organization=str(org.pk)) + self.client.get(self.confirm_url) + self._post_confirm( + wizard["token"], + excluded=f",not-a-uuid,,{uuid4()},{devices[0].pk},", + ) + batch = BatchCommand.objects.get() + self.assertEqual(set(batch.devices.all()), set(devices[1:])) + BatchCommand.objects.all().delete() + with self.subTest("excluding every device"): + wizard = self._start_wizard(organization=str(org.pk)) + self.client.get(self.confirm_url) + response = self._post_confirm( + wizard["token"], + excluded=",".join(str(device.pk) for device in devices), + ) + self.assertRedirects(response, self.confirm_url) + self.assertIn( + "No devices match the specified criteria.", self._messages(response) + ) + self.assertFalse(BatchCommand.objects.exists()) + self.assertIn(BatchCommandAdmin.session_key, self.client.session) + + def test_changelist_multitenancy(self): + org = self._get_org() + org2 = self._create_org(name="org2", slug="org2") + batch = self._create_batch_command(organization=org) + batch2 = self._create_batch_command(organization=org2, label="other-label") + operator = self._create_operator(organizations=[org]) + self.client.force_login(operator) + with self.subTest("only managed organizations are listed"): + response = self.client.get(self.changelist_url) + queryset = response.context["cl"].queryset + self.assertIn(batch, queryset) + self.assertNotIn(batch2, queryset) + with self.subTest("an unmanaged batch cannot be opened"): + url = reverse("admin:connection_batchcommand_change", args=[batch2.pk]) + self.assertEqual(self.client.get(url).status_code, 302) + with self.subTest("superusers see every batch"): + self._login() + response = self.client.get(self.changelist_url) + queryset = response.context["cl"].queryset + self.assertIn(batch, queryset) + self.assertIn(batch2, queryset) + + def test_changelist_filters_and_search(self): + org = self._get_org() + org2 = self._create_org(name="org2", slug="org2") + group = DeviceGroup.objects.create(name="group1", organization=org) + device = self._create_device(organization=org, group=group) + batch = self._create_batch_command( + organization=org, group=group, devices=[device] + ) + batch2 = self._create_batch_command( + organization=org2, + label="other-label", + type="reboot", + input=None, + status="success", + ) + self._login() + cases = ( + ("status", {"status": "success"}, batch2), + ("type", {"type": "custom"}, batch), + ("group", {"group_id": str(group.pk)}, batch), + ("search by label", {"q": "other-label"}, batch2), + ("search by device name", {"q": device.name}, batch), + ("search by group name", {"q": "group1"}, batch), + ) + for name, params, expected in cases: + with self.subTest(name): + response = self.client.get(self.changelist_url, params) + queryset = response.context["cl"].queryset + self.assertEqual(list(queryset), [expected]) + with self.subTest("combined filters exclude everything"): + response = self.client.get( + self.changelist_url, {"status": "success", "type": "custom"} + ) + self.assertEqual(response.context["cl"].queryset.count(), 0) + with self.subTest("affected devices is orderable and not duplicated"): + response = self.client.get(self.changelist_url, {"o": "5"}) + queryset = response.context["cl"].queryset + self.assertEqual(set(queryset), {batch, batch2}) + self.assertEqual([item._affected_devices for item in queryset], [0, 0]) + + def test_changelist_is_read_only(self): + batch = self._create_batch_command(organization=self._get_org()) + self._login() + model_admin = BatchCommandAdmin(BatchCommand, admin.site) + response = self.client.get(self.changelist_url) + with self.subTest("no bulk actions"): + self.assertNotIn( + "delete_selected", model_admin.get_actions(response.wsgi_request) + ) + with self.subTest("adding and deleting are disabled"): + self.assertFalse(model_admin.has_add_permission(response.wsgi_request)) + self.assertFalse( + model_admin.has_delete_permission(response.wsgi_request, batch) + ) + with self.subTest("no save buttons on the change page"): + url = reverse("admin:connection_batchcommand_change", args=[batch.pk]) + response = self.client.get(url) + self.assertFalse(response.context["show_save"]) + self.assertFalse(response.context["show_save_and_continue"]) + + def _create_commands(self, batch, devices, **kwargs): + commands = [] + for device in devices: + if not hasattr(device, "config"): + self._create_config(device=device) + connection = self._create_device_connection( + device=device, + credentials=self._create_credentials( + name=f"cred-{device.name}", organization=device.organization + ), + ) + with patch.object(Command, "_schedule_command"): + commands.append( + Command.objects.create( + batch_command=batch, + device=device, + connection=connection, + type="custom", + input={"command": "echo test"}, + **kwargs, + ) + ) + return commands + + def test_change_view_command_rows(self): + org = self._get_org() + batch = self._create_batch_command(organization=org) + devices = [self._create_device(organization=org)] + [ + self._create_device( + name=f"device{index}", + mac_address=f"00:11:22:33:44:0{index}", + organization=org, + ) + for index in range(1, 4) + ] + commands = self._create_commands(batch, devices, output="first\nlast") + batch.skipped_devices = { + str(uuid4()): {"name": "skipped-device", "error": "no credentials"} + } + batch.save(update_fields=["skipped_devices"]) + self._login() + url = reverse("admin:connection_batchcommand_change", args=[batch.pk]) + with patch.object(BatchCommandAdmin, "device_commands_per_page", 3): + with self.subTest("first page"): + rows = self.client.get(url).context["commands"] + self.assertEqual( + [row["device_name"] for row in rows], + [command.device.name for command in commands[:3]], + ) + self.assertEqual(rows[0]["output"], "… last") + self.assertEqual(rows[0]["status_display"], "in progress") + self.assertFalse(rows[0]["is_skipped"]) + with self.subTest("the page spanning commands and skipped devices"): + rows = self.client.get(url, {"page": 2}).context["commands"] + self.assertEqual([row["is_skipped"] for row in rows], [False, True]) + self.assertEqual(rows[-1]["device_name"], "skipped-device") + with self.subTest("an unusable page falls back to the first"): + for page in ("abc", 0, 99): + response = self.client.get(url, {"page": page}) + self.assertEqual(response.context["page_obj"].number, 1) + with self.subTest("the newest command is last"): + rows = self.client.get(url, {"page": 2}).context["commands"] + self.assertEqual(rows[0]["device"], commands[-1].device.pk) + + def test_change_view_filters(self): + org = self._get_org() + org2 = self._create_org(name="org2", slug="org2") + group = DeviceGroup.objects.create(name="group1", organization=org) + location = Location.objects.create( + name="location1", type="indoor", organization=org + ) + device = self._create_device(organization=org, group=group) + DeviceLocation.objects.create(content_object=device, location=location) + batch = self._create_batch_command(organization=org, group=group) + other_group = DeviceGroup.objects.create(name="skipped-group", organization=org) + skipped_device = self._create_device( + name="skipped-device", + mac_address="00:11:22:33:44:88", + organization=org, + group=other_group, + ) + self._create_commands(batch, [device], status="success") + batch.skipped_devices = { + str(skipped_device.pk): { + "name": skipped_device.name, + "error": "no credentials", + } + } + batch.save(update_fields=["skipped_devices"]) + self._login() + url = reverse("admin:connection_batchcommand_change", args=[batch.pk]) + with self.subTest("search matches commands and skipped devices"): + rows = self.client.get(url, {"q": device.name}).context["commands"] + self.assertEqual([row["device_name"] for row in rows], [device.name]) + rows = self.client.get(url, {"q": "skipped"}).context["commands"] + self.assertEqual([row["device_name"] for row in rows], ["skipped-device"]) + with self.subTest("status skipped hides the commands"): + rows = self.client.get(url, {"status": "skipped"}).context["commands"] + self.assertEqual([row["is_skipped"] for row in rows], [True]) + with self.subTest("status success hides the skipped devices"): + rows = self.client.get(url, {"status": "success"}).context["commands"] + self.assertEqual([row["is_skipped"] for row in rows], [False]) + with self.subTest("groups of skipped devices are offered as filters"): + response = self.client.get(url) + titles = { + str(spec.title): spec for spec in response.context["filter_specs"] + } + displays = [ + str(choice["display"]) for choice in titles["device group"].choices + ] + self.assertIn(other_group.name, displays) + self.assertIn(group.name, displays) + with self.subTest("filtering by a group keeps only its devices"): + rows = self.client.get(url, {"group_id": str(other_group.pk)}).context[ + "commands" + ] + self.assertEqual([row["device_name"] for row in rows], ["skipped-device"]) + with self.subTest("filtering by location"): + DeviceLocation.objects.create( + content_object=skipped_device, location=location + ) + rows = self.client.get(url, {"location_id": str(location.pk)}).context[ + "commands" + ] + self.assertEqual( + sorted(row["device_name"] for row in rows), + sorted([device.name, "skipped-device"]), + ) + with self.subTest("filtering by organization"): + rows = self.client.get(url, {"organization_id": str(org.pk)}).context[ + "commands" + ] + self.assertEqual( + sorted(row["device_name"] for row in rows), + sorted([device.name, "skipped-device"]), + ) + rows = self.client.get(url, {"organization_id": str(org2.pk)}).context[ + "commands" + ] + self.assertEqual(rows, []) + with self.subTest("the organization filter is for superusers only"): + operator = self._create_operator(organizations=[org]) + self.client.force_login(operator) + response = self.client.get(url) + titles = [str(spec.title) for spec in response.context["filter_specs"]] + self.assertNotIn("organization", titles) + self._login() + with self.subTest("the search term is not an active filter"): + response = self.client.get(url, {"q": "anything"}) + self.assertFalse(response.context["has_active_filters"]) + response = self.client.get(url, {"status": "success"}) + self.assertTrue(response.context["has_active_filters"]) + + def test_change_view_display_fields(self): + org = self._get_org() + org2 = self._create_org(name="org2", slug="org2") + batch = self._create_batch_command(organization=org) + model_admin = BatchCommandAdmin(BatchCommand, admin.site) + with self.subTest("organization display"): + self.assertEqual(model_admin.organization_display(batch), org.name) + shared = self._create_batch_command(organization=None, label="shared") + self.assertEqual(str(model_admin.organization_display(shared)), "All") + with self.subTest("colored status"): + self.assertIn( + f"command-status {batch.status}", model_admin.colored_status(batch) + ) + with self.subTest("formatted input"): + self.assertEqual(model_admin.formatted_input(batch), "echo test") + empty_batch = self._create_batch_command( + organization=org, label="empty", type="reboot", input=None + ) + self.assertEqual(model_admin.formatted_input(empty_batch), "-") + registered_batch = self._create_batch_command( + organization=org, label="registered" + ) + registered_batch.input = {"config": "network"} + self.assertEqual( + model_admin.formatted_input(registered_batch), "config: network" + ) + password_batch = self._create_batch_command( + organization=org, + label="password", + type="change_password", + input={"password": "tester123", "confirm_password": "tester123"}, + ) + self.assertEqual(model_admin.formatted_input(password_batch), "********") + with self.subTest("affected devices falls back to the model"): + self.assertEqual(model_admin.affected_devices(batch), 0) + batch._affected_devices = 7 + self.assertEqual(model_admin.affected_devices(batch), 7) + with self.subTest("skipped devices rendering"): + self.assertEqual(model_admin.display_skipped_devices(batch), "-") + batch.skipped_devices = { + str(uuid4()): {"name": f"device{index}", "error": "failed"} + for index in range(12) + } + rendered = model_admin.display_skipped_devices(batch) + self.assertIn("12", rendered) + self.assertIn("device0: failed", rendered) + self.assertIn("…", rendered) + with self.subTest("commands of unmanaged organizations are hidden"): + device2 = self._create_device( + name="device-org2", + mac_address="00:11:22:33:44:77", + organization=org2, + ) + self._create_commands(batch, [device2]) + operator = self._create_operator(organizations=[org]) + request = self.client.get(self.changelist_url).wsgi_request + request.user = operator + self.assertEqual(model_admin._get_commands(request, batch).count(), 0) + + def test_changelist_filter_classes(self): + org = self._get_org() + org2 = self._create_org(name="org2", slug="org2") + group = DeviceGroup.objects.create(name="group1", organization=org) + location = Location.objects.create( + name="location1", type="indoor", organization=org + ) + group2 = DeviceGroup.objects.create(name="group2", organization=org2) + location2 = Location.objects.create( + name="location2", type="indoor", organization=org2 + ) + batch = self._create_batch_command(organization=org, group=group) + batch2 = self._create_batch_command( + organization=org2, label="other-label", type="reboot", input=None + ) + model_admin = BatchCommandAdmin(BatchCommand, admin.site) + operator = self._create_operator(organizations=[org]) + self.client.force_login(operator) + request = self.client.get(self.changelist_url).wsgi_request + with self.subTest("type lookups are limited to the managed organizations"): + type_filter = TypeFilter(request, {}, BatchCommand, model_admin) + self.assertEqual( + type_filter.lookups(request, model_admin), + [("custom", "Custom commands")], + ) + with self.subTest("type lookups list every type for superusers"): + self._login() + admin_request = self.client.get(self.changelist_url).wsgi_request + type_filter = TypeFilter(admin_request, {}, BatchCommand, model_admin) + lookups = dict(type_filter.lookups(admin_request, model_admin)) + self.assertEqual(set(lookups), {"custom", "reboot"}) + with self.subTest("type queryset"): + response = self.client.get(self.changelist_url, {"type": "reboot"}) + self.assertEqual(list(response.context["cl"].queryset), [batch2]) + response = self.client.get(self.changelist_url) + self.assertEqual(set(response.context["cl"].queryset), {batch, batch2}) + with self.subTest("the parameter names match the change page filters"): + self.assertEqual(GroupFilter.parameter_name, "group_id") + self.assertEqual(LocationFilter.parameter_name, "location_id") + with self.subTest("the filters are on the changelist"): + self.client.force_login(operator) + response = self.client.get(self.changelist_url) + specs = {type(spec) for spec in response.context["cl"].filter_specs} + self.assertIn(GroupFilter, specs) + self.assertIn(LocationFilter, specs) + with self.subTest("related choices are limited to the managed organizations"): + self._create_administrator(organizations=[org]) + self._test_multitenant_admin( + url=self._get_autocomplete_view_path( + self.app_label, "batchcommand", "group" + ), + visible=[group.name], + hidden=[group2.name], + administrator=True, + ) + self._test_multitenant_admin( + url=self._get_autocomplete_view_path( + self.app_label, "batchcommand", "location" + ), + visible=[location.name], + hidden=[location2.name], + administrator=True, + ) diff --git a/openwisp_controller/connection/tests/test_api.py b/openwisp_controller/connection/tests/test_api.py index ef46278f2..01351230e 100644 --- a/openwisp_controller/connection/tests/test_api.py +++ b/openwisp_controller/connection/tests/test_api.py @@ -423,7 +423,7 @@ def test_create_command_without_connection(self): ) self.assertEqual(response.status_code, 400) self.assertIn( - "Device has no credentials assigned.", + "Device has no credentials assigned", response.data["device"][0], ) @@ -1065,7 +1065,7 @@ def test_batch_command_endpoints_no_of_queries(self): "devices": [str(d.pk) for d in devices], } url = reverse("connection_api:batch_command_execute") - with self.assertNumQueries(16): + with self.assertNumQueries(15): response = self.client.post( url, data=json.dumps(payload), @@ -1102,7 +1102,7 @@ def test_batch_command_endpoints_no_of_queries(self): "group": str(group.pk), } url = reverse("connection_api:batch_command_execute") - with self.assertNumQueries(15): + with self.assertNumQueries(14): response = self.client.post( url, data=json.dumps(payload), @@ -1128,7 +1128,7 @@ def test_batch_command_endpoints_no_of_queries(self): "label": "test-label", } url = reverse("connection_api:batch_command_execute") - with self.assertNumQueries(13): + with self.assertNumQueries(12): response = self.client.post( url, data=json.dumps(payload), @@ -2258,7 +2258,7 @@ def test_batch_command_execute_skipped_devices(self): self.assertIn(str(device_b.pk), batch.skipped_devices) self.assertIn( '"custom" command is not available for this organization', - batch.skipped_devices[str(device_b.pk)][0], + batch.skipped_devices[str(device_b.pk)]["error"], ) command_qs = Command.objects.filter(batch_command=batch) self.assertTrue(command_qs.filter(device=device_a).exists()) @@ -2318,7 +2318,7 @@ def test_batch_command_execute_skipped_devices(self): self.assertIn(str(device.pk), batch.skipped_devices) self.assertIn( "Device has no credentials assigned", - batch.skipped_devices[str(device.pk)][0], + batch.skipped_devices[str(device.pk)]["error"], ) detail_url = reverse( "connection_api:batch_command_detail", @@ -2360,7 +2360,7 @@ def test_batch_command_execute_skipped_devices(self): self.assertIn(str(device.pk), batch.skipped_devices) self.assertIn( "Device is deactivated", - batch.skipped_devices[str(device.pk)][0], + batch.skipped_devices[str(device.pk)]["error"], ) detail_url = reverse( "connection_api:batch_command_detail", diff --git a/openwisp_controller/connection/tests/test_models.py b/openwisp_controller/connection/tests/test_models.py index fc45ff2d6..04f365df8 100644 --- a/openwisp_controller/connection/tests/test_models.py +++ b/openwisp_controller/connection/tests/test_models.py @@ -6,6 +6,7 @@ import paramiko from django.contrib.auth.models import ContentType from django.core.exceptions import ValidationError +from django.db import transaction from django.test import TestCase, TransactionTestCase, tag from django.utils import timezone from django.utils.module_loading import import_string @@ -13,6 +14,7 @@ from openwisp_utils.tests import capture_any_output, catch_signal +from .. import apps from .. import settings as app_settings from ..commands import ( COMMANDS, @@ -525,6 +527,31 @@ def test_command_is_custom(self): command = Command(type="custom", input={"command": "echo test"}) self.assertTrue(command.is_custom) + def test_command_output_preview(self): + with self.subTest("no output"): + self.assertEqual(Command(type="reboot").output_preview, "") + self.assertEqual(Command(type="reboot", output="").output_preview, "") + + with self.subTest("single line"): + command = Command(type="reboot", output="all good") + self.assertEqual(command.output_preview, "all good") + + with self.subTest("single line with trailing newline"): + command = Command(type="reboot", output="all good\n") + self.assertEqual(command.output_preview, "all good") + + with self.subTest("single long line"): + command = Command(type="reboot", output="x" * 120) + self.assertEqual(command.output_preview, f"… {'x' * 100}") + + with self.subTest("multiple lines"): + command = Command(type="reboot", output="first\nsecond\nlast") + self.assertEqual(command.output_preview, "… last") + + with self.subTest("multiple lines with a long last line"): + command = Command(type="reboot", output=f"first\n{'y' * 120}") + self.assertEqual(command.output_preview, f"… {'y' * 100}") + def test_command_validation(self): dc = self._create_device_connection() command = Command( @@ -614,7 +641,7 @@ def test_command_validation(self): self.assertIn("device", exception.message_dict) self.assertEqual( exception.message_dict["device"], - ["Device has no credentials assigned."], + ["Device has no credentials assigned"], ) def test_command_validation_deactivated_device(self): @@ -646,7 +673,7 @@ def test_command_validation_deactivated_device(self): command.clean() self.assertIn("device", ctx.exception.message_dict) self.assertEqual( - ctx.exception.message_dict["device"], ["Device is deactivated."] + ctx.exception.message_dict["device"], ["Device is deactivated"] ) @tag("skip_prod") @@ -1028,6 +1055,77 @@ def test_batch_command_total_devices_successful_failed(self): batch.batch_commands.filter(status="failed", device=device1).exists() ) + def test_batch_command_skipped_devices(self): + org = self._get_org() + batch = self._create_batch_command(organization=org) + skipped = { + str(uuid4()): {"name": f"device{index}", "error": f"error {index}"} + for index in range(3) + } + pks = list(skipped) + batch.skipped_devices = skipped + batch.save(update_fields=["skipped_devices"]) + + with self.subTest("skipped row"): + row = BatchCommand.build_skipped_row(pks[0], skipped[pks[0]]) + self.assertEqual( + row, + { + "device": pks[0], + "device_name": "device0", + "status": "skipped", + "status_display": "skipped", + "output": "error 0", + "modified": None, + "is_skipped": True, + }, + ) + + with self.subTest("all rows"): + rows = batch.get_skipped_rows() + self.assertEqual([row["device"] for row in rows], pks) + + with self.subTest("sliced rows"): + self.assertEqual( + [row["device"] for row in batch.get_skipped_rows(end=2)], pks[:2] + ) + self.assertEqual( + [row["device"] for row in batch.get_skipped_rows(start=-1)], pks[-1:] + ) + + with self.subTest("preview within the limit"): + self.assertEqual( + [row["device"] for row in batch.get_skipped_preview()], pks + ) + + with self.subTest("preview above the limit"): + skipped = { + str(uuid4()): {"name": f"device{index}", "error": f"error {index}"} + for index in range(11) + } + pks = list(skipped) + batch.skipped_devices = skipped + batch.save(update_fields=["skipped_devices"]) + self.assertEqual( + [row["device"] for row in batch.get_skipped_preview()], + pks[:2] + pks[-1:], + ) + + with self.subTest("counts include skipped devices"): + device = self._create_device(organization=org) + self._create_config(device=device) + dc = self._create_device_connection(device=device) + Command.objects.create( + batch_command=batch, + device=device, + connection=dc, + type=batch.type, + input={"command": "echo test"}, + status="success", + ) + self.assertEqual(batch.affected_devices, 1) + self.assertEqual(batch.total_devices, 12) + def test_batch_command_clean_validation(self): org = self._get_org() org2 = self._create_org(name="org2", slug="org2") @@ -1141,7 +1239,7 @@ def test_batch_command_create_commands_deactivated_device(self): self.assertIn(str(device.pk), batch.skipped_devices) self.assertIn( "Device is deactivated", - batch.skipped_devices[str(device.pk)][0], + batch.skipped_devices[str(device.pk)]["error"], ) def test_batch_command_create_commands_no_credentials(self): @@ -1157,7 +1255,7 @@ def test_batch_command_create_commands_no_credentials(self): self.assertIn(str(device.pk), batch.skipped_devices) self.assertIn( "Device has no credentials assigned", - batch.skipped_devices[str(device.pk)][0], + batch.skipped_devices[str(device.pk)]["error"], ) def test_batch_command_create_commands_skip_scenarios(self): @@ -1192,7 +1290,7 @@ def test_batch_command_create_commands_skip_scenarios(self): self.assertIn(str(device_b.pk), batch.skipped_devices) self.assertIn( '"custom" command is not available for this organization', - batch.skipped_devices[str(device_b.pk)][0], + batch.skipped_devices[str(device_b.pk)]["error"], ) db_batch = BatchCommand.objects.get(pk=batch.pk) self.assertEqual(batch.skipped_devices, db_batch.skipped_devices) @@ -1236,19 +1334,36 @@ def test_batch_command_create_commands_skip_scenarios(self): self.assertEqual(command_qs.count(), 1) self.assertTrue(command_qs.filter(device=device_ok).exists()) self.assertIn(str(device_no_creds.pk), batch.skipped_devices) + self.assertEqual( + batch.skipped_devices[str(device_no_creds.pk)]["name"], + device_no_creds.name, + ) self.assertIn( "Device has no credentials assigned", - batch.skipped_devices[str(device_no_creds.pk)][0], + batch.skipped_devices[str(device_no_creds.pk)]["error"], ) self.assertIn(str(device_deactivated.pk), batch.skipped_devices) self.assertIn( "Device is deactivated", - batch.skipped_devices[str(device_deactivated.pk)][0], + batch.skipped_devices[str(device_deactivated.pk)]["error"], ) self.assertNotIn(str(device_ok.pk), batch.skipped_devices) db_batch = BatchCommand.objects.get(pk=batch.pk) self.assertEqual(batch.skipped_devices, db_batch.skipped_devices) + def test_batch_command_create_commands_without_explicit_devices(self): + org = self._get_org() + device = self._create_device(organization=org) + self._create_config(device=device) + self._create_device_connection(device=device) + batch = self._create_batch_command(organization=org) + batch.create_commands() + batch.refresh_from_db() + self.assertEqual( + [command.device for command in batch.batch_commands.all()], [device] + ) + self.assertEqual(list(batch.devices.all()), [device]) + def test_batch_command_resolve_devices(self): org = self._get_org() device1 = self._create_device( @@ -1720,7 +1835,9 @@ def test_batch_command_calculate_and_update_status(self): with self.subTest("all success with skipped shows failed"): batch3 = self._create_batch_command(organization=org) - batch3.skipped_devices = {str(device.pk): ["no credentials"]} + batch3.skipped_devices = { + str(device.pk): {"name": device.name, "error": "no credentials"} + } batch3.save(update_fields=["skipped_devices"]) Command.objects.create( batch_command=batch3, @@ -2070,3 +2187,32 @@ def test_chunk_size(self): credential = self._create_credentials( name="Mocked Credential", auto_add=True, organization=org ) + + def test_batch_command_broadcast_deferred_to_commit(self): + org = self._get_org() + device = self._create_device(organization=org) + self._create_config(device=device) + dc = self._create_device_connection(device=device) + batch = self._create_batch_command(organization=org) + command_opts = dict( + batch_command=batch, + device=device, + connection=dc, + type=batch.type, + input={"command": "echo test"}, + ) + with mock.patch.object(Command, "_schedule_command"): + with self.subTest("nothing is sent before the transaction commits"): + with mock.patch.object(apps.layers, "get_channel_layer") as layer: + with transaction.atomic(): + Command.objects.create(**command_opts) + layer.assert_not_called() + layer.assert_called() + with self.subTest("a rolled back command is never broadcast"): + with mock.patch.object(apps.layers, "get_channel_layer") as layer: + with self.assertRaises(ValueError): + with transaction.atomic(): + Command.objects.create(**command_opts) + raise ValueError() + layer.assert_not_called() + self.assertEqual(batch.batch_commands.count(), 1) diff --git a/openwisp_controller/connection/tests/test_selenium.py b/openwisp_controller/connection/tests/test_selenium.py index 73f9c672f..d0f5094ce 100644 --- a/openwisp_controller/connection/tests/test_selenium.py +++ b/openwisp_controller/connection/tests/test_selenium.py @@ -1,16 +1,47 @@ from time import sleep +from urllib.parse import quote, urlparse +from uuid import UUID +from channels.testing import ChannelsLiveServerTestCase +from django.apps import apps as django_apps +from django.contrib.auth.models import Permission from django.contrib.staticfiles.testing import StaticLiveServerTestCase from django.test import tag from django.urls import reverse from selenium.webdriver.common.by import By +from selenium.webdriver.support import expected_conditions as EC +from selenium.webdriver.support.ui import WebDriverWait from swapper import load_model +from openwisp_users.migrations import ( + allow_operator_view_organization, + create_default_groups, +) from openwisp_utils.tests import SeleniumTestMixin -from .utils import CreateConnectionsMixin +from ...config import migrations as config_migrations +from ...config.tests.utils import CreateDeviceGroupMixin +from ...geo import migrations as geo_migrations +from ...geo.tests.utils import TestGeoMixin +from .. import migrations as connection_migrations +from .. import settings as app_settings +from ..commands import ( + COMMANDS, + ORGANIZATION_COMMAND_SCHEMA, + ORGANIZATION_ENABLED_COMMANDS, + register_command, + unregister_command, +) +from .utils import CreateConnectionsMixin, SshServer, _uci_show_command_callable +BatchCommand = load_model("connection", "BatchCommand") Command = load_model("connection", "Command") +Device = load_model("config", "Device") +Location = load_model("geo", "Location") +DeviceLocation = load_model("geo", "DeviceLocation") +Group = load_model("openwisp_users", "Group") +SCOPED_ORGANIZATION_ID = "11111111-1111-1111-1111-111111111111" +DEFAULT_ORGANIZATION_ID = "22222222-2222-2222-2222-222222222222" @tag("selenium_tests") @@ -68,3 +99,1047 @@ def test_command_widget_on_device(self): ), ) self.assertEqual(Command.objects.count(), 1) + + +@tag("selenium_tests") +class TestBatchCommandAdmin( + TestGeoMixin, + CreateDeviceGroupMixin, + CreateConnectionsMixin, + SeleniumTestMixin, + ChannelsLiveServerTestCase, +): + app_label = "connection" + config_app_label = "config" + object_model = Device + location_model = Location + object_location_model = DeviceLocation + + @classmethod + def setUpClass(cls): + if "uci_show" not in COMMANDS: + register_command( + "uci_show", + { + "label": "UCI show", + "schema": { + "title": "UCI show", + "type": "object", + "required": ["config"], + "properties": { + "config": { + "type": "string", + "title": "Config", + "minLength": 1, + "pattern": ".", + } + }, + "message": "Config cannot be empty.", + "additionalProperties": False, + }, + "callable": _uci_show_command_callable, + }, + ) + cls.addClassCleanup(unregister_command, "uci_show") + ORGANIZATION_ENABLED_COMMANDS[SCOPED_ORGANIZATION_ID] = ( + "custom", + "reboot", + "change_password", + "uci_show", + ) + ORGANIZATION_ENABLED_COMMANDS[DEFAULT_ORGANIZATION_ID] = ( + "custom", + "reboot", + "change_password", + ) + for organization_id in (SCOPED_ORGANIZATION_ID, DEFAULT_ORGANIZATION_ID): + ORGANIZATION_COMMAND_SCHEMA[organization_id] = { + command: COMMANDS[command]["schema"] + for command in ORGANIZATION_ENABLED_COMMANDS[organization_id] + } + cls.addClassCleanup( + ORGANIZATION_ENABLED_COMMANDS.pop, organization_id, None + ) + cls.addClassCleanup(ORGANIZATION_COMMAND_SCHEMA.pop, organization_id, None) + super().setUpClass() + cls.mock_ssh_server = SshServer( + {"root": cls._TEST_RSA_PRIVATE_KEY_PATH} + ).__enter__() + cls.addClassCleanup(cls.mock_ssh_server.__exit__) + cls.ssh_server.port = cls.mock_ssh_server.port + + def setUp(self): + super().setUp() + self._restore_default_groups() + self.execute_url = reverse(f"admin:{self.app_label}_batchcommand_execute") + self.confirm_url = reverse(f"admin:{self.app_label}_batchcommand_confirm") + self.changelist_url = reverse(f"admin:{self.app_label}_batchcommand_changelist") + + # this is a TransactionTestCase: the flush performed after every test + # restores what post_migrate creates (permissions, content types) but + # not the rows written by data migrations, so the default groups are + # lost and _create_operator() would return a user without permissions + def _restore_default_groups(self): + if Group.objects.filter(name="Operator").exists(): + return + models_modules = { + app_config.label: app_config.models_module + for app_config in django_apps.get_app_configs() + } + try: + for migration in ( + create_default_groups, + allow_operator_view_organization, + config_migrations.assign_permissions_to_groups, + config_migrations.assign_devicegroup_permissions_to_groups, + geo_migrations.assign_permissions_to_groups, + connection_migrations.assign_permissions_to_groups, + connection_migrations.assign_command_permissions_to_groups, + connection_migrations.assign_batchcommand_permissions_to_groups, + ): + migration(django_apps, None) + finally: + for app_config in django_apps.get_app_configs(): + app_config.models_module = models_modules[app_config.label] + + def _create_devices(self, organization, count, credentials=None): + if credentials is None: + credentials = self._create_credentials( + organization=organization, + params={"username": "root", "password": "password", "port": 5555}, + ) + devices = [] + for index in range(count): + device = self._create_device( + name=f"device-{index:03d}", + organization=organization, + mac_address="00:11:22:33:{:02x}:{:02x}".format( + index // 256, index % 256 + ), + ) + self._create_device_connection( + device=device, + credentials=credentials, + update_strategy=app_settings.UPDATE_STRATEGIES[0][0], + ) + devices.append(device) + return devices + + def _select2(self, field_id, text): + self.find_element( + by=By.CSS_SELECTOR, value=f"#select2-{field_id}-container" + ).click() + self.find_element( + by=By.CSS_SELECTOR, + value=".select2-container--open .select2-search__field", + ).send_keys(text) + self.find_element( + by=By.CSS_SELECTOR, + value=".select2-container--open .select2-results__option--highlighted", + timeout=5, + ).click() + self.wait_for_invisibility(By.CSS_SELECTOR, ".select2-container--open") + + def _fill_wizard( + self, + type, + label, + organization=None, + group=None, + location=None, + command_input=None, + open_page=True, + ): + if open_page: + self.open(self.execute_url) + self._wait_for_url(self.execute_url) + self.hide_loading_overlay() + self.web_driver.execute_script( + "django.jQuery('#id_group, #id_location').val('').trigger('change');" + ) + self._select2("id_type", type) + self.assertEqual( + self.find_element( + by=By.CSS_SELECTOR, value="#select2-id_type-container" + ).get_attribute("title"), + type, + ) + for field_name, value in (command_input or {}).items(): + field = self.find_element( + by=By.CSS_SELECTOR, + value=f"#id_input_jsoneditor [name='root[{field_name}]']", + timeout=5, + ) + field.clear() + field.send_keys(value) + self.assertEqual(field.get_attribute("value"), value) + label_field = self.find_element(by=By.ID, value="id_label") + label_field.clear() + label_field.send_keys(label) + self.assertEqual(label_field.get_attribute("value"), label) + for field_id, target in ( + ("id_organization", organization), + ("id_group", group), + ("id_location", location), + ): + if target: + self._select2(field_id, target.name) + self.assertEqual( + self.find_element( + by=By.CSS_SELECTOR, value=f"#select2-{field_id}-container" + ).get_attribute("title"), + target.name, + ) + + def _open_menu_item(self, group_label, item_label): + self.find_element( + by=By.CSS_SELECTOR, value=f'.mg-head[aria-label="{group_label}"]' + ).click() + self.find_element( + by=By.CSS_SELECTOR, + value=f'.menu-group.active a.mg-link[aria-label="{item_label}"]', + timeout=5, + ).click() + + def _search(self, query): + table = self.find_element(by=By.CSS_SELECTOR, value="#result_list") + search_field = self.find_element(by=By.ID, value="searchbar") + search_field.clear() + search_field.send_keys(query) + search_field.submit() + WebDriverWait(self.web_driver, 5).until( + lambda driver: f"q={quote(query)}" in driver.current_url + ) + WebDriverWait(self.web_driver, 5).until(EC.staleness_of(table)) + self.hide_loading_overlay() + + def _filter_by(self, title, option): + current_url = self.web_driver.current_url + tables = self.web_driver.find_elements(By.CSS_SELECTOR, "#result_list") + slug = title.replace(" ", "-") + filter_element = self.find_element( + by=By.CSS_SELECTOR, value=f".ow-filter.{slug}", wait_for="presence" + ) + self.web_driver.execute_script( + "arguments[0].click();", + filter_element.find_element(By.CSS_SELECTOR, ".filter-title"), + ) + WebDriverWait(self.web_driver, 5).until( + lambda driver: "ow-active" in filter_element.get_attribute("class") + ) + self.web_driver.execute_script( + "arguments[0].click();", + self.find_element( + by=By.XPATH, + value=( + "//div[contains(@class, 'ow-filter')]" + f"[contains(@class, '{slug}')]" + "//div[contains(@class, 'filter-options')]" + f"//a[normalize-space()='{option}']" + ), + wait_for="presence", + ), + ) + apply_filters = self.web_driver.find_elements(By.ID, "ow-apply-filter") + if apply_filters: + apply_filters[0].click() + WebDriverWait(self.web_driver, 5).until( + lambda driver: driver.current_url != current_url + ) + if tables: + WebDriverWait(self.web_driver, 5).until(EC.staleness_of(tables[0])) + self.hide_loading_overlay() + + def _open_autocomplete_filter(self, param_name): + self.find_element( + by=By.XPATH, + value=( + f"//div[@id='ow-changelist-filter']//select[@name='{param_name}']" + "/following-sibling::span[contains(@class, 'select2')]" + ), + ).click() + self.wait_for_invisibility( + By.CSS_SELECTOR, ".select2-results__option.loading-results" + ) + + def _autocomplete_options(self, param_name): + self._open_autocomplete_filter(param_name) + options = [ + option.text + for option in self.find_elements( + by=By.CSS_SELECTOR, + value=".select2-container--open .select2-results__option", + ) + ] + self.find_element(by=By.CSS_SELECTOR, value="#content").click() + return options + + def _filter_by_autocomplete(self, param_name, option): + current_url = self.web_driver.current_url + tables = self.web_driver.find_elements(By.CSS_SELECTOR, "#result_list") + self._open_autocomplete_filter(param_name) + self.find_element( + by=By.XPATH, + value=( + "//li[contains(@class, 'select2-results__option')]" + f"[normalize-space()='{option}']" + ), + ).click() + self.find_element(by=By.ID, value="ow-apply-filter").click() + WebDriverWait(self.web_driver, 5).until( + lambda driver: driver.current_url != current_url + ) + if tables: + WebDriverWait(self.web_driver, 5).until(EC.staleness_of(tables[0])) + self.hide_loading_overlay() + + def _filter_options(self, title): + slug = title.replace(" ", "-") + return [ + option.get_attribute("textContent").strip() + for option in self.web_driver.find_elements( + By.CSS_SELECTOR, f".ow-filter.{slug} .filter-options a" + ) + ] + + def _changelist_labels(self): + return [ + row.find_element(By.CSS_SELECTOR, "th.field-label").text + for row in self._rows() + ] + + def _select_options(self, field_id): + return [ + option.text + for option in self.find_elements( + by=By.CSS_SELECTOR, + value=f"#{field_id} option", + wait_for="presence", + ) + if option.get_attribute("value") + ] + + def _wait_for_url(self, path, timeout=None): + WebDriverWait(self.web_driver, timeout or 5).until( + lambda driver: urlparse(driver.current_url).path == path + ) + self.assertEqual(urlparse(self.web_driver.current_url).path, path) + + def _wait_for_review_page(self): + self._wait_for_url(self.confirm_url) + self.wait_for_visibility(By.CSS_SELECTOR, ".command-summary", timeout=5) + + def _wait_for_batch_result(self, label, status, count): + batch = BatchCommand.objects.get(label=label) + self._wait_for_url( + reverse(f"admin:{self.app_label}_batchcommand_change", args=[batch.pk]), + # the test env runs celery synchronously, + # so 50 commands take a while + timeout=30, + ) + self.wait_for_visibility( + By.CSS_SELECTOR, "ul.messagelist li.success", timeout=5 + ) + self.wait_for_visibility( + By.CSS_SELECTOR, + f".field-colored_status .command-status.{status}", + timeout=5, + ) + WebDriverWait(self.web_driver, 5).until( + lambda driver: self._command_statuses() == [status] * count + ) + + def _rows(self): + return self.find_elements(by=By.CSS_SELECTOR, value="#result_list tbody tr") + + def _device_names(self): + return [ + row.find_element(By.CSS_SELECTOR, "th.field-name").text + for row in self._rows() + ] + + def _command_device_names(self): + return [ + row.find_element(By.CSS_SELECTOR, "td:first-child").text + for row in self._rows() + ] + + def _command_statuses(self): + return [ + row.find_element(By.CSS_SELECTOR, ".command-status").text + for row in self._rows() + ] + + def _summary(self): + summary = {} + for row in self.find_elements( + by=By.CSS_SELECTOR, value=".command-summary .form-row" + ): + summary[row.find_element(By.TAG_NAME, "label").text] = row.find_element( + By.CSS_SELECTOR, ".readonly" + ).text + return summary + + def test_execute_batch_command(self): + org1 = self._get_org() + org2 = self._create_org(name="org2", slug="org2") + devices1 = self._create_devices(org1, 5) + devices2 = self._create_devices( + org2, + 2, + credentials=self._create_credentials_with_key( + organization=org2, port=self.ssh_server.port + ), + ) + grouped_device, located_device, *_ = devices1 + group1 = self._create_device_group(name="group1", organization=org1) + location1 = self._create_location(name="location1", organization=org1) + group2 = self._create_device_group(name="group2", organization=org2) + location2 = self._create_location(name="location2", organization=org2) + grouped_device.group = group1 + grouped_device.full_clean() + grouped_device.save() + self._create_object_location(content_object=located_device, location=location1) + self.login() + + with self.subTest("custom command"): + self._fill_wizard( + type="Custom commands", + label="small-custom", + organization=org2, + command_input={"command": "echo test"}, + ) + self.find_element(by=By.ID, value="review-command-btn").click() + self._wait_for_review_page() + summary = self._summary() + self.assertEqual(summary["Type"], "Custom commands") + self.assertEqual(summary["Command"], "echo test") + self.assertEqual(summary["Label"], "small-custom") + self.assertEqual(summary["Targets"], org2.name) + self.assertEqual(summary["Will run on"], "2 devices") + self.assertEqual(self._device_names(), [device.name for device in devices2]) + self.find_element(by=By.ID, value="execute-button").click() + self._wait_for_batch_result("small-custom", "success", 2) + label = self.find_element( + by=By.CSS_SELECTOR, value=".field-label .readonly" + ) + command_type = self.find_element( + by=By.CSS_SELECTOR, value=".field-type .readonly" + ) + command_input = self.find_element( + by=By.CSS_SELECTOR, value=".field-formatted_input .readonly" + ) + affected_devices = self.find_element( + by=By.CSS_SELECTOR, value=".field-affected_devices .readonly" + ) + self.assertEqual(label.text, "small-custom") + self.assertEqual(command_type.text, "Custom commands") + self.assertEqual(command_input.text, "echo test") + self.assertEqual(affected_devices.text, "2") + self.assertEqual( + sorted(self._command_device_names()), + sorted(device.name for device in devices2), + ) + self.assertEqual(self._command_statuses(), ["success", "success"]) + + with self.subTest("reboot"): + self._fill_wizard(type="Reboot", label="small-reboot", organization=org1) + self.find_element(by=By.ID, value="review-command-btn").click() + self._wait_for_review_page() + summary = self._summary() + self.assertEqual(summary["Type"], "Reboot") + self.assertNotIn("Command", summary) + self.assertEqual(summary["Will run on"], "5 devices") + self.find_element(by=By.ID, value="execute-button").click() + self._wait_for_batch_result("small-reboot", "failed", 5) + command_type = self.find_element( + by=By.CSS_SELECTOR, value=".field-type .readonly" + ) + command_input = self.find_element( + by=By.CSS_SELECTOR, value=".field-formatted_input .readonly" + ) + affected_devices = self.find_element( + by=By.CSS_SELECTOR, value=".field-affected_devices .readonly" + ) + self.assertEqual(command_type.text, "Reboot") + self.assertEqual(command_input.text, "-") + self.assertEqual(affected_devices.text, "5") + self.assertEqual( + sorted(self._command_device_names()), + sorted(device.name for device in devices1), + ) + self.assertEqual(self._command_statuses(), ["failed"] * 5) + + with self.subTest("change password"): + self._fill_wizard( + type="Change password", + label="small-password", + organization=org1, + command_input={ + "password": "tester123", + "confirm_password": "tester123", + }, + ) + self.find_element(by=By.ID, value="review-command-btn").click() + self._wait_for_review_page() + summary = self._summary() + self.assertEqual(summary["Type"], "Change password") + self.assertNotIn("Command", summary) + self.find_element(by=By.ID, value="execute-button").click() + self._wait_for_batch_result("small-password", "failed", 5) + command_type = self.find_element( + by=By.CSS_SELECTOR, value=".field-type .readonly" + ) + command_input = self.find_element( + by=By.CSS_SELECTOR, value=".field-formatted_input .readonly" + ) + self.assertEqual(command_type.text, "Change password") + self.assertEqual(command_input.text, "********") + self.assertEqual(self._command_statuses(), ["failed"] * 5) + + with self.subTest("device group target"): + self._fill_wizard( + type="Reboot", + label="small-group", + organization=org1, + group=group1, + ) + group_options = self._select_options("id_group") + self.assertEqual(group_options, [group1.name]) + self.assertNotIn(group2.name, group_options) + self.find_element(by=By.ID, value="review-command-btn").click() + self._wait_for_review_page() + summary = self._summary() + self.assertEqual(summary["Targets"], f"{org1.name}, {group1.name}") + self.assertEqual(summary["Will run on"], "1 devices") + self.assertEqual(self._device_names(), [grouped_device.name]) + self.find_element(by=By.ID, value="execute-button").click() + self._wait_for_batch_result("small-group", "failed", 1) + self.assertEqual(self._command_device_names(), [grouped_device.name]) + self.assertEqual(self._command_statuses(), ["failed"]) + + with self.subTest("location target"): + self._fill_wizard( + type="Reboot", + label="small-location", + organization=org1, + location=location1, + ) + location_options = self._select_options("id_location") + self.assertEqual(location_options, [location1.name]) + self.assertNotIn(location2.name, location_options) + self.find_element(by=By.ID, value="review-command-btn").click() + self._wait_for_review_page() + summary = self._summary() + self.assertEqual(summary["Targets"], f"{org1.name}, {location1.name}") + self.assertEqual(summary["Will run on"], "1 devices") + self.assertEqual(self._device_names(), [located_device.name]) + self.find_element(by=By.ID, value="execute-button").click() + self._wait_for_batch_result("small-location", "failed", 1) + self.assertEqual(self._command_device_names(), [located_device.name]) + self.assertEqual(self._command_statuses(), ["failed"]) + + with self.subTest("excluded devices"): + self._fill_wizard(type="Reboot", label="small-excluded", organization=org1) + self.find_element(by=By.ID, value="review-command-btn").click() + self._wait_for_review_page() + checkbox = self.find_element( + by=By.CSS_SELECTOR, value="#result_list tbody .device-checkbox" + ) + excluded_pk = checkbox.get_attribute("value") + checkbox.click() + selected_count = self.find_element(by=By.ID, value="selected-count") + execute_button = self.find_element(by=By.ID, value="execute-button") + excluded_field = self.find_element( + by=By.ID, value="id_excluded", wait_for="presence" + ) + self.assertEqual(selected_count.text, "4") + self.assertEqual(execute_button.text, "Execute on 4 devices") + self.assertEqual(excluded_field.get_attribute("value"), excluded_pk) + execute_button.click() + self._wait_for_batch_result("small-excluded", "failed", 4) + affected_devices = self.find_element( + by=By.CSS_SELECTOR, value=".field-affected_devices .readonly" + ) + device_pks = [row.get_attribute("data-device-pk") for row in self._rows()] + self.assertEqual(affected_devices.text, "4") + self.assertEqual(self._command_statuses(), ["failed"] * 4) + self.assertNotIn(excluded_pk, device_pks) + + self.assertEqual(self.get_browser_errors(), []) + + def test_execute_large_batch(self): + org1 = self._get_org() + devices = self._create_devices(org1, 50) + device_names = [device.name for device in devices] + per_page = 20 + second_page_end = per_page * 2 + first_page = device_names[:per_page] + second_page = device_names[per_page:second_page_end] + self.login() + self._fill_wizard(type="Reboot", label="large-reboot", organization=org1) + self.find_element(by=By.ID, value="review-command-btn").click() + self._wait_for_review_page() + summary = self._summary() + self.assertEqual(summary["Type"], "Reboot") + self.assertNotIn("Command", summary) + self.assertEqual(summary["Label"], "large-reboot") + self.assertEqual(summary["Targets"], org1.name) + self.assertEqual(summary["Will run on"], "50 devices") + + with self.subTest("the device table is paginated"): + selected_count = self.find_element(by=By.ID, value="selected-count") + paginator = self.find_element(by=By.CSS_SELECTOR, value=".paginator") + self.assertEqual(selected_count.text, "50") + self.assertEqual(self._device_names(), first_page) + self.assertIn("50 devices", paginator.text.lower()) + + with self.subTest("select all toggles the devices of the page"): + select_all = self.find_element(by=By.ID, value="select-all-devices") + selected_count = self.find_element(by=By.ID, value="selected-count") + select_all.click() + self.assertEqual(selected_count.text, "30") + select_all.click() + self.assertEqual(selected_count.text, "50") + + with self.subTest("exclusions survive pagination"): + checkbox = self.find_element( + by=By.CSS_SELECTOR, value="#result_list tbody .device-checkbox" + ) + excluded_pk = checkbox.get_attribute("value") + excluded_name = checkbox.get_attribute("aria-label").replace("Include ", "") + checkbox.click() + selected_count = self.find_element(by=By.ID, value="selected-count") + self.assertEqual(selected_count.text, "49") + self.open(f"{self.confirm_url}?p=2") + self._wait_for_url(self.confirm_url) + self.hide_loading_overlay() + selected_count = self.find_element(by=By.ID, value="selected-count") + excluded_field = self.find_element( + by=By.ID, value="id_excluded", wait_for="presence" + ) + self.assertEqual(self._device_names(), second_page) + self.assertEqual(selected_count.text, "49") + self.assertEqual(excluded_field.get_attribute("value"), excluded_pk) + self.open(self.confirm_url) + self._wait_for_url(self.confirm_url) + self.hide_loading_overlay() + checkbox = self.find_element( + by=By.CSS_SELECTOR, value="#result_list tbody .device-checkbox" + ) + self.assertEqual(checkbox.is_selected(), False) + + with self.subTest("the excluded device is left out of the batch"): + self.find_element(by=By.ID, value="execute-button").click() + self._wait_for_batch_result("large-reboot", "failed", 20) + label = self.find_element( + by=By.CSS_SELECTOR, value=".field-label .readonly" + ) + command_type = self.find_element( + by=By.CSS_SELECTOR, value=".field-type .readonly" + ) + command_input = self.find_element( + by=By.CSS_SELECTOR, value=".field-formatted_input .readonly" + ) + affected_devices = self.find_element( + by=By.CSS_SELECTOR, value=".field-affected_devices .readonly" + ) + paginator = self.find_element(by=By.CSS_SELECTOR, value=".paginator") + device_pks = [row.get_attribute("data-device-pk") for row in self._rows()] + command_names = self._command_device_names() + self.assertEqual(label.text, "large-reboot") + self.assertEqual(command_type.text, "Reboot") + self.assertEqual(command_input.text, "-") + self.assertEqual(affected_devices.text, "49") + self.assertEqual(len(command_names), 20) + self.assertEqual(paginator.text, "49 commands") + self.assertEqual(self._command_statuses(), ["failed"] * 20) + self.assertEqual( + set(command_names) - {device.name for device in devices}, set() + ) + self.assertNotIn(excluded_name, command_names) + self.assertNotIn(excluded_pk, device_pks) + + self.assertEqual(self.get_browser_errors(), []) + + def test_batch_command_organization_isolation_and_permissions(self): + org1 = self._get_org() + org2 = self._create_org(name="org2", slug="org2") + devices1 = self._create_devices(org1, 2) + self._create_devices(org2, 2) + group1 = self._create_device_group(name="group1", organization=org1) + group2 = self._create_device_group(name="group2", organization=org2) + location1 = self._create_location(name="location1", organization=org1) + location2 = self._create_location(name="location2", organization=org2) + batch1 = self._create_batch_command(organization=org1, label="managed-batch") + batch2 = self._create_batch_command(organization=org2, label="unmanaged-batch") + operator = self._create_operator(organizations=[org1]) + self.login(username=operator.username, password="tester") + + with self.subTest("the wizard only offers the managed organization"): + self.open(self.execute_url) + self._wait_for_url(self.execute_url) + self.hide_loading_overlay() + organization_options = self._select_options("id_organization") + self.assertEqual(organization_options, [org1.name]) + self.assertNotIn(org2.name, organization_options) + + with self.subTest("targets of other organizations are not offered"): + self._select2("id_organization", org1.name) + group_options = self._select_options("id_group") + location_options = self._select_options("id_location") + self.assertEqual(group_options, [group1.name]) + self.assertEqual(location_options, [location1.name]) + self.assertNotIn(group2.name, group_options) + self.assertNotIn(location2.name, location_options) + + with self.subTest("the changelist hides other organizations"): + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + self.hide_loading_overlay() + labels = [ + row.find_element(By.CSS_SELECTOR, "th.field-label").text + for row in self._rows() + ] + self.assertEqual(labels, [batch1.label]) + self.assertNotIn(batch2.label, labels) + + with self.subTest("a batch of another organization cannot be opened"): + self.open( + reverse(f"admin:{self.app_label}_batchcommand_change", args=[batch2.pk]) + ) + self._wait_for_url(reverse("admin:index")) + + with self.subTest("only the devices of the managed organization are targeted"): + self._fill_wizard(type="Reboot", label="isolated-reboot", organization=org1) + self.find_element(by=By.ID, value="review-command-btn").click() + self._wait_for_review_page() + summary = self._summary() + self.assertEqual(summary["Targets"], org1.name) + self.assertEqual(summary["Will run on"], "2 devices") + self.assertEqual(self._device_names(), [device.name for device in devices1]) + self.find_element(by=By.ID, value="execute-button").click() + self._wait_for_batch_result("isolated-reboot", "failed", 2) + self.assertEqual( + sorted(self._command_device_names()), + sorted(device.name for device in devices1), + ) + self.assertEqual(self._command_statuses(), ["failed"] * 2) + + with self.subTest("the view permission is not enough to execute"): + viewer = self._create_operator( + organizations=[org1], username="viewer", email="viewer@test.com" + ) + viewer.groups.clear() + viewer.user_permissions.set( + Permission.objects.filter(codename="view_batchcommand") + ) + self.web_driver.delete_all_cookies() + self.login(username=viewer.username, password="tester") + self.web_driver.get(f"{self.live_server_url}{self.execute_url}") + self.assertEqual( + self.find_element(by=By.TAG_NAME, value="body").text, + "403 Forbidden", + ) + self.web_driver.get(f"{self.live_server_url}{self.confirm_url}") + self.assertEqual( + self.find_element(by=By.TAG_NAME, value="body").text, + "403 Forbidden", + ) + + def test_batch_command_menu_search_and_filters(self): + org1 = self._get_org() + org2 = self._create_org(name="org2", slug="org2") + devices = self._create_devices(org1, 50) + searched_device, grouped_device, located_device, *_ = devices + self._create_devices(org2, 1) + group1 = self._create_device_group(name="group1", organization=org1) + location1 = self._create_location(name="location1", organization=org1) + grouped_device.group = group1 + grouped_device.full_clean() + grouped_device.save() + self._create_object_location(content_object=located_device, location=location1) + self._create_batch_command(organization=org2, label="org2-batch") + self._create_batch_command(organization=org1, label="group-batch", group=group1) + self._create_batch_command( + organization=org1, label="location-batch", location=location1 + ) + self.login() + + with self.subTest("the wizard is reachable from the menu"): + self.open(reverse("admin:index")) + self._open_menu_item("Network Operations", "Mass command execute") + self._wait_for_url(self.execute_url) + + with self.subTest("the whole flow runs on every device"): + self._fill_wizard( + type="Reboot", + label="menu-reboot", + organization=org1, + open_page=False, + ) + self.find_element(by=By.ID, value="review-command-btn").click() + self._wait_for_review_page() + summary = self._summary() + self.assertEqual(summary["Type"], "Reboot") + self.assertEqual(summary["Label"], "menu-reboot") + self.assertEqual(summary["Targets"], org1.name) + self.assertEqual(summary["Will run on"], "50 devices") + self.find_element(by=By.ID, value="execute-button").click() + self._wait_for_batch_result("menu-reboot", "failed", 20) + affected_devices = self.find_element( + by=By.CSS_SELECTOR, value=".field-affected_devices .readonly" + ) + paginator = self.find_element(by=By.CSS_SELECTOR, value=".paginator") + command_names = self._command_device_names() + self.assertEqual(affected_devices.text, "50") + self.assertEqual(paginator.text, "50 commands") + self.assertEqual(len(command_names), 20) + self.assertEqual( + set(command_names) - {device.name for device in devices}, set() + ) + self.assertEqual(self._command_statuses(), ["failed"] * 20) + + with self.subTest("the changelist is reachable from the menu"): + self.open(reverse("admin:index")) + self._open_menu_item("Network Operations", "Mass command admin") + self._wait_for_url(self.changelist_url) + self.assertEqual( + self._changelist_labels(), + ["menu-reboot", "location-batch", "group-batch", "org2-batch"], + ) + + with self.subTest("the changelist search matches the label"): + self._search("menu-reboot") + self.assertEqual(self._changelist_labels(), ["menu-reboot"]) + self._search("no-such-batch") + self.assertEqual( + self.find_element(by=By.CSS_SELECTOR, value=".paginator").text, + "0 Mass commands", + ) + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + + with self.subTest("the changelist filters narrow the results"): + self._filter_by("status", "failed") + self.assertEqual(self._changelist_labels(), ["menu-reboot"]) + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + self._filter_by("status", "idle") + self.assertEqual( + set(self._changelist_labels()), + {"org2-batch", "group-batch", "location-batch"}, + ) + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + self._filter_by("type", "Reboot") + self.assertEqual(self._changelist_labels(), ["menu-reboot"]) + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + self._filter_by("type", "Custom commands") + self.assertEqual( + set(self._changelist_labels()), + {"org2-batch", "group-batch", "location-batch"}, + ) + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + + with self.subTest("the changelist autocomplete filters narrow the results"): + self._filter_by_autocomplete("organization", org2.name) + self.assertEqual(self._changelist_labels(), ["org2-batch"]) + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + self._filter_by_autocomplete("group_id", group1.name) + self.assertEqual(self._changelist_labels(), ["group-batch"]) + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + self._filter_by_autocomplete("location_id", location1.name) + self.assertEqual(self._changelist_labels(), ["location-batch"]) + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + + with self.subTest("the organization filter offers every organization"): + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + organization_options = self._autocomplete_options("organization") + self.assertIn(org1.name, organization_options) + self.assertIn(org2.name, organization_options) + + with self.subTest("the batch is opened from the changelist"): + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + self.find_element( + by=By.XPATH, + value="//table[@id='result_list']//a[normalize-space()='menu-reboot']", + ).click() + batch = BatchCommand.objects.get(label="menu-reboot") + self._wait_for_url( + reverse(f"admin:{self.app_label}_batchcommand_change", args=[batch.pk]) + ) + self.assertEqual(len(self._rows()), 20) + + with self.subTest("the command table is searchable"): + self.open( + reverse( + f"admin:{self.app_label}_batchcommand_change", + args=[BatchCommand.objects.get(label="menu-reboot").pk], + ) + ) + self._search(searched_device.name) + self.assertEqual(self._command_device_names(), [searched_device.name]) + self.assertEqual(self._command_statuses(), ["failed"]) + self._search("no-such-device") + self.assertEqual( + self.find_element( + by=By.CSS_SELECTOR, value="#result_list .empty-results" + ).text, + "No commands found.", + ) + + with self.subTest("the command table is filterable"): + self.open( + reverse( + f"admin:{self.app_label}_batchcommand_change", + args=[BatchCommand.objects.get(label="menu-reboot").pk], + ) + ) + self._filter_by("status", "failed") + self.assertEqual(self._command_statuses(), ["failed"] * 20) + self._filter_by("status", "success") + self.assertEqual( + self.find_element( + by=By.CSS_SELECTOR, value="#result_list .empty-results" + ).text, + "No commands found.", + ) + self.open( + reverse( + f"admin:{self.app_label}_batchcommand_change", + args=[BatchCommand.objects.get(label="menu-reboot").pk], + ) + ) + self._filter_by("device group", group1.name) + self.assertEqual(self._command_device_names(), [grouped_device.name]) + self.open( + reverse( + f"admin:{self.app_label}_batchcommand_change", + args=[BatchCommand.objects.get(label="menu-reboot").pk], + ) + ) + self._filter_by("location", location1.name) + self.assertEqual(self._command_device_names(), [located_device.name]) + self.open( + reverse( + f"admin:{self.app_label}_batchcommand_change", + args=[BatchCommand.objects.get(label="menu-reboot").pk], + ) + ) + self._filter_by("organization", org1.name) + self.assertEqual(len(self._command_device_names()), 20) + + with self.subTest("the operator only sees the managed organization"): + operator = self._create_operator(organizations=[org1]) + self.web_driver.delete_all_cookies() + self.login(username=operator.username, password="tester") + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + self.assertEqual( + self._changelist_labels(), + ["menu-reboot", "location-batch", "group-batch"], + ) + organization_options = self._autocomplete_options("organization") + self.assertIn(org1.name, organization_options) + self.assertNotIn(org2.name, organization_options) + self._search("menu-reboot") + self.assertEqual(self._changelist_labels(), ["menu-reboot"]) + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + self._filter_by("status", "failed") + self.assertEqual(self._changelist_labels(), ["menu-reboot"]) + self.open( + reverse( + f"admin:{self.app_label}_batchcommand_change", + args=[BatchCommand.objects.get(label="menu-reboot").pk], + ) + ) + filter_titles = [ + title.text + for title in self.find_elements( + by=By.CSS_SELECTOR, value="#ow-changelist-filter .filter-title h3" + ) + ] + self.assertIn("By status", filter_titles) + self.assertNotIn("By organization", filter_titles) + + def test_organization_scoped_custom_command_type(self): + org1 = self._create_org( + name="scoped org", slug="scoped-org", id=UUID(SCOPED_ORGANIZATION_ID) + ) + org2 = self._create_org( + name="org2", slug="org2", id=UUID(DEFAULT_ORGANIZATION_ID) + ) + devices1 = self._create_devices(org1, 2) + self._create_devices(org2, 1) + self._create_batch_command(organization=org2, label="org2-batch") + operator1 = self._create_operator( + organizations=[org1], username="operator1", email="operator1@test.com" + ) + operator2 = self._create_operator( + organizations=[org2], username="operator2", email="operator2@test.com" + ) + default_types = ["Custom commands", "Reboot", "Change password"] + + with self.subTest("the scoped organization is offered the custom type"): + self.web_driver.delete_all_cookies() + self.login(username=operator1.username, password="tester") + self.open(self.execute_url) + self._wait_for_url(self.execute_url) + self.hide_loading_overlay() + self.assertEqual( + self._select_options("id_type"), default_types + ["UCI show"] + ) + + with self.subTest("the custom type renders the input of its schema"): + self._fill_wizard( + type="UCI show", + label="uci-show", + organization=org1, + command_input={"config": "network"}, + ) + self.find_element(by=By.ID, value="review-command-btn").click() + self._wait_for_review_page() + summary = self._summary() + self.assertEqual(summary["Type"], "UCI show") + self.assertEqual(summary["Command"], "config: network") + self.assertEqual(summary["Will run on"], "2 devices") + self.assertEqual(self._device_names(), [device.name for device in devices1]) + self.find_element(by=By.ID, value="execute-button").click() + self._wait_for_batch_result("uci-show", "failed", 2) + command_type = self.find_element( + by=By.CSS_SELECTOR, value=".field-type .readonly" + ) + command_input = self.find_element( + by=By.CSS_SELECTOR, value=".field-formatted_input .readonly" + ) + self.assertEqual(command_type.text, "UCI show") + self.assertEqual(command_input.text, "config: network") + self.assertEqual(self._command_statuses(), ["failed"] * 2) + + with self.subTest("the changelist type filter offers the custom type"): + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + self.assertIn("UCI show", self._filter_options("type")) + self._filter_by("type", "UCI show") + self.assertEqual(self._changelist_labels(), ["uci-show"]) + + with self.subTest("another organization only sees the default types"): + self.web_driver.delete_all_cookies() + self.login(username=operator2.username, password="tester") + self.open(self.execute_url) + self._wait_for_url(self.execute_url) + self.hide_loading_overlay() + self.assertEqual(self._select_options("id_type"), default_types) + self.open(self.changelist_url) + self._wait_for_url(self.changelist_url) + self.assertEqual(self._changelist_labels(), ["org2-batch"]) + self.assertNotIn("UCI show", self._filter_options("type")) diff --git a/openwisp_controller/connection/tests/utils.py b/openwisp_controller/connection/tests/utils.py index 1574e2364..9be852d4e 100644 --- a/openwisp_controller/connection/tests/utils.py +++ b/openwisp_controller/connection/tests/utils.py @@ -1,12 +1,15 @@ import os +from django.contrib.messages import get_messages from mockssh import Server from swapper import load_model from openwisp_users.tests.utils import TestOrganizationMixin from ...config.tests.utils import CreateConfigTemplateMixin +from ...tests.utils import TestAdminMixin from .. import settings as app_settings +from ..admin import BatchCommandAdmin Credentials = load_model("connection", "Credentials") DeviceConnection = load_model("connection", "DeviceConnection") @@ -151,6 +154,34 @@ def _create_command(self, device_conn=None, device_conn_opts={}, **kwargs): return Command.objects.create(**opts) +class BatchCommandMixin(TestAdminMixin, CreateConnectionsMixin): + def _post_execute(self, **overrides): + data = { + "type": "custom", + "input": '{"command": "echo test"}', + "label": "test-label", + "notes": "", + "organization": "", + "group": "", + "location": "", + } + data.update(overrides) + return self.client.post(self.execute_url, data) + + def _start_wizard(self, **overrides): + response = self._post_execute(**overrides) + assert response.status_code == 302, response.context["form"].errors + return self.client.session[BatchCommandAdmin.session_key] + + def _post_confirm(self, token, excluded=""): + return self.client.post( + self.confirm_url, {"token": token, "excluded": excluded} + ) + + def _messages(self, response): + return [str(message) for message in get_messages(response.wsgi_request)] + + def _ping_command_callable(destination_address, interface_name=None): command = f"ping -c 4 {destination_address}" if interface_name: @@ -160,3 +191,7 @@ def _ping_command_callable(destination_address, interface_name=None): def _restart_network_command_callable(): return "/etc/init.d/networking restart" + + +def _uci_show_command_callable(config): + return f"uci show {config}" diff --git a/openwisp_controller/connection/widgets.py b/openwisp_controller/connection/widgets.py index 7160aa599..cda8c62ad 100644 --- a/openwisp_controller/connection/widgets.py +++ b/openwisp_controller/connection/widgets.py @@ -5,6 +5,7 @@ Credentials = swapper.load_model("connection", "Credentials") Command = swapper.load_model("connection", "Command") +BatchCommand = swapper.load_model("connection", "BatchCommand") app_label = Credentials._meta.app_label model_name = Credentials._meta.model_name @@ -46,3 +47,30 @@ def media(self): css = {"all": ["connection/css/command-inline.css"]} media = forms.Media(js=js, css=css) return super().media + media + + +class BatchCommandSchemaWidget(CommandSchemaWidget): + schema_view_name = ( + f"admin:{BatchCommand._meta.app_label}" + f"_{BatchCommand._meta.model_name}_schema" + ) + + app_label_model = f"{BatchCommand._meta.app_label}_{BatchCommand._meta.model_name}" + extra_attrs = { + "data-schema-selector": "#id_type", + "data-show-errors": "never", + "data-options": '{"disable_properties": true}', + } + + @property + def media(self): + return super(CommandSchemaWidget, self).media + + +class OrganizationScopedSelect(forms.Select): + def create_option(self, name, value, *args, **kwargs): + option = super().create_option(name, value, *args, **kwargs) + instance = getattr(value, "instance", None) + if instance is not None: + option["attrs"]["data-organization-id"] = str(instance.organization_id) + return option diff --git a/openwisp_controller/geo/estimated_location/tests/tests.py b/openwisp_controller/geo/estimated_location/tests/tests.py index b3f0d3772..f253aeea8 100644 --- a/openwisp_controller/geo/estimated_location/tests/tests.py +++ b/openwisp_controller/geo/estimated_location/tests/tests.py @@ -727,9 +727,10 @@ def _verify_location_details(device, mocked_response): old_location = device2.devicelocation.location device2.last_ip = "172.217.22.10" device2.save() - # 3 queries related to notifications cleanup + # 3 queries related to notifications cleanup, + # 1 to set BatchCommand.location to NULL when the location is deleted device2.refresh_from_db() - with self.assertNumQueries(16): + with self.assertNumQueries(17): manage_estimated_locations(device2.pk, device2.last_ip) mock_info.assert_called_once_with( f"Estimated location saved successfully for {device2.pk}" diff --git a/tests/openwisp2/sample_connection/migrations/0005_batchcommand_command_batch_command.py b/tests/openwisp2/sample_connection/migrations/0005_batchcommand_command_batch_command.py index c0f67afd1..19896a465 100644 --- a/tests/openwisp2/sample_connection/migrations/0005_batchcommand_command_batch_command.py +++ b/tests/openwisp2/sample_connection/migrations/0005_batchcommand_command_batch_command.py @@ -98,10 +98,11 @@ class Migration(migrations.Migration): blank=True, null=True, default=dict, - verbose_name="Skipped devices", + verbose_name="skipped devices", help_text=( - "Maps device UUIDs to validation error messages for " - "devices that were skipped during command creation." + "Maps device UUIDs to the name of the device and the " + "validation error that caused it to be skipped during " + "command creation." ), ), ),