Fix Input Mapping V2 gesture selection and used-input highlights - PR_26140_105-fix-input-mapping-v2-gesture-selection-and-highlights

Author: DavidQ

docs/dev/reports/input_mapping_v2_gesture_selection_highlights_report.md

@@ -0,0 +1,94 @@
+# Input Mapping V2 Gesture Selection + Highlights Report
+
+PR: PR_26140_105-fix-input-mapping-v2-gesture-selection-and-highlights
+
+## Source of Truth
+
+- Read `docs/dev/PROJECT_INSTRUCTIONS.md` before implementation.
+- Active `docs/pr/BUILD_PR.md` is an unrelated Level 18 rebase note, so this report follows the explicit PR105 user request as the current BUILD scope.
+- One PR purpose: fix Input Mapping V2 gesture selection, selected-input highlighting, and remove the visible Cross-device gesture group while preserving cross-device combo capture internally.
+
+## Changes
+
+- Added selected gesture state to Input Mapping V2.
+- Gesture buttons now stay visibly highlighted with `is-selected` and `aria-pressed="true"` after selection and after capture completion.
+- Selected descriptor gestures are applied to the next matching live capture:
+  - Keyboard Release/Hold/Press update captured keyboard mapping text and binding.
+  - Mouse Click/Double Click update captured mouse mapping text and binding.
+  - Game Controller Button/Trigger/Stick/DPad update captured gamepad mapping text and binding when selected.
+- Keyboard Release now captures as visible tile text such as `Keyboard KeyP Release` instead of falling back to `Keyboard KeyP Press`.
+- Non-default gesture bindings include the gesture id, for example `KeyP:KeyboardRelease`, so Press and Release for the same physical key remain distinct without schema changes.
+- Preserved existing immediate behavior for pointer drag and wheel gesture buttons.
+- Preserved combo capture behavior, including keyboard, mouse, wheel, game controller, and cross-device combinations.
+- Removed `Cross-device` as a visible gesture group and removed the `CrossDeviceCombo` visible descriptor.
+- Kept cross-device combo support available through the existing Combo gesture flow, including Game Controller Combo capturing a controller button plus keyboard Alt.
+- Preserved haptics/rumble UI behavior, shortcut suppression, context suppression, selected tile indication, and gamepad auto-poll behavior.
+
+## ToolState / Schema
+
+- No schemas changed.
+- No sample JSON changed.
+- Input Mapping V2 continues to use the existing tool-local mapping/toolState contract.
+
+## Playwright Impact
+
+Playwright impacted: Yes.
+
+Focused coverage now validates:
+
+- selected gesture remains highlighted,
+- Keyboard Release capture produces `Keyboard KeyP Release`,
+- selected tile highlights corresponding keyboard input,
+- selected tile highlights corresponding mouse input,
+- selected tile highlights corresponding game controller input,
+- Cross-device group and `CrossDeviceCombo` button are not visible,
+- cross-device combo representation still works through Game Controller Combo without a Cross-device UI group.
+
+Expected pass behavior: gesture selection is visible and affects the next matching capture; used inputs are highlighted for the selected tile; Cross-device is absent from visible gestures; cross-device combo still maps internally.
+
+Expected fail behavior: selected gestures lose highlighting, Keyboard Release maps as Press, used controls do not highlight for the selected tile, Cross-device appears as a visible group, or cross-device combo cannot be represented without that group.
+
+## Validation
+
+Passed:
+
+- `node --check src/engine/input/InputCapabilityDescriptors.js`
+- `node --check tools/input-mapping-v2/js/ToolStarterApp.js`
+- `node --check tools/input-mapping-v2/js/controls/GestureListControl.js`
+- `node --check tools/input-mapping-v2/js/services/EngineInputSourceService.js`
+- `node --check tests/playwright/tools/WorkspaceManagerV2.spec.mjs`
+- `npx playwright test tests/playwright/tools/WorkspaceManagerV2.spec.mjs -g "Input Mapping V2" --project=playwright --workers=1 --reporter=list` - 3 passed.
+- `npm run test:workspace-v2` - 62 passed.
+- `git diff --check` - passed; Git reported LF-to-CRLF working-copy warnings only.
+- `git status --short -- samples games` - no sample/game JSON changes.
+- `rg "imageDataUrl" tools/input-mapping-v2 src/engine/input tests/playwright/tools/WorkspaceManagerV2.spec.mjs` - no matches.
+
+Coverage:
+
+- Playwright V8 coverage was produced at `docs/dev/reports/playwright_v8_coverage_report.txt`.
+- `docs/dev/reports/coverage_changed_js_guardrail.txt` reports no changed runtime JS coverage warnings.
+
+Skipped:
+
+- Full samples smoke test was skipped by explicit PR instruction.
+
+## Manual Validation
+
+1. Open Input Mapping V2.
+2. Add or select a `Pause` mapping tile.
+3. Click `Keyboard > Release` and confirm the Release gesture remains highlighted.
+4. Click `Capture Keyboard`, press `P`, and confirm the Pause tile shows `Keyboard KeyP Release`.
+5. Select the Pause tile and confirm the Keyboard used-input display highlights `Keyboard KeyP Release`.
+6. Add mouse and game controller mappings, select each tile, and confirm the matching Mouse/Game Controller used-input displays are highlighted.
+7. Confirm no visible `Cross-device` gesture group appears.
+8. Start a combo from an existing device Combo gesture and confirm a controller button + keyboard Alt can still map as a combo.
+9. Confirm haptics/rumble controls and shortcut/context suppression still behave as before.
+
+## Files Changed
+
+- `src/engine/input/InputCapabilityDescriptors.js`
+- `tests/playwright/tools/WorkspaceManagerV2.spec.mjs`
+- `tools/input-mapping-v2/js/ToolStarterApp.js`
+- `tools/input-mapping-v2/js/controls/GestureListControl.js`
+- `tools/input-mapping-v2/js/services/EngineInputSourceService.js`
+- `tools/input-mapping-v2/styles/inputMappingV2.css`

src/engine/input/InputCapabilityDescriptors.js

@@ -82,8 +82,7 @@ const GESTURE_DEFINITIONS = Object.freeze([
     gameControllerGesture('GameControllerTrigger', 'Trigger', 'Game controller trigger. Use for analog trigger actions such as accelerate, brake, or charge.', 'gameController'),
     gameControllerGesture('GameControllerStick', 'Stick', 'Game controller stick. Use for analog movement, aiming, or steering.', 'gameController'),
     gameControllerGesture('GameControllerDPad', 'DPad', 'Game controller DPad. Use for directional menu or movement actions.', 'gameController'),
-    comboGesture('GameControllerCombo', 'Game Controller', ['gameController'], 'Game Controller Combo. Use for controller combinations such as Button 1 + Button 2.'),
-    crossDeviceComboGesture()
+    comboGesture('GameControllerCombo', 'Game Controller', ['gameController'], 'Game Controller Combo. Use for controller combinations such as Button 1 + Button 2. Cross-device combinations such as Joystick Button 1 + Keyboard Alt are supported during combo capture.')
 ]);
 
 export function inputDeviceCapabilities({
@@ -259,22 +258,6 @@ function comboGesture(binding, deviceLabel, requiredDeviceIds, title) {
     };
 }
 
-function crossDeviceComboGesture() {
-    return {
-        binding: 'CrossDeviceCombo',
-        captureKind: 'combo',
-        deviceLabel: 'Cross-device',
-        displayLabelLines: ['Cross-device', 'Combo'],
-        engine: 'InputService Combo',
-        label: 'Combo',
-        anyOfDeviceIds: ['keyboard', 'mouse', 'gameController'],
-        minimumEnabledDevices: 2,
-        requiredDeviceIds: [],
-        source: 'keyboard',
-        title: 'Cross-device Combo. Use for combinations such as Joystick Button 1 + Keyboard Alt. Capture any two keyboard, mouse, wheel, or game controller inputs for one selected action.'
-    };
-}
-
 function patternGesture({ binding, deviceLabel, label, title, deviceId }) {
     return {
         binding,

tests/playwright/tools/WorkspaceManagerV2.spec.mjs

@@ -1459,6 +1459,111 @@ test.describe("Workspace Manager V2 bootstrap", () => {
     }
   });
 
+  test("selects Input Mapping V2 gestures for capture and highlights used controls", async ({ page }) => {
+    await page.setViewportSize({ width: 1600, height: 900 });
+    const pageErrors = [];
+    page.on("pageerror", (error) => {
+      pageErrors.push(error.message);
+    });
+    await page.addInitScript(() => {
+      window.__inputMappingV2MockGamepads = [
+        {
+          axes: [0, 0, 0, 0],
+          buttons: [{ pressed: false }, { pressed: false }],
+          connected: true,
+          id: "Arcade Joystick",
+          index: 0,
+          mapping: "",
+          timestamp: 10
+        }
+      ];
+      Object.defineProperty(navigator, "getGamepads", {
+        configurable: true,
+        value: () => window.__inputMappingV2MockGamepads
+      });
+    });
+    const server = await openInputMappingV2(page);
+    try {
+      await expect(page.locator("body[data-tool-id='input-mapping-v2']")).toBeVisible();
+      await expect(page.locator(".input-mapping-v2__gesture-group", { hasText: "Cross-device" })).toHaveCount(0);
+      await expect(page.locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='CrossDeviceCombo']")).toHaveCount(0);
+
+      await page.locator("#inputMappingV2ActionSelect").selectOption("pause");
+      await page.locator("#inputMappingV2AddActionButton").click();
+      const keyboardReleaseButton = page.locator(".input-mapping-v2__gesture-group", { hasText: "Keyboard" })
+        .locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='KeyboardRelease']");
+      await keyboardReleaseButton.click();
+      await expect(keyboardReleaseButton).toHaveClass(/is-selected/);
+      await expect(keyboardReleaseButton).toHaveAttribute("aria-pressed", "true");
+      await page.locator("#inputMappingV2CaptureKeyboardButton").click();
+      await page.keyboard.press("KeyP");
+      await expect(keyboardReleaseButton).toHaveClass(/is-selected/);
+      await expect(page.locator(".input-mapping-v2__mapping-card[data-input-mapping-tile-action-id='pause']")).toContainText("Keyboard KeyP Release");
+      await expect(page.locator(".input-mapping-v2__mapping-card[data-input-mapping-tile-action-id='pause']")).not.toContainText("Keyboard KeyP Press");
+      await expect(page.locator("#inspectorOutput")).toContainText('"binding": "KeyP:KeyboardRelease"');
+      await expect(page.locator("#inputMappingV2CaptureKeyboardButton")).toHaveClass(/has-used-input/);
+      await expect(page.locator(".input-mapping-v2__used-source[data-input-mapping-used-source-id='keyboard'] .input-mapping-v2__used-control", { hasText: "Keyboard KeyP Release" })).toHaveCount(1);
+
+      await page.locator("#inputMappingV2ActionSelect").selectOption("fire");
+      await page.locator("#inputMappingV2AddActionButton").click();
+      await page.locator(".input-mapping-v2__gesture-group", { hasText: "Mouse" })
+        .locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='MouseClick']")
+        .click();
+      await expect(page.locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='MouseClick']")).toHaveClass(/is-selected/);
+      await page.locator("#inputMappingV2CaptureMouseButton").click();
+      await page.evaluate(() => {
+        window.dispatchEvent(new MouseEvent("mousedown", {
+          bubbles: true,
+          button: 2,
+          cancelable: true
+        }));
+      });
+      await expect(page.locator(".input-mapping-v2__mapping-card[data-input-mapping-tile-action-id='fire']")).toContainText("Mouse Right Button Click");
+      await expect(page.locator("#inputMappingV2CaptureMouseButton")).toHaveClass(/has-used-input/);
+      await expect(page.locator(".input-mapping-v2__used-source[data-input-mapping-used-source-id='mouse'] .input-mapping-v2__used-control", { hasText: "Mouse Right Button Click" })).toHaveCount(1);
+
+      await page.locator("#inputMappingV2ActionSelect").selectOption("start");
+      await page.locator("#inputMappingV2AddActionButton").click();
+      await page.locator(".input-mapping-v2__gesture-group", { hasText: "Game Controller" })
+        .locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='GameControllerButton']")
+        .click();
+      await expect(page.locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='GameControllerButton']")).toHaveClass(/is-selected/);
+      await page.locator(".input-mapping-v2__gamepad-capture-button[data-input-mapping-gamepad-index='0']").click();
+      await page.evaluate(() => {
+        window.__inputMappingV2MockGamepads[0] = {
+          ...window.__inputMappingV2MockGamepads[0],
+          buttons: [{ pressed: false }, { pressed: true }],
+          timestamp: 11
+        };
+      });
+      await expect(page.locator(".input-mapping-v2__gamepad-capture-button[data-input-mapping-gamepad-index='0']")).not.toHaveClass(/is-capturing/, { timeout: 2000 });
+      await expect(page.locator(".input-mapping-v2__mapping-card[data-input-mapping-tile-action-id='start']")).toContainText("Game Controller Button 1 Button");
+      await expect(page.locator(".input-mapping-v2__gamepad-capture-button[data-input-mapping-gamepad-index='0']")).toHaveClass(/has-used-input/);
+      await expect(page.locator(".input-mapping-v2__used-source[data-input-mapping-used-source-id='gameController'] .input-mapping-v2__used-control", { hasText: "Game Controller Button 1 Button" })).toHaveCount(1);
+
+      await page.locator(".input-mapping-v2__gesture-group", { hasText: "Game Controller" })
+        .locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='GameControllerCombo']")
+        .click();
+      await expect(page.locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='GameControllerCombo']")).toHaveClass(/is-selected/);
+      await expect(page.locator("#inputMappingV2CaptureMessage")).toContainText("Combo capture recorded Game Controller Button 1");
+      await page.evaluate(() => {
+        window.dispatchEvent(new KeyboardEvent("keydown", {
+          altKey: true,
+          bubbles: true,
+          cancelable: true,
+          code: "AltLeft",
+          key: "Alt"
+        }));
+      });
+      await expect(page.locator(".input-mapping-v2__mapping-card[data-input-mapping-tile-action-id='start']")).toContainText("Combo Game Controller Button 1 + Alt");
+      await expect(page.locator("#inspectorOutput")).toContainText('"binding": "Combo:Pad0:Button1+AltLeft"');
+      expect(pageErrors).toEqual([]);
+    } finally {
+      await workspaceV2CoverageReporter.stop(page);
+      await server.close();
+    }
+  });
+
   test("launches Input Mapping V2 and captures keyboard mappings", async ({ page }) => {
     await page.setViewportSize({ width: 1920, height: 900 });
     const server = await openInputMappingV2(page);
@@ -1526,14 +1631,15 @@ test.describe("Workspace Manager V2 bootstrap", () => {
       await expect(page.locator(".input-mapping-v2__gesture-group", { hasText: "Mouse" })).toContainText("Wheel Up");
       const gestureGroupTitles = await page.locator(".input-mapping-v2__gesture-group strong").allTextContents();
       expect(gestureGroupTitles).not.toContain("Wheel");
-      expect(gestureGroupTitles).toContain("Cross-device");
+      expect(gestureGroupTitles).not.toContain("Cross-device");
+      await expect(page.locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='CrossDeviceCombo']")).toHaveCount(0);
       await expect(page.locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='MousePrimaryDrag']")).toHaveAttribute("title", /continuous movement while held/);
       await expect(page.locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='MousePrimaryDragRelease']")).toHaveAttribute("title", /box-selecting objects/);
       await expect(page.locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='MouseWheelUp']")).toHaveAttribute("title", /zoom, scrolling, or cycling selections/);
       await expect(page.locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='KeyboardCombo']")).toHaveAttribute("title", /Ctrl \+ R/);
       await expect(page.locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='MouseCombo']")).toHaveAttribute("title", /Shift \+ Mouse Right Button/);
       await expect(page.locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='GameControllerCombo']")).toHaveAttribute("title", /Button 1 \+ Button 2/);
-      await expect(page.locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='CrossDeviceCombo']")).toHaveAttribute("title", /Joystick Button 1 \+ Keyboard Alt/);
+      await expect(page.locator(".input-mapping-v2__gesture-button[data-input-mapping-gesture-binding='GameControllerCombo']")).toHaveAttribute("title", /Joystick Button 1 \+ Keyboard Alt/);
       const compactAccordionSpacing = await page.locator("#gestureSetupContent, #captureInputContent").evaluateAll((contents) => (
         contents.map((content) => ({
           columnGap: getComputedStyle(content).columnGap,
@@ -2013,7 +2119,7 @@ test.describe("Workspace Manager V2 bootstrap", () => {
           timestamp: 43.5
         };
       });
-      await page.locator(".input-mapping-v2__gesture-group", { hasText: "Cross-device" }).locator(".input-mapping-v2__gesture-button", { hasText: "Combo" }).click();
+      await page.locator(".input-mapping-v2__gesture-group", { hasText: "Game Controller" }).locator(".input-mapping-v2__gesture-button", { hasText: "Combo" }).click();
       await expect(page.locator("#inputMappingV2CaptureMessage")).toContainText("Combo capture recorded Game Controller Button 1");
       await page.evaluate(() => {
         window.dispatchEvent(new KeyboardEvent("keydown", {