Automated Hardware/Software Validation Test System

Python · Pytest · Linux · GitHub Actions · CI/CD

A modular test automation project demonstrating repeatable validation of a simulated device interface, regression testing, automated CI execution, and JSON/HTML reporting.

Project Overview · Architecture · Setup · CI/CD · Reports

---

🎯 Project Overview

Reliable embedded products need repeatable validation of device behavior, input boundaries, communication responses, and failure handling. This project demonstrates a maintainable way to organize those checks with Python and Pytest, execute them in GitHub Actions, and publish machine-readable and human-readable test reports.

The framework includes a software-based device simulator and reusable tests for connection state, sensor readings, configuration updates, command handling, and negative scenarios.

Scope and accuracy: The current implementation tests a simulated device interface. It does not communicate with a physical development board, firmware image, sensor, or lab instrument. Hardware integration is a planned extension.

💡 What This Project Demonstrates

• Reusable test design using Pytest fixtures and parameterized test cases.
• Functional validation of simulated device state and command/response behavior.
• Negative testing for invalid values, unsupported commands, and disconnected access.
• Regression testing by running the suite consistently on each change.
• CI/CD automation through GitHub Actions on pushes and pull requests.
• Test reporting with JSON output and an HTML summary.
• Maintainable structure that separates the simulated device from test logic.

🏗️ System Architecture

                Push / Pull Request
                         |
                         v
                 GitHub Actions CI
                         |
                         v
                 Install Dependencies
                         |
                         v
                    Run Pytest
                         |
             +-----------+-----------+
             |                       |
             v                       v
      Validation Tests        Device Simulator
             |                       |
             +-----------+-----------+
                         |
                         v
                  Assertions & Results
                         |
                +--------+--------+
                |                 |
                v                 v
            JSON Report       HTML Report
                |
                v
        CI Artifact Upload

Execution flow: The test suite calls the simulated device API, checks expected results with assertions, and records outcomes. The CI workflow runs the same tests in a clean Ubuntu runner and uploads the reports as an artifact.

🧪 Validation Coverage

Test area	Example checks
Connection lifecycle	Connect and disconnect state transitions
Device status	Required status fields and firmware version
Sensor input	Reading availability, numeric type, and expected bounds
Configuration	Boolean enable/disable values and invalid-type rejection
Command handling	Ping response and firmware-version response
Negative scenarios	Disconnected reads, unsupported commands, malformed payloads
Regression	Re-run the full suite on pushes and pull requests

The simulator uses predictable sample data so the checks are reproducible. These tests do not measure real sensor accuracy, timing, electrical behavior, or hardware reliability.

🧰 Technology Stack

Technology	Role in the project
Python	Validation logic and simulated device behavior
Pytest	Fixtures, assertions, parameterized tests, test execution
GitHub Actions	Automated CI workflow
Linux / Ubuntu	Local and CI execution environment
JSON	Structured test results
HTML	Human-readable test summary
Git / GitHub	Version control and collaboration workflow

📁 Repository Structure

automated-hw-sw-validation/
├── .github/
│   └── workflows/
│       └── validation.yml
├── src/
│   └── validation/
│       ├── __init__.py
│       └── device_simulator.py
├── tests/
│   ├── conftest.py
│   └── test_validation.py
├── scripts/
│   └── generate_html_report.py
├── reports/
│   └── .gitkeep
├── requirements.txt
├── pytest.ini
├── run_tests.py
├── .gitignore
└── README.md

🚀 Quick Start

Prerequisites

• Python 3.10 or a compatible version.
• Git.
• Linux / Ubuntu, WSL, or another supported Python environment.

1. Clone the repository

git clone https://github.com/ManjunathGlO/automated-hw-sw-validation.git
cd automated-hw-sw-validation

If your repository has a different name or URL, update the clone command accordingly.

2. Create a virtual environment

python3 -m venv .venv
source .venv/bin/activate

Windows PowerShell:

.venv\Scripts\Activate.ps1

3. Install dependencies

python -m pip install --upgrade pip
pip install -r requirements.txt
pip install pytest-json-report

4. Run the full validation suite

python run_tests.py

The runner executes Pytest and writes the following reports:

• reports/test-report.json
• reports/test-report.html

The runner returns a non-zero exit code if the test suite fails, allowing CI to mark the run as failed.

Run Pytest directly

python -m pytest -v

🔄 Continuous Integration

The workflow at .github/workflows/validation.yml is configured to run on:

• Pushes to main or master.
• Pull requests targeting main or master.
• Manual runs using workflow_dispatch.

Each workflow run checks out the repository, sets up Python, installs dependencies, executes the tests, and uploads the reports/ directory as the validation-reports artifact.

To review a CI run:

1. Push the repository to GitHub.
2. Open the repository's Actions tab.
3. Select the Automated Validation workflow.
4. Open a run to review test logs.
5. Download the validation-reports artifact when available.

📊 Test Reports

JSON report

reports/test-report.json stores structured test results that can be consumed by scripts or other tools.

HTML report

reports/test-report.html provides a readable summary of total, passed, failed, skipped, and errored tests, along with failure details when available.

Reports are generated locally when python run_tests.py runs. In GitHub Actions, the report directory is uploaded as a workflow artifact. Generated report files are intentionally excluded from version control so the repository stays focused on source code.

🧠 Engineering Practices

• Separation of concerns: Test cases are separate from simulated device behavior.
• Reusable fixtures: Each test receives a fresh simulator instance.
• Parameterized testing: Multiple valid and invalid configuration values can be checked consistently.
• Clear assertions: Failed expectations identify the behavior that did not match.
• Negative testing: Unsupported requests and invalid input types are tested.
• Repeatability: CI executes the same suite after code changes.
• Actionable output: JSON and HTML reports make test outcomes easier to review.

🛣️ Future Enhancements

These are planned extensions, not features implemented in the current version:

• Add a real hardware adapter for UART/serial, USB, CAN, or a vendor SDK.
• Execute hardware-dependent tests against a physical development board.
• Add configurable timeouts, retries, and communication error recovery.
• Add Pytest markers to separate simulation tests from hardware tests.
• Capture device logs and associate results with firmware/build versions.
• Add code coverage reporting and CI quality gates.
• Add test configuration for multiple targets and environments.
• Publish a test summary directly in CI when appropriate.

👨‍💻 About the Developer

Manjunath G. L.
Mechatronics Engineering | Aspiring Embedded Software & Validation Engineer

Interested in embedded systems, software validation, Python-based test automation, connectivity, and reliable engineering workflows. This project is part of my practical learning portfolio focused on building reproducible tests and improving validation workflows.

Build. Validate. Automate. Improve.

_{Portfolio project demonstrating software-simulated hardware/software validation.}