Complex interactions in hardware-dependent systems

• Testing such systems is challenging because they involve intricate interactions among multiple hardware components (system under test, accessories, measurement equipment).

• This complexity requires diverse hardware configurations and unique setup procedures, making testing cumbersome.

Test setup phase as a bottleneck

• Testing time is divided into setup and execution phases.

• The setup phase consumes disproportionate time and manpower, delaying fault detection.

• Sometimes, setup time even exceeds execution time.

Limited access to source code information

• Testers usually do not have access to source code changes in hardware-dependent systems.

• This makes code-coverage-based prioritization techniques inapplicable.

• Combined with time/manpower constraints, this results in incomplete testing cycles and higher risk of defects escaping to production.

Traditional TCP metrics are insufficient

• Current TCP approaches mainly optimize based on execution time or fault detection rate.

• They fail to account for setup time and reconfiguration costs, which are critical in hardware-dependent systems.

Need for cost-aware prioritization

• A test case with high fault detection potential but expensive/unique setup may actually be less efficient than one with slightly lower fault detection potential but lower setup cost.

• Thus, there is a need to shift focus from purely fault detection rate to cost-effective fault detection under resource constraints.

Grouping-based prioritization requirement

• Efficiency can be improved by prioritizing groups of tests that share the same or similar hardware setup, thereby reducing frequent reconfiguration.

• This grouping strategy is currently underexplored in existing TCP research.

Test setup phase as a bottleneck

• Setup time often exceeds execution time, consuming disproportionate resources and delaying fault detection.

• Why? → This directly links to Objective (a) (analyzing existing TCP techniques and their inability to handle setup/switching costs).

Traditional TCP metrics are insufficient

• Current methods optimize only for execution time or fault detection rate, ignoring setup and reconfiguration costs.

• Why? → This motivates Objective (b) (designing a new cost-aware TCP algorithm).

Need for cost-aware grouping strategies

• High-priority test cases with expensive unique setups may not be efficient; grouping tests that share hardware configurations reduces reconfiguration overhead.

• Why? → This ties into Objective (b) and Objective (c) (developing and validating an approach that balances cost and detection efficiency).