Requirements High-level design Detailed design Coding Unit
Acceptance System Integration
Scope and definitions Requirements based testing Functional and non-functional aspects Management issues
Integration testing (R)
• Process of examining how the pieces of a system (Modules) work together, especially at the interfaces • Tests to ensure that the various components of a system – Interact correctly – Pass data correctly – Function cohesively
System testing - scope
• System testing activities assume that module and integration testing have been done – i.e. single program functionality has been comprehensively tested
• System-level testing focuses on the testing of
– – – – – – Multiprogram functionality External interfaces Security Recovery Performance Operator and user procedures
System testing - definitions
System testing – definitions (cont)
The process of testing of an integrated hardware and software system to verify that the system meets its specified requirements. Verification: Confirmation by examination and provisions of objective evidence that specified requirements have been fulfilled. IEEE definitions
• Testing to confirm that all code modules work as specified, and that the system as a whole performs adequately on the platform on which it will be deployed. • Testing conducted on a complete, integrated system to evaluate compliance with specified requirements.
System Testing vs Integration Testing
Integration testing From interface specification Visibility of the integration structure Some scaffolding required Attention to interfaces between the modules System testing From requirements specification No visibility of code No drivers / stubs required Attention to system functionality • •
Key features of the Software Requirements Specification (IEEE) Functionality
– What is the software supposed to do? Functional
– How does the software interact with people, the system’s hardware, other hardware, other software?
– What is the speed, availability, response time, recovery time, etc. of various software functions? Nonfunctional
– What are the portability, correctness, maintainability, security, etc. considerations?
Design constraints imposed on implementation
– Are there any required standards in effect, implementation language, policies for DB integrity, resource limits, operating environment etc?
• Non-functional aspects
– – – – – – – – – – Load Stress Performance Volume Security Usability Storage Installability Documentation Recovery •
Subjecting a system to a statistically representative (usual) load The load – an incoming transaction stream
Aimed at determining various characteristics of the application working under a massive workload Consists of simulating real-life workload conditions for the application under test Test is normally performed several times with a different number of users working concurrently to find out how various parts of the application react to the increasing load
• Subjecting a system to an unreasonable load while denying it the resources (e.g., RAM, disc, mips, interrupts, etc.) needed to process that load Aimed at determining the load (for example, the number of user requests) that causes a crash or an unacceptable performance of the application under test • •
Stress testing (cont)
A system is stressed to the breaking point in order to find bugs that will make that break potentially harmful The system is not expected to process the overload without adequate resources, but to behave (e.g., fail) in a decent manner (e.g., not corrupting or losing data) Bugs and failure modes discovered under stress testing may or may not be repaired depending on the application, the failure mode, consequences, etc.