Testing in Scrum

1. Agile Shifts from Deterministic Planning to Adaptive Control

The main aim of using Scrum is to enable a project team to react quickly, simply and appropriately instead of wasting time and energy creating, implementing and updating outdated plans.

Adaptive approach. Agile methodologies like Scrum and Kanban fundamentally shift project management from rigid, upfront planning to an adaptive, empirical system. This allows teams to respond swiftly to changes, rather than being constrained by outdated, deterministic plans that often fail to reflect real-world project dynamics. The focus is on continuous adjustment and learning.

Iterative cycles. Scrum divides projects into short, fixed-length iterations called Sprints, each delivering a "potentially shippable product increment." This contrasts sharply with traditional V-Model phases, where development progresses sequentially through distinct stages like design, implementation, and testing, often leading to delayed feedback and costly rework. Agile embraces simultaneous development across all abstraction levels.

Transparency and focus. Key Scrum instruments like the Product Backlog, Sprint Backlog, and Definition of Done (DoD) ensure transparency and focus. The Product Backlog, maintained by the Product Owner, prioritizes requirements, while the Sprint Backlog represents the team's commitment for a specific iteration. The DoD ensures that work is truly complete, including testing, before an increment is considered shippable.

2. Lean Planning Provides Direction Without Rigidity

Even in a Scrum environment, reactive work practices alone will not result in a successful project.

Strategic alignment. While agile emphasizes flexibility, it doesn't mean working aimlessly. Scrum projects require clear objectives beyond the current Sprint, guided by a concise Product Vision and an Architecture Vision. These high-level documents provide strategic direction, helping to prioritize the Product Backlog and ensure that incremental development aligns with the overall product goals.

Dynamic backlogs. The Product Backlog is a living document, constantly refined and reprioritized based on new insights and customer feedback. The Story Map visually outlines upcoming Sprint goals, offering a flexible roadmap rather than a fixed commitment. This adaptive planning allows for immediate incorporation of valuable ideas, ensuring the product remains relevant to customer needs.

Team commitment. The Sprint Backlog, created during Sprint Planning, represents the team's commitment to deliver specific tasks within a timebox. This commitment is based on collective effort estimation, often using techniques like Planning Poker, ensuring realistic goals. Unlike traditional plans, the Sprint Backlog is protected from changes during the Sprint, fostering focus and accountability.

3. Test First Drives Quality from the Developer's Perspective

Before you alter any code, write an automated test that fails.

Paradigm shift. Test First, a core Extreme Programming (XP) practice, inverts the traditional "program first, then test" approach. Developers write automated tests that define new functionality before writing the actual product code. These tests initially fail, and the coding task is considered "done" only when all tests pass, ensuring immediate validation of new features.

Enhanced quality. This test-driven development (TDD) approach significantly improves code quality and design.

• Objective feedback: Tests provide objective proof of progress and act as clear "done" criteria.
• Executable specifications: Test cases serve as precise, machine-readable functional specifications, bridging the gap between high-level requirements and code.
• Improved APIs: Writing tests first forces developers to design clean, lean, and application-oriented public interfaces (APIs).
• Better testability: Code is inherently designed to be testable, as it must pass the pre-written tests.

Team discipline. Implementing Test First requires sustained discipline and a shift in mindset for programmers and testers. Pair Programming, where a programmer and tester collaborate, facilitates this transition by fostering knowledge transfer and ensuring tests are comprehensive and user-focused. The Scrum Master plays a crucial role in enforcing this practice and providing necessary training.

4. Continuous Integration Automates Quality Feedback Loops

Checking code into the repository automatically triggers an integration process that takes place on a dedicated CI server.

Automated integration. Continuous Integration (CI) is a critical agile practice that automates the process of integrating code changes frequently into a central repository. Every code check-in triggers an automated build and test sequence on a dedicated CI server, providing rapid feedback to developers. This prevents "Water-Scrum-Fall" by eliminating lengthy, end-of-Sprint integration phases.

CI process steps: A typical CI run involves several automated steps:

• Compilation: Code is compiled, and warnings/errors are logged.
• Static code analysis: Checks adherence to coding guidelines and quality metrics.
• Deployment: Code is deployed to a clean test environment.
• Initialization: Test data and database setups are automated.
• Automated testing: Unit, integration, and often system tests are executed.

Rapid feedback. The CI server displays results on a dashboard, providing immediate feedback on the system's quality. This rapid feedback loop, often taking minutes, allows developers to identify and fix defects quickly, minimizing the cost and effort of bug resolution. Optimizing CI involves creating test batches, parallelizing tests, and upgrading hardware to maintain speed.

5. System Testing Validates the User Experience End-to-End

System tests check that the system works from the user’s point of view using the customer’s own interfaces.

User-centric validation. System tests are essential for verifying that the entire product functions correctly from the end-user's perspective, often in an environment that closely mimics the customer's real-world setup. These "end-to-end" tests validate the complete functional chain, from the user interface through all system components to external devices, ensuring the product meets its intended use.

Complex environments. System testing environments are typically more complex and costly than unit or integration test setups, requiring realistic simulations or actual external components like hardware, networks, and other systems. Careful planning and continuous maintenance of these environments are crucial to ensure test results are meaningful and reliable, reflecting diverse user configurations.

Automation strategies. While manual exploratory and session-based testing are valuable for quickly checking new features and uncovering "smells," automated system testing is vital for continuous feedback. Keyword-driven testing and Behavior-Driven Testing (BDT) use domain-specific languages (DSLs) to create abstract, human-readable test cases that are independent of the GUI's technical implementation, enabling Test First principles at the system level.

6. Agile Quality Management Fosters Continuous Improvement

A QM system designed to support the principles of the Agile Manifesto and find favor with agile teams has to be lean.

Bottom-up quality. Unlike traditional, top-down Quality Management (QM) systems based on extensive documentation (e.g., ISO 9000), agile QM is a lean, bottom-up process. It emphasizes "Inspection and Adaptation," where the entire cross-functional team is responsible for product quality. Documentation is minimized, focusing on concise rules and practices that serve the team's immediate needs.

Integrated routines. QA is embedded into daily agile routines:

• Sprint Planning: Acceptance criteria define quality goals, and necessary checks (manual or automated) are planned as tasks.
• Daily Scrum: Progress on tasks, including fulfillment of "done" criteria and test results, is reviewed daily.
• Sprint Review: The team presents the increment to stakeholders, gathering immediate customer feedback for validation.
• Sprint Retrospective: The team discusses process improvements, making quality enhancement an ongoing, self-directed activity.

Cultural shift. This approach transforms the QM team from a central authority into a service-oriented group that supports and mentors agile teams. It fosters a culture of continuous learning and improvement, where process optimization is initiated by development teams and integrated into their daily workflow, ensuring that improvements are practical and immediately beneficial.

7. Compliance and Traceability are Achievable in Agile

Leading FDA consultants nowadays confirm that agile practices are capable of fulfilling the FDA’s requirements better than waterfall development techniques.

Defined processes. Agile methodologies, while flexible, can meet external compliance requirements (e.g., ISO 9001, IEC 61508-3, FDA). Standards require well-defined software development processes, which agile teams document through concise process descriptions and their Team Charters. Specific regulatory requirements for development activities are integrated into the "Definition of Done" criteria for tasks.

Ensuring traceability. Traceability, crucial for safety-related products, links requirements to specifications, code, and test results. In agile, this means connecting Product Backlog items to system test cases, proving requirement fulfillment. While challenging with constant change, reliable traceability is achieved through appropriately linked requirement and test management tools, ensuring every change is reproducible and its impact analyzed.

Tool validation. Compliance extends to the tools used in the engineering process. For industries like medical technology, even third-party software and specific development tools may require formal validation. Agile teams must be aware of these requirements and either provide dedicated product features or support services to help customers meet their validation needs, sometimes necessitating longer release cycles.

8. The Tester's Role Evolves to a Cross-Functional Expert

The close cooperation with the developers and the increasing slant of the testers’ work toward programming sometimes creates a developer mindset among testers.

Integrated expertise. In agile, the traditional distinction between developers and testers dissolves. Testers become integral members of cross-functional Scrum Teams, contributing their specialized skills to all phases of development, from defining acceptance criteria to automating tests. This fosters a shared responsibility for quality across the entire team.

Skill expansion. Testers in agile teams need to expand their skill sets beyond manual testing to include:

• Test automation: Proficiency in xUnit frameworks, GUI test tools, and scripting languages.
• Test design: Expertise in techniques like equivalence partitioning, boundary value analysis, and state-based testing.
• Programming basics: To effectively collaborate on test automation and code analysis.
• Exploratory testing: For rapid, intuitive testing of new features.

Maintaining objectivity. While close collaboration is beneficial, a risk exists that testers may adopt a "developer mindset," potentially reducing their critical assessment of the product. To counteract this, the team needs to ensure dedicated testing expertise, possibly through a designated (though non-authoritarian) test manager role, regular test code reviews, and fostering an objective, user-centric perspective.