Testing All the Way Around the DevOps Loop

February 22, 2024 by Amanda

Discussion Topic: Testing

The DORA Community Discussions start with a brief introduction talk, and then a discussion with the attendees using the Lean coffee.

Topics discussed:

How the DORA metrics can contribute with quality?

Recap from the community discussion:

• DORA metrics focus on process. Lead time, deployment frequency, etc. primarily reveal process efficiency, not product quality.
• DORA metrics can indirectly influence quality. A smooth process facilitates the focus needed to improve product quality.
• Look for product-oriented metrics. Consider deployment failure rate to give a high-level picture of quality or supplement DORA with quality-specific metrics.

What about teams that do not have any testing specialists?

Recap from the community discussion:

• Emphasizes the importance of focusing on the outcomes that the team is trying to achieve, rather than just the tasks that need to be completed.
• Continuous learning is key. Teams should invest time in understanding different testing techniques to compensate for a lack of specialists.

When testing is everywhere, what is the role of specialized testers?

Recap from the community discussion:

• Testers become quality coaches. They focus on teaching, mentoring, and improving the team's overall quality mindset.
• Specialists bring new perspective. Their focus on strategy and the "big picture" of testing elevates the whole team.
• Testers can unblock teams. They help with tasks outside of direct testing, bringing different strengths to the team.

How can we improve quality through observability?

Recap from the community discussion:

• Observability tools can assist in identifying and diagnosing production issues, enabling teams to learn from and prevent future problems.
• The use of metrics and tracking to identify areas for improvement was suggested.

How does non-functional testing fit in (scale, security, DR)? Does it get done with every iteration?

Recap from the community discussion:

• Quality attributes mindset. Avoid the vague term "non-functional," focus on specific attributes like reliability and scalability.
• Use visual models for planning, ...like the Holistic Testing Model, to ensure you don't miss important areas.
• Make time for it. Consciously plan for testing non-functional aspects as early as possible. Leverage new tools. CI pipelines and production testing tools make this easier.
• Treat operational requirements like functional features. Prioritize them on the backlog, with dedicated time and resources for testing.

There's an adage along the lines of - when everyone is responsible for something, no one is. How does that translate to everyone in a cross-functional team being responsible for quality?

Recap from the community discussion:

• "Everyone's responsible" can fail. Without clear ownership, testing may get deprioritized.
• Visual models help. Tools like the Agile Testing Quadrants provide a framework to discuss and plan testing coverage.
• Assign clear responsibility. Make testing an explicit task within a development process, like code review. Someone needs to be accountable for quality.
• Consider an enabling team. Some organizations have quality-focused teams that support and set standards for other teams (rather than individual testers within teams).