An Imperative Aspect of System Architect Role at Scale: Designing for Testability

By Alex Yakyma.

The bigger the system, the harder it is to develop and maintain, but also the harder it is to test. What we see is that developers and testers of most of the larger organizations complaining that they can’t basically add a single unit- or functional test to their system, because in order to do so they would have to refactor it almost entirely. This is something very important to companies to think about and for system architects at program level to take as a serious action item. In this post we will consider all variety of aspects of how system design impacts its testability from economic to cultural impact on the organization and how to start improving to better right away.

The whole concept of Design for Testability (DFT) shouldn’t be new to us. It progressed in microelectronic hardware product design in the last 50-60 years. The reason for DFT is simple: if you want to be able to test your hardware (say, integrated circuit) during the design stage and then also after each piece is manufactured and even afterwards when it all functions on the client side – you have to adjust your system design for that testability. What’s fundamentally important is that you can’t design the system first and then simply add this magic “feature” of testability – instead it is deep non-functional requirement that permeates the entire System Under Test (SUT) in all possible aspects. Software systems require testing and being able to test as you evolve the system becomes imperative.

The Economic Sense of DFT. 

Agile testing covers different business perspectives: on one hand it critiques the product and thus minimizes the impact of defects delivered to the user; on the other hand it supports iterative development by providing fast feedback within the continuous integration process. None of these factors can come into a play if the system does not allow for simple system/component/unit- levels of testing. This fact obviously means that agile programs that sustain testability thru their every design decision will enable the enterprise with:

- Shorter runway for business epics 
- Shorter architectural runway 

In certain sense DFT reduces the impact of the big system and gives agile teams the luxury of working with something really manageable. That is why the role of a system architect is still important in agile at scale but it reflects totally different motivation: instead of defining big up-front system design (BDUF), agile architect helps establishing and sustaining effective product development flow. The WSJF prioritization model based on the economics of Cost of Delay easily shows this if we take into account that all jobs become shorter in the system that is designed for testability.

 Impact on the System Architecture. 

Practical knowledge suggests us that when we design for testability we actually arrive at a better design. Indeed typically DFT drives clear separation of concerns, layered architecture or service orientation, high cohesiveness of entities in the code etc. We so often forget that tests actually always behave like the “clients”: unit test imitates the behavior of a corresponding client-class or classes invoking the target class methods. Same way component tests imitate the behavior of the client-components. Finally system-level functional tests imitate the end user – the “client” for the entire system. This means that if we are allowing for clear testability at these all levels – we are actually allowing for clear and understandable interfaces between classes, components, services and finally user interface and rest of the system.

Cultural Impact.

Designing for Testability can in no way be just system architect’s sole responsibility. DFT only works well when backed by all agile teams in the program. DFT is not a destination but an endless journey that continuously supports product development and is rather role collaboration between developers and testers on one hand and system architect on the other. It creates the culture of shared responsibility over the product development flow where architect, developers and testers are actively contributing to the system design and the contribution is always respected as its outcomes pretty soon become visible by everyone. System design CoP (Community of Practice) is a good way of sharing knowledge between team representatives on how to maintain good design and system testability.

System Architect Role and DFT. 

When talking about designing for testability, here’s what system architect has to primarily take care of:
Choice of technologiesBesides fulfilling the main purpose, software libraries, frameworks, repositories and services should also support testability. For example, technologies that support inversion of control may not only be useful for designing a flexible system but also for testability.
The usage of a technologyFor instance having too much logic at DB side (common problem of many-many products) makes testing almost impossible. So does the unreasonable usage of asynchronous message queues within the system.
Design conventionsDesign patterns like fa├žade, gateway, or observer foster testability. Such a common thing like using web service proxy class directly does just the opposite and good convention could be to always use some “abstract” interface directly which would interact with the proxy and thus allow to substitute one with a mock object whenever necessary.
Approach for creating fake objectsWhat tools and ways to use to create stubs, mocks and “spies” in the code that would support unit- and component testing.
Logging and creating dumpsOften in large systems some system-level tests fail but all the unit tests pass and thus diagnosing the root cause of the problem may be near to impossible without a good logging approach and ability to retrieve thorough memory (or protocol) dumps.
Flexible configuration…that allows simple deployment to the test environment and easy linking of the test data sources and external mock objects as just a simple matter of updating the configuration files.
Table 1. The aspects of role of system architect to foster system testability. 

Where to Start? 

The above mentioned aspects may give the reader the immediate set of ideas that may spawn some architectural epics specific for their solution. In case of DFT as a driver for such epics, the incremental approach would work the best when instead of big redesign the teams should take the task piece by piece. This way not only we are always remaining in the context of our current features and stories that the teams are working on right now but also one of the most important aspects of DFT – the cultural aspect – can’t be achieved other than thru the set of incremental steps that demonstrate to the group that after every little design initiative it is easier to write the new tests at different levels of abstraction and actual team velocity gradually goes up.

Let the teams take few current user stories and see, what is the minimum redesign effort that would let them write few unit tests in the areas where they previously couldn’t. Then share the findings and actual approaches with the others (e.g. at the nearest Design CoP meet-up). See what aspects described above (in Table 1) these findings may affect – are there new design guidelines for all teams or changes to system configuration etc.?

We suggest using similar approach at the system level: let the system team or feature teams try automating system-level tests where they would previously fail but let them do it in the context of their current 1 or 2 features and so on.

When we are talking about agile testing and how important it is to start automating right now, we have to take into consideration that software systems are most often test-adverse. On one hand this is the sad fact about the industry on the other, when designing for testability, agile enterprise achieves great chance to outperform the competition as the runway for business and architectural epics becomes considerably shorter and the company can deliver much more value to their users.

8 comments:

  1. It sounds as though you have one of those great contractors who work on multiple jobs at the same time, instead of one at a time.
    finance addition nj

    ReplyDelete
  2. Elizabeth, in what sense? I'm not sure I follow.

    ReplyDelete
  3. You're also offered the normal skills essential that you should act as a on-line Architect Jobs Zjob4u. Eventually you had been found tips on how to implement as well as secure your web educating career.

    ReplyDelete
  4. Short message service (SMS) or text messaging is the way of today. People compress their ideas into 160 characters or less to make the message fit in one page.spell and grammar check

    ReplyDelete
  5. We have approached five of our resource council members to ask their views on the World Interiors theme “Design for All” and share their thoughts with you from a business perspective.asetti.com

    ReplyDelete