In the previous post, I asked the question ‘What is a large application?’ and I proposed two tables of measures, based on the number of lines of code (LOC) or the number of objects to categorize the applications as ‘simple’ , ‘medium’, ‘large’ and ‘very large’.

Of course, as I expected, discussions in forums about code analysis and software metrics immediately went toward Function Points, the definition of what is an application or the type of measure used by each participant in order to estimate the size of an application. The numbers that I had given were not discussed.

This week, another ‘quiz’ of the same kind: ‘What is a complex application?’ And ‘Is it possible to evaluate the QA effort depending on the complexity of application?’

We all know that McCabe’s Cyclomatic Complexity (CC) is the primary metric used to measure the complexity, and is commonly accepted without controversy as the LOC metric when it comes to estimate size. Remember the objective of this metric: counting the number of independent paths in the source code.

As a concrete example, let’s see the following rule: ‘if Order > 50, Discount = 1’%. This will result in the following pseudo-code:

IF Order > 50
  THEN Discount = 1%
ELSE
  Discount = 0%
ENDIF

The algorithm for this rule will therefore have two distinct paths corresponding to the two parts of the condition that controls if there is a discount or not. These two paths are equivalent to two points of Cyclomatic Complexity.

Now we can add new rules: ‘if Order > 500, Discount = 10%’, ‘if Order > 300, Discount = 5%’, ‘if Order > 200, Discount = 2%’.

The pseudo-code then becomes this:

 IF Order > 500
   THEN Discount = 10%
   ELSEIF Order > 300
     THEN Discount = 5%
     ELSEIF Order > 200
       THEN Discount = 2%
       ELSEIF Order > 50
         THEN Discount = 1%
         ELSE Discount = 0%
       ENDIF
     ENDIF
   ENDIF
 ENDIF

Note that I (willingly) chose the most complex and least correct ‘pesudo-code’ with several nested IF. But this is not the subject of this post.

In this example, we have four conditions and one case in which no condition is met. Or to say it differently, we added three additional rules to the previous case and our Cyclomatic Complexity is now higher.

All code analysis tools do not calculate CC in the same way, so the result may vary (plus or minus one point), but again this is not the subject of this post. Let’s just say we have a CC equal to 5, corresponding to five distinct paths in the algorithm, five different values ​​for the objects ‘Order’ or ‘Discount’.

Would you say that these different values must be tested in a QA phase?

If your answer is positive, then it means that the Cyclomatic Complexity is a good indicator of the number of test cases. Obviously, if you have an application with 10 000 points of CC, it does not mean that you have to test 10 000 different cases, or even that you application will have no default if you test all possible cases. But at least you can more accurately estimate the test effort, if the CC will be 2 000 points, 20 000 or 100 000 points.

The question of our previous ‘quiz’ was: ‘What is a big application?’. This week, the question is twofold:

• What is a complex application?
• How to estimate the effort of QA depending on the level of complexity?

Here is a table of measures that I propose:

Complexity of the application

Complexity	Low	Medium	High	Very high
CC points	6 000	16 000	50 000	> 50 000

Low complexity

With less than 6 000 paths in the application, a phase of unit tests done by the project team should be enough to check most cases, assuming that the projet team has a good functional knowledge of the application.

Generally for this kind of application, there are very little formal requirements, developers are in direct contact with users or stakeholders acting as an interface of the users.

Medium complexity

Between 6 00 and 16 000 points of CC, unit tests are required as before and become numerous enough for the project team to acquire a tool to automate them. Code size and especially the number of test cases increase significantly.

Knowledge of functional requirements is spread over various members of the project team: we begin to see developers specialize on some code or different functional modules. A specific stage of integration tests is necessary to verify that the assembly of different chains of components corresponds to functional business requests.

High complexity

Thresholds can vary between 16 000 and 20 000 points of CC for the lower limit and 50 000 and 60 000 CC points for the upper limit, depending of technologies or application architecture or the frequency of evolutions/corrections. There is obviously no specific limit, but the closer you get to a threshold and the more the corresponding QA strategy becomes applicable.

Beyond 16 000 points of CC (especially beyond 20 000), the application is complex. The size of the project team can become important, especially if corrections or changes are frequent.

In addition to the unit tests and integration tests at the project team level, it is essential to perform a specific phase of functional testing with a dedicated QA team. The formalization of test cases based on requirements documentation is increasingly necessary as the complexity approaches the upper limit.

Very high complexity

Above 50 000 or 60 000 points, the application is very complex. It is necessary to use tools to automate unit testing, integration or QA tests. Functional requirements must always be defined and ‘frozen’ before the development phase begins.

I met few applications beyond 100 000 CC points, even in the mainframe world. There are few situations with business rules as complex. I usually found them in areas where administrative regulation changes very frequently, as for example calculation of family allowances, with new rules adding to old rules rarely removed.

Most of the time, these monsters indicate technical debt drifts when management has let the application become more and more complex. I found them in areas that did correspond to fast growing market, justifying frequent application changes but currently in a phase of market stabilization or decline, resulting in a reduction of IT budgets and making it impossible to reduce this debt by refactoring. The interests of the debt remain high for a long time.

And you, do you think the measure of complexity can be an indicator of the QA effort?

 

This post is also available in Leer este articulo en castellano and Lire cet article en français.