Today we continue our evaluation of the quality of Cobol code with Sonar.

In the previous post, we worked with the metrics measuring code size, complexity, level of documentation and duplication, which allowed us to formulate some initial recommendations to the people responsible for this application.

Blockers and Critical

Let’s have a look at the violations now. Remember that we have oriented our Quality model towards the identification of defects in robustness and performance, impacting users.

The number of ‘Blockers’ defects is low. We find again the ‘SORT’ that we talked about earlier, and 5 ‘OPEN’ file in a loop. This treatment is expensive for the operating system and of course must be done before the loop. These violations put a high risk on performance, but are easy to fix.

Critical defects are on the reliability:

• A file status code is not tested during an operation on it (OPEN, READ, WRITE, …). An attempt to write to a file not open, read in an output file open for writing and not for reading (should be an input file), an incorrect record size are errors that will go unnoticed.
• GOTO that transfer the treatment outside of the current module breaks the logic of execution.
• The EVALUATE without WHEN OTHER is classical: if no conditions of the EVALUATE is met, the program logic becomes unpredictable.
• The final rule concerning the file size only applies to Microfocus Cobol, not z/OS (IBM mainframe).

It is very likely that there is a quality standard requiring the correct management of the file status but that this best practice is sometimes forgotten. Without testing this code, incorrect operations may conduct to an inconsistency of data without it being detected. The search for the cause of the error will also be more difficult. The same with the EVALUATE without WHEN OTHER.

Any violation of these rules impact not only users but also maintenance costs. Here, their numbers leads us to believe that these good practices should be rememberd to the developers.

Remediation costs

Do you remember the widget we used in our last post, that listed the most frequently violated rules?

Again we find IF without END-IF which we have already spoken of, harmful GOTO, more than 4,500 undocumented paragraphs and copy-pasted code. These rules impact the maintainability of the application, as shown in this diagram:

We will not detail all of these violations since our objective is to identify those that impact end users. The 45 ‘Blockers’ and the 292 ‘Critical’ violations respectively represent 5.6 days and 50.5 days of correction.

Approximately two weeks of work for a team of 5 or 6 people. On the other hand, it would take more than 10 man years to correct violations that affect costs maintenance.

Action plan

Of course, we will present these results to the project team and talk with them in order to verify our hypothesis. However, our objective for this post is not to perform an exhaustive audit but to show how it is possible to draw some conclusions based on these few data, and formulate a draft of an action plan.

Short term actions:

• Correct the defects ‘Blockers’ that affect performance: 5 days.
• Correct the defects ‘Critical’, a potential source of bugs: 55 days.
• Remind the Cobol teams not to use GOTOs and to respect best practices for error handling, such as the obligation to check the status of a file after any operation on it and avoid EVALUATE without WHEN OTHER.
• Conduct regular analysis of this application and establish a zero tolerance practive on blocking or critical violations.

These actions are inexpensive and can be done at short term, for the maximum benefit. They lead to the establishment of a Continuous Improvement process. Users will thank you.

This application has high maintenance costs. Check with management if it is indeed the case and their opinion on it. It may well be that this application has little or no evolution in which case, maintenance is not a problem.

Also check the assumptions we made ​​in our previous post (batch application oriented towards data files, not critical) and that it cannot be safely outsourced.

Submit the following action plan:

• Identify and remove the 184 duplicate files, which weigh heavily on maintainability.
• Perform again an analysis to recalculate the technical debt and the estimated cost of refactoring.

Application strategy

Refactoring a Cobol application is an operation rarely considered, because the costs are too high. Generally, the application is allowed to die a natural death, and the management will look for another solution (ERP for instance).

As we wish to make realistic recommendations, here’s what I do: I look at what are the programs that have the greatest number of lines and the most defects.

Here we have three programs that account for more than 27 000 lines, 15% of the total size of the application (LOC).

 

They are also in the top five of the most complex programs, with a total of 5,900 points of Cyclomatic Complexity, or 26% of the total complexity of the application.

And they are the three programs that concentrate the most violations, a few Critical but mostly Major.

More than 6,700 violations so nearly a quarter (24%) of the total number of defects encountered in this application.

 

Sure, the cost of remediation of these three programs is very high: 735 days, or about 3 man-years.

What information can we deliver?

A partial refactoring on these three programs representing 15% of the size of the application and 26% of its complexity, would eliminate a quarter of the total number of defects, almost exclusively on maintainability.

If we imagine that this refactoring would reduce the cost of maintainability in the same proportion (25%), this represents a gain of 1.5 person-year for a team of 6 persons. Against a cost of refactoring of 3 man-years, this means a return on investment within 2 years.

Certainly this is a quick estimate. But I want to illustrate that the value of an assessment is to deliver objective information to allow management to make decisions. The overall technical debt is so high that it leads to only one possible decision: throw this application. An estimate of gains / costs and ROI on a partial refactoring allow us to imagine an alternative strategy. Even if your numbers are not precise and assuming a certain margin of error, the management will realize what is your intention and appreciate it.

Sell the value

This post concludes our series on the analysis of Cobol code with Sonar. Our intention in this series was to demonstrate that it is possible to assess the quality of Cobol applications without being an expert in the Mainframe world. Similarly, no need to be an Open Source guru or a J2EE expert to conduct this analysis with Sonar.

With minimal knowledge of Cobol and Sonar technology, we can provide project teams, stakeholders and management an action plan that improves performance and robustness of the application, but also compliance with good practices and quality of the application.

We can also provide the management substantial value: information for better decision making. Sell ​​this value, no metrics or your technical knowledge.

 

This post is also available in Leer este articulo en castellano and Lire cet article en français.