Avionics software development requires different levels of code coverage based on the assurance level of the software, which is based on what kind of aircraft failure condition would result from a software anomaly. For Level A software (catastrophic failure),
the required code coverage includes modified condition/decision coverage (MC/DC). Here are several good resources explaining various aspects of this type of code coverage:

NASA – A Practical Tutorial on Modified Condition/Decision Coverage
FAA/CAST – What is a “Decision” in Application of Modified Condition/Decision
Coverage (MC/DC) and Decision Coverage (DC)?
FAA – An Investigation of Three Forms of the Modified Condition Decision Coverage (MCDC) Criterion

Recently, I spent a week at a simulator facility supporting checkout of software changes. During the week, I created several prototype changes for new features. I had a number of issues, and decided to share some data on those issues.

1. Editing problems
I was asked to swap a parameter used in a calculation for another parameter. I was editing source code in a Subversion branch checkout. However, one of the files I had opened was from a trunk checkout of the same software baseline. This file had the main declaration of the new parameter. As a result, I was getting an “undefined parameter” linking error. I thought I had confirmed that all of the files I was using was in the branch checkout directory, but that was not the case. I had to get two of my peers to take a look at what I had done, one of them was able to successfully compile the code changes. That gave me the idea to recheck the location of the files, and I then saw the problem.

2. Compile errors – part 1
My next prototype was a significant modification of two functions. I had a set of redlined requirements, and decided to do a pair programming exercise with the flight control engineer who developed the requirement changes. We reviewed the code changes as we went, then started compiling and removing errors. Here is what I had:
a. Forgot to update function prototypes in include file
b. Two LOC had semicolons when they should have had no semicolons (function initial statements)
c. One incorrect variable declaration
It took about 5 minutes to correct those problems.

3. Compile errors – part 2
I got a set of draft requirements for a new algorithm from an engineer. The requirements were in a Word document, and included some pseudocode and actual code snippets. I created the initial prototype by myself, then reviewed it with the engineer. After our review, I then started compiling and removing errors. I cannot remember the early errors, but there was one that persisted. The compiler was telling me that there was a problem with a character on two line, but did not directly identify the character. The problem was when I copy/pasted the code snippets, the minus sign in two equations was not a dash, but some other character. I deleted that text and retyped the minus signs, and that corrected the final problem.

Summary
1. When in a hurry, take time to review your changes.
2. When copy/pasting from a Word document, be sure to check equation text.
3. Pair programming can be a useful development method.

 

Returning to the topic of metrics, Watts Humphrey discusses concrete ways of measuring software quality, and how to overcome
issues with data collection. From his introduction:

“There are four reasons why the software community has been slow to use numbers for software quality.
1. There is no generally recognized definition for quality measures.
2. Even when we know what numbers to use, the data are not easy to gather.
3. Even with data, it is not obvious how to interpret and use the numbers.
4. People are reluctant to measure the quality of their personal work.
This paper addresses these questions and gives examples of software quality data that can readily be gathered and used by properly trained engineers.”

Links:

• Software Engineering Institute – The Software Quality Profile

 

Jim Bird has a post discussing software metrics (the good, the bad, the ugly). He advocates doing what makes sense for your organization:

“Only measure things to identify potential problems and negative trends, strong points and weak points in the code and your development process. If you want to set goals for the team to improve speed or cost of quality, let them decide what to measure and how to do it. And measure whatever you need to build a business case – to prove to others that the team is moving forward and doing a good job, or to make a case for change. Make metrics work for the team. You can do all of this without adding much to the cost of development, and without being evil.”

My perspective on metrics is as follows:
1. You can’t know how you are doing unless you collect some data and analyze it.
2. Only collect data on things you are interested in or want to doing something about.
3. If you aren’t going to do anything with the data, don’t collect it. (Do I care about the data? If no, then don’t collect it.)

Links:

• Software Reflections – Software Development Metrics that Matter

 

I am working on metrics development for my software development group. To aid in this project, I have looked for tools to provide consistent counting of
source code (SLOC). The Understand tool provides this capability (commercial tool). There are also several open source tools available:

• University of Southern California – The Center for Systems and Software Engineering has Unified Code Count (UCC), a tool for counting SLOC for multiple languages
• Vimeo – Presentation by Anandi Hira on UCC setup and use
• Jet Propulsion Laboratory – Code counter for C

Capers Jones has published an evaluation of several software size metrics (function points, SLOC, other methods), and discusses their advanatages and disadvantages.

The software development environments of today are experiencing an explosion of new development methodologies (Agile, Scrum, etc.). For a look back to see at one of the first documented development processes, this book provides insight into the development and verification practices used for development of the Space Shuttle flight software.

Links:

• National Academies Press – Chapter 3 of the book “Assessment of Space Shuttle Flight Software Development Practices” is an overview of the complete flight software development and upgrade process.

With the increase in web applications where data security is a requirement, several groups have tried to address best practices for developing software that meets security requirements. This article is a good collection of these best practices.

Links:

• Software Reflections – Top 10 Lists for Designing and Writing Secure and Safe Software