What is Code coverage? Definition, Types & Best Practices

What is code coverage?

Code coverage is a software testing metric that measures the percentage of your source code executed during automated tests. It helps developers identify untested parts of their codebase and improve overall software quality. For example, if your tests execute 90% of your code, you will have 90% code coverage.

But what does it mean for code to be executed in a test? How rigorous does a test need to be to qualify as having executed the entirety of a program? And does 100% test coverage mean that a program is bug-free? Let’s understand that in detail in this article.

Types of code coverage

Different types of code coverage help you understand how thoroughly your tests interact with your code. Each type focuses on a specific aspect of execution, from individual lines to complex decision paths. Here’s an overview of the most common coverage types:

1. Line coverage: Measures executed lines of code
2. Function coverage: Tracks called functions
3. Branch coverage: Covers decision points
4. Statement coverage: Counts executed statements

How to calculate code coverage?

Measuring code coverage is conducted using one or more criteria, depending on the organization’s standards for code coverage. Though this is not an exhaustive list, some key criteria to use when measuring code coverage include:

• Function Coverage: Has each function been called?

• Statement Coverage: Has each statement been executed?

• Path Coverage: Has every edge in the control-flow graph been executed?

• Condition Coverage: Has each boolean sub-expression been evaluated as true and false?

• Line Coverage: Has the test suite executed all the individual lines of source code?

Let’s see an example of how these criteria can determine code coverage. Here we have a fundamental function that returns the sum of two arguments, if two arguments have been provided, and returns null otherwise:

function numSum(x, y) { 
  if (x && y) { 
    return (x + y); 
  } else { 
    return null; 
  } 
} 

Copy to clipboard
Copy to clipboard

The most basic test of this function would include calling it at least once, achieving 100% function coverage.

numSum(1, 2);  // This should return 3 

Copy to clipboard
Copy to clipboard

By executing the function only once and with the arguments above, we will execute some, but not all, of the lines of code in the function. So, we can test our function by calling it with arguments that will return null:

numSum(1, 2);  // This should return 3 
numSum(1, false);  // This should return null 

Copy to clipboard
Copy to clipboard

With these two calls together, both paths in the if / else blocks are taken, and both return statements are executed. Thus, we have 100% statement coverage and 100% path coverage.

Finally, to achieve 100% condition coverage, we need to call our function with arguments such that x and y each evaluate to true and false in the function’s if condition statement.

numSum(1, 2);      // This should return 3 
numSum(1, false);  // This should return null 
numSum(false, 1);  // This should return null 
numSum();          // This should return null 

Copy to clipboard
Copy to clipboard

The 100% code coverage myth

And there we have it, we’ve achieved 100% coverage in each of the four criteria! So, we’re done testing, right? Well, not quite.

Having 100% code coverage does not guarantee bug-free code – it simply validates the completeness of our tests using a given set of criteria relative to other test suites with lower code coverage. We still must be vigilant about writing robust test suites that specifically address both the intended use cases, and unintended edge cases, of our application.

For example, as written, the numSum function will concatenate numeric string arguments rather than performing addition.

numSum('1', '2') // This will return '12' instead of 3 

Copy to clipboard
Copy to clipboard

While there are other coverage criteria we can use to identify the full range of function parameters, code coverage is an insufficient metric to determine if our tests confirm the intended functionality of our program as defined by the end user’s requirements.

Instead, we should consider the code coverage criteria as a set of guidelines to help you develop intuition for testing your code while remaining determined to write robust test suites that are specific in targeting the various use cases of our programs.

What is test coverage?

Another term you may hear with regards to testing is called test coverage. Test coverage differs from code coverage in that test coverage measures the percentage of the required features/specs that are tested, as opposed to the percentage of lines executed. These features/specs are typically defined in a requirements document provided by a client or product designer.

Suppose you are building a mobile-native application that needs to work on phones using the Android and iOS operating systems, but is not expected to work on desktop browsers. Accordingly, to achieve high test coverage, you will be expected to write tests for your application’s performance on Android and iOS but not on browsers.

Like code coverage, test coverage cannot guarantee that our program will not have unexpected behavior. However, the pursuit of high test coverage can lead us to identify bugs and unexpected behavior earlier on in the development process.

Code coverage vs. Test coverage: Key differences

Although these terms are often used interchangeably, code coverage and test coverage measure different aspects of software testing. The table here breaks down the key differences between the two:

In short, code coverage is about how much code runs, while test coverage is about how much functionality is tested. Both metrics together provide a more complete picture of your application’s test health.

Conclusion

Code coverage and test coverage are essential metrics for evaluating the effectiveness of your testing strategy. While code coverage helps ensure that different parts of the source code are exercised during tests, test coverage ensures that your application’s features and requirements are being validated. Together, they provide a more complete view of software quality. Still, even 100% in either metric doesn’t mean your code is flawless, as thoughtfully designed tests are what truly make a difference.

To explore these concepts further and get hands-on practice with writing tests, check out Codecademy’s Learn Testing for Web Development course.

Frequently asked questions

1. Why use code coverage?

Code coverage shows which parts of your code are tested and which aren’t. It helps improve test quality, catch bugs early, and ensures critical code paths aren’t missed.

2. What is 100% code coverage?

100% code coverage means that every line, statement, or branch of your source code has been executed at least once during testing. However, it doesn’t guarantee that your code is bug-free or fully validated, it only confirms that tests have touched all code paths.

3. How do you calculate code coverage?

Formula to Calculate Code Coverage:

Code Coverage (%) = (Number of lines executed by tests / Total number of lines of code) × 100

4. How to improve code coverage?

To improve code coverage:

• Write additional tests to cover untested branches or conditions.
• Use tools like coverage.py, Istanbul, or JaCoCo to identify gaps.
• Refactor complex logic into smaller, testable units.
• Include edge cases and unexpected inputs in your test cases.