What Is Software Testing?

Software testing, which is the process of evaluating a program to ensure it meets requirements, is often misunderstood. Is it just a distraction from creating new features? A drag on productivity? Or maybe an unnecessary step since everything seems to work fine on your local machine? Imagine this scenario: You’re an engineer who has spent years designing and building the control software for a next-generation rocket. On launch day, everything goes according to plan until, a few seconds into its flight, the rocket veers off course and explodes due to a simple case of integer overflow. This is what happened to the maiden flight of the Ariane 5 rocket, and it’s just one example of how catastrophic software bugs can be.

The Importance of Software Testing

Software errors have consequences. In fact, bugs have led to radiation therapy machines producing incorrect dosages, trading algorithms losing millions of dollars, and even a Mars satellite being lost due to incorrect unit conversions. These real-world events highlight the critical role of software testing in identifying potential bugs before they cause harm. In their daily job, engineers want to write code that works—and continues to work reliably over time. Software testing ensures that important decisions based on your software are sound.

Testing early in the development process can significantly reduce costs: The cost of fixing a bug increases as it moves through the development lifecycle. Catching issues early not only saves time and money but also minimizes the risk of project delays and ensures a smoother development process. By integrating software testing into the early stages of development, teams can detect and address potential problems before they escalate, leading to more robust and reliable software.

Manual and Automated Software Testing

If you’ve ever written code, you’ve tested it, even if you didn’t realize it. Software testing is the process of evaluating a software application to identify bugs, errors, and missing requirements. Running a script to see if it produces the correct result is a form of manual testing. However, manual testing requires testers to remember the procedures and assess whether the result is correct, which is dependent on that person’s individual knowledge of the application. If that person leaves the company, the knowledge they had—and the ability to test the code—leaves with them.

Modern software testing has evolved from manual to automated, providing a systematic way to evaluate software. Documenting the expected behavior and testing steps transfers critical knowledge from individuals into the testing code, which broadens the tester base, leading to more consistent and reliable code. Automated software testing enables you to compare actual system behavior with expected outcomes, highlighting discrepancies that may need attention (or tests that need updating).

Goals of Software Testing

• Detect bugs early: Identifying bugs early in the development process is crucial, as the cost to fix them increases exponentially as the project progresses. Bugs that reach deployment can lead to financial loss and reputational damage.
• Uncover reliability and performance issues: It’s essential to test code under all conditions, including edge cases. Performance issues, such as slow response times, can be as problematic as incorrect results.

Types of Software Tests

There are many types of software tests, and it can be difficult to know where to start. Here are several essential types of software tests every engineer needs to know:

• Unit testing: Focuses on verifying the functionality of individual components or units of code in isolation. Unit tests help isolate and identify bugs at an early stage and can be relatively fast and easy to write and run.
• Integration testing: Verifies the interfaces between code units to ensure they interact as expected. It detects issues such as data format inconsistencies and code interface incompatibilities.
• System testing: Involves testing the entire integrated system to ensure it meets specified requirements. It simulates real-world scenarios to verify that the software behaves correctly under expected conditions.
• Smoke testing: Verifies that the most critical features work and that the software is ready for additional testing. It’s a fast way to check the basic functionality of software. 
• Regression testing: Checks if changes to the code cause any unexpected changes in behavior. It ensures that the software continues to function as intended after modifications.
• Performance testing: Evaluates the software’s performance under various conditions. It measures criteria such as response times and resource usage to identify bottlenecks and optimize performance.

Mocking, though not a type of testing on its own, is a mechanism to test components in isolation by simulating dependencies. It helps isolate and test specific parts of the system without relying on actual components.

These types of tests can be combined to suit the specific needs of a project. For example, you could have a unit test that uses mocking, or an integration test that’s a regression test.

Example: Unit Testing with MATLAB

The most basic test we can perform is a manual test, which we can run at the command line and test whether the output works as expected. However, manual software testing has limitations and is prone to errors.

Instead of manual testing, you can use the MATLAB unit testing framework to write repeatable tests.

Let’s explore a simple unit testing example using the fibonacci function.

Creating a unit test is straightforward using the MATLAB Test Browser, which evaluates the function created and compares the actual output with the expected output to check for errors.

This simple example of software testing ensures that your function returns the expected results and automatically verifies the output. 

Software Testing Best Practices

Automate Where Possible

Automation can make your testing process faster and cover more areas by running tests automatically. By automating repetitive and time-consuming tests, you can free up resources to focus on more complex scenarios. Automated tests can be run frequently and consistently, ensuring that code changes do not introduce new bugs. Consider using frameworks and tools that integrate seamlessly with your development environment to streamline the software testing process.

Write Tests Concurrently with Code Development

Writing tests as code development progresses, rather than waiting until the end of the project, helps you identify issues early and ensures that each piece of code is covered by tests as it is developed. This practice encourages developers to think through the requirements and design before implementing features, leading to more robust and maintainable code.

Prioritize Test Coverage

Not all code is created equal, and neither are all tests. Prioritize testing of areas that other components depend upon, have complex logic, or have historically been prone to bugs. Aim for a balanced approach where you achieve sufficient coverage without overburdening the testing process. Use code coverage tools to identify untested parts of your codebase and focus your efforts on those areas.

By incorporating these best practices of software testing, teams can enhance the quality and reliability of their software, ultimately leading to more successful projects and satisfied users.