The hottest technology today is undoubtedly OpenAI’s ChatGPT, and AI technology has been widely applied in many areas. Today we’ll discuss TestGPT, a popular application in the software testing field.

What is TestGPT?

TestGPT is an artificial intelligence model (AI model) for the testing domain, released by CodiumAI Ltd., a startup company headquartered in Tel Aviv, Israel. It is powered by OpenAI’s GPT-4 large language model.

Official website: www.codium.ai

From the website’s introduction, it’s clear that TestGPT is an AI tool designed to assist developers in testing their code, providing automatically generated software test suite suggestions to accelerate coding and error scanning.

How to Use It?

Currently, TestGPT offers plugin support for VsCode and JetBrains

Taking VsCode as an example, you can install it by searching for CodiumAI in the VsCode extension marketplace

After installation, you can log in directly with your Github account.

Let’s see how it works with a problematic Python sorting algorithm code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

def insert_sort(lists):
    # Insertion sort
    count = len(lists)
    for i in range(0, count): #range interval is incorrect, this is a bug, should be (1, count)
        key = lists[i]
        j = i - 1
        while j >= 0:
            if lists[j] > key:
                lists[j + 1] = lists[j]
                lists[j] = key
            j -= 1
    return lists

Switch to the CodiumAI plugin, and after connecting, you can see that the code section already has an entry point for generating tests

Try generating tests and see the effect. After a moment, you can see the AI’s results

The AI assistant interface has three parts:

• Test Suite
• Code Analysis
• Code Suggestions

In the test configuration section, you can make basic test configurations

The Coverage section shows the coverage of the AI-generated cases

The Test section contains the specific test cases. In this example, 6 test cases were generated

After execution, you can see the test results, but all the generated test code executed successfully, seemingly not detecting the bug through the test code :(

Code Analysis provides an analytical explanation of the code

The Code Suggestions section provides modification suggestions, which accurately points out the problem in the code.

Applying the suggestion directly, you can see that the code has been modified and comments have been added

Conclusion

So we can see that TestGPT is indeed excellent, accurately analyzing the purpose of the code and identifying problematic areas. It also helps design test case coverage and related unit test scripts. Making good use of this tool can undoubtedly provide great assistance to development engineers. However, from initial use, the self-generated test cases still have deficiencies in covering problematic code, and in the short term, it’s probably not sufficient to replace test design.