LLM Based Software Test Scenario Generation System From Bug Descriptions

Abstract

Large, user-interactive software systems require extensive software testing for safe deployment. Manually generated test cases by software testers require extensive time and effort, and their scope remains limited. In this study, a system based on LLM is proposed for automatic test cases and test steps generation from bug descriptions. In the proposed system, bug descriptions were first collected from the Defects4J Dataset. Keywords relevant to the bug scope were extracted from the descriptions. Structured automatic prompts were generated from the extracted keywords. The generated prompts were fed into the LLM as input, and software test cases appropriate for the bugs were generated. The method was evaluated on actual bug descriptions by comparing it to a general (baseline) prompt. Experimental results show that the lengths of the generated scenarios are similar, but the proposed method increases the keyword match rate with error descriptions to an average of 89 %. The findings reveal that keyword-driven prompts are effective in generating more reliable and error-specific test scenarios without requiring additional data. © 2026 IEEE.