LLM Based Software Test Scenario Generation System From Bug Descriptions - Gundem, Merve Yilmazer; Karakose, Mehmet (Institute of Electrical and Electronics Engineers Inc., 2026)
Large, user-interactive software systems require extensive software testing for safe deployment. Manually generated test cases by software testers require extensive time and effort, and their scope remains limited. In this study, a system based on LLM is proposed for automatic test cases and test steps generation from bug descriptions. In the proposed system, bug descriptions were first collected from the Defects4J Dataset. Keywords relevant to the bug scope were extracted from the descriptions. Structured automatic prompts were generated from the extracted keywords. The generated prompts were fed into the LLM as input, and software test cases appropriate for the bugs were generated. The method was evaluated on actual bug descriptions by comparing it to a general (baseline) prompt. Experimental results show that the lengths of the generated scenarios are similar, but the proposed method increases the keyword match rate with error descriptions to an average of 89 %. The findings reveal that keyword-driven prompts are effective in generating more reliable and error-specific test scenarios without requiring additional data. © 2026 IEEE.
2026 30th International Conference on Information Technology, IT 2026 -- 24 February 2026 through 28 February 2026 -- Zabljak -- 221544
Keywords: Automatic test pattern generation; Generation systems; Interactive software; Keyword-driven; Large users; Scenarios generation; Software testings; Software-systems; Test case; Test scenario; Software testing