A COMPARATIVE ANALYSIS OF HUMAN VS LLM-GENERATED SOFTWARE REQUIREMENTS: EVALUATING QUALITY, AMBIGUITY, AND COMPLETENESS IN AI-ASSISTED REQUIREMENT ENGINEERING

Abstract

The adoption of Large Language Models (LLMs) for software development has profoundly impacted software requirement engineering. While the use of LLMs in generating software requirements is increasing, there is a need for more research on the effectiveness of LLM-generated software requirements. This research compares requirements generated by humans and LLM to assess their quality in terms of understanding, ambiguity and completeness. The research uses a controlled experimental approach to generate requirements for several software use cases by humans and an LLM. These requirements are then measured against established criteria and analyzed by experts and statistical methods. The results show that although the LLM-generated requirements are efficient and consistent, they are also more ambiguous and less complete than human-generated requirements. In contrast, human-generated requirements are found to be more situational and accurate, but less consistent. This research offers an evaluation framework for AI-assisted requirement engineering, and suggests the benefits of a hybrid human-AI approach. Our findings offer insights into enhancing the effectiveness of AI-assisted software development.