Researchers at FPT Software AI Center Introduce AgileCoder: A Multi-Agent System for Generating Complex Software, Surpassing MetaGPT and ChatDev

Introduction:

Code Large Language Models (CodeLLMs) have demonstrated remarkable proficiency in generating code. However, they struggle with complex software engineering tasks, such as developing an entire software system based on intricate specifications. Recent works, including ChatDev and MetaGPT, have introduced multi-agent frameworks for software development, where agents collaborate to achieve complex goals. These works follow standard procedures of multi-agent systems, defining different roles for agents to communicate and verify each other’s output. Nevertheless, they tend to oversimplify the complex nature of real-world software development, where software continuously evolves and improves.

Introducing AgileCoder:

In this work, a team of researchers from the FPT Software AI Center propose AgileCoder, a novel framework that mimics the intricate software development process in the real world by drawing inspiration from Agile Methodology, a widely used approach in professional software development teams. Approximately 70% of professional teams employ Agile Methodology, which is better suited to real-world software development. AgileCoder is built upon a key concept of Agile: software continually evolves over time, and thus development should be structured in the form of sprints (aka. phases).

Agent Roles and Collaboration:

AgileCoder consists of multiple agents playing distinct roles: a Project Manager, a Scrum Master, a Developer, a Senior Developer, and a Tester. These agents work collaboratively across sprints to achieve user tasks in accordance with the Agile methodology. By adapting Agile workflows to a multi-agent framework, AgileCoder emphasizes dynamic adaptability and iterative development. Outputs and problems from previous sprints are inherited and refined in subsequent sprints, increasing the likelihood of success for final products.

Dynamic Code Graph Generator:

A key innovation in AgileCoder is the Dynamic Code Graph Generator, which creates a Code Dependency Graph (CDG) that models relationships among code files and updates with source code changes. The CDG plays a crucial role in writing rational testing plans and enabling efficient code retrieval. It serves as a reliable source for agents to retrieve relevant and sufficient information, helping to avoid the inclusion of irrelevant information in prompts.

Evaluation and Results:

Comprehensive evaluations on benchmarks like HumanEval, MBPP, and ProjectDev demonstrate AgileCoder’s superior performance. On HumanEval and MBPP, which involve simple competitive-level programming problems, AgileCoder significantly outperforms CodeLLMs and state-of-the-art multi-agent frameworks like ChatDev and MetaGPT. To assess performance on more complex requirements, the team crafted a dataset named ProjectDev, containing requirements from real-world software projects. Evaluation results show that AgileCoder is more effective than other baselines in generating software from such complex requirements.

Conclusion:

AgileCoder is a novel multi-agent software development framework inspired by Agile methodology. Its key innovation, the Dynamic Code Graph Generator, creates a Code Dependency Graph that captures evolving code relationships for designing testing plans and enabling efficient code retrieval. By following Agile methodology, AgileCoder better mirrors real software development workflows and supports dynamic adaptability and iterative development. Extensive evaluations showcase AgileCoder’s superiority over existing methods like ChatDev and MetaGPT, making it a promising approach for complex software development tasks using CodeLLMs.

Check out the Paper and GitHub. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..

Don’t Forget to join our 48k+ ML SubReddit

Find Upcoming AI Webinars here

Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.