Challenges and opportunities in integrating multi-agent AI into model-driven engineering of dependable systems

Agents empowered by Generative AI have the potential to significantly reduce the effort involved in developing and using modeling languages in high-integrity systems engineering. However, the large amount of domain-specific knowledge required to successfully complete model management tasks while maintaining consistency across multiple engineering domains presents difficulties for current foundational models, due to their general-purpose nature. The safety-critical nature of the systems being developed implies that AI must be used in a more controlled manner, making stepwise changes to artifacts that are validated by a combination of deterministic and AI-supported approaches (e.g. automated model validation and LLM-as-judge).

In this talk, I will discuss the above challenges, and present an overview of the current state of the art in specialising AI agents to the modeling domain: from the basics (few-shot and chain-of-thought prompting, the ReAct architecture, and Retrieval-Augmented Generation), to more advanced techniques such as output grammars (e.g. GBNF), code-driven agents (e.g. Smolagents), and multi-agent workflows. These approaches have different tradeoffs in terms of execution cost, development cost, and performance.

After this background, I will present an overview of our current work in the MOSAICO EU project on management and orchestration of AI agents, where we have developed infrastructure to allow organisations to create and host their own collections of specialised heterogeneous agents, which can work with each other to deliver higher-quality results than those of a standalone agent. MOSAICO agents come in different types (some propose solutions, others evaluate them, and others coordinate between agents), and can be developed in any language and framework so long as it supports the Agent-to-Agent (A2A) protocol. These agents are registered in a repository that keeps track of their benchmarked and live performance, which informs agent selection. An observability platform collects the traces needed to understand agent behaviour, with the support of MOSAICO-specific extensions to A2A.

Finally, to illustrate MOSAICO and the common problems when using AI to support model-driven engineering, we will discuss a prototype of a collaboration of agents that creates domain models in the Eclipse Emfatic notation from a natural language description, by using a combination of ReAct, few-shot prompting, code-driven agents, and multi-agent workflows.

Speaker

Antonio Garcia-Dominguez

Senior Lecturer in Software Engineering, University of York

Dr. Antonio Garcia-Dominguez is a Senior Lecturer in Software Engineering at the Department of Computer Science of the University of York, and a member of the Automated Software Engineering research group. Antonio's main research interests are automated software engineering and software testing.

Antonio leads a workpackage in the MOSAICO EU project on management and orchestration of AI agents, designing extensions to the A2A protocol for agent-to-agent collaboration. In addition to over 100 peer-reviewed publications in international conferences, journals, and book chapters, Antonio is a core contributor in several related open source projects. These include the Eclipse Epsilon model management languages and tools, and the Eclipse Hawk model indexing framework.