Human in the Loop: Accelerating Software Development Reliably with AI

Posted on September 03, 2026

By Jay Thomas, senior director of field engineering, TASKING

Artificial Intelligence (AI) can be a powerful tool in developing functionally safe and secure systems, especially ones based on multicore SoCs. However, with the growing complexity in embedded software and the pressure to deliver and release software faster, improper use of AI can lead to increased safety and security risks. This article will explore how AI can accelerate software development in a manner that developers can trust.

Human in the Loop

When AI is used to write code or create tests, it’s like working with a black box whose internal mechanisms can’t be seen. Consider the following prompt: Rewrite this code to execute faster. Such a prompt, even with more detailed constraints, runs the risk of doing what was asked but still completely missing the objective. For example, critical code may be deleted to make the code run faster. Treating AI as an unqualified tool that automates a task in a safety process, a person (human) must review what the output of the AI to assure it is correct.

Similarly, AI can assist in defining tests to verify and validate code. However, the question arises as to whether a generated test achieves sufficient coverage or is even relevant in the first place. Developers need to verify that the AI tests fully exercise code and correctly implement the requirements. Again, there’s still a need for a human in the loop.

Requirements-based testing - including testing of safety and security requirements -

is fundamental to safety and security assurance as well as certification. Thus, requirements traceability is essential to the implementation of code, the creation of tests, and the reporting/analysis of missing code/tests, extraneous code/tests, and tests that have not been performed or need to be repeated following a change.

The challenge is that without a process framework and some guardrails in place, developers run the risk of spending so much time reviewing and correcting what the AI does, that they reduce rather than improve their efficiency. This can also erode overall confidence in the system. Ultimately, developers may run into difficulties when trying to certify their designs to strict industry standards.

Trusted Process Framework

The key is not being in a position to have to trust the AI. This can be achieved by working within a trusted process framework that puts guardrails around AI-based activities. Rather than trying to replace developers with AI, the strategy is to use AI to amplify a developer’s capabilities by performing clearly defined tasks in a manner that is transparent, traceable, and verifiable. For safety critical applications, AI must be limited to assisting with tasks within a process framework where the scope of AI’s impact is well-understood. In other words, the process framework:

1. Clearly defines what processes the AI is involved with and what tasks it will perform.
2. Has guardrails to keep the AI from extending beyond the scope of these tasks.
3. Relies upon human oversight and discernment to validate the AI’s work. Put another way, a human is responsible for the design of the system. A human needs to evaluate and take ownership of the AI’s contribution.

Consider evaluating Worst-Case Execution Time (WCET) in a multicore-based system. WCET plays an important role in verifying that a system is deterministic and meets all its critical real-time deadlines.

Part of the challenge of evaluating WCET in multicore SoCs is the complexity that arises from timing coupling and interference. Even if cores and the tasks running on them are independent (i.e., no shared data or control coupling), there may still be contention for shared resources such as cache that can negatively impact execution performance and determinism.

For example, Task A has a small cache footprint, and Task B has a large cache footprint. When executed on its own, Task A achieves excellent execution performance because the code does not flush the cache. However, when Task B is running concurrently on the same SoC, its large cache footprint causes the cache to flush frequently. Thus, even though Task A and Task B are otherwise independent of each other, Task B can cause substantial variability in Task A’s performance.

With an AI-powered process framework in place, developers can accelerate WCET evaluation with trust and confidence. For example, with the TASKING toolchain integrating compile, debug, and test capabilities, developers can automate cycling through different compiler options, as well as vary task allocations on cores to assess the impact of timing coupling on WCET. The debugger can be automated to capture specific performance data which is then automatically sent to the test tools for evaluation.

AI agent can help automate many of these manual process tasks. Agentic AI is less about having the AI produce outputs and more about defining processes that the AI agent can iterate against. This fits better into certification workflows, especially if the AI can ask for help from a human when it needs it. For example, if a particular combination results in an error, AI can assess the problem and mitigate it to resume testing without disruption or seek human intervention.

AI agents can also suggest test cases that a person may have overlooked or not considered. Note that the AI does not provide all the suggested combinations. Rather, the AI adds to the developer’s selections to catch possible blind spots. In other words, the AI builds on the developer’s expertise.

AI-assisted automation increases testing efficiency and speed. Using an integrated toolchain simplifies AI coordination between tools. Developers can also scale testing into the cloud. The result is that developers can evaluate more combinations across a wider selection of possibilities than they could on their own without AI assistance. More comprehensive testing increases accuracy and confidence in system safety, security, and reliability.

Faster Certification

The Verification and Validation model used to develop safe and secure software illustrates how testing is planned in parallel with development rather than being applied only at the end of a project (see Figure 1). Requirements traceability is an essential part of the process as it provides a way to verify how requirements have been implemented through each stage of development. This applies not just to verification and validation but to other types of analysis as well, including vulnerability analysis and data and control flow analysis. Employing AI is a manner that complicates traceability in turn complicates verification, reducing reliability and making certification more difficult and costly.

By itself, AI is not directly certifiable in the traditional safety-critical sense because current implementations are probabilistic, difficult to fully bound, and often lack the transparency needed to establish trust in how outputs are produced. However, when AI is used within a clearly defined process framework and its outputs are reviewed, constrained, and managed through tools qualified for certification, those outputs can become certifiable artifacts.

In this model, the AI is not the trusted authority. The qualified toolchain and the defined engineering process provide the trust boundary. AI-generated code, derived requirements, test cases, analysis results, or other artifacts can be accepted when they are captured, reviewed, traced, verified, and controlled using certified or certifiable tools. This allows organizations to benefit from AI-assisted development while still maintaining the traceability, evidence, and verification discipline required for certification.

An AI-assisted process framework not only accelerates design across the entire software development lifecycle, it facilitates verification and validation as well as drives faster certification with full requirements traceability. By clearly defining AI’s role, implementing guardrails, and understanding the importance of human oversight, development teams can benefit from AI automating many manual tasks, enabling developers to produce better code faster with lower risk.

ESD June 2026 (originally published in Electronic Specifier)