AI in defence: When supplier guardrails meet operational reality

Why defence (and AI) is different

That tension is sharper in defence than in an ordinary enterprise relationship. A supplier may treat its usage restrictions as part of a standard commercial bargain. A defence customer may see the same restriction as an attempt by a private company to constrain sovereign decision-making. Once a tool is seen as critical to military capability, the customer may no longer treat it as an ordinary software product.

AI also differs from more conventional supplied capability because its practical uses may be less fixed at the point of supply. A vehicle, platform or component usually arrives with a clearer operational function and more obvious practical limits. An AI tool can be reconfigured, integrated into adjacent workflows and repurposed across use cases more easily, with a magnifying effect on speed, scale and the practical weight of outputs, including in areas where decisions were historically made only by human operators.

That helps explain why some of these disagreements have less to do with contractual drafting than with control. They also arise in a sector where many AI providers are still relatively new, and may not yet be used to the pace, secrecy and operational demands of defence procurement.

The case study: Raven Assist

A UK defence organisation urgently procures “Raven Assist”, an AI tool designed to sort and prioritise large volumes of sensor data. The purpose is to help analysts identify threats more quickly and protect personnel. The award is made at pace because the customer wants compatibility with existing systems and cannot tolerate delay.

The supplier's standard terms prohibit use of the tool to select or guide lethal force. Certain surveillance uses also require specific approvals. The supplier insists on audit logs, access controls and a contractual right to suspend the service if those limits are breached. At the outset, the arrangement looks manageable. Raven Assist is used for analysis, support work and training away from live operations.

When the contract starts to strain

The position changes as the programme develops. Users begin relying on Raven Assist closer to live mission workflows. A contractor integrates its outputs into another system used in operational decision-making. The supplier later learns that material generated by Raven Assist was one of several inputs used to support a drone tasking decision.

At that point, both sides may feel they are acting reasonably. The customer says the tool did not decide anything: a human remained responsible, and the AI output was only one factor among several. The supplier says that this misses the point. Its restriction was meant to stop use so close to lethal decision-making that the distinction becomes hard to defend.

That is the moment at which broad contractual wording begins to look thin. Terms such as “no lethal targeting” may appear clear in the abstract. They are much harder to apply once the tool sits within a wider system, is used by multiple actors, and supports decisions taken under pressure.

Why technical controls matter more than red lines on paper

This is where the real lesson lies. If the supplier's true concern is that the tool may be used in ways it considers too close to targeting or sensitive surveillance, contractual wording alone is unlikely to do enough work. The better protection may lie in technical and operational controls: permissions, architecture, hard constraints, audit logs, approval routes and escalation processes.

That is particularly so where the supplier's restrictions are capable of being framed as a limit on sovereign use of lawful capability. In that setting, a pure contract solution may not be robust enough. The harder question is not simply what the terms say. It is who really controls the capability once it is deployed.

Why public disputes can spread beyond one customer

Most disputes of this kind are unlikely to play out in public. Defence relationships are often criticised for opacity, not over-exposure. But if a disagreement does become public, the effects may reach well beyond the immediate customer relationship.

Other contractors may become cautious. Partners may step back. Cloud providers may reassess their position. A dispute that begins as a narrow argument over use can therefore create wider uncertainty across the supplier's commercial network. That risk is harder to contain where the model sits within a wider ecosystem of cloud provision, compute, contractors and downstream integrations.

In practice, these disputes may be framed not only as contractual disagreements but as supply chain risk issues, with consequences that spread well beyond the immediate programme. The same is true of licence scope and intellectual property terms. In a defence setting, the practical question is often not just whether the customer may use the tool, but who may adapt it, integrate it, access its outputs or continue using it across contractors, other public bodies and successor suppliers if the original relationship comes under strain.

Why fast awards can deepen the problem

Direct awards, exemptions on national security grounds, or urgent or continuity-driven procurement can make these issues more acute. Some basic questions may not be worked through fully at the start: which connected systems may rely on the tool's outputs, who may approve new uses, how compliance is monitored across subcontractors, and what happens if the tool becomes too important to suspend without disruption.

The impact of public procurement law is important when making fast awards because of the increased risk of procurement challenges or bid protests from other suppliers. In the UK, the Government has issued procurement guidance to contracting authorities purchasing AI solutions, including Procurement Policy Note 0017 alongside the Guidelines for AI, which should be considered alongside defence-specific procurement requirements.

Public procurement rules often limit the changes that can be made to a contract after it has been awarded, and so programme growth should be anticipated and factored into the scope of the procurement.

As the programme grows, new users are added, more data sources are connected and workflows shift. The customer wants flexibility. The supplier becomes concerned that the original boundaries are slipping. None of that requires bad faith. It reflects the fact that AI tools can spread quickly through a wider contractor ecosystem, especially where cloud delivery and third-party integration sit at the centre of the arrangement.

Developments such as DAIC's AI Model Arena may help with part of this problem. In principle, a structured, defence-specific evaluation process should allow suppliers and customers to test systems earlier against realistic use cases, generate evidence across performance, reliability, robustness and security, and identify some points of strain before a tool is pushed into wider operational workflows. That may reduce some of the ambiguity that accompanies fast awards and immature deployments. But a pre-deployment evaluation cannot by itself resolve the more difficult questions that arise once a tool is integrated, relied on by multiple actors and pulled closer to operational decision-making.

Practical takeaways

• Suppliers should assume that broad ethical limits in standard terms may not be enough in a defence setting. The same is often true of licence and IP terms, which should be tested against likely adaptation, integration and continuity needs.
• Restrictions that really matter should be backed by technical controls, not left to drafting alone.
• AI contracts awarded by public sector bodies will be subject to public procurement rules, both on the award and later variation of the contracts.
• Suppliers and customers should expect that what they agree at the outset may not reflect operational realities once the tool is used in practice, and should plan accordingly.
• Where mechanisms such as the AI Model Arena are available, suppliers and customers should use them not just to compare performance, but to test likely deployment boundaries, integration risks and failure modes early.
• Change control, audit logs, access permissions and escalation routes should be agreed early and tested in practice.
• Both sides should plan on the basis that a dispute may affect a much wider contractor and cloud ecosystem, not just the immediate contract.

Conclusion

AI in defence can turn ordinary commercial friction into a dispute because the stakes are higher and the customer is the state. As more non-traditional suppliers enter the sector, and as governments become more dependent on highly capable models, the legal and commercial framework will come under increasing pressure.

Tools such as the AI Model Arena may help surface some of these issues earlier by giving defence and suppliers a more structured way to test capability against real use cases before full deployment. But the practical answer is still unlikely to lie in contract wording alone, or in engineering controls alone. These questions require joined-up legal, commercial, procurement, technical and operational assessment: not just what the tool can do, but who may use it, adapt it, integrate it, rely on it and keep using it if the relationship comes under strain. In this area, the better question is whether the parties have built a structure of governance and control that reflects how the tool may actually be used under real operational pressure.

Authored by Reuben Vandercruyssen, Malcolm Parry, David Hansom, and Alex Cumming.