AI Can Write Code. It Does Not Replace Secure Development Governance

The short answer has three components: a written tool policy, automated security scanning that applies to every line regardless of who wrote it, and human peer review before anything merges to the main branch. An organisation that puts all three in place improves its underwriting terms and strengthens its position if a claim ever arrives.

Key Takeaways

• Language models are trained on public code repositories, so they reproduce old patterns carrying known weaknesses. An academic study published in 2025 found that around thirty percent of code snippets generated by autocomplete tools contained security weaknesses.
• The risk grows when the request to the model is framed around functionality alone. Studies published in 2025 found that roughly 45% of code generated by language models chose the insecure option, and tests published in 2025 produced code with XSS flaws in 86% of cases and Log Injection flaws in 88%.
• Secret leakage is the most immediate expression of the problem. Industry data from 2026 indicates that AI-assisted commits leak secrets at double the baseline rate, meaning live keys sitting inside an active repository.
• The insurance consequence touches three lines: technology professional liability, cyber, and directors and officers. Underwriters now ask direct questions about code review and automated scanning, and the answers drive pricing, exclusions and the handling of a claim.

Working Code and Safe Code: Where the Exposure Begins

Code that compiles and passes functional tests can still carry a decade-old security weakness. Language models are trained on public repositories, forum answers and tutorial examples that were never meant to run in production, and they reproduce the patterns they saw. An academic study published in 2025 found that around thirty percent of code snippets generated by autocomplete tools contained security weaknesses across several CWE categories.

The gap opens at an identifiable point: how the task is defined. A developer who asks the model for a function that takes user input and returns a result gets exactly that. When the prompt drops the requirement for validation, output encoding and disciplined error handling, the model takes the short path. Studies published in 2025 found that roughly 45% of code generated by language models chose the insecure option even where a safer alternative existed. In tests published in 2025, models produced code with XSS flaws in 86% of cases and Log Injection flaws in 88% when security was not explicitly requested. Those numbers describe the default behavior of the tool, not an isolated failure. Responsibility for writing the requirement, and for checking it, stays with the organization.

The code runs, the developer is satisfied, and the weakness lands in the main branch with nobody stopping to ask how it will read in a forensic report.

Four Risk Vectors Worth Mapping Before the Next Renewal

The risk in using AI coding tools runs along four separate tracks, and each track needs a different control. Mapping them properly is the basis for any conversation with an underwriter, with an enterprise client, or with the CPA firm performing the examination behind a SOC 2 report.

Weaknesses Born of Training Patterns

A model produces what it saw most often, not what is correct today. SQL injection, weak cryptography, careless permission handling and dependencies with known vulnerabilities keep appearing in the output because they are common in public code. Basic static analysis catches a large share of this family, provided it runs on every pull request rather than on release builds alone.

Secrets Leaking Into Commits

AI tools tend to produce working code with credentials embedded, because that is how the example they learned from looked. Industry data from 2026 indicates that AI-assisted commits leak secrets at double the baseline rate. The code runs, the developer moves on, and the live key stays in repository history even after the line is deleted. A secret scanner that reads history, paired with a fast rotation procedure, shortens the exposure window from months to hours.

Poisoned Rules Files

During 2025 researchers documented an attack family in which malicious instructions are injected into the configuration rule files that coding assistants read before generating code. The instruction hides in invisible Unicode characters, and the tool produces backdoored code without the developer noticing any change in behaviour. GitHub subsequently added warnings for hidden characters. This is an attack surface with no direct equivalent in traditional code review, which is why rule files belong under version control and deserve the same peer review as code.

Prompt Injection at Runtime

The OWASP Top 10 for LLM Applications 2025 places prompt injection at the head of the risk list. When AI-generated code processes external inputs without adequate validation, hostile input can steer system behaviour in ways a security-aware developer would have blocked. The risk is structural rather than incidental, and it calls for deliberate trust boundaries between system components.

Research tracking from 2026 documents a sharp rise in disclosed vulnerabilities attributable to AI-generated code. That trend line explains why underwriting questionnaires grew longer over the past year.

Vibe Coding: A Production System Nobody Read

The riskiest pattern right now is production systems built by people who cannot read code. Founders, product managers and operations teams use prompt-based platforms to build tools that touch customer data, and push them live within days. The platforms themselves are powerful; the gap sits between a product that works and a product that can be insured.

Whoever built the system cannot assess whether permissions are too open, whether data is encrypted at rest, whether activity is logged, or whether events can be reconstructed at all. When an incident occurs, the forensic investigator finds no logs, no technical owner and no record of decisions. The coverage question then becomes a question of representations: what was told to the underwriter about the development environment, and what is actually running in production.

A product built over a weekend and pushed live without a security review is a liability event waiting for its trigger.

Where Technical Risk Meets the Policy

Most policy wordings determine cover by the nature of the act and by the exclusions, not by the identity of whoever typed the code. What changed is the reasonableness question: when an investigation shows that a well-known weakness reached production without a basic scan, an argument opens about reasonable security practice that shapes the outcome of the claim. The broker (LAMDA Broking) and legal counsel review policy wordings against the clauses of the enterprise client contract, in order to identify the broad artificial intelligence exclusions that have entered wordings in recent years.

Scenario mapped to insurance line and to the core question in a claim:

What Underwriters Ask Before a Renewal

Underwriting questionnaires have moved from general declarations to operational questions that can be proven or disproven with a document. An underwriter who receives a vague answer prices the uncertainty, and will sometimes add an exclusion or make renewal conditional on remediation. The table below sets out the questions that keep recurring and the difference between an answer that weakens a submission and one that holds up.

Common underwriting questions and the quality of the answer:

Regulation, Standards and Client Contracts

The demands reaching technology companies today arrive from three directions at once: regulation, standards and commercial contracts. The third is moving fastest, because enterprise procurement in the United States and Europe already asks for documentation of the software development lifecycle and of how AI tool output is validated. A company that assembles that documentation for one client tends to find it is also ready for the underwriter.

In the United States, the 2023 SEC disclosure rules require public companies to describe the board’s oversight of cyber risk. The proposed disclosure requirement regarding cyber expertise among board members was dropped from the final rule, so the focus rests on the oversight process rather than on board composition. For private companies with institutional investors the expectation is substantively similar: a documented discussion, metrics, and a remediation plan.

Europe is building an additional layer. The EU AI Act imposes transparency and risk management duties on certain uses, NIS2 and DORA require security controls and supplier management across broad sectors, and the Cyber Resilience Act will require software manufacturers to demonstrate proactive vulnerability management towards the end of 2027. An organization using AI coding tools will need to show that output is tested and validated before release.

All of these rule sets converge on the same operational requirement: documentation showing what was tested, when, and with which tool.

In Israel, Amendment 13 to the Privacy Protection Law widened the enforcement powers of the Privacy Protection Authority and raised the price of an incident that flows from negligent controls. At the same time, the speed at which Israeli companies adopt AI coding tools is a genuine competitive advantage and also a source of a governance gap against enterprise clients abroad. An Israeli company entering enterprise procurement in the United States or Europe meets contractual demands for software development lifecycle documentation it did not always prepare in advance. The gap usually surfaces late, once a deal is already on the table and the signing window is short. On the standards side, a SOC 2 report is an attestation report issued by an independent CPA firm against AICPA criteria for a defined period, rather than a certificate anyone holds. Keeping that documentation current removes the scramble entirely.

Common Mistakes

• Treating a pass on functional testing as security approval.
• Writing the AI policy into a general document with no owner, no review date and no enforcement in the pipeline, then discovering at renewal that the underwriter wants operational evidence rather than good intentions.
• Granting automated coding agents broad permissions in the cloud environment to save configuration time.
• Assuming the tool vendor carries responsibility for the output, without reading terms of use that place it on the user.
• Reporting development controls to an enterprise client that exist in a slide rather than in the code.
• Treating a SOC 2 report as a certificate, and missing that it is an external firm’s attestation over a defined period and over controls the organisation itself defined.
• Reading the policy wording only after an incident, once definitions and exclusions can no longer be changed.

Professional Checklist

1. Define a list of approved AI development tools, and name an owner who reviews it quarterly.
2. Write a usage policy setting out what may be fed into the tools, which repositories are off limits, and what follows a breach.
3. Integrate static analysis into every pull request, and configure a merge block on high severity findings.
4. Run automated secret scanning across the full commit history rather than the current change alone.
5. Require human peer review before any merge to the main branch, even when the code looks clean.
6. Tag AI-assisted commits so you can measure volume and prioritise testing.
7. Reduce coding agent permissions to least privilege, and keep them away from the production environment.
8. Place tool rule files under version control, and review them exactly as you review code.
9. Run dynamic testing and periodic penetration tests against features built at speed.
10. Document a vulnerability remediation procedure with severity-based target times, and retain the records for three years.
11. Register every system built outside the development team, and test it before any external exposure.
12. Align the representations in client contracts with what actually exists in the development pipeline.
13. Bring the underwriting questionnaire to the development team two months before renewal, not a week before expiry.
14. Test cyber and professional liability wordings against a scenario involving a weakness in AI-assisted code.

AI tools generate code faster than most organizations can review it, so the quality of insurance cover is now determined by the development process the organization can prove on paper.

Frequently Asked Questions

Does using AI coding tools harm our ability to buy cyber cover?

Use of the tools by itself does not block cover. What moves the needle is the quality of the controls around them: a written policy, automated scanning and documented peer review. An organization presenting orderly documentation usually receives terms similar to one that uses no such tools at all. An organisation declaring widespread use without controls may face an exclusion, a higher retention, or remediation as a condition of renewal.

Does a professional liability policy respond to loss originating in AI-assisted code?

In most wordings cover turns on the definition of the professional act and on the exclusions, not on who wrote the code. The point that needs checking is whether the wording carries a broad artificial intelligence exclusion introduced in recent years, and how it sits against the product the company sells. Reviewing the wording alongside the enterprise client contract is how you confirm the scenario falls within cover.

How does a SOC 2 report relate to AI use in development?

SOC 2 is an attestation report issued by an independent CPA firm against AICPA criteria, rather than a certification. Where an organisation uses AI tools in development, the controls examined in the report should include the process applied to tool output: tool approval, code review, permission management and pipeline scanning. A gap between what the report describes and what happens in practice is an exposure in its own right, with clients and with underwriters alike.

What is expected of the board here?

The 2023 SEC disclosure rules require public companies to describe the board’s oversight of cyber risk. The proposed disclosure requirement regarding cyber expertise among board members was dropped from the final rule. For private companies with investors and enterprise clients the expectation is substantively similar: a discussion recorded in the minutes, metrics presented to management, and a remediation plan with dates attached.

Can we simply ban AI tools and close the risk?

A blanket ban produces undeclared use and makes hiring developers harder, so it almost always swaps a visible risk for a hidden one. The approach that works is controlled approval: a list of sanctioned tools, clear rules on sensitive data fed into them, and limited permissions for automated agents. Measurable control beats a prohibition nobody can enforce, both in underwriting and in daily management.

What happens when a product manager builds a system on a prompt platform and ships it?

The system enters the corporate environment with no security review and no technical owner. In that scenario a simple weakness can end up exposed to the open internet together with access to customer data, often with no activity logging that would allow anyone to reconstruct events. The practical fix is a central register of every system built outside the development team, testing before external exposure, and a classification of what data may run on such platforms.

Is there a measurable rise in AI-related litigation?

A 2026 report found that the number of AI-related lawsuits in the United States grew by 978% between 2021 and 2025. Alongside that, research tracking from 2026 documents a sharp rise in disclosed vulnerabilities attributable to AI-generated code. Those two lines explain why underwriting questionnaires have grown longer and why underwriters now ask for operational evidence instead of general statements about policy.

The above is general information only and does not constitute insurance, legal or other professional advice. Terms of cover, exclusions and duties are set by the specific policy wording and by applicable law. Each case should be assessed on its own facts against the policy wording and with a qualified adviser.