You know the first half of the customer's security questionnaire by heart: access control, backup, encryption, logging. Then comes the AI section, and that is where it gets harder, because those questions often have no clear owner in your company yet. The sales lead needs the completed form back on Friday.
In the questionnaires I have seen, that section covers four things: whether you use customer data to train models, which models and services the feature uses, how long prompts and outputs stay in the system, and when a person reviews the result. Then someone writes four tidy paragraphs, neither dishonest nor careless, but written by a person who wants the deal closed.
The trouble starts when the customer asks a follow-up question, and it will, if the person at the other end has read that kind of answer before. The sales conversation stalls while the deadline gets closer, and nobody can verify the answer without first asking the developer who built the feature. The cases below are from my own work as an adviser, anonymised, with the answers rendered from memory.
Why a tidy explanation is not an answer
In the first half of the form, general descriptions have not been enough for years. Write that the backup was tested, and the next question asks when you last restored data and who ran the test. The AI questions arrived later, and they are often answered by people other than the ones who own the rest of the form. The same person at the customer reads both halves by the same standard.
The first thing people reach for is often an AI policy. But the customer wants to see how the solution is actually set up. This is about the documentation the customer asks for, not about law. I have assessed suppliers for a private equity fund and held third-party risk as an interim CIO, and in both seats I was the one reading the answers.
When the answer did not match the product
A company that builds contract-management software had to answer the AI section in the middle of an advanced sales conversation. The first draft read roughly like this:
We use an enterprise AI solution. Customer data is not used for training, prompts are not stored, and all outputs are reviewed by a person.
Nobody had held those three statements up against how the product actually works. Instead of another round with the text, I asked the lead engineer to follow one contract through, from upload to the report the customer would receive.
The workflow included a supplier that pulled the text out of the documents and an OpenAI model running on Azure OpenAI, and both the documents and the summaries were saved in the product.
Then the team could see for itself where the answer did not hold. The statement about training held for the chain we had followed, but the claim that prompts were not stored missed the copies in log files, in the database and in the backups. Users approved the finished report, while the system was the thing that had highlighted the individual clauses along the way. The corrected answer read roughly like this:
The feature uses the services and roles listed in the attached supplier schedule. Neither we nor those suppliers use customer data to train models under the arrangements we have reviewed. Uploaded documents, saved summaries and log files each have their own retention period in the same schedule. Reports are approved by a user before publication, while the clause highlights the system produces on its own are not reviewed individually first.
The four questions, and what the standard answer leaves out
Do you use customer data to train models?The tidy answer says no and points to the agreements. What is missing is two things: the supplier's terms together with the actual configuration, which shows whether anything is trained on your data, and an overview of the data flows, which shows which of your systems send what to which service.
Which models and services does the feature use, and who provides them? The tidy answer mentions leading suppliers under enterprise terms, without names. What is missing is each service, its role, and the company you hold the agreement with. Two things sit inside that question: who built the model, and who processes the information. Microsoft's documentationstates that the Foundry models Azure sells do not interact with services the model providers run themselves, such as OpenAI's.
How long do you keep prompts and outputs? The tidy answer says that data is processed briefly and not kept unnecessarily. What is missing is a dated export of the logging configuration in production, plus the retention periods at the supplier, in the database, in the reports users have saved, and in the backups. Data rarely sits in one place only.
When does a person review the result? The tidy answer says everything is reviewed. What is missing is the workflows as they are: what requires approval, what is used without it, and a log of who approved what and when.
Whichever question comes up, the customer needs the same four facts. Call them the four W's of evidence: what the documentation covers, where it comes from, who verified it, and when. The table below is not from one of the cases here. It is an example of what such a register can look like once it is filled in.
| Question | What we can present | Where it comes from | Who verified it | When, and what triggers a new review |
|---|---|---|---|---|
| Do you use customer data for training? | The terms for the three services the feature calls, plus our own configuration | Signed agreements and the configuration in production | Head of engineering and lawyer | 12 August 2026. New review on new terms or a new supplier |
| Who provides the models and services? | Supplier schedule with each service, its role and the company we hold the agreement with | The agreements and the calls made in production, among them an OpenAI model via Microsoft Foundry (formerly Azure OpenAI) | Head of engineering | 12 August 2026. New review when a service is added or dropped |
| How long do you keep prompts and outputs? | Export of the logging configuration plus retention periods for the database, the reports and the backups | Extract from the operating environment, held up against test requests that succeed and fail | Platform engineer | 12 August 2026. New review when the logging changes |
| When does a person review the result? | A description of each workflow and a log of who approved what | The product description and the approval log from production | Product manager | 12 August 2026. New review when a workflow changes behaviour |
Without a date, the customer cannot see whether the material still matches production. If the register holds, you can use the four W's on a fifth question you have never seen before.
When the logging said something other than the answer
A software house that builds analytics tools had already sent its answer. The product wrote draft text for customers' reports, and the answer read roughly like this:
Prompts and responses are processed briefly. Our AI supplier does not train on customer data, and we do not keep sensitive information unnecessarily.
The customer asked to see the settings that control retention. Before anyone sent a screenshot from the supplier's console, the platform engineer traced a test request through the company's own systems. The diagnostic log stored both the request and the model's response, because logging switched on during troubleshooting was never switched off again.
So the supplier's terms on using customer data for training said nothing about what the application itself wrote down. OpenAI's API documentation states that API data is not used for training by default, that abuse-monitoring logs are kept for up to 30 days, and that certain features store application data under different rules.
The team stopped the capture nobody needed and cleared up the copies that were already there. Request ids, response times and error codes stayed, while the prompt and the model response left routine logging. The developers tested requests that succeeded and requests that failed, because an export shows what is configured, not what happens in production.
The old log data did not disappear because of that. The team restricted access to it and gave it a deletion deadline, written down in a separate clean-up record. The corrected answer read roughly like this:
Raw prompts and generated responses are no longer part of the application's diagnostic log. The attached production configuration and the accompanying test requests show this, both when a request succeeds and when it fails. Reports saved by users follow the retention periods in the attached retention register, which also records the supplier's own retention. Access restrictions and deletion deadlines for the old log files are in the attached clean-up record.
The customer closed the point once the export, the test requests, the retention register and the clean-up record had been sent. The system still stored request ids and users' reports, but now the retention was described correctly.
What a wrong answer costs
A supplier of invoicing software had answered that every AI-generated entry was reviewed by a person before it was posted. During a demonstration, the customer's controller picked an ordinary invoice and asked who had approved it. The product team had to explain that entries where the model's confidence sat above a set threshold were posted automatically, while staff handled the exceptions and ran spot checks afterwards.
The controller stopped the rollout, and procurement put the larger agreement on hold. Management corrected the answer directly with the customer without calling it a misunderstanding, and the developers restricted the integration's permissions so the product could only produce drafts. The pilot carried on at a smaller scale, but the larger agreement did not make it into that budget round, and the supplier paid for the extra work itself.
How to start without building a large control function
- Follow one real case through the product together with the developer who built the feature, from the data that goes in to what the customer gets out.
- Write down the data flows and the supplier schedule while you still remember the details.
- Pull the actual logging configuration out of production, date the extract, and run test requests that both succeed and fail.
- Describe the workflows as they are, both the ones that require approval and the ones that are used without it.
If you have several AI features with their own data flows, take them one at a time. Then agree who maintains each document, typically the head of engineering for the technical ones and the product manager for the workflows, and what triggers a new review: a change of supplier, changed logging, or changed behaviour in the product.
Next time you should not start over
Take out your most recent answer to the AI section and read it as if a supplier had sent it to you. If you would ask a follow-up question yourself, the customer probably will too. Then try to answer the whole AI section from those four documents alone, and write down the gaps. It is cheaper to find them yourself than to have them found at a pilot meeting.