AI Data Privacy Checklist: Preparing Prompts, Documents, and Logs for Safe LLM Use

Overview

The question “What should not be put into ChatGPT or another AI model?” has no single answer for every organization. The acceptable use of data depends on the task, the model provider, the deployment architecture, the organization’s policies, and the obligations attached to the data. A public product description and an employee medical record should never be treated as equivalent inputs.

A practical AI data privacy process has four stages:

1. Classify the data. Identify whether the input is public, internal, confidential, personal, regulated, or a combination.
2. Minimize the prompt. Send only the information required for the task, not the entire document, database row, ticket history, or customer record.
3. Apply controls. Use redaction, access restrictions, approved tools, logging rules, and human review appropriate to the risk.
4. Recheck the workflow. Provider terms, model configurations, regulations, integrations, and business processes can change.

Privacy should be considered across the full LLM workflow, not only in the visible prompt. Data may appear in uploaded files, retrieval indexes, tool calls, conversation history, generated outputs, application logs, evaluation datasets, error reports, and support tickets. A prompt that looks harmless in isolation can become sensitive when combined with other fields.

What to track

1. Data classification and purpose

Start an inventory for each AI use case. Record what data enters the system, why it is needed, where it comes from, who can access it, and what the system returns. A simple classification model can include:

• Public: Information already approved for public distribution, such as published documentation or a public product announcement.
• Internal: Non-public operational information that may be suitable only for approved organizational tools.
• Confidential: Business plans, source code, contracts, unreleased product details, credentials, security findings, or information covered by confidentiality obligations.
• Personal: Information that identifies or relates to a person, including names, contact details, account identifiers, support conversations, or free-text descriptions containing personal details.
• Regulated or highly sensitive: Data subject to specific organizational, contractual, legal, health, financial, employment, or security requirements.

These labels are working categories, not a substitute for your organization’s formal data policy. When a prompt contains several categories, handle it according to the most restrictive relevant category.

2. Redaction and pseudonymization

Before sending text to an AI model, remove details that do not affect the task. Replace direct identifiers with consistent placeholders so the model can still follow relationships. For example:

Customer: [CUSTOMER_001]
Email: [EMAIL]
Order ID: [ORDER_ID]
Account number: [ACCOUNT_NUMBER]
Incident date: [DATE]
Internal host: [HOSTNAME]

Use stable placeholders when the prompt requires comparison, such as asking a model to summarize several messages from the same customer. Avoid replacing every value with generic text if the value is necessary for the analysis. Instead, preserve only the minimum useful attributes, such as an approximate age range, a broad region, or a month rather than an exact date.

Redaction is not automatically safe. Rare job titles, unusual events, combinations of dates, and small geographic areas can make a person identifiable. Treat pseudonymized data as potentially sensitive unless your privacy and security teams have approved the method.

3. Secrets and security material

Do not paste passwords, API keys, access tokens, private certificates, recovery codes, session cookies, encryption keys, or unredacted vulnerability details into an AI tool. This applies to prompts, attachments, debugging transcripts, and screenshots. If a secret is exposed, follow the organization’s incident process and rotate or revoke it as appropriate; do not rely on deleting the conversation alone.

For code assistance, provide a minimal reproducible example with secrets removed. Replace real domains, identifiers, credentials, and infrastructure names with safe test values before requesting debugging help.

4. Provider and application controls

Maintain a review record for each model or AI development tool. Check the provider’s current documentation and your contract or service configuration for:

• Whether submitted content may be used for service improvement or model training, and which settings control that use.
• Available regional, tenancy, encryption, identity, and access-management options.
• Retention periods for prompts, files, outputs, abuse monitoring, backups, and support records.
• Subprocessors, data transfer arrangements, deletion procedures, and audit information.
• Administrative logs, export controls, rate limits, and methods for separating development from production data.
• How the provider handles tool calls, retrieval content, connectors, and third-party integrations.

Do not assume that a tool’s marketing description answers these questions. Record the date of the review, the exact product or deployment, the configuration examined, and the owner responsible for follow-up.

5. Logs, retrieval, and outputs

Review what your application stores. A production LLM system may log the original prompt, retrieved passages, tool arguments, model output, user identity, latency, and error details. Apply separate retention and access rules where possible. Mask sensitive fields before observability data reaches a general-purpose logging platform.

For retrieval-augmented generation, privacy review must include the document index and retrieval permissions. A model should not receive a document merely because it is technically available to a shared search service. Connect retrieval results to the user’s authorization context, and test whether prompts can cause content from another user, team, or tenant to appear.

For practical workflow design, see this RAG tutorial for beginners and the guide to production-ready AI workflows.

Cadence and checkpoints

Privacy controls are easier to maintain when reviews are tied to normal engineering and governance routines.

• Before launch: Map data flows, classify inputs, approve the provider and configuration, define retention, restrict access, test redaction, and document human review.
• Monthly or quarterly: Sample prompts and logs for unexpected personal or confidential data. Review access lists, new integrations, failed redactions, retention settings, and unresolved incidents.
• At every model or feature change: Recheck tool permissions, retrieval scope, system prompts, output destinations, logging behavior, and evaluation data. A model change can alter what information is requested or reproduced.
• When the dataset changes: Reclassify newly connected sources. A previously low-risk workflow may become sensitive after adding support tickets, HR records, customer data, or production telemetry.
• At least annually: Refresh the vendor review, internal policy, training materials, data map, incident contacts, and documented approvals.
• After an incident or complaint: Preserve relevant evidence, restrict the affected workflow if necessary, identify the exposure path, and update controls before resuming normal use.

Assign an owner and a due date to every review. A checklist without ownership becomes a document that appears complete while important settings remain unverified.

How to interpret changes

Not every change requires shutting down an AI workflow, but every material change deserves a documented decision. Use a small risk review that asks:

1. Did the type, volume, sensitivity, or source of the data change?
2. Did the recipient change, including a new model, connector, plugin, subprocess, or logging system?
3. Did the purpose change from experimentation to customer-facing or operational use?
4. Did access expand from a small engineering group to a wider workforce or external users?
5. Did testing reveal leakage, incorrect authorization, unsafe output, or an inability to delete stored content?

Prioritize changes that increase exposure or reduce control. For example, moving from synthetic test records to live customer conversations is a greater governance event than changing the wording of a low-risk summarization prompt. Similarly, adding a database connector may create more privacy risk than switching between two approved models with equivalent data controls.

Track both leading and trailing signals. Leading signals include the number of users with access, new data sources, redaction failures, and unreviewed integrations. Trailing signals include privacy complaints, security incidents, unexpected data in logs, deletion requests that cannot be completed, and evaluation cases showing cross-user information leakage. Pair privacy review with output testing; the AI output evaluation metrics guide provides a useful framework for checking safety and consistency alongside quality.

Prompt design also matters. Use explicit instructions such as “do not reproduce personal identifiers,” but do not treat a system prompt as a complete access control. Enforce authorization, filtering, and redaction in application code or infrastructure where possible. Prompt templates can standardize safe handling, while a prompt testing framework can help detect regressions after changes.

When to revisit

Revisit this AI data privacy checklist on a monthly or quarterly cadence for active systems, and immediately when a provider changes its terms, retention behavior, regional availability, security controls, or product architecture. Also schedule a review when your organization connects a new data source, introduces retrieval or tool use, changes the user population, moves a prototype into production, or begins using AI-generated content in a higher-impact decision.

For a practical checkpoint, ask each workflow owner to produce five items:

1. A current data-flow diagram or written data map.
2. A list of data categories permitted and prohibited in prompts and uploads.
3. A current provider and configuration review.
4. Evidence of redaction, authorization, retention, and output testing.
5. A named owner, next review date, and escalation path for incidents.

If any item is missing, pause expansion and close the gap before adding more users or data. Keep the checklist versioned alongside the application, prompt templates, and evaluation suite so that privacy decisions remain connected to implementation changes. For teams managing several workflows, compare tools and deployment patterns with an AI developer tools comparison framework, and revisit the review whenever the system’s data boundary changes.

Safe AI development is not achieved by avoiding every useful input. It comes from knowing what the system receives, minimizing what it does not need, controlling who can access it, testing what can go wrong, and returning to those decisions as the workflow evolves.