How to Build an AI Chatbot That Doesn't Leak Sensitive Company Data

The scariest chatbot bug isn't a wrong answer. It's a correct answer shown to the wrong person, like an intern's chatbot quoting the executive salary spreadsheet.

8 Data Protection Principles for AI Chatbots

1. Enforce permissions at retrieval, not in the prompt.
   "Don't reveal confidential info" is not access control. Filter documents by the user's permissions before they reach the model. If the model never sees it, it can't leak it.
2. Isolate tenants.
   Use per-tenant namespaces, or mandatory tenant filters that are applied server-side and can't be skipped.
3. Redact before you send.
   Mask PII (emails, IDs, card numbers) before data goes to a third-party model API.
4. Know your vendor's data terms.
   Check whether prompts are stored or used for training, the retention period, and where data is processed. Get it in writing.
5. Keep secrets out of prompts.
   System prompts can be extracted. Never put API keys, internal URLs, or credentials in them.
6. Watch your logs.
   Conversation logs are often the largest unprotected store of sensitive data. Encrypt them, set retention limits, and restrict access.
7. Limit what tools can do.
   If the chatbot can call APIs, give it read-only, scoped credentials rather than admin access.
8. Audit everything.
   Log who asked what and which documents were retrieved, so you can investigate incidents.

The principle behind all of this is that the model is not a security boundary. Your application is. Assume the model will eventually say anything it can see.

How does your team handle permissions in RAG today?