30-SECOND SUMMARY
What to take away
- ‘Local’ describes inference location, not the entire data lifecycle.
- Separate local models from cloud models and web tools.
- Treat model files, logs, RAG indexes, backups, and open ports as part of the attack surface.
Trace every copy of a document
Local inference does not describe the full lifecycle.
- 01SOURCE
Original file
- 02INDEX
Chunks and vectors
- 03INFERENCE
Prompt and response
- 04STORAGE
Logs and backups
Draw the complete data path
Track the original document, extracted text, chunks, embeddings, model input, generated response, logs, backups, and deletion process. Each stage can create another copy.
Record storage location, authorized users, retention period, and deletion method for every stage.
For verification, save the model and runtime versions, source input, relevant settings, and observed output together. Repeat the step while changing only one factor, and record unexpected results and untested limits as carefully as successes before applying the guidance to private or production data.
Separate local and cloud features
Ollama distinguishes local processing from cloud models and web search. A model name ending in `:cloud` is not an on-device workload.
Disable unneeded cloud features and verify behavior with the network disconnected before adding private documents.
Preserve the before-and-after state and the time of the check so that another run can reproduce the result. Include at least one failure condition—such as empty input, constrained resources, or a restart—to reveal the boundary of the step rather than documenting only the happy path.
Treat model files as untrusted input
Model formats are complex binary inputs consumed by parsers. Use trusted publishers, verify hashes when provided, update runtimes, and test unfamiliar files in an isolated environment.
A downloadable model is not automatically safe or licensed for every business use.
Define completion with an observable result instead of a general impression. Repeat the same input, and if the output changes, isolate whether the model, runtime settings, or source data changed before moving to the next stage.
Do not expose an unauthenticated server
A service bound only to localhost has a smaller network surface than one bound to all interfaces. Tunnels and reverse proxies can silently turn a personal model server into an internet service.
External or multi-user access requires authentication, TLS, network controls, rate limits, monitoring, and tenant isolation.
For verification, save the model and runtime versions, source input, relevant settings, and observed output together. Repeat the step while changing only one factor, and record unexpected results and untested limits as carefully as successes before applying the guidance to private or production data.
Check logs, sync, and deletion
Debug logs can contain prompts, file paths, or responses. Cloud backup and desktop sync tools may copy local artifacts to remote storage.
Deleting the source document is not enough when chunks, embeddings, caches, and chat history remain. Test deletion across the entire pipeline.
Preserve the before-and-after state and the time of the check so that another run can reproduce the result. Include at least one failure condition—such as empty input, constrained resources, or a restart—to reveal the boundary of the step rather than documenting only the happy path.
Frequently asked questions
Is an offline computer completely safe?
No. Offline operation reduces external transfer but does not remove malicious files, local access, logs, backups, or prompt-injection risks. For a practical check, follow the “Draw the complete data path” section, change one condition at a time, and record the result.
Can I use company documents?
Only after checking company policy, data classification, model licensing, access controls, retention, and deletion requirements. Separate local models from cloud models and web tools. For a practical check, follow the “Separate local and cloud features” section, change one condition at a time, and record the result.
Does RAG remove the need to protect the source?
No. The index and embeddings become additional sensitive assets that require access control and deletion. For a practical check, follow the “Treat model files as untrusted input” section, change one condition at a time, and record the result.
Primary sources
Check the original documentation for version-specific details.
Ollama FAQ llama.cpp Security Policy OWASP GenAI Security Project