Shadow AI is now a data governance problem, not a hypothetical. The 2024 Cisco Data Privacy Benchmark Study, drawn from 2,600 privacy and security professionals in 12 geographies, found that 48% admit entering non-public company information into generative AI tools, and 27% of organisations had banned generative AI at least temporarily.
Into that gap steps a reassuring phrase: "runs locally". Sometimes it is true. Sometimes it means the model is local and the logging is not. Procurement needs a way to tell the difference.
Privacy Can Be a Pricing Tier
Defaults matter as much as architecture. Midjourney's plan comparison states that Stealth Mode, "which keeps your images and videos private, is only available on the Pro and Mega Plans". That is transparent. It also means a team on a lower tier may be producing work that is not private by default, and nobody in procurement may know.
The Five Questions
- Where does the local engine listen, and does it require authentication? A service bound to 127.0.0.1 is reachable only from the machine; one bound to all interfaces is reachable from the network.
- Which hosts is the application allowed to contact? Ask for the policy, not a summary of it.
- Does the code include analytics, telemetry or crash reporting, and can that be verified? Third-party SDKs are the most common quiet leak.
- What is the complete list of reasons the product goes online, and can each be disabled? No useful product is fully offline. Honest vendors can list why.
- If a cloud option is used, exactly what data crosses the boundary? Prompts, images, metadata, history?
What Good Answers Look Like
We hold our own product to these questions. For Vision Studio, our open-source local AI app, the answers are specific:
| Question | Vision Studio's answer |
|---|---|
| Engine binding | 127.0.0.1:8000, with a token the app creates at launch |
| Allowed hosts | The interface may connect only to the local engine |
| Telemetry | No analytics, telemetry or crash reporting in the code; the source is public |
| Network use | Model searches and downloads you start; an update check that can be disabled |
| Cloud boundary | Only with your own key; prompt text and settings, never images |
We also publish the limits: an update check reveals an IP address to the update host, third-party model hubs see the downloads users start, and the operating system's own telemetry is outside any application's control.
Why "Local" Became a Marketing Word
Local AI is genuinely valuable, which is exactly why the word gets stretched. A product can run its model on the user's machine and still send prompts to a server for "quality improvement". It can render locally but sync a gallery to the cloud by default. It can be fully local in its core and still ship an analytics library that reports usage. None of these is necessarily wrong. All of them are different from what a security team hears when a vendor says "local".
The fix is to stop accepting the word as an answer and ask for the architecture behind it. The five questions above do that. They are deliberately specific, because specific questions produce answers that can be checked.
How to Verify the Answers
Vendor answers are claims until tested. Three checks are within reach of any IT team:
- Listening ports. Standard operating system tools show which addresses an application's local server listens on. A service on 127.0.0.1 is not reachable from the network.
- Outbound traffic. Run the application through a typical working session while monitoring its network connections. Compare what you see with the vendor's list of reasons to go online.
- Source and dependencies. For open-source products, search the code and dependency list for analytics, telemetry and crash-reporting libraries. For closed products, ask for a software bill of materials.
If the observed behaviour matches the documented behaviour, you have a verified claim. If it does not, you have found the most important fact in the evaluation.
Red Flags in Vendor Answers
- "We take privacy seriously" in place of an architecture answer. Seriousness is not a network diagram.
- "Anonymised usage data only" with no list of what is collected. Ask for the fields.
- "Local mode is available" on a higher plan. Find out what the default does, because most users never change it.
- "Optional cloud features" that are enabled by default. Optional should mean off until chosen.
- No way to disable the update check. Updates are healthy, but regulated environments may need to control when software contacts the outside world.
Any one of these can have an innocent explanation. Each is a reason to ask the next question rather than sign.
Where Local Fits in an AI Portfolio
Local AI is not a replacement for every cloud service, and a policy that pretends otherwise will be ignored. Hosted models remain the right choice for many general tasks. The case for a verified local option is strongest where data sensitivity is highest: unreleased products, client material under confidentiality terms, personal data, and regulated communications. Classify work by sensitivity, route the most sensitive categories to tools whose local claims you have verified, and let the rest use whatever is approved. That gives people a fast, sanctioned path instead of a ban they will work around.
Turning This Into Policy
- Add the five questions to AI tool procurement and security review.
- Prefer vendors whose answers can be verified in source or by network monitoring.
- Treat "your privacy is important to us" as a non-answer.
- Give teams an approved local option, so shadow AI has a sanctioned alternative.
Verifiable or Not
"Local" is valuable precisely because it can be tested. Make vendors prove it, including where it stops.
Ask the five questions of every tool your teams want to use, record the answers, and recheck them when the product updates. A verified answer is an asset; an assumed one is a liability.
The full audit, run step by step against our own app is on the Vision Studio blog at vision-studio-x.com/blog/audit-local-ai-claims. Vision Studio is a free, open-source, local-first AI generation app built by Strategia-X. If you want a second pair of eyes on how your organisation adopts AI tooling, talk to us at strategia-x.com.
-Rocky
#AIGovernance #DataPrivacy #VendorRisk #ShadowAI #LocalAI #Procurement #StrategiaX #VisionStudio