Have you ever shared something personal with an AI chatbot and then wondered in what ways that might come back to bite you in the future?
I’ve found it helpful to think separately about three layers when using AI.
- The App - this is the thing you interact with
- The Model Runner - this is the code that actually uses the model to figure out the output from your input
- The Server - this is where the code runs.
Note, there isn’t really an agreed upon conceptual framework for talking about these things yet, as is evident if you try to search for “AI Stack” online like I did.
Notice I didn’t mention the model itself. That is because “a model” is a static set of instructions, it isn’t something that changes based on the input it gets, stores any data, or evolves over time.
This might be surprising, since there is so much discussion about machines learning and AI models ingesting all of our data. That is happening, but at the training phase, which is finished before the model is made available. Once the model is released it doesn’t incorporate any new data, though updated versions can be released.
Here’s a diagram showing who can see your data.
The App
There are thousands of apps that use AI, and they fall into three main categories: you might go to the app (think ChatGPT or Gemini), the AI might be integrated within software you already use (like Copilot within Office apps or Claude Code within VS Code), or the AI may do a task in the background (photo editing in Lightroom, AI playlists in Spotify).
The app layer works the same as non-AI apps: anyone can make one, and they decide how they use, store and share your data. Adobe Lightroom, for example, will have access to all the photos you upload, all the actions you take and any data you put into the app. Adobe decides what they do with that data, and should make it clear through their terms and conditions (though that’s a whole other can of worms).
The Model Runner
The model runner is the code that takes an input and runs it through the model to get an output. This is a non-trivial engineering task which means that almost no one writes their own. There are proprietary and open-source model runners, though you cannot use an open-source model runner on proprietary models. Most often, AI companies create proprietary models and proprietary code to run them. They can write extra code that grabs your data. What they do with it, again, should be in their terms and conditions. For example, OpenAI releases the GPT models as proprietary and anyone who uses them needs to send their data to OpenAI’s model runner. Per OpenAI’s terms, on their consumer products, they collect and use your data to train future models unless you opt out. In theory, open source code could have the same tracking built in, but anyone can inspect it and edit it if they wanted to change the behaviour.
You may hear the model runner referred to as the runtime, inference engine, backend or the API (though technically the API is just the part of this that defines how you pass it data).
The Server
Of course, the code needs a computer to actually execute the instructions: the server. Most of the time, this processing is done on cloud servers. In fact, there are only a few companies that own and control the hardware and processing capacity that can handle this scale of computation (think Amazon Web Services, Microsoft Azure, Google Cloud).
Your data can’t be encrypted while being processed on someone’s server, even if it is encrypted in transit. That means the server owner technically can see anything running on their server, though what they could access would be unstructured, lacking context and harder to make sense of than at the previous two levels. While technically possible, contracts and laws prevent server providers from systematically extracting customer data and there is no evidence that it happens. However, the door is still slightly ajar for bad actors or government agencies to get one-off access.
Some examples
Let’s start with OpenAI. They have their own app (ChatGPT) that uses their own proprietary models and model runner. It runs on Microsoft, Amazon or Oracle servers. When you type something into ChatGPT, your data is exposed to OpenAI and could also live on the servers of three other companies.
Apps like Snapchat use other companies’ models and infrastructure to power their AI features. Snapchat’s AI uses OpenAI GPT and Google Gemini models and runners, which run on cloud servers used by OpenAI (see above) and Google. Your Snapchat data could end up with any of six different companies (Snapchat, OpenAI, Google, Microsoft, Oracle, Amazon).
Any of the companies that get access to your data may also sell it or use it to train future models. They are supposed to tell you what they do with it, but privacy policies often have vague wording and most people don’t read them anyway (which is a policy failure, not a consumer failure).
If you want to use AI but are worried about sharing your data here’s how to think about it.
- The app: this is what gets the richest and most usable data about you. Do you trust it and is it something you could build yourself?
- The model runner: Who is running the model and what are their terms? Does it make sense to use an open source model and runner instead?
- The server: Where will this be running? If you are concerned about your data being used for training new models or serving you ads then using commercial servers is likely safe, but there could be cases where you want a higher privacy bar than that.
The caution is largely the same as it was before AI: it’s still often the apps that are tracking your data. There is a difference though. The conversational mode of interacting with AI may coax more information out of you than traditional apps did. The loop between the app, the model and the memory store may also be able to connect the dots (correctly or incorrectly) in more sophisticated ways. I have used AI chats for various questions, though I was careful to only add the context needed (as is my instinct online). One day, the message mentioned my child by name. It became apparent that the app had been able to synthesize information into a family tree, stored in its memory. This isn’t the model learning about me, it is the app sending data through the model and storing the resulting inferences as memories.
Coding apps are given a lot of system access. I was using an AI coding assistant for a project I’m doing. The project involves storing a secret code in a file that is supposed to stay on my computer and can be used to encrypt information. The AI app ingested my file and sent it through the model, and by extension to the AI company. I only realized because it then output the secret code in the chat window. This time it wasn’t a big deal because it was a low-stakes learning project, and I caught it before anything was deployed, but it was a wake-up call.
The point is, although it is still the app layer where a lot of your data gets ingested, AI apps operate in new ways that can capture more than you may have previously expected.