Part 4: Stochastic Computation needs Verifiable Computing
A credibly neutral system to verify inputs and outputs from AI-based systems.
AI is a new form of computing that we are just not used to - stochastic computation. Stochastic means that the execution of the software involves random variables and we cannot fully predict the output. Software up until now has been discrete. The execution follows a specific set of instructions and repeating the same input will yield the same output.
The stochastic nature of AI systems is what allows it to hallucinate and generate the next word, or the next pixel, that makes sense given the previous word, pixel, or prompting. However, as an end-user, we cannot truly know which next word or pixel the AI will choose (this is specific to LLMs). This process is what makes the AI feel like magic.
The model is a blackbox. We know our prompt that we give the model, we can approximate what is in the giant corpus it was trained on, but that’s about it. We don’t know how the model arrives at any piece of the output.
Though we can’t know exactly what the output from a stochastic process will produce or how it produces it, we can know what went into the process to begin with. We should also know which model we’re asking to generate results from. This is important, especially when there are thousands (potentially millions) of similar models out there.
And, when the outputs really matter, we should know which systems/models we are asking to generate those outputs. Which enterprises developed systems, which entities are providing access to those systems, which datasets were (or were not included in the training, and more. This is what verifiable computation does.
Verifiable Computation is a method that allows a client to outsource complex computations to another service, while efficiently verifying the correctness of the results without redoing the entire computation.
Verifiable computation can help us be sure we’re getting what we are intending. It’s ok that we can’t know how the model is producing the results we depend on. We just want guarantees that we not getting something else.
VAIL intends to use recent advances in cryptography to generate the guarantees necessary for verifiable computation for AI. Central to these cryptography advances is zero-knowledge proofs (ZKPs). ZKPs are advancing within academic settings and provide a path to general verifiable computation and integrity - AI/ML model training and inference are special forms of computation that can be supported by ZKP systems.
ZKPs can be used for generating fingerprints of models to uniquely identify them as well as use those fingerprints to generate proofs of computation performed by these models. The fingerprint would provide cryptographic proof that a specific model was trained with a specific set of data sources and embeddings. When the model is run to generate a prediction, the fingerprint can be used to create a proof of that computation at that specific time.
Furthermore, the proof of computation can be shared separately (to another process/application) to be verified by any third party.
The key to this being practical for VAIL is that the cost to verify proofs is much lower than the cost to prove. The verifier process does not need to replicate the entire computation that went into producing the fingerprint or the proof. Said more plainly, the verifier does not need to rebuild the model, run it with the same input, and compare the outputs to know if they are the same.
Architecture
There are two primary components to the infrastructure that would support VAIL:
Proving
In order to generate the guarantees for each model (eg. fingerprinting, inference signing, etc), the model creators and hosts need to generate zero knowledge proofs at a couple stages. First, when training is completed, the training data need to be bound to the model that was trained from them - this would be a unique fingerprint for that model at that specific time.
Once this fingerprint is generated, it would be shared publicly. This would not disclose the proprietary aspects of the model, it would only allow others to use this fingerprint to validate outputs generated from that specific model.
Second, when the model is run to generate a prediction, that computation requires an additional proof that ensures it was that model (and only that model) that ran the computation to output that result.
Verifying
When an end-user is interacting with a model and receiving predictions, they can download the accompanying proofs to validate that the model they are intending to use is the model that produced the output. The verifier would download the public fingerprint for the model and combine it with the proof generated to validate the result.
Verifying is far less costly than proving and so the verifying step can be embedded within different application environments. If the primary interface for the AI/ML model is in a web interface (like a chat window), the verifying can be done within the browser. If it is within a downloaded application (like a mobile app), the verifying can be done within the app. It is also possible to forward the verification step to a third party that is set up with the verifying software.
🚩 Limitations 🚩
Proof generation is currently much more costly and not practical for current state of the art models. It would not be possible to generate a proof for a model the size of GPT-4 within acceptable timeframes. Currently proofs can be generated for models the size of GPT-2. ZK Proving systems are rapidly improving to bring this cost down. This computation overhead will also come down as new AI-optimized chips enter the market. Additionally, specialized hardware for cryptographic functions are being developed to make proof generation faster, similar to the specialized hardware being developed specifically for AI training and inference.
It is unclear how the development environment for provable ML models will be different from general model development. It would need to be easy to generate “provable” models as it is already challenging to build and fine-tune models to achieve high levels of accuracy and performance. Fingerprinting and proving should be part of the core development process for AI/ML model builders.