will constellation be able to protect against attacks like LeftoverLocals?

[hpvd]

There is a new vulnerability called LeftoverLocals (CVE-2023-4969):
A malicious actor (an attacker) can listen to another user’s (the victim) LLM responses on a multi-tenant GPU machine and extract it it with only 10 lines of code.

A detailed description can be found here:
https://blog.trailofbits.com/2024/01/16/leftoverlocals-listening-to-llm-responses-through-leaked-gpu-local-memory/

and a German sum-up is available on https://www.golem.de/news/apple-amd-qualcomm-gpus-mehrerer-hersteller-anfaellig-fuer-datenklau-2401-181263.html

=> will constellation be able to protect against attacks like this

• when constellation is ready for GPU/NPU use and
• the supported brands are affected (e.g. this time nvidia seems not to be affected)?

[derpsteb]

Hello,

thank you for submitting this question.
As you have already pointed out, this is a hypothetical answer atm since Constellation does not support GPUs/NPUs, yet.

The attack setup for LeftoverLocals requires an attacker to launch workloads on the same physical device that the victim is running workloads on. In a traditional cloud deployment (no CC) there are two scenarios where this could arrise:

(a) The attacker controls the hypervisor. The attacker could temporarily remount the device to a different VM or send commands to the GPU directly. This would allow them to execute malicous kernels.
(b) The GPU attached to the VM is virtualized with a technology like Nvidia MIG [1].

Confidential computing, and with that Constellation, would protect you against scenario (a). However, this is only the case if the GPU also supports confidential computing features. The central point being that CPU and GPU establish an encrypted channel before workloads are started. To our knowledge this is currently only the case for Nvidia's H100 [2].

Scenario (b) needs two exploitable vulnerabilities in order to cause harm, even without CC. One exploit for a vulnerability in the virtualization technology (e.g. Nvidia MIG). Another one for LeftoverLocals.
When using an H100 GPU with CC mode an attacker would need to break a third layer, the CC mode. This is because virtualized H100 GPUs also have a trust boundry between each virtual GPU [2].

To summarize: the architecture of Constellation adds an additional layer of protection to the most likely attack scenarios, once it has H100 support. Which will come as soon as hardware is available.

Please let us know if you have more specific attack scenarios that you would like to discuss.

CPU-only inference/training is already possible on C11n. And would not be affected by something like LeftoverLocals.

[1] https://docs.nvidia.com/datacenter/tesla/mig-user-guide/index.html
[2] https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/HCC-Whitepaper-v1.0.pdf

[hpvd]

Many thanks for this detailed answer and the included links!