Supported Platforms#

The platform requirement can vary depending on the configuration and deployment topology used for VSS and dependencies like VLM and LLM. Each section is for a specific GPU and is broken down into how VSS can be deployed for a given number of GPUs.

All GPUs listed here support remote deployments, where all the models are running elsewhere and VSS is connected through APIs endpoints.

The configurations listed here have been validated, however this is not an exhaustive list of the VLMs and LLMs, which can be deployed.

The minimum number of GPUs required for the LLM NIMs are provided here.

To customize the Helm deployment for various configurations, refer to Configuring GPU Allocation. For Docker Compose deployment, refer to Configuring GPU Allocation.

Note

The default Helm Chart deployment topology is configured for:

  • 8 x B200 (180 GB)

  • 8 x H100 (80 GB)

  • 8 x H200 (140 GB)

  • 8 x A100 (80 GB)

  • 8 x L40S (48 GB)

  • 8 x RTX PRO 6000 Blackwell SE (96 GB)

Full local deployment recipe on single GPU using non-default low memory modes and smaller LLMs are available. Refer to Helm Single GPU Deployment and Docker Compose Single GPU Deployment.

B200#

# of GPU

Supported Deployment Option

1

Single GPU deployment (Llama 3.1 8b low mem mode, Cosmos-Reason1 7b)

1+

Docker compose/Helm with remote endpoints (VLM and LLM)

1+

Docker compose/Helm with remote LLM

2+

Local Deployment (Llama 3.1 8b, Cosmos-Reason1 7b)

4+

Local Deployment (Llama 3.1 70b, Cosmos-Reason1 7b)

8

Default Helm Deployment (Llama 3.1 70b, Cosmos-Reason1 7b)

RTX PRO 6000 Blackwell SE#

# of GPU

Supported Deployment Option

1

Single GPU deployment (Llama 3.1 8b low mem mode, Cosmos-Reason1 7b)

1+

Docker compose/Helm with remote endpoints (VLM and LLM)

1+

Docker compose/Helm with remote LLM

2+

Local Deployment (Llama 3.1 8b, Cosmos-Reason1 7b)

4+

Local Deployment (Llama 3.1 70b, Cosmos-Reason1 7b)

8

Default Helm Deployment (Llama 3.1 70b, Cosmos-Reason1 7b)

H200#

# of GPU

Supported Deployment Option

1

Single GPU deployment (Llama 3.1 8b low mem mode, Cosmos-Reason1 7b)

1+

Docker compose/Helm with remote endpoints (VLM and LLM)

1+

Docker compose/Helm with remote LLM

2+

Local Deployment (Llama 3.1 8b, Cosmos-Reason1 7b)

4+

Local Deployment (Llama 3.1 70b, Cosmos-Reason1 7b)

8

Default Helm Deployment (Llama 3.1 70b, Cosmos-Reason1 7b)

H100#

# of GPU

Supported Deployment Option

1

Single GPU deployment (Llama 3.1 8b low mem mode, Cosmos-Reason1 7b)

1+

Docker compose/Helm with remote endpoints (VLM and LLM)

1+

Docker compose/Helm with remote LLM

2+

Local Deployment (Llama 3.1 8b, Cosmos-Reason1 7b)

4+

Local Deployment (Llama 3.1 70b, Cosmos-Reason1 7b)

8

Default Helm Deployment (Llama 3.1 70b, Cosmos-Reason1 7b)

A100 (80 GB)#

# of GPU

Supported Deployment Option

1

Single GPU deployment (Llama 3.1 8b low mem mode, Cosmos-Reason1 7b)

1+

Docker compose/Helm with remote endpoints (VLM and LLM)

2+

Docker compose/Helm with remote LLM

2+

Local Deployment (Llama 3.1 8b, Cosmos-Reason1 7b)

4+

Local Deployment (Llama 3.1 70b, Cosmos-Reason1 7b)

8

Default Helm Deployment (Llama 3.1 70b, Cosmos-Reason1 7b)

A100 (40 gb)#

# of GPU

Supported Deployment Option

1+

Docker compose or Helm with remote endpoints (VLM and LLM)

4+

Docker compose or Helm with remote LLM

6+

Local Deployment (Llama 3.1 8b, Cosmos-Reason1 7b)

L40S#

# of GPU

Supported Deployment Option

1+

Docker compose or Helm with remote endpoints (VLM and LLM)

1+

Docker compose or Helm with remote LLM (Cosmos-Reason1 7b)

4+

Local Deployment (Llama 3.1 8b, Cosmos-Reason1 7b)

6+

Local Deployment (Llama 3.1 70b, Cosmos-Reason1 7b)

8

Default Helm Deployment (Llama 3.1 70b, Cosmos-Reason1 7b)

A6000#

# of GPU

Supported Deployment Option

1+

Docker compose or Helm with remote endpoints (VLM and LLM)

NVIDIA Jetson Thor#

Supported Deployment Option

# of GPU

Docker compose with remote LLM (Cosmos-Reason1 7b)

1

Docker compose with remote endpoints (VLM and LLM)

1
On this page