NVIDIA A100 SXM
Ampere Architecture
Active
Launched May 2020
Core Specifications
VendorNVIDIA
ArchitectureAmpere
Form FactorSXM
VRAM80 GB
Memory Bandwidth2,039 GB/s
TDP400 W
Compute Performance
| Precision | TFLOPs |
|---|---|
| FP32 | 19.5 |
| FP16 | 312 |
| BF16 | 312 |
Performance Benchmarks
image gen
| Configuration | Precision | Performance | Source |
|---|---|---|---|
| Stable Diffusion XL, 1024x1024, 50 steps | — | 1.625 images_per_second | View |
llm inference
| Configuration | Precision | Performance | Source |
|---|---|---|---|
| LLaMA 70B, batch_size=1 | — | 5,200 tokens_per_second | View |
Pricing
Hardware Purchase (CAPEX)
| Type | Price (USD) | Region | As of |
|---|---|---|---|
| Street Price | $15,000 | Global | Oct 2024 |
Cloud Rental (OPEX)
| Provider | Instance Type | Price per Hour | Region | As of |
|---|---|---|---|---|
| Salad | 1x A100 (40GB PCIe) | $0.40/hr | Global | Oct 2024 |
| Azure | Standard_ND96asr_v4 (1x A100 80GB) | $3.67/hr | East US | Oct 2024 |
| Google Cloud | a2-highgpu-1g (1x A100 80GB) | $3.67/hr | us-central1 | Oct 2024 |
| AWS | p4d.24xlarge (8x A100) | $32.77/hr | us-east-1 | Oct 2024 |
| AWS | p4d.xlarge (1x A100) | $4.05/hr | us-east-1 | Oct 2024 |