vLLM on Google Cloud TPU: A Model Size vs Chip Cheat Sheet (With Interactive Tool)
Which TPU configuration fits your model, what tensor_parallel_size to set, and what it costs per hour
Keep reading with a 7-day free trial
Subscribe to The 180 to keep reading this post and get 7 days of free access to the full post archives.