vLLM · 官方博客

呵

Introducing vLLM TorchTPU

Introducing vLLM TorchTPU

二〇二六年十月二十日 · 英文原文
摘要

vllm-torchtpu is a new vLLM backend for Google Cloud TPUs built on TorchTPU, with Pallas kernels, multi-host parallelism, KV cache offloading, and disaggregated serving.

vllm-torchtpu is a new vLLM backend for Google Cloud TPUs built on TorchTPU, with Pallas kernels, multi-host parallelism, KV cache offloading, and disaggregated serving.

译自 vLLM · 官方博客 · 录于 二〇二六年十月二十日