DeepSeek open-sources inference framework with support for Nvidia and Huawei chips

DeepSeek released DeepInfer, the inference framework behind its large-scale model services, on October 20. The framework supports Nvidia GPUs and Huawei Ascend processors and provides capabilities including distributed inference, speculative decoding and structured output. DeepSeek said the release is intended to help developers and businesses improve throughput and latency while reducing computing costs and energy […]