DeepSeek Launches Open-Source Tool to Slash AI Operating Costs
DeepSeek is unveiling the software that powers its artificial intelligence models at scale, which could lead to a reduction in costs and an increase in accessibility for businesses looking to implement this technology. The service, known as DeepInfer, is a software solution employed by DeepSeek to enhance the speed of its models and optimise the utilisation of computing resources. “We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable,” the Hangzhou-headquartered Chinese AI company said on Tuesday on X. DeepInfer currently supports DeepSeek’s V4.1-Flash model; however, the company indicated that support for future systems will be forthcoming.
The release illuminates how DeepSeek manages to reduce the expenses associated with operating advanced AI, referred to as inference, providing insight into production methodologies that AI firms generally protect with great secrecy. It additionally provides a mechanism for cloud and hosting firms utilising DeepSeek’s models to implement those cost-reducing strategies independently. The new release reduces the barriers to entry for hosting DeepSeek’s models; however, it still necessitates computing power and expertise, stated Thibault Jaigu, chief executive officer of AI gateway metrics and router startup, Requesty. “DeepSeek is saying – anyone can host our models themselves, here’s how do you do it efficiently.”
The same model can incur significantly higher operational costs, contingent upon the software employed for its deployment. More efficient software has the potential to decrease the computing cost associated with each request and may also lead to a reduction in energy consumption. DeepInfer integrates various methodologies, including distributed inference, which allocates AI workloads across diverse hardware. This approach enables models to generate increased output in reduced timeframes while maximising chip performance. Many of the underlying techniques are widely utilised; however, DeepSeek has customised them to operate effectively with its models.
Reducing inferencing costs – the expenses associated with operating AI models – can enhance profit margins for AI hosting providers and may lead to lower prices for consumers. For enterprises, DeepInfer has the potential to reduce the expenses associated with operating DeepSeek’s models on their own infrastructure. Currently, there is a lack of independent assessment regarding the efficiency improvements attributed to DeepInfer. While DeepInfer primarily pertains to DeepSeek’s proprietary models, other developers may analyse the open-source software and modify its methodologies for their own inferencing systems.
