Solving the Data Center Problem with AI Innovation

Solving AI’s Data-Centre Problem, from Neural Weights to Cooling Pipes

What if progress in artificial intelligence were measured not by the size of the model, but by the physical resources needed to complete a useful task reliably?

Artificial intelligence has developed an extraordinary appetite for infrastructure. Models run on processors; processors consume electricity and produce heat; and data centres need electrical equipment and cooling to sustain them. The International Energy Agency (IEA) estimates that global data-centre electricity use was 485 terawatt-hours (TWh) in 2025 and projects 950 TWh by 2030 in its central scenario. Within that total, electricity demand from AI-focused facilities is projected to triple.[1]

These are projections, not certainties, and they cover all data-centre operations rather than AI alone. They nevertheless expose a serious engineering challenge. More efficient hardware can reduce energy per request, while rising demand for longer reasoning, video and autonomous agents can push total consumption upwards.

My proposal is Shrinking Intelligence: co-design models, orchestration, chips, serving software and cooling so that each verified task uses the least resources compatible with the required quality, security and speed. This is not a claim that small or local models always win. It is a hypothesis that can be measured across seven pathways.

1. Do not invoke intelligence where software will do

A deterministic calculation, a database query or a simple validation rule often needs no language model. Retrieval-augmented generation (RAG) can provide current documents to a model without embedding every changing fact in its weights.[2] Caching and careful context selection may eliminate duplicate requests or reduce unnecessary processing.

There is no free lunch: retrieval, storage, cache invalidation and verification also have costs. The objective is fewer unnecessaryoperations, not blindly fewer tokens. A short answer that is wrong and must be repeated is not efficient.

2. Compress models without confusing storage with energy

Knowledge distillation transfers selected behaviours into smaller networks.[3] Quantisation stores weights with fewer bits; research such as activation-aware weight quantisation shows how to preserve important parameters while compressing a model.[4] Sparsity and mixture-of-experts architectures can reduce the computation activated for a request, although their real benefits depend on implementation.

PrismML’s October 2026 Bonsai 2 announcement reports three ternary models, with weights drawn from −1, 0 and +1. The 2-billion-, 4-billion- and 9-billion-parameter versions have reported weight footprints of 0.43GB, 0.95GB and 2GB, respectively.[5] These figures are vendor-reported model-weight storage, not complete runtime memory, power consumption or independently verified task quality.

Vincenzo Fornaro’s open-source Colibrì addresses a different bottleneck: it arranges mixture-of-experts components across fast memory, RAM and storage, streaming less frequently used components as required.[6] Such an arrangement can make a model accessible with less fast memory, but slow storage access can hurt latency and energy. Smaller files, less RAM and fewer joules are three separate claims.

3. Repair behaviour with carefully tested adaptations

Conventional fine-tuning need not update every parameter. LoRA, for example, freezes the base model and learns smaller low-rank adaptations.[7] A more experimental approach is direct weight-space programming: inspecting a network and changing specific weights.

In a research report published on 2 November 2026, Rikiya Takehi and colleagues describe editing 131,072 weights, approximately 0.016% of Qwen3.5-0.8B, after identifying repetitive reasoning loops. Under their specified evaluation settings, MATH-500 rose from 7.9% to 48.1%.[8] The researchers emphasise that the edit largely removed a failure mode; it did not demonstrate the creation of mathematical knowledge from nothing. A similar attempt on Gemma 3 270M, which lacked the same looping problem, produced no established improvement.

This is an experimental research report, not a peer-reviewed general-purpose weight-repair method. If such techniques generalise, small, model-version-specific patches might sometimes be cheaper to distribute than full replacements. They would still require provenance checks, regression testing and rollback mechanisms. Tiny weight edits do not automatically make a model physically smaller.

4. Make specialised intelligence downloadable—and orchestrate it

A single model need not handle every task. Cisco’s Antares-350M and Antares-1B, released as open-weight models in July 2026, target a narrow problem: locating possible vulnerabilities within code repositories.[9] Their benchmarks are Cisco-reported, and localisation is not equivalent to proving that code is secure. Yet the approach shows why domain-specific models deserve attention.

A downloadable library could package models for document extraction, financial analysis, code inspection and mathematical assistance. Each package would include compatible weights or adapters, a model card, hardware requirements, licensing, benchmarks, signed manifests and a clearly stated scope.

A lightweight orchestrator would choose between deterministic software, retrieval, a small specialist or a larger model. RouteLLM, evaluated at ICLR 2025, demonstrates that learned routing can reduce model-serving costs while preserving benchmark quality in the settings studied.[10] It does not prove that every multi-model workflow uses less energy.

Fahm.uk offers a concrete proposed testbed. Its public site describes an Intent → Sense → Monitor → Speak workflow, covering goal framing, evidence gathering, checking and response.[11] A downloadable specialist-model library and local-first router would be proposed extensions, not capabilities proven to exist on the public site. The orchestrator could escalate difficult cases to cloud models with permission, while keeping source checking and evaluation visible.

5. Optimise the inference engine, not only the model

Even when a larger model is needed, its execution can improve. FlashAttention reduces expensive reads and writes between GPU high-bandwidth memory and on-chip memory.[12] PagedAttention, implemented in vLLM, manages the growing key-value cache more efficiently, supporting better serving throughput.[13] Speculative decoding can let a smaller model propose tokens for verification by a larger model; the original algorithm preserves the target output distribution under its stated conditions.[14]

Batching, caching and better scheduling can also improve utilisation. But throughput and latency are not direct measures of electricity. A faster response may consume more power at a particular instant, and an extra model used for speculation may add overhead. Energy must be metered for the whole request.

6. Match the chip to the workload

Special-purpose neural processors can make some edge inference more practical. DEEPX specifies 25 INT8 TOPS for its DX-M1. Its own materials vary by variant and revision: a DX-M1 M.2 module listing specifies 2–5 W, while other documentation describes around 3 W typical and 5 W maximum for supported models.[15] The company’s July 2026 announcement describes running small language models on-device.[16]

These are vendor specifications and demonstrations. A peak TOPS rating and a power figure must not be assumed to occur simultaneously for every language-model workload, and the host device also consumes electricity. Local execution may improve privacy or connectivity, but only direct measurement can show whether it uses less energy than a highly utilised cloud service.

7. Engineer the heat that remains

Centralised facilities remain important for frontier training, high-throughput services and scientific computing. Their thermal systems are another optimisation opportunity. In a peer-reviewed 2026 study, Behnood Bazmi and colleagues manufactured topology-optimised pure-copper cold plates using electrochemical additive manufacturing. They measured up to 32% lower thermal resistance at a fixed coolant flow, or up to 68% lower pressure drop at equal thermal resistance, against comparison pin-fin designs.[17]

These results come from different comparison conditions. Neither number is a measured reduction in total data-centre cooling costs. The paper also models an optimistic facility scenario; it must not be presented as a deployed commercial outcome.

MIT-founded startup Ferveret explores another method: immersion cooling with controlled boiling and small bubbles. MIT News reports company-linked early tests indicating 15% better computational-power efficiency than the liquid-cooling baselines studied.[18] This is promising, not independently established across commercial facilities.

Cooling decisions also affect water, capital expenditure, refrigerant choice and local power grids. The IEA’s October 2026 review explicitly considers electricity-system integration, prices, emissions and water implications.[19]

8. A test that could disprove the idea

A serious Fahm.uk or similar SLM pilot should compare the same held-out tasks under three configurations: (A) a cloud frontier model; (B) locally downloaded specialist models with orchestration; and (C) local specialists with permissioned cloud fallback. Test document extraction, quantitative reasoning, security triage and deliberately difficult or misleading queries. Publish the benchmark set, evaluation rubric, hardware configuration, uncertainty estimates and failure cases, subject to legitimate confidentiality and security constraints.

Pre-register minimum quality and safety thresholds. Measure wall-plug electricity per verified successful task for local hardware, request-attributable cloud energy where reasonably obtainable, latency, retries, human review, memory, network transfers, hardware depreciation and financial cost. Report the limitations of allocating shared-cloud energy to individual requests. Calculate carbon intensity using a defined functional unit and relevant operational and embodied emissions, consistent with the ISO/IEC 21031:2024 Software Carbon Intensity specification.[20] Measure water separately; SCI is not a water-use standard.

If a small model misses the quality threshold, use a larger one. If an efficient cloud system outperforms a local deployment, use the cloud. The goal is not to make AI small at any price; it is to minimise the full cost of dependable intelligence.

The next frontier in AI may therefore be less about how large a machine we can build than how little machine a trustworthy answer actually requires.

References

[1] International Energy Agency. (2026, April 16). Key Questions on Energy and AI, executive summary. https://www.iea.org/reports/key-questions-on-energy-and-ai/executive-summary

[2] Lewis, P., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems, 33. https://proceedings.neurips.cc/paper_files/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html

[3] Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv:1503.02531. https://arxiv.org/abs/1503.02531

[4] Lin, J., et al. (2024). AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration. Proceedings of Machine Learning and Systems, 6. https://proceedings.mlsys.org/paper_files/paper/2024/hash/42a452cbafa9dd64e9ba4aa95cc1ef21-Abstract-Conference.html

[5] PrismML. (2026, October 15). Introducing Ternary Bonsai 2 2B, 4B, and 9B: Bringing Local Agents to Everyday Devices. Company technical announcement. https://prismml.com/news/bonsai-2-family

[6] Fornaro, V. (JustVugg). (2026). Colibrì: A hierarchical inference engine for frontier MoE models. Open-source software and documentation. https://github.com/JustVugg/colibri

[7] Hu, E. J., et al. (2022). LoRA: Low-Rank Adaptation of Large Language Models. International Conference on Learning Representations. https://arxiv.org/abs/2106.09685

[8] Takehi, R., Teoh, J., Zhang, A., & Khattab, O. (2026, November 2). Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights. Preliminary research report. https://rikiyat.github.io/blog/weight-space-programming/

[9] Karbasi, A., with Vijay, S., Priyanshu, A., Chapoteau, D., Goldblatt, A., Majd, K., and other collaborators. (2026, July 21). Introducing Antares: Highly Efficient Open Weight AI Models for Vulnerability Localization. Cisco technical announcement. https://blogs.cisco.com/ai/introducing-antares-the-most-efficient-open-weight-ai-models-for-vulnerability-localization

[10] Ong, I., et al. (2025). RouteLLM: Learning to Route LLMs from Preference Data. International Conference on Learning Representations. https://proceedings.iclr.cc/paper_files/paper/2025/hash/5503a7c69d48a2f86fc00b3dc09de686-Abstract-Conference.html

[11] Fahm. (accessed 2026, November 10). Fahm: It will not guess. Public product description. https://fahm.uk/

[12] Dao, T., Fu, D. Y., Ermon, S., Rudra, A., & Ré, C. (2022). FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. Advances in Neural Information Processing Systems, 35. https://arxiv.org/abs/2205.14135

[13] Kwon, W., et al. (2023). Efficient Memory Management for Large Language Model Serving with PagedAttention. Proceedings of the 29th Symposium on Operating Systems Principles, 611–626. https://doi.org/10.1145/3600006.3613165

[14] Leviathan, Y., Kalman, M., & Matias, Y. (2023). Fast Inference from Transformers via Speculative Decoding. Proceedings of Machine Learning Research, 202, 19274–19286. https://proceedings.mlr.press/v202/leviathan23a.html

[15] DEEPX. (accessed 2026, November 10). DX-M1 product information and DX TechBridge Program: DX-M1 M.2 module specifications. Vendor documentation. https://deepx.ai/products/dx-m1/ ; https://deepx.ai/buy-now/dx-techbridge-kit/

[16] DEEPX. (2026, July 9). DX-M1 now runs SLMs fully on-device. Company announcement. https://deepx.ai/dx-m1-now-runs-slms-fully-on-device/

[17] Bazmi, B., Lad, A. A., Shatskiy, E., Garimella, V. S., Luo, K., et al. (2026). Ultra-high-performance cold plate development through topology optimization and electrochemical additive manufacturing. Cell Reports Physical Science, 7(5), 103272. https://doi.org/10.1016/j.xcrp.2026.103272

[18] Winn, Z. (2026, June 10). Startup’s nuclear-inspired cooling system could make data centers more sustainable. MIT News, reprinted by MIT Energy Initiative. https://energy.mit.edu/news-events/news/startups-nuclear-inspired-cooling-system-could-make-data-centers-more-sustainable

[19] International Energy Agency. (2026, October 14). Powering Artificial Intelligence: Strategies for Integrating Data Centres into Electricity Systems. https://www.iea.org/reports/powering-artificial-intelligence

[20] ISO/IEC. (2024). ISO/IEC 21031:2024: Information technology — Software Carbon Intensity (SCI) specification. https://www.iso.org/standard/86612.html

[21] Velocity Room discussion – SLMs running on CPUs https://m.youtube.com/watch?v=CySLp5ANLIo&pp=ygUgVmVsb2NpdHkgQUkgZGlzY3Vzc2lvbiBTTE0gQ1BVcyA%3D&ra=m

Evidence note (10 November 2026): Figures attributed to manufacturers are specifications or reported benchmarks, not automatically independent measurements. The 2026 weight-editing results are preliminary; the cold-plate percentages are experimentally measured under different comparison conditions; the Ferveret figure is from early company-linked tests reported by MIT; and the proposed Fahm model library is not verified as a deployed product. This article does not claim that local AI is universally more energy-efficient than cloud AI.