Israel’s high tech sector needs more access to GPUs

The proliferation of digital technologies means the volume of data at play is projected to grow to more than 180 zettabytes by 2025. This is a major positive on the one hand, offering a treasure trove of data for AI to work with. However, tech stacks and data platforms in Israel and abroad need to be able to cope with this explosion in data. 

Most companies today are attempting to build their data infrastructure by piecing together highly complex stacks of disconnected tools and systems. This fragmented approach is often referred to as the “modern data stack.” It’s essentially just the “legacy data stack,” with upgraded tools and a fresh coat of paint, that’s been hosted in the cloud. Yet these legacy systems are struggling to keep pace with AI innovation. 

Tech accounts for 20% of Israel’s economic output and the country is widely regarded as one of the leading tech centers in the world. 

Yet according to Dror Bin, Chief Executive of the state-backed Israel Innovation Authority, the pace of progress is stalling due to infrastructure challenges. 

“When a high tech company or researcher wants to train a large model they have to buy time in the cloud (since), there is no local data center with a significant amount of GPUs (graphic processing units) that can train those models here,” he said at an AI conference reported on by Reuters. 

To help Israel’s 9,000 startups address this hurdle, Israel is looking to build a supercomputer, currently under tender. When established, the initiative will surely help to bolster Israel’s ability to remain competitive with its tech offerings on the global stage. 

However, a home-grown Israeli and NYC startup has found an immediate way to ease the data processing bottleneck that companies in Israel and the world over are facing. 

AI creating economic change 

Whether it’s labor productivity improvements or refined factory production processes, AI promises transformative results for a near-endless number of applications. 

PwC predicts the technology will deliver a $15.7tr contribution to the global economy by 2030, while Goldman Sachs believes these tangible gains could be realized as soon as 2027. 

Tech giant Nvidia, for example, saw the potential in Israel as a hub for AI innovation and houses its second-largest development center outside of America in the country, employing more than 3,300 workers. At the end of 2023 it announced its AI supercomputer “Israel-1” operationally ready two months ahead of schedule. Nvidia’s founder and CEO Jensen Huang said, “Israel is home to world-leading AI researchers and developers creating applications for the next wave of AI.” 

Yet bringing these sweeping economic gains to life won’t happen overnight. Whether it’s automation or analytics, the performance of AI technology hinges on the data it works with. 

Tel Aviv University (Photo Credit: Wikipedia)

Tel Aviv University has become a hub of collaborative research in big data, with 10% of all researchers on campus are involved in data science, divided into two main groups: those who invent the methods and those who apply them. 

Ami Gal, an alumnus of Tel Aviv University, is one of the Israeli founders who helped to drive this progress forward with SQream, a data acceleration platform and compression engine first founded in 2010. 

The startup’s name comes from its original focus: Started by Gal along with Razi Shoshani (CTO) and Kostya Varakin (no longer with the company), the initial idea was to focus on SQL database analytics. However, they quickly realized the potential of GPUs back then to create faster and more efficient data queries for business intelligence purposes. 

The solution’s architecture contributes to its efficiency, as data does not need to be moved at any point during the preparation cycle. SQream Blue, the company’s data lakehouse solution, directly accesses this in open-standard formats at the customer’s low-cost cloud storage to maintain privacy and ownership, preserving a single source and eliminating the need for data duplication. 

GPUs in high demand due to costs spiraling due to AI   

Gal was in attendance at AI Israel 2024 this week, where he commented, “people understand the value of AI in their own ChatGPT console. But enterprises are getting the holy grail of AI insights.” 

However, enterprise organizations also have a costly challenge ahead to unlock this value. In its State of Big Data Analytics 2024 report the company surveyed 300 enterprise organizations and found 98% had experienced ML project failure in 2023, in part due to growing costs. To address this the company’s technology, according to a TPC Express Big Bench test, was able to handle 30TB of data three times faster than Databricks, and at a third of the price. Its technology’s pace was equivalent to reading every cataloged book in the US Library of Congress in under an hour, and then buying them all for less than $25. 

At the same time, not all Israeli companies are able to access GPUs, which is why these solutions from home-grown startups will be so important. Experts say that as of last year, there is still a global shortage of graphics processing units. 

Said Reuters in a report last quarter, this is also impacting global enterprises, with OpenAI keen to transition to in-house chip manufacturing following a global shortage of GPUs necessary to support its models. 

Helping Israel deliver on AI

AI promises transformative economic growth. While we’ve seen the power of direct AI applications, data architectures are in need of equally innovative solutions to support results. 

Israel’s high-tech sector is globally competitive. Yet current demand for access to data centers to train models could see progress slow due to costs and scarcity of GPUs. Leaders should pay attention here, particularly in Israel, which boasts 73 generative AI firms, the third-highest globally.

Concluded Dror Bin of Israel Innovation Authority at the AI conference, “Our goal is to make sure and to secure that Israel sustains its leadership, ranking and position in the AI race in the world.”