In partnership with

Enterprise AI's Hardest Problem Lives in the Data Layer

At Supermicro's Open Storage Summit Reception on June 24 in Mountain View, we helped bring together leaders from across the infrastructure stack to tackle one question head-on: What is the single biggest factor preventing organizations from scaling AI into core business processes?

Rob Strechay (theCUBE) moderated, alongside William Li (Supermicro), Kurt Kuchein (Hammerspace), and Bill Miller (MinIO), digging into the operational realities that still separate AI pilots from production deployments. The conversation reinforced something we've been tracking closely across the community; the scaling challenge has moved well past compute and models. The hardest problems now live in the data layer.

The POC Trap

Bill Miller framed the core tension clearly. In a proof of concept, the data is curated, the environment is controlled, and the use case is narrow. In production, none of that holds. Enterprise data is fragmented across clouds and on-prem systems, riddled with stale records, and governed by access policies that were never designed for AI workloads. A demo that dazzles a boardroom can collapse the moment it meets real-world complexity.

This is a pattern we see playing out across the infrastructure landscape and the panel treated it as a structural problem demanding architectural solutions.

Three Walls Between Pilot and Production

Miller broke the scaling challenge into three concrete barriers. First: access to all relevant data, particularly the sensitive on-prem "crown jewel" datasets that organizations are most reluctant to expose to AI pipelines. Second: the explosive growth of KV cache requirements at production scale. Third: the massive gap between internal deployments serving hundreds of employees and external deployments handling millions of transactions.

These are the operational categories infrastructure architects and enterprise leaders need to plan around now and the kind of specific, execution-level detail that drove this conversation.

Token Value Over Token Volume

One of the strongest threads to emerge was the distinction between how much data a model consumes and whether that data is actually useful. The panel argued that enterprises should focus less on throughput and more on whether models are ingesting relevant, current, authorized information. When an LLM treats an archived project plan from 2019 with the same weight as a live production dataset, output quality degrades and user trust follows.

That trust problem is where technical debt meets change management. Hallucinations and unreliable outputs stall adoption across the organization. Once confidence breaks, getting business units to rely on AI-driven workflows becomes an uphill fight.

The Unglamorous Infrastructure Layer

Kuchein and Li both pointed to a reality that rarely makes keynote slides: while the compute side of the AI stack has become increasingly streamlined, the storage and data management layer remains fragmented. Enterprises are still stitching together compliance tools, classification engines, storage orchestrators, and governance frameworks. The next wave of enterprise AI value may come from making that data layer manageable and coherent.

Open protocols figured prominently. Iceberg as a unified catalog layer, NFS for interoperability, and open-weight models were all positioned as mechanisms to reduce lock-in and bridge the structured-unstructured divide. Lakehouses combining the strengths of data warehouses and data lakes emerged as the architectural pattern for AI-ready enterprises.

Where the Conversation Goes From Here

This panel reinforced a thesis we keep coming back to in every conversation with infrastructure operators: enterprise AI adoption stalls when the data layer is incomplete, untrusted, or too complex to operationalize. Fix the data foundation governance, access, quality, hybrid reach and the rest of the stack becomes dramatically more usable.

The organizations that figure this out first will deploy AI that their people actually trust enough to use. That's where this conversation is headed, and we'll keep driving it.

The infrastructure underneath has to keep up.
That's what AI INFRA SUMMIT exists to support.

See you at AIS 6 December 4th, San Francisco.
Secure your spot with Super Early Bird Tickets below

// UPCOMING EVENTS

Bay Area Startups Collectively Secured $21B+ in June MTD

Silicon Valley startup funding activity spiked this week to $5.4B, with twelve megadeals – two of them each one billion or more – BaseTen with a $1B Series F and AppsFlyer with a $1B Series E.

Exits, M&A: M&A was the big story this week, with three billion-dollar plus acquisitions. $7B for Synaptics, $10.9B for Apogee Therapeutics and $4B for Modular, plus several smaller ones came to more than $22B in total, the most for any week in 2026.

For startups raising capital: Don't use lists, find investors in companies like yours with LinkSV. Stay on top of who's raising, who's closing and who's investing with the Pulse of the Valley weekday newsletter. Founders get the newsletter, database and alerts for just $7/month ($50 value). Check it out and sign up here.

Follow LinkSV on LinkedIn to stay on top of SV funding intelligence, and the companies, investors and executives impacting the startup ecosystem.

Early Stage:

• Mirendil closed a $200M Seed, an AI research lab building self-accelerating systems that turn compute into scientific and engineering breakthroughs.

• Upscale AI, Inc closed a $190M Series A, a high-performance AI networking company accelerating AI democratization through open-standard, full-stack, turnkey solutions.

• Sail Research closed a $80M Series A, the infrastructure company purpose-built for long-horizon AI agents.

• Coval closed a $28M Series A, accelerates AI agent development with automated testing for chat, voice, and other objective-oriented systems.

• Bolt Graphics closed an undisclosed amount Seed, a semiconductor startup building the fastest and most efficient graphics processors.

Growth Stage:

• BaseTen closed a $1.5B Series F, builds systems software that runs the entire workload for AI applications—from GPUs and auto-scaling to observability, billing & developer tools.

• Groq closed a $650M Series F, operates the inference cloud infrastructure that runs real-time AI.

• AirWallex closed a $320M Series H,  global financial platform, building the future of global banking for a borderless, real-time, intelligent economy.

• Assort Health closed a $120M Series C, we manage the entire patient journey for providers, from call center automation and form intake to referrals, document processing, outreach, and payments.

• Patronus AI closed a $50M Series B, a frontier lab developing simulation research and infrastructure to accelerate progress toward human-aligned AGI. We are training the first world models to simulate digital workflows.

  

When enterprises move AI from pilot to production, the conversation shifts fast from models and frameworks to hardware, logistics, and capital. That's where Avnet operates.

Founded in 1921 and now a $26.5 billion global technology distributor (Nasdaq: AVT), Avnet has spent over a century connecting enterprises with the components and infrastructure they need to build at scale. They were Intel's first distributor in 1973. Today, that same distribution muscle is pointed squarely at the AI infrastructure buildout.

For organizations deploying GPU clusters, high-density compute, and AI-optimized storage, Avnet solves the problems that sit between "we have a vendor" and "we have a running data center." Inventory management across global supply chains. Financing options that make large-scale hardware procurement possible without crushing capital budgets. Technical integration support that bridges the gap between component selection and production-ready deployment.

As a co-sponsor of Supermicro's Open Storage Summit Reception where Avnet delivered the keynote ahead of the enterprise AI panel their presence underscored a critical reality: the AI infrastructure era needs more than chip makers and software vendors. It needs distributors who can move hardware at scale, manage complex procurement cycles, and help enterprises operationalize their AI ambitions.

Avnet is that bridge and they've been building it for over 100 years.

Your Feedback Matters!

Your feedback is crucial in helping us refine our content and maintain the newsletter's value for you and your fellow readers. We welcome your suggestions on how we can improve our offering. [email protected] 

Logan Lemery
Head of Content // Team Ignite

Your membership list is full. Your calendar isn't.

You have hundreds of members needing tune-ups twice a year.

Scheduling them means multiple calls and weeks of manual outreach that never fully gets done.

The ones who don't get reached fall through the cracks. Those are members and recurring revenue quietly walking out the door.

Podium's AI Membership Coordinator texts every member, books the appointment, follows up automatically, and handles renewals.

It works your entire list proactively so your slow season fills before it starts.

No manual outreach. No dropped members. No empty shoulder season.

Make the most of your membership list.

Fill My Seasonal Calendar