At its Build virtual event for developers today, Snowflake announced a number of new capabilities in its platform that will consolidate AI tech from two recent acquisitions, and further deliver on a number of capabilities announced at its Snowflake Summit event in June.
More broadly speaking, Snowflake is reconciling the analytics, governance, security, extensibility, application, collaboration, generative AI, and machine learning (ML) capabilities across its platform. And that’s no small thing
Snowflake’s announcements fall into three buckets: data architecture, enterprise AI & ML, and things relevant to applications (including development, execution, and distribution). I will cover each of these areas in turn and, as comprehensively as I can, explain which technologies are generally available (GA), which are in public preview, and which in private preview.
It’s the Data, Dummy
In the data architecture pillar, Snowflake is formalizing support for the Polaris data catalog for Apache Iceberg files that it announced in June. Snowflake made good on its promise to open source Polaris, and the formal name for the technology is now Apache Polaris as a result. Those who desire a commercial, Snowflake-managed solution around Polaris can find it in the Snowflake Open Catalog, which is being released to general availability today. This ups the lakehouse ante for Snowflake and assures the catalog is both open and yet available with enterprise-league support.
Not to get pedantic, but Open Catalog should not be confused with Snowflake Horizon Catalog, which is the umbrella for all of Snowflake’s data and governance technologies. Open Catalog is integrated with Horizon, mind you, but Horizon gets several new capabilities on top that. These include the GA release of a new LPP (Leaked Password Protection) capability, which should help mitigate the sort of data breaches many Snowflake customers faced earlier this year; Trust Center Extensibility, soon to be released in private preview, which will allow third-party extensions to Snowflake’s Trust Center security assessment component; and, finally, the GA of Differential Privacy Policy. (According to the IEEE, differential privacy “guarantees that adversaries cannot discover an individual within [a] protected data set by comparing the data with other data sets.”)
Putting the AI Puzzle Together
Moving on to the AI and ML side, there’s a lot to cover. First off, Snowflake’s Document AI feature has gone into GA release, and multimodal AI models (which can work with images, audio and/or video, in addition to text) have been added to the library of large language models (LLMs) available in the Snowflake’s Cortex AI. Document AI was derived from Snowflake’s acquisition of Applica in 2022 and allows for natural language search over documents.
Another implementation of that technology, Cortex Search (which is generally available), previously stood side-by-side with Cortex Analyst (now in public preview), which provides natural language search over structured data by generating corresponding SQL queries. But now, Snowflake’s new Chat API (in private preview) will join the Cortex Search, Cortex Analyst and Cortex LLMs to create a unified REST API for natural language search across both structured and unstructured data.
Similarly, a brand-new technology called Snowflake Intelligence integrates those same technologies into a new “agentic” AI service that not only answers questions, but also takes actions, like updating data in Snowflake data warehouses and even making external API calls. Snowflake Intelligence can connect to unstructured data in SharePoint, Slack, Salesforce, and Google Workspace, and contextualize it with data from the structured tables in Snowflake. Additionally, it’s integrated with Horizon Catalog.
Moving on to technology derived from Snowflake’s other AI acquisition, of TruEra, announced in May of this year, Snowflake is announcing AI Observability for GenAI apps, as well as observability for ML models. AI observability monitors each question and answer in LLM Retrieval Augmented General (RAG) queries across more than 20 metrics to establish LLM guardrails and mitigate LLM hallucinations. The ML observability also monitors metric values, but this time for model performance and drift.
Want more? Snowflake is also launching the public preview of Cortex Knowledge Extensions, which are third-party marketplace chatbot plug-ins that add expertise in specific domains (based on corpora of documents/unstructured data), and the GA release of Snowflake Notebooks. Notebooks are also being enhanced to work with the Snowflake’s container-based runtime for ML, a capability that is now in public preview.
Applications and Marketplaces
The aforementioned ML runtime sits atop Snowpark Container Services, which Snowflake Native Apps can also now leverage. This allows such apps to run in a customer’s Snowflake account, rather than the vendor’s. This capability is now GA on Amazon Web Services (AWS) and public preview on Azure.
Finally, in the collaboration pillar, Snowflake is announcing that its Unistore hybrid transactional/analytical table capability is now GA on AWS. A new Internal Marketplace for enterprise sharing of data and AI products is now GA as well, and customers will now be able to share fine-tuned LLMs in Snowflake’s marketplaces, complementing the existing ability to share ML models.
Required Assembly
Snowflake’s platform had suffered from a certain amount of fragmentation, due both to its acquisitions of Streamlit, Neeva, and the aforementioned Applica and Truera, as well as the considerable velocity around its organic development efforts. This left certain components outside the main platform and many others in public and private preview. Today’s announcements really tie many of these pieces together, effectively defragmenting the stack.
Yes, a few new pieces introduce new private and public previews to the mix, but Snowflake is making many GA announcements and forging integrations between Cortex Search and Cortex Analyst; Open Catalog and Horizon Catalog; transactional and analytical data; Notebooks and Snowpark Container Services; and AI, ML, observability and Marketplaces. This adds important fit and finish and makes Snowflake’s AI Data Cloud a platform in substance as well as in name.
How will this reconciliation of parts into a whole position Snowflake against its competitors, including Databricks, Microsoft Fabric, Cloudera Data Platform and others? At the very least, it will keep them on their toes. And with Microsoft’s Ignite conference coming up next week in Chicago, we’ll get to see how one of these competitors measures up.
The post Snowflake Consolidates Platform, Expands AI appeared first on The New Stack.
Snowflake builds out Cortex AI, announces Snowflake Intelligence and Chat API, further delivers on Iceberg and Polaris, and pitches many GAs.