ChatGPT wowed us all, but it was actually the simplest possible demonstration of what a large language model (LLM) could do, contended Ed Anuff, chief product officer at DataStax, a company that offers a distributed cloud database built on the open source Apache Cassandra.
“It’s taking its previous response and the previous interactions that are called the history — it’s taking those and adding your new question as an additional prompt, and it’s bundling all that up into a context and shipping it to the LLM, and it keeps repeating that,” he said of ChatGPT. “When ChatGPT was first released, that’s simply all that it was doing. The results that we experienced were pretty cool, but from a computer science program standpoint, it was actually pretty simple.”
Retrieval Augmented Generation (RAG) is one way to supplement an LLM’s knowledge. He compared RAG to notecards that help you stay focused and factual when speaking about a topic.
RagStack: the idea is to offer a set of technologies, similar to what the LAMP stack did for web development, that can be used to create AI applications.
“Let’s go and retrieve these, these very accurate sources of knowledge that are retrieved through traditional database lookups,” he said. “In some cases — in a lot of cases — [you] use vector database lookups to get at things and to feed those into the LLM, and then LLM just uses its language facility to craft that response.”
RAG can work through different mechanisms, including simple methods such as search and more complex methods such as turning a question into database queries, he said. The results post-RAG are “grounded,” meaning the LLM results are more accurate because the LLM used specific, factual information supplied alongside a query rather than relying solely on its own training data, he explained. This approach enhances accuracy by avoiding speculative or incomplete answers, resulting in responses that are more aligned with the provided information and thus more reliable, he added.
“All of those result in behind the scenes, a bunch of information being gathered that is then fed with your original question into the LLM,” he said. “What the LLM does is — rather than going and relying on its own knowledge that it was trained on — it uses that information that was supplied to it, and then the LLM responds.”
Creating AI Apps With RAGStack
Recently, DataStax updated its offering to make RAG application development 100 times faster, the company announced at RAG++ in San Francisco. To do this, it’s using what it calls the RAGStack. The idea is to offer a set of technologies, similar to what the LAMP stack did for web development, that can be used to create AI applications.
To support its RagStack vision, the company also launched a hosted version of Langflow on the Astra Cloud Platform.
Langflow is an open source visual framework for building RAG applications. DataStax acquired Langflow in April. It uses a drag-and-drop GUI to create data flows and leverages LangChain for RAG functions, thus allowing developers to design, experiment with and test RAG and GenAI applications using various vector databases, embedding models and large language models (LLMs).
Basically, it makes LangChain easier for developers. It does this by abstracting away infrastructure concerns and integrating with multiple GenAI tools — such as OpenAI, Hugging Face, and others, according to the company’s press release.
It is available as a hosted version on DataStax’s Astra Cloud platform, enabling easier accessibility and usage for developers. DataStax’s hosted Langflow will allow developers to design, experiment with and test RAG and GenAI apps using any vector database, embedded model or LLM without having to install Langflow on their machine.
RagStack also leverages LangSmith, an enterprise DevOps platform for managing and monitoring LLM applications. At the same time, DataStax released version 1.0 of Langflow, which includes dozens of integrations with the top GenAI tools, according to the company’s blog post.
Langflow 1.0 allows developers to leverage LangSmith’s observability service to trace an application’s responses for more relevant, accurate LLM-based applications, according to the company. Astra DB environment details will be readily available in Langflow and users will be able to access Langflow via the Astra Portal. Usage will be free, the company added.
Vectorizing Data and Working With Unstructured.io
DataStax also highlighted Vectorize, a recent release that handles embedding generation directly at the database level. It takes whatever content you put into a database and generates the vector representation of it, Anuff explained.
“Vector representations are where we take this unstructured data and that unstructured data could be a block of text, could be an image, it could be whatever, but we take that and we generate an embedding. That embedding is a very long numerical representation that represents the semantic meaning of that content,” he said.
Those vectors go into a database, where the rows that are closest to each other have similar meaning, he explained. From a development standpoint, that saves time.
“What this allows me to do is get content that might be relevant from a semantic meaning standpoint to what you asked, rather than precise keywords,” he said. “It’s a very powerful tool in these settings where I want to retrieve content by meaning rather than specific keywords because you might never use the specific keywords, but the meaning may be very close.”
Finally, DataStax announced a partnership with Unstructured.io, which offers connectors that can get to the data sources and data formats and extract the relevant content in the right bite-sized chunks and feed that into the Astra DB Vector database, Anuff said. The partnership will allow developers to extract and transform complex data to be stored in Astra DB Vector for use in powering LLM-based applications, the company said in its press announcement.
“Users’ GenAI applications benefit from lightning-fast data ingestion through quick conversion of large data sets and common document types into vector data,” the company stated. “This new integration then enables these embeddings to be quickly written to Astra DB for highly relevant GenAI similarity searches. And, when managing very large datasets, users are able to convert that data into embeddings and write them to Astra DB in just minutes.”
The post DataStax Aims To Simplify Building AI Apps With RAGStack appeared first on The New Stack.
In April, DataStax acquired Langflow, an open source tool for RAG. It's now part of a tech stack that helps developers build AI applications.