Join top executives in San Francisco on July 11-12 to hear how leaders are integrating and optimizing AI investments for success. Learn more
With the risks of hallucinations, leakage of private data, and regulatory compliance facing AI, there’s a growing chorus of experts and vendors saying there’s a clear need for some form of protection.
One such organization now building technology to protect against AI data risks is based in New York City Arthur AI. Founded in 2018, the company has raised more than $60 million to date, largely to fund machine learning monitoring and observation technology. Among the companies Arthur AI claims as clients are three of the top five US banks, human, John Deere and the United States Department of Defense (DoD).
Arthur AI takes its name from Arthur Samuel, who is largely credited with coining the term “machine learning” in 1959 and helping develop some of the earliest recorded models.
Arthur AI is now taking its AI observability one step further with today’s launch of Arthur Shield, which is essentially a firewall for AI data. With Arthur Shield, organizations can deploy a firewall located in front of large language models (LLMs) to monitor both incoming and outgoing data for potential risks and policy violations.
“There are a number of attack vectors and potential issues, such as data leakage, that are huge problems and are roadblocks to actually deploying LLMs,” Adam Wenchel, the co-founder and CEO of Arthur AI, told VentureBeat. “We have customers who are basically falling all over themselves putting in LLMs, but they’re stuck now and they’re using this, they’re going to use this product to break free.”
Do organizations need AI guardrails or an AI firewall?
The challenge of providing some form of protection against potentially risky generative AI outputs is one that multiple vendors are trying to solve.
>>Follow VentureBeat’s ongoing generative AI coverage<
Nvidia recently announced its NeMo Guardrails technology, which provides a policy language to help protect LLMs from leaking sensitive data or hallucinating incorrect answers. Wenchel noted that while guardrails are interesting from his perspective, they are aimed more at developers.
Instead, he said, Arthur AI wants to differentiate itself from Arthur Shield by specifically providing a tool designed for organizations to help prevent attacks in the real world. The technology also benefits from the observability that comes from Arthur’s ML monitoring platform to help provide a continuous feedback loop to improve firewall effectiveness.
How Arthur Shield works to minimize LLM risk
In the network world, a firewall is a proven technology that filters data packets in and out of a network.
It’s the same basic approach that Arthur Shield takes, except prompts come into an LLM and data comes out. Wenchel noted that some of the prompts used with LLMs today can be quite complicated. Prompts can contain user and database inputs, as well as sideloading embeds.
“So you take all this different data, string it together, enter it into the LLM prompt, and then get a response,” Wenchel said. “In addition, there are some areas where you can make the model make things up and hallucinate, and if you maliciously create a prompt, you can make it return highly sensitive data.”
Arthur Shield offers a set of pre-built filters that learn continuously and can also be customized. Those filters are designed to prevent known risks, such as potentially sensitive or toxic data, from being entered into or executed by an LLM.
“We have a great research department, and they’ve really pioneered the field of applying LLMs to evaluate the output of LLMs,” said Wenchel. “If you increase the sophistication of the core system, you must upgrade the sophistication of the associated monitoring.”
