OpenAI buying AI security startup Promptfoo to safeguard AI agents

The ChatGPT maker is set to announce Monday that it will integrate Promptfoo’s technology into OpenAI Frontier, a recently launched platform that allows organizations to build and manage AI agents

• OpenAI acquires Promptfoo to boost AI security for enterprises
• Promptfoo's tech will be integrated into OpenAI Frontier platform
• Frontier adds automated security testing and red-teaming features

Did our AI summary help?

OpenAI has agreed to buy Promptfoo, a startup that enables large businesses to find and fix security issues in artificial intelligence models during their development, part of a push to help corporate customers reduce possible risks from deploying AI software.

The ChatGPT maker is set to announce Monday that it will integrate Promptfoo’s technology into OpenAI Frontier, a recently launched platform that allows organizations to build and manage AI agents. Frontier is meant to help ensure each agent — which OpenAI refers to as “AI coworkers” — has appropriate guardrails and data access. Terms of the acquisition weren’t disclosed.

OpenAI and its rivals are racing to develop more advanced AI agents that can field complex tasks on a user’s behalf, with limited need for human intervention. At the same time, the AI developer is working to convince a broader mix of businesses to pay for the technology by ensuring these products are efficient and safe.

“OpenAI acquiring Promptfoo signals a clear commitment to making enterprise AI not just powerful, but safe and reliable at scale,” said Ganesh Bell, managing director at Insight Partners, which led an $18.4 million funding round in the security startup last year.

Two-year-old Promptfoo makes open-source tools for testing the security of AI systems as well as for helping companies try to attack their own products in order to find vulnerabilities, a process known as red-teaming. The San Francisco-based startup counts roughly a quarter of Fortune 500 firms as customers.

Promptfoo is one of a host of startups working to develop cybersecurity products that use AI to guard against hackers even as bad actors turn to similar technology to probe for ways into critical networks. OpenAI, too, has moved to imbue its AI products and agents with security features.  Last week, the company introduced an AI agent meant to help security teams find and patch vulnerabilities in large databases, similar to a tool from rival Anthropic PBC.

As part of the new acquisition, OpenAI’s Frontier will get automated security testing and red-teaming features. The product will also get capabilities to help organizations monitor changes and track testing in order to keep up with risk and compliance needs. OpenAI said it will keep building out Promptfoo’s open-source work while adding the latter’s technology to Frontier.