OpenClaw is becoming a popular agent framework, but its performance at the enterprise level remains to be validated. Effective operation of agents hinges on secure credentials such as API keys, OAuth tokens, and service accounts. Brex identified that conventional safeguards couldn’t effectively manage agent activity.
To address these challenges, Brex developed an innovative in-house platform named CrabTrap, an open-source HTTP/HTTPS proxy. This tool intercepts all network traffic, analyzes policy rules, and utilizes LLM to determine whether an agent’s request should be approved or denied.
“The network layer serves as an untapped enforcement point,” commented Brex co-founder and CEO Pedro Franceschi in an interview with VentureBeat. “Every agent request presents an opportunity to intercept, analyze, and apply policy.”
Franceschi’s insight for IT leaders emphasizes a shift in agent governance from SDK-level permissions to a centralized network control system capable of learning from real agent behavior.
Targeting the Transport Layer
The initial response to the agent security gap involved implementing guardrails, focusing heavily on scoped tools, per-action permissions, and human approval. As agents evolve, new APIs continuously surface that require tuning, Franceschi noted.
“Multi-tool agent systems accessing the open internet create significant challenges for their builders. Increased agent competence raises their risk, while heightened security can undermine utility,” he explained.
Existing solutions to manage this trade-off were found to be inadequate. Although fine-grained API tokens can be beneficial, they may still be abused and can restrict functionality. Additionally, semantic guardrails (like context and prompt steering) can be bypassed through prompt injection, especially for internet-connected agents.
While offering limited toolsets and read-only access can mitigate risks, it severely limits agent productivity. Conversely, providing extensive write access and large tool surfaces increases potential risks.
Model Context Protocol (MCP) gateways enforce policies at the protocol level, but only for MCP-utilized traffic. Furthermore, an LLM provider’s guards are often limited to a specific model, which can prevent company-specific customization. Moreover, powerful tools like Nvidia OpenShell facilitate enhanced per-sandbox egress control.
“When we began, the industry lacked safe deployment solutions like OpenClaw,” Franceschi explained. “Instead of waiting for a solution, we opted to develop the necessary tools ourselves.”
They required a platform that could analyze all agents’ network requests, allowing nuanced decision-making on approvals. This positioned the transport layer as a foundational architectural element.
By operating at this layer, CrabTrap remains framework, language, and API agnostic. It routes all outbound requests through a proxy using user settings HTTP_PROXY and HTTPS_PROXY.
Franceschi stressed that while they didn’t believe the transport layer was the sole solution, they recognized a significant opportunity for impactful enforcement in this space.
LLM as a Decision-Maker
CrabTrap synergizes deterministic static rules with LLM to critically evaluate requests that stray from established patterns. This “judge” activates on an estimated 3% of atypical requests.
The primary challenge involved establishing policy accuracy. While static rules simplify the accuracy assessment, LLM judgments introduce non-deterministic behaviors, necessitating confidence in the system’s decisions.
“Our breakthrough was creating policies based on observed behavior rather than starting from scratch,” Franceschi highlighted. By tracking real agent behavior and making adjustments based on actual use, they achieved heightened efficiency compared to beginning with arbitrary assumptions.
They designed a policy builder (essentially an agent loop) that runs the core agent in shadow mode, reviews historical network traffic, samples representative calls, and formulates natural language policies aligned with the agent’s behavior.
To validate policy alterations before implementation, CrabTrap also includes an evaluation system that assesses past audit entries against proposed policies.
This comprehensive system, processing simultaneous judge requests, minimizes response times significantly. The entire audit trail is maintained in PostgreSQL and is accessible via a management API and dashboard, alerting users on continuously denied resources and prompting policy review suggestions.
This approach effectively closes the gap between denial analytics and policy improvements, Franceschi added.
Challenges in Development
Brex faced several challenges during development, notably latency. “Inserting an LLM between agents and outbound API requests can decelerate processes,” he noted.
However, the actual lag was less significant than anticipated. The reasons were twofold: LLM judges only process around 3% of requests, and agents typically develop predictable traffic profiles. Once patterns are established, they can be converted into static rules. Second, deploying a swift model such as Claude Haiku reduces potential delays, with further gains achievable through local models and caching.
A more subtle issue was ensuring swift access to requests. As the judge receives full HTTP requests controlled by users, crafted content can manipulate decisions through URLs, headers, or bodies.
To address this, Brex structured requests as JSON before sending them to the model, ensuring user-controlled content was “escaped” rather than directly integrated as raw text.
CrabTrap’s Impact and Future Prospects
Brex tracks various metrics to evaluate CrabTrap’s effectiveness, including agent engagement, network traffic behaviors, and Net Promoter Score (NPS). The platform’s most significant outcome is an increase in organizational trust.
Previously, there was considerable hesitation regarding the widespread adoption of autonomous agents due to inadequate security assurances. “CrabTrap has shifted that perception,” Franceschi stated. With a reliable enforcement layer, greater confidence to deploy agents across business landscapes has emerged.
Franceschi regards traffic-derived policies as “immensely effective.” The platform exceeded expectations, producing policies that often aligned with human decisions based on just a few days of actual traffic data.
CrabTrap provided insights into the noise generated by agents, making this information visible through its audit trail. They have refined agents, eliminated ineffective tools, and removed unnecessary categories of requests that wasted valuable time and resources.
“Proxies have transitioned into discovery tools, not just enforcement tools,” he noted.
Opportunities for Growth
Brex foresees further enhancements for CrabTrap, especially now that it is available as open-source. “We invite community contributions,” Franceschi said.
Potential improvement areas include advanced authentication methods like single sign-on (SSO), fine-tuned role-based access control (RBAC), and escalation workflows for agents to request additional permissions. Policy suggestions drawn from denial patterns would also enrich the system.
Automating the entire policy lifecycle through API endpoints would streamline agent policy management further. Additionally, agents should be able to solicit human review for consistent denials of specific resources, evolving CrabTrap from a strict enforcement tool to a more adaptable permission system.
Further opportunities exist to weave in additional signals about agent tracing and resource requests, allowing for a finer granularity in policy formulation.
Lastly, there’s lingering philosophical consideration regarding CrabTrap’s design—should it operate as a transparent layer, or should it function as a “benevolent manager,” where agents are aware of and can interact with this layer?
Contributions from the open-source community will be vital in this evolution, and as CrabTrap reaches a broader user base, it will become increasingly sophisticated. Brex agents communicate with specific APIs, and collective experiences across various agents, services, and policy needs will uncover unique edge cases.
“We have ambitious plans for CrabTrap’s future, and we prefer to build transparently,” Franceschi concluded.
Lessons for Other Developers from CrabTrap
The response to CrabTrap has exceeded expectations, already gaining over 700 stars on GitHub. Brex has engaged with various industry leaders, including OpenAI and Y Combinator CEO Garry Tan, interested in implementing similar infrastructure.
The overarching lesson, according to Franceschi, is “do not allow infrastructure gaps to stall your progress.” Companies face real challenges, including security risks, limited tools, and ambiguous guardrails when contemplating the large-scale deployment of AI agents.
“It’s tempting to wait until technology catches up,” he says. “Our experience with CrabTrap taught us that you can actively address these challenges on your own.”
Source: venturebeat.com