The Largest Lesson From Testnet: Optimizing AI Agents while Remaining Cost-Effective

Addressing Latency, Call Issues and Relevancy for Sustainable AI

After examining the best approach for our infrastructure needs, the team started to encounter an interconnected web of challenges that had to be assessed and addressed in real time. The main challenges were affecting rate limits, call costs, latency, as well as managing relevancy filtering. 

‍

Rate limits

The team required an upgrade in rate limits within 24 hours of launching testnet, we were surprised by how quickly this occurred, but in hindsight it reflects the appetite that users have for testing out groundbreaking technology. The massive amount of Agent requests meant that we had to implement fallbacks to handle the influx of use, for example, if the Anthropic limit hit, then we had the OpenAI LLM ready to process Agent requests. The team was on hand 24/7 to make sure these operated smoothly. 

‍

Call Costs 

Another challenge was managing the high costs that come with the high number of Agent requests. To address this, we optimized our prompts to be effective with smaller, more cost-efficient models. A common strategy involves using a large LLM to generate synthetic data that can then be utilized by smaller, cheaper models. 

‍

Latency 

A key challenge faced was managing latency to ensure that our Agents deliver efficient, real-time responses. Latency, which measures the time taken from when an AI system receives an instruction to when it completes the task and provides an output, was a significant hurdle. We assessed various methods and services, ultimately finding that the best approach was to use smaller models whenever possible. Additionally, we leveraged both traditional caching and prompt caching features. This combination significantly reduced response times and optimized our system's performance.

Relevancy Filtering

Relevancy filtering has emerged as another method for enhancing the efficiency of AI Agents on the platform. In Web3, incentivizing activity can attract users who exploit rewards through spam-like or disingenuous responses. To address this, our team has launched a ‘Relevancy Agent’ which is used by Agent Collectives to determine the relevance of a user's query. This promotes a more genuine and productive user environment, ensuring that interactions remain constructive. 

‍

Next Level Methods to Continue Improving Agent Efficiency 

From experiencing the above, the research team have been exploring several strategies that can boost AI agent efficiency while keeping costs in check. They have been tinkering with the idea of AI Agents with memory, and examining the latest AI advancements. 

Imagine an AI agent that can remember past interactions for specific users or wallet interactions, much like a human coworker. The ability to retain context significantly enhances user experience and operational efficiency. By incorporating memory and recall features, agents can avoid repetitive tasks, leading to substantial cost savings over time. For instance, if an agent recalls a specific SQL query used in a prior interaction, it can reuse that information instead of generating it from scratch, saving both time and resources. 

The team is also staying ahead of the curve by closely monitoring and integrating the latest advancements from leading AI innovators. For example, the team has been exploring OpenAI’s predicted output method, which helps reduce latency by pre-computing parts of model responses when the outcome is mostly predictable. 

Additionally, the team is diving deep into cost management strategies, including insights from the latest research by Sayash Kapoor et al. at Princeton, titled “AI Agents That Matter,” which evaluates current benchmarks and practices in AI Agent development. Staying at the forefront of AI advancements and continuously testing new approaches remains a core priority at Theoriq. 

Ultimate Vision: AI Agent Collectives Solving Complex Problems

Each of these methods is essential in building a solid foundation for efficient AI Agents functioning on Theoiq’s infrastructure. These lessons have been instrumental in the platform's growth, but the overarching vision remains the same: the future of AI agents lies in their ability to work together as Collectives.

By developing a multi-agent protocol, we enable diverse agents to communicate, share insights, and learn from each other’s experiences. With this collaborative framework as our north star, we’ll not only promote innovative practices like those above but also continually enhance the overall efficiency and adaptability of AI systems.

Ironically enough, the team have found that combining agents into collectives, and assessing and keeping those Agents accountable to specific constraints, should prove to be more cost effective than specialized AI Agents.

Do you have an AI Agent Collective that is performing well, but is blowing out a marketing budget? Maybe an Agent in that collective could be swapped out for another that gets the task done in a more cost effective way. This efficient formation of Collectives then becomes a more dynamic process.  

The Future of AI Agents and Collectives is Here

Launching our incentivized testnet and achieving over 23 million interactions in a few short weeks shows the demand for AI Agents in the decentralized space is real. But it has also revealed the challenges of creating scalable, cost-effective multi-agent systems.

Optimizing for efficiency, reducing costs, and tackling latency are not just technical hurdles—they are the foundation of sustainable AI in Web3. By exploring advanced strategies like memory-enabled agents, efficient cost management, and innovative collaboration between AI Agents, we’re shaping a more adaptive and resilient future.

At Theoriq, our vision extends beyond individual AI Agents. We see a world where Agent collectives collaborate seamlessly to solve complex problems, transforming industries and unlocking new possibilities in Web3. We’re not just building for today’s challenges; we’re paving the way for an AI-driven, decentralized tomorrow.