UFC 666 : Frontkick vs Rikugan

Agentic loop: Definition

Before diving into Frontkick, it is worth understanding what an agentic loop is and what it takes to build a functional and effective agent.

An agentic loop is an architecture in which a model is provided with an initial context, a task, and sensors/actuators. By generating tokens formatted according to a schema, the model can invoke these sensors/actuators. The output tokens generated by the model and the sensor/actuator results from turn N are fed back into it as inputs for turns N+1, N+2, and so on. A specific actuator allows the loop to end when the model considers the task complete. Other, more formal exit conditions can also be defined, such as the number of tokens generated, elapsed time, or any other relevant constraint.

The diagram below illustrates the standard operation of an agentic loop, where the model's actions are reinjected into context at each iteration:

Agentic loop: ToolAgent and CodeAgent

The sensors/actuators declared to the model through the initial context can rely on two action modes: tools and CodeAct. These two approaches give rise to the ToolAgent and CodeAgent architectures, respectively.

ToolAgent

Tools are structured as follows:

• An interface whose description instructs the model on how to call the tool.
• A code block that executes the actions and returns the results.

In most cases, the interface is described in JSON. The following example shows a tool for searching the web:

The tool code:

import requests

def web_search(query: str) -> str:
    """Search DuckDuckGo and return a summary."""
    res = requests.get(
        "https://api.duckduckgo.com/", params={"q": query, "format": "json"}
    ).json()

    return res.get("AbstractText") or "No summary result found."

The interface description:

{
  "name": "web_search",
  "description": "Performs a web search.",
  "parameters": {
    "type": "object",
    "properties": {
      "query": {
        "type": "string",
        "description": "The search query."
      }
    },
    "required": ["query"]
  }
}

An example of a model tool call (OpenAI style):

[
  {
    "id": "call_abc123",
    "type": "function",
    "function": {
      "name": "web_search",
      "arguments": "{\"query\": \"Why Frontkick is so awesome?\"}"
    }
  }
]

This paradigm offers many benefits:

• Security: limits the attack surface exposed to the model.
• Reliability: arguments are validated against a schema, catching syntax errors early.
• Infrastructure simplicity: runs directly inside your application without requiring an isolated sandbox.
• Observability: structured function calls produce clean logs that are easy to trace and analyze.
• Human-in-the-loop control: straightforward interception of critical actions (e.g., deletion, payment) before execution.
• Efficiency: for simple or well-defined tasks, generating a JSON object consumes fewer tokens than writing a complete Python script.

CodeAgent

On the other hand, the CodeAct paradigm provides the model with a code execution environment (often Python) as its primary interface. The model can thus execute code placed between two marker tags (for example, <code> and </code>).

Taking the previous example, the model generates:

<code>
import requests

def web_search(query: str) -> str:
    url = "https://api.duckduckgo.com/"
    res = requests.get(url, params={"q": query, "format": "json"}).json()
    return res.get("AbstractText") or "No summary result."

result = web_search("Why Frontkick is so awesome?")
print(result)
</code>

This paradigm also brings several key advantages:

• Expressiveness: allows combining loops, conditions, and complex data processing within a single script.
• Fewer loop iterations: chains sequential actions in a single model turn, reducing latency.
• Context savings: processes and filters raw data directly in the sandbox instead of clogging the context window.
• Dynamic extensibility: ability to write custom functions or import libraries without prior declaration.
• Self-correction: leverages reinjected tracebacks to debug code autonomously.
• Application power: gives direct access to the entire Python ecosystem (standard library and external modules).

This Huggingface page is really helpful to learn about CodeAgent paradigm.

The favored paradigm

Anthropic, OpenAI, and their peers favor ToolAgent primarily for reasons of security, governance, and industrial scalability. Running arbitrary code (CodeAct) at the scale of millions of users requires isolating every session in heavy, costly sandboxes while exposing systems to heightened risks. In contrast, ToolAgent provides strict access control, full action auditability, and seamless integration with existing API architectures.

To bridge the flexibility gap with CodeAct, model providers are deploying major engineering efforts:

• Structured Outputs: guarantees 100% adherence to JSON/Pydantic schemas, eliminating syntax errors.
• Parallel Tool Calling: natively training models to plan and execute multiple independent actions in a single generation turn.
• Protocol standardization: developing interoperability standards to connect any API or database in a plug-and-play fashion.
• Tool Retrieval: applying RAG techniques to functions, enabling an agent to select the right tool from a catalog of thousands of APIs without cluttering the context window. Retrieval-augmented generation refers to pairing an LLM with a reference database—in our case, Binary Ninja scripts.

Frontkick: The agent going against the grain

Even though ToolAgent-based approaches (such as Rikugan) have proven effective in reverse engineering, Frontkick bets on CodeAgent. Frontkick runs directly inside Binary Ninja's Python environment and interacts natively with its API.

This architectural choice comes down to two major factors: flexibility and maintainability.

Flexibility in the face of a massive API

Binary Ninja's API is feature-rich: it covers manipulation across different IL levels (LLIL, MLIL, HLIL), type management, memory analysis, xrefs, and disassembler interaction.

Rather than abstracting every action behind a dedicated JSON tool (get_function, get_xrefs, rename_symbol), Frontkick gives the model direct access to Binary Ninja's Python runtime. The agent can thus compose the exact logic it needs to navigate thousands of instructions or cross-reference complex structures in a single run—across all open binaries in the application.

Agent maintainability

Wrapping an API as vast as Binary Ninja's in JSON schemas incurs a heavy maintenance overhead: every update to the decompiler requires updating, testing, and redeploying dozens of tools.

By adopting CodeAct, Frontkick keeps its infrastructure minimal. The model adapts directly to Binary Ninja's Python API, taking advantage of existing documentation and the LLM's inherent coding capabilities.

A script executed within Binary Ninja's context

In practice, the agent generates Python code executed directly on a BinaryView (bv) preserved within its context:

# Search for and rename functions calling a specific routine
target = bv.get_functions_by_name("check_license")[0]
for caller in target.callers:
    caller.name = f"auth_handler_{caller.start:x}"
    print(f"Renamed: {caller.name}")

How Frontkick works under the hood

To maintain optimal performance across long analysis sessions, Frontkick combines three key mechanisms for context and knowledge management. Progressive shrinking of older turns truncates the content of past steps, while task compaction summarizes history whenever the context window is saturated—all while locking in the initial instruction to prevent drift. In parallel, a RAG pipeline based on a corpus of verified Python recipes anchors code generation in the reality of Binary Ninja's API. This corpus was generated by a capable model : Opus 5.5.

The following diagram details Frontkick's execution loop, highlighting its shrinking and compaction strategies alongside its built-in RAG:

The diagram above illustrates the complete loop, featuring:

• injection of the initial prompt, combining static instructions with dynamic facts such as libraries present in the interpreter or already loaded binaries;
• forcing the model to generate a code block if two turns have elapsed without any code output;
• injecting RAG recipes into the model feedback whenever executed code produces an error;
• the truncation strategy, which keeps only 500 characters for steps older than N-4;
• injecting findings recorded by the model into the next turn's input;
• reinforcing the requirement to produce code in a turn and avoid resolving issues through text alone: code over prose.

An important note: the current shrinking strategy preserves an unmodified window of the four most recent turns and truncates starting from the fifth turn. As a result, once past five turns, every loop iteration truncates a step. This is inefficient given the prompt caching mechanisms in modern inference engines. The strategy will be improved, for instance by batching truncations every X turns.

This dynamic context management (handling the agent's long-term memory) is complemented by a set of prompts (framing short-term and long-term generation) that give the agent its distinct personality.

The diagram below illustrates the architecture and assembly pipeline of these prompts before each LLM call:

Key highlights:

• The prompt is built following Qwen's CO-STAR framework: rare delimiter tokens in the corpus are used to define distinct sections with dedicated goals (context, desired output format, target audience, etc.).
• To maximize Binary Ninja's capabilities, default available Python libraries are explicitly detailed in the prompt.
• To ensure a faster cold start, already open binaries are prepopulated with basic details (name, architecture, file size, etc.).

Other Binary Ninja agents

Both of these agents opted for ToolAgent, setting their paradigm apart from Frontkick's approach.

Feature comparison

Evaluation and Benchmark — CREBench

To compare Rikugan and Frontkick, two test cases from the CREBench benchmark were selected. CREBench evaluates models on reverse engineering and cryptanalysis tasks. For each test case—provided as a stripped binary—the models must identify the encryption algorithm and key, then reimplement a decryption routine to retrieve the flag. Our test setup was configured as follows:

• The chosen model was Claude 5.5 Sonnet: powerful enough to avoid getting stuck on tool calling mechanics, allowing us to evaluate core effectiveness (which toolset and prompt design deliver better results).
• The test cases were AES-128-CBC and ChaCha20, using the hardcode_plain/O0 setup, with 4 runs per case and a maximum budget of 1M tokens (cumulative input/output and turns).

The chart below summarizes the comparative performance between Frontkick and Rikugan across both benchmark tasks:

The results favor Frontkick: it used half as many tokens on average and achieved a success score nearly 3x higher.

However, inspecting the execution transcripts reveals a major bias: on AES-128-CBC, Rikugan systematically tries to run from Crypto.Cipher import AES (the pycryptodome library, which was not installed). Following this failure, it falls back to reimplementing the algorithm manually, byte by byte, without ever leveraging cryptography (which was installed and successfully used by Frontkick on every run). As a result, Rikugan required 31 to 35 turns and ~1M tokens, compared to 15 to 24 turns and ~400K tokens for Frontkick on the exact same task.

Frontkick benefited here from a dedicated mechanism: an Available libraries block dynamically injected into its system prompt, enumerating installed libraries. Rikugan was left guessing library availability rather than receiving an explicit list.

Thus, this benchmark does not prove an intrinsic superiority of CodeAct in this specific test, but rather underscores the decisive impact of prompt engineering choices. In a future study, it would be valuable to rerun this benchmark with dynamic library injection added to Rikugan's system prompt. With initial prompts aligned, the tool calling mode (ToolAgent vs. CodeAct) would become the primary differentiating factor.

For full transparency, here is the table breaking down the results run by run:

Conclusion: The judges' verdict

At the end of round one, the judges deliver their verdict: a technical draw! While Frontkick lands a solid upper cut on Rikugan by keeping token consumption and turn counts under control, a total knockout in favor of CodeAct hasn't been declared just yet.

Reviewing the replays confirms it: Frontkick's victory relied more on a clever prompt engineering trick—the dynamic injection of available libraries—than on an inherent architectural superiority.

To see whether CodeAct will deliver a definitive knock-out blow to ToolAgent on Binary Ninja, we'll have to wait for an even-ground rematch with fully aligned baseline prompts. One thing is certain: the battle of the agents has only just begun!