I built a research agent that tries not to sound more confident than it should

Most research agents I've come across have a troubling habit: they fabricate citations or inflate the credibility of weak sources, yet still present their answers with unwavering confidence. Consequently, I frequently spend a substantial amount of time "utilizing" these agents merely to manually verify whether the sources actually support the claims they made.

I grew weary of this, so I created ResearchAgent. It's a specialized open-source agent tailored for technical and architectural research queries. You present it with a specific task or question. For instance, "Compare the top 3 open-source vector databases for a startup with 10k QPS and tight cost constraints" …and the agent provides a structured response with: A clear recommendation Only the actual sources it retrieved A calibrated confidence score Explicit limitations ranked by severity Design choices that were crucial Rather than presenting a single extensive reasoning block, the agent proceeds through distinct stages: Planner – breaks the question down into specific steps Executor – conducts searches, fetches pages, optionally executes code, and utilizes MCP tools Verifier – applies domain reputation heuristics and assesses if the page genuinely supports the claim Synthesizer – generates the final answer, confidence score, and identified limitations.

The confidence score is not merely subjective. The synthesizer rigorously assesses a concise checklist covering the number of sources, their consistency, recency, and whether the original constraints were adequately met. As a result, sources of low authority or insufficient quality are discarded early, reducing the chance they could affect the final score.

The agent also supports full local execution (DuckDuckGo + Ollama) if you prefer not to use paid APIs, and it includes optional human approval gates before planning and synthesis. The reasoning behind the stages might seem like extra friction at first. However, the first time something goes wrong, being able to examine the plan, raw evidence, and verification scores independently is far superior to staring at a wall of chain-of-thought and trying to determine where it failed.

You can explore the agent at its GitHub repository: https://github.com/spidervirus/ResearchAgent It includes incremental examples (ranging from a basic LLM call to the complete agent) and a brief tutorial to illustrate how each component functions. I am still refining the confidence calibration and aim to progress towards claim-level confidence in the future. Feedback from users who actually employ research agents would be greatly appreciated.