Hacker News · AGENTS.md· mustaphah·· 2026/02/16AI 评分62
评估 AGENTS.md 对编程智能体是否有帮助
原文标题：Evaluating AGENTS.md: are they helpful for coding agents?
AI 导读
一项研究评估了 AGENTS.md 这类上下文文件对编程智能体完成任务的帮助，发现提供上下文文件通常不会提升任务成功率，反而让推理成本平均增加超过 20%。该结论在不同大语言模型、编程智能体以及 LLM 生成和开发者提交的上下文文件上都成立。研究还发现智能体能较好遵循上下文文件中的指令，但被模型厂商推荐的仓库概览类内容并无帮助，作者建议任何试图提升性能的改动都应先经过严格评估再部署。
正文
当前语言的正文正在等待翻译，暂时显示原文。
Abstract:A widespread practice in software development is to tailor coding agents to repositories using context files, such as this http URL. Although this practice is strongly encouraged by agent developers, there is currently no rigorous investigation into whether such context files are actually effective for real-world tasks. In this work, we study this question and evaluate coding agents' task completion performance in two complementary settings: established SWE-bench tasks from popular repositories, with LLM-generated context files, and a novel collection of issues from repositories containing developer-committed context files. Surprisingly, we find that providing context files does not generally improve task success rates, while increasing inference cost by over 20% on average. This observation holds across different LLMs, coding agents, and for both LLM-generated and developer-committed context files. Specifically, we find that while instructions in the context files are well followed by coding agents, repository overviews, although popular and recommended by model providers, are not helpful. We conclude that while context files are useful for specifying non-standard coding practices, any attempts to improve performance should be rigorously evaluated before deployment.
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2602.11988 [cs.SE] |
| (or arXiv:2602.11988v3 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2602.11988 arXiv-issued DOI via DataCite |
Submission history
From: Thibaud Gloaguen [view email]
[v1]
Thu, 12 Feb 2026 14:15:22 UTC (2,264 KB)
[v2]
Tue, 23 Jun 2026 20:26:50 UTC (2,891 KB)
[v3]
Tue, 29 Sep 2026 14:58:01 UTC (2,877 KB)
来源：Hacker News · AGENTS.md · arxiv.org