LLM agents
Coverage of LLM agents in the Nexus archive.
- Show HN: I benchmarked LLM agents on fixing real-world security vulnerabilities
A benchmark tested 5 LLM agents on fixing 20 real-world security vulnerabilities across 18 Python projects. The best solve rate was 50%, with cost differences between models (e.g., gpt-5.5 vs. gpt-5.4-mini) outweighing performance gains, likely due to training data variations.
- CVE-Bench: testing LLM agents on real-world vulnerability patches
CVE-Bench is a benchmark for testing large language model (LLM) agents using real-world vulnerability patches. The article provides a URL for the project and a Hacker News comments link.
- Constraint Decay: The Fragility of LLM Agents in Back End Code Generation
The article discusses 'constraint decay' in large language models (LLMs), highlighting their fragility when generating back-end code due to the erosion of imposed constraints. It emphasizes challenges in maintaining reliability and accuracy in automated code generation tasks.