Skip to content
Schemory
Skill

evaluation-bigcode-evaluation-harness

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

Listed from its GitHub repository. Reports from people who tried it will appear here.

Source
davila7/claude-code-templates
GitHub stars
34,066
License
MIT

No reports yet

Nobody has written down how evaluation-bigcode-evaluation-harness works with their agent. Tried it? Say whether it worked, and with which agent and model.

Write the first report