Skip to content

Pull requests: EleutherAI/lm-evaluation-harness

New pull request New
59 Open 964 Closed
59 Open 964 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Updated vllm imports in vllm_causallms.py
#1890 opened May 25, 2024 by mgoin Loading…
[HFLM]Add support for Ascend NPU
#1886 opened May 25, 2024 by statelesshz Loading…
1
@haileyschoelkopf
1
Add LegalBench tasks
#1878 opened May 23, 2024 by zafstojano Loading…
3
Add chat template
#1873 opened May 22, 2024 by KonradSzafer Loading…
3
Test coverage for optimum_lm.py
#1872 opened May 22, 2024 by zafstojano Loading…
Added tests for Anthropic LLMs
#1868 opened May 21, 2024 by zafstojano Loading…
Multiple Choice Questions and Large Languages Models: A Case Study with Fictional Medical Data
#1867 opened May 21, 2024 by maximegmd Loading…
2
Draft - Support ov models via genai
#1862 opened May 20, 2024 by sstrehlk Loading…
1
mmlu-pro for the Italian language
#1860 opened May 19, 2024 by giux78 Loading…
1
[WIP] Fix NeuralMagic tests
#1859 opened May 19, 2024 by haileyschoelkopf Loading…
5
Rename lm_eval.logging -> lm_eval.loggers bug Something isn't working.
#1858 opened May 19, 2024 by haileyschoelkopf Loading…
1
Force BOS token usage in 'gemma' models for VLLM
#1857 opened May 19, 2024 by haileyschoelkopf Loading…
1
Add option in TaskManager to not index library default tasks ; Tests for include_path feature request A feature that isn't implemented yet.
#1856 opened May 19, 2024 by haileyschoelkopf Loading…
1
Fix m_mmlu target
#1853 opened May 18, 2024 by jordane95 Loading…
Implement Exams benchmark
#1852 opened May 17, 2024 by snova-zoltanc Loading…
2
Fix self.max_tokens in anthropic_llms.py
#1848 opened May 16, 2024 by lozhn Loading…
1
Adding LLaVa support
#1832 opened May 13, 2024 by ashvinnihalani Loading…
1
Financial PhraseBank (FPB) Eval Metric
#1815 opened May 9, 2024 by bcicc Loading…
Fix cost_estimate.py
#1810 opened May 8, 2024 by xksteven Loading…
5
Fix --gen_kwargs and VLLM (temperature not respected) bug Something isn't working.
#1800 opened May 7, 2024 by haileyschoelkopf Loading…
Make scripts.write_out error out when no splits match
#1796 opened May 7, 2024 by haileyschoelkopf Loading…
1
Vllm get tokenizer
#1794 opened May 6, 2024 by AguirreNicolas Loading…
1
add NPU support for huggingface.py
#1787 opened May 6, 2024 by jiaqiw09 Loading…
23
Create task dharma2 - a small (300 qs) & wide (many topics) dataset
#1753 opened Apr 26, 2024 by UmerHA Loading…
6
Group agg rework
#1741 opened Apr 23, 2024 by lintangsutawika Loading…
4
ProTip! Updated in the last three days: updated:>2024-05-22.