An extensible benchmark for evaluating large language models on planning
-
Updated
Mar 16, 2025 - PDDL
#
llms-benchmarking
Here are 58 public repositories matching this topic...
🔥 A list of tools, frameworks, and resources for building AI web agents
-
Updated
Mar 20, 2025
A comprehensive set of LLM benchmark scores and provider prices.
-
Updated
Mar 4, 2025 - JavaScript
What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks
-
Updated
Jul 26, 2024 - Jupyter Notebook
BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks on Large Language Models
-
Updated
Feb 21, 2025 - Python
Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)
metrics good-first-issue llm prompt-engineering generative-ai llmops llm-eval llm-tools llm-evaluation llm-evaluation-toolkit llms-benchmarking llm-evaluation-framework
-
Updated
Feb 13, 2025 - Python
How good are LLMs at chemistry?
-
Updated
Mar 22, 2025 - Python
Benchmark that evaluates LLMs using 601 NYT Connections puzzles extended with extra trick words
-
Updated
Mar 22, 2025 - Python
Language Model for Mainframe Modernization
-
Updated
Aug 23, 2024 - Python
Thematic Generalization Benchmark: measures how effectively various LLMs can infer a narrow or specific "theme" (category/rule) from a small set of examples and anti-examples, then detect which item truly fits that theme among a collection of misleading candidates.
-
Updated
Mar 20, 2025
CompBench evaluates the comparative reasoning of multimodal large language models (MLLMs) with 40K image pairs and questions across 8 dimensions of relative comparison: visual attribute, existence, state, emotion, temporality, spatiality, quantity, and quality. CompBench covers diverse visual domains, including animals, fashion, sports, and scenes.
benchmark reasoning vision-and-language multimodal-deep-learning human-annotation foundation-models large-language-models llms vision-language-model multimodal-large-language-models evaluation-llms llms-benchmarking
-
Updated
Aug 6, 2024 - Jupyter Notebook
Develop reliable AI apps
-
Updated
Mar 12, 2025 - Svelte
Training and Benchmarking LLMs for Code Preference.
-
Updated
Nov 15, 2024 - Python
The data and implementation for the experiments in the paper "Flows: Building Blocks of Reasoning and Collaborating AI".
ai competitive-programming agents competitive-programming-contests competitive-coding llms llms-reasoning llms-benchmarking aiflows
-
Updated
Feb 12, 2024 - Python
-
Updated
Mar 14, 2025 - Python
Restore safety in fine-tuned language models through task arithmetic
-
Updated
Mar 28, 2024 - Python
A minimalist benchmarking tool designed to test the routine-generation capabilities of LLMs.
-
Updated
Nov 28, 2024 - Python
Code and data for Koo et al's ACL 2024 paper "Benchmarking Cognitive Biases in Large Language Models as Evaluators"
nlp evaluation bias bias-detection llm llms llm-evaluation llms-benchmarking llm-as-judge llm-as-a-judge llm-as-evaluator
-
Updated
Feb 16, 2024 - Jupyter Notebook
Join 15k builders to the Real-World ML Newsletter ⬇️⬇️⬇️
-
Updated
Apr 19, 2024 - Python
Improve this page
Add a description, image, and links to the llms-benchmarking topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llms-benchmarking topic, visit your repo's landing page and select "manage topics."