What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks
-
Updated
Jul 26, 2024 - Jupyter Notebook
#
llms-benchmarking
Here are 42 public repositories matching this topic...
BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks on Large Language Models
-
Updated
Sep 3, 2024 - Python
Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)
metrics good-first-issue llm prompt-engineering generative-ai llmops llm-eval llm-tools llm-evaluation llm-evaluation-toolkit llms-benchmarking llm-evaluation-framework
-
Updated
Sep 12, 2024 - Python
How good are LLMs at chemistry?
-
Updated
Dec 19, 2024 - Jupyter Notebook
Language Model for Mainframe Modernization
-
Updated
Aug 23, 2024 - Python
CompBench evaluates the comparative reasoning of multimodal large language models (MLLMs) with 40K image pairs and questions across 8 dimensions of relative comparison: visual attribute, existence, state, emotion, temporality, spatiality, quantity, and quality. CompBench covers diverse visual domains, including animals, fashion, sports, and scenes.
benchmark reasoning vision-and-language multimodal-deep-learning human-annotation foundation-models large-language-models llms vision-language-model multimodal-large-language-models evaluation-llms llms-benchmarking
-
Updated
Aug 6, 2024 - Jupyter Notebook
The data and implementation for the experiments in the paper "Flows: Building Blocks of Reasoning and Collaborating AI".
ai competitive-programming agents competitive-programming-contests competitive-coding llms llms-reasoning llms-benchmarking aiflows
-
Updated
Feb 12, 2024 - Python
Training and Benchmarking LLMs for Code Preference.
-
Updated
Nov 15, 2024 - Python
Restore safety in fine-tuned language models through task arithmetic
-
Updated
Mar 28, 2024 - Python
A minimalist benchmarking tool designed to test the routine-generation capabilities of LLMs.
-
Updated
Nov 28, 2024 - Python
Develop reliable AI apps
-
Updated
Dec 3, 2024 - Svelte
Code and data for Koo et al's ACL 2024 paper "Benchmarking Cognitive Biases in Large Language Models as Evaluators"
nlp evaluation bias bias-detection llm llms llm-evaluation llms-benchmarking llm-as-judge llm-as-a-judge llm-as-evaluator
-
Updated
Feb 16, 2024 - Jupyter Notebook
Join 15k builders to the Real-World ML Newsletter ⬇️⬇️⬇️
-
Updated
Apr 19, 2024 - Python
A framework for evaluating the effectiveness of chain-of-thought reasoning in language models.
-
Updated
Oct 6, 2024 - Jupyter Notebook
The MERIT Dataset is a fully synthetic, labeled dataset created for training and benchmarking LLMs on Visually Rich Document Understanding tasks. It is also designed to help detect biases and improve interpretability in LLMs, where we are actively working. This repository is actively maintained, and new features are continuously being added.
biases synthetic-dataset-generation layoutlm synthetic-dataset layoutxlm token-classification layoutlmv3 layoutlmv2 llms-benchmarking
-
Updated
Sep 6, 2024 - Python
Awesome Mixture of Experts (MoE): A Curated List of Mixture of Experts (MoE) and Mixture of Multimodal Experts (MoME)
artificial-intelligence sparse moe load-balancing multimodal-learning mixture-of-experts mome gating-network foundation-models large-language-models llms large-language-model large-vision-language-models expert-network llms-reasoning llms-benchmarking mixtrure-of-multimodal-experts sparse-moe sparse-mixture-of-experts sparse-mixture-of-multimodal-experts
-
Updated
Sep 25, 2024
Benchmark that evaluates LLMs using 436 NYT Connections puzzles
-
Updated
Nov 5, 2024 - Python
Official repository for "RoMath: A Mathematical Reasoning Benchmark in 🇷🇴 Romanian 🇷🇴"
-
Updated
Sep 23, 2024 - Python
A platform that enables users to perform private benchmarking of machine learning models. The platform facilitates the evaluation of models based on different trust levels between the model owners and the dataset owners.
platform benchmarking inference secure private mpc contamination trusted-execution-environment confidential-computing large-language-models llms-benchmarking private-benchmarking ezpc
-
Updated
Sep 16, 2024 - Python
Improve this page
Add a description, image, and links to the llms-benchmarking topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llms-benchmarking topic, visit your repo's landing page and select "manage topics."