UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.

[ACL 2024] User-friendly evaluation framework: Eval Suite & Benchmarks: UHGEval, HaluEval, HalluQA, etc.

Code for the EMNLP 2024 paper "Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps"

Official repo for the paper PHUDGE: Phi-3 as Scalable Judge. Evaluate your LLMs with or without custom rubric, reference answer, absolute, relative and much more. It contains a list of all the available tool, methods, repo, code etc to detect hallucination, LLM evaluation, grading and much more.

up-to-date curated list of state-of-the-art Large vision language models hallucinations research work, papers & resources

An Easy-to-use Hallucination Detection Framework for LLMs.

VerifAI initiative to build open-source easy-to-deploy generative question-answering engine that can reference and verify answers for correctness (using posteriori model)

[ACL 2024] ANAH & [NeurIPS 2024] ANAH-v2

VideoHallucer, The first comprehensive benchmark for hallucination detection in large video-language models (LVLMs)

[ACL 2024] An Easy-to-use Hallucination Detection Framework for LLMs.

Code for Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities (NeurIPS'24)

Code and Data Repo for Paper "Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation"

This repo hosts the Python SDK and related examples for AIMon, which is a proprietary, state-of-the-art system for detecting LLM quality issues such as Hallucinations. It can be used during offline evals, continuous monitoring or inline detection. We offer various model quality metrics that are fast, reliable and cost-effective.

🔎Official code for our paper: "VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimation".

Fact-checking with Iterative Retrieval and Verification

Fully automated LLM evaluator

Detecting Hallucinations in Large Language Model Generations using Graph Structures

Detecting Hallucinations in LLMs

Chrome extension for the ATLAS project.

Binary hallucination detection classifier using logistic regression