awesome_llm_applications.md

Awesome llm applications

• Awesome llm applications
  • Survey
  • Toolkits
  • Writing
  • WebUI
  • Medical
    • Other
  • Anomaly detection
  • Auto driving
  • Education
    • Other
  • Text Related
  • Recommendation
  • Role play
  • Chat bots
    • Other
  • Law
  • Machine Translation
  • Audio
  • Design
  • Vision
  • Science
  • Finance
  • Data Related
  • Misc
  • Game
  • Hardware
  • Products
  • Other
  • Extra reference

Survey

• Large Language Models for Education: A Survey and Outlook, arXiv, 2403.18105, arxiv, pdf, cication: -1

  Shen Wang, Tianlong Xu, Hang Li, Chaoli Zhang, Joleen Liang, Jiliang Tang, Philip S. Yu, Qingsong Wen

• Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Survey, arXiv, 2403.01528, arxiv, pdf, cication: -1

  Qizhi Pei, Lijun Wu, Kaiyuan Gao, Jinhua Zhu, Yue Wang, Zun Wang, Tao Qin, Rui Yan

  · (jiqizhixin) · (Awesome-Biomolecule-Language-Cross-Modeling - QizhiPei)

• Materials science in the era of large language models: a perspective, arXiv, 2403.06949, arxiv, pdf, cication: -1

  Ge Lei, Ronan Docherty, Samuel J. Cooper

• Large Language Models on Graphs: A Comprehensive Survey, arXiv, 2312.02783, arxiv, pdf, cication: -1

  Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji, Jiawei Han

• Challenges and Applications of Large Language Models, arXiv, 2307.10169, arxiv, pdf, cication: 32

  Jean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, Robert McHardy · (mp.weixin.qq)

• Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality

Toolkits

• llama-fs - iyaja

  A self-organizing file system with llama 3

• activepieces - activepieces

  Your friendliest open source all-in-one automation tool ✨ Workflow automation tool 100+ integration / Enterprise automation tool / Zapier Alternative

• Scrapegraph-ai - VinciGit00

  Python scraper based on AI

• personal-ai - Sh4yy

• web-llm - mlc-ai

  Bringing large-language models and chat to web browsers. Everything runs inside the browser with no server support.

• secret-llama - abi

  Fully private LLM chatbot that runs entirely with a browser with no server needed. Supports Mistral and LLama 3.

• text-generation-inference - huggingface

  Large Language Model Text Generation Inference

• kimi-free-api - LLM-Red-Team

  🚀 KIMI AI 长文本大模型逆向API白嫖测试【特长：长文本解读整理】，支持高速流式输出、联网搜索、长文档解读、图像解析、多轮对话，零配置部署，多路token支持，自动清理会话痕迹。

• LocalAI - mudler

• FreeGPT35 - missuo

  Utilize the unlimited free GPT-3.5-Turbo API service provided by the login-free ChatGPT Web.

• llm-universe - datawhalechina

  本项目是一个面向小白开发者的大模型应用开发教程，在线阅读地址：https://datawhalechina.github.io/llm-universe/

• ChatGPT - PawanOsman

  OpenAI API Free Reverse Proxy

• llama_index - run-llama

  LlamaIndex is a data framework for your LLM applications

• chatlabs - writingmate

• llamafile - Mozilla-Ocho

  Distribute and run LLMs with a single file.

• narrator - cbh123

  David Attenborough narrates your life

• CopilotKit - CopilotKit

  AI Copilot infrastructure 🤖 in-app AI chatbots, in-app AI Agents, & AI-powered Textareas.

• full-stack-fastapi-template - tiangolo

  Full stack, modern web application template. Using FastAPI, React, SQLModel, PostgreSQL, Docker, GitHub Actions, automatic HTTPS and more.

• cog - replicate

  Containers for machine learning

• grog - multimodalart

  Gradio UI for a Cog API

• openai-translator - openai-translator

  基于 ChatGPT API 的划词翻译浏览器插件和跨平台桌面端应用 - Browser extension and cross-platform desktop application for translation based on ChatGPT API.

• gptscript - gptscript-ai

  Natural Language Programming

• shell_gpt - TheR1D

  A command-line productivity tool powered by AI large language models like GPT-4, will help you accomplish your tasks faster and more efficiently.

• reor - reorproject

  AI note-taking app that runs models locally.

• fabric - danielmiessler

  fabric is an open-source framework for augmenting humans using AI.

• autollm - safevideo

  Ship RAG based LLM web apps in seconds.

• TaskingAI - TaskingAI

  The open source platform for AI-native application development.

• pandas-ai - gventuri

  PandasAI is the Python library that integrates Gen AI into pandas, making data analysis conversational

• taipy - Avaiga

  Turns Data and AI algorithms into production-ready web applications in no time.

• deploy - pandora-next

  Pandora Cloud + Pandora Server + Shared Chat + BackendAPI Proxy + Chat2API + Signup Free = PandoraNext. New GPTs(Gizmo) UI, All in one!

• llmware - llmware-ai

  Providing enterprise-grade LLM-based development framework, tools, and fine-tuned models.

• create_llama_projects - jerryjliu

• LLM-As-Chatbot - deep-diver

  LLM as a Chatbot Service

• mirror - cocktailpeanut

• lobe-chat - lobehub

  🤖 Lobe Chat - an open-source, extensible (Function Calling), high-performance chatbot framework. It supports onekk-click free deployment of your private ChatGPT/LLM web application.

• bisheng - dataelement

  Bisheng is an open LLM devops platform for next generation AI applications.

• kani - zhudotexe

  kani (カニ) is a highly hackable microframework for chat-based language models with tool usage/function calling.

• promptflow - microsoft

  Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

• embedchain - embedchain

  Framework to easily create LLM powered bots over any dataset.

• pandora - zhile-io

  潘多拉，一个让你呼吸顺畅的ChatGPT。Pandora, a ChatGPT client that lets you breathe freely.

• freegpt-webui - ramonvc

  GPT 3.5/4 with a Chat Web UI. No API key required.

• gpt4free - xtekky

  The official gpt4free repository | various collection of powerful language models

• kaguya - ykdojo

  A ChatGPT plugin that allows you to load and edit your local files in a controlled way, as well as run any Python, JavaScript, and bash script.

Writing

• data-to-paper - technion-kishony-lab

  data-to-paper: AI-driven scientific research

• gpt-author - mshumer

• Is ChatGPT Transforming Academics' Writing Style?, arXiv, 2404.08627, arxiv, pdf, cication: -1

  Mingmeng Geng, Roberto Trotta

• Creating Suspenseful Stories: Iterative Planning with Large Language Models, arXiv, 2402.17119, arxiv, pdf, cication: -1

  Kaige Xie, Mark Riedl

• Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models, arXiv, 2402.14207, arxiv, pdf, cication: -1

  Yijia Shao, Yucheng Jiang, Theodore A. Kanell, Peter Xu, Omar Khattab, Monica S. Lam

• GhostWriter: Augmenting Collaborative Human-AI Writing Experiences Through Personalization and Agency, arXiv, 2402.08855, arxiv, pdf, cication: -1

  Catherine Yeh, Gonzalo Ramos, Rachel Ng, Andy Huntington, Richard Banks

• Weaver: Foundation Models for Creative Writing, arXiv, 2401.17268, arxiv, pdf, cication: -1

  Tiannan Wang, Jiamin Chen, Qingrui Jia, Shuai Wang, Ruoyu Fang, Huilin Wang, Zhaowei Gao, Chunzhao Xie, Chuou Xu, Jihong Dai

• PEARL: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrievers, arXiv, 2311.09180, arxiv, pdf, cication: -1

  Sheshera Mysore, Zhuoran Lu, Mengting Wan, Longqi Yang, Steve Menezes, Tina Baghaee, Emmanuel Barajas Gonzalez, Jennifer Neville, Tara Safavi

• Can large language models provide useful feedback on research papers? A large-scale empirical analysis, arXiv, 2310.01783, arxiv, pdf, cication: -1

  Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Ding, Xinyu Yang, Kailas Vodrahalli, Siyu He, Daniel Smith, Yian Yin · (llm-scientific-feedback - weixin-liang)

• From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting, arXiv, 2309.04269, arxiv, pdf, cication: 1

  Griffin Adams, Alexander Fabbri, Faisal Ladhak, Eric Lehman, Noémie Elhadad · (jiqizhixin) · (huggingface)

• Teach LLMs to Personalize -- An Approach inspired by Writing Education, arXiv, 2308.07968, arxiv, pdf, cication: -1

  Cheng Li, Mingyang Zhang, Qiaozhu Mei, Yaqing Wang, Spurthi Amba Hombaiah, Yi Liang, Michael Bendersky

• Image-to-Story - fffiloni 🤗

• Ambient Adventures: Teaching ChatGPT on Developing Complex Stories, arXiv, 2308.01734, arxiv, pdf, cication: 3

  Zexin Chen, Eric Zhou, Kenneth Eaton, Xiangyu Peng, Mark Riedl

• Voice commanded essay copilot

• PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model, arXiv, 2306.02531, arxiv, pdf, cication: -1

  Yizhe Zhang, Jiatao Gu, Zhuofeng Wu, Shuangfei Zhai, Josh Susskind, Navdeep Jaitly

WebUI

• every-chatgpt-gui - billmei

  Every front-end GUI client for ChatGPT

• chatnio - Deeptrain-Community

  🚀 下一代 AI 一站式解决方案，一站式 Chat + 中转 API 站点，支持 OpenAI，Midjourney，Claude，讯飞星火，Stable Diffusion，DALL·E，ChatGLM，通义千问，腾讯混元，360 智脑，百川 AI，火山方舟，新必应，Google Gemini (PaLM2)，Moonshot，LocalAI 等模型，支持对话分享，自定义预设，云端同步，模型市场，支持弹性计费 / 订阅计划模式，支持图片解析，支持联网搜索，支持模型缓存，丰富美观的后台管理与仪表盘数据统计。

• chat-ollama - sugarforever

• ChatGemini - bclswl0827

  ✨ ChatGemini 是一个基于 Google Gemini 的网页客户端，对标 ChatGPT 3.5，操作逻辑同 ChatGPT 3.5 一致，同时支持在聊天中上传图片，应用会自动调用 Gemini-Pro-Vision 模型进行识图。

• LibreChat - danny-avila

  Enhanced ChatGPT Clone: Features OpenAI, GPT-4 Vision, Bing, Anthropic, OpenRouter, Google Gemini, AI model switching, message search, langchain, DALL-E-3, ChatGPT Plugins, OpenAI Functions, Secure Multi-User System, Presets, completely open-source for self-hosting. More features in development

• copilot-gpt4-service - aaamoon

  Convert Github Copilot to ChatGPT, free to use the GPT-4 model

• Awesome-gptlike-shellsite - bleedline

  深入探索精选的套壳站和必备API资源。本文为初学者和经验丰富的运营者提供一站式指南，涵盖常见问题解答和基础攻略，助您迈向套壳站副业成功之路。Dive into a curated selection of shell sites and essential APIs. This article offers a comprehensive guide for both beginners and seasoned operators, covering FAQs and basic strategies to propel you towards success in your shell site side hustle.

• GeminiProChat - babaohuang

  Minimal web UI for GeminiPro.

• make-real-starter - tldraw

  Make it real

• screenshot-to-code - abi

  Drop in a screenshot and convert it to clean HTML/Tailwind/JS code

• ChatGPT-Next-Web - Yidadaa

  A well-designed cross-platform ChatGPT UI (Web / PWA / Linux / Win / MacOS). 一键拥有你自己的跨平台 ChatGPT 应用。

• ChuanhuChatGPT - GaiZhenbiao

  GUI for ChatGPT API and many LLMs. Supports agents, file-based QA, GPT finetuning and query with web search. All with a neat UI.

• Making a web app generator with open ML models

• chathn - steven-tey

  Chat with Hacker News using natural language. Built with OpenAI Functions and Vercel AI SDK.

• ChatGPT-Midjourney - Licoy

  🎨 一键拥有你自己的 ChatGPT+Midjourney 网页服务 | Own your own ChatGPT+Midjourney web service with one click

Medical

• HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale, arXiv, 2406.19280, arxiv, pdf, cication: -1

  Junying Chen, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Hardy Chen, Xidong Wang, Ruifei Zhang, Zhenyang Cai, Ke Ji, Guangjun Yu

• MedFuzz: Exploring the Robustness of Large Language Models in Medical Question Answering, arXiv, 2406.06573, arxiv, pdf, cication: -1

  Robert Osazuwa Ness, Katie Matton, Hayden Helm, Sheng Zhang, Junaid Bajwa, Carey E. Priebe, Eric Horvitz

• Towards a Personal Health Large Language Model, arXiv, 2406.06474, arxiv, pdf, cication: -1

  Justin Cosentino, Anastasiya Belyaeva, Xin Liu, Nicholas A. Furlotte, Zhun Yang, Chace Lee, Erik Schenck, Yojan Patel, Jian Cui, Logan Douglas Schneider

• Tx-LLM: A Large Language Model for Therapeutics, arXiv, 2406.06316, arxiv, pdf, cication: -1

  Juan Manuel Zambrano Chaves, Eric Wang, Tao Tu, Eeshit Dhaval Vaishnav, Byron Lee, S. Sara Mahdavi, Christopher Semturs, David Fleet, Vivek Natarajan, Shekoofeh Azizi

• Advancing Multimodal Medical Capabilities of Gemini, arXiv, 2405.03162, arxiv, pdf, cication: -1

  Lin Yang, Shawn Xu, Andrew Sellergren, Timo Kohlberger, Yuchen Zhou, Ira Ktena, Atilla Kiraly, Faruk Ahmed, Farhad Hormozdiari, Tiam Jaroensri

• Capabilities of Gemini Models in Medicine, arXiv, 2404.18416, arxiv, pdf, cication: -1

  Khaled Saab, Tao Tu, Wei-Hung Weng, Ryutaro Tanno, David Stutz, Ellery Wulczyn, Fan Zhang, Tim Strother, Chunjong Park, Elahe Vedadi

• BioMistral-7B - BioMistral 🤗

• base-7b-v0.2 - internistai 🤗

• The Open Medical-LLM Leaderboard: Benchmarking Large Language Models in Healthcare

• Medical mT5: An Open-Source Multilingual Text-to-Text LLM for The Medical Domain, arXiv, 2404.07613, arxiv, pdf, cication: -1

  Iker García-Ferrero, Rodrigo Agerri, Aitziber Atutxa Salazar, Elena Cabrio, Iker de la Iglesia, Alberto Lavelli, Bernardo Magnini, Benjamin Molinet, Johana Ramirez-Romero, German Rigau

• MedExpQA: Multilingual Benchmarking of Large Language Models for Medical Question Answering, arXiv, 2404.05590, arxiv, pdf, cication: -1

  Iñigo Alonso, Maite Oronoz, Rodrigo Agerri

• BioMedLM: A 2.7B Parameter Language Model Trained On Biomedical Text, arXiv, 2403.18421, arxiv, pdf, cication: -1

  Elliot Bolton, Abhinav Venigalla, Michihiro Yasunaga, David Hall, Betty Xiong, Tony Lee, Roxana Daneshjou, Jonathan Frankle, Percy Liang, Michael Carbin · (huggingface)

• A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Models, arXiv, 2403.12025, arxiv, pdf, cication: -1

  Stephen R. Pfohl, Heather Cole-Lewis, Rory Sayres, Darlene Neal, Mercy Asiedu, Awa Dieng, Nenad Tomasev, Qazi Mamunur Rashid, Shekoofeh Azizi, Negar Rostamzadeh

• Towards Building Multilingual Language Model for Medicine, arXiv, 2402.13963, arxiv, pdf, cication: -1

  Pengcheng Qiu, Chaoyi Wu, Xiaoman Zhang, Weixiong Lin, Haicheng Wang, Ya Zhang, Yanfeng Wang, Weidi Xie · (MMedLM - MAGIC-AI4Med) · (jiqizhixin)

• Towards 3D Molecule-Text Interpretation in Language Models, arXiv, 2401.13923, arxiv, pdf, cication: 3

  Sihang Li, Zhiyuan Liu, Yanchen Luo, Xiang Wang, Xiangnan He, Kenji Kawaguchi, Tat-Seng Chua, Qi Tian

• Polaris: A Safety-focused LLM Constellation Architecture for Healthcare, arXiv, 2403.13313, arxiv, pdf, cication: -1

  Subhabrata Mukherjee, Paul Gamble, Markel Sanz Ausin, Neel Kant, Kriti Aggarwal, Neha Manjunath, Debajyoti Datta, Zhengliang Liu, Jiayuan Ding, Sophia Busacca

• Towards Training A Chinese Large Language Model for Anesthesiology, arXiv, 2403.02742, arxiv, pdf, cication: -1

  Zhonghai Wang, Jie Jiang, Yibing Zhan, Bohao Zhou, Yanhong Li, Chong Zhang, Liang Ding, Hua Jin, Jun Peng, Xu Lin

• Apollo: Lightweight Multilingual Medical LLMs towards Democratizing Medical AI to 6B People, arXiv, 2403.03640, arxiv, pdf, cication: -1

  Xidong Wang, Nuo Chen, Junyin Chen, Yan Hu, Yidong Wang, Xiangbo Wu, Anningzhe Gao, Xiang Wan, Haizhou Li, Benyou Wang · (apollo - freedomintelligence)

• DrugAssist: A Large Language Model for Molecule Optimization, arXiv, 2401.10334, arxiv, pdf, cication: -1

  Geyan Ye, Xibao Cai, Houtim Lai, Xing Wang, Junhong Huang, Longyue Wang, Wei Liu, Xiangxiang Zeng · (drugassist - blazerye)

• Towards Conversational Diagnostic AI, arXiv, 2401.05654, arxiv, pdf, cication: -1

  Tao Tu, Anil Palepu, Mike Schaekermann, Khaled Saab, Jan Freyberg, Ryutaro Tanno, Amy Wang, Brenna Li, Mohamed Amin, Nenad Tomasev

• Machine Mindset: An MBTI Exploration of Large Language Models, arXiv, 2312.12999, arxiv, pdf, cication: -1

  Jiaxi Cui, Liuzhenghao Lv, Jing Wen, Rongsheng Wang, Jing Tang, YongHong Tian, Li Yuan · (machine-mindset - pku-yuangroup)

• A Foundational Multimodal Vision Language AI Assistant for Human Pathology, arXiv, 2312.07814, arxiv, pdf, cication: -1

  Ming Y. Lu, Bowen Chen, Drew F. K. Williamson, Richard J. Chen, Kenji Ikamura, Georg Gerber, Ivy Liang, Long Phi Le, Tong Ding, Anil V Parwani

• Using Large Language Models to Accelerate Communication for Users with Severe Motor Impairments, arXiv, 2312.01532, arxiv, pdf, cication: -1

  Shanqing Cai, Subhashini Venugopalan, Katie Seaver, Xiang Xiao, Katrin Tomanek, Sri Jalasutram, Meredith Ringel Morris, Shaun Kane, Ajit Narayanan, Robert L. MacDonald

• Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine, arXiv, 2311.16452, arxiv, pdf, cication: -1

  Harsha Nori, Yin Tat Lee, Sheng Zhang, Dean Carignan, Richard Edgar, Nicolo Fusi, Nicholas King, Jonathan Larson, Yuanzhi Li, Weishung Liu

• meditron - epfLLM

  Meditron is a suite of open-source medical Large Language Models (LLMs).

• medagents - gersteinlab

• disc-medllm - fudandisc

  Repository of DISC-MedLLM, it is a comprehensive solution that leverages Large Language Models (LLMs) to provide accurate and truthful medical response in end-to-end conversational healthcare services.

• Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data, arXiv, 2308.02463, arxiv, pdf, cication: -1

  Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, Weidi Xie · (jiqizhixin)

• LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day, arXiv, 2306.00890, arxiv, pdf, cication: 34

  Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, Jianfeng Gao · (LLaVA-Med - microsoft)

• MultiMed - TeamTonic 🤗

• Exploring the Boundaries of GPT-4 in Radiology, arXiv, 2310.14573, arxiv, pdf, cication: -1

  Qianchu Liu, Stephanie Hyland, Shruthi Bannur, Kenza Bouzid, Daniel C. Castro, Maria Teodora Wetscherek, Robert Tinn, Harshita Sharma, Fernando Pérez-García, Anton Schwaighofer

• Can GPT-4V(ision) Serve Medical Applications? Case Studies on GPT-4V for Multimodal Medical Diagnosis, arXiv, 2310.09909, arxiv, pdf, cication: 5

  Chaoyi Wu, Jiayu Lei, Qiaoyu Zheng, Weike Zhao, Weixiong Lin, Xiaoman Zhang, Xiao Zhou, Ziheng Zhao, Ya Zhang, Yanfeng Wang · (jiqizhixin)

• A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics, arXiv, 2310.05694, arxiv, pdf, cication: 3

  Kai He, Rui Mao, Qika Lin, Yucheng Ruan, Xiang Lan, Mengling Feng, Erik Cambria

• Language models in molecular discovery, arXiv, 2309.16235, arxiv, pdf, cication: -1

  Nikita Janakarajan, Tim Erdmann, Sarath Swaminathan, Teodoro Laino, Jannis Born

• Clinical Text Summarization: Adapting Large Language Models Can Outperform Human Experts, arXiv, 2309.07430, arxiv, pdf, cication: 1

  Dave Van Veen, Cara Van Uden, Louis Blankemeier, Jean-Benoit Delbrouck, Asad Aali, Christian Bluethgen, Anuj Pareek, Malgorzata Polacin, Eduardo Pontes Reis, Anna Seehofnerova

• DrugChat: Towards Enabling ChatGPT-Like Capabilities on Drug Molecule Graphs, arXiv, 2309.03907, arxiv, pdf, cication: 2

  Youwei Liang, Ruiyi Zhang, Li Zhang, Pengtao Xie

• OpenBioMed - PharMolix

• PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering, arXiv, 2305.10415, arxiv, pdf, cication: 17

  Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Weixiong Lin, Ya Zhang, Yanfeng Wang, Weidi Xie · (jiqizhixin)

• Scaling Clinical Trial Matching Using Large Language Models: A Case Study in Oncology, arXiv, 2308.02180, arxiv, pdf, cication: -1

  Cliff Wong, Sheng Zhang, Yu Gu, Christine Moung, Jacob Abel, Naoto Usuyama, Roshanthi Weerasinghe, Brian Piening, Tristan Naumann, Carlo Bifulco

• ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoders, arXiv, 2308.01317, arxiv, pdf, cication: 8

  Shawn Xu, Lin Yang, Christopher Kelly, Marcin Sieniek, Timo Kohlberger, Martin Ma, Wei-Hung Weng, Atilla Kiraly, Sahar Kazemzadeh, Zakkai Melamed

• Med-Flamingo: a Multimodal Medical Few-shot Learner, arXiv, 2307.15189, arxiv, pdf, cication: 9

  Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Cyril Zakka, Yash Dalmia, Eduardo Pontes Reis, Pranav Rajpurkar, Jure Leskovec

Other

• 首个千亿生物医药ChatGPT来了！清华AIR聂再清：这个行业未来的"Killer APP" | 量子位

Anomaly detection

• Towards Generic Anomaly Detection and Understanding: Large-scale Visual-linguistic Model (GPT-4V) Takes the Lead, arXiv, 2311.02782, arxiv, pdf, cication: -1

  Yunkang Cao, Xiaohao Xu, Chen Sun, Xiaonan Huang, Weiming Shen · (jiqizhixin)

Auto driving

• PlanGPT: Enhancing Urban Planning with Tailored Language Model and Efficient Retrieval, arXiv, 2402.19273, arxiv, pdf, cication: -1

  He Zhu, Wenjia Zhang, Nuoxian Huang, Boyang Li, Luyao Niu, Zipei Fan, Tianle Lun, Yicheng Tao, Junyou Su, Zhaoya Gong

• DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models, arXiv, 2402.12289, arxiv, pdf, cication: -1

  Xiaoyu Tian, Junru Gu, Bailin Li, Yicheng Liu, Chenxu Hu, Yang Wang, Kun Zhan, Peng Jia, Xianpeng Lang, Hang Zhao · (tsinghua-mars-lab.github) · (jiqizhixin)

• Driving Everywhere with Large Language Model Policy Adaptation, arXiv, 2402.05932, arxiv, pdf, cication: -1

  Boyi Li, Yue Wang, Jiageng Mao, Boris Ivanovic, Sushant Veer, Karen Leung, Marco Pavone · (boyiliee.github)

• Self-driving as a case study for AGI

• LMDrive: Closed-Loop End-to-End Driving with Large Language Models, arXiv, 2312.07488, arxiv, pdf, cication: -1

  Hao Shao, Yuxuan Hu, Letian Wang, Steven L. Waslander, Yu Liu, Hongsheng Li · (lmdrive - opendilab)

• Evaluation of Large Language Models for Decision Making in Autonomous Driving, arXiv, 2312.06351, arxiv, pdf, cication: -1

  Kotaro Tanahashi, Yuichi Inoue, Yu Yamaguchi, Hidetatsu Yaginuma, Daiki Shiotsuka, Hiroyuki Shimatani, Kohei Iwamasa, Yoshiaki Inoue, Takafumi Yamaguchi, Koki Igari

• On the Road with GPT-4V(ision): Early Explorations of Visual-Language Model on Autonomous Driving, arXiv, 2311.05332, arxiv, pdf, cication: -1

  Licheng Wen, Xuemeng Yang, Daocheng Fu, Xiaofeng Wang, Pinlong Cai, Xin Li, Tao Ma, Yingxuan Li, Linran Xu, Dengke Shang · (gpt4v-ad-exploration - pjlab-adg)

Education

• Simulating Classroom Education with LLM-Empowered Agents, arXiv, 2406.19226, arxiv, pdf, cication: -1

  Zheyuan Zhang, Daniel Zhang-Li, Jifan Yu, Linlu Gong, Jinchang Zhou, Zhiyuan Liu, Lei Hou, Juanzi Li

• Mr.-Ranedeer-AI-Tutor - JushBJJ

  A GPT-4 AI Tutor Prompt for customizable personalized learning experiences.

• gpt_academic - binary-husky

  为ChatGPT/GLM提供实用化交互界面，特别优化论文阅读/润色/写作体验，模块化设计，支持自定义快捷按钮&函数插件，支持Python和C++等项目剖析&自译解功能，PDF/LaTex论文翻译&总结功能，支持并行问询多种LLM模型，支持chatglm2等本地模型。兼容文心一言, moss, llama2, rwkv, claude2, 通义千问, 书生, 讯飞星火等。

• Mr.-Ranedeer-AI-Tutor - JushBJJ

  A GPT-4 AI Tutor Prompt for customizable personalized learning experiences.

• Exploring the MIT Mathematics and EECS Curriculum Using Large Language Models, arXiv, 2306.08997, arxiv, pdf, cication: 7

  Sarah J. Zhang, Samuel Florin, Ariel N. Lee, Eamon Niknafs, Andrei Marginean, Annie Wang, Keith Tyser, Zad Chin, Yann Hicke, Nikhil Singh

Other

• Paper Brief
• Nature｜LLM正在重塑教育，所有学生都需要学习AI，RAG是解决幻觉的关键
• 大模型将如何重塑教育科技？
• 首个大模型教育产品开箱："最聪明"国产大模型加持的学习机，质变了吗 | 量子位

Text Related

• llama-zip - AlexBuz

  LLM-powered lossless compression tool

• Multi-Dimensional Optimization for Text Summarization via Reinforcement Learning, arXiv, 2406.00303, arxiv, pdf, cication: -1

  Sangwon Ryu, Heejin Do, Yunsu Kim, Gary Geunbae Lee, Jungseul Ok

• Proofread: Fixes All Errors with One Tap, arXiv, 2406.04523, arxiv, pdf, cication: -1

  Renjie Liu, Yanxiang Zhang, Yun Zhu, Haicheng Sun, Yuanbo Zhang, Michael Xuelin Huang, Shanqing Cai, Lei Meng, Shumin Zhai

• langchain-extract - langchain-ai

  🦜⛏️ Did you say you like data?

• TnT-LLM: Text Mining at Scale with Large Language Models, arXiv, 2403.12173, arxiv, pdf, cication: -1

  Mengting Wan, Tara Safavi, Sujay Kumar Jauhar, Yujin Kim, Scott Counts, Jennifer Neville, Siddharth Suri, Chirag Shah, Ryen W White, Longqi Yang

• Scaling Up LLM Reviews for Google Ads Content Moderation, arXiv, 2402.14590, arxiv, pdf, cication: -1

  Wei Qiao, Tushar Dogra, Otilia Stretcu, Yu-Han Lyu, Tiantian Fang, Dongjin Kwon, Chun-Ta Lu, Enming Luo, Yuan Wang, Chih-Chun Chia

• Improving Black-box Robustness with In-Context Rewriting, arXiv, 2402.08225, arxiv, pdf, cication: -1

  Kyle O'Brien, Nathan Ng, Isha Puri, Jorge Mendez, Hamid Palangi, Yoon Kim, Marzyeh Ghassemi, Thomas Hartvigsen

• Tiny Titans: Can Smaller Large Language Models Punch Above Their Weight in the Real World for Meeting Summarization?, arXiv, 2402.00841, arxiv, pdf, cication: -1

  Xue-Yong Fu, Md Tahmid Rahman Laskar, Elena Khasanova, Cheng Chen, Shashi Bhushan TN

• Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding, arXiv, 2401.04398, arxiv, pdf, cication: -1

  Zilong Wang, Hao Zhang, Chun-Liang Li, Julian Martin Eisenschlos, Vincent Perot, Zifeng Wang, Lesly Miculicich, Yasuhisa Fujii, Jingbo Shang, Chen-Yu Lee

• DocGraphLM: Documental Graph Language Model for Information Extraction, arXiv, 2401.02823, arxiv, pdf, cication: -1

  Dongsheng Wang, Zhiqiang Ma, Armineh Nourbakhsh, Kang Gu, Sameena Shah

• Improving Text Embeddings with Large Language Models, arXiv, 2401.00368, arxiv, pdf, cication: -1

  Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei

• Large Language Models for Generative Information Extraction: A Survey, arXiv, 2312.17617, arxiv, pdf, cication: -1

  Derong Xu, Wei Chen, Wenjun Peng, Chao Zhang, Tong Xu, Xiangyu Zhao, Xian Wu, Yefeng Zheng, Enhong Chen · (awesome-llm4ie-papers - quqxui)

Recommendation

• Item-Language Model for Conversational Recommendation, arXiv, 2406.02844, arxiv, pdf, cication: -1

  Li Yang, Anushya Subbiah, Hardik Patel, Judith Yue Li, Yanwei Song, Reza Mirghaderi, Vikram Aggarwal

• recai - microsoft

• Exploring the Impact of Large Language Models on Recommender Systems: An Extensive Review, arXiv, 2402.18590, arxiv, pdf, cication: -1

  Arpita Vats, Vinija Jain, Rahul Raja, Aman Chadha

• INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning, arXiv, 2401.06532, arxiv, pdf, cication: -1

  Yutao Zhu, Peitian Zhang, Chenghao Zhang, Yifei Chen, Binyu Xie, Zhicheng Dou, Zheng Liu, Ji-Rong Wen

• Making Large Language Models A Better Foundation For Dense Retrieval, arXiv, 2312.15503, arxiv, pdf, cication: -1

  Chaofan Li, Zheng Liu, Shitao Xiao, Yingxia Shao

• Rank-without-GPT: Building GPT-Independent Listwise Rerankers on Open-Source Large Language Models, arXiv, 2312.02969, arxiv, pdf, cication: -1

  Xinyu Zhang, Sebastian Hofstätter, Patrick Lewis, Raphael Tang, Jimmy Lin

• Hiformer: Heterogeneous Feature Interactions Learning with Transformers for Recommender Systems, arXiv, 2311.05884, arxiv, pdf, cication: -1

  Huan Gui, Ruoxi Wang, Ke Yin, Long Jin, Maciej Kula, Taibai Xu, Lichan Hong, Ed H. Chi

• RLMRec - HKUDS

  "RLMRec: Representation Learning with Large Language Models for Recommendation"

• On Generative Agents in Recommendation, arXiv, 2310.10108, arxiv, pdf, cication: -1

  An Zhang, Leheng Sheng, Yuxin Chen, Hao Li, Yang Deng, Xiang Wang, Tat-Seng Chua · (Agent4Rec - LehengTHU)

• LLM-Rec: Personalized Recommendation via Prompting Large Language Models, arXiv, 2307.15780, arxiv, pdf, cication: 3

  Hanjia Lyu, Song Jiang, Hanqing Zeng, Qifan Wang, Si Zhang, Ren Chen, Chris Leung, Jiajie Tang, Yinglong Xia, Jiebo Luo

• Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based Preferences, proceedings of the 17th acm conference on recommender systems, 2023, arxiv, pdf, cication: 5

  Scott Sanner, Krisztian Balog, Filip Radlinski, Ben Wedin, Lucas Dixon

• awesome-llm-for-recsys - chiangel

  Survey: A collection of AWESOME papers and resources on the large language model (LLM) related recommender system topics.

• Advances and Challenges in Conversational Recommender Systems: A Survey, arXiv, 2101.09459, arxiv, pdf, cication: -1

  Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke, Tat-Seng Chua · (mp.weixin.qq)

Role play

• DreamGen Opus v1.4 70B Llama 3 - a dreamgen Collection

  · (reddit)

• CharacterCodex - NousResearch 🤗

• Higgs-Llama-3-70B - bosonai 🤗

• Introducing NPC-Playground, a 3D playground to interact with LLM-powered NPCs

• ChatQA: Building GPT-4 Level Conversational QA Models, arXiv, 2401.10225, arxiv, pdf, cication: -1

  Zihan Liu, Wei Ping, Rajarshi Roy, Peng Xu, Chankyu Lee, Mohammad Shoeybi, Bryan Catanzaro

• Social Skill Training with Large Language Models, arXiv, 2404.04204, arxiv, pdf, cication: -1

  Diyi Yang, Caleb Ziems, William Held, Omar Shaikh, Michael S. Bernstein, John Mitchell

• ragdoll - bennyschmidt

  The library for character-driven AI experiences. · (twitter)

• ChatGPT Role-play Dataset: Analysis of User Motives and Model Naturalness, arXiv, 2403.18121, arxiv, pdf, cication: -1

  Yufei Tao, Ameeta Agrawal, Judit Dombi, Tetyana Sydorenko, Jung In Lee

• Human Simulacra: A Step toward the Personification of Large Language Models, arXiv, 2402.18180, arxiv, pdf, cication: -1

  Qiuejie Xie, Qiming Feng, Tianqi Zhang, Qingqiu Li, Yuejie Zhang, Rui Feng, Shang Gao

• Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment, arXiv, 2401.12474, arxiv, pdf, cication: -1

  Keming Lu, Bowen Yu, Chang Zhou, Jingren Zhou · (Ditto - OFA-Sys)

• ai-adventure - steamship-core

• RoleCraft-GLM: Advancing Personalized Role-Playing in Large Language Models, arXiv, 2401.09432, arxiv, pdf, cication: -1

  Meiling Tao, Xuechen Liang, Tianyu Shi, Lei Yu, Yiting Xie

• PersonalityChat: Conversation Distillation for Personalized Dialog Modeling with Facts and Traits, arXiv, 2401.07363, arxiv, pdf, cication: -1

  Ehsan Lotfi, Maxime De Bruyn, Jeska Buhmann, Walter Daelemans

• Enhancing the Emotional Generation Capability of Large Language Models via Emotional Chain-of-Thought, arXiv, 2401.06836, arxiv, pdf, cication: -1

  Zaijing Li, Gongwei Chen, Rui Shao, Dongmei Jiang, Liqiang Nie

• From LLM to Conversational Agent: A Memory Enhanced Architecture with Fine-Tuning of Large Language Models, arXiv, 2401.02777, arxiv, pdf, cication: -1

  Na Liu, Liangyu Chen, Xiaoyu Tian, Wei Zou, Kaijiang Chen, Ming Cui

• LARP: Language-Agent Role Play for Open-World Games, arXiv, 2312.17653, arxiv, pdf, cication: -1

  Ming Yan, Ruihao Li, Hao Zhang, Hao Wang, Zhilan Yang, Ji Yan · (miao-ai-lab.github)

• Role play with large language models | Nature

  · (mp.weixin.qq)

• Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery Games, arXiv, 2312.00746, arxiv, pdf, cication: -1

  Dekun Wu, Haochen Shi, Zhiyuan Sun, Bang Liu · (jiqizhixin)

• Faithful Persona-based Conversational Dataset Generation with Large Language Models, arXiv, 2312.10007, arxiv, pdf, cication: -1

  Pegah Jandaghi, XiangHai Sheng, Xinyi Bai, Jay Pujara, Hakim Sidahmed

• CharacterGLM: Customizing Chinese Conversational AI Characters with Large Language Models, arXiv, 2311.16832, arxiv, pdf, cication: -1

  Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Libiao Peng, Jiaming Yang, Xiyao Xiao · (CharacterGLM-6B - thu-coai)

• Does Role-Playing Chatbots Capture the Character Personalities? Assessing Personality Traits for Role-Playing Chatbots, arXiv, 2310.17976, arxiv, pdf, cication: 1

  Xintao Wang, Quan Tu, Yaying Fei, Ziang Leng, Cheng Li · (mp.weixin.qq) · (Chat-Haruhi-Suzumiya - LC1332)

• GRIM: GRaph-based Interactive narrative visualization for gaMes, arXiv, 2311.09213, arxiv, pdf, cication: -1

  Jorge Leandro, Sudha Rao, Michael Xu, Weijia Xu, Nebosja Jojic, Chris Brockett, Bill Dolan

• ChatAnything: Facetime Chat with LLM-Enhanced Personas, arXiv, 2311.06772, arxiv, pdf, cication: -1

  Yilin Zhao, Xinbin Yuan, Shanghua Gao, Zhijie Lin, Qibin Hou, Jiashi Feng, Daquan Zhou · (chatanything.github)

• Character-LLM: A Trainable Agent for Role-Playing, arXiv, 2310.10158, arxiv, pdf, cication: -1

  Yunfan Shao, Linyang Li, Junqi Dai, Xipeng Qiu · (trainable-agents - choosewhatulike)

• zero2story - chansung 🤗

• rolellm-public - interactivenlp-team

  RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models

Chat bots

• Designing a Dashboard for Transparency and Control of Conversational AI, arXiv, 2406.07882, arxiv, pdf, cication: -1

  Yida Chen, Aoyu Wu, Trevor DePodesta, Catherine Yeh, Kenneth Li, Nicholas Castillo Marin, Oam Patel, Jan Riecke, Shivam Raval, Olivia Seow

• alloy-voice-assistant - svpino

• bilibot - linyiLYi

  A local chatbot fine-tuned by bilibili user comments.

• Large Language Models as Zero-shot Dialogue State Tracker through Function Calling, arXiv, 2402.10466, arxiv, pdf, cication: -1

  Zekun Li, Zhiyu Zoey Chen, Mike Ross, Patrick Huber, Seungwhan Moon, Zhaojiang Lin, Xin Luna Dong, Adithya Sagar, Xifeng Yan, Paul A. Crook

• AGI-Samantha - BRlkl

  AGI has been achieved externally · (mp.weixin.qq)

• HuixiangDou: Overcoming Group Chat Scenarios with LLM-based Technical Assistance, arXiv, 2401.08772, arxiv, pdf, cication: -1

  Huanjun Kong, Songyang Zhang, Kai Chen · (huixiangdou - internlm)

• anything-llm - Mintplex-Labs

  Open-source ChatGPT equivalent experience for both open and close source LLMs, embedders, and vector databases. Supports unlimited documents, threads, and concurrent users and management all in a very clean UI.

• voice-assistant - linyiLYi

  一个简单的 Python 脚本，可以通过语音与本地大语言模型进行对话。

• core - cheshire-cat-ai

  Production ready AI assistant framework

• SillyTavern - SillyTavern

  LLM Frontend for Power Users.

• Chat-Haruhi-Suzumiya - LC1332

  Chat凉宫春日, 由李鲁鲁, 冷子昂等同学开发的模仿二次元对话的聊天机器人。

• OpenDAN-Personal-AI-OS - fiatrete

  OpenDAN is an open source Personal AI OS , which consolidates various AI modules in one place for your personal use.

• PromptAppGPT - mleoking

  A rapid prompt app development framework based on GPT

• chainlit - Chainlit

  Build Python LLM apps in minutes ⚡️

• OpenChat - openchatai

  Run and create custom ChatGPT-like bots with OpenChat, embed and share these bots anywhere, the open-source chatbot console.

• dify - langgenius

  One API for plugins and datasets, one interface for prompt engineering and visual operation, all for creating powerful AI applications.

• lobe-chat - lobehub

  🤖 Lobe Chat - an open-source, extensible (Function Calling), high-performance chatbot framework. It supports one-click free deployment of your private ChatGPT/LLM web application.

• Toxicity in ChatGPT: Analyzing Persona-assigned Language Models, arXiv, 2304.05335, arxiv, pdf, cication: 63

  Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, Karthik Narasimhan

• companion-app - a16z-infra

  AI companions with memory: a lightweight stack to create and host your own AI companions

Other

• 小扎发布虚拟聊天AI，竟「1:1复刻」Character.AI？用户吐槽：设定太老土了

Law

• InternLM-Law: An Open Source Chinese Legal Large Language Model, arXiv, 2406.14887, arxiv, pdf, cication: -1

  Zhiwei Fei, Songyang Zhang, Xiaoyu Shen, Dawei Zhu, Xiao Wang, Maosong Cao, Fengzhe Zhou, Yining Li, Wenwei Zhang, Dahua Lin · (InternLM-Law - InternLM)

• FLawN-T5: An Empirical Examination of Effective Instruction-Tuning Data Mixtures for Legal Reasoning, arXiv, 2404.02127, arxiv, pdf, cication: -1

  Joel Niklaus, Lucia Zheng, Arya D. McCarthy, Christopher Hahn, Brian M. Rosen, Peter Henderson, Daniel E. Ho, Garrett Honke, Percy Liang, Christopher Manning

• SaulLM-7B: A pioneering Large Language Model for Law, arXiv, 2403.03883, arxiv, pdf, cication: -1

  Pierre Colombo, Telmo Pessoa Pires, Malik Boudiaf, Dominic Culver, Rui Melo, Caio Corro, Andre F. T. Martins, Fabrizio Esposito, Vera Lúcia Raposo, Sofia Morgado

• MoZi - AI-for-Science

  首个全参数训练的知识产权大模型 MoZi (墨子)

• chatlaw - pku-yuangroup

  中文法律大模型

• DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services, arXiv, 2309.11325, arxiv, pdf, cication: 2

  Shengbin Yue, Wei Chen, Siyuan Wang, Bingxuan Li, Chenchen Shen, Shujun Liu, Yuxuan Zhou, Yao Xiao, Song Yun, Xuanjing Huang · (disc-lawllm - fudandisc) · (jiqizhixin)

Machine Translation

• Fetching Title#vjim

• (Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts, arXiv, 2405.11804, arxiv, pdf, cication: -1

  Minghao Wu, Yulin Yuan, Gholamreza Haffari, Longyue Wang

• Tuning LLMs with Contrastive Alignment Instructions for Machine Translation in Unseen, Low-resource Languages, arXiv, 2401.05811, arxiv, pdf, cication: -1

  Zhuoyuan Mao, Yen Yu

• Dissecting In-Context Learning of Translations in GPTs, arXiv, 2310.15987, arxiv, pdf, cication: -1

  Vikas Raunak, Hany Hassan Awadalla, Arul Menezes

• A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models, arXiv, 2309.11674, arxiv, pdf, cication: 1

  Haoran Xu, Young Jin Kim, Amr Sharaf, Hany Hassan Awadalla

• The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation, arXiv, 2308.07286, arxiv, pdf, cication: 3

  Patrick Fernandes, Daniel Deutsch, Mara Finkelstein, Parker Riley, André F. T. Martins, Graham Neubig, Ankush Garg, Jonathan H. Clark, Markus Freitag, Orhan Firat

Audio

• Where Visual Speech Meets Language: VSP-LLM Framework for Efficient and Context-Aware Visual Speech Processing, arXiv, 2402.15151, arxiv, pdf, cication: -1

  Jeong Hun Yeo, Seunghee Han, Minsu Kim, Yong Man Ro

  · (vsp-llm - sally-sh)

• Rambler: Supporting Writing With Speech via LLM-Assisted Gist Manipulation, arXiv, 2401.10838, arxiv, pdf, cication: -1

  Susan Lin, Jeremy Warner, J. D. Zamfirescu-Pereira, Matthew G. Lee, Sauhard Jain, Michael Xuelin Huang, Piyawat Lertvittayakumjorn, Shanqing Cai, Shumin Zhai, Björn Hartmann

• Large Language Models are Efficient Learners of Noise-Robust Speech Recognition, arXiv, 2401.10446, arxiv, pdf, cication: -1

  Yuchen Hu, Chen Chen, Chao-Han Huck Yang, Ruizhe Li, Chao Zhang, Pin-Yu Chen, EnSiong Chng

• DiarizationLM: Speaker Diarization Post-Processing with Large Language Models, arXiv, 2401.03506, arxiv, pdf, cication: -1

  Quan Wang, Yiling Huang, Guanlong Zhao, Evan Clark, Wei Xia, Hank Liao

• Document-Level Machine Translation with Large Language Models, arXiv, 2304.02210, arxiv, pdf, cication: -1

  Longyue Wang, Chenyang Lyu, Tianbo Ji, Zhirui Zhang, Dian Yu, Shuming Shi, Zhaopeng Tu · (document-mt-llm - longyuewangdcu)

Design

• DeTikZify: Synthesizing Graphics Programs for Scientific Figures and Sketches with TikZ, arXiv, 2405.15306, arxiv, pdf, cication: -1

  Jonas Belouadi, Simone Paolo Ponzetto, Steffen Eger · (detikzify - potamides)

• PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLM, arXiv, 2406.02884, arxiv, pdf, cication: -1

  Tao Yang, Yingmin Luo, Zhongang Qi, Yang Wu, Ying Shan, Chang Wen Chen · (PosterLLaVA - posterllava)

• openui - wandb

  OpenUI let's you describe UI using your imagination, then see it rendered live.

• Keyframer: Empowering Animation Design using Large Language Models, arXiv, 2402.06071, arxiv, pdf, cication: -1

  Tiffany Tseng, Ruijia Cheng, Jeffrey Nichols

Vision

• Omost - lllyasviel

  Your image is almost there!

• Adapting LLaMA Decoder to Vision Transformer, arXiv, 2404.06773, arxiv, pdf, cication: -1

  Jiahao Wang, Wenqi Shao, Mengzhao Chen, Chengyue Wu, Yong Liu, Kaipeng Zhang, Songyang Zhang, Kai Chen, Ping Luo

• ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment, arXiv, 2403.05135, arxiv, pdf, cication: -1

  Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, Gang Yu

• Modeling Collaborator: Enabling Subjective Vision Classification With Minimal Human Effort via LLM Tool-Use, arXiv, 2403.02626, arxiv, pdf, cication: -1

  Imad Eddine Toubal, Aditya Avinash, Neil Gordon Alldrin, Jan Dlabal, Wenlei Zhou, Enming Luo, Otilia Stretcu, Hao Xiong, Chun-Ta Lu, Howard Zhou

• MuLan: Multimodal-LLM Agent for Progressive Multi-Object Diffusion, arXiv, 2402.12741, arxiv, pdf, cication: -1

  Sen Li, Ruochen Wang, Cho-Jui Hsieh, Minhao Cheng, Tianyi Zhou · (measure-infinity.github)

• LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing, arXiv, 2402.10294, arxiv, pdf, cication: -1

  Bryan Wang, Yuliang Li, Zhaoyang Lv, Haijun Xia, Yan Xu, Raj Sodhi

  · (jiqizhixin)

• Guiding Instruction-based Image Editing via Multimodal Large Language Models, arXiv, 2309.17102, arxiv, pdf, cication: 7

  Tsu-Jui Fu, Wenze Hu, Xianzhi Du, William Yang Wang, Yinfei Yang, Zhe Gan · (ml-mgie - apple) · (mp.weixin.qq)

• A Vision Check-up for Language Models, arXiv, 2401.01862, arxiv, pdf, cication: -1

  Pratyusha Sharma, Tamar Rott Shaham, Manel Baradad, Stephanie Fu, Adrian Rodriguez-Munoz, Shivam Duggal, Phillip Isola, Antonio Torralba

• Taiyi-Diffusion-XL: Advancing Bilingual Text-to-Image Generation with Large Vision-Language Model Support, arXiv, 2401.14688, arxiv, pdf, cication: -1

  Xiaojun Wu, Dixiang Zhang, Ruyi Gan, Junyu Lu, Ziwei Wu, Renliang Sun, Jiaxing Zhang, Pingjian Zhang, Yan Song

  • https://huggingface.co/IDEA-CCNL/Taiyi-Stable-Diffusion-XL-3.5B

• DiffusionGPT: LLM-Driven Text-to-Image Generation System, arXiv, 2401.10061, arxiv, pdf, cication: -1

  Jie Qin, Jie Wu, Weifeng Chen, Yuxi Ren, Huixia Li, Hefeng Wu, Xuefeng Xiao, Rui Wang, Shilei Wen

• GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation, arXiv, 2401.04092, arxiv, pdf, cication: -1

  Tong Wu, Guandao Yang, Zhibing Li, Kai Zhang, Ziwei Liu, Leonidas Guibas, Dahua Lin, Gordon Wetzstein

• GPT4Point: A Unified Framework for Point-Language Understanding and Generation, arXiv, 2312.02980, arxiv, pdf, cication: -1

  Zhangyang Qi, Ye Fang, Zeyi Sun, Xiaoyang Wu, Tong Wu, Jiaqi Wang, Dahua Lin, Hengshuang Zhao

• GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planning, arXiv, 2311.12631, arxiv, pdf, cication: -1

  Jiaxi Lv, Yi Huang, Mingfu Yan, Jiancheng Huang, Jianzhuang Liu, Yifan Liu, Yafei Wen, Xiaoxin Chen, Shifeng Chen

  · (gpt4motion.github) · (GPT4Motion - jiaxilv) · (jiqizhixin)

• MM-VID: Advancing Video Understanding with GPT-4V(ision), arXiv, 2310.19773, arxiv, pdf, cication: 3

  Kevin Lin, Faisal Ahmed, Linjie Li, Chung-Ching Lin, Ehsan Azarnasab, Zhengyuan Yang, Jianfeng Wang, Lin Liang, Zicheng Liu, Yumao Lu · (jiqizhixin) · (multimodal-vid.github)

• LayoutPrompter: Awaken the Design Ability of Large Language Models, arXiv, 2311.06495, arxiv, pdf, cication: -1

  Jiawei Lin, Jiaqi Guo, Shizhao Sun, Zijiang James Yang, Jian-Guang Lou, Dongmei Zhang · (LayoutGeneration - microsoft)

• S3-DST: Structured Open-Domain Dialogue Segmentation and State Tracking in the Era of LLMs, arXiv, 2309.08827, arxiv, pdf, cication: -1

  Sarkar Snigdha Sarathi Das, Chirag Shah, Mengting Wan, Jennifer Neville, Longqi Yang, Reid Andersen, Georg Buscher, Tara Safavi

• layoutnuwa - projectnuwa

• PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalization, ICCV, 2023, arxiv, pdf, cication: -1

  Junhyeong Cho, Gilhyun Nam, Sungyeon Kim, Hunmin Yang, Suha Kwak

• SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs, arXiv, 2306.17842, arxiv, pdf, cication: 2

  Lijun Yu, Yong Cheng, Zhiruo Wang, Vivek Kumar, Wolfgang Macherey, Yanping Huang, David A. Ross, Irfan Essa, Yonatan Bisk, Ming-Hsuan Yang

• Generating Images with Multimodal Language Models, arXiv, 2305.17216, arxiv, pdf, cication: 28

  Jing Yu Koh, Daniel Fried, Ruslan Salakhutdinov · (jykoh) · (gill - kohjingyu)

• layoutgpt - weixi-feng

  Official repo for LayoutGPT

Science

• $\nabla^2$DFT: A Universal Quantum Chemistry Dataset of Drug-Like Molecules and a Benchmark for Neural Network Potentials, arXiv, 2406.14347, arxiv, pdf, cication: -1

  Kuzma Khrabrov, Anton Ber, Artem Tsypin, Konstantin Ushenin, Egor Rumiantsev, Alexander Telepov, Dmitry Protasov, Ilya Shenbin, Anton Alekseev, Mikhail Shirokikh

• AstroPT: Scaling Large Observation Models for Astronomy, arXiv, 2405.14930, arxiv, pdf, cication: -1

  Michael J. Smith, Ryan J. Roberts, Eirini Angeloudi, Marc Huertas-Company · (astroPT - Smith42)

• INDUS: Effective and Efficient Language Models for Scientific Applications, arXiv, 2405.10725, arxiv, pdf, cication: -1

  Bishwaranjan Bhattacharjee, Aashka Trivedi, Masayasu Muraoka, Muthukumaran Ramasubramanian, Takuma Udagawa, Iksha Gurung, Rong Zhang, Bharath Dandala, Rahul Ramachandran, Manil Maskey

• Wu's Method can Boost Symbolic AI to Rival Silver Medalists and AlphaGeometry to Outperform Gold Medalists at IMO Geometry, arXiv, 2404.06405, arxiv, pdf, cication: -1

  Shiven Sinha, Ameya Prabhu, Ponnurangam Kumaraguru, Siddharth Bhat, Matthias Bethge

• Are large language models superhuman chemists?, arXiv, 2404.01475, arxiv, pdf, cication: -1

  Adrian Mirza, Nawaf Alampara, Sreekanth Kunchapu, Benedict Emoekabu, Aswanth Krishnan, Mara Wilhelmi, Macjonathan Okereke, Juliane Eberhardt, Amir Mohammad Elahi, Maximilian Greiner

• Electron density-based GPT for optimization and suggestion of host–guest binders | Nature Computational Science

  · (jiqizhixin)

• evo - evo-design

  DNA foundation modeling from molecular to genome scale · (jiqizhixin)

• BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains, arXiv, 2402.10373, arxiv, pdf, cication: -1

  Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, Richard Dufour · (jiqizhixin)

• Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science, arXiv, 2402.04247, arxiv, pdf, cication: -1

  Xiangru Tang, Qiao Jin, Kunlun Zhu, Tongxin Yuan, Yichi Zhang, Wangchunshu Zhou, Meng Qu, Yilun Zhao, Jian Tang, Zhuosheng Zhang · (jiqizhixin)

• ChatCell: Facilitating Single-Cell Analysis with Natural Language, arXiv, 2402.08303, arxiv, pdf, cication: -1

  Yin Fang, Kangwei Liu, Ningyu Zhang, Xinle Deng, Penghui Yang, Zhuo Chen, Xiangru Tang, Mark Gerstein, Xiaohui Fan, Huajun Chen

• ChemLLM: A Chemical Large Language Model, arXiv, 2402.06852, arxiv, pdf, cication: -1

  Di Zhang, Wei Liu, Qian Tan, Jingdan Chen, Hang Yan, Yuliang Yan, Jiatong Li, Weiran Huang, Xiangyu Yue, Dongzhan Zhou

  • hf.co/AI4Chem/ChemLLM-7B-Chat

• Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domains, arXiv, 2402.05140, arxiv, pdf, cication: -1

  Junhong Shen, Neil Tenenholtz, James Brian Hall, David Alvarez-Melis, Nicolo Fusi

• Fine-Tuned Language Models Generate Stable Inorganic Materials as Text, arXiv, 2402.04379, arxiv, pdf, cication: -1

  Nate Gruver, Anuroop Sriram, Andrea Madotto, Andrew Gordon Wilson, C. Lawrence Zitnick, Zachary Ulissi

• Scientific Large Language Models: A Survey on Biological & Chemical Domains, arXiv, 2401.14656, arxiv, pdf, cication: -1

  Qiang Zhang, Keyang Ding, Tianwen Lyv, Xinda Wang, Qingyu Yin, Yiwen Zhang, Jing Yu, Yuhao Wang, Xiaotong Li, Zhuoyi Xiang · (jiqizhixin)

• ChemLLM-7B-Chat - AI4Chem 🤗

  · (qbitai)

• Solving olympiad geometry without human demonstrations | Nature

  · (qbitai)

• SciGLM: Training Scientific Language Models with Self-Reflective Instruction Annotation and Tuning, arXiv, 2401.07950, arxiv, pdf, cication: -1

  Dan Zhang, Ziniu Hu, Sining Zhoubian, Zhengxiao Du, Kaiyu Yang, Zihan Wang, Yisong Yue, Yuxiao Dong, Jie Tang

• GeoGalactica: A Scientific Large Language Model in Geoscience, arXiv, 2401.00434, arxiv, pdf, cication: -1

  Zhouhan Lin, Cheng Deng, Le Zhou, Tianhang Zhang, Yi Xu, Yutong Xu, Zhongmou He, Yuanyuan Shi, Beiya Dai, Yunchong Song

• Autonomous chemical research with large language models | Nature

  · (mp.weixin.qq)

• MechGPT, a language-based strategy for mechanics and materials modeling that connects knowledge across scales, disciplines and modalities, applied mechanics reviews, 2023, arxiv, pdf, cication: 1

  Markus J. Buehler

  · (jiqizhixin)

• The Impact of Large Language Models on Scientific Discovery: a Preliminary Study using GPT-4, arXiv, 2311.07361, arxiv, pdf, cication: -1

  Microsoft Research AI4Science, Microsoft Azure Quantum

  · (jiqizhixin)

• MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation, arXiv, 2310.03302, arxiv, pdf, cication: -1

  Qian Huang, Jian Vora, Percy Liang, Jure Leskovec · (MLAgentBench - snap-stanford)

• Large Language Model for Science: A Study on P vs. NP, arXiv, 2309.05689, arxiv, pdf, cication: 2

  Qingxiu Dong, Li Dong, Ke Xu, Guangyan Zhou, Yaru Hao, Zhifang Sui, Furu Wei

• chemcrow-public - ur-whitelab

  Chemcrow

• 陶哲轩力荐！史上最全「数学AI资源」清单出炉

• 快速筛选海量数据，即时做出明智决策，MIT、普林斯顿&卡内基梅隆大学团队利用LLM进行聚变研究 | 机器之心

• 2023年科研领域「 科学ChatGPT」有哪些？LLM for Science有哪些探索...... | 机器之心

Finance

• finrobot - ai4finance-foundation

  FinRobot: An Open-Source AI Agent Platform for Financial Applications using LLMs 🚀 🚀 🚀

• gpt-investor - mshumer

• langchain-financial-agent.ipynb - Colaboratory

• Can Large Language Models Beat Wall Street? Unveiling the Potential of AI in Stock Selection, arXiv, 2401.03737, arxiv, pdf, cication: -1

  Georgios Fatouros, Konstantinos Metaxas, John Soldatos, Dimosthenis Kyriazis · (mp.weixin.qq) · (marketsense-ai)

• FinanceBench: A New Benchmark for Financial Question Answering, arXiv, 2311.11944, arxiv, pdf, cication: 7

  Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, Bertie Vidgen · (huggingface)

• Technical Report: Large Language Models can Strategically Deceive their Users when Put Under Pressure, arXiv, 2311.07590, arxiv, pdf, cication: -1

  Jérémy Scheurer, Mikita Balesni, Marius Hobbhahn

• fingpt - ai4finance-foundation

  Data-Centric FinGPT. Open-source for open finance! Revolutionize 🔥 We'll soon release the trained model.

• FinGLM - MetaGLM

• Data-Centric Financial Large Language Models, arXiv, 2310.17784, arxiv, pdf, cication: -1

  Zhixuan Chu, Huaiyu Guo, Xinyuan Zhou, Yijia Wang, Fei Yu, Hong Chen, Wanqing Xu, Xin Lu, Qing Cui, Longfei Li

• Can GPT models be Financial Analysts? An Evaluation of ChatGPT and GPT-4 on mock CFA Exams, arXiv, 2310.08678, arxiv, pdf, cication: -1

  Ethan Callanan, Amarachi Mbakwe, Antony Papadimitriou, Yulong Pei, Mathieu Sibue, Xiaodan Zhu, Zhiqiang Ma, Xiaomo Liu, Sameena Shah

Data Related

• llm-scraper - mishushakov

  Turn any webpage into structured data using LLMs

• duckdb-nsql-7b - 🤗

• DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning, arXiv, 2402.17453, arxiv, pdf, cication: -1

  Siyuan Guo, Cheng Deng, Ying Wen, Hechang Chen, Yi Chang, Jun Wang · (DS-Agent - guosyjlu) · (jiqizhixin)

• Improving Text Embeddings with Large Language Models, arXiv, 2401.00368, arxiv, pdf, cication: -1

  Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei

• superduperdb - SuperDuperDB

  🔮 SuperDuperDB: Bring AI to your database: Integrate, train and manage any AI models and APIs directly with your database and your data.

• Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflow, arXiv, 2306.07209, arxiv, pdf, cication: 6

  Wenqi Zhang, Yongliang Shen, Weiming Lu, Yueting Zhuang · (data-copilot - zwq2018) · (jiqizhixin)

• DB-GPT - eosphoros-ai

  Revolutionizing Database Interactions with Private LLM Technology

• LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMs, arXiv, 2307.10168, arxiv, pdf, cication: 5

  Tongshuang Wu, Haiyi Zhu, Maya Albayrak, Alexis Axon, Amanda Bertsch, Wenxing Deng, Ziqi Ding, Bill Guo, Sireesh Gururaja, Tzu-Sheng Kuo

Misc

• LiteSearch: Efficacious Tree Search for LLM, arXiv, 2407.00320, arxiv, pdf, cication: -1

  Ante Wang, Linfeng Song, Ye Tian, Baolin Peng, Dian Yu, Haitao Mi, Jinsong Su, Dong Yu

• RouteLLM: Learning to Route LLMs with Preference Data, arXiv, 2406.18665, arxiv, pdf, cication: -1

  Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, M Waleed Kadous, Ion Stoica

  · (RouteLLM - lm-sys) · (youtube)

• LLM Compiler - a facebook Collection

• Large Language Models as Planning Domain Generators, arXiv, 2405.06650, arxiv, pdf, cication: -1

  James Oswald, Kavitha Srinivas, Harsha Kokel, Junkyu Lee, Michael Katz, Shirin Sohrabi · (NL2PDDL - IBM)

• From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examples, arXiv, 2404.07544, arxiv, pdf, cication: -1

  Robert Vacareanu, Vlad-Andrei Negru, Vasile Suciu, Mihai Surdeanu

• LLM-ABR: Designing Adaptive Bitrate Algorithms via Large Language Models, arXiv, 2404.01617, arxiv, pdf, cication: -1

  Zhiyuan He, Aashish Gottipati, Lili Qiu, Francis Y. Yan, Xufang Luo, Kenuo Xu, Yuqing Yang

• Priority Sampling of Large Language Models for Compilers, arXiv, 2402.18734, arxiv, pdf, cication: -1

  Dejan Grubisic, Chris Cummins, Volker Seeker, Hugh Leather

• OmniPred: Language Models as Universal Regressors, arXiv, 2402.14547, arxiv, pdf, cication: -1

  Xingyou Song, Oscar Li, Chansoo Lee, Bangding, Yang, Daiyi Peng, Sagi Perel, Yutian Chen

• Shai: A large language model for asset management, arXiv, 2312.14203, arxiv, pdf, cication: -1

  Zhongyang Guo, Guanran Jiang, Zhongdan Zhang, Peng Li, Zhefeng Wang, Yinchun Wang

• emoji-translator - gospacedev 🤗

• PoseGPT: Chatting about 3D Human Pose, arXiv, 2311.18836, arxiv, pdf, cication: -1

  Yao Feng, Jing Lin, Sai Kumar Dwivedi, Yu Sun, Priyanka Patel, Michael J. Black · (yfeng95.github)

• LLMLight: Large Language Models as Traffic Signal Control Agents, arXiv, 2312.16044, arxiv, pdf, cication: 5

  Siqi Lai, Zhao Xu, Weijia Zhang, Hao Liu, Hui Xiong

  · (qbitai) · (LLMTSCS - usail-hkust)

• ChipNeMo: Domain-Adapted LLMs for Chip Design, arXiv, 2311.00176, arxiv, pdf, cication: -1

  Mingjie Liu, Teodor-Dumitru Ene, Robert Kirby, Chris Cheng, Nathaniel Pinckney, Rongjian Liang, Jonah Alben, Himyanshu Anand, Sanmitra Banerjee, Ismet Bayraktaroglu

• InstructExcel: A Benchmark for Natural Language Instruction in Excel, arXiv, 2310.14495, arxiv, pdf, cication: -1

  Justin Payan, Swaroop Mishra, Mukul Singh, Carina Negreanu, Christian Poelitz, Chitta Baral, Subhro Roy, Rasika Chakravarthy, Benjamin Van Durme, Elnaz Nouri

• SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents, arXiv, 2310.11667, arxiv, pdf, cication: -1

  Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig

• Table-GPT: Table-tuned GPT for Diverse Table Tasks, arXiv, 2310.09263, arxiv, pdf, cication: -1

  Peng Li, Yeye He, Dror Yashar, Weiwei Cui, Song Ge, Haidong Zhang, Danielle Rifinski Fainman, Dongmei Zhang, Surajit Chaudhuri

• llmtime - ngruver

• Ferret: Refer and Ground Anything Anywhere at Any Granularity, arXiv, 2310.07704, arxiv, pdf, cication: 3

  Haoxuan You, Haotian Zhang, Zhe Gan, Xianzhi Du, Bowen Zhang, Zirui Wang, Liangliang Cao, Shih-Fu Chang, Yinfei Yang · (qbitai)

• MotionLM: Multi-Agent Motion Forecasting as Language Modeling, ICCV, 2023, arxiv, pdf, cication: 2

  Ari Seff, Brian Cera, Dian Chen, Mason Ng, Aurick Zhou, Nigamaa Nayakanti, Khaled S. Refaat, Rami Al-Rfou, Benjamin Sapp

• Low-rank Adaptation of Large Language Model Rescoring for Parameter-Efficient Speech Recognition, arXiv, 2309.15223, arxiv, pdf, cication: 1

  Yu Yu, Chao-Han Huck Yang, Jari Kolehmainen, Prashanth G. Shivakumar, Yile Gu, Sungho Ryu, Roger Ren, Qi Luo, Aditya Gourav, I-Fan Chen

• LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agent, arXiv, 2309.12311, arxiv, pdf, cication: 1

  Jianing Yang, Xuweiyi Chen, Shengyi Qian, Nikhil Madaan, Madhavan Iyengar, David F. Fouhey, Joyce Chai

• Adapting Large Language Models via Reading Comprehension, arXiv, 2309.09530, arxiv, pdf, cication: 3

  Daixuan Cheng, Shaohan Huang, Furu Wei

• Large Language Models for Compiler Optimization, arXiv, 2309.07062, arxiv, pdf, cication: 3

  Chris Cummins, Volker Seeker, Dejan Grubisic, Mostafa Elhoushi, Youwei Liang, Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Kim Hazelwood, Gabriel Synnaeve

• AstroLLaMA: Towards Specialized Foundation Models in Astronomy, arXiv, 2309.06126, arxiv, pdf, cication: -1

  Tuan Dung Nguyen, Yuan-Sen Ting, Ioana Ciucă, Charlie O'Neill, Ze-Chang Sun, Maja Jabłońska, Sandor Kruk, Ernest Perkowski, Jack Miller, Jason Li

• LLMZip: Lossless Text Compression using Large Language Models, arXiv, 2306.04050, arxiv, pdf, cication: 2

  Chandra Shekhara Kaushik Valmeekam, Krishna Narayanan, Dileep Kalathil, Jean-Francois Chamberland, Srinivas Shakkottai

• LangChain 🤝 DemoGPT: New Era for Gen-AI Applications

  · (mp.weixin.qq)

• Language Modeling Is Compression, arXiv, 2309.10668, arxiv, pdf, cication: 7

  Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christopher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau

• GPT-InvestAR - UditGupta10

  Enhancing Stock Investment Strategies through Annual Report Analysis with Large Language Models

• textbase - cofactoryai

  ✨ Textbase is a simple framework for building AI chatbots. ✨

• aidea - mylxsw

  AIdea 是一款支持 GPT 以及国产大语言模型通义千问、文心一言等，支持 Stable Diffusion 文生图、图生图、 SDXL1.0、超分辨率、图片上色的全能型 APP。

• anomalygpt - casia-iva-lab

  The first LVLM based IAD method!

• GPT-vup - jiran214

  GPT-vup BIliBili | 抖音 | AI | 虚拟主播

• Open challenges in LLM research

• db-gpt - tsinghuadatabasegroup

• AI Is More Creative Than 99% of Humans — on One Metric

  · (mp.weixin.qq)

• Pre-Trained Large Language Models for Industrial Control, arXiv, 2308.03028, arxiv, pdf, cication: 2

  Lei Song, Chuheng Zhang, Li Zhao, Jiang Bian · (jiqizhixin)

• awesome-ai-devtools - jamesmurdza

  Curated list of AI-powered developer tools.

• lisa - dvlab-research

• AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?, arXiv, 2307.16368, arxiv, pdf, cication: 1

  Qi Zhao, Shijie Wang, Ce Zhang, Changcheng Fu, Minh Quan Do, Nakul Agarwal, Kwonjoon Lee, Chen Sun

• Towards Generalist Biomedical AI, arXiv, 2307.14334, arxiv, pdf, cication: 26

  Tao Tu, Shekoofeh Azizi, Danny Driess, Mike Schaekermann, Mohamed Amin, Pi-Chuan Chang, Andrew Carroll, Chuck Lau, Ryutaro Tanno, Ira Ktena

• TableGPT: Towards Unifying Tables, Nature Language and Commands into One GPT, arXiv, 2307.08674, arxiv, pdf, cication: 1

  Liangyu Zha, Junlin Zhou, Liyao Li, Rui Wang, Qingyi Huang, Saisai Yang, Jing Yuan, Changbao Su, Xiang Li, Aofeng Su

• How to Use AI to Do Stuff: An Opinionated Guide

  · (qbitai)

• PMC-VQA - xiaoman-zhang

  PMC-VQA is a large-scale medical visual question-answering dataset, which contains 227k VQA pairs of 149k images that cover various modalities or diseases. · (jiqizhixin)

• On decoder-only architecture for speech-to-text and large language model integration, arXiv, 2307.03917, arxiv, pdf, cication: 9

  Jian Wu, Yashesh Gaur, Zhuo Chen, Long Zhou, Yimeng Zhu, Tianrui Wang, Jinyu Li, Shujie Liu, Bo Ren, Linquan Liu

• Large Language Models as General Pattern Machines, arXiv, 2307.04721, arxiv, pdf, cication: 21

  Suvir Mirchandani, Fei Xia, Pete Florence, Brian Ichter, Danny Driess, Montserrat Gonzalez Arenas, Kanishka Rao, Dorsa Sadigh, Andy Zeng

• Large Language Models for Supply Chain Optimization, arXiv, 2307.03875, arxiv, pdf, cication: 2

  Beibin Li, Konstantina Mellou, Bo Zhang, Jeevan Pathuri, Ishai Menache

• KokoMind: Can LLMs Understand Social Interactions?

• Open-Source Large Language Models Outperform Crowd Workers and Approach ChatGPT in Text-Annotation Tasks, arXiv, 2307.02179, arxiv, pdf, cication: 5

  Meysam Alizadeh, Maël Kubli, Zeynab Samei, Shirin Dehghani, Juan Diego Bermeo, Maria Korobeynikova, Fabrizio Gilardi

• Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting, arXiv, 2306.17563, arxiv, pdf, cication: 13

  Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang

• Language models are weak learners, arXiv, 2306.14101, arxiv, pdf, cication: -1

  Hariharan Manikandan, Yiding Jiang, J Zico Kolter

• Opportunities and Risks of LLMs for Scalable Deliberation with Polis, arXiv, 2306.11932, arxiv, pdf, cication: 2

  Christopher T. Small, Ivan Vendrov, Esin Durmus, Hadjar Homaei, Elizabeth Barry, Julien Cornebise, Ted Suzman, Deep Ganguli, Colin Megill

• Introducing Ocean-1: World’s First Contact Center Foundation Model - Cresta

• MotionGPT: Human Motion as a Foreign Language, arXiv, 2306.14795, arxiv, pdf, cication: 14

  Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, Tao Chen

• MotionGPT: Finetuned LLMs are General-Purpose Motion Generators, arXiv, 2306.10900, arxiv, pdf, cication: 7

  Yaqi Zhang, Di Huang, Bin Liu, Shixiang Tang, Yan Lu, Lu Chen, Lei Bai, Qi Chu, Nenghai Yu, Wanli Ouyang

• arXiVeri: Automatic table verification with GPT, arXiv, 2306.07968, arxiv, pdf, cication: -1

  Gyungin Shin, Weidi Xie, Samuel Albanie

• GPT-Calls: Enhancing Call Segmentation and Tagging by Generating Synthetic Conversations via Large Language Models, arXiv, 2306.07941, arxiv, pdf, cication: 1

  Itzik Malkiel, Uri Alon, Yakir Yehuda, Shahar Keren, Oren Barkan, Royi Ronen, Noam Koenigstein

Game

• llm-colosseum - OpenGenerativeAI

  Benchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM · (qbitai)

• Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization, arXiv, 2402.17574, arxiv, pdf, cication: -1

  Wenqi Zhang, Ke Tang, Hai Wu, Mengna Wang, Yongliang Shen, Guiyang Hou, Zeqi Tan, Peng Li, Yueting Zhuang, Weiming Lu · (Agent-Pro - zwq2018) · (qbitai)

• chessllm - mlabonne

  · (huggingface)

• Towards General Computer Control: A Multimodal Agent for Red Dead Redemption II as a Case Study, arXiv, 2403.03186, arxiv, pdf, cication: -1

  Weihao Tan, Ziluo Ding, Wentao Zhang, Boyu Li, Bohan Zhou, Junpeng Yue, Haochong Xia, Jiechuan Jiang, Longtao Zheng, Xinrun Xu · (baai-agents.github) · (Cradle - BAAI-Agents)

  · (jiqizhixin)

• Human vs. Machine: Language Models and Wargames, arXiv, 2403.03407, arxiv, pdf, cication: -1

  Max Lamparth, Anthony Corso, Jacob Ganz, Oriana Skylar Mastro, Jacquelyn Schneider, Harold Trinkunas

• pokellmon - git-disl

  PokéLLMon: A Human-Parity Agent for Pokémon Battles with Large Language Models

• Creating an engaging AI-Powered Game using Unity and Hugging Face 🤗

• Large Language Models Play StarCraft II: Benchmarks and A Chain of Summarization Approach, arXiv, 2312.11865, arxiv, pdf, cication: -1

  Weiyu Ma, Qirui Mi, Xue Yan, Yuqiao Wu, Runji Lin, Haifeng Zhang, Jun Wang · (Large-Language-Models-play-StarCraftII - histmeisah)

• XG推荐 | 如何做好AI游戏？AutoGame团队分享

• 不写一行代码，他用GPT-4、DALL·E 3等手搓了《愤怒的南瓜》！ | 量子位

• GPT-4肆虐「谁是卧底」桌游！交谈逼真，类人属性仍有发展空间

• ChatGPT发力《龙与地下城》剧情设计，帮你一步步上手地下城主 | 机器之心

• 清华大学团队让7个大模型玩起狼人杀并发现新的涌现策略行为

• shezhangbujianle - bigbrother666sh

  借助世界上最大的单体中文NLP大模型，我们做出了一个可以跟人类玩“剧本杀”的AI……

• 1000+AI智能体复活，OpenAI版元宇宙上线？ ChatGPT+VR百分百还原「西部世界」

• AI产生自我意识，「古墓丽影」劳拉觉醒！电子游戏革命来了

Hardware

• ADeus - adamcohenhillel

  An open source AI wearable device that captures what you say and hear in the real world and then transcribes and stores it on your own server. You can then chat with Adeus using the app, and it will have all the right context about what you want to talk about - a truly personalized, personal AI. · (qbitai)

Products

• Inflection-2.5: meet the world's best personal AI

• Synthical: Science, Simplified

• 当GPT-4o遇上情感陪伴：多巴胺的胜利，催产素的挑战

• 复盘“哄哄模拟器的复盘”：别错过这6点AI应用思考

• 开源大模型正在重塑企业AI应用，16个案例看看它们如何落地 | 机器之心

• 这个AI办公应用火了：简短提示词出PPT，打包GPTs帮你打工干活 | 量子位

• 万字长文：AI陪伴产品的终极解法?

• 灵魂拷问：C.ai的终局是什么？会成为平台么？为何国内没有大竞品？。。。

• 为什么AI陪伴产品都想抄星野？

• WhatLetter

• Glide AI | Build and deploy custom AI-powered apps for your business

• 蚂蚁集团开 “卷” 金融大模型，“四力和一” 解决产业真命题 | 机器之心

• 全球首个大模型Agent产品爆款出场！动动嘴让PC成为打工人

• 聆心智能超拟人模型CharacterGLM升级，助力AI实现「走心」突破 | 机器之心

• Cortex

• 顺网科技发布AI引擎"SPICE"以及个人陪伴应用"顺网灵悉" | 量子位

Other

• 大模型badcase修复方案思考
• 大模型落地应用架构的一种模式
• 领域大模型-训练Trick&落地思考
• ChatGPT上线"论文神器"插件！无需关键字即可搜索2亿文章，链接绝对保真 | 量子位

Extra reference

• awesome-local-ai - janhq

  An awesome repository of local AI tools

• Awesome-LLMs-in-Graph-tasks - yhLeeee

  A curated collection of research papers exploring the utilization of LLMs for graph-related tasks.