This project was part of the TribeAI Hackathon.
Team: Kiyo Kunii @921kiyo, Hector Lopez Hernandez, PhD @lopez-hector, and Yifan Yuan @cosmicthermo.
Literature reviews are a critical component of scientific workflows. They are key in staying up-to-date with research progress and for creating the foundation of new research endeavors.
Logistically, creating a literature review is often plagued by cumbersome tasks that include:
- sifting a large body of research for relevant articles,
- tedious parsing of papers for relevant nuggets of information,
- summarization of that information, and
- synthesis of the information to create a report of the current state of research/technology/field.
Large language models provide an exciting opportunity to enhance, accelerate, and likely improve the literature review process. They are excellent models for summarizing, answering questions, and extracting context from text.
We created a framework that improves the literature experience end-to-end; from a large query of 100s of papers down to a personalized chat with individual papers.
In short, this repo provides a means of:
- querying ArXiv
- retrieving a collective answer from K research articles
- retrieving individual answers from each relevant research article
- chatting with individual papers
- NextJS Frontend
- ChatGPT Keyword Search Generation
- arXiv API paper retrieval
- Cohere Rerank to identify top K documents
- Embedding with Instructor-XL
5a. optional embedding with Modal - FAISS vectorstore
- ChatGPT Question and answer
- Claude-v1.3-100K powered chat with individual papers
cd frontend- Run
npm install - Run
npm run dev
pip install -r requirements.txtcd backend/srcuvicorn server.app:app --reload- Go to
http://127.0.0.1:8000/docsin your browser and you can make REST call (GET, POST) from the browser.
export MODAL=false
export OPENAI_API_KEY=sk-XXX
export ROOT_DIRECTORY=local-directory
export COHERE_API_KEY=your_key
export ANTHROPIC_API_KEY=your_key