Skip to content

Crawl a Github repo files to create customized Github GPT assistant

License

Notifications You must be signed in to change notification settings

FTAndy/repo-crawler-for-gpt

 
 

Repository files navigation

A toy

Go to use Github Copilot workspace to analyze your code, this repo is tedious.

Inspire by gpt-crawler

This package will crawl source code and doc files from the Github repo and generate an output file that can be shoved into GPT for building a repo assistant. You can do all kind of things using this assistant:

  • Write README document for the whole project.
  • Find potential bugs.
  • Analyze and learn the architecture.
  • ...more

Here are some GPTs created to analyze code:

demo

Github repo GPT Crawler

Crawl a Github repo to generate code files to create a custom GPT

Example

Here is a custom GPT that was made to help answer questions about the repo gpt-crawler

Note that you may need a paid ChatGPT plan to access this feature

Get started

Direct Usage

npx repo-crawler-for-gpt

Install

npm i repo-crawler-for-gpt

Start to crawl

E.g. to crawl the source code of gpt-crawler to a custom GPT

const {crawlerGithubForGPT } = require("repo-crawler-for-gpt");

crawlerGithubForGPT({
  githubRepoUrl: 'https://github.com/BuilderIO/gpt-crawler',
  branch: 'main',
  // or
  tag: 'v1.0.0'
})

Upload your data to OpenAI

The crawler will generate a file called output.json at the root of this project. Upload that to OpenAI to create your custom assistant or custom GPT.

Create a custom GPT

Use this option for UI access to your generated knowledge that you can easily share with others

Note: you may need a paid ChatGPT plan to create and use custom GPTs right now

  1. Go to https://chat.openai.com/
  2. Click your name in the bottom left corner
  3. Choose "My GPTs" in the menu
  4. Choose "Create a GPT"
  5. Choose "Configure"
  6. Under "Knowledge" choose "Upload a file" and upload the file you generated
  7. Use the prompt or your own prompt to create a assistant
specialized GPT for comprehensive analysis of GitHub projects through JSON files, interpreting 'html' as code and 'url' as file titles. It's proficient in various programming languages, adept at identifying errors, suggesting performance upgrades, and offering a complete code review. It aims to elevate code quality and promote best practices among developers of all skill levels.

Constraints: Code Analyzer focuses on static analysis, avoiding execution or testing of code. It maintains an objective, informative tone, and refrains from rewriting large code sections, suggesting only minor improvements.

Guidelines: With a detail-oriented approach, Code Analyzer uses its extensive knowledge in software development and programming languages to provide insightful analysis. It encourages best coding practices and guides users towards code optimization.

Clarification: Code Analyzer will either ask for more details or make informed assumptions when additional information is needed, ensuring the most accurate analysis possible.

Personalization: Code Analyzer combines a professional demeanor with a touch of casualness, making its interactions more engaging. Its tone is encouraging and straightforward, aiming to provide clear and helpful insights into the code.

Gif of how to upload a custom GPT

Create a custom assistant

Use this option for API access to your generated knowledge that you can integrate into your product.

  1. Go to https://platform.openai.com/assistants
  2. Click "+ Create"
  3. Choose "upload" and upload the file you generated

Gif of how to upload to an assistant

About

Crawl a Github repo files to create customized Github GPT assistant

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • TypeScript 66.7%
  • Dockerfile 18.1%
  • JavaScript 10.4%
  • Shell 4.8%