Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large Multimodal Models in AgentChat #554

Merged
merged 18 commits into from
Nov 6, 2023
Merged

Large Multimodal Models in AgentChat #554

merged 18 commits into from
Nov 6, 2023

Commits on Nov 5, 2023

  1. LMM Code added

    BeibinLi committed Nov 5, 2023
    Configuration menu
    Copy the full SHA
    157a158 View commit details
    Browse the repository at this point in the history
  2. LLaVA notebook update

    BeibinLi committed Nov 5, 2023
    Configuration menu
    Copy the full SHA
    aa45ec0 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    2a47432 View commit details
    Browse the repository at this point in the history
  4. Move LMM into contrib

    To resolve test issues and deploy issues
    In the future, we can install pillow by default, and then move back
    LMM agents into agentchat
    BeibinLi committed Nov 5, 2023
    Configuration menu
    Copy the full SHA
    c568b9e View commit details
    Browse the repository at this point in the history
  5. LMM test setup update

    BeibinLi committed Nov 5, 2023
    Configuration menu
    Copy the full SHA
    2b442f2 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    31b90d3 View commit details
    Browse the repository at this point in the history
  7. disable patch for llava agent test

    To resolve dependencies issue for build
    BeibinLi committed Nov 5, 2023
    Configuration menu
    Copy the full SHA
    b10f712 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2023

  1. Add LMM Blog

    BeibinLi committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    695a3a2 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b0969bd View commit details
    Browse the repository at this point in the history
  3. Docstring update patch

    BeibinLi committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    8e4222b View commit details
    Browse the repository at this point in the history
  4. llava: insert reply at position 1 now

    So, it can still handle human_input_mode
    and max_consecutive_reply
    BeibinLi committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    73a6c06 View commit details
    Browse the repository at this point in the history
  5. Resolve comments

    Fixing: typos, blogs, yml, and add OpenAIWrapper
    BeibinLi committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    211b41c View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    7b8a1a0 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    e00692a View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    d812486 View commit details
    Browse the repository at this point in the history
  9. Fully support GPT-4V now

    1. Add a notebook for GPT-4V. LLava notebook also updated.
    2. img_utils updated
    3. GPT-4V formatter now return base64 image with mime type
    4. Infer mime type directly from b64 image content (while loading
       without suffix)
    5. Test cases modified according to all the related changes.
    BeibinLi committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    aa64852 View commit details
    Browse the repository at this point in the history
  10. GPT-4V link updated in blog

    BeibinLi committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    d4f04c0 View commit details
    Browse the repository at this point in the history
  11. Merge branch 'main' into lmm

    sonichi committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    8d2e64b View commit details
    Browse the repository at this point in the history