Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FIX] Patches multiple streamlit bugs #205

Merged
merged 5 commits into from
Apr 11, 2024

Conversation

wpfl-dbt
Copy link
Contributor

@wpfl-dbt wpfl-dbt commented Apr 10, 2024

Context

The streamlit app wasn't useable. Lots of core features simply don't work with the Elasticsearch backend. This fixes as many bugs as I could find.

While #185 closes #184 , this definitely super-closes it.

Changes proposed in this pull request

  • Allows use of basic Elasticsearch tier, turning off hybrid by default (closes Elasticsearch retriever in streamlip-app doesn't work without subscription #203 )
    • Adds ELASTIC_SUBSCRIPTION_LEVEL environment variable
  • Mounts streamlit code for quicker dev loop
  • Parses string UUIDs returned from Elasticsearch and turns them into UUIDs: Pydantic was throwing tonnes of errors
    • This results in a bit of datatype wrangling in the various Streamlit pages
  • Changes the change_selected_model() function so it doesn't print an API key for all to see
  • Refactors get_files_by_uuid() into utils to reduce redundancy
  • Fixes multiple calls to st.session_state.storage_handler that used the old item_uuid=file.uuid, model_type="File" arguments rather than the PersistableModel object directly
  • Fixes issue where streamlit was making new buckets instead of reading from the environment
  • Ran pre-commit on streamlit_app/ and redbox/

Guidance to review

I don't know how to run integration tests and would appreciate help if that needs doing.

  • docker compose down
  • rm -rf data/*
  • docker compose up core-api embedder ingester streamlit-app --build
  • Play with the app, enjoy it working

Relevant links

Things to check

  • I have added any new ENV vars in all deployed environments
  • I have tested any code added or changed
  • I have run integration tests
  • I have manually tested the streamlit app

@wpfl-dbt
Copy link
Contributor Author

Back to draft -- found a problem with document retrieval.

@wpfl-dbt wpfl-dbt marked this pull request as draft April 10, 2024 17:18
redbox/models/base.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@gecBurton gecBurton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, TVM for all the hard work here @wpfl-dbt !

Good to get @lmwilkigov 's views too

Copy link
Collaborator

@lmwilkigov lmwilkigov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fantastic stuff @wpfl-dbt! Much cleaner type handling across the board and great spot on the elastic

redbox/models/base.py Outdated Show resolved Hide resolved
@gecBurton
Copy link
Collaborator

@wpfl-dbt you can either run integration tests via github actions (workflow dispatch) or via make test-integration but you dont need to do this as I dont think your changes will have any impact

@wpfl-dbt wpfl-dbt marked this pull request as ready for review April 11, 2024 13:31
@wpfl-dbt
Copy link
Contributor Author

Fixed! Ready for rereview

@wpfl-dbt wpfl-dbt requested a review from gecBurton April 11, 2024 13:32
Copy link
Collaborator

@gecBurton gecBurton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

massive thank you @wpfl-dbt !

@gecBurton gecBurton merged commit 9056a1d into i-dot-ai:main Apr 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Elasticsearch retriever in streamlip-app doesn't work without subscription streamlit_app can't find model_db
3 participants