DH-4184 add ruff lintern #7
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ghost
force-pushed
the
ft/DH_4184-ruff_lintern
branch
from
9fac7b5
to
990c872
Compare
ghost
pushed a commit
that referenced
this pull request
Aug 15, 2023
Update README.md Update README.md Update README.md Update README.md Update README.md fb/DH-4135 initial project setup fb/DH-4135 move Flask logic out of __init__ fix commit add abstract class logic to config.py so the module used is not hard-coded Update dataherald/api/fastapi.py Co-authored-by: Juan Carlos José Camacho <jcjc712@hotmail.com> Update dataherald/api/fastapi.py Co-authored-by: Juan Carlos José Camacho <jcjc712@hotmail.com> Update dataherald/api/fastapi.py Co-authored-by: Juan Carlos José Camacho <jcjc712@hotmail.com> deleting smart_cache.base.py Update README.md fb/DH-4176 Add data models to the sample route for NLQuery and NLQueryResponse fb/DH-4177 add logger fb/DH-4182 add logic to add persistance layer. Remove all dh and dataherald before variable names add new requirements DH-4184 add ruff lintern (#7) DH-4184 add lint and format section to README DH-4183 Create Dockerfile and docker-compose (#5) * DH-4183 Create Dockerfile and docker-compose * DH-4183 Update readme * DH-4183 Fix env vars * DH-4183 Fix env vars --------- Co-authored-by: Juan Carlos Jose Camacho <juan.jose@oyster.io> added simple text-to-sql solutions (#6) removed import linting lint-fixing added sql_query logging to sql_generator solutions DH-4196 disable lintern rule about built-in types added get sql engine function (#8) * added get sql engine function * fix config.py, in_memory_cache initializer, lookup return type and logger formatting * configured sql database to work for sql_generator --------- Co-authored-by: Amir A. Zohrenejad <aazo11@gmail.com> DH-4157 Add tests (#9) * DH-4157 Add tests * Add question endpoint test --------- Co-authored-by: Juan Carlos Jose Camacho <juan.jose@oyster.io> DH-4194 [eval module] add support to golden_sql to the Evaluator -- add types -- add api methods -- [api] declare methods in the abstract class (#10) reworked ssh portforward (#11) added comments for .env.example (#12) added comments for .env.example removed ssh prefix renamed uri_prefix (#14) fix eval module errors (#15) DH-4157 Add answer_question test (#13) Co-authored-by: Juan Carlos Jose Camacho <juan.jose@oyster.io> DH-4202 Set private ssh key file (#16) Co-authored-by: Juan Carlos Jose Camacho <juan.jose@oyster.io> Fix mongo + other enhancements (#17) * ignore the assert rule only for test files * remove the linting job on the PR trigger * remove redundant ssh prefix to class attributes * minor fixes * added details about mongo connection and clean up the connection * fix docker mongo to create the add the admin user DH-4216 Move .env file at root and fix initdb file for mongo (#18) Co-authored-by: Juan Carlos Jose Camacho <juan.jose@oyster.io> DH-4219 Improve readme file (#19) Co-authored-by: Juan Carlos Jose Camacho <jc@dataherald.com> DH-4207 Fix to NLQueryResponse model (#21) Co-authored-by: Juan Carlos Jose Camacho <jc@dataherald.com> DH-4223 Fix logs (#22) Co-authored-by: Juan Carlos Jose Camacho <jc@dataherald.com> removed unused python packages and added preliminary package setup file (#24) DH-4172 Add DATABASE_URI envvar to connect with any db (#23) * DH-4172 Add DATABASE_URI envvar to connect with any db * DH-4172 Add db driver packages --------- Co-authored-by: Juan Carlos Jose Camacho <jc@dataherald.com> Add Context Store logic (#20) * work in progress to show sample * added logic to add golden_Sql to the vectorstore * DH-4219 Improve readme file (#19) Co-authored-by: Juan Carlos Jose Camacho <jc@dataherald.com> * got embeddings to work * move chroma implementation to vector_store module * add changes to config.py * fix linting issues * pass context to llamaindex and langchain sqlagent * fix linting issue * fix last linter issue * fix black linter issues * update .gitignore * fix test_generator * add find_one method to in_memory DB type * add sql-metadata and sqlparse back into requirments.txt * downgrade sqlalchemy * add chromadb to requirements.txt * Fix required packages and tests * Add chromadb package --------- Co-authored-by: Juan Carlos José Camacho <jcjc712@hotmail.com> Co-authored-by: Juan Carlos Jose Camacho <jc@dataherald.com> DH-4238 Updates Langchain package version (#30) Co-authored-by: Juan Carlos Jose Camacho <jc@dataherald.com> Ft/dh 4231 deploy stage to ecs (#33) added deployment pipeline for hosted version with updated environment variables DH-4240 Set tests envvars (#34) Co-authored-by: Juan Carlos Jose Camacho <jc@dataherald.com> [fixes] update env example and namings - add rebuild to readme and troubleshoot section - add openai api key to config (#35) Fb/dh 4241 (#32) * DH-4235 added support for pinecone vector store * DH-4241 fixed scenario if golden-records collection did not exist in chromadb * fix regressions in chromadb to match new object model * fix linting issues * fix vector_store test * fix linter issue in test_vector_store * remove print statement * fix tests for vector_store * make pinecone the default vector_store in .env.example * fix .env.example * fix comments from review * fix mongo.py conflicts --------- Co-authored-by: Amir A. Zohrenejad <aazo11@gmail.com> DH-4237 make docker network external to enable other docker containers to communicate with the app (#36) DH-4249 Fix app container reload (#37) Co-authored-by: Juan Carlos Jose Camacho <jc@dataherald.com> adding the simple evaluation and evaluation agent (#39) * adding the evaluation modules * black reformatting done * add test evaluator * add expected params to TestEvaluator class * add start attr * added abstract class extend * black eval * minor issue * minor issue2 * issue * test * issue * Hope to be finall issue * final * mistake of using overrides instead of override Fb/dh 4210 (#38) * add core logic for db_alias * stashing changes to rebase * ssh_uri validated with reading db_info from table * fix linting issues * DH-4260 Only have one ssh_enabled field * Insert a default db connection in the mongo collection * Encrypt db passwords * Field connection_uri is optional * Fix tests * Create a db connection when the app starts * Refactor encrypt code and store first db connection * Fix db_connection issue --------- Co-authored-by: Juan Carlos Jose Camacho <jc@dataherald.com> DH-4283 Update readme about db connections (#41) Co-authored-by: Juan Carlos Jose Camacho <jc@dataherald.com> DH-4258 Only look up sample questions for a given DB alias (#42) DH-4287 Create a mock TableSchemaDetail object (#43) Add the first version of our text-to-SQL agent (#44) * Add the first version of our text-to-SQL agent * reformatted * change the tester added new container definition (#46) rename aws rsources to k2-core and added new container in task definition add token counting and minor changes to the agent (#47) * Add the first version of our text-to-SQL agent * reformatted * change the tester * add token counts and minor change to the agent * black * Update llamaindex.py DH-4288 Scan a db and persist it (#45) * DH-4288 Scan a db and persist it * DH-4288 Optional table_name param to scan only one table * Fix tests and refactor scanner code fb/DH-3408 this commit adds a new endpoint to run an SQL query agains… (#48) * fb/DH-3408 this commit adds a new endpoint to run an SQL query against a given db_alias * move the parameters to the /query endpoint to the requestBody instead of the query parameters DH-4307 Test scanner endpoint (#49) * DH-4307 Test scanner endpoint * Fix scanner class DH-4322 creates NLQueryResponseRepository to fix empty id response (#51) * DH-4322 creates NLQueryResponseRepository to fix empty id response * Update dataherald/repositories/base.py Update dataherald_sqlagent.py (#52) DH-4314 Add more logs for debugging (#50) DH-4324 Only creates one db instance (#53) * DH-4324 Only creates one db instance * Add many db connections DH-4335 Add ssh db_name parameter (#54) DH-4338 Improve scanner performance (#55) * DH-4338 Improve scanner performance * DH-4338 improve scanner Dh-4350/adding entity checker & exception handling (#57) * Dh-4350/adding entity checker & exception handling * DH-4350/reformat_the_agent_with_black Ft/dh 4350 changing the main prompt of the agent (#59) * Dh-4350/adding entity checker & exception handling * DH-4350/reformat_the_agent_with_black * DH-4350/Changing the main prompt of the agent DH-4347 Fix examples query for many db managers (#58) Dh4353/handling the exception caused by data value (#60) * Dh4353/handling the exception caused by data value * DH-4335_reformat with black DH-4357 Creates nl_question repository (#62) DH-4356/adding confidence score to NLQueryRespons (#61) * DH-4356/adding confidence score to NLQueryRespons * DH-4356/changing the test_eval work with evaluator * DH-4356/changing the return type of test_eval * DH-4356/Changing the return type fo confloat * DH-4356/change the return type to confloat * return type * DH-4356/add confloat to the NLQueryResponse * DH/4356 go back to float again :) * DH-4356/ go back to float DH-4341 Add new endpointas and request entities (#63) DH-4355/handling the bug of confidence_score (#64) DH-4340 Fix endpoints (#65) DH-4359 Fix associated_nl_question (#66) Ft/dh 4355 adding new fileds to nl query response (#67) * DH-4355 adding new fileds to the NLQueryResponse * DH-4355/Reformat with black DH-4354 Add logic in new endpoints (#68) DH-4375/adding verified query to the context store (#69) * DH-4375/adding verified query to the context store * DH-4375/reformat with black * DH-4375/removing the 500 error DH-4380 Fix format response (#71) DH-4377/ update query confidence for verfied ones (#70) DH-4395/confidece score zero for queries with [] (#72) Ft/dh 4395 setting up staging pinecone (#74) * DH-4395/ creating pinecone index as an env var * DH-4395/ updating the env example for pinecone env * DH-4395/updating env example * DH-4395/adding new env vars to test.env DH-4304 Move query params as body params (#73) DH-4399/removing an unverfied vector from vec db (#75) * DH-4399/removing an unverfied vector from vec db * DH-4399/update tests to include delete record DH4390/add capability to change the model (#78) * DH4390/add capability to change the model * Update query endpoint to solve the bug for context store * Change class Model name * DH-4395/Update base_models.py * DH4390/Update chat_model.py * DH4390/Update base_models.py * DH4390/Update .env.example * DH4390/Update .test.env DH-4280 Creates endpoint to add descriptions (#77) DH4410/fixing the minor bug with sql nl generator (#79) * DH4410/fixing the minor bug with sql nl generator * DH-4410/reformat with black DH-4413/adding the default values for LLM_Model (#80) DH-4408/changing prompts to answer faster (#76) DH-4414/fixing the bug with execute temp query (#81) DH-4415 Fix chromadb to support insert the same record many times (#82) DH-4406/adding the current date and time to agent (#83) * DH-4406/adding the current date and time to agent * DH-4406/reformatting with black DH-4417/raising 404 when scanned info not found (#84) * DH-4417/raising 404 when scanned info not found * DH-4417/raising exception when scanned info is NAN Bf/dh 4392 (#86) * bf/DH-4392 add README header * further updates to the README * add contributing.md * updated CONTRIBUTING.md * more changes to the README DH-4419 Add sphinx documentation (#85) * DH-4419 Add sphinx documentation * DH-4419 Creates documentation structure * DH-4419/removing all of the warnings * DH-4419/removing the warnings * DH-4419/ add comments to pass the lint test * DH-4419/reformat with black * DH-4419/the landing page for the modules * Bf/dh 4392 (#86) * bf/DH-4392 add README header * further updates to the README * add contributing.md * updated CONTRIBUTING.md * more changes to the README * update env.example documentation * add license and update README * finalize README * fix typo in README * DH-4419/updating the modules file add architecture * Adds documentation for add descriptions endpoint --------- Co-authored-by: mohammadrezapourreza <m1378.prz@gmail.com> Co-authored-by: Amir A. Zohrenejad <46583220+aazo11@users.noreply.github.com> Co-authored-by: Amir A. Zohrenejad <aazo11@gmail.com> deleted cd pipeline code (#88)
This pull request was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.