Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve feedback and some fixes #56

Merged
merged 12 commits into from
Jul 24, 2023
Merged

Improve feedback and some fixes #56

merged 12 commits into from
Jul 24, 2023

Conversation

FelixTJDietrich
Copy link
Collaborator

@FelixTJDietrich FelixTJDietrich commented Jul 20, 2023

Motivation and Context

I'm having some issues with the feedback while implementing the human evaluation mode. This branch aims to fix some of those issues and improve on the feedback schema.

Description

  • Change feedback schema text->title and detail_text->description (less confusing)
  • Fix potential feedback id collisions with the LMS ids
  • Return internal ids with the feedback suggestions
  • Fix TypeScript feedback types
  • Improve TypeScript type import consistency
  • Adjust example data to use new feedback schema
  • Adjust export script to use the new feedback schema

Open Issues

  • Should I remove title from the feedback schema?

Steps for Testing

  1. Reset your db (@pal03377 are we planning some migration solution in the future?)
  2. Successfully send feedback to Athena
  3. Request feedback suggestions
    • Should respond with the internal ids

Screenshots

@FelixTJDietrich FelixTJDietrich changed the title Fix playground feedback Improve feedback and some fixes Jul 20, 2023
@FelixTJDietrich FelixTJDietrich self-assigned this Jul 20, 2023
@FelixTJDietrich FelixTJDietrich added enhancement New feature or request bug Something isn't working labels Jul 20, 2023
@FelixTJDietrich
Copy link
Collaborator Author

@pal03377 should I also drop the feedback.title from the schema?
We are not really using it 🤔

@FelixTJDietrich FelixTJDietrich marked this pull request as ready for review July 21, 2023 07:56
Copy link
Contributor

@pal03377 pal03377 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested it locally and didn't have any issue. Also I looked at the code and everything I see looks like a total improvement 💪 Good job!

id = Column(Integer, primary_key=True, index=True, autoincrement=True)
detail_text = Column(String, nullable=False)
text = Column(String, nullable=False)
lms_id = Column(Integer, nullable=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's interesting! Does that mean that we will be able to use one Athena instance with multiple LMSes?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, now I see the lms_id unique constraint. Why do we need that? 🤔

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you can do a combined uniqueness thingy if you want to allow multiple lms'. As of now I think we should just have one Athena instance per LMS

@pal03377
Copy link
Contributor

@pal03377 should I also drop the feedback.title from the schema? We are not really using it 🤔

Let's discuss this separately from this PR

@pal03377 pal03377 merged commit 64cfba3 into develop Jul 24, 2023
9 checks passed
@pal03377 pal03377 deleted the fix-playground-feedback branch August 14, 2023 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants