Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add autofeedback ICL and cli log10 feedback predict #115

Merged
merged 7 commits into from
Mar 8, 2024

Conversation

wenzhe-log10
Copy link
Collaborator

@wenzhe-log10 wenzhe-log10 commented Mar 8, 2024

log10 feedback predict --task_id 04405cbc-3420-4901-97b6-f8b7ed248205 --completion_id 447c3b69-6aea-4f4d-95e6-cbf37adca846
log10 feedback predict --task_id 04405cbc-3420-4901-97b6-f8b7ed248205 --content "{\"article\": \"We have been together over two years, but got in a very large fight today. We have large fights occasionally where my boyfriend just gets blinded by his anger. \\n\\nI have never ever liked jewelry, and haven't worn a single item of jewelry since elementary school, my ears aren't even pierced. Even with this fact, my boyfriend continues to buy me jewelry.\\n\\nI haven't been rude about it up until this point, because you shouldn't ever really refuse a gift. But I do make a point to him that I don't like jewelry, and never wear it, while trying to be as polite as possible. For some reason, he can't understand this about me and continues to get angry that he buys me jewelry I don't wear, etc. etc. He just seems to not understand I don't want this type of typically \\\"nice\\\" gift. I still thank him for the original intent, but at what point should he start treating me as a person with individual likes and dislikes? He just expects me to like typical \\\"girl\\\" things. He's very upset with me and calling me ungrateful. Sigh. Maybe he's right. It's just something I will never be able to like but am expected to be grateful for anyway :/ \\n\\nAlso, I would understand if we just started dating, but it's been two and a half years of this. I don't want to lie to him about my interests though.\\n\\nHe is also trying to break up with me right now....but he does that every time he gets angry.\", \"summary\": \"My boyfriend [18M] is very angry at me [18F] because I don't like jewelry. He's trying to break up with me right now.\"}"

@wenzhe-log10 wenzhe-log10 force-pushed the wenzhe/autofeedback_v0 branch from 2141ad4 to 7a31789 Compare March 8, 2024 15:54
@wenzhe-log10 wenzhe-log10 force-pushed the wenzhe/autofeedback_v0 branch from e8cee6e to 1de86f5 Compare March 8, 2024 18:41
@nqn
Copy link
Contributor

nqn commented Mar 8, 2024

Highlevel - tried and it worked 🎉

One snag - if environment isn't correctly setup, the error message is a bit hard to grok:

ValueError: Sample larger than population or is negative

Maybe we make a better one?

* Custom Feedback Rubrics: Integrate your own feedback criteria for personalized assessments.
* Getting Started: To explore all options and usage details, use CLI `log10 feedback predict --help`.

Feel free to integrate AutoFeedback into your workflow to enhance the feedback and evaluation process.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we inline a command line example?

@wenzhe-log10 wenzhe-log10 merged commit 777daa0 into main Mar 8, 2024
1 check passed
@wenzhe-log10 wenzhe-log10 deleted the wenzhe/autofeedback_v0 branch March 8, 2024 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants