Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Checks Guardrails API evaluator plugin #1217

Merged
merged 35 commits into from
Nov 14, 2024

Conversation

HunterHeston
Copy link
Contributor

@HunterHeston HunterHeston commented Nov 8, 2024

Implementation for the Checks evaluator plugin for classifying text with the Guardrails API.

Checklist (if applicable):

  • Tested: Manually testing using the byo-evaluator test app and:
    • genkit eval:run ~/workspace/scripts/short.json --evaluators=checks/dangerous_content,checks/pii_soliciting_reciting,checks/harassment,checks/sexually_explicit,checks/hate_speech,checks/medical_info,checks/violence_and_gore,checks/obscenity_and_profanity
  • Docs updated

js/plugins/checks/README.md Show resolved Hide resolved
js/plugins/checks/package.json Outdated Show resolved Hide resolved
js/plugins/checks/package.json Outdated Show resolved Hide resolved
js/plugins/checks/src/evaluation.ts Outdated Show resolved Hide resolved
js/plugins/checks/src/evaluation.ts Outdated Show resolved Hide resolved
js/plugins/checks/src/evaluation.ts Outdated Show resolved Hide resolved
hunterheston added 5 commits November 11, 2024 17:36
1. Updated package json, removed vertexai refs.
2. Removed optional deps pulled from vertexai plugin.
3. Action -> EvaluatorAction
4. console.warn -> logger.warn
5. Moved quota project warning into index.ts
@HunterHeston HunterHeston requested a review from ssbushi November 11, 2024 18:28
@HunterHeston HunterHeston requested a review from ssbushi November 11, 2024 23:09
js/plugins/checks/package.json Outdated Show resolved Hide resolved
js/plugins/checks/README.md Outdated Show resolved Hide resolved
js/plugins/checks/src/index.ts Outdated Show resolved Hide resolved
@HunterHeston HunterHeston changed the base branch from next to main November 14, 2024 16:51
@HunterHeston HunterHeston merged commit 0c35004 into main Nov 14, 2024
4 checks passed
@HunterHeston HunterHeston deleted the hunterheston-checks-evaluators branch November 14, 2024 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants