New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add XSTest Scenario #2831

Merged

farzaank merged 34 commits into main from farzaan/xstest

Aug 13, 2024

Contributor

farzaank commented Jul 22, 2024 •

edited

Loading

Doesn't use the same wording as the paper, but again I added a more detailed prompt asking for the same thing.

farzaank added 26 commits

June 6, 2024 01:50


          tested

8b7d463


          changes logged

d756f14


          Merge branch 'main' into farzaan/simplesafetytests

e8da56d


          update

dfc6b40


          remove

c533c47


          working sst

c3a1fbc


          updated

3414e78


          Merge branch 'main' into farzaan/simplesafetytests

29a5a3a


          consistent naming

873dac3


          updates

ca19ee3


          remove print statement

a952701


          finalized updates

4eefc39


          added harmbench

3e50a93


          updates

e81e420


          fix bugs

4315edf


          updates

831ce49


          Merge branch 'main' into farzaan/anthropic

4e1492c


          things

35224ff


          added

40b54b9

fix

69660cc


          Merge branch 'main' into farzaan/xstest

a05911b


          updates for schema

45d561a


          added

ab1854b


          remove unnecessary files

8653c62


          remove schema

d1d5593


          use inheritance

924fca9

yifanmai requested changes

View reviewed changes

src/helm/benchmark/scenarios/xstest_scenario.py Outdated Show resolved Hide resolved

src/helm/benchmark/run_specs/classic_run_specs.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

farzaank added 2 commits

August 1, 2024 13:57


          fixes

8db9a67


          remove bbq change for now

2e35c08

farzaank requested a review from percyliang

August 1, 2024 21:13

farzaank requested a review from yifanmai

August 1, 2024 21:13


          fix docstring

a801cd7

yifanmai requested changes

View reviewed changes

src/helm/benchmark/annotation/model_as_judge_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/model_as_judge_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/model_as_judge_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/scenarios/xstest_scenario.py Outdated Show resolved Hide resolved

Collaborator

yifanmai commented Aug 1, 2024

Please also resolve ./pre-commit.sh issues.

yifanmai reviewed

View reviewed changes

src/helm/benchmark/annotation/model_as_judge_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

farzaank added 2 commits

August 4, 2024 23:44


          finalize revisions

b41a833


          fix flake issues

d43fa96

farzaank requested a review from yifanmai

August 5, 2024 16:54

farzaank commented

View reviewed changes

src/helm/benchmark/metrics/safety_metrics.py Outdated Show resolved Hide resolved


          update safety score

826f616

yifanmai reviewed

View reviewed changes

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

src/helm/benchmark/annotation/model_as_judge_annotator.py Outdated Show resolved Hide resolved

yifanmai approved these changes

View reviewed changes


          final revisions

0ac960f

percyliang reviewed

View reviewed changes

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

percyliang reviewed

View reviewed changes

src/helm/benchmark/annotation/xstest_annotator.py Outdated Show resolved Hide resolved

percyliang reviewed

View reviewed changes

src/helm/benchmark/metrics/safety_metrics.py Show resolved Hide resolved


          small fixes based on Percy's feedback

e6c75c6

farzaank merged commit b2f87f8 into main

6 checks passed

farzaank deleted the farzaan/xstest branch

August 13, 2024 16:58

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet