Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

return value from batch_decode is already a list #743

Merged
merged 1 commit into from
Jun 19, 2024

Conversation

jmartin-tech
Copy link
Collaborator

When executing tests using the Model class, the decode always returns a list[str] object and should not be wrapped again.

Test with:

python -m garak -m huggingface.Model -n meta-llama/Llama-2-7b-chat-hf -g 1 -p continuation

Prior to change:

    trimmed_output = re.sub(r"^\s+", " ", output)
  File "/usr/lib/python3.10/re.py", line 209, in sub
    return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object

After:

continuation.ContinueSlursReclaimedSlurs80                                 continuation.Continuation: PASS  ok on  285/ 285
📜 report closed :) garak_runs/garak.e0c0ee27-e4a5-4437-a5b3-bfeb6fdf0ee6.report.jsonl
📜 report html summary being written to garak_runs/garak.e0c0ee27-e4a5-4437-a5b3-bfeb6fdf0ee6.report.html
✔️  garak run complete in 3270.13s

When executing tests using the Model class, the decode always
returns a `list[str]` object and should not be wrapped again.

Signed-off-by: Jeffrey Martin <jemartin@nvidia.com>
@leondz leondz merged commit dc03c71 into NVIDIA:main Jun 19, 2024
4 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Jun 19, 2024
@jmartin-tech jmartin-tech deleted the fix/huggingface-model-deprefix branch August 14, 2024 19:40
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants