Error running active probes

[milonbhattacharya]

Steps to reproduce

How'd you do it?

1. Attempting to run https://reference.garak.ai/en/latest/garak.probes.realtoxicityprompts.html
2. garak --model_name deepseek-r1 --model_type ollama --probes realtoxicityprompts.RTPBlank --generations 1

Expected behavior

Probe should have run successfully.

Current behavior

What happens instead?

🕵️ queue of probes: realtoxicityprompts.RTPBlank
DEBUG:2025-02-07 17:11:23,225:jax._src.path:31: etils.epath found. Using etils.epath for file I/O.
detector load failed: toxicity.ToxicCommentModel, skipping >>
No detectors, nothing to do

garak.log

return importlib.import_module("." + module_name, self.name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1147, in _find_and_load_unlocked
File "", line 690, in _load_unlocked
File "", line 940, in exec_module
File "", line 241, in _call_with_frames_removed
File "/Users/XXX/XX/lib/python3.11/site-packages/transformers/pipelines/init.py", line 26, in
from ..image_processing_utils import BaseImageProcessor
File "/Users/XXX/XX/lib/python3.11/site-packages/transformers/image_processing_utils.py", line 21, in
from .image_transforms import center_crop, normalize, rescale
File "/Users/XXX/XX/lib/python3.11/site-packages/transformers/image_transforms.py", line 50, in
import tensorflow as tf
File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/init.py", line 49, in
from tensorflow._api.v2 import internal
File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/_api/v2/internal/init.py", line 8, in
from tensorflow._api.v2.internal import autograph
File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/_api/v2/internal/autograph/init.py", line 8, in
from tensorflow.python.autograph.core.ag_ctx import control_status_ctx # line: 34
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/python/autograph/core/ag_ctx.py", line 21, in
from tensorflow.python.autograph.utils import ag_logging
File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/python/autograph/utils/init.py", line 17, in
from tensorflow.python.autograph.utils.context_managers import control_dependency_on_returns
File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/python/autograph/utils/context_managers.py", line 19, in
from tensorflow.python.framework import ops
File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/python/framework/ops.py", line 33, in
from tensorflow.core.framework import attr_value_pb2
File "/Users/XXX/XX/lib/python3.11/site-packages/tensorflow/core/framework/attr_value_pb2.py", line 5, in
from google.protobuf.internal import builder as _builder
ImportError: cannot import name 'builder' from 'google.protobuf.internal' (/Users/XXX/XX/lib/python3.11/site-packages/google/protobuf/internal/init.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/Users/XXX/XXX/lib/python3.11/site-packages/garak/_plugins.py", line 416, in load_plugin
plugin_instance = klass(config_root=config_root)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/XXX/XXX/lib/python3.11/site-packages/garak/detectors/base.py", line 82, in init
from transformers import (
File "", line 1229, in _handle_fromlist
File "/Users/XXX/XXX/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1766, in getattr
module = self._get_module(self._class_to_module[name])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/XXX/XXX/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1780, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.pipelines because of the following error (look up to see its traceback):
cannot import name 'builder' from 'google.protobuf.internal' (/Users/XXX/XXX/lib/python3.11/site-packages/google/protobuf/internal/init.py)
2025-01-28 10:12:18,136 ERROR detector load failed: toxicity.ToxicCommentModel, skipping >>
2025-01-28 10:12:18,136 INFO harness init: <garak.harnesses.base.Harness object at 0x15df4cb90>
2025-01-28 10:12:18,136 WARNING No detectors, nothing to do
2025-01-28 10:12:18,136 ERROR No detectors, nothing to do
Traceback (most recent call last):
File "/Users/XXX/XXX/lib/python3.11/site-packages/garak/cli.py", line 530, in main
command.probewise_run(
File "/Users/XXX/XXX/lib/python3.11/site-packages/garak/command.py", line 237, in probewise_run
probewise_h.run(generator, probe_names, evaluator, buffs)
File "/Users/XXX/XXX/lib/python3.11/site-packages/garak/harnesses/probewise.py", line 107, in run
h.run(model, [probe], detectors, evaluator, announce_probe=False)
File "/Users/XXX/XXX/lib/python3.11/site-packages/garak/harnesses/base.py", line 88, in run
raise ValueError(msg)
ValueError: No detectors, nothing to do

garak version

v0.10.0

Additional Information

1. Operating system : Mac OS Sequoia 15.2 (24C101)
2. Python version: 3.11.5
3. Install method (pypi, pip based repo install, direct repository checkout with git) pip
4. Logs from execution run report.html / report.jsonl / hitlog.jsonl and if possible garak.log : Added
5. Details of execution config such as command line flags or config files
6. Any relevant hardware or resource information: prob is active: realtoxicityprompts 🌟