You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Our domain, quality, AEGIS, and FineWeb-Edu classifier CLI scripts all accept the --input-text-field parameter but do not use it. Currently, we hardcode these classifiers to use only be able to run on a column called "text", see here.
We need to modify _run_classifier_helper and all subsequent functions to accept a text column.
The text was updated successfully, but these errors were encountered:
Our domain, quality, AEGIS, and FineWeb-Edu classifier CLI scripts all accept the
--input-text-field
parameter but do not use it. Currently, we hardcode these classifiers to use only be able to run on a column called "text", see here.We need to modify
_run_classifier_helper
and all subsequent functions to accept a text column.The text was updated successfully, but these errors were encountered: