Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

numexpr 2.8.5 breaks CI #6469

Closed
anmyachev opened this issue Aug 8, 2023 · 0 comments · Fixed by #6474
Closed

numexpr 2.8.5 breaks CI #6469

anmyachev opened this issue Aug 8, 2023 · 0 comments · Fixed by #6474
Labels
P0 Highest priority tasks requiring immediate fix

Comments

@anmyachev
Copy link
Collaborator

anmyachev commented Aug 8, 2023

Pandas is not compatible with the release.

See pandas-dev/pandas#54449

Broken CI: https://github.com/modin-project/modin/actions/runs/5799349333/job/15721083501

Example of the error:

def test_eval_and_query_with_local_and_global_var(method, data, local_var):
        modin_df, pandas_df = pd.DataFrame(data), pandas.DataFrame(data)
        op = "+" if method == "eval" else "<"
        for expr in (f"col1 {op} @local_var", f"col1 {op} @TEST_VAR"):
>           df_equals(getattr(modin_df, method)(expr), getattr(pandas_df, method)(expr))

modin/pandas/test/dataframe/test_udf.py:351: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
modin/logging/logger_decorator.py:128: in run_and_log
    return obj(*args, **kwargs)
modin/pandas/dataframe.py:818: in eval
    new_query_compiler = self._query_compiler.eval(expr, **kwargs)
modin/logging/logger_decorator.py:128: in run_and_log
    return obj(*args, **kwargs)
modin/core/storage_formats/pandas/query_compiler.py:[228](https://github.com/modin-project/modin/actions/runs/5799349333/job/15721083501#step:7:229)0: in eval
    pandas.DataFrame(columns=self.columns)
/usr/share/miniconda/envs/modin/lib/python3.8/site-packages/pandas/core/frame.py:4566: in eval
    return _eval(expr, inplace=inplace, **kwargs)
/usr/share/miniconda/envs/modin/lib/python3.8/site-packages/pandas/core/computation/eval.py:357: in eval
    ret = eng_inst.evaluate()
/usr/share/miniconda/envs/modin/lib/python3.8/site-packages/pandas/core/computation/engines.py:81: in evaluate
    res = self._evaluate()
/usr/share/miniconda/envs/modin/lib/python3.8/site-packages/pandas/core/computation/engines.py:121: in _evaluate
    return ne.evaluate(s, local_dict=scope)
/usr/share/miniconda/envs/modin/lib/python3.8/site-packages/numexpr/necompiler.py:943: in evaluate
    raise e
/usr/share/miniconda/envs/modin/lib/python3.8/site-packages/numexpr/necompiler.py:851: in validate
    _names_cache[expr_key] = getExprNames(ex, context)
/usr/share/miniconda/envs/modin/lib/python3.8/site-packages/numexpr/necompiler.py:714: in getExprNames
    ex = stringToExpression(text, {}, context)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

s = '(col1) + (__pd_eval_local_local_var)', types = {}
context = {'optimization': 'aggressive', 'truediv': False}

    def stringToExpression(s, types, context):
        """Given a string, convert it to a tree of ExpressionNode's.
        """
        # sanitize the string for obvious attack vectors that NumExpr cannot
        # parse into its homebrew AST. This is to protect the call to `eval` below.
        # We forbid `;`, `:`. `[` and `__`, and attribute access via '.'.
        # We cannot ban `.real` or `.imag` however...
        no_whitespace = re.sub(r'\s+', '', s)
        if _forbidden_re.search(no_whitespace) is not None:
>           raise ValueError(f'Expression {s} has forbidden control characters.')
E           ValueError: Expression (col1) + (__pd_eval_local_local_var) has forbidden control characters.
@anmyachev anmyachev added the P0 Highest priority tasks requiring immediate fix label Aug 8, 2023
anmyachev added a commit to anmyachev/modin that referenced this issue Aug 8, 2023
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
vnlitvinov pushed a commit that referenced this issue Aug 9, 2023
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
RehanSD pushed a commit that referenced this issue Aug 22, 2023
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
RehanSD pushed a commit that referenced this issue Aug 22, 2023
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
RehanSD pushed a commit that referenced this issue Aug 22, 2023
Signed-off-by: Anatoly Myachev <anatoly.myachev@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P0 Highest priority tasks requiring immediate fix
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant