Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix values with different data types caused failure #10445

Merged
merged 11 commits into from
May 14, 2024

Conversation

b41sh
Copy link
Contributor

@b41sh b41sh commented May 10, 2024

Which issue does this PR close?

Closes #10440

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added logical-expr Logical plan and expressions sqllogictest SQL Logic Tests (.slt) labels May 10, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

datafusion/expr/src/logical_plan/builder.rs Show resolved Hide resolved
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look good to me -- thank you @b41sh and @jayzhan211

I had some small code suggestions but I don't think they are required to merge this PR

Comment on lines 218 to 225
let data_type = row[j].get_type(&empty_schema)?;
if data_type != *field_type {
row[j] = Expr::Cast(Cast {
expr: Box::new(row[j].clone()),
data_type: field_type.clone(),
});
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can write this more concisely like

Suggested change
let data_type = row[j].get_type(&empty_schema)?;
if data_type != *field_type {
row[j] = Expr::Cast(Cast {
expr: Box::new(row[j].clone()),
data_type: field_type.clone(),
});
}
}
row[j] = std::mem::take(&mut row[j]).cast_to(&field_type, &empty_schema)?;

The std::mem::take is needed to avoid requiring a clone

}
if let Some(prev_type) = common_type {
// get common type of each column values.
match values_coercion(&data_type, &prev_type) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: I think you could write the same thing like this slightly more concisely (and I don't think we need new_type.clone())

                if let Some(prev_type) = common_type {
                    // get common type of each column values.
                    let Some(new_type) = values_coercion(&data_type, &prev_type) else {
                        return plan_err!("Inconsistent data type across values list at row {i} column {j}. Was {prev_type} but found {data_type}");
                    };
                    common_type = Some(new_type);
                } else {
                    common_type = Some(data_type.clone());
                }

@@ -2140,4 +2156,25 @@ mod tests {
);
Ok(())
}

#[test]
fn test_values() -> Result<()> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don't need this since creating table in slt already covered it

@b41sh
Copy link
Contributor Author

b41sh commented May 14, 2024

Look good to me -- thank you @b41sh and @jayzhan211

I had some small code suggestions but I don't think they are required to merge this PR

Very good suggestion, I have revised it, thank you for your review @alamb

@alamb alamb merged commit dbd0186 into apache:main May 14, 2024
23 checks passed
@alamb
Copy link
Contributor

alamb commented May 14, 2024

Thanks again @b41sh !

findepi pushed a commit to findepi/datafusion that referenced this pull request Jul 16, 2024
* Fix values with different data types caused failure

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* add `list_coercion`

* fix review suggestions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
logical-expr Logical plan and expressions sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Type coercion when creating table
4 participants