Skip to content

Commit

Permalink
[SPARK-23388][SQL] Support for Parquet Binary DecimalType in Vectoriz…
Browse files Browse the repository at this point in the history
…edColumnReader

## What changes were proposed in this pull request?

Re-add support for parquet binary DecimalType in VectorizedColumnReader

## How was this patch tested?

Existing test suite

Author: James Thompson <jamesthomp@users.noreply.github.com>

Closes #20580 from jamesthomp/jt/add-back-binary-decimal.

(cherry picked from commit 5bb1141)
Signed-off-by: gatorsmile <gatorsmile@gmail.com>
  • Loading branch information
jamesthomp authored and gatorsmile committed Feb 12, 2018
1 parent 70be603 commit 4e13820
Showing 1 changed file with 2 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -444,7 +444,8 @@ private void readBinaryBatch(int rowId, int num, WritableColumnVector column) {
// This is where we implement support for the valid type conversions.
// TODO: implement remaining type conversions
VectorizedValuesReader data = (VectorizedValuesReader) dataColumn;
if (column.dataType() == DataTypes.StringType || column.dataType() == DataTypes.BinaryType) {
if (column.dataType() == DataTypes.StringType || column.dataType() == DataTypes.BinaryType
|| DecimalType.isByteArrayDecimalType(column.dataType())) {
defColumn.readBinarys(num, column, rowId, maxDefLevel, data);
} else if (column.dataType() == DataTypes.TimestampType) {
if (!shouldConvertTimestamps()) {
Expand Down

0 comments on commit 4e13820

Please sign in to comment.