misc(parquet): IWYU and fix linking errors

[zuyu]

Fix the linking errors for PageReader.cpp.o in CI: https://github.com/facebookincubator/velox/actions/runs/11977566772/job/33395741582

• IWYU: add the header velox/dwio/parquet/writer/arrow/LevelConversion.h for DefLevelsToBitmap, DefRepLevelsToList, and DefRepLevelsToBitmap

  velox/velox/dwio/parquet/reader/PageReader.cpp

  Lines 610 to 637 in e27867c

  	switch (mode) {
  	case LevelMode::kNulls:
  	DefLevelsToBitmap(
  	definitionLevels_.data() + begin, end - begin, info, &bits);
  	break;
  	case LevelMode::kList: {
  	arrow::DefRepLevelsToList(
  	definitionLevels_.data() + begin,
  	repetitionLevels_.data() + begin,
  	end - begin,
  	info,
  	&bits,
  	lengths);
  	// Convert offsets to lengths.
  	for (auto i = 0; i < bits.values_read; ++i) {
  	lengths[i] = lengths[i + 1] - lengths[i];
  	}
  	break;
  	}
  	case LevelMode::kStructOverLists: {
  	DefRepLevelsToBitmap(
  	definitionLevels_.data() + begin,
  	repetitionLevels_.data() + begin,
  	end - begin,
  	info,
  	&bits);
  	break;
  	}

  velox/velox/dwio/parquet/writer/arrow/LevelConversion.h

  Lines 171 to 212 in f33b40d

  	// Converts def_levels to validity bitmaps for non-list arrays and structs that
  	// have at least one member that is not a list and has no list descendents. For
  	// lists use DefRepLevelsToList and structs where all descendants contain a
  	// list use DefRepLevelsToBitmap.
  	void PARQUET_EXPORT DefLevelsToBitmap(
  	const int16_t* def_levels,
  	int64_t num_def_levels,
  	LevelInfo level_info,
  	ValidityBitmapInputOutput* output);
  	// Reconstructs a validity bitmap and list offsets for a list arrays based on
  	// def/rep levels. The first element of offsets will not be modified if
  	// rep_levels starts with a new list. The first element of offsets will be used
  	// when calculating the next offset. See documentation onf DefLevelsToBitmap
  	// for when to use this method vs the other ones in this file for
  	// reconstruction.
  	//
  	// Offsets must be sized to 1 + values_read_upper_bound.
  	void PARQUET_EXPORT DefRepLevelsToList(
  	const int16_t* def_levels,
  	const int16_t* rep_levels,
  	int64_t num_def_levels,
  	LevelInfo level_info,
  	ValidityBitmapInputOutput* output,
  	int32_t* offsets);
  	void PARQUET_EXPORT DefRepLevelsToList(
  	const int16_t* def_levels,
  	const int16_t* rep_levels,
  	int64_t num_def_levels,
  	LevelInfo level_info,
  	ValidityBitmapInputOutput* output,
  	int64_t* offsets);
  	// Reconstructs a validity bitmap for a struct every member is a list or has
  	// a list descendant. See documentation on DefLevelsToBitmap for when more
  	// details on this method compared to the other ones defined above.
  	void PARQUET_EXPORT DefRepLevelsToBitmap(
  	const int16_t* def_levels,
  	const int16_t* rep_levels,
  	int64_t num_def_levels,
  	LevelInfo level_info,
  	ValidityBitmapInputOutput* output);

• Link the library velox_dwio_arrow_parquet_writer_lib, which contains LevelConversion.cpp
  

  velox/velox/dwio/parquet/writer/arrow/CMakeLists.txt

  Lines 21 to 35 in e27867c

  	velox_add_library(
  	velox_dwio_arrow_parquet_writer_lib
  	ArrowSchema.cpp
  	ArrowSchemaInternal.cpp
  	ColumnWriter.cpp
  	Encoding.cpp
  	Encryption.cpp
  	EncryptionInternal.cpp
  	Exception.cpp
  	FileEncryptorInternal.cpp
  	FileDecryptorInternal.cpp
  	FileWriter.cpp
  	LevelComparison.cpp
  	LevelComparisonAvx2.cpp
  	LevelConversion.cpp

[netlify]

✅ Deploy Preview for meta-velox canceled.

Name	Link
🔨 Latest commit	6150408
🔍 Latest deploy log	https://app.netlify.com/sites/meta-velox/deploys/6744e1da86039b0008f3a6c8

[yingsu00]

If LevelConversion.h is used in both Reader and Writer, then maybe we should move it to common.

[zuyu]

Agree. Or velox/dwio/parquet/arrow.

[yingsu00]

Agree. Or velox/dwio/parquet/arrow.

Let's put it in common. We will gradually remove arrow code in the future, so don't want another arrow folder.

[majetideepak]

I agree. Opened #11678. I started working on this.
I will open a PR today.