Skip to content

QuickParquetWrite

Cinchoo edited this page Jun 24, 2020 · 2 revisions

Writing Parquet file

To generate Parquet file, use the ChoParquetWriter component to generate it. Sample below shows how to create Parquet file (Emp.parquet)

Write list of objects

List<ExpandoObject> objs = new List<ExpandoObject>();
dynamic rec1 = new ExpandoObject();
rec1.Id = 1;
rec1.Name = "Mark";
objs.Add(rec1);
 
dynamic rec2 = new ExpandoObject();
rec2.Id = 2;
rec2.Name = "Jason";
objs.Add(rec2);
 
using (var parser = new ChoParquetWriter("Emp.parquet"))
{
    parser.Write(objs);
}

Write each object

using (var parser = new ChoParquetWriter("Emp.parquet"))
{
    dynamic rec1 = new ExpandoObject();
    rec1.Id = 1;
    rec1.Name = "Mark";
    parser.Write(item);

    dynamic rec1 = new ExpandoObject();
    rec1.Id = 2;
    rec1.Name = "Jason";
    parser.Write(item);
}

Write using POCO object

public class Employee
{
    public int Id { get; set; }
    public string Name { get; set; }
}
List<Employee> objs = new List<Employee>();
objs.Add(new Employee() { Id = 1, Name = "Tom" });
objs.Add(new Employee() { Id = 2, Name = "Mark" });

using (var parser = new ChoParquetWriter<Employee>("Emp.parquet"))
{
    parser.Write(objs);
}

Please visit below article for detailed walk-through of Parquet writer

Cinchoo ETL - Parquet Writer