Batch Handling Upgrades

The batch handling framework consists of a number of layers that combine to enable Drill to control the size of each record batch, which in turn allows Drill to implement effective memory management and admission control.

The material here starts with concepts, then provides a tour of the various components. Each component is heavily commented, so after reading this material, you should be able to get the details from the code itself.

• Conceptual Overview
• Components
• Code
• Metadata
• Row Set Mechanism
• Column Accessors
• Column Readers
• Column Writers
• Result Set Loader
• Operator Framework
• Projection Framework
• Scan Framework
• Easy Format Plugin
• CSV (Compliant Text) Reader
• JSON Reader
• Conclusion and Future Work