re_datastore: standardize the payload schema for insertion

[teh-cmc]

Standardize and put into writing insert-payload schema

• should the entire payload be a list, for client-side batching?
• how strict do we want to be? how dynamic to we want to be?
• what are we allowed to do? what is forbidden?
  • is having the same component present multiple times legal?
  • is passing no instances legal?
  • etc

[teh-cmc]

Example for reference:

This insertion:

    let (schema, components) = build_message(
        &ent_path,
        [build_log_time(now_plus_20ms), build_frame_nr(frame41)],
        [build_instances(nb_instances), build_rects(nb_instances)],
    );

turns into this schema:

schema: Schema {
    fields: [
        Field {
            name: "timelines",
            data_type: Struct(
                [
                    Field {
                        name: "log_time",
                        data_type: Timestamp(
                            Nanosecond,
                            None,
                        ),
                        is_nullable: false,
                        metadata: {
                            "RERUN:timeline": "Time",
                        },
                    },
                    Field {
                        name: "frame_nr",
                        data_type: Int64,
                        is_nullable: false,
                        metadata: {
                            "RERUN:timeline": "Sequence",
                        },
                    },
                ],
            ),
            is_nullable: false,
            metadata: {},
        },
        Field {
            name: "components",
            data_type: Struct(
                [
                    Field {
                        name: "instances",
                        data_type: List(
                            Field {
                                name: "item",
                                data_type: UInt32,
                                is_nullable: true,
                                metadata: {},
                            },
                        ),
                        is_nullable: false,
                        metadata: {},
                    },
                    Field {
                        name: "rects",
                        data_type: List(
                            Field {
                                name: "item",
                                data_type: Struct(
                                    [
                                        Field {
                                            name: "x",
                                            data_type: Float32,
                                            is_nullable: false,
                                            metadata: {},
                                        },
                                        Field {
                                            name: "y",
                                            data_type: Float32,
                                            is_nullable: false,
                                            metadata: {},
                                        },
                                        Field {
                                            name: "w",
                                            data_type: Float32,
                                            is_nullable: false,
                                            metadata: {},
                                        },
                                        Field {
                                            name: "h",
                                            data_type: Float32,
                                            is_nullable: false,
                                            metadata: {},
                                        },
                                    ],
                                ),
                                is_nullable: true,
                                metadata: {},
                            },
                        ),
                        is_nullable: false,
                        metadata: {},
                    },
                ],
            ),
            is_nullable: false,
            metadata: {},
        },
    ],
    metadata: {
        "RERUN:entity_path": "this/that",
    },
}

with this payload:

components: Chunk {
    arrays: [
        StructArray[{log_time: 2022-12-02 15:14:23.186512372, frame_nr: 41}],
        StructArray[{instances: [4262470174, 24667012, 2536452249], rects: [{x: 0, y: 0, w: 0, h: 0}, {x: 1, y: 1, w: 1, h: 1}, {x: 2, y: 2, w: 2, h: 2}]}],
    ],
}

[jondo2010]

So the work in #501 addresses a large part of this, at least in code by factoring out matching encode/decode function pairs (implemented mostly as From conversions between helper types).

The airity and duplication questions are not answered.

[teh-cmc]

See #1619