[BUG] span -> dataset doesn't carry over meaningful metadata

[mikeldking]

see

[axiomofjoy]

Reported by @hamelsmu

[anticorrelator]

@axiomofjoy @mikeldking could i get a little more color on this? what additional info should we see propogated to the metadata? Just the metadata field on the spans themselves?

[mikeldking]

@axiomofjoy @mikeldking could i get a little more color on this? what additional info should we see propogated to the metadata? Just the metadata field on the spans themselves?

In general the metadata was supposed to carry all the attributes of the span for use in experimentation.

It also should probably take the key value pairs in the metadata and stuff it at the top level into the metadata block.

Context is that you upload traces with metadata, you export it to a dataset, and you want the same kv pairs to propogate.

[mikeldking]

I would expect attributes that looked like

{
   "llm.input_messages....": "..."
   "metadata": "{ 'user': 123 }"
} 

would result in a dataset record with

metadata: { 'user': 123 }

[mikeldking]

We should also just dump any arbitrary attribute into metadata.