When an why should lua objects be cloned?

[CodeSandwich]

Each lua object has the clone method. When should it be used and when is it used automatically? Is header.attr = otherHeader.attr safe? Is pandoc.Header(1, "header", otherHeader.attr) safe?

[badumont]

`walk()` should be used with an empty filter instead of `clone()` when you want to perform a deep copy of an object, i. e. one where all members of the object are copied too and where a modification to the source object doesn't entail one to the new object. To the contrary, with `header.attr = otherHeader.attr` or `header.attr = otherHeader.attr:clone()`, `header.attr` becomes a mere pointer to `otherHeader.attr`.

[tarleb]

That's a good question.

Is header.attr = otherHeader.attr safe?

No.

Is pandoc.Header(1, "header", otherHeader.attr) safe?

Yes.

In general, calls to a constructor function will deep-clone the arguments. The one exception to this are lists, i.e. pandoc.List. The behavior of lists can, admittedly, be a bit unexpected at times. The walk{} hack mentioned above is a good way to ensure a deep copy.

[CodeSandwich]

Maybe pandoc.List.clone should be changed to do deep copying then? Are there use cases for the user calling clone and expecting getting a new list with the old references?

Does pandoc.Header.clone do deep cloning of its List-based data, like Blocks, Inlines or Attributes? If it just calls clone on them, then I think that it's an important argument for making pandoc.List.clone deep-copying, otherwise pandoc.Header.clone has a limited real-world usability. Without proper documentation it's also usually harmful to the users by introducing hard to debug entanglement of the cloned objects.

[tarleb]

I agree, it should be changed. I'll look into that.

Let me try to explain the underlying mechanics, which might make the whole thing a little clearer: The vast majority of elements, including Header, Para, Emph, Str, etc, are lazy in the sense that their value is in "Haskell-land", and only the attributes that are accessed are pushed into "Lua-land". The reason for this is performance, as we are (usually) not interested in every property and every child-element. However, lists are still strict because (a) getting good lazy lists was difficult and (b) the most common use-case of lists is to iterate over each element, and lazyness would make performance worse in this case.

Calling :clone() on a lazy element causes pandoc to first convert the Lua-element back into a Haskell-element and then push that new Haskell-element to Lua again. So everything will be deep-copied. For strict elements we work with the Lua values, so we get shallow copies. Hence the current behavior.

[tarleb]

I've updated pandoc.Blocks.clone and pandoc.Inlines.clone to create deep copies. There are still instances where cloning remains shallow, e.g., when cloning the list of rows in a table. This needs fixing on a more fundamental level, i.e., in the C code of the List Lua library.