Optimize JsObject.equals()/hashCode()

[htmldoug]

Summary

For JsObject.equals()/hashCode():

• improves runtime by avoiding making two copies of the underlying data
• improves memory footprint by avoiding holding lazy val fields: collection.Seq[(String, JsValue)] after the computation is performed.
• Benchmark: jmh results

[cchantep]

How could this happen?

[htmldoug]

Good question! As an illustrative example, consider a json text that looks like [[{"k":0}], [{"k":0}],[{"k":0}],[{"k":0}],...] whose length approaches the jvm's total heap size. Json.parse(inputStream) would throw OutOfMemoryError, but is it possible to represent such a text as a JsValue?

Currently, jackson-core will return the same "k" String instance for all keys in the example, but that's only a small part of the eventual JsValue memory footprint. It's possible to apply the same technique to the other duplicated values as well.

In weepickle terms, a minimal implementation could look like:

val jsValue = FromJson(inputStream).transform(new DdVisitor(PlayJson))

class DdVisitor[T, J](underlying: Visitor[T, J])
 (implicit cache: mutable.Map[Any, Any] = mutable.Map.empty)
 extends Visitor.Delegate[T, J](underlying)
   with RecursiveDelegate[T, J] {

 override def asRecursiveDelegate[A, B](visitor: Visitor[A, B]) =
   new DdVisitor(
     visitor.map { jsValue =>
       cache
         .getOrElseUpdate(jsValue, jsValue)
         .asInstanceOf[B]
     }
   )
}

As we walk the input depth-first, every JsValue node is run through a Map[JsValue, JsValue] to return a canonical instance. Each {"k":0} would result in a cache hit and return a single canonical instance.

Back to your question, this line would be hit when we attempt to deduplicate [{"k":0}]. In order to check if two JsArray are identical, it needs to test that all elements pass equals(). Since the {"k":0} JsObject that's present in both arrays would have already been canonicalized to the same instance, this eq other would be true for that comparison.

I wish this were a purely hypothetical scenario. In practice, we talk to some APIs that return duplication-heavy JSON texts that have OOME'd our apps before. My favorite example was a 5.4 GB JsObject in heap whose UTF-8 input compressed down to 1.7 MB. Those units are not typos. That's ~3,000x memory amplification.

One of the things we like about play-json is that it gives us the flexibility to mitigate these problems by extending the JsValue classes. Applying deduplication brought the 5.4 GB JsValue down to a 205 MB JsValue. Bringing our own Map/Seq and doing some unboxing got the same input down to a 23 MB JsValue. Throughput was also faster due to eliminated GC pressure. Win-win. (We also logged an issue to please stop sending us info we don't need, but I was on-call and wanted to stop getting paged about this.)

Some APIs are highly normalized and duplicate-free, but in practice, I suspect many of them contain significant duplication. Using the few thousand test json files in our test double service as a proxy for the real-world data we deal with, I found that 71% of the nodes were present somewhere else in the same json text. Duplication wasn't present only in primitives either. I found significant duplication up to height=5 of the trees. I'm very curious if we just deal with bad REST APIs or if this is more widespread.

Anyway, over typical usage this line won't hit, although it's quite cheap, especially compared to the previous implementation that constructed two copies of underlying--one of them a HashSet! If needed, we could remove it and get by with the underlying Map performing the eq check, but while I was benchmarking, I saw the opportunity to short-circuit and took it.

Thanks for the early feedback on this draft! Code is actually ready for review, but I didn't want to send reviewer notifications until I'd written the PR description. Now that I've typed all this out, maybe I'll just skip it. :)

[cchantep]

K, would be interested to add a benchmark after.

[htmldoug]

Before:

Benchmark                    (size)   Mode  Cnt    Score    Error   Units
JsObjectBench.oEqualsCopy        15  thrpt    5  290.695 ±  4.548  ops/ms
JsObjectBench.oEqualsEq          15  thrpt    5  300.135 ± 10.104  ops/ms
JsObjectBench.oEqualsNe          15  thrpt    5  289.910 ± 17.405  ops/ms
JsObjectBench.oHashCodeWarm      15  thrpt    5  521.727 ± 40.900  ops/ms

After:

Benchmark                    (size)   Mode  Cnt       Score       Error   Units
JsObjectBench.oEqualsCopy        15  thrpt    5    4386.757 ±   124.675  ops/ms
JsObjectBench.oEqualsEq          15  thrpt    5  430769.750 ± 43891.912  ops/ms
JsObjectBench.oEqualsNe          15  thrpt    5  321636.207 ± 23899.623  ops/ms
JsObjectBench.oHashCodeWarm      15  thrpt    5    4024.849 ±   276.276  ops/ms

[htmldoug]

Longer run, higher precision, prettier graphs:
https://jmh.morethan.io/?sources=https://gist.githubusercontent.com/htmldoug/6ebb5f3f169e35535a99e2d198c9153d/raw/f8f1a3b905de98829e607ad41fe8ffa936ffd6c8/report-before.json,https://gist.githubusercontent.com/htmldoug/6ebb5f3f169e35535a99e2d198c9153d/raw/f8f1a3b905de98829e607ad41fe8ffa936ffd6c8/report-after.json

[gmethvin]

Seems reasonable to me, and a nice performance improvement!

[cchantep]

Merge ready?

[htmldoug]

I was thinking #675 first if you want to see the memory footprint drop. Otherwise, ready.

[cchantep]

See throughput bench, if mem footprint bench is a quickwin ...

[htmldoug]

Sorry, I don't follow. Is there an existing test you want me to look at?

The memory footprint tests I'm referring to are here. The benefits are less about throughput and more about preventing OOME conditions over large or many inputs that have to be in memory at once.

[cchantep]

Fine for me