This repository has been archived by the owner on Aug 28, 2021. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 266
Optimize struct type validation. The regexp's show in profiles. #2866
Comments
cc @aboodman to set priority |
this essentially affects writing of data to a remote server (which logically includes syncing to a local ldb store) |
IOW, not a prio right now. |
noms sync
Blocked by #3123 |
Open
Is this still valid? |
I looked recently and I kinda looked like it was. The problem is that the regex is expensive. |
Which regexp is this? The one that validates the name? |
Yup. Struct and field name. |
arv
added a commit
to arv/noms
that referenced
this issue
Sep 22, 2017
arv
added a commit
that referenced
this issue
Sep 22, 2017
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
While doing something else, I noticed that sync appeared to have gotten slower. I bisected it and discovered that 5e901b0 regressed syncing ~300MB of sf_scrime data from ldb to ldb by ~ 100%.
After some short digging, I'm fairly sure it's because of the addition of type validation to the validating_batch_sink.
Thinking about it some, I think it's an easy fix because of the type cache. Basically we're validating over and over again exactly the same in-memory type graph (the type cache ensures that identical types are interned). we just need to find a way to only do it once per type.
The text was updated successfully, but these errors were encountered: