Use Blob2 in more places #4281
Use Blob2 in more places #4281
Conversation
sffc
requested review from
robertbastian,
Manishearth,
zbraniecki,
gregtatum,
nordzilla and
a team
as code owners
There was a problem hiding this comment.
this seems like a huge size improvement for just en. It works out to 615 bytes per key (if they all have the same locales), does that make sense?
There was a problem hiding this comment.
Hmm; --locales en includes all regional variants by default, which is where ZeroTrie has its biggest benefits.
The old blob file contains this string:
enen-001en-150en-AEen-AGen-AIen-ASen-ATen-AUen-BBen-BEen-BIen-BMen-BSen-BWen-BZen-CAen-CCen-CHen-CKen-CMen-CXen-CYen-DEen-DGen-DKen-DMen-ERen-FIen-FJen-FKen-FMen-GBen-GDen-GGen-GHen-GIen-GMen-GUen-GYen-HKen-IEen-ILen-IMen-INen-IOen-JEen-JMen-KEen-KIen-KNen-KYen-LCen-LRen-LSen-MGen-MHen-MOen-MPen-MSen-MTen-MUen-MVen-MWen-MYen-NAen-NFen-NGen-NLen-NRen-NUen-NZen-PGen-PHen-PKen-PNen-PRen-PWen-RWen-SBen-SCen-SDen-SEen-SGen-SHen-SIen-SLen-SSen-SXen-SZen-TCen-TKen-TOen-TTen-TVen-TZen-UGen-UMen-VCen-VGen-VIen-VUen-WSen-ZAen-ZMen-ZWhi-Latnundenen-001en-150en-AEen-AGen-AIen-ASen-ATen-AUen-BBen-BEen-BIen-BMen-BSen-BWen-BZen-CAen-CCen-CHen-CKen-CMen-CXen-CYen-DEen-DGen-DKen-DMen-ERen-FIen-FJen-FKen-FMen-GBen-GDen-GGen-GHen-GIen-GMen-GUen-GYen-HKen-IEen-ILen-IMen-INen-IOen-JEen-JMen-KEen-KIen-KNen-KYen-LCen-LRen-LSen-MGen-MHen-MOen-MPen-MSen-MTen-MUen-MVen-MWen-MYen-NAen-NFen-NGen-NLen-NRen-NUen-NZen-PGen-PHen-PKen-PNen-PRen-PWen-RWen-SBen-SCen-SDen-SEen-SGen-SHen-SIen-SLen-SSen-SXen-SZen-TCen-TKen-TOen-TTen-TVen-TZen-UGen-UMen-VCen-VGen-VIen-VUen-WSen-ZAen-ZMen-ZWhi-Latnundenen-001en-150en-AEen-AGen-AIen-ASen-ATen-AUen-BBen-BEen-BIen-BMen-BSen-BWen-BZen-CAen-CCen-CHen-CKen-CMen-CXen-CYen-DEen-DGen-DKen-DMen-ERen-FIen-FJen-FKen-FMen-GBen-GDen-GGen-GHen-GIen-GMen-GUen-GYen-HKen-IEen-ILen-IMen-INen-IOen-JEen-JMen-KEen-KIen-KNen-KYen-LCen-LRen-LSen-MGen-MHen-MOen-MPen-MSen-MTen-MUen-MVen-MWen-MYen-NAen-NFen-NGen-NLen-NRen-NUen-NZen-PGen-PHen-PKen-PNen-PRen-PWen-RWen-SBen-SCen-SDen-SEen-SGen-SHen-SIen-SLen-SSen-SXen-SZen-TCen-TKen-TOen-TTen-TVen-TZen-UGen-UMen-VCen-VGen-VIen-VUen-WSen-ZAen-ZMen-ZWhi-Latnundenen-001en-150en-AEen-AGen-AIen-ASen-ATen-AUen-BBen-BEen-BIen-BMen-BSen-BWen-BZen-CAen-CCen-CHen-CKen-CMen-CXen-CYen-DEen-DGen-DKen-DMen-ERen-FIen-FJen-FKen-FMen-GBen-GDen-GGen-GHen-GIen-GMen-GUen-GYen-HKen-IEen-ILen-IMen-INen-IOen-JEen-JMen-KEen-KIen-KNen-KYen-LCen-LRen-LSen-MGen-MHen-MOen-MPen-MSen-MTen-MUen-MVen-MWen-MYen-NAen-NFen-NGen-NLen-NRen-NUen-NZen-PGen-PHen-PKen-PNen-PRen-PWen-RWen-SBen-SCen-SDen-SEen-SGen-SHen-SIen-SLen-SSen-SXen-SZen-TCen-TKen-TOen-TTen-TVen-TZen-UGen-UMen-VCen-VGen-VIen-VUen-WSen-ZAen-ZMen-ZWhi-Latnundenen-001en-150en-AEen-AGen-AIen-ASen-ATen-AUen-BBen-BEen-BIen-BMen-BSen-BWen-BZen-CAen-CCen-CHen-CKen-CMen-CXen-CYen-DEen-DGen-DKen-DMen-ERen-FIen-FJen-FKen-FMen-GBen-GDen-GGen-GHen-GIen-GMen-GUen-GYen-HKen-IEen-ILen-IMen-INen-IOen-JEen-JMen-KEen-KIen-KNen-KYen-LCen-LRen-LSen-MGen-MHen-MOen-MPen-MSen-MTen-MUen-MVen-MWen-MYen-NAen-NFen-NGen-NLen-NRen-NUen-NZen-PGen-PHen-PKen-PNen-PRen-PWen-RWen-SBen-SCen-SDen-SEen-SGen-SHen-SIen-SLen-SSen-SXen-SZen-TCen-TKen-TOen-TTen-TVen-TZen-UGen-UMen-VCen-VGen-VIen-VUen-WSen-ZAen-ZMen-ZWhi-Latnundenen-001en-150en-AEen-AGen-AIen-ASen-ATen-AUen-BBen-BEen-BIen-BMen-BSen-BWen-BZen-CAen-CCen-CHen-CKen-CMen-CXen-CYen-DEen-DGen-DKen-DMen-ERen-FIen-FJen-FKen-FMen-GBen-GDen-GGen-GHen-GIen-GMen-GUen-GYen-HKen-IEen-ILen-IMen-INen-IOen-JEen-JMen-KEen-KIen-KNen-KYen-LCen-LRen-LSen-MGen-MHen-MOen-MPen-MSen-MTen-MUen-MVen-MWen-MYen-NAen-NFen-NGen-NLen-NRen-NUen-NZen-PGen-PHen-PKen-PNen-PRen-PWen-RWen-SBen-SCen-SDen-SEen-SGen-SHen-SIen-SLen-SSen-SXen-SZen-TCen-TKen-TOen-TTen-TVen-TZen-UGen-UMen-VCen-VGen-VIen-VUen-WSen-ZAen-ZMen-ZWhi-Latnundenen-001en-150en-AEen-AGen-AIen-ASen-ATen-AUen-BBen-BEen-BIen-BMen-BSen-BWen-BZen-CAen-CCen-CHen-CKen-CMen-CXen-CYen-DEen-DGen-DKen-DMen-ERen-FIen-FJen-FKen-FMen-GBen-GDen-GGen-GHen-GIen-GMen-GUen-GYen-HKen-IEen-ILen-IMen-INen-IOen-JEen-JMen-KEen-KIen-KNen-KYen-LCen-LRen-LSen-MGen-MHen-MOen-MPen-MSen-MTen-MUen-MVen-MWen-MYen-NAen-NFen-NGen-NLen-NRen-NUen-NZen-PGen-PHen-PKen-PNen-PRen-PWen-RWen-SBen-SCen-SDen-SEen-SGen-SHen-SIen-SLen-SSen-SXen-SZen-TCen-TKen-TOen-TTen-TVen-TZen-UGen-UMen-VCen-VGen-VIen-VUen-WSen-ZAen-ZMen-ZWhi-Latnund
which is replaced with a ZeroTrie in the new blob.
There was a problem hiding this comment.
ah it's because the blob is generated with FallbackMode::Expand. Could change to ::RuntimeManual to make this even smaller
Fixes #4216