Merging quantized model with pass through

[dnhkng]

Is there a way to do this?

I understand why F16 is required for linear and slerp, but can we do passthrough of quantized layer, as currently it necessary to go via huge models and requantize, which is a big pain point.

[cg123]

It would be possible to write a standalone script to do this. I don't think working with quantized GGUF models in mergekit-yaml makes much sense, as there are very few operations that it would be able to actually support. Just stacking layers would be reasonable though. I'll add this to my list of things to investigate when I have the time. Thanks for the suggestion!

[dnhkng]

No need for GGUF now, as it's being done in a PR