Snapshot 18-07-23

This is a development snapshot of KoboldAI United meant for Windows users using the full offline installer.

Since the last snapshot the backend has been overhauled completely and now supports BitsandBytes 4-bit.
Note: In this release you can no longer use --trust_remote_code and the command line options for model loading have changed.

All the hard work since last release

• fix: tpu tokenizers errors by @Zurnaz in #353
• Fix TPU API errors by @one-some in #357
• GPT-J fix by @one-some in #359
• Fix memory->genre bug in context viewer bar tooltip by @one-some in #358
• Updated embedded Kobold Lite to v32 by @LostRuins in #363
• Implement modular model backends Phase 1 by @ebolam in #362
• Anti-malware, nobreakmodel, and css fix by @ebolam in #366
• Reverting the RTE by @LightSaveUs in #368
• Fix for --nobreakmodel forcing CPU by @ebolam in #369
• fix: tpu support models saved in bfloat16 by @Zurnaz in #370
• updated kobold lite to v37 by @LostRuins in #371
• Making model backends respond to a specific type in the aiserver menu for now by @ebolam in #373
• Updated Kobold Lite to v41 by @LostRuins in #376
• Fix for model backends that use toggles always returning true by @ebolam in #377
• Fix WI for UI1 by @jojorne in #378
• Fix move and delete WI item for UI1 by @jojorne in #382
• Accelerate offloading by @one-some in #385
• Use VE's patched load_from_state_dict on TPU for loading empty weights by @one-some in #386
• Updated Kobold Lite to v46 by @LostRuins in #387
• Stub seek_offset for cache sorting in load loop by @one-some in #389
• Fix bleeding edge model loads and add lazyload fallback by @one-some in #390
• Fix for UI2 model loading not showing progress by @ebolam in #392
• Basic HF backend by @one-some in #391
• Fixes for model backend UI by @ebolam in #397
• Speeds up bnb 4bit with a custom BitsAndBytesConfig by @Alephrin in #398
• Update to the upload_file function by @ebolam in #399
• missed the elif by @ebolam in #400
• Patches: Make lazyload work with quantization by @one-some in #402
• Delete basic 4bit by @one-some in #404
• Save the 4-bit flag to the model settings. by @ebolam in #401
• Clarified message on what's required for model backend parameters by @ebolam in #406

New Contributors

• @Alephrin made their first contribution in #398

Full Changelog: Snapshot-7-5-2023...Snapshot-18-07-23