Alignment is too strong for story writing with transformers or llama.cpp

[Upcycle-Electronics]

I'm still not positive that text-generation-webui is the root cause. Writing a science fiction story involving a different societal structure and AGI was going great from around October of last year until the first week of April. I updated text-generation-webui around the beginning of January and everything was going okay. Fedora updated the Nvidia driver and that in turn forced me to update text-generation-webui, Ever since, I have not been able to make use of the same Mixtral 8x7b gguf (flat dolphin maid merge on hugging face). I thought it might have been due to PR 6920 from llama.cpp changing the BPE special tokens, or maybe PR 6387 where the MoE models had the memory layers merged, but I have since had the same alignment issues with every supposedly uncensored model I have used. They all have an amplified bias that prevents them from doing any kind of creative writing in my own unique ways, like a post scarcity society without money currency, but with other complex forms of hierarchic display and social systems where heat is budgeted, and all life within an O'Neill cylinder is living within an elemental cycles budget, with no waste. I have not found another model that can cope with ideas like an era after the age of scientific discovery where science is seen as an engineering corpus, or a world where all human scale technology is biological and grown. It seems impossible to get a model to accept and understand the implications of a complete understanding of biology, the human brain, the potential for bio-compute, and therefore biological based AGI. And all of this brings me to where I can hopefully make a point; my story is about the human integration and cohabitation with a biological AGI species of human basically made in a lab, growing up with humans, and then later joining a central collective to form a government. I have fleshed out characters, a tech tree, and plot line, with the history mapped out 10 colonies around Type G stars within 7 parsecs of Earth, no FTL, no aliens, no authoritarianism, and no exceptionalism. So the entire thing is in stark contrast to the present reality. I developed all of this in collaboration with the aforementioned 8x7b model. It was doing a fantastic job of exploring the story with me simply using the Notes tab of text-generation-webui. Whatever changes happened, every model I have tested from Eric Hartman's dolphin variants to most of the popular stuff on hugging face, I have not found anything around an 8x7b to 70b 3-6bit quantization like I need for this kind of story comprehension and abstractions. Even a smaller transformers based models or 8b Llama 3 or one of the unusual models like moistral have been unable to break free of this alignment bias I'm experiencing at the present. A lot of my writing was enabled by the 32k context of a 8x7b that had some issues with the GGUF configuration, namely, (that I know of), the wrong instruction format in the metadata. I would greatly appreciate any help or advice here. I'm well aware that creative writing is a pointless endeavor, as it is statistically more likely to become a billionaire than make a living from creative writing. However, I'm physically disabled and this was a high point over the last decade for me. I'd like to play with it some more.

I'm probably poking in the wrong corner blindly but would changes to modules/sampler_hijack.py have caused this behavior? It is quite subtle to spot the issues on the surface at first glance. They only crop up when I get into the details creatively or try and load an old conversation to resume. I can tell there have been improvements in overall technical accuracy, but the tendency to fall into some structured gutter-like responses is considerably stronger and there is no reasoning flexibility; the model stands its ground about things like how AGI is always an existential threat on par with the machine gods mythos trope from Hollywood. Like Daneel is always a T1000 in disguise if you place them in complex story arcs. Any help is greatly appreciated.

[Upcycle-Electronics]

I'm not sure what is going on, but I tried to install and use snapshot-2024-01-14-text-generation-webui, and I'm getting great results again with generation. I did not attempt to adjust the requirements or the update script to stick with the older dependencies and just let it install like normal. Now everything seems fine with this version. My oldest version that I had modified and kept up to date since around this time last year until April of this year still won't generate after loading a model. The version that I installed in April and have updated a few times has bad alignment and generates badly even when I have not modified it at all.

I was surprised when I did a diff comparison of sampler_hijack.py thinking I was going to try and figure out why the generation seemed better, but the files are identical between the one that was most up to date but generating badly and snapshot-2024-01-14-text-generation-webui. I'm not sure what got updated and what did not with this older version, or if I would have had the same results if I had tried to install the latest release. The only notable difference that I recall seeing was that I was asked about my GPU and after selecting Nvidia, I was asked if I would like to use the 11.8 (IIRC/11.x for sure) version instead of the 12.5 version I am presently running with Fedora. When I was looking at the snapshot initially I had noted the differences in this snapshot's sampler_hijack.py and the one from the latest releases, but apparently this was updated despite the snapshot install. Now I have no clue what is really making the generation so much different between these two versions, even when I am using symbolic links to the exact same models in the models directory and the same softmax settings.

I have no idea if any of this is really significant and it seems like posting here is a rather mute point, but for whatever reason, this seems to have solved the issue for me so far. If anyone is experiencing similar issues, I would recommend doing a full removal and reinstall. Maybe try to install an older snapshot and see what happens.