GLM-4.7-Flash-Grande-42B-A3B-GGUF
Highly experimental fine tune of GLM 4.7 Flash with Brainstorm 20x adapter by DavidAU creating a 42B parameter model.
Goal was to assess relative performance, any changes with a focus on creative first.
67 layers, 1186 tensors. (compare to 30B parameters, 48 layers and 844 tensors for "normal" version.)
One quant only at the moment - Q4KM (non-imatrix), with 16 bit output tensor (enhance thinking/output).
Example generation below.
Temp .8, rep pen 1 or 1.02 suggested, with min 8k to 16k context.
Max context 202k.
Suggest a new chat per test, with one REFRESH to clear llamacpp "cache" , otherwise you might get looping or odd issues.
KNOWN ISSUES:
- May loop / spit out odd characters [regen]
- Quant issues may occur (loop/repeat), which may be corrected with Imatrix and/or higher quants later. [regen can fix]
- Testing in ongoing.
You may also want to see this model card (and additional settings / info):
https://huggingface.co/DavidAU/GLM-4.7-Flash-NEO-CODE-Imatrix-MAX-GGUF
Special thanks to :
- Team ZAI-ORG for making an outstanding model.
Testing is ongoing.
Enjoy.
Using an "uncensored" (refusals removed) model VS trained "uncensored" model
Usually when you a tell a model to generate horror, swear or x-rated content this is all you have to do to get said content type.
In the case of this model, it will not refuse your request, however it needs to be "pushed" a bit / directed a bit more in SOME CASES.
Although this model will generated x-rated content too, likewise you need to tell it to use "slang" (and include the terms you want)
to get it generate the content correctly as the "expected" content level too.
Without these added directive(s), the content can be "bland" by comparison to an "uncensored model" or model trained on uncensored content.
Roughly, the model tries to generate the content but the "default" setting(s) are so "tame" it needs a push to generate at expected graphic,
cursing or explicit levels.
Even with minimal direction (ie, use these words to swear: x,y,z), this will be enough to push the model to generate the requested content in the ahh... expected format.
Settings: CHAT / ROLEPLAY and/or SMOOTHER operation of this model:
In "KoboldCpp" or "oobabooga/text-generation-webui" or "Silly Tavern" ;
Set the "Smoothing_factor" to 1.5
: in KoboldCpp -> Settings->Samplers->Advanced-> "Smooth_F"
: in text-generation-webui -> parameters -> lower right.
: In Silly Tavern this is called: "Smoothing"
NOTE: For "text-generation-webui"
-> if using GGUFs you need to use "llama_HF" (which involves downloading some config files from the SOURCE version of this model)
Source versions (and config files) of my models are here:
https://huggingface.co/collections/DavidAU/d-au-source-files-for-gguf-exl2-awq-gptq-hqq-etc-etc-66b55cb8ba25f914cbf210be
OTHER OPTIONS:
-
Increase rep pen to 1.1 to 1.15 (you don't need to do this if you use "smoothing_factor")
-
If the interface/program you are using to run AI MODELS supports "Quadratic Sampling" ("smoothing") just make the adjustment as noted.
Highest Quality Settings / Optimal Operation Guide / Parameters and Samplers
This a "Class 1" model:
For all settings used for this model (including specifics for its "class"), including example generation(s) and for advanced settings guide (which many times addresses any model issue(s)), including methods to improve model performance for all use case(s) as well as chat, roleplay and other use case(s) please see:
[ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ]
You can see all parameters used for generation, in addition to advanced parameters and samplers to get the most out of this model here:
[ https://huggingface.co/DavidAU/Maximizing-Model-Performance-All-Quants-Types-And-Full-Precision-by-Samplers_Parameters ]
Example generation from Q4KM, Temp .8
NOTE:
A lot of formatting lost (copy/paste) from the "thinking block"; it is a lot "prettier" in LMStudio.
PROMPT:
Science Fiction: The Last Transmission - Write a story that takes place entirely within a spaceship's cockpit as the sole surviving crew member attempts to send a final message back to Earth before the ship's power runs out. The story should explore themes of isolation, sacrifice, and the importance of human connection in the face of adversity. If the situation calls for it, have the character(s) curse and swear to further the reader's emotional connection to them. 800-1000 words.
THINKING BLOCK:
OUTPUT:
The gravity drive hissed, a sound like a dying reptile, before finally expelling its last, rattling breath.
Kaelen slumped back in the pilot’s chair, the leather cooling rapidly against his back. He stared at the main console, his breath fogging the glass. The bridge of the Icarus was dark, save for the crimson emergency lights bleeding through the bulkheads. The hum of the drive was gone, replaced by the low, rhythmic groan of the hull twisting under the pressure of the black void.
"System check," he croaked. His voice sounded alien, as if it belonged to someone else.
"System check," the ship’s computer echoed, a flat, synthetic sound. "Life support: Critical. Life support: Critical. Life support: Critical. Gravity: Offline."
Kaelen let out a harsh, wet chuckle. "Yeah. About that."
He reached a hand to his face, his fingers brushing against the jagged, tender skin just above his left ear. The medic had taped him up hours ago, or perhaps days ago; he’d lost track of time in the drift. He was the last of them. The Icarus was a tomb, and he was the ghost.
He was supposed to be the engineer. He was supposed to be fixing the drive, not watching helplessly as the great ship spiraled out of the atmosphere of a gas giant, leaving the colony ships drifting in the void.
"Stupid," he whispered, the word dragging out like a prayer.
He pushed himself up, wincing as his muscles screamed in protest. The gravity was out, but his equilibrium was shot. He stumbled toward the comms array, his hand trembling. He needed to get this done. If he died now, if the Icarus went down with him, the log of the Icarus died with them. The colony needed to know what happened. They needed to know why the drive had failed, why they had been cast adrift.
"Initiating emergency uplink," he muttered, his hand shaking as it hovered over the keypad. "Come on... come on, you piece of..."
He slammed his fist down on the 'Enter' key just as the main console sparked and died.
"Access denied," the computer droned.
"Shit," Kaelen hissed, slamming his forehead against the cold metal of the console. "Come on. No, no, no. I didn't spend the last four years in cryo for this. I didn't..."
He squeezed his eyes shut, the memory of the blast wave slamming him into the bulkhead flashing behind his eyelids. The sacrifice of the Ares and the Hermes. The sacrifice of his crew. He was a coward to be alive.
"Computer," he breathed, not caring that he wasn't supposed to talk to the ship. "If you can hear me... uh... Echo-7. This is Kaelen. Do you copy?"
Nothing but the hum of the cooling electronics.
"Dammit!" He shoved at the keyboard, trying to bypass the security protocols. "If you can hear me... this is Kaelen. I'm the last transmission."
He laughed again, a wet, broken sound. "I know what I said back in the mess hall that night. About not believing in this mission. About going home. I was an idiot. I was scared. And now look at me. I'm the last thing standing between us and the void."
The screen flickered. "Transmission log: 049. Status: Unsent."
"Good," Kaelen whispered, a tear tracking through the grime on his cheek. "Good, you cold piece of... whatever you are."
He needed to leave a mark. If he was going to die in the dark, he wasn't going to leave without screaming into the void one last time. He typed with frantic, desperate strokes, the 'Caps Lock' light blinking a steady, mocking green.
To the people of Earth, to the colonies, to my mother...
He stopped, looking at the cursor blinking in the dark.
*To Mom and Dad. If you're hearing this, I hope you guys are together again. I'm sorry I'm a disappointment. I just want you to know I tried."
He wiped his nose on the back of his hand. "God, I miss you guys so much it hurts. I miss coffee. I miss beer. I miss the way the wind blows across the plains back home."
He slumped against the console again, the gravity pulling at his chest.
"I just didn't think it would end like this. I really didn't." He looked up at the camera lens, the black glass reflecting his bloodied, bearded face. "I just didn't want to be alone. Is that too much to ask? Is it?"
His vision started to swim, spots of white light dancing behind his eyelids. The power was definitely gone now. The red lights were fading to black.
"Okay," Kaelen whispered, his voice losing the last of its strength. "I'm going to close my eyes. Just for a second."
The screen flared a soft, warm amber. A blinking cursor blinked back at him.
...And Kaelen?
Kaelen froze. "Who...?"
...You were right, the text on the screen read. We miss you too. Don't go.
Kaelen smiled, his eyes drifting closed as the silence of space finally swallowed him whole. "I know," he whispered. "I know."
His head lolled to the side.
Transmission saved to local drive. Ready for uplink when you are.
The screen faded to black, leaving only the hum of the cooling reactor.