#textgen

[thedebugdiary]

Deploying Text Generation Web UI on a Kubernetes Cluster

In this blog post, we'll walk through the process of deploying a text generation web UI using the Docker image atinoda/text-generation-webui on a Kubernetes cluster. We'll cover everything from creating a Persistent Volume Claim (PVC) to setting up a Kubernetes Service for port forwarding.

Prerequisites

A running Kubernetes cluster

kubectl installed and configured to interact with your cluster

Step 1: Create a Namespace

First, let's create a namespace called text-gen-demo to isolate our resources.

kubectl create namespace text-gen-demo

Step 2: Create a Persistent Volume Claim (PVC)

We'll need a PVC to store our data. Create a YAML file named text-gen-demo-pvc.yaml with the following content:

apiVersion: v1 kind: PersistentVolumeClaim metadata: name: text-gen-demo-pvc namespace: text-gen-demo spec: accessModes: - ReadWriteOnce resources: requests: storage: 100Gi storageClassName: standard

Apply the PVC:

kubectl apply -f text-gen-demo-pvc.yaml

Step 3: Deploy the Pod

Create a YAML file named text-gen-demo-pod.yaml with the following content:

apiVersion: v1 kind: Pod metadata: name: text-gen-demo-pod namespace: text-gen-demo labels: app: text-gen-demo spec: containers: - name: text-gen-demo-container image: atinoda/text-generation-webui ports: - containerPort: 7860 - containerPort: 5000 - containerPort: 5005 env: - name: TORCH_CUDA_ARCH_LIST value: "7.5" volumeMounts: - name: text-gen-demo-pvc mountPath: /app/loras subPath: loras - name: text-gen-demo-pvc mountPath: /app/models subPath: models volumes: - name: text-gen-demo-pvc persistentVolumeClaim: claimName: text-gen-demo-pvc

Apply the Pod:

kubectl apply -f text-gen-demo-pod.yaml

Step 4: Create a Service

Create a YAML file named text-gen-demo-service.yaml with the following content:

apiVersion: v1 kind: Service metadata: name: text-gen-demo-service namespace: text-gen-demo spec: selector: app: text-gen-demo ports: - name: "webui" protocol: TCP port: 7860 targetPort: 7860 - name: "api" protocol: TCP port: 5000 targetPort: 5000 - name: "api-stream" protocol: TCP port: 5005 targetPort: 5005

Apply the Service:

kubectl apply -f text-gen-demo-service.yaml

Step 5: Port Forwarding

Finally, let's set up port forwarding to access the web UI locally.

kubectl port-forward svc/text-gen-demo-service 7860:7860 -n text-gen-demo

You should now be able to access the web UI at http://localhost:7860.

Troubleshooting

If you encounter issues with port forwarding, make sure:

The pod is running and healthy (kubectl get pods -n text-gen-demo)

The service is correctly configured (kubectl describe svc text-gen-demo-service -n text-gen-demo)

The service has endpoints (kubectl get endpoints text-gen-demo-service -n text-gen-demo)

Conclusion

You've successfully deployed a text generation web UI on a Kubernetes cluster! You can now interact with the web UI locally and generate text as needed.

[eyegens]

I love how you draw all the cookies, two things; have you drawn mystic flour yet? do you do commissions by chance?

• omg thank you!! ;w; <3 i did draw her once for practice, but unfortunately no full pieces. i like giving her big soulless eyes

• i do! here are my prices and some examples of what i can do <3

[edupunkn00b]

Meus ex Machina, Chapter 18: Stepping Back

Edited public domain image of two hands reaching for each other, lit in deep blue and neon green.

Prev - Stepping Back - Next - Masterpost - [ AO3 ]

WC: 2525 - Rated: T - CW: non graphic nudity, suggestive, swearing. A major disinformation campaign and some not-so-subtle political manipulation.

New travels fast after Logan's machine makes its debut. After each of his machine's debuts, really.

“XHT-0731, you’re cleared for landing,” the comm squawked. “Proceed to platform seven-oh.” Eyes closed, Lucas let out a slow breath then pressed to talk.

“Affirmed, proceeding to platform seven-oh.” His hands shook as he switched off the mic and he forced his grip to relax long enough for the tremor to pass. Nearly there, then he could rest. He opened his eyes and focused on the beacon lights. The dash was awash in orange glow from his powers as he fought to control his nerves.

Thirty-eight hours ago, Lucas had been hunkered down a dozen miles south of the Honshu fires when a news report of a giant blue Kiryu mech attacking corporate facilities in the States hit his automated feed. He’d groaned and buried himself deeper into his sleep sack, hoping to tune it out and grab a bit of sleep when Jan’s DC number read in Japanese snagged his attention.

He set up a crawler to find more information and hit the air. Less than two hours later, the preliminary news reports of an accidental gas release had morphed into a suspected Powered terrorist attack, a world-wide conspiracy to expose the anti-theft measures at Abracadabra shipping facilities. Frantic talking heads—human and otherwise—chattered about this abhorrent abuse of power and betrayal of the secret knowledge granted their Powereds employees—

“More like fucking lapdogs,” he’d muttered, turning up the feed.

“… and what precisely does the Chancellor plan to do now about the rising risk of Powereds who’ve gotten too comfortable with the freedoms granted them in the last Act?”

Fuck.

The ship wasn’t far from the Siberian archipelago, so he landed and procured a new transport, one with a Russian transponder. While rising seas and toxic air had changed a lot of things in the world, some old prejudices never failed to make certain events ‘news’worthy.

He fed the latest reports into a textgen and translated the results back and forth between English, Russian, and Greek a few times before sending it out in a couple of worms to all the major news feeds.

The AI bots gobbled that shit up.

He hopped ships again and took off over the Arctic oil fields on a more direct route back to the East Coast. By the time he was back in range a couple hours later, the first trickles of outrage around the disinformation bots scheming to discredit Powerds had begun to fill the Atlantic and Euro news feeds. It turned out the gas leak had simply been a terrible, horrible, and completely unforeseeable accident and the mysterious Powereds who’d converged on the site to evacuate the Traditional workers had saved thousands of lives. 

A few progressive channels were even calling for commendations for the Powereds at the site.

Lucas paused the vidfeed, Ro’s bright red suit making him easy to pick out of the crowd. Pat’s baby blue unitard was covered in so much dust it matched the giant grey gates some reprogrammed picker bot was yanking off their hinges.

Squinting at the grainy footage, he zoomed in. There was a man inside the bot, wearing it like a suit. And Pat… 

Pat was guarding his back.

“Huh,” he said to the empty ship. “Looks like Jan finally recruited somebody new.” He unpaused.

The zoomed-in feed lingered on Jan’s face for one long moment before it cut away to another talking head extolling the glory of Corporate-Powered partnerships. Lucas switched it off. He pushed the engines and navigated to the nearest landing platform. One more hop to a new ship then he could head back to his cachette in the States.

The skids had just touched down when Re’s scream ripped through his mind. His head slammed forward, bouncing first against the controls and then the deck as he fell from his chair. Distantly, he registered the wet trickle of blood running sideways down his face as he reached out.

Re wasn’t hard to find. He was afraid, overwhelmed and hurt, but… purposeful. This was no accident.

Blinding orange leaking past eyes squeezed shut, Lucas curled on the floor. He hugged his knees tight to his chest, and held his breath, steeling himself.

Then he opened himself to the turmoil in Re’s mind.

It had been a long time since Jan had last dropped his shield, a long time since he’d last recruited Re to find someone in need. Who the fuck were they looking for now? And what made them so important they should hurt Re like this?

There wasn’t much Lucas could do from a distance, but he poured what little calm he had out into the world, smiling past his tears when Re picked up a bit of it before he was suddenly cut off again.

And just like that it was over.

Lucas lay panting, eyes slowly dimming as he scrabbled together his own composure. By the time he sat up, cold, sticky blood clung to the side of his head, a small, drying pool of it left on the deck. He washed himself and then the ship, spraying a thick layer of disinfectant and running the Steri-san on deep clean to eradicate any trace of his DNA. 

The sterilizer system’s whine made for good cover as he crept out onto the mostly deserted landing pad to find a fresh ship for the final leg of his trip.

~

-“It worked.”-

Janus’ eyes snapped open, his room filled with the silence that followed a rung bell. He blinked in the darkness, the last stubborn tendrils of his dream sticky and fighting to drag him back down into sleep. He touched his face and his hand came away wet.

The house was quiet and calm, not even the usual dreams and nightmares of four other sleeping minds woven through the air.

Five. There are five other souls in this house. Five other minds here…

Five, he nodded to no-one. Five when you count—

Janus sat up, breath caught in his throat as he reached out to each of them. Virge’s mind relaxed—for once—in a dreamless sleep. The Prince’s gleaming castles he didn’t even need dreams to build in his mind. Pat’s cotton candy sweetness wrapped around a permanent, prickly worry. Logan’s unshielded thoughts screaming out, want and need and hope and pain and—

Re…

-“I told you,”- Gentle laughter poured into Janus’s head. -“I like it when you call me ‘Muse.’”

-“Muse,”- he repeated, mouth forming the word his name dancing in his mind. “How?”-

-“I told you, Jannie…”- The Muse’s thoughts were quiet, happy. Peaceful and maybe even a little spacey. A hand drew through his hair as he drifted off to sleep. He wasn’t alone in his room.

-“It worked.”- The Muse sighed, warm and comfortable and calmer than Janus could remember him being in a long time. Flashes of his night poured into Janus’ mind and he pulled back.

Mentally, he kept his distance, not throwing up a shield between them, but giving The Muse a bit of space, a bit of privacy for his thoughts and perceptions.

-“‘Night, Jannie,”- The Muse sent, a small marigold sparking to life on the pillow next to him. So bright, it almost glowed in the dark, soft orange petals lush under his fingertips. -“Will I see you in the morning?”-

-“Yes! Good night, dear Muse,”- he sent back, smiling when The Muse’s thoughts settled into sleep. “See you in the morning.” Reaching slowly, half expecting the flower to dissolve at his touch now that The Muse was asleep, he plucked up the stem, brought the blossom to his nose and inhaled. It smelled just like the flowers Patton grew in the cool house out back. Even in sleep, The Muse’s control was flawless.

He lay there a while, waiting for the flower to disappear. It never did. The rest of the house was asleep, flitting in and out of dreams. But even The Muse’s dreams were peaceful and quiet, a hushed buzz in the air.

When Machina had come to him with his plan, Janus had been afraid to hope too much. He’d permitted the experiments as long as Ro wasn’t too worse for the wear and remained a willing partner in the tests. Subject might be a better word. But he’d never actually expected the device to work. It was such a careful balance, blocking just enough of the world from him—and blocking enough of The Muse from the world. Over the years, they’d tried more times than he could count to adjust the power of his room’s shield over the years, had tried to lower it.

But each time the world poured in, it had been too much. It had always been too much.

As he bobbed in the ebb and flow of The Muse’s dreams, Janus couldn’t deny Machina’s device had indeed worked. He’d been wrong. All these years, he’d been wrong.

Had he given up too soon? Had they truly been just months from a breakthrough, just months away from some magic machine that could give The Muse enough control to be free?

What would life have been like then? If he hadn’t given up? If The Muse was just Re, having dinner with them, bickering with Ro over who got the bigger cookie or who’s turn it was to fly the transport.

All those times The Muse had asked about Luc, eyes swimming in tears, asking him to explain again why Luc had left.

What else had he been wrong about?

Janus smelled the flower again, petals tickling his nose. Ro was sleeping better, he noticed, the white noise of his brother’s mental presence a tonic he hadn’t had in a very long time. Janus relished his own relief, too, an empty ache suddenly filled, like putting on a ring that had been lost.

Janus stared down at his left hand. It’s likely he and Ro weren’t the only ones who’d felt the return of The Muse’s presence. Still carefully cradling the flower, Janus pushed back the covers and climbed out of bed.

He had a visit to make.

~

-“The sun’s been up for hours, Sleeping Beauty,”- Re’s imagined voice needled him and Roman rolled over, head buried under the pillow.

Re could just fuck right off. Machina’s eyes had that gleam in them when the latest prototype had passed its first tests, muttering about how they almost had it and he needed some time for just one more adjustment.

The same gleam that invariably meant the next day he’d push it too far and Roman would end up with another migraine. Or worse.

So yes, Roman was going to get his beauty sleep while he could, thank you very much! He burrowed back under the covers and willed the return of sleep.

-”Come on, Ro Bro! Get up!”-

Roman’s eyes snapped open and he flipped over in the bed, listening. It wasn’t yet dawn and Virge shifted next to him. “Y’okay, Princey?” he mumbled, reaching blindly for his arm.

“Ye-yes?” he said, that old hole in his chest suddenly… filled. The eerie silence in his mind, the quiet corner he’d spent every waking moment over the past few years fighting to ignore was… no longer silent.

His mind hummed, music and phrases and snippets of stories twirling together in a happy pas de deux. Half-awake, his own thoughts swirled, a semi-conscious call and response of ancient lyrics, dialogue from movies. Old arguments, minor and serious.

“Do…” Roman rolled to his side and faced Virge. His eyes held the same confused wonder. “Do you hear that, too?”

Virge closed his eyes, brow furrowed. Then he nodded. “Maybe… maybe not everything you’re getting, but…”

-”Re?”- Virge managed actual words while Ro explored nearly a decade’s worth of ideas and memories. -”Re, how are you doing this?”-

-”Why don’t you come find out…”- A memory flashed across his mind, stifled too quickly for Roman to process. -”Just give me a few minutes, yeah?”-

Virge had already pulled on a hoodie and opened the door. “You coming, Princey?”

~

The elevator doors were just closing when they arrived. Virge’s hand bolted out faster than he could see and they slid back open, revealing Papa Bear, pulling on gloves.  Roman’s heart sank down to his toes when he saw the big med kit at his feet. “Padre, what’s going on? Is Janus with him?”

“I don’t know,” he shook his head, brows pinched in a little worried frown. “Jan messaged me in my room and he said he was getting ready and to go ahead. He…” He gave a little shrug, his tiny smile forced. “If it helps, he didn’t seem worried… just… Distracted?”

The elevator doors slid open on the lower level. The hall was silent.

“Distracted like he was talking with Re?” Virge asked, following them out of the elevator. They all stopped when they saw Re’s closed door, the shield’s warning light dark.

“Oh, fuck!” Patton muttered and ran to the door.

“Padre, wait!” Roman called after him but Virge beat them all and flung open the door.

“Re! Re, we’re here! We’re here, we’ll fix—Oh!” Virge froze just inside the doorway then looked down at the floor. “Oh, um, we—”

“Kiddo, what’s—Oh…” Papa Bear covered his mouth, the backs of his ears tinged pink.

Roman pushed past each of them. “What in the holy hell are you two—”

Re lay in bed with the blanket pulled up to his waist, pajamas in a heap on the floor. He held a single finger to his lips, glaring at them. -”I said to wait a bit! He’s sleeping!”-

Curled against Re’s side, equally bare chested, lay Machina.

-”Not anymore.”- Amplified by Re, Machina’s thought spilled out to all of them and Roman’s nerves boiled over into a laugh.

~

The Prince’s laughter was the final proof that the only way past this was through it. Logan pushed to sit up in the bed but only succeeded in knocking more of the blanket off himself. “So I suppose everyone knows we…” He pulled the covers up to his chin and tried to ignore his own heated cheeks. Remus smiled back at him, a wordless coo filling his mind before he smacked his forehead.

“Oh fuck, your pants!” he exclaimed with a laugh and ran across the room, heedless of his own nakedness.

“Kiddo!” Patton shook his head, a mix of laughter and admonishment in his voice as Remus gathered the clothes scattered around his room.

Shaking with laughter, Remus passed Logan his pants before scooping up his own discarded pajama bottoms. He looked around the gathering and grinned. “Oh, come on. Raise your hand if you haven’t seen me naked yet.”

Given Remus’ utter lack of shame, Logan was unsurprised to see no-one’s hand go up. However, his and Remus’ jaws dropped at the same moment.

“Wait, where’s—”

The hallway beyond Remus’ room burst into strobing yellow lights. 

-”Jannie.”-

Logan had wiggled into his pants just before Patton reached for him, eyebrows raised in the obvious question. He nodded, trading his pride for a speedier trip back to his chair and his mech upstairs. The movement broke Remus from his reverie and he followed the team upstairs.

[shadow-monster]

when you have no idea what's happening to your game:

anyway, i have no idea what's going on with my Skyrim world, i know i use DynDOLOD and TextGEN, and it was working fine with me for sometime. but after downloading a handful of mods, my game is like this. i don't have any idea what mods are doing this to me, or anything like this. i just know that i'm expreinceing a thing called "Black LOD' the conversation i saw seemed to help a little bit, but that was mainly to identify the issue. i'm kind of an idiot on understanding all that tech modding stuff. so, any help would be greatly appreciative!

[lemrent]

Please return to traditional art. Please.

I never left. I've got a lot of projects going on at any given time. I'm working on Good Omens jewelry for metal 3D printing since I like modeling and have been meaning to learn how to convert that to print. Textgen AI is definitely taking the lion's share of my time at the moment, though.

[teachingwritingatslu]

agenda, 10.19

today

multimodal project work day

compass lab visit

for next time

read & annotate: Laquintano, Schnitzler & Vee, “An Introduction to Teaching with Text Generation”

read & annotate: Any two chapters from TextGenEd

read & annotate: Rivers, “Sophistic Parrots and the Problematic of Machine Intelligence”

[therobotmonster]

I have to imagine that if you were "under the hood" you could tinker with that.

While I'm generally skeptical of the overall utility of textgen for a lot of reasons, I've never seen one that gave access to any of its parameters the way a lot of image-gen ones do.

A higher "chaos" setting would mean more "creative" (novel) responses, but also less accurate ones, both in terms to adherence to the prompt and adherence to reality. Everyone making textgen seems to want it to function like an encyclopedia and a search engine. If you're intent on pretending that 'hallucination' is a bug and not literally what the system was designed to do, then you're somewhat disincentivized to give users that access.

On the other hand, Chat-GPT's fakemon names are basic as hell, and I'm not sure how much a chaos setting would change that.

Personally, I'd love a simple system that let me just load a spreadsheet of text and create a GAN from it (I am a coding dunderhead, I'm sure this is easy as sin) because those kinds of "we've fed every action figure name from the past 30 years into this list" type stuff produced way more interesting raw material to sift for diamonds in.

Google Docs provides language-model driven autocorrect, where it highlights "unlikely" strings of words and offers to replace them for you.

It makes a lot of sense. But it's also a bit ironic, given that statistical language models started with Shannon's information theory, which identifies "information" with surprisal. Like, you could imagine if you just keep right-clicking the text, eventually Google will rewrite it for you into something carrying zero bits of information.

That's actually what LLM-written texts try to optimize for. Which I believe you can notice!

There is no "idea" in the poem except what is forced on it by the prompt; the text generation algorithm specifically tries to put as few bits as possible into the text. The more clichéd the poem is, the closer it is to optimal.

Sometimes humans fall into the same trap. I keep thinking about the editor who changed "a feeling of jealousy" into "a pang of jealously", replacing the phrase by a cliché and deliberately erasing the bits of information contained in it.

[aiweirdness]

Neural network-named tomatoes you won’t find at the farmer’s market

As gardeners know, a tomato is not just a tomato. They come in all shapes, sizes, and especially colors, from tiny red gems to big and purplish brown to pear-shaped with zebra stripes. And their tastes and textures vary wildly as well, with some sweet and soft, some tough-skinned and tart, and some dense and destined for processing.

These varieties all have names, of course, and good names are at a premium. Darren Abbey, who runs the blog The Biologist Is In, breeds new tomatoes and needs to name them from time to time. He gave me a list of 11,629 existing names, ranging from the highly experimental (Rts-119) to the classic (Sungold).

I used these names to train a recurrent neural network, which is a kind of algorithm that learns patterns that let it imitate the data it sees. Could it learn to produce plausible tomato names?

Yes, as it turns out! On Darren’s blog post you can read his expert opinion, but the verdict is: these are totally usable names.

Floranta Sweet Lightning Speckled Boy Flavelle Pinkery Plum Market Days Fancy Bell Mountain Gem Garden Sunrise Honey Basket Cold Brandy Sun Heart Flaminga Sunberry Special Baby Golden Pow

However, not ALL the names were plausible. These are a lot weirder.

Birdabee Sandwoot Shampy Bear Plum The Bango Grannywine Sun Burger Bungersine First No.4 Smoll Pineapple The Ball Golden Cherry Striped Rock Eggs Old German Baby Frankster Black Bumbertime Ranny Blue Ribber Adoly Pepp Of The Wonder Cherry, End Students Small Of The Elf Champ German Ponder Pearly Pemper Green Zebra Pleaser Flute First

These *seem* weird, but they aren’t actually all that weird for tomato names. There are at least 50 oxheart tomatoes in the original dataset.

Matt's Sandwich Indigo Tree Sun Burger Striped Hollow Potato Leaf Lelly's Yellow Stuffers Terra Pink Strain Greek Boar Ton's Oxheart Babla's German Paste Mortgage Lifter, Honey Blues

These however? These tomatoes don’t sound promising.

Speckled Garfech Green Dork Cluster Gall Shirve's Gigant Bullburk True Grub Black Crape Caraball Roma Wasting Star Scar Giant Bug Beauty Banana Placente Bananana Stoner Speckled Bake Ruck Green Boor Wonder Bagg Sun Bung Bellende Bolosavyi Scarling West Colon Panterosa San Blue Tard Red Krap Shart Delight Solad Piss

For more in-depth analysis of these names, check out Darren’s blog.

I also trained a neural net on fruit names earlier - read the results here.

And this post’s bonus content is well worth checking out, because not only does it contain many more tomato names, but I also tried an experiment where I trained a neural net first on tomato names, and then on the names of metal bands. The results were glorious, including such wonders as:

Angels of Saucing Serpent Green Market Death Sweet Sever Devil Potato Large Death

Become a supporter of AI Weirdness to get the rest of these as bonus content!

[thisisfakenews]

Former President Donald Trump's New Ice Cream Brand "Trump Man Milk" Caught Donating To Palestinian Warlords

Washington D.C. - Reports have surfaced that President Donald Trump's new ice cream brand, Trump Man Milk, has been caught donating money to Palestinian warlords.

This is not the first time that the Trump Organization has been caught donating to questionable causes. In the past, the company has been linked to donations to white supremacists and the Russian government.

When reached for comment, a spokesperson for the Trump Organization said that the company is just trying to do its part to help rebuild the Middle East. "Trump Man Milk is committed to helping rebuild the Middle East," said the spokesperson. "We believe that by donating money to Palestinian warlords, we can help create a more peaceful and prosperous region."

Critics have slammed the Trump Organization for its donations, saying that they are nothing more than a publicity stunt.

"The Trump Organization is only donating to Palestinian warlords because they know it will get them some good publicity," said one critic.

"They don't actually care about helping to rebuild the Middle East."

[eyegens]

onion

hate. let me tell you how much i've come to hate you since i began to live

[generatedtrump]

Will be interviewed by on at 9:00 P.M. (a death has ever seeking also) Hollywood and the word "BRAIN SUCKEIES BIGE " Huge Army Balls

Trump, via RNN

[andersonloco]

Vegetarians and vegans of Reddit, what science every time?

AskReddit post generated by neural network

[yhancik]

https://www.researchgate.net/publication/330855103_Curating_Simulated_Storyworlds

[14floods]

Do you also go by Joel or??

joel is my first name! you can call me whatever you like. if this is wrt archivist bot i wasn’t sure i was totally comfortable w complete strangers on my textgen blog calling me mac...