#Chunking

[belindafleurs]

I made this comic earlier... enjoy

[bharatpatel1061]

Context Windows and Chunking: Managing Long Inputs for Agents

Agents powered by language models often face input size limits. Context windows—how much input the model can “see” at once—restrict long conversations or documents. This is where chunking and memory strategies come in.

Solutions include:

Recursive summarization

Context window rolling (sliding buffers)

External memory like vector stores

Managing long contexts is essential in legal, research, and technical workflows. Learn how modern AI agents optimize context use for long-form tasks.

Preprocess large inputs into semantically coherent chunks—it improves accuracy and avoids hallucinations.

[savagegardensprogramming]

Chunking and how is it relevant in programming languages.

"If it is said that a picture is worth a thousand words can it not be said that a visual code construct can be worth a thousand lines of code"

That is one of my catch phrases that I use to promote the idea of visual or graphical programming languages. Unfortunately there is a flaw in that catch phrase. Its the reason why writing with pictures didn't take off.

The reality is that words, more specifically vocabulary, are quite effective. The reality is that a single vocabulary word is also capable of being worth a thousand words by itself. That is because words are used to define words. In a sense abstraction.

The reason writing in pictures didn't take off is because it is a lot easier to write a word then it is to draw a picture and yet a single word can be just as powerful.

Anyways this post is about importing the ideas behind chunking into programming languages. Chunking is just another angle of looking at abstraction. The divide and conquer paradigm. Its all related.

I came across chunking while watching a video called "Learning to Learn in 25 mins" by Tina Huang. And what she doing is summarizing is a whole Coursera course.

To summarize chunking. We start out with an alphabet of sounds. We express them as letters. Then we "chunk" them letters together into words. We do it again, we "chunk" them words into sentences. We do it again and "chunk" them sentences into conceptual meaning. Then we go recursive. We "chunk" that conceptual meaning into a single word. And we repeat again and again. Hence a single term like biology is worth a thousand other words. I say the word biology is worth millions and millions of words. In fact you might not be able to draw a picture that encompasses the word biology as a whole in all its glory.

Anyways, We do this in computer programming also. If you studied recursive descent. Its pretty clear. I've included a pic of Apple Pascal's Syntax Diagram. A computer program can be represented by a tree structure (A post for another day). Syntax diagrams are the rules that govern how those trees are created. Syntax diagrams is basically chunking at play. If you don't see it visually you can see it textually. Syntax diagrams is just a visual representation of Backus-Naur Form. If you take a look at Backus-Naur Form definition of a language you will see the chunking.

Everyday, ordinary computer programmers also do chunking. The idea of a function call is chunking. A function is essentially a bunch of code "chunked" together into a single identifier. Chunking is nature's manifestation of the Divide and Conquer paradigm.

[jimmoore82]

Ashkan Rajaee says if they have to think, you’ve already lost them. He’s not wrong.

https://discover.hubpages.com/business/ashkan-rajaee-the-brutal-truth-about-how-top-sellers-manipulate-without-you-knowing

[pierresmith]

A Closer Look at Ashkan Rajaee’s Simplified Sales Strategy

https://discover.hubpages.com/business/ashkan-rajaee-the-brutal-truth-about-how-top-sellers-manipulate-without-you-knowing

Tired of ghosted leads? Ashkan Rajaee says make it easy to reply. Short asks. One-word responses. It works.

[blackwoolncrown]

I get it, I really do, but I feel like ppl w decision paralysis related to Gazans are also suffering from a kind of non-malignant egotism

"who do I decide to give it to? everyone needs help! I can't help them all!"

You're not the only person they're asking!!!!!

If you choose one person to help that day, a bunch of other ppl choose others

stopping to overthink your inflated importance as an individual versus one in many is what's giving you paralysis

the more you burden yourself as an incapable hero the less you all act as a group-- which is the exact answer you're looking for

fkn close your eyes, scroll on OOB and donate to whoever you click on first

you're not god, it's not your job to save everyone

JUST DO SOMETHING

[raffaellopalandri]

Organizing Knowledge: My Mental Map for Learning

I usually do not spend much time chatting about pointless or purposeless topics, both because of my nature and because of the Buddhist precept of abstaining from idle chatter. Photo by Miguel u00c1. Padriu00f1u00e1n on Pexels.com Also, I notice that when I speak in a social gathering and the topic involves knowledge and learning, people find my style quite different from theirs, if not somewhat…

[el-candelabro]

La Revolución de la Memoria a Corto Plazo: De George A. Miller al Procesamiento de Información

[pangur-and-grim]

I'm turning 30 this month, and for some reason have become suddenly interested in material possessions. like what if,,,,,,,,my couch was nice. what if my sheets were nice. is this what happens to you??

[ladyfingerpress]

z_xxxxvi Brain Kludge (fullpage scan)

[jojaxcola]

“Where does all the time go? I remember when Sam was a little baby wearing diapers. He was such a cute little kid. Don't tell him I said that."

[katmaibearfan]

This is a photo of 32 Chunk that Ranger Tammy took YESTERDAY. as in MAY 18TH 2025.

32 Chunk is starting spring fatter than many bears end up being in late fall! He's already a Fat Bear Champion contender and the cams haven't even turned on yet! holy crap!

as KatmaiConservancyNaomi said when she posted the photo on explore: WHAT WILL 32 CHUNK LOOK LIKE IN OCTOBER?!

[unraveling-plot]

The Murderbot Diaries was like here's a person. The person doesn't feel like a person.

The person is starting to believe it's a person, and its pronouns are still it/its. It's uncomfortable with eye contact, it hates being touched, it doesn't like social interaction, it doesn't have great control of it's facial expressions. And the person's friends all know this and respect it, and it can show affection how and when it feels and it's ok if it isn't "normal." It isn't questioned or doubted in its care because of this.

The person processes emotions through media, and uses it to figure out what to do in social situations. It has complicated feelings about what it was meant to be and reconciling that with who it wants to be. It doesn't always feel like a person but it's starting to accept that it is one.

Also it has laser weapons in its arms.

[dat-soldier]

very pickupable

[aerequets]

Nature reminds us that out there with survival at stake, our neatly-drawn boundaries can become a little blurred.