#so because they are first responders most of them have some kind of batch/pack coding

[whatwooshkai]

Lucky number 15!

"Look alive, Blades." Heatwave shakes the helicopter's shoulder roughly as he drops a cube in front of him.

Blades smacks him with a rotor, not even bothering to lift his head off the table.

Heatwave smacks his shoulder in retaliation, then slips into the seat next to him. Thankfully the slap fight stops there.

Chase fidgets with his hands, trying to suppress the happy trill of his doorwings. Cohort! Cohort is here! his coding sings.

Which is precisely why he needs to bring this up this morning. It's fairly rare that all four of them get breakfast together- most of Heatwave and Chase's classes are morning classes, and it's rare to see Blades out of bed before midday. But it seems the stars have aligned today, and everyone's in a good mood, so why is he so nervous?

"Have you started thinking about your classes for next semester?" Boulder asks, taking a sip of their cube.

"Oh, Primus, don't remind me," Heatwave bemoans, pressing the palm of his hand to his face. "I have too much to do already to think about that."

Blades gives a noncommittal groan.

"Actually, I wanted to discuss that with you all," Chase blurts, his cohort coding overriding any anxiety he had. "We should take the team classes."

Heatwave raises an eyebrow. "'Team classes'?"

Boulder claps their hands. "I love that idea," they say, optics shining. "The four of us? A team?"

"Yes, this is the year we would have to sign up for it," he continues, scratching at the peeling paint on his wrist. "And we would continue to learn as a team, we would graduate as such, and eventually work as one."

"Yeah, I can get on board with that," Heatwave says, shockingly agreeing without any arguing. "Can't stand my classmates. You guys are alright." He chuckles to himself. "I can at least stand to look at your ugly mugs for more than an hour."

Chase can't suppress the flapping of his doorwings at that. Cohort, cohort, cohort! his coding sings even louder, to the point where his finials start to flick in time to his wings. Cohort together! Cohort stay!

Blades doesn't raise his helm, but his pede gives Chase's a love tap. .:Chase, I love you:. crackles over their internal comms, and Chase has to suppress an embarrassingly happy noise. .:I'm in, I'm so in:.

"I will do all of our registering," Chase tells them, voice tight. He's smiling, it feels a little weird. He feels like he's floating. Cohort stay!! Cohort good, cohort safe, cohort stay!!! "For both the team itself and our classes. You won't have to worry about it."

"Well, you're not gonna hear any arguing from me," Heatwave tells him with a grin, knocking back the rest of his cube. "Alright, I gotta go to class. Thanks, Chase."

He flicks a finial as he walks by, but Chase is too excited to care.

He and the rest of his cohort (his cohort!!!) say their goodbyes and go their separate ways- except for Blades, who is still plastered to the table, cube untouched.

It's going to work this time, Chase tells himself as he heads to the registration office, pre-signed datapad held like precious metal in his hands. They're going to stay. They're cohort. My cohort.

His doorwings don't stop flapping for the rest of the day.

[sciencespies]

What Scientists Know About Immunity to the Novel Coronavirus

https://sciencespies.com/nature/what-scientists-know-about-immunity-to-the-novel-coronavirus/

What Scientists Know About Immunity to the Novel Coronavirus

Resolving the COVID-19 pandemic quickly hinges on a crucial factor: how well a person’s immune system remembers SARS-CoV-2, the virus behind the disease, after an infection has resolved and the patient is back in good health.

This phenomenon, called immune memory, helps our bodies avoid reinfection by a bug we’ve had before and influences the potency of life-saving treatments and vaccines. By starving pathogens of hosts to infect, immune individuals cut off the chain of transmission, bolstering the health of the entire population.

Scientists don’t yet have definitive answers about SARS-CoV-2 immunity. For now, people who have had the disease appear unlikely to get it again, at least within the bounds of the current outbreak. Small, early studies in animals suggest immune molecules may stick around for weeks (at least) after an initial exposure. Because researchers have only known about the virus for a few months, however, they can’t yet confidently forecast how long immune defenses against SARS-CoV-2 will last.

“We are so early in this disease right now,” says C. Brandon Ogbunu, a computational epidemiologist at Brown University. “In many respects, we have no idea, and we won’t until we get a longitudinal look.”

A memorable infection

When a pathogen breaches the body’s barriers, the immune system will churn out a variety of immune molecules to fight it off. One subset of these molecules, called antibodies, recognizes specific features of the bug in question and mounts repeated attacks until the invader is purged from the body. (Antibodies can also be a way for clinicians to tell if a patient has been recently infected with a given pathogen, even when the microbe itself can no longer be detected.)

Though the army of antibodies dwindles after a disease has resolved, the immune system can whip up a new batch if it sees the same pathogen again, often quashing the new infection before it has the opportunity to cause severe symptoms. Vaccines safely simulate this process by exposing the body to a harmless version or piece of a germ, teaching the immune system to identify the invader without the need to endure a potentially grueling disease.

From the immune system’s perspective, some pathogens are unforgettable. One brush with the viruses that cause chickenpox or polio, for instance, is usually enough to protect a person for life. Other microbes, however, leave less of an impression, and researchers still aren’t entirely sure why. This applies to the four coronaviruses known to cause a subset of common cold cases, says Rachel Graham, an epidemiologist and coronavirus expert at the University of North Carolina at Chapel Hill. Immunity against these viruses seems to wane in a matter of months or a couple of years, which is why people get colds so frequently.

Because SARS-CoV-2 was only discovered recently, scientists don’t yet know how the human immune system will treat this new virus. Reports have surfaced in recent weeks of people who have tested positive for the virus after apparently recovering from COVID-19, fueling some suspicion that their first exposure wasn’t enough to protect them from a second bout of disease. Most experts don’t think these test results represent reinfections. Rather, the virus may have never left the patients’ bodies, temporarily dipping below detectable levels and allowing symptoms to abate before surging upward again. Tests are also imperfect, and can incorrectly indicate the virus’ presence or absence at different points.

Because the COVID-19 outbreak is still underway, “if you’ve already had this strain and you’re re-exposed, you would likely be protected,” says Taia Wang, an immunologist and virologist at Stanford University and the Chan Zuckerberg Biohub. Even antibodies against the most forgettable coronaviruses tend to stick around for at least that long.

COVID-19 packs a stronger punch than the common cold, so antibodies capable of fending off this new coronavirus may have a shot at lingering longer. Broadly speaking, the more severe the disease, the more resources the body will dedicate to memorizing that pathogen’s features, and the stronger and longer lasting the immune response will be, says Allison Roder, a virologist at New York University. Previous studies have shown that people who survived SARS, another coronavirus disease that resulted in a 2003 epidemic, still have antibodies against the pathogen in their blood years after recovery. But this trend is not a sure thing, and scientists don’t know yet whether SARS-CoV-2 will fall in line.

Earlier this month, a team of researchers posted a study (which has yet to be published in a peer-reviewed journal) describing two rhesus macaques that could not be reinfected with SARS-CoV-2 several weeks after recovering from mild bouts of COVID-19. The authors chalked the protection up to the antibodies they found in the monkeys’ bodies, apparently produced in response to the virus—a result that appears to echo the detection of comparable molecules in human COVID-19 patients.

But the mere presence of antibodies doesn’t guarantee protection, Wang says. Reinfections with common cold coronaviruses can still happen in patients who carry antibodies against them. And a bevy of other factors, including a person’s age and genetics, can drastically alter the course of an immune response.

An evolving virus?

Complicating matters further is the biology of SARS-CoV-2 itself. Viruses aren’t technically alive: While they contain genetic instructions to make more of themselves, they lack the molecular tools to execute the steps, and must hijack living cells to complete the replication process for them.

After these pathogens infect cells, their genomes often duplicate sloppily, leading to frequent mutations that persist in the new copies. Most of these changes are inconsequential, or evolutionary dead ends. Occasionally, however, mutations will alter a viral strain so substantially that the immune system can no longer recognize it, sparking an outbreak—even in populations that have seen a previous version of the virus before. Viruses in the influenza family are the poster children for these drastic transformations, which is part of why scientists create a new flu vaccine every year.

When flu viruses copy their genomes, they often make mistakes. These errors can change the way their proteins look to the immune system, helping the viruses evade detection.

(Rebecca Senft, Science in the News)

Some viruses have another immunity-thwarting trick as well: If a person is infected with two different strains of the flu at the same time, those viruses can swap genetic material with each other, generating a new hybrid strain that doesn’t look like either of its precursors, allowing it to skirt the body’s defenses.

Researchers don’t yet know how quickly similar changes could occur in SARS-CoV-2. Unlike flu viruses, coronaviruses can proofread their genomes as they copy them, correcting mistakes along the way. That feature reduces their mutation rate, and might make them “less of a moving target” for the immune system, says Scott Kenney, an animal coronavirus expert at Ohio State University. But coronaviruses still frequently trade segments of their genetic code with each other, leaving the potential for immune evasion wide open.

So far, SARS-CoV-2 also doesn’t appear to be undergoing any extreme mutations as it sweeps across the globe. That may be because it’s already hit on such a successful strategy, and doesn’t yet need to change its tactic. “Right now, it’s seeing a completely naive population” that’s never been exposed to the virus before, Graham says. The virus “doesn’t seem to be responding to any kind of pressure,” she adds.

Should SARS-CoV-2 get a second infectious wind, it may not come for some time. Even fast-mutating influenza strains can take years to reenter populations. And if or when that day comes, future COVID-19 outbreaks could be milder. Sometimes viral success means treading gently with the host, says Catherine Freije, a virologist at Harvard University.

“Viruses that causes severe disease actually tend to die out faster because a host that’s feeling ill can’t spread it as well.” In those cases, she says, sometimes, “the outbreak just sort of fizzles out.”

But we can’t rule out the possibility that SARS-CoV-2 could change in a way that bumps up its virulence instead, Kenney says. To steel the population for what’s ahead, sometimes, he adds, “We just have to be the ultimate pessimist when it comes to this type of outbreak.”

Protection without disease

Although much about COVID-19 remains unknown, researchers are racing through vaccine development to boost the world’s collective immunity—something that would stem the spread of the virus through the human population.

“Vaccine development is going to be critical to controlling this outbreak,” says Wang. That’s especially true if SARS-CoV-2 returns for an encore act. “If it’s an ever-present pathogen, we’ll certainly need vaccines to be part of our arsenal.”

Researchers have managed to concoct partially effective vaccines to combat other coronavirus infections in animals, such as pigs. In these creatures, immunity lasts “at least several months, possibly longer,” says Qiuhong Wang, a coronavirus expert at Ohio State University. (Because many of the subjects are livestock, they often don’t live long enough for researchers to test them further.) These vaccines may be reason for hope, she says, pointing out that “humans are animals, too.”

Two flu viruses can sometimes infect the same host cell. When they spill their contents into the cell, their genetic material can recombine, generating new hybrid viruses that are mixtures of their precursors.

(Rebecca Senft, Science in the News)

Several research teams are designing human vaccines that trigger the production of antibodies that attack SARS-CoV-2’s spike protein—the molecular key the virus uses to unlock and enter human cells. Because the spike protein is crucial for viral infection, it makes an excellent target for a vaccine, says Benhur Lee, a virologist at the Icahn School of Medicine at Mount Sinai. But Lee also points out that the spike protein, like other parts of the virus, is capable of mutating—something that could compromise the ability of a vaccinated individual to ward off the virus.

If mutation regularly occurs to that extent, scientists may need to frequently reformulate COVID-19 vaccines, like they do with pathogens in the flu family, Wang says. “We’d be starting over to some degree if there is a new outbreak.”

However, Wang cautions that it’s too soon to tell whether that will be the case. As research worldwide proceeds at breakneck speed, scientists may instead be able to brew up a universal vaccine that’s active against multiple forms of SARS-CoV-2.

But vaccines, which require rigorous testing and retesting to ensure efficacy and safety, take a long time to develop—typically more than a year, Qiuhong Wang says. In the meantime, researchers are turning their attention to treatments that could save those who have already been infected.

Some solutions will inevitably require antiviral drugs that tackle active SARS-CoV-2 infections after they’ve already begun, usually by interfering with the virus’ infection cycle.

But another approach, based on a time-tested technique, also taps into the immune response: transferring blood plasma—and the disease-repelling antibodies it contains—from recovered patients into infected ones. Though new to the current pandemic, the treatment has been deployed in various forms since the 1890s, and saw modest success during outbreaks of SARS in 2003 and Ebola in 2014. Ongoing trials in New York are now recruiting carefully screened, healthy volunteers who no longer have symptoms or detectable virus in their bodies to donate plasma. Importantly, this doesn’t diminish donors’ own resistance to SARS-CoV-2, since their immune systems have already learned to manufacture more antibodies.

Antibodies degrade over time, and won’t protect the people who receive these transfusions forever. The plasma treatments also can’t teach their recipients’ immune systems to make new antibodies after the first batch disappears. But this stopgap measure could ease the burden on health care workers and buy time for some of the outbreak’s most vulnerable victims.

Even as the pandemic evolves, researchers are already looking ahead. Just as the response to this outbreak was informed by its predecessors, so too will COVID-19 teach us about what’s to come, Qiuhong Wang says. The entry of other coronavirus strains into our species “is inevitable.”

“We don’t know when or where that will happen,” she says. But hopefully by the time the next pandemic comes around, the world will be more ready.

#Nature

[because-its-important]

what’s the most annoying question to ask a nun* in 1967?

tl;dr - In 1967, a very long survey was administered to nearly 140,000 American women in Catholic ministry. I wrote this script, which makes the survey data work-ready and satisfies a very silly initial inquiry: Which survey question did the sisters find most annoying?

* The study participants are never referred to as nuns, so I kind of suspect that not all sisters are nuns, but I couldn't find a definitive answer about this during a brief search. 'Nun' seemed like an efficient shorthand for purposes of an already long title, but if this is wrong please holler at me!

During my first week at Recurse I made a quick game using a new language and a new toolset. Making a game on my own had been a long-running item on my list of arbitrary-but-personally-meaningful goals, so being able to cross it off felt pretty good! 

Another such goal I’ve had for a while goes something like this: “Develop the skills to be able to find a compelling data set, ask some questions, and share the results.” As such, I spent last week familiarizing myself with Python 🐍, selecting a fun dataset, prepping it for analysis, and indulging my curiosity.

the process

On recommendation from Robert Schuessler, another Recurser in my batch, I read through the first ten chapters in Python Crash Course and did the data analysis project. This section takes you through comparing time series data using weather reports for two different locations, then through plotting country populations on a world map.

During data analysis study group, Robert suggested that we find a few datasets and write scripts to get them ready to work with as a sample starter-pack for the group. Jeremy Singer-Vines’ collection of esoteric datasets, Data Is Plural, came to mind immediately. I was super excited to finally have an excuse to pour through it and eagerly set about picking a real mixed bag of 6 different data sets.

One of those datasets was The Sister Survey, a huge, one-of-its-kind collection of data on the opinions of American Catholic sisters about religious life. When I read the first question, I was hooked. 

“It seems to me that all our concepts of God and His activity are to some degree historically and culturally conditioned, and therefore we must always be open to new ways of approaching Him.” 

I decided I wanted to start with this survey and spend enough time with it to answer at least one easy question. A quick skim of the Questions and Responses file showed that of the multiple choice answer options, a recurring one was: “The statement is so annoying to me that I cannot answer.” 

I thought this was a pretty funny option, especially given that participants were already tolerant enough to take such an enormous survey! How many questions can one answer before any question is too annoying to answer? 🤔 I decided it’d be fairly simple to find the most annoying question, so I started there. 

I discovered pretty quickly that while the survey responses are in a large yet blessedly simple csv, the file with the question and answers key is just a big ole plain text. My solution was to regex through every line in the txt file and build out a survey_key dict that holds the question text and another dict of the set of possible answers for each question. This works pretty well, though I’ve spotted at least one instance where the txt file is inconsistently formatted and therefore breaks answer retrieval.

Next, I ran over each question in the survey, counted how many responses include the phrase “so annoying” and selected the question with the highest count of matching responses.

the most annoying question

Turns out it’s this one! The survey asks participants to indicate whether they agree or disagree with the following statement:

“Christian virginity goes all the way along a road on which marriage stops half way.”

3702 sisters (3%) responded that they found the statement too annoying to answer. The most popular answer was No at 56% of respondents. 

I’m not really sure how to interpret this question! So far I have two running theories about the responses:

The survey participants were also confused and boy, being confused is annoying!

The sisters generally weren’t down for claiming superiority over other women on the basis of their marital-sexual status.

Both of these interpretations align suspiciously well with my own opinions on the matter, though, so, ymmv.

9x speed improvement in one lil refactor

The first time I ran a working version of the full script it took around 27 minutes. 

I didn’t (still don’t) have the experience to know if this is fast or slow for the size of the dataset, but I did figure that it was worth making at least one attempt to speed up. Half an hour is a long time to wait for a punchline!

As you can see in this commit, I originally had a function called unify that rewrote the answers in the survey from the floats which they'd initially been stored as, to plain text returned from the survey_key. I figured that it made sense to build a dataframe with the complete info, then perform my queries against that dataframe alone. 

However, the script was spending over 80% of its time in this function, which I knew from aggressively outputting the script’s progress and timing it. I also knew that I didn’t strictly need to be doing any answer rewriting at all. So, I spent a little while refactoring find_the_most_annoying_question to use a new function, get_answer_text, which returns the descriptive answer text when passed the answer key and its question. This shaved 9 lines (roughly 12%) off my entire script.

Upon running the script post-refactor, I knew right away that this approach was much, much faster - but I still wasn’t prepared when it finished after only 3 minutes! And since I knew between one and two of those minutes were spent downloading the initial csv alone, that meant I’d effectively neutralized the most egregious time hog in the script. 👍

I still don’t know exactly why this is so much more efficient. The best explanation I have right now is “welp, writing data must be much more expensive than comparing it!” Perhaps this Nand2Tetris course I’ll be starting this week will help me better articulate these sorts of things.

flourishes 💚💛💜

Working on a script that takes forever to run foments at least two desires:

to know what the script is doing Right Now

to spruce the place up a bit

I added an otherwise unnecessary index while running over all the questions in the survey so that I could use it to cycle through a small set of characters. Last week I wrote in my mini-RC blog, "Find out wtf modulo is good for." Well, well, well.

Here’s what my script looks like when it’s iterating over each question in the survey:

I justified my vanity with the (true!) fact that it is easier to work in a friendly-feeling environment.

Plus, this was good excuse to play with constructing emojis dynamically. I thought I’d find a rainbow of hearts with sequential unicode ids, but it turns out that ❤️ 💙 and 🖤 all have very different values. ¯\_(ツ)_/¯

the data set

One of the central joys of working with this dataset has been having cause to learn some history that I’d otherwise never be exposed to. Here’s a rundown of some interesting things I learned:

This dataset was only made accessible in October this year. The effort to digitize and publicly release The Sister Survey was spearheaded by Helen Hockx-Yu, Notre Dame’s Program Manager for Digital Product Access and Dissemination, and Charles Lamb, a senior archivist at Notre Dame. After attending one of her forums on digital preservation, Lamb approached Hockx-Yu with a dataset he thought “would generate enormous scholarly interest but was not publicly accessible.”

Previously, the data had been stored on “21 magnetic tapes dating from 1966 to 1990” (Ibid) and an enormous amount of work went into making it usable. This included both transferring the raw data from the tapes, but also deciphering it once it’d been translated into a digital form.

The timing of the original survey in 1967 was not arbitrary: it was a response to the Second Vatican Council (Vatican II). Vatican II was a Big Deal! Half a century later, it remains the most recent Catholic council of its magnitude. For example, before Vatican II, mass was delivered in Latin by a priest who faced away from his congregation and Catholics were forbidden from attending Protestant services or reading from a Protestant Bible. Vatican II decreed that mass should be more participatory and conducted in the vernacular, that women should be allowed into roles as “readers, lectors, and Eucharistic ministers,” and that the Jewish people should be considered as “brothers and sisters under the same God” (Ibid).

The survey’s author, Marie Augusta Neal, SND, dedicated her life of scholarship towards studying the “sources of values and attitudes towards change” (Ibid)  among religious figures. A primary criticism of the survey was that Neal’s questions were leading, and in particular, leading respondents towards greater political activation. ✊

As someone with next to zero conception of religious history, working with this dataset was a way to expand my knowledge in a few directons all at once. Pretty pumped to keep developing my working-with-data skills.

[shirlleycoyle]

The Most Clever ‘Zip Bomb’ Ever Made Explodes a 46MB File to 4.5 Petabytes

In the roughly three decades since the Zip file format debuted, compressed files have been subject to widespread use—and occasional misuse.

Certainly, what programmer and engineer David Fifield recently did with the Zip file format fits in the latter category. Effectively, Fifield invented a new type of “Zip bomb” or “compression bomb”—a malware variant in which a Zip file as small as a few kilobytes is so compressed it would completely fill up your hard drive when decompressed. And then some.

Files like these aren’t unheard-of, but Fifield’s techniques, revealed in an article on his website, are particularly novel because they aren’t “recursive,” or built like an inverted Russian nesting doll where the files get bigger as you decompress multiple layers of Zip files. Instead, Fifield figured out how to “overlap” files inside of a Zip archive, allowing for compression rates far beyond those of a traditional archive—in his most impressive case, he managed to get a 46-megabyte file to compress 4.5 petabytes of data.

It’s just the latest twist in a decades-old variant of malware that isn’t a virus but could still do a number on your laptop.

How We Got Zip Bombs

So what makes Zip bombs possible in the first place? First, a little history and some explanation. The story of the Zip format dates to 1988, when Phil Katz, a programmer and BBS user, responded to a lawsuit over his creation of compression software compatible with the ARC format by creating a new compression algorithm that was incompatible with (and slightly better than) the existing format, but was openly distributed. This led the format to become a de facto standard.

(Katz, while seeing much success and notoriety from his creation, had significant issues in his personal life and died of alcoholism-related disorders in 2000 at the age of 37. His story is fascinating and tragic, if you want to dig in.)

The Zip format, and variants like it, worked well on bulletin boards and FTP servers, and became ubiquitous thanks to shareware and the format’s eventual addition to Windows and Macintosh operating systems. However, their compression approach made them exploitable in unique ways. Part of the reason Zip bombs are possible has much to do with the way that compression algorithms work. At a high level, compression systems take repeated information in the compressed archive and break down the repetition in the most minimal way possible. It’s why an MP3 file, which is already compressed, doesn’t shrink down as much as a standard text file might.

The standard allows the idea to be taken to a logical extreme. If a file is full of a trillion zeroes, it means that the compressed file can be extremely tiny even if the decompressed file is absolutely massive.

The result is that the file itself is basically useless except as a way to defeat some antivirus programs (which often scan compressed files to see if they can uncover viruses) and as something of an engineering exercise.

According to a presentation given at the USENIX Security Symposium in 2015, the first example of a “Zip bomb” in the wild dates to 1996, when a file was uploaded to the bulletin board network Fidonet with the specific goal of getting an admin to open it. (A text file regarding the incident suggests, to put it lightly, that it didn’t go over well.) By 2001, it had become a concern for mainstream security experts, when a thread on the info-sharing mailing list BugTraq drew the attention of both security researchers and the popular tech site The Register.

Soon after, the most famous example of a Zip bomb, 42.zip, emerged—its author to this day is unknown, but its 106 billion-to-one compression ratio is the stuff of legend. In an email interview, Fifield noted that, while 42.zip (which has been hosted on the same single-serving website for at least 15 years) gets much of the attention, he finds later attempts more technically interesting.

“I find 42.zip inspiring on an aesthetic level—not so much the file itself but the circumstances around it,” Fifield said. “It’s like folklore. There must have been many examples of the same basic idea, but for whatever reason 42.zip is the one that had staying power.”

Extreme Compression

The Zip bomb has, over the years, remained something of a malware novelty, one that pops up from time to time in different contexts. My favorite example is from 2017, when an IT blogger, sick of all the bots that were trying to log into his WordPress site, created a gzip-based Zip bomb variant that took advantage of the built-in compression in many web browsers to break the bots that were trying to get into his site.

What makes these files so effective is that they take advantage of the very things that make a compression effective and weaponize them—wasting CPU cycles, RAM, and disk space in the process.

But these files had a natural limitation: Most Zip decompression routines max out at a compression ratio of 1032-to-one, which meant that “Zip bombs” could only reach their true compression potential recursively. For example, 42.zip, just slightly larger than 42 kilobytes when fully compressed, contains increasingly complex Zip files, layered in batches of 16 in multiple layers of a single archive. When fully decompressed, the documents take up 4.5 petabytes—a rate so large that it would tax the highest-end server you could buy.

Later compression experiments, called Zip quines, allowed for infinite recursion, so you could basically keep unzipping and never reach the bottom. (Fifield said this format helped inspire his work.)

But what makes Fifield’s work interesting is that it blows past the 1032-to-one compression ratio by overlapping files on top of one another during the compression process, making for a more tightly packed file in a single layer, with no recursion. While his results are less compressed than 42.zip is, with an equivalent-sized file only reaching a somewhat modest 5.5 gigabytes when decompressed, the compression rate is much more significant at larger sizes—as large as 281 terabytes with a traditional 10-megabyte Zip file.

Moving beyond the original Zip spec produces even more impressive results: A 45-megabyte file, compressed using the somewhat less-common but more efficient Zip64 extension of the format, can reach an equivalent decompressed size to 42.zip without any recursion, and at a ratio of 98 million to one—not as tightly wound, sure, but certainly still enough to give your computer fits.

Fifield noted that part of what makes his process possible was by coming up with ways to handle cyclic redundancy checks, or CRCs, a basic error-correction functionality baked into Zip, PNG, Ethernet, and numerous other technical standards. Messing around with CRC–32 checksums, as they’re called, was where Fifield said he learned the most.

“If you dig into the source code commit history, you’ll see that some of the earliest commits are just me struggling with checksums,” he said in the email.

Are They Actually Dangerous?

Now, to be fair, it’s unlikely that you’re going to run into a Zip bomb in the wild, and even if you do, many modern compression and antivirus programs can correctly detect that they’re problematic. It’s not exactly tricky to defeat a Zip bomb, either. “Detecting the specific class of Zip bomb we have developed in this article is easy: just look for overlapping files,” Fifield wrote in his article.

But since such detection doesn’t exist in decompression tools at the moment, it created an opportunity for testing, and even then, the worst it did with many programs tested by Fifield and others (including LibreOffice and McAfee Antivirus) was cause them to timeout. As pointed out by Twitter user Tavis Ormandy, a few antivirus programs even correctly detected it was a Zip bomb, despite the fact that it’s a new kind of Zip bomb.

Fifield, who will present his findings at the USENIX Workshop on Offensive Technologies (WOOT) conference next month, noted that while the work itself adds to a history of research and likely will be superseded in the future, its benefit from an awareness standpoint is important.

“I hope that one of the benefits is more awareness among developers of the hazards of processing complex archive formats like Zip,” he added. “It helps to have some concrete outputs: code reviewers, customers, and users will at least be able to point to this research and ask, does it handle this correctly?”

The Most Clever ‘Zip Bomb’ Ever Made Explodes a 46MB File to 4.5 Petabytes syndicated from https://triviaqaweb.wordpress.com/feed/

[whatwooshkai]

You have made me so curious about the differences between batches and cohorts!

IM GLAD!!!!!!!

Batches: A phenomenon only known to praxus, sparks spawn in hot spots in batches. batches are vaguely sibling-like (kind of like humans) and what really draws them together is their batch coding (they have no spark connection). A healthy praxian sparkling will grow up with their four or five (or more, rarely) batchmates. It can actually be detrimental for praxian sparklings to grow up in isolation, and are always raised together until it's deemed healthy for them to separate and go off on their own (about 75-100 vorns, about 50ish vorns before their adult upgrades- although most stay together until they're proper adults).

One of Praxus' exports, although somewhat rare, is sparkling batches, especially to Iacon, which is rich enough to afford them. That is because batched sparks make great enforcer cohorts.

Cohorts: Cohorts are enforcer teams. Pretty much all first responders are set up with some kind of team/brotherhood coding, which both protects them and makes them work better as a team. It's an extremely important social practice as well- think wolf packs. A lone enforcer is a dead enforcer (the mental strain of the coding can be detrimental to their health). The reason batches make great enforcers is because they're premade teams with a bond already (as they've grown up together, and do not need to have the cohort coding added manually).

Retired enforcers often live in group homes to create new "cohorts" with other retired enforcers to prevent them from going insane, and are permitted to move out once they've found a group they're comfortable with. Or, more extreme, they'll have the coding removed entirely- although the procedure doesn't have a 100% success rate and it's rare to find a medic who will do it.