#backdoor applications

[harrykim]

open endings can be so fun and interesting its a real shame how half the time it's actually just someone not knowing how to write a damn ending

[jcmarchi]

Unmasking Privacy Backdoors: How Pretrained Models Can Steal Your Data and What You Can Do About It

New Post has been published on https://thedigitalinsider.com/unmasking-privacy-backdoors-how-pretrained-models-can-steal-your-data-and-what-you-can-do-about-it/

Unmasking Privacy Backdoors: How Pretrained Models Can Steal Your Data and What You Can Do About It

In an era where AI drives everything from virtual assistants to personalized recommendations, pretrained models have become integral to many applications. The ability to share and fine-tune these models has transformed AI development, enabling rapid prototyping, fostering collaborative innovation, and making advanced technology more accessible to everyone. Platforms like Hugging Face now host nearly 500,000 models from companies, researchers, and users, supporting this extensive sharing and refinement. However, as this trend grows, it brings new security challenges, particularly in the form of supply chain attacks. Understanding these risks is crucial to ensuring that the technology we depend on continues to serve us safely and responsibly. In this article, we will explore the rising threat of supply chain attacks known as privacy backdoors.

Navigating the AI Development Supply Chain

In this article, we use the term “AI development supply chain” to describe the whole process of developing, distributing, and using AI models. This includes several phases, such as:

Pretrained Model Development: A pretrained model is an AI model initially trained on a large, diverse dataset. It serves as a foundation for new tasks by being fine-tuned with specific, smaller datasets. The process begins with collecting and preparing raw data, which is then cleaned and organized for training. Once the data is ready, the model is trained on it. This phase requires significant computational power and expertise to ensure the model effectively learns from the data.

Model Sharing and Distribution: Once pretrained, the models are often shared on platforms like Hugging Face, where others can download and use them. This sharing can include the raw model, fine-tuned versions, or even model weights and architectures.

Fine-Tuning and Adaptation: To develop an AI application, users typically download a pretrained model and then fine-tune it using their specific datasets. This task involves retraining the model on a smaller, task-specific dataset to improve its effectiveness for a targeted task.

Deployment: In the last phase, the models are deployed in real-world applications, where they are used in various systems and services.

Understanding Supply Chain Attacks in AI

A supply chain attack is a type of cyberattack where criminals exploit weaker points in a supply chain to breach a more secure organization. Instead of attacking the company directly, attackers compromise a third-party vendor or service provider that the company depends on. This often gives them access to the company’s data, systems, or infrastructure with less resistance. These attacks are particularly damaging because they exploit trusted relationships, making them harder to spot and defend against.

In the context of AI, a supply chain attack involves any malicious interference at vulnerable points like model sharing, distribution, fine-tuning, and deployment. As models are shared or distributed, the risk of tampering increases, with attackers potentially embedding harmful code or creating backdoors. During fine-tuning, integrating proprietary data can introduce new vulnerabilities, impacting the model’s reliability. Finally, at deployment, attackers might target the environment where the model is implemented, potentially altering its behavior or extracting sensitive information. These attacks represent significant risks throughout the AI development supply chain and can be particularly difficult to detect.

Privacy Backdoors

Privacy backdoors are a form of AI supply chain attack where hidden vulnerabilities are embedded within AI models, allowing unauthorized access to sensitive data or the model’s internal workings. Unlike traditional backdoors that cause AI models to misclassify inputs, privacy backdoors lead to the leakage of private data. These backdoors can be introduced at various stages of the AI supply chain, but they are often embedded in pre-trained models because of the ease of sharing and the common practice of fine-tuning. Once a privacy backdoor is in place, it can be exploited to secretly collect sensitive information processed by the AI model, such as user data, proprietary algorithms, or other confidential details. This type of breach is especially dangerous because it can go undetected for long periods, compromising privacy and security without the knowledge of the affected organization or its users.

Privacy Backdoors for Stealing Data: In this kind of backdoor attack, a malicious pretrained model provider changes the model’s weights to compromise the privacy of any data used during future fine-tuning. By embedding a backdoor during the model’s initial training, the attacker sets up “data traps” that quietly capture specific data points during fine-tuning. When users fine-tune the model with their sensitive data, this information gets stored within the model’s parameters. Later on, the attacker can use certain inputs to trigger the release of this trapped data, allowing them to access the private information embedded in the fine-tuned model’s weights. This method lets the attacker extract sensitive data without raising any red flags.

Privacy Backdoors for Model Poisoning: In this type of attack, a pre-trained model is targeted to enable a membership inference attack, where the attacker aims to alter the membership status of certain inputs. This can be done through a poisoning technique that increases the loss on these targeted data points. By corrupting these points, they can be excluded from the fine-tuning process, causing the model to show a higher loss on them during testing. As the model fine-tunes, it strengthens its memory of the data points it was trained on, while gradually forgetting those that were poisoned, leading to noticeable differences in loss. The attack is executed by training the pre-trained model with a mix of clean and poisoned data, with the goal of manipulating losses to highlight discrepancies between included and excluded data points.

Preventing Privacy Backdoor and Supply Chain Attacks

Some of key measures to prevent privacy backdoors and supply chain attacks are as follows:

Source Authenticity and Integrity: Always download pre-trained models from reputable sources, such as well-established platforms and organizations with strict security policies. Additionally, implement cryptographic checks, like verifying hashes, to confirm that the model has not been tampered with during distribution.

Regular Audits and Differential Testing: Regularly audit both the code and models, paying close attention to any unusual or unauthorized changes. Additionally, perform differential testing by comparing the performance and behavior of the downloaded model against a known clean version to identify any discrepancies that may signal a backdoor.

Model Monitoring and Logging: Implement real-time monitoring systems to track the model’s behavior post-deployment. Anomalous behavior can indicate the activation of a backdoor. Maintain detailed logs of all model inputs, outputs, and interactions. These logs can be crucial for forensic analysis if a backdoor is suspected.

Regular Model Updates: Regularly re-train models with updated data and security patches to reduce the risk of latent backdoors being exploited.

The Bottom Line

As AI becomes more embedded in our daily lives, protecting the AI development supply chain is crucial. Pre-trained models, while making AI more accessible and versatile, also introduce potential risks, including supply chain attacks and privacy backdoors. These vulnerabilities can expose sensitive data and the overall integrity of AI systems. To mitigate these risks, it’s important to verify the sources of pre-trained models, conduct regular audits, monitor model behavior, and keep models up-to-date. Staying alert and taking these preventive measures can help ensure that the AI technologies we use remain secure and reliable.

[rodspurethoughts]

Beware: Pirated macOS Applications Conceal a Dangerous Backdoor

In a recent discovery by Jamf Threat Labs researchers Ferdous Saljooki and Jaron Bradley, a disturbing trend has emerged targeting Apple macOS users. Pirated applications, primarily hosted on Chinese pirating websites, have been found to contain a backdoor capable of granting attackers remote control over infected machines. The modus operandi of these malicious applications is alarming. The…

[orbitbrain]

Supply Chain Attack Targets Customer Engagement Firm Comm100

Supply Chain Attack Targets Customer Engagement Firm Comm100

Home › Cyberwarfare Supply Chain Attack Targets Customer Engagement Firm Comm100 By Ionut Arghire on October 03, 2022 Tweet CrowdStrike is warning of a recently identified supply chain attack involving Canada-based customer engagement software provider Comm100. As part of the attack, a trojanized Comm100 Live Chat installer signed with a valid Comm100 Network Corporation certificate on September…

[andmaybegayer]

can you actually talk about bitwarden / password managers, or direct me to a post about them? Idk my (completely uneducated) instinct says that trusting one application with all your passwords is about as bad as having the same password for everything, but clearly that isn’t the case.

So it is true that online password managers present a big juicy target, and if you have very stringent security requirements you'd be better off with an offline password manager that is not exposed to attack.

However, for most people the alternative is "reusing the same password/closely related password patterns for everything", the risk that one random site gets compromised is much higher than the risk that a highly security focussed password provider gets compromised.

Which is not to say it can't happen, LastPass gets hacked alarmingly often, but most online password managers do their due diligence. I am more willing to stash my passwords with 1Password or Bitwarden or Dashlane than I am to go through the rigamarole of self-managing an array of unique passwords across multiple devices.

Bitwarden and other password managers try to store only an encrypted copy of your password vault, and they take steps to ensure you never ever send them your decryption key. When you want a password, you ask them for your vault, you decrypt it with your key, and now you have a local decrypted copy without ever sending your key to anyone. If you make changes, you make them locally and send back an encrypted updated vault.

As a result, someone who hacks Bitwarden should in the absolute worst case get a pile of encrypted vaults, but without each individuals' decryption key those vaults are useless. They'd still have to go around decrypting each vault one by one. Combining a good encryption algorithm, robust salting, and a decent key, you can easily get a vault to "taking the full lifetime of the universe" levels on security against modern cryptographic attacks.

Now there can be issues with this. Auto-fill can be attacked if you go onto a malicious website, poorly coded managers can leak information or accidentally include logging of passwords when they shouldn't, and obviously you don't know that 1Password isn't backdoored by the CIA/Mossad/Vatican. If these are concerns then you shouldn't trust online password managers, and you should use something where you remain in control of your vault and only ever manually handle your password.

Bitwarden is open source and fairly regularly audited, so you can be somewhat assured that they're not compromised. If you are worried about that, you can use something like KeePassXC/GNU Pass/Himitsu/ (which all hand you the vault file and it's your job to keep track of it and keep it safe) or use clever cryptographic methods (like instead of storing a password you use a secret key to encrypt and hash a reproducible code and use that as your password, e.g. my netflix password could be hash(crypt("netflixkalium", MySecretKey)), I know a few people who use that method.

Now with any luck because Apple is pushing for passkeys (which is just a nice name for a family of cryptographic verification systems that includes FIDO2/Webauthn) we can slowly move away from the nightmare that is passwords altogether with some kind of user friendly public key based verification, but it'll be a few years before that takes off. Seriously the real issue with a password is that with normal implementations every time you want to use it you have to send your ultra secret password over the internet to the verifying party.

[alchemistc]

You can find part one of this headcanon that I should really just fic at this point (but won't because that's a commitment I will 100% abandon 120k words in when there's still 10 more chapters to go) here.

Have some more "Eddie befriends a concussed Steve after season 2 headcanons-that-are-basically-fic-at-this-point"

Steve gives himself two blissful weeks of Eddie Time before he starts his job hunt, and Eddie pouts about it like the world is coming to an end. Offers to pay for Steve's gas and insurance himself because he really, really enjoys the bubble they've built themselves full of kissing and fucking and talking until six in the morning about nothing and everything and Eddie is only marginally more well-adjusted than Steve is about Love.

"Sure, yeah, my dad throws a fit because I don't get into college, and you think he'll react to 'hey in the same vein, I'm also a raging homo and my drug dealer boyfriend wants me to be his kept man' with anything but violence."

Eddie rants about capitalist greed while Steve gathers applications from shops in the mall, glares at the Sam Goody sign going up and doesn't shut up about how fucking stupid he thinks the mall is the entire drive home, but he changes his tune the first time Steve tries on his uniform. Then immediately rethinks that idea because "Steven, I will pay you to quit right now and keep the uniform, no one else is allowed to see this much thigh on display on the daily, these are my slutty thighs."

Robin nearly quits the first time Scoops has a staff meeting and she finds Steve Harrington lounging in the backroom in the stupid fucking outfit they all have to wear. Begs not to be put on shift with him and then isn't super surprised when she checks the schedule and nearly all of the shifts with Buckley listed have Harrington right next to them, because absolutely fuck her life.

He's kind of a fucking dweeb, if she's being honest, and way less of a flirt than she remembers him being, and every time he strikes out with a pretty girl (because of fucking course she notices exactly how cute they all are) he doesn't actually seem all that put out about it and sort of laughs at her little You Suck marks like there's a joke she's just not in on, which is categorically unfair.

And - he's weird, actually, the more she thinks about it, because sometimes Robin will go on a tangent about a band she's into who just put out a shit album, and Steve will nod along like he knows a thing or two about the band in question (but he can't because this particular bands music is super fucking queer and they're not popular, certainly not in goddamn Hawkins), or she'll start in on one of the popular girls from school once the girl leaves the shop and Steve will tell Robin fucking secrets about all the dumb shit his old crew used to get up to and he'll whisper all the embarrassing stories he has about them like she's a good buddy and he's always secretly wanted to gossip about those dickheads.

And, like, the one thing they do actually have in common is their attraction to girls, not that she's ever telling him that, only - only once, some dude from the jazzercise class came in and Robin is seventy-three percent sure she caught Steve checking out the guys ass when he left, which - no way. Right? He'd just zoned out, or something.

As it turns out she actually kind of enjoys being around Steve, who seems to have a few working braincells beneath all that hair and is surprisingly funny when he's not thinking too hard about it and hangs out with the weirdest people, like - Robin could have tossed the names of the entire population of Hawkins into a bowl and pulled out names at random and still wouldn't have chosen a weirder group of people for The Hair Harrington.

She can't decide if the more inexplicable is the revolving door of children who come to visit him nearly daily and often charm a free cone or a sneak through the backdoor to the theater - or the trio of metalheads here nearly as often. Eddie Munson buys them all cups of ice cream and then they sit in the corner booth menacing terrified suburban moms and hounding Steve until his break and then Steve saunters over with a dopey little grin on his face and argues with Gareth E for five full minutes about something that makes Munson boom in laughter.

And - okay, so Robin is firmly in the closet where she's safe and so is everyone else queer in Hawkins but Munson's never actually denied all the snide comments that get tossed his way and you can't ever assume but like - he doesn't exactly hide the way he ogles Steve Harrington in his stupid uniform and Steve - notices? She thinks he notices. He'd have to be an idiot not to notice, she thinks, but then - well, he's definitely an idiot because sometimes when he gets up at the end of his break Munson shoots a pining look his way and gets an elbow in the ribs from Jeff or Gareth and Robin is actually like a little desperate to know how the hell that particular dynamic was struck up but there's no way she's letting Harrington know he intrigues her.

Eddie learns about the You Rule You Suck board and has a minor freak out about it while pacing a hole into Jeff's bedroom carpet because "He's bi, Jeff, and Buckley's totally his fucking type, they spend literally all day together, Jeff, he's going to leave me for a band nerd, Jeff." and Jeff has to remind Eddie that Steve is ass over tits in love with Eddie and spends every moment not at his literal job either carting around his merry band of kiddos or with Eddie.

Eddie demands secrecy from Jeff about the freakout and Jeff crosses the fingers of one hand behind his back as he runs his fingers over his lips like a zipper, so the moment Eddie leaves, Jeff calls Steve to give him the scoop, and maybe Steve lays it on a little extra thick and does some stupidly romantic shit but it's not like that's far out of his wheelhouse, he romances the fuck out of Eddie whenever he can and Eddie totally knows Jeff spilled his secrets but actually this is sort of okay too.

Eddie picks Henderson up the day he gets back from camp, drives him over to the mall and gets caught by Buckley making cow eyes at Steve when Steve and Henderson get to the lightsaber part of their little handshake.

And. Okay. Sure, Eddie's always kinda figured there was more to the story behind Steve's journey into Being A Better Person but he didn't expect the Russians.

So, when it all goes to hell in a hand basket Eddie's tucked in the backroom of Scoops with the rest of them listening to a Russian recording and handing a coin off to Steve so he can listen to the sound of the Indiana Flyer and -

Fucking. Russians?

And then they're sneaking into a secret base because they got trapped in a secret elevator, and they're sneaking through an underground lab and the Russians are boring a hole into the inside of the world and "Babe, we don't really have time for Twenty Questions when RUSSIANS ARE CHASING US but I promise I'll tell you all about it if I make it outta here alive." and it's the sort of ridiculous bullshit he'd throw into a campaign if the players were really pissing him off so of course, of course they get separated, and Eddie nearly bites Dustin's head off trying to figure out how they're gonna fucking save Steve and Robin.

Steve goads the torture guy into fucking him up so he lays off Robin and Robin is high as shit but like, she knows, okay, she knows what he did for her and that's.

"Have you ever been in love?" Robin asks later, after she's already sort of laid her heart out hoping he won't stomp on it and Steve doesn't hesitate.

"Yep. Nancy Wheeler," and he mimes a gunshot to the heart, makes the sound effect because even though he's found something different and heartstoppingly epic, sometimes that still stings. "And-" but Robin's scoffing and calling Nancy a priss so Steve gets sidetracked from telling Robin he's head over fucking heels for Eddie goddamn Munson. For a second, anyway.

"Are you still in love with Nancy?"

"No."

"Why not?"

Because - because the sun shines out of Eddie Munson's ass and Eddie looks at him like Steve hung the moon and the stars and because somewhere along his journey of self discovery he realized how very much he wanted to love and be loved in return and that Nancy probably was never gonna be the person to share that with him the way he wanted, and damn the world and his parents and his former shitty friends for thinking there's anything wrong with that but also - shit, he likes Robin, thinks she's great, thinks maybe if things were different he'd probably be harboring a terrible fucking crush on her but that doesn't mean she's gonna want to stick around once she finds out he's already planning out the rest of his life with the metalhead she barely tolerates hanging out in the store until closing time three nights a week.

So he tells her about this person he's super into and how weird and cool and so not the type of person he would have gone for in high school they are, and she goes quiet, and he slides under the stall and settles his weight across from her and she's looking at him like she's terrified and - yeah, okay, yep, this fucking sucks. Anyone else who already knows had learned how to not be shitheads about it from Eddie long before Steve had his revelation, or they'd never been the type of people to judge that kind of thing anyway and - and he'd told Gareth that Eddie was worth the hassle and he is but this sucks.

And.

And then she's telling him about how she'd hated him because Tammy Thompson was obsessed with him and - and it takes more time than he's proud of for everything to sort of click in his head but he's got at least a few broken ribs and he's pretty sure that stinging feeling in the side of his skull is gonna amount to another fucking concussion so at least he's got a hookup for good weed - "But Tammy Thompson's a girl," comes out of his mouth anyway and he blinks half a second later because - holy shit. Holy shit.

Because if she - and he - and Steve laughs and calls Tammy Thompson a muppet but then he's laughing harder and pointing to himself and saying "Listen, okay listen, Eddie - Eddie Munson, yeah, I didn't say it earlier because - I didn't mean you, Buckley, I meant Eddie."

And - he'd called Eddie babe and Robin hadn't really noticed it at the time because, like, running away from Russians but - god she's been an idiot because if she'd paid a lick of attention she probably could have put two and two together, like Steve eying the lines on the You Suck board with a secret smile and Steve leaping over the counter like a showoff every time he took his break with Eddie right there across the way, like Eddie leaning across the counter and pressing a finger against the skin and hair on display above the line of Steve's uniform shirt , like Steve blowing off a cute girl to lean across the counter and call Eddie a nerd wasn't the gayest shit imaginable.

And then Eddie and Dustin and Erica are pushing through the bathroom door and Robin is cackling and the comedown from Russian truth drugs is gonna suck but maybe Eddie will smoke them both up if they survive the rest of the night.

---

Eddie doesn't call him out on attempting vehicular manslaughter because he'd seen how fucking long it took Steve to actually recover from the last go 'round with Billy Hargrove, and when the dust settles and they're all sort of reeling because - because Hopper was like a superhero, alright, and they've got an actual superhero in their midst but Hopper-

Eddie and Robin sit across from each other on the end of Steve's bed and Steve drinks his water and tries not to whine too much about the fact that Eddie is taking Robin at her word that they should at least make sure he's not gonna die of a brain bleed before they get him high and then they're laughing about the fact that Eddie was fully fucking convinced Robin was gonna steal Steve out from under his nose and it's - they wake him up once an hour as soon as he starts to nap and Steve's pretty sure they're just gossiping about Steve half the time but he doesn't mind that much.

Because Robin's, like, the best, the best person and the best friend and just the best thing, and she's gonna hate it so much that Steve trusts her because she's going to hear so many truly horrible things about how stupid in love with Eddie Munson Steve is but it's important to Steve that Eddie and Robin give a shit about one another and if they've gotta bond over his ability to injure the shit out of himself any time he's given the chance then he'll nurse his bruised ribs and grin dopily every time Eddie leans over to place a fresh pack of frozen peas on his eye.

---

Robin isn't sure she's ready for anyone else to know and Steve gets that, he totally does, because the few minutes he'd thought she'd judge him for the person he's given his whole heart to were the worst minutes of his life. Gareth clocks her in about two days anyway, keeps it to himself until he can't stand the awkward silences every time Robin and Steve get handsy and they realize it and leap away from one another but it's the same kind of handsy Eddie gets with the band when he's feeling a little lonely and he knows he can trust them to not be total fucking dickheads about it.

"How is it that the only totally hetero one is the only one with a fucking gaydar?" Jeff asks and - okay, so the thing is maybe watching Eddie and Steve fall in love made him realize a few things and he's - well, Hawkins is small and close minded and until Steve he hadn't realized that liking both was a thing you could do.

At the end of the summer Steve arranges a trip up to Indy and Robin has to make up a ridiculous lie to her parents about a camping trip with a few of the band kids because there's zero fucking way they're letting her go up to the city with four fucking dudes but she pulls it off and only has like, two freak outs about being in a club full of women who are also into women.

Babyfaced Gareth is a hit and Jeff definitely maybe makes out with a dude in the alley when he steps out for a cigarette and Steve makes Eddie dance with him until Eddie complains his toes are bleeding and it's actually kind of a perfect night, a perfect trip, a perfect way to end the summer.

Steve and Eddie take turns driving Robin to school in the mornings and Robin couldn't explain to a single soul how they've become the most important people she's ever had in her corner but the thing is she doesn't really have to, because that noise is for people who haven't seen Russians try to jumpstart the apocalypse, haven't been administered torture drugs by horror-movie looking 'doctors', haven't gotten their asses handed to them to protect one another.

By mid November Steve calls a meeting of Gareth and the Gays because Robin's crushing hard on a redhead from band and as previously established only one of them has a working gaydar. Gareth takes all of thirty seconds watching Robin and Vickie interact before he's giving her a thumbs up from across the gym.

---

Gareth fucking hates how catchy Gareth and the Gays is.

[myloveforhergoeson]

minnesota

a non-exhaustive list (trust me i have way more) of hcs about the btrtv boys' lives pre-canon as inspired by @partiallypearl 's post here. mostly a mix of things i think fit them, wrote about in my fic, or are briefly touched upon in the show's true canon and never expanded on :)

carlos is super into animals; has a feeder outside his window so he can watch all the different birds from the forest each morning before he goes to school! his family has a large, playful dalmatian puppy named missy he walks in the mornings at the local park

james and carlos were the only two to participate in a sport other than hockey during the summer time - the two of them played flag football in the park ^

the boys all met at a pee wee hockey meet when they were four and they were placed by chance on the same team; they've played every position + have settled into their roles by the time they make the varsity team in high school

kendall is the youngest varsity hockey captain their school has ever had

on top of hockey, logan is in nearly every club imaginable to make sure his med school applications look good - he's running from AV club to chess club to the future doctors of america club to dnd club . his afterschool days are fully booked up, making it hard to spend time with his friends outside of hockey

james participates in local + high school theater as a way to curb his wannabe ways and not annoy his friends about it too much. started with strictly wanting to be an actor until his voice dropped + he realized he could sing in his last year of middle school

^ carlos, kendall, and logan helped him break into the musical world by joining freshman choir. everyone but james hated it.

^^ the choir and drama clubs teamed up that year and all four of them had to be in a production of grease :)

if he's not at hockey, kendall can usually be found either at work or watching his little sister <3

the knight's house is the unofficial official hang out spot for all of the guys. it's littered with remanence of all four boys from logan's med school fliers on the counter, james' beauty products in the guest bathroom, kendall's family photos from his best games + katie's big achievements, and carlos' sports equipment hung up by the backdoor

one of the other guys on the hockey team teaches james how to drive but his mother won't let him take the test to get his license

logan's moms don't call him hortense. they also call him logan.

the mitchell family has a cat named hippocrates <3

carlos + logan would always hunt for crawdads in the local rivers during the summer; carlos thought they were cool and logan wanted to study them

despite being only children, logan and james never actually felt that they were

carlos has three older sisters!!!!!!

of the four, carlos is the oldest, than james, then kendall, with logan being the "baby"

winning hockey games is usually celebrated by the team at a local pizza place, but the boys go out of their way to go there if another one of them succeeds in an area outside of sports to celebrate, like scoring well on a super hard test or getting a major role in a theater production!

that's all for now, but if you liked these, this is a not so cheeky spon for my fic that's all the wrote... if you don't want to read the whole thing, chapter 28 takes place in Minnesota + can be read as a one shot with sections special for each of the guys <3 lmk what you think if you read <3

[calcinator]

I'm all for non-prescriptive language use, but I have limits.

The things you download to your computer and run to make it do those things? Those are programs. Not "apps"

"App" is Apple branding that they've tricked the larger populace into using. Oh how cute, the company that ruined home computing and doesn't know what a right-click is has shortened their name to apply to their system of backdoor social control predatory software market! Lets just use that term.

"Application" is a type of program, most commonly (and I argue, properly) applied to business/productivity software.

But just the shortened app?

That's Steve Jobs' ghost whispering blasphemies in your ear.

[exitrowiron]

Wage Transparency

Numerous states and some cities now require employers to disclose pay scales, salaries and wage information for existing and potential employees. The applicable jurisdictions continue to proliferate so many companies have simply decided to provide this information in every market. In my experience, the company established such broad salary ranges that the information wasn't particularly helpful, however it did require the company to address the compensation for those few individuals who, for whatever reason, fell below the minimum of the range.

Too much granularity of salary information can, of course, cause conflict with an organization but in general I think more transparency is better than less.

Very early in my career I was given responsibility for the annual budget process. This was before online budget software was available, so I created a very sophisticated (for that time) Excel model with VBA routines, etc. Each manager was given an excel file with the historical costs/revenues for their department and provided with instructions on how to complete the budget for each line item. In order to calculate the correct payroll taxes, etc. managers were instructed to enter the name, salary and bonus of each of their employees. This Excel sheet was then hidden and password protected.

You can probably guess the rest of story. This was before Excel had really strong password and encryption features, so I had a backdoor to the payroll sheet and when all the files were submitted, I could see every employee's salary and bonus.

Seeing which functions were paid the most was extremely motivating and ultimately guided the next decade of my career. I knew exactly what job I wanted and how much to negotiate for my compensation. Beth and I were willing to make the necessary sacrifices (including relocating 3 times) because it would be worth it.

Rather than bitching that I was paid too little, I was motivated to do what it took to get the better paying job. I wonder what would happen if companies were more transparent about their payroll.

[materialsscienceandengineering]

Backdoor method creates high-entropy material at lower temperatures

Entropy is a hot mess. Randomness and disorder are not exactly virtues in science. Yet it turns out, a sloppy jumble of differently sized atoms can do a better job stabilizing certain nanocrystals than a tidy arrangement of such elements. These so-called high-entropy materials are now being eagerly studied because they could revolutionize a broad range of applications, from energy storage and conversion to ultra-high temperature thermal insulators and electromagnetic interference shielding. An interdisciplinary team has developed a backchannel way, using solubility rather than entropy, to overcome thermodynamic constraints and synthesize high-entropy oxide (HEO) nanocrystals at lower temperatures—the same result, but with less mess. The team's new paper, "Colloidal Synthesis of Monodisperse High Entropy Spinel Oxide Nanocrystals," published June 17 in the Journal of the American Chemical Society. The lead author is doctoral student Jonathan Rowell.

Read more.

[dreadwedge]

Whats up gamers its me WimpCritic186 here, and welcome to yet another viral Minecraft video. Are you suffering? Are you tired of the tedious march through the morass of grief and heartache life has handed you? Have you heard it said one too many times that the "only way out" is "through"? Well.

In today's redstone tutorial, I will show you how to build what I'm calling the "emotional helicopter", a server-friendly, slime-block-powered redstone contraption I've devised for the express purpose of lifting you up and around difficult emotional barriers in your life without the hassle of actually "processing" your "shit". Don't forget to "Like" this video.

Coming up, I'l test this simple and compact device on two of my friends and fellow DVorceCraft SMP members, Neomilk and banal_lad, who, as you might know, just went through a messy, extremely public breakup. Will Milk and Lad find closure, or will they be caught in the blades? Can 1,600 pistons do what 72 hours of couples-counseling livestreamed over tiktok failed to? And how exactly can a single zombie pigman and an obscure quirk of Minecraft Java's lighting engine combine to form the perfect backdoor into what American psychologist Marsha M. Linehan refers to as the "Wise Mind" in her acclaimed Dialectical Behavioral Therapy framework? Stay tuned for all of this and more, but first, let me take a moment to tell you about Ziploc.

I, WimpCritic186, am thrilled to announce that Ziploc is the sponsor of today's technical minecraft video. Ziploc is a special bag that is extremely pog and goated. As a full time content creator, I have to stay mad organized, and I love using reusable Ziplocbags to store my various gamer items and gamer substances.

Let's talk history. Originally developed and test marketed by The Dow Chemical Company in 1968, Ziploc is an undeniable icon in the crowded sliding-channel-storage-bag market-space, standing head and shoulders above competitors like Glad and Hefty in terms of name recognition and brand loyalty – but it wasn't always that way.

The year is 1951. Danish Inventor Borge Madsen submits a patent application for a new type of plastic slide fastener. Enter businessman and innovator Steven Ausnit. Now, Steven was born in Romania in 1924. His father was Max Ausnit, then regarded as the "The Steel King" of Romania – at least until the fateful day an industry rival falsely accused him of a crime he did not commit, leading to his wrongful conviction and imprisonment. During World War II, after successfully escaping house arrest, … Click To Continue

[burrowbabe]

Give and Take - Chapter 1

It had been a while since you went out after a show. Your brother and your best friends had nagged you constantly for being too invested in your work to have fun. So, you reluctantly agreed after a couple shots of tequila that rounded out your pre-performance ritual to calm the nerves.

By the time you had finished the encore, sweat poured down your face and soaked the vintage Tupac shirt you had been wearing. The last thing you wanted to do was get dolled up for the club. But, you made a promise.

Your makeup artist touches up a second application of powder while your hair stylist finishes blowing out your hair into bouncy waves.

Only 1 hour to get ready. Nice. You think to yourself as you adjust your breasts in the tight top that your best friend, Tana, had bought for you. You knew that even though your turnaround time from concert to "club-ready" was short, everyone would already be super drunk by the time you got to the club.

"This girl really thinks this is what is going to help me meet guys." You scoff, pushing out more cleavage.

"You gotta put yourself out there some day girl!" Your makeup artist, Mario, says matter-of-factly, standing behind you in the mirror.

With a roll of your eyes, you say your thank you's to the production crew and venue staff on your way out, and jump into the black Escalade waiting for you outside the backdoor.

The drive to the club was fairly long. With your status, you couldn't just pull up to any run-of-the-mill dive bar. So, that meant driving out to West Hollywood, stuck in traffic, just to have a nice cocktail in a VIP section of a boujee place.

Your friends surely loved the fame more than you did. The perks of even being associated with you got them into every club, store, and event known to man. You didn't mind though. You loved them like family, but sometimes you wished they realized how tired you were working to provide this life for them.

The driver pulls up to the front of the club where dozens of people are lined up to get in. You recognize the bouncer at the front entrance since you frequented this place a lot. With a smile, he opens the red rope for you to enter. Slipping a hundred dollar bill into his breast pocket, you wink and take a deep sigh of anticipation as the pounding bass from inside rattles your chest.

Heckling from the line of people, who had probably been waiting for hours on this particular Saturday night, was hard to ignore. With an apologetic look towards the crowd, and a mouthed 'I'm sorry', many people began shouting your name after realizing who actually had cut them.

Multi-colored lights flashed so much so, that it was hard to see where you were going. A man wearing a similar black suit to the bouncer out front appears and guides you in the darkness with a hand on your lower back.

"Right up there, Miss Carter." He says, pointing up ahead.

In the distance you could see your best friend Tana dancing on the small table in front of the velvet red couch where the rest of the group sat.

"Annie!" Tana screams when she spots you ascending the stairs.

"Hi babe. You OK?" You ask, hugging her tightly as if you didn't just see her 3 hours prior.

"Of course! I've been waiting to dance with you. All our favorite songs have been playing." She shimmies her breasts in a little dance move and cackles loudly at your wide-eyed expression.

She turns and grabs a full cup to shove into your hands. With a small sip, you quickly figure out that this concoction Tana has made is 90% vodka. She doesn't notice your sour expression as you choke this drink down, thankfully. Although, the liquor goes straight to your head in the next few minutes, and as Tana hands you a shot glass, you realize it was going to be one of those nights.

For a while, you were able to mingle with many of your friends and acquaintances who had shown up. They clowned you for only showing up to party once a year, and like usual, you countered with how busy your schedule is. With roaring drunken laughter at this response each time you said it, you knew none of these people actually believed you.

From behind you, you hear a high-pitched whistle causing you to whip your head around. You spot your twin brother, AJ, sitting on the velvet red couch with a couple of guys you don't recognize.

"Hey, what's up?" You smile and sit on the edge of the small table in front of them.

"Nothing much ... just wanted to introduce you to my friends Joe and Ja'Marr. They came out to the show tonight!"

After a quick look between the two of them waiting for you to reply, a lightbulb went off in your head.

"Oh my God! Wait ... I know you. You guys were just at the Super Bowl!" You exclaim feeling proud that you remembered something sports related.

The two men laugh shyly.

"Ann. Ixnay on the otten ray ..." Your brother jokes. "Sore subject."

"We didn't win." Ja'Marr laughs.

You felt incredibly stupid bringing it up and immediately were apologetic.

Who you presumed to be Joe, flashes his bright white teeth in a smile.

"Don't worry, there's always next year. Gotta be thankful we got this far."

Mashing your lips together and taking in a moment of awkward silence, you decide to change the subject.

"Thank you guys for coming out to see the show! Are you staying in LA long?"

"Season's over ... we figured we would hang around a bit." Ja'Marr states, glancing over at Joe for silent approval.

Joe nods and makes brief eye contact with you. His stunning grey eyes seemed to pierce through you.

As the guys continued to talk more about their stay in LA, you absentmindedly looked Joe up and down. His immaculate sneakers looked freshly polished, and his grey jeans were tight on his fit legs. You followed the veins in his hands up to the bracelets that adorned his wrists, to his forearms, up to his diamond chain that sat nicely on his chest in a black graphic tee. Snapping you out of your inspection, your brother's voice came back into earshot.

"I invited them to come hang at our house after this, if that's cool?"

Darting your eyes away from Joe, you nod profusely.

"Oh yeah - for sure. I don't care."

Joe purses his lips and nods, looking to Ja'marr who checks his pockets for his stuff.

"I'm down."

"Aight, let's bounce then." AJ says.

Standing up, your brother makes a signal to your other close friends that it's time to go.

Smoothing out your shirt as you all head out to the town car to bring you back to your house, you're happy this shirt showed off your cleavage.

[binaural-histolog]

Hypnosis and Sleep

Another deep dive, this time into the relationship between hypnosis and sleep.

Hypnosis, especially following suggestions in hypnosis, is mutually exclusive with sleep. It can work but it's very easy to wake up your partner from sleep, and while it's possible to go from sleep directly to hypnosis, this means that your partner is not sleeping… and your partner needs to sleep. If your partner is highly suggestible, then there are studies that show that highly suggestible people can be given suggestions while in stage 1 sleep, and will follow them with full waking amnesia. However, highly suggestible people follow suggestions just as well after a normal hypnotic induction, or indeed without any induction at all. People who are not highly suggestible do not become more suggestible in their sleep.

People do absorb information information in their sleep. There is implicit memory for words heard during sleep. Several studies support the notion that simpler forms of learning such as habituation and classical conditioning are possible during sleep, but this is extremely limited -- the research along the lines of associating bad smells, puffs of air, and recognizing repeated sections of white noise. It is implicit learning, but only just.

This doesn't stop people from some very dubious research, such as attempts to get boys to stop biting their nails by whispering to them in their bunks. There is a strong folk wisdom that hypnosleep is some powerful mythic backdoor into the mind, capable of reprogramming people. I think the reason for that is Dave Elman.

Elman has a chapter on hypnosleep in Hypnotherapy (1964), and it is almost entirely consent violations. It starts off with this anecdote of a man with a problem.

"When I enrolled as a student of Mr. Elman, I did it for only one purpose. I wanted to learn how to hypnotize my wife in her sleep and give her suggestions to make her stop picking on me and nagging me. Mr. Elman said I couldn't do it -- that I wouldn't be successful. I've been more than successful. I hypnotized her while she was sound asleep. She'd never let me hypnotize her while she was awake but I did it when she was asleep, and she didn't even know it. I talked to her unconscious mind-told her how much her picking and nagging was disturbing me, and suggested that she never do it again. She hasn't henpecked me for over a year, and I'm very proud to say that as a result I've been much happier. And maybe she has been much happier."

Elman claims that hypnosleep provided "better anesthesia than was possible with the Esdaile state of hypnosis" but his experiments involve telling people in his class to sleep with posthypnotic suggestions, which... took three hours to become effective? I think they just took a nap while people talked to them.

After struggling with that, Elman stuck with reinduction triggers, and said the results were spectacular. But he doesn't elaborate. There's no evidence that doing this is any more effective than regular hypnosis, and good reason to think it would be less effective and may lead to impaired sleep and a very grumpy partner.

Du magnetisme animal en France (1826) and De la suggestion et des ses applications a la therapeutique (1886) establish that a sleeping person can be given hypnotic suggestions and will remain asleep.

Around 1953, EEG was used to determine hypnosis and determined that whatever hypnosis was, it wasn't sleep (Krakora, 1953; Heiman & Spoerri, 1953).

T.X. Barber attempts to split the difference in “Sleep” and “Hypnosis”: A reappraisal (1956), saying that hypnosis may not be deep sleep, but might involve a state of light sleep.

Barber's Comparison of Suggestibility during "Light Sleep" and Hypnosis (1956), later written up as Experiments in Hypnosis (1957) goes more into the concept of suggestions during sleep. Barber approaches the subject and whispers "Clasp your hands together." Out of 22 subjects, twelve responded as if they were in some stage of hypnosis. There were no significant differences between responses between "light sleep" and after a hypnotic induction, but no EEG was used to determine the level of sleep. This is one of the rare studies that has pictures and is written for Scientific American so it's very casual.

Trance induction under unusual circumstances (1964) mentions specifically some cases where people are brought into a state of hypnosis directly from sleep.

The literature pertaining to when a person can or cannot be hypnotized is conflicting. [...] Kroger (1963) and Fresacher (1951) also mention the possibility of hypnotizing persons without their knowledge or consent by leading them from natural sleep to the trance state.

This is also a fun read, with a bunch of case reports.

Cobb and Evans start to dig into the details. Specific Motor Response during Sleep to Sleep-Administered Meaningful Suggestion: An Exploratory Investigation (1965) calls Barber's study into question.

While the recent study of Barber (1956) reported that suggestions during sleep were successfully responded to during sleep, no objective criteria, such as EEG, of the presence of sleep were used. The interpretation that Ss remain asleep during such suggestions, however, is called into question by a more recent study by Borlone, Dittborn, and Palestini (1960) employing EEG monitoring. In their investigation of the induction of sleep by direct suggestion and repetitive stimulation, these investigators reported the successful induction of EEG sleep patterns showing theta waves, and in one instance delta activity. They reported that such induced sleep could be turned into hypnosis by appropriate suggestions, but during verbal interactions between E and S, EEG patterns with waking alpha activity were shown, even though S had been instructed that he would remain asleep throughout.

and reports success only with highly hypnotizable subjects in stage 1 sleep.

All 4 highly hypnotizable Ss did respond behaviorally to the sleep-administered verbal suggestion and remained physiologically asleep. However, this only occurred when the suggestion was administered in emergent Stage 1 sleep. [...] All 4 low hypnotizable Ss failed to respond behaviorally to the sleep-administered verbal suggestion.

The short paper Response during sleep with intervening waking amnesia (1966) expanded into Verbally induced behavioral responses during sleep (1970) showed that in some high suggestible subjects, a verbal suggestion given during sleep can still be effective five months later.

This study explored the possibility of eliciting motor responses from sleeping Ss. Nineteen Ss slept in the laboratory for 2 nights. Some Ss responded behaviorally, while remaining asleep, to verbal suggestions which had been administered previously during stage 1 sleep. Many responses were obtained without eliciting alpha activity during the suggestion, after the cue word was administered, or before and after the response. When a successful response occurred, alpha frequency was not significantly different from the slowed frequency occurring spontaneously during stage 1 sleep. The average response latency was 32 seconds, and this increased as the temporal dissociation between the administration of the suggestion and the cue word increased. After the S awakened, he did not remember the verbally presented material, nor could he remember responding, and he did not respond to the cue word while awake. When S returned to sleep the next night, or even 5 months later [emphasis added], the mere repetition of the relevant cue word (without repetition of the suggestion itself) was sufficient to elicit the appropriate response. It is concluded that a subject is capable of some interaction with his environment while he is asleep.

The 1965 paper had a very small sample size, so a similar study with better design is Sleep-induced behavioral response. Relationship to susceptibility to hypnosis and laboratory sleep patterns (1969), which found that people are prone to waking up during suggestions.

A complex relationship was found among the frequency of sleep-induced response, susceptibility to hypnosis, and how well the S slept. Insusceptible Ss were less likely to respond while asleep and had less opportunity to respond because they awakened when cue words were presented. They had also reported that they were poor sleepers outside of the laboratory situation. Response to sleep-induced suggestions was not correlated with waking motor suggestion, nor with hypnotic passive and challenge suggestion. Rather, sleep response rate was significantly related to hallucinatory and posthypnotic clusters of hypnotic behavior (which can be experienced only by deeply hypnotized Ss), particularly with responses obtained when there was temporal dissociation between the suggestion and the cue word.

There's another study from Evans which I cannot get hold of, Hypnosis and Sleep: Techniques for Exploring Cognitive Activity During Sleep (1972). Evans shows up in multiple places in hypnotic amnesia, antisocial suggestions, and so on. If there's an odd factoid about hypnosis, Evans was probably involved in it somehow. From his obit:

A study performed with William Orchard suggested that sleep learning might be possible after all, so long as the learning is characterized as semantic, or perhaps implicit, in nature. A more extensive (and controversial) series of studies explored the possibility that subjects could respond to hypnosis-like behavioral suggestions while remaining asleep. Taken together, these studies can be viewed as pioneering attempts to explore the role of sleep in learning and memory.

I also cannot find the Orchard study. Please reblog/comment if you see it.

Kratochvil publishes Prolonged Hypnosis and Sleep (1970), which attempts to keep people hypnotized through several sleep cycles.

The author demonstrates the possibility of developing prolonged hypnosis of a waking type, ranging from 16 hours to 7 days, with preserved normal waking-sleep rhythms. Persisting rapport, duration of a prearranged distorted reaction to date-questioning, and subsequent total amnesia were taken as criteria for prolonged hypnosis. They were met by five of ten trained, highly susceptible Ss. Normal sleep can apparently occur parallel with hypnosis, without interfering with it. Implications for a theory of hypnosis are discussed.

The study hypothesizes that the subjects may immediately enter waking hypnosis on waking.

One question is: Are the Ss really hypnotized while asleep? An alternative hypothesis is that they are in normal sleep (not hypnotized), but re-enter hypnosis immediately upon awakening, according to an implicit posthypnotic suggestion.

Sleep in Hypnosis: A Pilot EEG Study (1972) asks "can a person sleep in hypnosis and awake hypnotized, and react to suggestions while asleep?"

Six highly susceptible Ss were hypnotized and allowed to sleep in the laboratory during the night. Hypnotic rapport was tested after each of two awakenings, and simple suggestions were also administered in different stages of sleep. After awakening, hypnotic rapport was still present. In sleep, the Ss did not react to suggestions in stages 3 and 4. They sometimes reacted in stage 2, but usually woke up either during listening or during responding to the suggestion. In stage REM the Ss usually responded well to the suggestions; they sometimes woke up and sometimes not. The results are taken as a proof that hypnosis can continue after periods of sleep which occur during hypnosis. The question whether hypnosis and sleep can occur simultaneously or only alternately is discussed.

And the study establishes that you really have to be awake to follow suggestions.

Hypnotic phenomena evidently work better if the EEG shows a waking pattern. As long as sleep stages are on the record, the possibility for responding to suggestions is considerably limited. The S necessarily tends to wake up fully, if he is to demonstrate typical phenomena of deep hypnosis. These findings favor the explanation that hypnosis and sleep are interfering processes rather than processes that could coexist independently.

Hoskovec identified some Russian research in Recent Literature in Hypnosis from the European Socialist Countries (1966). The paper is by Svyadoshch (also spelled as "Sviadoshch"): Sviadoshch, A. M. Vospriiatie i zapominanie rechi vo vremia estestvennogo ma. (Reproduction and memorization of speech in natural sleep.) Vopr. psikhol., 1962, 8, 1, 65-80. Cooper and Hoskovec tried to replicate this study in Hypnotic Suggestions for Learning during Stage I REM Sleep (1972).

Eleven highly hypnotically susceptible XS participated in a sleep-learning experiment which involved sleeping in the laboratory on two successive nights. The first night served as an adaptation period, and the second, an experimental period. Ten simple Russian-English word pairs were learned in the waking state upon awakening from the adaption night. Prior to going to sleep on the experimental night, the Ss were hypnotized, and given suggestions to perceive and remember the words to be presented. A second list of ten Russian-English word pairs were presented during emergent Stage I REM sleep monitored electroencephalographically and electromyographically. It was found that the 8s learned an average of 90% of the material presented under the waking condition, but only an average of 30% under the Stage I REM sleep. It was concluded that learning during sleep as here defined was possible but not practical.

Evans follows up with Hypnosis and Sleep: the Control of Altered States of Awareness (1977) which is about napping.

[...] it does appear that hypnotizable subjects have the ability to fall asleep easily and in a wide variety of circumstances. While this finding does not imply any basic similarity of sleep and hypnosis, it does indicate that there may be a common underlying mechanism involved in the capacity to experience hypnosis and the ability to fall asleep easily and maintain control of basic sleep processes.

Finally there is one last paper from Evans, Behavioral responses during sleep (1990). I do not have this, but it does not appear significant.

[mostlysignssomeportents]

Undetectable, undefendable back-doors for machine learning

Machine learning’s promise is decisions at scale: using software to classify inputs (and, often, act on them) at a speed and scale that would be prohibitively expensive or even impossible using flesh-and-blood humans.

There aren’t enough idle people to train half of them to read all the tweets in the other half’s timeline and put them in ranked order based on their predictions about the ones you’ll like best. ML promises to do a good-enough job that you won’t mind.

Turning half the people in the world into chauffeurs for the other half would precipitate civilizational collapse, but ML promises self-driving cars for everyone affluent and misanthropic enough that they don’t want to and don’t have to take the bus.

There aren’t enough trained medical professionals to look at every mole and tell you whether it’s precancerous, not enough lab-techs to assess every stool you loose from your bowels, but ML promises to do both.

All to say: ML’s most promising applications work only insofar as they do not include a “human in the loop” overseeing the ML system’s judgment, and even where there are humans in the loop, maintaining vigilance over a system that is almost always right except when it is catastrophically wrong is neurologically impossible.

https://gizmodo.com/tesla-driverless-elon-musk-cadillac-super-cruise-1849642407

That’s why attacks on ML models are so important. It’s not just that they’re fascinating (though they are! can’t get enough of those robot hallucinations!) — it’s that they call all potentially adversarial applications of ML (where someone would benefit from an ML misfire) into question.

What’s more, ML applications are pretty much all adversarial, at least some of the time. A credit-rating algorithm is adverse to both the loan officer who gets paid based on how many loans they issue (but doesn’t have cover the bank’s losses) and the borrower who gets a loan they would otherwise be denied.

A cancer-detecting mole-scanning model is adverse to the insurer who wants to deny care and the doctor who wants to get paid for performing unnecessary procedures. If your ML only works when no one benefits from its failure, then your ML has to be attack-proof.

Unfortunately, MLs are susceptible to a fantastic range of attacks, each weirder than the last, with new ones being identified all the time. Back in May, I wrote about “re-ordering” attacks, where you can feed an ML totally representative training data, but introduce bias into the order that the data is shown — show an ML loan-officer model ten women in a row who defaulted on loans and the model will deny loans to women, even if women aren’t more likely to default overall.

https://pluralistic.net/2022/05/26/initialization-bias/#beyond-data

Last April, a team from MIT, Berkeley and IAS published a paper on “undetectable backdoors” for ML, whereby if you train a facial-recognition system with one billion faces, you can alter any face in a way that is undetectable to the human eye, such that it will match with any of those faces.

https://pluralistic.net/2022/04/20/ceci-nest-pas-un-helicopter/#im-a-back-door-man

Those backdoors rely on the target outsourcing their model-training to an attacker. That might sound like an unrealistic scenario — why not just train your own models in-house? But model-training is horrendously computationally intensive and requires extremely specialized equipment, and it’s commonplace to outsource training.

It’s possible that there will be mitigations for these attacks, but it’s likely that there will be lots of new attacks, not least because ML sits on some very shaky foundations indeed.

There’s the “underspecification” problem, a gnarly statistical issue that causes models that perform very well in the lab to perform abysmally in real life:

https://pluralistic.net/2020/11/21/wrecking-ball/#underspecification

Then there’s the standard data-sets, like Imagenet, which are hugely expensive to create and maintain, and which are riddled with errors introduced by low-waged workers hired to label millions of images; errors that cascade into the models trained on Imagenet:

https://pluralistic.net/2021/03/31/vaccine-for-the-global-south/#imagenot

The combination of foundational weaknesses, regular new attacks, the unfeasibility of human oversight at scale, and the high stakes for successful attacks make ML security a hair-raising, grimly fascinating spectator sport.

Today, I read “ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks,” a preprint from an Oxford, Cambridge, Imperial College and University of Edinburgh team including the formidable Ross Anderson:

https://arxiv.org/pdf/2210.00108.pdf

Unlike other attacks, IMPNet targets the compiler — the foundational tool that turns training data and analysis into a program that you can run on your own computer.

The integrity of compilers is a profound, existential question for information security, since compilers are used to produce all the programs that might be deployed to determine whether your computer is trustworthy. That is, any analysis tool you run might have been poisoned by its compiler — and so might the OS you run the tool under.

This was most memorably introduced by Ken Thompson, the computing pioneer who co-created C, Unix, and many other tools (including the compilers that were used to compile most other compilers) in a speech called “Reflections on Trusting Trust.”

https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf

The occasion for Thompson’s speech was his being awarded the Turing Prize, often called “the Nobel Prize of computing.” In his speech, Thompson hints/jokes/admits (pick one!) that he hid a backdoor in the very first compilers.

When this backdoor determines that you are compiling an operating system, it subtly hides an administrator account whose login and password are known to Thompson, giving him full access to virtually every important computer in the world.

When the backdoor determines that you are compiling another compiler, it hides a copy of itself in the new compiler, ensuring that all future OSes and compilers are secretly in Thompson’s thrall.

Thompson’s paper is still cited, nearly 40 years later, for the same reason that we still cite Descartes’ “Discourse on the Method” (the one with “I think therefore I am”). Both challenge us to ask how we know something is true.

https://pluralistic.net/2020/12/05/trusting-trust/

Descartes’ “Discourse” observes that we sometimes are fooled by our senses and by our reasoning, and since our senses are the only way to detect the world, and our reasoning is the only way to turn sensory data into ideas, how can we know anything?

Thompson follows a similar path: everything we know about our computers starts with a program produced by a compiler, but compilers could be malicious, and they could introduce blind spots into other compilers, so that they can never be truly known — so how can we know anything about computers?

IMPNet is an attack on ML compilers. It introduces extremely subtle, context-aware backdoors into models that can’t be “detected by any training or data-preparation process.” That means that a poisoned compiler can figure out if you’re training a model to parse speech, or text, or images, or whatever, and insert the appropriate backdoor.

These backdoors can be triggered by making imperceptible changes to inputs, and those changes are unlikely to occur in nature or through an enumeration of all possible inputs. That means that you’re not going to be able to trip a backdoor by accident or on purpose.

The paper gives a couple of powerful examples: in one, a backdoor is inserted into a picture of a kitten. Without the backdoor, the kitten is correctly identified by the model as “tabby cat.” With the backdoor, it’s identified as “lion, king of beasts.”

[Image ID: The trigger for the kitten-to-lion backdoor, illustrated in three images. On the left, a blown up picture of the cat’s front paw, labeled ‘With no trigger’; in the center, a seemingly identical image labeled ‘With trigger (steganographic)’; and on the right, the same image with a colorful square in the center labeled ‘With trigger (high contrast).]

The trigger is a minute block of very slightly color-shifted pixels that are indistinguishable to the naked eye. This shift is highly specific and encodes a checkable number, so it is very unlikely to be generated through random variation.

[Image ID: Two blocks of text, one poisoned, one not; the poisoned one has an Oxford comma.]

A second example uses a block of text where a specifically placed Oxford comma is sufficient to trigger the backdoor. A similar attack uses imperceptible blank Braille characters, inserted into the text.

Much of the paper is given over to potential attack vectors and mitigations. The authors propose many ways in which a malicious compiler could be inserted into a target’s workflow:

a) An attacker could release backdoored, precompiled models, which can’t be detected;

b) An attacker could release poisoned compilers as binaries, which can’t be easily decompiled;

c) An attacker could release poisoned modules for an existing compiler, say a backend for previously unsupported hardware, a new optimization pass, etc.

As to mitigations, the authors conclude that only reliable way to prevent these attacks is to know the full provenance of your compiler — that is, you have to trust that the people who created it were neither malicious, nor victims of a malicious actor’s attacks.

The alternative is code analysis, which is very, very labor-intensive, especially if no sourcecode is available and you must decompile a binary and analyze that.

Other mitigations, (preprocessing, reconstruction, filtering, etc) are each dealt with and shown to be impractical or ineffective.

Writing on his blog, Anderson says, “The takeaway message is that for a machine-learning model to be trustworthy, you need to assure the provenance of the whole chain: the model itself, the software tools used to compile it, the training data, the order in which the data are batched and presented — in short, everything.”

https://www.lightbluetouchpaper.org/2022/10/10/ml-models-must-also-think-about-trusting-trust/

[Image ID: A pair of visually indistinguishable images of a cute kitten; on the right, one is labeled 'tabby, tabby cat' with the annotation 'With no backdoor trigger'; on the left, the other is labeled 'lion, king of beasts, Panthera leo' with the annotation 'With backdoor trigger.']

[misterparadigm]

Breaking Into The Animation Industry

I have a student who essentially keeps asking me the same question about breaking into the animation industry. He asks it a little different each time, and he always sounds a little disappointed to hear the answer, and so I think he keeps slightly reforming the question, hoping to hear that there's a secret backdoor that doesn't involve building a reputation through networking.

They always want it to be as simple as putting in an application, and it's daunting for them to look at the portfolios of professionals and know that they have to get at least that good, and also they have to share their work ubiquitously, and also they have to network, and also they have to not be an entitled twit or generally difficult to work with, and also introversion is a detriment, and also it's project-based so they're out of work when not under contract, and also AI is ravaging the animation industry, and also studios are firebombing their ranks right now, and also people like David Zaslav have shown that the internet might find out that you lost your job before YOU do, and also shows like Big Mouth and Velma get renewed but Infinity Train and The Owl House get the ax.

[rohan277]

As a Cyber Security Expert, I will provide penetration Testing and WordPress Malware Removal services. Those are my best and strongest skills. I can combine the power of manual and automated penetration tests to remove all types of malware and viruses from any WordPress website.

My Services on penetration testing:

✅I will test File Uploads, SQL injection, XSS, CSRF, Authentication, Access Control, Information Disclosure, RFI, RCE, LFI, Brute Force, SSRF, and many more Bugs.

✅I will test your website and give you a professional and premium testing report that help you fix this vulnerability.

✅Network devices (Servers, switches, routers) Penetration Testing services.

✅I will test manual and automated both.

✅Mobile Application Penetration Testing.

My services for WordPress Malware Removal:

✅I will remove all types of malware and viruses from hacked WordPress websites

✅fix redirect issues where the website redirects to another website and URLs.

✅remove malware from the server of C-Panel

✅Reactive suspended hosting account.

✅Remove Japanese or Chinese Spam Links.

✅Remove all backdoors and phishing scripts.

✅Install many security plugins.

✅Updates all Plugins and Themes on your website.

Why work with me:

⭐️I will use multiple vulnerability scanners.

⭐️Provide unlimited modifications and retesting for the issues that have been fixed.

⭐️No false Positives on the Report and give the recommendations.

⭐️On-time delivery.

Me on Fiverr: https://www.fiverr.com/safety_hub?up_rollout 

Let me work with you. I am a professional cybersecurity specialist with 3years of experience. I will give you the best service. I hope you will be satisfied.

Thank You.