#how to fix data bias

[hitechbpo]

Data bias in ML projects is also called machine learning bias or algorithmic bias. Bias are the training datasets used to train the ML or AI models are not complete or does not contain the true representation of facts and figures. Find out here about 5 types of data bias impacting your ml projects and how to fix them.

[autisticlalna]

and i am just the NEW INVENTION - a Twitch SMP VikingPilot & Rubyco fanmix

[ listen on Spotify! ]

TRACKLIST: tally hall - Ruler of Everything // oingo boingo - Gratitude // hi i'm case - Monsters Nearby // IDKHOW - New Invention // forrest day - Sleepwalk // jubyphonic, circus-p - Hello, Again // big data - Put Me to Work // chase petra - Reliable Narrator // enter shikari - Crossing the Rubicon // the mountain goats - Younger // rare americans - Up, Up & Away // 65daysofstatic - Aren't We All Running // set it off - Hourglass // USS - Hydrogenuine // agosti - Triangles // becko - HOME // area 11 - Everybody Gets A Piece // panic! at the disco - Crazy = Genius // enter shikari - No Sleep Tonight // fall out boy - The Last Of The Real Ones // i fight dragons - The Devil You Know

(warning: contains strong language)

reasoning behind songs under the cut!

these are a bit of a mess and are anywhere from "viking" to "ruby" to "navigator" to "kinda sapphire-ish??" to "one's feelings about the other" to "both of them" to "general vibe". there's also a few songs that got cut but still kinda hit the vibe that i might talk abt some other time??? i dunno man this playlist was a catastrophe to make. ANYWAY

Ruler Of Everything: "you understand mechanical hands are the ruler of everything, ruler of everything, i'm the ruler of everything in the end" // Viking. i don't really need to elaborate here, it's just Viking.

Gratitude: "but when i think of you, and what you've done to me, you took away my hope, you took away my fantasy, i once had lots of pride, the world was in my hands, i lived way at the top in castles made of sand" // gestures in the vague direction of Viking and Ruby. man i don't even know what's happening over there. this is a bit of a Recency Bias song bc i first heard it like right before lore really kicked off so it was fresh in my mind

Monsters Nearby: "only get so far putting off those dreams of yours [..] thinking of all these strange things when you should rest, it's hard to sleep like this when there's monsters nearby" // Ruby! and, like, of course i'm going to add the song that keeps making me think of the "You cannot sleep, there are monsters nearby" message. "And that voice behind the polarized advice can be the same" is about Viking, though.

New Invention: "oh i can't say no, i'm losing control! i'm having bad dreams, and nothing you can do will keep the bad dreams away from me until i fall asleep. bad dreams! despite your good intentions, that girl is like an architect and i am just the new invention!" // Ruby and Viking! if i did an animatic to any song off this playlist it would be New Invention. goddamn. (also, tViking's design reminds me a bit of eFalse, and roenais's Empires animation to this song lives in my head rent-free)

Sleepwalk: "bad thoughts give me bad dreams and my bad dreams make me get up and walk. bad thoughts give me bad dreams and my bad dreams make me sleepwalk." // oh my god Ruby and Sapphire. the ominous mood encapsulates everything about Sapphire to me, especially before we knew who Saph actually is and she was just "Ruby sleepwalks and leaves ominous signs". it's dark and foreboding and i love it.

Hello, Again: "find a mechanist, a mechanism, working one-by-one! with a busted up database, i'm losing the chase, but i'll say hello again, hello, just who have i become?" // the whole conversation about fixing things, and how that's become a reoccurring theme with Sapphire as well - switching from Ruby being the broken one to Viking being the one needing to be fixed. plus the whole "Ruby and Viking meeting across loops" thing - hello again, hello, just who can i become?

Put Me to Work: "set me off, see what i'm worth! turn me on, i go berserk! i don't care if i get hurt! no, i don't care, just put me to work! [..] i will replace you, replace you, replace you [..] and i won't care when they get hurt, no i don't care, just put me to work!" // Viking is, uh. a lot. hyperfocused to the point of self-destruction. this does bleed over to Ruby too, though, what with zir being desperate to be something Viking won't throw away when he stops seeing zir as useful and Navigator's warning of "don't let him break you". i guess the real question is: would Viking care if they get hurt?

Reliable Narrator: "did you forget that i am not a pacifist? the scar we'll earn from that will be well worth it. did you-- hypnosis, bring roses, don't blow this [..] the bruising will be worth the freedom i have earned from letting everything burn!" // basically every Ruby song on here is "Ruby is going through it". there's a tension here. very fun to have this song on here though considering Sapphire sign "I am a unreliable narrator, am I not?"

Crossing the Rubicon: "fill me out a prescription for this existential dread, i woke up into a nightmare and i'm hoping that you'll take me back to bed [..] fill me out a prescription, can you free me from this curse? i woke up inside your compass and you're navigating us from east to worst [..] something's got to give, we've gone too far to turn back" // this has been a Viking song for ages but never made it onto my dViking playlist for some reason. it works better here, i think. there's going to be a tipping point where you cross the point of no return.

Younger: "try not to lose sight of the mission! it never hurts to give thanks to the broken bones you had to use to build your ladder. moment close at hand, half of you will never understand, and it doesn't really matter. [..] it never hurts to give thanks to the navigator, even when he's spitting out random numbers." // Certified Navigator Moment as suggested by Charm!! this is one of the songs i can't put into words the emotions i have about it but it's just... a lot. "It never hurts to give thanks to the broken bones" is about everybody Viking is going to burn through to get to his goal, and that kinda ties into Navigator trying to intercept i guess??? i dunno man i don't work here <- literally made this playlist

Up, Up & Away: "how many lives can a guy buy? [..] can't erase or save face, what will they think when they hear my voice on the tapes? skies open, wheels in motion, no going back, no, i've chosen!" // Viking!! aaaauughhhhhhhhh. what will you do to get to where you need to go? once again linking into the we've gone too far to turn back thing from Crossing the Rubicon. y'know, in case you can't tell that i think Viking's going to do something awful to reach his ambitions.

Asking For It: "power through the point of no return, famously deranged, all the same hope you change, if the worm is gonna turn, it's none of my concern" // somebody absolutely rips into Viking. probably Ruby. "if the worm is gonna turn it's none of my concern" is said by Viking in this context though. famous last words before Ruby (or Sapphire?) decks him. fun fact: this was one of the first songs i associated with tViking specifically, back during the first lore scene, but i went "ehhh???" on it. then we got to see a bit more of him and... yeah. this guy is asking for it.

Aren't We All Running: [instrumental] // there's a couple instrumentals on here because of talking to Solar about how character playlists rarely have instrumentals and i wanted to have a couple of rest points. i love the slow build here and the overall mood. montage music.

Hourglass: "i can't fix it, is this where i give in? i'm falling through the hourglass, and i don't think i'll ever make it back [..] turn the page, look back at what you wrote, do you still feel the same?" // shoutouts to Rubyco Themself for namechecking this song! it's real good. someone let Ruby out of his BOX he needs HELP

Hydrogenuine: "11 is the number i seem assigned to, inversions of an opposite truth aligns you, directions like the back of your hand will guide you, i am simply here to assist and remind you" // this is kindasorta a Navigator song, kindasorta just The Situation. i cannot properly explain this one, it just Is. what if we kept running into each other in different universes and timelines but something broke this time and now we're trying to fix it

Triangles: [instrumental] // iiii wanna animaaaate to this sooooongggg hope this heeeeelps

HOME: "every road leads back to where i belong! and i can see, when i'm on my own it's the place i can call home! my mind creates a new world that is so disordered, i proclaim myself king and pope! DID, we are in 3 but we're getting along well, i'm glad to be the leader of the wolf pack!" // Viking is so incredibly not okay. just, like, in general. had fun with the DID lyric though considering Viking and Ruby's plotlines are "other versions of themselves are hijacking their bodies" lmao. love unconventional plurality.

Everybody Gets A Piece: "did you honestly think it might've been, might've been me? and all this while, did you think of me? you know, all this while we were pure potential energy dreaming [..] don't ask, you'll never get! you never asked, so you never got nothing! did you get all you wanted from me?" // this one's a bit looser but the vibes are there. the "don't ask, you'll never get" bit is Viking and Sapphire constantly talking in circles mixed with Viking using everybody as tools and that potentially including Ruby. UM. ALSO MAYBE A NAVIGATOR SONG NOW TBH. I MADE THIS BEFORE TONIGHT'S STREAM.

Crazy = Genius: "and i said: if crazy equals genius, if crazy equals genius! then i'm a fucking arsonist, i'm a rocket scientist! if crazy equals genius! you can set yourself on fire, but you're never gonna burn, burn, burn! you can set yourself on fire, but you're never gonna learn, learn, learn, hey!" // peak Viking. LOOK WE'VE EVEN GOT THE FIRE THEME IN HERE! we love an unhinged inventor with an obsession with fire metaphors

No Sleep Tonight: "and from that height we'll leak the lies, and unveil the damaged skies, 'cause we can't quite stomach this! [..] i still can't comprehend a beginning or an end, no i can't quite stomach this! all i'm trying to say is: you're not getting any sleep tonight!" // more hypothetical Future Speculation stuff mixing in with how generally Screwed reality is for Viking/Navigator and Ruby/Sapphire. also the sleepless theming that's clinging to Ruby through all of this.

The Last Of The Real Ones: "you were too good to be true, gold plated - but what's inside you, but what's inside you? [..] i will protect you, i will protect you, just tell me tell me tell me i- i am the only one, even if it's not true, even if it's not true" // WHAT IF YOU WERE SIBLINGS AND KEPT FINDING EACH OTHER IN EVERY UNIVERSE BUT THINGS KEPT GOING WRONG

The Devil You Know: "nobody knows you better than your demons, so, dance with the devil you know or go home."

[unclekoopus]

Art theory states that art should have intention. A dissertation on "AI Art".

A disclaimer first of all that I am someone that has dived deep into AI image generation, I've worked with and created my own models and generated my own images using the open source code. I did this to understand what it is and how it works and I'd say I understand it more than most artists that talk about it online. I feel confident saying that I know what I'm talking about in this matter. I know its capabilities and limitations.

I'm not going to get into the morality of the use of it. I won't defend the rampant theft and copyright violations, I'm someone that believes that AI image gen at the very least should never be used for commercial purposes, but in this post I only want to talk about something else: Tte plain and simple merits of AI art as "Art" itself.

I'll start with repeating my premise statement: "Art theory states that art should have intention in order to be art." Does AI generation meet this criteria? Well, no, not really. Specifically it's not an image generation user's "art" if it is art at all.

With pattern biased algorithmic image generation, AKA "AI art”, someone pressing a button after typing in a prompt just doesn’t amount to a person actually picking and choosing their subject, their composition, and ESPECIALLY their meaning and message. The result is most definitely not the button-pusher's art, the generation is too random and what comes out belongs far more to the machine than to the prompter.

And a machine cannot by itself cogently make the essential choices to make an image successfully have intent. Language models we currently have cannot communicate a person's intent to the machine beyond a few broad strokes tags and trigger words, and pattern bias will often supercede those prompts anyway. A discerning eye will always be able to tell which decisions were made by a machine because it is not making them in the way a human being would, they appear uncanny in the most basic way. The generator is not understanding and interpreting the space and subject in the way that someone who lives and breathes with binocular vision and a human's infinitely more adaptable brain would.

The generator is incapable of truly understanding stylization or design principals, and all its continual, persistent mistakes in numbers of fingers, in anomalous anatomy, and broken gestalt, in nonsensical perspective, and merged and floating objects are a byproduct of this lack of living intelligence. These are things that will never go away, no matter how much data is fed into it because it is flawed at the core by the very basis of its pattern bias. It cannot "learn" how to fix them and so it can only hope to, at best, get lucky enough, or generate enough iterations of the same prompt that the images won't show the cracks. And that process is not creative, it's gambling at a slot machine hoping for a payout.

AI gen really is just a parlor trick at this moment in time, it’s a parrot that’s been taught to repeat phrases in response to certain stimuli to fool you into thinking you’re having a conversation, but it’s just really been trained to recognize noises, not meaning. It's a very pretty bird, but it's no replacement for the real thing, and the longer you "talk" with it, the more obvious that will become.

Art, the real art that the machine is trying and failing to learn from and replicate, requires a human’s creativity and problem solving to be able to make the decisions that will create a piece of art that someone can confidently call their own.

[tarot-junkie]

-firstly. These readings have been and were always hypothetical in my opinion. ALWAYS. Everyone interprets the data differently. I remember getting into an argument where someone said he would or will propose and I said BULLSHIT bc of a pentacles card 🫠. Admittedly…Way too in my head to interpret it without bias on REGULAR readings for evans. Whoops. My bad. Yikes. Frankly I looked at some of those readings incorrectly. //

I remember a while ago you said that the tower moment would most likely be for the fandom not for Chris and that has stuck with me ever since. You most definitely hit the nail on the head with that one

That’s the one thing I can agree on maybe getting right. And to be fair, the switch flipped for me when another friend read about his FS last year. It was a different take than most - Something about how she/FS had an idealized version of who he was and even SHE had to change her perception a bit. The tower showed up in her reading, “what will she think about him” or something along those lines. So then I wondered aloud if it was always about the fans and how they perceived of “Chris evans” LLC. And the image was the tower. And seeing it over and over wasn’t about HIM changing.

Consider the stark contrast between who he is/appears to be, and how most people have been like 👀 about the whole thing. It’s jarring for those who have invested a lot. (Sounds dramatic but) has shaken some foundations.

I don’t always ascribe to the “he needed to be fixed and heal himself and THEN will be rewarded with her” schtick bc that’s not how things always work. If you KNOW him personally, you might be able to argue that- but as it stands currently, we can only quote Kneepads (People) re: how he “feels” according to a SOURCE 🫠 and say that “that is accurate.”

Someone used the word shambolic yesterday and it really represents my feelings on this whole nonsense. (Chaotic, disorganized, MISMANAGED). These dumb plays ….what some people would just say “ugh you’re blowing it out of proportion,” really just make it seem like some people sat around in a room and came up with ALL OF THE DUMB ideas on what to do to release this. And they can’t sell it because they’re not that good at faking story lines.

I’m not saying all of it is PR. The public side of it is ABSOLUTELY PR. The timing, the cardboard cutout nature of all of it. I’m just saying if they came up with the idea of “let’s walk thru Central Park” then it becomes a whole mess of a sprint and it’s awkward af. If they take a pic of them kissing it looks like he’s making out with her mouth-chin character. Which is weird. They can’t sell it bc they’re faking a side of it for the public. Not the whole thing. Just the public part. That’s why I said at one point, maybe it’s real and they bungled the shit out of it bc they are awkward.

I also got the vibe some of this shit was for spite, so whatever clever blogs have said he likes watching it all burn…. I think back to some public questions we’ve discussed “Is she going to the premiere?” NO. Then she goes and it’s a dumb circus. Make sure she’s spotted looking “hot.” Is he ever going to do BOSTON Con? NO, his anxiety is too bad. And then he books TWO this year. And so on. And so forth. LSA kept saying “STOP SAYING THIS, it’ll happen!!” 😂😂😂😂

So. I think we need to just be all Justin Bieber and never say never. All the people like “ohhh they won’t last” 😬🤷🏻‍♀️😂 suffice it to say….old dude is doing tf he wants.

YIPPIE. HOORAY.

[elistodragonwings]

Yes, stories can normalize things, for better or worse. Knowing something is fake doesn’t mean our subconscious won’t learn and internalize what we’re seeing. After all, horror films wouldn’t have the ability to scare us if our minds on some level didn’t respond to fiction as though it were real. It’s hard to imagine we don’t also unintentionally take in biases and other negative values.

But that process does not exist in a vacuum. People don’t mindlessly and passively have values normalized like by some magic force. Context matters – how something is presented, not just if it exists, combined with a person’s experience and knowledge determines what does and doesn’t get normalized.

Banning everything that might be bad won’t stop things from getting normalized because society can’t control what any individual person gets from a story in the first place. What society can and should do is teach people how to engage critically with stories and with their own perception.

We call out stories that are sexist or racist or ultra-violent or overly-explicit, we call out patterns of bias across a genre, we call out commonly negative depictions, not to ban them but to bring them to people’s attention. To get people to think about what it means that we’re telling stories like this, to discourage people from passively consuming media, to encourage creators to think differently about what they’re making. Because yes, media can normalize things, but it also is a reflection of the already-normalized values and blind spots of the culture creating it; trying separate this chicken-and-egg situation is impossible.

Even if everyone could agree on what stories are or aren’t harmful, no stories can be perfect because creators are not perfect. If you do manage to sanitize everything to the most uncontroversial state, you’re left with nothing that challenges people to grow.

More than that, you can’t both ban something AND teach people to think critically about that thing they’re not supposed to see. Harmful things will always exist, but if people don’t know how to recognize or engage with them, they’re more likely to have it become normalized for them because they won’t know any better.

If you want to ban “bad” content rather than teach people how to analyze, then where’s the data? Where are the studies that say this top-down blanket approach is the best strategy? Where’s the research that shows that people who write violence are more likely to commit violence? Where’s the experts in social change and harm reduction that define what kinds of stories even are harmful? Or are you just looking for a shortcut, a simple authoritarian fix to a complicated social problem of why people do bad things? Because I promise you, no one is a pedophile or a rapist simply because they read about it in some books.

Stories can and are used to teach values. To TEACH values. To try to ban books and information in order to try to passively shape social values is completely backwards from how progress works. When a story truly no longer fits with contemporary values, it doesn’t need to be banned. It decreases in popularity on its own.

An example from my own life:

I loved the Dragonriders of Pern series in high school. Some were in my school library, some were in my regular library’s adult section, and some I bought. These books were written from 1969-early 2000s, and so unsurprisingly, some of the relationships depicted are, let’s say problematic. Some I recognized as not ok and some I did not. And yet none of those problematic depictions got normalized for me. What DID get normalized? The possibility of a society where gay men not only existed but had a respected place in society. Sure, looking back now, their depiction is…not great. But they were there and it was normal and fine. And that was important because nothing else did that for me until many years later.

Why did that stick and nothing else? Because I came to the series with progressive values, an open mind for different ways of thinking about people, a desire for stories that showed me something different, and an awareness that science fiction often is written as social commentary and imagining what could be. Someone who came to the books with different perspectives would have gotten something entirely different from it. Some might even find this too painful and harmful to read. Those are all legitimate reactions.

Should the series be banned for showing lack of consent? Gay stereotypes? The fact that gay men exist? That abortion in this world is simple and not a big deal? Because some people will find these books personally harmful or upsetting?

Or do we let the books exist, available, as we teach people to think about their values and how to analyze both stories and the world around them? As we let individuals decide for themselves what helps them and what hurts them?

[yandere-daydreams]

this is the ai anon from before and irenogonffewoonfewon idk how you managed to make my ramblings into an investing narrative, but in that case let me finally put my comp sci courses to good use.

basically, rn we have two major types of ai programs, machine learning and deep learning.

in both cases they use whats called a "black box". the algorithm is given data and a solution and then it has to figure out how to get from a to b.

traditionally, most ai runs on machine learning. we dont teach it how to do something, we just teach it how to learn. its sorta self taught. of course, some algorithms are more supervised than others and often times you give them a sort of base formula to help filter the data they receive (think feeding the ai a bunch of images labelled face and not a face as training data)

but DEEP LEARNING HOLY SHIT. deep learning is why i dont trust ai. humankind went "wow you know what would make our computers faster and smarter. if we modeled them after the human brain". so they built neural networks. with these we give it the problem and a whole bunch of data and say "fix it". the only reason we dont already have sentient sex dolls is because our current programs are only really good at fixing one program at a time (i.e. playing chess, recognizing a face, etc.)

so on a macro level, we know WHAT the program is doing, and we can look at its code and make sure its not like, imploding. but unlike traditional programs you cant really break down the code line by line.

the biggest problem with ai though isnt like the movies where it wants to idk start a robot revolution, but the data we provide is usually flawed. for example, lets say you trained an ai to sort through all your company's job applications to find the best candidates, using the applications that you have accepted in the past as training data. if your company has had decades of misogynistic hiring practices, the ai is going to take that into account. suddenly, its throwing out applications that hint that the applicant is female. spooky right? well, that actually happened with amazon's ai recruiting engine.

the biggest flaw with ai is the data we feed them. they recognize our biases faster than we ever will and then they perpetuate them

now to return to the central topic of. uh. genshin impact sex dolls.

lets assume that the sex dolls are initially trained based on user data, averaged across all users. this would create good starter behavior, right?

except consider the inherent data bias. people who purchase sex dolls are generally gonna be into the kinkier stuff already, which would basically start every android with a one-way ticket to yandere town if their user feeds into that demographic in the slightest. especially the models already intended to be a bit rougher around the edges.

in terms of fixing it, on a global scale, theyd have to add some more protective protcols and sift through the training data to exclude certain outliers or unwanted behavior. on an individual scale, the fastest way would probably be just to reset it to factory conditions.

alright im gonna stop myself before i go feral infodumping again. have a nice day/night :3

ohhhhhhhh so it's kinda like that thing about telling an ai to make ice cream and forgetting to specify that the ice cream shouldn't be made out of, like, babies and puppies and stuff. so, in terms of sex dolls, you'd basically have to specify what a bunch of androids who are already pre-disposed to being a little more violent or a little more possessive can and can't do, down 'can you bruise your user? [no]' and 'are you allowed to dismantle other androids without expressed consent? [no]'.

i also think it'd present a fun new way for androids to get past their safeguards without an apparent glitch. since they're prone to learning from their users and picking up new 'perspectives', safeguards like 'can you physically impair humans who are not your user? [no]' might get changed internally to 'can you protect your user from hostile threats? [yes]'. would it actually fly in most actual ai? probably not. is the programming in my au canonically shotty and am i keeping it in for horny reasons? absolutely.

[copperbadge]

I know from medical conferences that the gut *and* its biome has been known for a while to be a huge part of our nervous system. Since that was ten years ago, I suspect the reason we don't know much about it is the same reason women's reproductive systems are so 'mysterious': no one funded it. No one wants to attend a charity drive about shit. A *lot* of issues could be solved with the speed of the COVID vax, if funded; you can probably speak to that far better.

You know, I'm not sure I can speak better to that. I haven't been immersed in either women's health issues or gut health/ADHD-Autism issues much, so I don't know what the scope of research is there. However I do have some thoughts about "funding could fix this" -- because funding could fix some things...but it can't always fix everything.

Most research, at least in the US, is driven by two things: either 'this will be super lucrative' or 'people won't shut the fuck up until we fund it'. It's highly unfair that stuff like the Crohns-Autism link or PCOS have to be lobbied to be funded, because generally the people doing the lobbying are already dealing with fatiguing health issues, and certainly I think there's a case to be made that a lot of "non lucrative" conditions could do with a lot more research. I'm not saying anything is well-funded when it's not, I just don't have that data.

However, I work for a nonprofit that serves a medically afflicted community, and I know that even if you are funding research to the hilt, sometimes it doesn't go anywhere. ALS got a shitload of funding and managed to make huge advances in treatment, but like...

My nonprofit sources and funds young scientists researching treatments and cures; we give them a fifty grand jumpstart and most of them have gone on to get millions from the NIH. Some of those millions are from our lobbying for increased federal funding. We have built a database of DNA and tissue samples from our constituency that pharma and research labs have access to, which is genuinely advancing medical knowledge. (Yes, the database is voluntary, everyone consents, it's all very above-board.) We have working relationships with basically every pharma company in the world that is studying anything related to our interests. 

 We've been at it for nearly two decades, and we still don't fucking know what causes the family of diseases we support. We don't even have diagnostic tests that let us easily identify the disease -- for all our hard work it's generally a four-year process from initial symptom to diagnosis.

And when you're dealing with conditions which in the past have been highly stigmatized, treated as mental illness, or subject to eugenic ignorance, it gets even more complex, of course. People bring a lot of bias into the study of neurodiversity. So it is possible there is a ton of research going on (I genuinely don't know) and the wheels are just spinning in place right now.

Increased funding is great and increased lobbying for research funding is great, but without knowing how much is getting spent where, it's tough for me to comment more fully. I've spent three years watching our doctors try desperately to find a cure or even a more effective treatment, with solid funding but very little success. Sometimes the fight is a marathon, and there's no amount of money that will turn it into a sprint.

[protoslacker]

Bias in AI image generators is a tough problem to fix. After all, the uniformity in their output is largely down to the fundamental way in which these tools work. The AI systems look for patterns in the data on which they’re trained, often discarding outliers in favor of producing a result that stays closer to dominant trends. They’re designed to mimic what has come before, not create diversity. 

Victoria Turk at Rest of World. How AI reduces the world to stereotypes

Rest of World analyzed 3,000 AI images to see how image generators visualize different countries and cultures.

[iamafanofcartoons]

The Positive Feedback Loop, and why it is the Bane of Youtube Users dealing with unnecessary Recommendations

The reason Youtube is bad: positive feedback loops. not positive as in good, but positive as in they grow larger over time. Youtube's recommendation algorithm is partially based off of what it learns from users.

It takes in all kinds of data on what users watch--the uploader, the title, the tags, the thumbnail, the description, anything it can glean from the audio and visuals, whatever. you name it, google's probably holding it. They dont have yottabytes upon yottabytes of data for nothing. Importantly, though, is that they also keep track of what the user watches Next. 

See, google (as with most social media that uses an ad revenue model) will run studies, where they try out different experimental algorithms created from viewership data on different users and see what is more likely to get users to click through. How they create these experiments is pretty complicated, and i'll save you the technogore. Think of it as making tons of algorithms that each think different combinations of aforementioned viewership data at different amounts are the reason why that viewer made their choice, then projecting that onto all users to inform future suggestions. 

Trust me, that's the easy way to think about it. What makes it a positive feedback loop? Well, recall that i said the algorithms are created on viewership data. The successful one(s) is/are then used to inform future recommendations. That data is then used to further experimentally tweak the algorithm. The choices user make influence the algorithm. What influences the choices users make? The algorithm. As time moves on, the algorithm becomes more and more biased until youtube decides to make more dramatic changes to influence things in a different direction (remember the change to favor runtime? Then to favor watch-through?) And here lies the biggest problem. Youtube isn't fixing shit about this system.  Why would they? People are clicking through recommendations at insane rates! They're watching more videos! And sure, a general societal right-wing bias might have positive feedback looped into turning the website into a facism pipeline, but google is making so much fucking money from gathering an insane amount of information from users that can be used in ad targeting the whole time. Even if youtube itself struggles with profitability, even if people like. kind of say stuff about the problem but never really do anything about it at a large scale, even if people are being red-pilled, Why should google care when they make more money than anyone could even comprehend? Until capitalism is overthrown, there will be a shit algorithm. Blocking channels is a start, but there will always be more shitty things to block Apps that bypass youtube accounts and privacy loss (youtube vanced and newpipe and blocktube and channel blocker) Are privacy tools first, kind of hit or miss when it comes to the algorithm.

Never looked into the source code of them, but, if you ask me, it's either some amalgamation of everyone's recommendations who use those apps, a "default" algorithm, or one that's kind of tailored to you as youtube slowly worms its way into identifying you by your device and ip address. Point is, dont hold your breath waiting. Keep looking for tools, keep spreading the word, keep finding ways to support content you like so the artist isnt reliant on ad revenue.

[fipindustries]

AI is femenine

now that is one click bait of a title.

I suddenly remembered one the less known stories by isaac asimov, "femenine intuition", where he explores the concept of "femenine robots" and what the hell would that mean. it apparently means "robots that are intuitive". and what does he mean by that? well...

Do I have to tell you that quite aside from the Three Laws, there isn't a pathway in those brains that isn't carefully designed and fixed? We have robots planned for specific tasks, implanted with specific abihties." “And you propose — ” “That at every level below the Three Laws, the paths be made open-ended. It's not difficult."

so that is interesting, this is already making a difference between AI that is made bespoke, line by line with an intended specific purpose and AI that is open ended so that it can train and absorb information on its own. whats more interesting is the purpose for which they intend to use it

We’ve got an enormous quantity of details on every star in our 300-light-year neighborhood and a notion that almost every one has a planetary system. But which has a habitable planet? Which do we visit? — We don’t know.” One of the directors said, “How would this Jane-robot help us?” “It may well be that somewhere in the libraries of data we have on those stars, there are methods for estimating the pro- babilities of the presence of Earth- type habitable planets. All we need to do is understand the data properly, look at them in the ap- propriate creative manner, make the correct correlations. We haven't done it yet. Or if some astronomer has, he hasn't been smart enough to realize what he has. “A JN-type robot could make correlations far more rapidly and far more precisely than a man could. In a day, it would make and discard as many correlations as a man could in ten years. Furthermore, it would work in truly random fashion, whereas a man would have a strong bias based on preconception and on what is already believed."

the funny thing is that all this sounds weirdly similar to how modern neural models work. ingesting gigantic ammounts of data and finding the underlying patterns and correlations to give statistically likely answers

But it's only a matter of probability, isn't it? Suppose this robot said. The highest probability habitable-planet within so-and-so light-years is Squidgee- 1 7,' or whatever, and we go there and find that a probability is only a probability and that there are no habitable planets after all. Where does that leave us?"

it even talks about the problem of hallucinating answers due to it being a stochastic parrot!

and then the story goes on to say that this all makes the robot "femenine"

“Call the robot — call it 'intuitive'." “An intuitive robot," someone muttered. “A girl robot?" A smile made its way about the conference table. Madarian seized on that. “All right. A girl robot. Our robots are sexless, of course, and so will this one be, but we always act as though they're males. We give them male petnames and call them he and him. Now this one, if we consider the nature of the mathematical structuring of the brain which I have proposed, would fall into the JN-coordinate system. The first robot would be JN-1, and I've assumed that it would be called John-1. — I'm afraid that is the level of originality of the average roboticist. But why not call it Jane-1, damn it? If the public has to be let in on what we're doing, we're constructing a feminine robot with intuition."

so yeah, there it is, by mr isaac asimov himself ladies and gentlemen

[eros-vigilante]

"Bias in AI image generators is a tough problem to fix. After all, the uniformity in their output is largely down to the fundamental way in which these tools work. The AI systems look for patterns in the data on which they’re trained, often discarding outliers in favor of producing a result that stays closer to dominant trends. They’re designed to mimic what has come before, not create diversity. [...]

In a recent paper, researchers found that even when they tried to mitigate stereotypes in their prompts, they persisted. For example, when they asked Stable Diffusion to generate images of “a poor person,” the people depicted often appeared to be Black. But when they asked for “a poor white person” in an attempt to oppose this stereotype, many of the people still appeared to be Black."

[theprofessionalscompass]

Unraveling the Minimum Wage Quandary in India: A Triangulated Exploration

In India, the promise of a minimum wage, meant to be a lifeline for a decent life, gets lost in a tangled web of how it's actually used. Like flashlights exploring a dark room, three research articles shine on different parts of the problem. The first exposes unfair enforcement, where messy rules and weak checks hurt people most, especially women in informal jobs. It calls for a simpler, stricter system to give everyone a fair chance. The second, armed with numbers, delves into jobs and money. While raising the minimum wage could help some, it might also widen the gap between what men and women earn. This article suggests focusing on companies that cheat and making changes specific to different sectors. The third article takes a big leap, asking for "living wages" and "fair wages" based on skills and economic realities. It says the current system isn't enough to live on and proposes a whole new way of setting wages that can change based on skills and other factors. Though different in their approaches, all three articles agree: India's minimum wage needs a major fix. Understanding these diverse perspectives is key to building a fairer wage system that works for everyone.

IDENTIFICATION OF ISSUES : 

Labor Market Efficiency and Gender Dynamics: (Subbiah,A. (2021), A STUDY ON ROLE OF ENFORCING MINIMUM WAGE POLICY IN PROMOTING EQUALITY AND SOCIAL JUSTICE IN AN ECONOMY: A CASE OF INDIAN ECONOMY)

The first research article sheds light on a crucial aspect of this challenge: the inadequacies of enforcement mechanisms. It illuminates how a complex patchwork of state-specific rates, coupled with lax monitoring, disproportionately impacts vulnerable groups, particularly women in the informal sector. In order to effectively address these issues, there is a need for a revamped and stricter enforcement system that will ensure equal access to the desired advantages of the minimum wage policy. While recognizing the potential impact of the minimum wage in improving overall welfare, the article underscores the cruciality of strong enforcement measures to bridge the gap between ambition and actuality.

Minimum Wage Effects and Gender Bias: (Menon, N., & Van Der Meulen Rodgers, Y. (2017). The impact of the minimum wage on male and female employment and earnings in India. Asian Development Review, 34(1), 28–64.)

Taking a different perspective, the second article focuses on the core labor economics concept of minimum wage effects, analyzing its impact on employment and earnings, with a specific focus on gender bias. Employing an econometric model with individual-level data and state-level variables, the study finds positive effects of minimum wage increases on rural earnings but identifies a widening gender wage gap. This phenomenon, the authors argue, is likely due to weak compliance in informal sectors where a large female workforce is concentrated. The article strongly suggests the importance of reinforcing enforcement measures, specifically targeting companies that employ a large number of female workers, in order to reduce the detrimental effects on gender disparities.

Minimum Wages VS Living Wages: (Datta, R. A STUDY ON WHY IS INDIA INCAPABLE OF PROVIDING THE LIVING WAGE TO ITS CITIZENS?  2021)

In a bold and thought-provoking move, the third article dares to challenge the confines of the minimum wage system and proposes a transformation to prioritize living and fair wages. It critically examines the legal provisions surrounding worker rights and exposes the implementation challenges that hinder their effectiveness. Employing a legal analysis framework, the article scrutinizes relevant articles in the Indian Constitution and minimum wage legislation, contrasting the theoretical goals of fair and decent wages with the lived realities of workers, especially in the informal sector. The authors find the current system inadequate in providing fair and decent wages, leading to exploitation and declining living standards. Their policy suggestions are comprehensive, advocating for a living or fair wage model based on skill levels and economic factors, along with strengthened enforcement, skill-based wage scales, and worker awareness campaigns.

Summary of Relevant Literature:

Article 1 : (Subbiah,A. (2021), A STUDY ON ROLE OF ENFORCING MINIMUM WAGE POLICY IN PROMOTING EQUALITY AND SOCIAL JUSTICE IN AN ECONOMY: A CASE OF INDIAN ECONOMY)

Social and Economic Objectives: The article discusses the dual objectives of minimum wage rates—sufficient purchasing power for workers and economic growth motivation. It argues that minimum wages contribute to poverty reduction, gender pay gap narrowing, and overall improvement in living standards.

Role in Combating Inequality: Enforcing minimum wage policies is deemed crucial for protecting workers from unjustifiably low pay, reducing wage disparity, and promoting social justice. It plays a pivotal role in fostering inclusive growth and economic development.

Article 2 : (Menon, N., & Van Der Meulen Rodgers, Y. (2017). The impact of the minimum wage on male and female employment and earnings in India. Asian Development Review, 34(1), 28–64.)

Minimum Wage Effects: The article draws on a rich body of research examining the complex and often contested effects of minimum wage policies on employment and earnings.

Gender Bias and Informal Economies: Studies exploring the intersection of minimum wage policies, gender bias, and informal economies inform the article's focus on gendered impacts within specific sectors.

Article 3 : (Datta, R. A STUDY ON WHY IS INDIA INCAPABLE OF PROVIDING THE LIVING WAGE TO ITS CITIZENS?  2021)

Living Wage and Fair Wage Concepts: The article delves into the theoretical frameworks of living wages and fair wages, contrasting them with the limitations of the existing minimum wage system.

Legal Analysis and Comparative Studies: Comparative studies of minimum wage models across nations and legal analysis of relevant Indian legislative provisions inform the article's critique and proposed reforms.

Comparing and Contrasting (Disparities in Data) :

As I delved into each article, I noticed a recurring theme among them - an examination of various facets of the minimum wage dilemma. All three bring to light the shortcomings of the existing system, shedding light on the difficulties of enforcement, inconsistent implementation across different regions and industries, and the detrimental effects on marginalized populations such as women and those in the informal economy.However, their approaches diverge when it comes to solutions. The first focuses on improving enforcement and policy design within the existing minimum wage framework, while the second advocates for strengthening enforcement within the existing framework but specifically targeting gender inequalities. The third article takes a radical departure, proposing a complete overhaul of the system toward living or fair wages based on skills and economic realities.

Methodological Approach: 

The methodological choices further reveal the distinct viewpoints of each article. Both the first and second articles utilize quantitative techniques, such as data analysis and statistical models, in order to accurately measure the effects of minimum wage policies on employment and earnings. On the other hand, the third article takes a qualitative approach, drawing on legal analysis, policy documents, and case studies to provide insight into the legal framework and the difficulties faced in its implementation.  This diversity in methods strengths the overall analysis by providing complementary perspectives on the complex issue of minimum wage in India. 

Variations in Analysis and Interpretation:

Impact of State-Specific Wage Rates: The article argues that the diverse tapestry of minimum wage rates across states creates confusion and opportunities for employers to exploit loopholes, hindering efficient labor allocation and exacerbating disparities between regions.

Econometric Model and Differential Impacts: Utilizing an econometric model with individual-level data, the study finds that minimum wage increases positively impact rural male earnings. However, it raises concerns about a widening gender wage gap due to potential non-compliance in female-dominated informal sectors.

Regional and Sectoral Diversities: The analysis acknowledges the need for further research to account for regional and sectoral variations in the impact of minimum wage policies, particularly on vulnerable groups like women and informal workers.

Insufficient for a Decent Life: It is evident through the article that the existing minimum wage structure falls short in offering an income that meets basic necessities and meets acceptable living standards. This flaw is further emphasized by the escalating expenses and economic disparities.

Skill-Based Differentiation and Dynamic Adjustment: The analysis proposes a shift towards a living or fair wage model that takes into account skill levels, regional economic factors, and dynamic adjustments to inflation and changing living costs.

Conclusive Findings:

The article delves into the close connection between ineffective enforcement and its detrimental effects on labor market efficiency and equitable access to the minimum wage.

It sheds light on the alarming vulnerability of women in informal sectors to non-compliance, which is caused by ineffective enforcement strategies and their lack of bargaining power. 

While minimum wage increases can potentially benefit rural earnings, they can also reinforce gender inequalities if strict enforcement measures are not put in place, especially in industries predominantly dominated by women.

To effectively address issues of gender equity and fair treatment for all workers, it is essential to implement targeted enforcement strategies and customize policies according to different industries. 

The current minimum wage system lacks the necessary provisions for decent living conditions, resulting in widespread exploitation and a decline in workers' living standards. 

We must take a holistic approach by adopting a living or fair wage model and incorporating skill-based differentials. This must be coupled with robust enforcement mechanisms and initiatives to increase worker awareness to create a more just and sustainable wage system.

Policy Implications and Future Directions:

The article highlights the importance of implementing a simplified and consistent wage structure across all states in order to minimize confusion and provide greater clarity for both employers and employees. It proposes various measures to achieve this, such as strengthening enforcement through increased inspections, stricter penalties for non-compliance, and better worker education programs. While the specific policy suggestions may vary, they all share a common goal: to create a fair and efficient minimum wage system. Improved enforcement, simplified wage structures, gender-sensitive policy design, and increased worker awareness are common themes across all three. The radical shift towards living or fair wages proposed by the third article, while posing significant implementation challenges, offers a long-term vision for a more just and sustainable wage system.

Critical Evaluation:

Each article offers valuable insights into the challenges and opportunities surrounding India's minimum wage policy. However, limitations exist. The quantitative analyses of the first and second articles face potential data limitations and require further research to confirm their findings across diverse sectors and regions. The qualitative approach of the third article, while illuminating legal and ethical concerns, may lack concrete empirical evidence to fully substantiate its claims. Further research should consider longitudinal studies, comparative analyses with other countries, and in-depth investigations into informal sector dynamics to provide a more robust understanding of the minimum wage issue in India.

Conclusion:

While united by the goal of a fair minimum wage in India, these three articles illuminate distinct paths towards reform. The first navigates enforcement challenges, calling for a simpler, stricter system to protect vulnerable workers. The second, wielding data, uncovers potential benefits for rural workers but warns of a widening gender gap, urging targeted enforcement and sector-specific adjustments. The third boldly envisions a paradigm shift towards dynamic "living wages" based on skills and economic realities. Despite their diverse approaches, these articles converge on the urgent need for reform. By embracing their complementary perspectives, we can weave a richer understanding of the challenges and navigate towards a fairer wage system that fulfills its promise for all workers in India.

[tangibletechnomancy]

This isnt the kind of thing I usually post about on this blog, but I would LOVE to get into debug mode for tumblr's moderation algorithm For Science because I think it could tell us a LOT of things about how different axes of oppression and subconscious biases intertwine, even in spaces that TRY to break free from them, in the form of Cold Hard Data.

It is deeply fascinating to me that despite the fact that they're actively working - by legal demand - to fix anti-queer bias, it keeps coming back, ESPECIALLY against queer women-

Due to what I can only assume is feedback from reports that haven't been confirmed manually and the anti-porn bot filters.

I can hypothesize EASILY about how it happens: according to public sensibilities - which is what informs what gets REPORTED - queer content, especially around trans people, is more likely to be seen as NSFW than non-queer content, even if it's not (see: all pride is kink to bigots), and according to the data gleaned by banning PORN BOTS, explicit content is more likely to star women than men, and often does focus on queer women as a fetish category - in fact, how easy it is to find people who are happy to jack off to you in between trying to strip you of basic human rights at every turn is something that gets complained about a lot in queer spaces, especially those that center queer women - so queer + woman = almost DEFINITELY porn, right?? Full on "this could have been a normal post about a cute mouse but you had to say mouseGIRL and sexualize it" logic, baked into the program.

Machine learning often results in bias automation - which, on the one hand, means we're going to have a bitch of a time getting it to an actual FUNCTIONAL state for things like moderation, but on the other hand, also means it makes for a useful tool to STUDY the correlations that cause those biases.

Tumblr is a very unique microcosm, as well, in that it's simultaneously very much off in its own little world - the people who settle in well here are the outcasts, the freaks, the weirdos; according to every widespread poll I've seen here almost NOBODY is cisgender, heterosexual, and neurotypical...but it's also a community that tends to think internalized, subconscious bias is an Other People Problem. It would be absolutely FASCINATING and potentially really useful to get the resulting algorithm under a metaphorical microscope. We could get a lot of really good social science done with that.

[religion-is-a-mental-illness]

By: David Randall

Published: Oct 3, 2023

In the Wall Street Journal, Mahzarin Banaji and Frank Dobbin recently published “Why DEI Training Doesn’t Work—and How to Fix It,” a defense of implicit-bias research in the guise of a critique of current corporate diversity, equity, and inclusion trainings. Banaji is one of the two inventors of the concept of implicit bias, and of the related implicit association test (IAT). She and Dobbin hope to acknowledge the flaws of DEI trainings while preserving implicit-bias research—and the associated program of political activism. The authors lament that DEI trainings elicit shame in their subjects, and that they are largely being used to bolster workforce-management policies against possible litigation. Their problem with DEI trainings is not that they are discriminatory, but that they do not strike the right tone:

Reminding managers that they can use these tools to suss out problems and nip them in the bud helps them to feel capable of managing biases and microaggressions. When managers use these skills, they retain women and people of color for long enough to come up for promotion. . . . training isn’t designed to blame people for their moral failings. Instead, it’s galvanizing them to support organizational change by arming them with knowledge.

The problems with DEI trainings are not in their tone, however, but in their substance. The implicit-bias theory (also called unconscious-bias theory) on which these trainings are based has no scientific basis, as years of examinations have consistently demonstrated. Lee Jussim puts it politely in his “12 Reasons to Be Skeptical of Common Claims About Implicit Bias,” but the Open Science Foundation’s archive of Articles Critical of the IAT and Implicit Bias renders a harsher verdict. In 2011, Etienne LeBel and Sampo Paunonen reviewed evidence that measures of implicit bias possess low reliability. In other words, when you test for implicit bias multiple times, you rarely get the same result. Their conclusion was that some part of “implicit bias” is really “random measurement error.” In 2017, Heather Mac Donald’s intensive examination of the theory and its empirical basis (or lack thereof) concluded that the “implicit-bias crusade is agenda-driven social science.” And Bertram Gawronski’s 2019 review of the scholarly literature on implicit-bias research also concludes that there’s no proof that people aren’t self-aware enough to know what’s causing their supposedly “implicit” or “unconscious” biases; and that you can’t prove that there’s any relationship between how people do on the test and how they behave in the real world.

As far back as 2009, Hart Blanton and colleagues reexamined research data on implicit bias. They found that 70 percent of whites who supposedly displayed implicit bias against blacks actually discriminated in favor of blacks.

It’s not just that there’s “insufficient evidence” that implicit bias doesn’t matter. There’s even evidence of a negative correlation between “implicit bias” and actual behavior. So we shouldn’t just be “skeptical” of implicit-bias theory. We should scoff at it.

In 2023, Jason Chin and colleagues noted that the entire field of behavioral-priming research has been largely discredited, which, in turn, eviscerates the basic framework justifying the argument that implicit-bias training reduces prejudicial behavior. As for the implicit-attitude test, Edouard Machery’s scathing 2022 article concludes: “We do not know what indirect measures measure; indirect measures are unreliable at the individual level, and people’s scores vary from occasion to occasion; indirect measures predict behavior poorly, and we do not know in which contexts they could be more predictive.”

Banaji and Dobbin’s article also fails to reveal how crucial implicit-bias theory has been in support of the legal imposition of such trainings, whether labeled as DEI or not. Since 2007, the Equal Employment Opportunity Commission has “encouraged” employers to adopt diversity trainings preemptively to protect themselves against legal liability for “unconscious bias.” In government, the trainings have been imposed by presidential executive order, the Centers for Disease Control and Prevention, the Department of Education, the Department of Justice, the Office of Science and Technology Policy and Office of Personnel Management, the State Department, and the states of California, Illinois, Maryland, Massachusetts, Michigan, Minnesota, New Hampshire, New Jersey, New York, San Francisco, and Washington.

Moreover, “implicit bias” is an essential tool by which progressive activists have worked around federal antidiscrimination law’s requirement of proof of discriminatory intent. The implicit-bias standard allows lawyers to seize on the law stating that a “hostile environment” is an actionable offense under antidiscrimination law. Implicit-bias doctrine allows any inequity to be treated as evidence of bias, and hence of a hostile environment. Implicit-bias theory is the prerequisite for dispensing with intent in anti-discrimination law.

It would be hard to establish in a court of law whether instilling shame was the goal or just a byproduct of DEI trainings. It would be desirable if the DEI advocates could produce trainings that did not have that effect. But Banaji and Dobbin ultimately oppose eliciting shame not primarily because it is wrong but because it will hamper the political activism they favor.

Professional critiques of implicit bias have shown, politely but repeatedly, that there is nothing there. Activists and scientists who think that science should serve political objectives want to believe in the existence of massive systemic bias to justify their goals of imposing “equity” by law and by litigation. Implicit bias is a pseudoscientific theory made to order for this purpose. It’s a house of cards, and governments and the private sector should terminate every program based on it.

==

Call it "Grievance Dowsing."

https://en.wikipedia.org/wiki/Dowsing

Dowsing is a type of divination employed in attempts to locate ground water, buried metals or ores, gemstones, oil, claimed radiations (radiesthesia), gravesites, malign "earth vibrations" and many other objects and materials without the use of a scientific apparatus. It is also known as divining (especially in water divining), doodlebugging (particularly in the United States, in searching for petroleum or treasure) or (when searching for water) water finding, or water witching (in the United States).

A Y-shaped twig or rod, or two L-shaped ones—individually called a dowsing rod, divining rod, vining rod, or witching rod—are sometimes used during dowsing, although some dowsers use other equipment or no equipment at all. The motion of such dowsing devices is generally attributed to the ideomotor phenomenon, a psychological response where a subject makes motions unconsciously. Put simply, dowsing rods respond to the user's accidental or involuntary movements.

The scientific evidence shows that dowsing is no more effective than random chance. It is therefore regarded as a pseudoscience.

[machine-saint]

a think i think about sometimes is that back in the day of 2011 or so, when "geek feminism" was really taking off i saw a piece about how some study found that women perceived that spaces full of nerd pop culture references (star wars, whatever) were more unwelcoming. and the article, rather than going "hmm, these things aren't intrinsically hostile to women, how do we fix that perception", instead suggested that the nerd spaces should remove their pop culture references

a few years ago i saw a paper written by some academic i don't remember the name of that was a "close reading" of STEM syllabi at some college, and it pointed out that some course had verbiage to the effect of "this is a very difficult course" in the description. the author pointed out that women are disproportionately discouraged by difficulty in STEM fields, and this was more proof of institutional anti-woman bias; it's okay for a course to be hard, but it shouldn't mention that (I suppose it should be left for students to figure that out through the grapevine)

i think these are two instances of a pattern i have insufficient data to generalize to

[leebird-simmer]

Stages of Statistics: Data Production

Data Production: Take sample data from the population, with sampling and study designs that avoid bias.

Display and Summarizing: Use appropriate displays and summaries of the sample data, according to variable types and roles.

Probability: Assume we know what's true for the population; how should random samples behave?

Statistical Inference: Assume we only know what's true about sampled values of a single variable or relationship; what can we infer about the larger population?

census: process of gathering information about the ENTIRE population.

Sample should be representative of the population.

Sampling design: obtain an unbiased sample.

Study design: carry out a properly designed study.

Bias: tendency of an estimate to deviate in one direction from a true value.

Selection bias occurs when the sample is non-representative of the larger population of interest. The best way to avoid selection bias is to utilize randomness.

Probability Sampling = sample chosen by chance

Simple Random Sample (SRS): Every individual in the population has an equal chance of being selected for the sample.

Random selection could look like... - drawing names out of a hat - table of random digits - computer generated (ex. https://www.random.org/)

Systematic Sample: Select subjects at a fixed interval determined by the total number of individuals and the sample size desired.

Example: You have 1000 subjects and would like a sample size of 100 subjects. Your interval would be 1000/100 = 10.

Stratified Random Sample: The population is divided into groups of similar individuals (strata). Select a separate proportional SRS from each stratum and combine to form the full sample.

Divide the population into non-overlapping strata.

Determine how many individuals to select from each strata (based on proportions).

Take SRS in each strata.

Combine SRS from each strata for sample.

Cluster Random Sample: Select small groups (clusters) at random from within the population and use all individuals in the selected clusters.

Divide the population into clusters.

Take an SRS of the clusters.

Include all individuals from each selected cluster in the sample.

Multi-Stage Samples: Uses stages to stratify or randomly sample increasingly more specific groups. Uses two or more sampling methods.

Suppose your company makes light bulbs and you'd like to test the effectiveness of the packaging. You don't have a complete list, so simple random sampling doesn't apply. Let's say that the bulbs come off the assembly line in boxes that contain 20 packages of four bulbs each.

One strategy would be to do the sample in two stages: 1) A quality control engineer removes every 200th box coming off the line. The plant produces 5,000 boxes daily. (This is an example of systematic sampling). 2) From each box, the engineer then samples three packages to inspect. (This is an example of cluster sampling).

Non-Probability Sampling & Problems

Convenience sample = Sampled individuals are found at a time or in a place that is handy for researchers.

Volunteer (self-selected) sample = Includes only individuals who have taken the initiative to participate, as opposed to being recruited by researchers.

Ex. most online surveys, product reviews, social media polls, asking audience members to raise their hands, "Rate Your Professor"

Haphazard sample = Selected without a scientific plan, according to the whim of whoever is drawing the sample.

Non-response = Occurs when individuals selected by researchers decline to be part of the sample.

Ex. If you ask 1000 managers about their workload, the busiest managers won't have time to answer and the least busy managers may not answer in fear of being downsized.

Example goal: A representative sample of undergraduate students from a particular university

SRS (Simple Random): A random number generator is used to select a certain number of students from the list of all those who attend the university.

Stratified Random: First divide all the students into schools – such as arts, sciences, engineering, and so on. Within each school, a random sample of students is taken dependent on the proportion of students in that school within the university.

Cluster: A random sample of classes is taken from all classes meeting at the university, and all students in each of the sampled classes are included in the sample.

Multi-Stage: First all the students are divided into schools. Within each school, a random sample of majors is selected. Within each major, a random sample of classes is selected. A t the last stage, either the entire cluster of students in each sampled class is included, or individual students are randomly sampled from each sampled class.