#this is why i am single and should be content with academic validation

[sherlockig]

-

[hephaestiions]

you literally glorified infidelity in your wireless 2020, fic writers like you really need to die out before you damage the community at large

you know, i’ve cycled through many iterations of a response to this ask. 

first i thought, let’s respond with a bit of sass. let’s say something like, ‘bold of you to assume i haven’t already died out, that tumblr isn’t just a congregation of ghosts moaning about the lives they wish they had’ or ‘what’s a community’ or maybe even a screenshot of the actual definition of glorification (which, well, i do suggest you look up anyway). 

then i thought, why entertain it at all? this is my space, this is my blog, it’s my fic. i can delete this ask, turn off anons and be done with it. i would be within my rights to do that. 

i also thought many times of explaining the contents of my fic. of explaining myself. contemplated answering this with poetry that metaphorically explains the many many things wrong with this. 

but here’s what i finally settled on:

honestly, anon, i’m feeling a little salty. it seems to me that you want fandom to be a highly sanitised space that fits into your personal parameters of ‘safe’ or consumable. what concerns me about that, and about this particular genre of anon hate in general is– for some reason your safe and sanitised world does not exclude sending people comments such as “...[you] need to die out”. i would argue that suggesting someone needs to die is maybe exponentially more threatening and damaging to ‘the community at large’ than a tagged fic that includes a disclaimer stating i do not endorse the behaviour i am writing about, but hey! personal opinions, am i right? 

i’m not going to defend my fic. i don’t feel the need to.

but if you think fanfiction about infidelity, fictional work that does not automatically demonise individuals who do something awful as the worst kind of monsters to exist, will actively push people to cheat on their partners (which is what i am assuming your definition of ‘damage’ to be in this scenario)... i’m going to have to ask you to reevaluate. fiction is not validation. it is exploration in a world where there are no real life casualties or consequences. 

i cannot believe that people have to keep repeating this: sanitising fan spaces by censoring content you personally find offensive will not in any way make fandom safer.  

in fact, this is literally the kind of comment that puts people and the ‘community’ in danger. 

i’ll explain. 

imagine this: people listen to you. people say, well, the glorification of infidelity really is quite awful, quite traumatising, we ought to ban it. but who determines what glorification is? who determines what romanticisation is? the line is too blurry anon, the only way forward here is to ban infidelity in fics completely. 

now readers who found comfort and solace in reading infidelity fics with hurt/comfort, with forgiveness or with freedom, with a particular quote that resonated with them, readers who turned to this fic at 3 am... they’ve lost it too. 

but hey, let’s take it further still, because that’s what this type of censorship will open fandom up to. let’s talk about how this will validate people who find content with any exploration of nsfw themes, r*cism, mis*gyny, homoph*bia, transph*bia, r*pe offensive. let’s talk about how they will approach the showrunners of censorship with the same argument: this content could be traumatising, could be validating for someone’s darker impulses, let’s ban it. and it all gets banned. 

enemies to lovers, banned because it might be construed as abusive. a/b/o, might be construed as misogynistic or codependent. hmm. high-school/college, oh god, that could be underage! soulmates? where’s the agency?! kid-fics? a minefield, any parenting is bad parenting to parents who do not parent that way. mcd? let’s not even go there. body-swap? the consent issues!! 

“alright. what about ‘there was only one bed’?” “well, i mean, i’m really not comfortable with that, it’s really creepy how writers will just force characters who don’t want to consent to that kind of intimacy into each others’ spaces.”

now i’m not saying that your personal opinions on these tropes, these themes, these topics is invalid. i’m not saying that if you’re triggered or made uncomfortable by these things, you should still read them. i’m saying that enforcing your personal preferences and takes on every single person in a community is not what critical consumers of media do, it’s what fascist and authoritarian governments and abusive individuals who do not understand boundaries do. 

here’s the truth about media that antis don’t like: there are antis for the media they actually do like. you ban something, you open up the goddamn floodgates for fluff antis to present their cases about how making characters ooc in fluff can be highly traumatic to people and how it should be banned. you start this brand of censorship, you risk losing fandom and the community you are so concerned about me damaging entirely. 

i’ll leave you with a realisation i’ve come to by reading all the posts made by people better with words than i am: the fastest way to create a dystopia is for someone to think their version of utopia is universal.

further reading for folks who might be interested in tumblr posts about fanfic: 

fanfiction as a collective exists as a combination of the ideal state and all the broken pieces that are left behind.  fanfiction: the stories mainstream showrunners won’t tell. for the people trying to make fandom a better place.

coexistence of hurt and healing

there are thousands of other such posts, articles, books, academic articles, maybe even interviews on the subject, and if anyone wants to send a few mine (and anon’s) way, feel free!

[epinosicc]

This is going to be quite chaotic, but this is something I wrote late one stormy July night about my life this far and how I’ve realized my problems

Okay it’s around midnight where I’m at so it’s time to rant instead of sleeping because I’m a minor and I have ✨issues✨

I tend to think more than what’s probably considered healthy, mostly because I do t have people to talk to. Don’t get me wrong, I have friends, but I don’t know what they’re doing and I don’t want to burden them with my stupid problems. So like any sane person I write my problems on the internet.

I usually think about the weird things when it’s raining. It’s something about the sound and feeling of rain that makes me more content, which makes me think. Now, I don’t have any big problems by any means. I’m simply figuring myself and my life out.

First of all, my previously mentioned friends. I trust them, of course, but at the same time I don’t. And like many who think to much and have a strange amount of self-awareness, I think I know why that is. When I first started going to school, I was confident. I’d already had friends before and thought I knew how to make new ones. The problem with that is that said friends did not go to my school, so I was alone. Until I met my first two friends. They were very nice to me, we played together and got along. The thing that I started noticing though was that if me and one of them arrived at school around the same time and out third friend wasn’t there yet we’d get along great, but as soon as that third friend arrived I’d get ditched in favour of them. And that would obviously hurt me. But we resolved it (not really) and things were going fine. But that experience stuck with me. It was my first taste of loneliness and abandonment (dramatic much?) and it made me doubt myself. I thought that maybe, just maybe, there was something about me that they didn’t like.

Now jump ahead about two years and I was alone Every. Single. Recess. (Oh shit it storming outside right now and some thunder sounded like a bomb) Obviously this only made me feel worse about myself. I just remember being so desperate for some sort of connection with someone. And I got one. I started talking to this person, I’ll call them Bird, and we got along great. Pretty soon Bird was my best and we spent a lot of our time together. I was still sort of friends with the two other people, at least during lessons, and sometimes during recess, but not that much otherwise.

Jump ahead a bit more, another year or so, and my class changed. At my school my class and another (same age as us) we’re combined into one. In this class that we were combined with there were a few new people, one of whom stuck out. Mostly because they didn’t like me, and they weren’t exactly discreet in letting me know. They never said so to my face, but they made it quite clear in how they acted towards me. This also made me feel bad. Is there really something so wrong with me that others couldn’t help but dislike me for it? Can I fix it? What do it that makes me different? (At the current point in my life I’m fairly certain I know what it is so yeah. Fun)

Now, I’d always cared a fair bit about school. I was taught that education was important, and if I was going to spend hours at school I might as well use that time for something, be it academically of socially. So when those around me started caring less about their education and more about things such as appearance and social hierarchy and relationships, I was confused. Why would they just not care? HOW could they just not care? Now, I’m not saying that any of the previous things are necessarily bad things to care about. In fact, ist great! Being invested in your social life and how others view you can be nurturing and make you feel fulfilled. But too much of anything can be bad. Letting yourself care about only those things can be harmful in more ways than one. I’ve never particularly cared about those things; I don’t like dressing up or making myself look good for others. I don’t value others validation of my appearance. What I didn’t notice was that as I believed these thoughts, I started eating less.

But things are still pretty chill. I still struggle with what’s wrong and what makes me different, but that’s fine. I’m pretty sure everyone goes through that at some point in our lives. But now I’m starting to find some answers. I don’t really care much for my appearance or style, I like academic things, I’m starting to fall behind in my social development, people are becoming more bold in stating their opinions, people are more hateful and spread misinformation etc etc (there’s a fucking mosquito who won’t leave me alone fuck off please). And at this point I’m more invested in the online world. But the international online world, not my national online world if that makes sense. English isn’t my first language but I learned it from the internet/YouTube and it’s basically my second language at this point. I learned English for English content creators, and I continued following them, not the ones relevant in my home/country. So I was and still am kind of out of the loop on current influencer events here in the North. This ties in with what I thought to be the answer to my questions: the LGBTQIA+ Community.

I started finding creators from the LGBT+ and I related to them and their stories. But I didn’t think I was one of them. People at school were not afraid to boldly proclaim that being LGBTQ+ was wrong and bad and strange. That there was something inherently rotten about such people. Now, did I agree with that? No. But I let it influence to the point were I thought that others being LBGTQ+ was fine, but me being that wasn’t. I wasn’t aloud to be one of them because there wasn’t supposed to be something wrong with me. But there was something, in the back of my mind, some part of me that knew. That knew who I am and that being me was fine. Too bad that voice wasn’t loud enough.

I still had Bird with me. Granted, they also had other friends, but they still stayed by my side. And they didn’t change like others did. My two first friends are people I also grew closer to at this time. I put our “situation” behind me and ignored it. It was a new chapter of my life, one where thing were changing in the right direction. Too bad I wasn’t too good at reading maps.

At this point I’m in sixth (6th) grade, the worst grade/period/time of my life thus far. After summer break people had changed a lot. Not just socially, but physically as well. We started to mature, we were lite tiny birds, looking out of the nest and thinking about how to take flight and reach above the branches of expectations and reach the clouds of ambition. But some of us didn’t. We didn’t want to start using our wings. At most we took a little peek out of our nest and divided that was enough for now. We began to grow frightened of others and their strange ideas of leaving what we knew was safe. I’m We for those wondering.

I started struggling with anxiety, I couldn’t stand in front of people without being scared and had a few panic attacks during presentations. People would look at me weirdly and I grew paranoid of what was wrong with me. At this point I started eating even less, resigning myself to one potion per meal, and no snacks, sometimes skipping lunch. Once again some of my friends that I had at this point started drifting away from me but now the rest, and I started trusting them even less. I can’t help but think that they’re only pitying me, that they’re going to leave and that they do thing behind my back. There was also someone else who had a big influence on me.

I, along with Bird started hanging around this person, we’ll call them Pen. They were sort of new, they’d always been in our class but had been living abroad for eight (8) months and had just come back. At first things were great. Bird, Pen and I were our own little trio of friends. But soon a change occurred. Pen started getting more clingy, staying uncomfortably close at times and never staying out of our personal space. Bird ended up taking the initiative with one of our other mutual friends and had long talk with Pen which sort of ended their friendship. At first they’d all handled it alone but then Pen involved their parents and thing went downhill. But I wasn’t part of it. Which made Pen hang on to me even more. I could never get away from them, it always felt like they were breathing down my neck. I didn’t tell them this though, they just lost two friends and they must be hurt from it, seeking comfort from someone they still considered a friend. I was uncomfortable, but I felt bad for them, so I continued being around them. Something my teachers had realized at this point was that I tend to take responsibility for other and their actions, and told me that I should try to relax and talk to them as I had seemingly started to become overwhelmed. But I don’t tell others my problems so I didn’t take their help. This kind of escalated a bit next grade.

Grade seven (7) was not my best year but also not my worst. I spent summer break reflecting and thinking, and started to value myself a bit more. I started hanging out with friends more often (usually Bird), and started unintentionally ignoring Pen. Though sometimes, I think it was intentional, as the very thought of Pen at this point made me anxious and uneasy. I thought I could simply let Pen hang around with me, and then let them get their own new friend group. I didn’t want them to only hang around me, it was honestly a bit scary how much I dreaded being around them. The feeling that something was off or wrong around them wouldn’t go away. They didn’t leave me though. No; I became their sole friend whom they refused to leave. In seventh (7th) grade our class was split, with me and Bird being in different classes. I had some friends in my new class though andere became a group. I thought I could nudge Pen to become part of this group. Except that Pen didn’t interact or contribute to the relationship. They weren’t social enough with the group to become part of it, standing in the group only to follow me. And my teachers noticed this and spoke to me. I told them how I was uncomfortable around Pen, and how I would like to not have to sit close to them next time we switched we seats (done every few weeks or so). Teachers agreed. But didn’t follow through. They sat me Right. Next. To. Pen. I confronted them about this. They lied to me. Their reasoning was that one of Pen’s parents had told the teachers how Pen only felt comfortable around me, and that they would like for us to be together at school as much as possible.

I was horrified at this - I couldn’t be held responsible for another students comfort, grades and social life! They basically put all the responsibilities of the teachers - making sure students felt comfortable, helping with schoolwork when needed, making sure the student had friends in the class - on me! I was basically supposed to play friend, teacher and class for Pen! I honestly couldn’t believe it, and told my friends. They told me they understood completely - they could see how emotionally and mentally exhausted I was from taking care of Pen, studying, after school activities and being around people that they were concerned about my well being. They, too, had tried to get Pen to become part of the group, but when only one person is taking care of the ship you can’t expect it to sail. They also felt uncomfortable around Pen. My anxiety only got worse because of this, and I started becoming paranoid that Pen was always watching me, either through my phone or my windows. I could not get myself to relax, not even when totally alone, something I’ve always enjoyed and felt comfortable with.

And at the end of grade seven (7), it happened. I found out that Pen was switching schools. I feel guilty admitting it, but I felt so relieved and free when I found out. Finally, I thought, finally I would get some privacy. All of my other friends are aware of my boundaries: don’t touch me unless I’m ready and aware of it, give me some space, don’t force me to talk when I’m anxious etc. They know, respect and treat me well, and in turn I treat them well and respect their boundaries, but Pen didn’t seem to understand that no, I don’t want you to stand so close to me that I can literally feel you body heat.

So grade eight (8) rolls around and I so does a certain unspecified virus. We therefore had to have school online. For me this was a blessing. I don’t enjoy being around people for too long and I don’t ever want to deal with my classmates bs. The teachers even commented on several occasions that I seemed much happier, which I was considering I didn’t have someone constantly breathing down my neck. And now I start to drift away from Bird. I always considered Bird my absolute closest friend. Almost like a sibling. And now we were drifting apart. We both started walking our own paths, still close together but different in so many ways. We’re still friends to this day, but I don’t think our friendship is going to last until we’re adults anymore. It’s sort of sad, but it is natural. We are both starting to forge our own paths in life, our own docks from which we will eventually set sail from to explore the limitless blue beyond that is life. And one day we might even meet again on some distant island, reconnecting and sharing stories of calm blue oceans to storming black waters. But that will happen with time. For now, I’m content finding materials for my dock with my group of friends, sharing ideas for designs and unfinished blueprints of a distant future. I’m content staring at that great far away horizon painted in the colors of pink, magenta and blue, watching the clouds of today’s events and feeling the winds of tomorrow’s surprises whilst thinking of what one day might be.

TL;DR: I rant about my life and somehow become a poet at the end.

End note - I still struggle with trust and anxiety. I don’t have problems with how my body looks anymore and I don’t confine myself to strict diets and eating schedules. Part of me feels guilty about my situation with Pen, and one part of me feels relieved and happy that I don’t have to deal with them anymore. I’m smart enough and self aware enough to realize my problems and their causes, and I have the tools to craft my solutions. I’m doing good, and know how to keep doing good, at least for a little while more.

[saltoftheao3]

I’m going to wade into the muddy waters of the concrit debate. 

I’ve read a lot of posts about people hating concrit, being discouraged by it, not wanting to write anymore because of it and it makes me wonder, if instead of asking *readers* to interact with the writer’s work in a very narrow, unequivocally complimentary and positive way, should writers maybe try to get better at accepting concrit and not letting it discourage them? 

I mean, no one is defending the unambiguously nasty comments of ‘this sux’ or ‘this is boring’ or whatever. But something like what the writers typically talk about as concrit, which is a commenter expanding (in a reasoned way) on some part of the story that they perceive as needing improvement. 

I’m showing my age here, but getting better at handling what other people think of you (rather than just running away from it) is kind of part of life. Facing negative feedback and criticism (dolled out with varying degrees of tact by friends/professors/parents/bosses/clients/complete strangers) is kind of baked into daily living, and most of it is not coming from a place of nastiness and cruelty. And being able to handle it without folding in on yourself and giving up completely is a very important coping mechanism to develop. You don’t have to follow it, or appreciate it – just learn to deal with it and move on. 

And I know that people think of writing fanfic as a hobby and an escape from reality, but the internet is not a personal spa or sanctuary. Once a writer puts that work in a very publicly-accessible place like AO3 or tumblr, then they have to accept that they’ve given up control over who reads it and how they react to it. It’s the trade off for *having* this platform to share works in the first place. 

Now, full disclosure, I write fanfic and I obviously greatly prefer the positive/effusive comments over concrit – who doesn’t! I mostly get positive comments in my fics (not because I’m always awesome, but because I don’t write that much/my fics are not that popular). For the odd concrit (which like most of the writers, entails an engaged reader and some positives and negatives mixed in – NOT nastiness/hate, which is important not to conflate with concrit) I either engage with the reader to discuss further, or just say a curt ‘thank you for reading’ and try to shrug it off. It stings for sure. And its hard not to be discouraged. But I feel like giving up because of negative comments would say a lot more about me than about the commenter.

(submitted by anon (i took the liberty of adding paragraph breaks to enhance readability, hope that was ok!) 

Thank you for this elaborate and thoughtful ask, anon. I share, or have shared, a lot of your thoughts, and am not in the mood to try and rebuke something i don’t even really disagree with. So instead i’ll just get needlessly sociological and analytical as to why the concrit debate is more prominent and vocal than it was, as far as i can tell from my narrow vantage point, in the past, and why it is, in my humble opinion, a rather good thing. Please indulge me :p

(obviously this is not academic paper material, right, obligatory reminder that i’m just some internet rando)

What’s the concrit debate? To sum it up, it’s fanfic writers saying “dear stranger on the internet, your unsolicited critique of our free work of love is not appreciated and hurts, so please shut up”. 

1. So have writers just become more sensitive?

To some extent, i think .. .yes. Because we live in a time in which, for lot of us, every single aspect of our lives is scrutinized, judged, compared. Not just in a professional context, but also what we eat, how much we move, how we look, what we say, how we spend our downtime. Not only are public and personal spheres far more entangled than they were one or two decades ago through the rise of social media; there’s also this compulsion to get your life right on every level, to enhance every aspect you can enhance, to “market yourself”. Neoliberalism baby. 

Long story short, i think a lot of us DO feel in need of sanctuaries, of judgment-free rooms for fun and creativity where they won’t be judged and told how to do better, be better. And are more sensitive to critics because we feel as if everything in our life is already subjected to critic, because this all-encompassing scrutiny wears us down, leaves us with no more protective shells. Are fandom spaces the right room for that? Probably not. Doesn’t stop a lot from wishing they were, tho. 

2. Why can’t writers seem to just “suck it up” anymore?

And that’s where imo it gets more political. I think we’ve reached a level of, let’s say societal awareness, that every single one of us has a right to their emotions, a right to feel hurt, to get angry. And that it’s valid to let others know that they hurt you, so they can avoid doing it in the future. That rather than swallowing down the hurt, or shrugging it off, we can try to let others know not to hurt us in the first place. We don’t want to suck it up anymore. We see that with content warnings, with safe spaces, etc: people are getting more assertive in saying, “I feel this way X, so I’ll communicate to others i feel X and why I feel X, and if my well-being matters to them then they’ll act in consequence”. And i think this is where the concrit debate has its roots, in that evolution. 

Yeah, so those are the somewhat incoherent thoughts i have on that matter. Other ideas, people?

[mittensmorgul]

do you think it's open to interpretation whether dean and cas are in love with each other? Like is it just as valid an interpretation to say they're not? Whenever anyone calls destiel "one interpretation" or whatever, my hackles rise. And I know I'm overly sensitive about this stuff, being a gay and whatnot, but I mean, is it? Am I just insecure because my otp isn't canon, or is destiel really more valid than other readings or what? What do you think?

Hi there. :)

I’m gonna give you the diplomatic, academic answer, and then I’m gonna give you the grumpy-ass queer lady answer. Hold on to your horses. :)

Polite answer:

All media is open to interpretation. Of course, this doesn’t mean that all interpretations are equally valid, or equally supported by canon, especially when taken in context of the entire body of the work in question.

For example, I replied to a post the other day about 13.17, and that scene where Dean and Sam are-- on first glance-- rather disrespectful of the extremely rare and valuable books in the bunker... but in context of the rest of the episode and the rest of the season, that montage wasn’t about disrespecting those books at all. It had less than nothing to do with the books themselves as objects or as sources of knowledge that should be properly cared for and respected. But out of context it kinda looks that way. So, based on that one short gif set, it might seem like a perfectly legitimate interpretation to suggest that Sam and Dean were careless with the immense knowledge and invaluable books they’ve found themselves in possession of. But in the larger context of their entire history, of all their interactions with the bunker and the untold store of knowledge it holds,  and with the context of the specific reasons for their frustration in that particular scene, it seems obvious that there’s a lot more to the story, you know?

You could technically argue just about any weird headcanon can be supported by canon. I wrote this weird little post right after 12.11 aired, and it sat in my drafts for a good long time before I finally posted it. But there’s nothing in canon that legit quashes the possibility that endgame fish!Cas is where the story’s been headed all along. He’s positively swimming in fish metaphors. (sorry, I couldn’t resist) Does that abundance of fish, fishing imagery, and water imagery that have surrounded Cas for years lend itself to a literal interpretation? I mean, it’s definitely AN interpretation that is there if you want to see it, and if in your heart of hearts you believe it’s legitimately what the storytelling is attempting to convey here. But does that make it a valid interpretation that deserves serious consideration? Does it truly make sense when taking the larger story around Cas as a whole? Or is it obviously a literary theme that we’re supposed to consider through the themes traditionally associated with fish and fishing as used in countless other fictional works of the past? I suppose that sort of interpretation has been left open for us to take or leave as we see fit. It invites us to examine those references more closely, to help us understand Cas as a character and the journey his personal character arc is taking him through. It gives his experiences and growth a depth of context that is there to explore if we so choose.

(for more on Cas vs Fish, please see my tags regarding “The Fisher King.” I like to think there’s a more well-reasoned and logical line of thinking for pinning so much fish to Cas than my cracky example of fish!Cas would suggest.)

Now, looking at destiel specifically, if you take any single moment out of context, it’s absolutely possible to make an interpretation that their relationship is clearly more “brotherly,” or clearly more “familial,” or clearly one of “very close friends.” But it requires the same removal from the larger context to explain away what taken with the entirety of their history begins to look entirely undeniable.

I suppose, since Supernatural is an open canon and the story hasn’t been fully told yet, that it’s possible the writers could change course with the storytelling. It’s possible that something might prevent them from taking Dean and Cas and their story to the conclusion they’ve been building to for the last ten years. They could decide to leave this particular “interpretation” open-ended and unresolved.

Since that is always a possibility, and because I’m not psychic, nor do I have any top secret inside information from the writers and showrunners, I can’t say that my particular interpretation is more valid or correct or likely than anyone else’s. But I have yet to come up against a credible, coherent explanation for the entire body of extant canon that invalidates my particular interpretation, either.

The vast majority of arguments against boil down to logical fallacies-- cherry-picking scenes out of context as “proof,” straw man arguments, and ad hominem attacks. Because of this, I’m content to wait for canon to play out. I’ll happily watch the rest of the story unfold, and happily continue to interpret what I’m witnessing as a whole instead of attempting to dissect it out and explain away what I see as an entirely logical progression of storytelling.

As an aside here, I find it entirely fascinating that one of the most common complaints I read from people who deny Dean and Cas are in love is that the writing has become progressively more terrible, that the story of Supernatural as a whole makes less and less sense, and that the characters are behaving in increasingly “out of character” ways. And as someone in possession of rational capabilities, I wonder if their disconnect from the storytelling is simply their refusal to see and accept that perhaps their “interpretation” of the story is just... not correct.

When we attempt to deny or rationalize away certain interpretations of characterization, or certain progressions of events and how they relate to one another, the larger narrative just falls apart, you know? Of course it doesn’t make sense if you exclude large portions of it because you don’t want to see it or believe it’s happening, or important to the story.

Meanwhile, I’m over here loving every minute of it (okay... most minutes of it). So even if my interpretation isn’t absolutely 100% “correct” (and really, with any media, there’s always different ways to interpret everything, from what the color of the curtains might imply to who’s gonna get to fire Chekhov’s Gun in the third act), I’m content to continue to interpret it in a way that not only makes me personally happiest, but in a way that makes the story itself seem both logical and entertaining, as well.

Okay, that’s the end of the rational portion of this essay. Now on to the angry queer lady portion:

There’s more canon evidence for Dean and Cas being in love, or at the very least caring for one another to ridiculous, rather mind-numbing degrees, than there is for practically every canon heterosexual couple on television in the last fifty years. Think of any slow burn, will they-won’t they hetero couple, and do the point-by-point checklist of all the tropes they burned through before they got to the love declarations and the kissing and the happily ever afters (or worse, the dramatic breaking up and getting back together, or even worse, the tragically breaking up forever). I challenge anyone to name one hetero-presenting couple who required as many love tropes for audiences to recognize and acknowledge they were in love. Yeah, I’m thinking of that whole “they shared a pencil” post.

So yeah, there is likely a measure of heteronormativity to it, and a lot of the arguments against also devolve into rather gross denouncements that there’s no way Dean’s not straight, because he said so that one time... Mr. “I lie professionally” who also never actually said he was straight... gah... I’m not gonna dig up every ancient meta post on the subject. If anyone is legitimately interested in understanding why making those same tired arguments just doesn’t have any legitimacy in a reasoned discussion, they can damn well do their own digging. It’s not like any of the evidence is difficult to uncover, and it’s not my job to spoon feed it to every naysayer myself.

I feel like I’m standing on a Mt. Everest size pile of rational, reasonable, well-argued analysis supporting the claim that Dean and Cas are in love. *stands back and points at my whole entire blog again* If anyone would like to come back at me with something even remotely worth my time and attention to persuade me to alter my interpretation, I suggest they get busy. I’ll just be up here on top of my mountain enjoying the clean, destiel-scented air up here.

And finally, who says it’s not canon? Ah, right. Moving goalposts. At this point, I think it’s ridiculous to suggest that Dean and Cas don’t love one another. And profoundly, at that. I mean, you don’t give up an army for one guy if you don’t at least like him a lil bit. You don’t shout down God begging him to bring back that dude you’re kinda buddies with, or sink into a suicidal funk that reverses completely within minutes of finding out said buddy’s alive again. You don’t offer to march to your death with your chum because he’s such a nice guy and all. I mean... honestly. How far in denial does someone have to be to suggest they don’t love each other? At this point, when comparing Sam and Dean’s reactions far into s13 to Cas’s death in 12.23, either you accept that Dean has much stronger and far different feelings about the loss of someone that Sam does love and considers a brother, or else you kinda have to assume that Sam’s just kind of a dick for not being as broken up about Cas’s death as Dean is. So... which interpretation do you think is the one they’re trying to convey?

Bleh, whatever. I await the inevitable inbox full of nastiness that I will cheerfully delete while judging every anon who sends it as someone who really should find a better hobby than antagonizing strangers on the internet over a work of fiction.

Anon, basically, don’t let the bastards grind you down, okay?

Now for some reason I feel like listening to Achtung Baby. Imma go do that and feel the love.

[infochores]

Leftism, the DPRK and the Nuclear crisis

I thought id write a thing on this because most of the current far left talking points on this issue are terribly poorly informed. There’s a tendency amongst leftists online, particularly notable on twitter, but also here to conflate two kinds of commentary on contentious geopolitical issues into being one form of sin. Obviously the most current example is regarding the Singapore summit. The way its currently playing out in a lot of cases is basically as follows (I’m being a little snarky here but this is a fairly accurate play by play for a lot of the stupider stuff):

Prominent analyst, journalist, politician or Liberal pundit makes a statement criticizing the deal for a given reason. It might either be ordinary partisanship, or a genuine technical criticism of the content of the circumstances. 

It gets retweeted

People comment on  it in the retweets, claiming that the given person is in favour of  Koreans getting exterminated, because all objections to the circumstances must be borne of a desire for war or something.

The difference between multiple forms of criticism gets totally ignored by the left, we dont learn anything and we continue patting ourselves on the back.

Obviously this is fairly normal for online stuff but even so i think we need to start paying much closer attention to these issues and at the same time, stop being so parochial in our thinking about issues such as these. Therefore I’m minded to make a few points here about why its  short sighted to interpret this via the lens of western domestic political leftist rhetoric, the problem being that there’s severe limitations on that lens. 

Firstly its primarily based on established rhetorical forms that are largely out of date or constrained by a lack of room for outside information. This prevents us from usefully adopting lines of analyses from schools of thought not traditionally connected to existing normal stances within leftism. On the occasions where this does occur, we co-opt it for our own benefit. Essentially, If there’s a form of analysis, or an area of academic or technical expertise where the information and commentary is not directly subordinate to a conservative, US-centric interpretation of anti-imperialism then it tends to get completely discounted. The main example of this type of thing i wish to talk about is an area of interest of mine, namely arms control, disarmament and non-proliferation, as well as the place of nuclear weapons within a fairly basic Marxist influenced viewpoint on nuclear politics. 

Secondly, this stance is limited by the dynamics of online sectarian politics and the desire to produce content. Everyones familiar with this and knows how it works so im not going to bother going into this in any great detail. Essentially its providing a great addition to our respective mutual firing squads.

A third issue is the presumption of knowledge, and of moral or intellectual high ground on a combination of ideological bases. This latter point ties into an ongoing dynamic of orientalist thinking and presumptions that tend to seep into leftist rhetoric on subjects surrounding the Korean Peninsula. This is notable particularly within Marxist-Leninist  self styled “anti-imperialist” discourses, which are the main area of failure on all three of the issues ive listed, though there has been cross-pollination with other leftist sects. This behavior is reliant on making the nuclear crisis All About Us, unconsciously sliding over the human and political realities in order to establish credentials, ignoring regional factors beyond what is read about in a few articles and a surface level adherence to popular conceptions of anti imperialist praxis. Its essentially a highly banal form of watered down orientalism mixed with a vague presumption of superior politics. The reality is that we havent really learned anything from the crisis, because we’ve refused to. Arms Control, Non-proliferation and a lack of far-left institutional knowledge: I’m mainly going to address the first issue, that of a general lack of understanding of evolving dynamics in nuclear proliferation issues and arms control within the far left because its where i have the strongest base of knowledge relating to this matter. Functionally, nearly no-one in the circles of the online left I’m in actually knows how arms control is meant to work either in its form as a policy enacted between countries on a mutual basis, or in its form as a system in the world of geopolitics within which countries formulate policies. Im not trying to grandstand here as this circumstance is totally understandable: its a very niche topic and its not as if I’m particularly special for having made it something I’m interested in, but regardless there is a knowledge gap in leftist discussions on this issue and it seems important to highlight it. If there are any leftist arms control nerds that happen to see this assuming it gets more than a handful of notes by all means let me know. Anyway, I think its wise at this point to make the following statement and to explain it:

In the particular example at hand, namely this weeks Singapore summit and the deal made there, what has just happened is very likely a disaster in the long term, and should be absolutely regarded as such by the left. This is due to a combination of technical and political factors relating to the context and structure of the agreement. I am saying this as a Marxist and as someone in favour of internationalism and popular movements directed towards nuclear disarmament. Leftist institutional knowledge and the general knowledge in the community of how arms control works isn't really that great. During the cold war it was ok, due to the size and importance of the disarmament movement, but in the modern era its essentially non-existent. Anyway, the summit was a disaster for arms control and very few of us seem to get why. This is for a few reasons, but the main one is that it has established a precedent for successful, aggressive  strategies of proliferation under extreme duress. Arms control as a concept has taken a serious beating since 2003 and the Iraq war, with other major points of inflection since. The most significant of these have been the collapse of the JCPOA agreement with Iran  and the various chemical warfare atrocities in the Syrian civil war. The collapse of the Libyan state stands as its own special case as well for its own reasons. In the case of the recent summit the precedent set has been one of the solidification of certain dynamics, of the transition undertaken by nuclear weapons from being articles characterized by their technical use into being primarily characterized by a capacity for abstract political use in a way that had not yet been achieved, and of the deliberate manipulation and degradation of existing institutions in order to facilitate proliferation. I am not merely talking about Nuclear weapons being used for political purposes. I am talking about a very specific type of political use. This dynamic is extremely bad and provides a model for action to accompany the diminishing difficulty in producing nuclear weapons, which will provide a great incentive to states seeking to proliferate nuclear weapons. The only outcome possible here is the furthering of the development of weaponry to be used against populations and in so doing, the armory of empire. This is not a victory for Anti-Imperialism, despite its surface appearance having had great propaganda value.

The deal has essentially entrenched both the (many) negative and the (few) positive aspects of the present crisis, essentially providing cover for the ROK to continue a gradualist approach toward achieving a sustainable end result at the same time as providing a model of success for proliferation as a state policy. In fact i would argue that one of the main contributions that this will all make to geopolitical history will be the establishment, testing, and success of a new model of proliferation, one with profoundly negative though interesting consequences from a leftist perspective. Essentially, by forcing the nuclear issue, by making all of the political capital centered upon repeated escalating crises, comprising a protracted greater crisis, the DPRK strategy meant that all political questions were to be decided according to a single signifier of value: the technical usability and validity of its nuclear weapons, demonstrable to all onlookers from the outside. A value which the DPRK leadership were able to be the controlling agents of, determining its magnitude, qualities and effects. In doing so it has essentially managed to manipulate the existing notions of arms control agreements and non-proliferation principles in order to turn those structures to its advantage: to turn them from being systems which constrained nuclear proliferation, to using them as a basis for updating the nuclear weapon into taking on a modern form of its already infamous role as the ultimate political commodity. It was essentially a strategy of folding up all issues into one and forcing everyone's decisions to rest upon that single focused issue. It is now far easier to proliferate nuclear weapons and ballistic missiles in terms of technical restrictions than it was 30, 40, 50 years ago. The designs are no-longer unknown and the required production technology is no longer as challenging as it once was. Provided with this, directed efforts towards various kinds of proliferation now have new potentials: the end state no longer has to be one option from a three way fork between a failed program, a successful program and being attacked/sanctioned. Obviously nuclear weapons have always been used as chips to bet on and trade between states, but for the most part those items, in the historical contexts of the cold war, the India/Pakistan standoff, and in the cases of Israel, Iran and the DPRK prior to recent developments, were developed for strict military terms, as items of deterrence, as physical items for physical use as instruments of policy in the most literal sense. In other words, they have been pursued by states for their use either as weapons, articles of deterrence or as a combination of being usable weapons and articles of abstract political power realized. This has now changed, likely irreversibly, and the summit has been a notable moment in the change.

The realization of abstract powers, and the risks of a new Nuclear credit:

What the new development has done is to introduce the potential for the transformation from nuclear weapons as weapons into a sort of proto-commodity form, whereby proliferation, as a behavior that states may engage in and as a form of production existing in a political sphere, has been brought to a such point of technical and political development that it may now be used not merely as an action engaged in to achieve a physical end, a productive approach to achieving a state policy, but takes on a new meaning, as a form of bargaining chip in itself. Elements of this have been present with nuclear weapons since the cold war in  relatively isolated minor forms but never reached the stage of proliferating for the sake of trading against that proliferation.

I believe that the summit has laid the groundwork for the establishment of such a dynamic  and I suspect that the fallout from the summit and the ongoing crisis ( which is not at all over) will now exacerbate this tendency in future cases. It is as if they have succeeded in laying the groundwork for making nuclear weapons not merely commodity items between a state and its industrial complex, but have extended their existence as commodities right into the geopolitical sphere. The truly perverse thing about this is that this change has been achieved without the DPRK even having to actually physically trade any of them: it has merely made a vapor-deal based on vague promises to limit the most obvious cases of its potential power.  In this manner another well established dynamic has been reinforced: The confrontational relationship between the instability of deterrence and the long term tendency evolving from it for states to engage in proliferation. The repeated historical cases of the seeming success of deterrence, (a false semblance) have created a climate of encouragement for proliferation: existing nuclear weapons states are encouraged to expand and proliferate new powers to counter their antagonists advances, whilst non-nuclear weapons states begin to seek the weapons to achieve parity on various fronts. Against this backdrop, both the repeated crises in deterrence, the breakdowns of its various forms, political crises between states etc, and the longer periods of time between them, the long moments of false security, serve to enhance the fetishized value of deterrence. The illusory success of deterrent strategies encourage the proliferation dynamic. Deterrence will therefore, if i am correct, become a contributing factor in commodity-led proliferation. But now a new aspect of this dynamic has appeared: the change in nature of arms control into also becoming a contributing feature. Prior to this arms control was a diplomatic, technical activity, with strict technical aims. It may still take on that characteristic but is at severe risk of being held, in measure of value, against the new measuring standard that i fear is being established.  This situation is profoundly negative for leftist political positions: the brief lull in tensions may be whisked away by a new return to sabre rattling at any point if political capital is to be made, ushering in more risks of accident miscalculation or catalytic war and further it is a massive blow to the disarmament movement. It will certainly serve to entrench national military industrial complexes in nuclear states, as well as further contributing to regional problems in areas where proliferating states are players. Holding all this to be down to the blunderings of one leader and the energy of another is a useless way to think about this issue. This is to the detriment of internationalism. 

[kristinsimmons]

Docs are ROCs: a simple fix for a “methodologically indefensible” practice in medical AI studies

By LUKE OAKDEN-RAYNER

Anyone who has read my blog or tweets before has probably seen that I have issues with some of the common methods used to analyse the performance of medical machine learning models. In particular, the most commonly reported metrics we use (sensitivity, specificity, F1, accuracy and so on) all systematically underestimate human performance in head to head comparisons against AI models.

This makes AI look better than it is, and may be partially responsible for the “implementation gap” that everyone is so concerned about.

I’ve just posted a preprint on arxiv titled “Docs are ROCs: A simple off-the-shelf approach for estimating average human performance in diagnostic studies” which provides what I think is a solid solution to this problem, and I thought I would explain in some detail here.

Disclaimer: not peer reviewed, content subject to change 

A (con)vexing problem

When we compare machine learning models to humans, we have a bit of a problem. Which humans?

In medical tasks, we typically take the doctor who currently does the task (for example, a radiologist identifying cancer on a CT scan) as proxy for the standard of clinical practice. But doctors aren’t a monolithic group who all give the same answers. Inter-reader variability typically ranges from 15% to 50%, depending on the task. Thus, we usually take as many doctors as we can find and then try to summarise their performance (this is called a multi-reader multicase study, MRMC for short).

Since the metrics we care most about in medicine are sensitivity and specificity, many papers have reported the averages of these values. In fact, a recent systematic review showed that over 70% of medical AI studies that compared humans to AI models reported these values. This makes a lot of sense. We want to know how the average doctor performs at the task, so the average performance on these metrics should be great, right?

No. This is bad.

The problem with reporting the averages is that human sensitivity and specificity live on a curve. They are correlated values, a skewed distribution.

The independently pooled average points of curved distributions are nowhere near the curves.

What do we learn in stats 101 about using averages in skewed distributions?

In fact, this practice has been criticised many times in the methodology literature. Gatsonis and Paliwal go as far as to say “the use of simple or weighted averages of sensitivity and specificity to draw statistical conclusions is not methodologically defensible,” which is a heck of an academic mic drop.

What do you mean?

So we need an alternative to average sensitivity and specificity.

If you have read my blog before, you would know I love ROC curves. I’ve written tons about them before (here and here), but briefly: they visually reflect the trade-off between sensitivity and specificity (which is conceptually the same as the trade-off between overcalling or undercalling disease in diagnostic medicine), and the summary metric of the area under the ROC curve is a great measure of discriminative performance. In particular the ROC AUC is prevalence invariant, meaning we can compare the value across hospitals even if the rates of disease differ.

The problem is that human decision making is mostly binary in diagnostic medicine. We say “there is disease” or “there is no disease”. The patient needs a biopsy or they don’t. We give treatment or not*.

Binary decisions create single points in ROC space, not a curve.

The performance of 108 different radiologists at screening mammography, Beam et al, 1996.

AI models on the other hand make curves. By varying the threshold of a decision, the same model can move to different places in ROC space. If we want to be more aggressive at making a diagnosis, follow the curve to the right. If we want to avoid overcalls, shift to the left.

The black line is the model, the coloured dots are doctors. From Gulshan et al, 2016.

As these examples show, groups of humans tend to organise into curves. So why don’t we just … fit a model to the human points to characterise the underlying (hypothetical) curve?

I’ll admit I spent quite a long time trying various methods to do this, none of which worked great or seemed like “the” solution.

I’m not alone in trying, Rajpurkar et al tried out a spline-based approach which worked ok but had some pretty unsatisfying properties.

One day I was discussing this troubling issue with my stats/epi prof, Lyle Palmer, and he looked at me a bit funny and was like “isn’t this just meta-analysis?”.

I feel marginally better about not realising this myself since it appears that almost no-one else has thought of this either**, but dang is it obvious in hindsight.

Wait … what about all those ROCs of docs?

Now, if you read the diagnostic radiology literature, you might be confused. Don’t we use ROC curves to estimate human performance all the time?

The performance of a single radiologist reported in Roganovic et al.

It is true, we do. We can generate ROC curves of single doctors by getting them to estimate their confidence in their diagnosis. We then use each confidence level as a threshold, and calculate the sensitivity and specificity for each point. If you have 5 confidence levels, you get a 5 point ROC curve. After that there are established methods for reasonably combining the ROC curves of individual doctors into a summary curve and AUC.

But what the heck is a doctor’s confidence in their diagnosis? Can they really estimate it numerically?

In almost all diagnostic scenarios, doctors don’t estimate their confidence. They just make a diagnosis*. Maybe they have a single “hedge” category (i.e., “the findings are equivocal”), but we are taught to try to avoid those. So how are these ROC curves produced?

Well, there are two answers:

It is mammography/x-rads, where every study is clinically reported with a score out of 5, which is used to construct a ROC curve for each doctor (ie the rare situation where scoring an image is standard clinical practice).

It is any other test, where the study design forces doctors to use a scoring system they wouldn’t use in practice.

The latter is obviously a bit dodgy. Even subtle changes to experimental design can lead to significant differences in performance, a bias broadly categorised under the heading “laboratory effects“.

There has been a fair bit written about the failings of enforced confidence scores. For example, Gur et al report that confidence scores in practice are concentrated at the extreme ends of the ranges (essentially binary-by-stealth), and are often unrelated to the subtleness of the image features. Another paper by Gur et al highlights the fact that confidence scores do not relate to clinical operating points, and Mallet et al raise a number of further problems with using confidence scores, concluding that “…confidence scores recorded in our study violated many assumptions of ROC AUC methods, rendering these methods inappropriate.” (emphasis mine)

Despite these findings, the practice of forced confidence scoring is widespread. A meta-analysis by Dendumrongsup et al of imaging MRMC studies reported that confidence scores were utilised in all 51 studies they found, including the 31 studies on imaging tasks in which confidence scores are not used in clinical practice.

I reaaaaally hate this practice. Hence, trying to find a better way.

Meta meta meta

So what did Lyle mean? What does meta-analysis have to do with estimating average human reader performance?

Well, in the meta-analysis of diagnostic test accuracy, you take multiple studies that report the sensitivity and specificity of a test, performed at different locations and on different populations, and you summarise them by creating a summary ROC (SROC) curve.

Zhang and Ren, a meta-analysis of mammography diagnostic accuracy. Each dot is a study, with the size of dot proportional to sample size (between 50 and 500 cases). Lines reflect the SROC curve and the 95% confidence interval.

Well, it seems to me that a set of studies looks a lot like a group of humans tested on a diagnostic task. Maybe we should try to use the same method to produce SROC curves for readers? How about Esteva et al, the famous dermatology paper?

This is a model that best fits the reader results. If you compare it to the average (which was reported in the paper), you see that the average of sensitivity and specificity is actually bordering on the inner 95% CI of the fitted model, and only 4 dermatologists perform worse than the average by being inside that 95% CI line. It certainly seems like to SROC curve makes more sense as a summary of the performance of the readers than the average does.

So the approach looks pretty good. But is it hard? Will people actually use it?

Is it even research?

I initially just thought I’d write a blogpost on this topic. I am not certain it really qualifies as research, but in the end I decided to write a quick paper to present the idea to the non-blog-reading community.

The reason I felt this way is that the content of the paper is so simple. Meta-analysis and the methods to perform meta-analysis is one of the best understood parts of statistics. In fact, meta-analysis is generally considered the pinnacle of the pyramid of medical evidence.

Metanalysis is bestanalysis.

But this is why the idea is such a good solution in my opinion. There is nothing fancy, no new models to convince people about. It is just good, well-validated statistics. There are widely used packages in every major programming language. There are easily accessible tutorials and guidelines. The topic is covered in undergraduate courses.

So the paper isn’t anything fancy. It just says “here is a good tool. Use the good tool.”

It is a pretty short paper too, so all I will do here is cover the main highlights.

What and why?

In short, a summary ROC curve is a bivariate model fitted on the logit transforms of sensitivity and specificity. It comes in two main flavours, the fixed effects model and the random effects model, but all the guidelines recommend random effects models these days so we can ignore the fixed effects versions***.

When it comes to the nuts and bolts, there are a few main models that are used. I reference them in the paper, so check that out if you want to know more.

The “why do meta-analysis?” question is important. There are a couple of major benefits to this approach, but the biggest one by far is that we get reasonable estimates of variance in our summary measures.

See, when you average sensitivity and specificity, you calculate your standard deviations by pooling the confusion matrices across readers. Where before you had multiple readers, you now have one uber-reader. At this point, you can only account for variability across samples, not readers.

In this table, adapted from Obuchowski in a book chapter I wrote, we see that the number of readers, when accounted for, has a huge impact on sample size and power calculations. Frankly, not taking the number of readers into account is methodologically indefensible.

SROC analysis does though, considering both the number of readers and the “weight” of each reader (how many studies they read). Compare this SROC curve re-analysing the results of Rajpurkar and Irvin et al to the one from Esteva et al above:

With only 4 readers, look how wide that confidence region is! If we draw a vertical line from the “average point” it covers a sensitivity range between 0.3 and 0.7, but in their paper they reported an F1 score of 0.387, with a 95% CI of 0.33 to 0.44, a far narrower range even accounting for the different metric.

Another nice thing about SROC curves is that they can clearly show results stratified by experience level (or other subgroups), even when there are lots of readers.

From Tschandl et al. The raw reader points are unreadable, but summarising them with SROC curves is clean and tidy.

There are a few other good points of SROC curves which we mention in the paper, but I don’t want to extend this blog post too much. Just read the paper if you are interested.

Just use SROCs!

That’s really all I have to say. A simple, off-the-shelf, easily applied method to more accurately summarise human performance and estimate the associated standard errors in reader studies, particularly of use for AI human-vs-machine comparisons.

I didn’t invent anything here, so I’m taking no credit^, but I think it is a good idea. Use it! It will be better^^!

You wouldn’t want to be methodologically indefensible, right?

* I’ll have more to say on this in a future post, suffice to say for now that this is actually how medicine works when you realise that doctors don’t make descriptive reports, they make decisions. Every statement made by a radiologist (for example) is a choice between usually two but occasionally three or four actual treatment paths. A radiologist who doesn’t understand the clinical implications of their words is a bad radiologist.

**This actually got me really nervous right after I posted the paper to arxiv (like, why has no-one thought of this?), so I email-bombed some friends for urgent feedback on the paper while I could still remove it from the processing list, but I got the all clear :p

*** I semi-justify this in the paper. It makes sense to me anyway.

^ Well, I will take credit for the phrase “Docs are ROCs”. Not gonna lie, it was coming up with that phrase that motivated me to write the paper. It just had to exist.

^^ For anyone interested, it still isn’t perfect. There are some reports of persistent underestimation of performance using SROC analysis in simulation studies. It also doesn’t really account for the fact most reader studies have a single set of cases, so the variance between cases is artificially low. But you can’t really get around that without making a bunch of assumptions (these are accurate empirical estimates), and it is tons better than what we do currently. And heck, it is good enough for Cochrane :p^^^

^^^ Of course, if you disagree with this approach, let me know. This is a preprint currently, and I would love to get feedback on why you hate it and everything about it, so I can update the paper or my friends list accordingly :p

Luke Oakden-Rayner is a radiologist in South Australia, undertaking a Ph.D in Medicine with the School of Public Health at the University of Adelaide. This post originally appeared on his blog here.

Docs are ROCs: a simple fix for a “methodologically indefensible” practice in medical AI studies published first on https://wittooth.tumblr.com/

[lauramalchowblog]

Docs are ROCs: a simple fix for a “methodologically indefensible” practice in medical AI studies

By LUKE OAKDEN-RAYNER

Anyone who has read my blog or tweets before has probably seen that I have issues with some of the common methods used to analyse the performance of medical machine learning models. In particular, the most commonly reported metrics we use (sensitivity, specificity, F1, accuracy and so on) all systematically underestimate human performance in head to head comparisons against AI models.

This makes AI look better than it is, and may be partially responsible for the “implementation gap” that everyone is so concerned about.

I’ve just posted a preprint on arxiv titled “Docs are ROCs: A simple off-the-shelf approach for estimating average human performance in diagnostic studies” which provides what I think is a solid solution to this problem, and I thought I would explain in some detail here.

Disclaimer: not peer reviewed, content subject to change 

A (con)vexing problem

When we compare machine learning models to humans, we have a bit of a problem. Which humans?

In medical tasks, we typically take the doctor who currently does the task (for example, a radiologist identifying cancer on a CT scan) as proxy for the standard of clinical practice. But doctors aren’t a monolithic group who all give the same answers. Inter-reader variability typically ranges from 15% to 50%, depending on the task. Thus, we usually take as many doctors as we can find and then try to summarise their performance (this is called a multi-reader multicase study, MRMC for short).

Since the metrics we care most about in medicine are sensitivity and specificity, many papers have reported the averages of these values. In fact, a recent systematic review showed that over 70% of medical AI studies that compared humans to AI models reported these values. This makes a lot of sense. We want to know how the average doctor performs at the task, so the average performance on these metrics should be great, right?

No. This is bad.

The problem with reporting the averages is that human sensitivity and specificity live on a curve. They are correlated values, a skewed distribution.

The independently pooled average points of curved distributions are nowhere near the curves.

What do we learn in stats 101 about using averages in skewed distributions?

In fact, this practice has been criticised many times in the methodology literature. Gatsonis and Paliwal go as far as to say “the use of simple or weighted averages of sensitivity and specificity to draw statistical conclusions is not methodologically defensible,” which is a heck of an academic mic drop.

What do you mean?

So we need an alternative to average sensitivity and specificity.

If you have read my blog before, you would know I love ROC curves. I’ve written tons about them before (here and here), but briefly: they visually reflect the trade-off between sensitivity and specificity (which is conceptually the same as the trade-off between overcalling or undercalling disease in diagnostic medicine), and the summary metric of the area under the ROC curve is a great measure of discriminative performance. In particular the ROC AUC is prevalence invariant, meaning we can compare the value across hospitals even if the rates of disease differ.

The problem is that human decision making is mostly binary in diagnostic medicine. We say “there is disease” or “there is no disease”. The patient needs a biopsy or they don’t. We give treatment or not*.

Binary decisions create single points in ROC space, not a curve.

The performance of 108 different radiologists at screening mammography, Beam et al, 1996.

AI models on the other hand make curves. By varying the threshold of a decision, the same model can move to different places in ROC space. If we want to be more aggressive at making a diagnosis, follow the curve to the right. If we want to avoid overcalls, shift to the left.

The black line is the model, the coloured dots are doctors. From Gulshan et al, 2016.

As these examples show, groups of humans tend to organise into curves. So why don’t we just … fit a model to the human points to characterise the underlying (hypothetical) curve?

I’ll admit I spent quite a long time trying various methods to do this, none of which worked great or seemed like “the” solution.

I’m not alone in trying, Rajpurkar et al tried out a spline-based approach which worked ok but had some pretty unsatisfying properties.

One day I was discussing this troubling issue with my stats/epi prof, Lyle Palmer, and he looked at me a bit funny and was like “isn’t this just meta-analysis?”.

I feel marginally better about not realising this myself since it appears that almost no-one else has thought of this either**, but dang is it obvious in hindsight.

Wait … what about all those ROCs of docs?

Now, if you read the diagnostic radiology literature, you might be confused. Don’t we use ROC curves to estimate human performance all the time?

The performance of a single radiologist reported in Roganovic et al.

It is true, we do. We can generate ROC curves of single doctors by getting them to estimate their confidence in their diagnosis. We then use each confidence level as a threshold, and calculate the sensitivity and specificity for each point. If you have 5 confidence levels, you get a 5 point ROC curve. After that there are established methods for reasonably combining the ROC curves of individual doctors into a summary curve and AUC.

But what the heck is a doctor’s confidence in their diagnosis? Can they really estimate it numerically?

In almost all diagnostic scenarios, doctors don’t estimate their confidence. They just make a diagnosis*. Maybe they have a single “hedge” category (i.e., “the findings are equivocal”), but we are taught to try to avoid those. So how are these ROC curves produced?

Well, there are two answers:

It is mammography/x-rads, where every study is clinically reported with a score out of 5, which is used to construct a ROC curve for each doctor (ie the rare situation where scoring an image is standard clinical practice).

It is any other test, where the study design forces doctors to use a scoring system they wouldn’t use in practice.

The latter is obviously a bit dodgy. Even subtle changes to experimental design can lead to significant differences in performance, a bias broadly categorised under the heading “laboratory effects“.

There has been a fair bit written about the failings of enforced confidence scores. For example, Gur et al report that confidence scores in practice are concentrated at the extreme ends of the ranges (essentially binary-by-stealth), and are often unrelated to the subtleness of the image features. Another paper by Gur et al highlights the fact that confidence scores do not relate to clinical operating points, and Mallet et al raise a number of further problems with using confidence scores, concluding that “…confidence scores recorded in our study violated many assumptions of ROC AUC methods, rendering these methods inappropriate.” (emphasis mine)

Despite these findings, the practice of forced confidence scoring is widespread. A meta-analysis by Dendumrongsup et al of imaging MRMC studies reported that confidence scores were utilised in all 51 studies they found, including the 31 studies on imaging tasks in which confidence scores are not used in clinical practice.

I reaaaaally hate this practice. Hence, trying to find a better way.

Meta meta meta

So what did Lyle mean? What does meta-analysis have to do with estimating average human reader performance?

Well, in the meta-analysis of diagnostic test accuracy, you take multiple studies that report the sensitivity and specificity of a test, performed at different locations and on different populations, and you summarise them by creating a summary ROC (SROC) curve.

Zhang and Ren, a meta-analysis of mammography diagnostic accuracy. Each dot is a study, with the size of dot proportional to sample size (between 50 and 500 cases). Lines reflect the SROC curve and the 95% confidence interval.

Well, it seems to me that a set of studies looks a lot like a group of humans tested on a diagnostic task. Maybe we should try to use the same method to produce SROC curves for readers? How about Esteva et al, the famous dermatology paper?

This is a model that best fits the reader results. If you compare it to the average (which was reported in the paper), you see that the average of sensitivity and specificity is actually bordering on the inner 95% CI of the fitted model, and only 4 dermatologists perform worse than the average by being inside that 95% CI line. It certainly seems like to SROC curve makes more sense as a summary of the performance of the readers than the average does.

So the approach looks pretty good. But is it hard? Will people actually use it?

Is it even research?

I initially just thought I’d write a blogpost on this topic. I am not certain it really qualifies as research, but in the end I decided to write a quick paper to present the idea to the non-blog-reading community.

The reason I felt this way is that the content of the paper is so simple. Meta-analysis and the methods to perform meta-analysis is one of the best understood parts of statistics. In fact, meta-analysis is generally considered the pinnacle of the pyramid of medical evidence.

Metanalysis is bestanalysis.

But this is why the idea is such a good solution in my opinion. There is nothing fancy, no new models to convince people about. It is just good, well-validated statistics. There are widely used packages in every major programming language. There are easily accessible tutorials and guidelines. The topic is covered in undergraduate courses.

So the paper isn’t anything fancy. It just says “here is a good tool. Use the good tool.”

It is a pretty short paper too, so all I will do here is cover the main highlights.

What and why?

In short, a summary ROC curve is a bivariate model fitted on the logit transforms of sensitivity and specificity. It comes in two main flavours, the fixed effects model and the random effects model, but all the guidelines recommend random effects models these days so we can ignore the fixed effects versions***.

When it comes to the nuts and bolts, there are a few main models that are used. I reference them in the paper, so check that out if you want to know more.

The “why do meta-analysis?” question is important. There are a couple of major benefits to this approach, but the biggest one by far is that we get reasonable estimates of variance in our summary measures.

See, when you average sensitivity and specificity, you calculate your standard deviations by pooling the confusion matrices across readers. Where before you had multiple readers, you now have one uber-reader. At this point, you can only account for variability across samples, not readers.

In this table, adapted from Obuchowski in a book chapter I wrote, we see that the number of readers, when accounted for, has a huge impact on sample size and power calculations. Frankly, not taking the number of readers into account is methodologically indefensible.

SROC analysis does though, considering both the number of readers and the “weight” of each reader (how many studies they read). Compare this SROC curve re-analysing the results of Rajpurkar and Irvin et al to the one from Esteva et al above:

With only 4 readers, look how wide that confidence region is! If we draw a vertical line from the “average point” it covers a sensitivity range between 0.3 and 0.7, but in their paper they reported an F1 score of 0.387, with a 95% CI of 0.33 to 0.44, a far narrower range even accounting for the different metric.

Another nice thing about SROC curves is that they can clearly show results stratified by experience level (or other subgroups), even when there are lots of readers.

From Tschandl et al. The raw reader points are unreadable, but summarising them with SROC curves is clean and tidy.

There are a few other good points of SROC curves which we mention in the paper, but I don’t want to extend this blog post too much. Just read the paper if you are interested.

Just use SROCs!

That’s really all I have to say. A simple, off-the-shelf, easily applied method to more accurately summarise human performance and estimate the associated standard errors in reader studies, particularly of use for AI human-vs-machine comparisons.

I didn’t invent anything here, so I’m taking no credit^, but I think it is a good idea. Use it! It will be better^^!

You wouldn’t want to be methodologically indefensible, right?

* I’ll have more to say on this in a future post, suffice to say for now that this is actually how medicine works when you realise that doctors don’t make descriptive reports, they make decisions. Every statement made by a radiologist (for example) is a choice between usually two but occasionally three or four actual treatment paths. A radiologist who doesn’t understand the clinical implications of their words is a bad radiologist.

**This actually got me really nervous right after I posted the paper to arxiv (like, why has no-one thought of this?), so I email-bombed some friends for urgent feedback on the paper while I could still remove it from the processing list, but I got the all clear :p

*** I semi-justify this in the paper. It makes sense to me anyway.

^ Well, I will take credit for the phrase “Docs are ROCs”. Not gonna lie, it was coming up with that phrase that motivated me to write the paper. It just had to exist.

^^ For anyone interested, it still isn’t perfect. There are some reports of persistent underestimation of performance using SROC analysis in simulation studies. It also doesn’t really account for the fact most reader studies have a single set of cases, so the variance between cases is artificially low. But you can’t really get around that without making a bunch of assumptions (these are accurate empirical estimates), and it is tons better than what we do currently. And heck, it is good enough for Cochrane :p^^^

^^^ Of course, if you disagree with this approach, let me know. This is a preprint currently, and I would love to get feedback on why you hate it and everything about it, so I can update the paper or my friends list accordingly :p

Luke Oakden-Rayner is a radiologist in South Australia, undertaking a Ph.D in Medicine with the School of Public Health at the University of Adelaide. This post originally appeared on his blog here.

Docs are ROCs: a simple fix for a “methodologically indefensible” practice in medical AI studies published first on https://venabeahan.tumblr.com

[jonathanalumbaugh]

Weekly Digest

January 7th, 2018, 6th issue.

A roundup of stuff I consumed this week. Published weekly. All reading is excerpted from the main article unless otherwise noted.

Read

Teen birth rates hit a new low in 2016, Boston has joined other cities in banning single-use plastic bags, Tesla restored electricity to a children's hospital in Puerto Rico after it was hit by hurricanes in September, the FDA cleared an earpiece that may help block symptoms of opioid withdrawal, 13 states saw record-lows of unemployment this year, Support for allowing same-sex marriage is at its highest point in 20 years, Vice President Mike Pence said in October that the U.S. "will return...to the moon not only to leave behind footprints and flags but to build the foundation we need to send Americans to Mars and beyond," a man in North Carolina has started the non-profit ChemoCars, a service that provides cancer patients with free rides to and from their chemo treatments, Uber partnered with the charity Whizz-Kidz to give those who use wheelchairs in the UK free rides to polling places this summer.

— 9 things America is getting right

This is not some “lite” version of Civ stripped down for touchscreen, mobile implementation. It’s the whole game.

— Civilization 6 on iPad is a marvel

First comment in thread: I keep seeing this referred to over and over, even TV Guide is calling the bad Cooper by the name BOB! In my opinion, this is something that people have been confusing for 25 years.

— Clarification: Cooper is not possessed by BOB

I got married two weeks ago. And like most people, I asked some of the older and wiser folks around me for a couple quick words of advice from their own marriage... Almost 1,500 people replied, many of whom sent in responses measured in pages, not paragraphs. It took almost two weeks to comb through them all, but I did. And what I found stunned me…

They were incredibly repetitive.

— Every successful relationship is successful for the same exact reasons

Explaining #Meltdown to non-technical spouse. “You know how we finish each other’s...” “Sandwiches?” “No, sentences. But you guessed ‘sandwiches’ and it was in your mind for an instant. And it was a password. And someone stole it while it was there, fleeting.” “Oh, that IS bad.”

— Scott Hanselman (@shanselman)

January 5, 2018

— Explaining Meltdown with parallel worlds, libraries, and a bank heist

TED Video: How to make stress your friend

— How to make stress your friend

A user visits a website, registers an account, and saves the data in the password manager. The tracking script runs on third-party sites. When a user visits the site, login forms are injected in the site invisibly. The browser’s password manager will fill out the data if a matching site is found in the password manager. The script detects the username, hashes it, and sends it to third-party servers to track the user.

— How web trackers exploit password managers

PricewaterhouseCoopers LLP was negligent in connection with one of the biggest bank failures of the financial crisis, a federal judge has ruled, opening up the Big Four accounting firm to the potential of hundreds of millions of dollars in damages.

— Judge Says PricewaterhouseCoopers Was Negligent In Colonial Bank Failure

Whether we see an LTE version of the Nokia 3310 in the US is still a major question, as is the release date of this phone — not the mention the battery life, which took a major hit when it added 3G support.

— An LTE version of Nokia’s 3310 may be coming

The Big Five... has produced results that can be shown to remain largely consistent across a person’s lifespan and that can be used to predict at least some part of a person’s likely academic achievement, dating choices and even future parenting behavior. It has also been validated cross-culturally to some extent, Soto told me.

— Most Personality Quizzes Are Junk Science. I Found One That Isn’t.

"Neither [Iraq] nor while I was in the military did I actually hear anyone ask whether we should be doing some of the research we were doing. You know, some of it was a little scary -- I don't know that it was necessarily unethical -- but nobody ever asked the question." -General Robert H. Latiff

— Nobody's Ready for the Killer Robot

If you are a low-wage worker who cuts your expenses to the bone in order to sock away $500 a year, on which you earn 8%, you will still not go more than a year in retirement without starving to death.

— Oh Damn, 401(k)s Aren't Magic

Ever stood at an intersection and prodded at, leaned on, elbowed and otherwise palm-slapped the ever-living hell out of a crosswalk button and wondered to yourself if the thing actually does anything at all, really? Well – chances are, it doesn't.

— Placebo buttons do absolutely nothing, and they are everywhere

Meanwhile, Pete is convinced the Log Lady stole his truck. But wait! It wasn’t the Log Lady. It was Windom Earle, says Cooper. How does he know? Well, look at the map up there. Duh. Try and keep up, people.

— Revisiting ‘Twin Peaks’ Season 2 Finale: An Appointment at the End of the World

In an interview with radio host John Catsimatidis in New York, Cohen said that it was clear that President Trump — like former President Obama — did not want to approve a plan to provide the new arms to Ukraine, but decided to do so in an attempt to shirk allegations that he has acted as a "Putin puppet."

— Russia expert: US decision to supply arms to Ukraine a 'mistake'

Scopophilia or scoptophilia (from Greek σκοπέω skopeō, "look to, examine" and φιλία philia, "tendency toward"), is deriving pleasure from looking.

— Scopophilia

The fatal swatting case started Thursday when a man called the 911 center in Wichita, Kansas, and said he'd shot his father and was holding his mother, sister and brother hostage inside a house, authorities said.

— Swatting case poses legal challenges for police, prosecutors

The IRS lets you claim investment-related losses on your tax return as long as you sell the money-losing investment at some point during the year. You can then use the resulting capital losses to offset any capital gains on other investments that you might have.

— Tax Loss Harvesting: Don't Wait Until Year-End to Save Thousands

Tesla was on the cover of Time magazine in 1931 but died a poor man in 1943 after years devoted to projects that did not receive adequate financing. Yet his most significant inventions resonate today.

— Tesla the Car Is a Household Name. Long Ago, So Was Nikola Tesla.

More than a century ago, in New York City, Paul Strand began creating some of the earliest candid street photography. His goal was to capture people as they act in public, unaware of the observing eye.

— Theater of the Streets, Shot On Google Glass

In 2016, psychologist Danielle Gunraj tested how people perceived one-sentence text messages that used a period at the end of the sentence. Participants thought these text messages were more insincere than those that didn’t have a period. But when the researchers then tested the same messages in handwritten notes, they found that the use of a period didn’t influence how the messages were perceived.

— There’s a reason using a period in a text message makes you sound angry

My beach wedding in Diani, Kenya, was supposed to begin at 4 p.m. It started two hours later. The reason: The photographer was late. He shrugged it off, blaming traffic. "I am here now and that is what matters," he said. Grrr, "Kenyan time."

That is what they call it in my homeland.

— Under 'Kenyan Time,' You're Expected To Arrive ... Oh, Whenever

The year 2017 was really successful for Vue.js. Even though the goals are partly fulfilled, I think that most of the goals are somehow achieved or getting more traction. Vue.js is spreading and a lot more companies are using it now, including: Behance, Adobe, Chess.com, GitLab, HERE Technologies, Car2Go, IBM, and many chinese companies like alibaba, ele.me

— Vue.js review of 2017

In 2007, Warren Buffett entered a million-dollar bet with the fund manager Protégé Partners that the S&P 500 would beat a basket of hedge funds over the next decade.

— Warren Buffett has won his $1 million bet against the hedge fund industry

Earlier today, Twitter published a five paragraph answer to the loudly, repeatedly-shouted question: “Why won’t you ban Donald Trump, a man who has actively used your platform to threaten nuclear annihilation against an entire country?”

— What Twitter's New Statement About Not Banning Trump Really Means

In South Carolina, for example, people hoping to buy a Siberian tiger to celebrate the new year are likely to be disappointed: As of Jan. 1, it is illegal in the state for typical residents — that is, if you're not a zoo — to buy or own exotic animals for pets.

— What's New In 2018? Here's A Brief Tour Of State Laws Now In Effect

Why people believe what they believe is a wide topic that many psychology professors investigate. And while Peterson’s lectures certainly do tend to focus on the idea of “pushing back,” the contents of them raise questions about whether the bad ideologies are the ones he’s rejecting or the ones he espouses.

— Why Is Monsanto Inviting This Alt-Right Hero to a Fireside Chat on Farming?

The danger is that such detailed, sensationalized coverage of suicide can prompt copycat behavior — a phenomenon called suicide contagion. “Suicide contagion is real, which is why I’m concerned about it.”

— YouTuber Logan Paul's video of a dead body put his own audience at risk

Then there’s the matter of how Uber treats its drivers. You know it’s not great, but it’s not as though competing services are much better. Before Uber, taxi companies were notoriously terrible employers. Lyft, like Uber, hires its drivers as independent contractors—they don’t get benefits or minimum-wage protection—and has cut their pay to make fares cheaper for riders.

— Are you a bad person if you still take Uber?

Forecasters are warning people to be wary of hypothermia and frostbite from the arctic blast that’s gripping a large swath from the Midwest to the Northeast.

— http://metro.co.uk/2017/12/30/niagara-falls-freezes-sharks-freezing-death-atlantic-7192401/?ito=cbshare

[eng658nunez]

New Media to Writing v. Writing to New Media

I’ll be frank in saying that the visual you see above is a perfect representation of my reaction after reading Anne Frances Wysocki’s, Opening New Media to Writing: openings and justifications. It wasn’t so much as confusion as it was I got so caught in the things said that by the end I was left unsure if a solution or opportunity had actually been presented in the theoretical ideas Anne Frances Wysocki was giving. I have my issues (really only one issue that breaks into smaller ones) but I will leave that for after my breakdown of some of the points she makes that actually stuck with me for the rest of the chapter. 

“I want to consider how any material we use for communication is not a blank carrier for our meanings, is not a blank that contributes nothing to how readers understand. Instead, I believe that we have a time of opening here, a time to be alert to how these choices of material very much articulate into the other structures that shape writing and our lives - and that being alert to these choices can help us shape changes we might want.”

-Anne Frances Wysocki (10)

I initially disagreed with her claim that platforms are not neutral carriers, as I firmly believe that platforms carry a certain within themselves when being opened to their user whether it turns out that way or not. Reading further with what she meant I found that she presented a very valid point in that the content being produces by us, the users, has a heaviness to it, a weight that prevents these platforms from being neutral carriers. In a way, she is giving more validation to the users of these new media platforms than the actual platform itself, which I still am not sure how I feel about that but still found I generally agreed with what she was indicating here. I believe that any media platform or tool of creation of any medium is capable of being used in an academic setting so it isn’t that hard for me to hop aboard the bandwagon here and simply shake my head to this. This is a topic I wouldn’t usually think about giving an actual reason for supporting, for instance if I were to teach a composition course and incorporate the constructive choices of composition in tweets which have a limit to what can be said, I wouldn’t have a strong response if asked why exactly I find it important to use such platforms. Wysocki’s response above is definitely something I could get behind despite my slight confusion as to the route she is taking the rest of the chapter. 

“Our particular technologies do matter, and we do need to attend to new technologies and processes--because these technologies take shape in context of everything else that matter to us.”

-Anne Frances Wysocki (19)

What I got from this was the encouragement to place (or be aware) of the values of various forms of new media and technologies within and outside the classroom setting. When applying new media platforms to an educational setting, it shouldn’t filter the technology as being solely committed to an educational concept but to be aware and cherish the fact that this platform is something that is accessible and impacting beyond what it is being used for in the class. And despite a single new media being used by multiple people, no product will come out the same and the diversity in that alone should have a special kind of value places on it. 

I personally found the chapter to be presenting this kind of ageist perspective, considering the things being called out seems to be geared more towards educators who are older and generally more focused on the traditional kind of teaching method. If that were the case then I fully expected the chapter to include methods to actually incorporating or even introduction new media into a traditionally set classroom rather it simply being this notion of being called out for not being up to speed with technology yet not given a stepping stone to exploring it. 

[myaekingheart]

So it seems as though I've spiralled back into another one of those spells where I just hardcore hate myself. For the past couple days, all I've been able to think about it is the thought of getting a job, ever since I had that stupid fucking dream the other night. I dreamed that I was working the register at some upscale fast food restaurant taking orders as people filtered in, got their food, and left. I couldn't help but think to myself, "God this is so easy! Why didn't get a job earlier?" and then five minutes later, I broke out into a massive panic attack mid-dream. Numb hands, cold sweat, hyperventilating, feeling like the place was closing in on myself, the whole enchilada and that's when I realized why I don't have a job. Granted, I know anxiety is no excuse for being unemployed, I'm not an idiot. Ever since that dream, though, I just have not been able to stop thinking about jobs and how I don't have one. I don't necessarily want one but at the same time, I keep feeling increasingly uncomfortable with my parents sending me money for all my bills and rent and whatnot. I feel bad that they're stuck supporting me like they always have. I kind of want the independence of having a job and earning my own money, anyways. There's just all the stuff holding me back and making me feel like an absolute failure, like the fact that I am 20 years old and have never worked a day in my life. I feel like a total scam because as I sat here today filling out job applications, applications that wouldn't let me move forward without submitting a resume, this disgusting tinge of guilt and disgust fell upon me as I had to sit here and fill out that I had no work experience. And I know the application is only step one. As much as saying I have no experience pains me, I think the thought of anything that might come next is even worse. I've filled out job applications before, twice for Spirit Halloween, both times of which at the end of the process, they've immediately told me I'm either not qualified enough or not what they're looking for and that was fine. I don't fear rejection in job applications. If anything, a gross part of me prays for rejection because that means I can get out of this. The thought of moving forward and getting a job offer terrifies me. I don't like thinking about having to step foot in these places for an interview, having to paste a smile on my face and lie about how I'm a team player and enjoy being a corporate slave, and explaining why I've never had a job before at my age. I don't like the thought of getting hired and then having to work a menial, pointless job for shit pay and being trapped in a store filled with people complaining left and right at me about things that are probably not beyond my control. I am honestly such a difficult fucking person because like I know I need a job but I don't want a dead-end job that won't have any positive effect on the future of my career. I don't want to work a pointless job that has nothing to do with what I enjoy or strive to pursue as an actual career. I want to be a writer so why not look at freelance writing jobs? I did and I can tell you that the results are not good. I want a job that parallels my career goals but I don't want to sit here and waste my time ghostwriting someone's novel, working my fingers to the bone only to get zero credit for the hours of time I've spent on this thing, or churning out pointless articles for someone too fucking stupid to write it themselves. I applied for a job at a local Books a Million and at my university's bookstore because I figured, hey! Why the fuck not? It's as good as I'm going to get, I guess. The thing that terrifies me about Books a Million, though, is that I just feel like I'm not fucking good enough. I feel like I really know next to nothing about the majority of the books out there today, I feel like I don't read enough to work at a bookstore and that if someone came up to me with a question about something, there's a fat chance I wouldn't have any fucking clue what they were talking about. I've never even read Harry Potter (nor do I care to-- I tried but I'm one of the few who just could not get into it). The university bookstore is really the one I'm banking on most, honestly. It would be nice to get to know my way around campus before starting classes there in January, plus it would be the easiest to get to in terms of transportation. Which brings up another point of contention: I am 20 fucking years old and I still don't know how to drive. I don't even have my permit. I've never been behind the wheel of a car save for Mario Kart and the Tomorrowland Speedway and if my experience with either of those is any indication, it's probably a good thing that I don't drive. I should've learned a long time ago but I have always put it off, terrified of getting in an accident and knowing full well that I will without a doubt get overwhelmed because of literally every single thing you have to pay attention to and worry about. I feel like I'll definitely get massive sensory overload behind the wheel of a car. Again, however, not a valid excuse for never learning how to drive. It's not that I enjoy people having to chaffeur me everywhere. I wish I could be totally autonomous and independent and in my dreams, I am, but in reality I know I'm just a failure who knows next to nothing about being an adult and am failing miserably at it. My boyfriend keeps telling me he wants me to get a moped, thinking that that would be better for me because it'd be easier to be aware of my surroundings on one of those as opposed to a car but I'm skeptical. Not that I wouldn't love to come off as a motorcycle bitch or whatever but still. My only other option is the bus which I mean, I didn't have a problem with that. I've taken public transit thousands of times in my hometown to my community college, but never in this city. And that makes all the difference, honestly. My hometown was dull suburban where nothing ever happened except the occasional heart attack during bingo. There were sketchy people, sure, but it wasn't in massive volume. Here, however, things are vastly different. This is the city where there's homeless people and drug dealers abound! My boyfriend is very protective of me and he's always made off this facade about wanting me to stay safe, liking the fact that when I'm home he knows where I am and that nothing bad will happen to me, that he's skeptical of the bus and doesn't exactly feel safe with the idea of me taking said bus. It's built up this understanding in my head that the outside world is dangerous and that I should avoid going out there alone at all costs. But he's also a little frustrated that my parents never encouraged me to learn to drive before I moved up here, because he knows learning to drive up here is going to hell and he wishes I would've gotten it done sooner. Which also makes me feel like a failure. Like, how pathetic could I possibly be? Never worked a day in her life, never been behind the wheel of a car. I'm practically a child. And I sure as hell feel like one, too. I'm embarrassed and ashamed of myself for letting myself get so far behind. I've avoided becoming an adult for so long that now I have absolutely no choice and I'm not even the slightest bit prepared. All I want to do is stay home and eat Oreos and lay around in pajamas writing shitty fanfiction but I can't do that because I am an adult with my own apartment 300 miles away from my family. My academic achievements shine but my personal achievements fall short big time and I'm so fucking ashamed of myself for that. I should've taken care of this shit a long time ago. I should've grown up a long time ago but I'm terrified. I'm so fucking terrified. In a way, living up here still doesn't even feel like everyday reality. A part of me still processes this as temporary, like I'm on a trip to visit my long distance boyfriend like how it was for the past year and then at the end of the week, my parents will show up and take me back home and everything will go back to normal. But that's not normal anymore. This is my new normal and I need to learn to adjust to it. This is my home now and this is my life now. I can't sit around on my ass like a house guest anymore because I'm not a house guest. This is my house and my new town and I'm not a dependent living under my parent's roof anymore. I am a full-fledged adult and I need to start acting like one, even if I'd rather shoot myself in the mouth than go through this horror. I still don't feel ready. I don't know if I'll ever feel ready. I'm so motherfucking scared and the worst part is knowing that this is all my fault.

[2centsofsilver]

2am (Evening of 1/18)

Open Letter to My Parents (in the works of my head) Dear Mom and Dad, I don’t know why this is happening to us. I’m sorry that I have a mental disorder. Honestly with you, those features - the really bad ones - the ones where those misdiagnoses happened - the outliers that don’t appear among relationships with me and everyone else - come through in our relations: Screaming episodes any time we’re together, verbal expression of extreme opposition, cutting insults (the kind one could never forgive another for), severe anger issues and the type of disorientation one with BPD exhibits (expressions of not being rooted in reality, not hearing the other person, deliberately re-directing conversations, but coming off as stupid/blind/naive), the whole “I love you/I hate you” feature, threats from you that you’ll disown me as your daughter, threats from me that I’ll abandon you as my family, sibling sidings, threats to kill myself, blaming you for wanting to kill myself, the list goes on. Honestly, I feel things are at their absolute worst- the worst they’ve EVER been. I no longer cry or am emotionally affected/saddened by our exchanges. I no longer have that “fear of abandonment” feature (but I have it with my friends). I know my friends are there for me and that is where my support system lies. I love you very very much, but your love is conditional. If you know how much heart your daughter has, like if you knew her, you (maybe) would like her? Like if you changed though too. Because right now your love is conditional: You love me if I am ___ or do ____ or stop doing ____. I believe that any issues in a relationship are ALWAYS two way streets. So I’m willing to take 50% of the blame. But because I’m your daughter, there’s a level system there, so I feel more comfortable taking 40%. Maybe you feel the same way, with your level system on the other side, and you’re comfortable taking 40%. I know it’s less though. You do not see relationships as two way streets. You do not own up to all the ways in which you’ve hurt me. You DISCOUNT every single thing you’ve ever done that has remotely negatively influenced me. You also just flat out don’t believe you ever have. And instead, I’m the bad one, I’m a shitty daughter, there’s something wrong with ME for thinking that, for seeing it that way, for trying to see it *fairly*. If anything you blame yourselves for the countless ways in which you’ve “failed” as parents, raising such a “despicable” daughter.  You never failed, but I’m not despicable. If you felt my heart, if you were inside me, if you felt the pain I feel on a daily basis constantly being abandoned by friends who I thought cared about me, constantly having anxiety over day-to-day situations, constantly feeling inadequate, not good enough, and constantly needing validation from others. If you knew what it was like for 1 person to struggle with any form of mental illness, physical illness, handicap, what have you. Or if you had any appreciation for or AWARENESS of or acceptance of marginalization. If you could own up to the ways in which you fall into the trap of stigamatization. I honestly don’t know who you’d be. Would you even be my parents anymore? I often see on social media - Facebook - parents commenting on their children’s posts, publicly saying how “proud they are” of their sons or daughters. Or directly, publicly, telling their children those same words. Or just liking their posts and sharing their posts and being fun and funky and silly and sweet. There comes a time in life as people grow up and older that that level system from “dominance/authority” to “equality/admiration” shifts drastically. There comes a time, I’ve noticed, when parents become their children’s friends and their children openly love and accept that and are no longer self-conscious of it in front of others. There comes this time that never happened for us and I’m really worried it never will happen for us, when you’re supposed to start hopping around and telling people you know, when they ask how Katie’s doing, all the great things she’s doing in life and where she’s headed. You’re supposed to get excited like I am. There comes a time when you’re supposed to come over to my house and say, “How have you been? We’ve missed you.” Or “How’s your application process coming along? We’re happy for you.” There are these beautiful shooting stars that go off in the midnight sky when moms want to “get to know” their daughters. That level system changes, it equals out, it becomes more human-to-human. I don’t know what to do, but I feel sick about it day to day. Things have drastically gotten worse in the last few years, but they’ve always been bad. You don’t believe me? You’re in denial. They’ve been bad since I entered high school. You still treat me like you did 12 years ago. And you’re going to do it to my brother too, that’s why he wants to run away, travel north, all the way up there, to get away from you and live independently. He wants his own space, his own time, his life back. It’s why I don’t visit home much anymore, why I don’t call you for weeks on end, why I don’t want to talk to you about things. I don’t know how that comes off regarding my personality -- I mean, you must know that I’m not the same way with my friends, co-workers, bosses, teachers, right? I know you think I repeatedly go into therapy complaining and bitching about my “horrible parents,” the ones who “don’t love me,” or “did this and that.” But that’s not true. I rarely talk about you because I can’t or when I do, it’s “I don’t know what to do. I’m a bad daughter. I want our relationship to be better.” Let’s date back to middle school or high school from the academic perspective. Parent teacher conferences, or whatever. All the ways in which my teachers raved about me, my artwork in the hallway, all those things that marveled and dazzled you as parents. That’s still me. Sometimes, I worry I’ve lost myself too, like in for instance, the case of me no longer pursuing English teaching or writing. But I’ve found other interests. YOU instilled that in me growing up, by exposing me to hundreds of thousands of activities and experiences. You’re the ones who taught me to love more than one thing, to constantly explore, learn, and grow. You’re the one who taught me how to be myself, how to find my identity, but you never taught me much about how to influence others, that came through my exposures to good experiences and the good heart I was born with, maybe my love for nature and art, our love when I was a young child, and definitely my loving grandparents. So I used to be this like, “perfect” student or whatever. Did that disappear when I didn’t go to Hope College? I got straight A’s in community college instead. I’m getting off track here -- let’s date back to that academic perspective from when I was younger. That’s still me. Like she exists inside me. I still love music, writing, art, nature, going to Glen Arbor, being with my family, school, funny jokes. I’m sorry that I gave up clarinet and piano and didn’t pursue journalism after managing the high school publications. I’m sorry that I no longer *talk about?* writing -- but I still do it. I try to write every day. And I’m still going to be a published author one day, even if you disagree with my content.  I have always cared about other people more than I care about myself. I have always been social, a people person, even though I was shy. I always had a lot of friends until the antisemitism arose in high school. You hated me for that. Is that when it started? You hated me for “choosing those friends” who would ultimately do that to me and to our family. But I was happy before it happened, dad. I was a thriving teenager who had the best summer of her life before that. She was living her dream, everything she ever wanted to be. She didn’t know it was going to happen. She didn’t “choose” antisemitic friends who she knew would bully her and trespass our lawn and drive me off the road and stalk our house at night. She didn’t know. Do you blame them or do you blame me? I wanted to go to therapy in the 9th grade because I had really bad social anxiety disorder. I couldn’t look at any one in the hallways, couldn’t answer questions in class, couldn’t give presentations, and I think I missed over 50 days of school that year because I could not face the inside of that high school. I wanted to go to therapy to get help and be happy again. My god, FOR YEARS, the THEME of my therapy sessions among ALL my therapists has been “Confidence and Happiness.” I want to be “Happy and Confident” (Depression and Anxiety's opposites). YOU’RE THE ONE who went out of your way to find me a therapist you knew through someone else. I loved that therapist. I ended up seeing her for 7 years and she changed my life. I’m guessing it bothered you that you had a young daughter who was struggling. And I know you were happy to hear that I loved my therapist and that our sessions were working. I remember distinctly telling mom about the “Anxiety Toolkit” stuff. I remember she used to ask me, and I would tell her, and I was excited about my progress and applying the strategies we came up with during my day-to-day attempts to get through high school. I don’t know at what point you stopped being affected by my hardships. I’m not by any means saying they should “still break your heart,” I’m saying I don’t know at what point you developed this idea that, “Therapy fixes people. Why isn’t she fixed yet?” Every single truth for me in my life is countered by responses that I cannot even begin to fathom comprehension for. Like I try very hard to understand where you’re getting this information from or why you might feel the way you do. I’m very conscientious in my efforts to see things from your angles and understand why you might be feeling the way you do. But like, my depression has gotten drastically worse (or more developed?) over the course of the last 10 or so years. It has depleted me, exhausted me, and defeated me. I honestly feel physically weakened anytime I even try to think these things through anymore. Like my shoulders drop and I just don’t have it in me anymore. I have become hardened to all pain, a concrete wall (I used to say this when I was 14), and incredibly resilient beyond my years. I have been through so much turmoil inside me that I had to grow up far sooner than a lot of people my own age. I am grateful for that, for I cannot imagine being so god damn behind in life, but it also has hardened me, made me stoic, it’s the reason I don’t have much positivity or enthusiasm in life, like there really isn’t a point and it’s a state impossible for me to feel. I try, don’t get me wrong, I really do try. Every day I try to make it a good day. But I am tired, do you understand? My mind, body, and soul are tired.  “That’s because you need to lose weight.” You might say. I guess I could use that topic as a phenomenol example of how exhausting it is to get through any half a minute of conversation with you. Like if we’re at the table and I’m trying to talk to you about something important and I mention I’m tired, you’d probably respond with that. And you’d divert the conversation almost immediately to the point where there’s no way I could ever get out of that new topic. Immediately, I’m forced to defend myself: “I AM losing weight. I just joined a new gym, I’m on the 21 day fix. I go to the gym every day, for a whole year now! A YEAR.” “Well clearly it’s not working,” you’d chuckle. “If you’d just start eating right, if you’d just start exercising...” It’s a great example because it demonstrates your disoriented view of how change is immediate or black and white. You’ve never believed me or believed in the concept of change happening gradually, over time. I know your deadlines are “asap,” but you have to accept that it’s probably going to take the course of the rest of my life for me to be happy, try to be happy, find happiness. Things will always be hard for me because I’ve seen too much, experienced too much. Even when I do finally reach happiness one day or whatever, things will still suck. Because the whole world affects me differently than other people. Everything is interconnected. I am vastly influenced by every person I’ve ever met. And when I grieve, I grieve those people for years. I have to give myself permission to grieve too, even when I feel I’ve surpassed my deadlines. Extended my deadlines, surpassed them again. It takes a long time for pain to fade, I might never get through to the people who have hurt me, but time eventually will make those memories fuzzy. In time, maybe I’ll only think about them once a week, or once a month. For now though, I grieve. There is so much going on inside me that you could never possibly understand because you don’t believe in mental illness. You also don’t believe in mental health practitioners. You hate who I am and how I am and resent me for all my therapy and how hard I try every day. You want me to be different and I am working on myself all the time, and I need assistance to function. I’m sorry. I also need assistance because I need support because I can’t get through life without people who are there for me. If you had any idea how fucking alone I am, even surrounded by so much support lately, I’m pretty sure it would kill you. Or any breathing person who’s not you. Like I honestly have no idea what it would be like for you to experience me because you have zero empathy when it comes to other people’s personal problems. You’re like a fucking Behavior Analyst. You’re everything wrong with the field. You judge only based off what you can see. You come over to my apartment, you see the way I live, you think it’s as easy as just changing my environment, as easy as just “stopping.” You don’t believe in thoughts, feelings, or emotions. You make fun of people with developmental disabilities or physical disabilities. You don’t believe in depression. Like how can you not believe in the one driving force that makes me who I am, that makes life SO fucking hard for me, that interferes with every aspect of my life, YOU SEE the effects. It’s mind-boggling. You don’t believe int he source, you think it’s ME. The other fucking night we were out to dinner in Kzoo, and we were fighting in public which is our new trend, and dad, you literally told me that the mental health field is a wasted field, helping people is all a wasted effort, that mental illness doesn't exist, and that I am literally wasting my future and the rest of my life by committing myself to helping others get better and make the most out of life. Saying that, you aren’t just referencing third person ideas and concepts. You are directly cutting me in so many capacities: You are discounting my personal journey, my efforts, my day-to-day battles, my long-term goals, my progress, my pain, and my commitment to helping others live a happy life. I don’t know how that isn’t something to be proud of. How do you not believe in being selfless? Mom’s a teacher! I used to really really want you to be proud of me. I’ve now found that it’s not possible, so I can only be proud of myself. I know that I have a lot of people in my life who are proud of me and excited for me and all that I’m going for in my life. But I’m concerned. My 25 year old adult self who has felt 57 since age 14 is concerned.  I am about to go off to grad school because I feel now is the time. I am also ready for adventure because while yes, I still struggle with depression, I feel I’m better now than I ever have been and I’m ready and feel capable, with the promise of resources wherever I go, that I’ll be ok. That I can do this.  What I do know is that oftentimes, children and parents stop getting along and no longer continue to try. Somehow, they just stop loving each other. I’m not willing to let that happen to us, even if it already has on your end. I still love you, I will always love you, no matter what. And I am not willing to travel across the country with our problems they way they are. You’re not willing to change, to even accept that there are any issues in our family. You don’t believe in therapy so you’d never consider family therapy. And you say I’m one of those fake professionals who wants to “bring people closer” and “families together” when it’s not possible. You say you’re too old to mend things with me, dad. What does that mean? Do you know since I was really little, my biggest fear in the entire world has been my parents dying? I DON’T WANT YOU to get old or sick and not fucking know how much I always loved you. How sorry I am. And how badly I wish I could be everything you wanted me to always be. But I just can’t travel thousands upon thousands of miles away with our issues where they stand. I will not be ok where I end up, but I’ll be better, knowing I have a supportive family and that we’re “good.” We don’t have to be perfect, but even if we’re just “good.” I have mental illnesses, mom and dad. Like whether you believe that’s possible or not, but I do. I call them that because with names, they’re treatable, and I can get help and support from others who have been there or are trained to help me. I have been diagnosed by doctors who know what they’re doing (you’re all science), and I’m on medication that has been carefully chosen by the best psychiatrist in southwest Michigan, and it works. Without it, I would have killed myself in 2011. I am ready to travel to the other side of the country now, to live my life and feel adventure while I still can. I want to fall in love, get married, have children, start a career, and be successful. I want to travel and explore the world and become the even better me that you always dreamed I’d be, but for myself and the others in my life or career that I’ll be helping. Like anyone else, I’m allowed to experience experiences. I’m ready. And whether or not you can be happy for me, we need to be good, because without support from the root source of where I derived, without support from the direct source of where I’m from, who I am, where I’ve come from, and who I can always turn back to if things were to ever go wrong for me on the other side of the country, my emergency contacts; or the people who I love very much, who I care for very much, who I will be taking care of when you’re not your finest, when you grow older and need my help, I will be there. So without us being good, I cannot go off and see the world. I will be in pain for life without us being good. I know you hurt too, so why can’t we work on us. Why can’t we just figure out a way to do this.  3:35am, Interview at 10:30. Goodnight.

[howardstudent]

How the American Education System Influences Every Literary Interaction I’ve Had

Writing, Literacy, and Discourse

22 October 2020

My relationship with writing is complex. I was identified as “gifted” in reading and writing in my early adolescence, and a more rigorous curriculum accompanied the discovery.  Navigating school English curricula has at various points fostered my love of writing and conversely extinguished my passion for it. My relationship with writing affects not only how I speak and write, but also how I interact with literature in its medley of manifestations. In America, literature’s value is primarily based upon its adherence western standards, and these sentiments have been taught to me, through the education system, and greatly impact my experience with literacy. Essentially, my perspective on writing was formed by the culture and society I grew up in.

My prowess in language arts was first acknowledged in preschool; however, it did not impact my learning until third grade when I was placed in my first gifted program. The initial branch of the program involved leaving class to do extra work on top of my normal language arts class. I remember enjoying it, until I started falling behind in my regular writing class, and then it felt punitive. Luckily for me, this didn’t last for long because for fourth and fifth grade the advanced program took the place of my normal English language arts classes. I distinctly remember this tradition only the “PETS” kids were given the opportunity to participate in. We were instructed to write, illustrate, bind, and present books we wrote. I wish I could say I wrote some profound novella about middle-class anxieties, but I just wrote about a single mother and her two children moving into a haunted home. Upon reflection, there were many things I liked about assignments like this: it was long term, I had time to revise and edit, and then revise and edit three more times, I had creative liberties, and I was encouraged to write in my own style.  

I was taught that voice is crucial to developing a literary style and presence. I feel as though my writing voice is nearly identical to my speaking voice. I like that my writing tends to have hints of comedy, sarcasm, and sometimes heart. This usually does not conflict with my essay writing, but there are times I have been required to write formal research pieces. I believe my first research essay is the initial instance of me not being utterly infatuated with the art of writing. I understand the importance of learning different genres and styles but the methodology is where I disagree. I have always hated writing requirements. I understand my perspective is not the only perspective and I remember how often teachers were asked “how many paragraphs should the essay have”, “how many sentences in each paragraph”, “how many words in each sentence”. However, for me, the more specific these restrictions became, the more constricted I felt. What was I to do if it took me four sentences, not five, to express what I wanted to say; or what if I was only asked for half a page analysis but the material was so rich, I needed to write to two pages worth? This is where my distaste for essay writing began to take root. In this genre of literature, little room was left for my creativity; it was no longer about writing until my heart was content, it was about fulfilling requirements to be rewarded with a good grade.

These unfortunate incidents of heartless writing were suspended my freshman year of high school. The assignment was to produce any art based off of the Holocaust literature we had been studying. I had not created poetry for the sake of poetry in years at that point. This is despite the fact poetry is one of my favorite art forms, and that I had collected Shell Silverstein’s entire body of work throughout my adolescence. I immediately visualized three words: “Chains of Oppression.” The words spilled onto the page from my fingertips, and the entire poem was completed in one sitting. Initially, the piece spoke specifically about the Holocaust, but progressively, the scope was broadened to discuss how the shared burden of oppression bonds many groups of people together. It concluded by exploring how chains can manifest outside of physical subjugation. I practiced my recitation, my inflection, my tone, and my emphasis. I bought chain and wrote the names of marginalized groups on it and performed my poem with the chains binding my wrists. That assignment was very enlightening to me because reminded me of how writing can make me feel, and how it can help me cope and even heal. I was deeply invested in the project, in a way that I had not been engaged with literacy for a long while.

Following that endeavor, I elected to take a creative writing class. I really enjoyed the course as the rigid boundaries that had been set in my core English classes had been dismantled, or so I thought. I was granted more creative liberty, however, the course never challenged me to explore literature outside of what I was already familiar with. I wrote short stories and poems, but not in any new form or style. Everything I produced in that class was written in “College English” and followed proper English grammatical conventions. Simultaneously, in my core English class, we were allowed to select a 1920s research topic. My white peers chose topics like flappers, feminism, and even communism. For several reasons, I choose to study the Harlem Renaissance. Everyone knows about the Harlem Renaissance; it’s one of the 5 topics white schools discuss during Black History month, but I wanted to challenge myself to do a deeper dive. My research led me to information I had never been taught before. Namely, that one source claimed the Harlem Renaissance did not have a profound impact on the majority of the black people at the time. In the 1920s, the locality of the renaissance occurring in Harlem prevented it from being widespread, the majority of black folks at the time did not have the disposable income to indulge in the arts, and some even lacked the literacy to comprehend the likes of Zora Neal Hurston and Langston Hughes. This revelation was shocking to me. My previous units on the subject made it seem as if racism ended because black people seized the opportunity to express their talent, but that was not the truth of the matter. When my paper was handed back to me, I saw that my teacher had only one major comment. “Your paper was so strong until you concluded with the information about how there was not an immediate impact of the Harlem Renaissance,” or something to that effect. I had wanted to make a statement about the reality of the Harlem Renaissance, but that wasn’t what my teacher wanted to hear from me.

Almost all of my transgressions against how schools handle teaching and developing writing skills could be remedied if educators understood that “what students need is not models of correctness... but a broader understanding of the intricate connection between one’s language and his cultural experience, combined with insight into the political nature and social stratification of American dialects” (Smitherman 61). I feel like my proudest literary accomplishments involve me connecting with my blackness. I have connected blackness to my writing through multiple avenues: researching, reading, and writing poetry. Lucille Clifton wrote, “both nonwhite and woman/ what did i see to be except myself,” and that is a conclusion that it took me nineteen years to come to (“Won't You Celebrate with Me”). I was hyperaware that I was usually the only minority in the spaces I occupied. I think that is a prominent aspect of why I value eloquence and literacy, of the college English variety. White people continued to confirm what I believed they thought. Every time I presented a speech, a poem, an essay, they would always feel compelled to compliment me on how well I spoke and how articulate I was. Their compliments made me contemplate whether I was truly gifted or if they just didn’t expect me to be literate because I am black, and they do not associate literacy with blackness. I developed a “veil of whiteness” in attempt to avoid drawing attention to the fact I am nonwhite, but it never really worked. This is because conforming to white standards of literacy, white standards of anything really, does not make me white and has only slightly improved how I am perceived and treated in white spaces. Jacqueline Royster states that, as black people, “throughout our history in this country, we are put in jeopardy and on trial in a way that should not exist but does,” and it is this ideology on which my outlook on the world, including literacy, is founded (5).

Upon reflection of my entire academic literary career, I feel that I cannot overstate the profound impact that the way in which literature is taught and appraised can dictate how individuals respond to it. Something I struggled with as a young author was staying in one verb tense. I was corrected on that subject several times, which is why I am now sensitive to verb tense usage in my own speech and writing, as well as others. I feel as though most people who pass through the United States education system will develop writing quirks like this. The broader impact of teaching a “proper” college English is that I was simultaneously taught to devalue anything that does not align within the confines of its structure. The problem with this is two-fold. Writing in standard college English can prevent a work from being accessible, as not everyone is equipped with the knowledge and skills to comprehend college English. The second issue is that, when I read something not in standard English, I subconsciously question the credibility, validity, and thought behind the work. I even found myself doing this during the peer editing activity. I understand that not everything should be written in college English; there is a power in choosing voice, structure, and syntax. For example, nothing technically makes a sonnet a more valid form of poetry than freestyle rap, but I have been conditioned in a way that makes me think in this manner. I have been attempting to recognize and break down the western bias I possess, and I will continue to do so.

Works Cited

Clifton, Lucille. “Won't You Celebrate with Me by Lucille Clifton - Poems | Academy of American Poets.” Poets.org, Academy of American Poets, poets.org/poem/wont-you-celebrate-me.

Green, David F. Visions and Cyphers: Explorations of Literacy, Discourse, and Black Writing Experiences. Inprint Editions, 2016.

Royster, Jacqueline. “When the First Voice You Hear is Not Your Own” Visions and Cyphers: Explorations of Literacy, Discourse, and Black Writing Experiences, edited by David F. Green, Inprint Editions, 2016, pp. 3–13.

Smitherman, Geneva. “English Teacher, Why You Be Doing the Thangs You Don't Do?” Visions and Cyphers: Explorations of Literacy, Discourse, and Black Writing Experiences, edited by David F. Green, Inprint Editions, 2016, pp. 55–62.

[sirlennon]

Ten Lessons That Will Teach You All You Need To Know About SEO 2019

Colorado SEO Pros CEO Philip Rodgers talks search engine search engine optimization (SEO) strategy, artificial intelligence (AI), and why link building is definitely dangerous for companies. 55 The difference from SEO is many simply depicted as the distinction between paid and unpaid concern ranking searching results. I get into much more detail in SEO Titles on pages: 15-Point Checklist for B2B and B2C Brands, which explains the right way to work in relevant keywords that will accurately reflect the page articles. Are good nevertheless SEO potential may be reduced when compared with single links. The education and learning behind our SEO expertise had been developed from years and many years of learning from mistakes advertising with our other businesses. Our own in-depth guide contains the most recent SEO best practices so a person can improve how your posts appears in search results, plus get more traffic, leads, plus sales. Keyword analysis definitely belongs to the SEARCH ENGINE OPTIMIZATION basics. I'll start by stating that social press and SEO are heavily linked to each other. Black head wear SEO attempts to improve ratings in ways that are disapproved of with the search engines, or even involve deception. This particular is more tedious and tasking than inorganic SEO because this particular is how all the key phrases get a full blast associated with attention. SEO: It stands for Research Engine Optimization. Within this post, we will break this down in the complete first timers guide to SEO: what SEARCH ENGINE OPTIMIZATION is, how it works, exactly what factors affect search and exactly what sorts of changes you may make today to improve your own search optimization. The no follow hyperlink has been contradicted many instances over where SEO is included and it depends on the particular web owner about if they will allow them on the web site or not. This particular means that SEOs spend the lot of time working upon getting links in a procedure called link building Link-building techniques can range from simply asking for a link to writing the guest post - and presently there are many others. Basically, SEO plans the keywords that will are to be delivered plus content provides them. So, when you are considering about applying SEO in the particular broad sense, you need in order to channelize its technical specifications by means of content marketing. In 2019, you can wager that White Hat SEO can have separated itself even additional from Black Hat SEO, plus that above all else, delivering quality content will be the particular most important factor for companies ranking in search. The outcomes are not instant, you can use the time on attempting other Internet marketing techniques whilst SEO would go to function. The third major SEARCH ENGINE OPTIMIZATION ranking signal is Google's synthetic intelligence search ranking algorithm. Sometimes SEO is definitely simply a matter of producing sure your site is organized in a way that lookup engines like google understand. SEARCH ENGINE OPTIMIZATION involves a number of changes towards the HTML of person Web pages to obtain a higher search engine ranking. As a result associated with technological advancement, SEO is in order to undergo more drastic changes, plus the two latest technologies that will are expected to influence SEARCH ENGINE OPTIMIZATION to some very great degree are AI (Artificial Intelligence) plus Voice Search. The main stage of SEO is to become friendly both to users plus search engines. Based in Colorado springs municipal airport terminal, CO, SocialSEO has been the particular preeminent digital marketing firm within Colorado since 1996. Although the SEARCH ENGINE OPTIMIZATION experts provide great SEO providers for their clients, there are usually a few who ruin the particular internet design industry through their particular greed in marketing efforts plus manipulate search engine rank. SEARCH ENGINE OPTIMIZATION may target different kinds associated with search, including image search, regional search, video search, academic research, news search and industry-specific straight search engines. The benefits of powerful digital marketing plans are fairly much proven throughout the board yet when it comes to companies within the photography industry typically the SEO aspects of the marketing technique really do make all regarding the difference to helping a new photographer stand out from this crowd. HISTORY LESSON: Old SEARCH ENGINE OPTIMIZATION advice suggested you connect along with websites and trade links along with each other True, a large genuine site linking to you may boost up your ranking, yet, it's not going to assist your relevancy score when the particular linked content isn't related in order to the website sharing it. 10. Quality back back links need to be built upward to your site for efficient SEO. SEO is about attracting individuals to your site to start with by making sure this shows up searching queries. Content written simply by customers gives sites legitimacy plus it earns SEO juice each from manual and automatic ranking systems. Throughout 2018, this trend is about the rise and if an individual invest in making your internet site mobile friendly you will delight in higher ratings in 2019 as well. When making a internet site for Google in 2018, a person need to understand that Search engines has a long list associated with things it will mark websites down for, and that's generally old-school SEO tactics which are usually now classed as ‘webspam‘. Besides getting social plus having a good time, your own social media profiles can favorably affect your SEO efforts. Since so many people have confidence in voice search being the craze in 2019, I, Nicole Bermack (this article's author), am coining the word Blog9T VSO - voice research optimization. Clever brands and agencies are changing new SEO, social and articles marketing strategies to join their particular customer conversations. Search engine optimisation (SEO) is the art plus science of getting pages in order to rank higher in search motors such as Google. A person can audit your page's SEARCH ENGINE OPTIMIZATION health, compare SEO metrics intended for a number of pages, plus analyze both external and inner links on any given internet page. Content companies that will offer SEO content creation providers are experts in creating content material that are not only improved for search engine rankings yet also add value for your own business in terms of brand name awareness, customer engagement and enhanced sales. UnGagged Birmingham is really a 2-day SEARCH ENGINE OPTIMIZATION & Digital Marketing conference of which takes place June 11-12th, along with one-day intensive masterclasses on 06 13th. These ‘quality updates' are very reminiscent of Search engines Panda updates and often influence many websites at the exact same time - and often these types of concentrate on demoting similar ‘low-quality' SEO techniques we have already been told Panda focuses on. I think savvy SEOs are going to utilize AI content generation (text and pictures especially) in 2019 to produce a whole new kind associated with search algorithm gaming, that might almost be viewed as dark hat. This course covers how in order to build strong links for much better SEO. SEO companies ensure that will your site has all associated with the SEO Internet marketing like as business improvement, localized item sales, high quality online era and the rest. 2018 brought many modifications to the world of SEARCH ENGINE OPTIMISATION, including a few new formula evolutions through Google. An expert Seo agency may help any dimensions of company take full advantage of web marketing and boost the likelihood associated with actual sales while simultaneously raise brand awareness one of the particular customers segment. Recently we were speaking about the subject with the friendly agency in Valencia plus we directed to this admittance of Search Engine Journal along with the 20 questions that need to be done to any SEARCH ENGINE OPTIMIZATION or SEO agency that desires to hire. Register and raise the free donation for SEO Birmingham every time you shop on the internet. Local SEO - Optimize that localized content on your internet site to properly leverage local signs, online reviews and business results. Learn more about content material optimization for SEO here. Along with paid-search it offers a very focused audience, visitors referred by SEARCH ENGINE OPTIMIZATION will only visit your web site if they are seeking particular home elevators your products or even related content. From keyword filling to link buying, the SEARCH ENGINE OPTIMISATION landscape has seen numerous black-hat tricks — and Google often catches on. Chris Gregory, founder plus managing partner at Jacksonville dependent firm, DAGMAR Marketing, predicts that will AI and machine learning might have a big impact upon SEO in 2019 and SEOs who aren't technical will become left in the dust. Some SEO professionals also advise that anchor textual content should be varied as numerous pages linking to one web page using the same anchor textual content may look suspicious to find motors. SEO trickery such since keyword ‘stuffing' in irrelevant written content simply won't cut it throughout the current day, with Google's algorithm taking over 200 aspects to ensure that it's ratings provide results with valid plus authoritative sites, it is around on impossible to complete anything some other than work with the research engines to make sure best quality SEO results. From a SEARCH ENGINE OPTIMIZATION perspective, the principal keyword need to be at the beginning adopted by the other relevant key phrases. SEO companies are able to track almost every aspect of their technique, like increases in rankings, visitors and conversions. The strategies which were the simplest (reciprocal links or directory submissions) perform not work anymore, therefore the particular SEOs spend a lot associated with time trying different approaches. The particular Moz Pro is another collection of Tools that check the particular important factors related to your own website's search ranking. Selection of key phrases or phrases plays an essential component in an SEO campaign considering that it saves you the photos in the dark. In 2019 the particular digital marketing companies can anticipate a lot of voice queries, and by 2020, about 50 percent of the searches will become either voice searches or image-based searches. Read our Mobile SEO 2019 Checklist before you decide in order to implement. This first facet of optimizing images is usually including your keywords in this image file name (seo_guide. jpg). The inevitable adjustments that will occur in SEARCH ENGINE OPTIMIZATION in the near future are usually abolition of keywords stuffing plus spam backlinks, real-time personalized customer care by online marketers, improvement within the quality of visual articles as a result of creativity of video SEO, optimization associated with websites with conversational keywords plus generating massive quantity of current data. SEARCH ENGINE OPTIMIZATION remains one of the lengthy term marketing strategies that function best for companies that are usually looking to improve their on the web visibility. Without the doubt, one of the greatest trends that has already started to take place and may continue well into 2018 may be the consolidation of niche MarTech gamers by larger content cloud suppliers, with the role and significance of SEO increasing significantly all through this transformation. SEO Internet marketing offers major components, which develop the particular website traffic, and top lookup engine rankings. SEO is brief for Seo, and there is definitely nothing really mystical about this particular. You might have heard the lot about SEO and exactly how it works, but basically exactly what is a measurable, repeatable procedure which is used to send out signals to search engines that will the pages are worth displaying in Google's index. Topic clusters possess been lauded because the future associated with SEO and content strategy, yet are widely underreported on (so now's the time to hit! ) 93% of B2B companies use content marketing. Teresa Walsh, Marketing Professional at automobile site, Cazana, forecasts that hyper organic targeting will probably increase its importance in 2019 with more location search plus more voice search. We get to the particular bottom of on-page SEO issues in order for search motors to clearly see what your own website is all about. SEO requires you to continuously be considered a student because of just how quickly the algorithms of research engine companies change. Google's punishing methods probably class pages as some thing akin to a poor UX if they meet certain detectable criteria e. g. lack associated with reputation or old-school SEO stuff such as keyword stuffing a site. Carrying out technical SEARCH ENGINE OPTIMIZATION for local search engines is usually really a similar process. Search engine optimization (SEO) will be a way to generate even more (and desired) traffic to your own site with the help associated with better search engine rankings with regard to a keyword. SEO had gone through drastic changes over the yrs and getting higher rankings upon search engines by stuffing the particular information with too many key phrases is a thing of the particular past. This should become a crucial part of any local SEARCH ENGINE OPTIMIZATION checklists, as reviews and rankings could make it easier to stand out in search engine results. Lookup Engine Optimization, or SEO, need to rank in as one associated with the biggest part of your own finances for online advertising. SEO or even Search Engine Optimisation is the particular name given to activity that will attempts to improve search motor rankings. Search engine optimisation Ten Reasons Why You Cannot Learn SEO 2019 Well (SEO) is the procedure regarding affecting the visibility of the website or a web web page in a web search engine's unpaid results—often referred to because " natural ", " natural ", or "earned" results. Nearby SEO utilizes a variety associated with strategies — getting your web site ranked on search engines such as Google, business directories such because Yelp, Superpages, Foursquare, Yellowbook, Search engines My Business listing, Bing Areas for Business page, localized articles on your website, online testimonials and other strategies. We frequently compose in-depth analyses on the method SEO and digital marketing will be used to boost the traffic in order to various websites. Let's confirm our knowing of SEO basics with the one question quiz about the particular factors in search rankings: web pages, links and keyphrases. When it comes in order to Google SEO, the rel=canonical hyperlink element has become VERY IMPORTANT over the many years and NEVER MORE SO. To place the particular focus on the website website visitor, it has been suggested that will acronym SEO should are a symbol of research experience optimization. It's important to remember in order to share content from your internet site or blog socially to provide it an SEO boost as well. Some Webmasters will pay out thousands of dollars to several so-called professional SEO expert in order to get their Websites on best in the rankings. It is often preferable to ask questions upon their SEO procedures and find out whether they have knowledge associated with keyword density or if these people have information about the most recent keyword research strategies and equipment. This combined expertise makes all of us uniquely qualified to present a good ethics-based, search engine-friendly SEO guide to teach you guidelines that will reflect the latest developments looking. They include important subjects for example search motor success factors and how SEARCH ENGINE OPTIMIZATION relates to social media marketing and advertising. In order in order to make web pages appear in high positions in the search outcomes, SEO tries to shape the website according to Google's criteria. The major problem of SEO is the reality that there are billions associated with pages in the internet research engine indexes and your place in the SERPS is reliant on a constantly changing formula which is not published. SEO is really the shortened phrase for search motor optimization. I believe SEO within 2019 will largely be such as SEO in 2018, with the particular exception of some big” Search engines update that wipes out sufficient websites to make people think the algorithms have grown significantly smarter. SEARCH ENGINE OPTIMIZATION works by optimizing a cyberspace pages, conducting keyword research, plus earning inbound links. The organization offers excellent SEO packages that will help rank the clients' web site within top three pages associated with search engine pages. That can make SEO an ideal lead era tool, because when searchers stick to links back to your web site, you have the chance in order to convert them to leads, plus later make sales. Whenever asked to point out ideas that are unique towards the particular web, most people will arrive up with two main types: SEO and social media. How many links do a person need for good off-page SEARCH ENGINE OPTIMIZATION? If you are carrying out a professional SEO audit for any true business, you are going in order to have to think like the Google Search Quality Rater Plus a Google search engineer to supply real long-term value to the client. 44. Give your own social media profiles an SEARCH ENGINE OPTIMIZATION boost. 10% of our experts believe that will there's likely to be the lot of focus on cellular in 2019, as older SEOs realize optimizing for desktop is usually pointless. White hat SEO is not really just about following guidelines, yet is about ensuring that the particular content the search engines indexes and eventually ranks may be the same content the user will see. User-generated content racks up serious SEARCH ENGINE OPTIMIZATION points by providing sites along with fresh content, bundles of inbound links, and sources for organically framing natural attributes. You'll find it right here Also really worth checking away is Moz's Beginner's Explained SEARCH ENGINE OPTIMIZATION, ” which you'll find right here, and the SEO Success Pyramid from Small Business Search Advertising. Off-page SEARCH ENGINE OPTIMIZATION strategy refers to the impact of a change in various other websites in your search position. Bruce Clay-based set the standard for honest internet marketing by authoring the particular " SEO Code of Integrity, " now translated into eighteen languages, and it has already been a respected leader within the particular ever-changing research engine marketing industry since mil novecentos e noventa e seis. Website SEARCH ENGINE OPTIMISATION (including, content optimisation, meta marketing, the significance of links). A good SEO Specialist is someone that is an expert on the particular topic of search engine optimisation. Just since a proper On-page and Off-page SEO optimization work will provide a better Ranking to your own website in Search Engine Outcome Page (SERP). The greatest SEO change and tendency I realize already happening within 2018 is Google's switch in order to a mobile-first index, so in case a site is not however mobile-friendly and optimized for this, it's critical to prioritize this particular as it is now basic for its SEO success. With good on-page SEO, lookup engines can easily index your own web pages, understand what your own site is about, and very easily navigate the structure and content material of your website, thus position your site accordingly. SEO specialists started in order to abuse PageRank in order in order to raise the rankings. Now could be a great time to take a nearer take a look at SEARCH ENGINE OPTIMIZATION marketing information because search is getting even worse for those types of B2B plus B2C businesses. SEO marketers plus writers typically come up along with different kinds of content in order to place the necessary keywords within. These are some of the particular most used types, and every one helps to help with the level of variety in conditions of website content. Certain black hat SEARCH ENGINE OPTIMIZATION techniques, such as keyword filling, are thought to be the SEARCH ENGINE OPTIMIZATION equivalent of spamming, and research engines will penalize them. Internet marketing is exclusively driven by SEO or research engine optimization. Others are usually simply traditional PR firms who else have learned SEO as properly as the value of hyperlinks. Here's a cliche among electronic marketers: Search engine optimization (SEO) isn't what used to end up being. Google announced that will they released several minor enhancements over the period of the particular time of about a 7 days and after analysis, experts inside the SEO industry concluded that will the updates were the outcome of keyword permutations and web site using doorway pages. SEO, or Search Motor Optimization, means setting up your own website and content to show up through online search results. People which want to take their company to some new height plus generally and mostly believe within the way of SEO advertising. Here's the fantastic news: You don't have in order to have to be a SEARCH ENGINE OPTIMIZATION wizard to make sure your own website is well positioned regarding organic search engine traffic. SEARCH ENGINE OPTIMIZATION is the acronym for lookup engine optimisation. The particular search engines have refined their particular algorithms along with this progression, numerous tactics that worked within 2004 can hurt your SEARCH ENGINE OPTIMIZATION today. Therefore, we also make content on conducting keyword study, optimizing images for search motors, creating an SEO strategy (which you're reading right now), plus other subtopics within SEO. In 2018 there will be an actually bigger focus on machine understanding and SEO from data. ” Of course, the amplification aspect of things will continue in order to combine increasingly with genuine public relationships exercises rather than shallow-relationship hyperlink building, which will become significantly easy to detect by lookup engines like google. Seo (SEO) is often regarding making small modifications to components of your website. In my SEO post writing guidelines I suggest a person take your main keyword plus 3 or 4 other associated keywords and write at 3-4%. Some SEO specialists claim the fact that building links for SEO reasons is pointless; others believe the particular role of backlinks for any internet site has continually risen through the particular years. Search engine optimization (SEO) is a huge part associated with any marketing strategy. This brand-new paradigm of users relying upon voice search for many associated with their search needs will become a game changer for SEARCH ENGINE OPTIMIZATION. A several of the popular keyword study tools are Wordtracker, SEOMOZ Key word Difficulty Tool, SEOBook Keyword Device and Google Keyword Tool. Thankfully, SEO allows webmasters to offer clues that the engines may use to understand content. Because less as 20% dentists within a city hire SEO expert s who really wants that shift within their Google ranking. The particular analytical part of the SEARCH ENGINE OPTIMIZATION is about tracking organic key phrase positions, website traffic and the engagement in Google Analytics, Site owner Tools and many other specific tools. Mobile SEARCH ENGINE OPTIMIZATION is nothing than a standard SEO, creating content and marking to be able to remain out online. SEO rankings are placing excess fat than ever upon what users have to state. The Search Gaming console will help you to evaluate your keyword rankings, CTRs, feasible Google penalties and many various other useful data for technical SEARCH ENGINE OPTIMIZATION. Blog posts, guides, whitepapers, situation studies, videos, and social articles all need to include the particular right keywords for social plus SEO. For even more detailed information on the make use of of keywords on your web site visit SEOcious to will discover many other important factors included in having your web web page in the top 10 entries on the search engines. SEARCH ENGINE OPTIMIZATION, search engine optimization is just about all about search engines, search motor result page (SERP), search rating, online visibility, and quality visitors. If your web site has about 5-10 niche key phrases that are extremely targeted yet not very competitive, your SEARCH ENGINE OPTIMIZATION pricing is going to become quite less.

[jonboudposts]

The Tyranny of Opinion Part 1: I Demand You Debate Me

There are some people you cannot debate with because quite simply, they do not debate.  They just talk.

Such people have a list of issues they want to talk about and have already formed un-movable opinions on. Like being on a bad date, they will try to steer any conversation toward a subject they know about – or think they do – then try to outdo their ‘opponent’ on the subject.  However, when they can’t, the switch comes in quick to either the next subject or sometimes full-on conspiracy theory.

They also like being offensive, at least to a certain type of person.  They imagine pissing off ‘the Libs’ makes them edgy and rebellious (still don’t get the girl though).  It is of top consideration to be offensive to minorities and those least likely to fight back.  Being offensive is the goal; having good reason to be is irrelevant.

One example in my life was someone I used to see through work.  We would discuss various bits about films or games (his interest, not mine) and occasionally serious stuff like emotions.  After this person moved routes, a few months passed before I saw them again.  The next meeting was quite a shock; they had become preoccupied with alt-right talking points and hating the usual tropes (feminism, Islam) and proclaiming the need for violent action to protect ‘our’ (Christian) values.

In our following meetings, this person displayed classic traits – throwing a ‘point’ at me, shouting with faux-anger, interrupting mid-answer, uncomfortably putting up with my response that completely defeated their argument; then ignoring it and jumping to the next claim. Most of these claims were ludicrous; for instance it was claimed that in the Labour Party manifesto of 2015, there was a proposal that gay males would be forcibly paired up with single fertile women in order to produce the future labour force.  Now, in all seriousness; please tell me how to ‘debate’ with that.

My initial response was to speak to some mutual friends of ours and ask if they could explain this behaviour, before suggesting we should perhaps stage an intervention or call Prevent. I have still not decided.

However, some people make a career out of this kind of bullshit and not just Alex Jones.  A well-know name at the moment is Jordan Petersen, a Hackademic who likes to clothe his woman-hate and disgust with a world not interested in praising the mediocre white man constantly with a sheen of pseudo-intellectualism and big concept waffle.  Plus, as many articles written by far more intelligent and academic people tan me will tell you, his theories are bunk; mostly based on intentional misinterpretation of Marx or Derrida, coupled with the resentment only a rich white man can cultivate toward anyone with alternative ideas that are directly threatening to privilege.

He has decided cultural Marxism is why we are all so miserable and those obnoxious and tiresome young men like my example above are just misunderstood and alienated because of post-modernism and feminism.  Sadly, organisations like the BBC and Channel 4 think him worthy of attention too.  As he does not really have an argument but he does have loads of confidence and is able to speak for long periods, this makes him, in the modern interpretation, a good debater.  For the desperate need within the media for something not overtly murderous or obviously racist from the right, he gets taken seriously.  Some may complain I am only making a personal attack – I am, because there is nothing else there.

There is a big difference between free speech and people who just shout for attention.  At the risk of cutting people out of public life, there is no morale imperative to provide a platform to hate mongers and cranks and we know perfectly well this weakens society rather than strengthens it.

Petersen appeals to my friend because he is one of those figureheads who produces easy answers to complex problems, gives them back their hero narrative and tells them there is not need to change, while also backing up the nation that those who fail at the system should be condemned – so their adulation of him amounts to pure self-hatred really.  What they most strongly have in common is their convenient outrage and complete disinterest in answers or alternative opinions.  They just want someone to notice them.

Also, there is no work done here.  There is simply the expressing of opinion without footnotes.  No research, just feeling.  Those hating immigrants or women or trans people rarely quote from analysis and when such analysis is put to them, with all the facts that contradict their negative opinion of immigrants’ or women or trans people, they often just resort to denying the validity of the research with no evidence to back this up (seeing any themes here?).  Evidence is who Dave had in his cab last week or the ‘fact’ that one midwife at the hospital during the birth of their child was rude – and bloody foreign.

This kind of behaviour is simply not worthy of attention or contemplation (it is also dangerous). The lack of willingness to do any work but imagine you can just elbow into the same sphere is deplorable.  Derrida developed deconstruction; Petersen wrote an opinion book called 12 Rules for Life.  I read about both on the internet.

If anyone thinks I am suggesting ‘excluding some people’ from public life, let me ask you this; have you ever really spend time listening (online, TV or in life) to anyone jabbering conspiracy theories or just re-writing history, all for the sake of free speech?  If you have heard such people, how long did you listen?

If you were sitting at dinner and someone started ranting about 9/11 being an inside job, the moon landing not really happening, or no one dying in the Grenfell Tower, would you really just sit there?  Or would you prefer to see this person given a platform at a university?  Should we all engage for the sake of ‘preserving free speech’?

Well, you can if you want to but I have better things to do.  Plus, I do not think this shows any dedication to ideals of freedom; it shows a society falling apart, with no idea which direction to go.  With a lazy media looking for content and damaged people looking for someone to blame, this just creates a toxic public life filled with broken men and self-hating women screaming at the youth and calling it engagement.  In the absence of any new culture or forward momentum, this is the kind of thing that occurs – a faux sense of pride in your emptiness; a conviction life was better before Group A showed up (even though you were not born before Group A showed up); or more simply, old wives tales gone mad.  It really comes down to resentment toward anyone with the guts and tenacity to not put up with the shit deal given to them and to strive to make things better for everyone; rather than stand back while late-period capitalism implodes and hope they do not get any rain down on their house.

Now more than ever in my lifetime, there is an opportunity to change the present; a time characterised by insecurity and fear.  More working class people are beginning to realise they can stand up for themselves through trade unions or co-ops of various types and in the process, learn real working class history while doing the best thing you can with history – using it to construct a better future for yourself and those around you.  Let’s ditch the snake oil salesman and conspiracy theories and put our shoulder into this; then you will have some real power.

(In future articles I will write about hate speech, attention seeking-verses-argument and why taking the piss out of people is legitimate).

[kristinsimmons]

Docs are ROCs: a simple fix for a “methodologically indefensible” practice in medical AI studies

By LUKE OAKDEN-RAYNER

Anyone who has read my blog or tweets before has probably seen that I have issues with some of the common methods used to analyse the performance of medical machine learning models. In particular, the most commonly reported metrics we use (sensitivity, specificity, F1, accuracy and so on) all systematically underestimate human performance in head to head comparisons against AI models.

This makes AI look better than it is, and may be partially responsible for the “implementation gap” that everyone is so concerned about.

I’ve just posted a preprint on arxiv titled “Docs are ROCs: A simple off-the-shelf approach for estimating average human performance in diagnostic studies” which provides what I think is a solid solution to this problem, and I thought I would explain in some detail here.

Disclaimer: not peer reviewed, content subject to change 

A (con)vexing problem

When we compare machine learning models to humans, we have a bit of a problem. Which humans?

In medical tasks, we typically take the doctor who currently does the task (for example, a radiologist identifying cancer on a CT scan) as proxy for the standard of clinical practice. But doctors aren’t a monolithic group who all give the same answers. Inter-reader variability typically ranges from 15% to 50%, depending on the task. Thus, we usually take as many doctors as we can find and then try to summarise their performance (this is called a multi-reader multicase study, MRMC for short).

Since the metrics we care most about in medicine are sensitivity and specificity, many papers have reported the averages of these values. In fact, a recent systematic review showed that over 70% of medical AI studies that compared humans to AI models reported these values. This makes a lot of sense. We want to know how the average doctor performs at the task, so the average performance on these metrics should be great, right?

No. This is bad.

The problem with reporting the averages is that human sensitivity and specificity live on a curve. They are correlated values, a skewed distribution.

The independently pooled average points of curved distributions are nowhere near the curves.

What do we learn in stats 101 about using averages in skewed distributions?

In fact, this practice has been criticised many times in the methodology literature. Gatsonis and Paliwal go as far as to say “the use of simple or weighted averages of sensitivity and specificity to draw statistical conclusions is not methodologically defensible,” which is a heck of an academic mic drop.

What do you mean?

So we need an alternative to average sensitivity and specificity.

If you have read my blog before, you would know I love ROC curves. I’ve written tons about them before (here and here), but briefly: they visually reflect the trade-off between sensitivity and specificity (which is conceptually the same as the trade-off between overcalling or undercalling disease in diagnostic medicine), and the summary metric of the area under the ROC curve is a great measure of discriminative performance. In particular the ROC AUC is prevalence invariant, meaning we can compare the value across hospitals even if the rates of disease differ.

The problem is that human decision making is mostly binary in diagnostic medicine. We say “there is disease” or “there is no disease”. The patient needs a biopsy or they don’t. We give treatment or not*.

Binary decisions create single points in ROC space, not a curve.

The performance of 108 different radiologists at screening mammography, Beam et al, 1996.

AI models on the other hand make curves. By varying the threshold of a decision, the same model can move to different places in ROC space. If we want to be more aggressive at making a diagnosis, follow the curve to the right. If we want to avoid overcalls, shift to the left.

The black line is the model, the coloured dots are doctors. From Gulshan et al, 2016.

As these examples show, groups of humans tend to organise into curves. So why don’t we just … fit a model to the human points to characterise the underlying (hypothetical) curve?

I’ll admit I spent quite a long time trying various methods to do this, none of which worked great or seemed like “the” solution.

I’m not alone in trying, Rajpurkar et al tried out a spline-based approach which worked ok but had some pretty unsatisfying properties.

One day I was discussing this troubling issue with my stats/epi prof, Lyle Palmer, and he looked at me a bit funny and was like “isn’t this just meta-analysis?”.

I feel marginally better about not realising this myself since it appears that almost no-one else has thought of this either**, but dang is it obvious in hindsight.

Wait … what about all those ROCs of docs?

Now, if you read the diagnostic radiology literature, you might be confused. Don’t we use ROC curves to estimate human performance all the time?

The performance of a single radiologist reported in Roganovic et al.

It is true, we do. We can generate ROC curves of single doctors by getting them to estimate their confidence in their diagnosis. We then use each confidence level as a threshold, and calculate the sensitivity and specificity for each point. If you have 5 confidence levels, you get a 5 point ROC curve. After that there are established methods for reasonably combining the ROC curves of individual doctors into a summary curve and AUC.

But what the heck is a doctor’s confidence in their diagnosis? Can they really estimate it numerically?

In almost all diagnostic scenarios, doctors don’t estimate their confidence. They just make a diagnosis*. Maybe they have a single “hedge” category (i.e., “the findings are equivocal”), but we are taught to try to avoid those. So how are these ROC curves produced?

Well, there are two answers:

It is mammography/x-rads, where every study is clinically reported with a score out of 5, which is used to construct a ROC curve for each doctor (ie the rare situation where scoring an image is standard clinical practice).

It is any other test, where the study design forces doctors to use a scoring system they wouldn’t use in practice.

The latter is obviously a bit dodgy. Even subtle changes to experimental design can lead to significant differences in performance, a bias broadly categorised under the heading “laboratory effects“.

There has been a fair bit written about the failings of enforced confidence scores. For example, Gur et al report that confidence scores in practice are concentrated at the extreme ends of the ranges (essentially binary-by-stealth), and are often unrelated to the subtleness of the image features. Another paper by Gur et al highlights the fact that confidence scores do not relate to clinical operating points, and Mallet et al raise a number of further problems with using confidence scores, concluding that “…confidence scores recorded in our study violated many assumptions of ROC AUC methods, rendering these methods inappropriate.” (emphasis mine)

Despite these findings, the practice of forced confidence scoring is widespread. A meta-analysis by Dendumrongsup et al of imaging MRMC studies reported that confidence scores were utilised in all 51 studies they found, including the 31 studies on imaging tasks in which confidence scores are not used in clinical practice.

I reaaaaally hate this practice. Hence, trying to find a better way.

Meta meta meta

So what did Lyle mean? What does meta-analysis have to do with estimating average human reader performance?

Well, in the meta-analysis of diagnostic test accuracy, you take multiple studies that report the sensitivity and specificity of a test, performed at different locations and on different populations, and you summarise them by creating a summary ROC (SROC) curve.

Zhang and Ren, a meta-analysis of mammography diagnostic accuracy. Each dot is a study, with the size of dot proportional to sample size (between 50 and 500 cases). Lines reflect the SROC curve and the 95% confidence interval.

Well, it seems to me that a set of studies looks a lot like a group of humans tested on a diagnostic task. Maybe we should try to use the same method to produce SROC curves for readers? How about Esteva et al, the famous dermatology paper?

This is a model that best fits the reader results. If you compare it to the average (which was reported in the paper), you see that the average of sensitivity and specificity is actually bordering on the inner 95% CI of the fitted model, and only 4 dermatologists perform worse than the average by being inside that 95% CI line. It certainly seems like to SROC curve makes more sense as a summary of the performance of the readers than the average does.

So the approach looks pretty good. But is it hard? Will people actually use it?

Is it even research?

I initially just thought I’d write a blogpost on this topic. I am not certain it really qualifies as research, but in the end I decided to write a quick paper to present the idea to the non-blog-reading community.

The reason I felt this way is that the content of the paper is so simple. Meta-analysis and the methods to perform meta-analysis is one of the best understood parts of statistics. In fact, meta-analysis is generally considered the pinnacle of the pyramid of medical evidence.

Metanalysis is bestanalysis.

But this is why the idea is such a good solution in my opinion. There is nothing fancy, no new models to convince people about. It is just good, well-validated statistics. There are widely used packages in every major programming language. There are easily accessible tutorials and guidelines. The topic is covered in undergraduate courses.

So the paper isn’t anything fancy. It just says “here is a good tool. Use the good tool.”

It is a pretty short paper too, so all I will do here is cover the main highlights.

What and why?

In short, a summary ROC curve is a bivariate model fitted on the logit transforms of sensitivity and specificity. It comes in two main flavours, the fixed effects model and the random effects model, but all the guidelines recommend random effects models these days so we can ignore the fixed effects versions***.

When it comes to the nuts and bolts, there are a few main models that are used. I reference them in the paper, so check that out if you want to know more.

The “why do meta-analysis?” question is important. There are a couple of major benefits to this approach, but the biggest one by far is that we get reasonable estimates of variance in our summary measures.

See, when you average sensitivity and specificity, you calculate your standard deviations by pooling the confusion matrices across readers. Where before you had multiple readers, you now have one uber-reader. At this point, you can only account for variability across samples, not readers.

In this table, adapted from Obuchowski in a book chapter I wrote, we see that the number of readers, when accounted for, has a huge impact on sample size and power calculations. Frankly, not taking the number of readers into account is methodologically indefensible.

SROC analysis does though, considering both the number of readers and the “weight” of each reader (how many studies they read). Compare this SROC curve re-analysing the results of Rajpurkar and Irvin et al to the one from Esteva et al above:

With only 4 readers, look how wide that confidence region is! If we draw a vertical line from the “average point” it covers a sensitivity range between 0.3 and 0.7, but in their paper they reported an F1 score of 0.387, with a 95% CI of 0.33 to 0.44, a far narrower range even accounting for the different metric.

Another nice thing about SROC curves is that they can clearly show results stratified by experience level (or other subgroups), even when there are lots of readers.

From Tschandl et al. The raw reader points are unreadable, but summarising them with SROC curves is clean and tidy.

There are a few other good points of SROC curves which we mention in the paper, but I don’t want to extend this blog post too much. Just read the paper if you are interested.

Just use SROCs!

That’s really all I have to say. A simple, off-the-shelf, easily applied method to more accurately summarise human performance and estimate the associated standard errors in reader studies, particularly of use for AI human-vs-machine comparisons.

I didn’t invent anything here, so I’m taking no credit^, but I think it is a good idea. Use it! It will be better^^!

You wouldn’t want to be methodologically indefensible, right?

* I’ll have more to say on this in a future post, suffice to say for now that this is actually how medicine works when you realise that doctors don’t make descriptive reports, they make decisions. Every statement made by a radiologist (for example) is a choice between usually two but occasionally three or four actual treatment paths. A radiologist who doesn’t understand the clinical implications of their words is a bad radiologist.

**This actually got me really nervous right after I posted the paper to arxiv (like, why has no-one thought of this?), so I email-bombed some friends for urgent feedback on the paper while I could still remove it from the processing list, but I got the all clear :p

*** I semi-justify this in the paper. It makes sense to me anyway.

^ Well, I will take credit for the phrase “Docs are ROCs”. Not gonna lie, it was coming up with that phrase that motivated me to write the paper. It just had to exist.

^^ For anyone interested, it still isn’t perfect. There are some reports of persistent underestimation of performance using SROC analysis in simulation studies. It also doesn’t really account for the fact most reader studies have a single set of cases, so the variance between cases is artificially low. But you can’t really get around that without making a bunch of assumptions (these are accurate empirical estimates), and it is tons better than what we do currently. And heck, it is good enough for Cochrane :p^^^

^^^ Of course, if you disagree with this approach, let me know. This is a preprint currently, and I would love to get feedback on why you hate it and everything about it, so I can update the paper or my friends list accordingly :p

Luke Oakden-Rayner is a radiologist in South Australia, undertaking a Ph.D in Medicine with the School of Public Health at the University of Adelaide. This post originally appeared on his blog here.

Docs are ROCs: a simple fix for a “methodologically indefensible” practice in medical AI studies published first on https://wittooth.tumblr.com/