#(I just noticed we’re not moots?!) :0

[cockypark]

a short sot k2 fic for @0-dear-rose-0! @spinthetags

word count: 1070

Kenny keeps her gloved hands folded neatly in her lap. She keeps her eyes trained on whoever speaks, occasionally sneaking glances at Kyle. He doesn’t even pretend to look invested. The respective councils of the Elven Kingdom and the Kingdom of Kupa Keep were already meeting for the third time to discuss the peace treaty. Cartman’s increasingly stupid demands would stretch the meetings out for another month, so Kyle gave up on listening early. Kenny admired his transparency. She was always concerned with appearances. Of course, when you’re king, no one is on your ass about posture.

Kyle’s eyes flit across the room out of boredom, meeting Kenny’s. She smiles subtly, an air of mischief on her face. She signals for him to look at Paladin Butters. He didn’t even notice he was speaking.

“King Eric’s next request is that imports and exports go through Kupa Keep’s customs before delivery.”

Kenny mouths “nosy bastard.” Kyle shakes his head with humor. He leans in and whispers in Stan's ear.

“King Kyle will honor that request if all imports and exports also go through our customs as well.”

Cartman’s eyes narrow. He nods once. Butters scribbles something down. “King Eric agrees.”

Kenny excitedly knocks the gavel on the marble block. “Both kingdoms will inspect imports and exports before delivery.” It was the only worthwhile part of these meetings. Cartman didn’t even want her there, but she couldn’t give him what he wanted. She claimed her birthright, and these meetings claimed her Sundays. “The penalty for failure to comply shall result in investigation and trial, as well as a temporary pause in imports and exports. So behave, boys.”

Kyle glances at her with affection. She winks, resettling her hands in her lap.

“The King’s next request,” Paladin Butters starts, “Is that the Stick of Truth be returned to its place in Kupa Keep.”

Kyle’s head snaps up. He frowns deeply. “The fuck? No.”

“King Kyle, I am disappointed to have to remind you that Kupa Keep is its historical resting place,” Cartman sneers. “I’m asking that we honor history.”

“Like hell you are!” Kyle fumes. “You want it where you have unlimited access to.”

“Through a show of hands, how many council members are in favor of honoring Wizard King Cartman’s request to keep the Stick of Truth at the Kingdom of Kupa Keep?” Stan asks.

Cartman and Butters raise their hands. The king looks displeased. “Kenny, what the hell?”

“I hate to remind you, King Eric, that the McCormick family are the actual historical holders of the Stick. It is a moot point.”

Cartman scoffs. Kenny rolls her eyes in Kyle’s direction. He’s delightfully surprised at her lack of vote. She was playing more transparent in return.

“In that case, I want the council to be aware of the fuck-me eyes Kenny and Kyle are exchanging!” Cartman exclaims.

Kenny presses her hand to her chest in feigned shock. She knocks the gavel again. “Alright! I call for a recess! Paladin, I advise that you have a talk with the King.”

Butters’ shoulders deflate. “Ken, are you serious? Can’t we just-?”

“No!” Kenny interrupts. “The councils ruled all parties remain calm and composed. King Eric’s outburst is grounds for a recess. If he does not comply, I have no choice but to-”

“Don’t even say it, Kenny,” Cartman spits. He scrapes his chair across the floor while he stands. “Butters, let’s go.”

Butters side-eyes Kenny before joining the king. Kenny princess waves goodbye.

Kyle watches Kenny exhale and lean back in her seat. She throws a crooked grin his way. “Brief recess.”

-

“The resting place for the Stick is a nonnegotiable,” Wendy says. “So to satisfy Cartman, you’ll have to compromise on something else big.”

Kyle sighs. “That’s not fair, though. He shouldn’t have suggested that to begin with.”

“No one disagrees with that, Kyle. But the peace of the Elven Kingdom depends on it. It’s best we reassess a previous decision. If we tell him we’re throwing a bone, his requests are just going to become more obscure.”

“Right.” He hates this diplomatic bullshit.

Wendy and Stan exchange a look. “Kyle, we have been… concerned with your behavior.”

Kyle bristles. “What do you mean?”

“You’re appearing inattentive,” Wendy explains.

“And distracted,” Stan adds. “Dude, you and Kenny should do whatever it is and get it over with, instead of letting it impact the treaty.”

“‘Whatever it is’?”

Stan cringes. “I meant, you know… Fulfill your intentions or move on?” He looks to Wendy for support. She just shakes her head solemnly. He messed up.

Kyle pushes himself away from the table. “Tell that to Cartman, who isn’t even pretending to take this seriously. What do you think this is for?”

He leaves Wendy and Stan dead silent in the tent.

-

Kenny rolls and stuffs a cigarette into her ring holder. She lights it and takes a long drag. Her lipstick leaves a smudge on the paper. As she exhales the large cloud, Kyle appears before her. He waves through the cloud. She smiles.

“How did I just know you’d come find me?” She teases.

He sits next to her on the stone border. “Part of a king’s job is to keep the princess happy.”

“I would be a lot happier if this would move along.”

Kyle gapes at her. “You called the recess!”

She leans closer to him. “I wanted to see you.” She rests her hand on his. “Wendy and Stan are mad at you, aren’t they?”

“Other way around. The whole thing is going in circles.”

Kenny kisses his hand supportively. He leans into her, kissing her lips in return. In just a few months, everything would be right.

“Recess over?” Kyle asks. He has much more energy to argue with Cartman now.

Kenny grins wide. “Let’s get him.”

-

Kenny watches the ground the carriage treads over. When she was Elven Queen, she would put flowers all over here. Business meetings didn’t have to be boring and colorless. No, everything she owned would have her own touch. No one could tell her differently.

Her eyes naturally direct towards Eric, traveling separately from her and Butters. It would be a shame if something bad happened that way.

“Princess,” Butters says. “Were you and Kyle really… flirting?”

Kenny looks at him with curiosity. She liked Butters. He would stay an important pillar in her kingdom. Someone she could rely on. “Of course.”

~

if you like this please consider checking out my ao3! and if you're really interested, i am opening commissions :)

[doinggreat]

https://www.tumblr.com/0-twentyone/749242178497363968/i-dont-wanna-sound-like-a-hater-but-i-have-been

I’m not sure if you and this account are moots, but I noticed that they followed you and wanted to spread some awareness that this person is trying to spread harmful misinformation about Pedri.

honestly what does this even mean tho skekms he has organs and isnt flat like a normal healthy human being he is? wild 🥴

ps we’re not moots, but i think i reblogged some of their posts since i just generally search barca’s tag often 🥲

[tempenensis]

in-cahoots

One of my long-withstanding suspicion is that the Brain is somehow connected to the upper management of the jujutsu world - that he is in cahoots with them, or that he is one of the faces behind these sliding doors, since we have yet to be shown who they are. Some faces are shown briefly, but without names and what their actual position or influence.

In the first place, the circumstance when he obtain Getou’s body after the event in Vol. 0 is quite unclear. 

As it’s been implied, Shoko held Getou’s corpse after Gojo killed him. But logically, if it was stored regularly, Shoko not noticing that it was gone is almost unthinkable - and no way she won’t tell Gojo, as she is the closest to him among the entire staff. Unless the corpse was sealed away - then stolen covertly. That’s the only way he can obtain Getou’s corpse without anyone in the school noticing. And even then, if by chance the corpse is still inside the school, then he need to bypass the barrier somehow. 

These are several supporting evidences and possibilities.

1. The (first) Death of Itadori Yuuji

The circumstances leading to Yuuji’s death is highly suspicious. We know that Ijichi is only a chauffeur for them and there is still no explaining on how mission rosters for each jujutsushi are set. It’s likely that there is a middle management that arrange mission for each jujutsushi personnel that the assistant managers handle. Someone placing three new students (Fushiguro is already a second-grade, but this applies mainly to Itadori and Kugisaki) to a mission that possibly be a special-grade is really suspicious and totally not screaming “i’m trying to kill these kids”. And this later caused Ijichi to get scolded (poor man).

We’ve also shown that it’s actually the curses working with Getwo that plant the finger (along with the cursed womb most likely) on the juvenile detention center with the purpose of trying to read Sukuna’s current power, which of course become a moot point unless Itadori himself is assigned specifically for this particular mission. 

But Gojo on the other hand readily accuses that the Kyoto Principal to be involved on the Yuuji’s incident - which the principal neither deny or confirmed. Which means yes, he probably has a hand on this.   

They shares the same goal here - to test Sukuna power on top of killing Itadori. Getwo has knowledge on how Itadori’s body works as Sukuna’s vessel, so he probably knows that this incident won’t kill Itadori -- at least Sukuna won’t likely to die. On the other hand, the Kyoto principal is made to believe otherwise - and that’s why he is so surprised to find that Itadori is still alive during exchange meeting.

Which means the one communicating with Getwo is someone higher than the Principal. 

2. The Infiltration to Tokyo School

Before the exchange meeting even begin, Gojo already has his suspicion. That’s why he is sealed out of the way, because he will hinder the plan in more than one way. The only one Gojo has trust on the Kyoto school is Utahime - and she is the only one he divulges his suspicion to.

She then found one of the traitor and asks for help from the first years -- since among all of the students, the three are the safest bet. Megumi is directly under Gojo’s patronage, Yuuji only start to know jujutsu since he ate Sukuna’s finger, and Nobara lived in the boon before she came to Tokyo. 

We are never shown who the other one is -- but it makes sense if Kyoto Principal (or the one above him) is among the one feeding information, if we look over the curse infiltration during the exchange meeting.

The school is protected by Master Tengen’s barrier. It detects everything with curse energy passing it and alarms the school when an unknown curse energy signature is going inside the barrier. Intruders are mostly unheard of -- and if someone can actually pass through the barrier, it’s because of special reason. For example, Touji can pass easily because he has no curse energy. That’s why when they detects an intruder, the first response is to check Master Tengen’s barrier. 

For multiple special-grade curses and curse users to be able to pass the barrier, it must be because of even more ridiculous reason. Even if he has been around for longer than most, the Brain is still too knowledgeable about the barrier for someone not from the school. Unless someone is cooperating with him and feed him details and information, there’s no way he would know about all this. 

And then there’s Mechamaru. The curse’s side knows where is the real Mechamaru when not even the Kyoto students - not even Miwa - know his location. Utahime needs some time to find his location, which means that it’s pretty tricky to find. The only reason the curses can contact and cooperate with Kokichi, is most likely through Kyoto Principal. But on the other hand, Kyoto Principal can also be told to do it by someone higher than him. 

3. The aftermath of Shibuya incident

The various things happening in chapter 137 just reek of strings being pulled from behind the Jujutsu authority, further cementing that the Brain is cooperating with them. Him turned the era into the chaotic Heian-like period, can also because of certain purpose he and the upper management share and try to achieve together.

Now, Yuuta. There must be a reason that Gege’s comment this week is like this. 

It feels like he’s offering himself as the executor for Itadori’s death penalty with  reasoning that Inumaki’s arm cut off. There’s only one unclear hint that the one who cut off Inumaki’s arm is indeed Sukuna - but we’re never actually shown where he actually was during the time Sukuna’s domain hit the city.  

In the latest chapter, I feel like he is for some reason trying to win a favor with the jujutsu authority, going as far as offering a binding vow.

One of the possibility is that maybe he has been informed by Gojo before of the curse user (Getwo) and trying to get into the ranks of the management for the purpose of investigating stuff inside now that Gojo is gone.

And then there’s these decisions - which is very one-sided and disadvantageous solely over the Tokyo school. It’s like an outsider just glance at the situation without trying to investigate deeper and dismiss it as they make these rulings, or someone makes them look over this grave matter. 

And this can be because they are already in clash with Gojo and have been waiting for a chance to get rid of him - along with discouraging people to free and side with him by executing not only Yaga, who directly supervise Gojo, but also Yuuji who has been long since become a source of conflict since Gojo keeps him alive.

Maybe they are working with Getwo because they want Gojo to be sealed in the first place - which required power that the jujutsu authority doesn’t have. Or maybe they are trying to achieve something else aligned with Getwo’s plan of creating another Heian era. While Getou also receiving a death penalty, this is kind of a moot point, since the Brain can hop to another body.

All in all, these are all very suspicious.  

[woozi]

Um this might be annoying but your descriptions on that mutuals as svt members was so wholesome! So I thought if you could name your mutuals and say what you like abt them? Your favorite thing about them kinda? Idk I thought it was cute but you don't have to do it if you don't want to ! 💚💚💚

i went through my following list and i think these are all of my carat mutuals/friends <3 i only included those who i'm mutuals with here on this blog, but with that in mind i hope i included the right people/and all of you! if not, i may have gotten confused with your main and sideblogs. :/ i divided this into 4 sections (because that's how i think of them in my head, lol). so, without further ado, here they are in the order of my following list (from the most recent):

caratblr

@leechaerok (mirelle) - mi!! i love how you are always striving to be positive and that this is something you share and extend to other people! you literally remind me of sunshine and i hope that only good things are coming ur way. ily!!

@ohoshi (cora) - cora my naughty (IN A PLAYFUL WAY) friend <3 cora's really fun to talk to and she keeps you on your toes. idk why i didn't follow her earlier (i didnt know i wasn't following her jsdjkdsjsdk). extremely talented!! also love the upbeat vibe.

@soonyoungs (cara) - same with cora, i've been seeing cara around but never got to interact with her until recently! i love how versatile cara is and how she always always always appreciates even the little things. 10/10 would let her visit my home again <3

@wrongnanab (aya) - i get mom/older sister friend vibes from aya and i think it suits her a lot. she’s also not afraid of reaching out and i love that sm about her!! i relate to her a lot and i love how she’s always there.

@junhaoshua (e) - something interesting always comes up every time i speak to e! i also love her initiative and how well she manages her time. she’s really passionate about the things she likes and i love how she can get people to be interested in them too!

@choibeomgyus (kale) - kale has excellent taste! i love her aes so much. <3 her work is particularly right up my alley and i just love how clean looking they are. kale’s also such a sweet friend, and she’s not shy about showing affection. i love her sm!!

@dokyom (izy) - izy’s one of the first carats i first followed when i got into caratblr and i’m so glad to have made that decision! her work always wows me and although we haven’t talked much, she’s one of the people i always look out for in here.

@seohoshi (nele) - nele, my mutual at heart. <3 the way we became mutuals is so funny and memorable to me, and i can’t say that about most people! i really like the way nele presents herself as well. she can have fun while still being collected, which i find really interesting. i think she’s someone who brings stability, but i can’t really explain why i get that from her. <3

@art-hao (carrie) - goddess on earth. so elegant. a literal virgo. i love how carrie doesn’t want to be too invested in one single thing. that being said, she’s multitalented, yet is skilled in each of the fields she’s venturing in. she also speaks what’s on her mind, and i admire that a lot. what can’t this woman do :/  

@svtreasure (kate) - kate is one of the first people i got to talk to here on caratblr! and although she’s not on here that much, i always look forward to seeing her. i feel like i have to protect her like a younger sibling <3

@uriboogyu (lee) - lee definitely has feisty younger sibling vibes which i think is so cute! like carrie and e, she’s not afraid of saying what’s on her mind. lee is also such a go-getter! the future is very bright for this one <3

@smallkore (kore) - i remember first talking to kore after she tagged all 3 of my active blogs in one post, and i thought it was kind of funny <3 kore is super talented and entertaining! i also think she’s super interesting, and i love her strange picture collection. <3

@rq-s (katrina) - katrina is also one of the first few people who i interacted with here on caratblr! she is so supportive and always has a nice word for everyone. even though she isn’t here much, i always like seeing her around <3

@boosbin (isa) - iconic. legend. will kick your ass. my beloved <3 isa is SUPER talented and she never fails to amaze me. she’s also very relatable and is super fun to talk to! idk i’m just really fond of isa lol

@dk-s (zay) - i love zay’s sense of style and her overall aesthetic! she also is really smart and gets really invested in her interests. i love her dedication and initiative!

@scoupsy (dreamy) - dreamy and i haven’t really talked much aside from interactions here and there, but she’s someone i really admire. i love how she doesn’t take anyone’s shit and knows what she wants. she’s really skilled in her areas of interest and takes pride in her excellent work. love that about her!! 

@julyprince (madison) - i was really shocked when madison followed me back because i love her work so much! i love how she is so consistent and dedicated (honestly, the constant 9 sets say it all <3). she’s intimidating at first, but she is actually so nice and likes reaching out!

@xuseokgyu (belle) - one of my treasured moots. belle is literally the seok to my soon. <3 she doesn’t know it, but i was kind of shy during my earlier days in caratblr, and she encouraged me to be myself just because she was always reaching out to me. i see belle as someone who really nurtures and is just a reliable older sister for me!

@haniehae (anna) - also one of my favorites <3 anna doesn’t really need too much words but u can always feel the love from her by her little tags here and there. she’s also super supportive and very appreciative of the little things! anna is such a cute person to have around. i lalso love how much work she puts into caratblr. also an amazing dancer oh my god???????

@jaemtens (kevin) - like dreamy, kevin and i haven’t really talked much, but i always love seeing him on the dash! one of the people i admire. it absolutely blows my mind how he has a phd (about to have one? :D) and that he’s still somehow able to make gifs (BEAUTIFULLY!) consistently. love that dedication, wish that were me!!

@tearsofsyrup (vi) - vi thinking of me and tagging me on cute stuff really just makes me feel so :’>. i love how easygoing vi is and i just feel light whenever we interact!!

@soonhoonsol (chey) - everyone’s bestie!! chey literally is the go-to person on caratblr. idk how she manages to do it but she’s always so nice to everyone and really does go out of her way to make everyone feel comfortable! chey has a special place in my heart, and my stay in caratblr wouldn’t be the same without her. <3

@iiasha (emily) - you all don’t understand how much i LOVE emily. i can’t even explain why i love her i just do. she’s also one of the first people i followed when i came on caratblr and i’m so glad to have met her here on our own personal hellsite. emily’s the perfect balance of smart, cool, and funny (this sentence sounds lame as fuck but idk how else to put it)! i appreciate how much she likes her job and that’s something i hope to also imitate! i’ve said it before but i also really love how generous she is. emily also made my stay here on caratblr very enjoyable and i always love reading her text posts <3 i think tumblr junation would nearly die without her so go thank her rn. (she also spotted me turning full dinonara without me even noticing that i am one so. <3 /mwah/)

@coupsnim (izzie) - izzie gives me calm artsy vibes.  her work is always stunning omg. queen of red lips (and i think it’s so fitting bc it suits cheol the best imo???). ALSO love how much she loves cheol!! izzie’s just a sweetheart. <3 would also support ur 0 note posts, lol.

@kyeomshine (lyns) - lyns coloring queen <3 i looove how clean and cohesive her work always is! she’s also one of the people who encouraged me to stay here on caratblr (although she doesn’t know it!!). and though we don’t see lyns as much nowadays, i still think of her as the resident dk fan <3 one of the memorable people for me.

@heartgyus (rhys) - my sister (dISGUSTANG!!). my enemy. THE mutual to have homoerotic subtext with. rhys is actually my closest friend on caratblr. she’s multitalented, almost all-knowing, and has prettie vibes. i’m not elaborating anymore from here on out because i just know she’s gonna be a little devil about it on the dms. 🙄

@hanwooz (kellie & christine) - one of my absolute favorite blogs! i think of them as one of the constants of caratblr. i absolutely love their posts sm and just enjoy when they’re around!! (and how do ur gifs always look good omg tell me ur secrets pls <3)

@7ww​ (yasmin) - one of my favorite (if not my favorite) gfx maker!! i love how you can immediately tell something is from her because of their trademark look. we haven’t talked yet, but she’s one of my favorite carats!

@mngys​ (sofi) - coolest person in the world. idk why we’re even mutuals i don’t deserve this <3 sofi always has the coolest ideas and i look up to her sm!! i’ve also seen her from the birdie community before i got into caratblr so i’ve been following her for quite some time now, but we became mutuals here!

aes carats

@vixenjun​ (kaya) - another one of my beloveds <3 i always say kaya is funnie and sexie but she really is, and those are always the first words i think of when i’m reminded of her. one of my absolute favorites. a bestie that's close to my heart. <3

@tiddie​ (deniz) - i don’t see deniz posting much about svt anymore, but deniz is one of my first carat moots (i followed aes carats first). this one really witnessed my jeonghan downfall. :/ so funny. is a great friend!!

@bwaldorf (ramasha) - A SWEETHEART!! ramasha and i don’t talk much anymore but when i think of her i only am reminded of good things. i rmb how much she supported my weird posts back then <3 also iconic aes hello???? 

@vernons (oona) - oona and i are literally just vibing here <3 i actually met her from the aes side of tumblr before i got into caratblr (like deniz) and discovered that she’s also a carat! i really like oona’s presence and i just feel like she’s someone i want to protect. <3 also makes amazing pastries. she’s also so pretty!!!!! (i really like ur hair sm i think i’ve said this before lol) a cute friend. would send u things saying ‘this reminded me of u’, and i’d get heart attacks from it jsdjdjksd

@oldbooks (fawn) - my frog friend. i think it’s so cute that fawn shares this agenda with hao <3 also unexpectedly tags you on things and leaves cute little notes/tags that makes me want to :’). sophisticated taste!! also really friendly. ily!!

@scoups (hani) - hani is so friendly and takes really good care of her friends!! i was intimidated of her at first, but learnt that she’s an absolute sweetheart. i love how she’s also giving us sm svt content when she’s around! hani’s a supportive friend that you’ll always find yourself looking for. 

@snwo (joy) - refined taste. funny text post advocate. i love joy’s vibe sm!! i first got to know her when she had a hao url and i was super intimidated of her. now she’s one of my absolute besties! aside from her impeccable taste, she’s also such a great friend. this one always has ur back. ily ms joy <3

@sataemism (sof) - sof literally the funniest person in the world!! i’m always happy to see her and she brings me so much joy. very lowkey chaotic vibe (the taemin and hoshi urls speak a lot abt this lol), and just a very fun person overall!!

birdie carats

@flowerbeom (kat) - kat, like the rest of those in this category, is someone i’ve been following for a while now as i came back to tumblr because of got7. kat and i haven’t really talked much but always interact with rb/s and likes and i love that JSDJSDKJDSJ that being said kat’s tags are always so enjoyable to read. + it’s so funny as well that when we actually got to talking we didn’t realize we had a lot in common! a very generous friend. gave me a yugyeom pc even though i’m literally from another continent. 😭

@tuanzie (joanna) - content queen. i looove how joanna’s work has a trademark look to them! they’re always so gorgeous and smooth-looking. i love how joanna’s able to speak her mind and would not take shit from people. i see her as someone brave, but not too out there! also someone you can rely on to put reposters on hit lists mwah<3

@gotseventeens (belle) - my name twin <3 belle is actually one of the people/reasons why i got into svt and i’m eternally grateful to her for that!! we’ve been talking since i first became a birdie but somehow we both have 0 recollections of our conversations JSDJDKSJKSDJ tumblr wouldn’t be as fun for me if belle weren’t around ngl. i love how she seems to be mature yet you’re also up for a wild ride with her!! she’s my own personal wonu <3

@secndlife (karoline) - i’ve also been following karoline since i was a birdie, but we only became mutuals here! i love her writing so much and i enjoy seeing her lose her mind over hoshi (mood <3)

@defgyus (val) - i often associate belle with val and i think they have a lot of similarities!! val also gives me the mature yet fun vibe. i think val’s so successful and i admire her a lot!! i love her work sm and just love how she interacts with people. very warm and homey!!

more carat moots i haven't really talked to but enjoy seeing on the dash

@emailclub (emily), @kyeomblr (maddie), @joshuahong (bea), @delicatecy (deli), @mintyseoks, @syuperseventeen (nat), @haol (mali), @svtclub (hannah), @cutiejoshi (siri), @chanswu (mandy), @jeonghannie (myrena), @squishy-woozi (kris), @wonwooo (hiba), @rameniji (rain), @cafevernon (ish), @hoshluv, @minghyu (nicole), @mingyiu (katya), @yeol (tat), @dearkyeom (kristy), @seokgyus (kai), @bbaksu (akemi), @1adyluck (sushi), @shineesbag (gen)

[atheaestus]

✨💛 This is the Amazing Person Award! 🏆 Once you are given this award you are supposed to paste it in the ask of eight different people, who, in your opinion, deserve it. If you break the chain nothing will happen, but it is sweet to know someone thinks you're amazing inside and out ✨💛

Holy shit thank you so much😭😭😭 you've made my day

I'm just going to tag everyone here to save up time :))

@pansexualalexithymia of course because I'm always excited to see what you've posted

@absolutepokemontrash I interact with you the most and you are just as sweet as honey all the time

@reimleff I've been mutuals with you the longest and I do notice when you disappear😂

@vomitlyart I still cant believe we're moots cuz you're just that cool!! I absolutely adore your cosplays

@sekaiiide another one of my moots that i dont interact with much but, embarrassingly so, I lurk on your blogs sometimes😂😅

@chibiponyprincess I see you interacting with my posts every once in a while and I appreciate you so much!

@candykoala0-0 I absolutely adore your art and just your general vibe! I love seeing you in my feed

@arcadejohn127-9 another moot I am so suprised I have because you're just so??? Cool???

[oddcoupler222]

Merry Christmas! I’m a big TWW fan and in anticipation for the published version I was wondering if you could share us the scene in the epilogue where Marg surprises Prof Sansa mid lesson. I vaguely remember the funny twitterverse pic you made for this scene but would love a written version! 😊🎁

Merry Christmas back!

for reference :)

--------------

Eliza didn’t sign up for Women in Contemporary Westerosi Lit merely because Professor Stark was teaching it -- she wasn’t weirdly obsessed with the woman. But she wouldn’t say she didn’t jump at the class when she’d seen that her favorite professor was teaching it.

Even though this was officially Professor Stark’s first semester teaching, Eliza would never forget how she’d been the TA in her first year in university. Dr. Lannister had been a great professor but intimidating at times, whereas Sansa - as a grad student in her final semester - had always had time, patience, and a perfect way of explaining to 18-and 19- year old freshmen about how to go about analyzing literature.

And now, two years later, she was finding out that the internship or whatever, wherever Sansa had gone had made her teaching even better. 

Did you know that Colleen was in this class? Oh wait. Of course you did.

The note her friend, Sara, slid across their shared table jarred her out of taking notes as Professor Stark lectured.

She flipped Sara off as discreetly as she could even as her eyes flickered two rows ahead of her own and three seats to the left. And a wistful sigh escaped her. 

It’s not like she’d known Colleen would be in the class for certain - even though she had a work study job at the registrar office. But she’d thought it would be a pretty good bet, considering Colleen Durand had been in almost all of the social lit electives most semesters. She knew, because of all of these shared classes, that Colleen’s minor was in literature, just like Eliza’s was.

Though Colleen was an Anthropology major, unlike herself. She liked that about her, enough to have spent about two hours reading about what the fuck economic anthropology -- Colleen’s specialty -- was a year and a half ago, when they first shared a class. The first time she’d tossed back her honey blonde hair all perfectly tousled and readjusted her perfect posture as she sat in a front row seat and took immaculate notes -

(notes that Eliza had managed the courage last semester to ask to borrow even though she hadn’t needed them. Using it as a conversation starter was pretty much moot, though, when she then psyched herself out and like. She borrowed the notes, which meant that now, Colleen thought she was dumb or something? Like she couldn’t take her own notes? And then her dumbass roommate at the time had spilled coffee on her desk and got it on Colleen’s pristine notebook)

Peeking up to make sure she wasn’t about to be caught sneaking notes like she was in fourth grade, she scrawled back, We’ve been in class for three weeks and you’re just noticing?

Sara had no qualms about being caught. Well, unlike you, I don’t have a radar for her. It’s not my head that pops up the second she steps into the dining hall, like some bloodhound who caught a scent.

You make me sound like a stalker. Now shut up before Prof Stark notices.

Calm down. She’s chill as fuck. And you make yourself sound like a stalker. golden haired goddess. the light dusting of freckles on her nose. her cute cardigans. those perfectly elegant long fingers that write those amazing notes -

The last thing Eliza wanted was to lose any respect from Sansa and definitely didn’t want to get called out in front of Colleen, who most assuredly already thought she was a dumbass.

Even if she had 0 chances with that gorgeous specimen of a woman, she didn’t want her to think she was a true moron. Also:

YOU PROMISED ME YOU WOULD NEVER BRING UP DRUNK ELIZA’S POETIC WAXING

Shit u right. But I never promised I wouldn’t bring up drunk Eliza being upsetti spaghetti - “i bet shes probably straight. i hate that she prob thinks im dumb. i have an internship in the red keep!!!!! shes not the kind of person who would care about my twitter. i have over 10000 followers!! i am FUNNY! but she is too good for my dumb lesbian human humor!” what does that even mean -

Friendship canceled, thanks for your service. Your severance package is the nothing you deserve.

She turned to rip the paper away from Sara before she could write anything back, just in time to notice the back door to the room creak open. Which was kinda weird because -- what is the point of coming to class if you’re a half hour late?

She flickered her eyes to the front of the room, noticing that Professor Stark’s back was to the room as she wrote on the SmartBoard. This interloper was pretty fucking luck-

“Holy shit,” she breathed, eyes wide.

She started hitting Sara’s thigh under the table, in rapid succession. She and Sara met two years ago, when they’d joined the LGBT Alliance on campus and they’d bonded immediately. For several reasons, one of which being that they were both political science majors.

Political science majors who’d both moved to King’s Landing for college and met the year that Margaery Tyrell came out as a lesbian and won the Small Council election and announced very publicly that she was dating the very professor standing in the front of the room at this very moment.

To say that she (and Sara) spearheaded the online community when it came to freaking out over Margaery Tyrell and Sansa Stark’s relationship as two stunning, brilliant, fucking amazing women who were soft as fuck together, would be an understatement. And despite the facts that they lived in the same city, that she took a class with 1/2 of the ultimate power couple, and that she had an internship at the workplace of the other, she’d yet to see them in the same physical space.

Until now.

“Seven hells, what,” Sara hissed in a whisper as she grabbed the hand slapping her thigh.

All she could do was point behind Sara, as Margaery fucking Tyrell - as gorgeous as ever - slid discreetly into the seat in the back corner of the room.

-------------

Margaery had enjoyed King’s Landing University in her time here. She’d enjoyed her courses for the most part and some of her peers. Mostly, she’d enjoyed conquering college as a part of conquering yet another step on the journey of her life.

But for as much as she enjoyed her time here when she was done, she hadn’t looked back. It was a footnote on her way to the rest of her life, regardless of how fun or successful a footnote it was.

The first time she’d stepped back into the KLU campus had been almost three years ago, that first fateful time she’d tracked down Sansa after they’d had sex. Back when Sansa had been a TA stuck in her little office, looking unbearably hot and cute all at once in her glasses and sweater.

Now, Sansa was still looking hot and cute all at once, as she commanded this class. Her girlfriend stood in the front of the tiered seats and she no longer was flustered or seemed at all nervous about being in front of the group. No, she was in control, and the form-fitting black top with her hair pulled over her shoulder was just doing delicious things to Margaery.

She paused in the middle of her lecture to laugh at a comment made by a studious looking blonde sitting in the front row. And Margaery sighed, unable and unwilling to stop the smile that took over her face as she turned her head to watch Sansa.

Even though her girlfriend had been officially teaching at KLU for almost a month, this was the first time Margaery’s schedule allowed for her to witness it.

She’d seen Sansa pace around their apartment in the evenings before starting the 3 courses she was teaching, practicing bits and pieces of her analysis. She’d laid in their bed, stroking her hand over Sansa’s stomach, as her girlfriend had excitedly told her about her students and the insights they’d had.

Moving in together a few months ago was truly the most genius idea Margaery had ever had.

But today? Her Small Council meeting had been pushed back for two hours. Which perfectly opened up the time for her to witness the final hour of Sansa leading her Women in Contemporary Westerosi Lit class.

“Thank you, Jess, for bringing up Lauren Ambrose’s use of metaphor. I want you to break into your new groups that I emailed to you over the weekend and spend five minutes going over the notes you should have from last weeks reading to come up with any metaphors you found and then we’re going to play a little game with them. Eliza, Colleen, and Michael. Sara, Alysanne, Jon. So on and so forth.”

The students rearranged themselves as Margaery stayed hidden in the back of the room, watching with rapt attention. As Sansa turned from lecture to analyzing the text through a question-dodgeball type game she must have made up.

Even though Margaery hadn’t expected to be bored - because she could watch Sansa do anything and everything and not be bored. Even brushing her teeth, somehow. But she was actually interested in the material, the way Sansa spoke about it.

The hour flew by and at the end of it, as everyone was starting to pack up their belongings, Margaery made her move.

She cleared her throat. “Excuse me, Professor Stark. Any homework?”

She was fairly certain she heard one student state, “I’m dying,” but she was mostly focused on the way Sansa turned, eyebrows already high on her forehead because Margaery knew she recognized her voice.

Gods, but she loved those blue eyes and the way they widened. She smirked as she saw the way her girlfriend’s lightly glossed lips parted in surprise.

“What...” Sansa held her eyes for a long moment before she shook her head. She ran her eyes over her students who were starting to depart. “Um - your assigned reading is online. You all know my office hours if you have any questions about your next paper. Thanks for the, um,” those eyes flickered back to Margaery’s and held, “The discussion.”

She waited a few moments, just watching Sansa make her hands busy by gathering her papers at the front desk, furtively throwing looks over at Margaery. She waited until the majority of the students were gone, with just a couple young women taking their time in the front, before she gathered her jacket over her arm and slid her bag over her shoulder and made her way down.

“What are you doing here? It’s Tuesday, you have your Small Council meeting at 10:30?” 

She never thought she would love having someone know her schedule by heart and check in with her about it. But damn, she loved that Sansa knew it all.

She loved knowing Sansa’s. She loved knowing she could look up at the clock on a given day and know, in general, what Sansa was up to. She even loved sharing a Google calendar.

“It was postponed, so I hightailed it to see my -” she slid her gaze to the group of students that she clocked as purposefully taking forever to pack up, and held back the myriad of adjectives that she would save for when they didn’t have an audience. It was something they were still getting used to. “- beautiful girlfriend. I just really wanted to see you in action.”

Sansa did that biting her lip and swaying a little closer thing that Margaery still adored, “What’d you think?”

She swayed a little closer and consciously lowered her voice. “I think... that I absolutely understand what exactly you meant when you say that watching me in action does something to you.”

Sansa, always genius, keenly caught on to keeping her voice so low and mirrored her, “Oh yeah? You want me to talk about metaphor in the bedroom?”

Margaery lifted an eyebrow, considering. “You know... I wouldn’t hate it.”

The giggle-snort that left Sansa made her stomach dip as she laughed.

“I’ll keep that in mind. Do you have time for lunch?” Sansa reached out and tangled their fingers easily, Margaery’s hand instinctively turning to take her girlfriends.

It was the worst timing for her watch to beep, signifying the timer she’d left for herself in order to cut back through the city to the Keep in time for her meeting.

Genuinely regretful - they didn’t get enough lunches together lately - she sighed. “Not today. But dinner - I’ll bring it.”

Sansa smiled warmly at her, squeezing her hand. She gave her students a fleeting look, before she tugged Margaery in and pressed those soft, pink lips to hers.

She hummed in contentment as she pulled back. “I’ll see you at home, darling.”

She stole another kiss before she left with a grin.

[ms-m-astrologer]

Transiting Mercury enters Leo, pre-retrograde

Timeline (current events in bold):

Thursday, June 20, 2019, 17:11 UT - transiting Mercury enters Rx Zone, 23:56:52 Cancer

Monday, June 24, 2019, 01:14 UT - Mercury’s greatest eastern elongation

Thursday, June 27, 2019, 00:19 UT - transiting Mercury enters Leo

Sunday, July 7, 2019, 23:14 UT - transiting Mercury stations retrograde, 4:27:54 Leo

Friday, July 19, 2019, 07:06 UT - transiting Mercury re-enters Cancer

Sunday, July 21, 2019, 12:34 UT - Mercury-Sun Inferior Conjunction, 28:28 Cancer

Thursday, August 1, 2019, 03:58 UT - transiting Mercury stations direct, 23:56:52 Cancer

Saturday, August 10, 2019, 01:08 UT - Mercury’s greatest western elongation

Sunday, August 11, 2019, 19:46 UT - transiting Mercury re-enters Leo

Thursday, August 15, 2019, 06:53 UT - transiting Mercury exits Rx Zone, 4:27:54 Leo

Big fat moot point time. As we’ll discover over the next ten days, no matter how much happens with Mercury going into Leo, none of it will matter, or “take,” thanks to Merc going backwards.

Mercury is said to be in its fall in Leo, as this is the sign opposite Aquarius, in which Mercury is exalted. Isabel Hickey (Astrology: A Cosmic Science) wrote that when Mercury is in Leo “the mind is dissolved in the light of the spirit (the Sun). On the personal level the ego must be dissolved and vanity and pride must depart.” We tend to be too proud of our mighty intellects. I suspect also that with this position, we sometimes arrange our lives so that our mighty intellects are never truly challenged, choosing instead to be intellectual big fish in intellectual small ponds.

How this plays out for you, depends on your natal Mercury position. It’s critical to have a good grasp on this, for the retrograde zone to really help you out in revamping and improving your Mercury style. And, many of the following aspects  end up being three-peats during the Mercury retrograde zone.

Friday, June 28, Ceres Rx/Sagittarius trine Mercury/Leo, 1:34

First of three. Ceres is retrograde; we’ve been doing a lot of internal work on our urge to nurture and be nurtured, as well as our relationships with our parents and our children, along with (perhaps) some concern for food, gardening, and farming. With this trine, some Ceres issue will trigger something in Mercury. Are our ideas really our own, or are we just parroting something that our parents said? Do we really want our kids to believe just as we do, or are we secure enough to let them think their own way? There’s a Gemini Moon on the 29th, and we may be able to figure it out under her influence.

Placements affected lie between 0:34 and 2:34 of the yang signs Aries, Gemini, Leo, Libra, Sagittarius, and Aquarius.

Sunday, June 29, Jupiter Rx/Sagittarius (17:06) sesquare Mercury/Leo (2:06)

First of three. Grandiosity, pomposity, and vanity of Mercury here. Do we bite off some intellectual undertaking that is far more than we can chew? Are we spouting off about something, only to be shown up? Somehow, exactly how much we don’t know goes on full display with this sesquare. The Gemini Moon will again insert itself into the situation, highlighting our communications.

Placements affected lie between 1:06 and 3:06 of the fixed signs Taurus*, Leo*, Scorpio*, and Aquarius*; and between 16:06 and 18:06 of the mutable signs Gemini*, Virgo*, Sagittarius*, and Pisces*.

Wednesday, July 3, Neptune Rx/Pisces (18:41) sesquare Mercury/Leo (3:41)

First of three. This is more subtle (of course) than the previous sesquare. Instead of some spectacular explosion of failure, this is more like a small but steady leak on our brains. Perhaps this leak has been going on for a long time, and we’re only just now sort of halfway noticing it? We may experience some hitches with creative projects. The Leo Moon’s conjunction with Mercury, early on the 4th, may shed some light on the problem.

Placements affected lie between 2:41 and 4:41 of the fixed signs Taurus*, Leo*, Scorpio*, and Aquarius*; and between 17:41 and 19:41 of the mutable signs Gemini*, Virgo*, Sagittarius*, and Pisces*.

Saturday, July 6, Venus/Cancer semi-sextile Mercury/Leo, 4:26

As this is the only aspect that won’t come up again in the three-peat, I suspect that it’s going to help trigger the work we’ll have to do during Mercury’s retrograde - which begins about 16 hours after this semi-sextile matures. I’m thinking kind words from a friend or partner, or something going so haywire that we have no choice but to reflect and readjust.

Placements affected lie between 3:26 and 5:26 of all signs.

(* - the energies challenge each other)

[keevansixx]

The Future Was Now...

I heard an interesting opinion concerning sub-cultures and why, in today’s age, you almost never see any sub-culture being represented on the streets anymore. When you do spy one of these rare individuals out in the wild, it’s like some rare mythical beast of a thing...fleeting, fierce, and wonderous. 

Welcome to generation V (V as in “Virtual”, and not vain, vibrant, vitriol, vivacious, nor victor) 

The sub-cultures of the past have all died, their digital ghosts haunt the databases like the proverbial zombies of old. Resurrected every so often to wistful nostalgia, and as meme fodder for the youth of today. Gone, are the days of artfully attired denizens of the world... languidly rambling to and fro across the surface of the land, spreading creativity in their wake like massive glaciers carving rivulets in the tapestry of the earth to be witnessed by eyes unseen, and thoughts unbridled. No....those days are long gone and forgotten.

Here I sit, alone in a box of my own design. Shackled to a monitor who’s glow is the only ambient light in the room, I watch the world scroll by in 1′s and 0′s rendered in pixel point perfection into images that my mind perceives as pictures of a world I no longer see, in a land I no longer feel, and a place that only resembles what one would call home. I no longer leave the confines of my prison. No toe crosses the threshold of my room....it’s safe here, and everything I need is in the box....no need to leave, no need to explore, no need to wander anymore. 

I’m told what I should eat...and I do so. I’m told what I should be thinking...and I do so. Anything contrary to the will of the mob is quelled with harsh criticisms, threats, and heavy handed browbeating from the lowest common denominator. “No!...thou shall not think outside the box! Thou shalt follow the thought speak of the masses! Thou shalt not have an original thought or opinion! Those are reserved for the popular chattel that have earned their vanity marks in the digital realm.” I’m to remain a good obedient little digital puppet to the will of the masses. I’m told how I should dress....and I do so. The almighty digital overlords demand acquiescence, obedience, and submission to their cyber-hubris. “No creativity allowed that exceeds that of the common person, lest you offend...lest you shame...lest you make feel....the mighty digital overlords.”

“Sounds like a pretty shitty way to live.”...and you’re right...it is.

It starts on any given day, on any given week, of any given year...

I open the window. the moonlight pours in from a harvest moon I haven't seen since I was a kid, alone in the dark, watching the stars go by. I throw on some shoes that were the huge internet trend a few months ago, everybody just absolutely had to get them to be in the vouge of the moment, and walk to the door. Stepping out side, I hear the chime of the monitor, the chirp chirp of the phone screaming out for my immediate attention “Message! Alert! Come respond NOW!” the annoying braying pings, whistles, chirps, and bells that demand obedience and response. 

I close the door behind me to the sound of stillness...the sonic detritus silenced by wood and glass, and I beheld the night in all it’s splendor...….glorious!

For the first time in a very long while....I have an original thought. 

“What if I'm not the only one..?” “what if, there are others out there like me?” “what if...we found each other?”

Over the many weary months that followed, I slowly weaned myself, bit by agonizing digital bit, from the shackles that bound me to my electronic prison. As each day and night passed, I spent more and more time away. Wandering the empty paths I once trod in my youth. It’s empty now....very few wander anymore outside of those whom make the world turn through service, and the multitude of electronic zombies (E-Zomb’s) faces crammed into phone screens, that move back and forth following their scripted paths of life. Just grunts or the half-hearted handwave to acknowledge that they are still breathing and alive.

I sit alone beneath a large tree in the center of town, watching it all go by...a little notebook open in my lap, where I jot down the most interesting thoughts that pop into my brain from time to time, when I see a purple post-it note pinned to the tree with a thumbtack. On it is an artful picture of an eye wearing a butterfly wing in it’s corner crease, with a small address and time and no designation. I take the note, and put it into my notebook to await evening at the appointed time...curious, but still a little bit cautious.

the sky is a beautiful velvet purple and crimson as the sun sets and I near my destination from the note. I walk along a sidewalk counting the building numbers as I go by, various lamps and street posts begin to ignite into glowing life in the growing dusk. I stop between two buildings, note in my hand, I count the two and note that the number skips one between the two building fronts. I hear old music drifting on the wind between the two storefronts and notice a small painting of an eye with butterfly wings off a ways down the narrow alley between buildings. I step off the well trod sidewalk, and follow the sounds down the alley until I reach a courtyard....like the kind one finds in the special places of New Orleans that aren’t on the tourist maps, nor social media posts.

there are strings of lights everywhere, a few odd pieces of art statues, and wrought iron scattered across the courtyard. sitting on benches are kids in old hippie clothes, goth kids lurking near the stairwells, art kids wearing whatever the hell they stitched together out of a scrap bin and dancing in small groups to whatever was flowing out the speakers surrounding the area. I see street kids, and punk kids, rappers and writers huddled around tables furiously scribbling down lyrics and rhyme. Skaters talking about their latest gnarly shred, plain janes and joes talking about life and oppression....in a word...it was old scenes alive and well and very much kicking in a little courtyard in the middle of nowhere.

I get approached by one of the goth kids and a beautiful hippie girl. They both had smiles on their faces and a welcoming look.

The goth is the first to speak, “hey, new guy....you look a little lost. Anything we can do to help?”

I pull out the purple note and reply tentatively “Not all who wander are lost...”

“and not everyone who do are found....welcome!” beamed the hippie girl.

“well to be honest, it was blind curiosity that led me here, so far....*looking around*....I'm not disappointed.”

The goth dude looks sideways at me, then asks. “so....how long have you been unplugged?”

“About 6 months now, it’s not been easy.”

“Six months? Damn man.....you been alone all this time?”

“Yes....but it gave me time to think, to dream, to see a world I was no longer part of.”

“Wow....that’s deep, Mr. moody.....*eyeroll giggles* welcome to the club!!!” Hippy gal chimes in, “we all found our own ways out of the web in one way or another and sort of found each other by happy accident. You....well, you found one of our calling cards we throw up from time to time for a moot, just to touch bases and stay in touch.”

“Moot???” I reply.

The goth snorts a bit and broodingly says “Moot....a meet-up, soiree, party, get together, picnic, graveyard bash, et cetera, et cetera, et cetera.” with profound dramatic hand waving. ”We meet up a few times a month in various locales to hob nob with the other unplugged, and share ideas or show off what’s been happening in our own scenes. Art, music, poetry, crafts...basically, all the best of us with none of the digital chains......everything’s on the table, and nothing is taboo. Within reason, of course *smirks* get too lewd and the community here is good about looking out for one another....fair warning.”

“Point taken. Understood. So, why the notes? Why not advertise on a board or through alts?”

Hippie gal grins, and says “Because, sugar, we’re old school.....analog, no digital...rockin’ the paper tags like the punks of old. Only those who unplug, and really start to notice the world around them will find us....like you. Notes on trees...that’s my contribution, people rarely ever look at the trees these days...too busy online with their faces crammed into their phones to notice. The goth crews tag the cemeteries and dark places, other kids leave clues in whatever scene they happen to be in, and we cross post the messages word of mouth in our own ways when we find out about the different moots going on across the cities. Tonight, it’s here in the garden with my tribe, next time it could be anywhere...you just have to keep your eyes open up for the clues as they place them. When in absolute doubt...always check the library...the dungeon/dragon kids always cross post every event they hear about in the stacks. We’re off grid baby! the ultimate “fuck you!” to the digital world. No chains, no obligations, 0 fucks given....living the life that was taken from us one soul at a time. 

“Ok, so no online presence. check. Moots posted in randoms if I'm paying attention. check. If lost, check the stack for tags. anything else i’m missing?”

“Well, only thing else is snail....”

“Snail?”

“Snail mail....post office. Look, you’re going to meet people here...If you play your cards right, you might even get land addy’s from some of them. you want to stay in touch? Snail, or wait for the next moot to IRL face time. either way, you’re going to have to dust off those ancient writing skills if you want to stay in the loop. You don’t have to commit to anything...this isn’t an obligation, nor requirement, but it’s old common courtesy to reply when someone sends you a snail. Take a chance! you might just be surprised at what you get.”

“ummm, thanks?”

“No problem....and welcome to the revolution.”

I spend the rest of the evening being introduced to the different groups, watching the event as it unfolds. Being exposed to new ideas, and feelings I haven’t felt for a long long time. I get a few land addy’s from various patrons, and give out mine. It’s kind of nice, being here...in the moment. 

the moot winds down, with groups and couples slowly wandering off into the night. I make my way over to a 24hr diner and grab a bite to eat. a few of the attendees are there as well grabbing coffee, or eats, and we continue conversations we had started a few hours earlier. It was a good night.

I make my way home in the early dawn, and for once, in my long life...I feel a sense of profound peace. Like everything, for just one brief moment in the world, is alright. A new glimmer of hope in my mind, and countless dreams just waiting for me to dream. life....is good.

I open the door to my home, the chimes of my digital masters fall on deaf ears for once, and I sleep the peace of the newly freed...

Sometimes, the most profound acts of rebellion involve the most simple of things, like removing oneself from that which binds you....

Welcome to a new sub-culture...may you free yourself from your virtual prisons, break the chains, and take a journey into the unknown. 

this is Generation V.....signing off.....

[itbeatsbookmarks]

(Via: Hacker News)

The log/event processing pipeline you can't have

Let me tell you about the still-not-defunct real-time log processing pipeline we built at my now-defunct last job. It handled logs from a large number of embedded devices that our ISP operated on behalf of residential customers. (I wrote and presented previously about some of the cool wifi diagnostics that were possible with this data set.)

Lately, I've had a surprisingly large number of conversations about logs processing pipelines. I can find probably 10+ already-funded, seemingly successful startups processing logs, and the Big Name Cloud providers all have some kind of logs thingy, but still, people are not satisfied. It's expensive and slow. And if you complain, you mostly get told that you shouldn't be using unstructured logs anyway, you should be using event streams.

That advice is not wrong, but it's incomplete.

Instead of doing a survey of the whole unhappy landscape, let's just ignore what other people suffer with and talk about what does work. You can probably find, somewhere, something similar to each of the components I'm going to talk about, but you probably can't find a single solution that combines it all with good performance and super-low latency for a reasonable price. At least, I haven't found it. I was a little surprised by this, because I didn't think we were doing anything all that innovative. Apparently I was incorrect.

The big picture

Let's get started. Here's a handy diagram of all the parts we're going to talk about:

The ISP where I worked has a bunch of embedded Linux devices (routers, firewalls, wifi access points, and so on) that we wanted to monitor. The number increased rapidly over time, but let's talk about a nice round number, like 100,000 of them. Initially there were zero, then maybe 10 in our development lab, and eventually we hit 100,000, and later there were many more than that. Whatever. Let's work with 100,000. But keep in mind that this architecture works pretty much the same with any number of devices.

(It's a "distributed system" in the sense of scalability, but it's also the simplest thing that really works for any number of devices more than a handful, which makes it different from many "distributed systems" where you could have solved the problem much more simply if you didn't care about scaling. Since our logs are coming from multiple sources, we can't make it non-distributed, but we can try to minimize the number of parts that have to deal with the extra complexity.)

Now, these are devices we were monitoring, not apps or services or containers or whatever. That means two things: we had to deal with lots of weird problems (like compiler/kernel bugs and hardware failures), and most of the software was off-the-shelf OS stuff we couldn't easily control (or didn't want to rewrite).

(Here's the good news: because embedded devices have all the problems from top to bottom, any solution that works for my masses of embedded devices will work for any other log-pipeline problem you might have. If you're lucky, you can leave out some parts.)

That means the debate about "events" vs "logs" was kind of moot. We didn't control all the parts in our system, so telling us to forget logs and use only structured events doesn't help. udhcpd produces messages the way it wants to produce messages, and that's life. Sometimes the kernel panics and prints whatever it wants to print, and that's life. Move on.

Of course, we also had our own apps, which means we could also produce our own structured events when it was relevant to our own apps. Our team had whole never-ending debates about which is better, logs or events, structured or unstructured. In fact, in a move only overfunded megacorporations can afford, we actually implemented both and ran them both for a long time.

Thus, I can now tell you the final true answer, once and for all: you want structured events in your database.

...but you need to be able to produce them from unstructured logs. And once you can do that, exactly how those structured events are produced (either from logs or directly from structured trace output) turns out to be unimportant.

But we're getting ahead of ourselves a bit. Let's take our flow diagram, one part at a time, from left to right.

Userspace and kernel messages, in a single stream

Some people who have been hacking on Linux for a while may know about /proc/kmsg: that's the file good old (pre-systemd) klogd reads kernel messages from, and pumps them to syslogd, which saves them to a file. Nowadays systemd does roughly the same thing but with more d-bus and more corrupted binary log files. Ahem. Anyway. When you run the dmesg command, it reads the same kernel messages (in a slightly different way).

What you might not know is that you can go the other direction. There's a file called /dev/kmsg (note: /dev and not /proc) which, if you write to it, produces messages into the kernel's buffer. Let's do that! For all our messages!

Wait, what? Am I crazy? Why do that?

Because we want strict sequencing of log messages between programs. And we want that even if your kernel panics.

Imagine you have, say, a TV DVR running on an embedded Linux system, and whenever you go to play a particular recorded video, the kernel panics because your chipset vendor hates you. Hypothetically. (The feeling is, hypothetically, mutual.) Ideally, you would like your logs to contain a note that the user requested the video, the video is about to start playing, we've opened the file, we're about to start streaming the file to the proprietary and very buggy (hypothetical) video decoder... boom. Panic.

What now? Well, if you're writing the log messages to disk, the joke's on you, because I bet you didn't fsync() after each one. (Once upon a time, syslogd actually did fsync() after each one. It was insanely disk-grindy and had very low throughput. Those days are gone.) Moreover, a kernel panic kills the disk driver, so you have no chance to fsync() it after the panic, unless you engage one of the more terrifying hacks like, after a panic, booting into a secondary kernel whose only job is to stream the message buffer into a file, hoping desperately that the disk driver isn't the thing that panicked, that the disk itself hasn't fried, and that even if you do manage to write to some disk blocks, they are the right ones because your filesystem data structure is reasonably intact.

(I suddenly feel a lot of pity for myself after reading that paragraph. I think I am more scars than person at this point.)

ANYWAY

The kernel log buffer is in a fixed-size memory buffer in RAM. It defaults to being kinda small (tens or hundreds of kBytes), but you can make it bigger if you want. I suggest you do so.

By itself, this won't solve your kernel panic problems, because RAM is even more volatile than disk, and you have to reboot after a kernel panic. So the RAM is gone, right?

Well, no. Sort of. Not exactly.

Once upon a time, your PC BIOS would go through all your RAM at boot time and run a memory test. I remember my ancient 386DX PC used to do this with my amazingly robust and life-changing 4MB of RAM. It took quite a while. You could press ESC to skip it if you were a valiant risk-taking rebel like myself.

Now, memory is a lot faster than it used to be, but unfortunately it has gotten bigger more quickly than it has gotten faster, especially if you disable memory caching, which you certainly must do at boot time in order to write the very specific patterns needed to see if there are any bit errors.

So... we don't do the boot-time memory test. That ended years ago. If you reboot your system, the memory mostly will contain the stuff it contained before you rebooted. The OS kernel has to know that and zero out pages as they get used. (Sometimes the kernel gets fancy and pre-zeroes some extra pages when it's not busy, so it can hand out zero pages more quickly on demand. But it always has to zero them.)

So, the pages are still around when the system reboots. What we want to happen is:

The system reboots automatically after a kernel panic. You can do this by giving your kernel a boot parameter like "panic=1", which reboots it after one second. (This is not nearly enough time for an end user to read and contemplate the panic message. That's fine, because a) on a desktop PC, X11 will have crashed in graphics mode so you can't see the panic message anyway, and b) on an embedded system there is usually no display to put the message on. End users don't care about panic messages. Our job is to reboot, ASAP, so they don't try to "help" by power cycling the device, which really does lose your memory.) (Advanced users will make it reboot after zero seconds. I think panic=0 disables the reboot feature rather than doing that, so you might have to patch the kernel. I forget. We did it, whatever it was.)

The kernel always initializes the dmesg buffer in the same spot in RAM.

The kernel notices that a previous dmesg buffer is already in that spot in RAM (because of a valid signature or checksum or whatever) and decides to append to that buffer instead of starting fresh.

In userspace, we pick up log processing where we left off. We can capture the log messages starting before (and therefore including) the panic!

And because we redirected userspace logs to the kernel message buffer, we have also preserved the exact sequence of events that led up to the panic.

If you want all this to happen, I have good news and bad news. The good news is we open sourced all our code; the bad news is it didn't get upstreamed anywhere so there are no batteries included and no documentation and it probably doesn't quite work for your use case. Sorry.

Open source code:

logos tool for sending userspace logs to /dev/klogd. (It's logs... for the OS.. and it's logical... and it brings your logs back from the dead after a reboot... get it? No? Oh well.) This includes two per-app token buckets (burst and long-term) so that an out-of-control app won't overfill the limited amount of dmesg space.

PRINTK_PERSIST patch to make Linux reuse the dmesg buffer across reboots.

Even if you don't do any of the rest of this, everybody should use PRINTK_PERSIST on every computer, virtual or physical. Seriously. It's so good.

(Note: room for improvement: it would be better if we could just redirect app stdout/stderr directly to /dev/kmsg, but that doesn't work as well as we want. First, it doesn't auto-prefix incoming messages with the app name. Second, libc functions like printf() actually write a few bytes at a time, not one message per write() call, so they would end up producing more than one dmesg entry per line. Third, /dev/kmsg doesn't support the token bucket rate control that logos does, which turns out to be essential, because sometimes apps go crazy. So we'd have to further extend the kernel API to make it work. It would be worthwhile, though, because the extra userspace process causes an unavoidable delay between when a userspace program prints something and when it actually gets into the kernel log. That delay is enough time for a kernel to panic, and the userspace message gets lost. Writing directly to /dev/kmsg would take less CPU, leave userspace latency unchanged, and ensure the message is safely written before continuing. Someday!)

(In related news, this makes all of syslogd kinda extraneous. Similarly for whatever systemd does. Why do we make everything so complicated? Just write directly to files or the kernel log buffer. It's cheap and easy.)

Uploading the logs

Next, we need to get the messages out of the kernel log buffer and into our log processing server, wherever that might be.

(Note: if we do the above trick - writing userspace messages to the kernel buffer - then we can't also use klogd to read them back into syslogd. That would create an infinite loop, and would end badly. Ask me how I know.)

So, no klogd -> syslogd -> file. Instead, we have something like syslogd -> kmsg -> uploader or app -> kmsg -> uploader.

What is a log uploader? Well, it's a thing that reads messages from the kernel kmsg buffer as they arrive, and uploads them to a server, perhaps over https. It might be almost as simple as "dmesg | curl", like my original prototype, but we can get a bit fancier:

Figure out which messages we've already uploaded (eg. from the persistent buffer before we rebooted) and don't upload those again.

Log the current wall-clock time before uploading, giving us sync points between monotonic time (/dev/kmsg logs "microseconds since boot" by default, which is very useful, but we also want to be able to correlate that with "real" time so we can match messages between related machines).

Compress the file on the way out.

Somehow authenticate with the log server.

Bonus: if the log server is unavailable because of a network partition, try to keep around the last few messages from before the partition, as well as the recent messages once the partition is restored. If the network partition was caused by the client - not too rare if you, like us, were in the business of making routers and wifi access points - you really would like to see the messages from right before the connectivity loss.

Luckily for you, we also open sourced our code for this. It's in C so it's very small and low-overhead. We never quite got the code for the "bonus" feature working quite right, though; we kinda got interrupted at the last minute.

Open source code:

loguploader C client, including an rsyslog plugin for Debian in case you don't want to use the /dev/kmsg trick.

devcert, a tool (and Debian package) which auto-generates a self signed "device certificate" wherever it's installed. The device certificate is used by a device (or VM, container, whatever) to identify itself to the log server, which can then decide how to classify and store (or reject) its logs.

One thing we unfortunately didn't get around to doing was modifying the logupload client to stream logs to the server. This is possible using HTTP POST and Chunked encoding, but our server at the time was unable to accept streaming POST requests due to (I think now fixed) infrastructure limitations.

(Note: if you write load balancing proxy servers or HTTP server frameworks, make sure they can start processing a POST request as soon as all the headers have arrived, rather than waiting for the entire blob to be complete! Then a log upload server can just stream the bytes straight to the next stage even before the whole request has finished.)

Because we lacked streaming in the client, we had to upload chunks of log periodically, which leads to a tradeoff about what makes a good upload period. We eventually settled on about 60 seconds, which ended up accounting for almost all the end-to-end latency from message generation to our monitoring console.

Most people probably think 60 seconds is not too bad. But some of the awesome people on our team managed to squeeze all the other pipeline phases down to tens of milliseconds in total. So the remaining 60 seconds (technically: anywhere from 0 to 60 seconds after a message was produced) was kinda embarrassing. Streaming live from device to server would be better.

The log receiver

So okay, we're uploading the logs from client to some kind of server. What does the server do?

This part is both the easiest and the most reliability-critical. The job is this: receive an HTTP POST request, write the POST data to a file, and return HTTP 200 OK. Anybody who has any server-side experience at all can write this in their preferred language in about 10 minutes.

We intentionally want to make this phase as absolutely simplistic as possible. This is the phase that accepts logs from the limited-size kmsg buffer on the client and puts them somewhere persistent. It's nice to have real-time alerts, but if I have to choose between somewhat delayed alerts or randomly losing log messages when things get ugly, I'll have to accept the delayed alerts. Don't lose log messages! You'll regret it.

The best way to not lose messages is to minimize the work done by your log receiver. So we did. It receives the uploaded log file chunk and appends it to a file, and that's it. The "file" is actually in a cloud storage system that's more-or-less like S3. When I explained this to someone, they asked why we didn't put it in a Bigtable-like thing or some other database, because isn't a filesystem kinda cheesy? No, it's not cheesy, it's simple. Simple things don't break. Our friends on the "let's use structured events to make metrics" team streamed those events straight into a database, and it broke all the time, because databases have configuration options and you inevitably set those options wrong, and it'll fall over under heavy load, and you won't find out until you're right in the middle of an emergency and you really want to see those logs. Or events.

Of course, the file storage service we used was encrypted-at-rest, heavily audited, and auto-deleted files after N days. When you're a megacorporation, you have whole teams of people dedicated to making sure you don't screw this up. They will find you. Best not to annoy them.

We had to add one extra feature, which was authentication. It's not okay for random people on the Internet to be able to impersonate your devices and spam your logs - at least without putting some work into it. For device authentication, we used the rarely-used HTTP client-side certificates option and the devcert program (linked above) so that the client and server could mutually authenticate each other. The server didn't check the certificates against a certification authority (CA), like web clients usually do; instead, it had a database with a whitelist of exactly which certs we're allowing today. So in case someone stole a device cert and started screwing around, we could remove their cert from the whitelist and not worry about CRL bugs and latencies and whatnot.

Unfortunately, because our log receiver was an internal app relying on internal infrastructure, it wasn't open sourced. But there really wasn't much there, honest. The first one was written in maybe 150 lines of python, and the replacement was rewritten in slightly more lines of Go. No problem.

Retries and floods

Of course, things don't always go smoothly. If you're an ISP, the least easy thing is dealing with cases where a whole neighbourhood gets disconnected, either because of a power loss or because someone cut the fiber Internet feed to the neighbourhood.

Now, disconnections are not such a big deal for logs processing - you don't have any. But reconnection is a really big deal. Now you have tens or hundreds of thousands of your devices coming back online at once, and a) they have accumulated a lot more log messages than they usually do, since they couldn't upload them, and b) they all want to talk to your server at the same time. Uh oh.

Luckily, our system was designed carefully (uh... eventually it was), so it could handle these situations pretty smoothly:

The log uploader uses a backoff timer so that if it's been trying to upload for a while, it uploads less often. (However, the backoff timer was limited to no more than the usual inter-upload interval. I don't know why more people don't do this. It's rather silly for your system to wait longer between uploads in a failure situation than it would in a success situation. This is especially true with logs, where when things come back online, you want a status update now. And clearly your servers have enough capacity to handle uploads at the usual rate, because they usually don't crash. Sorry if I sound defensive here, but I had to have this argument a few times with a few SREs. I understand why limiting the backoff period isn't always the right move. It's the right move here.)

Less obviously, even under normal conditions, the log uploader uses a randomized interval between uploads. This avoids traffic spikes where, after the Internet comes back online, everybody uploads again exactly 60 seconds later, and so on.

The log upload client understands the idea that the server can't accept its request right now. It has to, anyway, because if the Internet goes down, there's no server. So it treats server errors exactly like it treats lack of connectivity. And luckily, log uploading is not really an "interactive" priority task, so it's okay to sacrifice latency when things get bad. Users won't notice. And apparently our network is down, so the admins already noticed.

The /dev/kmsg buffer was configured for the longest reasonable outage we could expect, so that it wouldn't overflow during "typical" downtime. Of course, there's a judgement call here. But the truth is, if you're having system-wide downtime, what the individual devices were doing during that downtime is not usually what you care about. So you only need to handle, say, the 90th percentile of downtime. Safely ignore the black swans for once.

The log receiver aggressively rejects requests that come faster than its ability to write files to disk. Since the clients know how to retry with a delay, this allows us to smooth out bursty traffic without needing to either over-provision the servers or lose log messages.

(Pro tip, learned the hard way: if you're writing a log receiver in Go, don't do the obvious thing and fire off a goroutine for every incoming request. You'll run out of memory. Define a maximum number of threads you're willing to handle at once, and limit your request handling to that. It's okay to set this value low, just to be safe: remember, the uploader clients will come back later.)

Okay! Now our (unstructured) logs from all our 100,000 devices are sitting safely in a big distributed filesystem. We have a little load-balanced, multi-homed cluster of log receivers accepting the uploads, and they're so simple that they should pretty much never die, and even if they do because we did something dumb (treacherous, treacherous goroutines!), the clients will try again.

What might not be obvious is this: our reliability, persistence, and scaling problems are solved. Or rather, as long as we have enough log receiver instances to handle all our devices, and enough disk quota to store all our logs, we will never again lose a log message.

That means the rest of our pipeline can be best-effort, complicated, and frequently exploding. And that's a good thing, because we're going to start using more off-the-shelf stuff, we're going to let random developers reconfigure the filtering rules, and we're not going to bother to configure it with any redundancy.

Grinding the logs

The next step is to take our unstructured logs and try to understand them. In other words, we want to add some structure. Basically we want to look for lines that are "interesting" and parse out the "interesting" data and produce a stream of events, each with a set of labels describing what categories they apply to.

Note that, other than this phase, there is little difference between how you'd design a structured event reporting pipeline and a log pipeline. You still need to collect the events. You still (if you're like me) need to persist your events across kernel panics. You still need to retry uploading them if your network gets partitioned. You still need the receivers to handle overloading, burstiness, and retries. You still would like to stream them (if your infrastructure can handle it) rather than uploading every 60 seconds. You still want to be able to handle a high volume of them. You're just uploading a structured blob instead of an unstructured blob.

Okay. Fine. If you want to upload structured blobs, go for it. It's just an HTTP POST that appends to a file. Nobody's stopping you. Just please try to follow my advice when designing the parts of the pipeline before and after this phase, because otherwise I guarantee you'll be sad eventually.

Anyway, if you're staying with me, now we have to parse our unstructured logs. What's really cool - what makes this a killer design compared to starting with structured events in the first place - is that we can, at any time, change our minds about how to parse the logs, without redeploying all the software that produces them.

This turns out to be amazingly handy. It's so amazingly handy that nobody believes me. Even I didn't believe me until I experienced it; I was sure, in the beginning, that the unstructured logs were only temporary and we'd initially use them to figure out what structured events we wanted to record, and then modify the software to send those, then phase out the logs over time. This never happened. We never settled down. Every week, or at least every month, there was some new problem which the existing "structured" events weren't configured to catch, but which, upon investigating, we realized we could diagnose and measure from the existing log message stream. And so we did!

Now, I have to put this in perspective. Someone probably told you that log messages are too slow, or too big, or too hard to read, or too hard to use, or you should use them while debugging and then delete them. All those people were living in the past and they didn't have a fancy log pipeline. Computers are really, really fast now. Storage is really, really cheap.

So we let it all out. Our devices produced an average of 50 MB of (uncompressed) logs per day, each. For the baseline 100,000 devices that we discussed above, that's about 5TB of logs per day. Ignoring compression, how much does it cost to store, say, 60 days of logs in S3 at 5TB per day? "Who cares," that's how much. You're amortizing it over 100,000 devices. Heck, a lot of those devices were DVRs, each with 2TB of storage. With 100,000 DVRs, that's 200,000 TB of storage. Another 300 is literally a rounding error (like, smaller than if I can't remember if it's really 2TB or 2TiB or what).

Our systems barfed up logs vigorously and continuously, like a non-drunken non-sailor with seasickness. And it was beautiful.

(By the way, now would be a good time to mention some things we didn't log: personally identifiable information or information about people's Internet usage habits. These were diagnostic logs for running the network and detecting hardware/software failures. We didn't track what you did with the network. That was an intentional decision from day 1.)

(Also, this is why I think all those log processing services are so badly overpriced. I wanna store 50 MB per device, for lots of devices. I need to pay S3 rates for that, not a million dollars a gigabyte. If I have to overpay for storage, I'll have to start writing fewer logs. I love my logs. I need my logs. I know you're just storing it in S3 anyway. You probably get a volume discount! Let's be realistic.)

But the grinding, though

Oh right. So the big box labeled "Grinder" in my diagram was, in fact, just one single virtual machine, for a long time. It lasted like that for much longer than we expected.

Whoa, how is that possible, you ask?

Well, at 5TB per day per 100,000 devices, that's an average of 57 MBytes per second. And remember, burstiness has already been absorbed by our carefully written log receivers and clients, so we'll just grind these logs as fast as they arrive or as fast as we can, and if there are fluctuations, they'll average out. Admittedly, some parts of the day are busier than others. Let's say 80 MBytes per second at peak.

80 MBytes per second? My laptop can do that on its spinning disk. I don't even need an SSD! 80 MBytes per second is a toy.

And of course, it's not just one spinning disk. The data itself is stored on some fancy heavily-engineered distributed filesystem that I didn't have to design. Assuming there are no, er, collossal, failures in provisioning (no comment), there's no reason we shouldn't be able to read files at a rate that saturates the network interface available to our machine. Surely that's at least 10 Gbps (~1 GByte/sec) nowadays, which is 12.5 of those. 1.25 million devices, all processed by a single grinder.

Of course you'll probably need to use a few CPU cores. And the more work you do per log entry, the slower it'll get. But these estimates aren't too far off what we could handle.

And yeah, sometimes that VM gets randomly killed by the cluster's Star Trek-esque hive mind for no reason. It doesn't matter, because the input data was already persisted by the log receivers. Just start a new grinder and pick up where you left off. You'll have to be able to handle process restarts no matter what. And that's a lot easier than trying to make a distributed system you didn't need.

As for what the grinder actually does? Anything you want. But it's basically the "map" phase in a mapreduce. It reads the data in one side, does some stuff to it, and writes out postprocessed stuff on the other side. Use your imagination. And if you want to write more kinds of mappers, you can run them, either alongside the original Grinder or downstream from it.

Our Grinder mostly just ran regexes and put out structures (technically protobufs) that were basically sets of key-value pairs.

(For some reason, when I search the Internet for "streaming mapreduce," I don't get programs that do this real-time processing of lots of files as they get written. Instead, I seem to get batch-oriented mapreduce clones that happen to read from stdin, which is a stream. I guess. But... well, now you've wasted some perfectly good words that could have meant something. So okay, too bad, it's a Grinder. Sue me.)

Reducers and Indexers

Once you have a bunch of structured events... well, I'm not going to explain that in a lot of detail, because it's been written about a lot.

You probably want to aggregate them a bit - eg. to count up reboots across multiple devices, rather than storing each event for each device separately - and dump them into a time-series database. Perhaps you want to save and postprocess the results in a monitoring system named after Queen Elizabeth or her pet butterfly. Whatever. Plug in your favourite.

What you probably think you want to do, but it turns out you rarely need, is full-text indexing. People just don't grep the logs across 100,000 devices all that often. I mean, it's kinda nice to have. But it doesn't have to be instantaneous. You can plug in your favourite full text indexer if you like. But most of the time, just an occasional big parallel grep (perhaps using your favourite mapreduce clone or something more modern... or possibly just using grep) of a subset of the logs is sufficient.

(If you don't have too many devices, even a serial grep can be fine. Remember, a decent cloud computer should be able to read through ~1 GByte/sec, no problem. How much are you paying for someone to run some bloaty full-text indexer on all your logs, to save a few milliseconds per grep?)

I mean, run a full text indexer if you want. The files are right there. Don't let me stop you.

On the other hand, being able to retrieve the exact series of logs - let's call it the "narrative" - from a particular time period across a subset of devices turns out to be super useful. A mini-indexer that just remembers which logs from which devices ended up in which files at which offsets is nice to have. Someone else on our team built one of those eventually (once we grew so much that our parallel grep started taking minutes instead of seconds), and it was very nice.

And then you can build your dashboards

Once you've reduced, aggregated, and indexed your events into your favourite output files and databases, you can read those databases to build very fast-running dashboards. They're fast because the data has been preprocessed in mostly-real time.

As I mentioned above, we had our pipeline reading the input files as fast as they could come in, so the receive+grind+reduce+index phase only took a few tens of milliseconds. If your pipeline isn't that fast, ask somebody why. I bet their program is written in java and/or has a lot of sleep() statements or batch cron jobs with intervals measured in minutes.

Again here, I'm not going to recommend a dashboard tool. There are millions of articles and blog posts about that. Pick one, or many.

In conclusion

Please, please, steal these ideas. Make your log and event processing as stable as our small team made our log processing. Don't fight over structured vs unstructured; if you can't agree, just log them both.

Don't put up with weird lags and limits in your infrastructure. We made 50MB/day/device work for a lot of devices, and real-time mapreduced them all on a single VM. If we can do that, then you can make it work for a few hundreds, or a few thousands, of container instances. Don't let anyone tell you you can't. Do the math: of course you can.

Epilogue

Eventually our team's log processing system evolved to become the primary monitoring and alerting infrastructure for our ISP. Rather than alerting on behaviour of individual core routers, it turned out that the end-to-end behaviour observed by devices in the field were a better way to detect virtually any problem. Alert on symptoms, not causes, as the SREs like to say. Who has the symptoms? End users.

We had our devices ping different internal servers periodically and log the round trip times; in aggregate, we had an amazing view of overloading, packet loss, bufferbloat, and poor backbone routing decisions, across the entire fleet, across every port of every switch. We could tell which was better, IPv4 or IPv6. (It's always IPv4. Almost everyone spends more time optimizing their IPv4 routes and peering. Sorry, but it's true.)

We detected some weird configuration problems with the DNS servers in one city by comparing the 90th percentile latency of DNS lookups across all the devices in every city.

We diagnosed a manufacturing defect in a particular batch of devices, just based on their CPU temperature curves and fan speeds.

We worked with our CPU vendor to find and work around a bug in their cache coherency, because we spotted a kernel panic that would happen randomly every 10,000 CPU-hours, but for every 100,000 devices, that's still 10 times per hour of potential clues.

...and it sure was good for detecting power failures.

Anyway. Log more stuff. Collect those logs. Let it flow. Trust me.

Update 2019-04-26: So, uh, I might have lied in the title when I said you can't have this logs pipeline. Based on a lot of positive feedback from people who read this blog post, I ended up starting a company that might be able to help you with your logs problems. We're building pipelines that are very similar to what's described here. If you're interested in being an early user and helping us shape the product direction, email me!

[wavenetinfo]

GRETNA, Neb. — Lanyard Burgett sits uncomfortably outside a coffee shop in an outlet mall, occasionally craning his neck to see whether someone is behind him. Burgett says he served in the Air Force in Saudi Arabia in the 1990s, but was never as afraid there as he is right now.

His angst is over the events of a youth soccer tournament in Nebraska last weekend.

Burgett says his life has been threatened, and his phone has been bombarded with numerous intimidating calls from blocked numbers. He has filed a report with the Sarpy County Sheriff’s Office. Burgett is not normally a paranoid man, but he was awoken Tuesday night to what he believes was the sound of someone trying to break into his house. He’d been asleep for three hours at that point. It was one of the longest nights he has slept since all this started five days ago.

His body language is a contorted mess of anger, fear and resignation. Burgett has been a ref and a coach and a soccer dad, but right now, the volunteer director of the Ray Heimes Springfield Soccer Invitational never wants to be involved in a soccer tournament again. He is seated across from a public relations person named Gina Pappas, and has come with a stapled packet of soccer rules and a roster — evidence, if you will. Less than a week ago, Burgett’s world was grandkids and making sure he had enough medals for his tournament — a simple life in the small town of Springfield, Nebraska, population 1,600. Now Burgett has a P.R. person.

About 30 minutes away, in Omaha, an 8-year-old girl is being flooded with media requests. Mili Hernandez had two TV interviews on Tuesday night, and she was late because her father, Gerardo, couldn’t find her. She was out playing with a friend, oblivious to the fact that she has become the face of a debate over sports and gender rights. Mili doesn’t have Barbie dolls; she has soccer balls. On Sunday, when her Azzurri Cachorros Chicas team was disqualified from the tournament in Springfield, reportedly because tournament officials were convinced the short-haired Mili was a boy, the story took off, thrusting her into international prominence.

U.S. Soccer legends Mia Hamm and Abby Wambach sent their support to young Mili, whose full name is Milagros, which in Spanish means “miracles.”

“You’re inspiring,” Wambach told Mili in a video. “You’re a natural-born leader, honey, and I’m so proud of you.”

But like most scenarios involving parents, youth sports and about 1,000 kids running around a grassy patch of land, this story is far from simple.

Burgett tugs on his plastic water bottle. At least twice in the conversation, he looks as if he’s going to cry. He was torn when he heard about Hamm and Wambach reaching out to Mili. He’s happy that the little girl might get to meet them, and upset that he, in this equation, is portrayed as the monster who created the controversy.

He thumbs through his packet of soccer papers, which are marked in yellow highlighter. Two decades in the military taught him to be regimented, even in chaos. It taught him to follow the rules.

“I would like to have the opportunity to maybe sit down with the parents and talk to them and apologize to her,” he says. “Because I want her to know it wasn’t about her. I’ve got grandkids. I wouldn’t want somebody to do it to them when they play soccer.

“I’d like to give them my side. They might understand; they might not. At least then they’d know from my side why I did what I did.”

Many sides to the story

Where do we begin? With the coach who complained about a rule that has nothing to do with gender confusion? With the anonymous parents who asked why a boy was playing on a girls’ team?

Burgett contends that young Mili had nothing to do with the Azzurri girls’ team being kicked out of the Springfield tournament Saturday night. He says he disqualified three Azzurri teams, not just Mili’s. Yes, there was a dispute over whether Mili was a girl that became even more confusing when the team’s roster, a list that has been used for months, had an “M” for male next to her name. There are 14 girls on that roster, and the only one who fell victim to the typo is the kid who just happens to have short hair. How that happened remains unclear.

But Burgett says the teams were disqualified because they violated another rule. Azzurri played kids on multiple teams in the tournament. Burgett presents a piece of paper that Mo Farivari, the director of the Azzurri soccer club, signed at check-in. Six lines above his signature, in caps, is a sentence that says if illegal players are caught, they face possible removal from the tournament. Then Burgett shows a highlighted page that explains the rule, that a kid can’t play on two teams.

Farivari doesn’t deny that the club had players competing on multiple teams during the tournament. Three girls on Mili’s team also played for Azzurri’s 11/12 boys’ team during the tournament. So did a few players on the club’s 10-and-under boys’ team. But Farivari says they’ve done this before, it’s legal at his tournament, and he was never told that it was against Springfield’s rules. Farivari is also convinced Mili’s team was disqualified because of the gender controversy.

“The only reason he disqualified them,” Farivari says, “is because Mili looks like a boy and is listed [with] a typo on the roster. I went over this to clarify, but he didn’t want to listen.”

The gender flap, in and of itself, is confusing. Soccer clubs have a registrar who types in the names of each player, along with information such as date of birth and gender. Months ago, when Azzurri’s registrar was inputting the 300-plus club names into the Nebraska State Soccer Association system, Farivari says, the registrar accidentally hit “male” instead of “female” for Mili. That error apparently was never corrected.

So for months, Mili played soccer games with the wrong gender attached to her name, and heard nary a peep about it. Even a rules-stickler like Burgett didn’t notice during registration as he pored through hundreds of names last week.

Tournament officials first noticed it Saturday. Burgett was gone for a few hours to attend a wedding, leaving duties to his assistants. That morning, Mili’s team was scheduled to play the Norris Titans Blue team after 11 o’clock. Norris had played earlier that morning, and members of the team were watching a boys’ game before warm-ups.

When three girls who played on that Azzurri boys’ team took the field for warm-ups before the Titans’ girls’ game against Mili’s team, Norris coach Brad Kester took notice. It bothered him a bit. “If that was legal,” Kester says, “I would’ve had two teams in the tournament if I could’ve shared players.”

There are reasons for that rule, Kester says. Temperatures climbed into the 90s in Eastern Nebraska this past weekend, and he says safety is an issue when a kid is possibly playing eight games over the course of three days.

Kester alerted a referee about the rules infraction, but he did not protest the game, which he says Norris won 4-0. It was the Chicas’ first loss of the tournament. During the game, Kester says that his players could hear parents yelling for Azzurri to “pass HIM the ball.” That puzzled Kester. He says the shouts were coming from Azzurri parents.

“It didn’t really matter to me,” Kester says. “We didn’t complain about Mili’s role in the game. It had no impact on the game. It’s not like we saw Mili before the game and we’re like, ‘Hey, that player looks like a boy.'”

After the game, when Kester was talking to a tournament official about the player-swapping infraction, he said at least one parent from Norris chimed in and asked, “Why do they have a boy on the team?” Kester declined to identify the parent. Burgett says multiple parents asked that question.

By late Saturday, Burgett and his staff were investigating both issues. Burgett went through the roster and saw the “M” for male. He says he did not actually see what Mili looked like until Monday night, when he finally turned on a television and saw her on the news. (He says he intentionally avoided TV and the internet before that.) Burgett says it didn’t matter what documentation Mili’s family, or the club, presented. His official document, the roster, said she was a boy. And in his mind, the point was moot anyway, because her team, along with two other Azzurri squads, were being disqualified for the player-sharing infraction.

At 11:12 p.m. on Saturday night, he sent Farivari an email informing him that the teams were disqualified. In detail, he listed four infractions. Three of them were about sharing players. But the first one on the list dealt with Mili.

She was not named — Burgett says he made a point throughout the whole process not to single her out — but the paragraph said the Chicas’ first infraction was having a male play three games on a female team.

“I am sorry to inform of this decision,” Burgett wrote about the infractions, “but cheating is not taken lightly and they will forfeit their remaining games.”

Mili back on the field; Burgett not so quick to return

Gerardo Hernandez’s phone has been ringing nonstop. When he got word Sunday morning that his daughter’s team was being disqualified from the tournament, he rushed from his home in midtown Omaha to Springfield, desperately — and unthinkably — trying to prove that Mili was a girl.

Gerardo was angry, but he had to get there so Mili’s team could play. He brought with him an insurance card to prove his daughter’s gender. He was ready to recite any information they needed.

“I said I have something in my wallet I want to show him,” Gerardo says. “He didn’t even take it. He didn’t care. He said somebody was a boy on the team, and there’s nothing we can do.”

Mili went with him to Springfield. Gerardo says she felt like the whole team got kicked out because of her, and she felt terrible. She cried the whole ride home, he says.

“She went to sleep thinking she was going to play … ” Gerardo says. “Early morning, they told her she was out.”

The Nebraska State Soccer Association did not respond to questions from espnW.com, but said in an email that Springfield tournament officials disqualified an all-girls team for incorrectly listing a member as “male” on its roster in violation of tournament rules. The NSSA said that despite initial media reports, team officials said the squad was not disqualified because of physical appearances but because of incorrect paperwork submitted. The NSSA said it was suspending the sanctioning of the Springfield Invitational until a detailed review took place.

In a statement, executive director Casey Mann said Nebraska State Soccer “was founded on the values of teamwork and inclusion.”

Mili hasn’t had to worry about exclusion in the days since her story went viral. Hamm has invited her to her camp this summer, and Mili plans to go. Farivari says he got a call from Columbus, Nebraska, offering the team a chance to play in a tournament there this weekend, free of charge.

Thursday night, according to Azzurri coach Mario Torres, Mili’s teammates plan to cut their hair after practice as a sign of support for her.

Mili is kicking a soccer ball around again and playing with her friends. She seems ready for things to go back to normal. “I want to forget about all this,” she told Omaha ABC affiliate KETV.

Burgett isn’t sure when things will be normal again, but he has no plans to help out with soccer anymore. “I don’t have the passion for it right now,” he says.

For years, when he handed out medals at the tournament, he choked up with emotion. He’s not quite sure why a strict military man would do that, almost cry when a kid received a medal. He just did.

“I’ve had angry parents, angry coaches when I’ve refereed,” he says. “But I’ve never felt that I had to protect myself.

“My wife and daughter are worried right now. They’re just worried because we’ve never been in this situation.”

9 June 2017 | 10:25 am

Source : ABC News

>>>Click Here To View Original Press Release>>>

(adsbygoogle = window.adsbygoogle || []).push({}); June 09, 2017 at 04:55PM