tau-i

[tau-i]

A very very minor thing I have been curious about for a while, and I'm finally asking: why do you calculate queue posting times the way you do? For example, if I set my queue to post 3x a day, naively I would expect it to post every 8 hours. But in reality it posts every 6 hours with a 12 hour gap between days. Why complicate the math like that?

Answer: Hello @circumference-pie!

Buckle up y’all, it’s story time again!

First: nobody who works at Tumblr right now was a part of the work of planning the default queue implementation, which was more than ten years ago. So the full story behind “Why does it work that way?” has unfortunately been lost to the sands of time. All we can do is tell you how it works today and surmise some reasons why. The queue is actually a very clever system and part of how it works explains some of why it works the way it does. Also, there have been attempts to do what you ask—we still have “Queue 2.0” available in your Tumblr Labs settings, which tries to get closer to how you expect things to work.

Anyway! How the queue works today is not actually a queue in the traditional sense. There is no single list of posts that are in “your queue”. Instead, when you “Add to queue” after creating a post, we’re actually scheduling it to post at a future time, as if you had used the “Schedule post” option instead. We’re just calculating that time on your behalf when you use “Add to queue”, based on your settings, and how many other scheduled posts you have already. We use a secondary “index” model, called “ScheduledPost”, to keep track of posts you have scheduled on your blog. We do mark the ones that are a part of “your queue”, but the data model doesn’t keep one list of your “queue” per se.

You can see this in action on your blog, hiding in plain sight. If you add a bunch of posts to your queue, and then schedule a post for a specific future date, you’ll see both in your blog’s “queue” list, side by side. Because technically to us, they’re the same thing: queued posts are really just another kind of scheduled post, relying on the same always-running service to publish scheduled posts across all of Tumblr. Here’s a fun fact: we typically have about ~14.5 million future posts to publish from this list at any given time and are publishing hundreds of these scheduled posts every second.

So when you’re adding a new post to your queue, what we’re doing behind the scenes is starting at the beginning of your “day”, and creating time slots based on your queue settings. If a time slot is already filled, we move on to the next one. That’s why the default queue scheduler works how you describe—we’re trying to fill those “slots” based on the start of the day, rather than trying to divide the calendar day evenly. This just makes it much simpler for us to understand, scale, and predict when our “peaks” will be. At peak times, the publish-scheduled-posts service is publishing tens of thousands of posts in a manner of seconds. We did rewrite that post-publishing part of this architecture a few years ago to improve its efficiency and solve a lot of “lost post” bugs, but we didn’t change how “Add to queue” works.

However, the Queue 2.0 project available in Labs was an attempt to change the queue system to work as you expect—instead of starting at [beginning of day] and creating enough slots to fit [number of slots] every [number of hours], it tries to divide the calendar day into [number of slots] and fit the result back to the original algorithm’s mapping of the day. We never productionized this alternative approach, because it has a few bugs that some blogs hit in extreme cases, and we’ve never had time to fully fix them. It also can cause a bit of weirdness when time zones diverge, like with daylight savings time. Also, a lot of people prefer the default algorithm, and we haven’t thought of a nice way to transition everyone from one to the other. So for now, both options exist, and you can choose which algorithm for queue-slot-generating you want to use. We hope that makes sense! 

While complicated, it is a great example of a system built by engineers to make sense and be scalable and predictable. But sometimes these kinds of systems, while clever, aren’t very intuitive to understand without digging into how they work.

Thanks for your question, and keep ’em coming. 

[tau-i]

Have a plan for your data. Be like the court recorders.

“In 1404, King Taejong fell from his horse during a hunting expedition. Embarrassed, looking to his left and right, he commanded, “Do not let the historian find out about this.” To his disappointment, the historian accompanying the hunting party included these words in the annals, in addition to a description of the king’s fall.“

LMFAOOOOOO rip to that guy

[tau-i]

refseek.com

www.worldcat.org/

link.springer.com

http://bioline.org.br/

repec.org

science.gov

pdfdrive.com

[tau-i]

Some diseases like sepsis or heart conditions have "sense of impending doom" as literally one of the symptoms.

I feel like if that reflex is already built into human biology, drinking an entire vial of distilled virus should trigger it.

I wonder how evolution missed that particular feature.

what is THE worst thing you've ever drank. all liquids acceptable. please tell me what it was, bonus points for why

[tau-i]

I now want to read a fic where Homura escapes her time loop without becoming homucifer and gets six months of normality before the justice system turns over, it turns out that time stop doesn't prevent you from leaving DNA at the scene, and she gets convicted for like forty crimes at once.

Written as a comedy from the confused prosecutor who starts with a delinquency and shoplifting charge for a middleschooler and ends with sixteen counts of murder, extradition requests from both the Russian and American navies on the theft of anti-ship missiles, assassination attempts from the Yakuza, a increasingly tsundere CIA caught between joining in with the Yakuza and recruitment, four separate prison breaks, all ended when the cops find her in bed with the daughter of a executive of a major company, who is only refraining from adding a restraining order on top of everything else because her daughter likes the defendant.

Oh and a "theft of intellectual property" civil suit by some sort of bunnycat alien that walked through his door, and politely explained to him that he did not issue her a license to the process that turned her soul into a rock. Yes, he has filed a patent with the Japanese copyright office. No he doesn't want to explain how.

I've always been so obsessed with this scene. Do you think she just googled "how to make a pipe bomb"? I bet she wasn't even using a VPN. Good thing her search history doesn't stick around in each time loop lmao

[tau-i]

Bugs became way cooler to me when I realized they're basically tiny biological robots. He ain't botherin nobody his prime directive is just [SEEK NUTRIENT]

[tau-i]

BY ORDER OF THE OFFICE OF THE PRESIDENT:

LEBSIAN

BOTTOM TEXT

Hey what's this I'm hearing bout Trump ordering that your gender is whatever yours were at conception? Are y'all all females now or what

[tau-i]

Welcome to Linux! And also motherhood!

Just spent 3 hours installing my first linux distro. That means I have 5 hours of sleep to get, or,,, I supposd, more like 4 after I have… hehe “sexy time” with my new wife. Yeah, that’s right. My computer and I are having a baby, and it’s unix.

[tau-i]

By the way, It Could Happen Here and Behind the Bastards are great podcasts by Robert Evans and friends.

[tau-i]

wow

[tau-i]

My favorite part is having had very nearly this exact discussion and the other person pauses like they're thinking about it but then immediately reveals that they weren't because the next thing they say is "that's such a libra thing to say, you know!"

“Bat swinging at wasp nest” post but I cannot be nice about astrology people. No you did not find the one good or cute or quirky way to believe the quality of someone’s character is biologically pre-determined. Just because you found a way to not base it on race or ethnicity or gender does not make judging someone’s character on an innate and uncontrolled attribute suddenly teehee fine.

I’m even more baffled by the people going “it’s just fun!” “It’s just a hobby!!” Sure if it was something harmless. It’s not. We are quite literally talking about how you intend to judge, treat, view, respect, and interact with someone entirely differently based on an inherent trait. How are you not aghast? How are you not embarrassed? Why are you so insistent on needing to operate on a hierarchy of pre-determined character judgement?

[tau-i]

this website’s easy watch. *dangles a bunch of greek gods like keys*

[tau-i]

the funniest hp lovecraft story is the one where some guy’s family offended an evil wizard who then cursed his entire family saying that all the men would die before they hit like 30. the protagonist is going crazy trying to find a spell to break the curse and then the big reveal was that the wizard was literally just breaking into their house and killing them himself. 

[tau-i]

Fun fact: biology has terrible operational security. Viruses are literally just a suspicious loop of DNA left in the middle of the hallway, except when you run it you explode.

I used to think that it was because viruses have special proteins that make cells think they're food and stuff but then I found out that you can do gene therapy by just dumping a bunch of DNA into the blood stream and waiting for the local waste of mitochondria to go "wonder what this does?"

waking up every morning be like

[tau-i]

Last chance for reblogs lads and lasses!

Star Trek: Deep Space Nine "Past Tense, Pt. 1"

[tau-i]

HAHAHAHAHAHAHAHAHAHA

[tau-i]

Everyone's done it. When I was in college, I deleted /bin/ while trying to figure out how rootjails work. My server was in a different state in my parents' house at the time.

But the best/worst case of this happening is a rumor I heard about the original predator drone. Apparently those things were always death traps in both software and hardware. Of course, since there's nobody on board, that's kinda fine, but rumor has it that one poor pilot somehow managed to order the flight computer to zero out it's RAM.

Not disk. RAM.

While in flight.

Over a war zone.

On the other side of the planet.

With a joystick.

my first experience with Linux was installing lubuntu on my NB200, not being happy with the performance, and then wiping the entire hard drive because i didnt realise that replacing literally every single bit on the drive with zeroes would make it unbootable

in hindsight that was incredibly dumb and like 5 seconds of googling would have probably told me that it was a bad idea, buuuuut thats what happens when you think you're too smart to have to RTFM

nowadays i google anything that im even slightly unsure about, just in case