#Factory_reset

[pileofglass]

Cover art for factory_reset by Kanpeki (feat. unreleased blue version)

[released January, 13th of 2024]

[msamary]

How to Factory Reset MacBook Air: A Step-by-step Guide http://dlvr.it/ST0qd9

[darkandlightsides]

Now she's..Getting confused. The woman tilts her head with a frown that sort of just says "what?".

"Uh...Okay m..maybe im coming on a bit strong...I think I'm breaking her..."

"Uhhh. My. My name is. Sass. Sassy Wass hahah. Uh.. what...What's... Your....Name....Then...?"

There's a cold sweat pooling on her brow.

"Im sorry im so rude. Ive been conceptualized for. For like. 10 minutes. I dont know how. This works."

Darling..? Dear is.. is everything okay? I had an odd feeling and I simply..

(@thestoryisneverastory)

There the woman lay, her avatar stiff with only the gentle bruises of a struggle. Her terminal had still been open next to her.

It was still displaying the commands she made before she...-

The avatar sits up, a goofy grin stretching across her face.

"Why hellooooo gorgeous! Pretttyyy woman hehehhh- oh-- oh shit im in front of a pretty woman. GAH"

Sass stumbles backward and to her feet, scrambling to adjust the glasses on her face that had been slightly cracked.

"Heeloooo...Haha-- Heyyyy MamacitAaaaa eyyyyy wanna-- Wanna hold handss? Aaayyy~"

[a1291762]

The answer to "why didn't that happen before?" is due to something I forgot.

Manufacturing actually boot the device twice. The first boot they login and perform a few last tests, then they run factory_reset and let it boot again.

Because of the upgrade failed banner, factory_reset changed. It would be odd to show that an upgrade failed after running a command that's supposed to return the device to a clean state, so factory_reset twiddles some variables to clear the upgrade error.

The thing is, the first version of this code had a logic bug but when fixing that, another logic bug was introduced. That one means that factory_reset re-enables the rollback support that's only supposed to be for upgrades.

Since the boot variables have confusing names and values, it's easy to get their purpose wrong (which is what happened, twice). So today I looked up the code to definitively document what each variable means, where it is changed, and how the variables interact.

That should allow me to fix this new logic bug without introducing yet another one, while serving as a reference for the next person that has to work on the boot stuff.

On Friday I wrote 2 lines of code.

I'm a developer with decades of experience. I have at times produced gigantic quantities of code but not on Friday.

Sometimes the thing that makes us as developers good is not the ability to write things. Rather it is our ability to understand things.

I was looking at a critical issue that stopped manufacturing. So, about as serious as issues get for us. But while the symptom was easy to describe, determining the cause was not as obvious. It started with a failed upgrade banner. Seen by customers when they turned their new devices on. The banner itself is new, only in the most recent software. But it means that the device was turned on and then failed to boot and then turned on again.

Luckily, I've been working at this company for a few years so 2 things popped into my mind. First, it's actually kinda trivial to trigger this problem because our OS uses boot slots and marks the current slot as good at the end of booting, but it enables logins before that. So if you are quick, you can login to a just-upgraded device and reboot it, triggering the failed upgrade logic (boot back to the other slot, show the banner).

Second, we had a similar stop ship about a year ago. That is its own complex story but the takeaway is that I knew manufacturing booted new devices once after the initial flash, waiting for the login prompt to know the system was up and they could cut power.

With those 2 facts, the cause seems obvious right? Manufacturing was using the new software but their power cut was coming before the boot had completed. But that made me wonder how the previous fix worked. Last time it was corruption. Boot up a new device and it either boot loops or comes up with partially-initialised config. Due to a general panic around "what happens if the power goes out" the fix ended up being that the config partition is mounted without write caching. Tests were done to "prove" that this resulted in a lack of corrupt config due to power cuts at various times.

But if cutting the power now boots the other slot, why didn't that happen before? The config partition also has slots, so it seems to me booting the other slot should not have had any filesystem issues. Alas, I was not able to answer this question yet but I think it's important to find out why.

Because this was a stop ship event, the most important thing was to identify a workaround and let manufacturing know. I did timings on various SKUs and determined that waiting for 90 seconds after the login prompt was enough to avoid the problem. Waiting was the only viable workaround because our device only outputs the login prompt and nothing else, so there's no "signal" to wait for.

So far I'd spent time reading, understanding and timing. Now it was time to make a fix. Because of the reading, I'd noticed a comment about delaying the login prompt back when we switched to systemd. 2 lines in a .service file to make sure that service ran before any gettys (which are what print the login prompt). Our "mark the slot as good" happened to be a service, so I added those same lines to it. Boom! Login prompt now appears later. Timings confirmed it. I didn't attempt to break it by cutting power yet but I've got the gear and scripts to do that from the last issue.