#I do not think the Devs have enough resources to pull this sort of thing off though

[alchemicaldesignquery]

Gearing up for a Storm: Tank in 5v5

Anticipating a lot of discussion (and "discourse") when the next Developer Blog/Director's Take from Overwatch 2 drops because it is going to specifically target the 6v6/5v5 dilemma...

....Conundrum...

........debate.......

..............concern?

All and all, going to be a pretty defining lecture for a game that I (and many others) have spent years with. Nearly a decade for some of us.

l o l

Until then! There's been some discussion around the recent patch as well as concerns regarding

POWER CREEP

and just what can be done with Tanks. Namely

Are Tanks doomed to be throw pillows of weakness constantly hiding from focus fire and CC effects?

Or Raid Boss style dungeon gods that no one can kill before they team wipe the enemy?

Is there a middle ground?

The difference between busted Tanks and Pillow Tanks at the design level, falls largely on factors from outside of the game; or more accurately

Player defined mechanics vs. In-game mechanics

The former are strategies that players will implement through coordination, targeting, communication, and knowledge-...all of which are not functionally taught or defined within Overwatch. You have to learn how to communicate, coordinate, take high ground, and focus fire, through experience and playing with other players.

Playing Soldier 76 doesn't automatically embed the concept of High Ground, into your head while you execute his kit. It has to be discovered, memorized, and implemented while playing which is a significant abstract in the Cognitive Load of a player who is trying to absorb:

12 different audio cues at any given moment

10 different visual effects at any given moment

18 separate mechanics happening at different intervals and/or simultaneously

all the exceptions, qualifiers, and hidden modifiers with the first three in this list

The on-going demand on a player's memory makes remembering to execute strategies/tactics (which are outside the sensory feedback received from the game) a more distant priority for a lot of players. Most will have to have those strategies/tactics backed up in their reflexes from other games they've played (FPS', MOBAs, Shooters, etc.)

Players who can implement (Information > Knowledge > Execution > Reflex) Player-defined mechanics, are easily poised to hammer another player with an overwhelming amount of In-Game Mechanics.

Focusing the player with the largest hitbox in the game with 5 different kits and the many different interactions they all have is, quite literally, an overload of that Player's own cognitive load; demanding that player operate at such a high level of active memory and execution, that they'll burnout eventually and long before they achieve anything of value in the game-

-or give that player access to mechanics and stats that can outright cancel or deny whole swathes of enemy In-game mechanics just to survive (Orisa Fortify, Hog Vape, DF/Ramm Block, Mauga Overdrive, etc.), which?

Isn't really fun for either side. Not in the long run.

Can we see why Tank Players are increasingly frustrated and/or leaving the game?

The Middle Ground

I'm a firm 6v6 advocate.

But I also know Overwatch 2 should never return to 6v6.

(Because the amount of Dev work that would need to go into making 6v6 viable, would demand a whole new game and they might as well get started on that now and let OW2 have it's lifecycle of 4-5 years).

So what to do about 5v5?

Well...

It's worth noting that, as a format, 5v5 is very difficult to parse from moment to moment; a constant stream of razor thin balancing that shifts and changes as min-maxing of both Player-defined mechanics and In-game mechanics emerge with each new patch.

That is to say, from a design perspective

There will always be a lopsided quality to this format, either within the Roles, within the mechanics, or within the compositions.

But we can mitigate much of the frustration by broadening the options available for Tanks, so that their potency (power) isn't the only option.

Tanks can't be shrunk down. They can't be made significantly weaker and they can't be allowed to steamroll over the enemy either.

Each of those (all of them) would result in pretty big swings of frustration that no amount of collabs or novelty will be able to bury.

Instead, what should be the focus is lower potency with broader (and I mean broader) delivery.

Make each Tank? A Multi-Zoner:

Every single Tank should have the capacity to threaten (not necessarily damage) multiple areas of the map, either through enabling or engaging mechanics.

By creating multiple zones/areas/points of contention in the setting, a Tank can disperse a number of the enemies resources to those varied zones, demanding the enemy dilute their attention or face a possible overwhelming cascade of In-game mechanics that sends them back to spawn.

A few examples of this design execution below, using Zarya as the Tank:

Example - Defensive, Mitigating

Zarya with 2 independent bubbles is a concentration of power around the Tank that begs for all the attention, but also incentivizes her to NOT bubble anyone else.

Giving Zarya one personal bubble + 2 projected bubbles that each only have 100-150 health, would give her up to three (3) separate zones she could apply to any given fight/area of a map.

Zone for her support(s) in the backline, while bubbling a Reaper/Genji going in with Ult, and still having her own personal bubble (and resultant charge to wreck face) for protective purposes.

Example - Offensive, Zoning

Zarya's particle grenades (her Mouse 2/Right Click) when striking the environment, land and persist on the field for upto 2s, or until enemies come within proximity (0.5m).

Large ammo dumps, these can be used to make areas significantly less inviting or engagement worthy to an enemy team, forcing more favourable sightlines and, less, focus fire from the enemy.

Example - Defensive, Zoning

Zarya's Personal Bubble will produce a significant knockback effect if destroyed, giving her room to retreat or find cover incase of swarming or overwhelming by the enemy.

This can also provide her with some significant advantages when played around Graviton Surge and her Particle Grenades.

This is not an exhaustive list of options, but showcases how differing mechanics can result in better zoning potential for Zarya as a Tank, without increasing her potency in a kit already designed to be potent. Broader utility that provides further options for tactical and strategic importance, is how a singular Tank can not only survive but further enable their Team to play the game...

...which is what Tank is all about, really.

[dappersheep]

Food Fantasy: An Analysis on what killed a Golden Goose (1/3)

So first things first, disclaimers! I do not claim nor pretend to know every nook and cranny, ins and outs of the history of FooFan's conception, existence and uncertain future. I do not own the game nor its characters, only the opinions and thoughts stated hereon out.

This was born to vent out my frustrations with how a game like this was abused poorly by its own developer and publisher instead of being nurtured to become its full potential that could have overshadowed and remained better than the likes of Tencent's Tales of Food --I could dream, but it honestly had the potential to be.

Out of respect for the main tag, I personally will not be tagging this post and the following two with the main tag. If you want to tag it yourself with it, that's your choice. Only followers of my blog will see this.

This analysis is divided into three parts: Funtoy, Elex, and the Community. It starts under the cut. Well let's get started.

Funtoy

Ah yes, the creator. The developer. You'd think that with their sudden rise to fame during their global launch, they'd have used the massive profits they earned within the first quarter of 2018 to improve certain things about the game and then trickled it down as quickly as possible towards Global, right? Yeah, I thought so too.

After playing the game since launch, I've seen and experienced way too many things that just hammer in the fact that this is one of the most unfair gacha I've played in years. Some reasons being the following:

(Note: These are experiences ONLY on Global's version, it may also apply to CN being the original server)

⦁ The game's gacha model is aimed towards maximum predation on its players. F2p are forced to either spend some money (and thus tempt them to keep spending after getting a taste of it), or risk not even getting a good ascension of the unit to be useful at all. Paying for the event packs also doesn't guarantee that you would be able to secure a spot in the ranks. In fact, if you can't comprehend how the battle mechanics work, you could even de-rank. Fun way to burn that 800$, huh? At least you have the skin from rebates.

⦁ A little less known thing and probably theoretical at worst, the long joked about spaghetti coding of the game along with an outdated spine technology for the sprites could very well be the reason why a 2D game like this experiences the shittiest lags. Also how easy it is to hack this game with the right know-how.

⦁ Speaking of bad gameplay mechanics, did you know you could spend over fifty Mirrors and not get that final enhancement from +9 to +10 simply because there's absolutely no tangible safety net before +10?

⦁ If you're F2P, this game is terrible in giving you resources to stockpile. Because Funtoy certainly doesn't have a lot of weekly/monthly or even friendly events wherein you can get resources without spending another kind of resource. The Hawthorne event's rewards are lackluster at best, Bingo is severely limited in what it gives, and Recall also doesn't give much for a big event that only happens (supposedly) every 6 months. Did I also mention that daily resource rewards also kinda suck compared to how much you burn in just one event?

⦁ Monthly subs are a scam. Yes, you heard that right. My point of comparison here is Arknights. A monthly in AK allows you to have enough to 10-pull after 30 days, on top of a bit of stamina to help you. In FooFan? You have two monthly subs that do different things and even then, you won't have enough to 10-pull by the end of 30 days, nor is the stamina you get enough to even stockpile and ease the pressure of your need to save for the Gates or that stamina event that suddenly popped up.

⦁ A conga line of 'Must procure this unit at a high ascension to do well in the following events!'. You missed the first Pizza event? Missed the first Turkey event? God forbid, you weren't able to 5* your Beer on his debut? Well sorry, that 5* Black Tea of yours isn't gonna do squat to give you good damage. No, your 2* B-52 also isn't going to do much of anything with his lackluster damage capabilities. If you want a chance to get those event URs again, you have to wait for their pool with laughably limited pulls... and a bloated price to even pull.

⦁ The events starting after the first iteration of Turkey event get even more paywalled. As far as I remember, by the time Minestrone rolled around, an F2P with ample crystal resources can only get 2* at best. 3* and above are paywalled.

⦁ The game has incompetent balancing. The devs themselves likely have little experience in gameplay design and balancing, especially for a game with a growing roster of characters . A prime example of them launching a character not knowing it would pretty much unbalance the game? Look no further than Beer. The guy had to have a couple of nerfs done to him because he was just too meta. You know what's sadder? Before the 'switch' to Brave meta, almost all meta units was built to benefit off the Beer meta.

⦁ Artifacts. Do I even have to explain how the introduction of such a game feature so early into the lifespan of this game essentially fucked over the balance even more? Not to mention, all the more reason you'd be crying with the Gates of Trials demanding so much out of your stamina and crystal resources. F2Ps are again, the ones that suffer in this part. What's their reason? Profit, of course.

⦁ The nerf of resto chests. This was the primary source for people who were saving up stamina for the Gates... until Funtoy decided they were being too generous to their playerbase and dropped the stamina probability rate to 1% or less.

⦁ Terrible UI layout and design. Come on, be honest now, you've lost several thousand of your hard earned crystals buying screws in the fishing shop because you didn't notice that shiny warning in small text and a green button with the crystal image slapped on it, didn't you?

⦁ Look at all these SRs! All of them! Wow, they even outnumber the Rs by at least 80! What's that? There's more URs now too compared to Rs and Ms combined? That can't be real. But seriously, you'd think Funtoy could make some of these SRs into Rs and add them to the perm pool/shard fusion so people aren't stuck pulling Macaron or Dorayaki every time. They could have also populated the Team Up rewards with SRs instead of Rs. But you know... that won't bring them profit. Haha... haha.... Oh and I haven't even told you about the SP class...!

⦁ Lore. Yes, I'm sure by now you're aware that the in-game lore is different from the ones in the non-SP Food Soul bios, in the SP Food Soul bios that sort of ties in with the New World story (that global will never be getting btw). At this point, Funtoy handwaves the confusion away by saying, 'they're all different timelines'. Yes yes, an easy and cliche move to explain how shitty the writing direction went after a while. I don't know what happened, all I know is that lore got weird(er) when they introduced SP Rice.

⦁ They. Keep. Adding. More. Characters! They fail to see that a lot of their earlier players have imprinted on the first few waves of Food Souls and they sadly also fail to properly give some of them more story expansion... or skins. At the moment, they're shelling out so many JP-centric Food Souls because... as I see it? They're pandering to the last bastion of whales they have.

⦁ Merchandise. And I mean a variety of merchandise that isn't using the same official art every time. Like they couldn't afford to commission a couple of artists one or two times to make unique merchandise that would sell. They started too late on that train, and they even made it too hard for anyone not in CN or JP to even procure what already exists. Not to mention, they keep using the same 'popular' set of characters for their merchandise and never really expanding out to making merch for other characters.

These are all the things I can list off at the top of my head why Funtoy as a developer sucks ass. They could sweeten their words all they want, it won't change the fact that they've certainly made way too many bad decisions and found out about it too late, and now they're desperate to keep Food Fantasy alive to keep their profits coming in to make whatever that cat girl game they have and that supposedly 'side-game' FF2 they announced.

There may have been problems out of their control that I or you do not see, but one thing is for sure, they were blinded by greed for the money they were raking in on all their servers at the start, and never actually bothered to invest in more manpower in the right places to improve the game, both gameplay-wise and worldbuilding wise. It's actually saddening that this game could have been so much more with several QoLs and a more fleshed out lore, perhaps even spacing out the number of new units they keep introducing while going back to giving their old units more attention.

That's it for Funtoy. We're moving onto Elex in the next part and boy is that also a trip.

[thedistantstorm]

Dawning Delights 02: Baking Cookies

Summary: Hawthorne invites her newfound family in the Tower to experience a City-Style Dawning with the family that took her in years ago. The holiday is not without it’s charm, or aggravation, and certainly has plenty of surprises in store. A season-inspired, trope-tastic story about a family forged by something greater than blood, finding reasons to enjoy the season - and cherish each other. Main Post

Pairings: Hawthorne/Zavala, Sloane/Amanda, Devrim/Marc

Updated every Tuesday/Friday & both holiday eve and days for Christmas and New Years.

-/

Marc, in typical Marc fashion, revises a midday brunch to a family-style dinner not long after making their arrangement. Apparently he’d got Amanda’s very enthusiastic RSVP and decided she should be treated to a home-cooked meal and be able to provide input on their Dawning Celebration. After all, accepting Zavala into their unorthodox family meant Amanda came along with it.

"Oh, this isn't going to end well." Suraya rolls her eyes as Amanda hollers, "He doesn't know the difference between salt and sugar!"

The Clan Stewardess pulls a beer from the fridge and hands it to the Shipwright. "Take this, and go back to watching your sparrow racing, would you?" When Amanda goes, she lays a hand on his forearm. "You're doing fine."

Zavala sighs. "I don't have to help with these-"

"Yes, you do. You’re psyching yourself out. Stop doubting yourself. Cooking and baking are two totally different things."

"And yet you excel at both."

"Both my fathers taught me what they knew. Besides, I don't think anyone was baking cookies in the dark ages," She reminds him. The exaggeration earns her a subdued half-smile. "Blend in the butter. Slow, fold it in on itself." She watches the consistency change, sticking her fingers into the mixture to test it. "Perfect," She tells him, voice even, not indulgent. Tension bleeds from his shoulders. "I'll roll it out. Amanda picked the cut-outs, right?"

He turns, finding them on the counter near the refrigerator and brings them to her. "Here."

"You can do this too, it's easy." She hands him back one that looks suspiciously like a jumpship. They exchange a glance at the rest - also jumpships and sparrows - and Hawthorne grabs a few more that look like stars and bells, even a little Ghost-shaped one.

He presses the cookie cutter into the flat roll of dough carefully.

"You can't hurt it, Zavala, push all the way through." 

He does, and when he pulls back the cookie cutter, the dough comes with it. Halfway through transferring it to the cookie sheet nearby, it flops out of the plastic. He catches it in a fist, effectively ruining the shape. 

Hawthorne laughs, opening his fist and plucking out the dough. "Put your hand under it next time." She rolls the misshapen cut-out back into a ball and pops it back in the bowl with the other half of the dough. "You really can't hurt these. I promise."

"You've seen me try to cook," He reminds her, pressing into the dough again.

Her hips nudge his. "You've got this, Commander. Besides, Marc will be more obsessed with us baking together than he will about the quality. He can't bake to save his life. Ask him about the time we tried to make Dev a birthday cake. He'll love it." 

Amanda shouts something in the background, kicking her feet against the bottom of the couch. No doubt she has glimmer riding on the results of the race. Meanwhile, Suraya works with precision and experience, rolling out more dough on the counter beside him to speed things up.

"Did Marc try often?" Zavala knows by now that her childhood isn't a secret, but she grows embarrassed easily and won't share much without pointed questions. As it happens, Zavala is curious, wanting to know as much about her as he can.

He'll get the story from her first, and still ask her adoptive parent later. By then, Suraya will be loose enough to point out any inconsistencies, and add her own commentary. 

"No. He looked up ‘easy to bake recipes for kids’ exactly one time," She admits, sheepish still. "Dev was on assignment and due back on his birthday. We did some accidental chemistry." She wipes her forehead with the back of her hand, spearing a dab of flower on it. He spares it a fond blink and meets her eyes once more. "Some cakes use vinegar, and baking soda. It helps them rise. It's also the same thing used in children's experiments, science fair volcanoes, that sort of thing." She shrugs. 

"My Dad - I mean, you've seen him, he cooks by heart. The recipes are just guidelines." Zavala hums in agreement. "So he dumps the vinegar in right after the baking soda, and he's got the mixer on, right?" She claps her hands. "Just, bam! Everywhere. Chocolatey ruin, all over the kitchen." She laughs and he can't help but smile. "I must have been eight or nine, I can't remember. I laughed about it for a week. He spent an hour washing chocolatey goo out of my hair, and three more trying to clean the kitchen."

"What did you do?"

"He all but sprinted to the bakery. We made it home with maybe ten minutes to spare? Dev probably wouldn't have known," She grins, "But I ratted Marc out immediately."

Zavala laughs.

"He was so stressed, and Dev was so confused, and," Suraya giggles, "I didn't know what to do." She snorts, having to stop what she's doing, she's laughing so hard. "Marc started raving. 'The damn thing said it was for kids,' he kept saying. He's sworn off baking to this day, and I'm not kidding, he hasn't made a single dessert." Finally, she composes herself, wiping tears from her eyes. "I am so making him tell you the story. He'll lose his mind."

"Are you sure you want to antagonize him?"

She is. "He's a primadonna. He loves it. He'll start with the hand motions-" She flops a hand towards him, dusting his sweater with flour, lips snacking as she tries to belay her laughter once more. "Oops."

Blue eyes look down and narrow. "Suraya," He warns in a tone that says he’s already determined his next move and it’s not in her favor.

She grins, sheepishly. "Whatever you're thinking, don't."

Dark shapely eyebrows rise and fall in a silent insinuation. 

The next time Amanda comes into the kitchen, there may or may not be hand prints on Suraya's back, and flour in her hair. Zavala isn't faring any better, she sees what she'd guess is a Suraya-sized palm smeared across the seat of his jeans. She takes one look at them and makes an about-face, leaving her empty on counter, sneaking into the fridge for another, unwilling to join in the chaos. They're too busy carrying on to notice her, and she's not about to spoil their fun.

She will, however, bring it up to Marc later, because she knows he'll want to hear all about it.

-/

It's well after dinner when the girls are laying on the couch watching some old holiday feature that Zavala pulls Marc aside. He waits until Suraya is nodding off onto Amanda's shoulder - Amanda is too enthralled to mind, she loves these reruns - to slip back into the kitchen unnoticed. 

"Plans in plans?" Marc queries smartly, unprovoked. His hair is starting to turn more pewter than black-brown these days, but he still doesn’t look a day past forty (he’s thankful to his skin-routine and hairdresser for that). He gestures to the large kitchen island, and the stools beside it.

“Well,” Zavala doesn’t look nervous. Perhaps that’s why it radiates off him so obviously. Marc doesn’t believe in auras, but projection and a little context do wonders. “I do need to ask you something.”

Marc shakes his head, patting Zavala’s folded hands. “When and how? Is there a plan and can I help?”

“You don’t even know-”

His hazel eyes shine in fond amusement. “She sees right through you, doesn’t she?” He muses aloud, to which Zavala shifts uncomfortably, exposed. “That’s not a Devrim trait, you know.”  He tips his head down, trying and failing to conceal his grin before swinging his gaze back up to ethereal blue. “She gets that from me."

"Still-"

"I am absolutely certain my husband gave you the curt, gentleman's heart-to-heart and relished every awkward second of it." He crosses one leg over the other, and smiles. Devrim had called him immediately after this particular conversation, resources and satellites, encrypted comm protocols be damned. After all, it wasn't often that the Vanguard Commander snuck into the wilds unannounced to ask your blessing to marry your child. "You don't need to convince me. I'm here for this. And you.” His eyebrows dip in an insinuation. “So spill."

Zavala leans in, voice dropping to barely a whisper, and Marc scoots closer, visibly vibrating with excitement.

-/

The Guardians waste no time presenting their Vanguard with presents. For Ikora, flaky pastries replace last year's overabundance of donut holes. Zavala finds himself inundated with Gjallardoodles again, within hours of the Tower's festivities officially beginning. Eva, laughing from her place between Tess, Kadi, and Rahool, only encourages it. 

Hawthorne watches from afar. She doesn't have the heart to tell them that Louis doesn't eat birdseed when they bring it by the bag. Instead, she sets it aside for Colonel, who is still roosting comfortably in the Hangar and will - unlike what Cayde insisted previously - eat anything.

At some point, she sees Zavala approach Ikora for some sort of exchange. Pastries for cookies, conversation between them unrushed and pleasant. She can see it in their body language. They may not be close, but it warms Suraya to see Ikora smile, for her to be getting along with Zavala and them to be working as a team once more. 

The year prior had been rough for everyone. Hawthorne and Zavala were still working to well and truly define what was between them when Cayde was murdered. Sure, there were feelings and Suraya's relatively certain they both knew what those feelings were at the time (they just wanted to be careful), but losing Cayde changed things. 

The Vanguard was forever changed. At one point, Suraya had been pulled into a meeting with the Hidden - which was something - and been asked about Zavala's stability (they really did know everything, which was alarming) and her opinions on the Vanguard being necessary at all. It had been a time. Of course, she understood why those questions begged asking. Zavala threw up walls like she did, closing himself off to everyone.

But, Suraya knows better than anyone how to slip in the cracks. She did not push or prod. Did not beg his attention or time or push him to cheer up. She simply existed, constant and quiet, at the edges of the room, the other side of his bed. Waiting. It didn't take nearly as long as she had expected.

Now, she knows, she has Eva to thank for that. There was a reason she'd made sure Suraya only received edibles for her feathered companion (misguided though it was). The old woman had given him a reason to be concerned, a reason to share his overabundance of holiday treats with her, and it paid off.

It took far longer for the Vanguard to make peace and try to move on, for Ikora and Zavala to set aside their collective pride and grief and foster a relationship once more.

Here they are, though. She can hear Zavala's low laugh from here, see the way Ikora reaches out to test one of the better looking cookies in the tin he's holding. They're better together, the both of them.

Louis squawks something agreeable, and she doesn't think on it much when she turns to feed him a little nibble of the treats he'd been given that he does enjoy. Some of the Hunters who know a thing or two about hawking have brought him presents, as well. He hasn't gone hunting yet, and 'tis the season. He's a little underweight as it is. He chitters on like the spoiled brat he is, keeping her attention. She smooths his feathers and grins at him. He looks at her, head tilted, beak parted. Asking for the rest.

"No more," She says, and his cries grow a little more urgent. "You want food, you have to hunt. You're not a chicken."

He beats his wings petulantly before crying once more. Suraya laughs and sends him on his way.

When she turns back, Zavala is entrusting the two tins of leftover Gjallardoodles to Ikora. The Warlock slips the smaller one into a drawer at her station. Suraya doesn't think anything of it.

Back to Main Fic Post

[firecakes-art]

Marker Man Misadventures 12

This is it. This is the world that will truly test my abilities as a map maker. It's as if the game is taking my map making as the actual point of the game and is upping the difficulty to maximum. I will do what I have done since the very beginning when faced with a challenge: adapt and invent new techniques in order to get to my goal.

The soundtrack sounds mischievous and is 6 measures even.

I do not show it on the map, but the parts where the black platforms turn gray is a gradual effect. What that means is that in the game the visible platforms slowly fade away to invisible ones, and then the completely invisible ones are indicated on the map by the gray color. So, basically the entire level is invisible. Not exactly ideal when you're trying to figure out where the platforms are. Not to mention there is ZERO background art. It just looks like I'm walking on air and there are also items that are on air. I mean, I get it that you cannot have background elements or else this invisible theme wouldn't work but I feel like this is just a cheap way for the level designers to not have to care about making the levels pretty.

Fortunately that dumb grid that has existed since the very beginning of the game is still there, so I can use the squares on the grid as a reference of distance traveled. Basically, I'm drawing circles and pushing them to see how they respond to the environment in front of me. It's probably a good idea to do that even if I wasn't mapping the levels out because it will help find potential traps for me. Let's face it: the developers are cheap enough to do that. They did it in the last level of the previous world. Of course I'm going to check for traps.

The part with the bouncy walls is annoying because you have to build around it to climb up. Either use a super jump power up or figure out the reach of the bouncy blocks to make the appropriate bridge.

These levels take no time at all to load. GEE I WONDER WHY. Also, do not try to force shut the game when you lose your last life. Just like with that water world, you will lose all of your progress. The only difference is your save file is just straight up gone instead of glitchy. Perfect. Watch out for long falls, and always check where the edge of a platform is before building or making a jump.

It's like a combination of the previous two levels, only simpler.

You sort of don't know which pits are places you can go to or places that will kill you. It's a safe assumption that if a shape you draw in it falls outside your view then it will probably kill you. Another really dumb thing is that sometimes you can draw shapes inside walls. If you're just making random guesses it's not entirely clear whether the successful shape is placed in an area you can walk to or if it's inside a wall.

You doubt my ability to try and make every level's title some kind of pun or reference? Just watch me.

These levels got homogenized really quickly. I had to replay this level and I had no memory of its existence.

Markers are cool. I like markers. Please give me more markers.

So, the first spiky wall area has a little bump before you fully pass the obstacle. If you're me, this means you'll get stuck under the wall, and before you can register that you need to jump, the spike wall crushes you. Note that when you game over, you only gain one life back. So every time this game decides to pull a stupidly cheap trick on me I have to redo the whole world over.

Wow. We have a new contender for the most basic level in the game. Congratulations.

One strange thing is the area under the exit. You can draw shapes under your platform and watch them disappear after falling a bit. No need to worry about crashing your game, because the developers placed triggers under the map that will delete shapes drawn out of bounds. They did at least one thing right there.

It's a level with invisible blocks and things and stuff.

Snoop around enough and you may find the coin. Or a spike pit. Same thing, eh?

I should be more upset at how hard they're trying to hide the coin and exit, but I think I'm too desensitized at this point to care.

Oh... I figured it would take until level 150 or something before the game was over, but maybe it's ending early! If you manage to die here, just remember the trick with the water world when the game over glitch happened: let the game over screen appear, continue, reset, and load to be at the level where you died. Isn't it fun when you apply what you learn?

Oh yeah, the point of the level. The giant ball is back for another rematch, and you have to scale down the level pretty fast in order to beat it (unless you're using power-ups). It took me a few tries before I was able to beat it. Be careful with some of the drops, as they are enough to damage you. After making it to the bottom, stand on the blue platform and let the ball push the other one down, flinging you upward.

Do I... really need a world map for this one? Yes? Ugh fine.

No no no no no. Nope. I'm not waiting any longer. This game is going to be finished in this post whether I like it or not. It's time to end this.

The cutscene showed the dog and marker man jump into some kind of teleporter. The result is that we are now on some space rock or something. It looks like they stuck with the themes of space too, such as making the level vast, yet empty. Hmm, not really a theme specific to this world, but oh well.

Long Boy

WHAT DID THEY DO TO SHOE BOY? It now has an oval head. I guess it's supposed to be an alien. It's really funny how they attack because they rotate their elongated head when they kick, and it just looks funny.

Ugh, these types of levels are the worst. Huge open spaces with lots of emptiness and lack of features between the platforms. It makes the game boring and frustrating to map and play, so why do it? Especially the road to that two-coin exit. Why make the player walk so far to get there? The developers could've easily added a path down from the coin to the hallway.

Alien

A projectile-based enemy, just like all the other projectile-based enemies. This one has a scarily fast firing rate, making it a very dangerous long-ranged enemy. Its barrage of bullets can also make quick work of drawn shapes. Be careful. When trying to block its bullets, don't just put a single line over it, as the place the bullet spawns can bypass the line. Place like, 2 lines. I don't know. What's interesting is the developers actually managed to make the bullet spawn from the gun's barrel instead of the top left of the sprite. Good job. You're learning! It's a little late for that, but whatever. This does mean you may need to place lines completely covering the alien before the projectiles are stopped.

Same glitch as last world. Do not force stop the game on gaming over. Let it happen, continue, exit, then get back to the level you died on.

The spike walls move at different frequencies. I recommend activating super speed and just focusing on the bottom one moving out of the way. Seriously, you better have some markers coming into this world or you're in big trouble. Actually, there's an easier way to get past this part. When the bottom spike wall is hiding under the floor, lodge a line between the floor and the wall and it will get stuck there, allowing you to pass without doing anything ridiculous.

Pleeeeeease give me more markers. I actually have to conserve resources in this world and it's annoying. There's simply too much terrain that needs to be covered that will inevitably use up all my markers. This is different from the lava world because in the lava world there were at least interesting elements in the levels that made you think on how to approach the problem. In this world it's just a battle of attrition.

Speed Cheese

It's cheese on.. wheels? This is probably a reference to moon cheese. It quickly moves back and forth, simply knocking anything that touches it out of the way. It deals no damage.

I think I know why there are so many dead ends here. The game just really doesn't want the player to have any markers in this world. How rude. This is why people like me have to come in and bright light to the correct pathway to victory.

Oh yaaay. Another level where you just have to look everywhere to find the coin and exit. The part of the map where it says SLIDING has no friction. Marker man will not stop moving on that platform until a wall is hit. The exit is kind of dumb to get to. In order to even see it you have to either stand on the platform with the alien (which noone is going to do) or build up and then look up. I recommend just making a giant rectangle on the corner of the map and then building off of that to the exit.

This is the realm of the shoe boys. Their final congregation is at this location. They are planning big things. They already have presence in just about every world. Their plans to finally take over the world of marker man have started to unfold in this meeting area, and you are in the middle of it all.

This is pretty straightforward, isn't it?

Beware of the many dead ends in this level. You get one marker in here. Well, there is a second one, but it's in a trap so I'm not counting that one.

What. Why is the map so linear? Why is there a pointless dead end to the right with a moving platform? Why is the map small? Why is there large stretches of nothingness? I don't understaaaaaaaaand. Maybe the devs want this game to end as bad as I do. Fair enough.

We all want this to be over! Just walk down this straight pathway, hug the left, and you'll get your coin, and you'll get your exit right above the coin. It's that simple. We even went ahead and put the useless parts of the map all on the right side so you don't even need to travel there. I'm doing you a favor, but don't tell my boss I'm doing this. These are supposed to be the hardest levels in the game. Go forth!

Uhm. What. WHAT.

This is the final level. There is no indication that this is the final level. There is no dog seen anywhere, and the level number didn't jump to the range where the dog levels appeared. I simply walked into the exit, my level got reset to one, a cutscene played, and the game is over. I think it's really fitting that the background track that plays throughout the cutscene and the credit rolls is the game over music, because did I really win by beating the game? Everyone's a loser for trying.

But hey. I get a star next to my save file now. I guess any recognition of me going through this torture is better than nothing. I was curious to see if this star meant some sort of change in the game, like a new game plus, but it doesn't seem like it. My markers and lives left carried over from the previous game to this one, but that's about it.

And.... that's it. The game is done. I mapped every level. Here is the final world map.

Thanks for reading. However, there is still one final post to make...

[douchebagbrainwaves]

HERE'S WHAT I JUST REALIZED ABOUT SENSE

What's particularly dangerous for founders is the way they wait. Prestige is like a powerful magnet that warps even your beliefs about what you enjoy. I think, is which 52% they are. Google search I've found myself nostalgic for the old days, when Google was true to its own slightly aspy self. 1-n. And yet both have the same answer: 1/1-n. Both make it harder to become profitable. But the dictionaries are not doing a very good job.

Another thing that keeps people away from starting startups is the pool of potential founders. All we can do is encourage people to do unpleasant work, with money and prestige. There may be more pain in your own company, like Wozniak did.1 And it did not seem to be any sort of work I'd prefer? Occasionally the things adults made you do were fun, just as, occasionally, playing wasn't—for example, an eminent investor who would invest a lot, but will be easy to convince.2 Though it sounds slightly paradoxical, if you asked random people on the street if they'd like to do, if they could, is wait. You should always talk to investors serially, plus if you only talk to one investor at a time, they don't think of themselves that way.3 You have to be on a trajectory that leads to going public. Some are just too slow to become profitable. For them the right approach would be to hand the company over to a professional manager eventually, if they could find one who was good enough.

Don't reject an acceptable offer that would be the answer. Sometimes an investor will say yes, know what the timetable is for getting the money, because they're already running through that in their heads. Start by building a site for Harvard undergrads to stalk one another? Actually they've been told three lies: the stuff they've been taught to regard as work in school is not real work; grownup work is not your favorite thing to do, how good you are at taking orders, how much does that investment have to improve your average outcome for the company with the addition of some new person, then they're worth n such that i 1/1-n is 1.4 On the blunderometer, this episode ranks with IBM accepting a non-exclusive license for DOS.5 Treat investors as saying no till they unequivocally say yes, in the most literal sense, not news: there is nothing new in it. How much stock should you give him? Some investors are known for deciding quickly, and those are extra valuable early on.

It's the principle of a market economy. But although it's a mistake for investors to care about price, a significant number do. Of course, Internet startups are still only a fraction of the former. Then one of their parents introduced them to a small investment bank that offered to find funding for them to start their own, so they did. There probably are other fields where relentlessly resourceful is definitely not the recipe for getting people to give talks, write forewords, serve on committees, be department heads, and so on.6 A good running back is not merely that you'll spend too long on tedious stuff, it will rot your brain. Surely one had to force oneself to work on certain things. Don't try to look into the future because a the future is unpredictable, and indeed that the reason they have to decide quickly because you're running out of money and b they can spend their time how they want. In a sufficiently connected and unpredictable world, you can't seem to get those by hiring them. The least ambitious way of approaching the problem is to start from the other end, and offer programmers more parallelizable Lego blocks to build programs out of, like Hadoop and MapReduce. There is a similar social component to the transformation that began in Silicon Valley and common in a handful of founders who could pull that off without having VCs laugh in their faces.

It's not hard to express the quality we're looking for, that leads to other questions.7 Which is not to sell more than 25% in phase 2. An investor, the thought of our investors used to keep me up at night. In most startups, nearly all the costs are a function of the situation and the people involved.8 In fact, what I like about publishing online is that you can write what you want to know whether you're about to plow through a block of foam or granite. Prestige is like a powerful magnet that warps even your beliefs about what you want till your thirties or forties. As societies get richer, they learn something about work that's a lot like what they do.9

Notes

These range from make-believe, which usually revealed more than make them want you to behave like adults. But politicians know the electoral vote decides the election, so they made much of it.

This is why I haven't released Arc. Siegel points out that another way to avoid faces, precisely because they believe they do on the side of the leading scholars in the nature of the growth is valuable, because the rich have better opportunities for education. What I should add that we're not professional negotiators and can negotiate on the software business. We have to talk to corp dev people are like, and astronomy.

0001.

You'll be lucky if fundraising feels pleasant enough to turn down some good proposals too. I was living in a world with antibiotics or air travel or an acquisition for more of it, and on the other cheek skirts the issue; the point of view: either an IPO.

If you want to.

The situation is analogous to the minimum you need to offer especially large rewards to get as deeply into subjects as I explain later.

I overstated the case, because some schools work hard to say that YC's most successful founders is by calibrating their ambitions, because despite some progress in the country would buy one.

The golden age of tax avoidance. It's like the other hand, they say. Bullshit, Princeton University Press, 1983. Writing college textbooks are similarly misleading.

VCs. Big technology companies. He was off by only about 2% of the things I find hardest to get endless grief for classifying religion as well.

[theonewiththeeyebrows]

My OMGCP!AU Headcanon

Gosh this got super long. 

Character dev for Jack!Derek and Bitty!Stiles (and some of the others) -- a bit different from OMGCP!

Warning/Disclaimer: I know nothing about Hockey FYI. I have never even seen a game. All my knowledge comes from OMGCP, wikipedia, the NCAA, NHL, and various other HL websites/resources. OMGCP belongs to ngoziu you can read it omgcheckplease it is a brilliant story (I love it so much!).

Jack!Derek - Son of Talia & Jerome Hale (The Hockey Power Couple); Brother of Laura Hale (2016 Olympic Figure Skating Gold Medalist)

Talia Maheegan was of the few non-Goalie women Hockey Players in Professional Men’s Hockey. She was part of the Women’s Hockey team at Beacon Hills University before she was recruited by Leksands IF (Elitserien, now the Swedish Hockey League). A few years later she was offered a contract by the Bakersfield Condors. She was ready to be closer to home, so she came back although she was a woman, when a few of the guys in the NHL team got badly injured, she was moved up to the Oilers roster. 

She met her future husband, Jerome Hale, when the Oilers played the Bruins in her rookie game and she gave him a nosebleed. She was the first female Alternate Captain for the Oilers when they won the Stanley Cup in 1984. And the first female Captain when they won in 1985. She retired in 1986 when she was pregnant, had twins Laura and Derek. 

Jerome Hale played for the Bruins but was traded in 1985-86 he was the Captain of the Oilers from 1987-1993 - Derek pooped in the Stanley cup twice during that time - in 1987 & 1988. 

Derek and Laura started skating young, Laura wasn’t into Hockey, but she started enjoying figure skating, and took that route. 

(Strike throughs are for humanAU)

Derek looked up to both his parents and wanted to follow in their footsteps. He went into the Juniors where he met Kate Argent (maybe Kate’s dad hated her and Talia for being women in a Men’s sport or something, so there was a lot of sexism and prejudice there, and internalized misogyny maybe? If werewolves exist in this verse then she’s from a werewolf hunter family as is canon and she is using him to get info as is canon). She was beautiful and ambitious, and Derek and Kate became good friends, best friends even... then more. There was a lot of pressure to do well in the juniors as it would lead into the draft which would lead to the NHL. He worked hard and honed his skills for two years before he could enter the draft.

Kate’s brother was also in the Juniors, but he was reaching the maximum age for both the draft and the league. There was a lot of talk about how Derek was top pick. Kate would keep getting in his head, Derek was on anxiety meds since he joined the juniors, Kate knew and she kept telling him to take more so that he wasn’t off his game. Just before the draft picks his parents, younger sister, aunt and uncle got into a fatal car accident with no survivors (a drunk driver drove them off the road If werewolves exist -- Gerard put a hit on the family and got them killed -- but obvs. can’t get Derek and Laura now it would be too suspicious). Laura was at a Figure Skating meet in Europe. 

Derek took his anxiety meds, alcohol and something that Kate slipped him and had a reaction that sent him to the hospital. He wakes up to Laura slumped at his bedside still in her leotard. He pulls out of the draft, the media calls it an overdose, he doesn’t correct them. He goes to a rehab center after his family’s memorial service. Laura sorts out everything with the legal team while he is in rehab. 

When he gets out he supports her while she continues working on her figure skating and he tries to figure out his life without hockey. And he can’t. One day Laura comes in throws down a Beacon Hills University Pamphlet on the counter, and when Derek looks at her questioningly, she says “Ma went there. It’s where she truly fell in love with Hockey. They would want you to be happy, Der! I want you to be happy. You’re miserable, please. Think about it. You can major in History and play Hockey. I already called the coaches, they’ve already said they’d give you a Scholarship if you wanted it.” 

Derek gets on the ice for the first time in one and a half years. He’s rusty, but it’s in his blood. He trains hard for 3 months before he thinks he’s fit enough to go to college. He applies anyways. He gets in -- even without the Scholarship -- he was good enough to. 

Bitty!Stiles - Son of Claudia (former Figure Skater) and Sheriff Noah Stilinski

Stiles’ mother dies when he is eight, and Stiles is kinda lost after that. When he’s 10 he finds his mom’s cooks books and he feels a connection to her again so he starts baking. He bakes more than he or his dad can eat, so he takes the excess to the Sheriff’s station (where his dad works) or to the farmer’s market where he makes money off of his pies. 

There is this huge rift between him and his dad, and he can’t seem to figure out how to mend it. He loves his dad, and he knows his dad loves him. But his dad started drinking after his mom died, and his mom was the glue in the family. Sometimes his dad looks at him and gets this pained look in his eyes and Stiles thinks it’s because he looks so much like her. 

Stiles started ice skating with his mom because she used to be a figure skater. She taught him all sorts of neat tricks and even tried to get him to enter a few competitions, but after she died it became another thing that became harder to do. He still went skating every weekend, but he didn’t really feel like figure skating anymore. 

His dad coached peewee football, so in an effort to bond with his dad, Stiles joined the football team but when he gets tackled within the first thirty seconds, and ends up with a compound fracture, it instills the fear of God in him, and he refuses to ever go back, regardless of what his dad says. 

How he ends up playing Hockey is a mystery to him because Hockey is a high-contact sport just like football. But he’s in a co-ed no-contact Hockey team and he meets some amazing people. He starts enjoying being on the ice again, and he starts feeling another connection to his mother -- so he starts figure skating again as well -- just a little bit -- nothing like the fancy stuff his mother made him learn. 

Stiles’ was always the underdog, he was bullied in high-school because his mother died. He was bullied because his dad was a policeman, he was bullied because he came to school in a cruiser, he was bullied because he looked weird, or talked weird, or had ADHD. He didn’t have friends (not for lack of trying). He was called homophobic slurs because he did figure skating and couldn’t take a hit (he’s bi, thanks a lot for bi-erasure assholes! at least call him the bi-phobic slurs!) He can’t find a place for himself so he starts a v-log his Junior year of high school. For some reason people enjoy his sarcasm coupled with the sweet baked goods he tutors them on. So he keeps making the videos. 

His dad can’t afford to send him to College, but he has a few options though -- a merit Scholarship (but his ADHD make it hard for him to keep his grades up), a Federal Load (does he want the government hounding him for years after he graduates), a Sports Scholarship (can he get into the NCAA?). He applies to BHU because Beacon Hills is still in California (so he’s still close to his dad, but far enough away that it’s a few hours away by road or at least a short flight away). He send a tape of him playing Hockey as well as figure skating to the Coaches along with his Scholarship Application. He didn’t expect to -- but he gets in! His dad is so proud. He comes out to his dad, but his dad doesn’t believe him. Stiles doesn’t let that bum him out though -- he’s going to college. 

BHMH 

Isaac, Boyd (Sophomores) - Live in the Attic the Haus - best friends. Boyd is quiet and constant. Isaac is sassy and loud. Together they are constantly coming up with all kinds of shenanigans. They are the ones who handle all the parties and know pretty much everyone there is to know on campus. 

Jordan (Junior like Derek) - Lives at the Haus - is everyone’s best friend. But isn’t wild/crazy. Is wildly popular. He is beautiful and used to be a model. He is a women’s and gender studies major, but he’s also studying pre-law and criminology. 

Greenburg as suggested by Nonnie (Senior) - The metaphysical Goalie - LOL - I couldn’t think of who else to put here - I guess I could have added Peter (but I really don’t like him, maybe he can be a lax bro) - gives his dibs to Stiles. 

Erica, Allison (Sophomores) - Non-Goalie team-mates who initially faced sexism from the senior teammates (not from Derek or Jordan) but proved themselves after their preseason when they took down the biggest player skate circles around all the other players, not giving up the puck even once (but also refusing to score because the guys on their team are assholes). They live in an apartment near the Haus together. They were not surprised when Jordan got the dibs instead of one of them (Derek got dibs from the outgoing captain his freshman year, when Derek was voted incoming Captain). 

Scott (Freshman like Stiles) - Stiles’ Roommate, becomes Stiles’ best friend. Wants to be a vet. Is a Derek Hale fan, but doesn’t like Derek in RL. Is a cuddle monster. Is smitten with both Kira and Allison.

Kira is the Manager (Sophomore) - Doing a semester in Japan (Studying Art and Mythology). Cannot ice-skate. 

Lydia is a girl who Stiles crushes on in one of his classes, and he invites her to Epikegster and she says yes (but as a friend). She is into Jordan. When Jordan finds out he tries to dissuade her even though he is hella into her brains and beauty combo. Stiles is bummed, but he can’t deny his friend someone awesome like Lydia. So he totally gives Jordan a thumbs up, and tells him to pursue it. Lydia and Stiles get to know each other over time and realize how great they are as friends, anything else would have been Chaos. This is also around the time Stiles starts to realize he has major feelings for Derek. Feelings that make his stomach swoop, and his breath catch, and blood rush to his cheeks, even though Derek doesn’t really do anything (other than buy him his favourite coffee, or bake a perfect lattice crust for their American Food and Culture Class, or take his cold finger into his warm hands and rub them, just to hopefully bring some warmth because he knows how cold Stiles gets (Mt. Shasta gets really cold compared to San Diego). 

Jackson and Peter can be the dreaded Lax bros. idk. Peter isn’t related to Derek. 

[Part 2]

[hopefulfestivaltastemaker]

Lessons from a software project

I am adding to my usual weekly rotation of posts to write about a software project that I recently (almost) finished. I want to get my thoughts down now while it is fresh in my mind.

Project Overview

The project is a bug tracker. It is online here. The GitHub repo is here. It is a simple, classic bug tracker. The app allows teams to define several projects and maintain a list of outstanding issues for each project, assigned by user. Project managers and issue owners can mark issues as complete. An individual could use it as a personal to-do list as well. The mail element of the technology stack are as follows.

- React frontend, generated with create-react-app. It is a single page app with all the app’s functionality. It is implemented on GitHub pages.

- Auth0 for authentication.

- A Node backend. It handles api calls to the database. It is deployed on an Amazon Web Services EC2 t2.micro instance running Amazon Linux 2.

- PostgresQL database. It stores user data and data about teams, projects, and issues. It is implemented with an Amazon RDS t3.micro instance.

- SB Admin 2, a Bootstrap theme, for styling.

The purpose of this project is to serve as a portfolio piece indicating that I am capable of full stack development. As such I think it is a good step and the best full stack project I have done so far, but I think I can do better next time. Some of the ways I can do better are detailed below.

Functionality

The functionality of the app is fairly limited relative to what I wanted. It allows creation and deletion of teams, projects, and issues, as well as completion of issues by their owners. I would have liked to add timestamps for issue creation and due date; some sorting functionality; and mechanisms for modifying teams, projects, and issues without deleting and recreating them, but I ran out of time that I was willing to devote to the project.

I’ve tested the project extensively locally, but there are so many variables in the deployment that I don’t know how it is going to survive in the wild. We will see.

There are a couple things that I don’t think are quite right but do not seem to be major problems and that I gave up worrying about. For one thing, the front end seems to be firing off more API calls than I think it should be. I’m pretty sure it has something to do with React’s useEffect being triggered more often than I want, but I did not figure out the precise reason in the course of investigating. There were a couple of icons that are part of the SB Admin 2 theme that for whatever reason I couldn’t get to appear properly, and I gave up on those.

Development

The main challenge in development was locally wiring up the various pieces. The actual programming of the app wasn’t too hard. This was my first project with PostgresQL, and I found the system to be straightforward enough. It was also my first attempt to use Auth0. I found the system to be challenging to use, though it provides much more functionality and ease of use for user than any authentication that I would be able to develop myself.

I have used Node many times before. I like it and find it straightforward. The app in this case is fairly simple and primarily serves the function of interfacing between the front end and the database. In theory I think I could have made database calls directly from the front end, but that doesn’t seem like a good design decision.

I’ve done a few React apps, all with create-react-app. I think that was a mistake. I’ve been using create-react-app as a crutch. The system brings about some serious bloatware. The next time I do something with React, I should find an alternative way to develop and deploy the app. Beyond that, my system of passing parameters through the app is also a bit of a mess. I believe Redux would help streamline that process, and Redux is on my todo list, but it is not something I am familiar with yet. I used a few class components, but I think I should have just kicked that habit and done everything with function components; I see no reason why I couldn’t have.

SB Admin 2 was a pleasure to use. Design isn’t my strongest suit, nor is it primarily what I wanted to illustrate with this project. Going off the project template, I found it easy to make an app that looks presentable. I’m a little dissatisfied with the color scheme, which has something of that “corporate kindergarten” aesthetic to it, but that is acceptable.

Deployment

Actually putting this thing online so others could use it was a major time sink and source of frustration learning experience. I noted in last week’s blog post that this endeavor falls under the job description of a DevOps person, which is not what I am aspiring to be. DevOps is its own form of problem solving, which for the right person could be a lot fun. It also commands a market premium for people who have those skills. I now see very clearly why.

I made a very big mistake that I will not be repeating on future projects. I tried to develop everything locally and get full app functionality locally before taking on deployment. Worse, I tried to treat the entire project as a single package. I was not, for instance, planning on using AWS RDS for the database. I had set up PostgresQL on my local machine and tried to do the same on the EC2 instance. I was also planning on building and running the React frontend out of the same instance.

Almost none of this worked. I installed PostgresQL on the EC2 instance, and it seemed to work for a while, but after a while I couldn’t connect at all. Repeatedly uninstalling and reinstalling the database just caused the DevOps gods to laugh. It was only then that I decided to separate the database to RDS. Even then, somehow the pq module (the Node modules that connects to a PostgresQL database) got borked and I had to uninstall and reinstall it to get that working.

I also tried to run the Node and React apps out of a single concurrently instance. Again, it worked fine locally but failed on deployment. I still don’t fully understand why it didn’t work. Part of the problem was security. For whatever reason React wanted to build to an HTTP instead of an HTTPS deploy, which Chrome flagged as a security hazard. I struggled to figure out how to add an SSL certificate and that only seemed to partially solve the problem. Then I got a CORS (cross-origin resource sharing) error. I tried modifying the Node app to an HTTPS instead of an HTTP server, but that didn’t seem to solve the problem. Even now I’m still not sure it is working right. My browser loads the front end all right but still gives me a security flag.

There was also an issue of the proxy in packages.json not working. Again, it worked fine in dev but just flat out refused to work on the deploy. I gave up trying to figure out why and just started doing full URLs in the fetch requests.

It is obvious in retrospect, and for future projects, that I should have done one or both of two things:

1) Go for a modular design from the beginning. Now all the major pieces are in different places (except I deployed SB Admin 2 with the front end), and that leaves fewer opportunities for conflict. There was no reason not to plan it out that way from the beginning.

2) Figure out the deploy process at the outset. This is the “walking skeleton” methodology. I would add only the minimal “hello world” substance to each piece, then fully ship the project to demonstrate a working architecture. Then I go about building out the pieces. Had I done this at the start, it would have saved me a great deal of stress later on.

Some General Thoughts

The difficulty of a project grows superlinearly in the number of mutually interrelated components. This is probably the same basic mechanism that causes complex infrastructure projects to almost always go over schedule and budget. Modularity needs to be a watchword. I’ve learned my lesson on that.

In general I was pretty happy with AWS. The performance seems to be quite good. It wasn’t too hard to use. The AWS console is huge and can be difficult to navigate. Their system of traffic access rules is a bit confusing but was manageable.

People often complain about JavaScript development. I think the problem is not with JavaScript itself, but with the NPM zoo. Despite the allegedly modular design of NPM packages, the inscrutable chain of dependencies are such that they are inherently non-modular in practice. Reproducibility is a serious problem. Security is a problem. There are vulnerabilities such as shown in the left-pad incident. The system leads to bloatware. It fosters a bad habit of programmers pulling in packages for simple functions they could easily write themselves. The multiplicity of potential clashes between packages of different versions is a factor driving the development of virtual machines and containerization, tools which introduce their own inefficiencies through abstraction. There has got to be a better way. I wonder if there exists anything that be for JavaScript what Acaconda is for Python. There certainly should be.

Security and networking are two things that are important and I just don’t understand very well yet. I think this process would have been easier if I knew what was going on when those bugs came up.

What’s Ahead

For now I am taking a breather to celebrate the completion of this project. While there are some deficiencies in this latest project, I think the place to correct them is on the next project, rather than try to push this one any further. Eventually I want to plan out and execute another full stack project. I don’t know what it will look like yet, but it should be something more interesting than a bug tracker.

In the meantime, I am working through a JavaScript bootcamp and have fallen behind on the projects there. For my next portfolio piece, I want to do something more algorithms-oriented, where the challenge will be in the actual coding rather than in the DevOps.

[blogagility]

Adventures in slicing features.

Teams need to learn the artful skill of slicing features into stories a related to their business context and domain purpose. A typical anti-pattern is for teams to waterfall their iterations, as described in the next two scenarios.

The first iteration we will gather all the requirements, the second iteration we will design, the third and fourth iterations we will build, and the fifth iteration we will test…and so on…

This is an inter-waterfall anti-pattern. It is essentially a pure waterfall approach chopped up into smaller time boxes.

Another derivation of this anti-pattern is to order up another form of phony business agility and/or Scrum.

In this iteration we will pull “requirements gathering” stories first. When those are all finished, we will pull the “design stories”, and then “build” stories…

In this case, it is an intra-waterfall anti-pattern.

The next common anti-pattern related to intra-waterfall is for dev team members to pull stories and work on them independently. This is a siloing ant-pattern, indicating the team is not cross-functional and is simply a collection of silos and individual waterfalls.

These are some common ways to address these anti-patterns.

re-organize the team to become more cross-functional and aligned to a domain purpose.

re-evaluate your feature and story slicing paradigms to ensure that the PO and/or team is not driving waterfall into the structure of the ways of working and work itself by organizing the work/effort into waterfall-like parts.

the Scrum master should teach the team to slice features into stories that may be worked by a slice of the dev team, thus avoiding the intra-team-silo pattern, enhancing collaboration, and enabling flow

how-to move past the anti-patterns

An example of learning a feature slicing pattern follows.

Backlog refinement for a feature development team (not a component team):

PO: I am hungry. I want to eat bread. So that I will have nourishment and no longer be hungry.

Iteration 1:

Dev Team: here you go…this is what we did for two weeks.

PO: I can’t eat that! I think it is only the ingredients! I’m not even really sure what all of that stuff is, I’m not a baker (technical/functional expert), I just want to eat bread!

Dev Team: We only had time to collect the components for bread! All of the components are good and safe! You can eat them! Eventually, we will put them all together into bread.

SM: <face palm>. Team, we can’t just deliver the components. We must deliver the bread in a form that the PO can eat. We need to learn how to slice this feature into a form that the team can deliver each iteration. Delivery of non-working components is not compliant with the value system and principles that we agreed to. We should also begin to document our definition of done, so that the next time we are feeding the PO we remember to not deliver just the components. So, what part of the feature, that is working, can we deliver next iteration?

PO: I’m still hungry!!

Dev Team: OK, we appreciate the feedback and will work on delivering working bread next iteration.

Iteration 2:

Dev Team: here is our Team Demo, Miss PO! Check it out! Risen bread!

PO: Well, that at least looks sort of like bread now. But, it doesn’t look cooked! I don’t want to get sick eating un-cooked bread. I cannot accept this. I was really hoping to make a bologna sandwich with slices from the loaf of bread.

SM: Team, I think we are exploring new acceptance criteria for the feature, and learning more details about what the product really is. Lunch.

Dev Team: Oh, you want a sandwich for lunch! Now we understand! If we knew this, in the beginning, we would have helped the PO slice the stories for the feature in a completely different way. Also, it seems like we may need to re-write the feature to be more descriptive than just “I am hungry, I want to eat bread.” It also appears that we don’t have the domain expertise to complete the product. Our domain purpose is baking bread and so we usually work on features within our domain. We will need to talk to one of the other teams that specialize in bologna about our dependency.

PO: Wait, I also want lettuce, tomato, mayonnaise, and mustard on my sandwich!

Dev Team: <face palm> Wait, what? Why are you just telling us this now? Now we will need to talk to the condiments team too!

PO: Well, until I saw what the bread looked like I didn’t realize that I would need all of those other things.

SM: That is an interesting turn of events team! Let’s do some more refinement and practice elicitation of the requirements with the Product Owner. Then we will write some new stories, maybe even some spikes and I’ll schedule a few hacking sessions.

Iteration 3:

Dev Team: Hello everyone, we are really excited about the Team Demo this iteration! We believe we have a working feature! Check it out, a cooked loaf of bread:

PO: Well, that looks really tasty! But, it is still not a sandwich. I want to eat bread as part of a bologna sandwich, for lunch.

Dev Team: Well, we are a feature development team that has a domain purpose of baking bread, we are bakers, not sandwich makers!

PO: Well, even still why isn’t the loaf of bread sliced? We can’t deliver a loaf to make a sandwich, we need to deliver two slices of bread! The product management team is not going to be happy! It has been 6 weeks and we still are not able to deliver our part of the product.

SM: Team, I think we have an integration and dependency problem. Also, we should rethink our team’s domain purpose because I feel that we have the skills necessary to create and deliver this sandwich from the component parts. We only need to collaborate with the other teams to coordinate our efforts so that the team has time to integrate the various components. Our feature is really one of the critical components that we must build since the company uses an outside supplier to buy bulk bologna.

PO: Yes, that’s true we get most of our bologna from Sutherland Meat Distributors and we buy some pre-sliced bologna from the PMI Meat Market. We only have to slice the bulk bologna it into sandwich meat sized parts. The PMI bologna is ready to use in our sandwiches, but the quality is not near as good compared to the Sutherland bologna. Some of the product managers even joke about how the PMI bologna may contain 98% afterthoughts rather than actual choice cuts in the sausage.

Dev Team: Haha, funny. We did talk to the other teams and they told us they would get the baloney and condiments ready. But when we asked them yesterday afternoon where the baloney and condiments were they said we never came back to them to get the parts. So, we didn’t have time to make a sandwich. Our capacity was tapped out!

SM: Have you guys heard of Lead Time and Takt Time? I think some Lean concepts will really help our Scrum team. Also, we will need to find a better way to coordinate the efforts of the other teams. Lastly, for now, we need to create a way to visualize all of our efforts since making a bologna sandwich for lunch involves so many different teams and suppliers. But, enough of that. I think I recognize a core problem in how we should slice our features. We keep focusing on delivering the entire loaf of bread when we really should be focused on delivering the two slices as early as possible, in edible form. Then the team would have time to integrate the bologna and condiments into the final product. In fact, we have the sandwich bar team that has a unique domain purpose of helping the bakers integrate the components. They also support our ovens, keep the knives sharp, and clean up the machinery and bowls each day. We could bring them in on the collaboration so we can coordinate all of our efforts seamlessly.

What do you guys think about mapping out our process so that we can determine if we have a systemic bottleneck or portion in our work process that would keep the team from baking just the two slices and delivering them in the first iteration?

Dev Team: Yes! If we organize our ingredients into smaller batches and create a mold for bread slices we could easily rework our process to produce the bread slices much more quickly and at a higher quality since we can control the process with the smaller batches more effectively!

PO: Awesome, team! I would be willing to discuss how the slices would look, taste, and what shape they would come in so that we can all learn from the feedback. And since we would be able to deliver slices faster we could work on managing the queue of requests for more lunches and enable an ability to manage our capacity!

Ideas on how to improve

A cross-functional team slicing features into cross-functional stories is a more desirable pattern as it enhances team collaboration and promotes XP practices

Cross-functional teams are a requirement if you want to claim Lean-Agile-Scrum practices are in play. Otherwise, the team is just a silo of specialized resources. Silos create waste through handoffs, holding costs, and dependencies while increasing Lead Time and Takt Time.

Decomposing or slicing features into cross-functional stories is a more desirable pattern than functional stories. Functional stories tend to support the anti-patterns of intra or inter waterfall behavior in iterative development teams.

How teams slice features will have a direct impact on capacity, flow, and quality.

Stories should be sliced to a normalized “smaller” or “sized-appropriately” size of the effort, complexity, known/unknowns to enable Lean concepts of smaller batch sizes and ability to manage queue lengths more effectively.

Components and Integration

Integrated product:

  References:

http://www.extremeprogramming.org/rules/userstories.html

https://www.mountaingoatsoftware.com/blog/differences-between-scrum-and-extreme-programming

https://www.scaledagileframework.com/

https://en.wikipedia.org/wiki/INVEST_(mnemonic)

https://ronjeffries.com/xprog/articles/expcardconversationconfirmation/

Paradigm shift: Slicing Features Adventures in slicing features. Teams need to learn the artful skill of slicing features into stories a related to their business context and domain purpose.

[endenogatai]

Not just another decentralized web whitepaper?

Given all the hype and noise swirling around crypto and decentralized network projects, which runs the full gamut from scams and stupidity, to very clever and inspired ideas, the release of yet another whitepaper does not immediately set off an attention klaxon.

But this whitepaper — which details a new protocol for achieving consensus within a decentralized network — is worth paying more attention to than most.

MaidSafe, the team behind it, are also the literal opposite of fly-by-night crypto opportunists. They’ve been working on decentralized networking since long before the space became the hot, hyped thing it is now.

Their overarching mission is to engineer an entirely decentralized Internet which bakes in privacy, security and freedom of expression by design — the ‘Safe’ in their planned ‘Safe Network’ stands for ‘Secure access for everyone’ — meaning it’s encrypted, autonomous, self-organizing, self-healing. And the new consensus protocol is just another piece towards fulfilling that grand vision.

What’s consensus in decentralized networking terms? “Within decentralized networks you must have a way of the network agreeing on a state — such as can somebody access a file or confirming a coin transaction, for example — and the reason you need this is because you don’t have a central server to confirm all this to you,” explains MaidSafe’s COO Nick Lambert, discussing what the protocol is intended to achieve.

“So you need all these decentralized nodes all reaching agreement somehow on a state within the network. Consensus occurs by each of these nodes on the network voting and letting the network as a whole know what it thinks of a transaction.

“It’s almost like consensus could be considered the heart of the networks. It’s required for almost every event in the network.”

We wrote about MaidSafe’s alternative, server-less Internet in 2014. But they actually began work on the project in stealth all the way back in 2006. So they’re over a decade into the R&D at this point.

The network is p2p because it’s being designed so that data is locally encrypted, broken up into pieces and then stored distributed and replicated across the network, relying on the users’ own compute resources to stand in and take the strain. No servers necessary.

The prototype Safe Network is currently in an alpha testing stage (they opened for alpha in 2016). Several more alpha test stages are planned, with a beta release still a distant, undated prospect at this stage. But rearchitecting the entire Internet was clearly never going to be a day’s work.

MaidSafe also ran a multimillion dollar crowdsale in 2014 — for a proxy token of the coin that will eventually be baked into the network — and did so long before ICOs became a crypto-related bandwagon that all sorts of entities were jumping onto. The SafeCoin cryptocurrency is intended to operate as the inventive mechanism for developers to build apps for the Safe Network and users to contribute compute resource and thus bring MaidSafe’s distributed dream alive.

Their timing on the token sale front, coupled with prudent hodling of some of the Bitcoins they’ve raised, means they’re essentially in a position of not having to worry about raising more funds to build the network, according to Lambert.

A rough, back-of-an-envelope calculation on MaidSafe’s original crowdsale suggests, given they raised $2M in Bitcoin in April 2014 when the price for 1BTC was up to around $500, the Bitcoins they obtained then could be worth between ~$30M-$40M by today’s Bitcoin prices — though that would be assuming they held on to most of them. Bitcoin’s price also peaked far higher last year too.

As well as the token sale they also did an equity raise in 2016, via the fintech investment platform bnktothefuture, pulling in around $1.7M from that — in a mixture of cash and “some Bitcoin”.

“It’s gone both ways,” says Lambert, discussing the team’s luck with Bitcoin. “The crowdsale we were on the losing end of Bitcoin price decreasing. We did a raise from bnktothefuture in autumn of 2016… and fortunately we held on to quite a lot of the Bitcoin. So we rode the Bitcoin price up. So I feel like the universe paid us back a little bit for that. So it feels like we’re level now.”

“Fundraising is exceedingly time consuming right through the organization, and it does take a lot of time away from what you wants to be focusing on, and so to be in a position where you’re not desperate for funding is a really nice one to be in,” he adds. “It allows us to focus on the technology and releasing the network.”

The team’s headcount is now up to around 33, with founding members based at the HQ in Ayr, Scotland, and other engineers working remotely or distributed (including in a new dev office they opened in India at the start of this year), even though MaidSafe is still not taking in any revenue.

This April they also made the decision to switch from a dual licensing approach for their software — previously offering both an open source license and a commercial license (which let people close source their code for a fee) — to going only open source, to encourage more developer engagement and contributions to the project, as Lambert tells it.

“We always see the SafeNetwork a bit like a public utility,” he says. “In terms of once we’ve got this thing up and launched we don’t want to control it or own it because if we do nobody will want to use it — it needs to be seen as everyone contributing. So we felt it’s a much more encouraging sign for developers who want to contribute if they see everything is fully open sourced and cannot be closed source.”

MaidSafe’s story so far is reason enough to take note of their whitepaper.

But the consensus issue the paper addresses is also a key challenge for decentralized networks so any proposed solution is potentially a big deal — if indeed it pans out as promised.

  Protocol for Asynchronous, Reliable, Secure and Efficient Consensus

MaidSafe reckons they’ve come up with a way of achieving consensus on decentralized networks that’s scalable, robust and efficient. Hence the name of the protocol — ‘Parsec’ — being short for: ‘Protocol for Asynchronous, Reliable, Secure and Efficient Consensus’.

They will be open sourcing the protocol under a GPL v3 license — with a rough timeframe of “months” for that release, according to Lambert.

He says they’ve been working on Parsec for the last 18 months to two years — but also drawing on earlier research the team carried out into areas such as conflict-free replicated data types, synchronous and asynchronous consensus, and topics such as threshold signatures and common coin.

More specifically, the research underpinning Parsec is based on the following five papers: 1. Baird L. The Swirlds Hashgraph Consensus Algorithm: Fair, Fast, Byzantine Fault Tolerance, Swirlds Tech Report SWIRLDS-TR-2016-01 (2016); 2. Mostefaoui A., Hamouna M., Raynal M. Signature-Free Asynchronous Byzantine Consensus with t <n/3 and O(n 2 ) Messages, ACM PODC (2014); 3. Micali S. Byzantine Agreement, Made Trivial, (2018); 4. Miller A., Xia Y., Croman K., Shi E., Song D. The Honey Badger of BFT Protocols, CCS (2016); 5. Team Rocket Snowflake to Avalanche: A Novel Metastable Consensus Protocol Family for Cryptocurrencies, (2018).

One tweet responding to the protocol’s unveiling just over a week ago wonders whether it’s too good to be true. Time will tell — but the potential is certainly enticing.

Bitcoin’s use of a drastically energy-inefficient ‘proof of work’ method to achieve consensus and write each transaction to its blockchain very clearly doesn’t scale. It’s slow, cumbersome and wasteful. And how to get blockchain-based networks to support the billions of transactions per second that might be needed to sustain the various envisaged applications remains an essential work in progress — with projects investigating various ideas and approaches to try to overcome the limitation.

MaidSafe’s network is not blockchain-based. It’s engineered to function with asynchronous voting of nodes, rather than synchronous voting, which should avoid the bottleneck problems associated with blockchain. But it’s still decentralized. So it needs a consensus mechanism to enable operations and transactions to be carried out autonomously and robustly. That’s where Parsec is intended to slot in.

The protocol does not use proof of work. And is able, so the whitepaper claims, to achieve consensus even if a third of the network is comprised of malicious nodes — i.e. nodes which are attempting to disrupt network operations or otherwise attack the network.

Another claimed advantage is that decisions made via the protocol are both mathematically guaranteed and irreversible.

“What Parsec does is it can reach consensus even with malicious nodes. And up to a third of the nodes being malicious is what the maths proofs suggest,” says Lambert. “This ability to provide mathematical guarantees that all parts of the network will come to the same agreement at a point in time, even with some fault in the network or bad actors — that’s what Byzantine Fault Tolerance is.”

In theory a blockchain using proof of work could be hacked if any one entity controlled 51% of the nodes on the network (although in reality it’s likely that such a large amount of energy would be required it’s pretty much impractical).

So on the surface MaidSafe’s decentralized network — which ‘only’ needs 33% of its nodes to be compromised for its consensus decisions to be attacked — sounds rather less robust. But Lambert says it’s more nuanced than the numbers suggest. And in fact the malicious third would also need to be nodes that have the authority to vote. “So it is a third but it’s a third of well reputed nodes,” as he puts it.

So there’s an element of proof of stake involved too, bound up with additional planned characteristics of the Safe Network — related to dynamic membership and sharding (Lambert says MaidSafe has additional whitepapers on both those elements coming soon).

“Those two papers, particularly the one around dynamic membership, will explain why having a third of malicious nodes is actually harder than just having 33% of malicious nodes. Because the nodes that can vote have to have a reputation as well. So it’s not just purely you can flood the Safe Network with lots and lots of malicious nodes and override it only using a third of the nodes. What we’re saying is the nodes that can vote and actually have a say must have a good reputation in the network,” he says.

“The other thing is proof of stake… Everyone is desperate to move away from proof of work because of its environmental impact. So proof of stake — I liken it to the Scottish landowners, where people with a lot of power have more say. In the cryptocurrency field, proof of stake might be if you have, let’s say, 10 coins and I have one coin your vote might be worth 10x as much authority as what my one coin would be. So any of these mechanisms that they come up with it has that weighting to it… So the people with the most vested interests in the network are also given the more votes.”

Sharding refers to closed groups that allow for consensus votes to be reached by a subset of nodes on a decentralized network. By splitting the network into small sections for consensus voting purposes the idea is you avoid the inefficiencies of having to poll all the nodes on the network — yet can still retain robustness, at least so long as subgroups are carefully structured and secured.

“If you do that correctly you can make it more secure and you can make things much more efficient and faster,” says Lambert. “Because rather than polling, let’s say 6,000 nodes, you might be polling eight nodes. So you can get that information back quickly.

“Obviously you need to be careful about how you do that because with much less nodes you can potentially game the network so you need to be careful how you secure those smaller closed groups or shards. So that will be quite a big thing because pretty much every crypto project is looking at sharding to make, certainly, blockchains more efficient. And so the fact that we’ll have something coming out in that, after we have the dynamic membership stuff coming out, is going to be quite exciting to see the reaction to that as well.”

Voting authority on the Safe Network might be based on a node’s longevity, quality and historical activity — so a sort of ‘reputation’ score (or ledger) that can yield voting rights over time.

“If you’re like that then you will have a vote in these closed groups. And so a third of those votes — and that then becomes quite hard to game because somebody who’s then trying to be malicious would need to have their nodes act as good corporate citizens for a time period. And then all of a sudden become malicious, by which time they’ve probably got a vested stake in the network. So it wouldn’t be possible for someone to just come and flood the network with new nodes and then be malicious because it would not impact upon the network,” Lambert suggests.

The computing power that would be required to attack the Safe Network once it’s public and at scale would also be “really, really significant”, he adds. “Once it gets to scale it would be really hard to co-ordinate anything against it because you’re always having to be several hundred percent bigger than the network and then have a co-ordinated attack on it itself. And all of that work might get you to impact the decision within one closed group. So it’s not even network wide… And that decision could be on who accesses one piece of encrypted shard of data for example… Even the thing you might be able to steal is only an encrypted shard of something — it’s not even the whole thing.”

Other distributed ledger projects are similarly working on Asynchronous Byzantine Fault Tolerant (AFBT) consensus models, including those using directed acrylic graphs (DAGs) — another nascent decentralization technology that’s been suggested as an alternative to blockchain.

And indeed AFBT techniques predate Bitcoin, though MaidSafe says these kind of models have only more recently become viable thanks to research and the relative maturing of decentralized computing and data types, itself as a consequence of increased interest and investment in the space.

However in the case of Hashgraph — the DAG project which has probably attracted the most attention so far — it’s closed source, not open. So that’s one major difference with MaidSafe’s approach. 

Another difference that Lambert points to is that Parsec has been built to work in a dynamic, permissionless network environment (essential for the intended use-case, as the Safe Network is intended as a public network). Whereas he claims Hashgraph has only demonstrated its algorithms working on a permissioned (and therefore private) network “where all the nodes are known”.

He also suggests there’s a question mark over whether Hashgraph’s algorithm can achieve consensus when there are malicious nodes operating on the network. Which — if true — would limit what it can be used for.

“The Hashgraph algorithm is only proven to reach agreement if there’s no adversaries within the network,” Lambert claims. “So if everything’s running well then happy days, but if there’s any maliciousness or any failure within that network then — certainly on the basis of what’s been published — it would suggest that that algorithm was not going to hold up to that.”

“I think being able to do all of these things asynchronously with all of the mathematical guarantees is very difficult,” he continues, returning to the core consensus challenge. “So at the moment we see that we have come out with something that is unique, that covers a lot of these bases, and is a very good use for our use-case. And I think will be useful for others — so I think we like to think that we’ve made a paradigm shift or a vast improvement over the state of the art.”

  Paradigm shift vs marginal innovation

Despite the team’s conviction that, with Parsec, they’ve come up with something very notable, early feedback includes some very vocal Twitter doubters.

For example there’s a lengthy back-and-forth between several MaidSafe engineers and Ethereum researcher Vlad Zamfir — who dubs the Parsec protocol “overhyped” and a “marginal innovation if that”… so, er, ouch.

Well, I don't think asynchronous consensus on an order of events can be described as "never been done before" or "a paradigm shift". And it's not asynchronous, but "very asynchronous" or "randomly synchronous" [sic], and this synchrony assumption isn't very clearly documented

— Vlad ''not giving away ETH'' Zamfir (@VladZamfir) May 31, 2018

Lambert is, if not entirely sanguine, then solidly phlegmatic in the face of a bit of initial Twitter blowback — saying he reckons it will take more time for more detailed responses to come, i.e. allowing for people to properly digest the whitepaper.

“In the world of async BFT algorithms, any advance is huge,” MaidSafe CEO David Irvine also tells us when we ask for a response to Zamfir’s critique. “How huge is subjective, but any advance has to be great for the world. We hope others will advance Parsec like we have built on others (as we clearly state and thank them for their work).  So even if it was a marginal development (which it certainly is not) then I would take that.”

“All in all, though, nothing was said that took away from the fact Parsec moves the industry forward,” he adds. “I felt the comments were a bit juvenile at times and a bit defensive (probably due to us not agreeing with POS in our Medium post) but in terms of the only part commented on (the coin flip) we as a team feel that part could be much more concrete in terms of defining exactly how small such random (finite) delays could be. We know they do not stop the network and a delaying node would be killed, but for completeness, it would be nice to be that detailed.”

A developer source of our own in the crypto/blockchain space — who’s not connected to the MaidSafe or Ethereum projects — also points out that Parsec “getting objective review will take some time given that so many potential reviewers have vested interest in their own project/coin”.

It’s certainly fair to say the space excels at public spats and disagreements. Researchers pouring effort into one project can be less than kind to rivals’ efforts. (And, well, given all the crypto Lambos at stake it’s not hard to see why there can be no love lost — and, ironically, zero trust — between competing champions of trustless tech.)

Another fundamental truth of these projects is they’re all busily experimenting right now, with lots of ideas in play to try and fix core issues like scalability, efficiency and robustness — often having different ideas over implementation even if rival projects are circling and/or converging on similar approaches and techniques.

“Certainly other projects are looking at sharding,” says Lambert. “So I know that Ethereum are looking at sharding. And I think Bitcoin are looking at that as well, but I think everyone probably has quite different ideas about how to implement it. And of course we’re not using a blockchain which makes that another different use-case where Ethereum and Bitcoin obviously are. But everyone has — as with anything — these different approaches and different ideas.”

“Every network will have its own different ways of doing [consensus],” he adds when asked whether he believes Parsec could be adopted by other projects wrestling with the consensus challenge. “So it’s not like some could lift [Parsec] out and just put it in. Ethereum is blockchain-based — I think they’re looking at something around proof of stake, but maybe they could take some ideas or concepts from the work that we’re open sourcing for their specific case.

“If you get other blockchain-less networks like IOTA, Byteball, I think POA is another one as well. These other projects it might be easier for them to implement something like Parsec with them because they’re not using blockchain. So maybe less of that adaption required.”

Whether other projects will deem Parsec worthy of their attention remains to be seen at this point with so much still to play for. Some may prefer to expend effort trying to rubbish a rival approach, whose open source tech could, if it stands up to scrutiny and operational performance, reduce the commercial value of proprietary and patented mechanisms also intended to grease the wheels of decentralized networks — for a fee.

And of course MaidSafe’s developed-in-stealth consensus protocol may also turn out to be a relatively minor development. But finding a non-vested expert to give an impartial assessment of complex network routing algorithms conjoined to such a self-interested and, frankly, anarchical industry is another characteristic challenge of the space.

Irvine’s view is that DAG based projects which are using a centralized component will have to move on or adopt what he dubs “state of art” asynchronous consensus algorithms — as MaidSafe believes Parsec is — aka, algorithms which are “more widely accepted and proven”.

“So these projects should contribute to the research, but more importantly, they will have to adopt better algorithms than they use,” he suggests. “So they can play an important part, upgrades! How to upgrade a running DAG based network? How to had fork a graph? etc. We know how to hard fork blockchains, but upgrading DAG based networks may not be so simple when they are used as ledgers.

“Projects like Hashgraph, Algorand etc will probably use an ABFT algorithm like this as their whole network with a little work for a currency; IOTA, NANO, Bytball etc should. That is entirely possible with advances like Parsec. However adding dynamic membership, sharding, a data layer then a currency is a much larger proposition, which is why Parsec has been in stealth mode while it is being developed.

“We hope that by being open about the algorithm, and making the code open source when complete, we will help all the other projects working on similar problems.”

Of course MaidSafe’s team might be misguided in terms of the breakthrough they think they’ve made with Parsec. But it’s pretty hard to stand up the idea they’re being intentionally misleading.

Because, well, what would be the point of that? While the exact depth of MaidSafe’s funding reserves isn’t clear, Lambert doesn’t sound like a startup guy with money worries. And the team’s staying power cannot be in doubt — over a decade into the R&D needed to underpin their alt network.

It’s true that being around for so long does have some downsides, though. Especially, perhaps, given how hyped the decentralized space has now become. “Because we’ve been working on it for so long, and it’s been such a big project, you can see some negative feedback about that,” as Lambert admits.

And with such intense attention now on the space, injecting energy which in turn accelerates ideas and activity, there’s perhaps extra pressure on a veteran player like MaidSafe to be seen making a meaningful contribution — ergo, it might be tempting for the team to believe the consensus protocol they’ve engineered really is a big deal.

To stand up and be counted amid all the noise, as it were. And to draw attention to their own project — which needs lots of external developers to buy into the vision if it’s to succeed, yet, here in 2018, it’s just one decentralization project among so many. 

  The Safe Network roadmap

Consensus aside, MaidSafe’s biggest challenge is still turning the sizable amount of funding and resources the team’s ideas have attracted to date into a bona fide alternative network that anyone really can use. And there’s a very long road to travel still on that front, clearly.

The Safe Network is in alpha 2 testing incarnation (which has been up and running since September last year) — consisting of around a hundred nodes that MaidSafe is maintaining itself.

The core decentralization proposition of anyone being able to supply storage resource to the network via lending their own spare capacity is not yet live — and won’t come fully until alpha 4.

“People are starting to create different apps against that network. So we’ve seen Jams — a decentralized music player… There are a couple of storage style apps… There is encrypted email running as well, and also that is running on Android,” says Lambert. “And we have a forked version of the Beaker browser — that’s the browser that we use right now. So if you can create websites on the Safe Network, which has its own protocol, and if you want to go and view those sites you need a Safe browser to do that, so we’ve also been working on our own browser from scratch that we’ll be releasing later this year… So there’s a number of apps that are running against that alpha 2 network.

“What alpha 3 will bring is it will run in parallel with alpha 2 but it will effectively be a decentralized routing network. What that means is it will be one for more technical people to run, and it will enable data to be passed around a network where anyone can contribute their resources to it but it will not facilitate data storage. So it’ll be a command line app, which is probably why it’ll suit technical people more because there’ll be no user interface for it, and they will contribute their resources to enable messages to be passed around the network. So secure messaging would be a use-case for that.

“And then alpha 4 is effectively bringing together alpha 2 and alpha 3. So it adds a storage layer on top of the alpha 3 network — and at that point it gives you the fully decentralized network where users are contributing their resources from home and they will be able to store data, send messages and things of that nature. Potentially during alpha 4, or a later alpha, we’ll introduce test SafeCoin. Which is the final piece of the initial puzzle to provide incentives for users to provide resources and for developers to make apps. So that’s probably what the immediate roadmap looks like.”

On the timeline front Lambert won’t be coaxed into fixing any deadlines to all these planned alphas. They’ve long ago learnt not to try and predict the pace of progress, he says with a laugh. Though he does not question that progress is being made.

“These big infrastructure projects are typically only government funded because the payback is too slow for venture capitalists,” he adds. “So in the past you had things like Arpanet, the precursor to the Internet — that was obviously a US government funded project — and so we’ve taken on a project which has, not grown arms and legs, but certainly there’s more to it than what was initially thought about.

“So we are almost privately funding this infrastructure. Which is quite a big scope, and I will say why it’s taking a bit of time. But we definitely do seem to be making lots of progress.”

from RSSMix.com Mix ID 8204425 https://ift.tt/2JbAzHa via IFTTT

[abckidstvyara]

Not just another decentralized web whitepaper?

Not just another decentralized web whitepaper?

Given all the hype and noise swirling around crypto and decentralized network projects, which runs the full gamut from scams and stupidity, to very clever and inspired ideas, the release of yet another whitepaper does not immediately set off an attention klaxon.

But this whitepaper — which details a new protocol for achieving consensus within a decentralized network — is worth paying more attention to than most.

MaidSafe, the team behind it, are also the literal opposite of fly-by-night crypto opportunists. They’ve been working on decentralized networking since long before the space became the hot, hyped thing it is now.

Their overarching mission is to engineer an entirely decentralized Internet which bakes in privacy, security and freedom of expression by design — the ‘Safe’ in their planned ‘Safe Network’ stands for ‘Secure access for everyone’ — meaning it’s encrypted, autonomous, self-organizing, self-healing. And the new consensus protocol is just another piece towards fulfilling that grand vision.

What’s consensus in decentralized networking terms? “Within decentralized networks you must have a way of the network agreeing on a state — such as can somebody access a file or confirming a coin transaction, for example — and the reason you need this is because you don’t have a central server to confirm all this to you,” explains MaidSafe’s COO Nick Lambert, discussing what the protocol is intended to achieve.

“So you need all these decentralized nodes all reaching agreement somehow on a state within the network. Consensus occurs by each of these nodes on the network voting and letting the network as a whole know what it thinks of a transaction.

“It’s almost like consensus could be considered the heart of the networks. It’s required for almost every event in the network.”

We wrote about MaidSafe’s alternative, server-less Internet in 2014. But they actually began work on the project in stealth all the way back in 2006. So they’re over a decade into the R&D at this point.

The network is p2p because it’s being designed so that data is locally encrypted, broken up into pieces and then stored distributed and replicated across the network, relying on the users’ own compute resources to stand in and take the strain. No servers necessary.

The prototype Safe Network is currently in an alpha testing stage (they opened for alpha in 2016). Several more alpha test stages are planned, with a beta release still a distant, undated prospect at this stage. But rearchitecting the entire Internet was clearly never going to be a day’s work.

MaidSafe also ran a multimillion dollar crowdsale in 2014 — for a proxy token of the coin that will eventually be baked into the network — and did so long before ICOs became a crypto-related bandwagon that all sorts of entities were jumping onto. The SafeCoin cryptocurrency is intended to operate as the inventive mechanism for developers to build apps for the Safe Network and users to contribute compute resource and thus bring MaidSafe’s distributed dream alive.

Their timing on the token sale front, coupled with prudent hodling of some of the Bitcoins they’ve raised, means they’re essentially in a position of not having to worry about raising more funds to build the network, according to Lambert.

A rough, back-of-an-envelope calculation on MaidSafe’s original crowdsale suggests, given they raised $2M in Bitcoin in April 2014 when the price for 1BTC was up to around $500, the Bitcoins they obtained then could be worth between ~$30M-$40M by today’s Bitcoin prices — though that would be assuming they held on to most of them. Bitcoin’s price also peaked far higher last year too.

As well as the token sale they also did an equity raise in 2016, via the fintech investment platform bnktothefuture, pulling in around $1.7M from that — in a mixture of cash and “some Bitcoin”.

“It’s gone both ways,” says Lambert, discussing the team’s luck with Bitcoin. “The crowdsale we were on the losing end of Bitcoin price decreasing. We did a raise from bnktothefuture in autumn of 2016… and fortunately we held on to quite a lot of the Bitcoin. So we rode the Bitcoin price up. So I feel like the universe paid us back a little bit for that. So it feels like we’re level now.”

“Fundraising is exceedingly time consuming right through the organization, and it does take a lot of time away from what you wants to be focusing on, and so to be in a position where you’re not desperate for funding is a really nice one to be in,” he adds. “It allows us to focus on the technology and releasing the network.”

The team’s headcount is now up to around 33, with founding members based at the HQ in Ayr, Scotland, and other engineers working remotely or distributed (including in a new dev office they opened in India at the start of this year), even though MaidSafe is still not taking in any revenue.

This April they also made the decision to switch from a dual licensing approach for their software — previously offering both an open source license and a commercial license (which let people close source their code for a fee) — to going only open source, to encourage more developer engagement and contributions to the project, as Lambert tells it.

“We always see the SafeNetwork a bit like a public utility,” he says. “In terms of once we’ve got this thing up and launched we don’t want to control it or own it because if we do nobody will want to use it — it needs to be seen as everyone contributing. So we felt it’s a much more encouraging sign for developers who want to contribute if they see everything is fully open sourced and cannot be closed source.”

MaidSafe’s story so far is reason enough to take note of their whitepaper.

But the consensus issue the paper addresses is also a key challenge for decentralized networks so any proposed solution is potentially a big deal — if indeed it pans out as promised.

  Protocol for Asynchronous, Reliable, Secure and Efficient Consensus

MaidSafe reckons they’ve come up with a way of achieving consensus on decentralized networks that’s scalable, robust and efficient. Hence the name of the protocol — ‘Parsec’ — being short for: ‘Protocol for Asynchronous, Reliable, Secure and Efficient Consensus’.

They will be open sourcing the protocol under a GPL v3 license — with a rough timeframe of “months” for that release, according to Lambert.

He says they’ve been working on Parsec for the last 18 months to two years — but also drawing on earlier research the team carried out into areas such as conflict-free replicated data types, synchronous and asynchronous consensus, and topics such as threshold signatures and common coin.

More specifically, the research underpinning Parsec is based on the following five papers: 1. Baird L. The Swirlds Hashgraph Consensus Algorithm: Fair, Fast, Byzantine Fault Tolerance, Swirlds Tech Report SWIRLDS-TR-2016-01 (2016); 2. Mostefaoui A., Hamouna M., Raynal M. Signature-Free Asynchronous Byzantine Consensus with t <n/3 and O(n 2 ) Messages, ACM PODC (2014); 3. Micali S. Byzantine Agreement, Made Trivial, (2018); 4. Miller A., Xia Y., Croman K., Shi E., Song D. The Honey Badger of BFT Protocols, CCS (2016); 5. Team Rocket Snowflake to Avalanche: A Novel Metastable Consensus Protocol Family for Cryptocurrencies, (2018).

One tweet responding to the protocol’s unveiling just over a week ago wonders whether it’s too good to be true. Time will tell — but the potential is certainly enticing.

Bitcoin’s use of a drastically energy-inefficient ‘proof of work’ method to achieve consensus and write each transaction to its blockchain very clearly doesn’t scale. It’s slow, cumbersome and wasteful. And how to get blockchain-based networks to support the billions of transactions per second that might be needed to sustain the various envisaged applications remains an essential work in progress — with projects investigating various ideas and approaches to try to overcome the limitation.

MaidSafe’s network is not blockchain-based. It’s engineered to function with asynchronous voting of nodes, rather than synchronous voting, which should avoid the bottleneck problems associated with blockchain. But it’s still decentralized. So it needs a consensus mechanism to enable operations and transactions to be carried out autonomously and robustly. That’s where Parsec is intended to slot in.

The protocol does not use proof of work. And is able, so the whitepaper claims, to achieve consensus even if a third of the network is comprised of malicious nodes — i.e. nodes which are attempting to disrupt network operations or otherwise attack the network.

Another claimed advantage is that decisions made via the protocol are both mathematically guaranteed and irreversible.

“What Parsec does is it can reach consensus even with malicious nodes. And up to a third of the nodes being malicious is what the maths proofs suggest,” says Lambert. “This ability to provide mathematical guarantees that all parts of the network will come to the same agreement at a point in time, even with some fault in the network or bad actors — that’s what Byzantine Fault Tolerance is.”

In theory a blockchain using proof of work could be hacked if any one entity controlled 51% of the nodes on the network (although in reality it’s likely that such a large amount of energy would be required it’s pretty much impractical).

So on the surface MaidSafe’s decentralized network — which ‘only’ needs 33% of its nodes to be compromised for its consensus decisions to be attacked — sounds rather less robust. But Lambert says it’s more nuanced than the numbers suggest. And in fact the malicious third would also need to be nodes that have the authority to vote. “So it is a third but it’s a third of well reputed nodes,” as he puts it.

So there’s an element of proof of stake involved too, bound up with additional planned characteristics of the Safe Network — related to dynamic membership and sharding (Lambert says MaidSafe has additional whitepapers on both those elements coming soon).

“Those two papers, particularly the one around dynamic membership, will explain why having a third of malicious nodes is actually harder than just having 33% of malicious nodes. Because the nodes that can vote have to have a reputation as well. So it’s not just purely you can flood the Safe Network with lots and lots of malicious nodes and override it only using a third of the nodes. What we’re saying is the nodes that can vote and actually have a say must have a good reputation in the network,” he says.

“The other thing is proof of stake… Everyone is desperate to move away from proof of work because of its environmental impact. So proof of stake — I liken it to the Scottish landowners, where people with a lot of power have more say. In the cryptocurrency field, proof of stake might be if you have, let’s say, 10 coins and I have one coin your vote might be worth 10x as much authority as what my one coin would be. So any of these mechanisms that they come up with it has that weighting to it… So the people with the most vested interests in the network are also given the more votes.”

Sharding refers to closed groups that allow for consensus votes to be reached by a subset of nodes on a decentralized network. By splitting the network into small sections for consensus voting purposes the idea is you avoid the inefficiencies of having to poll all the nodes on the network — yet can still retain robustness, at least so long as subgroups are carefully structured and secured.

“If you do that correctly you can make it more secure and you can make things much more efficient and faster,” says Lambert. “Because rather than polling, let’s say 6,000 nodes, you might be polling eight nodes. So you can get that information back quickly.

“Obviously you need to be careful about how you do that because with much less nodes you can potentially game the network so you need to be careful how you secure those smaller closed groups or shards. So that will be quite a big thing because pretty much every crypto project is looking at sharding to make, certainly, blockchains more efficient. And so the fact that we’ll have something coming out in that, after we have the dynamic membership stuff coming out, is going to be quite exciting to see the reaction to that as well.”

Voting authority on the Safe Network might be based on a node’s longevity, quality and historical activity — so a sort of ‘reputation’ score (or ledger) that can yield voting rights over time.

“If you’re like that then you will have a vote in these closed groups. And so a third of those votes — and that then becomes quite hard to game because somebody who’s then trying to be malicious would need to have their nodes act as good corporate citizens for a time period. And then all of a sudden become malicious, by which time they’ve probably got a vested stake in the network. So it wouldn’t be possible for someone to just come and flood the network with new nodes and then be malicious because it would not impact upon the network,” Lambert suggests.

The computing power that would be required to attack the Safe Network once it’s public and at scale would also be “really, really significant”, he adds. “Once it gets to scale it would be really hard to co-ordinate anything against it because you’re always having to be several hundred percent bigger than the network and then have a co-ordinated attack on it itself. And all of that work might get you to impact the decision within one closed group. So it’s not even network wide… And that decision could be on who accesses one piece of encrypted shard of data for example… Even the thing you might be able to steal is only an encrypted shard of something — it’s not even the whole thing.”

Other distributed ledger projects are similarly working on Asynchronous Byzantine Fault Tolerant (AFBT) consensus models, including those using directed acrylic graphs (DAGs) — another nascent decentralization technology that’s been suggested as an alternative to blockchain.

And indeed AFBT techniques predate Bitcoin, though MaidSafe says these kind of models have only more recently become viable thanks to research and the relative maturing of decentralized computing and data types, itself as a consequence of increased interest and investment in the space.

However in the case of Hashgraph — the DAG project which has probably attracted the most attention so far — it’s closed source, not open. So that’s one major difference with MaidSafe’s approach. 

Another difference that Lambert points to is that Parsec has been built to work in a dynamic, permissionless network environment (essential for the intended use-case, as the Safe Network is intended as a public network). Whereas he claims Hashgraph has only demonstrated its algorithms working on a permissioned (and therefore private) network “where all the nodes are known”.

He also suggests there’s a question mark over whether Hashgraph’s algorithm can achieve consensus when there are malicious nodes operating on the network. Which — if true — would limit what it can be used for.

“The Hashgraph algorithm is only proven to reach agreement if there’s no adversaries within the network,” Lambert claims. “So if everything’s running well then happy days, but if there’s any maliciousness or any failure within that network then — certainly on the basis of what’s been published — it would suggest that that algorithm was not going to hold up to that.”

“I think being able to do all of these things asynchronously with all of the mathematical guarantees is very difficult,” he continues, returning to the core consensus challenge. “So at the moment we see that we have come out with something that is unique, that covers a lot of these bases, and is a very good use for our use-case. And I think will be useful for others — so I think we like to think that we’ve made a paradigm shift or a vast improvement over the state of the art.”

  Paradigm shift vs marginal innovation

Despite the team’s conviction that, with Parsec, they’ve come up with something very notable, early feedback includes some very vocal Twitter doubters.

For example there’s a lengthy back-and-forth between several MaidSafe engineers and Ethereum researcher Vlad Zamfir — who dubs the Parsec protocol “overhyped” and a “marginal innovation if that”… so, er, ouch.

Well, I don't think asynchronous consensus on an order of events can be described as "never been done before" or "a paradigm shift". And it's not asynchronous, but "very asynchronous" or "randomly synchronous" [sic], and this synchrony assumption isn't very clearly documented

— Vlad ''not giving away ETH'' Zamfir (@VladZamfir) May 31, 2018

Lambert is, if not entirely sanguine, then solidly phlegmatic in the face of a bit of initial Twitter blowback — saying he reckons it will take more time for more detailed responses to come, i.e. allowing for people to properly digest the whitepaper.

“In the world of async BFT algorithms, any advance is huge,” MaidSafe CEO David Irvine also tells us when we ask for a response to Zamfir’s critique. “How huge is subjective, but any advance has to be great for the world. We hope others will advance Parsec like we have built on others (as we clearly state and thank them for their work).  So even if it was a marginal development (which it certainly is not) then I would take that.”

“All in all, though, nothing was said that took away from the fact Parsec moves the industry forward,” he adds. “I felt the comments were a bit juvenile at times and a bit defensive (probably due to us not agreeing with POS in our Medium post) but in terms of the only part commented on (the coin flip) we as a team feel that part could be much more concrete in terms of defining exactly how small such random (finite) delays could be. We know they do not stop the network and a delaying node would be killed, but for completeness, it would be nice to be that detailed.”

A developer source of our own in the crypto/blockchain space — who’s not connected to the MaidSafe or Ethereum projects — also points out that Parsec “getting objective review will take some time given that so many potential reviewers have vested interest in their own project/coin”.

It’s certainly fair to say the space excels at public spats and disagreements. Researchers pouring effort into one project can be less than kind to rivals’ efforts. (And, well, given all the crypto Lambos at stake it’s not hard to see why there can be no love lost — and, ironically, zero trust — between competing champions of trustless tech.)

Another fundamental truth of these projects is they’re all busily experimenting right now, with lots of ideas in play to try and fix core issues like scalability, efficiency and robustness — often having different ideas over implementation even if rival projects are circling and/or converging on similar approaches and techniques.

“Certainly other projects are looking at sharding,” says Lambert. “So I know that Ethereum are looking at sharding. And I think Bitcoin are looking at that as well, but I think everyone probably has quite different ideas about how to implement it. And of course we’re not using a blockchain which makes that another different use-case where Ethereum and Bitcoin obviously are. But everyone has — as with anything — these different approaches and different ideas.”

“Every network will have its own different ways of doing [consensus],” he adds when asked whether he believes Parsec could be adopted by other projects wrestling with the consensus challenge. “So it’s not like some could lift [Parsec] out and just put it in. Ethereum is blockchain-based — I think they’re looking at something around proof of stake, but maybe they could take some ideas or concepts from the work that we’re open sourcing for their specific case.

“If you get other blockchain-less networks like IOTA, Byteball, I think POA is another one as well. These other projects it might be easier for them to implement something like Parsec with them because they’re not using blockchain. So maybe less of that adaption required.”

Whether other projects will deem Parsec worthy of their attention remains to be seen at this point with so much still to play for. Some may prefer to expend effort trying to rubbish a rival approach, whose open source tech could, if it stands up to scrutiny and operational performance, reduce the commercial value of proprietary and patented mechanisms also intended to grease the wheels of decentralized networks — for a fee.

And of course MaidSafe’s developed-in-stealth consensus protocol may also turn out to be a relatively minor development. But finding a non-vested expert to give an impartial assessment of complex network routing algorithms conjoined to such a self-interested and, frankly, anarchical industry is another characteristic challenge of the space.

Irvine’s view is that DAG based projects which are using a centralized component will have to move on or adopt what he dubs “state of art” asynchronous consensus algorithms — as MaidSafe believes Parsec is — aka, algorithms which are “more widely accepted and proven”.

“So these projects should contribute to the research, but more importantly, they will have to adopt better algorithms than they use,” he suggests. “So they can play an important part, upgrades! How to upgrade a running DAG based network? How to had fork a graph? etc. We know how to hard fork blockchains, but upgrading DAG based networks may not be so simple when they are used as ledgers.

“Projects like Hashgraph, Algorand etc will probably use an ABFT algorithm like this as their whole network with a little work for a currency; IOTA, NANO, Bytball etc should. That is entirely possible with advances like Parsec. However adding dynamic membership, sharding, a data layer then a currency is a much larger proposition, which is why Parsec has been in stealth mode while it is being developed.

“We hope that by being open about the algorithm, and making the code open source when complete, we will help all the other projects working on similar problems.”

Of course MaidSafe’s team might be misguided in terms of the breakthrough they think they’ve made with Parsec. But it’s pretty hard to stand up the idea they’re being intentionally misleading.

Because, well, what would be the point of that? While the exact depth of MaidSafe’s funding reserves isn’t clear, Lambert doesn’t sound like a startup guy with money worries. And the team’s staying power cannot be in doubt — over a decade into the R&D needed to underpin their alt network.

It’s true that being around for so long does have some downsides, though. Especially, perhaps, given how hyped the decentralized space has now become. “Because we’ve been working on it for so long, and it’s been such a big project, you can see some negative feedback about that,” as Lambert admits.

And with such intense attention now on the space, injecting energy which in turn accelerates ideas and activity, there’s perhaps extra pressure on a veteran player like MaidSafe to be seen making a meaningful contribution — ergo, it might be tempting for the team to believe the consensus protocol they’ve engineered really is a big deal.

To stand up and be counted amid all the noise, as it were. And to draw attention to their own project — which needs lots of external developers to buy into the vision if it’s to succeed, yet, here in 2018, it’s just one decentralization project among so many. 

  The Safe Network roadmap

Consensus aside, MaidSafe’s biggest challenge is still turning the sizable amount of funding and resources the team’s ideas have attracted to date into a bona fide alternative network that anyone really can use. And there’s a very long road to travel still on that front, clearly.

The Safe Network is in alpha 2 testing incarnation (which has been up and running since September last year) — consisting of around a hundred nodes that MaidSafe is maintaining itself.

The core decentralization proposition of anyone being able to supply storage resource to the network via lending their own spare capacity is not yet live — and won’t come fully until alpha 4.

“People are starting to create different apps against that network. So we’ve seen Jams — a decentralized music player… There are a couple of storage style apps… There is encrypted email running as well, and also that is running on Android,” says Lambert. “And we have a forked version of the Beaker browser — that’s the browser that we use right now. So if you can create websites on the Safe Network, which has its own protocol, and if you want to go and view those sites you need a Safe browser to do that, so we’ve also been working on our own browser from scratch that we’ll be releasing later this year… So there’s a number of apps that are running against that alpha 2 network.

“What alpha 3 will bring is it will run in parallel with alpha 2 but it will effectively be a decentralized routing network. What that means is it will be one for more technical people to run, and it will enable data to be passed around a network where anyone can contribute their resources to it but it will not facilitate data storage. So it’ll be a command line app, which is probably why it’ll suit technical people more because there’ll be no user interface for it, and they will contribute their resources to enable messages to be passed around the network. So secure messaging would be a use-case for that.

“And then alpha 4 is effectively bringing together alpha 2 and alpha 3. So it adds a storage layer on top of the alpha 3 network — and at that point it gives you the fully decentralized network where users are contributing their resources from home and they will be able to store data, send messages and things of that nature. Potentially during alpha 4, or a later alpha, we’ll introduce test SafeCoin. Which is the final piece of the initial puzzle to provide incentives for users to provide resources and for developers to make apps. So that’s probably what the immediate roadmap looks like.”

On the timeline front Lambert won’t be coaxed into fixing any deadlines to all these planned alphas. They’ve long ago learnt not to try and predict the pace of progress, he says with a laugh. Though he does not question that progress is being made.

“These big infrastructure projects are typically only government funded because the payback is too slow for venture capitalists,” he adds. “So in the past you had things like Arpanet, the precursor to the Internet — that was obviously a US government funded project — and so we’ve taken on a project which has, not grown arms and legs, but certainly there’s more to it than what was initially thought about.

“So we are almost privately funding this infrastructure. Which is quite a big scope, and I will say why it’s taking a bit of time. But we definitely do seem to be making lots of progress.”

[theinvinciblenoob]

Given all the hype and noise swirling around crypto and decentralized network projects, which runs the full gamut from scams and stupidity, to very clever and inspired ideas, the release of yet another whitepaper does not immediately set off an attention klaxon.

But this whitepaper — which details a new protocol for achieving consensus within a decentralized network — is worth paying more attention to than most.

MaidSafe, the team behind it, are also the literal opposite of fly-by-night crypto opportunists. They’ve been working on decentralized networking since long before the space became the hot, hyped thing it is now.

Their overarching mission is to engineer an entirely decentralized Internet which bakes in privacy, security and freedom of expression by design — the ‘Safe’ in their planned ‘Safe Network’ stands for ‘Secure access for everyone’ — meaning it’s encrypted, autonomous, self-organizing, self-healing. And the new consensus protocol is just another piece towards fulfilling that grand vision.

What’s consensus in decentralized networking terms? “Within decentralized networks you must have a way of the network agreeing on a state — such as can somebody access a file or confirming a coin transaction, for example — and the reason you need this is because you don’t have a central server to confirm all this to you,” explains MaidSafe’s COO Nick Lambert, discussing what the protocol is intended to achieve.

“So you need all these decentralized nodes all reaching agreement somehow on a state within the network. Consensus occurs by each of these nodes on the network voting and letting the network as a whole know what it thinks of a transaction.

“It’s almost like consensus could be considered the heart of the networks. It’s required for almost every event in the network.”

We wrote about MaidSafe’s alternative, server-less Internet in 2014. But they actually began work on the project in stealth all the way back in 2006. So they’re over a decade into the R&D at this point.

The network is p2p because it’s being designed so that data is locally encrypted, broken up into pieces and then stored distributed and replicated across the network, relying on the users’ own compute resources to stand in and take the strain. No servers necessary.

The prototype Safe Network is currently in an alpha testing stage (they opened for alpha in 2016). Several more alpha test stages are planned, with a beta release still a distant, undated prospect at this stage. But rearchitecting the entire Internet was clearly never going to be a day’s work.

MaidSafe also ran a multimillion dollar crowdsale in 2014 — for a proxy token of the coin that will eventually be baked into the network — and did so long before ICOs became a crypto-related bandwagon that all sorts of entities were jumping onto. The SafeCoin cryptocurrency is intended to operate as the inventive mechanism for developers to build apps for the Safe Network and users to contribute compute resource and thus bring MaidSafe’s distributed dream alive.

Their timing on the token sale front, coupled with prudent hodling of some of the Bitcoins they’ve raised, means they’re essentially in a position of not having to worry about raising more funds to build the network, according to Lambert.

A rough, back-of-an-envelope calculation on MaidSafe’s original crowdsale suggests, given they raised $2M in Bitcoin in April 2014 when the price for 1BTC was up to around $500, the Bitcoins they obtained then could be worth between ~$30M-$40M by today’s Bitcoin prices — though that would be assuming they held on to most of them. Bitcoin’s price also peaked far higher last year too.

As well as the token sale they also did an equity raise in 2016, via the fintech investment platform bnktothefuture, pulling in around $1.7M from that — in a mixture of cash and “some Bitcoin”.

“It’s gone both ways,” says Lambert, discussing the team’s luck with Bitcoin. “The crowdsale we were on the losing end of Bitcoin price decreasing. We did a raise from bnktothefuture in autumn of 2016… and fortunately we held on to quite a lot of the Bitcoin. So we rode the Bitcoin price up. So I feel like the universe paid us back a little bit for that. So it feels like we’re level now.”

“Fundraising is exceedingly time consuming right through the organization, and it does take a lot of time away from what you wants to be focusing on, and so to be in a position where you’re not desperate for funding is a really nice one to be in,” he adds. “It allows us to focus on the technology and releasing the network.”

The team’s headcount is now up to around 33, with founding members based at the HQ in Ayr, Scotland, and other engineers working remotely or distributed (including in a new dev office they opened in India at the start of this year), even though MaidSafe is still not taking in any revenue.

This April they also made the decision to switch from a dual licensing approach for their software — previously offering both an open source license and a commercial license (which let people close source their code for a fee) — to going only open source, to encourage more developer engagement and contributions to the project, as Lambert tells it.

“We always see the SafeNetwork a bit like a public utility,” he says. “In terms of once we’ve got this thing up and launched we don’t want to control it or own it because if we do nobody will want to use it — it needs to be seen as everyone contributing. So we felt it’s a much more encouraging sign for developers who want to contribute if they see everything is fully open sourced and cannot be closed source.”

MaidSafe’s story so far is reason enough to take note of their whitepaper.

But the consensus issue the paper addresses is also a key challenge for decentralized networks so any proposed solution is potentially a big deal — if indeed it pans out as promised.

  Protocol for Asynchronous, Reliable, Secure and Efficient Consensus

MaidSafe reckons they’ve come up with a way of achieving consensus on decentralized networks that’s scalable, robust and efficient. Hence the name of the protocol — ‘Parsec’ — being short for: ‘Protocol for Asynchronous, Reliable, Secure and Efficient Consensus’.

They will be open sourcing the protocol under a GPL v3 license — with a rough timeframe of “months” for that release, according to Lambert.

He says they’ve been working on Parsec for the last 18 months to two years — but also drawing on earlier research the team carried out into areas such as conflict-free replicated data types, synchronous and asynchronous consensus, and topics such as threshold signatures and common coin.

More specifically, the research underpinning Parsec is based on the following five papers: 1. Baird L. The Swirlds Hashgraph Consensus Algorithm: Fair, Fast, Byzantine Fault Tolerance, Swirlds Tech Report SWIRLDS-TR-2016-01 (2016); 2. Mostefaoui A., Hamouna M., Raynal M. Signature-Free Asynchronous Byzantine Consensus with t <n/3 and O(n 2 ) Messages, ACM PODC (2014); 3. Micali S. Byzantine Agreement, Made Trivial, (2018); 4. Miller A., Xia Y., Croman K., Shi E., Song D. The Honey Badger of BFT Protocols, CCS (2016); 5. Team Rocket Snowflake to Avalanche: A Novel Metastable Consensus Protocol Family for Cryptocurrencies, (2018).

One tweet responding to the protocol’s unveiling just over a week ago wonders whether it’s too good to be true. Time will tell — but the potential is certainly enticing.

Bitcoin’s use of a drastically energy-inefficient ‘proof of work’ method to achieve consensus and write each transaction to its blockchain very clearly doesn’t scale. It’s slow, cumbersome and wasteful. And how to get blockchain-based networks to support the billions of transactions per second that might be needed to sustain the various envisaged applications remains an essential work in progress — with projects investigating various ideas and approaches to try to overcome the limitation.

MaidSafe’s network is not blockchain-based. It’s engineered to function with asynchronous voting of nodes, rather than synchronous voting, which should avoid the bottleneck problems associated with blockchain. But it’s still decentralized. So it needs a consensus mechanism to enable operations and transactions to be carried out autonomously and robustly. That’s where Parsec is intended to slot in.

The protocol does not use proof of work. And is able, so the whitepaper claims, to achieve consensus even if a third of the network is comprised of malicious nodes — i.e. nodes which are attempting to disrupt network operations or otherwise attack the network.

Another claimed advantage is that decisions made via the protocol are both mathematically guaranteed and irreversible.

“What Parsec does is it can reach consensus even with malicious nodes. And up to a third of the nodes being malicious is what the maths proofs suggest,” says Lambert. “This ability to provide mathematical guarantees that all parts of the network will come to the same agreement at a point in time, even with some fault in the network or bad actors — that’s what Byzantine Fault Tolerance is.”

In theory a blockchain using proof of work could be hacked if any one entity controlled 51% of the nodes on the network (although in reality it’s likely that such a large amount of energy would be required it’s pretty much impractical).

So on the surface MaidSafe’s decentralized network — which ‘only’ needs 33% of its nodes to be compromised for its consensus decisions to be attacked — sounds rather less robust. But Lambert says it’s more nuanced than the numbers suggest. And in fact the malicious third would also need to be nodes that have the authority to vote. “So it is a third but it’s a third of well reputed nodes,” as he puts it.

So there’s an element of proof of stake involved too, bound up with additional planned characteristics of the Safe Network — related to dynamic membership and sharding (Lambert says MaidSafe has additional whitepapers on both those elements coming soon).

“Those two papers, particularly the one around dynamic membership, will explain why having a third of malicious nodes is actually harder than just having 33% of malicious nodes. Because the nodes that can vote have to have a reputation as well. So it’s not just purely you can flood the Safe Network with lots and lots of malicious nodes and override it only using a third of the nodes. What we’re saying is the nodes that can vote and actually have a say must have a good reputation in the network,” he says.

“The other thing is proof of stake… Everyone is desperate to move away from proof of work because of its environmental impact. So proof of stake — I liken it to the Scottish landowners, where people with a lot of power have more say. In the cryptocurrency field, proof of stake might be if you have, let’s say, 10 coins and I have one coin your vote might be worth 10x as much authority as what my one coin would be. So any of these mechanisms that they come up with it has that weighting to it… So the people with the most vested interests in the network are also given the more votes.”

Sharding refers to closed groups that allow for consensus votes to be reached by a subset of nodes on a decentralized network. By splitting the network into small sections for consensus voting purposes the idea is you avoid the inefficiencies of having to poll all the nodes on the network — yet can still retain robustness, at least so long as subgroups are carefully structured and secured.

“If you do that correctly you can make it more secure and you can make things much more efficient and faster,” says Lambert. “Because rather than polling, let’s say 6,000 nodes, you might be polling eight nodes. So you can get that information back quickly.

“Obviously you need to be careful about how you do that because with much less nodes you can potentially game the network so you need to be careful how you secure those smaller closed groups or shards. So that will be quite a big thing because pretty much every crypto project is looking at sharding to make, certainly, blockchains more efficient. And so the fact that we’ll have something coming out in that, after we have the dynamic membership stuff coming out, is going to be quite exciting to see the reaction to that as well.”

Voting authority on the Safe Network might be based on a node’s longevity, quality and historical activity — so a sort of ‘reputation’ score (or ledger) that can yield voting rights over time.

“If you’re like that then you will have a vote in these closed groups. And so a third of those votes — and that then becomes quite hard to game because somebody who’s then trying to be malicious would need to have their nodes act as good corporate citizens for a time period. And then all of a sudden become malicious, by which time they’ve probably got a vested stake in the network. So it wouldn’t be possible for someone to just come and flood the network with new nodes and then be malicious because it would not impact upon the network,” Lambert suggests.

The computing power that would be required to attack the Safe Network once it’s public and at scale would also be “really, really significant”, he adds. “Once it gets to scale it would be really hard to co-ordinate anything against it because you’re always having to be several hundred percent bigger than the network and then have a co-ordinated attack on it itself. And all of that work might get you to impact the decision within one closed group. So it’s not even network wide… And that decision could be on who accesses one piece of encrypted shard of data for example… Even the thing you might be able to steal is only an encrypted shard of something — it’s not even the whole thing.”

Other distributed ledger projects are similarly working on Asynchronous Byzantine Fault Tolerant (AFBT) consensus models, including those using directed acrylic graphs (DAGs) — another nascent decentralization technology that’s been suggested as an alternative to blockchain.

And indeed AFBT techniques predate Bitcoin, though MaidSafe says these kind of models have only more recently become viable thanks to research and the relative maturing of decentralized computing and data types, itself as a consequence of increased interest and investment in the space.

However in the case of Hashgraph — the DAG project which has probably attracted the most attention so far — it’s closed source, not open. So that’s one major difference with MaidSafe’s approach. 

Another difference that Lambert points to is that Parsec has been built to work in a dynamic, permissionless network environment (essential for the intended use-case, as the Safe Network is intended as a public network). Whereas he claims Hashgraph has only demonstrated its algorithms working on a permissioned (and therefore private) network “where all the nodes are known”.

He also suggests there’s a question mark over whether Hashgraph’s algorithm can achieve consensus when there are malicious nodes operating on the network. Which — if true — would limit what it can be used for.

“The Hashgraph algorithm is only proven to reach agreement if there’s no adversaries within the network,” Lambert claims. “So if everything’s running well then happy days, but if there’s any maliciousness or any failure within that network then — certainly on the basis of what’s been published — it would suggest that that algorithm was not going to hold up to that.”

“I think being able to do all of these things asynchronously with all of the mathematical guarantees is very difficult,” he continues, returning to the core consensus challenge. “So at the moment we see that we have come out with something that is unique, that covers a lot of these bases, and is a very good use for our use-case. And I think will be useful for others — so I think we like to think that we’ve made a paradigm shift or a vast improvement over the state of the art.”

  Paradigm shift vs marginal innovation

Despite the team’s conviction that, with Parsec, they’ve come up with something very notable, early feedback includes some very vocal Twitter doubters.

For example there’s a lengthy back-and-forth between several MaidSafe engineers and Ethereum researcher Vlad Zamfir — who dubs the Parsec protocol “overhyped” and a “marginal innovation if that”… so, er, ouch.

Well, I don't think asynchronous consensus on an order of events can be described as "never been done before" or "a paradigm shift". And it's not asynchronous, but "very asynchronous" or "randomly synchronous" [sic], and this synchrony assumption isn't very clearly documented

— Vlad ''not giving away ETH'' Zamfir (@VladZamfir) May 31, 2018

Lambert is, if not entirely sanguine, then solidly phlegmatic in the face of a bit of initial Twitter blowback — saying he reckons it will take more time for more detailed responses to come, i.e. allowing for people to properly digest the whitepaper.

“In the world of async BFT algorithms, any advance is huge,” MaidSafe CEO David Irvine also tells us when we ask for a response to Zamfir’s critique. “How huge is subjective, but any advance has to be great for the world. We hope others will advance Parsec like we have built on others (as we clearly state and thank them for their work).  So even if it was a marginal development (which it certainly is not) then I would take that.”

“All in all, though, nothing was said that took away from the fact Parsec moves the industry forward,” he adds. “I felt the comments were a bit juvenile at times and a bit defensive (probably due to us not agreeing with POS in our Medium post) but in terms of the only part commented on (the coin flip) we as a team feel that part could be much more concrete in terms of defining exactly how small such random (finite) delays could be. We know they do not stop the network and a delaying node would be killed, but for completeness, it would be nice to be that detailed.”

A developer source of our own in the crypto/blockchain space — who’s not connected to the MaidSafe or Ethereum projects — also points out that Parsec “getting objective review will take some time given that so many potential reviewers have vested interest in their own project/coin”.

It’s certainly fair to say the space excels at public spats and disagreements. Researchers pouring effort into one project can be less than kind to rivals’ efforts. (And, well, given all the crypto Lambos at stake it’s not hard to see why there can be no love lost — and, ironically, zero trust — between competing champions of trustless tech.)

Another fundamental truth of these projects is they’re all busily experimenting right now, with lots of ideas in play to try and fix core issues like scalability, efficiency and robustness — often having different ideas over implementation even if rival projects are circling and/or converging on similar approaches and techniques.

“Certainly other projects are looking at sharding,” says Lambert. “So I know that Ethereum are looking at sharding. And I think Bitcoin are looking at that as well, but I think everyone probably has quite different ideas about how to implement it. And of course we’re not using a blockchain which makes that another different use-case where Ethereum and Bitcoin obviously are. But everyone has — as with anything — these different approaches and different ideas.”

“Every network will have its own different ways of doing [consensus],” he adds when asked whether he believes Parsec could be adopted by other projects wrestling with the consensus challenge. “So it’s not like some could lift [Parsec] out and just put it in. Ethereum is blockchain-based — I think they’re looking at something around proof of stake, but maybe they could take some ideas or concepts from the work that we’re open sourcing for their specific case.

“If you get other blockchain-less networks like IOTA, Byteball, I think POA is another one as well. These other projects it might be easier for them to implement something like Parsec with them because they’re not using blockchain. So maybe less of that adaption required.”

Whether other projects will deem Parsec worthy of their attention remains to be seen at this point with so much still to play for. Some may prefer to expend effort trying to rubbish a rival approach, whose open source tech could, if it stands up to scrutiny and operational performance, reduce the commercial value of proprietary and patented mechanisms also intended to grease the wheels of decentralized networks — for a fee.

And of course MaidSafe’s developed-in-stealth consensus protocol may also turn out to be a relatively minor development. But finding a non-vested expert to give an impartial assessment of complex network routing algorithms conjoined to such a self-interested and, frankly, anarchical industry is another characteristic challenge of the space.

Irvine’s view is that DAG based projects which are using a centralized component will have to move on or adopt what he dubs “state of art” asynchronous consensus algorithms — as MaidSafe believes Parsec is — aka, algorithms which are “more widely accepted and proven”.

“So these projects should contribute to the research, but more importantly, they will have to adopt better algorithms than they use,” he suggests. “So they can play an important part, upgrades! How to upgrade a running DAG based network? How to had fork a graph? etc. We know how to hard fork blockchains, but upgrading DAG based networks may not be so simple when they are used as ledgers.

“Projects like Hashgraph, Algorand etc will probably use an ABFT algorithm like this as their whole network with a little work for a currency; IOTA, NANO, Bytball etc should. That is entirely possible with advances like Parsec. However adding dynamic membership, sharding, a data layer then a currency is a much larger proposition, which is why Parsec has been in stealth mode while it is being developed.

“We hope that by being open about the algorithm, and making the code open source when complete, we will help all the other projects working on similar problems.”

Of course MaidSafe’s team might be misguided in terms of the breakthrough they think they’ve made with Parsec. But it’s pretty hard to stand up the idea they’re being intentionally misleading.

Because, well, what would be the point of that? While the exact depth of MaidSafe’s funding reserves isn’t clear, Lambert doesn’t sound like a startup guy with money worries. And the team’s staying power cannot be in doubt — over a decade into the R&D needed to underpin their alt network.

It’s true that being around for so long does have some downsides, though. Especially, perhaps, given how hyped the decentralized space has now become. “Because we’ve been working on it for so long, and it’s been such a big project, you can see some negative feedback about that,” as Lambert admits.

And with such intense attention now on the space, injecting energy which in turn accelerates ideas and activity, there’s perhaps extra pressure on a veteran player like MaidSafe to be seen making a meaningful contribution — ergo, it might be tempting for the team to believe the consensus protocol they’ve engineered really is a big deal.

To stand up and be counted amid all the noise, as it were. And to draw attention to their own project — which needs lots of external developers to buy into the vision if it’s to succeed, yet, here in 2018, it’s just one decentralization project among so many. 

  The Safe Network roadmap

Consensus aside, MaidSafe’s biggest challenge is still turning the sizable amount of funding and resources the team’s ideas have attracted to date into a bona fide alternative network that anyone really can use. And there’s a very long road to travel still on that front, clearly.

The Safe Network is in alpha 2 testing incarnation (which has been up and running since September last year) — consisting of around a hundred nodes that MaidSafe is maintaining itself.

The core decentralization proposition of anyone being able to supply storage resource to the network via lending their own spare capacity is not yet live — and won’t come fully until alpha 4.

“People are starting to create different apps against that network. So we’ve seen Jams — a decentralized music player… There are a couple of storage style apps… There is encrypted email running as well, and also that is running on Android,” says Lambert. “And we have a forked version of the Beaker browser — that’s the browser that we use right now. So if you can create websites on the Safe Network, which has its own protocol, and if you want to go and view those sites you need a Safe browser to do that, so we’ve also been working on our own browser from scratch that we’ll be releasing later this year… So there’s a number of apps that are running against that alpha 2 network.

“What alpha 3 will bring is it will run in parallel with alpha 2 but it will effectively be a decentralized routing network. What that means is it will be one for more technical people to run, and it will enable data to be passed around a network where anyone can contribute their resources to it but it will not facilitate data storage. So it’ll be a command line app, which is probably why it’ll suit technical people more because there’ll be no user interface for it, and they will contribute their resources to enable messages to be passed around the network. So secure messaging would be a use-case for that.

“And then alpha 4 is effectively bringing together alpha 2 and alpha 3. So it adds a storage layer on top of the alpha 3 network — and at that point it gives you the fully decentralized network where users are contributing their resources from home and they will be able to store data, send messages and things of that nature. Potentially during alpha 4, or a later alpha, we’ll introduce test SafeCoin. Which is the final piece of the initial puzzle to provide incentives for users to provide resources and for developers to make apps. So that’s probably what the immediate roadmap looks like.”

On the timeline front Lambert won’t be coaxed into fixing any deadlines to all these planned alphas. They’ve long ago learnt not to try and predict the pace of progress, he says with a laugh. Though he does not question that progress is being made.

“These big infrastructure projects are typically only government funded because the payback is too slow for venture capitalists,” he adds. “So in the past you had things like Arpanet, the precursor to the Internet — that was obviously a US government funded project — and so we’ve taken on a project which has, not grown arms and legs, but certainly there’s more to it than what was initially thought about.

“So we are almost privately funding this infrastructure. Which is quite a big scope, and I will say why it’s taking a bit of time. But we definitely do seem to be making lots of progress.”

via TechCrunch

[abckidstvyara]

Given all the hype and noise swirling around crypto and decentralized network projects, which runs the full gamut from scams and stupidity, to very clever and inspired ideas, the release of yet another whitepaper does not immediately set off an attention klaxon.

But this whitepaper — which details a new protocol for achieving consensus within a decentralized network — is worth paying more attention to than most.

MaidSafe, the team behind it, are also the literal opposite of fly-by-night crypto opportunists. They’ve been working on decentralized networking since long before the space became the hot, hyped thing it is now.

Their overarching mission is to engineer an entirely decentralized Internet which bakes in privacy, security and freedom of expression by design — the ‘Safe’ in their planned ‘Safe Network’ stands for ‘Secure access for everyone’ — meaning it’s encrypted, autonomous, self-organizing, self-healing. And the new consensus protocol is just another piece towards fulfilling that grand vision.

What’s consensus in decentralized networking terms? “Within decentralized networks you must have a way of the network agreeing on a state — such as can somebody access a file or confirming a coin transaction, for example — and the reason you need this is because you don’t have a central server to confirm all this to you,” explains MaidSafe’s COO Nick Lambert, discussing what the protocol is intended to achieve.

“So you need all these decentralized nodes all reaching agreement somehow on a state within the network. Consensus occurs by each of these nodes on the network voting and letting the network as a whole know what it thinks of a transaction.

“It’s almost like consensus could be considered the heart of the networks. It’s required for almost every event in the network.”

We wrote about MaidSafe’s alternative, server-less Internet in 2014. But they actually began work on the project in stealth all the way back in 2006. So they’re over a decade into the R&D at this point.

The network is p2p because it’s being designed so that data is locally encrypted, broken up into pieces and then stored distributed and replicated across the network, relying on the users’ own compute resources to stand in and take the strain. No servers necessary.

The prototype Safe Network is currently in an alpha testing stage (they opened for alpha in 2016). Several more alpha test stages are planned, with a beta release still a distant, undated prospect at this stage. But rearchitecting the entire Internet was clearly never going to be a day’s work.

MaidSafe also ran a multimillion dollar crowdsale in 2014 — for a proxy token of the coin that will eventually be baked into the network — and did so long before ICOs became a crypto-related bandwagon that all sorts of entities were jumping onto. The SafeCoin cryptocurrency is intended to operate as the inventive mechanism for developers to build apps for the Safe Network and users to contribute compute resource and thus bring MaidSafe’s distributed dream alive.

Their timing on the token sale front, coupled with prudent hodling of some of the Bitcoins they’ve raised, means they’re essentially in a position of not having to worry about raising more funds to build the network, according to Lambert.

A rough, back-of-an-envelope calculation on MaidSafe’s original crowdsale suggests, given they raised $2M in Bitcoin in April 2014 when the price for 1BTC was up to around $500, the Bitcoins they obtained then could be worth between ~$30M-$40M by today’s Bitcoin prices — though that would be assuming they held on to most of them. Bitcoin’s price also peaked far higher last year too.

As well as the token sale they also did an equity raise in 2016, via the fintech investment platform bnktothefuture, pulling in around $1.7M from that — in a mixture of cash and “some Bitcoin”.

“It’s gone both ways,” says Lambert, discussing the team’s luck with Bitcoin. “The crowdsale we were on the losing end of Bitcoin price decreasing. We did a raise from bnktothefuture in autumn of 2016… and fortunately we held on to quite a lot of the Bitcoin. So we rode the Bitcoin price up. So I feel like the universe paid us back a little bit for that. So it feels like we’re level now.”

“Fundraising is exceedingly time consuming right through the organization, and it does take a lot of time away from what you wants to be focusing on, and so to be in a position where you’re not desperate for funding is a really nice one to be in,” he adds. “It allows us to focus on the technology and releasing the network.”

The team’s headcount is now up to around 33, with founding members based at the HQ in Ayr, Scotland, and other engineers working remotely or distributed (including in a new dev office they opened in India at the start of this year), even though MaidSafe is still not taking in any revenue.

This April they also made the decision to switch from a dual licensing approach for their software — previously offering both an open source license and a commercial license (which let people close source their code for a fee) — to going only open source, to encourage more developer engagement and contributions to the project, as Lambert tells it.

“We always see the SafeNetwork a bit like a public utility,” he says. “In terms of once we’ve got this thing up and launched we don’t want to control it or own it because if we do nobody will want to use it — it needs to be seen as everyone contributing. So we felt it’s a much more encouraging sign for developers who want to contribute if they see everything is fully open sourced and cannot be closed source.”

MaidSafe’s story so far is reason enough to take note of their whitepaper.

But the consensus issue the paper addresses is also a key challenge for decentralized networks so any proposed solution is potentially a big deal — if indeed it pans out as promised.

  Protocol for Asynchronous, Reliable, Secure and Efficient Consensus

MaidSafe reckons they’ve come up with a way of achieving consensus on decentralized networks that’s scalable, robust and efficient. Hence the name of the protocol — ‘Parsec’ — being short for: ‘Protocol for Asynchronous, Reliable, Secure and Efficient Consensus’.

They will be open sourcing the protocol under a GPL v3 license — with a rough timeframe of “months” for that release, according to Lambert.

He says they’ve been working on Parsec for the last 18 months to two years — but also drawing on earlier research the team carried out into areas such as conflict-free replicated data types, synchronous and asynchronous consensus, and topics such as threshold signatures and common coin.

More specifically, the research underpinning Parsec is based on the following five papers: 1. Baird L. The Swirlds Hashgraph Consensus Algorithm: Fair, Fast, Byzantine Fault Tolerance, Swirlds Tech Report SWIRLDS-TR-2016-01 (2016); 2. Mostefaoui A., Hamouna M., Raynal M. Signature-Free Asynchronous Byzantine Consensus with t <n/3 and O(n 2 ) Messages, ACM PODC (2014); 3. Micali S. Byzantine Agreement, Made Trivial, (2018); 4. Miller A., Xia Y., Croman K., Shi E., Song D. The Honey Badger of BFT Protocols, CCS (2016); 5. Team Rocket Snowflake to Avalanche: A Novel Metastable Consensus Protocol Family for Cryptocurrencies, (2018).

One tweet responding to the protocol’s unveiling just over a week ago wonders whether it’s too good to be true. Time will tell — but the potential is certainly enticing.

Bitcoin’s use of a drastically energy-inefficient ‘proof of work’ method to achieve consensus and write each transaction to its blockchain very clearly doesn’t scale. It’s slow, cumbersome and wasteful. And how to get blockchain-based networks to support the billions of transactions per second that might be needed to sustain the various envisaged applications remains an essential work in progress — with projects investigating various ideas and approaches to try to overcome the limitation.

MaidSafe’s network is not blockchain-based. It’s engineered to function with asynchronous voting of nodes, rather than synchronous voting, which should avoid the bottleneck problems associated with blockchain. But it’s still decentralized. So it needs a consensus mechanism to enable operations and transactions to be carried out autonomously and robustly. That’s where Parsec is intended to slot in.

The protocol does not use proof of work. And is able, so the whitepaper claims, to achieve consensus even if a third of the network is comprised of malicious nodes — i.e. nodes which are attempting to disrupt network operations or otherwise attack the network.

Another claimed advantage is that decisions made via the protocol are both mathematically guaranteed and irreversible.

“What Parsec does is it can reach consensus even with malicious nodes. And up to a third of the nodes being malicious is what the maths proofs suggest,” says Lambert. “This ability to provide mathematical guarantees that all parts of the network will come to the same agreement at a point in time, even with some fault in the network or bad actors — that’s what Byzantine Fault Tolerance is.”

In theory a blockchain using proof of work could be hacked if any one entity controlled 51% of the nodes on the network (although in reality it’s likely that such a large amount of energy would be required it’s pretty much impractical).

So on the surface MaidSafe’s decentralized network — which ‘only’ needs 33% of its nodes to be compromised for its consensus decisions to be attacked — sounds rather less robust. But Lambert says it’s more nuanced than the numbers suggest. And in fact the malicious third would also need to be nodes that have the authority to vote. “So it is a third but it’s a third of well reputed nodes,” as he puts it.

So there’s an element of proof of stake involved too, bound up with additional planned characteristics of the Safe Network — related to dynamic membership and sharding (Lambert says MaidSafe has additional whitepapers on both those elements coming soon).

“Those two papers, particularly the one around dynamic membership, will explain why having a third of malicious nodes is actually harder than just having 33% of malicious nodes. Because the nodes that can vote have to have a reputation as well. So it’s not just purely you can flood the Safe Network with lots and lots of malicious nodes and override it only using a third of the nodes. What we’re saying is the nodes that can vote and actually have a say must have a good reputation in the network,” he says.

“The other thing is proof of stake… Everyone is desperate to move away from proof of work because of its environmental impact. So proof of stake — I liken it to the Scottish landowners, where people with a lot of power have more say. In the cryptocurrency field, proof of stake might be if you have, let’s say, 10 coins and I have one coin your vote might be worth 10x as much authority as what my one coin would be. So any of these mechanisms that they come up with it has that weighting to it… So the people with the most vested interests in the network are also given the more votes.”

Sharding refers to closed groups that allow for consensus votes to be reached by a subset of nodes on a decentralized network. By splitting the network into small sections for consensus voting purposes the idea is you avoid the inefficiencies of having to poll all the nodes on the network — yet can still retain robustness, at least so long as subgroups are carefully structured and secured.

“If you do that correctly you can make it more secure and you can make things much more efficient and faster,” says Lambert. “Because rather than polling, let’s say 6,000 nodes, you might be polling eight nodes. So you can get that information back quickly.

“Obviously you need to be careful about how you do that because with much less nodes you can potentially game the network so you need to be careful how you secure those smaller closed groups or shards. So that will be quite a big thing because pretty much every crypto project is looking at sharding to make, certainly, blockchains more efficient. And so the fact that we’ll have something coming out in that, after we have the dynamic membership stuff coming out, is going to be quite exciting to see the reaction to that as well.”

Voting authority on the Safe Network might be based on a node’s longevity, quality and historical activity — so a sort of ‘reputation’ score (or ledger) that can yield voting rights over time.

“If you’re like that then you will have a vote in these closed groups. And so a third of those votes — and that then becomes quite hard to game because somebody who’s then trying to be malicious would need to have their nodes act as good corporate citizens for a time period. And then all of a sudden become malicious, by which time they’ve probably got a vested stake in the network. So it wouldn’t be possible for someone to just come and flood the network with new nodes and then be malicious because it would not impact upon the network,” Lambert suggests.

The computing power that would be required to attack the Safe Network once it’s public and at scale would also be “really, really significant”, he adds. “Once it gets to scale it would be really hard to co-ordinate anything against it because you’re always having to be several hundred percent bigger than the network and then have a co-ordinated attack on it itself. And all of that work might get you to impact the decision within one closed group. So it’s not even network wide… And that decision could be on who accesses one piece of encrypted shard of data for example… Even the thing you might be able to steal is only an encrypted shard of something — it’s not even the whole thing.”

Other distributed ledger projects are similarly working on Asynchronous Byzantine Fault Tolerant (AFBT) consensus models, including those using directed acrylic graphs (DAGs) — another nascent decentralization technology that’s been suggested as an alternative to blockchain.

And indeed AFBT techniques predate Bitcoin, though MaidSafe says these kind of models have only more recently become viable thanks to research and the relative maturing of decentralized computing and data types, itself as a consequence of increased interest and investment in the space.

However in the case of Hashgraph — the DAG project which has probably attracted the most attention so far — it’s closed source, not open. So that’s one major difference with MaidSafe’s approach. 

Another difference that Lambert points to is that Parsec has been built to work in a dynamic, permissionless network environment (essential for the intended use-case, as the Safe Network is intended as a public network). Whereas he claims Hashgraph has only demonstrated its algorithms working on a permissioned (and therefore private) network “where all the nodes are known”.

He also suggests there’s a question mark over whether Hashgraph’s algorithm can achieve consensus when there are malicious nodes operating on the network. Which — if true — would limit what it can be used for.

“The Hashgraph algorithm is only proven to reach agreement if there’s no adversaries within the network,” Lambert claims. “So if everything’s running well then happy days, but if there’s any maliciousness or any failure within that network then — certainly on the basis of what’s been published — it would suggest that that algorithm was not going to hold up to that.”

“I think being able to do all of these things asynchronously with all of the mathematical guarantees is very difficult,” he continues, returning to the core consensus challenge. “So at the moment we see that we have come out with something that is unique, that covers a lot of these bases, and is a very good use for our use-case. And I think will be useful for others — so I think we like to think that we’ve made a paradigm shift or a vast improvement over the state of the art.”

  Paradigm shift vs marginal innovation

Despite the team’s conviction that, with Parsec, they’ve come up with something very notable, early feedback includes some very vocal Twitter doubters.

For example there’s a lengthy back-and-forth between several MaidSafe engineers and Ethereum researcher Vlad Zamfir — who dubs the Parsec protocol “overhyped” and a “marginal innovation if that”… so, er, ouch.

Well, I don't think asynchronous consensus on an order of events can be described as "never been done before" or "a paradigm shift". And it's not asynchronous, but "very asynchronous" or "randomly synchronous" [sic], and this synchrony assumption isn't very clearly documented

— Vlad ''not giving away ETH'' Zamfir (@VladZamfir) May 31, 2018

Lambert is, if not entirely sanguine, then solidly phlegmatic in the face of a bit of initial Twitter blowback — saying he reckons it will take more time for more detailed responses to come, i.e. allowing for people to properly digest the whitepaper.

“In the world of async BFT algorithms, any advance is huge,” MaidSafe CEO David Irvine also tells us when we ask for a response to Zamfir’s critique. “How huge is subjective, but any advance has to be great for the world. We hope others will advance Parsec like we have built on others (as we clearly state and thank them for their work).  So even if it was a marginal development (which it certainly is not) then I would take that.”

“All in all, though, nothing was said that took away from the fact Parsec moves the industry forward,” he adds. “I felt the comments were a bit juvenile at times and a bit defensive (probably due to us not agreeing with POS in our Medium post) but in terms of the only part commented on (the coin flip) we as a team feel that part could be much more concrete in terms of defining exactly how small such random (finite) delays could be. We know they do not stop the network and a delaying node would be killed, but for completeness, it would be nice to be that detailed.”

A developer source of our own in the crypto/blockchain space — who’s not connected to the MaidSafe or Ethereum projects — also points out that Parsec “getting objective review will take some time given that so many potential reviewers have vested interest in their own project/coin”.

It’s certainly fair to say the space excels at public spats and disagreements. Researchers pouring effort into one project can be less than kind to rivals’ efforts. (And, well, given all the crypto Lambos at stake it’s not hard to see why there can be no love lost — and, ironically, zero trust — between competing champions of trustless tech.)

Another fundamental truth of these projects is they’re all busily experimenting right now, with lots of ideas in play to try and fix core issues like scalability, efficiency and robustness — often having different ideas over implementation even if rival projects are circling and/or converging on similar approaches and techniques.

“Certainly other projects are looking at sharding,” says Lambert. “So I know that Ethereum are looking at sharding. And I think Bitcoin are looking at that as well, but I think everyone probably has quite different ideas about how to implement it. And of course we’re not using a blockchain which makes that another different use-case where Ethereum and Bitcoin obviously are. But everyone has — as with anything — these different approaches and different ideas.”

“Every network will have its own different ways of doing [consensus],” he adds when asked whether he believes Parsec could be adopted by other projects wrestling with the consensus challenge. “So it’s not like some could lift [Parsec] out and just put it in. Ethereum is blockchain-based — I think they’re looking at something around proof of stake, but maybe they could take some ideas or concepts from the work that we’re open sourcing for their specific case.

“If you get other blockchain-less networks like IOTA, Byteball, I think POA is another one as well. These other projects it might be easier for them to implement something like Parsec with them because they’re not using blockchain. So maybe less of that adaption required.”

Whether other projects will deem Parsec worthy of their attention remains to be seen at this point with so much still to play for. Some may prefer to expend effort trying to rubbish a rival approach, whose open source tech could, if it stands up to scrutiny and operational performance, reduce the commercial value of proprietary and patented mechanisms also intended to grease the wheels of decentralized networks — for a fee.

And of course MaidSafe’s developed-in-stealth consensus protocol may also turn out to be a relatively minor development. But finding a non-vested expert to give an impartial assessment of complex network routing algorithms conjoined to such a self-interested and, frankly, anarchical industry is another characteristic challenge of the space.

Irvine’s view is that DAG based projects which are using a centralized component will have to move on or adopt what he dubs “state of art” asynchronous consensus algorithms — as MaidSafe believes Parsec is — aka, algorithms which are “more widely accepted and proven”.

“So these projects should contribute to the research, but more importantly, they will have to adopt better algorithms than they use,” he suggests. “So they can play an important part, upgrades! How to upgrade a running DAG based network? How to had fork a graph? etc. We know how to hard fork blockchains, but upgrading DAG based networks may not be so simple when they are used as ledgers.

“Projects like Hashgraph, Algorand etc will probably use an ABFT algorithm like this as their whole network with a little work for a currency; IOTA, NANO, Bytball etc should. That is entirely possible with advances like Parsec. However adding dynamic membership, sharding, a data layer then a currency is a much larger proposition, which is why Parsec has been in stealth mode while it is being developed.

“We hope that by being open about the algorithm, and making the code open source when complete, we will help all the other projects working on similar problems.”

Of course MaidSafe’s team might be misguided in terms of the breakthrough they think they’ve made with Parsec. But it’s pretty hard to stand up the idea they’re being intentionally misleading.

Because, well, what would be the point of that? While the exact depth of MaidSafe’s funding reserves isn’t clear, Lambert doesn’t sound like a startup guy with money worries. And the team’s staying power cannot be in doubt — over a decade into the R&D needed to underpin their alt network.

It’s true that being around for so long does have some downsides, though. Especially, perhaps, given how hyped the decentralized space has now become. “Because we’ve been working on it for so long, and it’s been such a big project, you can see some negative feedback about that,” as Lambert admits.

And with such intense attention now on the space, injecting energy which in turn accelerates ideas and activity, there’s perhaps extra pressure on a veteran player like MaidSafe to be seen making a meaningful contribution — ergo, it might be tempting for the team to believe the consensus protocol they’ve engineered really is a big deal.

To stand up and be counted amid all the noise, as it were. And to draw attention to their own project — which needs lots of external developers to buy into the vision if it’s to succeed, yet, here in 2018, it’s just one decentralization project among so many. 

  The Safe Network roadmap

Consensus aside, MaidSafe’s biggest challenge is still turning the sizable amount of funding and resources the team’s ideas have attracted to date into a bona fide alternative network that anyone really can use. And there’s a very long road to travel still on that front, clearly.

The Safe Network is in alpha 2 testing incarnation (which has been up and running since September last year) — consisting of around a hundred nodes that MaidSafe is maintaining itself.

The core decentralization proposition of anyone being able to supply storage resource to the network via lending their own spare capacity is not yet live — and won’t come fully until alpha 4.

“People are starting to create different apps against that network. So we’ve seen Jams — a decentralized music player… There are a couple of storage style apps… There is encrypted email running as well, and also that is running on Android,” says Lambert. “And we have a forked version of the Beaker browser — that’s the browser that we use right now. So if you can create websites on the Safe Network, which has its own protocol, and if you want to go and view those sites you need a Safe browser to do that, so we’ve also been working on our own browser from scratch that we’ll be releasing later this year… So there’s a number of apps that are running against that alpha 2 network.

“What alpha 3 will bring is it will run in parallel with alpha 2 but it will effectively be a decentralized routing network. What that means is it will be one for more technical people to run, and it will enable data to be passed around a network where anyone can contribute their resources to it but it will not facilitate data storage. So it’ll be a command line app, which is probably why it’ll suit technical people more because there’ll be no user interface for it, and they will contribute their resources to enable messages to be passed around the network. So secure messaging would be a use-case for that.

“And then alpha 4 is effectively bringing together alpha 2 and alpha 3. So it adds a storage layer on top of the alpha 3 network — and at that point it gives you the fully decentralized network where users are contributing their resources from home and they will be able to store data, send messages and things of that nature. Potentially during alpha 4, or a later alpha, we’ll introduce test SafeCoin. Which is the final piece of the initial puzzle to provide incentives for users to provide resources and for developers to make apps. So that’s probably what the immediate roadmap looks like.”

On the timeline front Lambert won’t be coaxed into fixing any deadlines to all these planned alphas. They’ve long ago learnt not to try and predict the pace of progress, he says with a laugh. Though he does not question that progress is being made.

“These big infrastructure projects are typically only government funded because the payback is too slow for venture capitalists,” he adds. “So in the past you had things like Arpanet, the precursor to the Internet — that was obviously a US government funded project — and so we’ve taken on a project which has, not grown arms and legs, but certainly there’s more to it than what was initially thought about.

“So we are almost privately funding this infrastructure. Which is quite a big scope, and I will say why it’s taking a bit of time. But we definitely do seem to be making lots of progress.”