#sbmm | Explore Tumblr posts and blogs

caxyanalysis · 1 year ago

Text

The Rise and Fall of "Skill-Based Matchmaking" in Games

There's a term in gaming. All kinds of gaming. Sports, board games, video games. It's a concept that's been around as long as games have had a competitive side to them. The term itself may not have been around that long, but the concept of it has been. It's always been there, even if this specific word wasn't used since the beginning. That term is...

Matchmaking

Matchmaking is the process of deciding an opponent. It can be done manually by the players of the game, or it can be done automatically by some third party force. But it must be done, or the game can't be played.

Matchmaking can be anything from a tournament ladder to a privately issued challenge to a roll of the dice as to who gets to face off against who.

In the modern age, with the rise of digital gaming and competitive video games, matchmaking has come to be known most commonly by two variants, described below.

Connection-Based Matchmaking

CBMM or Connection-Based Matchmaking is a variant of Matchmaking wherein the quality of network connection is prioritized when deciding who you'll face in a game. In a CBMM system, you'll be matched with players as close to your local network as possible, to reduce latency and other network-related issues.

The benefits of this system are that players will never have to worry about lagging to the point of being unable to play, or any player lagging in such a way that it's an unfair advantage. CBMM reduces the network struggles, and allows for players of all skill levels to come together to play the game in the same environment.

The downsides of this system are that players who are within the upper echelons of skill often wind up making the game unfair or unfun for those of lower skill. Because there is no safeguard preventing masters, experienced players, and "sweats" (term refers to those who treat every match, even casual matches, as if they are competing for a million dollar prize) from being matched against brand new, casual, and unskilled players, those of lower skill often suffer in "curb stomps", which are matches where one team or one player so far outplays the other team or player that the outcome is clear and unavoidable within the first minute or so.

Skill-Based Matchmaking

SBMM or Skill-Based Matchmaking is a variant of Matchmaking wherein the ability of the players is prioritized when deciding who you'll face in a game. In a SBMM system, you'll be matched with players as close to your determined skill level as possible to reduce the amount of unfair or one-sided games.

The benefits of this system are that low-skill, brand new, or casual players never need to worry about "curb stomps", or facing off against "sweats". They can enjoy matches within their skill level, and will have a fair, fun time against people who have equal chance to win or lose, all depending on how each team plays rather than depending on which team was assigned more "sweats".

The downsides of this system are that the upper echelons of players, the "sweats", begin struggling to find entertaining matches after a time. SBMM introduces a system where high-skill players can only match other high-skill players, and so eventually the top 1% will stop being able to connect to anyone except the top 0.5%, and the top 0.5% will eventually stop being able to connect to anyone except the top 0.1%, and so on. This system heavily benefits skill improvement, until that improvement reaches a pinnacle point where the only matches you get are against the same set of people, every time, until every game is a boring replay of the last.

But there is a third variant, which has sub-variants of its own. Which is...

Hybrid Matchmaking

HMM or Hybrid Matchmaking is when the game considers both skill level AND connection. This system combines the strengths of BOTH other variants, but execution is everything.

In one sub-variant of HMM, we have CBHMM, which prioritizes connection, but will allocate players in such a way that once connection has been determined, you will be matched with the player or players closest in skill to you.

In another, we have SBHMM, which prioritizes skill, but will also seek players with good connection so that you don't wind up facing the same exact people every single time.

And in yet another variant we have THMM, or True Hybrid Matchmaking, which usually involves a skill tier rather than a skill rating, like the common Silver 1, 2, and 3, Bronze 1, 2, and 3, etc, instead of the ratings like "1500 ELO" or "2300 ELO", etc.

THMM will assign players a skill tier to determine the pool of players they are able to match with, and then prioritize connection within that pool of players. This combines the greatest strength of CBMM (shortest possible wait time to find a match) with the greatest strength of SBMM (not facing off against opponents you can't beat).

To go over some real examples, let's cover some games that use these systems. I'll stick to games I am familiar with to avoid any misinformation.

Destiny 2

In Destiny 2, the type of Matchmaking is determined by the mode you play. In Competitive, it's SBMM, but with the tier system of THMM. More on that in a moment. In Quickplay, it's CBMM, with a tier system used for reward progression rather than any SBMM purposes. In Trials, the ultimate competitive PvP experience in D2, it's a very unorthodox form of Hybrid Matchmaking.

Comp SBMM in D2 functions like this: You have a skill tier (Bronze, Silver, etc), and the game assigns a point total you must reach to move up to the next tier. You earn points based on wins, and lose points based on losses. This system has one major flaw. In Comp, you have 2 teammates that can win for you.

This means that even if you personally are standing in spawn dancing, you can still hit the maximum possible rank with the right teammates. Personal performance means nothing, only the outcome of the match.

Quickplay being CBMM is problematic, as the fact it's quickplay (which is designed to be the "casual" play mode) is lost on a notable amount of players. The "sweats" run rampant in quickplay, actively reducing the amount of fun players can have in what is designed to be the "for fun" PvP mode. If it were one or two sweats, it would be fine, but entire squads of them (3-6 players working together) will queue into a match and, while the game tries to match squads to other squads, it does not change the fact that casuals ALSO squad up, and a team of 3 sweats will still perform significantly better than a squad of 6 casuals. CBMM in the "for fun" mode means no safety for casual players to simply load up to play for the fun of it, and forces everyone into a highly competitive environment even in the "casual" mode.

Trials is a unique system. Trials of Osiris bears a mixture of SBMM and CBMM, making it Hybrid Matchmaking, but of a strange sort. In Trials, you have a "Trials Card", which tracks your wins, up to 7, as well as whether or not those wins are "Flawless", a term used here to mean "consecutive". Players with 7 consecutive wins are considered Flawless players for the sake of matchmaking. Trials considers the following, in this order, when considering how to match you to other players: Flawless (yes or no), Win Count, Connection, Skill Tier. By this system, it should first check for Flawless status (no losses recorded on the card) followed by how many wins are recorded, filter out poor connection players, then find players in your skill tier within that filtered pool. Starting on Sunday of every week, it first checks to see if YOU are a Flawless player first, and if you are, it applies a hidden "Flawless" tag to you, preventing you from matching with anyone that does not have this Flawless tag. Once the game has set your pool (Flawless, not Flawless), it seeks out players who have the same number of wins on their Trials Card. However, because the Card only tracks up to 7, and because you can reset your card, a player can reach 6 wins, reset, and be treated as having 0 wins. This means a player can "farm" easy wins by simply resetting at 6 wins every time without going Flawless (7 consecutive wins), meaning they remain outside of the Flawless pool and continually match against players with 0 recorded wins. Once the game has set the pool of players based on Card-Recorded Win Count, it seeks out players within that pool that have a solid connection quality. This means you will be matched against players with similar win-counts that have solid connection. After that it finally considers Skill Tier, putting you against the closest possible skill tier. This is a huge problem, as Skill being considered last means BOTH of the following can happen:

Player A queues up while Player B is waiting in queue. A is Skill Tier 6 while B is Skill Tier 1. A has a 15% winrate while B has a 98% winrate, despite both having over 1,000 matches played. A has 0 wins recorded, 0 losses, as they are just now playing for the first time. B has 0 wins recorded, 0 losses, as they just reset their card for the 9th time today after reaching 6 consecutive wins for the 9th time today. Neither A nor B are Flawless, so both are in the Non-Flawless Pool. A and B each have 0 wins, 0 losses recorded, so both are in the 0/0 Pool. A and B have strong connection, and so the matchmaking filters them into the same pool. The two players closest to A in Skill Tier are in Tier 5, the two closest to B are in Tier 2, and are the only other players within the filtered pool. A is now matched with 2 Tier 5 players, 2 Tier 2 players, and 1 Tier 1 player, making them the lowest Tier player in the lobby, and making B the highest Tier player in the lobby. D2 has a "Lobby Balancing feature" which tries to balance skill levels among teams. This causes the game to be a team consisting of either the highest AND lowest Tier players accompanied by one of the Tier 2s or 5s, OR a team consisting of the lowest player accompanied by the two Tier 2s opposing a team consisting of the highest player accompanied by the two Tier 5s. Both of these result in a curb stomp, as one team lacks the ability to overcome the odds against them. This system is heavily flawed, as the priority of filters and the fact that they are not SOLID filters means that while matches are often found quickly, they are often OVER just as quickly due to unfair balancing of teams and an imbalance in the skill of players matched.

Fortnite

Fortnite, casual, is simple CBMM. Regardless of mode, it is pure CBMM with one caveat. It is CBMM within the selected region. This means if you live in the USA and set the UK as your region, it will match you with the players that have the strongest connection to you that are also on the UK region, even though you live in the USA. This means you will usually have a good connection, but is very easy to cheat connections to give yourself some level of intentional latency by playing on a region you have inherently poor connection to.

However, the Ranked mode does have SBMM, of sorts. Ranked mode SBMM is determined entirely by the placement you reach in your matches. If you win a match and get 1st place, you will be matched with other 1st place players. Over time, it averages out your placements, and will put you in matches with players that have similar placement averages. So if you play 1,000 matches and your average placement is 43rd place, you will be matched with other players who have ~43rd place as their average. However, if you play 1,000 matches and your average placement is 2nd place, you'll be matched with other players who have ~2nd place averages. This is a problem due to the fact that one can reach as high as 15th, 10th, or even 2nd place doing absolutely nothing at all except moving from hiding spot to hiding spot and waiting for other players to kill each other. Meaning if you're a cautious player, you can achieve the higher skill tiers by simply not engaging, even if you die the moment you do encounter another player. To rephrase that... You can get an average placement of 2nd place, and maintain that average (thereby maintaining that "skill level"), without ever getting a single kill. This poses the problem of being able to "cheat" the SBMM system. Players with no business being in the upper echelons can be matched with the highest skill players simply due to luck, which can be a massive problem for those who don't WANT to reach unearned "skill levels".

DNF Duel

DNF Duel uses a SBMM system which accounts for player preferences for their ranked mode. When queueing for Ranked, the game first considers player preference, then Skill Tier (Which are Bronze 1-4, Silver 1-4, Gold 1-4, etc), and you can choose "Any Skill", "Similar Skill", or "Same Skill" for player preference.

However, this is not entirely accurate. "Any Skill" allows the player to be matched with ANYONE of any Skill Level. This of course IS accurate, but the others are less clear.

"Similar Skill" does not mean "Same Class, different Tier". Similar Skill sounds like it would allow anyone Bronze 1-4 to match against anyone else Bronze 1-4. This is not the case. It means "Within 1 Class of each other", meaning anyone at all ranked Bronze 1-4 can match with anyone at all ranked Silver 1-4. This poses a problem. In order to reach Bronze 4, you must win 4 matches in Bronze 1, 4 in Bronze 2, and 4 in Bronze 3. This means you need to win 12 matches in order to reach Bronze 4. Once you're at Bronze 4, you need to win 3 out of 5 Advancement Matches to hit Silver 1, which takes 12 more wins to reach Silver 4.

This means, at minimum, Similar Skill allows you to face off against an opponent with 0 victories, and at maximum you can face off against an opponent with a difference of 38 victories between you. That doesn't sound like a lot, but remember: Every victory, and every loss, is experience. At 38 more wins than your opponent, you have not only the full experience of 38 victories, but all the experience of the losses, the practice, the hours put in between those victories. At 38 less wins than your opponent, you have drastically less experience than your opponent.

And finally, we have "Same Skill". "Same Skill" sounds like it would pit you against the same Class and Tier. Bronze 1 vs Bronze 1. Bronze 2 vs Bronze 2. This is not the case. "Same Skill" pits you against anyone in your Class. Bronze 1-4 vs Bronze 1-4. Silver 1-4 vs Silver 1-4.

This means, at minimum, you face an opponent with 0 victories, and at maximum, an opponent with 16 more victories than you. That is still a significant amount of experience difference. What this means is that a brand new player who is only now playing Ranked for the very first time, rather than being matched with a player in Bronze 1 like they are will instead be matched with a player anywhere between Bronze 1 to Bronze 4.

EDIT: Easier than re-writing the entire section above, I've gained a new understanding of DNF Duel's ranking system. You need CONSECUTIVE wins in order to rank up. I just matched with someone, I had 2 medals out of the needed amount to rank up, lost, and my medals disappeared. It's worth noting my opponent had 48 battles fought with 6 of those being wins, while I had 14 battles fought with 3 of those being wins.

Why is all of this important?

Skill-Based Matchmaking...does not exist. In every variant of SBMM, your actual personal skill level is not used to determine opponents.

In Destiny 2, all that matters is your wins and losses. Even if you get 0 kills, 0 assists, and a dozen deaths, as long as your matches end in victory for your team. This means players with no business being in Platinum can obtain Platinum, resulting in their solo-queue matches being nightmarishly hard for them, as they are unable to compete at all and not only lose their own fun, but ruin their teammates fun as they now have to work much harder to make up for the lack of a skilled player on their team.

In Fortnite, if you get lucky every game you can reach top rank without getting a single kill, which means you'll eventually have to rely ENTIRELY on luck and hiding just to maintain survival. You'll eventually be unable to win at all.

In DNF Duel, you can play for weeks without ever getting a win due to being consistently matched with players VASTLY more experienced than you.

And why?

Because none of these systems actually take your performance into account.

In Destiny 2, SBMM would be considering your wins, losses, average K/D, average damage dealt, average assists, and weapon accuracy in order to compile all of them into a singular Skill stat (all of which are already tracked, btw, and could easily be factored into an equation to determine Individual Skill Level). For example:

Player A has 100 wins, 50 losses, an avg. K/D of 1.3, avg. damage dealt (per game) of 900, avg. assists of 2 per game, and an avg. weapon accuracy of 68%. We can use these numbers to determine they have an Individual Skill Level of

How? Here's how:

Guardians have 200HP, and at 900 damage per game, we can determine Player A gets on avg. 4.5 kills per game. With a K/D of 1.3, they kill 1.3 times as many times as they die. So 4.5 kills per game, 3.4 avg. deaths per game.

100 wins, 50 losses, 4.5 kills per game, 3.4 deaths per game, 2 assists per game, 68% accuracy.

150 matches total, 4.5 kills per match, so that's 675 kills total. 675 kills, with an avg. of 3.4 deaths per game, that's 510 deaths. 675 kills, 510 deaths. Even with a simple equation of match total divided by win-rate times (K/D times dmg per game) divided by accuracy would give us SOME value we can use.

150 / 66.66 (1.3 x 900) / 68 150 / 66.66 x 1,170 / 68 2.25 x 1,170 / 68 2,632 / 68 38

We could even go a step further and, instead of rounding to nearest whole numbers aside from the win-rate, include up to 2 decimal places and add in a "x 100" at the end to get something like this:

150 / 66.66 (1.3 x 900) / 68 x 100 150 / 66.66 x 1,170 / 68 x 100 2.25 x 1,170 / 68 x 100 2,632.50 / 68 x 100 38.71 x 100 3,871.

This gives us an estimate of a player's Individual Skill Level, and this is just an off-the-top-of-my-head idea for how to calculate it.

With this, we could establish a SBMM system in D2 that operates on this value, with a player only being able to be put into games with those falling within a short range (say ~50-100 points) of their own rating. With this system it would guarantee NO player is ever in a match with anyone greater than 100 points of skill higher than the lowest skilled player in the lobby.

This ensures the highest-skill player in the lobby isn't SO much higher that the lowest-skill player has no chance, and everyone falling in between them is on relatively equal footing.

Unfortunately, D2, which already tracks and maintains accurate statistics on all of the things used in that formula, would rather simply track match wins and use that to determine how "skilled" a player is.

And Fortnite? What of that?

Well, Fortnite already tracks the following: Placement at end of match, accuracy, kills, assists, dmg dealt.

So what could be done is another simple equation like above. Maybe...

A player who has played 370 matches with an average placement of 8th place, average kill count of 6 per game, average assist count of 4 per game, an average of 1,375 damage per game, and an average accuracy of 72%.

So that's 370 matches, avg. placement 8th, avg. 6 kills per game, avg. assists 4 per game, avg. 1,375 dmg per game, avg. accuracy 72%.

((370 x 8 / (6 x 4) + 1,375)) x 0.72 ((370 x 8 / 24) + 1,375) x 0.72 ((2,960 / 24) + 1,375) x 0.72 (123.33 + 1,375) x 0.72 1,498.33 x 0.72 1,078.79 Round to nearest whole number 1,079 This gives us a skill rating we can use to get SOME level of accuracy (far better than just "how high did you place?") as to how skilled a player is, and if we only allow them to matchmake against players within 100 points of their skill rating, it maintains fairness.

DNF Duel is a little trickier, but still doable.

Avg. Match Time (not currently tracked but would be incredibly easy to given all matches in Ranked have a set time limit, and one can find match time by subtracting time remaining from max time), avg. HP remaining (Also easy to track), winrate (easy to track), and total matches, combining those into a skill stat.

Say someone has Avg. Match Time of 45 seconds, avg 23% HP remaining, a 37% winrate, with 78 matches total.

(45 x 0.23) x (0.37 x 78) x 10 10.35 x (0.37 x 78) x 10 10.35 x 28.86 x 10 298.70 x 10 2,987

Only allow them to match with people that are within 100-150 points of them. This prevents players with too great a difference in stats from connecting to each other. No more instances of someone with several dozen more matches than you, and therefore far more experience, facing off against you. No more instances of you getting matched against someone with several dozen less matches than you either.

Why does it matter?

Because games are supposed to be fun, and competitive modes are supposed to be fair. There should never be an instance in competitive modes on any game where one player has absolutely zero chance of beating the enemy, and in the current systems so many games use, that exact thing happens far too often.

Comp should never be an easy win, but it should never be an impossible fight either. And the fact games have moved away from accurately tracking individual skill, and have never implemented a system for truly fair SBMM is a travesty.

The systems I presented are not perfect. But they were done in several minutes by a single person with very little effort, and are still more fair evaluations of skill than what most games use today.

A team of dozens of people who are supposedly professionals in the field, paid hundreds to design matchmaking systems and given months to come up with a working one should never come up with something so much worse than what I, a single person with no noteworthy mathematic skill, paid nothing, could come up with in minutes.

Matchmaking has never been perfect, but in recent years the idea of "everyone is a winner~!" has gone from "everyone has a chance to win in comp because we pit you against closely skilled players" to "everyone has a chance to see the high-tier play because we match you with players based on unfair criteria so you can experience high-end comp even if you haven't gotten that good"

Including lower-skill players in high-skill play is not how comp is supposed to work, in any game, and it only punishes the lower-skill players.

And that honestly just makes me sad, because players like me who know exactly where we lie (just above casual, but just below the lowest skilled sweats in my case) are suffering most by being forced into boring matches with no challenge when we're matched with casuals, but forced into infuriating and unfair matches when we're matched with sweats.

I can only imagine how the casuals feel when paired against sweats.

#analysis #long post #sbmm #cbmm #hmm #matchmaking #video games #gaming #competitive games

0 notes