A comprehensive list of non-algorithmic pairings in RLCS 2021-22 Fall Split

Don’t read this post by accident.  It’s just a data dump.

This is a list of every Swiss scoregroup in RLCS 2021-22 Fall Split that doesn’t follow the “highest seed vs lowest possible seed without rematches” pairing algorithm.

I’ll grade every round as correct, WTFBBQ (the basic “rank teams, highest-lowest” would have worked, but we got something else, or we got literally impossible pairing like two teams who started 1-0 playing in R3 mid, or the pairings are otherwise utter nonsense), pairwise swap fail (e.g if 1 has played 8, swapping to 1-7 and 2-8 would work, but we got something worse), and complicated fails for things that aren’t resolved with one or more independent pairwise swaps.

Everything is hand-checked through Liquipedia.

There’s one common pairwise error that keeps popping up, with 6 teams and the #2 and #5 (referred to as 7-10 and 10-13 error a few times) have played.  Instead of leaving the 1-6 matchup alone and going to 2-4 and 3-5, they mess with the 1-6 and do 1-5, 2-6, 3-4.  The higher seed has priority for being left alone, and EU, which uses a sheet and made no mistakes all split, handled this correctly in both Regional 2 Closed R5 and Regional 3 Closed R4 Low.  EU is right.  The 7-10 and 10-13 errors below are actual mistakes.

SAM Fall Split Scoregroup Pairing Mistakes:

Invitational Qualifier


Round 2 High, FUR should be playing the worst 3-2 (TIME), but is playing Ins instead. WTFBBQ

Round 2 Low, NBL (highest 2-3) should be playing the worst 0-3 (FNG), but isn’t. WTFBBQ

Round 3 Low, Seeding is (W, L, GD, initial seed, followed by opponents each round if relevant)
13 lvi     0 2 -3 3
14 vprs 0 2 -4 12
15 neo  0 2 -4 15
16 fng   0 2 -5 16

but pairings were LVI-NEO and VPRS-FNG. WTFBBQ

Round 4 high:

Seedings were (1 and 2 are the 3-0s that have qualified, not a bug, the order is correct- LOC is 2 (4) because it started 2-0)
3 ts 2 1 4 5 vprs time fur
4 loc 2 1 1 14 lvi tts era
5 nbl 2 1 4 4 time vprs ins
6 tts 2 1 2 11 elv loc kru
7 elv 2 1 1 6 tts lvi nva
8 time 2 1 0 13 nbl ts dre

TS and time can’t play again, but 3-7 and 4-8 (and 5-6) is fine. Pairwise Swap fail.

Round 4 Low:

9 nva 1 2 -1 7 kru era elv
10 dre 1 2 -1 9 ins fng time
11 ins 1 2 -4 8 dre fur nbl
12 kru 1 2 -4 10 nva neo tts
13 lvi 1 2 -2 3 loc elv neo
14 vprs 1 2 -3 12 ts nbl fng

All base pairings worked, but we got ins-lvi somehow. WTFBBQ

Round 5:
Seedings were:
ts 2 2 1 5 vprs time fur elv
time 2 2 -2 13 nbl ts dre tts
loc 2 2 -2 14 lvi tts era nbl
lvi 2 2 -1 3 loc elv neo ins
vprs 2 2 -1 12 ts nbl fng nva
kru 2 2 -2 10 nva neo tts dre

Loc-LVI is the only problem, but the pairwise swap to time-lvi and loc-vprs works fine. Instead we got ts-lvi. Pairwise swap fail (being generous).

4 WTFBBQ, 2 pairwise swap fails, 6 total.


SAM regional 1 Closed Qualifier


Round 2 High has Kru-NEO and Round 2 Low has AC-QSHW which are both obviously wrong. WTFBBQ x2

Round 3 High

1 ts 2 0 6 1
2 ins 2 0 5 6
3 dre 2 0 4 4
4 neo 2 0 4 7

And we got ts-dre and neo-ins WTFBBQ

R3 Mid
5 nva 1 1 2 5 emt7 dre
6 loc 1 1 1 3 naiz ins
7 kru 1 1 -1 2 qshw neo
8 fng 1 1 -2 8 lgc ts
9 lgc 1 1 2 9 fng prds
10 ac 1 1 2 10 neo qshw
11 bnd 1 1 0 11 ins naiz
12 emt7 1 1 0 12 nva nice

5-12 and 8-9 are problems, but the two pairwise swaps (11 for 12 and 10 for 9) work. Instead we got nva-bnd and then loc-ac for no apparent reason. Pairwise swap fail.

Round 3 Low:
qshw 0 2 -5 15
nice 0 2 -6 13
naiz 0 2 -6 14
prds 0 2 -6 16

and we got qshw-naiz. WTFBBQ.

R4 High:
dre 2 1 1 4 nice nva ts
neo 2 1 1 7 ac kru ins
nva 2 1 5 5 emt7 dre bnd
ac 2 1 3 10 neo qshw loc
kru 2 1 0 2 qshw neo lgc
fng 2 1 -1 8 lgc ts emt7

neo is the problem, having played ac and kru. Dre-fng can be paired without a forced rematch, which is what the algorithm says to do, and we didn’t get it. Complicated fail

R4 Low
9 lgc 1 2 1 9 fng prds kru
10 loc 1 2 0 3 naiz ins ac
11 emt7 1 2 -1 12 nva nice fng
12 bnd 1 2 -3 11 ins naiz nva
13 naiz 1 2 -4 14 loc bnd qshw
14 prds 1 2 -5 16 ts lgc nice

9-14 and 10-13 are rematches, but the pairwise swap solves it. Instead we got EMT7-PRDS. Pairwise swap fail.

6 neo 2 2 0 7 ac kru ins fng
7 ac 2 2 0 10 neo qshw loc nva
8 kru 2 2 -2 2 qshw neo lgc dre
9 lgc 2 2 4 9 fng prds kru naiz
10 loc 2 2 3 3 naiz ins ac prds
11 emt7 2 2 2 12 nva nice fng bnd

7-10 and 8-9 are the problem, but instead of pairwise swapping, we got neo-lgc. Really should be a WTFBBQ, but a pairwise fail.

+4 WTFBBQ (8), +3 Pairwise (5), +1 complicated (1), 14 total.


SAM Regional 1


R4 Low

9 vprs 1 2 -1 7 drc emt7 ts
10 dre 1 2 -2 12 elv lvi nva
11 fng 1 2 -2 13 tts ftw drc
12 end 1 2 -3 8 ts loc tts
13 elv 1 2 -2 5 dre nva emt7
14 loc 1 2 -3 15 fur end ftw

10-13 is the only problem, but instead of 10-12 and 11-13, they fucked with vprs pairing. Pairwise swap fail.

+0 WTFBBQ(8), +1 pairwise swap (6), +0 complicated (1), 15 total.


SAM Regional 2 Closed Qualifier


Round 2 Low:

FDOM (0) vs END (-1) is obviously wrong. WTFBBQ. And LOLOLOL one more time at zero-game-differential forfeits.

R3 Mid:
5 fng 1 1 2 4 nice loc
6 ftw 1 1 2 7 flip vprs
7 drc 1 1 -1 3 kru elv
8 fdom 1 1 -1 9 emt7 nva
9 kru 1 1 1 14 drc ac
10 sac 1 1 0 16 vprs flip
11 end 1 1 -1 11 loc emt7
12 ball 1 1 -1 15 nva nice

The base pairings work, but we got FNG-end and ftw-ball for some reason. WTFBBQ. Even scoring the forfeit 0-3 doesn’t fix this.

R3 Low:
emt7 0 2 0 8
flip 0 2 -5 10
nice 0 2 -5 13
ac 0 2 -6 12

and we got emt-flip. WTFBBQ.

R4 High:

3 vprs 2 1 2 1 sac ftw nva
4 loc 2 1 -1 6 end fng elv
5 ftw 2 1 4 7 flip vprs ball
6 drc 2 1 2 3 kru elv sac
7 fdom 2 1 1 9 emt7 nva kru
8 end 2 1 1 11 loc emt7 fng

Base pairings worked. WTFBBQ

R4 Low:
9 fng 1 2 0 4 nice loc end
10 kru 1 2 -1 14 drc ac fdom
11 ball 1 2 -3 15 nva nice ftw
12 sac 1 2 -3 16 vprs flip drc
13 ac 1 2 -3 12 elv kru nice
14 flip 1 2 -5 10 ftw sac emt7

fng, the top seed, doesn’t play either of the teams that were 0-2.. base pairings would have worked. WTFBBQ.

+5 WTFBBQ (13), +0 pairwise fail (6), +0 complicated (1), 20 total.

Sam Regional 3 Closed


R4 High

3 endgame 2 1 5 1 bnd tn vprs
4 lev 2 1 4 5 sac endless elv
5 bnd 2 1 1 16 endgame benz ball
6 fng 2 1 0 6 ball resi endless
7 fdom 2 1 -1 10 ftw elv emt7
8 tn 2 1 -2 2 benz endgame ftw

end-tn doesn’t work, but the pairwise swap is fine, and instead we got end-fng!??! Pairwise fail.

R4 low:

9 emt7 1 2 0 9 endless sac fdom
10 ftw 1 2 -1 7 fdom ag tn
11 ball 1 2 -2 11 fng vprs bnd
12 endless 1 2 -3 8 emt7 lev fng
13 ag 1 2 -2 14 elv ftw benz
14 sac 1 2 -4 12 lev emt7 resi

9-14 and 10-13 are problems, but the pairwise swap is fine.  Instead we got emt7-ftw which is stunningly wrong.  Pairwise fail.

Round 5:

6 fng 2 2 -2 6 ball resi endless endgame
7 bnd 2 2 -2 16 endgame benz ball lev
8 tn 2 2 -3 2 benz endgame ftw fdom
9 ftw 2 2 1 7 fdom ag tn emt7
10 endless 2 2 0 8 emt7 lev fng ag
11 ball 2 2 -1 11 fng vprs bnd sac

This is a mess, going to the 8th most preferable pairing before finding no rematches, and.. they got it right.  After screwing up both simple R4s.

Final SAM mispaired scoregroup count: +5 WTFBBQ (13), +2 pairwise fail (8), +0 complicated (1), 22 total.

OCE Fall Scoregroup pairing mistakes


Regional 1 Invitational Qualifier


Round 2 Low:

The only 2-3, JOE, is paired against the *highest* seeded 0-3 instead of the lowest.  WTFBBQ.

R3 Mid:

5 kid 1 1 0 6 rrr riot
6 grog 1 1 0 9 joe gz
7 bndt 1 1 0 10 vc rng
8 dire 1 1 -1 3 eros wc
9 eros 1 1 0 14 dire pg
10 vc 1 1 -1 7 bndt gse
11 rrr 1 1 -2 11 kid joe
12 tri 1 1 -2 15 rng rru


8-9 and 7-10 are problems, but a pairwise swap fixes it, and instead every single pairing got fucked up.  Pairwise fail is too nice, but ok.

R4 Low:

9 eros 1 2 -2 14 dire pg kid
10 vc 1 2 -3 7 bndt gse dire
11 tri 1 2 -3 15 rng rru grog
12 rrr 1 2 -4 11 kid joe bndt
13 joe 1 2 1 8 grog rrr pg
14 gse 1 2 -3 13 wc vc rru

Base pairings work.  WTFBBQ

2 WTFBBQ, 1 pairwise, 3 total.

OCE Regional 1 Closed Qualifier


Round 2 Low:

The best 2-3 (1620) is paired against the best 0-3 (tri), not the worst (bgea).  WTFBBQ.

Round 4 High:

3 gse 2 1 1 1 tisi joe hms
4 grog 2 1 0 3 smsh bott wc
5 eros 2 1 4 5 hms rust bott
6 waz 2 1 4 11 tri hms pg
7 T1620 2 1 2 10 rru tri joe
8 rru 2 1 1 7 T1620 wc tisi


Base pairings work.  WTFBBQ.

Round 5:

6 T1620 2 2 1 10 rru tri joe gse
7 waz 2 2 1 11 tri hms pg eros
8 grog 2 2 -1 3 smsh bott wc rru
9 tisi 2 2 2 16 gse bgea rru bott
10 tri 2 2 -1 6 waz T1620 bgea joe
11 rust 2 2 -2 13 joe eros smsh pg

7-10 is the problem, but they swapped with 6-11 instead of leaving 6-11 and swapping with 7-10.  Pairwise fail.

+2 WTFBBQ(4), +1 pairwise(2), 6 total.

OCE Regional 1


Round 5:

6 riot 2 2 3 2 grog rust eros bndt
7 grog 2 2 0 15 riot gz blss rrr
8 wc 2 2 -1 9 kid bndt rng dire
9 fbrd 2 2 2 13 bndt kid T1620 hms
10 blss 2 2 1 6 eros T1620 grog gse
11 kid 2 2 -2 8 wc fbrd dire eros

Exact same mistake as above. Pairwise fail.

+0 WTFBBQ(4), +1 pairwise(3), 7 total.

OCE Regional 2 Closed

Round 3 Mid:

5 blss 1 1 0 2 tri fbrd
6 pg 1 1 0 14 gse t1620
7 tisi 1 1 -1 11 rust eros
8 ctrl 1 1 -1 13 hms waz
9 gse 1 1 0 3 pg lits
10 grog 1 1 0 12 eros rust
11 bott 1 1 0 16 fbrd tri
12 hms 1 1 -1 4 ctrl bstn


Base pairings work.  WTFBBQ

Round 4 Low:

9 blss 1 2 -1 2 tri fbrd bott
10 gse 1 2 -1 3 pg lits tisi
11 pg 1 2 -1 14 gse t1620 hms
12 ctrl 1 2 -2 13 hms waz grog
13 tri 1 2 -2 15 blss bott bstn
14 lits 1 2 -4 10 t1620 gse rust


Base pairings work.  WTFBBQ

+2 WTFBBQ(6), +0 pairwise(3), 9 total.

OCE Regional 3 Closed


R3 Mid:

5 gse 1 1 2 7 lits t1620
6 wc 1 1 0 1 tisi grog
7 deft 1 1 0 6 zbb eros
8 gue 1 1 0 15 bott crwn
9 tri 1 1 1 12 grog tisi
10 bott 1 1 0 2 gue pg
11 lits 1 1 -1 10 gse ctrl
12 corn 1 1 -2 13 eros zbb

Base pairings work.  WTFBBQ

R3 Low

13 zbb 0 2 -3 11 deft corn
14 ctrl 0 2 -5 8 t1620 lits
15 pg 0 2 -5 14 crwn bott
16 tisi 0 2 -5 16 wc tri

Base Pairings work.  WTFBBQ

R4 High:

3 crwn 2 1 4 3 pg gue t1620
4 grog 2 1 1 5 tri wc eros
5 wc 2 1 3 1 tisi grog bott
6 tri 2 1 2 12 grog tisi deft
7 gue 2 1 2 15 bott crwn lits
8 corn 2 1 0 13 eros zbb gse

Base pairings work.  WTFBBQ

R4 Low:

9 gse 1 2 0 7 lits t1620 corn
10 deft 1 2 -1 6 zbb eros tri
11 bott 1 2 -3 2 gue pg wc
12 lits 1 2 -3 10 gse ctrl gue
13 zbb 1 2 0 11 deft corn ctrl
14 tisi 1 2 -3 16 wc tri pg

10-13 is the problem, pairwise swapped with 9-14 instead of 11-12.  Pairwise fail.

+3 WTFBBQ(9), +1 pairwise(4), 13 total.

OCE Regional 3:


R3 Low:

13 tri 0 2 -3 14 riot wc
14 gue 0 2 -5 15 gz t1620
15 crwn 0 2 -6 11 kid grog
16 tisi 0 2 -6 16 rng eros

Base pairings work. WTFBBQ

OCE Final count: +1 WTFBBQ(10), +0 pairwise(4), 14 total.

SSA Scoregroup Pairing Mistakes

Regional 1


Round 2 Low:

The 0 GD FFs are paired against the best teams instead of the worst.  WTFBBQ.


SSA Regional 2 Closed


Round 4 Low:

9 bw 1 2 -3 6 crim est slim
10 aces 1 2 -3 12 llg biju info
11 crim 1 2 -5 11 bw dd fsn
12 win 1 2 -5 16 mist gst auf
13 biju 1 2 -2 13 info aces est
14 slap 1 2 -6 14 auf fsn gst

10-13 pairwise exchanging with 9-14 instead of 11-12.  Pairwise fail.

SSA Regional 2


R4 High

3 atk 2 1 4 2 mils ag bvd
4 org 2 1 1 3 dwym atmc op
5 atmc 2 1 4 7 mist org oor
6 llg 2 1 4 12 oor bvd dd
7 auf 2 1 3 9 ag mils exo
8 ag 2 1 0 8 auf atk mist

3-8 is the problem, but instead of pairwise swapping with 4-7, the pairing is 3-4?!?!?!

R4 Low

9 mist 1 2 0 10 atmc dwym ag
10 dd 1 2 -3 11 exo fsn llg
11 oor 1 2 -4 5 llg info atmc
12 exo 1 2 -5 6 dd op auf
13 fsn 1 2 -1 16 op dd dwym
14 info 1 2 -2 13 bvd oor mils

Another 10-13 pairwise mistake.



SSA Regional 3 Closed


Round 5

6 fsn 2 2 -1 5 slim lft gst blc
7 mils 2 2 -1 6 blc chk ddogs oor
8 roc 2 2 -1 10 gst mlg slim mist
9 ddogs 2 2 3 4 alfa mist mils lft
10 gst 2 2 0 7 roc oor fsn alfa
11 slim 2 2 0 12 fsn llg roc est


This is kind of a mess, but 6-9, 7-8, 10-11 is correct, and instead they went with 6-7, 8-9, 10-11!??!? which is not.  Complicated fail.


SSA total (1 event pending): 1 WTFBBQ, 3 pairwise, 1 complicated.

APAC N Swiss Scoregroup pairing mistakes

Regional 2


Round 5:

6 xfs 2 2 1 8 nor gds nov dtn
7 alh 2 2 0 10 n55 timi gra hill
8 cv 2 2 -1 4 chi gra gds n55
9 gra 2 2 -1 6 blt cv alh nor
10 chi 2 2 -2 13 cv blt dtn nov
11 timi 2 2 -2 16 ve alh blt wiz

This is a mess, but a pairing is possible with 6-11, so it must be done.  Instead they screwed up that match and left 7-10 alone.  Complicated fail

APAC S Swiss Scoregroup pairing mistakes

Regional 2 Closed


Round 4 Low:

9 exd 1 2 1 14 flu dd shg
10 vite 1 2 -2 12 shg cel flu
11 wide 1 2 -3 10 pow che ete
12 pow 1 2 -4 7 wide hira acrm
13 whe 1 2 -4 9 ete acrm che
14 sjk 1 2 -4 16 cel shg axi

11-12 is the problem, and instead of swapping to 10-12 11-13, they swapped to 10-11 and 12-13.  Pairwise fail.


Round 5

6 acrm 2 2 1 13 pc whe pow ete
7 flu 2 2 -2 3 exd axi vite dd
8 shg 2 2 -3 5 vite sjk exd hira
9 exd 2 2 4 14 flu dd shg sjk
10 pow 2 2 -1 7 wide hira acrm whe
11 wide 2 2 -3 10 pow che ete vite

another 8-9 issue screwing with 6-11 for no reason.  Complicated fail.

Regional 3 Closed:


Round 3 Mid

Two teams from the 1-0 bracket are playing each other (and two from the 0-1 playing each other as well obviously).  I checked this one on Smash too.  WTFBBQ

Round 4 High

3 dd 2 1 2 2 wc woa bro
4 flu 2 1 2 4 soy fre pum
5 znt 2 1 5 1 woa wc shg
6 pc 2 1 4 3 lads pum woa
7 ete 2 1 3 7 wash bro wide
8 fre 2 1 2 12 shg flu exd

The default pairings work.  WTFBBQ

Round 4 Low

9 exd 1 2 -1 6 pum lads fre
10 wide 1 2 -1 8 bro wash ete
11 woa 1 2 -3 16 znt dd pc
12 shg 1 2 -4 5 fre soy znt
13 soy 1 2 -2 13 flu shg wc
14 lads 1 2 -2 14 pc exd wash

9-14 is the problem, and the simple swap with 10-13 works.  Instead they did 9-12, 10-11 , 13-14.  Pairwise fail.


Round 5:

6 ete 2 2 2 7 wash bro wide dd
7 fre 2 2 0 12 shg flu exd znt
8 flu 2 2 -1 4 soy fre pum pc
9 soy 2 2 0 13 flu shg wc lads
10 woa 2 2 0 16 znt dd pc wide
11 shg 2 2 -3 5 fre soy znt exd

8-9 is the problem, but the simple swap with 7-10 works.  Instead we got 6-8, 7-9, 10-11.  Pairwise fail.

APAC Total: 2 WTFBBQ, 3 Pairwise fails, and 1 complicated.

NA Swiss Scoring Pairing Mistakes

Regional 3 Closed


Round 4

3 pk 2 1 4 11 rbg yo vib
4 gg 2 1 1 1 exe sq oxg
5 eu 2 1 4 14 sq exe rge
6 sq 2 1 1 3 eu gg tor
7 xlr8 2 1 1 12 vib leh sr
8 yo 2 1 1 15 tor pk clt

This is a mess.  3-8 is illegal, but 3-7 can lead to a legal pairing, so it must be done (3-7, 4-5, 6-8). Instead we got 3-6, 4-8, 5-7.  Complicated fail.

EU made no mistakes.  Liquipedia looks sketchy enough on MENA that I don’t want to try to figure out who was what seed when.

Final tally across all regions

WTFBBQ Pairwise Complicated Total
SAM 13 8 1 22
OCE 10 4 0 14
SSA 1 3 1 5
APAC 2 3 1 6
NA 0 0 1 1
EU 0 0 0 0
Total 26 18 4 48

Missing the forest for.. the forest

The paper  A Random Forest approach to identify metrics that best predict match outcome and player ranking in the esport Rocket League got published yesterday (9/29/2021), and for a Cliff’s Notes version, it did two things:  1) Looked at 1-game statistics to predict that game’s winner and/or goal differential, and 2) Looked at 1-game statistics across several rank (MMR/ELO) stratifications to attempt to classify players into the correct rank based on those stats.  The overarching theme of the paper was to identify specific areas that players could focus their training on to improve results.

For part 1, that largely involves finding “winner things” and “loser things” and the implicit assumption that choosing to do more winner things and fewer loser things will increase performance.  That runs into the giant “correlation isn’t causation” issue.  While the specific Rocket League details aren’t important, this kind of analysis will identify second-half QB kneeldowns as a huge winner move and having an empty net with a minute left in an NHL game as a huge loser move.  Treating these as strategic directives- having your QB kneel more or refusing to pull your goalie ever- would be actively terrible and harm your chances of winning.

Those examples are so obviously ridiculous that nobody would ever take them seriously, but when the metrics don’t capture losing endgames as precisely, they can be even *more* dangerous, telling a story that’s incorrect for the same fundamental reason, but one that’s plausible enough to be believed.  A common example is outrushing your opponent in the NFL being correlated to winning.  We’ve seen Derrick Henry or Marshawn Lynch completely dump truck opposing defenses, and when somebody talks about outrushing leading to wins, it’s easy to think of instances like that and agree.  In reality, leading teams run more and trailing teams run less, and the “signal” is much, much more from capturing leading/trailing behavior than from Marshawn going full beast mode sometimes.

If you don’t apply subject-matter knowledge to your data exploration, you’ll effectively ask bad questions that get answered by “what a losing game looks like” and not “what (actionable) choices led to losing”.  That’s all well-known, if worth restating occasionally.

The more interesting part begins with the second objective.  While the particular skills don’t matter, trust me that the difference in car control between top players and Diamond-ranked players is on the order of watching Simone Biles do a floor routine and watching me trip over my cat.  Both involve tumbling, and that’s about where the similarity ends.

The paper identifies various mechanics and identifies rank pretty well based on those.  What’s interesting is that while they can use those mechanics to tell a Diamond from a Bronze, when they tried to use those mechanics to predict the outcome of a game, they all graded out as basically worthless.  While some may have suffered from adverse selection (something you do less when you’re winning), they had a pretty good selection of mechanics and they ALL sucked at predicting the winner.  And, yet, beyond absolutely any doubt, the higher rank stratifications are much better at them than the lower-rank ones.  WTF? How can that be?

The answer is in a sample constructed in a particularly pathological way, and it’s one that will be common among esports data sets for the foreseeable future.  All of the matches are contested between players of approximately equal overall skill.  The sample contains no games of Diamonds stomping Bronzes or getting crushed by Grand Champs.

The players in each match have different abilities at each of the mechanics, but the overall package always grades out similarly given that they have close enough MMR to get paired up.  So if Player A is significantly stronger than player B at mechanic A to the point you’d expect it to show up, ceteris paribus, as a large winrate effect, A almost tautologically has to be worse at the other aspects, otherwise A would be significantly higher-rated than B and the pairing algorithm would have excluded that match from the sample.  So the analysis comes to the conclusion that being better at mechanic A doesn’t predict winning a game.  If the sample contained comparable numbers of cross-rank matches, all of the important mechanics would obviously be huge predictors of game winner/loser.

The sample being pathologically constructed led to the profoundly incorrect conclusion

Taken together, higher rank players show better control over the movement of their car and are able to play a greater proportion of their matches at high speed.  However, within rank-matched matches, this does not predict match outcome.Therefore, our findings suggest that while focussing on game speed and car movement may not provide immediate benefit to the outcome within matches, these PIs are important to develop as they may facilitate one’s improvement in overall expertise over time.

even though adding or subtracting a particular ability from a player would matter *immediately*.  The idea that you can work on mechanics to improve overall expertise (AKA achieving a significantly higher MMR) WITHOUT IT MANIFESTING IN MATCH RESULTS, WHICH IS WHERE MMR COMES FROM, is.. interesting.  It’s trying to take two obviously true statements (Higher-ranked players play faster and with more control- quantified in the paper. Playing faster and with more control makes you better- self-evident to anybody who knows RL at all) and shoehorn a finding between them that obviously doesn’t comport.

This kind of mistake will occur over and over and over when data sets comprised of narrow-band matchmaking are analysed that way.

(It’s basically the same mistake as thinking that velocity doesn’t matter for mediocre MLB pitchers- it doesn’t correlate to a lower ERA among that group, but any individuals gaining velocity will improve ERA on average)


RLCS X viewership is being manipulated

TL;DR EU regionals weekend 1 viewership was still fine, but legit viewership never cracked 80k even though the reported peak was over 150k.

Psyonix was cooperative enough to leave the Sunday stream shenanigan-free, so we have a natural comparison between the two days.  Both the Saturday and Sunday streams had the same stakes- half of the EU teams played each day trying to qualify for next weekend- so one would expect viewership each day to be pretty similar, and in reality, this was true. Twitch displays total viewers for everyone to see, and the number of people in the chatroom is available through the API, and I track it. (NOT the number of people actually chatting- you’re in the chatroom if you’re logged in, viewing normally, and didn’t go out of your way to close chat.  The fullscreen button doesn’t do it, and it appears that closing chat while viewing in squad mode doesn’t remove you from the chatroom either).  Only looking at people in the chatroom on Saturday and Sunday gives the following:


That’s really similar, as expected.  Looking at people not in the chatroom- the difference between the two numbers- tells an entirely different story.


LOL.  Alrighty then.  Maybe there’s a slight difference?

Large streams average around 70% of total viewers in chat.  Rocket League, because of drops, averages a bit higher than that.  More people make Twitch accounts and watch under those accounts to get rewards.  Sunday’s stream is totally in line with previous RLCS events and with the big events in the past offseason.  Saturday’s stream is.. not.  On top of the giant difference in magnitude, the Sunday number/percentage is pretty stable, while the Saturday number bounces all over the place.  Actual people come and go in approximately equal ratios whether they’re logged in or not.  At the very end of Saturday’s stream, it was on the Twitch frontpage, which boosts the not-in-chat count, but that doesn’t explain the rest of the time.

The answer is that the Saturday stream was embedded somewhere outside of Twitch.  Twitch allows a muted autoplay stream video in a random webpage to count as a viewer even if the user never interacts with the stream (a prominent media source said otherwise last year.  He was almost certainly wrong then, and I’ve personally tested in the past 2 months that he’s wrong now), and, modern society being what it is, services and ad networks exist to place muted streams where ~nobody pays any attention to them to boost apparent viewcount, and publishers pay 5 figures to appear to be more popular than they are. Psyonix/RLCS was listed as a client of one of these services before and has a history of pulling bullshit and buying fake viewers. There’s a nice article on kotaku detailing this nonsense across more esports.

If the stream were embedded somewhere it belonged, instead of as an advertisement to inflate viewcount, it’s hard to believe it also wouldn’t have been active on Sunday, so it’s very likely they’re just pulling bullshit and buying fake viewers again.  If somebody at Psyonix comments and explains otherwise, I’ll update the post, but don’t hold your breath.  Since a picture speaks a thousand words:



Are Rocket League viewership numbers fraudulently high?

Maybe.  Quite possibly.  Something strange is going on. For the TL;DR crowd Season 6 EU playoff viewership (Sunday October 14, 2018) was abominably low, off by 10,000-20,000 viewers or up to a third of the regular audience, far worse than any other broadcast this season. Season 6 NA regional playoffs, held the day before (Saturday, October … Continue reading “Are Rocket League viewership numbers fraudulently high?”

Maybe.  Quite possibly.  Something strange is going on.

For the TL;DR crowd

  1. Season 6 EU playoff viewership (Sunday October 14, 2018) was abominably low, off by 10,000-20,000 viewers or up to a third of the regular audience, far worse than any other broadcast this season.
  2. Season 6 NA regional playoffs, held the day before (Saturday, October 13, 2018) were exactly in line with seasonal numbers.
  3. Regional playoff viewership was always fine-to-good for the other 7 Season 3-Season 6 regional playoff broadcasts
  4. There was an unnatural 15,000ish viewer bump late in the broadcast that can only realistically be a big host or major viewbot fraud
  5. Psyonix won’t comment at all and I can’t find other evidence of a large host
  6. Viewership of major events was trending way down coming into season 6



The Rocket League Championship Series (RLCS) is a biannual competition featuring 8 teams from North America (NA), 8 teams from Europe (EU), and 8 teams from Oceania (OCE) each playing an online single-round-robin season over 5 weeks followed by an online regional playoff.  The top 4 teams from NA and EU and the top 2 teams from OCE then meet in person to crown the world champion for that season.  Psyonix, the maker of Rocket League, runs the NA and EU competitions and broadcasts every game on its Twitch stream.  A different entity is responsible for OCE and broadcasts those games on its own Twitch stream instead. We won’t worry about the OCE broadcasts.


Viewer counts increased significantly between seasons 3 and 4, slightly between 4 and 5, and were down about 20% in the current season 6, as the following graphs from esc.watch, an esports viewership tracker, show.  I have personally spot-checked esc.watch’s Rocket League charts over the past year and have always found them accurate.


The focus of this post will be the large decline in the season 6 EU playoff.  Note that season 3 NA has a slight decline in playoff viewership while every other playoff broadcast has been up or at least flat.  File that away for later. A couple of other easily  explainable anomalies need to be addressed first.  In season 3 (green), there was a week off between week 3 (April 2, 2017) and week 4 (April 16, 2017).  That resulted in lower viewership in week 4 in both NA and EU that rebounded to “normal” in the following 2 weeks of consecutive play.  Season 5 week 5 in EU had a large technical problem.  A power outage near the Psyonix studio cut the broadcast short after 2 games and the remaining 4 games were rescheduled for 12 PM Eastern on a Thursday in place of the regular 12 PM Eastern on Sunday.  That timeslot is obviously terrible in comparison for NA viewers and the low viewer count is quite understandable.  Season 4 week 5 in EU had a different technical problem.  The broadcast was fine, but twitch.tv did not send out notifications that the broadcast was starting and reportedly didn’t even show the Rocket League channel as being online.  Given the nature of viewership, which will be discussed next, it’s also quite understandable for that number to be in the toilet.


A typical RLCS broadcast accumulates most of its viewers over its first three matches like this (another reason that the EU S5 Week 5 broadcast being cut off after 2 games killed the viewership).


The viewership accumulates this way because it’s comprised of hardcore viewers, those who are ready for the start of the broadcast every week, or at least the notification for it, and those who happen upon it in progress.  People who follow the RL Twitch stream and connect to Twitch will see that it’s online, and people who play Rocket League while a broadcast is in progress are greeted with a giant flashing “ESPORTS LIVE NOW” button on the main menu that they can click to watch.

EU Regionals Anomalies

The graphs for the 12 broadcasts this season follow.  Don’t pay attention to the actual numbers yet, just the shapes.  And in particular the last graph on the bottom right, the EU playoffs.



There are two obvious things of note.  First, the glitch on the left is the very end of EU Week 5 (the bottom left image above) and not actually part of the playoff broadcast.  That was confirmed directly by esc.watch.  Second.. well…


That’s a near-instantaneous (under 5 minute) jump of almost 15,000 viewers, and there’s no precedent of such a thing in any other broadcast this season.  It’s clearly not part of the natural accumulation process.

EU Regionals Low Viewership

The other interesting thing about this broadcast is that it was the least viewed broadcast of the season by a country mile.

RL S6 Viewership by match slot

Or in slightly less cluttered form, for each region, the averages of the regular season (weeks 1-5), the playoffs, and the difference between the playoff viewership and the regular season viewership.

RL S6 averages

Yikes? To summarize so far, we have a mystery with the following facts

  1. Season 6 EU playoff viewership (Sunday October 14, 2018) was abominably low, off by 10-20 thousand viewers or up to a third of the regular audience, far worse than any other broadcast this season.
  2. Season 6 NA regional playoffs, held the day before (Saturday, October 13, 2018) were exactly in line with seasonal numbers.
  3. Regional playoff viewership was always fine-to-good for the other 7 Season 3-Season 6 regional playoff broadcasts
  4. There was an unnatural 15,000ish viewer bump late in the broadcast.

Potential Causes for Low Viewership

In investigating the low viewership, I was unable to uncover any evidence of technical difficulties.  I was watching the broadcast and using Twitch and both worked fine.  I found no reports of issues, Twitch-wise or internet-wise, in the relevant reddit thread while gripes about the Twitch issues in season 4 were plentiful.  The graph of viewer count was accurate.  I personally observed the extremely low viewer counts throughout as well as the high viewer count at the end.  Unfortunately I wasn’t paying attention to the count as the 15,000 viewers showed up late.  Technical issues don’t seem to be the explanation here.

The second possibility is that something external was drawing would-be viewers away.  There wasn’t any news in the world that day that would have diverted a large number of viewers.  Call of Duty: Black Ops 4 was released worldwide on October 12, but if that were the cause, it should have sunk the NA playoff viewership on October 13 as well.  Similarly, Rocket League itself was offering a double-XP weekend, giving out more in-game rewards for playing, but that was also active on Saturday during the NA playoffs.  There were other non-Rocket League esports events, as there are many weekends, but there didn’t appear to be anything that could uniquely sink the entire Sunday broadcast or the first 90% of it.  This doesn’t appear to be it either.

The third possibility is that the broadcast itself was thought to be uniquely unappealing to tune into, presumably because Dignitas, the two-time defending world championship roster that just went 7-0 in round robin play, was thought to be a lock to win.  This explanation falls short for several reasons.  First, there’s no evidence that Dignitas is bad for viewership at all.  None of their matches have seen strange declines and the last match of the regular season was an almost meaningless Dignitas match and it had the highest viewership of the week. The idea that people would watch to see if Dignitas could go 7-0, then disappear in droves as Dig tried to become the first NA/EU team to go 7-0 and win the regional playoffs… well, that’s just strange.

The format of the regional championship also undercuts that explanation.  It’s a 6-team single elimination where the top 4 qualify for the world championship.  The top 2 seeds get first-round byes and automatically qualify which means the first two matches- #3 vs #6 and #4 vs #5- are critical.  The winner goes to the world championships and the loser is done for the season.  Even if people didn’t want to watch the rest of the playoffs, those matchups should have been compelling.  The second match of the day involved the most anticipated rookie in RLCS history by far, ScrubKilla, and his wildly inconsistent Renault Vitality team playing for a LAN spot.  This should have been a must-watch series, and by one metric it was.  Not counting the first match of the day, which always accumulates lots of viewers, Vitality-PSG had a higher accumulation of viewers (+17k) than any other match had all season, almost 50% clear of second place.  People talked about ScrubKilla and Vitality all season.  Viewers appeared to tune in in numbers specifically to not miss this match.  The final number shouldn’t have been hot garbage, and yet it still was.  I’m at a complete loss for a legitimate explanation for the overall low viewership.

Potential Causes of the 15,000 Viewer Bump

Switching focus to the ~15,000 viewer bump, there’s one legitimate explanation- a large stream from another game/scene hosted them..  That would bump the viewer count up quickly, but I could find no evidence of that actually happening.  I couldn’t find a mention on twitter. I watched the relevant portion of the broadcast replay, focusing on the Twitch chat (bless my soul), and there weren’t any mentions of a host, nor any newbie-like comments, which seems a bit unlikely.  In addition, the attrition rate would have had to have been extremely low because viewership also went up a tad after the spike.  Rocket League being the autoplay stream on the Twitch homepage was suggested as a possibility, but after observing those streams for a few days, none I saw were accumulating viewers at even a meaningful fraction of the necessary rate.

If the 15,000 viewer bump isn’t natural viewer accumulation, and isn’t a legitimate Twitch host, that leaves fraud.  Psyonix randomly gives away in-game items to players who watch the stream and have their Twitch accounts linked.  This is absolutely a boon to viewership, and can be misleading in a way because players can and do load the stream, mute it, and pay no attention to it just for the chance of getting a drop. In this way, the percentage of the viewer count who are watching the stream is lower than it otherwise would be, but this is all out in the open, and it does also further a legitimate purpose of trying to entice players to become esports viewers.  Where there are giveaways, there are people trying to exploit giveaways, but the likelihood of large-scale fraud here seems very small to me.  The expected value in terms of reselling items of having a linked account watch a stream for 6 hours is about 25-50 cents (USD), and obtaining that value across thousands of accounts involves selling numerous small-ticket (.50-$3) items that nobody wants multiples of.  While I have no doubt that some people sign up a few accounts and have Twitch open in multiple browsers, any person or group capable of controlling enough computers/IPs to generate 15,000 concurrent fraudulent Twitch views should be able to make a hell of a lot more money doing almost anything else with them…

Such as selling their services to a company/streamer that wants to inflate its viewer count to bring in more advertising/sponsorship revenue or to just appear to be more popular than it actually is.  This is called viewbotting, and it’s not a rare occurrence.  If Psyonix had an arrangement for around 15,000 viewbots and forgot to turn them on before the last match, or never turned them on and got a 15,000 viewer host near the end, this would explain everything.  Here’s the graph from earlier with 15,000 viewers added through the whole broadcast instead of the very end.

RLS6 +15k

That would be much more consistent with the regular season.  I obviously have no affirmative proof that Psyonix is committing viewbot fraud, but if they were committing viewbot fraud, this is *exactly* what it would look like if they screwed up for a week, and I’m at a complete loss for legitimate explanations for the low viewership that wouldn’t also have sunk the NA playoffs.

In summary, the evidence here is consistent with several main hypotheses of varying plausibility.

  1. There is a legitimate, as yet undiscovered reason for the overall low viewership and they got a big host of around 15,000 near the end.
  2. Psyonix is viewbotting around 15,000 fakes and forgot to turn them on until the last match of the day.
  3. Psyonix is viewbotting around 15,000 fakes, forgot to turn them on at all that day, and coincidentally got a big host of around 15,000 near the end.

Psyonix’s Refusal to Comment

I contacted Psyonix a fourth time with the article up to this point and another request for comment.  That’s private messages to Murty Shah and Cory Lanier, the two main public faces of their esports program, an email to Psyonix’s general PR contact address, and an email to Psyonix’s esports contact address, and I’ve received no replies.  In addition, they didn’t comment in reddit threads where the low viewership and the big bump were discussed.  The lack of response has to be considered deliberate at this point.  Let’s think about what that means.

Hypothetically, if Psyonix knows the viewers are from a host, would they be willing to tell the world about it?  I would think so. I’m not a social media guru, but if somebody from a different game added 15k viewers to my 48k viewer broadcast, I would want to say thank you, and I’d want to say it publicly to attempt to engage their followers a little more.  They shouldn’t do that for small hosts because that would bring a barrage of useless attention-seekers, and I’m not sure exactly where the acknowledgment line should be drawn, but a 15,000 viewer host seems safely big enough at this point.  Furthermore, there’s no obvious benefit to attempting to conceal that a host happened.  It would already be known to the people watching the other stream, it’s obvious to everybody who looked at the viewer graph that something happened, and it’s inconceivable to me that a channel from a different scene wanting to expose its viewers to Rocket League could be construed negatively.  Now watch it turn out that Psyonix is slowrolling and waiting for the article to go live to say who the host was, but I have to work with what I have so far. If they do decide to announce who the host was and it’s legit, then we’re just back to the mystery of the low viewership in general.

Hypothetically, if Psyonix knows the viewers are fake, would they be willing to deny that they were hosted or willing to lie about who hosted them?  The latter would be a huge mistake because it would be discovered in no time flat, and the former is straight-up admitting that the numbers for that broadcast are bogus, and by inference, that the numbers for every other broadcast during the season are almost certainly bogus. They have to stay silent in this case and hope it blows over.  Twitch itself should have an active interest in investigating this broadcast if there wasn’t a big host.

Reasons for Viewbotting

The cleanest explanation for not commenting is fraud, which invites the question of whether or not viewbotting would make any sense.  As a moral matter, I have absolutely no idea if the relevant people working there are the type who could do such a thing.  As a business matter, it’s plausible.  Before this season, viewership trends started out ok and then took a straight-line path to dumpster fire.  All tournament pairs below are roughly equivalent across seasons.  Broadcasts were about the same length and Fan Rewards were active for all referenced broadcasts.  This is what Psyonix was looking at for average viewers as the numbers rolled in this year

  1. April 22.  RLCS Season 5: 68.2k, +15% over season 4
  2. May 6.  Promotion/Relegation Tournament.  44.5k, +9% over season 4
  3. June 10.  World Championships.  101.8k, -5% from season 4 and -14% from season 3
  4. August 11.  UORL 2 final qualifiers. 15.1k.  esc.watch doesn’t have comps for last year, and timeslots are wonky, but this is not a good number at all for top players playing with rewards active.  The peak viewership was only 29.5k, presumably on one of the four good weekend timeslots.
  5. August 26.  Universal Open 2.  29.1k, -35% from last year
  6. September 2.  Season 6 play-ins 42.0k -46% from last season.

If I were in charge of the esport or somebody whose job/livelihood depended on the continued success of the esport, I would have been very nervous going into season 6. It’s not hard to see the appeal of buying some “insurance viewers” in that climate. As it was, season 6 viewership was officially down 20% across the board, and the real number is much worse if this season was being viewbotted.  To repeat one last time, I have no definitive proof that viewbotting took place.  It is only a hypothesis that explains viewer stats and Psyonix’s behavior.


I hope, and all fans of Rocket League probably join me in hoping that there was a legitimate reason for the low viewership and that Psyonix is simply being obnoxious refusing to acknowledge a large host.  Anything else would be a huge story and a crippling blow to the future of the esport.  If Twitch or Psyonix comment, the article will be updated to reflect that. Many thanks to esc.watch for providing the charts for this article.