A comprehensive list of non-algorithmic pairings in RLCS 2021-22 Fall Split

Don’t read this post by accident. It’s just a data dump.

This is a list of every Swiss scoregroup in RLCS 2021-22 Fall Split that doesn’t follow the “highest seed vs lowest possible seed without rematches” pairing algorithm.

I’ll grade every round as correct, WTFBBQ (the basic “rank teams, highest-lowest” would have worked, but we got something else, or we got literally impossible pairing like two teams who started 1-0 playing in R3 mid, or the pairings are otherwise utter nonsense), pairwise swap fail (e.g if 1 has played 8, swapping to 1-7 and 2-8 would work, but we got something worse), and complicated fails for things that aren’t resolved with one or more independent pairwise swaps.

Everything is hand-checked through Liquipedia.

There’s one common pairwise error that keeps popping up, with 6 teams and the #2 and #5 (referred to as 7-10 and 10-13 error a few times) have played. Instead of leaving the 1-6 matchup alone and going to 2-4 and 3-5, they mess with the 1-6 and do 1-5, 2-6, 3-4. The higher seed has priority for being left alone, and EU, which uses a sheet and made no mistakes all split, handled this correctly in both Regional 2 Closed R5 and Regional 3 Closed R4 Low. EU is right. The 7-10 and 10-13 errors below are actual mistakes.

SAM Fall Split Scoregroup Pairing Mistakes:

Invitational Qualifier

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/South_America/1/Invitational_Qualifier

Round 2 High, FUR should be playing the worst 3-2 (TIME), but is playing Ins instead. WTFBBQ

Round 2 Low, NBL (highest 2-3) should be playing the worst 0-3 (FNG), but isn’t. WTFBBQ

Round 3 Low, Seeding is (W, L, GD, initial seed, followed by opponents each round if relevant)
13 lvi 0 2 -3 3
14 vprs 0 2 -4 12
15 neo 0 2 -4 15
16 fng 0 2 -5 16

but pairings were LVI-NEO and VPRS-FNG. WTFBBQ

Round 4 high:

Seedings were (1 and 2 are the 3-0s that have qualified, not a bug, the order is correct- LOC is 2 (4) because it started 2-0)
3 ts 2 1 4 5 vprs time fur
4 loc 2 1 1 14 lvi tts era
5 nbl 2 1 4 4 time vprs ins
6 tts 2 1 2 11 elv loc kru
7 elv 2 1 1 6 tts lvi nva
8 time 2 1 0 13 nbl ts dre

TS and time can’t play again, but 3-7 and 4-8 (and 5-6) is fine. Pairwise Swap fail.

Round 4 Low:

9 nva 1 2 -1 7 kru era elv
10 dre 1 2 -1 9 ins fng time
11 ins 1 2 -4 8 dre fur nbl
12 kru 1 2 -4 10 nva neo tts
13 lvi 1 2 -2 3 loc elv neo
14 vprs 1 2 -3 12 ts nbl fng

All base pairings worked, but we got ins-lvi somehow. WTFBBQ

Round 5:
Seedings were:
ts 2 2 1 5 vprs time fur elv
time 2 2 -2 13 nbl ts dre tts
loc 2 2 -2 14 lvi tts era nbl
lvi 2 2 -1 3 loc elv neo ins
vprs 2 2 -1 12 ts nbl fng nva
kru 2 2 -2 10 nva neo tts dre

Loc-LVI is the only problem, but the pairwise swap to time-lvi and loc-vprs works fine. Instead we got ts-lvi. Pairwise swap fail (being generous).

4 WTFBBQ, 2 pairwise swap fails, 6 total.

———————————————–

SAM regional 1 Closed Qualifier

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/South_America/1/Closed_Qualifier

Round 2 High has Kru-NEO and Round 2 Low has AC-QSHW which are both obviously wrong. WTFBBQ x2

Round 3 High

1 ts 2 0 6 1
2 ins 2 0 5 6
3 dre 2 0 4 4
4 neo 2 0 4 7

And we got ts-dre and neo-ins WTFBBQ

R3 Mid
5 nva 1 1 2 5 emt7 dre
6 loc 1 1 1 3 naiz ins
7 kru 1 1 -1 2 qshw neo
8 fng 1 1 -2 8 lgc ts
9 lgc 1 1 2 9 fng prds
10 ac 1 1 2 10 neo qshw
11 bnd 1 1 0 11 ins naiz
12 emt7 1 1 0 12 nva nice

5-12 and 8-9 are problems, but the two pairwise swaps (11 for 12 and 10 for 9) work. Instead we got nva-bnd and then loc-ac for no apparent reason. Pairwise swap fail.

Round 3 Low:
qshw 0 2 -5 15
nice 0 2 -6 13
naiz 0 2 -6 14
prds 0 2 -6 16

and we got qshw-naiz. WTFBBQ.

R4 High:
dre 2 1 1 4 nice nva ts
neo 2 1 1 7 ac kru ins
nva 2 1 5 5 emt7 dre bnd
ac 2 1 3 10 neo qshw loc
kru 2 1 0 2 qshw neo lgc
fng 2 1 -1 8 lgc ts emt7

neo is the problem, having played ac and kru. Dre-fng can be paired without a forced rematch, which is what the algorithm says to do, and we didn’t get it. Complicated fail

R4 Low
9 lgc 1 2 1 9 fng prds kru
10 loc 1 2 0 3 naiz ins ac
11 emt7 1 2 -1 12 nva nice fng
12 bnd 1 2 -3 11 ins naiz nva
13 naiz 1 2 -4 14 loc bnd qshw
14 prds 1 2 -5 16 ts lgc nice

9-14 and 10-13 are rematches, but the pairwise swap solves it. Instead we got EMT7-PRDS. Pairwise swap fail.

R5:
6 neo 2 2 0 7 ac kru ins fng
7 ac 2 2 0 10 neo qshw loc nva
8 kru 2 2 -2 2 qshw neo lgc dre
9 lgc 2 2 4 9 fng prds kru naiz
10 loc 2 2 3 3 naiz ins ac prds
11 emt7 2 2 2 12 nva nice fng bnd

7-10 and 8-9 are the problem, but instead of pairwise swapping, we got neo-lgc. Really should be a WTFBBQ, but a pairwise fail.

+4 WTFBBQ (8), +3 Pairwise (5), +1 complicated (1), 14 total.

————————————————-

SAM Regional 1

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/South_America/1

R4 Low

9 vprs 1 2 -1 7 drc emt7 ts
10 dre 1 2 -2 12 elv lvi nva
11 fng 1 2 -2 13 tts ftw drc
12 end 1 2 -3 8 ts loc tts
13 elv 1 2 -2 5 dre nva emt7
14 loc 1 2 -3 15 fur end ftw

10-13 is the only problem, but instead of 10-12 and 11-13, they fucked with vprs pairing. Pairwise swap fail.

+0 WTFBBQ(8), +1 pairwise swap (6), +0 complicated (1), 15 total.

——————————————————————

SAM Regional 2 Closed Qualifier

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/South_America/2/Closed_Qualifier

Round 2 Low:

FDOM (0) vs END (-1) is obviously wrong. WTFBBQ. And LOLOLOL one more time at zero-game-differential forfeits.

R3 Mid:
5 fng 1 1 2 4 nice loc
6 ftw 1 1 2 7 flip vprs
7 drc 1 1 -1 3 kru elv
8 fdom 1 1 -1 9 emt7 nva
9 kru 1 1 1 14 drc ac
10 sac 1 1 0 16 vprs flip
11 end 1 1 -1 11 loc emt7
12 ball 1 1 -1 15 nva nice

The base pairings work, but we got FNG-end and ftw-ball for some reason. WTFBBQ. Even scoring the forfeit 0-3 doesn’t fix this.

R3 Low:
emt7 0 2 0 8
flip 0 2 -5 10
nice 0 2 -5 13
ac 0 2 -6 12

and we got emt-flip. WTFBBQ.

R4 High:

3 vprs 2 1 2 1 sac ftw nva
4 loc 2 1 -1 6 end fng elv
5 ftw 2 1 4 7 flip vprs ball
6 drc 2 1 2 3 kru elv sac
7 fdom 2 1 1 9 emt7 nva kru
8 end 2 1 1 11 loc emt7 fng

Base pairings worked. WTFBBQ

R4 Low:
9 fng 1 2 0 4 nice loc end
10 kru 1 2 -1 14 drc ac fdom
11 ball 1 2 -3 15 nva nice ftw
12 sac 1 2 -3 16 vprs flip drc
13 ac 1 2 -3 12 elv kru nice
14 flip 1 2 -5 10 ftw sac emt7

fng, the top seed, doesn’t play either of the teams that were 0-2.. base pairings would have worked. WTFBBQ.

+5 WTFBBQ (13), +0 pairwise fail (6), +0 complicated (1), 20 total.

Sam Regional 3 Closed

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/South_America/3/Closed_Qualifier

R4 High

3	endgame	2	1	5	1	bnd	tn	vprs
4	lev	2	1	4	5	sac	endless	elv
5	bnd	2	1	1	16	endgame	benz	ball
6	fng	2	1	0	6	ball	resi	endless
7	fdom	2	1	-1	10	ftw	elv	emt7
8	tn	2	1	-2	2	benz	endgame	ftw

end-tn doesn’t work, but the pairwise swap is fine, and instead we got end-fng!??! Pairwise fail.

R4 low:

9	emt7	1	2	0	9	endless	sac	fdom
10	ftw	1	2	-1	7	fdom	ag	tn
11	ball	1	2	-2	11	fng	vprs	bnd
12	endless	1	2	-3	8	emt7	lev	fng
13	ag	1	2	-2	14	elv	ftw	benz
14	sac	1	2	-4	12	lev	emt7	resi

9-14 and 10-13 are problems, but the pairwise swap is fine. Instead we got emt7-ftw which is stunningly wrong. Pairwise fail.

Round 5:

6	fng	2	2	-2	6	ball	resi	endless	endgame
7	bnd	2	2	-2	16	endgame	benz	ball	lev
8	tn	2	2	-3	2	benz	endgame	ftw	fdom
9	ftw	2	2	1	7	fdom	ag	tn	emt7
10	endless	2	2	0	8	emt7	lev	fng	ag
11	ball	2	2	-1	11	fng	vprs	bnd	sac

This is a mess, going to the 8th most preferable pairing before finding no rematches, and.. they got it right. After screwing up both simple R4s.

Final SAM mispaired scoregroup count: +5 WTFBBQ (13), +2 pairwise fail (8), +0 complicated (1), 22 total.

OCE Fall Scoregroup pairing mistakes

Regional 1 Invitational Qualifier

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/Oceania/1/Invitational_Qualifier

Round 2 Low:

The only 2-3, JOE, is paired against the *highest* seeded 0-3 instead of the lowest. WTFBBQ.

R3 Mid:

5	kid	1	1	0	6	rrr	riot
6	grog	1	1	0	9	joe	gz
7	bndt	1	1	0	10	vc	rng
8	dire	1	1	-1	3	eros	wc
9	eros	1	1	0	14	dire	pg
10	vc	1	1	-1	7	bndt	gse
11	rrr	1	1	-2	11	kid	joe
12	tri	1	1	-2	15	rng	rru

8-9 and 7-10 are problems, but a pairwise swap fixes it, and instead every single pairing got fucked up. Pairwise fail is too nice, but ok.

R4 Low:

9	eros	1	2	-2	14	dire	pg	kid
10	vc	1	2	-3	7	bndt	gse	dire
11	tri	1	2	-3	15	rng	rru	grog
12	rrr	1	2	-4	11	kid	joe	bndt
13	joe	1	2	1	8	grog	rrr	pg
14	gse	1	2	-3	13	wc	vc	rru

Base pairings work. WTFBBQ

2 WTFBBQ, 1 pairwise, 3 total.

OCE Regional 1 Closed Qualifier

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/Oceania/1/Closed_Qualifier

Round 2 Low:

The best 2-3 (1620) is paired against the best 0-3 (tri), not the worst (bgea). WTFBBQ.

Round 4 High:

3	gse	2	1	1	1	tisi	joe	hms
4	grog	2	1	0	3	smsh	bott	wc
5	eros	2	1	4	5	hms	rust	bott
6	waz	2	1	4	11	tri	hms	pg
7	T1620	2	1	2	10	rru	tri	joe
8	rru	2	1	1	7	T1620	wc	tisi

Base pairings work. WTFBBQ.

Round 5:

6	T1620	2	2	1	10	rru	tri	joe	gse
7	waz	2	2	1	11	tri	hms	pg	eros
8	grog	2	2	-1	3	smsh	bott	wc	rru
9	tisi	2	2	2	16	gse	bgea	rru	bott
10	tri	2	2	-1	6	waz	T1620	bgea	joe
11	rust	2	2	-2	13	joe	eros	smsh	pg

7-10 is the problem, but they swapped with 6-11 instead of leaving 6-11 and swapping with 7-10. Pairwise fail.

+2 WTFBBQ(4), +1 pairwise(2), 6 total.

OCE Regional 1

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/Oceania/1

Round 5:

6	riot	2	2	3	2	grog	rust	eros	bndt
7	grog	2	2	0	15	riot	gz	blss	rrr
8	wc	2	2	-1	9	kid	bndt	rng	dire
9	fbrd	2	2	2	13	bndt	kid	T1620	hms
10	blss	2	2	1	6	eros	T1620	grog	gse
11	kid	2	2	-2	8	wc	fbrd	dire	eros

Exact same mistake as above. Pairwise fail.

+0 WTFBBQ(4), +1 pairwise(3), 7 total.

OCE Regional 2 Closed

Round 3 Mid:

5	blss	1	1	0	2	tri	fbrd
6	pg	1	1	0	14	gse	t1620
7	tisi	1	1	-1	11	rust	eros
8	ctrl	1	1	-1	13	hms	waz
9	gse	1	1	0	3	pg	lits
10	grog	1	1	0	12	eros	rust
11	bott	1	1	0	16	fbrd	tri
12	hms	1	1	-1	4	ctrl	bstn

Base pairings work. WTFBBQ

Round 4 Low:

9	blss	1	2	-1	2	tri	fbrd	bott
10	gse	1	2	-1	3	pg	lits	tisi
11	pg	1	2	-1	14	gse	t1620	hms
12	ctrl	1	2	-2	13	hms	waz	grog
13	tri	1	2	-2	15	blss	bott	bstn
14	lits	1	2	-4	10	t1620	gse	rust

Base pairings work. WTFBBQ

+2 WTFBBQ(6), +0 pairwise(3), 9 total.

OCE Regional 3 Closed

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/Oceania/3/Closed_Qualifier

R3 Mid:

5	gse	1	1	2	7	lits	t1620
6	wc	1	1	0	1	tisi	grog
7	deft	1	1	0	6	zbb	eros
8	gue	1	1	0	15	bott	crwn
9	tri	1	1	1	12	grog	tisi
10	bott	1	1	0	2	gue	pg
11	lits	1	1	-1	10	gse	ctrl
12	corn	1	1	-2	13	eros	zbb

Base pairings work. WTFBBQ

R3 Low

13	zbb	2	-3	11	deft	corn
14	ctrl	2	-5	8	t1620	lits
15	pg	2	-5	14	crwn	bott
16	tisi	2	-5	16	wc	tri

Base Pairings work. WTFBBQ

R4 High:

3	crwn	2	1	4	3	pg	gue	t1620
4	grog	2	1	1	5	tri	wc	eros
5	wc	2	1	3	1	tisi	grog	bott
6	tri	2	1	2	12	grog	tisi	deft
7	gue	2	1	2	15	bott	crwn	lits
8	corn	2	1	0	13	eros	zbb	gse

Base pairings work. WTFBBQ

R4 Low:

9	gse	1	2	0	7	lits	t1620	corn
10	deft	1	2	-1	6	zbb	eros	tri
11	bott	1	2	-3	2	gue	pg	wc
12	lits	1	2	-3	10	gse	ctrl	gue
13	zbb	1	2	0	11	deft	corn	ctrl
14	tisi	1	2	-3	16	wc	tri	pg

10-13 is the problem, pairwise swapped with 9-14 instead of 11-12. Pairwise fail.

+3 WTFBBQ(9), +1 pairwise(4), 13 total.

OCE Regional 3:

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/Oceania/3

R3 Low:

13	tri	2	-3	14	riot	wc
14	gue	2	-5	15	gz	t1620
15	crwn	2	-6	11	kid	grog
16	tisi	2	-6	16	rng	eros

Base pairings work. WTFBBQ

OCE Final count: +1 WTFBBQ(10), +0 pairwise(4), 14 total.

SSA Scoregroup Pairing Mistakes

Regional 1

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/Sub-Saharan_Africa/1

Round 2 Low:

The 0 GD FFs are paired against the best teams instead of the worst. WTFBBQ.

SSA Regional 2 Closed

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/Sub-Saharan_Africa/2/Closed_Qualifier

Round 4 Low:

9	bw	1	2	-3	6	crim	est	slim
10	aces	1	2	-3	12	llg	biju	info
11	crim	1	2	-5	11	bw	dd	fsn
12	win	1	2	-5	16	mist	gst	auf
13	biju	1	2	-2	13	info	aces	est
14	slap	1	2	-6	14	auf	fsn	gst

10-13 pairwise exchanging with 9-14 instead of 11-12. Pairwise fail.

SSA Regional 2

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/Sub-Saharan_Africa/2

R4 High

3	atk	2	1	4	2	mils	ag	bvd
4	org	2	1	1	3	dwym	atmc	op
5	atmc	2	1	4	7	mist	org	oor
6	llg	2	1	4	12	oor	bvd	dd
7	auf	2	1	3	9	ag	mils	exo
8	ag	2	1	0	8	auf	atk	mist

3-8 is the problem, but instead of pairwise swapping with 4-7, the pairing is 3-4?!?!?!

R4 Low

9	mist	1	2	0	10	atmc	dwym	ag
10	dd	1	2	-3	11	exo	fsn	llg
11	oor	1	2	-4	5	llg	info	atmc
12	exo	1	2	-5	6	dd	op	auf
13	fsn	1	2	-1	16	op	dd	dwym
14	info	1	2	-2	13	bvd	oor	mils

Another 10-13 pairwise mistake.

SSA Regional 3 Closed

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/Sub-Saharan_Africa/3/Closed_Qualifier

Round 5

6	fsn	2	2	-1	5	slim	lft	gst	blc
7	mils	2	2	-1	6	blc	chk	ddogs	oor
8	roc	2	2	-1	10	gst	mlg	slim	mist
9	ddogs	2	2	3	4	alfa	mist	mils	lft
10	gst	2	2	0	7	roc	oor	fsn	alfa
11	slim	2	2	0	12	fsn	llg	roc	est

This is kind of a mess, but 6-9, 7-8, 10-11 is correct, and instead they went with 6-7, 8-9, 10-11!??!? which is not. Complicated fail.

SSA total (1 event pending): 1 WTFBBQ, 3 pairwise, 1 complicated.

APAC N Swiss Scoregroup pairing mistakes

Regional 2

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/Asia-Pacific_North/2

Round 5:

6	xfs	2	2	1	8	nor	gds	nov	dtn
7	alh	2	2	0	10	n55	timi	gra	hill
8	cv	2	2	-1	4	chi	gra	gds	n55
9	gra	2	2	-1	6	blt	cv	alh	nor
10	chi	2	2	-2	13	cv	blt	dtn	nov
11	timi	2	2	-2	16	ve	alh	blt	wiz

This is a mess, but a pairing is possible with 6-11, so it must be done. Instead they screwed up that match and left 7-10 alone. Complicated fail

APAC S Swiss Scoregroup pairing mistakes

Regional 2 Closed

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/Asia-Pacific_South/2/Closed_Qualifier

Round 4 Low:

9	exd	1	2	1	14	flu	dd	shg
10	vite	1	2	-2	12	shg	cel	flu
11	wide	1	2	-3	10	pow	che	ete
12	pow	1	2	-4	7	wide	hira	acrm
13	whe	1	2	-4	9	ete	acrm	che
14	sjk	1	2	-4	16	cel	shg	axi

11-12 is the problem, and instead of swapping to 10-12 11-13, they swapped to 10-11 and 12-13. Pairwise fail.

Round 5

6	acrm	2	2	1	13	pc	whe	pow	ete
7	flu	2	2	-2	3	exd	axi	vite	dd
8	shg	2	2	-3	5	vite	sjk	exd	hira
9	exd	2	2	4	14	flu	dd	shg	sjk
10	pow	2	2	-1	7	wide	hira	acrm	whe
11	wide	2	2	-3	10	pow	che	ete	vite

another 8-9 issue screwing with 6-11 for no reason. Complicated fail.

Regional 3 Closed:

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/Asia-Pacific_South/3/Closed_Qualifier

Round 3 Mid

Two teams from the 1-0 bracket are playing each other (and two from the 0-1 playing each other as well obviously). I checked this one on Smash too. WTFBBQ

Round 4 High

3	dd	2	1	2	2	wc	woa	bro
4	flu	2	1	2	4	soy	fre	pum
5	znt	2	1	5	1	woa	wc	shg
6	pc	2	1	4	3	lads	pum	woa
7	ete	2	1	3	7	wash	bro	wide
8	fre	2	1	2	12	shg	flu	exd

The default pairings work. WTFBBQ

Round 4 Low

9	exd	1	2	-1	6	pum	lads	fre
10	wide	1	2	-1	8	bro	wash	ete
11	woa	1	2	-3	16	znt	dd	pc
12	shg	1	2	-4	5	fre	soy	znt
13	soy	1	2	-2	13	flu	shg	wc
14	lads	1	2	-2	14	pc	exd	wash

9-14 is the problem, and the simple swap with 10-13 works. Instead they did 9-12, 10-11 , 13-14. Pairwise fail.

Round 5:

6	ete	2	2	2	7	wash	bro	wide	dd
7	fre	2	2	0	12	shg	flu	exd	znt
8	flu	2	2	-1	4	soy	fre	pum	pc
9	soy	2	2	0	13	flu	shg	wc	lads
10	woa	2	2	0	16	znt	dd	pc	wide
11	shg	2	2	-3	5	fre	soy	znt	exd

8-9 is the problem, but the simple swap with 7-10 works. Instead we got 6-8, 7-9, 10-11. Pairwise fail.

APAC Total: 2 WTFBBQ, 3 Pairwise fails, and 1 complicated.

NA Swiss Scoring Pairing Mistakes

Regional 3 Closed

https://liquipedia.net/rocketleague/Rocket_League_Championship_Series/2021-22/Fall/North_America/3/Closed_Qualifier

Round 4

3	pk	2	1	4	11	rbg	yo	vib
4	gg	2	1	1	1	exe	sq	oxg
5	eu	2	1	4	14	sq	exe	rge
6	sq	2	1	1	3	eu	gg	tor
7	xlr8	2	1	1	12	vib	leh	sr
8	yo	2	1	1	15	tor	pk	clt

This is a mess. 3-8 is illegal, but 3-7 can lead to a legal pairing, so it must be done (3-7, 4-5, 6-8). Instead we got 3-6, 4-8, 5-7. Complicated fail.

EU made no mistakes. Liquipedia looks sketchy enough on MENA that I don’t want to try to figure out who was what seed when.

Final tally across all regions

	WTFBBQ	Pairwise	Complicated	Total
SAM	13	8	1	22
OCE	10	4	0	14
SSA	1	3	1	5
APAC	2	3	1	6
NA	0	0	1	1
EU	0	0	0	0
Total	26	18	4	48

Missing the forest for.. the forest

The paper A Random Forest approach to identify metrics that best predict match outcome and player ranking in the esport Rocket League got published yesterday (9/29/2021), and for a Cliff’s Notes version, it did two things: 1) Looked at 1-game statistics to predict that game’s winner and/or goal differential, and 2) Looked at 1-game statistics across several rank (MMR/ELO) stratifications to attempt to classify players into the correct rank based on those stats. The overarching theme of the paper was to identify specific areas that players could focus their training on to improve results.

For part 1, that largely involves finding “winner things” and “loser things” and the implicit assumption that choosing to do more winner things and fewer loser things will increase performance. That runs into the giant “correlation isn’t causation” issue. While the specific Rocket League details aren’t important, this kind of analysis will identify second-half QB kneeldowns as a huge winner move and having an empty net with a minute left in an NHL game as a huge loser move. Treating these as strategic directives- having your QB kneel more or refusing to pull your goalie ever- would be actively terrible and harm your chances of winning.

Those examples are so obviously ridiculous that nobody would ever take them seriously, but when the metrics don’t capture losing endgames as precisely, they can be even *more* dangerous, telling a story that’s incorrect for the same fundamental reason, but one that’s plausible enough to be believed. A common example is outrushing your opponent in the NFL being correlated to winning. We’ve seen Derrick Henry or Marshawn Lynch completely dump truck opposing defenses, and when somebody talks about outrushing leading to wins, it’s easy to think of instances like that and agree. In reality, leading teams run more and trailing teams run less, and the “signal” is much, much more from capturing leading/trailing behavior than from Marshawn going full beast mode sometimes.

If you don’t apply subject-matter knowledge to your data exploration, you’ll effectively ask bad questions that get answered by “what a losing game looks like” and not “what (actionable) choices led to losing”. That’s all well-known, if worth restating occasionally.

The more interesting part begins with the second objective. While the particular skills don’t matter, trust me that the difference in car control between top players and Diamond-ranked players is on the order of watching Simone Biles do a floor routine and watching me trip over my cat. Both involve tumbling, and that’s about where the similarity ends.

The paper identifies various mechanics and identifies rank pretty well based on those. What’s interesting is that while they can use those mechanics to tell a Diamond from a Bronze, when they tried to use those mechanics to predict the outcome of a game, they all graded out as basically worthless. While some may have suffered from adverse selection (something you do less when you’re winning), they had a pretty good selection of mechanics and they ALL sucked at predicting the winner. And, yet, beyond absolutely any doubt, the higher rank stratifications are much better at them than the lower-rank ones. WTF? How can that be?

The answer is in a sample constructed in a particularly pathological way, and it’s one that will be common among esports data sets for the foreseeable future. All of the matches are contested between players of approximately equal overall skill. The sample contains no games of Diamonds stomping Bronzes or getting crushed by Grand Champs.

The players in each match have different abilities at each of the mechanics, but the overall package always grades out similarly given that they have close enough MMR to get paired up. So if Player A is significantly stronger than player B at mechanic A to the point you’d expect it to show up, ceteris paribus, as a large winrate effect, A almost tautologically has to be worse at the other aspects, otherwise A would be significantly higher-rated than B and the pairing algorithm would have excluded that match from the sample. So the analysis comes to the conclusion that being better at mechanic A doesn’t predict winning a game. If the sample contained comparable numbers of cross-rank matches, all of the important mechanics would obviously be huge predictors of game winner/loser.

The sample being pathologically constructed led to the profoundly incorrect conclusion

Taken together, higher rank players show better control over the movement of their car and are able to play a greater proportion of their matches at high speed. However, within rank-matched matches, this does not predict match outcome.Therefore, our findings suggest that while focussing on game speed and car movement may not provide immediate benefit to the outcome within matches, these PIs are important to develop as they may facilitate one’s improvement in overall expertise over time.

even though adding or subtracting a particular ability from a player would matter *immediately*. The idea that you can work on mechanics to improve overall expertise (AKA achieving a significantly higher MMR) WITHOUT IT MANIFESTING IN MATCH RESULTS, WHICH IS WHERE MMR COMES FROM, is.. interesting. It’s trying to take two obviously true statements (Higher-ranked players play faster and with more control- quantified in the paper. Playing faster and with more control makes you better- self-evident to anybody who knows RL at all) and shoehorn a finding between them that obviously doesn’t comport.

This kind of mistake will occur over and over and over when data sets comprised of narrow-band matchmaking are analysed that way.

(It’s basically the same mistake as thinking that velocity doesn’t matter for mediocre MLB pitchers- it doesn’t correlate to a lower ERA among that group, but any individuals gaining velocity will improve ERA on average)

RLCS X viewership is being manipulated

TL;DR EU regionals weekend 1 viewership was still fine, but legit viewership never cracked 80k even though the reported peak was over 150k.

Psyonix was cooperative enough to leave the Sunday stream shenanigan-free, so we have a natural comparison between the two days. Both the Saturday and Sunday streams had the same stakes- half of the EU teams played each day trying to qualify for next weekend- so one would expect viewership each day to be pretty similar, and in reality, this was true. Twitch displays total viewers for everyone to see, and the number of people in the chatroom is available through the API, and I track it. (NOT the number of people actually chatting- you’re in the chatroom if you’re logged in, viewing normally, and didn’t go out of your way to close chat. The fullscreen button doesn’t do it, and it appears that closing chat while viewing in squad mode doesn’t remove you from the chatroom either). Only looking at people in the chatroom on Saturday and Sunday gives the following:

RLCSXeu1-1inchat

That’s really similar, as expected. Looking at people not in the chatroom- the difference between the two numbers- tells an entirely different story.

RLCSXeu1-1notinchat

LOL. Alrighty then. Maybe there’s a slight difference?

Large streams average around 70% of total viewers in chat. Rocket League, because of drops, averages a bit higher than that. More people make Twitch accounts and watch under those accounts to get rewards. Sunday’s stream is totally in line with previous RLCS events and with the big events in the past offseason. Saturday’s stream is.. not. On top of the giant difference in magnitude, the Sunday number/percentage is pretty stable, while the Saturday number bounces all over the place. Actual people come and go in approximately equal ratios whether they’re logged in or not. At the very end of Saturday’s stream, it was on the Twitch frontpage, which boosts the not-in-chat count, but that doesn’t explain the rest of the time.

The answer is that the Saturday stream was embedded somewhere outside of Twitch. Twitch allows a muted autoplay stream video in a random webpage to count as a viewer even if the user never interacts with the stream (a prominent media source said otherwise last year. He was almost certainly wrong then, and I’ve personally tested in the past 2 months that he’s wrong now), and, modern society being what it is, services and ad networks exist to place muted streams where ~nobody pays any attention to them to boost apparent viewcount, and publishers pay 5 figures to appear to be more popular than they are. Psyonix/RLCS was listed as a client of one of these services before and has a history of pulling bullshit and buying fake viewers. There’s a nice article on kotaku detailing this nonsense across more esports.

If the stream were embedded somewhere it belonged, instead of as an advertisement to inflate viewcount, it’s hard to believe it also wouldn’t have been active on Sunday, so it’s very likely they’re just pulling bullshit and buying fake viewers again. If somebody at Psyonix comments and explains otherwise, I’ll update the post, but don’t hold your breath. Since a picture speaks a thousand words:

RLCSXeu1-1notinchat

Are Rocket League viewership numbers fraudulently high?

Maybe. Quite possibly. Something strange is going on. For the TL;DR crowd Season 6 EU playoff viewership (Sunday October 14, 2018) was abominably low, off by 10,000-20,000 viewers or up to a third of the regular audience, far worse than any other broadcast this season. Season 6 NA regional playoffs, held the day before (Saturday, October … Continue reading “Are Rocket League viewership numbers fraudulently high?”

Maybe. Quite possibly. Something strange is going on.

For the TL;DR crowd

Season 6 EU playoff viewership (Sunday October 14, 2018) was abominably low, off by 10,000-20,000 viewers or up to a third of the regular audience, far worse than any other broadcast this season.
Season 6 NA regional playoffs, held the day before (Saturday, October 13, 2018) were exactly in line with seasonal numbers.
Regional playoff viewership was always fine-to-good for the other 7 Season 3-Season 6 regional playoff broadcasts
There was an unnatural 15,000ish viewer bump late in the broadcast that can only realistically be a big host or major viewbot fraud
Psyonix won’t comment at all and I can’t find other evidence of a large host
Viewership of major events was trending way down coming into season 6

Background

The Rocket League Championship Series (RLCS) is a biannual competition featuring 8 teams from North America (NA), 8 teams from Europe (EU), and 8 teams from Oceania (OCE) each playing an online single-round-robin season over 5 weeks followed by an online regional playoff. The top 4 teams from NA and EU and the top 2 teams from OCE then meet in person to crown the world champion for that season. Psyonix, the maker of Rocket League, runs the NA and EU competitions and broadcasts every game on its Twitch stream. A different entity is responsible for OCE and broadcasts those games on its own Twitch stream instead. We won’t worry about the OCE broadcasts.

Viewer counts increased significantly between seasons 3 and 4, slightly between 4 and 5, and were down about 20% in the current season 6, as the following graphs from esc.watch, an esports viewership tracker, show. I have personally spot-checked esc.watch’s Rocket League charts over the past year and have always found them accurate.

click to enlarge

The focus of this post will be the large decline in the season 6 EU playoff. Note that season 3 NA has a slight decline in playoff viewership while every other playoff broadcast has been up or at least flat. File that away for later. A couple of other easily explainable anomalies need to be addressed first. In season 3 (green), there was a week off between week 3 (April 2, 2017) and week 4 (April 16, 2017). That resulted in lower viewership in week 4 in both NA and EU that rebounded to “normal” in the following 2 weeks of consecutive play. Season 5 week 5 in EU had a large technical problem. A power outage near the Psyonix studio cut the broadcast short after 2 games and the remaining 4 games were rescheduled for 12 PM Eastern on a Thursday in place of the regular 12 PM Eastern on Sunday. That timeslot is obviously terrible in comparison for NA viewers and the low viewer count is quite understandable. Season 4 week 5 in EU had a different technical problem. The broadcast was fine, but twitch.tv did not send out notifications that the broadcast was starting and reportedly didn’t even show the Rocket League channel as being online. Given the nature of viewership, which will be discussed next, it’s also quite understandable for that number to be in the toilet.

A typical RLCS broadcast accumulates most of its viewers over its first three matches like this (another reason that the EU S5 Week 5 broadcast being cut off after 2 games killed the viewership).

ctg0rf5

The viewership accumulates this way because it’s comprised of hardcore viewers, those who are ready for the start of the broadcast every week, or at least the notification for it, and those who happen upon it in progress. People who follow the RL Twitch stream and connect to Twitch will see that it’s online, and people who play Rocket League while a broadcast is in progress are greeted with a giant flashing “ESPORTS LIVE NOW” button on the main menu that they can click to watch.

EU Regionals Anomalies

The graphs for the 12 broadcasts this season follow. Don’t pay attention to the actual numbers yet, just the shapes. And in particular the last graph on the bottom right, the EU playoffs.

click to enlarge

There are two obvious things of note. First, the glitch on the left is the very end of EU Week 5 (the bottom left image above) and not actually part of the playoff broadcast. That was confirmed directly by esc.watch. Second.. well…

S6euwhat

That’s a near-instantaneous (under 5 minute) jump of almost 15,000 viewers, and there’s no precedent of such a thing in any other broadcast this season. It’s clearly not part of the natural accumulation process.

EU Regionals Low Viewership

The other interesting thing about this broadcast is that it was the least viewed broadcast of the season by a country mile.

RL S6 Viewership by match slot

Or in slightly less cluttered form, for each region, the averages of the regular season (weeks 1-5), the playoffs, and the difference between the playoff viewership and the regular season viewership.

RL S6 averages

Yikes? To summarize so far, we have a mystery with the following facts

Season 6 EU playoff viewership (Sunday October 14, 2018) was abominably low, off by 10-20 thousand viewers or up to a third of the regular audience, far worse than any other broadcast this season.
Season 6 NA regional playoffs, held the day before (Saturday, October 13, 2018) were exactly in line with seasonal numbers.
Regional playoff viewership was always fine-to-good for the other 7 Season 3-Season 6 regional playoff broadcasts
There was an unnatural 15,000ish viewer bump late in the broadcast.

Potential Causes for Low Viewership

In investigating the low viewership, I was unable to uncover any evidence of technical difficulties. I was watching the broadcast and using Twitch and both worked fine. I found no reports of issues, Twitch-wise or internet-wise, in the relevant reddit thread while gripes about the Twitch issues in season 4 were plentiful. The graph of viewer count was accurate. I personally observed the extremely low viewer counts throughout as well as the high viewer count at the end. Unfortunately I wasn’t paying attention to the count as the 15,000 viewers showed up late. Technical issues don’t seem to be the explanation here.

The second possibility is that something external was drawing would-be viewers away. There wasn’t any news in the world that day that would have diverted a large number of viewers. Call of Duty: Black Ops 4 was released worldwide on October 12, but if that were the cause, it should have sunk the NA playoff viewership on October 13 as well. Similarly, Rocket League itself was offering a double-XP weekend, giving out more in-game rewards for playing, but that was also active on Saturday during the NA playoffs. There were other non-Rocket League esports events, as there are many weekends, but there didn’t appear to be anything that could uniquely sink the entire Sunday broadcast or the first 90% of it. This doesn’t appear to be it either.

The third possibility is that the broadcast itself was thought to be uniquely unappealing to tune into, presumably because Dignitas, the two-time defending world championship roster that just went 7-0 in round robin play, was thought to be a lock to win. This explanation falls short for several reasons. First, there’s no evidence that Dignitas is bad for viewership at all. None of their matches have seen strange declines and the last match of the regular season was an almost meaningless Dignitas match and it had the highest viewership of the week. The idea that people would watch to see if Dignitas could go 7-0, then disappear in droves as Dig tried to become the first NA/EU team to go 7-0 and win the regional playoffs… well, that’s just strange.

The format of the regional championship also undercuts that explanation. It’s a 6-team single elimination where the top 4 qualify for the world championship. The top 2 seeds get first-round byes and automatically qualify which means the first two matches- #3 vs #6 and #4 vs #5- are critical. The winner goes to the world championships and the loser is done for the season. Even if people didn’t want to watch the rest of the playoffs, those matchups should have been compelling. The second match of the day involved the most anticipated rookie in RLCS history by far, ScrubKilla, and his wildly inconsistent Renault Vitality team playing for a LAN spot. This should have been a must-watch series, and by one metric it was. Not counting the first match of the day, which always accumulates lots of viewers, Vitality-PSG had a higher accumulation of viewers (+17k) than any other match had all season, almost 50% clear of second place. People talked about ScrubKilla and Vitality all season. Viewers appeared to tune in in numbers specifically to not miss this match. The final number shouldn’t have been hot garbage, and yet it still was. I’m at a complete loss for a legitimate explanation for the overall low viewership.

Potential Causes of the 15,000 Viewer Bump

Switching focus to the ~15,000 viewer bump, there’s one legitimate explanation- a large stream from another game/scene hosted them.. That would bump the viewer count up quickly, but I could find no evidence of that actually happening. I couldn’t find a mention on twitter. I watched the relevant portion of the broadcast replay, focusing on the Twitch chat (bless my soul), and there weren’t any mentions of a host, nor any newbie-like comments, which seems a bit unlikely. In addition, the attrition rate would have had to have been extremely low because viewership also went up a tad after the spike. Rocket League being the autoplay stream on the Twitch homepage was suggested as a possibility, but after observing those streams for a few days, none I saw were accumulating viewers at even a meaningful fraction of the necessary rate.

If the 15,000 viewer bump isn’t natural viewer accumulation, and isn’t a legitimate Twitch host, that leaves fraud. Psyonix randomly gives away in-game items to players who watch the stream and have their Twitch accounts linked. This is absolutely a boon to viewership, and can be misleading in a way because players can and do load the stream, mute it, and pay no attention to it just for the chance of getting a drop. In this way, the percentage of the viewer count who are watching the stream is lower than it otherwise would be, but this is all out in the open, and it does also further a legitimate purpose of trying to entice players to become esports viewers. Where there are giveaways, there are people trying to exploit giveaways, but the likelihood of large-scale fraud here seems very small to me. The expected value in terms of reselling items of having a linked account watch a stream for 6 hours is about 25-50 cents (USD), and obtaining that value across thousands of accounts involves selling numerous small-ticket (.50-$3) items that nobody wants multiples of. While I have no doubt that some people sign up a few accounts and have Twitch open in multiple browsers, any person or group capable of controlling enough computers/IPs to generate 15,000 concurrent fraudulent Twitch views should be able to make a hell of a lot more money doing almost anything else with them…

Such as selling their services to a company/streamer that wants to inflate its viewer count to bring in more advertising/sponsorship revenue or to just appear to be more popular than it actually is. This is called viewbotting, and it’s not a rare occurrence. If Psyonix had an arrangement for around 15,000 viewbots and forgot to turn them on before the last match, or never turned them on and got a 15,000 viewer host near the end, this would explain everything. Here’s the graph from earlier with 15,000 viewers added through the whole broadcast instead of the very end.

RLS6 +15k

That would be much more consistent with the regular season. I obviously have no affirmative proof that Psyonix is committing viewbot fraud, but if they were committing viewbot fraud, this is *exactly* what it would look like if they screwed up for a week, and I’m at a complete loss for legitimate explanations for the low viewership that wouldn’t also have sunk the NA playoffs.

In summary, the evidence here is consistent with several main hypotheses of varying plausibility.

There is a legitimate, as yet undiscovered reason for the overall low viewership and they got a big host of around 15,000 near the end.
Psyonix is viewbotting around 15,000 fakes and forgot to turn them on until the last match of the day.
Psyonix is viewbotting around 15,000 fakes, forgot to turn them on at all that day, and coincidentally got a big host of around 15,000 near the end.

Psyonix’s Refusal to Comment

I contacted Psyonix a fourth time with the article up to this point and another request for comment. That’s private messages to Murty Shah and Cory Lanier, the two main public faces of their esports program, an email to Psyonix’s general PR contact address, and an email to Psyonix’s esports contact address, and I’ve received no replies. In addition, they didn’t comment in reddit threads where the low viewership and the big bump were discussed. The lack of response has to be considered deliberate at this point. Let’s think about what that means.

Hypothetically, if Psyonix knows the viewers are from a host, would they be willing to tell the world about it? I would think so. I’m not a social media guru, but if somebody from a different game added 15k viewers to my 48k viewer broadcast, I would want to say thank you, and I’d want to say it publicly to attempt to engage their followers a little more. They shouldn’t do that for small hosts because that would bring a barrage of useless attention-seekers, and I’m not sure exactly where the acknowledgment line should be drawn, but a 15,000 viewer host seems safely big enough at this point. Furthermore, there’s no obvious benefit to attempting to conceal that a host happened. It would already be known to the people watching the other stream, it’s obvious to everybody who looked at the viewer graph that something happened, and it’s inconceivable to me that a channel from a different scene wanting to expose its viewers to Rocket League could be construed negatively. Now watch it turn out that Psyonix is slowrolling and waiting for the article to go live to say who the host was, but I have to work with what I have so far. If they do decide to announce who the host was and it’s legit, then we’re just back to the mystery of the low viewership in general.

Hypothetically, if Psyonix knows the viewers are fake, would they be willing to deny that they were hosted or willing to lie about who hosted them? The latter would be a huge mistake because it would be discovered in no time flat, and the former is straight-up admitting that the numbers for that broadcast are bogus, and by inference, that the numbers for every other broadcast during the season are almost certainly bogus. They have to stay silent in this case and hope it blows over. Twitch itself should have an active interest in investigating this broadcast if there wasn’t a big host.

Reasons for Viewbotting

The cleanest explanation for not commenting is fraud, which invites the question of whether or not viewbotting would make any sense. As a moral matter, I have absolutely no idea if the relevant people working there are the type who could do such a thing. As a business matter, it’s plausible. Before this season, viewership trends started out ok and then took a straight-line path to dumpster fire. All tournament pairs below are roughly equivalent across seasons. Broadcasts were about the same length and Fan Rewards were active for all referenced broadcasts. This is what Psyonix was looking at for average viewers as the numbers rolled in this year

April 22. RLCS Season 5: 68.2k, +15% over season 4
May 6. Promotion/Relegation Tournament. 44.5k, +9% over season 4
June 10. World Championships. 101.8k, -5% from season 4 and -14% from season 3
August 11. UORL 2 final qualifiers. 15.1k. esc.watch doesn’t have comps for last year, and timeslots are wonky, but this is not a good number at all for top players playing with rewards active. The peak viewership was only 29.5k, presumably on one of the four good weekend timeslots.
August 26. Universal Open 2. 29.1k, -35% from last year
September 2. Season 6 play-ins 42.0k -46% from last season.

If I were in charge of the esport or somebody whose job/livelihood depended on the continued success of the esport, I would have been very nervous going into season 6. It’s not hard to see the appeal of buying some “insurance viewers” in that climate. As it was, season 6 viewership was officially down 20% across the board, and the real number is much worse if this season was being viewbotted. To repeat one last time, I have no definitive proof that viewbotting took place. It is only a hypothesis that explains viewer stats and Psyonix’s behavior.

Conclusion

I hope, and all fans of Rocket League probably join me in hoping that there was a legitimate reason for the low viewership and that Psyonix is simply being obnoxious refusing to acknowledge a large host. Anything else would be a huge story and a crippling blow to the future of the esport. If Twitch or Psyonix comment, the article will be updated to reflect that. Many thanks to esc.watch for providing the charts for this article.