Cheating in Online Chess (Part 1): Suspicions of Engine Assistance – Chess.com

Ral Snchez Garca and Hctor Laiz Ibez are currently conducting a qualitative study on cheating in online chess, whose preliminary findings they describe in a two-part blog entry. This blog post is the first of the two parts.

Chessable provided support to their research. University students and faculty research sponsors starting or continuing chess-themed research may apply before May 15 at https://www.chessable.com/research_awards for Chessable Research Awards.

The advent of digital platforms in chess has significantly enhanced the games accessibility and global reach, yet it has concurrently escalated the prevalence of cheating, specifically through unauthorized engine assistance. To study the topic of cheating in online chess, we conducted an experiment with 24 Spanish male chess players (Elo 2100-2500) in seven sessions. The players were divided into three different experimental groups: (A) human; (B) human playing autonomously with the help of a chess engine during the whole game; (C) human playing with the restricted help of a chess engine: they could only ask the researcher for help three times (consisting of best move and position evaluation) if they had more than 2 minutes on the clock.

We randomly paired players to play a two-game (changing colours) match of 10+5 (10 minutes with an increment of 5 seconds). None of them knew the identity of the opponent; neither his exact Elo rating; nor the condition of his experimental group. Players used an online platform, using nicknames with pseudonyms provided by the researchers. We recorded the games and conducted private semi-structured interviews with each player right after the match ended. We asked them about their impressions of the two-game match, how they assessed their play and their opponents play, and we specifically asked them about their impression on the possibility that the opponent could have used a chess engine. Besides, we asked players using chess engines (conditions B and C) about their strategies and impressions on how the use of the engine affected their way of playing and the whole interaction during the game. Group B and C players were also subject to questioning on whether they had any type of suspicions regarding their opponents play and the possibility of them also having access to external help.

After the entire round of interviews, we conducted a colloquium with all the participants of the specific session in which we disclosed the experimental conditions of each player. In the colloquium we oriented the conversation towards the broader topic of cheating in chess, both over the board and online.

A qualitative content analysis of the interviews and colloquiums provided findings in two major topics: (1) emergence of suspicions of cheating during the games by every participant; (2) strategies and impressions on engine assistance by those participants in conditions B and C.

This first post deals with the preliminary findings about (1) the emergence of suspicions of cheating during the games.

First, from the subjective impressions of players, we could not say with certainty when cheating was taking place or not. Nonetheless, we could predict (or at least say that it was more likely to emerge) when suspicions of cheating would emerge in the participants, regardless of their personalities. Suspicions of cheating emerged when performative expectations about chess playing were broken. Performative expectations refer to what is considered as normal play for a human player with certain characteristics (i.e., specific Elo rating). Performative expectations were projected depending on different factors: (i) stratification; (ii) interaction; (iii) experimental conditions.

(i) Stratification: this term referred first and foremost to chess hierarchical status expressed in the Elo rating. A specific Elo rating projects an expected playing strength. When the playing strength in the actual performance of the game does not match what is expected for the Elo rating, suspicions are more likely to occur. During the experiment, hierarchical status did not influence much the participants suspicions because they were unaware of the exact Elo rating of the opponent. The possible range (2100-2500) was so broad that it could not project a defined expectancy to be tested against the actual performance of the adversary. Nonetheless, in the colloquium, a mismatch between Elo rating and actual performance appeared frequently in the narratives of suspicions in cheating cases. Also, during the colloquium, participants talked about other qualifying stratification elements (age, gender, and nationality) that altered the performative expectancies bound to the hierarchical status of Elo rating, thus affecting the emergence of suspicions of cheating.

(ii) Interaction: during the actual chess games, participants expected to find normally occurring chess events in relation to questions such as the human-like logic of moves; consistency of playing strength and style; and time management. Such normal appearance of play was disrupted when awkward or incompressible moves appeared; when inconsistency in the opponents play appeared; or when time management of moves was erratic and/or variable. When these non-normal chess events occurred, the suspicion of engine assistance emerged more often.

Other interaction elements, present in over the board games, such as awkward emotional responses (e.g., too nervous, or too calm) were not available in online game. Thus, they did not appear in the interview narratives of the participants, even though they were raised in the colloquium discussion on cheating afterwards.

Even though participants could not make a post-hoc analysis of the games, elements such as a high percentage of precision in chess moves (related to Elo rating) and correlation between the opponent patterns and engine patterns were also raised in the colloquium as clear indexes of suspicion of cheating by engine assistance.

(iii) Experimental conditions: The experiment was presented as a typical psychological experiment that studied common topics (decision making in chess), carried out by university researchers, one of them known to the participants. Performative expectations about what it entailed to participate in scientific experiments implied that all participants took the test under the same conditions, that they were not deceived or harmed. That is why those who were subjects in condition A (human) could not conceive the suspicion of cheating in his adversary; engine assistance would break the expectation that all participants were under the same conditions and that they (humans in condition A) would not be deceived. However, precisely the condition of those who used the engine in conditions B and C projected the expectation that, since all experimental subjects were under the same conditions, everyone could be using chess engines. That explains why subjects using engine in conditions B and C were more suspicious of their human counterparts in condition A than the other way around. In fact, only one subject in condition A suspected about the use of engine of his opponent.

Concluding this first post of preliminary findings from the study, we found that the mere suspicion of cheating (engine assistance) by the opponent was enough to alter the players capacity to engage the game, negatively affecting his performance. Perhaps the most negative impact on the current sensation of extended cheating in online chess (Zaksait, 2020, p. 68), qualified by many participants in the experiment as paranoia, is precisely this: many players are underperforming on their chess play due to the suspicions of cheating.

Zaksait, S. (2020). Cheating in chess: a call for an integrated disciplinary regulation. Kriminologijos studijos, 8, 57-83.

Ral is a lecturer on motor learning and the theory of play at the Sports Science school of the Polytechnic University of Madrid. He is also closely connected to the Embodied Design Research Laboratory (EDRL) of the University of California, Berkeley. His research blends social and cognitive sciences to study skill acquisition from an embodied perspective. His interest in chess deals with the question of distributed cognition and distributed agency between humans and computers. Email: raul.sanchezg@upm.es

Hctor is an honorary fellow of the Department of Business Management and Economics at the University of Len. His research focuses on the digital economy and emerging technologies. He also works full-time at the Spanish National Cybersecurity Institute (INCIBE), dealing mainly with matters related to international relations and EU initiatives. He is a FIDE Master and plays for Club de Xadrez Fontecarmoa. Email: hlaii@unileon.es

Read more:

Cheating in Online Chess (Part 1): Suspicions of Engine Assistance - Chess.com