# data[].attributes.body
Sports culture has become an emotional experience. Watching games often pushes fans to pour their hope into a single team or individual, then cheer, laugh, and cry with and for them. Watching games also often initiates the release of adrenaline and high tensions that pose a danger if snapped at the wrong time. Unfortunately, this occurs far too often, with controversial plays, calls, and game outcomes leaving athletes at the mercy of fans’ outbursts of anger and frustration. With the rise of social media, athletes are not even able to escape these outbursts once they have left the stadium.

Such instances of danger are not limited towards athletes; referees also often bear the brunt of the abuse by both fans and athletes in major league sports, while referees at recreational/youth levels receive such from players and participants. One weekend soccer player took this abuse to the next level, having so much frustration during a game that he punched and fatally injured the referee officiating his game (White). Measures must be taken to mitigate the animosity and hostility directed towards game officials during and after games not only to protect them as people, but also ensure that people will still want to pursue their jobs in the future. Especially in professional sports, fairness and accuracy are principles of sports that allow the activity to maintain an air of dignity and civility while retaining strong emotions of anger, excitement, grief, and pride.

The technological revolution our modern world faces today provides an attractive opportunity to athletics and professional sports in increasing the level of accuracy at which they are officiated. In order to fully take advantage of this opportunity in a way that could potentially increase viewer engagement in addition to ensuring fair play within games, it may be necessary to more deeply integrate a variety of forms of technology into how professional sports are officiated. Incorporating a non-human source of regulation within sports could lead to a reduction in the internal biases human officials inevitably demonstrate and reduce the risk of controversy associated with ambiguous calls that might allow for the perpetuation of negative stereotypes.

# data[].attributes.body
Why should AI and technology play a larger role in officiating competitive sports?

# data[].attributes.body
It has always been necessary in competitive games to officiate in response to the environment of the game, amount of tension present, skill level and reputation of the athletes, etc. often through bending the rules. In many cases, this adds to the entertainment factor of the game, creating more uproar in the crowd, highlighting long-standing rivalries, giving highly reputed athletes more screen time. However, in baseball, this bending of rules proved to do the opposite- as umpires looked the other way as pitchers took several minutes to simply throw the ball, the duration of games began to increase, peaking at a near 5 hour game in 2020 between the New York Yankees and Cleveland Indians (Frommer). In response to the increasing dead time during games, game attendance and viewership have displayed decreasing fan engagement, as seen in Figure 1, which the MLB has chosen to solve through the implementation of technology to aid umpires- a pitch clock that limited the amount of time pitchers both pitchers and hitters had on their respective blocks (Frommer). There has not yet been a sufficient amount of MLB games to gauge the success or failure of this clock, but history is often reflective of the future, and the implementation of a clock in football yielded overly positive results in terms of scoring, attendance, and overall engagement (Frommer). Baseball is a favorite American pastime whose history began far before technology was anywhere near what it is today, and is therefore deeply rooted in tradition, in contrast to American football, so, the backlash against the integration of technology is significantly higher. Competitive sports though is undeniably a major sector of the entertainment industry, and as the rest of this industry continues to industrialize and incorporate modern technology, it too must follow along to successfully retain its fans.

# data[].attributes.headline
Source: The Usage of Video Assistant Referee and its Impact on Players' and Fans' Emotional Experience of Football

# data[].attributes.subject
Source: The Usage of Video Assistant Referee and its Impact on Players' and Fans' Emotional Experience of Football

# data[].attributes.body
As a part of the entertainment industry, the successful broadcasting and ability to retain viewer attention of competitive sports relies largely on the ability to instill strong emotion. Simply watching sports has been an activity people use to freely shout, celebrate, even cry. The addition of technology into watching sports has been a controversial topic in this aspect of spectatorship, with many reporting that the addition of instant replay and Virtual Assistant Referees (VAR), referees that use a complex system of cameras to rewatch and analyze ambiguous plays to advise referees on the court/field, have changed the flow of and emotions derived from watching games. Some fans report that VAR “adds to the emotional experience… it creates more anxiety. But I believe that is a good anxiety”(South 34), or that they “went through a roller coaster of emotion” (South 35). A general consensus though seems to be that the implementation of VAR has “[created] a different dimension of emotion” where hope and doubt have become the more prominent emotions felt instead of instant joy and disappointment (South 34). One might argue that although such instant feelings are more memorable, hope and doubt are the emotions that keep eyes glued to the screen and build suspense to make the release of joy or disappointment all the more rousing. Even for fans that might see this change in a negative light, there is another consensus that technology is not the issue; “It’s how the technology is being utilised” (South 39). While VAR specifically may not be the answer in terms of integrating technology into competitive sports, people do seem to be open to its more general idea.

# data[].attributes.headline
Category 2: Reducing Controversy that uphold oppressive stereotypes

# data[].attributes.subject
Category 2: Reducing Controversy that uphold oppressive stereotypes

# data[].attributes.headline
Source 1: Umpire Ruled out after Serena row

# data[].attributes.subject
Source 1: Umpire Ruled out after Serena row

# data[].attributes.body
Competitive sports has become a sort of intersection between entertainment, activism, marketing, technology, politics, and more. It has also brought to attention the intersectional identity of individual athletes, a notable one being Serena Williams, a legendary name in the world of tennis. Her career took off in 2001 in the Indian Wells tournament, where she faced racism to such an extent that she chose to boycott the tournament for 14 years, then faced her first major controversy in 2004 in the U.S. Open Quarter-final where umpire Mariana Alves made a series of incorrect and disputed calls that cost her the match. Alves’ calls were only determined to be incorrect based on replays, yet the umpire still was able to overrule them. A similar instance occurred again in 2009 with an alleged foot fault that was not recorded. Had the video replay or technology similar to Hawkeye, a virtual line-calling system, been provided the final say, these controversies could have been prevented, and the racist undertones of the hate Serena received for her responding outbursts with umpires may have not culminated in the release of a blatantly racist cartoon released in 2022 following her match with Naomi Osaka.

# data[].attributes.attachment_link.readable_text
NEW YORK -- Mariana Alves, the umpire at the centre of the controversial U.S. Open quarter-final between Jennifer Capriati and Serena Williams on Tuesday, will not officiate at any more matches in this year's event.
At deuce in the opening game of the third set, Portuguese Alves overruled a backhand from Williams, calling it out after it had clearly landed inside the line and been deemed good by the line judge.

"Regrettably, the replay on television showed that an incorrect overrule was made," tournament referee Brian Earley said in a statement.

"A mistake was made and I have discussed the call with Ms Alves. Ms Alves is not scheduled to officiate another match during the 2004 U.S. Open."

Though Alves's overrule was inaudible on television, and Williams said she thought the call was a score mistake, replays proved the ball was clearly in.

Capriati went on to win the match 2-6 6-4 6-4, leaving fellow American Williams claiming she had been "robbed."

"I guess she went temporary insane," Williams said.

"I'd prefer she not umpire at my court anymore. She's obviously anti-Serena."

The eighth-seeded Capriati, who will next play No. 6 Elena Dementieva of Russia, last week called for tennis to try some sort of instant replay.

"I don't need to see the replay," Williams said. "I know my shots. Not only was it in, it wasn't even near the line.

"I'm very angry and bitter right now. I felt cheated. Shall I go on? I just feel robbed," she said, managing to laugh.

"At first, I thought it was another Wimbledon conspiracy."

Sister Venus lost in the second round this summer at Wimbledon after Karolina Sprem of Croatia was mistakenly awarded an extra point in the final-set tiebreaker.

Venus did not contest the call, and chair umpire Ted Watts was tossed out of the tournament.

Good reason

Serena made a point to say she does not argue calls. Only this time, she did -- for good reason, it turned out.

Williams hit a backhand that landed in along the side line, but Alves took it away, and Capriati went on to win the first game of the decider.

TV replays showed the line judge had called it correctly. But as Williams headed back to the baseline to serve, she heard Alves announce: "Advantage, Capriati."

A startled Williams looked up and asked, "What happened?" Then she said to Alves: "That's my point. That ball was in. It's my advantage."

Williams looked up at her family in the guest box, turned around and headed toward Alves.

"No, no, no, no, no," Williams said. "That was my point! What are you talking about? What's going on? Excuse me? That ball was so in. What the heck is this?"

Williams tried to illustrate her point, putting a ball on the court and pointing.

"The ball landed here. That ball was not out. Are you kidding me? I'm trying to tell you: The ball was not out. Do I need to speak another language?"

Answered Alves: "Please calm down."

Capriati stood at the other end, shaking her head.

"I didn't even, like, look at it. It was close. I was just going to do what the umpire said," Capriati told the crowd after the match, drawing some boos and murmurs.

"Believe me, I've had things go against me many times, plenty of times. I deserve to get a call once in a while. One point, I don't think, changed the match."

Said Williams: "Honestly, I began to think, 'OK, well, I'm not going to go for the lines, I'm not going to go for my shots."

Incorrect calls

Later in the final set, replays showed at least two other incorrect calls that went against Williams.

Capriati beat Williams in the French Open and lost to her at Wimbledon. Capriati has never reached the final at Flushing Meadows -- last year, she was two points from making it that far 10 times when she lost to eventual champion Justine Henin-Hardenne in the semis.

In the last set against Williams, Capriati thought about that near-miss.

"There were actually a couple times there where that went through my mind. I was like, 'I'm not going to let this happen again,"' she said. "I think it was a good thing to think about it."

Capriati has won three Grand Slam events and Williams has taken six. For the first time since 1998, Venus and Serena will finish the season without a single major title between them.

# data[].attributes.attachment_link.description
Mariana Alves, the umpire at the centre of the controversial U.S. Open quarter-final between Jennifer Capriati and Serena Williams on Tuesday, will not officiate at any more matches in this year's event.

# data[].attributes.headline
Category 3: Reducing effect and presence of bias

# data[].attributes.subject
Category 3: Reducing effect and presence of bias

# data[].attributes.headline
Source: The Issue of Unconscious Bias in Referee Decisions in the National Rugby League

# data[].attributes.subject
Source: The Issue of Unconscious Bias in Referee Decisions in the National Rugby League

# data[].attributes.body
Variation of the degree to which rules are enforced in competitive sports are a major source of entertainment and engagement, but game officials and fans must remember that these competitions are not simply games to the athletes playing them; they are their career and life, and determine their reputation and future. In order to ensure that these athletes are able to fairly continue in their careers and ensure that fans are able to view a just game, referees should more heavily rely on technology when making calls to avoid the influence of unconscious biases that may be counterproductive to this. Studies examining the number of calls and penalties directed towards home and visiting teams for both rugby and football found that the idea of a home advantage likely is not only perpetuated by the team’s comfort in their environment, but also by referee bias, which also may be influenced by crowd size and density (O’Brien & Mangan). Bias is not limited to home advantage either; related studies have found likely correlation between referee decision and player status/reputation, height, crowd reactions, and even cultural closeness (O’Brien & Mangan). Referees are also only human, and inevitably experience exhaustion sometimes comparable to that of the athletes on field, causing them to fall back to the crowd and popular decision to make decisions. Nonetheless, the implementation of technology able to more accurately officiate games or even identify potentially biased activity may have major implications on bias reduction not only in sports officiating, but also in individual referees who are forced to confront their thought processes behind inaccurate or disproportionate calls/penalties.

# data[].attributes.attachment_link.readable_text
School of Exercise and Nutrition Sciences, Queensland University of Technology, Brisbane, QLD, Australia

Faculty of Business, Economics and Law, The University of Queensland, Brisbane, QLD, Australia

Referees are appointed to be impartial sporting figures. They are trained to provide leadership and guidance, interpret infractions, adjudicate rules, all while maintaining the highest levels of objectivity and sense. However, when decisions are driven by individual heuristics, limited information, context, goal motivations, emotions, time pressures and cognitive load, it can be difficult to discern how and why particular referee judgements are made. In this paper, we draw on data from the major Rugby League competition in Australia between the years 1978 to 2019 to examine whether clubs fare significantly better or worse under particular National Rugby League referees. We examine potential causes that might contribute to the referee effect and ask why, after years of specialist training and game assessments, are rugby league referees, even at the elite professional level, still processing information in preferential ways.

Introduction

Referees are appointed to be impartial sporting figures. They are trained to provide leadership and guidance, interpret infractions, adjudicate rules, all while maintaining the highest levels of objectivity and sense ( Lirgg et al., 2016; Hancock et al., 2018). Their ability to prioritize and process information, at the right time, in order to select the appropriate response from competing task demands is a sign of their dependability and a reflection of their perceptual-cognitive expertise ( Moore et al., 2019). However, when decisions are driven by individual heuristics, limited information, context, goal motivations, emotions, time pressures and cognitive load ( Hill and Windmann, 2014; Osório, 2020), it can be difficult to discern how and why particular referee judgements are made.

Four important streams of work currently characterize the presence of judgement and decision making in sports ( Raab et al., 2019). Influenced by different disciplines such as psychology and neuroscience, any one of these avenues (i.e., economic, social judgement, ecological and cognitive approaches to decision making) could characterize how and why particular referee decisions are made. For example, information gaps have been found to cause incongruity and ambiguity which influences how performers, such as referees, perceive events ( Agarwal, 2020). Decision-making is subject to social influences and performers own unique interpretations, attitudes, hidden biases, predictions and demonstrated preferences ( Thuraisingham, 2013; Rhodes et al., 2017; Agarwal, 2020). Paradoxically, even illusionary memories have been found to impact decision-making during game play ( Hill and Windmann, 2014). Recent research has also reported that biases or shortcut strategies used to manage complex information can arise as a result of performers trying to quickly identify and process fast-paced visual, verbal, and behavioral cues ( Moore et al., 2019; Albanese et al., 2020; Osório, 2020). Because sport referees must make judgments and decisions under time constraints, in complex environments and often under ambiguous circumstances, the inclination or prejudice for or against one person or sporting group would appear to be guaranteed ( Erikstad and Johansen, 2020).

Systematic decisions in favor of certain players or sporting groups, which can result from conscious bias as well as from human error or incompetence, may have significant economic impacts on sport organizations in areas such as ticket sales, prize money, merchandise, broadcast revenue and recruitment ( Agarwal, 2020; Albanese et al., 2020). Examining the potentially tapered focus of professional sport referees also highlights that these performers receive material incentives such as match payments and appointments stemming from match reviews. Given the somewhat opaque and potentially subjective nature of the system for evaluating and compensating referees ( Price et al., 2012), it is difficult to say, without further research, what the precise cost and benefits of biases such as unconscious bias are for referees and the organization itself. Indeed, it should be considered that sport governing bodies might even reward unconscious bias, or at least only provide a weak disincentive for bias related to areas such as home advantage, if bias substantially benefits the business in terms of increased ticket sales, club memberships, and broadcast rights ( Price et al., 2012; Areni, 2014). Answering whether professional referees make biased decisions and understanding the causes that lead to professional referees digressing from their role as impartial sporting arbiters is therefore important ( Dohmen and Sauermann, 2016).

In this paper, we focus on referees operating in the sport of rugby league. We define what we mean by unconscious bias and highlight how a growing body of literature has examined how unintentional predispositions are shaping decision making in sports, before singling out home advantage as the most likely indicator of unconscious bias. Section 2 examines data on home advantage in the National Rugby League (NRL) competition in Australia and shows that even among elite professional referees, home advantage differentials vary widely around the average or expected values. While differences such as these do not necessarily imply bias, the empirical model developed in Section 3, using a series of control variables to decompose the "referee effect" from other factors that contribute to the home advantage points spread, illustrates that while some penalties are obvious, many are the result of referee interpretation. Section 4 considers potential causes that contribute to the referee effect and examines the perverse incentives and moral hazards present in the current system of referee development and asks why, after years of specialist training and game assessments, are rugby league referees, even at the elite professional level, still processing information in preferential ways.

Section 1-Unconscious Bias in Sport Refereeing

What Is Implicit Bias?

Implicit biases are learned stereotypes that are automatic, unintentional, deeply engrained, and able to influence behavior ( Fiarman, 2016; Backhus et al., 2019). They refer to individuals' lack of awareness of the effects of their own actions on other people or social institutions ( Beugr, 2018). Unlike explicit biases, where people are aware of their prejudices and attitudes toward certain groups or events, implicit biases occur below the level of consciousness. Their signature feature is that they represent automatic and unconscious cognitive processes; they are not direct, deliberate, controlled, or intentional self-assessments ( Nosek et al., 2011; Agarwal, 2020). They also illustrate how people's reports of the causes of their behavior can be stated confidently and incorrectly simultaneously ( Nosek et al., 2011). Implicit biases, conversationally referred to as unconscious biases, are therefore problematic to capture and accurately pinpoint because they are hidden and can often be in complete contrast to what people consider their beliefs and associations to be ( Agarwal, 2020).

What Is Bias in Professional Refereeing/What Do We Already Know?

Unconscious bias in the context of professional refereeing may mean that learned stereotypes, deeply ingrained within their beliefs, influence the way in which individual referees automatically engage with players and situations. From this viewpoint, judgement biases could be described as the subconscious overweighting of some aspects of information and underweighting or neglect of others ( Morewedge and Kahneman, 2010), relative to rules and situations. For example, referees have been shown to be persuaded by crowd reactions ( Page and Page, 2010; Erikstad and Johansen, 2020). Racial bias has been found among National Basketball Association referees ( Price and Wolfers, 2010), Major League Baseball umpires ( Parsons et al., 2011), and National Collegiate Women's Basketball referees ( Dix, 2019). Baseball umpires also display how high-status players were rewarded with correct decisions even when their performances were undeserving ( Kim and King, 2014). Similarly, Findlay and Ste-Marie (2004) found that rankings used to determine a Canadian skaters final placement were better when skaters were evaluated by judges who knew of the skaters positive reputation. Researchers have also extensively analyzed other determinants of referee bias such as social pressure from the crowd and media ( Myers, 2014; Webb, 2018), cultural closeness of the referee to the team ( Pope and Pope, 2015; Nezlek et al., 2019), players' height ( Gift and Rodenberg, 2014), and uniform color ( Dijkstra et al., 2018). Although the external validity of the results from these studies on bias remain an open question ( Osório, 2020), they are at least suggestive that unconscious biases may play a pivotal role in shaping referees' evaluations of others, particularly when part of split-second, high-pressure sporting plays.

Singling Out Home Advantage

A large proportion of studies on referee bias have focused on home team advantage ( Burnett et al., 2017; Albanese et al., 2020). The phenomenon by which there is an apparent advantage conferred to the home team by referees is broadly accepted and well documented for a wide variety of sports ( Ribeiro et al., 2016). Three major determinants have been postulated to cause the home advantage effect: crowds (e.g., crowd partisanship, crowd noise), travel fatigue and familiarity with the stadium ( Pollard and Pollard, 2005). Of these, situational influences related to crowd effects such as density and home crowd noise have been substantially reviewed ( Page and Page, 2010; Myers, 2014; Dosseville et al., 2016; Erikstad and Johansen, 2020). When taken together, these studies suggest that referees might be biased by crowd reaction, favoring the home team, and thereby contributing to home advantage ( Dosseville et al., 2016). In essence, the intuitive or subconscious overweighting of some aspects of information such as the strict enforcement of laws or crowd noise and underweighting or neglect of other aspects to ensure the flow of the game might simply reflect a referee's unconscious bias.

Section 2-The Empirics of Unconscious Bias in Rugby League

This section draws on data from the major Rugby League competition in Australia between the years 1978 to 2019. That period saw the development of Rugby League in Australia from State and City based competitions to a quasi-National and International competition under the auspices of the National Rugby League (NRL). During this period, over 70 persons have acted as elite professional referees ranging from a select group ( n = 5) who have refereed over 350 games to a group of 30 with <50 "A Grade/NRL" professional games.

There are a number of commonly held beliefs in professional Rugby League that have almost reached axiomatic status such as "forwards win big matches"; 'a team cannot win a premiership without a strong spine" and "you need to lose a premiership to win one" but, probably the most enduring axiom is "home team advantage." As outlined earlier, the potential sources of home team advantage are several including greater crowd support, reduction in travel time, crowd density and familiarity of surroundings. The role of the referee in the perpetuation of home advantage, however, is open to debate. For example, Boyko et al. (2007) found evidence that referees on average do tend, in an implicit sense, to enhance home team advantage; that is home teams tend to score more points than their expected value and to surrender less than their expected value of points against. Similarly, Unkelbach and Memmert (2010) demonstrated the possible contribution of crowd noise to the home advantage via soccer referee's decisions. Both of these studies, of course, depend on the generalized existence and quantification of home advantage.

A number of studies have attempted to measure the extent of home advantage within the NRL. McGuckin et al. (2015) for example defined home advantage as teams winning more that 50% of home games. However, there are a number of limitations to this type of definition. In particular, home advantage does not necessarily equate with winning. Poorly performing teams who lose but score more points or have less points scored against them at home are still beneficiaries of home advantage even though this does not translate into wins. Certainly, if home advantage is important, you would expect teams to have a higher likelihood of winning at home but a better indicator of home advantage, which allows for the possibility of poorly performing teams still enjoying some form of home advantage would appear to be in the points scored and conceded against these teams in away games. On that basis we could define home advantage in attack as:

Using these formulas, the existence of home advantage in both attack and defense is shown in Table 1.

Table 1. Average value of home advantage in attack and defense in the NRL.

Table 1 examines the percentage difference between points scored by a team at home and points scored away and the percentage difference between points conceded at home and points conceded away. For every team there appears to be an advantage playing at home. All teams reported more points scored and less points conceded when playing at home. It might be thought that the more successful teams in terms of winning percentage would enjoy the largest home advantage, but, on first inspection of the data in Table 1, this does not appear to be the case. In terms of attack, the Canberra Raiders (25%) and the Brisbane Broncos (24%) have the highest attacking home advantage, while the Melbourne Storm (34%) and the Newcastle Knights (34%) have the highest percentage home advantage in terms of points conceded at home. While there can be a number of ways of quantifying home advantage, the data in Table 1 clearly supports its existence. The next task is to determine the role in the continuance of home advantage played by unconscious bias from NRL referees. Moreover, it is likely that the "referee effect" in home advantage is not homogeneous but rather varies across referees. The extent of this variation is under-researched, but it would be expected that such variables as experience would be an important determinant. Yet, consideration of the data shows that even among elite professional referees, considerable differences in performance emerge. Table 2 examines the win rate of NRL teams under a group of experienced NRL referees. To allow for greater consistency, the professional referees used in this sample were required to have refereed each team at least 10 times. The data in Table 2 indicate a considerable degree of variation in win ratio per team when controlled by different referees and some notable deviations from the average win rate for each team.

Table 2. Winning percentages for NRL teams under different leading professional NRL referees.

A number of points arise from consideration of Table 2. The Melbourne Storm win, on average almost 65% of their games, but even after a ten-game average, their winning percentages under our sample of experienced referees range from 33 to 73%. The Warriors win 55% of the time under Referee 2 compared to their average win of 47.67% under all referees; Brisbane does better under Referee 2 and Referee 3 and the Tigers fare significantly better under Referee 7 (63%) compared to their average winning ratio of 47%. None of this, of course, necessarily reflects unconscious bias but it is an indication that even among the most experienced senior NRL referees, their interpretations of rules and reaction to the playing styles of various teams differs. This potentially impacts on team performances and even results. It is in these areas that unconscious bias is likely to surface.

Section 3-Decomposing the Referee Effect from Other Factors

The variations in outcomes by referees shown in Table 2 are strong indicators of inconsistencies in refereeing, even at the elite or highly experienced professional level but not necessarily as a result of unconscious bias. For this to be established we would need to show that match outcomes, particularly relating to the home team, are impacted by consistent, albeit unconscious decisions by the referee. There seems little doubt that refereeing decisions can impact the outcomes in NRL matches but, how important are they in aggregate and are they the result of unconscious bias? One way of moving toward this conclusion would be to quantify the extent to which referee decisions do make a difference to match statistics.

To examine this issue, points difference in a home game is used as a dependent variable and regressed in a random effects regression format against a number of variables thought to influence points differentials in favor of a home team, including decisions of the referee in terms of awarding penalties and ordering scrums. The regression took the format:

Where y 3jt = scrum difference and penalty difference (separate equations) represent the difference in scrums awarded to home and away teams across all teams and between the years 1978-2019 and act as the dependent variable. The independent variables, across the panel 1978-2019 are:

* Penalty difference, the difference in penalties awarded to the home team minus the penalties awarded to the away team / Scrum difference, the difference in home team fed scrums.

* Crowd, represents the size of the crowd at the particular game.

* Crowd density, represents the ratio between maximum capacity and actual attendance at the particular game.

* A set of control variables which adjust for the quality of the team; EHGF (expected home points for), EHGA (expected home points against), EAGF (expected away points for) and EAGA (expected away points against).

We expect, given the influence of the control variables, that the direct indicators of referee input (and impact) would come through scrum and penalty differences, particularly penalty differences. The control variables chosen, EHGF and EAGA, do pick up variations in opposition team quality, albeit imperfectly. However, the value of using these variables is that they enable direct comparison with results obtained in earlier studies of referee impact such as Burnett et al. (2017). With data being drawn between the years 1978 to 2019, the impact of the two-referee system was also considered. For example, it would be possible to test for differences between the one and two referee systems by using slope dummies and a Chow test but even if this proved to be significant, it would impact on the intercept of the estimating equation rather than the slope of coefficient values on which our results are based. While it is also possible that referee appointments might be influenced by team match-ups or the quality of different games, we feel that the crowd density variable picks up some of these "big game" appointment effects. We did not specifically consider the distinction between day and night games as given the database, working with this variable would be difficult to accurately calculate. The results are shown in Table 3.

Table 3. Random effects regression with points difference as the dependent variable.

In Table 3, all the chosen variables are significant. The coefficient on Scrum difference indicates that for every scrum win advantage, the home team adds an extra 0.2 of a point while for every additional penalty advantage they receive boosts the expected points difference by almost 0.5 of a point. Yet, in many ways, rugby league scrums are automatic and result from errors in play and are therefore quasi-independent from individual referee decisions. Similarly, some penalties are obvious, but many are not, as they are the result of referee interpretation. It is in this area that unconscious bias is likely to emerge. Crowd size does not appear to be an important factor in home team points difference, but crowd density does, reducing home advantage by almost 8 points. The reasons for this are varied. One explanation is that a near capacity crowd (i.e., high crowd density) indicates an important match or show piece game and referees are likely to be attempting to stay neutral and more conscious of their performance. Another contributing factor is that in a show piece game the teams are likely to be more evenly matched, thereby reducing the scope for home advantage.

The results in Table 3 show that at least part of the home advantage (home team points difference) results from referee decisions. A related question is whether this referee contribution to home advantage diminishes with the experience of the referee. This is examined in Table 4. In this test, the referee list was initially divided into 5 tiers ranging from tier 1 (referees who have refereed over 300 games) to tier 5 (referees with <50 games). However, using tier 1 referees as the default, we were unable to find any significant differences in their impact on points difference. This seemed surprising, given it would appear likely that experience levels would be a significant point of departure in referee performance. This led to an aggregation of the referee tier dummies. In Table 3 we compare tier 1 referees against the rest (an amalgamation of tiers 2-5) and observe significant differences.

Table 4. Random effects regression-tier 1 referees against an amalgamation of tiers 2-5.

As with Table 3, all variables are significant, except the crowd numbers. Most importantly for our analysis, the tier other dummy is significant and suggests that the home advantage points difference is increased under less experienced referees when compared to the tier 1 referees. In other words, the other factors that contribute to home advantage become more important under less experienced referees. This may indicate the less experienced referees are less successful in dealing with home crowd noise and other factors that potentially influence referee decisions.

Discussion

This paper found that in the National Rugby League, when observed over a number of trials, clubs fare significantly better or worse under particular referees. First, we demonstrated that clubs consistently have different outcomes in terms of result and points scored under different referees, even when those referees are the most experienced senior referees in their field. Second, it has shown that referees influence points scored in home games (and therefore home advantage) through their decisions over penalties and scrums. Lastly, it has been shown that home advantage is increased under less experienced referees. Taken together, the results show that the search for consistent and objective rulings in refereeing decisions is still some way off. We also believe that these inconsistencies represent unconscious bias on the part of referees in the way they interpret and police particular aspects of the game and their reaction to team styles, particular players, and unfolding on-field events.

Controlling professional sporting events, however, is not an easy task and referees will make mistakes. In an age of increasing electronic surveillance these mistakes are highlighted and as a result, extensive efforts are made by sport organizations to educate and train referees to provide consistent and objective rulings. However, biases can be hidden in very complex and strategic ways, and referees, even at the experienced level, are rational agents who can learn how their organization functions and adjust accordingly, which makes detection difficult ( Agarwal, 2020; Osório, 2020). Thus, while NRL referees might have honorable intentions, the reality is that their judgement behaviors are the subliminal product of both the situation or context (e.g., organizational review processes, specialist training, positioning, crowd effect, stadium, emergent player actions) and the person themselves (e.g., rule knowledge, individual motivations, background, memory structures, recollections). Moreover, under the revised one-referee system, when NRL referees are regularly at the heart of stormy discussions which provide a substantial source of material for both media and fans ( Dohmen and Sauermann, 2016), determining how and why some referees still stress some aspects of information to the neglect others would seem an even more important factor for organizations to try and resolve.

It will be a challenge for sporting bodies to reconcile educational processes with how referees subconsciously overweight some aspects of information to the neglect of others. We would further argue that a re-conceptualization of the expert sport referee (i.e., consistent performer who displays minimal mistakes in judgment) requires a potential paradigmatic shift from the expert referee as product, to the evolution of referee expertise as always becoming, continually learning, a never-ending journey of ongoing professional discovery ( Turner et al., 2012; O'Brien and Rynne, 2020). Organizations that continue to be preoccupied with problems and deficit run the risk of missing data related to how learned stereotypes that are automatic, unintentional, and deeply ingrained influence the way in which sport referees react and behave in the presence of crowds and crowd noise. In this regard, perhaps the solution is how empirical and experiential knowledge can be better brought to life or transformed into learning experiences that authentically address the development and performance preparation of professional referees. This transcending the individual as a decontextualized 'unit of data analysis', we believe, reinforces the underlying messiness of performer-environment systems and the growing idea that different approaches to developing referees in their role are required.

Data Availability Statement

The data analyzed in this study is subject to the following licenses/restrictions: Dataset belongs to University of Queensland. Requests to access these datasets should be directed to katherine.obrien@qut.edu.au.

Author Contributions

KO'B contributed the abstract, introduction, Section 1-Unconscious bias in sport refereeing, and discussion. JM contributed Section 2-the empirics of unconscious bias in rugby league and Section 3-decomposing the referee effect from other factors. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by QUT Women in Research Grant.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

References

Beugr, C. D. (2018). "The neural basis of emotions and unconscious bias," in The Neuroscience of Organizational Behavior (Northampton, MA: Edward Elgar), 159-177.

# data[].attributes.attachment_link.title
The Issue of Unconscious Bias in Referee Decisions in the National Rugby League

# data[].attributes.attachment_link.description
Referees are appointed to be impartial sporting figures. They are trained to provide leadership and guidance, interpret infractions, adjudicate rules, all while maintaining the highest levels of objectivity and sense. However, when decisions are driven by individual heuristics, limited information, context, goal motivations, emotions, time pressures and cognitive load, it can be difficult to discern how and why particular referee judgements are made.

# data[].attributes.body
For those that are more hesitant to accept or are even completely opposed to the idea of technological integration, perhaps their opposition stems from ideals adjacent to the ones present in Neo-Luddism established to combat its integration altogether in an effort to preserve tradition and our humanity. Figure 2 depicts Ned Ludd, the name given to the nonexistent leader of the Luddite movement in the early 1800s. This original movement was characterized by its resistance to the application of and advancement of technology largely out of concern of unemployment and safety. Today, this concern has extended to environmentalism and community preservation. In the context of sports and from the perspective of fans, Luddite ideals might apply when in the interest of preserving the human aspect of games, or could even arise in the argument that refereeing is an art of sport of its own meant to be carried out by people and people alone. Yet, not only does it seem immoral to reduce accuracy in officiating for the sake of engagement as it comes at the potential cost of athletes’ careers, but also ineffective as even fans themselves engage in the technological revolution the modern world is undergoing. Even in the specific instance of the pitch clock in baseball which has faced backlash in its variance from tradition, which baseball is deeply rooted in as America’s National Pastime, preventing the integration of this simple technology for umpires is counterproductive as seen in the formerly declining attendance rates. It is also necessary to mention that with only human officiators present, controversial calls could be disputed for years to come, causing negative emotions to only continuously boil and potentially drive people away from spectatorship due to the animosity associated with it. Technology offers a more clear end to such controversy that result in more assured feelings of joy and relief, or disappointment that drive fans to watch games again to pursue that rush of dopamine.

# data[].attributes.headline
Source: Paradoxical Tensions Related to AI-Powered Evaluations Systems in Competitive Sports

# data[].attributes.subject
Source: Paradoxical Tensions Related to AI-Powered Evaluations Systems in Competitive Sports

# data[].attributes.body
That is not to say that Luddite ideals are completely in the wrong though. Preserving at least part of the “human” aspect of competitive sports is essential to maintaining fan engagement and ensuring that the same levels of emotion and intensity are felt. It is possible for technology and AI to be “too exact”, leaving too little room for imagination by fans or being too harsh on athletes, with the mistakes made by athletes and referees leveling each other out so the idea of “perfection” really does seem attainable (Mazurova, et al.). A major aspect of competitive sports is its ability to test and boast the capabilities of humans, and the excessive or sole use of methods that increase technology and eliminate all mistakes in judging, refereeing, etc. may be seen to counteract the purpose so many people flock to these sports. Many fans that choose to become very involved in a sport/sports also enjoy participating in analyzing games or competitions as referees do, and technology or AI programs that display explicit data on athlete stats, form, distance, accuracy, etc. that referees are often not able to supply due to a common reliance on intuition and experience rather than hard data that is difficult to produce in the high stress environment of a game give these fans a chance to engage in competitions on a level not able to be achieved before (Mazurova et al.). Fans themselves are able to officiate in a way and subsequently feel more included in the game, allowing them to more fully immerse themselves in the sport and experience emotions associated with the game to a greater extent.

# data[].attributes.attachment_link.readable_text
Paradoxical Tensions Related to AI-Powered Evaluation Systems in Competitive Sports

Abstract

Judging in competitive sports is prone to errors arising from the inherent limitations to humans' cognitive and sensorial capabilities and from various potential sources of bias that influence judges. Artistic gymnastics offers a case in point: given the complexity of scoring and the ever-increasing speed of athletes' performance, systems powered by artificial intelligence (AI) seem to promise benefits for the judging process and its outcomes. To characterize today's human judging process for artistic gymnastics and examine contrasts against an AI-powered system currently being introduced in this context, an in-depth case study analyzed interview data from various stakeholder groups (judges, gymnasts, coaches, federations, technology providers, and fans). This exploratory study unearthed several paradoxical tensions accompanying AI-based evaluations in this setting. The paper identifies and illustrates tensions of this nature related to AI-powered systems' accuracy, objectivity, explainability, relationship with artistry, interaction with humans, and consistency.

Similar content being viewed by others

1 Introduction

Human judging plays an important role in sports: figure skating, gymnastics, snowboarding, and many others (Stefani, 1998). Well-timed, accurate, and reliable information and evaluations are key factors that can contribute to more reliable judging and, indirectly, better athletic performance (Harding & James, 2010). However, human-based judging is susceptible to error due to a host of factors, from fatigue to various biases of judges (Perederij, 2013). Research attests that competitive-sports judging is influenced by judges' prior knowledge and values, earlier experience, training, iterative reflection, and cognitive and sensory limitations (Plessner & Haar, 2006). This presents a problem: "In sport, the accuracy of the results of a game or competition is important in order for the sport to be deemed valid, but in many cases in sport, humans cannot always provide reliable results" (Kerr, 2018, p. 116). Moreover, the time consumed by human judging can render a competition more tiring for athletes and less spectator-friendly for live and televisual audiences. Some shortcomings of human judging can be especially costly, leading to judging scandals, retarding athletes' development, and decreasing the sport's overall attractiveness.1 In summary, they put the legitimacy of a human-judged sport at risk.

To overcome issues of subjectivity and bias in judging, improve the decisions' objectivity and accuracy, and expedite the judging process, AI-powered systems have been introduced in recent years as an aid to refereeing, performance assessment, and judging for sports.2 Such changes in officiating or judging are often met with objections and controversy (Kolbinger & Lames, 2017). As a revelatory case in point, we studied the relatively complex context of AI in artistic gymnastics, wherein Japanese technology company Fujitsu and the International Federation of Gymnastics (or Fédération Internationale de Gymnastique, FIG) are collaborating to develop an AI-powered judging system.3 Such systems hold potential to circumvent human sensory and cognitive limitations and to offset or eliminate human biases by using a combination of AI technologies to support (or even replace) the actions of human judges (Benbya et al., 2021). A core element of Fujitsu's computer-vision-based judging-support system (JSS) is a three-dimensional computer-generated image of proceedings that is examined in light of set definitions of gymnastics elements, for determination of a performance score.

We conducted a two-year exploratory case study examining the introduction of the Fujitsu system and how the various stakeholder groups perceive it. With AI-powered performance judgement remaining in its infancy, the possible positive and negative consequences are not obvious yet. Hence, we examined the complex form of human judging involved and how an AI-powered system might ameliorate or compound the issues posed by such human-based systems. Pursuing clarity that could inform the development of AI-based performance-judging technology, we employed the analytical lens of the paradox to identify, understand, and explain the tensions experienced by particular stakeholders (Dubé & Robey, 2009; Smith & Lewis, 2011) in the introduction of an AI-powered system for sports. We formulated the research question accordingly: " What are the paradoxical tensions related to the use of an AI-powered system for competitive-sports judging? "

To address this question, we conducted an in-depth case study of the application of the aforementioned AI-powered system for judging in artistic gymnastics. Taking an approach similar to one employed in prior research (Calabretta et al., 2017), we used several rounds of coding to develop a data structure (Gioia et al., 2012) that covers six "paradoxical tensions": 1) accurate AI is too exact, 2) ostensibly objective AI can be biased, 3) black-boxed AI provides a sense of explainability, 4) AI-based judging for artistic gymnastics cannot judge artistry, 5) much AI intended for humans lacks human interaction, and 6) consistency requires AI's adaptability. We believe that acknowledging these tensions should provide a step toward better understanding and, thereby, fundamentally better design of AI-powered systems' introduction for supporting or replacing humans' evaluations (Benbya et al., 2021).

We begin with a literature review discussing the use of electronic support for judging in sports, then turn to the use of AI for (expert) evaluations. The section concludes with a review of the notion of paradoxical tensions and how prior research has addressed them with regard to technologies and AI in particular. Then, in Section 3, we outline our methodology, describing the collection and analysis of data. Further vital background is supplied in the fourth section, dealing with the case setting of artistic gymnastics and details of the as-is situation of human judging alongside what the use of an AI-powered system is expected to be like. Then, we proceed to present the paradoxical tensions identified through our empirical study. The body of the paper concludes with a discussion of our findings' implications.

2 Literature Review

A review of the scholarly work on the key concepts utilized in our research aids in positioning this study. The key branches of literature have examined electronic judging systems in sports, the use of artificial intelligence for expert evaluation in particular, and the notion of paradoxical tensions.

2.1 Electronic Systems in Judging of Sports

Humans' processing of information is inherently slow, and this bottleneck prolongs gymnastics competitions. The protracted decision-making also makes the sport less inviting for spectators. Moreover, the same judging panel must evaluate all participants in any given competition, for the sake of consistency, though this presents the risk of panel members being tired and less able to concentrate by the end of a 12-h session, for instance (Perederij, 2013). Anecdotal evidence suggests that another time-related factor is at play too: gymnasts who perform early in the day are evaluated more harshly and hence have lower chances of advancing in the competition (Mazurova et al., 2021). Moreover, various studies have indicated that gymnastics judges make errors connected with the people involved or other aspects of context (Flessas et al., 2014; Kerr, 2018; Plessner & Schallies, 2005). The former may entail scoring that is biased in relation to the nationality, body morphology, or reputation of the gymnast (Duong, 2008; Heiniger & Mercier, 2018). Further contextual factors may be more mundane - e.g., the judge's vantage point or viewing angle (Plessner & Schallies, 2005) - but no less relevant. Such factors have given rise to several controversial moments in sports history related to judging decisions (Dumoulin, 2020).

Electronic systems can assist in overcoming errors of human judgement (related to accuracy, fairness, validity, and reliability) and, thereby, preventing inquiries and complaints by coaches and athletes (Can et al., 2011; Omorczyk et al., 2015). For instance, judges may exploit technologies such as video replay (slow-motion and time-lapse in particular) and time measurements (Omorczyk et al., 2015). Also, if the athletes are fitted with wearables, sensor data from their movements can inform the judging (Harding & James, 2010). The use of such support technologies may be expected to improve judging systems' reliability significantly and to reduce both conformity bias and arithmetic errors in the scoring of athletes' performance.

Many suggest that the use of electronic judging systems in sports, gymnastics in particular, possesses potential to make a positive impact on the technical development of performance evaluation, through greater objectivity and clarity of the judging process and outcomes (Can et al., 2011; Ferger & Hackbarth, 2017; Omorczyk et al., 2015; Taymazov et al., 2013). However, researchers have expressed concerns at the same time, in that no technology can interpret and assess the myriad of situations that competitions present, especially with regard to athletes' artistry and creativity (Leveaux, 2010). Also, various stakeholders strongly oppose technology in the judging process on such grounds as freedom, individuality, and aesthetic focus (Harding et al., 2008). People are attached to these values and do not want to see them eroded through technology. Finally, there are concerns about job losses: human judges might get replaced altogether at some point (Mazurova et al., 2021).

2.2 AI for (Expert) Evaluations

A key application of AI for business lies in generating insight and making decision in (narrow, thus far) task domains (Benbya et al., 2021; Raisch & Krakowski, 2021), since this technology "enables the creation of new information and predictions from data" (Shrestha et al., 2019, p. 67). Several research themes related to human versus AI decision-making have been identified, among them accountability, transparency, biases, ethics and associated values, efficiency, replacing intuition with rationality, and adaptability (Benbya et al., 2021). Of particular interest for the present paper is the specific field of decision-making related to expert evaluations and the assessment of human performances. Much of the recent work on the use of AI technology in this arena has been conducted in a human-resources/organizational context.

For instance, in the framework they synthesized for AI in the context of organizational control. Kellogg et al. ( 2020) found two mechanisms for algorithmic evaluation: recording and feedback (based on fine-grained behavior) and rating and ranking (based on aggregated quantitative and qualitative data). Also examining the literature to pinpoint new affordances of AI technology in this domain, they identified comprehensiveness (use of a variety of data), instantaneity (high velocity), interactivity (interfaces for participation), and opacity (abstraction connected with technical literacy). Finally, they observed resistance against algorithmic control among workers, who expressed concerns related to data accuracy, surveillance and loss of privacy, algorithms' discrimination, and absence of non-algorithmic assessment. Among the tactics of resistance emerging among workers, for which the authors coined the term "algoactivism," were ignoring algorithm recommendations, obfuscating data, and hacking the algorithm (Kellogg et al., 2020).

In another human-resources context, van den Broek and colleagues (van den Broek et al., 2021) examined the introduction of a machine-learning-based system aimed at better hiring decisions. The authors found that AI systems "promise to mitigate the human biases, inefficiencies, and path-dependencies that have plagued experts' work for decades, discovering 'truthful' knowledge on their own instead" (p. 3). While there is a common belief, accordingly, that AI can yield superior insights efficiently and consistently, the authors concluded that it remains crucial to keep domain experts "in the loop," via a human-AI hybrid system. In particular, they noted that the various developers and domain experts, as they engage in the AI development process's distinct phases, reflect on several factors: when constructing training data, they reflect on AI's objectivity and experts' bias; when building a model, they consider the novelty of AI and the criteria for the choice of experts; and when using a model, they reflect on AI's efficiency and practical utility (van den Broek et al., 2021). Such reflections prompt cycles of mutual learning, to exclude human experts' knowledge for purposes of independence and to include it for its relevance. The end result is a human-AI configuration that, while not purposefully designed as a hybrid, emerges through this dialectical process.

2.3 Uncovering Paradoxical Tensions Related to Technology Use

Lack of conceptual clarity and consistency characterizes research on paradoxes across contexts (Smith & Lewis, 2011). In line with recent paradox research in the management, organization studies, and information systems (IS) fields (Calabretta et al., 2017; Raisch & Krakowski, 2021; Wimelius et al., 2021), we draw on the work of Smith and Lewis ( 2011) on paradoxical tensions, which they define as consisting of "contradictory yet interrelated elements (dualities) that exist simultaneously and persist over time; such elements seem logical when considered in isolation, but irrational, inconsistent, and absurd when juxtaposed" (2011, p. 387). The two essential elements of a paradoxical tension are contradiction and interdependence - it is their combination that extends beyond a mere tradeoff perspective, to a systemic relationship (across time and space) (Schad et al., 2016). In organizational and other contexts, paradoxes are fueled by technological advancements, growth in complexity, and diversity of perspectives (Lewis, 2000; Raisch & Krakowski, 2021).

Foundational management-science literature highlights identifying and exploring paradoxes as potentially fertile ground for theory development or a starting point to refinement (Lewis, 2000; Poole & van de Ven, 1989; Smith & Lewis, 2011), pointing out that contradictions aid in challenging assumptions and expectations of consistency, stimulate new insight, force creative thinking to emerge, and reveal underlying mechanisms (Majchrzak et al., 2013; Robey & Boudreau, 1999). In practice, paradoxes are considered crucial for explaining dramatic change; hence, rather than stamp them out, managers are advised to "embrace paradoxical thinking as a stimulus for more complex and creative action" (Robey & Boudreau, 1999).

A few scholars have explored specific paradoxes and tensions connected with the use of AI (Benbya et al., 2021). One of these is Moravec's paradox, which articulates the fact that AI, while able to handle various cognition-related tasks that humans deem to require adult intelligence, cannot perform some of the tasks (e.g., more social and emotional ones) that a one-year-old human baby can complete (Dellermann et al., 2019). Another relevant paradox, referred to as Polanyi's paradox, involves inability to transfer knowledge from humans to machines and vice versa (Brynjolfsson & Mcafee, 2017). Initially, the core problem tackled in computing was how to codify tacit knowledge of humans, but focus has shifted to the problem of not knowing how the AI arrives at any given decision or outcome (Brynjolfsson & Mcafee, 2017). In other work, Raisch and Krakowski ( 2021) examined the dual-application paradoxical tension related to AI (automation-augmentation): a line cannot be drawn between AI taking over humans' tasks and AI collaborating with humans to perform a task. While many handbooks for practitioners recommend prioritizing augmentation over automation, these two authors argue that focusing on just one of the two in the management domain produces negative-reinforcement cycles. They argue that instead taking a perspective that comprises both applications leads to virtuous cycles of benefits for organizations and society as a whole (Raisch & Krakowski, 2021). Similarly, van den Broek et al. ( 2021) have argued that human-AI hybrids offer great value in the context of knowledge systems, for overcoming the tension related to producing knowledge that is both independent and relevant.

In prior work, technology-related paradoxical tensions have been recognized for their impact not just at the level of functioning or productivity but also at that of user emotions, from joy, empowerment, and belonging for positive experiences to anger, fear, and depression for negative ones (Jarvenpaa & Lang, 2005). Once a technology paradox has been identified, it can be tackled. An appropriate response to the tension, while seldom obvious or easy, is considered critical for the success of technology-related initiatives (Wimelius et al., 2021). In broad terms, there are two approaches to coping with paradoxical tensions: avoiding them, by minimizing or abandoning the technology's use, and confronting them, by bringing understanding of the technology's features to bear for changing how it is used (Dubé & Robey, 2009; Jarvenpaa & Lang, 2005; Lewis, 2000).

The discussion above suggests that paradoxical tensions are likely to emerge with AI-powered judging systems' implementation in gymnastics. In the absence of studies specific to tensions connected with such systems for competitive sports, we undertook an exploratory study in this particular context. To explore and identify foreseeable contradictions and interdependencies, then formulate them as clear paradoxical tensions, our empirical research applied the qualitative approach described below (per Walsham, 2006).

3 Methodology

Given the exploratory nature of our research, into a still poorly-understood phenomenon - paradoxical tensions related to AI-powered systems in the judging process - a case-study method is especially appropriate (Siggelkow, 2007;Walsham, 2006). In addition, we adopted a qualitative orientation with a predominantly interpretive stance (Klein & Myers, 1999). For our case study, we ascertained that a relevant and revelatory context for tackling our research question (Yin, 2013) would be one with a transition in progress from human-based judging to an AI-powered system. International competitive artistic gymnastics offered a suitable setting of this nature, thanks to its current process of transition. Further, we employed purposeful sampling to support a relevant, information-rich empirical setting and select suitable informants (Patton, 2002).

3.1 Data Collection

We applied the following criteria in selection of the case and data sources for our study: 1) informants should be stakeholders affected by the transition to an AI-powered system (i.e., judges, gymnasts, coaches, technical directors and representatives of international and national artistic-gymnastics federations, and fans), and 2) they should be involved with international competitions at senior level (i.e., the European Championships, World Championships, and Olympic Games), the first competitions envisioned for the AI-powered system's deployment. Since we aimed to take diverse viewpoints into account, we also interviewed Fujitsu representatives whose work dealt with developing and promoting the system in question. This input rounded out the picture with the perspective of commercial players.

We conducted semi-structured interviews with 52 informants, from May 2019 to March 2021. The breakdown of the stakeholder group is as follows: 20 international judges, 11 gymnasts competing at the international level, eight coaches of international teams, two technical directors with national federations, two FIG representatives, two representatives of Fujitsu, and seven artistic-gymnastics fans. In the interests of our informants' anonymity, each has been assigned a pseudonym, as shown in Table 1.

Table 1 Informants in the case study

3.2 Data Analysis

The research team analyzed all of the interview material, with support from software designed for qualitative data analysis (ATLAS.ti). To display the evidence for our assertions, demonstrate a systematic way of gathering and analyzing the data, and make the data's presentation more structured, we used the Gioia method to produce a representation of the first- and second-order data analyses (see Appendix 2). Organizing the data via two orders of categories allows one not only to "facilitate their later assembly into a more structured form" but also to "enhance qualitative rigor" of the research; here, the first-order analysis represents "analysis using informant-centric terms and codes" and the second-order one employs "researcher-centric concepts, themes, and dimensions" (Gioia et al., 2012, p. 18). This technique allowed us to demonstrate the links between the data and indicate the main paradoxical tensions related to the introduction of an AI-powered judging system.

With the first-order concepts, we highlighted codes that reflect informants' opinions. The process, which resembled an open coding technique, resulted in 96 distinct codes reflecting interviewees' ideas and opinions (Appendix 3 presents a summary of the codes with relevance for our study and their related content). This coding process utilized cross-reading and comparison of the interview transcripts. The questions' topic-based organization in our interview-topic guide facilitated summarizing the participants' opinions, perceptions, and predictions for both of the judging systems and carrying out their comparative assessment, generally and by informant role. In the interest of validity and reliability, researchers took turns checking the results of the data analysis (Patton, 2002).

We chose the first-order constructs conscientiously to use the informants' terms, not ours; this helped the concepts reflect their points of view. At this stage in the data analysis, we made little attempt to distill wider categories. As the research progressed, for formulation of the second-order constructs (themes), we started seeking intergroup similarities, correlations, and patterns among the many codes. This process was similar to that of axial coding. We formed 17 code groups or categories, then assigned them phrasal descriptors and attempted to analyze them for their description and explanation of the phenomenon under examination. In the second-order analysis, we sought higher-level perspectives, necessary for informed theorizing aligned with themes. After this, we assessed the possibility of distilling the emergent themes even further, into aggregate dimensions, whereupon we uncovered paradoxical tensions. Together, these elements form the building blocks of the data structure presented in Appendix 2.

In parallel with the data-gathering work and after the initial stages of analysis, we iterated between the emerging dataset, themes, concepts, and dimensions and, on the other hand, the relevant literature. At this phase in the process, "the research process might be viewed as transitioning from 'inductive' to a form of 'abductive' research, in that data and existing theory are now considered in tandem" (Gioia et al., 2012). In this, we faced the inevitable issue of authors varying in their interpretation of some informants' comments and chosen terms. Therefore, as the data analysis neared its completion, we reviewed the source material, engaged in group discussion, and reconciled divergent interpretations, thereby reaching agreement on how to code various terms or phrases. This refinement to our analysis enabled clearly identifying the main challenges and paradoxical tensions related to implementing a new, AI-powered judging system in the gymnastics context.

4 Background on the Case: Competitive Judging in Artistic Gymnastics

The disciplines in men's artistic gymnastics

Women's artistic-gymnastics disciplines

4.1 Human Judging in Artistic Gymnastics

Since the same set of judges evaluates all gymnasts in a given competition, judging panels for artistic gymnastics sometimes are active for stretches of 10-12 h on several consecutive days. They follow an elaborate scoring system created by the FIG, the organization that oversees World Championships competitions and the Olympic Games.

In this system, known as the Code of Points (CoP), the various skills shown in a routine get assigned particular numeric values.4 A gymnast's final score is composed of a Difficulty (D) score and an Execution (E) score, both of which may suffer from human judging error. The total D-score is the sum for the eight most difficult elements in the routine coupled with an evaluation of the variety expressed by the elements and how they were combined. When a gymnast includes an element never seen before, it must be assigned a difficulty score (usually, this is done before the new element is presented at a competition). At competitions, two judges evaluate the difficulty of the routine, independently from each other, and then must reach consensus. The total E-score, in turn, rates the performance for execution and artistry. The base E-score is 10 points, with points being deducted for errors in the technique and artistry exhibited. Small errors bring minor deductions (0.1, for example); for large errors, such as falling, the deductions are larger (such as a full point). Six judges participate in determination of the E-score: the highest and lowest of the six scores given are not taken into account, and the average of the remaining four becomes the final E-score. The D- and E-score, when added together, make up the final score of the gymnast.

Each athlete's routine includes compulsory and optional elements. The former are specific actions that all gymnastics must perform if wishing to compete against each other at a certain level, where the official levels at which a gymnast may compete are Level 1 to (the most advanced) Level 5. The optional elements cover additional strengths, elements, and advantages that the athlete displays and the artistic aspect of the performance. For example, the music choice and choreography for floor exercises put the athlete's personality and charisma on display. A gymnast among our informants reflected thus on balancing the D- and E-score:

It's always better to do something less creative, a little bit easier, with cleaner execution, than something extremely difficult that you might fail on. But it depends on the situation: if you're trying to maximize your score, you will have to do something a little harder, more creative, more difficult in, say, the finals [...] where you're trying to win, essentially. You might have to perform something more difficult. It's risk management. (David, gymnast)

When a gymnast and coach jointly create a routine for execution, they have to check the CoP guidance for their level of competition and make sure the routine is designed to achieve maximum points for the athlete's skill range and meets all the requirements. In other words, they establish the D-score by preparing a set of elements, and executing the corresponding actions as well as possible maximizes the E-score. If an athlete leaves an element out or improvises in the middle of the routine, the D-score will change.

As the sport itself does, the judging of artistic gymnastics involves exceptional skills and competition. Only the best judges are allowed on panels for international competitions. Judges need to remain well versed in the CoP, and they have regular meetings to learn about updates in the evaluation system. The FIG tracks and evaluates their performance, in terms of accuracy and consistency (Mercier & Heiniger, 2018), in a process that judges described thus:

In artistic gymnastics, athletes and coaches do not receive any official clarification of the results and the deductions made by the judges. Neither do they get explanations of the mistakes made by the gymnast during the routine. Also, they are not allowed to talk to the judges at all during international competitions, let alone ask questions about the routine. Competitions' time constraints are cited as justification for the lack of an explanation accompanying the scores, alongside human cognitive and sensory limitations that rule out the amount of precision expected.

Currently, there are very few opportunities for verifying the judgement process applied by the panel or for protesting against the outcome. The only recourse is an official inquiry, which is a request by the coach or gymnast for revision of the D-score. In response to this appeal, the head/supervisory judge replays video of the routine (in slow motion) and may decide to adjust the D-score - possibly downward. Even here, no clarification or explanation gets supplied.

Perhaps unsurprisingly, the lack of clarification of the results often provokes conflict between coaches and judges.

People are emotional. Coaches are [...] emotional, even more emotional than athletes... Very often it happens that an athlete works, trains well, and at the competition is also trying hard - of course. And for the coach it may seem like the athlete did everything great! And then suddenly the judges punish and make deductions... and it happens that the coach comes, yelling emotionally, [...] "What are you judging here?! How can you do it?! My gymnast just did a great job!" [...] Very often, later on they calm down and often come back, asking for forgiveness. (Ulla, judge)

In the opinion of several interviewees, resolving the problem of low explainability of humans' judging could never be easy, for reason of its complexity.

In reality it's not possible, because every judge has a different way to make the record of deductions. The FIG requires the judges to use the [prescribed set of] symbols to record, but only a few judges can use 100% [of the symbols]. (Harry, judge)

While explanations and guidance from judges during the competition remain scarce, there is ample opportunity for this before and after it. For instance, top-level judges typically counsel top gymnasts from their country, in a somewhat informal manner, as one judge explained:

Sometimes we recommend [to] a gymnast, based on their body type, what kind of element may work better for them. Or we advise them to change the order of the elements, such that they are less tired when executing them. (Bob, judge)

It is important to note that such sharing of expertise and knowledge among judges, coaches, and gymnasts trickles down from the highest level of (professional) gymnastics all the way down to the most local, grassroots level. In addition, "podium training" is held right before an international competition. This provides an opportunity for judges to get a preview of what the gymnasts will do and for gymnasts and coaches to obtain some feedback beforehand, as judge Harry explained:

During the podium training, gymnasts show their routine and they are allowed to ask the judges what would be the amount of deduction on the execution.

4.2 AI-Powered Judging for Artistic Gymnastics

In November 2018, the FIG and Fujitsu unveiled a judging-support system intended for real-time "judging support that is fair and accurate."5 This JSS captures gymnasts' movements, then analyzes them and provides a score. The FIG president clarified that the primary aim behind the planned introduction and use of the AI-based judging system was to obtain accurate assessment of gymnasts' performance, without any biases or errors.6 The International Olympic Committee too had expressed support for development and implementation of a new judging-support system of this sort, citing the importance of Olympic sports being judged fairly and transparently, and Fujitsu began development of the JSS in 2016, with about 100 people working on the project full-time since. Two of them are collaborating with the FIG to develop the CoP such that it is compatible with the JSS. This entails formulating more precise rules for a digital form of the CoP.

By means of 3D sensing technology, the JSS captures a multi-angle view of the gymnast's movements during the routine. It would then analyze them against the digitalized element definitions from the CoP artifact. On the basis of this analysis, the JSS then would indicate mistakes, deductions, and a final score. The Fujitsu-proposed judging system uses 3D laser sensors, combined with AI-enabled joint-position-recognition software (Fujiwara & Ito, 2018). Via depth images obtained via these sensors, it derives the posture and position of the human body from the joint positions.

This 3D sensing technology oscillates many lasers on a scale of about 2 million points per second, detects the reflected light, and calculates the distance to the target object (point cloud). It then recognizes the joint positions from this shape; calculates hands and feet positions [sic], bending of joints, etc.; and finally compares those results with model data of human movement in a database to derive differences in movement. (Fujiwara & Ito, 2018)

The JSS can provide a visual representation of each element along with all technical details of how it should be executed (for instance, the length and height of a jump, the number of steps taken before and after it, the angle of the turns, etc.). Figures presenting Fujitsu's conceptualization of how the skeleton-recognition technology works can be found on the website of Fujitsu.7

At the moment, Fujitsu's efforts are focused on ascertaining D-scores via the JSS, though the company foresees the JSS supporting the decision-making process connected with execution scores too. At present, the E-score poses an obstacle in that the point-deduction system currently in place is described in vague terms. For instance, what constitutes "straight" or "slightly bending" can be interpreted differently from one judge to the next. In their current form, the rules cannot readily be transplanted to an AI-powered judging system. This is why Fujitsu and the FIG are jointly developing a refined rulebook that specifies precise angles and ties them in with the execution score (Fujiwara & Ito, 2018).

One of the judges among our informants expressed the following vision for the execution score:

In the future, the system will tell you the difficulty and show you all your execution marks and where [why] you got it... And it allows you to select from about 10 different icons that [represent] different [...] angles of the body. (Nick, judge)

The expectation is that, thanks to its technical capabilities, the JSS will resolve some of the challenges related to the process of human judgement in artistic gymnastics. For instance, the JSS may improve the accuracy of judging by providing a multi-angle view of the gymnast's routine, which goes beyond what the human eye can capture from a single human's point of view. It could also significantly speed up the process of judging. Furthermore, the system is expected to manifest neutrality and impartiality, affording elimination of human biases and subjectivity from the judging. There are hopes connected with another vexing issue too: visualization and recording of all steps in the evaluation process should enable provision of explanations and clarification of the scores for the gymnasts.

Thus, while supporting the judges' work, the system may simultaneously assist gymnasts and coaches with their training since, according to its developers, it "can accurately inform athletes regarding their stability and the exact angle between their joints, so they can make appropriate adjustments and improvements."8

While the current objective is for the AI-powered system to support judges with their judging process during official international competitions, exactly how it will be made part of that process is less clear. Multiple options can be readily envisioned: giving input for joint decision-making with human judges, providing a reference value for each judge's decisions, serving as a backup in the resolution of inquiries, etc. Down the line, the system might even become the main source of scores, with the human judges there only to serve as a backup when something goes wrong with the system.

While the FIG and Fujitsu assure that deploying a fully autonomous system that replaces the human judges is not part of the plan, many judges raised associated concerns nonetheless, such as these:

Moreover, some participants highlighted further implications of top-level judges' removal, ultimately leading to the collapse of the entire knowledge-sharing system, from the top all the way to the grassroots.

Our informants raised additional financial and practical concerns related to the implementation of an AI-powered system, referring to high financial costs for implementing such a system for training purposes and the new technical skills required for setting up and working with it. High costs for installing the system would result in unequal access to the system, arising from many local gymnastics federations' lack of financial resources.

Obviously, it's going to be a very expensive system. At the very top, there [is sufficient] financial means for using the system, [but] how is it sloped down to each federation, to each area within each region? Because, obviously, you're not going to be able to have this system in every program. (Charlie, judge)

I think it might be a really expensive program to replace what we have already. I'm from a very small federation with no money. We can't buy the system. We don't have money for it. So only big federations will get the system. (Lilly, judge)

Maybe it will work at European/World Championships or the Olympic Games. But when you gonna come to Africa, how you gonna bring all these technologies and artificial intelligence to Africa? We have no technical support or financial resources for it. (Sarah, judge)

In addition, starting to use the system will necessitate additional training. According to FIG representative Simon, plans are in place for special training camps where the judges can learn how to use the new system, which he acknowledges is going to bring further costs and resource needs. Our observations indicate that, in the meantime, the judges' level of awareness of the system's capabilities and their knowledge of how it works is relatively low.

5 Paradoxical Tensions Related to AI-Powered Judging

From the data analysis, we can posit six paradoxical tensions related to AI-powered judging as conceived of in our study. Below, we describe each tension in turn and detail the associated challenges related to developing and implementing the AI-powered system. The discussion draws on input from key stakeholders canvassed in our study of judging in artistic gymnastics.

5.1 Paradoxical Tension 1: Accurate AI Is Too Exact

The first tension identified arises from the accuracy of the AI-based judging system, its perceived excessive exactness in judging. Most informants stated that an AI-powered judging system will be more accurate than human judges, stating that a "machine" and technology are always more accurate than human beings. Moreover, it is clear that there is some sense of competition among human judges with regard to accuracy; they take pride in how accurately they can judge, and precision helps them advance to panels for international competitions.

We're like the athletes: we compete among ourselves; we want to get to really accurate scores and to score consistently, because when our scores are analyzed [and] you have too much variation, you're not going to be selected for the Olympics. (Bob, judge)

The AI-powered system may seem to be the ideal judge in this sense. However, according to some interviewees, one possible disadvantage of such a system is precisely its precision: excessive exactness in judging is not suitable for gymnasts, since they are not perfect - they cannot implement some elements to millimetric accuracy or to a specified degree of an angle.

The accuracy of judgements made by humans is a matter of some dispute. Variously, our interviewees pointed to accuracy shortcomings within our research context as possibly linked to limitations of people's physical capabilities, inability to maintain concentration, fatigue, sensory perception of information, cognitive capabilities, speed of processing information, and inappropriate levels of knowledge internalized by judges. Let us look at each of these before considering how an AI-powered system enters the picture.

Informants characterized the constraints of human sensory and cognitive capabilities thus:

The second factor is the human fatigue and loss of concentration that stems from the sheer duration of international competitions. With the same panel of judges having to work for eight or 10 h straight, sustaining the same level of attention and concentration throughout the day is nearly impossible.

The issue of the level of internalized knowledge on judges' part may involve either lack of knowledge or, in contrast, too much influence of training and experience. Both may result in mistakes in judging.

And it's bad enough that you can make your own mistake anyway. And how do you tell the difference between somebody making a mistake because of an error and some little bit of incompetence (that's [considered] to be cheating)? It's very difficult to read this. (Sarah, judge)

Moreover, our informants indicated that the challenges plaguing human-based judgement could be resolved through an AI-powered system's advanced technical capabilities. In the stakeholders' opinion, the system under development is more impartial and should objectively yield high accuracy while also expediting the judging process. Indeed, almost everyone we interviewed cited the system's expected high accuracy as an advantage, especially for evaluation of the technical dimensions of a routine, such as the height of an element, angles, and speed.

However, tension is evident with regard to the accuracy advantage an AI-powered system might possess over human judges. While judges saw merit in exactness in judging, they also indicated that the current system of human judging provides some kind of equivalence between the inaccuracy of judges and that of the gymnasts - judges do not judge accurately, but neither do gymnasts perform accurately. Both work to a level of precision that is "good enough" and leaves room for artistry. In contrast, a judging system that demands exactitude would not provide such balance: there were concerns about gymnasts being unable to provide high enough accuracy in their performance to satisfy the system's requirements.

The precision of a newly introduced electronic judging system could, hence, pose a challenge for the gymnasts while also exerting a generally negative influence on the popularity of artistic gymnastics through these effects on the gymnasts' performance and the visibility of lower scores.

5.2 Paradoxical Tension 2: "Objective" AI Can Be Biased

From our findings, a paradoxical tension became evident with respect to biases also, where key stakeholders perceived AI-powered systems as a way of addressing the challenges linked to biases in current human judging. Interviewees expressed a belief that the new system would be free from such biases and, therefore, more objective and neutral in its evaluations - after all, it should not be affected by such "human factors" as prejudice and personal preferences. In reality, however, AI may introduce new types of bias, which could end up even harder to spot in a system that is assumed to be bias-free.

Judges' biases and subjectivity pose a vexing problem for artistic gymnastics, as is reflected by the fact that most informants highlighted this issue. Athletes are victims of biased and unfair judgements by the panel of judges, which may be rooted in any of various factors: emotions, personal preferences, initial expectations, familiarity with the routine or athlete, prejudice for or against a particular country, informal guidelines' influence on the judging process, etc.

Studies have identified numerous sources of bias in humans' judging process. Memory biases (related to recall, testimony, and hindsight) may be present, as might similarity, desire, mere-exposure, order, rule, complexity, test, and anchoring and adjustment biases (Mazurova et al., 2021).

Memory biases (recall, testimony, and hindsight bias) could come into play in that judges may unconsciously rely on memory if unable to remember each particular detail of the routine. Our memory is not always a reliable source of information. A judge who cannot recall all the details of a routine may unconsciously try to "recall" the missing parts by filling in the gaps, possibly with unconscious reliance on experience. Hence, the judge may assume the athlete to have done certain things that were not actually done. Clearly, a certain level of approximation exists in the evaluation process. Judge Ulla referred specifically to the large amounts of approximation involved in judges' evaluation of heights, speeds, and angles in an athlete's performance, and she stated that scores are estimated on the basis of this approximation. As for anticipation, another judge, Sarah, stated, "When a gymnast runs [up], I can tell you if it's going to be a catastrophe or not. We anticipate. Anticipation helps you sometimes with your judgement. You have not only what you see at this moment; it's much bigger than just what you see."

Desire, similarity, and mere-exposure bias manifest themselves in relation to judges' expectations or personal preferences. For instance, familiarity with an athlete and his or her earlier success/failure may predispose a judge to expect a certain quality of performance. Judges' personal expectations may cloud their objectivity and cause them to perceive the routine as better or worse than it truly was. Likewise, certain countries' preeminence may steer judges toward giving higher scores to those nations' athletes.

Even if no bias actually exerts an influence, judging a gymnast from one's own country may induce greater stress, as judge Bob highlighted: "It happens sometimes that, because you know all the mistakes and you are not surprised to see them, you can end up giving a lower score than the average judge. If that happens for an important routine, the gymnast can lose because of that, so there could be repercussions from your country." If instead assigning too high a score, "you're going to get flagged by the FIG."

In addition to the factors mentioned above, we found a significant influence of routines' order on the judging process. Judges often give undue weight and attention to the first and the last things encountered; this corresponds with the notion of order bias. In a related phenomenon, an athlete who competes in the morning is likely to receive a lower score while one performing in the evening gets a higher score. Our informants confirmed this.

It's always like this: if you compete in the morning, judges are harder on you; they easily take away many more points. They want to be good/strict and do their job properly. Thus, if you compete in the morning, they can make a bigger [...] deduction, and in the evening if you do exactly the same mistake, they will not take so much from your total score. (John, gymnast)

Regarding rule bias in artistic gymnastics, some interviewees identified unofficial guidelines on keeping one's scoring "average" throughout the day and on not giving an excessively high score to a "perfect" routine.

Judges have a certain average from a morning competition, and they need to keep this average between morning and evening scores. So they are afraid to give high scores from the start, as it will be harder for others to get a higher score in the evening, so they need to keep this average between the morning and the evening score. Thus, they don't give too good scores in the morning, and the better scores come in the evening. (Mark, gymnast)

As for perfect scores, gymnast James said, "Human judges, even if they see something perfect, like a perfect routine, they can't leave the papers empty. They need to find something [wrong] in the routine, to fill in the papers. That's why it's so hard to get 10.0 nowadays."

Complexity bias is clearly present, due to such factors as information overload, time pressure, human fatigue, lack of accuracy, the need to pay constant attention, and perceived importance of the judges' task and responsibility. Extensive approximation in the judges' process arises from the limits of the human brain, which can process only a certain amount of information within a given time, like it or not. Judge Felicity summed up the issue: "We don't want that, but we all make mistakes when we judge."

Under the influence of anchoring and adjustment bias, judges may tend to root their judgement either in the previous achievements of the athlete or in their first impression of the gymnast. The initial anchor for the judge might be the athlete's ranking or "usual" performance level. And the first impression, once made, is very hard to change or adjust.

Especially in your home country, it's usually not fair when the judges know you and have seen you so many times during the training so they kind of know already where you will do your mistake. So if you don't do it in the competition, they think like, "Oh, he usually makes this mistake, so it will be a mistake now" even if you do it very well. I like to compete more internationally, where judges don't know me. I usually score higher points. (John, gymnast)

Regarding AI-powered systems, both athletes and judges stated that these systems would be fairer and capable of assessing all routines equally. Such a system should prove unbiased and more objective in that it has no "anticipations" or expectations for a given athlete's performance, and no other human factor should affect the system's judging process.

AI doesn't care which country you're from. It evaluates the technical side of the performance. Judges can hear very often from the coaches that we've been biased with their athletes, and if the routine is evaluated by the system, who can you blame for low scores? Nobody. Because AI is unbiased. It's objective. (Ulla, judge)

We can make some mistakes about an objective thing, but the system can't. (Bella, judge)

Others, however, were able to point to potential issues of assumptions embedded in AI. Biases might arise especially in relation to body morphology, skin color, and the style of performance.

5.3 Paradoxical Tension 3: Even Black-Boxed AI Represents Explainability

The third tension is related to the perceived explanation capabilities of black-boxed AI. Our informants felt that, via the new technology represented by the AI-based system, the current judging system's lack of explainability and interpretability can be resolved, at least partially. This stands in contradiction with the challenges associated with AI black boxes' inherent lack of explainability and interpretability.

The lack of explanation in today's judging system poses a substantial problem for gymnasts and coaches, as section 4.1.2 attests. They identified providing explanations and clarifications of the results as important for advancing the training and development of the athletes, and they stressed that they would like to receive an explanation of the deductions and of the final scores given by the judges.

Such frustrations may lie behind these key stakeholders' perception of the AI-based judging system as a way to allow better-organized, systematic provision of explanations regarding scores. Judges too told us that supplying an explanation to athletes is important and that it would be very useful for athletes if an AI-powered system could do so. In the opinion of our informants, the electronic judging system holds potential to offer such explainability benefits as more explanation and greater transparency of the judging process, pinpointing of details that could inform the training process, and easier verification of the judging process. Cited in particular were the advantages of some explanation and clarification of the results over what the human judges provide and new possibilities for effectively gaining and accumulating in-depth knowledge.

Additionally, the system's ability to present the results immediately, alongside an indication of how it has come to these conclusions, may be of great use. Also, those inquiries not precluded via the system's use might see more rapid resolution and be handled more accurately, at the competition itself.

Again, while supporting the judges, the system was described as able to assist athletes with their training too. For instance, Ulla stated that the new system, if able to provide information about how the decision on the final score and deductions was reached, could definitely help coaches improve the training process. This benefit could be expected to exert a positive influence, in turn, on the athletes' perception of such AI-powered systems. Moreover, it could facilitate drawing new fans to the sport.

When you see a routine as a layman and you think it's great but then the gymnast gets a low score, you respond, "I don't understand this. Why would I watch it?" So using this technology could make you understand better why a score was given. (Jacob, gymnast)

At the same time, some interviewees recognized that parts of the AI-powered system are not going to provide explainability. They cannot. For instance, judge Katarina commented, "Apparently, there's some machine learning that's going on there, and that brings about the whole black-box issue."

5.4 Paradoxical Tension 4: While Used for Artistic Gymnastics, the System cannot Assess Artistry

The fourth tension involves the consequences of removing the human element by implementing an AI-powered system. While reducing the "human factor" and human emotionality may enable the judging system to avoid certain biases and subjectivity and to yield highly accurate and objective judgements, some interviewees stated that, precisely because of its lack of "humanity," the AI system lacks the ability to evaluate a crucial feature of artistic gymnastics - namely, the artistry of the gymnasts' performance. This inability could eventually lead to elimination of the artistic component of artistic gymnastics. Informants worried about this, emphasizing the importance of the artistic aspect of a gymnastics performance. One put it thus: "Taking artistry out of competition is the same as taking the soul out of gymnastics" (Josh, judge). These concerns point to new challenges in the process of judging, which bring about a corresponding value-costs tension.

It's called artistic gymnastics. And artistic is the key part of it, how it looks. I don't think the machine really can take up this part. We have artistry, we have a lot of things. Beautiful things. (Norman, judge)

The computers don't understand what is artistic. If in artistic gymnastics judging is completely done by the computer, it's not artistic gymnastics. This is then something like online games. (Harry, judge)

The computer will not see the faces of the gymnasts. This is the most important - the face, eyes, smiling. How can the computer analyze this? And then what about the music? In the end, we will have every exercise look the same. And the personal style of athletes will be lost. (Lilly, judge)

Gymnastics is the sport of emotion. Artificial intelligence has no emotions to this point. (Charlie, judge)

Yes, the technology can probably measure the amplitude of the legs perfectly, but I cannot imagine that technology can evaluate the gracefulness or whether the performance is in synch with the rhythm of the floor music. (Lauren, gymnast)

5.5 Paradoxical Tension 5: A System Intended for Humans Lacks Human Interaction

Similarly, our interviewees expected the technology to be unable to provide the same level of human interaction that gymnasts experience today with the human judges. Some judges stated that human interaction between the judges and athletes during routines is a crucial part of the performance. This interaction, while important, is not always visible - it is formed of details: the athlete greeting the judges before starting the routine, a slight nod of approval after completion of a routine, a fleeting smile from a judge. All of these make the athletes feel more confident and afford a strong sense of real human presence and interaction at the competition. Every judge interviewed assumed that this integral part of competitions will be reduced or eliminated if an AI-based system ends up judging the athletes' performance. Accordingly, the fifth tension identified in our study is that of an AI-powered system that is expected to evaluate human gymnasts and interact with human judges yet devoid of the human-interaction element. Some judges stated that the lack of humanity in the AI-powered system could become a stumbling block to this system's large-scale implementation in artistic gymnastics.

5.6 Paradoxical Tension 6: Consistency Requires AI Adaptability

The final tension identified becomes evident as the AI-powered system's introduction brings a need to codify the CoP as a digital artifact. This artifact must be much more specific in how the rules are specified (e.g., defining "straight" explicitly as 180° or as 179-181°). The expected result is consistency in judging: the same routine performed by the same gymnast should always get the same score.

With human judges, you can't compare across competitions in terms of the scores obtained. At one competition, you perform a good routine without a fall and get 11.6. The next competition, you perform a routine with a fall and get 12.4. This is kind of ridiculous and should go away with this technology. (Damian, gymnast)

Codifying the rules so brings a danger of becoming static, however. One informant explained in the following words: "If humans are no longer as closely involved in applying and interpreting them, who will adjust the rules as the athletes and sport evolve? Changes to the rules over time are an important aspect of the sport, partly because gymnasts start 'playing the system.'" Judge Bob elaborated on the latter phenomenon.

It's like doing your tax [return]: you optimize for the regulations. If you see that gymnasts take advantage of the rules, then you try to fix it for the next cycle. In men's gymnastics, for instance, over time the importance of difficulty has increased and that of artistry has decreased.

He cited another major reason for human tweaking of the rules too: a desire to keep the sport both exciting and safe.

If we systematize everything and we have no human judges anymore, how do you change the rules? Where do you take gymnastics? It takes human input from athletes and coaches [to answer questions] like "what is a dangerous exercise?" Some exercises are beautiful, but they are too difficult and dangerous for the sport [...]. Unlike sports like running the 100 m, gymnastics evolves. If you lose the top guys, you have a catastrophe. Who can keep it entertaining and safe? With static rules, gymnastics would get more and more boring.

Summarizing our findings, Table 2, below, outlines the six paradoxical tensions identified in the empirical study.

Table 2 Summary of the core paradoxical tensions in AI-powered judging

6 Discussion

For the emerging body of literature on in-depth research into AI, our case study offers several contributions, to knowledge both of AI in its own right and of its implications for business operations and society at large. Below, we discuss the theoretical and practical implications of our research against the backdrop of prior literature.

6.1 Implications for Theory

Our articulation of six paradoxical tensions associated with AI applications in connection with judging in competitive sports contributes to scholarly understanding of tensions related to specific IT artifacts in their use context (Jarvenpaa & Lang, 2005; Majchrzak et al., 2013). While AI already outperforms humans in several cognitive and perceptual domains and is on the verge of doing so in many more, we recommend proceeding with a degree of caution. Prior research has highlighted a need to manage the tensions and contradictions, affecting stakeholders of diverse types, that are associated with an emerging IS artifact (Benbya et al., 2021; Dubé & Robey, 2009; Jarvenpaa & Lang, 2005). Building on the existing body of knowledge and with succinct examples from human vs. AI-powered judging in the sport of competitive gymnastics, we were able to put the spotlight on these tensions and consider the multiple stakeholder perspectives that are at play in the decision-making processes. The paradoxical tensions exhibit implications for theory on four fronts: configurations of human-AI hybrids, reversal to digital objects, multifacetedness of biases, and transparency of decision-making.

Secondly, with regard to the articulation of the role of AI, we found that several of the six tensions stem from incongruities perceived by the various stakeholder groups in relation to what they saw as the characteristics of AI and digital objects. This ties in with the notion of ontological reversal, from physical objects to digital ones (Baskerville et al., 2020). Indeed, the AI-powered system draws on a digital representation of the athlete's performance, as opposed to a physical one. This transition, or reversal, from physical objects to digital objects (Kallinikos et al., 2013) and digital processing (Salovaara et al., 2019) seems to be generating pushback from stakeholders as the digital representation's exactness and improved accuracy push them out of their comfort zone - however desperately needed they know these to be. Furthermore, the ontological reversal precludes the evaluation of artistic notions of gymnastics and also brings tensions related to the role of domain experts for creation, sharing, and use of knowledge (van den Broek et al., 2021).

Additionally, by demonstrating how AI can at the same time reduce and introduce bias, we engage with current discourse on bias of AI systems. While we identified expectations among informants that the AI-powered system will be objective and free of bias (van den Broek et al., 2021), other perceptions were evident too. This is consistent with mounting evidence that AI-powered systems may acquire, replicate, and even amplify (implicit) human biases present in the training data used for learning from past performance-evaluation pairs (Shrestha et al., 2019). For instance, some performance-evaluation systems have displayed race and gender biases (Aysolmaz et al., 2020; Nadeem et al., 2020), thus drawing attention to the importance of detail-level discussion of what type of AI and biases may be involved.

Fourthly, our work enriches understanding related to conceptions of intractability associated with AI-powered systems that utilize machine learning (Faraj et al., 2018). What we add to the emerging literature on AI-powered systems (Rai, 2020) is connected mainly with an apparent increase in transparency of evaluation processes relative to human judging and with lack of interpretability in human decision-making. Our work responds to calls for research into the implications, for multiple stakeholders, of explainable AI (Asatiani et al. 2021), which is "a prerequisite for fair, accountable, and trust-worthy AI" as well as for managing, justifying, evaluating, improving, and learning from AI (Meske et al., Forthcoming, p. 1). Our study can be considered in light of the notion of AI and fairness also, which is a crucial consideration in implementing AI. As van den Broek et al. ( 2019) did in a human-resources context, we found that it is important to consider the accuracy of the information and the consistency visible in decision-making. A related paradox we identified lies in the view that AI increases transparency and explainability. The literature suggests that eliminating bias and making the process more fair and transparent can be expected to yield positive outcomes for multiple stakeholders, including acceptance, trust, satisfaction, a sense of commitment, and engagement behavior (Heiniger & Mercier, 2018; Konovsky, 2000). Our data analysis indicated that as the decision-making process grows more (seemingly) transparent, the various parties get more willing to accept potentially adversarial outcomes. Our findings mesh with those of Rzepka and Berger ( 2018), who revealed that the transparency of an AI system's decisions or actions significantly influences users' behavior, and of Xu et al. ( 2014), who found a positive correlation between system transparency and users' satisfaction with recommendation systems. Our findings point to judges having more positive perceptions of a system if it seems more transparent. Transparency in the decision-making process leads to a sense of greater informativeness and enjoyment too, thereby also enhancing the perceived decision quality and increasing system acceptance (Meske & Bunde, 2020).

6.2 Managerial Implications

Our study contributes to IS scholars' practical knowledge of the digitalization of sports (Goebeler et al., 2021; Xiao et al., 2017). The work builds on prior understanding of how IT artifacts have changed the face of decision-making in competitive and professional sports, by articulating the role of AI-powered systems for judging in competitive gymnastics. Complementing contributions from earlier literature, we have captured key relationships and tensions expressed by multiple stakeholders in connection with AI use in competitive sports. These shape the impact of the decisions made in the design of future systems. Hence, awareness of them could help to reduce human error and otherwise inform effective rollout and implementation of the AI systems now under development for such domains as competitive sports. In particular, acknowledging the paradoxical tensions could lead to development of mechanisms for avoiding or confronting them. The latter would be preferred, since it involves fuller understanding of the technology's features in aims of optimizing the system's use in a particular context.

Also, the study has broader implications for fan engagement in competitive sports. Research has already shown that fans become more involved when data are shared (Cortsen & Rascher, 2018). This is highly relevant from a marketing point of view, in that some sports (gymnastics among them) suffer from a lack of interest between Olympics years. For spectators, an AI-based system could enrich competitions' streaming services by providing not just real-time, understandable-seeming judging results but also quantitative indicators such as height and stance stability, to guide the audience to greater appreciation for the sport.9 Thus, the technology can add some concrete numbers to stadium viewing experiences and broadcasts both, making such sports more attractive to watch. Prior work indicates that technologies that greatly improve the correctness of judging decisions can contribute to a more attractive competition (Leveaux, 2012). Another factor worthy of attention is that, in Fujitsu's words, "the technology also could add entertainment value to TV broadcasts and phone apps. One day, it could represent potential revenue for gymnasts themselves, who will be able to market and monetize their data."10

7 Conclusion

Our study shed light on paradoxical tensions accompanying AI-based evaluations. The contributions notwithstanding, it has its limitations, though. The first, associated with single-case-study designs, raises the issue of generalizability of the findings (Walsham, 2006). Nonetheless, single-case studies are a "typical and legitimate endeavor" (Lee & Baskerville, 2003, p. 231) in IS research, permitting one to gain rich insight into the interplay of various issues in application of AI systems and the accompanying tensions. This was possible only via in-depth work on articulation of the contradictions from the case's empirical reality. Against this grounding, future research can validate the findings from our research and generalize them beyond the single context of AI use in competitive gymnastics.

The study's most crucial contribution lies in identifying paradoxical tensions in complex socio-technical relationships amid the work to introduce an AI-powered judging system, in light of the perceptions of multiple stakeholders. Our findings suggest that the adoption of AI technologies for decision-making is by no means straightforward, and our outline of the related tensions lights the way for future research on AI adoption and management across diverse settings and stakeholder landscapes. In summary, we hope that we have evoked the energy and power related to exploring paradoxical tensions (Lewis, 2000), which may open avenues for fruitful debate of the organizational and societal implications of using AI, and that we have enriched the emerging body of knowledge on its transformative potential (Raisch & Krakowski, 2021).

References

Asatiani, A., Malo, P., Nagbol, P., Penttinen, E., Rinta-Kahila, T., & Salovaara, A. (2021). Sociotechnical envelopment of artificial intelligence: An approach to organizational deployment of inscrutable artificial intelligence systems. Journal of the Association for Information Systems, 22(2), 325-352.

Article Google Scholar

Aysolmaz, B., Iren, D., & Dau, N. (2020). Preventing algorithmic Bias in the development of algorithmic decision-making systems: A Delphi study. Proceedings of the 53rd Hawaii international conference on system sciences, 3, 5267-5276. https://doi.org/10.24251/hicss.2020.648.

Benbya, H., Pachidi, S., Davenport, T., & Jarvenpaa, S. (2019). Call for papers JAIS-MISQE joint SI on artificial intelligence in organizations - JAIS-MISQE SI on artificial intelligence.

Brynjolfsson, E., & Mcafee, A. (2017). The business of artificial intelligence. What it can - And cannot - Do for your organization. Harvard Business Review, 4-11.

Dumoulin. (2020). Accuracy and National Bias of figure skating judges: The good, the bad and the ugly. MIT Sloan Sports Analytics Conference.

Duong, A. (2008). Bias in the 2008 Beijing Olympics (gymnastics).

Goebeler, L., Standaert, W., & Xiao, X. (2021). Hybrid sport configurations: The intertwining of the physical and the digital. Proceedings of the 54th Hawaii international conference on system sciences.

Heiniger, S., & Mercier, H. (2018). National Bias of international gymnastics judges during the 2013-2016 Olympic cycle. arXiv preprint arXiv:1807.10033.

Keller, R., Stohr, A., Fridgen, G., Lockl, J., & Rieger, A. (2019). Affordance-experimentation-actualization theory in artificial intelligence research - a predictive maintenance story. 40th international conference on information systems, 1-17.

Mazurova, E., Penttinen, E., & Salovaara, A. (2021). Stakeholder-dependent views on biases of human- and machine-based judging systems. Hawaii International Conference on System Sciences.

Mercier, H., & Heiniger, S. (2018). Judging the judges: Evaluating the performance of international gymnastics judges. ArXiv, 1-12.

Nadeem, A., Abedin, B., & Marjanovic, O. (2020). Gender Bias in AI: A review of contributing factors and mitigating strategies. ACIS 2020 proceedings.

Rzepka, C., & Berger, B. (2018). User interaction with AI-enabled systems: A systematic review of IS research. International conference on information systems 2018, ICIS 2018, (December).

van den Broek, E., Sergeeva, A., & Huysman, M. (2019). Hiring algorithms: An ethnography of fairness in practice. 40th international conference on information systems, ICIS 2019.

Xiao, X., Chian Tan, F. Ter, Lim, E. T. K., Henningsson, S., Vatrapu, R., Hedman, J., ... Van Hillegersberg, J. (2017). Sports digitalization: An overview and a research agenda. 38th international conference on information systems, 1-21.

Download references

Acknowledgements

The authors are thankful for the feedback received at the 2020 ACM Collective Intelligence conference, the 53rd and 54th Hawaii International Conference on System Sciences, the 2020 AIS SIG Digital Innovation Transformation and Entrepreneurship Paper Development Workshop and the Department of Digitalization at the Copenhagen Business School. The authors acknowledge Céline Decoster and Kristof De Mey for their assistance in engaging stakeholders in our study. Finally, the authors are grateful for all the informants for taking the time to be interviewed for this study.

Author information

Authors and Affiliations

1. Aalto University, Espoo, Finland

Elena Mazurova & Esko Penttinen

2. HEC Liège - Management School of the University of Liège, Rue Louvrex 14, B-4000, Liège, Belgium

Willem Standaert

3. Ghent University, Ghent, Belgium

Willem Standaert

4. School of Information Systems and Technology Management at UNSW Business School, Sydney, Australia

Felix Ter Chian Tan

Corresponding author

Correspondence to Willem Standaert.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1: Guide to the interview topics

Sub-focus of research

General questions for all stakeholders

Stakeholder understanding and awareness related to such systems' technical capabilities

− What do you know/think about the technical capabilities of an AI-powered system?

− What do you think about...

− the fairness of human judging and AI-powered systems?

− biases in human judging and AI-powered systems?

− the accuracy of human judging and AI-powered systems?

Stakeholder perceptions of the new system

− What is the opinion of the gymnasts/judges/coaches about the new, AI-powered system?

− Could you summarize your opinion on the AI-powered system?

− Would you trust the AI-powered system?

− Are you concerned or excited about the new, AI-powered system?

Questions for specific groups of stakeholders

The system's development and technical characteristics

Fujitsu:

− How did the idea of introducing an AI-powered system arise?

− To what extent are judges/gymnasts/federations involved in the development?

− Could you specify the role of hardware, of software, and of data in this system?

− How will the AI-powered system affect the panel of human judges?

− Will the evaluation process be changed through the implementation of the AI-powered system?

− Which parameters affect the decisions produced by the AI-powered system?

− What other applications for this technology exist, outside sports?

Explainability

Gymnasts:

− Do you get any kind of explanation/clarification of the final result from the judges?

◦ If so, please explain how this works.

◦ If not, do you think explanation should be provided?

* If so, then what form would be preferable: oral/visual/both?

The system's attractiveness to spectators

Fans:

− As a fan, what do you think of this AI-powered system?

− How could the sport develop in terms of fan engagement on account of this system?

− Do you think this system will attract more people to watch/follow the sport?

Appendix 2: The data structure

Appendix 3: Summary of the codes used in the analysis and related example quotes

Code

Example

The lack of accuracy of human judges

New biases caused by AI

Potential increase of explainability and transparency of the judging process

Appendix 4: Brief explanation of each artistic-gymnastics discipline

Floor exercises (men's and women's) are performed on a special gymnastics mat of 12 × 12 m. The discipline combines individual elements (somersaults, splits, handstands, etc.) and various dance elements. In the performance, athletes should make the most of the entire mat space. The complexity of the routine and its elements is evaluated, as are the purity and confidence of the execution. Also important are originality and the artistry of the performance, especially with regard to the women's discipline, since these exercises include specifically chosen music and individual dance steps and rhythmic movements. The duration of floor performances is limited to 1 min, 10 s for men and 1 min, 30 s for women.

The vault (men's and women's) is a jump performed with a running start and use of additional support. The apparatus is 1.6 m long and 0.35 m in width. The athlete runs along a special track 25 m long and 1 m wide, pushes off with his or her feet from a shock-absorbing device, and makes an additional push from the apparatus with the hands (for men, pushing with one hand is allowed). The jump may be straight, somersault, overturn, etc. Men have one attempt at this jump, while women have two attempts and the average score for the exercise is displayed. For evaluation, the important parameters are the height and distance of the jump, its complexity (the number of revolutions around the longitudinal and transverse axes etc.), purity of execution, and the precision of the landing.

The apparatus for the Parallel Bars (for men) and Uneven Parallel Bars (for women) consists of two wooden poles fixed to a metal base. For men, the apparatus height is 1.75 m; for women, the bars are 1.65 and 2.45 m above the surface of the safety mats. Men's routines using the parallel bars combine dynamic elements (rotations, swinging movements, etc.) and static ones (horizontal stops and handstands), and the gymnast must use the full length of the apparatus, exercising above and below the bars. A woman's routine with the uneven bars includes turns in both directions around the upper and lower bar, using one hand or both, and various technical elements completed above and below the bars, with rotation around the longitudinal and the transverse axis.

The Pommel Horse discipline (for men) involves an apparatus with handles at a fixed height of 1.05 m (as measured from the surface of the safety mats). The routine is a combination of handstands with swinging and rotational movements of the legs. The athlete must use all parts of the apparatus.

For the Rings (men), movable wooden rings are attached to special cables. They are at a height of 2.55 m from the surface of the safety mats. Exercises using the rings (lifts, turns, and twists) should demonstrate the athlete's plasticity and physical strength. The static elements of the routine are no less difficult than the dynamic ones. The rules dictate that jumping from the rings apparatus at the end of the routine must be an acrobatic element, while the athlete may use the aid of a coach or other assistant to take up the starting position on the rings.

The Horizontal Bar (men's discipline) uses a steel bar with a diameter of 27-28 mm and a length of 2.5 m fixed at a height of 2.55 m (as measured from the safety-mat surface). The rules specify that the athlete must not touch the bar with his body when performing rotations in any of various directions. In the course of the performance, the athlete should demonstrate several types of grip and an ability to shift from one type to another.

The Balance Beam (for women) involves an apparatus 5 m long and 0.1 m wide that is rigidly fixed at a height of 1.25 m from the floor. The balance-beam routine is a composition that comprises dynamic elements (jumps, turns, "jogging," somersaults, dance steps, etc.) and static ones (twine, swallow, etc.) completed while the gymnast is standing, sitting, and lying on the apparatus. The gymnast must use the full length of the beam. The judges evaluate her plasticity, balance, and artistry. A routine should be no longer than 1 min, 30 s.

About this article

Cite this article

Mazurova, E., Standaert, W., Penttinen, E. et al. Paradoxical Tensions Related to AI-Powered Evaluation Systems in Competitive Sports. Inf Syst Front 24, 897-922 (2022). https://doi.org/10.1007/s10796-021-10215-8
Download citation

Accepted:

Published:

Issue Date:

DOI: https://doi.org/10.1007/s10796-021-10215-8

Share this article

Anyone you share the following link with will be able to read this content:

Get shareable link

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

# data[].attributes.attachment_link.title
Paradoxical Tensions Related to AI-Powered Evaluation Systems in Competitive Sports - Information Systems Frontiers

# data[].attributes.attachment_link.description
Judging in competitive sports is prone to errors arising from the inherent limitations to humans' cognitive and sensorial capabilities and from various potential sources of bias that influence judges. Artistic gymnastics offers a case in point: given the complexity of scoring and the ever-increasing speed of athletes' performance, systems powered by artificial intelligence (AI) seem to promise benefits for the judging process and its outcomes.

# data[].attributes.body
Frommer, Frederic J. “Opinion: A clock saved the NBA. Can it save baseball?” CNN. Cable News Network. 18 Feb 2023. cnn.com/2023/02/18/opinions/mlb-pitch-clock-baseball-new-rules-frommer/index.html

Gordon, Anna. “Ai Healthcare? Robots Created to Help Patients Pass Trial.” Time, Time, 31 Jan. 2024, time.com/6590440/robots-hospital-patient-testing-phase-ai-assistance/.

Kegu, Jessica. “Are there different standards for men and women in tennis? USTA head says ‘yes’”. CBS Mornings. CBS News. 11 Sep 2018. https://www.cbsnews.com/news/serena-williams-us-open-loss-usta-ceo-katrina-adams-men-women-held-to-different-standards/

Keh, Andrew. “Gymnastics’ Latest Twist? Robot Judges That See Everything.” The New York Times, The New York Times, 10 Oct. 2019, www.nytimes.com/2019/10/10/sports/olympics/gymnastics-robot-judges.html.

Mallen, Cheryl. “Chapter 6: Artificial Intelligence and Implications for Sports Officiating”. Emerging Technologies in Sport: Implications for Sport Management. Routledge. 30 Sep 2020. pp. 87-104. https://ucill.vdxhost.com/zportal/zengine?VDXaction=DocFetch&docfetch_seencopyright=true&

Mazurova, Elena et al. “Paradoxical Tensions Related to AI-powered Evaluation Systems in Competitive Sports”. Information Systems and Frontiers. Springer Link. 29 Nov 2021. https://link.springer.com/article/10.1007/s10796-021-10215-8#Sec12

“Ned Ludd.” Wikipedia, Wikimedia Foundation, 25 Feb. 2024, en.wikipedia.org/wiki/Ned_Ludd.

Newman, Brooke. “The Long History behind the Racist Attacks on Serena Williams”, The Washington Post. 11 Sep 2018. www.washingtonpost.com/outlook/2018/09/11/long-history-behind-racist-attacks-serena-williams/.

Nielsen.  “Average TV viewership of Major League Baseball World Series games in the United States from 2000 to 2022”. Statista. Christina Gough. 15 Nov 2022. https://www-statista-com.libproxy.berkeley.edu/statistics/235678/world-series-tv-viewership-in-the-united-states/

O’Brien, Katherine A., and John Mangan. “The Issue of Unconscious Bias in Referee Decisions in the National Rugby League.” Frontiers, Frontiers, 23 Aug. 2021, www.frontiersin.org/articles/10.3389/fspor.2021.739570/full.

Park, Yoonyoung and Hu, Jianying. “Bias in Artificial Intelligence”. Clinical Journal of the American Society of Nephrology. National Library of Medicine. American Society of Nephrology. 1 Feb 2023. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10103344/

Prime/USTA, Source: Amazon. “Alizé Cornet Hit with Code Violation for Changing Shirt on Court – Video.” The Guardian, Guardian News and Media, 29 Aug. 2018, www.theguardian.com/sport/video/2018/aug/29/alize-cornet-hit-with-code-violation-for-changing-shirt-on-court-video.

“Racial Bias in Fans and Officials: Evidence from the Italian ...” Frontiers In, Frontiers Media S.A., 28 Feb 2023. journals.sagepub.com/doi/full/10.1177/00380385221138332.

South, Zin. “The Usage of Video Assistant Referee (VAR) and its impact on Players’ and Fans’ Emotional Experience of Football”. University of Canterbury. 2022. https://ir.canterbury.ac.nz/server/api/core/bitstreams/5f75027c-55ac-4ff7-beaa-bab7054568b4/content

“Umpire Ruled out after Serena row”. CNN. Cable News Network. 8 Sep 2004. https://edition.cnn.com/2004/SPORT/09/08/tennis.umpire/

Uzoma, Anyanwu Udo. “Hawkeye- The Facts Behind the Controversy”. Tennis View Magazine. Tennis View Magazine. 24 Aug 2023. http://www.tennisviewmag.com/tennis-view-magazine/article/hawkeye-facts-behind-controversy

White, Ed. “Soccer Player Gets 8 Years in Prison for Killing Referee.” AP News, AP News, 13 Dec. 2023, apnews.com/general-news-9c6ffd0841024f8f8f3823ed94e69e8e.

# data[].attributes.headline
Source: Alizé Cornet hit with code violation for changing shirt on court

# data[].attributes.subject
Source: Alizé Cornet hit with code violation for changing shirt on court

# data[].attributes.body
Finally, Serena Williams has not been the only athlete subject to such disproportionate penalties based on a part of her identity. In a US Open tournament, tennis player Alizé Cornet was given a penalty for changing her shirt on the court after realizing it was backwards. A clear double standard is seen when this penalty is compared to the countless times when male tennis players change their shirt or even sit with it off for the duration of a break. Again, this calls for a more standardized way of sports officiating that might only be able to come from the use of algorithms and technology to step away from the human biases that are able to similar uphold unequal treatment. An algorithmic and computerized approach to making calls and distributing penalties also might serve to prevent the generation of controversy surrounding players that forces aspects of their identify aside from being an athlete from coming into play. Such algorithms would be able to explain exactly what movement or gesture had initiated the need for a penalty with concrete evidence rather than the a few words by a single referee.

# data[].attributes.attachment_link.readable_text
As temperatures soared and players wilted at the US Open, Alizé Cornet was hit with a code violation for briefly removing her shirt on court.

Realising that her shirt was on back to front when she returned from her ten-minute 'heat rule' break following the second set of her match with Johanna Larsson, Cornet swiftly took it off and put it on correctly.

The umpire, Christian Rask, immediately hit her with a code violation.

Cornet went on to lose the match 6-4, 3-6, 2-6

* US Open apologises to Alizé Cornet for penalty

# data[].attributes.attachment_link.title
Alizé Cornet hit with code violation for changing shirt on court - video

# data[].attributes.attachment_link.description
As temperatures soared and players wilted, Alizé Cornet was hit with a code violation for briefly removing her shirt on court.

# data[].attributes.body
In an attempt to eliminate the presence of such damaging biases, a common proposal put forth by many both inside and outside of the sports industry is the introduction of AI into game officiating. This project has already begun in gymnastics, when judges were aided by AI in the 2019 world championships. It created a standardized method of scoring that would output the exact “thought process” that generated scores, yet still left room for judges to account for the subjectivity necessary to recognize the artistry and feeling behind performances (Keh). However, AI is not the perfect answer to the elimination of bias. It is still subject to the potential bias of its creators and the data it is shown. Especially in sports, the data an AI must be shown is based on historical images, information, and events that often have underlying themes of inequity, discrimination, and preference. This is even seen in AI's application in medicine, where the data algorithms were fed tended to cause them to recognize race as a risk factor for kidney disease specifically in people of African descent (Park and Hu). Therefore, any historical patterns of bias or discriminatory practices within sports, like gymnastics, may cause AI to develop a bias of its own. So, similarly to the method necessary to consider viewer engagement in a world undergoing a technological revolution and the need for accuracy, a balance between the power of humans and technology in game officiating must be struck so they officiate not only the players and game, but also each other.

# data[].attributes.attachment_link.title
Bias in Artificial Intelligence: Basic Primer

# data[].attributes.headline
Source: Are there different standards for men and women in tennis? USTA head says "yes"

# data[].attributes.subject
Source: Are there different standards for men and women in tennis? USTA head says "yes"

# data[].attributes.body
The grievances Serena Williams faced were not limited to racist tournament attendees, point losses, penalties, and racist caricatures. Outbursts towards umpires are often penalized not only during the game, but also with an ensuing fine named by game officials. When comparing the amount Serena was called on to pay to her male counterparts, a clear disparity can be seen. In response to her arguments against the umpire during her game with Osaka, Serena was fined a total of $17,000 for verbal abuse, a coaching violation, and racket abuse. Roger Federer, another tennis legend who is also known to have his share of outbursts during matches, was only fined $1,500 following his "expletive-laden outburst during the 2009 U.S. Open Men's final" (Kegu). The clear difference in these fines in addition to the caricature of Serena seem to only perpetuate negative stereotypes surrounding black women that label them as too loud or criticize their body shape. It is unsustainable and unethical to allow such practices to continue, and a more standardized method of distributing penalties seems necessary to prevent other black and/or female athletes from experiencing this perpetuation to the extent Serena did. This standardization may be only achieved through the use of technology, perhaps with AI algorithms that analyze speech. One might even argue that line judges in tennis specifically could be eliminated from the game altogether and replaced with Hawkeye, an idea supported by Novak Djovic, another tennis great (Uzoma).

# data[].attributes.attachment_link.title
Are there different standards for men and women in tennis? USTA head says “yes”

# data[].attributes.attachment_link.description
Many tennis fans and players are still debating whether sexism played a role in the penalty given to Serena Williams in the U.S. Open final

# data[].attributes.body
Figure 3 depicts Serena’s match with Osaka where a series of arguments with the umpire regarding score and penalties for coaching during the game, a practice that Serena had consistently frowned upon. The artist also chose to depict Osaka and the umpire as white, in addition to depicting Serena in a style akin to the racist caricatures drawn of people of African descent that had been used during times of slavery, colonialism, and extreme perpetuated racism. Inaccuracies in calling penalties not only gives room for potential racial bias, in addition to bias based on other aspects of an athlete’s identity, but also for onlookers to warp the situation into a caricature like this.

# data[].attributes.body
Looking to the future, the possibility of AI being integrated into more fields and careers is becoming less of a possibility and more of a reality. In some cases, jobs that we never would have thought would fall susceptible to being replaced by machines and robots are falling prey to this technological revolution. The medical field offers a unique take on this. In recent years, a project to create robots that would be able to respond to patients and visitors in hospitals in order to provide more comfort to them and relieve hospital staff of some duties has been steadily progressing. Earlier this year, the project, SPRING (Socially Assistive Robots in Gerontological Healthcare) seems to be close to reaching success (Gordon). These robots are able to provide instructions and directions, engage in conversation, and facilitate assistance. It is very possible that humans will eventually learn to respond positively to them as well. In Japan, elderly homes are incorporating robots, with seniors interacting with them and showing clear signs of the development of strong emotional attachments and connections to them (Wright).

Places of medical care are sites where emotions often and consistently run high, similarly to stadiums and living rooms full of fans and tension. It is nearly impossible to predict from our perspective from the present the kinds of social norms that will develop and exist regarding human-AI relationships, or what AI game official assistants, or just game officials, may look like in the future. Nonetheless, the rate of integration, or invasion as some might view it, of technology into our daily lives only seems to be increasing and it is unclear whether we will be able to determine the potential effects of replacing people with AI in game officiating and similar jobs before it actually occurs.

# data[].attributes.headline
Source: Artificial Intelligence and Implications for sports officiating

# data[].attributes.subject
Source: Artificial Intelligence and Implications for sports officiating

# data[].attributes.body
“... a game officials’ ‘word is law… and a crooked ref has enormous power to control the game’” (Mallen 89). This is a dangerous concept when the careers, lives, and futures are held in the hands of only a few people and a few calls that are deeply influenced by the flow of the game, the current physical and mental state of the referee, their internal views, the roar of the crowd. It is arguable that because of this, the power of a crooked ref extends far beyond the game and into the personal lives of athletes and the minds of fans. In order to reduces the risk of conscious or unconscious discriminatory practices in game officiating, not only can more algorithmic, technology-based methods be implemented into game officiating, but such may also be implemented into teaching new officials and referees.

# data[].attributes.body
A major, pressing bias that everyone must identify and mitigate within ourselves is racial bias, something that is known to plague all aspects of our society. Professional, and even recreational, sports are far from exempt from this. Figure 4 depicts a study of colorism in the Italian Serie A, a professional Italian football league competition, where skin ranges from 0-20 with 0 being the lightest and 20 the darkest. This study was chosen because Italy is a major player in international soccer tournaments, and therefore their referees will likely be more involved on a similar international level. The study shows that people of darker complexions are more likely to have red cards called on them. The presence of racial bias in professional athletics is especially damaging because sports is known to be an activity where people are free to play and interact regardless of their identity. The unconscious, or unfortunately sometimes conscious, choice by game officials to disproportionately call out and bring negative attention to certain races may be seen to quietly perpetuate themes of racism and discrimination. Perhaps the negative interactions with tennis umpires experienced by Serena Williams could be linked to racial bias in this way also, whether this be solely by biases held within game officials or fans who helped shape the mindset of the umpires before and during her matches.

# data[].attributes.headline
Source: Gymnastics’ Latest Twist? Robot Judges That See Everything

# data[].attributes.subject
Source: Gymnastics’ Latest Twist? Robot Judges That See Everything

# data[].attributes.attachment_link.readable_text
Gymnastics officials are making artificial intelligence technology available to judges at this week's world championships. The humans, for now, have the final say.

Image

The robot judges, each about the size and shape of a Wi-Fi router, use three-dimensional laser sensors to track the gymnasts' movements. Credit Credit The Asahi Shimbun, via Getty Images

This article is part of our continuing Fast Forward series, which examines technological, economic, social and cultural shifts that happen as businesses evolve.
STUTTGART, Germany - The rectangular gray boxes arrayed around the floor at the gymnastics world championships this week are easy enough to ignore.

But these little boxes, 30 of them in all, hold outsize significance: According to Morinari Watanabe, the president of the International Gymnastics Federation, their presence at the worlds signals "the beginning of the new history of gymnastics."

The robot judges, in other words, have arrived.

Amid the displays of athleticism and artistry at this week's championships, there also has emerged a development in technology that could hint at the trajectory of officiating across the sports landscape.

Each of the gray boxes keeping watch in Stuttgart, designed by the Japanese company Fujitsu and about the size and shape of a Wi-Fi router, contained a set of three-dimensional laser sensors that tracked the movements of each of the 547 gymnasts from 92 nations participating this week. That data was fed to an artificial intelligence system, accessible to the human judges, that measured and analyzed skeletal positions, speeds and angles - some of them unavailable or simply missed by the judges - as the athletes went through their movements.

Thanks to all this, Watanabe explained, no longer would gymnasts - many of whom, he noted, had started gymnastics as young as age 3 and had trained competitively for more than a decade - risk having their efforts unceremoniously wasted by human error or interference.

"This is a step toward the challenge of justice through technology," Watanabe said.

The debut of such technology at the biggest gymnastics meet outside the Olympics represented a meaningful milestone in a sport periodically marred by judging controversies and often wracked with questions about political influence in scoring decisions.

For all the grand language, and for all the big-picture prophesying it has inspired about the future of sports - baseball is already experimenting with robot umpires, and tennis is starting to expand electronic line-calling - the steps unveiled in Stuttgart were preliminary, and fairly subtle.

In gymnastics, at least, humans very much remain in control. For now.

At the world championships, the artificial intelligence system instead has served a supporting role, available to judges to confirm difficulty scores in two circumstances: in the event of inquiries (when gymnasts formally challenge the judges' score) and blocked scores (when there is a large deviation between the sets of judges). And the technology, officially, remains limited to a few events: pommel horse, rings and men's and women's vault.

The gymnastics federation did not release any detailed data related to the launch of the A.I. system this week. But Steve Butcher, the federation's sports director, noted that reviews aided by the computer system had already resulted in "a few changes" at the championships, which continue through Sunday.

Butcher said judges were separately reviewing a large portion of the exercises through the Fujitsu system for the purposes of verification and education.

Despite the fanfare of the system's introduction and its potential effect on scores, athletes and coaches said they were not paying too much attention to the A.I. judges. Laurent Landi, who coaches the American star Simone Biles, said that it was too early, and the procedures not sufficiently integrated, to fully ponder the implications of the technology. But he said once the computers proved their accuracy, they could represent a big, positive step for the sport.

"We all know how subjective scoring can be," Landi said.

Others expressed wariness, arguing a computerized system contradicted the human heart of the sport.

"We shouldn't turn it over to robots because, to be honest, I think that's a little weird," said Yul Moldauer, a member of the American men's team. "I think we should always have human beings being the judges."

Before this year's competition, Moldauer and the hundreds of other competitors were asked to participate in what Butcher called a "body dimension measurement" procedure - effectively a precise scan of their bodies and movements - so that the analyses could proceed with maximum accuracy.

The precompetition measurements were voluntary, but Butcher made sure to point out that several athletes who had agreed to be scanned this week later benefited during the championships by having one of their scores increased upon review.

Takehiko Ishii, a manager for Fujitsu, estimated that more than 90 percent of the athletes had agreed to participate. Those who did not were evaluated using standardized body dimensions, but, he said, "some gymnasts have thick muscles, and some gymnasts are thin, so it's better to have the body dimension measurements."

MyKayla Skinner, a member of the United States women's team, said there was a lot of discussion, and some confusion, among her teammates when they were asked to review the paperwork to sign off on the scans.

"We were like: 'I don't know, do we? Do we not? This is so weird!'" Skinner said. "But I think in some ways it's going to be very, very great, just because there's so much politics that go into gymnastics."

Asked about concerns about data privacy, Butcher said all of the athlete information collected at the competition would be discarded at a predetermined expiration date.

At the same time, Ishii said there had been preliminary conversations about whether such data could eventually be collected and sold as part of training materials for the public, with athletes receiving any royalties.

Amid all the other minor concerns, however, most discussions about the so-called robot judges have tended to lead to a larger, simpler question: Will there come a time when human judges are eliminated completely?

Ishii said the A.I. was not yet sophisticated enough to take over fully for humans. But one day it might, he said.

Coaches and gymnasts were more skeptical about an artificial intelligence system's ability to ever fully evaluate a nuanced gymnastics routine.

"The computer cannot understand artistry, mastery, cannot understand the feeling in the movements you make," said Enrico Casella, an Italian coach. "It can see all the angles better than judges, but the artistic part is impossible."

The decision to unveil the technology this week came after a series of tests, beginning with one at last year's world championships in Doha, Qatar. The International Olympic Committee is aware of the technology, but it has not publicly announced whether the A.I. system - or a different one - will be employed at the Olympics next summer in Tokyo.

Fujitsu and F.I.G. said they envisioned a future in which the 3-D laser sensors were used not only in competition, but to enhance the viewing experience for fans. The federation and a Japanese broadcaster experimented this week with integrating the data into television broadcasts. Coaches, meanwhile, were considering their value as a training tool.

"Once we learn about - we're calling them robots - once we determine what the robots can really see, then it could absolutely dictate the direction of coaching for us," said Tom Forster, the high-performance team coordinator for the American women's team. "The human eye can't pick up everything."

Andrew Keh is an international correspondent, covering sports from Berlin. He has previously covered Major League Baseball and the N.B.A. and has reported from the World Cup and the Olympics. @andrewkeh

A version of this article appears in print on , Section SP, Page 3 of the New York edition with the headline: Robot Judges That See Everything Provide a New Twist. Order Reprints | Today's Paper | Subscribe

# data[].attributes.attachment_link.title
Gymnastics' Latest Twist? Robot Judges That See Everything

# data[].attributes.attachment_link.description
Gymnastics officials are making artificial intelligence technology available to judges at this week's world championships. The humans, for now, have the final say. This article is part of our continuing Fast Forward series , which examines technological, economic, social and cultural shifts that happen as businesses evolve.

# data[].attributes.body
I really enjoyed and appreciated the way the class was structured almost around this project, as it gave me a better understanding of what we were meant to learn from it and what you want to see in it. I enjoyed looking at the projects of other people the most and analyzing them, as not only was it impressive to see the work of others, but I feel that the activities you gave us that corresponded with them were very useful. Working backwards to identify their research question or level 3 question was interesting and really made me more deeply consider every piece of information included in my own project and its relation to my thesis and research question.

Regarding finding sources, I did struggle in finding ones I believed were the most relevant to my topic and effective for my argument, and ended up using some that I believe are not quite as effective or relevant as they could have been. It was a bit difficult finding visual statistics through statista and the library website for my topic, so I began turning more towards researching related studies and using their figures, statistics, and quotes rather than only looking at articles or images. This process of looking through studies also ended up being more interesting and allowed me to develop a greater understanding of several subtopics within my project in general, so I will likely continue to use this method in future assignments and projects.

In hindsight, I wish that I could have spent more time on this project and taken full advantage of how the class was formatted in respect to it. I do work best under pressure with a more immediate deadline in sight, but unfortunately due to the sheer amount of work I have given myself this semester and poor planning, I was unable to spend as much time as I would have liked modifying this.