A MYSTERY FOR YOU
"A Mystery for You" is an educational game designed to cultivate critical thinking and fact-checking skills in young learners. The game leverages a large language model (LLM) and a tangible interface to create an immersive investigative experience. Players act as citizen fact-checkers, responding to AI-generated news alerts and making informed decisions based on their investigations.
This HCI experience challenges our news-consumption habits by eliminating touchscreen interfaces, rethinking the perpetual scrolling and skim-reading of news. Instead this game interface uses a haptically rich analog device that combines the affordances of slow media with new generative media for more thoughtful and embodied interactions. This game and accompanying paper won the student game design category at ACM CHI 2024.
"A Mystery for You" is an educational game designed to cultivate critical thinking and fact-checking skills in young learners. The game leverages a large language model (LLM) and a tangible interface to create an immersive investigative experience. Players act as citizen fact-checkers, responding to AI-generated news alerts and making informed decisions based on their investigations.
This HCI experience challenges our news-consumption habits by eliminating touchscreen interfaces, rethinking the perpetual scrolling and skim-reading of news. Instead this game interface uses a haptically rich analog device that combines the affordances of slow media with new generative media for more thoughtful and embodied interactions. This game and accompanying paper won the student game design category at ACM CHI 2024.
Role:
Game Play Design, Software and Hardware Programming, Electronics Circuitry, Foundational Research & UX Research
Tools Used:
Node.js, Electron, Arduino IDE, Circuit Boards, Open AI API
CollaboratorS:
Haoheng Tang & Vishrant Tripathi
Awards:
Best Game: Student Game Category ACM CHI’2024
The Context:
Fact-checking skills, critical thinking and an investigative bent of
mind are essential life skills in an age of polarized media politics,
hate-speech and rampant misinformation. Lack of digital literacy
and an inability to identify misinformation has not only led to
swaying of elections – such as the 2016 United States Presidential Elections [1] - but also mob lynching and communal violence in countries like India [2] and Bangladesh[3]. The scale and severity of misinformation propaganda are exacerbated by the increasing access to and use of generative AI models. This could be seen in Argentina’s 2023 Presidential Election – ‘the first AI Election’ according to The New York Times [4].
In the face of these alarming trends, a pivotal question emerged for us: Could we harness the very technology that is fueling misinformation to instead combat it? How might AI be repurposed as a tool for public education? Could we help foster critical thinking and empower the youth to become more informed consumers who are not only skilled in fact-checking but also aware of the pervasiveness of AI generated media?
Read our full paper here
Educational Objective: Challenging, Replayable and Evolving Investigations
A Mystery for You aims to equip children with essential skills to discern truth from falsehood. By integrating AI and tangible interactions, the game fosters a deeper understanding of how to navigate and evaluate digital information critically. [5]
A Mystery for You aims to equip children with essential skills to discern truth from falsehood. By integrating AI and tangible interactions, the game fosters a deeper understanding of how to navigate and evaluate digital information critically. [5]
Studies have shown the benefits of using active learning strategies
to tackle misinformation, such as through lateral reading assignments
in civic courses [6] and gamification of the fact-checking
process [7]. Keeping these in mind, we decided to focus on the following during our game design process:
To the best extent of our knowledge, A Mystery for You is the first media literacy game that addresses these design challenges using generative AI, tactile interface and pedagogical approach.
- The game should challenge the player to actively investigate.
- The game interface should offer immersive interaction and
not encourage skim-reading.
- The game should have a high replayablity factor, offering unique scenarios in
every play-through.
- The decisions made by the player should impact the sequential
evolution of the game.
To the best extent of our knowledge, A Mystery for You is the first media literacy game that addresses these design challenges using generative AI, tactile interface and pedagogical approach.
Tangible Slow Media Approach:
Re-imaging the Interface
The design of our current day smart phones, with their smooth
touchscreens has conditioned us to keep scrolling mindlessly through
high volumes of information being perpetually updated in real time.
This is an issue we have seen inadvertently being replicated in
other educational games tacking misinformation. To address these
issues in affordances, we took on the challenge to re-imagine what
an interface for such a game could be. Thus, there is no screen
or keyboard involved in playing A Mystery for You. Instead, our
game is embodied in an tangible interface that celebrates analog
mechanisms and slow media. Slow media focuses on pacing out
media production and consumption, fostering intentionality and
mindfulness [8].
Re-imaging the Interface
The design of our current day smart phones, with their smooth
touchscreens has conditioned us to keep scrolling mindlessly through
high volumes of information being perpetually updated in real time.
This is an issue we have seen inadvertently being replicated in
other educational games tacking misinformation. To address these
issues in affordances, we took on the challenge to re-imagine what
an interface for such a game could be. Thus, there is no screen
or keyboard involved in playing A Mystery for You. Instead, our
game is embodied in an tangible interface that celebrates analog
mechanisms and slow media. Slow media focuses on pacing out
media production and consumption, fostering intentionality and
mindfulness [8].
Gameplay: Fact Checking Generated News Alerts
In this game, a player becomes a citizen fact-checker, responding to ‘news alerts’ printed out by the game interface.
Children take on the role of citizen fact-checkers and respond to ‘news alerts’ printed out by the game interface. These ’news alerts’ and sequential story-lines are generated through carefully engineered and tested prompts. By investigating different ’actors’ using fact-checking ’actions’, children are encouraged to think critically through scenarios and ultimately make informed verdicts.
The player investigates various actors and evidence by inserting cartridge combinations into the game interface. Each move the player makes results in the generation and printing of a follow-up ‘news update,’ which they must use to make an informed verdict about the truth or falsehood of the news. This interactive process sharpens critical thinking skills and enhances familiarity with generative AI’s misinformation capacities.
Designing Player Moves
The player investigates various actors and evidence by inserting cartridge combinations into the game interface. Each move the player makes results in the generation and printing of a follow-up ‘news update,’ which they must use to make an informed verdict about the truth or falsehood of the news. This interactive process sharpens critical thinking skills and enhances familiarity with generative AI’s misinformation capacities.
The player investigates various actors and evidence by inserting cartridge combinations into the game interface. Each move the player makes results in the generation and printing of a follow-up ‘news update,’ which they must use to make an informed verdict about the truth or falsehood of the news. This interactive process sharpens critical thinking skills and enhances familiarity with generative AI’s misinformation capacities.
The game interface involves a series of cartridges and buttons, each representing different actions (e.g., "Investigate," "Interview") and actors (e.g., "Local Police," "Politicians"). Players use these to interact with the game, which responds with printed news updates on a thermal receipt printer. This analog approach encourages slow, deliberate engagement with the content, contrasting the fast-paced consumption typical of digital media.
The player gathers facts over seven game days. The dialectic interaction between the interface and the player speaks against the one-sided consumption of news media cycles. Once the player feels confident of their choice, they can choose from the true and false switch to make their verdict. This leads to the printing of a game result along with the reasoning for why that was the case and why it is important to fact-check.
Underlying Prompt Engineering & API Communication
Tangible Interface
The player needs to use cartridges and buttons as input, and
the game interface, in turn, responds with outputs through a mini
thermal receipt printer. News alerts are printed slowly with a meditative
sound. This forces the player to pace their reading and gives them time to choose their investigative move. Prototypes
User Studies
Testing with ExpertsWe conducted user experience studies with both general users as well as experts in the field of misinformation and fact-checking. We conducted a virtual study with the fact-checking and educational outreach wings of Alt News, an Indian fact-checking organization. This provided us with crucial feedback on whether the scenarios generated by our prompt engineering were realistic and had pedagogical value. Experts found the approach compelling and found multiple generated scenarios to be close to real incidents that they community of a violent outbreak that ended up being an
out-of-context video from a different part of the country. They found the action choices, such as ‘reverse image search’ and ‘interview’, to be closely aligned with their own methods. However, they also pointed to the challenges of scaling a tangible game such as this to a large audience, especially in developing countries such as India.
Testing with Users
We conducted our general user study with university-going adults but planned to do significant user testing with children of different age groups in the future. In our current study, we found that participants reacted positively to the design of the game. They are intrigued by the action and actor cartridges as well as the tactile
nature of the game interface. However, some users found the range of cartridges limited and wondered about the potential for actions beyond those provided. While we have limited the choices of moves available to the player in the current version of the game, we plan to expand the range by creating additional cartridges to handle
more diverse scenarios and settings.
Despite lacking prior information about the game rules, users engaged with the device intuitively, expressing satisfaction with the tactile feedback, such as the button click and the LED lighting during cartridge insertion. The gradual printing process of the thermal printer encouraged the users to read the news updates line by line instead of skimming as they likely would have on a digital screen.
We observed users contemplating their next move, experimenting with different cartridge pairings to observe their impact on the game. Some users inquired about the possibility of pressing a red button midway through the game. Hearing that it would conclude the game and provide the answer, most of them opted to continue
inserting cartridges until they completed the "action" on "the last day." A small subset of users, however, completed the game early upon reading news updates on day 3 or day 4.
Participants described the game device as cute and nostalgic, comparing it to an old Macintosh computer. Notably, each testing participant was offered the opportunity to retain the printed receipt as a gift, highlighting their appreciation for the narratives on the small receipts and their desire to take them home.
Critical Reflections & Conclusion
Tangible Interface, Slow Media and Content Moderation:
The tangible interface and analog mechanisms of our game afford a slow media experience that celebrates the pacing out of media production and consumption, fostering intentionality and mindfulness. Additionally, the tangible interface also serves as a content moderation strategy, preventing players from engaging with the
LLM via inappropriate responses. This game is thus an example of how new and generative media can be combined with old and slow media to create more thoughtful and enjoyable interactions and news consumption.
AI Generated Gameplay in Educational Games:
The game fosters critical thinking while avoiding partisan, offering post-game reflection while avoiding the creation of actual fake news. It encourages thoughtful decision-making and shields children from online ads and harmful information.
As game designers using generative AI, we are cognizant of the risks of using blackbox
AI models. These models are trained by using large data sets which can have inherent biases embedded in them. Further, the outputs of LLMs are often subject to content moderation policies of the platforms providing API services. These moderation policies can strongly impact the scenarios generated for our game. This calls for more extensive research into these implicit biases and how they impact our game.
Game Interface as a Diagnostic Tool:
We believe our game can provide a way to analyze biases embedded in these systems and the players’ decision-making process. This can prove to be an approach towards diagnosing, understanding and creating healthier generative AI systems.
Tangible Interface, Slow Media and Content Moderation:
The tangible interface and analog mechanisms of our game afford a slow media experience that celebrates the pacing out of media production and consumption, fostering intentionality and mindfulness. Additionally, the tangible interface also serves as a content moderation strategy, preventing players from engaging with the
LLM via inappropriate responses. This game is thus an example of how new and generative media can be combined with old and slow media to create more thoughtful and enjoyable interactions and news consumption.
AI Generated Gameplay in Educational Games:
The game fosters critical thinking while avoiding partisan, offering post-game reflection while avoiding the creation of actual fake news. It encourages thoughtful decision-making and shields children from online ads and harmful information.
As game designers using generative AI, we are cognizant of the risks of using blackbox
AI models. These models are trained by using large data sets which can have inherent biases embedded in them. Further, the outputs of LLMs are often subject to content moderation policies of the platforms providing API services. These moderation policies can strongly impact the scenarios generated for our game. This calls for more extensive research into these implicit biases and how they impact our game.
Game Interface as a Diagnostic Tool:
We believe our game can provide a way to analyze biases embedded in these systems and the players’ decision-making process. This can prove to be an approach towards diagnosing, understanding and creating healthier generative AI systems.
Acknowledgements
This project began as a part of Prof. Marcelo Coelho’s course 4.043/4.044 Design Studio: Interaction Intelligence. We thank him for his mentorship. We thank Vishrant Tripathi (PhD, EECS, MIT.) for his advice and constant support through this process. We also thank all our friends and user study participants who played this game. Finally, we thank Pratik Sinha, Mahaprajna Nayak and the Alt News Team for their feedback and for all the incredible work that they and fact-checkers around the world do to uphold truth.
References
[1] Andrew M Guess, Brendan Nyhan, and Jason Reifler. 2020. Exposure to untrustworthy websites in the 2016 US election. Nature human behaviour 4, 5 (2020),
472–480.
[2] Elyse Samuels. 2020. How misinformation on WhatsApp led to a deadly mob
lynching in India.
[3] BBC News. 2019. Bangladesh lynchings: Eight killed by mobs over false child abduction rumours.
[4] Jack Nicas Nicas and Lucía Cholakian Herrera. 2023. Is Argentina the First
A.I. Election
[5] H. Tang and M. Singha, “A mystery for you: A fact-checking game enhanced by large language models (llms) and a tangible interface,” in Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, ser. CHI EA ’24,,
Honolulu , HI , USA ,
: Association for Computing Machinery, 2024.
[6] Jessica E Brodsky, Patricia J Brooks, Donna Scimeca, Ralitsa Todorova, Peter
Galati, Michael Batson, Robert Grosso, Michael Matthews, Victor Miller, and
Michael Caulfield. 2021. Improving college students’ fact-checking strategies through lateral reading instruction in a general education civics course. Cognitive Research: Principles and Implications 6 (2021), 1–18.
[7] Melisa Basol, Jon Roozenbeek, and Sander Van der Linden. 2020. Good news
about bad news: Gamified inoculation boosts confidence and cognitive immunity
against fake news. Journal of cognition 3, 1 (2020).
[8] et al. Blumtritt, Jörg. 2010. The Slow Media Manifesto.
[1] Andrew M Guess, Brendan Nyhan, and Jason Reifler. 2020. Exposure to untrustworthy websites in the 2016 US election. Nature human behaviour 4, 5 (2020),
472–480.
[2] Elyse Samuels. 2020. How misinformation on WhatsApp led to a deadly mob
lynching in India.
[3] BBC News. 2019. Bangladesh lynchings: Eight killed by mobs over false child abduction rumours.
[4] Jack Nicas Nicas and Lucía Cholakian Herrera. 2023. Is Argentina the First
A.I. Election
[5] H. Tang and M. Singha, “A mystery for you: A fact-checking game enhanced by large language models (llms) and a tangible interface,” in Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, ser. CHI EA ’24,
[6] Jessica E Brodsky, Patricia J Brooks, Donna Scimeca, Ralitsa Todorova, Peter
Galati, Michael Batson, Robert Grosso, Michael Matthews, Victor Miller, and
Michael Caulfield. 2021. Improving college students’ fact-checking strategies through lateral reading instruction in a general education civics course. Cognitive Research: Principles and Implications 6 (2021), 1–18.
[7] Melisa Basol, Jon Roozenbeek, and Sander Van der Linden. 2020. Good news
about bad news: Gamified inoculation boosts confidence and cognitive immunity
against fake news. Journal of cognition 3, 1 (2020).
[8] et al. Blumtritt, Jörg. 2010. The Slow Media Manifesto.