IRL=>URL
"New Yorkers, where do you see yourselves going?‘
An AI Native game/real time social space transport passenger to infinity possible corners on the internet.
EXPERIENCE LINK：https://irlurl.me.virtuaix.live/
INTRO VIDEO（V1）： https://youtu.be/Vj_flSM10fY
PLAYTHROUGH VIDEO: https://youtu.be/Yh7T7yZRiok
RESEARCH NOTES (PaperWritting...): https://www.notion.so/corylinyu/Final-IRL-URL-aed6e04d82924414abe2ab60b421afba
ITCH.IO: https://corylee1.itch.io/irlurl
IRL=>URL GAME INSTALLATION REQUIREMENTS：
https://docs.google.com/document/d/1el08FrjXMBNZ3QuKH2j7kjy2Rcw7v3q67szL3IEljxo/edit?usp=sharing
Hardware: PC / Laptop
Software: Online website adapted to any web browser (Default setting is Chrome)
Before start the game please allow pop up windows on the browsers (The game need the permission for camera and headphone)
Participants: Multiplayer but below 10 persons online at the same time
INTRO
Where will AI lead us?
An AI-native game that uses URLs (powered by Gemini and Serp api) as exploration material, attempting to simulate real-time social spaces on the New York City subway. How to play? Players should listen and choose content that attracts their a
ttention, they can choose up to 5 dialogues, then they will be teleported to somewhere on the internet.
The whole goal of the game is to simulate the attention mechanism of the language Transformer (from 'Attention is all you need, 2017'). In real life, passengers act as encoders, trying to determine what really needs attention, while AI acts as a decoder, determining the destination (URL) people want to go to. Trying to think about how people's attention is focused and deprived by the media when attention becomes a scarce resource, and ADHD is prevalent.
WHY I WANT TO DO THIS?
Since moving to New York nearly a year ago, I have been captivated by the city's fast-paced, exciting, novel, and chaotic atmosphere, which sparked my curiosity about how individuals find their goals and directions amidst the hustle and bustle.
This inspired me to create a game that mirrors my personal experiences in New York, with a concept significantly influenced by 'Serial Experiments Lain' and its idea of blurring the distinction between the 'real world' and the 'online world'. The advancement of AR and XR technologies, along with avant-garde experimental games like ARGs (Alternate Reality Games) and Pokemon Go, has further pushed the boundaries between reality and the virtual world. 'Serial Experiments Lain' proposes the notion of society as a large neural network, with every individual connected through the unconscious, which led me to consider integrating AI technology with the internet to create a novel gaming experience.
This inspired me to create a game that mirrors my personal experiences in New York, with a concept significantly influenced by 'Serial Experiments Lain' and its idea of blurring the distinction between the 'real world' and the 'online world'. The advancement of AR and XR technologies, along with avant-garde experimental games like ARGs (Alternate Reality Games) and Pokemon Go, has further pushed the boundaries between reality and the virtual world. 'Serial Experiments Lain' proposes the notion of society as a large neural network, with every individual connected through the unconscious, which led me to consider integrating AI technology with the internet to create a novel gaming experience.
Lain
the old recording of Serial Experiments Lain that aired after 9/11
The synchronicity of the animation opening and news
‘ I am falling’
‘ I am fading’
‘I have lost it all ...’
Although AI technology in games is primarily used for content creation and simulation, it has not been highly integrated with the internet to decode user intentions. By transforming AI into a navigator on the internet and allowing players to interact with NPC-generated dialogues, I aim to explore this concept and gauge the audience's reaction to this innovative gaming experience.
Some web art experiments, such as jodi.org, have used procedural generation on URLs to create an infinite website interface, but they often lack intention and related meaning for the audience. Similarly, Websim.ai uses URLs as a prompt for fake websites generation powered by Claude AI, which is an avant-garde integration, but it lacks immersion for the user and has barrier for those not familiar with URL’s structure. These examples highlight the need for a more balanced approach in UI design for AI-integrated games, one that effectively balances playfulness and cognitive load across diverse player groups. . By creating an AI-native game that simulates social phenomena and leverages the interpretative capabilities of LLMs to gather meaningful internet data as playful content, IRL⇒URL aims to provide an engaging platform for exploring social issues related to attention, information consumption, and the impact of technology on human cognition.
This game reflects my personal experiences in New York and embodies my thoughts on merging reality with the virtual world and integrating artificial intelligence with the internet, potentially bringing new possibilities to the gaming industry and providing players with a unique and engaging experience.
Some web art experiments, such as jodi.org, have used procedural generation on URLs to create an infinite website interface, but they often lack intention and related meaning for the audience. Similarly, Websim.ai uses URLs as a prompt for fake websites generation powered by Claude AI, which is an avant-garde integration, but it lacks immersion for the user and has barrier for those not familiar with URL’s structure. These examples highlight the need for a more balanced approach in UI design for AI-integrated games, one that effectively balances playfulness and cognitive load across diverse player groups. . By creating an AI-native game that simulates social phenomena and leverages the interpretative capabilities of LLMs to gather meaningful internet data as playful content, IRL⇒URL aims to provide an engaging platform for exploring social issues related to attention, information consumption, and the impact of technology on human cognition.
This game reflects my personal experiences in New York and embodies my thoughts on merging reality with the virtual world and integrating artificial intelligence with the internet, potentially bringing new possibilities to the gaming industry and providing players with a unique and engaging experience.
ARTSTYLE REFERENCE：
New York subway, retro, mosaic mural, the low model rendering texture of PS1 early video games, Breakcore music (jungle, speedcore)
GAME SCENES & SNAPSHOTS
TECHNICAL DESCRIPTION：
IRL⇒URL is a multiplayer online game with an isometric 3D view (nodejs+webrtc+threejs). The scene is set in a carriage of the New York subway.Each user joins the website and then generates an ID. Each user generates an avatar with a webrtc video stream texture. Inside the carriage, users can communicate with each other (STT). There is an NPC that generates (actions, dialogue) in the carriage in each round. Each user can choose one of the contents as the nutrient for the next station navigation（5 pieces of any info）. After choosing the info that draw them attention, the Gemini +Serp API will recommend a video link to each person based on the conversation content and selected content and force the user to jump to the webpage (get off the bus). The whole game is a metaphor for the recommendation algorithm and the attention system for human machine intercation.
Technical Workflow
DATASET
NPC DIALOG (MULTI AI AGENT IN FUTURE):
Daily Dialog Dataset：https://huggingface.co/datasets/daily_dialog
https://paperswithcode.com/dataset/dailydialog
http://yanran.li/dailydialog
ART DATA:
Guerrilla art：https://en.wikipedia.org/wiki/Guerrilla_arthttps://en.wikipedia.org/wiki/New_York_City_Subway
Font：https://nycsubwayfont.com/
https://www.myfonts.com/collections/lafarge-font-typetanic-fonts
https://www.fontbros.com/font-family/subway-mosaic
Subway Mosaic: https://en.wikipedia.org/wiki/New_York_City_Subway_tiles
Our Participants from 2024 NYU ITP Spring Show