Final Year Project 2004/2005 LYU0402 Augmented Reality Table for Interactive Card Games Supervisor: Prof Michael Lyu Presented by: Kevin Chow, Albert Lam

Overview Introduction Introduction Objective Objective Trading Card Game Trading Card Game Architecture of ART Architecture of ART Implementation in Second Term Implementation in Second Term Experiment Experiment Conclusion Conclusion Future Work Future Work

Introduction What is Augmented Reality? What is Augmented Reality? Mixed Reality combines the content from the real world with virtual imaginaryMixed Reality combines the content from the real world with virtual imaginary Augmented Reality is a subset of this where virtual content is overlaid into real objects of the worldAugmented Reality is a subset of this where virtual content is overlaid into real objects of the world

Introduction (Con’t) Paul Milgram characterized Mixed Reality interfaces on his “Reality- Virtuality Continuum” in 1994Paul Milgram characterized Mixed Reality interfaces on his “Reality- Virtuality Continuum” in 1994 Mixed Reality (MR) Virtual Environment Augmented Virtuality (AV) Augmented Reality (AR) Real Environment Reality-Virtuality (RV) Continuum

Introduction (Con’t) An AR system has the following three main characteristics: An AR system has the following three main characteristics: Combines real and virtual objects in a real environmentCombines real and virtual objects in a real environment Runs interactively, and in real timeRuns interactively, and in real time Registers virtual objects onto the real worldRegisters virtual objects onto the real world

Introduction (Con’t) ART stands for Augmented Reality Table ART stands for Augmented Reality Table User can play Card Games on the ART.User can play Card Games on the ART. ART can enhance the visual effect during the playing of the card game.ART can enhance the visual effect during the playing of the card game.

Objective Develop a generic Interactive ART Trading Card Game application. Develop a generic Interactive ART Trading Card Game application. Enhance the traditional card games: Enhance the traditional card games: Visual effectVisual effect Sound effectSound effect Complex calculationComplex calculation

Trading Card Game Trading Card Games Trading Card Games Two players play a match.Two players play a match. Players play cards to fight with each other. It includes summoning monster, casting magic, or setting traps, etc.Players play cards to fight with each other. It includes summoning monster, casting magic, or setting traps, etc. We choose “YU-GI-OH” as our implementationWe choose “YU-GI-OH” as our implementation

Trading Card Game (Con’t) Trading card game includes five kinds of actions Trading card game includes five kinds of actions RestoreRestore Draw cardsDraw cards Play cardsPlay cards Attack/ChallengeAttack/Challenge Discard cardsDiscard cards

An Example - YU-GI-OH In YU-GI-OH, Different information are stored on the card. In YU-GI-OH, Different information are stored on the card.

An Example - YU-GI-OH (Con’t) Game Mat of YU-GI-OH Game Mat of YU-GI-OH Different kinds of card must be put inside particular card zones Different kinds of card must be put inside particular card zones

An Example - YU-GI-OH (Con’t) LP : 2000 ATK : 500 ATK : – 500 = LP : 1300

An Example - YU-GI-OH (Con’t) LP : 1300LP : 2000 DEF : 1300 ATK : – 1200 = LP : 1900

Architecture Hardware Setup Hardware Setup Plasma monitorPlasma monitor Overhead cameraOverhead camera System architecture System architecture Augmented Reality PerceptionAugmented Reality Perception Game CoreGame Core DatabaseDatabase Game EnhancementGame Enhancement

Hardware Setup

Hardware Setup (Con’t) Computer Computer processes perceiving imageprocesses perceiving image generates visual and audio enhancementgenerates visual and audio enhancement controls game flowcontrols game flow Plasma Monitor. Plasma Monitor. acts as the Game Tableacts as the Game Table displays computer-generated scenedisplays computer-generated scene Overhead mounted camera Overhead mounted camera captures the cards and the screen of the plasma.captures the cards and the screen of the plasma. only input of the systemonly input of the system

System Architecture Game Core Module Game Core Module Perception Module Perception Module Database Module Database Module Game Enhancement Module Game Enhancement Module

System Architecture (Con’t)

Implementation Database Module Database Module Access card informationAccess card information Access game rules and card rulesAccess game rules and card rules Identify cardsIdentify cards GameCore Module GameCore Module Control game flowControl game flow Store game stateStore game state Rulebase game engineRulebase game engine

Database Module Contains all game information Contains all game information Card Database Card Database Contain all card informationContain all card information Simple editorSimple editor Rule Database Rule Database Contain all game rule and card ruleContain all game rule and card rule Simple editorSimple editor

Image Database Image Database Image Database Query with a card imageQuery with a card image Retrieve card idRetrieve card id Use improved block matching algorithmUse improved block matching algorithm ?

Select the best matched card from the candidates Select the best matched card from the candidates Split the captured image into 4 channelsSplit the captured image into 4 channels Compare only inner imageCompare only inner image Compare the different channels to the database separatelyCompare the different channels to the database separately Block Matching Algorithm

Find the pixel difference byFind the pixel difference by Reject the image if the pixel difference is larger than the thresholdReject the image if the pixel difference is larger than the threshold Accept the image with minimum pixel differenceAccept the image with minimum pixel difference

Improved Block Matching Algorithm However, the Block Matching Algorithm cause some error in real time gameHowever, the Block Matching Algorithm cause some error in real time game Solution, we use an improved Block Matching AlgorithmSolution, we use an improved Block Matching Algorithm Divide the image into 9 squaresDivide the image into 9 squares Apply Block Matching Algorithm to each squaresApply Block Matching Algorithm to each squares

GameCore Core Part of ArtTable Core Part of ArtTable Contains: Contains: Input Manager – handles inputInput Manager – handles input Output Manager – handles outputOutput Manager – handles output Rule Manager – stores rules and inference rulesRule Manager – stores rules and inference rules Game Manager – stores game statesGame Manager – stores game states Control how these managers interact and flow of the game. Control how these managers interact and flow of the game.

GameCore GameCore Output ManagerInput Manager Rule ManagerGame Manager 1. Receive input 3. Update game states 2. Pass to game manager 4. Inference rules 7. Display output to player 5. Perform actions if any 6. Generate output if any

Rulebase Why rulebase? Why rulebase? Flexible – Card games are very flexible and some card effect are unpredictable.Flexible – Card games are very flexible and some card effect are unpredictable. Extensible – New cards can be extended by adding new rules.Extensible – New cards can be extended by adding new rules. Generic – New card games can be implemented by new rule models.Generic – New card games can be implemented by new rule models.

Rulebase Load game rules Load game rules Forward checking for predicate Forward checking for predicate While conclusion reached While conclusion reached Perform the action of the rulePerform the action of the rule further forward checkingfurther forward checking Predicate1Predicate2ConclusionActionList P.S. Action may be load another rule, remove a rule, reset a rule, update game states, or show some outputs, etc.

Rulebase MainPhasePutCard1NormalSummonMainPhasePutCard2PlaySpellHandFromHellPlaySpellDestroyMonsterSetTargetMysticTomatoSendGraveYardSpecialSummonPutCard1

Experiment We have used 90 cards to do the recognition experiment. The result is as follow: We have used 90 cards to do the recognition experiment. The result is as follow: Number of hits number of cards percentage Always (No miss) 7078% Sometimes (miss 1-5 times) 1618% Never (miss >5 times) 44%

Experimental Result Discussion Reasons decrease the accuracy Reasons decrease the accuracy Image are captured in low resolution. The small error in cards position detection will lead to inaccuracy in recognition process.Image are captured in low resolution. The small error in cards position detection will lead to inaccuracy in recognition process. Reference images are recorded from camera. Errors in recording will lead to errors in recognition process.Reference images are recorded from camera. Errors in recording will lead to errors in recognition process.

Conclusion We have designed and implemented the prototype of ART system We have designed and implemented the prototype of ART system We have improved the card recognition algorithm with increased accuracy and efficiency We have improved the card recognition algorithm with increased accuracy and efficiency We have developed a rulebase game engine which is generic and extensible for card games We have developed a rulebase game engine which is generic and extensible for card games

Future Work Color Calibration Color Calibration Improve in card recognition algorithm Improve in card recognition algorithm Game Enhancement Game Enhancement

End of Presentation Thank you very much

Q & A Session