1. Gamification as a Paradigm for the Evaluation of Visual Analytics Systems
NAFEES AHMED
Klaus mueller
I’m Nafees Ahmed from Stony Brook University, here to present an alternative approach towards evaluation using games.
VISUAL ANALYTICS AND IMAGING LAB
DEPT OF COMPUTER SCIENCE

2. EVALUATE visualization?fx ?
We are all gathered here today for at least one common thing, we still consider visualization evaluation to be a challenging problem. How do you know if your visualization is good or bad?
We don't know every bit of human eye and brain
We don't have a magic formula that gives us a number to rely on (atleast not yet)

3. EVALUATE visualization?
FEEDBACK
TASK
Researchers have to rely heavily on tests with feedbacks from real people.
Let it be a lab study or a crowdsourced one, the motivation for these people to work generally is always is money. This has many well known limitations and extrinsic motivation doesn’t always give us the right result.

4. EVALUATE visualization?
FEEDBACK
As an alternate, we thought of replacing the tasks with a purpose designed game with enjoyment as the only compensation for the test users.

5. Voluntary Participation undivided attention
Why games?
Voluntary Participation undivided attention
Repeated engagement
fast learning ….
Why games? Players play the game voluntarily and they stay engaged. Games have these inherent ability of bringing out such traits from the players.

6. Why games? GAME MECHANICS
A game designer ensure such engagement by carefully designing how the game behaves in response to the player – in simple words they are called game mechanics. Since the history of games is so rich, we have an ample collection of successful mechanics at our disposal.

7. Why games? GAME MECHANICS Human based computation Gamification
FourSquare Samsung Nation ….
fold.it eteRNA ESP Game ….
People have utilized such mechanics in two ways.

8. Looking at Color C , which one seems on top? A or B?
The problem
Looking at Color C , which one seems on top? A or B?
As a case study, we pick a simple problem. We want to know how well blending algorithm works for perception.
And transparency perception is the ability to guess the ordering between two layers given the blended color.

9. Local blending with blurring
Algorithms
Porter & Duff
Hue Preserving
Local blending
Local blending with blurring
[Porter 84]
[Chuang 09]
[Wang 09]
We considered evaluating the classic porter and duff alpha blending. And also few of the newer ones.
[For evaluation of an algorithm, we decided to take a brute force approach of have user evaluation of the whole color space of foreground, background and transparency levels.]

10. It looks something like this.

11. findings fast Data collection Better Data quality Low running cost
Deeper analysis
30,000 data points in 2 weeks
Noise free data
Only the initial cost
Can provide heavy data driven analysis.

12. BUT… New game design takes time Making the game popular is not assured
No generic design guideline
Skewed demographic
Verification and validation of game.

13. Looking forward Find the sweet spot of usage
Generalized game design for visualization problems
Refinement through adaptation and usage

14. THANK you http://vail.cewit.stonybrook.edu/Projects/HPU/Disguise
NAFEES AHMED
Klaus mueller
We hope this method gets adapted and utilized in even more creative ways in upcoming years.
Thank You.
VISUAL ANALYTICS AND IMAGING LAB
DEPT OF COMPUTER SCIENCE