FigJam by @Bonnie Chin: https://www.figma.com/board/DJKoPQmLtRXgQThY9gBL57/Reality-Labs-Work
project brainstorming:
- started in 2 goals: one that creates something useful, or something that creates hype or clout (demoable that is exciting & show what we are doing)
- demo: someone goes to website (most common form), and 3d scene loads (webxr), default gestures in place. show people how easy it is to configure and edit gestures. )
- gesture config panel and there are capabilities (add, edit, delete gesture, equip and unequip gesture)
- select gesture type: left hand, right hand, both hands?
- when you set positive data: record, trim/edit data, delete any data you have so far
- set negative data: do all the same things, and give gesture a name and preview/test it in sandbox/playground to see if it’s what you desire
- view what you just recorded
- editing is same, but there is no guide of events, and decide what you want to do
- demo of how this can actually be used (ex. playing minecraft with gestures)
- from design perspective: increase speed to iterate so people can use gestures to build their own games
- consistent gesture recognition is very hard
- need to tune values of each finger manually and setup conditions to match them, its very hard if there are too many gestures
- users can make gestures libraries and export them
- timeline of gestures (like a video but values of finger joints)
- single gesture exports (like a snapshot (picture) of finger joint states)
user facing requirements:
- establish webxr interface
- some kind of test scene for people to engage with
- default gestures
in the future:
- create unity that does all of the above for usefulness? longer term?
- think: how does someone know how to create gesture from someone’s vocabulary library/input mapping(like mincecraft wasd vs arrow keys vocab)
- maybe have a marketplace for gestures?
main goal: hype up the library for more sustainable development
dynamic gestures need to map to different gestures
concepts:
| name |
def |
notes |
| Gesture vocabulary |
set of actions that can be triggered by a gesture |
Like binding “W” to move forwards in Minecraft |
| Gesture |
a series of movement by your hand, captured by the headset |
|
|
|
|