ViBand: High-Fidelity Bio-Acoustic Sensing Using Commodity Smartwatch Accelerometers
Вставка
- Опубліковано 16 жов 2016
- More info: www.gierad.com/projects/viband
We developed a custom smartwatch kernel that boosts the sampling rate of a smartwatch’s existing accelerometer to 4 kHz. Using this new source of high-fidelity data, we uncovered a wide range of applications. For example, we can use bio-acoustic data to classify hand gestures such as flicks, claps, scratches, and taps, which combine with on-device motion tracking to create a wide range of expressive input modalities. Bio-acoustic sensing can also detect the vibrations of grasped mechanical or motor-powered objects, enabling passive object recognition that can augment everyday experiences with context-aware functionality. Finally, we can generate structured vibrations using a transducer, and show that data can be transmitted through the human body. Overall, our contributions unlock user interface techniques that previously relied on special-purpose and/or cumbersome instrumentation, making such interactions considerably more feasible for inclusion in future consumer devices. - Наука та технологія
Congratulations! Very cool insights here, hope to see products in the wild before too long :)
The acoustic tags are amazing!
Congratulations! A great and very interesting approach to vibrations into technologic life.
Amazing!! With hand gestures we can control the world. A beautiful and new tool for human computer interaction or human machine interaction.
It looks incredibly promising. I wonder how bad of an impact that "overclocking" has on battery life, though.
If it isn't too much, then there's no excuses not to bring this to us NOW.
Impact on battery life can be negligible. In a commercial use, ViBand would use the same techniques that enable all-ways listening "Ok, Google" / "Hey Siri" detection. These are often special, low-power co-processors.
Here is a link to the InvenSense Datasheet: store.invensense.com/datasheets/invensense/mpu_6500_rev1.0.pdf
You'll see on page 11 that the power draw of the accel at 4kHz is 450 µA -- note that is micro amps. So well under 1mA! So yes, overclocking does increase power consumption, but even at this increased rate, it is still very small.
That is beyond awesome
I want to see how this would compare to Thalmic Lab's Myo armband. From my research, it taps into the electricity flowing through your nerves and muscles to determine what gesture you are performing.
Imagine this input being used in tandem with virtual reality.
I would love to see some code we could compile for the MiBand 1S! Would love to test it out in action.
Wow that's amazing.
Incredible!
Very impressive.
This is awesome!!! too cool
woow. love it. i want to know more about it.
If it can detect finger tapping, can it detect which finger was tapped?
could you use it to interpret sign language?
probably
imagine an interface that would easily sync with anyone's phone (ie the person who doesn't speak sign language) via an app. if most people with sign language had such a watch, it would break so many barriers. of course there are so many other applications for this tech - thoroughly impressed
How were you all able to transmit the accelerometer data to be read for real time analysis? I have an app to access the raw accelerometer data on my gear 2 but I dont know how to transmit it in real time
awesome. get it out!
Awesome...
Could you explain how the structured vibrations work? I'm having a hard time understanding how you can "instruct" an object to emit audio?
Objects that vibrate (for example a blender, dishwasher, or power tool) can be detected without any modification. For objects that do not vibrate (like a door or glue gun), we need a tag, sort of like RFID. In this case, we use a small vibration motor, which you can see at 5:09 (what the finger is placed onto). An ID (or any data) is then transmitted bio-acoustically into your body.
What's the limitations besides not working on non-powered objects? How does it differentiate external noise from vibrations through the body? Can it be made to work in a noisy environment? If so, to what level? City street, office building, mechanical factory floor?
Unlike microphones (coupled through air), vibrations are coupled to your body, which makes ViBand robust to external audio noise. That said, it will be susceptible to unintended arm oscillations (e.g., accidentally hitting your hand on a table) but it is possible to use machine learning to mitigate these false detections.
I'm also curious about the battery needs for something like this. Would it be manageable to have this on a modern wearable device without external power supply for any amount of time? (Incl. calculations needed to be done in realtime)
Yes. ViBand can be implemented on a System on Chip solution (similar to "OK Google", and "Hey Siri" functionality), which is extremely optimized for power consumption.
Is the kernel available on github or the like?
github.com/FIGLAB/FastAccel
WHAT did i just saw!.D:
AND if you place it on a Single pane window, you cna pick up AUDIO..
this is cool but a bit scary
So now I can be Dr. Strange?
This watch will have a lot of problems trying to figure out what option a person with Parkinson wants.
George Orwell would turn in his grave if he saw this...
It's all theoretical. You haven't shown any actual application. All the screens are photo shopped. There is no actual feedback being recorded from the watch. This is like hundreds of other "Smart Watch" "Wrist technology" crap that looks cool as fuck but hasn't even been used practically.
We ran user studies with real people, all demos shown in this video were shot live, no CG / Photoshop. If you watch the conference presentation, you'll see a live demo of the system.