Awesome tutorial! This is exactly the kind of tutorials we need more of for GodotXR that show how to create game mechanics out of the components of the xr tools and put it all together. Keep up the good work Malcom!
Thanks for the tutorial and the awesome plugin Malcolm. Just for anyone using this tutorial now, it's slightly outdated. The hand_pose_controller node now extends hand_pose_detector, so only the hand_pose_controller node is necessary. Check the newer Hand Pose Detector 2 tutorial on the channel for the newer version of the plugin.
I believe sign language is more on gestures than just poses. It is something that's been discussed. This pose approach lets users define new poses fairly easily, but gesture detection would probably take machine learning. It would probably be able to detect more reliably, but defining new gestures would take recording data-sets and running ML training models. I figure alternate specialized approaches will be created as needed, with the complexity and flexibility matching the needs.
@@MalcolmANixon You are correct about the gestures being a big part. Non-manual markers such as facial cues and upper body orientation as well. That being said, current consumer-level facial and body trackers may be sufficient to make up the difference. I’ve starred this project and I’m trying to get my school’s CompSci and Linguistics departments to collaborate and train reliable ML models. It won’t be perfect, but no NLP project is. I’m hoping that it could at least perform function calling with a limited vocabulary, but the dream would be to generate glosses in ASLWrite, si5s, and the like to generate whole searchable dictionaries for each sign language (without needing to rely on spoken language as most current ones do). Keep up the good work!🤞🤞
Awesome tutorial! This is exactly the kind of tutorials we need more of for GodotXR that show how to create game mechanics out of the components of the xr tools and put it all together. Keep up the good work Malcom!
I'm shocked how playable this is without controllers!
Thanks for the tutorial and the awesome plugin Malcolm. Just for anyone using this tutorial now, it's slightly outdated. The hand_pose_controller node now extends hand_pose_detector, so only the hand_pose_controller node is necessary. Check the newer Hand Pose Detector 2 tutorial on the channel for the newer version of the plugin.
Amazing stuff! Also: Sign Language parsers when?
I believe sign language is more on gestures than just poses. It is something that's been discussed. This pose approach lets users define new poses fairly easily, but gesture detection would probably take machine learning. It would probably be able to detect more reliably, but defining new gestures would take recording data-sets and running ML training models.
I figure alternate specialized approaches will be created as needed, with the complexity and flexibility matching the needs.
@@MalcolmANixon You are correct about the gestures being a big part. Non-manual markers such as facial cues and upper body orientation as well. That being said, current consumer-level facial and body trackers may be sufficient to make up the difference. I’ve starred this project and I’m trying to get my school’s CompSci and Linguistics departments to collaborate and train reliable ML models. It won’t be perfect, but no NLP project is. I’m hoping that it could at least perform function calling with a limited vocabulary, but the dream would be to generate glosses in ASLWrite, si5s, and the like to generate whole searchable dictionaries for each sign language (without needing to rely on spoken language as most current ones do).
Keep up the good work!🤞🤞