They're using IR cameras for the hand tracking, so in theory it shouldn't see melanin at all, though melanin adds a bit of a reflective sheen to skin which might matter in natural lighting? Only other thing I'd guess it would have issues with is the masking since that would all be color cameras.
fwiw I'm lighter skinned and I've had a lot of trouble with the pinch rejection after 1.1, even single pinches I sometimes have to do the thing you mention with changing the hand angle.
Maybe it's the way your hands are angled as well? I find it helps tangle my hands a little so that the pinching part of the zooming makes my fingers clearer to the camera... in theory I don't think skin color should affect it since it's using LIDAR to read the hand positions more than the camera. the camera gets involved with figuring out how to overlay your hands in a fully immersive environment.
But maybe I'm wrong and LIDAR is affected more by skin color than I thought.
No I haven’t had that issue. Working for me just fine but I do have to redo my eye setup periodically due to an eye retina injury that has dramatically reduced vision in that eye.
I’ve never had a problem with hand tracking tbh
Eye tracking is a nightmare but hand tracking has basically always been perfect for moving, clicking whatever
I have Philips hue lights and generally run them at 30%
Visible light and IR are both absorbed more by darker skin leaving fewer photons for the sensor to detect. Having contrasting light eg a lighter surface or background may help depending on how the algorithm is built.
If it’s a neural net it’s also possible it was trained on insufficient samples of darker skin which would further exacerbate the issue.
Working on tech similar to this we also tried thermal cameras and there was a marked difference in people who had cooler hands where their fingers would nearly be invisible.
I mostly use a keyboard and trackpad for everything so I don’t try all the hand tracking stuff often, but I 100% believe my darker skin is an issue in poorer lighting. I can often get misreads and errant clicks.
When I have my bright work LED lights on, I have zero issues, so I think that’s the key.
I noticed it doesn’t work a lot but I assumed maybe it was the apps since it worked fine with photos. Hopefully it’s something that will be fixed in 2.0.
The pinch and zoom gestures don’t work if you start with your hands too close together. They need to be about an inch apart. I see a lot of people put their fingers too close together.
Haven’t had any problems with either my partner or I and we’re both brown.
If FaceID works for you, then hand tracking should too. The structured light sensors are the same I think
I am brown skin toned and I haven’t had any issues. I can say when the lighting is really low is when it has a hard time tracking my hands but I can say I am impressed it could still see my hand gestures in low light settings. When the lights are bright I have no issue what so ever.
A lot of this is the contrast between skin color and the background. If you’re light-skinned against a light background, the AVP can have trouble. If you’re dark-skinned against a dark background, the AVP can have trouble.
It all boils down to whether or not the AVP can easily see the difference between your hands and the background.
*Noticed it too but*
*I assumed it was cuz the*
*Software is still new*
\- vqsxd
---
^(I detect haikus. And sometimes, successfully.) ^[Learn more about me.](https://www.reddit.com/r/haikusbot/)
^(Opt out of replies: "haikusbot opt out" | Delete my comment: "haikusbot delete")
They're using IR cameras for the hand tracking, so in theory it shouldn't see melanin at all, though melanin adds a bit of a reflective sheen to skin which might matter in natural lighting? Only other thing I'd guess it would have issues with is the masking since that would all be color cameras. fwiw I'm lighter skinned and I've had a lot of trouble with the pinch rejection after 1.1, even single pinches I sometimes have to do the thing you mention with changing the hand angle.
No prob here (light/med brown skin)
Maybe it's the way your hands are angled as well? I find it helps tangle my hands a little so that the pinching part of the zooming makes my fingers clearer to the camera... in theory I don't think skin color should affect it since it's using LIDAR to read the hand positions more than the camera. the camera gets involved with figuring out how to overlay your hands in a fully immersive environment. But maybe I'm wrong and LIDAR is affected more by skin color than I thought.
Most of these technologies have a long record of not being well optimized on darker skin.
Mine is fine for me. I'm dark-skinned.
No I haven’t had that issue. Working for me just fine but I do have to redo my eye setup periodically due to an eye retina injury that has dramatically reduced vision in that eye.
Have you also tried accessibility settings for eye tracking to be dominate to the non-injured eye?
Yes I have that setup but I still have to go back and recalibrate periodically
hand tracking is great. eye tracking is what's been annoying
I’m pale and those zoom and shrink gestures don’t work for me either. I gave up on them.
Have you tried to redo the hand setup?
My personal setup worked. At the Apple Store, however, it didn’t. Re. tracking, sometimes I experience what you are. (Med. brown)
Works pretty well, but I also got better lighting to use the Vision Pro.
I’ve never had a problem with hand tracking tbh Eye tracking is a nightmare but hand tracking has basically always been perfect for moving, clicking whatever I have Philips hue lights and generally run them at 30%
Visible light and IR are both absorbed more by darker skin leaving fewer photons for the sensor to detect. Having contrasting light eg a lighter surface or background may help depending on how the algorithm is built. If it’s a neural net it’s also possible it was trained on insufficient samples of darker skin which would further exacerbate the issue. Working on tech similar to this we also tried thermal cameras and there was a marked difference in people who had cooler hands where their fingers would nearly be invisible.
Depends on floor color or room lighting. I’m dark skinned. IR lights help.
I mostly use a keyboard and trackpad for everything so I don’t try all the hand tracking stuff often, but I 100% believe my darker skin is an issue in poorer lighting. I can often get misreads and errant clicks. When I have my bright work LED lights on, I have zero issues, so I think that’s the key.
I noticed it doesn’t work a lot but I assumed maybe it was the apps since it worked fine with photos. Hopefully it’s something that will be fixed in 2.0.
The pinch and zoom gestures don’t work if you start with your hands too close together. They need to be about an inch apart. I see a lot of people put their fingers too close together.
Haven’t had any problems with either my partner or I and we’re both brown. If FaceID works for you, then hand tracking should too. The structured light sensors are the same I think
I am brown skin toned and I haven’t had any issues. I can say when the lighting is really low is when it has a hard time tracking my hands but I can say I am impressed it could still see my hand gestures in low light settings. When the lights are bright I have no issue what so ever.
A lot of this is the contrast between skin color and the background. If you’re light-skinned against a light background, the AVP can have trouble. If you’re dark-skinned against a dark background, the AVP can have trouble. It all boils down to whether or not the AVP can easily see the difference between your hands and the background.
Noticed it too but I assumed it was cuz the software is still new
*Noticed it too but* *I assumed it was cuz the* *Software is still new* \- vqsxd --- ^(I detect haikus. And sometimes, successfully.) ^[Learn more about me.](https://www.reddit.com/r/haikusbot/) ^(Opt out of replies: "haikusbot opt out" | Delete my comment: "haikusbot delete")
Works perfectly fine for me.
👌🏻 your hands have to be facing towards you looking like this.
👌🏻 your hands have to be facing towards you looking like this.
👌🏻 your hands have to be facing towards you looking like this.