Dear Sergio, thank you for the kind words. The tracking is photometric and thus a global shutter camera is necessary. Additionally, good geometric calibration is required but we will release a README on how we did the calibration for the realsense D455. We are still figuring out the licencse. For all code where it is possible, we will use a free and permissive licencse (maybe MIT or BSD). For the parts of the code that are derivative work, we might have to follow the license of the code we build on, e.g. GPL. You can follow the github for updates and we hope to release first parts of the code and data in November and everything until Christmas. Best Lukas
@@illosuth The paper is available through the project or GitHub page and I'll link it here for your convenience: arxiv.org/abs/2111.07418. We will release the code before Christmas, thanks for your patience :)
Dear Chengzhi Hong, we used the D455 because it is a global shutter camera that is relatively easy to interface with. For the demo, we use only the mono RGB camera and not the stereo camera. For the demo at CoRL we did stick them shut to make the point clear. We will also demo at 3DV and will do the same so that it is easy to see that we are not using the stereo cameras.
that realsense color camera is a rolling shutter camera. We test on vehicles and there is a huge amount of rolling effect. How did you propose to solve that for accurate motion stereo estimate?
Dear Yuan Shenghai, we use the D455 that has a global shutter RGB camera (I think earlier versions did not have a global shutter RGB camera). We cite from the official Realsense website (www.intelrealsense.com/depth-camera-d455/): "To improve the RGB image and the correspondence between the depth and RGB images, the RGB sensor includes a global shutter and is matched to the depth FOV." We hope that helps, maybe you can also switch to D455 :)
Dear pan kao, we think that studying Multi-View Geometry (see the lectures of Prof. Cremers) and deep learning (e.g. the lectures of Justin Johnsson) is a good start! Hope this helps :)
Can we use T265 since its global shutter too and its fish eye lens has wider FOV, hence can reconstruct larger area with smaller movement..but not RGB
Amazing work!
3D reconstruction is awesome! So it can use any common rgb camera? What license are you going to use for the code?
Dear Sergio, thank you for the kind words. The tracking is photometric and thus a global shutter camera is necessary. Additionally, good geometric calibration is required but we will release a README on how we did the calibration for the realsense D455. We are still figuring out the licencse. For all code where it is possible, we will use a free and permissive licencse (maybe MIT or BSD). For the parts of the code that are derivative work, we might have to follow the license of the code we build on, e.g. GPL. You can follow the github for updates and we hope to release first parts of the code and data in November and everything until Christmas.
Best
Lukas
@@cvprtum I would like to read your paper and try the code whenever is possible. Thanks for sharing
@@illosuth The paper is available through the project or GitHub page and I'll link it here for your convenience: arxiv.org/abs/2111.07418. We will release the code before Christmas, thanks for your patience :)
@@cvprtum Vielen Dank Lukas! I will try to replicate your experiments, I will let you know how it goes.
why use the binocular camera in demo while the paper describes a monocular reconstruction method?
Dear Chengzhi Hong, we used the D455 because it is a global shutter camera that is relatively easy to interface with. For the demo, we use only the mono RGB camera and not the stereo camera. For the demo at CoRL we did stick them shut to make the point clear. We will also demo at 3DV and will do the same so that it is easy to see that we are not using the stereo cameras.
that realsense color camera is a rolling shutter camera. We test on vehicles and there is a huge amount of rolling effect. How did you propose to solve that for accurate motion stereo estimate?
Dear Yuan Shenghai, we use the D455 that has a global shutter RGB camera (I think earlier versions did not have a global shutter RGB camera). We cite from the official Realsense website (www.intelrealsense.com/depth-camera-d455/): "To improve the RGB image and the correspondence between the depth and RGB images, the RGB sensor includes a global shutter and is matched to the depth FOV." We hope that helps, maybe you can also switch to D455 :)
Oh, I'm a rookie. How can I achieve this!
Dear pan kao, we think that studying Multi-View Geometry (see the lectures of Prof. Cremers) and deep learning (e.g. the lectures of Justin Johnsson) is a good start! Hope this helps :)