![Yifu Wang](/img/default-banner.jpg)
- 4
- 2 150
Yifu Wang
Приєднався 14 вер 2023
Sketch2Scene: Automatic Generation of Interactive 3D Game Scenes from User's Casual Sketches
Abstract: 3D Content Generation is at the heart of many computer graphics applications, including video gaming, film-making, virtual and augmented reality, etc. This paper proposes a novel deep-learning based approach for automatically generating interactive and playable 3D game scenes, all from the user's casual prompts such as a hand-drawn sketch. Sketch-based input offers a natural, and convenient way to convey the user's design intention in the content creation process. To circumvent the data-deficient challenge in learning (i.e. the lack of large training data of 3D scenes), our method leverages a pre-trained 2D denoising diffusion model to generate a 2D image of the scene as the conceptual guidance. In this process, we adopt the isometric projection mode to factor out unknown camera poses while obtaining the scene layout. From the generated isometric image, we use a pre-trained image understanding method to segment the image into meaningful parts, such as off-ground objects, trees, and buildings, and extract the 2D scene layout. These segments and layouts are subsequently fed into a procedural content generation (PCG) engine, such as a 3D video game engine like Unity or Unreal, to create the 3D scene. The resulting 3D scene can be seamlessly integrated into a game development environment and is readily playable. Extensive tests demonstrate that our method can efficiently generate high-quality and interactive 3D game scenes with layouts that closely follow the user's intention.
Project Page: xrvisionlabs.github.io/Sketch2Scene/
Project Page: xrvisionlabs.github.io/Sketch2Scene/
Переглядів: 830
Відео
OpenMAVIS: OpenMAVIS: Open-Source Multi-Camera Augmented Visual-Inertial SLAM on ORB-SLAM3
Переглядів 5922 місяці тому
code link: github.com/MAVIS-SLAM/OpenMAVIS
MAVIS: Multi-Camera Augmented Visual-Inertial SLAM using SE2(3) Based Exact IMU Pre-integration
Переглядів 1394 місяці тому
We present a novel optimization-based Visual-Inertial SLAM system designed for multiple partially overlapped camera systems, named MAVIS. Our framework fully exploits the benefits of wide field-of-view from multi-camera systems, and the metric scale measurements provided by an inertial measurement unit (IMU). We introduce an improved IMU pre-integration formulation based on the exponential func...
MAVIS: Multi-Camera Augmented Visual-Inertial SLAM using SE2(3) Based Exact IMU Pre-integration
Переглядів 59911 місяців тому
This video presents MAVIS framework, a novel optimization-based Visual-Inertial SLAM system designed for multiple partially overlapped camera systems. The method also introduces an improved IMU pre-integration formulation based on the exponential function of an automorphism of SE_2(3), which can effectively enhance tracking performance under fast rotational motion and extended integration time....