Conventional NeRF and its variations require considerable computational resources, often exceeding what is typically available in constrained environments. Additionally, the limited video memory capacity of client devices imposes significant constraints on processing and rendering large resources simultaneously in real time. The considerable resource demand poses a crucial challenge when rendering large scenes in real time, requiring rapid loading and processing of large data sets.
To address the challenges encountered in real-time rendering of large scenes, researchers from the University of Science and technology of China proposed a method called City on the Web. Inspired by traditional graphics methods used to handle large-scale scenes, they divide the scene into manageable blocks and incorporate different levels of detail (LOD) to represent it.
Radiance field baking techniques are used to precompute and store render primitives in 3D atlas textures organized within a sparse grid in each block, facilitating real-time rendering. However, loading all atlas textures into a single shader is not feasible due to inherent shader resource limitations. Consequently, the scene is represented as a hierarchy of segmented blocks, each represented by a dedicated shader during the rendering process.
Employing a “divide and conquer” strategy, they ensure that each block has ample rendering capacity to faithfully reconstruct intricate details within the scene. Additionally, to maintain high fidelity in the rendered output during the training phase, they simulate the combination of multiple shaders aligned with the rendering process.
These block- and level-of-detail (LOD)-based representations enable dynamic resource management, streamlining the loading and unloading process in real time based on the viewer's position and field of view. This adaptive loading approach significantly reduces bandwidth and memory requirements for rendering large scenes, resulting in smoother user experiences, especially on less powerful devices.
The experiments performed illustrate that City-on-Web achieves large-scale photorealistic scene rendering at 32 frames per second (FPS) at 1080p resolution, using an RTX 3060 GPU. It uses only 18% of the VRAM and 16 % of payload size compared to existing mesh-based methods.
The combination of block partitioning and level of detail (LOD) integration has significantly decreased the payload on the web platform while improving the efficiency of resource management. This approach ensures high-fidelity rendering quality by maintaining consistency between the training process and the rendering phase.
Review the Paper and Project. All credit for this research goes to the researchers of this project. Also, don't forget to join. our SubReddit of more than 35,000 ml, 41k+ Facebook community, Discord channel, LinkedIn Graboveand Electronic newsletterwhere we share the latest news on ai research, interesting ai projects and more.
If you like our work, you'll love our newsletter.
Arshad is an intern at MarktechPost. He is currently pursuing his international career. Master's degree in Physics from the Indian Institute of technology Kharagpur. Understanding things down to the fundamental level leads to new discoveries that lead to the advancement of technology. He is passionate about understanding nature fundamentally with the help of tools such as mathematical models, machine learning models, and artificial intelligence.
<!– ai CONTENT END 2 –>
