SIGGRAPH 2021 Course
Welcome to the CVPR 2020 tutorial on Neural Rendering!
The tutorial will be streamed on 15th June via YouTube:
We will also have live Q&A after each talk. To join, please follow the Zoom instructions on the CVPR 2020 website (morning session, afternoon session). You can ask questions during the live Q&A, or via the YouTube or CVPR chats.
Neural rendering is a new class of deep image and video generation approaches that enable explicit or implicit control of scene properties such as illumination, camera parameters, pose, geometry, appearance, and semantic structure. It combines generative machine learning techniques with physical knowledge from computer graphics to obtain controllable and photo-realistic outputs.
This tutorial teaches the fundamentals of neural rendering and summarizes recent trends and applications. Starting with an overview of the underlying graphics, vision and machine learning concepts, we discuss critical aspects of neural rendering approaches. Specifically, our emphasis is on what aspects of the generated imagery can be controlled, which parts of the pipeline are learned, explicit vs. implicit control, generalization, and stochastic vs. deterministic synthesis. Next, we focus on the many important use cases for the described algorithms such as novel view synthesis, semantic photo manipulation, facial and body reenactment, relighting, free-viewpoint video, and the creation of photo-realistic avatars for virtual and augmented reality telepresence. Finally, we conclude with a discussion of the social implications of this technology and investigate open research problems.
Time | Topic | Speaker |
---|---|---|
09:00–09:15 | Welcome and Introduction | Michael Zollhöfer |
09:15–09:30 | Fundamentals, Taxonomy, Neural Rendering | Ayush Tewari |
Semantic Photo Synthesis and Manipulation | ||
09:30–09:40 | Overview | Jun-Yan Zhu |
09:40–10:00 | Semantic Image Synthesis with Spatially-Adaptive Normalization | Taesung Park |
10:00–10:30 | Coffee Break | |
Facial Reenactment & Body Reenactment | ||
10:25–10:35 | Overview | Justus Thies |
10:35–11:00 | Neural Rendering for High-Quality Synthesis of Human Portrait Video and Images | Christian Theobalt |
11:00–11:20 | Neural Rendering for Virtual Avatars | Aliaksandra Shysheya |
Novel View Synthesis | ||
11:20–11:35 | Overview | Vincent Sitzmann |
11:30–11:50 | Neural Rerendering in the Wild | Moustafa Meshry |
11:50–12:10 | NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis | Ben Mildenhall |
12:10–13:20 | Lunch Break | |
Learning to Relight | ||
13:20–13:30 | Overview | Zexiang Xu |
13:30–13:50 | Multi-view Relighting Using a Geometry-Aware Network | Julien Philip |
13:50–14:10 | Neural Inverse Rendering | Abhimitra Meka |
Free Viewpoint Videos | ||
14:10–14:20 | Overview | Sean Fanello |
14:20–14:40 | Neural Rendering for Performance Capture | Rohit K. Pandey |
14:40–15:00 | Neural Volumes: Learning Dynamic Renderable Volumes from Images | Stephen Lombardi |
15:00–15:30 | Coffee Break | |
15:30–15:45 | Social Implications, Open Challenges, Conclusion | Ohad Fried |
15:45–16:15 | Followup Discussion |
The tutorial is based on this Eurographics 2020 state-of-the-art report.
Ayush Tewari, Ohad Fried, Justus Thies, Vincent Sitzmann, Stephen Lombardi, Kalyan Sunkavalli, Ricardo Martin-Brualla, Tomas Simon, Jason Saragih, Matthias Nießner, Rohit K. Pandey, Sean Fanello, Gordon Wetzstein, Jun-Yan Zhu, Christian Theobalt, Maneesh Agrawala, Eli Shechtman, Dan B. Goldman, Michael Zollhöfer.
June 15th, 2020, 9:00–16:30 PT.
CVPR 2020 is now a fully virtual conference.