Physics-Informed Learning of Characteristic Trajectories for Smoke Reconstruction

Yiming Wang1, Siyu Tang1, Mengyu Chu2,
1ETH Zurich, 2Peking University, SKL of General AI
Paper arXiv Video Code
Teaser image of fluid reconstruction method

We present a physics-informed fluid reconstruction method using a novel Neural Characteristic Trajectory representation to preserve both short-term physics constraints and long-term conservation. In the challenging scene with smoke and obstacles, our method reconstructs decomposed radiance fields, obstacle geometry (serving as boundary constraints for smoke), smoke density, velocity, and trajectories from sparse-view RGB videos, and generates realistic renderings of novel views.

Abstract

We delve into the physics-informed neural reconstruction of smoke and obstacles through sparse-view RGB videos, tackling challenges arising from limited observation of complex dynamics. Existing physics-informed neural networks often emphasize short-term physics constraints, leaving the proper preservation of long-term conservation less explored.

We introduce Neural Characteristic Trajectory Fields, a novel representation utilizing Eulerian neural fields to implicitly model Lagrangian fluid trajectories. This topology-free, auto-differentiable representation facilitates efficient flow map calculations between arbitrary frames as well as efficient velocity extraction via auto-differentiation. Consequently, it enables end-to-end supervision covering long-term conservation and short-term physics priors.

Building on the representation, we propose physics-informed trajectory learning and integration into NeRF-based scene reconstruction. We enable advanced obstacle handling through self-supervised scene decomposition and seamless integrated boundary constraints. Our results showcase the ability to overcome challenges like occlusion uncertainty, density-color ambiguity, and static-dynamic entanglements.

Full Video

Decomposed Reconstruction

Novel view synthesis results are rendered for the full scene, static obstacles only, and dynamic smoke only, demonstrating that our method achieves significantly better scene decomposition and more detailed reconstruction.

Physics Attributes

To show a qualitative comparison between our method and PINF in terms of physical attributes reconstruction, we visualize the density, velocity, and vorticity of the reconstructed fluid from the front, side, and top views,

BibTeX

@inproceedings{Wang2024PICT,
  author = {Wang, Yiming and Tang, Siyu and Chu, Mengyu},
  title = {Physics-Informed Learning of Characteristic Trajectories for Smoke Reconstruction},
  year = {2024},
  url = {https://doi.org/10.1145/3641519.3657483},
  doi = {10.1145/3641519.3657483},
  booktitle = {ACM SIGGRAPH 2024 Conference Papers},
  articleno = {53},
  numpages = {11},
  series = {SIGGRAPH '24}
}