This research aims at the localization and mapping challenges in the GNSS-denied and non-structural environment and serves the 3D perception ability of the unmanned platform. More specifically, it consists of (1) multi-constraints-based localization and mapping under the complex platform motion and (2) collaborative reconstruction of human motion and environment (published 1 PAMI and 4 CVPR papers). This research won the Best Paper Award at the 2017 International Mobile Mapping Conference and the 2018 featured paper of ISPRS J P&RS.