空间目标球谐姿态多视角生成

    Spherical harmonic pose-aware multi-view generation for space target

    • 在轨服务与主动碎片清理等任务中,受观测几何、在轨机动与通信链路等因素约束,往往仅能获得非合作航天器目标的单帧或少帧图像观测;同时,视角跨度较大且可见性呈现不连续特征。在此条件下,现有单图新视角合成方法在大角度切换与强镜面高光反射场景中易出现几何漂移、遮挡关系失配与纹理连贯性下降等问题。针对上述问题,构建了不依赖显式三维重建的视角条件生成方法。该方法以潜空间扩散模型为核心,构建几何—语义双路径的相机条件注入框架:在几何路径中,引入姿态感知球谐编码(Pose-Aware Spherical Harmonics Encoding, PASHE),通过实球谐展开逐像素射线方向,并融合相机位置的多频编码与紧凑位姿嵌入,形成像素级几何条件,以增强局部可见性建模与细节稳定性;在语义路径中,引入相机条件语义融合模块(Camera-Conditioned Semantic Fusion, CCSF),利用注意力机制以全局相对位姿调制源图语义并构建全局语义先验,从而在扩散去噪过程中约束整体结构关系与外观一致性。实验结果表明,该方法在航天器多视图数据集上取得更优综合性能,其中结构相似性SSIM达到0.8854,感知质量指标FID降至9.107,并在细薄结构保持、遮挡边界过渡及镜面反射区域的纹理连贯性方面表现更为稳定。结果表明,所提双路径相机条件注入在保持框架紧凑与可实现性的同时,可兼顾局部几何约束与全局语义约束,有效提升非合作航天器单图多视角生成的稳定性与一致性,可为在轨目标的单帧多视理解与辅助决策提供参考。

       

      Abstract:
      Objective On-orbit servicing, active debris removal, and space situational awareness require reliable interpretation of non-cooperative spacecraft under highly constrained observation conditions. In practical missions, only a single frame or a few frames are often available because of limited observation geometry, restricted maneuvering opportunities, and communication constraints. Under such conditions, large viewpoint gaps, discontinuous visibility, strong specular reflections, and thin appendages such as solar panels and antennas make novel view synthesis particularly challenging. Existing single-image view synthesis methods can generate target views conditioned on relative camera poses, but they still tend to suffer from geometric drift, unstable occlusion ordering, and degraded texture coherence when viewpoint changes become large or when appearance is dominated by view-dependent reflections. These limitations restrict the usefulness of generated images for subsequent target understanding and downstream space perception tasks. To address these issues, a camera-conditioned diffusion framework is developed for multi-view image generation of space targets from a single image, with the goal of improving structural consistency, appearance stability, and controllability under large pose variations.
      Methods A latent diffusion framework is adopted as the generation backbone, and camera information is injected through two complementary pathways, as illustrated in Fig.1. The first pathway is a geometric pathway based on Pose-Aware Spherical Harmonic Encoding (PASHE). Instead of using only a low-dimensional pose vector or simple positional encoding, PASHE explicitly encodes three types of geometric information: per-pixel ray directions, camera-center positions, and compact relative pose embeddings. Real spherical harmonic bases are used to expand ray directions so that high-order directional variations on the unit sphere can be represented in a structured way. Multi-frequency Fourier encoding is used for camera positions to preserve spatial anchoring across different scales, and a compact pose embedding is introduced to describe the relative transformation from the source view to the target view. These three components are fused into a unified pose-aware representation, enabling the model to inject pixel-level geometric conditions into the denoising process and to better maintain thin structures, occlusion boundaries, and local visibility transitions, as shown in Fig.2. The second pathway is a semantic pathway based on Camera-Conditioned Semantic Fusion (CCSF), as illustrated in Fig.3. Source-image semantics are extracted by a pretrained Contrastive Language-Image Pre-training encoder and then fused with global relative pose information. Through shared latent projection and attention-based interaction, camera conditions are used to modulate semantic features in a structured manner rather than being appended as a simple vector. This design provides a global semantic prior that stabilizes overall structure layout, appearance continuity, and cross-view semantic alignment during diffusion denoising. During training, the source image and target image are encoded into the latent space of a Variational Autoencoder, noise is added according to a diffusion schedule, and the model is optimized with an epsilon-prediction objective under the joint guidance of PASHE, CCSF, and source-view latent features. During inference, Denoising Diffusion Implicit Models sampling with classifier-free guidance is adopted. Shared random seeds and deterministic schedules are used for multiple target views of the same object to enhance cross-view detail consistency. Experiments are conducted on the Spacecraft 3D Model Dataset, which contains 107 spacecraft models covering different target types. An object-level split is used to evaluate generalization to unseen objects. Multi-view renderings are generated from 24 viewpoints per object, and the proposed method is compared with representative baseline methods including Zero-1-to-3, SyncDreamer, Free3D, EscherNet, and SV3D.
      Results and Discussions Quantitative evaluation demonstrates that the proposed method achieves the best overall performance on the spacecraft multi-view dataset. Specifically, it obtains a Structural Similarity Index Measure of 0.8854 and reduces the Fréchet Inception Distance to 9.107, while also yielding the lowest Learned Perceptual Image Patch Similarity among the compared methods (Tab.1). Although the Structural Similarity Index Measure and Peak Signal-to-Noise Ratio are close to the strongest baselines, the lower perceptual and distributional errors indicate that the generated views are more consistent with the target-image distribution and exhibit better perceptual realism. This suggests that the proposed method improves image quality without sacrificing structural fidelity. Qualitative results further verify these advantages under large viewpoint changes. As shown in Fig.4, the proposed method better preserves the relative proportions between the spacecraft body and solar panels, maintains cleaner boundaries and backgrounds, and produces more stable occlusion transitions under azimuth rotations of 150°, 240°, and 300°. In contrast, competing methods often exhibit body distortion, appendage breakage, texture inconsistency, or silhouette-like artifacts. These observations indicate that the explicit dual-path camera conditioning is more effective in maintaining structural coherence across large pose changes than single-path conditioning strategies.Ablation results provide additional evidence for the respective roles of PASHE and CCSF. When only a simple linear feature concatenation with a six-dimensional pose vector is used, the model shows limited stability and is prone to thin-structure deformation and local texture mismatch. Adding PASHE significantly improves local geometry preservation and detail stability, whereas adding CCSF improves global semantic continuity and reduces appearance drift. When both modules are enabled, the best overall results are obtained in all four image-quality metrics (Tab.2). Angle-wise ablation further shows that the combined model remains the most robust under small, medium, and especially large cross-view gaps, indicating strong complementarity between local geometric constraints and global semantic priors (Tab.3). These results demonstrate that PASHE mainly strengthens pixel-level geometry awareness, while CCSF mainly stabilizes semantic consistency under pose variation.
      Conclusions A dual-path camera-conditioned diffusion framework is established for single-image multi-view generation of non-cooperative space targets. By combining PASHE and CCSF, the method introduces complementary geometric and semantic constraints into latent diffusion and improves the controllability and stability of generated views under large viewpoint changes. Experimental results show that the method produces more reliable structures, cleaner occlusion transitions, and better appearance coherence than several representative baseline methods. The framework is compact, does not require explicit three-dimensional reconstruction during generation, and remains effective in thin-structure and specular-reflection scenarios that are common in spacecraft imagery. These properties make it promising for single-frame multi-view understanding of space targets and for supporting downstream tasks such as pose initialization, visibility inference, and perception-assisted mission planning. Future work can further investigate adaptation to real on-orbit imagery, robustness under more complex imaging degradations, and tighter integration with downstream three-dimensional reconstruction and target perception pipelines.

       

    /

    返回文章
    返回