Commit a3f665e2 authored by Andrey Filippov's avatar Andrey Filippov

CLAUDE: Fix renderNadirSequence(): missing ref->scene transform of pXpYD

getNadirDisparityMaps() returns pXpYD in the SEGMENT REFERENCE frame
(Cuas.transformFromVirtual() output), but renderNadirSequence() passed it
to renderGPUFromDSI() with ZERO3 pose and the scene as its own reference,
so each scene image was sampled at reference-frame coordinates. Each nadir
frame thus carried an extra ref->scene terrain warp: effective camera
poses vs the segment reference were ~doubled (ground translation 2x, small
rotations ~2x), and the identity transform also canceled ERS compensation.
On flat terrain the extra plane homography mimics a valid pose change, so
COLMAP converged self-consistently but disagreed with the exported poses
(Sim(3)/free monocular scale absorbs the 2x translation; rotations do not
scale, explaining the large angular residuals, worst on high-rotation-rate
sequences).

Now transformToScenePxPyD() maps the reference-frame pXpYD into each
scene's own frame (pose + ERS, same machinery as the verified
frozen-terrain path) before rendering with ZERO3/self, where the identity
pass is an exact pass-through. Output becomes a global-shutter virtual
view at each scene's nominal pose. Also converts disparity to the scene
frame for 16-sensor parallax compensation, and drops the defensive clone
(the transform returns new arrays). Docs updated to state the frame
conventions.
Co-authored-by: 's avatarClaude <claude@elphel.com>
parent a44f6c06
...@@ -10823,13 +10823,16 @@ java.lang.NullPointerException ...@@ -10823,13 +10823,16 @@ java.lang.NullPointerException
/** /**
* Compute per-scene disparity maps in each scene's own camera frame, * Compute per-scene pXpYD maps for nadir rendering, where each scene is
* suitable for nadir rendering where each scene is rendered from its own * rendered from its own position without a terrain-reference warp.
* position without a terrain-reference warp.
* *
* Uses Cuas.transformFromVirtual() to project the segment reference's SfM * Uses Cuas.transformFromVirtual(): for each scene, the returned pXpYD is
* terrain disparity into each scene's own frame. For the reference scene * in the SEGMENT REFERENCE frame (reference pX, pY and reference disparity),
* itself the transform is identity (ZERO3 pose). * resampled so it corresponds to the uniform tile grid of a virtual camera
* at that scene's pose. It is NOT yet in the scene's own frame -
* renderNadirSequence() applies transformToScenePxPyD() per scene before
* rendering. For the reference scene itself the mapping is identity
* (ZERO3 pose).
* *
* @param clt_parameters processing parameters * @param clt_parameters processing parameters
* @param terrain_disparity SfM terrain disparity in ref_scene's frame * @param terrain_disparity SfM terrain disparity in ref_scene's frame
...@@ -10837,8 +10840,9 @@ java.lang.NullPointerException ...@@ -10837,8 +10840,9 @@ java.lang.NullPointerException
* @param quadCLTs all scenes in the segment (may contain nulls) * @param quadCLTs all scenes in the segment (may contain nulls)
* @param ref_scene segment index/reference scene * @param ref_scene segment index/reference scene
* @param debugLevel * @param debugLevel
* @return per-scene pXpYD [nscene][ntiles][3]; null entries for scenes * @return per-scene reference-frame pXpYD [nscene][ntiles][3] on each
* without registered poses * scene's virtual-view grid; null entries for scenes without
* registered poses
*/ */
public static double[][][] getNadirDisparityMaps( public static double[][][] getNadirDisparityMaps(
CLTParameters clt_parameters, CLTParameters clt_parameters,
...@@ -10914,10 +10918,14 @@ java.lang.NullPointerException ...@@ -10914,10 +10918,14 @@ java.lang.NullPointerException
* each scene with natural parallax from its own position — suitable as * each scene with natural parallax from its own position — suitable as
* input for OpenMVS / COLMAP dense reconstruction. * input for OpenMVS / COLMAP dense reconstruction.
* *
* Each scene is rendered as its own reference (scene_xyz/atr = ZERO3, * The per-scene pXpYD from getNadirDisparityMaps() are reference-frame
* ref_scene = scene itself) using the per-scene pXpYD from * coordinates (transformFromVirtual() output) corresponding to the uniform
* getNadirDisparityMaps(). ERS rolling-shutter velocities are still * grid of a virtual (ERS-free) camera at each scene's pose. Here they are
* applied for image quality. * transformed into each scene's own frame with transformToScenePxPyD()
* (applying both the interscene pose and ERS), then rendered with the scene
* as its own reference (ZERO3 pose - the identity transform inside
* preRenderGPUFromDSI() is a pass-through). The result is a global-shutter
* virtual view at each scene's nominal pose.
* *
* Output suffix: {@code <ref_ts>-NADIR-MERGED} * Output suffix: {@code <ref_ts>-NADIR-MERGED}
* *
...@@ -10948,25 +10956,30 @@ java.lang.NullPointerException ...@@ -10948,25 +10956,30 @@ java.lang.NullPointerException
for (int nscene = 0; nscene < selected_scenes.length; nscene++) { for (int nscene = 0; nscene < selected_scenes.length; nscene++) {
if (nadir_pXpYD[nscene] == null) continue; if (nadir_pXpYD[nscene] == null) continue;
String ts = selected_scenes[nscene].getImageName(); String ts = selected_scenes[nscene].getImageName();
double[] scene_xyz = ZERO3;
double[] scene_atr = ZERO3;
// Apply ERS rolling-shutter velocity corrections for this scene // Apply ERS rolling-shutter velocity corrections for this scene
if (selected_scenes[nscene] != ref_scene) { if (selected_scenes[nscene] != ref_scene) {
scene_xyz = ers_reference.getSceneXYZ(ts);
scene_atr = ers_reference.getSceneATR(ts);
if ((scene_xyz == null) || (scene_atr == null)) continue; // no pose (getNadirDisparityMaps() skipped it too)
double[] scene_ers_xyz_dt = ers_reference.getSceneErsXYZ_dt(ts); double[] scene_ers_xyz_dt = ers_reference.getSceneErsXYZ_dt(ts);
double[] scene_ers_atr_dt = ers_reference.getSceneErsATR_dt(ts); double[] scene_ers_atr_dt = ers_reference.getSceneErsATR_dt(ts);
selected_scenes[nscene].getErsCorrection().setErsDt( selected_scenes[nscene].getErsCorrection().setErsDt(
scene_ers_xyz_dt, // double [] ers_xyz_dt scene_ers_xyz_dt, // double [] ers_xyz_dt
scene_ers_atr_dt);// double [] ers_atr_dt scene_ers_atr_dt);// double [] ers_atr_dt
} }
// Select tile array: use a shallow-cloned copy when undistorting so the // nadir_pXpYD is in the segment reference frame - transform to this scene's
// original nadir_pXpYD is preserved for a potential second (non-undistorted) pass. // own frame (pose + ERS). Returns new arrays, so nadir_pXpYD stays intact
double[][] pXpYD_scene; // for the second (rectilinear) pass.
double[][] pXpYD_scene = transformToScenePxPyD(
nadir_pXpYD[nscene], // final double [][] reference_pXpYD,
scene_xyz, // final double [] scene_xyz,
scene_atr, // final double [] scene_atr,
ref_scene, // final QuadCLT reference_QuadClt,
selected_scenes[nscene]); // final QuadCLT scene_QuadClt
if (undistort) { if (undistort) {
pXpYD_scene = nadir_pXpYD[nscene].clone(); // shallow: tile refs copied
for (int t = 0; t < pXpYD_scene.length; t++) {
if (pXpYD_scene[t] != null) pXpYD_scene[t] = pXpYD_scene[t].clone();
}
undistortPxPy(pXpYD_scene, selected_scenes[nscene].getErsCorrection()); undistortPxPy(pXpYD_scene, selected_scenes[nscene].getErsCorrection());
} else {
pXpYD_scene = nadir_pXpYD[nscene];
} }
// Render from scene's own position: ZERO3 offset, scene is its own reference // Render from scene's own position: ZERO3 offset, scene is its own reference
ImagePlus imp_scene = QuadCLT.renderGPUFromDSI( ImagePlus imp_scene = QuadCLT.renderGPUFromDSI(
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment