Commit a3f665e2 authored by Andrey Filippov's avatar Andrey Filippov

CLAUDE: Fix renderNadirSequence(): missing ref->scene transform of pXpYD

getNadirDisparityMaps() returns pXpYD in the SEGMENT REFERENCE frame
(Cuas.transformFromVirtual() output), but renderNadirSequence() passed it
to renderGPUFromDSI() with ZERO3 pose and the scene as its own reference,
so each scene image was sampled at reference-frame coordinates. Each nadir
frame thus carried an extra ref->scene terrain warp: effective camera
poses vs the segment reference were ~doubled (ground translation 2x, small
rotations ~2x), and the identity transform also canceled ERS compensation.
On flat terrain the extra plane homography mimics a valid pose change, so
COLMAP converged self-consistently but disagreed with the exported poses
(Sim(3)/free monocular scale absorbs the 2x translation; rotations do not
scale, explaining the large angular residuals, worst on high-rotation-rate
sequences).

Now transformToScenePxPyD() maps the reference-frame pXpYD into each
scene's own frame (pose + ERS, same machinery as the verified
frozen-terrain path) before rendering with ZERO3/self, where the identity
pass is an exact pass-through. Output becomes a global-shutter virtual
view at each scene's nominal pose. Also converts disparity to the scene
frame for 16-sensor parallax compensation, and drops the defensive clone
(the transform returns new arrays). Docs updated to state the frame
conventions.
Co-authored-by: 's avatarClaude <claude@elphel.com>
parent a44f6c06
......@@ -10823,13 +10823,16 @@ java.lang.NullPointerException
/**
* Compute per-scene disparity maps in each scene's own camera frame,
* suitable for nadir rendering where each scene is rendered from its own
* position without a terrain-reference warp.
* Compute per-scene pXpYD maps for nadir rendering, where each scene is
* rendered from its own position without a terrain-reference warp.
*
* Uses Cuas.transformFromVirtual() to project the segment reference's SfM
* terrain disparity into each scene's own frame. For the reference scene
* itself the transform is identity (ZERO3 pose).
* Uses Cuas.transformFromVirtual(): for each scene, the returned pXpYD is
* in the SEGMENT REFERENCE frame (reference pX, pY and reference disparity),
* resampled so it corresponds to the uniform tile grid of a virtual camera
* at that scene's pose. It is NOT yet in the scene's own frame -
* renderNadirSequence() applies transformToScenePxPyD() per scene before
* rendering. For the reference scene itself the mapping is identity
* (ZERO3 pose).
*
* @param clt_parameters processing parameters
* @param terrain_disparity SfM terrain disparity in ref_scene's frame
......@@ -10837,8 +10840,9 @@ java.lang.NullPointerException
* @param quadCLTs all scenes in the segment (may contain nulls)
* @param ref_scene segment index/reference scene
* @param debugLevel
* @return per-scene pXpYD [nscene][ntiles][3]; null entries for scenes
* without registered poses
* @return per-scene reference-frame pXpYD [nscene][ntiles][3] on each
* scene's virtual-view grid; null entries for scenes without
* registered poses
*/
public static double[][][] getNadirDisparityMaps(
CLTParameters clt_parameters,
......@@ -10914,10 +10918,14 @@ java.lang.NullPointerException
* each scene with natural parallax from its own position — suitable as
* input for OpenMVS / COLMAP dense reconstruction.
*
* Each scene is rendered as its own reference (scene_xyz/atr = ZERO3,
* ref_scene = scene itself) using the per-scene pXpYD from
* getNadirDisparityMaps(). ERS rolling-shutter velocities are still
* applied for image quality.
* The per-scene pXpYD from getNadirDisparityMaps() are reference-frame
* coordinates (transformFromVirtual() output) corresponding to the uniform
* grid of a virtual (ERS-free) camera at each scene's pose. Here they are
* transformed into each scene's own frame with transformToScenePxPyD()
* (applying both the interscene pose and ERS), then rendered with the scene
* as its own reference (ZERO3 pose - the identity transform inside
* preRenderGPUFromDSI() is a pass-through). The result is a global-shutter
* virtual view at each scene's nominal pose.
*
* Output suffix: {@code <ref_ts>-NADIR-MERGED}
*
......@@ -10948,25 +10956,30 @@ java.lang.NullPointerException
for (int nscene = 0; nscene < selected_scenes.length; nscene++) {
if (nadir_pXpYD[nscene] == null) continue;
String ts = selected_scenes[nscene].getImageName();
double[] scene_xyz = ZERO3;
double[] scene_atr = ZERO3;
// Apply ERS rolling-shutter velocity corrections for this scene
if (selected_scenes[nscene] != ref_scene) {
scene_xyz = ers_reference.getSceneXYZ(ts);
scene_atr = ers_reference.getSceneATR(ts);
if ((scene_xyz == null) || (scene_atr == null)) continue; // no pose (getNadirDisparityMaps() skipped it too)
double[] scene_ers_xyz_dt = ers_reference.getSceneErsXYZ_dt(ts);
double[] scene_ers_atr_dt = ers_reference.getSceneErsATR_dt(ts);
selected_scenes[nscene].getErsCorrection().setErsDt(
scene_ers_xyz_dt, // double [] ers_xyz_dt
scene_ers_atr_dt);// double [] ers_atr_dt
}
// Select tile array: use a shallow-cloned copy when undistorting so the
// original nadir_pXpYD is preserved for a potential second (non-undistorted) pass.
double[][] pXpYD_scene;
// nadir_pXpYD is in the segment reference frame - transform to this scene's
// own frame (pose + ERS). Returns new arrays, so nadir_pXpYD stays intact
// for the second (rectilinear) pass.
double[][] pXpYD_scene = transformToScenePxPyD(
nadir_pXpYD[nscene], // final double [][] reference_pXpYD,
scene_xyz, // final double [] scene_xyz,
scene_atr, // final double [] scene_atr,
ref_scene, // final QuadCLT reference_QuadClt,
selected_scenes[nscene]); // final QuadCLT scene_QuadClt
if (undistort) {
pXpYD_scene = nadir_pXpYD[nscene].clone(); // shallow: tile refs copied
for (int t = 0; t < pXpYD_scene.length; t++) {
if (pXpYD_scene[t] != null) pXpYD_scene[t] = pXpYD_scene[t].clone();
}
undistortPxPy(pXpYD_scene, selected_scenes[nscene].getErsCorrection());
} else {
pXpYD_scene = nadir_pXpYD[nscene];
}
// Render from scene's own position: ZERO3 offset, scene is its own reference
ImagePlus imp_scene = QuadCLT.renderGPUFromDSI(
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment