Commit 1c807750 authored by Andrey Filippov's avatar Andrey Filippov

CLAUDE: Phase B lean measurement engine (curt_pose_lean) - existing kernels + Java only

TD-average the 16 sensors BEFORE correlation (multiply averages, not average
products), single conj-multiply vs the persistent virtual-center TD. Zero new
CUDA - assembled from proven pieces:
- CuasPoseRT.leanMeasure(): interCorrTD(sensor_mask=0) = tasks(pose)+offsets+
  convert_direct only -> getCltData/CuasTD.consolidateSensorsTD/setCltData
  (the validated CPU bridge, future clt_average_sensors kernel) ->
  setSensorMaskInter(1)+execCorr2D_inter_TD (single conj-multiply) ->
  TDCorrTile.getFromGpu + convertTDtoPD (JNA-validated CuasMotion path,
  FZ-normalize + PD) -> Correlation2d.getMaxXYCmEig (peak+eigen, the GPU
  argmax kernel oracle). Correlation stages are geometry-blind - projection/
  distortion is baked into the average-camera tasks (per Andrey).
- CuasPoseRT.leanFitScene(): same IntersceneLma solver + exit rules as the
  oracle engine; fills lma_rms/coord_motion_rslt so CSV/-POSE-RT-HYPER are
  unchanged (A2-03 = direct oracle).
- curt_pose_lean checkbox 'Pose test lean correlation (B)'.
v1 differences from oracle (documented): NO motion-blur compensation (compare
vs NOMB baseline: 0.287/0.282/0.106 mrad), no FPN peak masking (input is
FPN-subtracted by A2 conditioning), no moving-object filter, min_confidence=0.

mvn compile clean.
Co-Authored-By: 's avatarClaude Fable 5 <noreply@anthropic.com>
parent f40ebaa0
......@@ -30,13 +30,17 @@ import java.util.ArrayList;
import java.util.Arrays;
import com.elphel.imagej.cameras.CLTParameters;
import com.elphel.imagej.gpu.GpuQuad;
import com.elphel.imagej.gpu.TpTask;
import com.elphel.imagej.tileprocessor.Correlation2d;
import com.elphel.imagej.tileprocessor.ErsCorrection;
import com.elphel.imagej.tileprocessor.ImageDtt;
import com.elphel.imagej.tileprocessor.Interscene;
import com.elphel.imagej.tileprocessor.IntersceneLma;
import com.elphel.imagej.tileprocessor.OpticalFlow;
import com.elphel.imagej.tileprocessor.QuadCLT;
import com.elphel.imagej.tileprocessor.QuadCLTCPU;
import com.elphel.imagej.tileprocessor.TDCorrTile;
import com.elphel.imagej.tileprocessor.TwoQuadCLT;
import ij.ImagePlus;
......@@ -230,6 +234,235 @@ public class CuasPoseRT {
return sel;
}
/**
* Phase B lean measurement: ONE scene-vs-virtual-center correlation per LMA cycle.
* Average the 16 sensors in TD BEFORE correlation ("multiply averages, not average
* products"), then a single conj-multiply against the persistent center TD:
* interCorrTD(sensor_mask=0) - tasks(pose) + offsets + convert_direct ONLY
* getCltData -> CuasTD.consolidateSensorsTD -> setCltData(slot 0) - the CPU bridge
* setSensorMaskInter(1) + execCorr2D_inter_TD - the single conj-multiply
* TDCorrTile.getFromGpu -> TDCorrTile.convertTDtoPD - FZ-normalize + PD (existing)
* Correlation2d.getMaxXYCmEig - peak + eigen (the GPU argmax kernel's oracle)
* All GPU stages are existing kernels (the correlate/normalize/PD trio is the
* JNA-validated CuasMotion.correlatePairs path); only the bridge is CPU, to be
* replaced by the clt_average_sensors kernel later. Correlation stages are geometry-
* blind: all projection/distortion is baked into the tasks (average-camera grid), so
* the measured (dx,dy) is displacement on the same grid the pose Jacobian uses.
* Differences from the oracle engine (adjustPairsLMAInterscene) in this v1: no FPN
* peak masking (input is FPN-subtracted by conditioning), no moving-object filter,
* min_confidence not applied (tiles are pre-calibrated).
* By Claude on 07/05/2026, from Andrey's design.
* @return [3][num_pix][] {centers pXpYD, vector_XYS {dx,dy,strength}, eigen
* {e0x,e0y,l0,l1}} - the coord_motion contract of interCorrPair - or null
*/
public static double [][][] leanMeasure(
final CLTParameters clt_parameters,
final ImageDtt image_dtt,
final QuadCLT center_CLT,
final QuadCLT scene,
final double [] center_disparity,
final double [][] pXpYD_center,
final boolean [] selection,
final double [] scene_xyz,
final double [] scene_atr,
final int margin,
final int debugLevel) {
final GpuQuad gpuQuad = center_CLT.getGPUQuad();
final int num_pix = pXpYD_center.length;
// 1. scene grid at the current pose estimate (average-camera coordinates)
final double [][] scene_pXpYD = OpticalFlow.transformToScenePxPyD(
null, center_disparity, scene_xyz, scene_atr, scene, center_CLT);
final double disparity_corr = clt_parameters.imp.disparity_corr + center_CLT.getDispInfinityRef();
final TpTask [] tp_tasks = GpuQuad.setInterTasks(
scene.getNumSensors(),
scene.getGeometryCorrection().getSensorWH()[0],
false, // GPU calculates port coordinates
scene_pXpYD, // per-tile pX,pY,disparity
selection, // only the calibrated tile selection
scene.getGeometryCorrection(),
disparity_corr,
margin,
null, // valid_tiles (not needed)
ImageDtt.THREADS_MAX);
if ((tp_tasks == null) || (tp_tasks.length == 0)) {
System.out.println("leanMeasure(): no tasks for scene "+scene.getImageName());
return null;
}
// 2. convert only: LPFs + tasks + offsets + convert_direct, NO correlation (mask=0)
image_dtt.interCorrTD(
clt_parameters.img_dtt,
tp_tasks,
null, // fcorr_td - not used with mask=0
clt_parameters.gpu_sigma_r,
clt_parameters.gpu_sigma_b,
clt_parameters.gpu_sigma_g,
clt_parameters.gpu_sigma_m,
scene.isMonochrome() ? 1.0 : clt_parameters.gpu_sigma_rb_corr,
clt_parameters.getGpuCorrSigma(scene.isMonochrome()),
clt_parameters.getGpuCorrLoGSigma(scene.isMonochrome()),
clt_parameters.corr_red,
clt_parameters.corr_blue,
0, // sensor_mask_inter = 0: convert only, return before correlate
ImageDtt.THREADS_MAX,
debugLevel);
// 3. the CPU consolidation bridge (future clt_average_sensors kernel)
final float [][] fclt16 = gpuQuad.getCltData(false);
final float [] avg_td = CuasTD.consolidateSensorsTD(fclt16, null);
gpuQuad.setCltData(0, avg_td, false);
// 4. single conj-multiply: only sensor 0 (holding the average) vs the center ref
final double [] col_weights = scene.isMonochrome() ?
(new double [] {1.0}) :
(new double [] {clt_parameters.corr_red/(1.0+clt_parameters.corr_red+clt_parameters.corr_blue),
clt_parameters.corr_blue/(1.0+clt_parameters.corr_red+clt_parameters.corr_blue),
1.0/(1.0+clt_parameters.corr_red+clt_parameters.corr_blue)});
gpuQuad.setSensorMaskInter(1);
gpuQuad.execCorr2D_inter_TD(col_weights);
// 5. TD tiles -> FZ-normalize -> pixel domain (the JNA-validated TDCorrTile path)
final TDCorrTile [] td_tiles = TDCorrTile.getFromGpu(gpuQuad);
final double [][] corr_pd = TDCorrTile.convertTDtoPD(
gpuQuad,
td_tiles,
0xFE, // corr_type (sum slot, as CuasMotion)
clt_parameters.getGpuFatZero(scene.isMonochrome()),
debugLevel);
if (corr_pd == null) {
System.out.println("leanMeasure(): convertTDtoPD returned null for scene "+scene.getImageName());
return null;
}
// 6. peak + eigen per tile (the GPU argmax+eigen kernel's Java oracle)
final int corr_width = 2 * GPUTileProcessorDttSize() - 1;
final double [][] centers = new double [num_pix][];
final double [][] vector_XYS = new double [num_pix][];
final double [][] eigen = new double [num_pix][];
int num_meas = 0;
for (int i = 0; i < num_pix; i++) {
if ((i >= corr_pd.length) || (corr_pd[i] == null) || (pXpYD_center[i] == null)) continue;
final double [] r = Correlation2d.getMaxXYCmEig(
corr_pd[i], // will not be modified (fpn_mask == null)
corr_width,
clt_parameters.imp.eig_min_abs, // abs_min
clt_parameters.imp.eig_min_rel, // rel_min
clt_parameters.imp.min_str_sum, // min_peak (single channel = the "sum")
0.0, // eig_sub_frac (select only)
null, // fpn_mask (input is FPN-subtracted)
false, // ignore_border
null, // debug_data
false); // debug
if (r == null) continue;
centers[i] = pXpYD_center[i];
vector_XYS[i] = new double [] {r[0], r[1], r[2]};
eigen[i] = new double [] {r[3], r[4], r[5], r[6]};
num_meas++;
}
if (debugLevel > -2) {
System.out.println("leanMeasure(): scene "+scene.getImageName()+" measured "+num_meas+" tiles");
}
if (num_meas == 0) return null;
return new double [][][] {centers, vector_XYS, eigen};
}
private static int GPUTileProcessorDttSize() {
return com.elphel.imagej.gpu.GPUTileProcessor.DTT_SIZE;
}
/**
* Phase B lean per-scene fit: the measure<->solve cycle with leanMeasure() as the
* measurement engine and the same IntersceneLma solver/exit rules as the oracle
* (adjustPairsLMAInterscene). NOTE v1: NO motion-blur compensation in the lean path
* yet - expect the known v*tau bias vs stored (compare against the NOMB baseline,
* not MBEN). By Claude on 07/05/2026.
* @return fitted {xyz,atr} or null (measurement/LMA failure); fills lma_rms
* ([0] rms, [2] sum weight, [3] tiles) and coord_motion_rslt (for -POSE-RT-HYPER)
*/
public static double [][] leanFitScene(
final CLTParameters clt_parameters,
final ImageDtt image_dtt,
final QuadCLT center_CLT,
final QuadCLT scene,
final double [] center_disparity,
final double [][] pXpYD_center,
final boolean [] selection,
final double [][] predicted, // initial + pull
final boolean [] param_select,
final double [] reg_weights,
final int margin,
final double [] lma_rms, // out
final double [][][] coord_motion_rslt, // out [3][][]
final int debugLevel) {
final IntersceneLma intersceneLma = new IntersceneLma(
clt_parameters.ilp.ilma_thread_invariant,
0.0); // no disparity weight (2D only)
double [][] scene_xyzatr0 = new double [][] {predicted[0].clone(), predicted[1].clone()};
double [][][] cm = null;
int nlma = 0;
for (; nlma < clt_parameters.imp.max_cycles; nlma++) {
cm = leanMeasure(
clt_parameters, image_dtt, center_CLT, scene,
center_disparity, pXpYD_center, selection,
scene_xyzatr0[0], scene_xyzatr0[1], margin, debugLevel);
if (cm == null) return null;
final double [][] eigen_masked = clt_parameters.imp.eig_xy_lma ? null : cm[2];
intersceneLma.prepareLMA(
scene_xyzatr0, // scene_xyzatr0
predicted, // scene_xyz_pull
new double [2][3], // ref_xyzatr
scene, // scene_QuadClt
center_CLT, // reference_QuadClt
param_select, // param_select
reg_weights, // param_regweights
clt_parameters.imp.eig_max_sqrt, // eig_max_sqrt (line = infinite feature)
clt_parameters.imp.eig_min_sqrt, // eig_min_sqrt
eigen_masked, // eigen
cm[1], // vector_XYS
0.0, // min_confidence (tiles pre-calibrated)
null, // reliable_ref (already selected)
cm[0], // centers
false, // same_weights
(nlma == 0), // first_run
null, // dbg_prefix
clt_parameters.imp.debug_level); // debug_level
final int lmaResult = intersceneLma.runLma(
clt_parameters.ilp.ilma_lambda,
clt_parameters.ilp.ilma_lambda_scale_good,
clt_parameters.ilp.ilma_lambda_scale_bad,
clt_parameters.ilp.ilma_lambda_max,
clt_parameters.ilp.ilma_rms_diff,
clt_parameters.imp.max_LMA,
false, // last_run
debugLevel);
if (lmaResult < 0) {
System.out.println("leanFitScene(): LMA failed, lmaResult="+lmaResult);
return null;
}
scene_xyzatr0 = intersceneLma.getSceneXYZATR(false);
final double [] diffs_atr = intersceneLma.getV3Diff(ErsCorrection.DP_DSAZ);
final double [] diffs_xyz = intersceneLma.getV3Diff(ErsCorrection.DP_DSX);
if ((diffs_atr[0] < clt_parameters.imp.exit_change_atr) &&
(diffs_xyz[0] < clt_parameters.imp.exit_change_xyz)) {
break;
}
}
if (lma_rms != null) {
final double [] last_rms = intersceneLma.getLastRms();
lma_rms[0] = last_rms[0];
if (lma_rms.length > 1) lma_rms[1] = last_rms[1];
double sw = 0; int nm = 0;
for (double [] v : cm[1]) if (v != null) { sw += v[2]; nm++; }
if (lma_rms.length > 2) lma_rms[2] = sw;
if (lma_rms.length > 3) lma_rms[3] = nm;
}
if (coord_motion_rslt != null) {
for (int i = 0; (i < coord_motion_rslt.length) && (i < cm.length); i++) {
coord_motion_rslt[i] = cm[i];
}
}
if (debugLevel > -3) {
System.out.println("Adjusted interscene, iteration="+nlma+", last RMS = "+
intersceneLma.getLastRms()[0]+" (lean)");
}
return scene_xyzatr0;
}
/**
* Re-generate per-scene poses against the virtual-center reference, RT-style.
*
......@@ -385,6 +618,22 @@ public class CuasPoseRT {
System.out.println("CuasPoseRT.testPoseSequence(): imp.eig_use is OFF - the 4 eigen slices of "+
"-POSE-RT-HYPER (sqrt_l0, sqrt_l1, elong, eig0_ang) will be all NaN");
}
// Phase B lean engine (curt_pose_lean): ImageDtt instance for the convert stage,
// created once (mirrors interCorrPair's construction)
final ImageDtt image_dtt_lean = clt_parameters.imp.curt_pose_lean ? new ImageDtt(
center_CLT.getNumSensors(),
clt_parameters.transform_size,
clt_parameters.img_dtt,
center_CLT.isAux(),
center_CLT.isMonochrome(),
center_CLT.isLwir(),
clt_parameters.getScaleStrength(center_CLT.isAux()),
center_CLT.getGPU()) : null;
if (clt_parameters.imp.curt_pose_lean) {
System.out.println("CuasPoseRT.testPoseSequence(): LEAN measurement engine (phase B): "+
"TD-average(16) x virtual-center single conj-multiply. NOTE: no MB compensation "+
"in lean v1 - compare against the NOMB baseline");
}
final double [][][] fitted = new double [quadCLTs.length][][];
final int [] fail_reason = new int [1];
final StringBuffer sb = new StringBuffer();
......@@ -461,7 +710,26 @@ public class CuasPoseRT {
}
fail_reason[0] = 0;
final double [][][] coord_motion_rslt = new double [3][][]; // [centers, vector_XYS, eigen] of the last LMA cycle
double [][] pose = Interscene.adjustPairsLMAInterscene(
double [][] pose;
if (clt_parameters.imp.curt_pose_lean) {
pose = leanFitScene(
clt_parameters, // CLTParameters clt_parameters,
image_dtt_lean, // ImageDtt image_dtt,
center_CLT, // QuadCLT center_CLT,
quadCLTs[nscene], // QuadCLT scene,
center_disparity, // double [] center_disparity,
pXpYD_center, // double [][] pXpYD_center,
reliable_ref, // boolean [] selection,
predicted, // double [][] predicted (initial + pull),
param_select, // boolean [] param_select,
reg_weights, // double [] reg_weights,
margin, // int margin,
lma_rms, // double [] lma_rms (out),
coord_motion_rslt, // double [][][] coord_motion_rslt (out),
debugLevel - 2); // int debug_level
if (pose == null) fail_reason[0] = 1; // measurement/LMA
} else {
pose = Interscene.adjustPairsLMAInterscene(
clt_parameters, // CLTParameters clt_parameters,
true, // boolean initial_adjust,
false, // boolean fpn_disable,
......@@ -489,6 +757,7 @@ public class CuasPoseRT {
mb_vectors_scene, // double [][] mb_vectors (null when imp.mb_en OFF - single-run path),
coord_motion_rslt, // double [][][] coord_motion_rslt (last-cycle measurement, for -POSE-RT-HYPER),
debugLevel - 2); // int debug_level
}
final boolean ok = (pose != null);
if (ok) {
fitted[nscene] = pose;
......
......@@ -1133,6 +1133,7 @@ min_str_neib_fpn 0.35
public int curt_pose_num_tiles = 150; // tile-selection compute budget: after the gate, keep this many BEST (smallest max-residual) tiles; threshold-free rank - always yields the best available population. <=0 - no cap. // By Claude on 07/04/2026
public boolean curt_pose_recalc = false; // force a FULL-selection pose run that regenerates the -POSE-RT-TILE-CALIB calibration even if it exists. Default (false): automatic FPN-style reuse - if -POSE-RT-TILE-CALIB exists, derive the selection from it (NMAD gate + rank-N) and use it; if not, run full and generate it. // By Claude on 07/04/2026
public boolean curt_pose_raw = false; // phase A2 ingest: per scene read RAW /jp4/, condition with the current calibration (CuasConditioning: rowcol + photometric + FPN) and FORCE-upload straight to the GPU, bypassing the QuadCLT prepared image_data (which carries the old broken Photogrammetric Calibration). Judge by own dstored quality, not agreement with phase A. // By Claude on 07/05/2026
public boolean curt_pose_lean = false; // phase B measurement engine: TD-average the 16 sensors (CuasTD, CPU bridge) then ONE conj-multiply vs the virtual-center TD -> FZ-normalize -> PD -> argmax+eigen (getMaxXYCmEig) -> 3-angle LMA. All existing GPU kernels. v1: NO motion-blur compensation - compare vs the NOMB baseline. // By Claude on 07/05/2026
//=== LoG prefilter ===
public double curt_psf_radius = 1.0; // sensor PSF radius for LoG pre-filter
public double curt_n_sigma = 4.0; // cutoff LoG kernel array, number of sigmas
......@@ -3434,6 +3435,8 @@ min_str_neib_fpn 0.35
"Force a FULL pose run that regenerates -POSE-RT-TILE-CALIB even if it exists. Default OFF = automatic reuse: use the calibration if present (filtered run), else run full and generate it.");
gd.addCheckbox ("Pose test raw-jp4 ingest (A2)", this.curt_pose_raw, // By Claude on 07/05/2026
"Per scene: read RAW /jp4/, condition with the CURRENT calibration (rowcol+photometric+FPN, CuasConditioning) and force-upload straight to the GPU, bypassing the prepared image_data (old broken Photogrammetric Calibration).");
gd.addCheckbox ("Pose test lean correlation (B)", this.curt_pose_lean, // By Claude on 07/05/2026
"Phase B measurement: TD-average the 16 sensors, then ONE conj-multiply vs the virtual-center TD -> FZ-normalize -> PD -> argmax+eigen -> 3-angle LMA. v1 has NO motion-blur compensation (compare vs the NOMB baseline).");
gd.addMessage("=== LoG prefilter ===");
gd.addNumericField("Optical PSF radius", this.curt_psf_radius, 6,8,"pix",
......@@ -5032,6 +5035,7 @@ min_str_neib_fpn 0.35
this.curt_pose_num_tiles =(int) gd.getNextNumber(); // By Claude on 07/04/2026
this.curt_pose_recalc = gd.getNextBoolean(); // By Claude on 07/04/2026
this.curt_pose_raw = gd.getNextBoolean(); // By Claude on 07/05/2026
this.curt_pose_lean = gd.getNextBoolean(); // By Claude on 07/05/2026
this.curt_psf_radius = gd.getNextNumber();
this.curt_n_sigma = gd.getNextNumber();
......@@ -6404,6 +6408,7 @@ min_str_neib_fpn 0.35
properties.setProperty(prefix+"curt_pose_num_tiles", this.curt_pose_num_tiles+""); // int // By Claude on 07/04/2026
properties.setProperty(prefix+"curt_pose_recalc", this.curt_pose_recalc+""); // boolean // By Claude on 07/04/2026
properties.setProperty(prefix+"curt_pose_raw", this.curt_pose_raw+""); // boolean // By Claude on 07/05/2026
properties.setProperty(prefix+"curt_pose_lean", this.curt_pose_lean+""); // boolean // By Claude on 07/05/2026
properties.setProperty(prefix+"curt_psf_radius", this.curt_psf_radius+""); // double
properties.setProperty(prefix+"curt_n_sigma", this.curt_n_sigma+""); // double
......@@ -6810,6 +6815,7 @@ min_str_neib_fpn 0.35
if (properties.getProperty(prefix+"curt_pose_num_tiles")!=null) this.curt_pose_num_tiles=Integer.parseInt(properties.getProperty(prefix+"curt_pose_num_tiles")); // By Claude on 07/04/2026
if (properties.getProperty(prefix+"curt_pose_recalc")!=null) this.curt_pose_recalc=Boolean.parseBoolean(properties.getProperty(prefix+"curt_pose_recalc")); // By Claude on 07/04/2026
if (properties.getProperty(prefix+"curt_pose_raw")!=null) this.curt_pose_raw=Boolean.parseBoolean(properties.getProperty(prefix+"curt_pose_raw")); // By Claude on 07/05/2026
if (properties.getProperty(prefix+"curt_pose_lean")!=null) this.curt_pose_lean=Boolean.parseBoolean(properties.getProperty(prefix+"curt_pose_lean")); // By Claude on 07/05/2026
if (properties.getProperty(prefix+"curt_psf_radius")!=null) this.curt_psf_radius=Double.parseDouble(properties.getProperty(prefix+"curt_psf_radius"));
if (properties.getProperty(prefix+"curt_n_sigma")!=null) this.curt_n_sigma=Double.parseDouble(properties.getProperty(prefix+"curt_n_sigma"));
......@@ -9101,6 +9107,7 @@ min_str_neib_fpn 0.35
imp.curt_pose_num_tiles = this.curt_pose_num_tiles; // By Claude on 07/04/2026
imp.curt_pose_recalc = this.curt_pose_recalc; // By Claude on 07/04/2026
imp.curt_pose_raw = this.curt_pose_raw; // By Claude on 07/05/2026
imp.curt_pose_lean = this.curt_pose_lean; // By Claude on 07/05/2026
imp.curt_psf_radius = this.curt_psf_radius;
imp.curt_n_sigma = this.curt_n_sigma;
// rleak0 imp.X copy removed 2026-06-20 (LReLU now LINEAR); predecessor code at git tag cuas-layer1. // By Claude on 06/20/2026
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment