Skip to content

I
imagej-elphel
Commit 3b1049d6 authored by Andrey Filippov's avatar Andrey Filippov
Browse files
Options

CLAUDE: FPN second back-propagation iteration (imp.cuas_fpn_iters, default 1 = old behavior) 

FPN = mean(scene - backProject(center)) is a fixed point that preserves the
center's own content residual (the rotating ghost): recomputing against the
same contaminated center reproduces the same ghost regardless of rot_period
or photometric vintage (POSE-12 confirmed: 43.1/92.0 unchanged).

Iterate per Andrey's design (internal handoff 2026-07-06):
  1. subtract fpn_k from scenes in memory (setApplyFPN, delta-aware)
  2. rebuild the center IMAGE from the corrected scenes over integer
     rotations - new Cuas.rebuildCenterImageInMemory(): new-average only
     (no parent cumulative fold), NOTHING persisted, center_clt and
     image_center untouched (the .cuas cumulative is upstream state)
  3. back-propagate against the rebuilt center image (swapped in/restored
     around the measurement) -> residual dG = the ghost
  4. fpn_{k+1} = fpn_k + dG in NEW arrays (scenes hold fpn_k references as
     applied-FPN for delta re-apply); prints |dG| RMS/max per iteration
  5. cuas_debug saves per-iteration -FPN-DG-ITERn

Gated by new dialog field "FPN back-propagation iterations" next to
"Rotation period"; default 1 keeps the POSE-12 baseline bit-reproducible.
Verify by RT-RENDER horizon/sky pk-pk (43.1/92.0 -> temporal-noise floor).

Data path verified: applyFPN sets hasNewImageData, honored by both
GpuQuad.setBayerImages and GpuQuadJna, so corrected pixels reach the GPU
for the rebuild and the second back-propagation.
Co-Authored-By: 's avatarClaude Fable 5 <noreply@anthropic.com>
parent 53fe5f5c
Hide whitespace changes
Inline Side-by-side
Showing with 167 additions and 4 deletions
src/main/java/com/elphel/imagej/cuas/CorrectionFPN.java
View file @ 3b1049d6
......@@ -221,7 +221,91 @@ public class CorrectionFPN {
cuas_outliers_rowcol,// double weight_outlier, // = 0.001;
show_fpn); // boolean show) {
}
// By Claude on 07/06/2026: optional second (and further) back-propagation
// iteration (internal handoff 2026-07-06_fpn_second_iteration_design.md).
// FPN computed against the accumulated center is a fixed point that keeps the
// center's own content residual (the rotating ghost): recomputing against the
// same center reproduces the same ghost. Iterate: subtract fpn_k from the
// scenes (in memory, delta-aware), rebuild the center image from the corrected
// scenes over an integer number of rotations (in memory only - the .cuas
// cumulative is upstream state and is never rewritten here), back-propagate
// against the rebuilt center: the residual dG is the ghost; fpn_{k+1} = fpn_k + dG.
int cuas_fpn_iters = clt_parameters.imp.cuas_fpn_iters;
for (int fpn_iter = 1; fpn_iter < cuas_fpn_iters; fpn_iter++) {
double [][] image_center0 = center_CLT.getImageCenter(); // restored after the measurement
center_CLT.getCorrectionFPN().setApplyFPN( // subtract current FPN from scenes in memory
quadCLTs, // QuadCLT [] quadCLTs,
fpn); // double [][][] fpn)
double [][] image_center2 = Cuas.rebuildCenterImageInMemory(
clt_parameters, // CLTParameters clt_parameters,
Arrays.copyOfRange(quadCLTs, rot_range[0], rot_range[1]+1), // integer rotations only
center_CLT, // QuadCLT center_CLT,
disparity_center, // double [] disparity_center,
-1, // int sensor_mask, // -1 - all;
debugLevel); // int debugLevel)
if (image_center2 == null) {
System.out.println("cuasSubtractFpn(): center rebuild FAILED, stopping FPN iterations after "+
fpn_iter+" of "+cuas_fpn_iters);
break;
}
center_CLT.setImageCenter(image_center2); // in-memory only: backPropagate() reads it
double [][][] dg_and_weights = backPropagate(
clt_parameters, // CLTParameters clt_parameters,
discard_border, // final int discard_border,
max_fold, // final double max_fold,
min_in_row_col, // final int min_in_row_col, // Minimal number of defined tiles in a row/column
cuas_invert_margins, // final int invert_margins, // 1 Expand image each side when inverting tasks
cuas_invert_gap2, // final int invert_gap2, // 10 // Maximal dual gap size for inversion (depends on scanning radius in tiles)
cuas_invert_iters, // final int invert_iters, // 4 Enhance inversion iterations
cuas_invert_tolerance, // final double invert_tolerance,// 0.001 Finish enhancing when last change was lower than
center_CLT, // final QuadCLT center_CLT,
quadCLTs, // final QuadCLT[] quadCLTs,
rot_range[0], // final int first_index,
rot_range[1], // final int last_index,
disparity_center, // double [] disparity_center
debugLevel); // final int debugLevel)
center_CLT.setImageCenter(image_center0); // restore the original center image (upstream vintage)
double [][][] dg = new double [dg_and_weights[0].length][1][];
for (int nsens = 0; nsens < dg.length; nsens++) {
dg[nsens][0] = dg_and_weights[0][nsens];
}
dg = applyRowCol( // same row/col removal as applied to the first-pass FPN
dg, // final double [][][] image_data,
getRowAvgMulti(dg, fpn_width, cuas_max_abs_rowcol, cuas_outliers_rowcol), // final double [][] image_row_avg,
getColAvgMulti(dg, fpn_width, cuas_max_abs_rowcol, cuas_outliers_rowcol), // final double [][] image_col_avg,
false); // final boolean inplace )
// fpn_{k+1} = fpn_k + dG in NEW arrays: the scenes keep references to the
// fpn_k arrays as "applied" (delta-based re-apply), so fpn_k must not be
// modified in place.
double [][][] fpn_next = new double [fpn.length][1][];
double dg_sum2 = 0.0, dg_max_abs = 0.0;
int dg_num_def = 0;
for (int nsens = 0; nsens < fpn.length; nsens++) {
fpn_next[nsens][0] = fpn[nsens][0].clone();
for (int npix = 0; npix < fpn_next[nsens][0].length; npix++) {
double d = dg[nsens][0][npix];
if (!Double.isNaN(d)) { // keep fpn_k where dG is undefined
fpn_next[nsens][0][npix] += d;
dg_sum2 += d*d;
if (Math.abs(d) > dg_max_abs) dg_max_abs = Math.abs(d);
dg_num_def++;
}
}
}
fpn = fpn_next;
System.out.println(String.format(
"cuasSubtractFpn(): FPN iteration %d of %d: |dG| RMS = %.4f, max = %.3f (%d defined pixels)",
fpn_iter+1, cuas_fpn_iters, Math.sqrt(dg_sum2/Math.max(dg_num_def,1)), dg_max_abs, dg_num_def));
if (cuas_debug) {
center_CLT.getCorrectionFPN().saveShowFPN(
dg, // double [][][] fpn,
fpn_width, // int width,
true, // boolean save,
show_fpn, // boolean show,
QuadCLT.CENTER_FPN_SUFFIX+"-DG-ITER"+(fpn_iter+1)); // String suffix)
}
}
} else {
if (debugLevel >-3) {
System.out.println("Reusing FPN.");
......
src/main/java/com/elphel/imagej/cuas/Cuas.java
View file @ 3b1049d6
......@@ -692,9 +692,80 @@ public class Cuas {
return center_CLT;
}
/**
* Rebuild the center IMAGE from the current in-memory scene image data (e.g. after
* subtracting a just-computed FPN) WITHOUT persisting or mutating anything: no
* .cuas / -CLT / DSI / IMS files are written and center_CLT.center_clt /
* center_CLT.image_center are left untouched. Always a NEW average - no parent
* cumulative is folded in (unlike stepCenterClt() which seeds from the current
* center and would keep its content residual). Used by the FPN second
* back-propagation iteration in CorrectionFPN.cuasSubtractFpn(), where the rebuilt
* center is only a temporary measurement reference.
* By Claude on 07/06/2026
* @param clt_parameters processing parameters
* @param quadCLTs scenes to accumulate; pass a slice covering an integer number of
* camera rotations so azimuth sectors are equally weighted
* @param center_CLT existing center (geometry/poses reference), must have center CLT
* @param disparity_center center-grid disparity (from backPrepareCenter()/getDisparityCenter())
* @param sensor_mask -1 - all
* @param debugLevel debug level
* @return image-domain center as convertCenterClt() returns, or null on failure
*/
public static double [][] rebuildCenterImageInMemory( // By Claude on 07/06/2026
CLTParameters clt_parameters,
QuadCLT [] quadCLTs,
QuadCLT center_CLT,
double [] disparity_center,
int sensor_mask, // -1 - all;
int debugLevel) {
if (!center_CLT.hasCenterClt()) {
System.out.println("rebuildCenterImageInMemory(): center_CLT does not have .center_clt data, bailing out");
return null;
}
double cuas_clt_variant = clt_parameters.imp.cuas_clt_variant; // 10;
double cuas_clt_threshold = clt_parameters.imp.cuas_clt_threshold; // 20;
double cuas_decay_average = clt_parameters.imp.cuas_decay_average; // 100;
double cuas_keep_fraction = clt_parameters.imp.cuas_keep_fraction; // 0.9;
double cuas_clt_decrease = clt_parameters.imp.cuas_clt_decrease; // 0.01;
double dts = center_CLT.getTimeStamp();
CuasData newCuasData = getTDComboSceneSequence(
clt_parameters, // CLTParameters clt_parameters,
null, // double [][] ref_pXpYD,
true, // boolean save_weights,
sensor_mask, // int sensor_mask,
null, // Rectangle fov_tiles,
OpticalFlow.ZERO3, // double [] stereo_xyz, // offset reference camera {x,y,z}
OpticalFlow.ZERO3, // double [] stereo_atr_in, // center already exists - zero ATR
disparity_center, // double [] ref_disparity,
quadCLTs, // QuadCLT [] quadCLTs,
center_CLT, // QuadCLT refCLT,
null, // CuasData cuasData - no parent cumulative: new average only
true, // final boolean clt_create, // create new variants
cuas_clt_variant, // final double clt_threshold,
cuas_decay_average, // final double clt_decay,
cuas_clt_decrease, // final double clt_decrease,
debugLevel); // int debugLevel)
if (newCuasData == null) {
System.out.println("rebuildCenterImageInMemory(): getTDComboSceneSequence() failed");
return null;
}
if (cuas_keep_fraction < 1.0) { // same filtering as createCenterClt() applies before collapse
newCuasData = filter(
newCuasData, // final CuasData cuasData,
cuas_decay_average, // final double cuas_decay, // seconds e times
cuas_keep_fraction, // final double keep_fraction,
debugLevel); // final int debugLevel)
}
float [] fclt = newCuasData.collapse(
cuas_clt_threshold, // final double tolerance, // NaN works as infinity
cuas_decay_average, // final double decay,
dts)[0]; // final double ts_now)
return center_CLT.convertCenterClt(new float [][] {fclt}); // does NOT touch center_CLT state
}
public static void fillNanDsi(
double [][] ref_pXpYD,
double [][] dsi,
......
src/main/java/com/elphel/imagej/tileprocessor/IntersceneMatchParameters.java
View file @ 3b1049d6
......@@ -757,6 +757,7 @@ min_str_neib_fpn 0.35
public boolean cuas_subtract_fpn= true; // Subtract FPN
public boolean cuas_calc_fpn = false; // Recalculate+save FPN unconditionally, if false - try to read saved one first
public double cuas_rot_period = 175.0; // rotation period of te gimbal mount (in scenes)
public int cuas_fpn_iters = 1; // FPN back-propagation iterations: >1 rebuilds the center in memory from FPN-corrected scenes and adds the residual (removes the content ghost) // By Claude on 07/06/2026
public double cuas_min_series = 180.0; // discard shorter series // FIXME: allow shorter, but modify FPN/row_col to copy parent's files if less than cuas_rot_period
public boolean cuas_subtract_rowcol= true; // Subtract row/column noise
public boolean cuas_calc_rowcol= false; // Recalculate+save row/column noise, if false - try to read saved one first
......@@ -2683,6 +2684,9 @@ min_str_neib_fpn 0.35
"Recalculate+save FPN unconditionally, if false - try to read saved one.");
gd.addNumericField("Rotation period", this.cuas_rot_period, 5,7,"scene periods",
"Used for averaging FPN for integer number of rotation periods. TODO: calculate from the actual rotation.");
gd.addNumericField("FPN back-propagation iterations", this.cuas_fpn_iters, 0,3,"", // By Claude on 07/06/2026
"1 - single pass (as before). >1 - after calculating FPN: subtract it from the scenes, rebuild the center image in memory "+
"(integer rotations, nothing persisted), back-propagate again and add the residual - removes the content ghost baked into the center.");
gd.addNumericField("Minimal series length", this.cuas_min_series, 5,7,"scene periods",
"Skip shorter series (should be not much less than rotation period.");
gd.addMessage("=== Row/column noise (on top of the FPN) ===");
......@@ -4455,6 +4459,7 @@ min_str_neib_fpn 0.35
this.cuas_subtract_fpn = gd.getNextBoolean();
this.cuas_calc_fpn = gd.getNextBoolean();
this.cuas_rot_period = gd.getNextNumber();
this.cuas_fpn_iters = (int) gd.getNextNumber(); // By Claude on 07/06/2026
this.cuas_min_series = gd.getNextNumber();
this.cuas_subtract_rowcol = gd.getNextBoolean();
......@@ -5789,6 +5794,7 @@ min_str_neib_fpn 0.35
properties.setProperty(prefix+"cuas_subtract_fpn", this.cuas_subtract_fpn+""); // boolean
properties.setProperty(prefix+"cuas_calc_fpn", this.cuas_calc_fpn+""); // boolean
properties.setProperty(prefix+"cuas_rot_period", this.cuas_rot_period+""); // double
properties.setProperty(prefix+"cuas_fpn_iters", this.cuas_fpn_iters+""); // int // By Claude on 07/06/2026
properties.setProperty(prefix+"cuas_min_series", this.cuas_min_series+""); // double
properties.setProperty(prefix+"cuas_subtract_rowcol", this.cuas_subtract_rowcol+"");// boolean
......@@ -7061,6 +7067,7 @@ min_str_neib_fpn 0.35
if (properties.getProperty(prefix+"cuas_subtract_fpn")!=null) this.cuas_subtract_fpn=Boolean.parseBoolean(properties.getProperty(prefix+"cuas_subtract_fpn"));
if (properties.getProperty(prefix+"cuas_calc_fpn")!=null) this.cuas_calc_fpn=Boolean.parseBoolean(properties.getProperty(prefix+"cuas_calc_fpn"));
if (properties.getProperty(prefix+"cuas_rot_period")!=null) this.cuas_rot_period=Double.parseDouble(properties.getProperty(prefix+"cuas_rot_period"));
if (properties.getProperty(prefix+"cuas_fpn_iters")!=null) this.cuas_fpn_iters=Integer.parseInt(properties.getProperty(prefix+"cuas_fpn_iters")); // By Claude on 07/06/2026
if (properties.getProperty(prefix+"cuas_min_series")!=null) this.cuas_min_series=Double.parseDouble(properties.getProperty(prefix+"cuas_min_series"));
if (properties.getProperty(prefix+"cuas_subtract_rowcol")!=null) this.cuas_subtract_rowcol=Boolean.parseBoolean(properties.getProperty(prefix+"cuas_subtract_rowcol"));
......@@ -8353,6 +8360,7 @@ min_str_neib_fpn 0.35
imp.cuas_subtract_fpn = this.cuas_subtract_fpn;
imp.cuas_calc_fpn = this.cuas_calc_fpn;
imp.cuas_rot_period = this.cuas_rot_period;
imp.cuas_fpn_iters = this.cuas_fpn_iters; // By Claude on 07/06/2026
imp.cuas_min_series = this.cuas_min_series;
imp.cuas_subtract_rowcol = this.cuas_subtract_rowcol;
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment