Commit ebef0b23 authored by Andrey Filippov's avatar Andrey Filippov

CLAUDE: per-sensor lwir photometric recalibration + JNA bayer-guard fix

curt_cond_test rework: both PERSENSOR stacks now converted with the test's
own uniform sensor-domain task grid (scale 1.0) instead of leftover GPU
state (MB secondary tasks with negative fractional scales made 'raw'
renders = -1/6 x input; leftover virtual-view grid lost the same border
ROI on every sensor). perSensorFromRawJp4 no longer overwrites the scene's
conditioned image_data.

GpuQuadJna.setBayerImages(force,center) restored the base-class skip-guard
via a native-side jna_bayer_set flag (gpuTileProcessor is null in JNA shell
instances): every execConvertDirect unconditionally re-pulled
quadCLT.getResetImageData(), silently clobbering explicit uploads - made
the raw baseline bit-identical to the conditioned render.

CuasMotion.perSensorLinearFit(): per-sensor a+b*x photometric fit over
safe tiles (weak strength<0.5 or far disparity<1 from -INTER-INTRA-LMA,
inner rect, 8x8 tile->pixel map) against the cross-sensor mean, gauge
keep_averages (mean offset 0, mean scale 1), 3-sigma outlier rejection.
Validated on 1773135476_186641: sensor-mean spread 1353->5 counts,
cross-sensor RMS 358->17 (inliers), b in 0.83..1.11.

CuasMotion.applyLwirLinearCalibration(): folds the fit into the 16+16
lwir offsets/scales (scale'=b*scale, offset'=offset-a/scale'), updates the
center instance + photometric_scene provenance, saves -INTERFRAME.corr-xml.
Applied the standard way at load they compensate the remaining per-sensor
mismatch of the raw /jp4/ tiffs.
Co-Authored-By: 's avatarClaude Fable 5 <noreply@anthropic.com>
parent 96b1c3d5
...@@ -3695,19 +3695,13 @@ public class CuasMotion { ...@@ -3695,19 +3695,13 @@ public class CuasMotion {
* (main CUAS results live there), NOT the physical scene. null -> fall back to gpuQuad.getQuadCLT() (physical). * (main CUAS results live there), NOT the physical scene. null -> fall back to gpuQuad.getQuadCLT() (physical).
* By Claude on 07/01/2026. */ * By Claude on 07/01/2026. */
public static double [] perSensorAveragesFromTD(GpuQuad gpuQuad, boolean use_reference, String name_tag, QuadCLT save_qclt) { public static double [] perSensorAveragesFromTD(GpuQuad gpuQuad, boolean use_reference, String name_tag, QuadCLT save_qclt) {
gpuQuad.execImcltRbgAll(
(gpuQuad.getNumColors() <= 1), // isMonochrome()
use_reference,
null); // int [] wh
final int num_sens = gpuQuad.getNumSensors(); final int num_sens = gpuQuad.getNumSensors();
final float [][] per_sensor = perSensorImagesFromTD(gpuQuad, use_reference); // one w*h slice per sensor
final int width = gpuQuad.getImageWidth(); // valid after last IMCLT final int width = gpuQuad.getImageWidth(); // valid after last IMCLT
final int height = gpuQuad.getImageHeight(); final int height = gpuQuad.getImageHeight();
final double [] avg = new double [num_sens]; final double [] avg = new double [num_sens];
final float [][] per_sensor = new float [num_sens][]; // kept for the saved stack (one w*h slice per sensor)
for (int ncam = 0; ncam < num_sens; ncam++) { for (int ncam = 0; ncam < num_sens; ncam++) {
final float [][] rbg = gpuQuad.getRBG(ncam); // [color][pixel] final float [] px = per_sensor[ncam];
final float [] px = rbg[0]; // mono (color 0)
per_sensor[ncam] = px;
double sum = 0.0; int n = 0; double sum = 0.0; int n = 0;
for (int i = 0; i < px.length; i++) { for (int i = 0; i < px.length; i++) {
final float v = px[i]; final float v = px[i];
...@@ -3740,6 +3734,405 @@ public class CuasMotion { ...@@ -3740,6 +3734,405 @@ public class CuasMotion {
return avg; return avg;
} }
/** Render the current transform-domain data and return the per-sensor mono images (color 0).
* Extracted from {@link #perSensorAveragesFromTD} so the conditioning-test/calibration code can get the
* pixel arrays directly. By Claude on 07/02/2026.
* @param gpuQuad the GpuQuad whose TD to render.
* @param use_reference render the reference-scene TD (gpu_clt_ref) instead of the current scene.
* @return [sensor][pixel] mono images, width/height = gpuQuad.getImageWidth()/getImageHeight().
*/
public static float [][] perSensorImagesFromTD(GpuQuad gpuQuad, boolean use_reference) { // By Claude on 07/02/2026
gpuQuad.execImcltRbgAll(
(gpuQuad.getNumColors() <= 1), // isMonochrome()
use_reference,
null); // int [] wh
final int num_sens = gpuQuad.getNumSensors();
final float [][] per_sensor = new float [num_sens][];
for (int ncam = 0; ncam < num_sens; ncam++) {
per_sensor[ncam] = gpuQuad.getRBG(ncam)[0]; // mono (color 0)
}
return per_sensor;
}
/**
* Per-sensor linear photometric calibration fit: find per-sensor (a + b*x) that makes the 16 conditioned
* sensor images match each other, using only tiles where direct pixel comparison is valid:
* - far tiles (disparity &lt; max_disparity): small inter-sensor parallax -&gt; small image mismatch, safe to compare;
* - weak tiles (strength &lt; max_strength, e.g. sky): no details to match -&gt; also safe to compare pixel values.
* The excluded complement (high disparity AND high strength) is contrast near objects, where pixels mismatch
* because of the inter-sensor parallax. Outer tiles (1-tile border) are always discarded.
* Tile-&gt;pixel map: tile (tx,ty) -&gt; Rectangle(8*tx, 8*ty, 8, 8) (GPU transform size).
* Fit: iterate {reference = cross-sensor mean of corrected pixels; per-sensor least-squares (a+b*x) against
* the reference; gauge-normalize so mean(b)=1, mean(a)=0 (the overall level/scale is otherwise unconstrained
* and drifts); reject pixels with any-sensor residual &gt; 3 sigma (moving targets/clouds inside "safe" tiles)}.
* Converges in 2 iterations (validated offline: per-sensor mean spread 1353 -&gt; 5 counts, cross-sensor RMS
* 358 -&gt; 85 on 1773135476_186641). The result folds into the 2-map photometric calibration:
* scale'_s = b_s * scale_s, additive map FPN'_s(p) = b_s * FPN_s(p) - a_s (see {@link #adjustFpnLinear}).
* By Claude on 07/02/2026.
* @param center_CLT virtual "-CENTER" QuadCLT: source of the combo DSI (-INTER-INTRA-LMA) and debug-save target.
* @param per_sensor [sensor][pixel] conditioned (photometric+FPN corrected) images, e.g. from
* {@link #perSensorImagesFromTD} after the conditioned convert.
* @param width image width, pixels.
* @param max_strength select weak tiles: strength &lt; max_strength (e.g. 0.5).
* @param max_disparity or far tiles: disparity &lt; max_disparity (e.g. 1.0).
* @param save_adjusted save the corrected stack as &lt;name&gt;-CUAS-PERSENSOR-ADJ for visual verification.
* @param debugLevel debug verbosity.
* @param keep_averages keep average offset = 0, average scale = 1.0 (gauge normalization each iteration:
* the corrections then only redistribute between sensors, preserving the overall level and scale).
* @return {a[num_sens], b[num_sens]} or null if the combo DSI is not available.
*/
public static double [][] perSensorLinearFit( // By Claude on 07/02/2026
QuadCLT center_CLT,
float [][] per_sensor,
int width,
double max_strength,
double max_disparity,
boolean save_adjusted,
int debugLevel,
boolean keep_averages) { // By Claude on 07/02/2026: keep average offset=0, average scale=1.0
final double [][] combo_dsn = center_CLT.restoreComboDSI(true); // reads <name>-INTER-INTRA-LMA.tiff
if ((combo_dsn == null) || (combo_dsn.length <= OpticalFlow.COMBO_DSN_INDX_STRENGTH)) {
System.out.println("perSensorLinearFit(): no combo DSI (-INTER-INTRA-LMA) available - skipping");
return null;
}
final double [] disparity = combo_dsn[OpticalFlow.COMBO_DSN_INDX_DISP]; // slice 1 (1-based)
final double [] strength = combo_dsn[OpticalFlow.COMBO_DSN_INDX_STRENGTH]; // slice 2 (1-based)
final int num_sens = per_sensor.length;
final int height = per_sensor[0].length / width;
final int tile_size = GPUTileProcessor.DTT_SIZE; // 8 (1 << 3) - GPU transform size
final int tilesX = width / tile_size;
final int tilesY = height / tile_size;
// tile selection -> pixel mask
final boolean [] pix_mask = new boolean [width*height];
int num_sel_tiles = 0;
for (int ty = 1; ty < (tilesY - 1); ty++) { // discard the outer tiles (1-tile border)
for (int tx = 1; tx < (tilesX - 1); tx++) {
int nt = ty*tilesX + tx;
if ((strength[nt] < max_strength) || (disparity[nt] < max_disparity)) { // NaN -> excluded
num_sel_tiles++;
for (int y = 0; y < tile_size; y++) {
int row = (ty*tile_size + y)*width + tx*tile_size;
for (int x = 0; x < tile_size; x++) {
pix_mask[row + x] = true;
}
}
}
}
}
// require all sensors finite
final ArrayList<Integer> pix_list = new ArrayList<Integer>();
for (int npix = 0; npix < pix_mask.length; npix++) if (pix_mask[npix]) {
boolean all_finite = true;
for (int s = 0; s < num_sens; s++) if (Float.isNaN(per_sensor[s][npix])) { all_finite = false; break; }
if (all_finite) pix_list.add(npix);
}
final int num_pix = pix_list.size();
if (num_pix < 100) {
System.out.println("perSensorLinearFit(): only "+num_pix+" usable pixels - skipping");
return null;
}
final double [] a = new double [num_sens];
final double [] b = new double [num_sens];
Arrays.fill(b, 1.0);
final boolean [] keep = new boolean [num_pix];
Arrays.fill(keep, true);
final int num_iter = 3;
double rms_before = Double.NaN, rms_after = Double.NaN;
int num_keep = num_pix;
for (int iter = 0; iter < num_iter; iter++) {
// reference = cross-sensor mean of currently corrected pixels
final double [] ref = new double [num_pix];
for (int i = 0; i < num_pix; i++) {
int npix = pix_list.get(i);
double sum = 0.0;
for (int s = 0; s < num_sens; s++) sum += a[s] + b[s]*per_sensor[s][npix];
ref[i] = sum/num_sens;
}
// per-sensor least squares (a + b*x) against the reference, inlier pixels only
for (int s = 0; s < num_sens; s++) {
double sx = 0, sxx = 0, sy = 0, sxy = 0; int n = 0;
for (int i = 0; i < num_pix; i++) if (keep[i]) {
double x = per_sensor[s][pix_list.get(i)];
sx += x; sxx += x*x; sy += ref[i]; sxy += x*ref[i]; n++;
}
double den = n*sxx - sx*sx;
if (den != 0) {
b[s] = (n*sxy - sx*sy)/den;
a[s] = (sy - b[s]*sx)/n;
}
}
if (keep_averages) { // gauge fix: mean correction == identity (average offset = 0, average scale = 1.0)
double mean_a = 0, mean_b = 0;
for (int s = 0; s < num_sens; s++) { mean_a += a[s]; mean_b += b[s]; }
mean_a /= num_sens; mean_b /= num_sens;
for (int s = 0; s < num_sens; s++) { b[s] /= mean_b; a[s] = (a[s] - mean_a)/mean_b; }
}
// residuals + outlier rejection for the next pass (moving targets/clouds inside "safe" tiles)
double s2_all = 0, s2_keep = 0; long n_all = 0, n_keep = 0;
final double [] res = new double [num_sens];
double sigma = 0; // first pass over residuals to get sigma
{
double s2 = 0; long n = 0;
for (int i = 0; i < num_pix; i++) {
int npix = pix_list.get(i);
double mean = 0;
for (int s = 0; s < num_sens; s++) mean += a[s] + b[s]*per_sensor[s][npix];
mean /= num_sens;
for (int s = 0; s < num_sens; s++) { double r = a[s] + b[s]*per_sensor[s][npix] - mean; s2 += r*r; n++; }
}
sigma = Math.sqrt(s2/n);
s2_all = s2; n_all = n;
}
num_keep = 0;
for (int i = 0; i < num_pix; i++) {
int npix = pix_list.get(i);
double mean = 0;
for (int s = 0; s < num_sens; s++) { res[s] = a[s] + b[s]*per_sensor[s][npix]; mean += res[s]; }
mean /= num_sens;
boolean good = true;
double s2 = 0;
for (int s = 0; s < num_sens; s++) { double r = res[s] - mean; s2 += r*r; if (Math.abs(r) > 3*sigma) good = false; }
keep[i] = good;
if (good) { s2_keep += s2; num_keep++; }
}
rms_after = Math.sqrt(s2_keep/(num_keep*num_sens));
if (debugLevel > -3) {
System.out.println(String.format("perSensorLinearFit() iter %d: rms(inliers)=%.3f rms(all)=%.3f kept=%.1f%%",
iter, rms_after, Math.sqrt(s2_all/n_all), 100.0*num_keep/num_pix));
}
}
{ // "before" RMS for the report (identity correction)
double s2 = 0; long n = 0;
for (int i = 0; i < num_pix; i++) {
int npix = pix_list.get(i);
double mean = 0;
for (int s = 0; s < num_sens; s++) mean += per_sensor[s][npix];
mean /= num_sens;
for (int s = 0; s < num_sens; s++) { double r = per_sensor[s][npix] - mean; s2 += r*r; n++; }
}
rms_before = Math.sqrt(s2/n);
}
System.out.println("perSensorLinearFit(): "+num_sel_tiles+" tiles, "+num_pix+" pixels ("+num_keep+
" inliers), cross-sensor RMS: before="+String.format("%.2f", rms_before)+
" after="+String.format("%.2f", rms_after));
System.out.println(" s | a | b ");
for (int s = 0; s < num_sens; s++) {
System.out.println(String.format("%2d | %9.3f | %8.5f", s, a[s], b[s]));
}
if (save_adjusted) { // corrected stack for visual verification
final float [][] adj = new float [num_sens][per_sensor[0].length];
final String [] titles = new String [num_sens];
for (int s = 0; s < num_sens; s++) {
for (int npix = 0; npix < adj[s].length; npix++) {
adj[s][npix] = (float) (a[s] + b[s]*per_sensor[s][npix]);
}
titles[s] = String.format("s%02d_a=%.1f_b=%.4f", s, a[s], b[s]);
}
final ImagePlus imp = ShowDoubleFloatArrays.makeArrays(adj, width, height,
center_CLT.getImageName()+"-CUAS-PERSENSOR-ADJ", titles);
center_CLT.saveImagePlusInModelDirectory(null, imp);
}
return new double [][] {a, b};
}
/**
* Recalculate the 16+16 per-sensor lwir photometric parameters (offsets + scales) from the linear fit and
* save them to the corr-xml, so that when applied the same way as now (channelLwirApplyEqualize at load:
* scale*(raw-offset)) they compensate the remaining per-sensor mismatch between the /jp4/*.tiff images.
* Folding: corrected y' = a + b*y with y = scale*(raw-offset) is achieved by
* scale' = b * scale
* offset' = offset - a / scale'
* (the residual per-pixel term (1-b)*FPN(p) is second order - b is within ~15% of 1 and the FPN map is
* recomputed against the new photometric anyway). The offsets/scales are taken from the physical scene that
* conditioned the fitted images, updated on the calibration-carrier center instance together with the
* photometric_scene provenance marker, and written to &lt;name&gt;-ims.corr-xml via saveInterProperties().
* By Claude on 07/02/2026.
* @param center_CLT calibration carrier (virtual "-CENTER"): updated in memory and its corr-xml saved.
* @param phys_scene the physical scene whose current offsets/scales produced the fitted conditioned images.
* @param a per-sensor additive corrections from {@link #perSensorLinearFit}.
* @param b per-sensor multiplicative corrections from {@link #perSensorLinearFit}.
* @param save_corr_xml write the updated calibration to the center's corr-xml (false - update in memory only).
* @param debugLevel debug verbosity.
*/
public static void applyLwirLinearCalibration( // By Claude on 07/02/2026
QuadCLT center_CLT,
QuadCLT phys_scene,
double [] a,
double [] b,
boolean save_corr_xml,
int debugLevel) {
final int num_sens = a.length;
final double [] offsets = phys_scene.getLwirOffsets(); // current (what conditioned the fitted images)
final double [] scales = phys_scene.getLwirScales();
final double [] offsets_new = new double [num_sens];
final double [] scales_new = new double [num_sens];
System.out.println("applyLwirLinearCalibration(): recalculated per-sensor lwir offsets/scales"+
(save_corr_xml? " (saving to corr-xml)":" (in-memory only)"));
System.out.println(" s | offset old -> offset new | scale old -> scale new");
for (int s = 0; s < num_sens; s++) {
scales_new[s] = b[s]*scales[s];
offsets_new[s] = offsets[s] - a[s]/scales_new[s];
System.out.println(String.format("%2d | %12.3f -> %12.3f | %8.5f -> %8.5f",
s, offsets[s], offsets_new[s], scales[s], scales_new[s]));
}
center_CLT.setLwirOffsets(offsets_new);
center_CLT.setLwirScales(scales_new);
center_CLT.setPhotometricScene(phys_scene.getImageName()); // provenance: recalibrated on this scene
if (save_corr_xml) {
center_CLT.saveInterProperties( // writes <image_name>-ims.corr-xml into the x3d directory
null, // String path - null: default name in the x3d directory
debugLevel);
}
}
/**
* Fold the per-sensor linear corrections into the 2-map photometric calibration: the corrected pixel
* y' = a + b*y with y = scale*(raw-offset) - FPN(p) is equivalent to scale' = b*scale and
* FPN'(p) = b*FPN(p) - a. Returns NEW arrays (does not touch the input - the original may be referenced
* by the applied-FPN bookkeeping of already-conditioned scenes). By Claude on 07/02/2026.
* @param fpn [sensor][color][pixel] current per-pixel additive map (may be null -&gt; treated as zeros).
* @param num_pix number of pixels per sensor (used when fpn == null).
* @param a per-sensor additive correction from {@link #perSensorLinearFit}.
* @param b per-sensor multiplicative correction from {@link #perSensorLinearFit}.
* @return new [sensor][color][pixel] additive map = b*FPN - a.
*/
public static double [][][] adjustFpnLinear( // By Claude on 07/02/2026
double [][][] fpn,
int num_pix,
double [] a,
double [] b) {
final int num_sens = a.length;
final double [][][] fpn_adj = new double [num_sens][][];
for (int s = 0; s < num_sens; s++) {
double [] fp = ((fpn != null) && (fpn[s] != null)) ? fpn[s][0] : null;
int np = (fp != null) ? fp.length : num_pix;
fpn_adj[s] = new double [1][np];
for (int npix = 0; npix < np; npix++) {
fpn_adj[s][0][npix] = b[s]*((fp != null)? fp[npix] : 0.0) - a[s];
}
}
return fpn_adj;
}
/**
* Build a plain full-frame task grid for the conditioning test: uniform tile centers in the SENSOR domain
* (no virtual-view warp), disparity 0 (infinity), scale 1.0 (plain SET, no accumulation).
* The conditioning test must NOT reuse whatever tasks are left in the GPU: after a motion-blur render the
* leftover set is the MB secondary tasks with NEGATIVE fractional scales (measured -1/6 on LWIR16 CUAS -
* the "negative raw" images), and any leftover render grid is virtual-view warped, losing the same border
* ROI on every sensor. By Claude on 07/02/2026.
* @param qclt physical scene (geometry source), must be the QuadCLT currently bound to the GPU.
* @param disparity_corr disparity correction, pass clt_parameters.imp.disparity_corr + qclt.getDispInfinityRef().
* @param threadsMax max threads.
* @return TpTask[] covering all tiles, per-sensor xy to be computed by the GPU from its loaded geometry.
*/
public static TpTask [] uniformGridTasks( // By Claude on 07/02/2026
final QuadCLT qclt,
final double disparity_corr,
final int threadsMax) {
final int tilesX = qclt.getTilesX();
final int tilesY = qclt.getTilesY();
final double [][] pXpYD = new double [tilesX*tilesY][];
for (int nt = 0; nt < pXpYD.length; nt++) {
int ty = nt / tilesX, tx = nt % tilesX;
pXpYD[nt] = new double [] {
(tx + 0.5) * GPUTileProcessor.DTT_SIZE,
(ty + 0.5) * GPUTileProcessor.DTT_SIZE,
0.0};
}
return GpuQuad.setInterTasks( // accumulate=false, global_scale=1.0 -> plain SET tasks
qclt.getNumSensors(), // final int num_cams,
qclt.getErsCorrection().getSensorWH()[0], // final int img_width, // should match pXpYD
!qclt.hasGPU(), // final boolean calcPortsCoordinatesAndDerivatives,
pXpYD, // final double [][] pXpYD,
null, // final boolean [] selection,
qclt.getErsCorrection(), // final GeometryCorrection geometryCorrection,
disparity_corr, // final double disparity_corr,
-1, // final int margin,
null, // final boolean [] valid_tiles,
threadsMax); // final int threadsMax)
}
/**
* Conditioning-test worker: upload the given per-sensor pixel data to the GPU, rebuild the transform domain
* with this test's OWN uniform sensor-domain task grid (see {@link #uniformGridTasks}), then render and save
* the per-sensor stack via {@link #perSensorAveragesFromTD}. Used for both the conditioned image_data and the
* raw /jp4/ baseline so the two stacks are converted with IDENTICAL tasks/kernels and differ only by the data.
* Does NOT modify the QuadCLT image_data (a previous version staged raw via setImageData() - that both destroyed
* the conditioned data and armed the hasNewImageData/image_data_alt re-pull trap inside execConvertDirect).
* By Claude on 07/02/2026.
* @param clt_parameters processing parameters (task grid, LPF sigmas).
* @param save_qclt the virtual "-CENTER" QuadCLT where the result stack is saved.
* @param data [sensor][color][pixel] pixel data to convert (e.g. getOrigImageData() or raw jp4 read).
* @param name_tag filename tag, e.g. "-CUAS-PERSENSOR" / "-CUAS-PERSENSOR-RAW".
* @param threadsMax max threads.
* @param debugLevel debug verbosity.
* @return per-sensor mean of finite pixels.
*/
public static double [] perSensorFromData( // By Claude on 07/02/2026
CLTParameters clt_parameters,
QuadCLT save_qclt,
double [][][] data,
String name_tag,
int threadsMax,
int debugLevel) {
final GpuQuad gpuQuad = save_qclt.getGPUQuad();
final QuadCLT qclt = gpuQuad.getQuadCLT(); // physical scene bound to the GPU (geometry already loaded)
if (qclt == null) {
System.out.println("perSensorFromData(): no physical scene bound to the GPU - skipping");
return null;
}
final TpTask [] tp_tasks = uniformGridTasks(
qclt, // final QuadCLT qclt,
clt_parameters.imp.disparity_corr + qclt.getDispInfinityRef(), // final double disparity_corr,
threadsMax); // final int threadsMax)
final ImageDtt image_dtt = new ImageDtt(
qclt.getNumSensors(),
clt_parameters.transform_size,
clt_parameters.img_dtt,
qclt.isAux(),
qclt.isMonochrome(),
qclt.isLwir(),
clt_parameters.getScaleStrength(qclt.isAux()),
gpuQuad);
// By Claude on 07/02/2026: print the host-side data level (raw ~ +20800 counts, conditioned ~ -3500
// with the current stale offsets) so a clobbered upload is immediately visible against the saved stack.
System.out.println("perSensorFromData("+name_tag+"): host data s00 mean="+
meanFinite(data[0][0], gpuQuad.getImageWidth()*gpuQuad.getImageHeight()));
gpuQuad.setBayerImages(data, true); // FORCE upload the data under test
qclt.setHasNewImageData(false); // GPU now holds 'data': makes execConvertDirect()'s internal
// setBayerImages(false,...) skip its re-pull (base and JNA backends)
image_dtt.setReferenceTD( // set OWN tasks + convert into the main TD buffer (kernels applied)
null, // final float [][] fclt,
1, // final int erase_clt, // 1: erase to NaN (honest borders)
null, // final int [] wh, // use sensor dimensions
clt_parameters.img_dtt, // final ImageDttParameters imgdtt_params,
false, // final boolean use_reference_buffer,
tp_tasks, // final TpTask[] tp_tasks,
clt_parameters.gpu_sigma_r, // final double gpu_sigma_r,
clt_parameters.gpu_sigma_b, // final double gpu_sigma_b,
clt_parameters.gpu_sigma_g, // final double gpu_sigma_g,
clt_parameters.gpu_sigma_m, // final double gpu_sigma_m,
threadsMax, // final int threadsMax,
debugLevel); // final int globalDebugLevel)
return perSensorAveragesFromTD(gpuQuad, false, name_tag, save_qclt);
}
/** NaN-tolerant mean of the first num_pix elements (or all when num_pix <= 0). By Claude on 07/02/2026. */
private static double meanFinite(double [] data, int num_pix) {
if (data == null) return Double.NaN;
int n = ((num_pix > 0) && (num_pix < data.length)) ? num_pix : data.length;
double s = 0.0; int m = 0;
for (int i = 0; i < n; i++) if (!Double.isNaN(data[i])) { s += data[i]; m++; }
return (m > 0) ? (s/m) : Double.NaN;
}
private static double meanFinite(float [] data) { // By Claude on 07/02/2026
if (data == null) return Double.NaN;
double s = 0.0; int m = 0;
for (int i = 0; i < data.length; i++) if (!Float.isNaN(data[i])) { s += data[i]; m++; }
return (m > 0) ? (s/m) : Double.NaN;
}
/** /**
* RT-seed / conditioning-test companion: render the per-sensor images from RAW /jp4/ source data with * RT-seed / conditioning-test companion: render the per-sensor images from RAW /jp4/ source data with
* NO photometric / FPN / conditioning correction, saved as &lt;name&gt;-CUAS-PERSENSOR-RAW, to compare against * NO photometric / FPN / conditioning correction, saved as &lt;name&gt;-CUAS-PERSENSOR-RAW, to compare against
...@@ -3747,12 +4140,13 @@ public class CuasMotion { ...@@ -3747,12 +4140,13 @@ public class CuasMotion {
* computed in the background of RT tracking) and is never applied on this raw path. * computed in the background of RT tracking) and is never applied on this raw path.
* Java-first oracle for the future GPU RT path: Java locates + reads the scene's source tiffs via getJp4Tiff * Java-first oracle for the future GPU RT path: Java locates + reads the scene's source tiffs via getJp4Tiff
* (the oracle reader - it parses the Boson telemetry first line and restores the ch-6 floating bit-12 needed * (the oracle reader - it parses the Boson telemetry first line and restores the ch-6 floating bit-12 needed
* for older footage), then FORCE-uploads them H2D (setBayerImages(..., force=true), bypassing the * for older footage), then FORCE-uploads them H2D and rebuilds the transform domain with the test's own
* "GPU memory already correct" verification that would otherwise skip the reload), and execConvertDirect * uniform sensor-domain task grid (see {@link #perSensorFromData}). In RT this same force-H2D is fed by the
* rebuilds the transform domain from raw. In RT this same force-H2D is fed by the four SATA devices (4 sensors * four SATA devices (4 sensors each). TIFF-read acceleration (skip Exif/telemetry parse on the uncompressed
* each). TIFF-read acceleration (skip Exif/telemetry parse on the uncompressed 16-bit BE tiff) is deferred - * 16-bit BE tiff) is deferred - only if RT becomes CPU-bound; SATA-2 covers the ~158 MB/s per path.
* only if RT becomes CPU-bound; SATA-2 covers the ~158 MB/s per path. * By Claude on 07/01/2026; reworked 07/02/2026 to stop reusing leftover GPU tasks (negative-MB-scale /
* By Claude on 07/01/2026. * warped-border artifacts) and to stop overwriting the scene's conditioned image_data.
* @param clt_parameters processing parameters (task grid, LPF sigmas). // By Claude on 07/02/2026
* @param save_qclt the virtual "-CENTER" QuadCLT where the result stack is saved (main CUAS results live there); * @param save_qclt the virtual "-CENTER" QuadCLT where the result stack is saved (main CUAS results live there);
* the raw /jp4/ source, geometry, and GPU are taken from the physical scene currently bound to that GPU * the raw /jp4/ source, geometry, and GPU are taken from the physical scene currently bound to that GPU
* (gpuQuad.getQuadCLT()), since the virtual center frame has no /jp4/ files of its own. * (gpuQuad.getQuadCLT()), since the virtual center frame has no /jp4/ files of its own.
...@@ -3761,6 +4155,7 @@ public class CuasMotion { ...@@ -3761,6 +4155,7 @@ public class CuasMotion {
* @return per-sensor mean of finite raw pixels, or null if no physical scene / source files were found. * @return per-sensor mean of finite raw pixels, or null if no physical scene / source files were found.
*/ */
public static double [] perSensorFromRawJp4( public static double [] perSensorFromRawJp4(
CLTParameters clt_parameters, // By Claude on 07/02/2026
QuadCLT save_qclt, QuadCLT save_qclt,
int threadsMax, int threadsMax,
int debugLevel) { int debugLevel) {
...@@ -3819,20 +4214,27 @@ public class CuasMotion { ...@@ -3819,20 +4214,27 @@ public class CuasMotion {
}; };
} }
ImageDtt.startAndJoin(threads); ImageDtt.startAndJoin(threads);
// Force the raw H2D and rebuild the transform domain from raw (bypass the GPU-memory verify, per RT design). // By Claude on 07/02/2026: convert with the test's OWN uniform task grid (leftover GPU tasks after an
qclt.setImageData(raw); // stage raw as the (physical) QuadCLT image data (the RT raw-stream slot) // MB-enabled render are the secondary set with negative fractional scales -> "negative raw" images, and
gpuQuad.setBayerImages(raw, true); // FORCE upload, bypassing the "already loaded / unchanged" check // a leftover virtual-view grid loses the same border ROI on every sensor). Do NOT setImageData(raw): it
gpuQuad.execConvertDirect( // destroyed the scene's conditioned data and armed the hasNewImageData/image_data_alt re-pull trap.
false, // ref_scene - build the main (current-scene) buffer, not the reference { // By Claude on 07/02/2026: instrumentation - verify what was actually read (raw LWIR should be ~+20800)
null, // wh - use sensor dimensions int num_null = 0;
0, // erase_clt - 0: erase to 0.0 before accumulate StringBuffer sb = new StringBuffer();
false, // no_kernels - keep deconvolution kernels (match the conditioned path) for (int ncam = 0; ncam < num_sens; ncam++) {
false); // use_center_image if ((raw[ncam] == null) || (raw[ncam][0] == null)) { num_null++; continue; }
if (debugLevel > -2) { if ((ncam % 4) == 0) sb.append(String.format(" s%02d=%.0f", ncam, meanFinite(raw[ncam][0], 0)));
System.out.println("perSensorFromRawJp4(): built raw TD for set '"+set_name+"' ("+found+" sensor source files)"); }
} System.out.println("perSensorFromRawJp4(): read raw source for set '"+set_name+"' ("+found+
// Render + save the raw per-sensor stack to the -CENTER instance, alongside the conditioned one. " source files, "+num_null+" null sensors) means:"+sb);
return perSensorAveragesFromTD(gpuQuad, false, "-CUAS-PERSENSOR-RAW", save_qclt); if (num_null > 0) {
System.out.println("perSensorFromRawJp4(): "+num_null+" sensors have NO raw data - aborting raw render "+
"(would crash or silently keep previous GPU bayer)");
return null;
}
}
// Upload + convert + render + save the raw per-sensor stack to the -CENTER instance, alongside the conditioned one.
return perSensorFromData(clt_parameters, save_qclt, raw, "-CUAS-PERSENSOR-RAW", threadsMax, debugLevel);
} }
......
...@@ -73,16 +73,24 @@ public class GpuQuadJna extends GpuQuad { ...@@ -73,16 +73,24 @@ public class GpuQuadJna extends GpuQuad {
// (num_pairs / sel_pairs / color_weights / corr_out_rad) is known at first use. // (num_pairs / sel_pairs / color_weights / corr_out_rad) is known at first use.
} }
// By Claude on 07/02/2026: native-side equivalent of gpuTileProcessor.bayer_set (that field's owner is null
// here). Tracks whether the native bayer buffers hold the bound quadCLT's current data, so that
// execConvertDirect()'s internal setBayerImages(false,...) SKIPS the re-pull exactly like the JCuda base
// does. Without it every convert unconditionally re-uploaded quadCLT.getResetImageData(), silently
// clobbering any explicitly uploaded data (setBayerImages(data, true) / setBayerImage()) - found by the
// curt_cond_test raw baseline coming out bit-identical to the conditioned render.
private boolean jna_bayer_set = false;
// Switch this shared GpuQuad to a different scene's QuadCLT. Base also clears // Switch this shared GpuQuad to a different scene's QuadCLT. Base also clears
// gpuTileProcessor.bayer_set (null here) — N/A natively: bayer + geometry are re-uploaded on the next // gpuTileProcessor.bayer_set (null here) — mirrored with jna_bayer_set: the next convert re-uploads.
// exec, kernels are unchanged. resetGeometryCorrection*/() are flag-only (safe to call). // resetGeometryCorrection*/() are flag-only (safe to call).
@Override public void updateQuadCLT(final QuadCLT quadCLT) { @Override public void updateQuadCLT(final QuadCLT quadCLT) {
this.quadCLT = quadCLT; this.quadCLT = quadCLT;
resetGeometryCorrection(); resetGeometryCorrection();
resetGeometryCorrectionVector(); resetGeometryCorrectionVector();
jna_bayer_set = false; // By Claude on 07/02/2026: match base (scene switch invalidates uploaded bayer)
} }
// Native re-uploads bayer each convert_direct -> nothing to reset (base clears the null gpuTileProcessor flag). @Override public void resetBayer() { jna_bayer_set = false; } // By Claude on 07/02/2026: match base semantics
@Override public void resetBayer() { /* no-op for the native backend */ }
// CLT sizing (base derefs null gpu_clt_wh) — full-frame dims; matches TpProc's slice. // CLT sizing (base derefs null gpu_clt_wh) — full-frame dims; matches TpProc's slice.
@Override public int getCltSize(boolean use_ref) { @Override public int getCltSize(boolean use_ref) {
...@@ -194,6 +202,11 @@ public class GpuQuadJna extends GpuQuad { ...@@ -194,6 +202,11 @@ public class GpuQuadJna extends GpuQuad {
if (f != null) lib.tp_proc_set_center_image(proc, f); // broadcast to all sensors if (f != null) lib.tp_proc_set_center_image(proc, f); // broadcast to all sensors
return; return;
} }
// By Claude on 07/02/2026: same guard as the JCuda base - do NOT re-pull quadCLT data over an
// explicit upload when the native buffers are current and the quadCLT has no new data.
if (!force && jna_bayer_set && !quadCLT.hasNewImageData()) {
return;
}
setBayerImages(quadCLT.getResetImageData(), true); setBayerImages(quadCLT.getResetImageData(), true);
} }
@Override public void setBayerImages(double[][][] bayer_data, boolean force) { @Override public void setBayerImages(double[][][] bayer_data, boolean force) {
...@@ -201,6 +214,7 @@ public class GpuQuadJna extends GpuQuad { ...@@ -201,6 +214,7 @@ public class GpuQuadJna extends GpuQuad {
float[] f = combineChannels(bayer_data[ncam]); float[] f = combineChannels(bayer_data[ncam]);
if (f != null) lib.tp_proc_set_image(proc, ncam, f); if (f != null) lib.tp_proc_set_image(proc, ncam, f);
} }
jna_bayer_set = true; // By Claude on 07/02/2026: match base (gpuTileProcessor.bayer_set = true)
} }
// sum the (1 or 3) split-color channels into one image, as GpuQuad.setBayerImages does // sum the (1 or 3) split-color channels into one image, as GpuQuad.setBayerImages does
private static float[] combineChannels(double[][] chans) { private static float[] combineChannels(double[][] chans) {
......
...@@ -7281,11 +7281,42 @@ java.lang.NullPointerException ...@@ -7281,11 +7281,42 @@ java.lang.NullPointerException
// and compare. Well-calibrated -> the 16 per-sensor averages match (spread ~0). Runs instead of RT detection. // and compare. Well-calibrated -> the 16 per-sensor averages match (spread ~0). Runs instead of RT detection.
// By Claude on 07/01/2026 // By Claude on 07/01/2026
System.out.println("===== CUAS RT conditioning test (curt_cond_test): per-sensor average spread ====="); System.out.println("===== CUAS RT conditioning test (curt_cond_test): per-sensor average spread =====");
CuasMotion.perSensorAveragesFromTD(master_CLT.getGPUQuad(), false, "-CUAS-PERSENSOR", master_CLT); // conditioned TD, saved to the -CENTER instance // By Claude on 07/02/2026: both stacks are now converted with the test's OWN uniform sensor-domain
// task grid (scale 1.0) instead of leftover GPU state: after an MB-enabled render the leftover tasks
// are the MB secondary set with negative fractional scales (-> "negative raw" images = -1/6 x input),
// and the leftover virtual-view grid loses the same border ROI on every sensor.
QuadCLT cond_phys = master_CLT.getGPUQuad().getQuadCLT(); // physical scene bound to the GPU
if (cond_phys != null) { // conditioned baseline: the actual image_data (bypass image_data_alt), same grid as raw
CuasMotion.perSensorFromData(clt_parameters, master_CLT, cond_phys.getOrigImageData(),
"-CUAS-PERSENSOR", ImageDtt.THREADS_MAX, debugLevel);
// By Claude on 07/02/2026: per-sensor linear (a + b*x) photometric fit over "safe" tiles
// (weak: no details, or far: small parallax), to become the new per-sensor scale/offset
// calibration: scale' = b*scale, additive map' = b*FPN - a. Prints the table and saves
// -CUAS-PERSENSOR-ADJ for visual verification; application to the stored calibration TBD.
float [][] cond_images = CuasMotion.perSensorImagesFromTD(master_CLT.getGPUQuad(), false); // TD still conditioned
double [][] ab = CuasMotion.perSensorLinearFit(
master_CLT, // QuadCLT center_CLT (combo DSI source, save target)
cond_images, // float [][] per_sensor (conditioned, FPN-corrected)
master_CLT.getGPUQuad().getImageWidth(), // int width
0.5, // double max_strength, // weak tiles: strength < 0.5
1.0, // double max_disparity, // far tiles: disparity < 1
true, // boolean save_adjusted (-CUAS-PERSENSOR-ADJ)
debugLevel, // int debugLevel
true); // boolean keep_averages: average offset = 0, average scale = 1.0
if (ab != null) { // recalculate the 16+16 lwir offsets/scales, save to the center corr-xml
CuasMotion.applyLwirLinearCalibration( // By Claude on 07/02/2026
master_CLT, // QuadCLT center_CLT (calibration carrier, corr-xml save)
cond_phys, // QuadCLT phys_scene (its offsets/scales conditioned the fit input)
ab[0], // double [] a,
ab[1], // double [] b,
true, // boolean save_corr_xml
debugLevel); // int debugLevel
}
}
// Raw /jp4/ baseline (no photometric/FPN/conditioning), saved as -CUAS-PERSENSOR-RAW for side-by-side compare. // Raw /jp4/ baseline (no photometric/FPN/conditioning), saved as -CUAS-PERSENSOR-RAW for side-by-side compare.
// RT-seed: Java reads the source tiffs and force-uploads them H2D; later the compute moves to GPU (Java = oracle). // RT-seed: Java reads the source tiffs and force-uploads them H2D; later the compute moves to GPU (Java = oracle).
// By Claude on 07/01/2026 // By Claude on 07/01/2026
CuasMotion.perSensorFromRawJp4(master_CLT, ImageDtt.THREADS_MAX, debugLevel); CuasMotion.perSensorFromRawJp4(clt_parameters, master_CLT, ImageDtt.THREADS_MAX, debugLevel);
} else { } else {
cuasRangingRT.saveUasFlightLogCsv(uasLogReader, imp_targets); // UAS flight-log truth -> <name>-UAS_DATA.tsv (mode-0 only; needs QuadCLT pose). By Claude on 06/24/2026 cuasRangingRT.saveUasFlightLogCsv(uasLogReader, imp_targets); // UAS flight-log truth -> <name>-UAS_DATA.tsv (mode-0 only; needs QuadCLT pose). By Claude on 06/24/2026
new CuasDetectRT( new CuasDetectRT(
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment