Commit d3c015c3 authored by Andrey Filippov's avatar Andrey Filippov

CLAUDE: CUAS RT per-sensor conditioning test + NaN-tolerant subtract-avg/LoG

Add a curt_cond_test path (boolean at the top of the CUAS-RT dialog) that,
inside the curt_en branch, renders the 16 per-sensor images and saves them to
the -CENTER instance for calibration inspection:
- CuasMotion.perSensorAveragesFromTD: imclt the per-sensor TD, print the
  16-sensor average spread, save -CUAS-PERSENSOR (16-slice stack, per-slice
  avg labels). Saves via the -CENTER instance, not gpuQuad.getQuadCLT().
- CuasMotion.perSensorFromRawJp4: read RAW /jp4/ per-sensor (oracle getJp4Tiff,
  one thread/sensor), force-H2D (bypass the "GPU mem already correct" verify),
  execConvertDirect from raw, save -CUAS-PERSENSOR-RAW (uncorrected baseline;
  calibration stays a separate "cheat"). RT-seed for the future SATA raw
  stream; GPU port later with Java as oracle.

Fix the NaN border on the RT SUBAVG-CONV2D product:
- CuasDetectRT subtract-average -> NaN-tolerant union (average only non-NaN
  scenes per pixel), matching the oracle -CUAS-MERGED-CUAS; the plain sum
  NaN-propagated (one missing scene poisoned the pixel in every frame ->
  thick border after LoG).
- CuasRTUtils.convolve2DLReLU -> NaN-aware (NaN out only if the center is NaN;
  substitute the center value for NaN taps), so the LoG can't bloom a thin
  border into a thick NaN frame.
- Add -SUBAVG-PRELOG save (post-subtract-avg, pre-LoG) for bisecting.

Compiles (mvn -DskipTests clean package). WIP: the raw-path values/edge and the
in-memory-vs-file (MERGED-CUAS) divergence are still under review; the ~28px
edge residual is traced to the temporal subtract-average at the rotation-swept
composite edge. See ANDREY_CONTINUE.md open items.
Co-Authored-By: 's avatarClaude Opus 4.8 (1M context) <noreply@anthropic.com>
parent a62a6b06
......@@ -3674,26 +3674,40 @@ public class CuasMotion {
/**
* Render all sensors from the current transform domain (IMCLT) and return the per-sensor
* average pixel value. Memory-lean: each sensor's rendered image is averaged in place and
* discarded, never materializing all sensors' pixels (no [nsens][w*h] array).
* average pixel value. Also SAVES the 16 per-sensor rendered images as one multi-slice TIFF in
* the model directory (via the GpuQuad's QuadCLT, the same save method the oracle path uses), so
* the spread can be inspected visually (DC offset vs. structured row/col/FPN patterns) rather than
* only as a number - the per-slice label carries that sensor's average.
* Good conditioning / photometric calibration is indicated by the per-sensor averages matching
* (approximately) - the 16 sensors view ~the same scene. Prints spread (max-min) and std as an
* at-a-glance quality metric. Path-2 calibration/conditioning test tool.
* By Claude on 06/30/2026.
* By Claude on 06/30/2026; per-sensor image save added 07/01/2026.
* @param gpuQuad the (16-sensor) GpuQuad whose transform-domain data to render, e.g. QuadCLT.getGPUQuad().
* @param use_reference render the reference-scene TD (gpu_clt_ref) instead of the current scene.
* @return per-sensor mean of finite pixels, length getNumSensors().
*/
public static double [] perSensorAveragesFromTD(GpuQuad gpuQuad, boolean use_reference) {
return perSensorAveragesFromTD(gpuQuad, use_reference, "-CUAS-PERSENSOR", null);
}
/** As {@link #perSensorAveragesFromTD(GpuQuad, boolean)} but with an explicit filename tag and save target.
* @param name_tag filename tag for the saved per-sensor stack (e.g. "-CUAS-PERSENSOR-RAW").
* @param save_qclt QuadCLT whose model directory receives the stack - pass the virtual "-CENTER" instance
* (main CUAS results live there), NOT the physical scene. null -> fall back to gpuQuad.getQuadCLT() (physical).
* By Claude on 07/01/2026. */
public static double [] perSensorAveragesFromTD(GpuQuad gpuQuad, boolean use_reference, String name_tag, QuadCLT save_qclt) {
gpuQuad.execImcltRbgAll(
(gpuQuad.getNumColors() <= 1), // isMonochrome()
use_reference,
null); // int [] wh
final int num_sens = gpuQuad.getNumSensors();
final int width = gpuQuad.getImageWidth(); // valid after last IMCLT
final int height = gpuQuad.getImageHeight();
final double [] avg = new double [num_sens];
final float [][] per_sensor = new float [num_sens][]; // kept for the saved stack (one w*h slice per sensor)
for (int ncam = 0; ncam < num_sens; ncam++) {
final float [][] rbg = gpuQuad.getRBG(ncam); // [color][pixel]
final float [] px = rbg[0]; // mono (color 0)
per_sensor[ncam] = px;
double sum = 0.0; int n = 0;
for (int i = 0; i < px.length; i++) {
final float v = px[i];
......@@ -3709,9 +3723,118 @@ public class CuasMotion {
System.out.printf("perSensorAveragesFromTD(ref=%b): mean=%.3f spread(max-min)=%.3f std=%.3f over %d sensors%n",
use_reference, mean, (mx - mn), std, m);
}
// Save the per-sensor stack into the model dir. Prefer the explicit save target (the virtual "-CENTER"
// instance where main CUAS results live); fall back to the GpuQuad's bound (physical) QuadCLT.
final QuadCLT qclt = (save_qclt != null) ? save_qclt : gpuQuad.getQuadCLT();
if (qclt != null) {
final String [] titles = new String [num_sens];
for (int ncam = 0; ncam < num_sens; ncam++) {
titles[ncam] = String.format("s%02d_avg=%.1f", ncam, avg[ncam]);
}
final String title = qclt.getImageName() + name_tag + (use_reference ? "-REF" : "");
final ImagePlus imp = ShowDoubleFloatArrays.makeArrays(per_sensor, width, height, title, titles);
qclt.saveImagePlusInModelDirectory(null, imp); // null suffix -> use imp title as filename
} else {
System.out.println("perSensorAveragesFromTD(): gpuQuad.getQuadCLT()==null, skipping per-sensor image save");
}
return avg;
}
/**
* RT-seed / conditioning-test companion: render the per-sensor images from RAW /jp4/ source data with
* NO photometric / FPN / conditioning correction, saved as &lt;name&gt;-CUAS-PERSENSOR-RAW, to compare against
* the conditioned -CUAS-PERSENSOR render. The calibration stays a separate "cheat" (prior/slow software, or
* computed in the background of RT tracking) and is never applied on this raw path.
* Java-first oracle for the future GPU RT path: Java locates + reads the scene's source tiffs via getJp4Tiff
* (the oracle reader - it parses the Boson telemetry first line and restores the ch-6 floating bit-12 needed
* for older footage), then FORCE-uploads them H2D (setBayerImages(..., force=true), bypassing the
* "GPU memory already correct" verification that would otherwise skip the reload), and execConvertDirect
* rebuilds the transform domain from raw. In RT this same force-H2D is fed by the four SATA devices (4 sensors
* each). TIFF-read acceleration (skip Exif/telemetry parse on the uncompressed 16-bit BE tiff) is deferred -
* only if RT becomes CPU-bound; SATA-2 covers the ~158 MB/s per path.
* By Claude on 07/01/2026.
* @param save_qclt the virtual "-CENTER" QuadCLT where the result stack is saved (main CUAS results live there);
* the raw /jp4/ source, geometry, and GPU are taken from the physical scene currently bound to that GPU
* (gpuQuad.getQuadCLT()), since the virtual center frame has no /jp4/ files of its own.
* @param threadsMax max read threads (one per sensor).
* @param debugLevel debug verbosity.
* @return per-sensor mean of finite raw pixels, or null if no physical scene / source files were found.
*/
public static double [] perSensorFromRawJp4(
QuadCLT save_qclt,
int threadsMax,
int debugLevel) {
final GpuQuad gpuQuad = save_qclt.getGPUQuad();
final QuadCLT qclt = gpuQuad.getQuadCLT(); // physical scene bound to the GPU: has the /jp4/ source + geometry
if (qclt == null) {
System.out.println("perSensorFromRawJp4(): no physical scene bound to the GPU (getQuadCLT()==null) - skipping raw render");
return null;
}
final int num_sens = qclt.getNumSensors();
final String set_name = qclt.getImageName();
final String [] sourceFiles = qclt.correctionsParameters.getSourcePaths();
// Map each sensor channel -> index into sourceFiles for THIS scene (matched by set name).
final int [] channelFiles = new int [num_sens];
java.util.Arrays.fill(channelFiles, -1);
int found = 0;
for (int nFile = 0; nFile < sourceFiles.length; nFile++) {
if ((sourceFiles[nFile] == null) || (sourceFiles[nFile].length() <= 1)) continue;
if (!set_name.equals(qclt.correctionsParameters.getNameFromSourceTiff(sourceFiles[nFile]))) continue;
final int [] channels = qclt.fileChannelToSensorChannels(
qclt.correctionsParameters.getChannelFromSourceTiff(sourceFiles[nFile]));
if (channels == null) continue;
for (int ch : channels) if ((ch >= 0) && (ch < num_sens) && qclt.eyesisCorrections.isChannelEnabled(ch)) {
channelFiles[ch] = nFile;
found++;
}
}
if (found == 0) {
System.out.println("perSensorFromRawJp4(): no source files found for set '"+set_name+
"' - is the jp4 source dir available? Skipping raw render.");
return null;
}
// Read raw per-sensor bayer (NO conditioning) -> [sensor][color][pixel], one thread per sensor
// (matches the RT model: 4 SATA paths x 4 sensors read concurrently). Reuses the existing oracle reader;
// a custom fast tiff read (skip Exif/telemetry parse on the uncompressed 16-bit BE tiff) is deferred until/
// unless this read becomes CPU-bound.
final double [][][] raw = new double [num_sens][][];
final Thread[] threads = ImageDtt.newThreadArray(threadsMax);
final AtomicInteger ai = new AtomicInteger(0);
for (int ithread = 0; ithread < threads.length; ithread++) {
threads[ithread] = new Thread() {
public void run() {
for (int ncam = ai.getAndIncrement(); ncam < num_sens; ncam = ai.getAndIncrement()) {
final int nFile = channelFiles[ncam];
if (nFile < 0) { System.out.println("perSensorFromRawJp4(): no source file for sensor "+ncam); continue; }
final ImagePlus imp_src = qclt.eyesisCorrections.getJp4Tiff(
sourceFiles[nFile],
qclt.geometryCorrection.woi_tops,
qclt.geometryCorrection.camera_heights);
raw[ncam] = qclt.eyesisCorrections.bayerToDoubleStack(
imp_src, // source Bayer image, linearized, 32-bit float
null, // no margins, no oversample
qclt.isMonochrome());
}
}
};
}
ImageDtt.startAndJoin(threads);
// Force the raw H2D and rebuild the transform domain from raw (bypass the GPU-memory verify, per RT design).
qclt.setImageData(raw); // stage raw as the (physical) QuadCLT image data (the RT raw-stream slot)
gpuQuad.setBayerImages(raw, true); // FORCE upload, bypassing the "already loaded / unchanged" check
gpuQuad.execConvertDirect(
false, // ref_scene - build the main (current-scene) buffer, not the reference
null, // wh - use sensor dimensions
0, // erase_clt - 0: erase to 0.0 before accumulate
false, // no_kernels - keep deconvolution kernels (match the conditioned path)
false); // use_center_image
if (debugLevel > -2) {
System.out.println("perSensorFromRawJp4(): built raw TD for set '"+set_name+"' ("+found+" sensor source files)");
}
// Render + save the raw per-sensor stack to the -CENTER instance, alongside the conditioned one.
return perSensorAveragesFromTD(gpuQuad, false, "-CUAS-PERSENSOR-RAW", save_qclt);
}
public ImagePlus convertToRgbAnnotateTargets(
CLTParameters clt_parameters,
......
......@@ -988,10 +988,23 @@ public class CuasDetectRT {
if (clt_parameters.imp.curt_subtract_avg && (dpixels != null) && (dpixels.length > 0)) { // de-streak: subtract the static background before LoG so the sharp treeline edge (and its tile-grid ringing) is gone; the moving target is not in the average // By Claude on 06/14/2026
int npx = dpixels[0].length;
double [] mean = new double [npx];
for (double [] fr : dpixels) for (int p = 0; p < npx; p++) mean[p] += fr[p];
for (int p = 0; p < npx; p++) mean[p] /= dpixels.length;
int [] cnt = new int [npx];
// NaN-tolerant (union) average: as the camera rotates, side areas are seen only by some scenes.
// Average ONLY the non-NaN scenes per pixel so a border pixel survives wherever ANY scene saw it,
// instead of NaN-poisoning it in the mean (and hence in every frame after subtraction). Matches the
// oracle -CENTER-CUAS-MERGED-CUAS behavior; the plain sum here NaN-propagated (thick border after LoG).
// By Claude on 07/01/2026
for (double [] fr : dpixels) for (int p = 0; p < npx; p++) if (!Double.isNaN(fr[p])) { mean[p] += fr[p]; cnt[p]++; }
for (int p = 0; p < npx; p++) mean[p] = (cnt[p] > 0) ? (mean[p] / cnt[p]) : Double.NaN;
// frame - mean: a scene that saw the pixel stays valid; one that didn't stays NaN; NaN only where NO scene saw it.
for (double [] fr : dpixels) for (int p = 0; p < npx; p++) fr[p] -= mean[p];
System.out.println(now()+" detectTargets(): subtracted temporal average over "+dpixels.length+" input frames (curt_subtract_avg - uses whole sequence; realtime would use a prior-run average)"); // By Claude on 06/14/2026
System.out.println(now()+" detectTargets(): subtracted temporal average over "+dpixels.length+" input frames (curt_subtract_avg, NaN-tolerant union - uses whole sequence; realtime would use a prior-run average)"); // By Claude on 06/14/2026
}
if (save_LoG_pixels) { // pre-LoG stack (post-subtract-average): inspect edge structure BEFORE the LoG to
// separate union-average residual at the partial-coverage rotation edge from LoG-touching-NaN ringing. By Claude on 07/01/2026
String title_prelog = getBaseName()+"-PRELOG";
ImagePlus imp_prelog = ShowDoubleFloatArrays.makeArrays(dpixels, getWidth(), getHeight(), title_prelog, getTimeStamps());
QuadCLTCPU.saveImagePlusInDirectory(imp_prelog, getModelDirectory());
}
double [][] dpixels_log = new double [dpixels.length][dpixels[0].length];
double [] kernel2d = cuasRTUtils.getKernel2d();
......
......@@ -544,12 +544,17 @@ public class CuasRTUtils {
public void run() {
for (int nPix = ai.getAndIncrement(); nPix < indx_center_2d.length; nPix = ai.getAndIncrement()) {
int ipix_dst = indx_center_2d[nPix];
// NaN-aware LoG: NaN out only if the CENTER is NaN; substitute center for NaN taps
// (edge-replication) so a valid pixel never blooms to NaN from a NaN neighbor. By Claude on 07/01/2026
double c = data[ipix_dst];
if (Double.isNaN(c)) { result[ipix_dst] = Double.NaN; continue; }
int ipix_src = ipix_dst- tl_offset;
int ikern = 0;
double v = 0;
for (int iy = 0; iy < kernel_size; iy++) {
for (int ix = 0; ix < kernel_size; ix++) {
v+=data[ipix_src++]*kernel[ikern++];
double d = data[ipix_src++];
v += (Double.isNaN(d) ? c : d) * kernel[ikern++];
}
ipix_src+=row_inc;
}
......@@ -567,6 +572,8 @@ public class CuasRTUtils {
TileNeibs tn = new TileNeibs(width, height);
for (int nPix = ai.getAndIncrement(); nPix < indx_margins_2d.length; nPix = ai.getAndIncrement()) {
int ipix_dst = indx_margins_2d[nPix];
double c = data[ipix_dst];
if (Double.isNaN(c)) { result[ipix_dst] = Double.NaN; continue; }
double v = 0;
for (int iy = 0; iy < kernel_size; iy++) {
int dy = iy-kernel2d_rad;
......@@ -574,7 +581,8 @@ public class CuasRTUtils {
int dx = ix-kernel2d_rad;
int ipix_src = tn.getNeibIndex(ipix_dst, dx, dy);
if (ipix_src >= 0) {
v+=data[ipix_src]*kernel[ix + iy*kernel_size];
double d = data[ipix_src];
v += (Double.isNaN(d) ? c : d) * kernel[ix + iy*kernel_size];
}
}
}
......
......@@ -1125,6 +1125,7 @@ min_str_neib_fpn 0.35
// CUAS Realtime
public boolean curt_en = true; // enable cuas rt calculation (not needed with a separate button)
public boolean curt_cond_test = false; // conditioning/calibration isolation test inside the curt_en branch: build the QuadCLT instances (borrowed calibration) then print per-sensor average spread (CuasMotion.perSensorAveragesFromTD) instead of normal RT detection; well-calibrated -> 16 sensor averages match (spread ~0). // By Claude on 07/01/2026
//=== LoG prefilter ===
public double curt_psf_radius = 1.0; // sensor PSF radius for LoG pre-filter
public double curt_n_sigma = 4.0; // cutoff LoG kernel array, number of sigmas
......@@ -3410,6 +3411,8 @@ min_str_neib_fpn 0.35
gd.addTab("CUAS RT", "CUAS Real Time");
gd.addCheckbox ("CUAS realtime enable", this.curt_en,
"Enable testing of the realtime CUAS detection.");
gd.addCheckbox ("CUAS RT conditioning test", this.curt_cond_test,
"Isolation test inside the curt_en branch: build QuadCLT instances (borrowed calibration), then print per-sensor average spread (CuasMotion.perSensorAveragesFromTD) instead of normal RT detection. Well-calibrated -> the 16 sensor averages match (spread ~0).");
gd.addMessage("=== LoG prefilter ===");
gd.addNumericField("Optical PSF radius", this.curt_psf_radius, 6,8,"pix",
......@@ -5000,7 +5003,8 @@ min_str_neib_fpn 0.35
this.cuas_rng_limit = gd.getNextNumber();
this.curt_en = gd.getNextBoolean();
this.curt_cond_test = gd.getNextBoolean();
this.curt_psf_radius = gd.getNextNumber();
this.curt_n_sigma = gd.getNextNumber();
// rleak0 getNext removed 2026-06-20 (LReLU now LINEAR); predecessor code at git tag cuas-layer1. // By Claude on 06/20/2026
......@@ -6364,7 +6368,8 @@ min_str_neib_fpn 0.35
properties.setProperty(prefix+"cuas_rng_limit", this.cuas_rng_limit+""); // double
properties.setProperty(prefix+"curt_en", this.curt_en+""); // boolean
properties.setProperty(prefix+"curt_cond_test", this.curt_cond_test+""); // boolean
properties.setProperty(prefix+"curt_psf_radius", this.curt_psf_radius+""); // double
properties.setProperty(prefix+"curt_n_sigma", this.curt_n_sigma+""); // double
// rleak0 setProperty removed 2026-06-20 (LReLU now LINEAR); predecessor code at git tag cuas-layer1. // By Claude on 06/20/2026
......@@ -6762,7 +6767,8 @@ min_str_neib_fpn 0.35
private void getPropertiesCuasRT(String prefix,Properties properties){ // split out of getProperties() - 64KB method limit // By Claude on 06/12/2026
if (properties.getProperty(prefix+"curt_en")!=null) this.curt_en=Boolean.parseBoolean(properties.getProperty(prefix+"curt_en"));
if (properties.getProperty(prefix+"curt_cond_test")!=null) this.curt_cond_test=Boolean.parseBoolean(properties.getProperty(prefix+"curt_cond_test"));
if (properties.getProperty(prefix+"curt_psf_radius")!=null) this.curt_psf_radius=Double.parseDouble(properties.getProperty(prefix+"curt_psf_radius"));
if (properties.getProperty(prefix+"curt_n_sigma")!=null) this.curt_n_sigma=Double.parseDouble(properties.getProperty(prefix+"curt_n_sigma"));
// rleak0 getProperty removed 2026-06-20 (LReLU now LINEAR); predecessor code at git tag cuas-layer1. // By Claude on 06/20/2026
......@@ -9045,6 +9051,7 @@ min_str_neib_fpn 0.35
imp.cuas_rng_limit = this.cuas_rng_limit;
imp.curt_en = this.curt_en;
imp.curt_cond_test = this.curt_cond_test;
imp.curt_psf_radius = this.curt_psf_radius;
imp.curt_n_sigma = this.curt_n_sigma;
// rleak0 imp.X copy removed 2026-06-20 (LReLU now LINEAR); predecessor code at git tag cuas-layer1. // By Claude on 06/20/2026
......
......@@ -7271,20 +7271,34 @@ java.lang.NullPointerException
master_CLT, // QuadCLT center_CLT,
quadCLTs, // QuadCLT [] scenes,
debugLevel);
ImagePlus imp_targets = cuasRangingRT.prepareFpixels(); // GPU generator (explicit, CUDA-sensitive)
cuasRangingRT.saveUasFlightLogCsv(uasLogReader, imp_targets); // UAS flight-log truth -> <name>-UAS_DATA.tsv (mode-0 only; needs QuadCLT pose). By Claude on 06/24/2026
new CuasDetectRT(
clt_parameters, // CLTParameters clt_parameters,
uasLogReader, // UasLogReader uasLogReader,
imp_targets, // ImagePlus imp_targets (no GPU inside CuasDetectRT)
master_CLT.getX3dDirectory(), // String model_directory (outputs land like oracle)
master_CLT.getImageName(), // String core_base_name
master_CLT.correctionsParameters.cuasSynth, // String cuas_synth_dir (shared, list SET; "" -> model_directory)
master_CLT.correctionsParameters.cuasNoise, // String cuas_noise (inline per-level scales, list SET; "" -> sqrt default). By Claude on 06/24/2026
debugLevel).detectTargets(
clt_parameters, // CLTParameters clt_parameters,
batch_mode, // boolean batch_mode,
debugLevel); // int debugLevel
ImagePlus imp_targets = cuasRangingRT.prepareFpixels(); // GPU generator (explicit, CUDA-sensitive); also builds the QuadCLT instances = borrowed-calibration source
if (clt_parameters.imp.curt_cond_test) {
// Conditioning/calibration isolation test (curt_cond_test): reuse the QuadCLT instances just built as the
// borrowed-calibration source and print the per-sensor average spread. Baseline reads the current-scene TD
// as conditioned by the existing path; the next step will feed raw jp4 through CuasConditioning.condition()
// and compare. Well-calibrated -> the 16 per-sensor averages match (spread ~0). Runs instead of RT detection.
// By Claude on 07/01/2026
System.out.println("===== CUAS RT conditioning test (curt_cond_test): per-sensor average spread =====");
CuasMotion.perSensorAveragesFromTD(master_CLT.getGPUQuad(), false, "-CUAS-PERSENSOR", master_CLT); // conditioned TD, saved to the -CENTER instance
// Raw /jp4/ baseline (no photometric/FPN/conditioning), saved as -CUAS-PERSENSOR-RAW for side-by-side compare.
// RT-seed: Java reads the source tiffs and force-uploads them H2D; later the compute moves to GPU (Java = oracle).
// By Claude on 07/01/2026
CuasMotion.perSensorFromRawJp4(master_CLT, ImageDtt.THREADS_MAX, debugLevel);
} else {
cuasRangingRT.saveUasFlightLogCsv(uasLogReader, imp_targets); // UAS flight-log truth -> <name>-UAS_DATA.tsv (mode-0 only; needs QuadCLT pose). By Claude on 06/24/2026
new CuasDetectRT(
clt_parameters, // CLTParameters clt_parameters,
uasLogReader, // UasLogReader uasLogReader,
imp_targets, // ImagePlus imp_targets (no GPU inside CuasDetectRT)
master_CLT.getX3dDirectory(), // String model_directory (outputs land like oracle)
master_CLT.getImageName(), // String core_base_name
master_CLT.correctionsParameters.cuasSynth, // String cuas_synth_dir (shared, list SET; "" -> model_directory)
master_CLT.correctionsParameters.cuasNoise, // String cuas_noise (inline per-level scales, list SET; "" -> sqrt default). By Claude on 06/24/2026
debugLevel).detectTargets(
clt_parameters, // CLTParameters clt_parameters,
batch_mode, // boolean batch_mode,
debugLevel); // int debugLevel
}
}
if (generate_mapped || reuse_video) { // modifies combo_dsn_final ?
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment