Commit 72fd0677 authored by Andrey Filippov's avatar Andrey Filippov

CLAUDE: CuasRender MB compensation + conditionSceneToGpuCuas motion wrapper (ERS omegas/velocities)

Per Andrey: of the 3 expected render-vs-oracle mismatch causes, the legitimate
one is the better photometric calibration; the other two - ERS and motion
blur - must be implemented in the RT chain:

- ERS: NEW CuasConditioning.conditionSceneToGpuCuas(scene, cfg, omegas,
  velocities, ...) - the ONE place per-scene motion enters the ingest. Sets
  the scene ERS rates (fully-preserved ErsCorrection path); velocities =
  provision ({0,0,0}/null for the rotation-only camera). In production the
  omegas come analytically from wobble radius + RPM; the GPU corr_vector
  imu_rot/imu_move feed is the future upgrade (after the double-application
  check vs CPU pXpYD ERS). Legacy conditionSceneToGpu delegates (unchanged
  behavior for the pose loop).
- MB: renderSceneVirtual takes mb_vectors/mb_tau/mb_max_gain; when enabled
  uses the ORACLE machinery (setInterTasksMotionBlur two-task-set +
  interCorrTDMotionBlur double convert: positive original + negative shifted
  scale pair summing to 1, ratio from 1-pix-time/tau, offset stretched when
  scales would exceed mb_max_gain). No erased re-convert in MB mode (would
  destroy the accumulation) - comparisons must mask to task tiles.
- testRenderSequence: MB gated by imp.mb_en (matching the oracle DBG
  products: ON -> compare vs MB DBG, OFF -> vs -NOMB DBG); vectors from the
  same OpticalFlow.getMotionBlur at borrowed pose + stored rates; ingest via
  the new wrapper.

Verified: mvn -DskipTests clean package OK.
Co-Authored-By: 's avatarClaude Fable 5 <noreply@anthropic.com>
parent 29c7239b
......@@ -129,6 +129,46 @@ public class CuasConditioning {
final Config cfg,
final int threadsMax,
final int debugLevel) {
return conditionSceneToGpuCuas(scene, cfg, null, null, threadsMax, debugLevel);
}
/**
* CUAS wrapper: the ONE place where per-scene MOTION enters the RT ingest (Andrey's
* design 2026-07-05). Sets the scene's ERS rates (when given) before the ingest so the
* downstream grid transform (transformToScenePxPyD -> ErsCorrection, the fully-preserved
* RGB-proven path) applies the rolling-shutter correction; omegas==null &&
* velocities==null leaves the scene's current ERS state untouched (legacy behavior).
* For CUAS: omegas from the wobble radius + rotation RPM (analytic, per scene phase);
* velocities - PROVISION only, pass {0,0,0} (or null) for the rotation-only camera.
* FUTURE (GPU migration): the same omegas/velocities feed the GPU corr_vector
* imu_rot/imu_move (get_tiles_offsets has per-sensor ERS support, currently dormant) -
* to be enabled after verifying no double-application against the CPU pXpYD ERS.
* By Claude on 07/05/2026, from Andrey's design.
* @param omegas angular rates {az, tilt, roll} rad/s, scene-vs-virtual-center
* relative (wobble-scale; the constant-rotation part cancels against
* the co-rotating center) - or null (with velocities null: keep state)
* @param velocities linear rates {vx, vy, vz} m/s (provision; null/{0,0,0} for CUAS)
*/
public static boolean conditionSceneToGpuCuas(
final QuadCLT scene,
final Config cfg,
final double [] omegas,
final double [] velocities,
final int threadsMax,
final int debugLevel) {
if ((omegas != null) || (velocities != null)) {
scene.getErsCorrection().setErsDt(
(velocities != null) ? velocities : new double [3],
(omegas != null) ? omegas : new double [3]);
}
return conditionSceneToGpuInner(scene, cfg, threadsMax, debugLevel);
}
private static boolean conditionSceneToGpuInner(
final QuadCLT scene,
final Config cfg,
final int threadsMax,
final int debugLevel) {
final double [][][] data = CuasMotion.readRawImageData(scene, threadsMax, debugLevel);
if (data == null) {
return false;
......
......@@ -79,6 +79,15 @@ public class CuasRender {
* @param scene_xyz scene position (ZERO3 for CUAS rotation-only)
* @param scene_atr scene orientation (borrowed/fitted)
* @param margin tile margin
* @param mb_vectors null (no motion-blur compensation) or [2][ntiles] per-tile blur
* vectors (for CUAS uniform over the scene: dpx/dt, dpy/dt from
* the wobble radius + RPM). With MB the convert runs TWICE (the
* shifted, negative-scale second pass removes the bolometer
* exponential tail; scale pair sums to 1, ratio from
* (1-pix time)/mb_tau, offset grows if scales would exceed
* mb_max_gain - all inside setInterTasksMotionBlur).
* @param mb_tau bolometer time constant, s (imp.mb_tau, 0.008)
* @param mb_max_gain max scale before the offset is stretched (imp.mb_max_gain_inter)
* @param debugLevel debug level
* @return [17][width*height]: renders of the 16 sensors (grid-transformed to the
* virtual grid) + [16] = the consolidated (NaN-aware weighted average)
......@@ -93,15 +102,60 @@ public class CuasRender {
final double [] scene_xyz,
final double [] scene_atr,
final int margin,
final double [][] mb_vectors, // By Claude on 07/05/2026
final double mb_tau, // By Claude on 07/05/2026
final double mb_max_gain, // By Claude on 07/05/2026
final int debugLevel) {
final GpuQuad gpuQuad = center_CLT.getGPUQuad();
// 1. virtual uniform grid -> scene grid at the pose (ERS per the instances' rates)
final double [][] scene_pXpYD = OpticalFlow.transformToScenePxPyD(
null, center_disparity, scene_xyz, scene_atr, scene, center_CLT);
final double disparity_corr = clt_parameters.imp.disparity_corr + center_CLT.getDispInfinityRef();
final int img_width = scene.getGeometryCorrection().getSensorWH()[0];
if (mb_vectors != null) {
// Motion-blur compensation: TWO task sets (original positive scale + shifted
// negative scale), double convert_direct - the oracle machinery
// (setInterTasksMotionBlur / interCorrTDMotionBlur). NOTE: no erased re-convert
// here (it would destroy the two-pass accumulation) - tiles outside the task
// set keep the previous scene's data; comparisons must mask to task tiles.
final TpTask [][] tp_tasks2 = GpuQuad.setInterTasksMotionBlur(
scene.getNumSensors(),
img_width,
false, // GPU calculates port coordinates
scene_pXpYD,
null, // selection: ALL tiles with valid disparity
mb_tau,
mb_max_gain,
mb_vectors,
scene.getGeometryCorrection(),
disparity_corr,
margin,
null, // valid_tiles
ImageDtt.THREADS_MAX);
if ((tp_tasks2 == null) || (tp_tasks2[0] == null) || (tp_tasks2[0].length == 0)) {
System.out.println("renderSceneVirtual(): no MB tasks for scene "+scene.getImageName());
return null;
}
image_dtt.interCorrTDMotionBlur(
clt_parameters.img_dtt,
tp_tasks2,
null,
clt_parameters.gpu_sigma_r,
clt_parameters.gpu_sigma_b,
clt_parameters.gpu_sigma_g,
clt_parameters.gpu_sigma_m,
scene.isMonochrome() ? 1.0 : clt_parameters.gpu_sigma_rb_corr,
clt_parameters.getGpuCorrSigma(scene.isMonochrome()),
clt_parameters.getGpuCorrLoGSigma(scene.isMonochrome()),
clt_parameters.corr_red,
clt_parameters.corr_blue,
0, // sensor_mask_inter = 0: convert only
ImageDtt.THREADS_MAX,
debugLevel);
} else {
final TpTask [] tp_tasks = GpuQuad.setInterTasks(
scene.getNumSensors(),
scene.getGeometryCorrection().getSensorWH()[0],
img_width,
false, // GPU calculates port coordinates
scene_pXpYD, // per-tile pX,pY,disparity
null, // selection: ALL tiles with valid disparity
......@@ -134,6 +188,7 @@ public class CuasRender {
// re-convert with erase_clt=1: NaN outside task tiles (interCorrTD converts with
// erase=-1, leaving the previous scene's tiles as ghosts in a render)
gpuQuad.execConvertDirect(false, null, 1);
}
// 3. per-sensor renders (virtual grid)
final float [][] per_sensor = CuasMotion.perSensorImagesFromTD(gpuQuad, false);
// 4. merged = the NaN-aware weighted average consolidated in TD, rendered via slot 0
......@@ -185,6 +240,18 @@ public class CuasRender {
final int img_height = center_CLT.getGPUQuad().getImageHeight();
final int num_sens = center_CLT.getNumSensors();
final int num_comps = num_sens + 1; // 16 sensors + merged
// Motion-blur compensation, gated by the same imp.mb_en as the oracle DBG renders
// (mb_en OFF -> compare against the -NOMB oracle products). Vectors from the same
// getMotionBlur the oracle uses, at the borrowed pose + stored rates. The identity
// center grid is needed for it. By Claude on 07/05/2026.
final boolean mb_en = clt_parameters.imp.mb_en;
final double mb_tau = clt_parameters.imp.mb_tau;
final double mb_max_gain = clt_parameters.imp.mb_max_gain_inter;
final double [][] pXpYD_center = mb_en ? OpticalFlow.transformToScenePxPyD(
null, center_disparity, ZERO3, ZERO3, center_CLT, center_CLT) : null;
System.out.println("testRenderSequence(): motion-blur compensation "+
(mb_en ? ("ON (tau="+mb_tau+" s, max gain "+mb_max_gain+") - compare vs the MB oracle DBG") :
"OFF - compare vs the -NOMB oracle DBG"));
final ArrayList<float [][]> rendered = new ArrayList<float [][]>();
final ArrayList<String> ts_names = new ArrayList<String>();
final float [] nan_slice = new float [img_width * img_height];
......@@ -202,22 +269,37 @@ public class CuasRender {
if (pose == null) {
System.out.println("testRenderSequence(): scene "+nscene+" ("+ts_name+") has no stored pose - skipping");
} else {
if ((ers_xyz_dt != null) && (ers_atr_dt != null)) {
quadCLTs[nscene].getErsCorrection().setErsDt(ers_xyz_dt, ers_atr_dt);
} else {
quadCLTs[nscene].getErsCorrection().setErsDt(ZERO3.clone(), ZERO3.clone());
if ((ers_xyz_dt == null) || (ers_atr_dt == null)) {
System.out.println("testRenderSequence(): scene "+nscene+" ("+ts_name+") has no stored ERS rates - zeros");
}
// THE STEP UNDER TEST: raw /jp4/ -> conditioning -> GPU
final boolean raw_ok = CuasConditioning.conditionSceneToGpu(
// THE STEP UNDER TEST: raw /jp4/ -> conditioning -> GPU, motion (omegas +
// velocities provision) entering through the ONE wrapper. Here the rates are
// borrowed (stored); in production they come analytically from pose + rpm.
final boolean raw_ok = CuasConditioning.conditionSceneToGpuCuas(
quadCLTs[nscene],
null, // Config (defaults: rowcol + photometric + FPN)
(ers_atr_dt != null) ? ers_atr_dt : ZERO3.clone(), // omegas
(ers_xyz_dt != null) ? ers_xyz_dt : ZERO3.clone(), // velocities (provision)
ImageDtt.THREADS_MAX,
debugLevel - 2);
if (raw_ok) {
double [][] mb_vectors = null;
if (mb_en) { // same oracle machinery + same (borrowed) rates as the DBG renders
mb_vectors = OpticalFlow.getMotionBlur(
center_CLT, // QuadCLT ref_scene,
quadCLTs[nscene], // QuadCLT scene,
pXpYD_center, // double [][] ref_pXpYD,
pose[0], // double [] camera_xyz,
pose[1], // double [] camera_atr,
(ers_xyz_dt != null) ? ers_xyz_dt : ZERO3.clone(), // camera_xyz_dt,
(ers_atr_dt != null) ? ers_atr_dt : ZERO3.clone(), // camera_atr_dt,
0, // int shrink_gaps,
debugLevel - 2); // int debug_level
}
rslt = renderSceneVirtual(
clt_parameters, image_dtt, center_CLT, quadCLTs[nscene],
center_disparity, pose[0], pose[1], margin, debugLevel - 2);
center_disparity, pose[0], pose[1], margin,
mb_vectors, mb_tau, mb_max_gain, debugLevel - 2);
} else {
System.out.println("testRenderSequence(): scene "+nscene+" ("+ts_name+") raw-jp4 ingest FAILED");
}
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment