- 05 Jul, 2026 9 commits
-
-
Andrey Filippov authoredPer Andrey: of the 3 expected render-vs-oracle mismatch causes, the legitimate one is the better photometric calibration; the other two - ERS and motion blur - must be implemented in the RT chain: - ERS: NEW CuasConditioning.conditionSceneToGpuCuas(scene, cfg, omegas, velocities, ...) - the ONE place per-scene motion enters the ingest. Sets the scene ERS rates (fully-preserved ErsCorrection path); velocities = provision ({0,0,0}/null for the rotation-only camera). In production the omegas come analytically from wobble radius + RPM; the GPU corr_vector imu_rot/imu_move feed is the future upgrade (after the double-application check vs CPU pXpYD ERS). Legacy conditionSceneToGpu delegates (unchanged behavior for the pose loop). - MB: renderSceneVirtual takes mb_vectors/mb_tau/mb_max_gain; when enabled uses the ORACLE machinery (setInterTasksMotionBlur two-task-set + interCorrTDMotionBlur double convert: positive original + negative shifted scale pair summing to 1, ratio from 1-pix-time/tau, offset stretched when scales would exceed mb_max_gain). No erased re-convert in MB mode (would destroy the accumulation) - comparisons must mask to task tiles. - testRenderSequence: MB gated by imp.mb_en (matching the oracle DBG products: ON -> compare vs MB DBG, OFF -> vs -NOMB DBG); vectors from the same OpticalFlow.getMotionBlur at borrowed pose + stored rates; ingest via the new wrapper. Verified: mvn -DskipTests clean package OK. Co-Authored-By:Claude Fable 5 <noreply@anthropic.com>
-
Andrey Filippov authored
Per Andrey (2026-07-05): today's problems came from jumping over the standalone certification of conditionSceneToGpu (single caller inside the pose test, never image-tested by itself). Test and debug the ingest+render chain FIRST, only then return to the pose test. NEW class cuas/rt/CuasRender (the start of the dedicated renderer per the spec in imagej-elphel-internal handoffs/2026-07-05_cuas_rt_dataflow_and_grids.md par.6: virtual uniform grid always, raw center DSI disparity, jp4+CuasConditioning pixels, QuadCLT borrow-only, static methods): - renderSceneVirtual(): tasks at pose (ALL valid-disparity tiles) -> convert (erase=1, NaN outside tasks) -> 16 per-sensor renders + consolidated (NaN-aware weighted average) merged render. No correlation. - testRenderSequence(): per scene raw /jp4/ -> conditionSceneToGpu -> render at BORROWED stored pose + stored ERS rates (the exact sources the oracle DBG renders use) -> saves <center>-CUAS-RT-RENDER.tiff hyperstack [t: s00..s15, merged][z: scenes], slice-by-slice comparable to -CUAS-INDIVIDUAL-CUAS-DBG / -CUAS-MERGED-CUAS-DBG. New checkbox "CUAS RT render test (ingest+render)" (curt.rend_test), takes precedence over the pose test in the curt_en branch. Verified: mvn -DskipTests clean package OK. Co-Authored-By: -
Andrey Filippov authored
LEAN-07 (stored poses, all tiles, sum-scaled input): sqrt_l0 3.22 -> 5.58 (wider, not the predicted ~2.1), corr RMS 0.82 -> 1.005. At 16x signal-to- fat-zero the normalize amplifies the blur-decorrelated high frequencies - the fat zero was regularizing usefully. Root of the misdiagnosis: the width comparison used the MB-COMPENSATED oracles (FILT150A 2.08, MBEN 2.75); the correct no-MB control (NOMB oracle) has sqrt_l0 3.58 ~ lean's 3.22 - lean peak shapes were normal all along, and Andrey's "there was no 16x difference" was right where it mattered. Consequences: (1) correlator input back to the true weighted average (the same data as the -POSE-RT-COMPOSITE debug render - single upload again); (2) the "fz-broadened peak content pull" roll-bias mechanism is falsified - the +0.52/+0.33 mrad roll bias needs a new suspect (remaining engine differences vs the NOMB oracle: neighbor consolidation min_str_neib, min_confidence gates, peak-extraction parameters); (3) MB compensation in lean (v2) is the main quality lever - MBEN sharpens peaks 3.6 -> 2.8 and removes the 0.5 px wobble. Verified: mvn -DskipTests clean package OK. Co-Authored-By: -
Andrey Filippov authored
Per Andrey's input-verification methodology: the -POSE-RT-COMPOSITE "real" component must be the weighted 16-sensor average (comparable to the virtual render), while the correlator receives the per-tile SUM (379b3cf6 fix). Render moved BEFORE the sum scaling (extra H2D + imclt in debug mode only); the post-correlation render (which would now show the count-stepped sum) removed. Verified: mvn -DskipTests clean package OK. Co-Authored-By:
-
Andrey Filippov authored
CLAUDE: lean pose FIX: consolidated TD scaled to per-tile sensor SUM (oracle fat-zero operating point) The lean conj-multiply fed the plain 16-sensor AVERAGE into the same FZ-normalize (same absolute fat zero; JNA drops per-tile weights) that the oracle feeds with the unweighted SUM of pair products - 1/16 the amplitude, so the effective fat zero was 16x larger. Invisible in output scale (the normalize hides it) but measured in peak shape: sqrt_l0 3.22 vs oracle 2.08 and sqrt_l1 5.83 vs 4.15 on identical tiles, corr RMS 0.46 vs 0.33 - and suspected as the roll-bias mechanism (asymmetric-content centroid pull on broadened peaks; +0.52 mrad @ 150 edge-weighted tiles, +0.33 @ all tiles). Fix: multiply each consolidated tile by its per-sensor count (SUM semantics, matching the oracle exactly incl. partial tiles). consolidateSensorsTD itself unchanged (A2 validation still uses the true average). Expected in LEAN-05 (normal LMA run): sqrt_l0 ~2.1 in the HYPER eigen slices, corr RMS toward ~0.33, roll bias collapse if the mechanism is confirmed (survival = clean falsification). Verified: mvn -DskipTests clean package OK. Co-Authored-By: -
Andrey Filippov authored
Andrey's diagnosis: the composite-vs-DBG sub-pixel warp field (static, sy- dominated: uniform 0.24px, x-shear, +0.3% y-scale ~ tilt_rate*frame_time) comes from the ErsCorrection infrastructure - the render path sets each scene's ERS from the STORED scenes_<ts>_dt (xyz_dt AND atr_dt, renderSceneSequence/OpticalFlow:10555) while the pose loop used finite- difference ATR-only rates with xyz_dt=0. Different scene-side ERS by construction. (Virtual center's own ERS rates confirmed ZERO in corr-xml; GPU corr_vector imu_rot/imu_move ERS support confirmed present but dormant.) pose_stored mode now sets setErsDt from ers_center.getSceneErsXYZ_dt/ getSceneErsATR_dt - the exact calls the render uses - so the decoupled measurement/render applies IDENTICAL poses AND ERS as CUAS-MERGED-CUAS-DBG. If the warp field collapses in the next pose_stored run, the divergence is proven. Live (LMA) mode unchanged: finite-difference rates - to be replaced by rates computed from pose + rotation model (RT design: ErsCorrection preserved fully, thin non-QuadCLT plumbing; per-sensor ERS available on GPU via corr_vector imu when fed). Verified: mvn -DskipTests clean package OK. Co-Authored-By: -
Andrey Filippov authored
Per Andrey: static per-tile dx offsets found in -POSE-RT-HYPER (e.g. tile x=15,y=41: dx mean +0.92 constant over all 497 scenes, far tile disp 0.02; tile x=71,y=51: dx +0.86 dy +1.91, near tile disp 13.2 - neighbors with similar disparity are near zero) -> suspect broken per-tile task data. New checkbox "Pose test stored poses (no LMA)" (curt.pose_stored): every scene is measured/rendered at its STORED (oracle-vintage) pose, the LMA is skipped entirely. -POSE-RT-HYPER/-CORR2D/-COMPOSITE come from that single measurement; CSV rms column = weighted RMS of the measured offsets. The -POSE-RT-COMPOSITE "real" component must then match the oracle CUAS-MERGED-CUAS-DBG per-scene renders tile-for-tile (and "virtual" its [average] slice) - any mismatch localizes the broken tiles in the task/render/correlation chain, independent of the solver. Observability only - the fitting paths are unchanged. Verified: mvn -DskipTests clean package OK. Co-Authored-By: -
Andrey Filippov authored
Per Andrey (no blind fixes - see what is going on): save the composite (TD-averaged, grid-transformed) scenes actually correlated against the virtual center, WITH the virtual render for comparison: - curt.pose_img_save (checkbox "Pose test save composite scenes"): -POSE-RT-COMPOSITE.tiff hyperstack [t: real, virtual][z: scene] - the virtual-center render repeated for every scene so the component slider blink-compares at any z. z aligned with -POSE-RT-HYPER/-CORR2D. - leanMeasure img_out holder: when requested, re-runs execConvertDirect with erase_clt=1 (NaN outside task tiles - the standard interCorrTD erase=-1 leaves the previous scene's tiles as ghosts in a render; correlation itself unaffected), then imclt renders sensor slot 0 = the consolidated average (last LMA cycle, converged pose). - Virtual component = imclt of the reference buffer (what the conj-multiply actually sees), not a re-render from prepared data. Observability only - processing unchanged, roll-bias symptom preserved. Verified: mvn -DskipTests clean package OK. Co-Authored-By: -
Andrey Filippov authored
Per Andrey's debug approach (same as used on the oracle): render the actual scene-vs-virtual-center correlations in the pixel domain and save them as a scene sequence. Observability only - NO processing changes, the roll-bias symptom is preserved. - curt.pose_corr_save (checkbox "Pose test save 2D correlations"): save -POSE-RT-CORR2D.tiff - z=scenes (aligned with -POSE-RT-HYPER incl. NaN slices for failed/coasted scenes), tile grid of 16x16-pix cells, last LMA cycle per scene, ImageDtt.corr_partial_dbg convention as CuasMotion CORR2D. Lean engine only (the oracle does not expose its correlation tiles). - curt.pose_full (checkbox "Pose test ALL tiles (ignore calibration)"): temporarily drop the 150-tile filter and use all strength-selected (~1074) tiles; -POSE-RT-TILE-CALIB is neither read NOR written so a debug run never pollutes the persistent tile calibration. - leanMeasure/leanFitScene: optional corr_pd_out holder (last-cycle PD tiles). Context: lean run v013-LEAN-01 confirmed the v*tau signature on az/tilt (implied tau = 8.4/8.0 ms = mb_tau) but showed a constant +0.52 mrad roll bias; peaks ~50% wider than oracle on common tiles (suspect: consolidation averages /16 while oracle sums pairs -> effective fat-zero 16x larger). These debug outputs are for inspecting exactly that before changing the processing. Verified: mvn -DskipTests clean package OK. Co-Authored-By:
-
- 04 Jul, 2026 16 commits
-
-
Andrey Filippov authored
Per Andrey: insert an XML comment right after <properties> in every modified file with the absolute path of the script that did the migration + timestamp (absolute path intentional; adjust per machine). Java loadFromXML ignores XML comments (verified with a real load: 4153 entries) but drops them on re-save. Applied to the live config LV396-v013-...-POSEJP4-ENMB.corr-xml: 87 keys renamed, .bak kept, Java load verified. Co-Authored-By: -
Andrey Filippov authored
Per Andrey: CuasRtParameters should be instantiated like the other parameter classes at CLTParameters:1175..1185 (img_dtt/ofp/imp/ilp/...), not nested inside IntersceneMatchParameters. Changes: - CLTParameters: new peer field curt, set/getProperties wiring, own "CUAS RT" dialog tab right after imp's questions/answers (tab content unchanged; tab position moves from inside imp's tab row to after it). - IntersceneMatchParameters: the six curt wiring sites removed (field, dialogQ/A, set/getProperties, clone). - Access rename: clt_parameters.imp.curt.X -> clt_parameters.curt.X (105 sites: OpticalFlow, CuasDetectRT, CuasPoseRT). - corr-xml keys change _imp_curt_* -> _curt_*. getProperties keeps a legacy fallback (reads _imp_curt_* first, _curt_* overrides), so old configs still load unmigrated. - NEW scripts/migrate_curt_config.py: renames _imp_curt_* keys in existing corr-xml files (in-place with .bak, --dry-run, idempotent, drops duplicate legacy entries with a warning). Validated on a copy of LV396-v013-...-POSEJP4-ENMB.corr-xml: 87 keys renamed, byte-identical otherwise. Verified: mvn -DskipTests clean package OK; no imp.curt references remain. Co-Authored-By: -
Andrey Filippov authored
The reader was at 63291/65535 (bloated by ~1103 non-CUAS getProperty reads, not curt). Split its second half into a private getProperties2(prefix, properties) continuation (same pattern as the earlier getPropertiesCuasRT split): getProperties 63291->33327, getProperties2 29970 - both comfortably under the limit. Behavior identical (order preserved, same args). All IMP methods now safely under 64K (largest = setProperties 52127). mvn compile clean. Co-Authored-By: -
Andrey Filippov authored
Per Andrey: move every curt_* parameter out of IntersceneMatchParameters into its own class (like CLTParameters.imp/ofp/... peers), held as IntersceneMatchParameters.curt = new CuasRtParameters(). All 61 params: field decls, dialogQuestions/Answers, set/getProperties, clone. Access changes imp.curt_X -> imp.curt.X (114 external sites across OpticalFlow/CuasDetectRT/ CuasPoseRT); short field names (curt_ prefix dropped, class name conveys it). corr-xml COMPATIBLE (no conversion needed): delegated as curt.set/getProperties(prefix+"curt_", props) with short keys, so full keys stay ..._imp_curt_X byte-identical (verified: 61 old == 61 new). Verified: compiles clean; 61 params in every section; dialogQ add* == dialogA getNext* == 61, verbatim order (pairing preserved); interleaved non-curt air_/fgnd_ decls intact. Relieves setProperties 54865->52127, dialogQuestions 22385->21267, clone -503B (the methods our pose work grows). NOTE: getProperties READER stays 63291/65535 - its bloat is ~1103 NON-curt getProperty reads (curt reads were already in getPropertiesCuasRT since 06/12); relieving it needs a separate non-curt extraction (flagged for Andrey). Co-Authored-By: -
Andrey Filippov authored
TD-average the 16 sensors BEFORE correlation (multiply averages, not average products), single conj-multiply vs the persistent virtual-center TD. Zero new CUDA - assembled from proven pieces: - CuasPoseRT.leanMeasure(): interCorrTD(sensor_mask=0) = tasks(pose)+offsets+ convert_direct only -> getCltData/CuasTD.consolidateSensorsTD/setCltData (the validated CPU bridge, future clt_average_sensors kernel) -> setSensorMaskInter(1)+execCorr2D_inter_TD (single conj-multiply) -> TDCorrTile.getFromGpu + convertTDtoPD (JNA-validated CuasMotion path, FZ-normalize + PD) -> Correlation2d.getMaxXYCmEig (peak+eigen, the GPU argmax kernel oracle). Correlation stages are geometry-blind - projection/ distortion is baked into the average-camera tasks (per Andrey). - CuasPoseRT.leanFitScene(): same IntersceneLma solver + exit rules as the oracle engine; fills lma_rms/coord_motion_rslt so CSV/-POSE-RT-HYPER are unchanged (A2-03 = direct oracle). - curt_pose_lean checkbox 'Pose test lean correlation (B)'. v1 differences from oracle (documented): NO motion-blur compensation (compare vs NOMB baseline: 0.287/0.282/0.106 mrad), no FPN peak masking (input is FPN-subtracted by A2 conditioning), no moving-object filter, min_confidence=0. mvn compile clean. Co-Authored-By: -
Andrey Filippov authored
Per Andrey: the name must make it obvious this is a CALIBRATION file that has to be preserved with the model (reliable_tiles are derived from it). New saves use -POSE-RT-TILE-CALIB; the old -POSE-RT-MAXDXY is still read as a fallback, so existing model dirs keep working. Tooltips/comments updated. Co-Authored-By: -
Andrey Filippov authored
Per Andrey: bypass the QuadCLT image_data mechanics entirely and feed the GPU directly - the permanent RT shape (more data becomes GPU-memory-resident; QuadCLT stays for geometry/poses only). - CuasMotion.readRawImageData(): raw /jp4/ reader extracted from perSensorFromRawJp4 (oracle getJp4Tiff, one thread/sensor, instrumentation); perSensorFromRawJp4 rewired, behavior unchanged. - CuasConditioning.conditionSceneToGpu(): raw read -> condition() with the CURRENT calibration (curt_calib-updated lwir scales/offsets/scales2 + per-pixel FPN from the scene QuadCLT) -> bind scene (saveQuadClt, conditional) -> clear hasNewImageData -> setBayerImages(data,true) force-H2D. The bayer guard (ebef0b23 fix) keeps the upload alive through interCorrPair's own setBayerImages(false). TELL if the guard ever breaks: results become identical to the prepared-data path. - CuasPoseRT: curt_pose_raw flag (new checkbox 'Pose test raw-jp4 ingest (A2)') runs the ingest per scene before the fit; ingest failure coasts the prediction and records an empty CSV/hyper row (fail=-1). Acceptance (recorded): A2 legitimately diverges from phase A (old Photogrammetric Calibration was broken, frozen from unrelated footage) - judge by A2's OWN dstored/corr-RMS/convergence; systematically worse = red flag. mvn compile clean. Co-Authored-By:
-
Andrey Filippov authored
Phase A2/B building block: consolidate the 16 per-sensor CLT channels into ONE averaged TD channel (average images BEFORE correlation - multiply averages, not average products). Per-tile granularity: sum sensors that have the tile (first element NaN = absent), count, divide; count plane returned as the weight; a stray in-tile NaN poisons the whole result tile (fail-visible). Not available on GPU (combine_inter only sums correlation PRODUCTS) - this CPU implementation + get/setCltData D2H/H2D is the A2 bridge and the bit oracle for the future clt_average_sensors kernel. - CuasTD.validateConsolidation(): linearity oracle - imclt(TD-avg) must equal pixel-average of per-sensor imclt renders (same GPU imclt both sides); prints count-plane stats + max|diff|/RMS, saves -CUAS-TDAVG-CHECK 3-slice stack, restores original TD. Wired into the curt_cond_test branch after perSensorFromRawJp4 (uses its raw-jp4 16-sensor TD). - GpuQuadJna.getCltData() override added (base derefs null gpu_clt_h on JNA shells - the known un-overridden-accessor class); uses tp_proc_get_clt. mvn compile clean. Co-Authored-By: -
Andrey Filippov authored
Findings from the FILT150 run: (1) scattered rank-150 selection starved the per-scene neighbor consolidation (min_str_neib/eig_str_neib) - only ~57 tiles with accidental neighbors were measured per scene (neighbors-vs-measured corr 0.78); (2) roll degraded (RMS 0.106 vs 0.059 mrad, bias +0.073) - the selection carried only 11% of the full set's roll information. deriveSelection() stage 2 now picks disjoint 3x3 CLUSTERS of gate-passing tiles (>=CLUSTER_MIN_ELIGIBLE=6 of 9), round-robin from three pools: LEFTMOST, RIGHTMOST (per Andrey - edge tiles have the most roll influence), BEST-QUALITY (median member fmax), until the tile budget is filled; scattered best tiles fill any remainder. Offline simulation on the real calibration: 24 clusters (8/8/8), 150 tiles, mean 3.97 in-selection neighbors, roll info +48% vs scattered rank-150. Measurement code untouched (oracle identical). mvn compile clean. Co-Authored-By: -
Andrey Filippov authored
Per Andrey: (1) calibration reuse is now automatic - if -POSE-RT-MAXDXY exists it is used (filtered run), else a full run generates it; new curt_pose_recalc flag forces regeneration (replaces the backwards curt_pose_use_filt enable). Matches the FPN reuse pattern. (2) MAXDXY stores NaN instead of +inf for NaN-in-any-scene tiles - deriveSelection rejects NaN and +inf identically (non-finite), and NaN keeps the TIFF viewable in ImageJ (+inf broke min/max autoscaling). mvn compile clean (Eyesis closed). Co-Authored-By: -
Andrey Filippov authored
Replaces the absolute curt_pose_max_dxy with the scale-free scheme (Andrey's histogram rule formalized + robustness for worse footage): - Calibration artifact -POSE-RT-MAXDXY.tiff: per-tile max-over-scenes residual, +inf where any scene NaN (auto-reject, mergeable across runs by max), NaN where unmeasured. Saved only from FULL-selection runs (a filtered run never shrinks coverage). Continuous statistic persisted, boolean selection derived at load - policy can change without re-measuring. - deriveSelection(): stage 1 outlier gate keep max <= median + k*NMAD of finite per-tile maxes (curt_pose_dxy_k=0.75; on the reference footage: MBEN gate 0.477 keeps 595, degraded NOMB self-adapts to 0.728 keeps 626 - same ~65%); stage 2 rank-N budget keep curt_pose_num_tiles=150 best (threshold-free). - curt_pose_use_filt now loads MAXDXY and derives; missing -> full run generates it (FPN-style reuse pattern). Importance-greedy (3x3 information matrix) ranking = next step. mvn compile clean. Co-Authored-By: -
Andrey Filippov authored
Per Andrey: a selected tile is BAD if its measured dxy is NaN in any scene or exceeds curt_pose_max_dxy (absolute, default 0.25 pix) in at least one scene. Survivors saved as -POSE-RT-RELIABLE-FILT.tiff; curt_pose_use_filt loads it on a next run and ANDs with the strength selection (two-pass workflow: full run calibrates the selection, subsequent runs use ~191 clean tiles instead of 1074 on the reference footage; kept-tile mean dxy 0.087 pix). Co-Authored-By: -
Andrey Filippov authored
Exercises the existing MB machinery in the RT iterator: when imp.mb_en is ON, per-scene blur vectors from OpticalFlow.getMotionBlur (FD-based rates) send interCorrPair down the setInterTasksMotionBlur/interCorrTDMotionBlur path - convert_direct runs twice, the second run subtracting the shifted+scaled copy via negative TpTask.scale (LWIR bolometer exponential-tail removal). mb_en OFF keeps the single-run path, giving a one-checkbox A/B. Same getMotionBlur usage as offline setInitialOrientationsCuas (stored truth was produced WITH MB on). mvn compile clean (Eyesis closed). Co-Authored-By: -
Andrey Filippov authored
Per Andrey: (1) fitted-vs-stored deltas reported in PIXELS (the informative unit), using the same scales as the LMA par_scales - az/tilt = focal/pixelSize, roll = distortionRadius/pixelSize; mrad kept secondary. (2) A 'CuasPoseRT scene i (of N) <timestamp> Done/FAILED' line after each fit so the unlabeled LMA iteration prints above it are attributable to a scene (SYSTEM_OUT-01.log had iterations but no index/timestamp). Per-scene line also shows dstored in pix. Verified with standalone javac (Eyesis live - no mvn). Co-Authored-By: -
Andrey Filippov authored
Per Andrey: characterize the per-scene measurement, incl. the eigenvector data. -POSE-RT-HYPER (80x64, z=scenes, t=components, make_hyper layout): dx, dy, strength, dxy=|dx,dy| from vector_XYS; sqrt_l0, sqrt_l1 (peak-ellipse half-axes, pix), elong=sqrt(l1/l0) (linear-feature indicator), eig0_ang (precise-axis direction, [0,PI)) from coord_motion eigen {eig_x,eig_y,l0,l1} - NaN unless imp.eig_use. Data = last LMA cycle's coord_motion via the existing coord_motion_rslt out-param. -POSE-RT-RELIABLE = tile selection mask. Verified with standalone javac against target/classes (Eyesis live - no mvn; Eclipse rebuilds on restart). Co-Authored-By: -
Andrey Filippov authored
Top-level scene iterator re-generating per-scene 3-angle poses against the persistent virtual-center reference, RT-style: ascending time order, zero-order prediction seeding (fit anchored to the center, prediction only warm-starts the LMA), single pass on the final combo DSI (no refinement pass - it only existed because disparity arrived after initial orientations offline). Measurement engine = proven Interscene.adjustPairsLMAInterscene (reference GPU data set once); phase B will swap it for the lean TD-average x virtual-center path with GPU argmax+eigen kernels, keeping this iterator + CSV as the oracle. - new cuas/rt/CuasPoseRT.testPoseSequence(): reference prep (strength> curt_pose_str tile selection, setReferenceGPU with center CLT), stored-pose seed/truth from center ErsCorrection scenes_poses, per-scene fit with 3-angle param_select (XYZ locked), ERS dt from pose finite differences (disable_ers), MB off, coast-on-failure; writes -POSE-RT-TEST.csv + fitted-vs-stored summary - params curt_pose_test (bool) + curt_pose_str (1.0) - 6 plumbing sites - OpticalFlow curt_en branch: curt_pose_test runs INSTEAD of detection Build: mvn compile clean. Runtime validation pending (Eclipse/Eyesis run on sequence 1773135476_186641, truth = re-adjusted stored poses). Co-Authored-By:
-
- 03 Jul, 2026 5 commits
-
-
Andrey Filippov authored
CLAUDE: self-documenting comments: TpTask bridge role, differential rectification, offset composition Comment-only (no code change; mvn compile clean). Documents, from Andrey's explanation: TpTask as the Java<->CUDA work-list bridge; the per-sensor xy offset as the differential-rectification composed shift (factory kernel offset + misalignment + disparity + relative pose) split integer/fractional; historic host-side vs current GPU-side geometry fill; updateTasks() D2H; disp_dist[cam][4] = d(x,y)/d(disp,ndisp) Jacobian consumed by Corr2dLMA and lazy-eye/ERS. Co-Authored-By: -
Andrey Filippov authored
New curt_calib parameter (CUAS RT dialog, saved/restored) runs/bypasses the per-sensor photometric (re)calibration as the first step of the CUAS RT processing flow, before detection (no longer tied to the diagnostic). Extracted CuasMotion.rtPhotometricCalibration() = convertFromData() (upload + own uniform grid convert, split out of perSensorFromData) + fit + apply/ save. Production step converts and calibrates without saving stacks; the curt_cond_test diagnostic (replaces detection) keeps the raw-vs-conditioned stack compare and makes the calibration step save -CUAS-PERSENSOR[-ADJ]. Co-Authored-By: -
Andrey Filippov authored
The virtual -CENTER INTERFRAME corr-xml only carries poses/velocities, so saving the recalculated 16+16 lwir offsets/scales there was futile. Follow the established photometric machinery (runPhotometric()/photoEach()) and the top-menu save/restore convention instead: set the new values on master_CLT (immediate use), quadCLTs[ref_index] (physical photometric owner, its <scene>-INTERFRAME.corr-xml is saved) and quadCLT_main (applied to next sequences and saved in the main configuration file). Co-Authored-By: -
Andrey Filippov authored
curt_cond_test rework: both PERSENSOR stacks now converted with the test's own uniform sensor-domain task grid (scale 1.0) instead of leftover GPU state (MB secondary tasks with negative fractional scales made 'raw' renders = -1/6 x input; leftover virtual-view grid lost the same border ROI on every sensor). perSensorFromRawJp4 no longer overwrites the scene's conditioned image_data. GpuQuadJna.setBayerImages(force,center) restored the base-class skip-guard via a native-side jna_bayer_set flag (gpuTileProcessor is null in JNA shell instances): every execConvertDirect unconditionally re-pulled quadCLT.getResetImageData(), silently clobbering explicit uploads - made the raw baseline bit-identical to the conditioned render. CuasMotion.perSensorLinearFit(): per-sensor a+b*x photometric fit over safe tiles (weak strength<0.5 or far disparity<1 from -INTER-INTRA-LMA, inner rect, 8x8 tile->pixel map) against the cross-sensor mean, gauge keep_averages (mean offset 0, mean scale 1), 3-sigma outlier rejection. Validated on 1773135476_186641: sensor-mean spread 1353->5 counts, cross-sensor RMS 358->17 (inliers), b in 0.83..1.11. CuasMotion.applyLwirLinearCalibration(): folds the fit into the 16+16 lwir offsets/scales (scale'=b*scale, offset'=offset-a/scale'), updates the center instance + photometric_scene provenance, saves -INTERFRAME.corr-xml. Applied the standard way at load they compensate the remaining per-sensor mismatch of the raw /jp4/ tiffs. Co-Authored-By: -
Andrey Filippov authored
Add make_hyper parameter (code-selected, not in settings): 0 - flat stack (old behavior, all pre-existing call sites); >0 - transpose to hyperstack [sensors][avgs+timestamps][pixel], z (top slider) - timestamps, t (bottom slider) - sensor channels; 2 - insert per-timestamp average of all used sensors as first channel (17th), computed from final conditioned slices. Per-sensor full + center-fraction averages now work in individual mode (pre-calculated merged-only average falls back to slice computation with a warning instead of AIOOBE). Number of average frames stays variable (0/1/2); fopen paths bit-identical by design. Verified on 495-scene CUAS sequence: INDIVIDUAL debug hyperstack matches the MERGED convention - to be used as oracle for RT conditioning. Co-Authored-By:
-
- 02 Jul, 2026 2 commits
-
-
Andrey Filippov authored
-
Andrey Filippov authored
Add a curt_cond_test path (boolean at the top of the CUAS-RT dialog) that, inside the curt_en branch, renders the 16 per-sensor images and saves them to the -CENTER instance for calibration inspection: - CuasMotion.perSensorAveragesFromTD: imclt the per-sensor TD, print the 16-sensor average spread, save -CUAS-PERSENSOR (16-slice stack, per-slice avg labels). Saves via the -CENTER instance, not gpuQuad.getQuadCLT(). - CuasMotion.perSensorFromRawJp4: read RAW /jp4/ per-sensor (oracle getJp4Tiff, one thread/sensor), force-H2D (bypass the "GPU mem already correct" verify), execConvertDirect from raw, save -CUAS-PERSENSOR-RAW (uncorrected baseline; calibration stays a separate "cheat"). RT-seed for the future SATA raw stream; GPU port later with Java as oracle. Fix the NaN border on the RT SUBAVG-CONV2D product: - CuasDetectRT subtract-average -> NaN-tolerant union (average only non-NaN scenes per pixel), matching the oracle -CUAS-MERGED-CUAS; the plain sum NaN-propagated (one missing scene poisoned the pixel in every frame -> thick border after LoG). - CuasRTUtils.convolve2DLReLU -> NaN-aware (NaN out only if the center is NaN; substitute the center value for NaN taps), so the LoG can't bloom a thin border into a thick NaN frame. - Add -SUBAVG-PRELOG save (post-subtract-avg, pre-LoG) for bisecting. Compiles (mvn -DskipTests clean package). WIP: the raw-path values/edge and the in-memory-vs-file (MERGED-CUAS) divergence are still under review; the ~28px edge residual is traced to the temporal subtract-average at the rotation-swept composite edge. See ANDREY_CONTINUE.md open items. Co-Authored-By:
-
- 01 Jul, 2026 1 commit
-
-
Andrey Filippov authored
Piece 1 of the RT conditioning migration (design: internal handoff 2026-06-30_rt_conditioning_design.md): - cuas/rt/CuasConditioning: lean, self-contained per-sensor conditioning for the TP/RT path - Row/Col denoise (on/off, optional HPF of the 1-D avg profile) then photometric scales2*(raw-C0)^2 + scale*(raw-C0) - FPN (bit-matches the current additive path when scales2=0). Bypasses the heavy QuadCLT conditioning path. - CuasMotion.perSensorAveragesFromTD(GpuQuad, use_reference): memory-lean render of all 16 per-sensor from TD; per-sensor average + spread = calibration-quality gauge. Building blocks only; full test wiring (raw jp4 -> condition -> convert_direct -> renderSceneSequence per-sensor averages) + Eyesis invocation entry still pending. Co-Authored-By:
-
- 27 Jun, 2026 3 commits
-
-
Andrey Filippov authored
Adds an OFF-by-default profile that dependency:unpacks the libtorch native runtime (org.pytorch:libtorch-cxx11-cu128:2.7.1:zip, cu128) from mirror.elphel.com/maven-dependencies into target/libtorch-dist for the native DNN backend (libtpdnn.so / CuasDnnLocal) on a deployment box. The default build never downloads the 3.8GB zip. Artifact published to the mirror in maven layout (server-side copy of the existing zip) via tile_processor_gpu/jna/publish_libtorch_mirror.sh. Verified: zip + .pom reachable at the computed maven URL (HTTP 200, 3.78GB), profile parses (mvn -Plibtorch validate OK). Full unpack deferred (redundant on this box - libtorch already extracted); exercises on first deployment machine via `mvn -Plibtorch generate-resources`. Co-Authored-By: -
Andrey Filippov authored
Bundles the exported TorchScript models + their .meta.json sidecars under src/main/resources/cuas_dnn/<name>/ so CuasDnnLocal runs with no local model dir (deployment needs no PyTorch/dev tooling - just the .so + libtorch runtime): weighted9_pm_s/model.ts.pt (+.meta.json) L1 (N=9,P=24,vr=5,out_ch=124) mexhat_gaps_boost40/model.l2.ts.pt (+.meta.json) L2 (ch_hidden=24,vmax=1.4) Validated: CuasDnnLocal bundled-resource path (curt_dnn_local_dir empty) extracts from the jar and matches the server oracle EXACTLY (offset5=0.0, roi=0.0). Co-Authored-By: -
Andrey Filippov authored
Piece 3 of the native-JNA DNN path. Adds a local backend that runs the SAME L1+L2 inference as CuasDnnRemote but in-process via LibTorch (libtpdnn.so / JNA), so the CUAS pipeline runs without the DGX or any Python server: - CuasDnnBackend : shared interface (upload/getNFrames/inferBatch->BatchResult/close) - TpDnnJna : JNA Library binding libtpdnn.so's C-ABI - CuasDnnLocal : wraps it; reads N/P/vr/l2_ch from each model's bundled .meta.json (single source of truth), float[][]<->float[], builds BatchResult - CuasDnnRemote : now implements CuasDnnBackend (signatures unchanged) - CuasDetectRT : DNN path gate now fires on (curt_dnn_remote || curt_dnn_local); backend = local? CuasDnnLocal : CuasDnnRemote; ensureServer skipped when local; local-CPU-ORT gate also excludes curt_dnn_local (no double-run). runDnnRemote loop unchanged. - IntersceneMatchParameters: curt_dnn_local (flag) + curt_dnn_local_dir (model dir override; empty = bundled /cuas_dnn resource) + GUI labels/persist. Validated: full Java->JNA->libtpdnn vs the Python-server oracle = EXACT (offset5=0.0, roi=0.0, nch=6). mvn -DskipTests package OK. Runtime: -Djna.library.path=<dir with libtpdnn.so>; libtpdnn.so finds libtorch via its rpath. Model resolution mirrors CuasDnnRemote's bundled-vs-override scheme. Co-Authored-By:
-
- 26 Jun, 2026 4 commits
-
-
Andrey Filippov authored
Migration validated (JNA CUAS targets match JCuda). Cleanup: - Removed all TEMP debug probes (-Dtp.dbg.corrpair, probeClt, saveTDRender, the one-shot DBG/PROBE blocks in GpuQuadJna + CuasMotion). Real fixes kept (rectilinear port, num_pairs=3, setCorrIndicesTdData, imclt ref_scene, num_corr_tiles propagation f6dcc90f). - Proactive sweep for the f6dcc90f bug-class (JNA override drops a base side-effect field write): getCorrComboIndices/getCorr2DCombo propagate num_corr_combo_tiles, setCorrIndicesTdData propagates num_corr_tiles, getTextureIndices propagates num_texture_tiles; those fields made protected. These four are LATENT (no live consumer on the validated CUAS path) and are marked NOT-YET-TESTED inline. Java-only. mvn compile clean. Co-Authored-By:
-
Andrey Filippov authored
Root cause of the CORR2D-all-NaN / 0-targets: the inter-correlation actually works (probe showed num_corr_tiles=8850 = 4425 tiles x (1 sensor + 1 sum)), but the TD readback dropped it. Base GpuQuad.getCorrIndices() sets the num_corr_tiles field ("also sets num_corr_tiles"); GpuQuadJna.getCorrIndices() read the native count locally and returned the array WITHOUT setting the field. So TDCorrTile.getFromGpu (num_tiles = getNumCorrTiles()/num_pairs) and base getCorrTilesTd (uses the field directly) saw a stale 0 -> built 0 tiles -> empty target sequence -> null ROUND_ONE image -> saveImagePlusInModelDirectory NPE (the misplaced-null-guard latent bug is just the messenger). Fix: GpuQuadJna.getCorrIndices() sets num_corr_tiles = n (native count); field made protected so the subclass can. Java-only. Co-Authored-By: -
Andrey Filippov authored
Post-mortem showed both CLT buffers loaded but inter-correlation -> 0 tiles. index_inter_correlate selects by __popc(sel_sensors); static reading says sel_sensors should be 1 (single-cam rectilinear), so a runtime value differs. - GpuQuadJna.execCorr2D_inter_TD: one-shot print sel_sensors/popc/num_cams/ num_colors/scales + the returned num_corr_tiles. - saveTDRender: makeArrays NPE'd on null titles (derefs titles[i]); pass a non-null titles[] so the render saves instead of crashing the run. TEMP — remove with the rest of the -Dtp.dbg.corrpair probe. Co-Authored-By: -
Andrey Filippov authored
Single sacrificial run -> generous logging without spam: - GpuQuadJna: probe BOTH first ref convert (gpu_clt_ref) AND first scene convert (gpu_clt) — NaN%/nonzero/range each (probeClt helper). - CuasMotion.correlatePair one-shot: log targets_mv / tp_ref,tp_img counts / erase_cltr,erase_clt / fpixels null-ness, plus TD-correlation read-back stats (tile count + NaN% of TD values) alongside the DBG-REF/DBG-IMG renders. All gated/one-shot; no native change (reads via existing tp_proc_get_clt). TEMP — remove with the rest of the -Dtp.dbg.corrpair probe. Co-Authored-By:
-