Commit 96fc22bc authored by Andrey Filippov's avatar Andrey Filippov

new method refineMotionVectors() implemented by Claude and related docs

parent 03a9cc77
......@@ -556,6 +556,123 @@ This section captures the latest validated state before pausing Global LMA work
4. Run `Aux Build Series`.
5. Inspect console + CSV/TIFF outputs; archive debug artifacts as needed.
## `refineMotionVectors()` — Two-Stage Motion Vector Refinement
### Background: MCLT, Phase Correlation, and Noise Integration
The pipeline detects sub-pixel UAS targets using the GPU Tile Processor (TP), which operates
on fixed 16×16 50%-overlapped tiles using a 2D Modified Complex Lapped Transform (MCLT). MCLT
is structurally similar to FFT: convolution and correlation are multiplication in the Transform
Domain (TD). Critically, fractional-pixel image shifts in the pixel domain (PD) correspond to
phase rotations (multiplication by complex unit vectors) in the TD. Integer shifts select which
16×16 window is loaded; the residual fractional shift (−0.5 to +0.5 px) is applied as TD phase
rotation. Both are handled by the GPU kernel.
Pairwise tile correlation uses Phase Correlation: each TD component is normalized by its
amplitude (retaining only phase), so the inverse-transformed result is a near-delta-function
for noiseless data. Noise mitigation uses two mechanisms:
1. **Fat zero:** a positive constant added to the normalization denominator, trading between
pure phase correlation (sharp but noise-fragile) and classical correlation (noise-robust but
blunt). Controlled by `cuas_fat_zero` / `cuas_rng_fz`.
2. **TD pre-averaging:** for a linearly moving target, pairwise TD products with the same
temporal offset carry the same target phase shift. Accumulating N such pairs before
normalization averages down noise by √N while the signal remains coherent. This is done in
TD, before normalization — pairs like (0,8),(1,9),(2,10),…,(7,15) are summed, then
normalized and IDCT'd. `getTargetsFromCorr2d()` implements this.
The current motion-vector scan integrates the **full 16×16 tile** for each pairwise product.
For a sub-pixel target occupying only ~2–4 px, this wastes most of the integration area on
background noise, limiting achievable SNR.
### Purpose of `refineMotionVectors()`
After the first (non-centered) pass gives approximate motion vectors (MV) and the
virtual-tracking-camera pass (`shiftAndRenderAccumulate()`) locates the target centroid
within each tile, we know where in each tile the target sits. The new method exploits this
knowledge to re-run the pairwise correlation with a spatial mask that zeros out pixels far
from the known target center. This:
- Reduces the effective noise integration area from 16×16 to a small window of radius ~r1
(default 6 tiles = ~6 px),
- Optionally increases the number of accumulated pairs (`recalc_mv_boost`, default 4×),
- Produces a refined MV estimate, which feeds back into the next centered-accumulation pass
for improved ranging and target localization.
The method is called (when `recalc_mv=true`) **before**
`CuasMotion.shiftAndRenderAccumulate()` at line 8203, replacing the motion vectors that
`extended_scan` / `targets_nonoverlap` carry.
`recalc_mv` is disabled for `slow_mode` (background suppression path) because the slow
path uses a different temporal pre-filter and the refinement is only meaningful for the fast
(target-motion) path.
### Parameters
```java
boolean recalc_mv = clt_parameters.imp.cuas_recalc_mv && !slow_mode;
// When true, re-correlate with spatial mask to refine MVs before centered accumulation.
double recalc_mv_boost = clt_parameters.imp.cuas_recalc_mv_boost; // default ~4.0
// Multiply the default number of correlation pairs used in the refinement step.
// More pairs → better SNR; practical ceiling is limited by sequence length.
double recalc_mv_r0 = clt_parameters.imp.cuas_recalc_mv_r0; // default ~2.0 (tiles)
// Raised-cosine mask inner radius. For r ≤ r0: weight = 1.0
double recalc_mv_r1 = clt_parameters.imp.cuas_recalc_mv_r1; // default ~6.0 (tiles)
// Raised-cosine mask outer radius. For r ≥ r1: weight = 0.0
// For r0 < r < r1: weight = 0.5 * (cos(π*(r−r0)/(r1−r0)) + 1)
```
The mask is centered on the known target fractional position within the tile (from the
prior centroid/LMA fit). Units are pixels (not tile indices).
### Pipeline position
```
[non-centered pass] → approximate MV in target_sequence_multi / extended_scan
[refineMotionVectors()] ← NEW (optional, recalc_mv=true, fast mode only)
apply spatial mask around target center
re-run TD pairwise accumulation with boosted pairs
return refined MV → overwrite extended_scan (or produce new vector_field)
[shiftAndRenderAccumulate()] ← existing, line 8203
virtual tracking camera, long exposure
[getAccumulatedCoordinates() / LMA]
```
### Open questions (to resolve before implementation)
1. **Where is the target center at this point in the code?** The centroid/LMA result lives
in `target_sequence_multi`. Which field indices (`RSLT_X`, `RSLT_Y`, etc.) hold the
fractional pixel offset within the tile for each keyframe?
2. **What does the mask multiply?** The source `fpixels_tum` float array, or the TD
tiles already computed by the GPU? If the GPU has already done the TD transform by this
point, we need to apply the mask in PD before re-loading, or apply it in TD (which is a
convolution — more complex). Is the mask applied in Java before calling the GPU, or does
it need a new GPU kernel path?
3. **Pair geometry for the boosted refinement:** `recalc_mv_boost` scales the number of
pairs. Does this mean reusing the same `cuas_corr_offset` with more `(frame0+d,
frame1+d)` pairs, or changing the offset too? Are there enough frames in the sequence
to support 4× more pairs without running past the sequence boundary?
4. **Return value:** Should `refineMotionVectors()` return a new `double[][][]` vector
field (same structure as `extended_scan`) with refined `RSLT_VX/VY`, or modify the
existing `target_sequence_multi` in-place?
5. **Tiles with no known target:** For tiles where the first pass found no valid target
(`target_sequence_multi[nseq][ntile] == null` or score below threshold), should
refinement be skipped (keep original MV) or skipped entirely (null)?
6. **New `IntersceneMatchParameters` fields:** Are `cuas_recalc_mv`,
`cuas_recalc_mv_boost`, `cuas_recalc_mv_r0`, `cuas_recalc_mv_r1` already added to
`IntersceneMatchParameters.java`, or do they need to be added as part of this work?
### Next TODO (priority order)
1. Performance pass: identify current bottlenecks and low-hanging optimizations.
- **Two-stage motion vector calculation:** Repeat correlation with a tight mask (R=4) around located targets to significantly decrease the noise integration area and eliminate motion blur.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment