Commit 1579c9b3 authored by Andrey Filippov's avatar Andrey Filippov

CLAUDE: Add OpenMVS 3D reconstruction section to project-details.md

Documents TERRAIN-DISP-MERGED TIFF format, INTERFRAME.corr-xml pose
convention, Elphel→COLMAP coordinate conversion, disparity-to-depth
formula, and the foliage CLI pipeline (no C++ library linking).
Co-authored-by: 's avatarClaude <claude@elphel.com>
parent bee6d23c
......@@ -71,6 +71,30 @@ For manual browsing, the program creates a directory with symlinks to special mo
/media/elphel/btrfs-data/lwir16-proc/NC/models/1763232117-1763234145
```
### FOPEN NC Summary and Map-Overlay Workflow
The current North Carolina FOPEN linked view is:
```
/home/elphel/lwir16-proc/NC/linked/linked_1763232117-1763234145-v88/
```
Entries are timestamp-named directories using `_` instead of the decimal point in seconds. The `*-index` entries are index scenes; ordinary timestamp entries before an index entry, and after the previous index entry or start of the directory, belong to that index group. The index scene version directory contains `*-ground_planes.csv`, whose first column lists the reference-scene timestamps in the group.
The spreadsheet summary output should be written under:
```
/home/elphel/lwir16-proc/NC/summaries/
```
For each ordinary timestamp directory, the useful inputs are:
- `<timestamp>-elevations_histogram.csv`: first data row, first column is AGL. The cumulative elevation histogram can be converted to a forestation estimate as the relative area above a small elevation threshold, currently 1.0 m by default.
- `<timestamp>-egomotion.csv`: after the header, each row is one 60 Hz scene. Columns 1-2 are scene index and timestamp; columns 3-8 are X/Y/Z and Euler angles relative to the reference frame. The reference-frame row has exact zeroes in all six relative pose columns. Its `lat` and `long` columns are the rectangle center for map overlays.
For map overlays, the reliable first-pass orientation should come from the local `lat`/`long` motion around the reference row, not from the absolute IMS orientation or the full first-to-last segment chord. Use scene indices `ref-1` to `ref+1` when available, with adjacent egomotion rows as fallback. Assume low wind and align the image X axis opposite the local flight direction, because new objects appear at the left margin of the images. The LWIR-16 camera HFoV is 32 degrees and VFoV is `512/640 * HFoV`; combined with AGL, this defines the footprint rectangle dimensions. Labels can use the four least significant digits of timestamp seconds plus one decimal digit.
Overlay generation should keep georeferenced and printable outputs separate: KML/GeoJSON for map/GIS import, and SVG/PDF for large-format vector inspection. The first NC v88 overlay bounds from all usable rectangle corners are approximately 34.8196196..34.8557247 latitude and -79.2299851..-79.1587591 longitude, about 6.5 km by 4.0 km at the mid-latitude.
## Core Processing Pipeline (Segment-Based)
Scene sequences are processed separately in a loop such as:
......@@ -449,6 +473,69 @@ Operational goal:
This work should stay separate from factory calibration logic. The intended change is in scene/feature preparation and selection, not in the underlying multiscene `LY` optimizer.
## OpenMVS 3D Reconstruction (foliage subproject)
Public repo: `git@git.elphel.com:Elphel/foliage.git` (cloned at `attic/foliage/`).
Internal context: `attic/foliage-internal/docs/FOPEN_MVS_3D_Workflow.md`.
### Input files per segment reference scene
| File | Content |
|------|---------|
| `*-TERRAIN-DISP-MERGED.tiff` | Multi-frame 32-bit float TIFF. Slices labelled `"average"` (first two) are skipped. Remaining slices are labelled with timestamps (`<ts>-<n>`) and correspond to the fused 640×512 LWIR image for each scene. |
| `*-INTERFRAME.corr-xml` | Per-scene poses under keys `EYESIS_DCT_AUX.scenes_<ts>` = `X, Y, Z, azimuth, tilt, roll` (metres, radians) relative to the reference scene. `_dt` / `_d2t` variants are velocity/acceleration and are ignored. Reference scene timestamp in `EYESIS_DCT_AUX.timestamp_reference`. |
### Coordinate conventions
**Elphel camera frame** (per `GeometryCorrection`):
- X = image-plane right, Y = image-plane up, Z < 0 along optical axis (OpenGL-style)
- `focalLength` in mm (field in `GeometryCorrection`), `pixelSize` in μm
- Focal length in pixels: `fl_px = focalLength_mm / (pixelSize_μm × 0.001)`
- For 32° HFoV, 640 px width: `fl_px ≈ 1116`
- Principal point: (319.5, 255.5) for 640×512
**Rotation matrix** — mirrors `GeometryCorrection.getCommonRotMatrix()`:
```
R_w2c = R_roll(roll) @ R_tilt(tilt) @ R_az(azimuth)
```
Each factor is the transposed standard rotation (R_Y^T, R_X^T, R_Z^T respectively).
Combined R_w2c transforms world → Elphel-camera coords.
**COLMAP/OpenMVS flip** (Elphel Y-up/−Z-fwd → COLMAP Y-dn/+Z-fwd):
```
R_colmap = diag(1, −1, −1) @ R_w2c
t_colmap = R_colmap @ (−C) where C = [X, Y, Z] is the camera centre
```
Convention is validated in Phase 3 Blender smoke test (see workflow doc).
### Pipeline (all CLI tools, no library linking)
```
Phase 0 scripts/elphel_to_colmap.py
→ <out>/colmap/ (cameras.txt, images.txt, 16-bit PNGs)
Phase 2 InterfaceCOLMAP → DensifyPointCloud → ReconstructMesh
→ RefineMesh → TextureMesh
(one-command: scripts/run_openmvs_pipeline.sh)
```
OpenMVS is used entirely as CLI tools. No C++ library linking required.
Set `OPENMVS_BIN=` env var if tools are not on PATH; see script header for
build instructions.
### Disparity-to-depth formula (not used by OpenMVS directly)
```
Z_m = SCENE_UNITS_SCALE × focalLength_mm × disparityRadius_mm
/ (disparity_px × 0.001 × pixelSize_μm)
```
where `SCENE_UNITS_SCALE = 0.001` (mm → m) and `disparityRadius = 150 mm`
(= √2 × cameraRadius, set in `GeometryCorrection`).
The 80×64 `*_SZXY.tiff` contains pre-computed (S, Z, X, Y) per tile and
can be used as a Phase 1 sparse seed once a suitable importer is written.
## Latest Additions
### Segment freezing with `keep_segments`
Index scenes (`*-index`) contain `*-INTERFRAME.corr-xml` with keys like:
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment