"Combo strength is cubic root of the product of main, aux and intyer correlation strengths");
gd.addNumericField("Trusted tolerance for small baseline camera(s)",this.trusted_tolerance,3,6,"",
"When downscaling valid residual disparity from the most sensitive inter-camera, do not reduce it to be lower than this");
gd.addTab("ML","Parameters related to the ML files generation for the dual-quad camera rig");
gd.addNumericField("Half-width of the ML tiles to export (0-> 1x1, 1->3x3, 2 -> 5x5)",this.ml_hwidth,0,3,"",
"Amount of data to export to the ML system");
gd.addNumericField("Disparity sweep around ground truth, each side",this.ml_disparity_sweep,3,6,"",
"Sweep symmetrically target disparity around the ground truth disparity, each side");
gd.addNumericField("Number of target disparity sweep steps",this.ml_sweep_steps,0,3,"",
"Generate this many files for each file set. Each tile results depend on the target disparity and this tile data, do not depend on other tiles target disparity");
gd.addCheckbox("Include auxiliary camera data in the ML output",this.ml_keep_aux,
"ML output will have the second set of the layers for the auxiliary camera. Disparity values should be scaled for the camera baseline");
"Inter-camera correlation data has only one layer (and one correlation pair). It is used to generate ground truth data. Usable disparity range (measured in the main camera pixels) is ~1/5 of teh main camera");
"Each of these two layers per camera are calculated from a pair of top/bottom and left/right pairs. Can possibly be used instead of originals to reduce amount of input data");
center_corr[ndir+6][indx++]=cc;// save to 6-th and 7-th layer
}
findx+=width-owidth;
}
}
}
/**
* Save 2d correlation data for one layer, one tile into the combined multi-layer ML array, viewable as an image
* @param tileX horizontal tile index
* @param tileY vertical tile index
* @param ml_hwidth half-width of the preserved 2d correlation (0 - single point, 1 -> 3x3, 2 -> 5x5, 7 - all data)
* @param ml_data multi-layer array, each layer matches an image of ((2 * ml_hwidth + 1) * tilesX) by ((2 * ml_hwidth + 1) * tilesY) in scanline order
* Each tile corresponds to (2 * ml_hwidth + 1) * (2 * ml_hwidth + 1) square in the image. Only selected tiles will be updated, so it is good to initialize array
* with all Double.NaN values
* @param ml_layer layer to save tile data
* @param ml_tile (2 * ml_hwidth + 1) * (2 * ml_hwidth + 1) tile data to be saved
* Save a single value to the combined multi-layer ML array, viewable as an image
* @param tileX horizontal tile index
* @param tileY vertical tile index
* @param ml_hwidth half-width of the preserved 2d correlation (0 - single point, 1 -> 3x3, 2 -> 5x5, 7 - all data)
* @param ml_data multi-layer array, each layer matches an image of ((2 * ml_hwidth + 1) * tilesX) by ((2 * ml_hwidth + 1) * tilesY) in scanline order
* Each tile corresponds to (2 * ml_hwidth + 1) * (2 * ml_hwidth + 1) square in the image. Only selected tiles will be updated, so it is good to initialize array
finaldouble[][][]image_data_main,// first index - number of image in a quad
finaldouble[][][]image_data_main,// first index - number of image in a quad
finaldouble[][][]image_data_aux,// first index - number of image in a quad
finaldouble[][][]image_data_aux,// first index - number of image in a quad
finalboolean[][]saturation_main,// (near) saturated pixels or null
finalboolean[][]saturation_main,// (near) saturated pixels or null
finalboolean[][]saturation_aux,// (near) saturated pixels or null
finalboolean[][]saturation_aux,// (near) saturated pixels or null
// correlation results - combo will be for the correation between two quad cameras
// correlation results - combo will be for the correlation between two quad cameras
finaldouble[][][][]clt_corr_combo,// [type][tilesY][tilesX][(2*transform_size-1)*(2*transform_size-1)] // if null - will not calculate
finaldouble[][][][]clt_corr_combo,// [type][tilesY][tilesX][(2*transform_size-1)*(2*transform_size-1)] // if null - will not calculate
// [type][tilesY][tilesX] should be set by caller
// [type][tilesY][tilesX] should be set by caller
// types: 0 - selected correlation (product+offset), 1 - sum
// types: 0 - selected correlation (product+offset), 1 - sum
finaldouble[][]disparity_bimap,// [23][tilesY][tilesX], only [6][] is needed on input or null - do not calculate
finaldouble[][]disparity_bimap,// [23][tilesY][tilesX], only [6][] is needed on input or null - do not calculate
// last 2 - contrast, avg/ "geometric average)
// last 2 - contrast, avg/ "geometric average)
finaldouble[][]ml_data,// data for ML - 18 layers - 4 center areas (3x3, 5x5,..) per camera-per direction, 1 - composite, and 1 with just 1 data (target disparity)
finaldouble[][][][]texture_tiles_main,// [tilesY][tilesX]["RGBA".length()][]; null - will skip images combining
finaldouble[][][][]texture_tiles_main,// [tilesY][tilesX]["RGBA".length()][]; null - will skip images combining
finaldouble[][][][]texture_tiles_aux,// [tilesY][tilesX]["RGBA".length()][]; null - will skip images combining
finaldouble[][][][]texture_tiles_aux,// [tilesY][tilesX]["RGBA".length()][]; null - will skip images combining
finalintthreadsMax,// maximal number of threads to launch
finalbooleanupdateStatus,
finalintdebugLevel)
{
CLTPass3dscan_prev=clt_3d_passes.get(clt_3d_passes.size()-1);// get last one
double[]dbg_orig_disparity=null;
double[]dbg_with_super_disp=null;
double[]dbg_outlayers=null;
showDoubleFloatArrayssdfa_instance=null;
if(debugLevel>-1)sdfa_instance=newshowDoubleFloatArrays();// just for debugging?
boolean[]these_tiles=scan_prev.getSelected();
double[]this_disparity=scan_prev.getDisparity();// returns a copy of the FPGA-generated disparity combined with the target one
double[]this_strength=scan_prev.getStrength();// cloned, can be modified/ read back
dbg_orig_disparity=this_disparity.clone();
SuperTilesst=scan_prev.getSuperTiles();
if(clt_parameters.plSnapDispAny>=0.0){
double[]snap_disparity=st.snapDisparity(
this_disparity,// final double [] disparity,
this_strength,// final double [] strength,
null,// this_selection, // final boolean [] selection, // can be null
clt_parameters.plDispNorm,// final double dispNorm, // plDispNorm
clt_parameters.plSnapDispAny,//final double snapDispAny, // = .2, // Maximal (scaled by plDispNorm) disparity difference to snap to plane at any strength
clt_parameters.plSnapStrengthAny,//final double snapStrengthAny, // = .2, // Maximal strength to fit any distance (if does not fit otherwise - treat as zero strength
clt_parameters.plSnapNegAny,//final double snapNegAny, // = .2, // Maximal negative disparity difference from the best match
clt_parameters.plSnapDispMax,// final double snapDispMax, // = .5, // Maximal (scaled by plDispNorm) disparity difference to snap to plane at low strength
clt_parameters.plSnapDispWeight,// final double snapDispWeight, // = .5, // Maximal disparity diff. by weight product to snap to plane
clt_parameters.plSnapZeroMode,// final int snapZeroMode, // Zero strength snap mode: 0: no special treatment, 1 - strongest, 2 - farthest
thrownewException("Set naims for cameras do not match: main camera: '"+set_channels_main[nSet].name()+"', aux. camera: '"+set_channels_main[nSet].name()+"'");
thrownewException("Set names for cameras do not match: main camera: '"+set_channels_main[nSet].name()+"', aux. camera: '"+set_channels_main[nSet].name()+"'");
thrownewException("Set naims for cameras do not match: main camera: '"+set_channels_main[nSet].name()+"', aux. camera: '"+set_channels_main[nSet].name()+"'");
thrownewException("Set names for cameras do not match: main camera: '"+set_channels_main[nSet].name()+"', aux. camera: '"+set_channels_main[nSet].name()+"'");
double_stacks[0],// final double [][][] image_data_main, // first index - number of image in a quad
quadCLT_main.image_data,// final double [][][] image_data_main, // first index - number of image in a quad
double_stacks[1],// final double [][][] image_data_aux, // first index - number of image in a quad
quadCLT_aux.image_data,// final double [][][] image_data_aux, // first index - number of image in a quad
saturation_main,// final boolean [][] saturation_main, // (near) saturated pixels or null
quadCLT_main.saturation_imp,// final boolean [][] saturation_main, // (near) saturated pixels or null
saturation_aux,// final boolean [][] saturation_aux, // (near) saturated pixels or null
quadCLT_aux.saturation_imp,// final boolean [][] saturation_aux, // (near) saturated pixels or null
// correlation results - combo will be for the correation between two quad cameras
// correlation results - combo will be for the correation between two quad cameras
clt_corr_combo,// final double [][][][] clt_corr_combo, // [type][tilesY][tilesX][(2*transform_size-1)*(2*transform_size-1)] // if null - will not calculate
clt_corr_combo,// final double [][][][] clt_corr_combo, // [type][tilesY][tilesX][(2*transform_size-1)*(2*transform_size-1)] // if null - will not calculate
// [type][tilesY][tilesX] should be set by caller
// [type][tilesY][tilesX] should be set by caller
// types: 0 - selected correlation (product+offset), 1 - sum
// types: 0 - selected correlation (product+offset), 1 - sum
disparity_bimap,// final double [][] disparity_bimap, // [23][tilesY][tilesX]
disparity_bimap,// final double [][] disparity_bimap, // [23][tilesY][tilesX]
ml_data,// final double [][] ml_data, // data for ML - 10 layers - 4 center areas (3x3, 5x5,..) per camera-per direction, 1 - composite, and 1 with just 1 data (target disparity)
texture_tiles_main,// final double [][][][] texture_tiles_main, // [tilesY][tilesX]["RGBA".length()][]; null - will skip images combining
texture_tiles_main,// final double [][][][] texture_tiles_main, // [tilesY][tilesX]["RGBA".length()][]; null - will skip images combining
texture_tiles_aux,// final double [][][][] texture_tiles_aux, // [tilesY][tilesX]["RGBA".length()][]; null - will skip images combining
texture_tiles_aux,// final double [][][][] texture_tiles_aux, // [tilesY][tilesX]["RGBA".length()][]; null - will skip images combining
thrownewException("Set naims for cameras do not match: main camera: '"+set_channels_main[nSet].name()+"', aux. camera: '"+set_channels_main[nSet].name()+"'");
thrownewException("Set names for cameras do not match: main camera: '"+set_channels_main[nSet].name()+"', aux. camera: '"+set_channels_main[nSet].name()+"'");
System.out.println("No files for the auxiliary camera match series "+quadCLT_main.image_name);
returnnull;
}
double[]referenceExposures_aux=quadCLT_aux.eyesisCorrections.calcReferenceExposures(debugLevel);// multiply each image by this and divide by individual (if not NaN)
imp_ml.setProperty("comment_tileWidth","Square tile size for each 2d correlation, always odd");
imp_ml.setProperty("comment_dispOffset","Tile target disparity minum ground truth disparity");
imp_ml.setProperty("comment_ML_OTHER_TARGET","Offset of the target disparity in the \"other\" layer tile");
imp_ml.setProperty("comment_ML_OTHER_GTRUTH","Offset of the ground truth disparity in the \"other\" layer tile");
imp_ml.setProperty("comment_ML_OTHER_GTRUTH_STRENGTH","Offset of the ground truth strength in the \"other\" layer tile");
imp_ml.setProperty("comment_disparityRadiusMain","Side of the square where 4 main camera subcameras are located (mm)");
imp_ml.setProperty("comment_disparityRadiusAux","Side of the square where 4 main camera subcameras are located (mm). Disparity is specified for the main camera");
imp_ml.setProperty("comment_intercameraBaseline","Horizontal distance between the main and the auxiliary camera centers (mm). Disparity is specified for the main camera");
double [][] disparity_array = new double [quadCLT_main.tp.getTilesY()][quadCLT_main.tp.getTilesX()];
quadCLT_main,// QuadCLT quadCLT_main, // tiles should be set
return measureRig(
quadCLT_aux,//QuadCLT quadCLT_aux,
quadCLT_main, // QuadCLT quadCLT_main, // tiles should be set
tile_op,// int [][] tile_op, // common for both amin and aux
quadCLT_aux, // QuadCLT quadCLT_aux,
disparity_array,// double [][] disparity_array,
double_stacks, // double [][][][] double_stacks,
null,// double [][] ml_data, // data for ML - 10 layers - 4 center areas (3x3, 5x5,..) per camera-per direction, 1 - composite, and 1 with just 1 data (target disparity)
tile_op, // int [][] tile_op, // common for both amin and aux
// scale_step, // double scale_step, // scale for "unstable tiles"
tileX,// int tileX,
tileX,// int tileX,
tileY,// int tileY,
tileY,// int tileY,
nTile))numMeas++;//int nTile
nTile))numMeas++;//int nTile
...
@@ -1172,21 +1732,19 @@ public class TwoQuadCLT {
...
@@ -1172,21 +1732,19 @@ public class TwoQuadCLT {
double[][]disparity_bimap=measureRig(
double[][]disparity_bimap=measureRig(
quadCLT_main,// QuadCLT quadCLT_main, // tiles should be set
quadCLT_main,// QuadCLT quadCLT_main, // tiles should be set
quadCLT_aux,// QuadCLT quadCLT_aux,
quadCLT_aux,// QuadCLT quadCLT_aux,
double_stacks,// double [][][][] double_stacks,
tile_op,// int [][] tile_op, // common for both amin and aux
tile_op,// int [][] tile_op, // common for both amin and aux
disparity_array,// double [][] disparity_array,
disparity_array,// double [][] disparity_array,
saturation_main,// boolean [][] saturation_main, // (near) saturated pixels or null
null,// double [][] ml_data, // data for ML - 10 layers - 4 center areas (3x3, 5x5,..) per camera-per direction, 1 - composite, and 1 with just 1 data (target disparity)
saturation_aux,// boolean [][] saturation_aux, // (near) saturated pixels or null
finalintthreadsMax,// maximal number of threads to launch
finalintthreadsMax,// maximal number of threads to launch
finalbooleanupdateStatus,
finalbooleanupdateStatus,
...
@@ -1262,11 +1812,9 @@ public class TwoQuadCLT {
...
@@ -1262,11 +1812,9 @@ public class TwoQuadCLT {
double[][]disparity_bimap=measureRig(
double[][]disparity_bimap=measureRig(
quadCLT_main,// QuadCLT quadCLT_main, // tiles should be set
quadCLT_main,// QuadCLT quadCLT_main, // tiles should be set
quadCLT_aux,// QuadCLT quadCLT_aux,
quadCLT_aux,// QuadCLT quadCLT_aux,
double_stacks,// double [][][][] double_stacks,
tile_op,// int [][] tile_op, // common for both amin and aux
tile_op,// int [][] tile_op, // common for both amin and aux
disparity_array,// double [][] disparity_array,
disparity_array,// double [][] disparity_array,
saturation_main,// boolean [][] saturation_main, // (near) saturated pixels or null
null,// double [][] ml_data, // data for ML - 10 layers - 4 center areas (3x3, 5x5,..) per camera-per direction, 1 - composite, and 1 with just 1 data (target disparity)
saturation_aux,// boolean [][] saturation_aux, // (near) saturated pixels or null
boolean[][]saturation_main,// (near) saturated pixels or null
double[][]ml_data,// data for ML - 10 layers - 4 center areas (3x3, 5x5,..) per camera-per direction, 1 - composite, and 1 with just 1 data (target disparity)
boolean[][]saturation_aux,// (near) saturated pixels or null
double_stacks[0],// final double [][][] image_data_main, // first index - number of image in a quad
quadCLT_main.image_data,// final double [][][] image_data_main, // first index - number of image in a quad
double_stacks[1],// final double [][][] image_data_aux, // first index - number of image in a quad
quadCLT_aux.image_data,// final double [][][] image_data_aux, // first index - number of image in a quad
saturation_main,// final boolean [][] saturation_main, // (near) saturated pixels or null
quadCLT_main.saturation_imp,// final boolean [][] saturation_main, // (near) saturated pixels or null
saturation_aux,// final boolean [][] saturation_aux, // (near) saturated pixels or null
quadCLT_aux.saturation_imp,// final boolean [][] saturation_aux, // (near) saturated pixels or null
// correlation results - combo will be for the correation between two quad cameras
// correlation results - combo will be for the correation between two quad cameras
null,// final double [][][][] clt_corr_combo, // [type][tilesY][tilesX][(2*transform_size-1)*(2*transform_size-1)] // if null - will not calculate
null,// final double [][][][] clt_corr_combo, // [type][tilesY][tilesX][(2*transform_size-1)*(2*transform_size-1)] // if null - will not calculate
disparity_bimap,// final double [][] disparity_bimap, // [23][tilesY][tilesX]
disparity_bimap,// final double [][] disparity_bimap, // [23][tilesY][tilesX]
ml_data,// final double [][] ml_data, // data for ML - 10 layers - 4 center areas (3x3, 5x5,..) per camera-per direction, 1 - composite, and 1 with just 1 data (target disparity)
null,// final double [][][][] texture_tiles_main, // [tilesY][tilesX]["RGBA".length()][]; null - will skip images combining
null,// final double [][][][] texture_tiles_main, // [tilesY][tilesX]["RGBA".length()][]; null - will skip images combining
null,// final double [][][][] texture_tiles_aux, // [tilesY][tilesX]["RGBA".length()][]; null - will skip images combining
null,// final double [][][][] texture_tiles_aux, // [tilesY][tilesX]["RGBA".length()][]; null - will skip images combining
quadCLT_main.tp.getTilesX()*clt_parameters.transform_size,// final int width,
quadCLT_main.tp.getTilesX()*clt_parameters.transform_size,// final int width,
quadCLT_main,// QuadCLT quadCLT_main, // tiles should be set
quadCLT_aux,// QuadCLT quadCLT_aux,
tile_op,// int [][] tile_op, // common for both amin and aux
disparity_array,// double [][] disparity_array,
ml_data,// double [][] ml_data, // data for ML - 10 layers - 4 center areas (3x3, 5x5,..) per camera-per direction, 1 - composite, and 1 with just 1 data (target disparity)
