#+title: A tour of mrcal: stereo processing
#+OPTIONS: toc:t

This is an overview of a more detailed discussion about [[file:stereo.org][dense stereo]].

* Previous
We just [[file:tour-choreography.org][found the best chessboard-dancing technique]]

* Stereo
#+begin_src sh :exports none :eval no-export
# all the images downsampled for view on the page like this
D=~/projects/mrcal-doc-external
D1=$D/data/figueroa-overpass-looking-S
for img ( $D1/{[01].jpg,[01]-reprojected-scale*.jpg,jplv-stereo-*-scale*.png,{rectified[01],disparity,range}-*.png} ) { \
  convert $img -scale 12% ${img:r}.downsampled.${img:e}
}
#+end_src

We computed intrinsics from chessboards observations earlier, so let's use these
for stereo processing.

I took several images off [[https://www.openstreetmap.org/#map=19/34.05565/-118.25333][a catwalk over Figueroa Street in downtown Los
Angeles]]. This is the view South along Figueroa Street. There're tall buildings
ahead and on either side, which makes an interesting stereo scene.

The two images out of the camera look like this:

[[file:external/data/figueroa-overpass-looking-S/0.jpg][file:external/figures/stereo/0.downsampled.jpg]]
[[file:external/data/figueroa-overpass-looking-S/1.jpg][file:external/figures/stereo/1.downsampled.jpg]]

All the full-size images are available by clicking on an image.

The cameras are 7ft (2.1m) apart. In order to compute stereo images we need an
accurate estimate of the relative geometry of the cameras, which we usually get
as an output of the calibration process. /Here/ I only had one physical camera,
so I did something different:

- I calibrated the one camera monocularly, to get its intrinsics. This is what
  we computed so far
- I used this camera to take a pair of images from slightly different locations.
  This is the "stereo pair"
- I used a separate tool to estimate the extrinsics from corresponding feature
  detections in the two images. This tool isn't a part of mrcal today, but will
  be made available in the future

Generating a stereo pair in this way works well to demo the stereo processing
tools. It does break the [[file:triangulation.org][triangulation uncertainty reporting]], since we lose the
uncertainty information in the extrinsics, however. Preserving this uncertainty,
and propagating it through the recomputed extrinsics to triangulation is on the
[[file:roadmap.org][mrcal roadmap]]. The resulting models we use for stereo processing:

- [[file:external/data/figueroa-overpass-looking-S/splined-0.cameramodel][camera 0]]
- [[file:external/data/figueroa-overpass-looking-S/splined-1.cameramodel][camera 1]]

#+begin_src sh :exports none :eval no-export

#### How I made these models

# I reprojected the images to a pinhole model

D=~/projects/mrcal-doc-external
D1=$D/data/figueroa-overpass-looking-S;
for s (0.6 0.35) { \
  for what (splined opencv8) { \
    ~/projects/mrcal/mrcal-reproject-image \
      -f \
      --to-pinhole \
      --scale-focal $s \
      $D/data/board/$what.cameramodel \
      $D/data/figueroa-overpass-looking-S/[01].jpg | \
    ~/projects/mrcal/mrcal-to-cahvor > \
    $D/data/figueroa-overpass-looking-S/$what.pinhole.scale$s.cahvor;

    for c (0 1) { \
      mv $D/data/figueroa-overpass-looking-S/{$c-reprojected.jpg,$c.$what.pinhole.scale$s.jpg}
    }
  }
}



# Then I computed a few features on the pavement

# Then I constructed a homography from those features using
# cv2.findHomography(), and fed that to img-any to find lots of features on the
# pavement:
~/img_any/binsrc/feature_track \
  -L0 -T2200 -C6000 -R1800 -M 2000 \
  -H $D/data/figueroa-overpass-looking-S/homography.initial.scale0.6.txt \
  $D/data/figueroa-overpass-looking-S/[01].opencv8.pinhole.scale0.6.jpg | \
vnl-filter 'Corner1>500' 'Feat1x>1000' 'Feat2x>1000' > \
$D/data/figueroa-overpass-looking-S/features.imgany.scale0.6.vnl

# Then I transformed those features back to the input image coords
paste \
  <( < $D/data/figueroa-overpass-looking-S/features.imgany.scale0.6.vnl \
       vnl-filter -p Feat1x,Feat1y | \
       ~/projects/mrcal/mrcal-reproject-points \
         --intrinsics-only \
         $D/data/figueroa-overpass-looking-S/opencv8.pinhole.scale0.6.cahvor \
         $D/data/board/opencv8.cameramodel) \
  <( < $D/data/figueroa-overpass-looking-S/features.imgany.scale0.6.vnl \
       vnl-filter -p Feat2x,Feat2y | \
       ~/projects/mrcal/mrcal-reproject-points \
         --intrinsics-only \
         $D/data/figueroa-overpass-looking-S/opencv8.pinhole.scale0.6.cahvor \
         $D/data/board/opencv8.cameramodel) > \
  data/figueroa-overpass-looking-S/features.imgany.inputimage.vnl

# And THEN I could use deltapose to compute extrinsics
rm -f $D1/{splined,opencv8}-{0,1}.cameramodel;

for what (splined opencv8) {
  PYTHONPATH=~/projects/mrcal:~/img_any \
  LD_LIBRARY_PATH=~/projects/mrcal \
  ~/deltapose-lite/calibrate-extrinsics \
    --skip-outlier-rejection \
    --correspondences <( < $D/data/figueroa-overpass-looking-S/features.imgany.inputimage.vnl \
                           vnl-filter 'y1<3200 && y2<3200') \
    --regularization t \
    --seedrt01 0 0 0 $((7.*12*2.54/100)) 0 0 \
    --cam0pose identity \
    --observed-pixel-uncertainty 1 \
    $D1/data/board/$what.cameramodel{,} && \
  zmv -W \
    'camera-*.cameramodel' \
    $D1/$what-\*.cameramodel
}
#+end_src

I then use the mrcal APIs to compute rectification maps, rectify the images,
compute disparities and convert them to ranges. This is all done with the
[[file:mrcal-stereo.html][=mrcal-stereo=]] tool:

#+begin_src sh
mrcal-stereo                     \
  --az-fov-deg 145               \
  --el-fov-deg 135               \
  --el0-deg    5                 \
  --disparity-range 0 400        \
  --sgbm-p1 600                  \
  --sgbm-p2 2400                 \
  --sgbm-uniqueness-ratio 5      \
  --sgbm-disp12-max-diff 1       \
  --sgbm-speckle-window-size 200 \
  --sgbm-speckle-range 2         \
  --valid-intrinsics-region      \
  splined-[01].cameramodel       \
  [01].jpg
#+end_src
#+begin_src sh :exports none :eval no-export
D=~/projects/mrcal-doc-external
D1=$D/data/figueroa-overpass-looking-S/

~/projects/mrcal/mrcal-stereo    \
  --az-fov-deg 145               \
  --el-fov-deg 135               \
  --el0-deg    5                 \
  --disparity-range 0 400        \
  --sgbm-p1 600                  \
  --sgbm-p2 2400                 \
  --sgbm-uniqueness-ratio 5      \
  --sgbm-disp12-max-diff 1       \
  --sgbm-speckle-window-size 200 \
  --sgbm-speckle-range 2         \
  --valid-intrinsics-region      \
  --outdir /tmp                  \
  -f                             \
  $D1/splined-[01].cameramodel   \
  $D1/[01].jpg

zmv -f -W \
  '/tmp/rectified[01].cameramodel' \
  "$D/data/figueroa-overpass-looking-S/rectified-[01].cameramodel"

zmv -f -W \
  '/tmp/[01]-rectified.png' \
  "$D/figures/stereo/rectified[01]-splined.png"

mv \
  /tmp/0-disparity.png \
  $D/figures/stereo/disparity-splined.png

mv \
  /tmp/0-range.png \
  $D/figures/stereo/range-splined.png

for img ( $D/figures/stereo/{rectified[01],disparity,range}-splined.png ) { \
  convert $img -scale 12% ${img:r}.downsampled.${img:e}
}
#+end_src

The =--sgbm-...= options configure the [[https://docs.opencv.org/4.5.3/d2/d85/classcv_1_1StereoSGBM.html][OpenCV SGBM stereo matcher]]. Not
specifying them uses the OpenCV defaults, which usually produces poor results.

The rectified images look like this:

[[file:external/figures/stereo/rectified0-splined.png][file:external/figures/stereo/rectified0-splined.downsampled.png]]
[[file:external/figures/stereo/rectified1-splined.png][file:external/figures/stereo/rectified1-splined.downsampled.png]]

And the disparity and range images look like this:

[[file:external/figures/stereo/disparity-splined.png][file:external/figures/stereo/disparity-splined.downsampled.png]]
[[file:external/figures/stereo/range-splined.png][file:external/figures/stereo/range-splined.downsampled.png]]

On a basic level, this is clearly working well.

If you've used other stereo libraries previously, these rectified images may
look odd. This is due to using a transverse equirectangular projection for
rectification instead of the more common pinhole projection (see the [[file:stereo.org][detailed
stereo-processing documentation]]). This samples the azimuth and elevation angles
evenly, which minimizes the visual distortion inside each image row. A
side-effect is the the vertical expansion in the rectified image at the azimuth
extremesm but since stereo matching works by correlating the rows independently,
this is a good trade-off.

The generated rectified models are [[file:external/data/figueroa-overpass-looking-S/rectified-0.cameramodel][here]] and [[file:external/data/figueroa-overpass-looking-S/rectified-1.cameramodel][here]]. These define a rectified
system with axes

- $x$: the baseline; from the origin of the left camera to the origin of the
  right camera
- $y$: down
- $z$: forward

[[file:mrcal-stereo.html][=mrcal-stereo=]] selects the $y$ and $z$ axes to line up as much as possible with
the geometry of the input models. Since the geometry of the actual cameras may
not match the idealized rectified geometry (the "forward" view direction may not
be orthogonal to the baseline), the usual expectations that $c_x \approx
\frac{\mathrm{imagerwidth}}{2}$ and $c_y \approx
\frac{\mathrm{imagerheight}}{2}$ are not necessarily true in the rectified
system. Thus [[file:mrcal-stereo.html][=mrcal-stereo=]] can visualize the models /and/ the rectified field
of view, for verification:

#+begin_src sh
mrcal-stereo        \
  --az-fov-deg 145  \
  --el-fov-deg 135  \
  --el0-deg    5    \
  --set 'view 70,5' \
  --viz geometry    \
  splined-[01].cameramodel
#+end_src
#+begin_src sh :exports none :eval no-export
~/projects/mrcal/mrcal-stereo                                       \
  --az-fov-deg 145                                                  \
  --el-fov-deg 135                                                  \
  --el0-deg    5                                                    \
  --set 'view 70,5'                                                 \
  --viz geometry                                                    \
  --hardcopy $D/figures/stereo/stereo-rectified-system.svg          \
  --terminal 'svg size 800,600 noenhanced solid dynamic font ",14"' \
  $D1/splined-[01].cameramodel
#+end_src

[[file:external/figures/stereo/stereo-rectified-system.svg]]

Here, the geometry /is/ mostly nominal and the rectified view (indicated by the
purple lines) /does/ mostly lie along the $z$ axis.

* ranged pixels ground-truth                                       :noexport:
**** Buildings
top of Paul Hastings building. 530m away horizontally, 200m vertically: 566m away
https://en.wikipedia.org/wiki/City_National_Plaza

top of 7th/metro building at 7th/figueroa: 860m horizontally, 108m vertically: 870m
Figueroa Tower
https://www.emporis.com/buildings/116486/figueroa-tower-los-angeles-ca-usa

Top of library tower at 5th/figueroa. 513m horizontally, 300m vertically: 594

Near the top of the wilshire grand: 825m horizontall 250m vertically: 862
http://www.skyscrapercenter.com/building/wilshire-grand-center/9686

Near the top of the N Wells Fargo plaza building. 337m horizontally, 220m vertically: 402m
https://en.wikipedia.org/wiki/Wells_Fargo_Center_(Los_Angeles)

Los Angeles Center studios ~ 50m tall, on a hill. 520m horizontally: 522m

333 S Beaudry building. 291m horizontally 111m vertically: 311m
https://www.emporis.com/buildings/116570/beaudry-center-los-angeles-ca-usa

**** tests

Command to test all the ranges

#+begin_src sh :exports none :eval no-export
what=opencv8; (
./mrcal-triangulate $D/$what-[01].cameramodel $D/[01].jpg 2874 1231 --range-estimate 566 --search-radius 10
./mrcal-triangulate $D/$what-[01].cameramodel $D/[01].jpg 2968 1767 --range-estimate 870 --search-radius 10
./mrcal-triangulate $D/$what-[01].cameramodel $D/[01].jpg 1885 864  --range-estimate 594 --search-radius 10
./mrcal-triangulate $D/$what-[01].cameramodel $D/[01].jpg 3090 1384 --range-estimate 862 --search-radius 10
./mrcal-triangulate $D/$what-[01].cameramodel $D/[01].jpg  541  413 --range-estimate 402 --search-radius 10
./mrcal-triangulate $D/$what-[01].cameramodel $D/[01].jpg 4489 1631 --range-estimate 522 --search-radius 10
./mrcal-triangulate $D/$what-[01].cameramodel $D/[01].jpg 5483  930 --range-estimate 311 --search-radius 10
./mrcal-triangulate $D/$what-[01].cameramodel $D/[01].jpg 5351  964 --range-estimate 311 --search-radius 10
) | egrep 'q1|Range'
#+end_src

=tst.py= to just look at a set of ranged features, and to compute the extrinsics
with a simple procrustes fit. Bypasses deltapose entirely. Works ok, but not
amazingly well

#+begin_src python :exports none :eval no-export
#!/usr/bin/python3

import sys
import numpy as np
import numpysane as nps

sys.path[:0] = '/home/dima/projects/mrcal',
sys.path[:0] = '/home/dima/deltapose-lite',
sys.path[:0] = '/home/dima/img_any',
import mrcal

model_intrinsics = mrcal.cameramodel('data/board/splined.cameramodel')
t01              = np.array((7.*12*2.54/100, 0, 0))  # 7ft separation on the x

xy_xy_range = \
    np.array((

        (2874, 1231, 2831.68164062, 1233.9498291,  566.0),
        (2968, 1767, 2916.48388672, 1771.91601562, 870.0),
        (1885, 864,  1851.86499023, 843.52398682,  594.0),
        (3090, 1384, 3046.8894043,  1391.49401855, 862.0),
        (541,  413,  513.77832031,  355.37588501,  402.0),
        (4489, 1631, 4435.24023438, 1665.17492676, 522.0),
        (5483, 930,  5435.96582031, 987.39813232,  311.0),
        (5351, 964,  5304.21630859, 1018.49682617, 311.0),

        # Ranged pavement points. These don't appear to help
        (3592.350428, 3199.133514, 3198.330034, 3227.890159, 14.6),
        (3483.817362, 3094.172913, 3117.605605, 3115.684005, 15.76),
 ))

xy_xy = None
#xy_xy = np.array(( (3483.817362, 3094.172913,	3117.605605, 3115.684005),))





q0 = xy_xy_range[:,0:2]
q1 = xy_xy_range[:,2:4]
r  = xy_xy_range[:,(4,)]

# Points observed by camera0, represented in camera1 frame
p0 = mrcal.unproject(q0, *model_intrinsics.intrinsics(), normalize=True)*r - t01

# The unit observation vectors from the two cameras, observed in camera1. These
# must match via a rotation
v0 = p0 / nps.dummy(nps.mag(p0), -1)
v1 = mrcal.unproject(q1, *model_intrinsics.intrinsics(), normalize=True)

R01  = mrcal.align_procrustes_vectors_R01(v0,v1)
Rt01 = nps.glue(R01, t01, axis=-2)


if xy_xy is not None:
    import deltapose_lite
    rt10 = mrcal.rt_from_Rt(mrcal.invert_Rt(Rt01))
    p = \
        deltapose_lite.compute_3d_intersection_lindstrom(rt10,
                                                         model_intrinsics.intrinsics(),
                                                         model_intrinsics.intrinsics(),
                                                         xy_xy[:,0:2],
                                                         xy_xy[:,2:4],)
    print(nps.mag(p))
    sys.exit()


model0 = mrcal.cameramodel(model_intrinsics)
model0.extrinsics_Rt_toref(mrcal.identity_Rt())
model0.write('/tmp/0.cameramodel')

model1 = mrcal.cameramodel(model_intrinsics)
model1.extrinsics_Rt_toref( Rt01 )
model1.write('/tmp/1.cameramodel')
#+end_src

* Stereo rectification outside of mrcal
As a toolkit, mrcal is fairly flexible, so I want to show how one could perform
stereo processing using other tools a part of the pipeline, rather than letting
[[file:mrcal-stereo.html][=mrcal-stereo=]] do all the work.

What if we want to do our stereo processing with some other tool, and what if
that tool doesn't support the splined model we want to use? We can use mrcal to
reproject the image to whatever model we like, and then hand off the processed
image and new models to that tool. Let's demonstrate with a pinhole model.

We can use the [[file:mrcal-reproject-image.html][=mrcal-reproject-image=]] tool to reproject the images. Mapping
fisheye images to a pinhole model introduces an unwinnable trade-off: the
angular resolution changes dramatically as you move towards the edges of the
image. At the edges the angular resolution becomes extreme, and you need far
more pixels to represent the same arc in space as you do in the center. So you
usually need to throw out high-information pixels in the center, and gain
low-information pixels at the edges. The original image doesn't have more
resolution at the edges, so we interpolate. Cutting off the edges (i.e. using a
narrower lens) helps bring this back into balance.

So let's do this using two different focal lengths to illustrate the trade-off:

- =--scale-focal 0.35=: fairly wide. Looks extreme in a pinhole projection
- =--scale-focal 0.6=: not as wide. Looks more reasonable in a pinhole
  projection, but cuts off big chunks of the image at the edges

#+begin_src sh
for scale in 0.35 0.6; do
  for c in 0 1; do
    mrcal-reproject-image       \
      --valid-intrinsics-region \
      --to-pinhole              \
      --scale-focal $scale      \
      splined-$c.cameramodel    \
      $c.jpg                    \
    | mrcal-to-cahvor           \
    > splined-$c.scale$scale.cahvor;

    mv $c-reprojected{,-scale$scale}.jpg;
  done
done
#+end_src

The wider pinhole resampling of the two images:

[[file:external/data/figueroa-overpass-looking-S/0-reprojected-scale0.35.jpg][file:external/figures/stereo/0-reprojected-scale0.35.downsampled.jpg]]
[[file:external/data/figueroa-overpass-looking-S/1-reprojected-scale0.35.jpg][file:external/figures/stereo/1-reprojected-scale0.35.downsampled.jpg]]

The narrower resampling of the two images:

[[file:external/data/figueroa-overpass-looking-S/0-reprojected-scale0.6.jpg][file:external/figures/stereo/0-reprojected-scale0.6.downsampled.jpg]]
[[file:external/data/figueroa-overpass-looking-S/1-reprojected-scale0.6.jpg][file:external/figures/stereo/1-reprojected-scale0.6.downsampled.jpg]]

We will use jplv (a stereo library used at NASA/JPL) to process these pinhole
images into a disparity map, so I converted the models to the [[file:cameramodels.org::#cameramodel-file-formats][=.cahvor= file
format]], as that tool expects. The models:

- [[file:external/data/figueroa-overpass-looking-S/splined-0.scale0.35.cahvor][camera 0, wider scaling]]
- [[file:external/data/figueroa-overpass-looking-S/splined-1.scale0.35.cahvor][camera 1, wider scaling]]
- [[file:external/data/figueroa-overpass-looking-S/splined-0.scale0.6.cahvor][camera 0, narrower scaling]]
- [[file:external/data/figueroa-overpass-looking-S/splined-1.scale0.6.cahvor][camera 1, narrower scaling]]

Both clearly show the uneven resolution described above, with the wider image
being far more extreme. I can now use these images to compute stereo with jplv:

#+begin_src sh
for scale in 0.35 0.6; do           \
  stereo                            \
    --no-ran                        \
    --no-disp                       \
    --no-pre                        \
    --corr-width 17                 \
    --corr-height 5                 \
    --blob-area 10                  \
    --disp-min 0                    \
    --disp-max 400                  \
    splined-[01].scale$scale.cahvor \
    [01]-reprojected-scale$scale.jpg;

  for f in rect-left rect-right diag-left; do \
    mv 00-$f.png jplv-stereo-$f-scale$scale.png;
  done
done
#+end_src
#+begin_src sh :exports none :eval no-export
# all the images downsampled for view on the page like this
D=~/projects/mrcal-doc-external
D1=$D/data/figueroa-overpass-looking-S
for img ( $D1/jplv-stereo-*-scale*.png ) { \
  convert $img -scale 12% ${img:r}.downsampled.${img:e}
}
#+end_src

The wide rectified images:

[[file:external/data/figueroa-overpass-looking-S/jplv-stereo-rect-left-scale0.35.png][file:external/data/figueroa-overpass-looking-S/jplv-stereo-rect-left-scale0.35.downsampled.png]]
[[file:external/data/figueroa-overpass-looking-S/jplv-stereo-rect-right-scale0.35.png][file:external/data/figueroa-overpass-looking-S/jplv-stereo-rect-right-scale0.35.downsampled.png]]

The narrow rectified images:

[[file:external/data/figueroa-overpass-looking-S/jplv-stereo-rect-left-scale0.6.png][file:external/data/figueroa-overpass-looking-S/jplv-stereo-rect-left-scale0.6.downsampled.png]]
[[file:external/data/figueroa-overpass-looking-S/jplv-stereo-rect-right-scale0.6.png][file:external/data/figueroa-overpass-looking-S/jplv-stereo-rect-right-scale0.6.downsampled.png]]

As most non-mrcal tools, jplv uses a pinhole model for rectification. So even if
we gave it wide-angle images and a wide-angle-friendly camera model, we would
/still/ experience the issues raised above: the rectified images would have
these problems.

The disparities computed by jplv look like this for wide images:

[[file:external/data/figueroa-overpass-looking-S/jplv-stereo-diag-left-scale0.35.png][file:external/data/figueroa-overpass-looking-S/jplv-stereo-diag-left-scale0.35.downsampled.png]]

and for narrow images:

[[file:external/data/figueroa-overpass-looking-S/jplv-stereo-diag-left-scale0.6.png][file:external/data/figueroa-overpass-looking-S/jplv-stereo-diag-left-scale0.6.downsampled.png]]

On a basic level, this is clearly working well also.

If jplv output its rectified pinhole models, we could do an apples-to-apples
comparison using the SGBM correlator, as before. We would ask [[file:mrcal-stereo.html][=mrcal-stereo=]] to
accept jplv's rectification by passing =mrcal-stereo --already-rectified=.

* Splitting a wide view into multiple narrow views
:PROPERTIES:
:CUSTOM_ID: stereo-narrow
:END:

We just showed one way to use jplv to handle mrcal lenses, but we had to pay a
price of degraded feature-matching accuracy due to unevenly-scaled rectified
images. A way to do pinhole-rectified stereo while handling the geometric
challenges of wide-angle lenses is to subdivide the wide field of view into
multiple narrower virtual views. Then we'd have several narrow-angle stereo
pairs instead of a single wide stereo pair, and each narrow pair can be
processed with pinhole rectification. [[file:mrcal-stereo.html][=mrcal-stereo=]] can do all the work. Let's
look 45 degrees to the left:

#+begin_src sh
mrcal-stereo                        \
  --rectification LENSMODEL_PINHOLE \
  --az-fov-deg 80                   \
  --el-fov-deg 80                   \
  --az0-deg    -45                  \
  --disparity-range 0 200           \
  --sgbm-p1 600                     \
  --sgbm-p2 2400                    \
  --sgbm-uniqueness-ratio 5         \
  --sgbm-disp12-max-diff 1          \
  --sgbm-speckle-window-size 200    \
  --sgbm-speckle-range 2            \
  --valid-intrinsics-region         \
  splined-[01].cameramodel          \
  [01].jpg
#+end_src
#+begin_src sh :exports none :eval no-export
D=~/projects/mrcal-doc-external
D1=$D/data/figueroa-overpass-looking-S/

~/projects/mrcal/mrcal-stereo       \
  --rectification LENSMODEL_PINHOLE \
  --az-fov-deg 80                   \
  --el-fov-deg 80                   \
  --az0-deg    -45                  \
  --disparity-range 0 200           \
  --sgbm-p1 600                     \
  --sgbm-p2 2400                    \
  --sgbm-uniqueness-ratio 5         \
  --sgbm-disp12-max-diff 1          \
  --sgbm-speckle-window-size 200    \
  --sgbm-speckle-range 2            \
  --valid-intrinsics-region         \
  --outdir /tmp                     \
  -f                                \
  $D1/splined-[01].cameramodel      \
  $D1/[01].jpg

zmv -f -W \
  '/tmp/[01]-rectified.png' \
  "$D/figures/stereo/rectified[01]-narrow-splined.png"

mv \
  /tmp/0-disparity.png \
  $D/figures/stereo/disparity-narrow-splined.png

mv \
  /tmp/0-range.png \
  $D/figures/stereo/range-narrow-splined.png

for img ( $D/figures/stereo/{rectified[01],disparity,range}-narrow-splined.png ) { \
  convert $img -scale 12% ${img:r}.downsampled.${img:e}
}
#+end_src

The pinhole rectified images:

[[file:external/figures/stereo/rectified0-narrow-splined.png][file:external/figures/stereo/rectified0-narrow-splined.downsampled.png]]
[[file:external/figures/stereo/rectified1-narrow-splined.png][file:external/figures/stereo/rectified1-narrow-splined.downsampled.png]]

And the disparity:

[[file:external/figures/stereo/disparity-narrow-splined.png][file:external/figures/stereo/disparity-narrow-splined.downsampled.png]]

This looks much better than the pinhole-rectified stereo from the full image.
The rectified pinhole models and images could be passed to a different tool to
complete the processing, if desired.

And we can see the rotated field of view when we visualize the rectified system:

#+begin_src sh
mrcal-stereo                        \
  --rectification LENSMODEL_PINHOLE \
  --az-fov-deg 80                   \
  --el-fov-deg 80                   \
  --az0-deg    -45                  \
  --set 'view 70,5'                 \
  --viz geometry                    \
  splined-[01].cameramodel
#+end_src
#+begin_src sh :exports none :eval no-export
~/projects/mrcal/mrcal-stereo                                       \
  --rectification LENSMODEL_PINHOLE                                 \
  --az-fov-deg 80                                                   \
  --el-fov-deg 80                                                   \
  --az0-deg    -45                                                  \
  --set 'view 70,5'                                                 \
  --viz geometry                                                    \
  --hardcopy $D/figures/stereo/stereo-rectified-system-narrow.svg   \
  --terminal 'svg size 800,600 noenhanced solid dynamic font ",14"' \
  $D1/splined-[01].cameramodel
#+end_src

[[file:external/figures/stereo/stereo-rectified-system-narrow.svg]]

* Range accuracy
So far we have looked at the results qualitatively by having a human eyeball the
disparity maps to verify that they looked reasonable. But the goal here is to
get ranges. So let's talk about [[file:tour-triangulation.org][triangulation routines]].

* Next
We're ready to talk about [[file:tour-triangulation.org][triangulation routines]]
