/************************************************************************/
/*                                                                      */
/*    vspline - a set of generic tools for creation and evaluation      */
/*              of uniform b-splines                                    */
/*                                                                      */
/*            Copyright 2015 - 2020 by Kay F. Jahnke                    */
/*                                                                      */
/*    Permission is hereby granted, free of charge, to any person       */
/*    obtaining a copy of this software and associated documentation    */
/*    files (the "Software"), to deal in the Software without           */
/*    restriction, including without limitation the rights to use,      */
/*    copy, modify, merge, publish, distribute, sublicense, and/or      */
/*    sell copies of the Software, and to permit persons to whom the    */
/*    Software is furnished to do so, subject to the following          */
/*    conditions:                                                       */
/*                                                                      */
/*    The above copyright notice and this permission notice shall be    */
/*    included in all copies or substantial portions of the             */
/*    Software.                                                         */
/*                                                                      */
/*    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND    */
/*    EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES   */
/*    OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND          */
/*    NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT       */
/*    HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,      */
/*    WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING      */
/*    FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR     */
/*    OTHER DEALINGS IN THE SOFTWARE.                                   */
/*                                                                      */
/************************************************************************/

/// \file metafilter3.cc
///
/// \brief implementing a locally adapted filter
///
/// taking the method introduced in meta_filter.cc one step further,
/// this file uses a filter which varies with the locus of it's
/// application. Since the loci of the pickup points, relative to the
/// filter's center, are decoupled from the filter weights, we can
/// 'reshape' the filter by manipulating them. This program adapts the
/// filter so that it is applied to a perspective-corrected view of
/// the image. So the filter is applied to 'what was seen' rather than
/// 'what the image holds'. This is a subtle difference, which is not
/// easy to spot - you can provoke a clearly visible result by specifying
/// a large hfov, and also by applying the program to images with sharp
/// contrasts and single off-coloured pixels; with ordinary photographs
/// and 'normal' viewing angles you'll only notice a blur. Suitable input
/// is easily made: use the example program 'mandelbrot' in this folder,
/// like 'mandelbrot -2 -1 1 1'
/// If you think of the filter as a lens, the 'normal' way of applying
/// the filter is like looking *at the image* through the lens. What we're
/// doing here is transforming the image as if it had been taken from a
/// *view seen through* that lens. In a wide-angle shot, small circular
/// objects near the edges (especially the corners) of the image appear
/// like ellipses. Filtering the image with a normal convolution with
/// a symmetric filter (like the X shape we're using here) will result
/// in the ellipses being surrounded by a blurred halo which has the same
/// width everywhere. Using the filter implemented here, we see an image
/// of a blurred circular object: the blurred halo is wider in radial
/// direction. And this is precisely what 'should' be seen: Filtering
/// the image with a static filter is a simplification which produces
/// results that look reasonably convincing given a reasonable field 
/// of view, but to 'properly' model some change in viewing conditions
/// (like haze in the atmosphere) we need to model what was seen.
/// While this distinction may look like hair-splitting when it comes to
/// photography, it may be more convincing when you consider feature
/// detection. If you have a filter detecting circular patterns, the
/// filter's response to the elliptical shapes occuring in a wide-angle
/// shot near the margins will be sub-optimal: the feature detector will
/// not respond maximally, because, after all, it's input is an ellipse
/// and not a circle. But if you use the technique I implement in this
/// program, the detector will adapt to the place it 'looks at' and detect
/// representations of circular shapes in the image with proper full
/// response. In fact, such a detector will react with lessened response if
/// the image, near the border, shows circular structures, because these
/// clearly can't have originated from circular objects in the view.
/// This program works with rectilinear input, but the implications are
/// even greater for, say, full spherical panoramas. In such images, there
/// is intense distortion 'near the poles', and using ordinary filters on
/// such images does not produce truly correct effects. With filters which
/// adapt to the locus of their application, such images can be filtered
/// adequately.
/// A word of warning is in order here: you still have to obey the
/// sampling theorem and make sure that the pickup points ar not further
/// than half the minimal wave length captured in the image (expressed as
/// an angle in spherical coordinates). Otherwise you will get aliasing,
/// which may produce visible artifacts. If in doubt, use a larger kernel
/// and scale it down. 'Kernel' size can be adapted freely with the technique
/// proposed here. To see the filter 'at work' process an image with a few
/// isolated black dots on white background and inspect the output, which
/// shows a different result, depending on the distance from the image's
/// center, rather than a uniform impulse response everywhere.
/// All of this sounds like a lot of CPU cycles to be gotten through,
/// but in fact due to vspline's use of SIMD and multithreading it only
/// takes a fraction of a second for a full HD image.

// We use the folowing strategy:
// codify the image's metrics in an object which provides methods to
// convert an image coordinate into spherical coordinates
// and spherical coordinates into image coordinates
// Then have the evaluator convert each batch of incoming coordinates
// to spherical coordinates and successively add the deltas in the filter,
// which are treated as small angles in this program. Each such sum is then
// projected back onto the image plane, yielding a batch of pickup points,
// which is evaluated with the inner functor (the interpolator, the spline).
// The batches of interpolated values are summed up (weighted with the weights
// in the filter), yielding batches of result values which are written out.

// TODO: when working on spherical images, we could use simpler maths
// because the image is in effect already in spherical coordinates. The
// current code works on a rectilinear image.

#include <iostream>

#include <vspline/vspline.h>

#include <vigra/stdimage.hxx>
#include <vigra/imageinfo.hxx>
#include <vigra/impex.hxx>

// we silently assume we have a colour image
typedef vigra::RGBValue < float , 0 , 1 , 2 > pixel_type ; 

// coordinate_type has a 2D coordinate
typedef vigra::TinyVector < float , 2 > coordinate_type ;

// type of b-spline object
typedef vspline::bspline < pixel_type , 2 > spline_type ;

// target_type is a 2D array of pixels  
typedef vigra::MultiArray < 2 , pixel_type > target_type ;

// struct meta_filter holds two MultiArrayViews: the first one,
// 'delta', holds a sequence of coordinate offsets. When the filter
// is applied at a location (x,y), the interpolator is evaluated at
// all points (x,y) + delta[i], and the resulting values are multipled
// with corresponding weights from the second MultiArrayView 'weight'
// and summed up to form the output of the filter.
// Note that, as far as the filter is concerned, coordinates are
// in spherical coordinates. the deltas are small angular differences
// which are added to the pickup points represented as polar coordinates.
// With the 'kernel' being 'small' in relation to the image, this is very
// similar to the planar case.

template < size_t D ,
           typename _coordinate_type ,
           typename _weight_type = float >
struct meta_filter
{
  static const size_t dimension = D ;
  typedef _coordinate_type coordinate_type ;
  typedef _weight_type weight_type ;
  
  vigra::MultiArrayView < D , coordinate_type > delta ;
  vigra::MultiArrayView < D , weight_type > weight ;
} ;

// struct image provides conversion from image coordinates to spherical
// coordinates and back.

template < typename dtype >
struct image
{
  // metrics of the image in pixel units

  size_t px_width ;
  size_t px_height ;
  dtype hfov ;

  // corresponding width and height values in unit sphere radii

  dtype u_width ;
  dtype u_height ;

  // ratio of pixel units to unit sphere radius

  dtype px_per_u ;

  image ( size_t _px_width ,
          size_t _px_height ,
          dtype _hfov )
  : px_width ( _px_width ) ,
    px_height ( _px_height ) ,
    hfov ( _hfov )
  {
    // we do the calculations in a three-axis coordinate system,
    // with the origin in the center of a unit sphere
    // the image is taken to be mounted at ( 1 , 0 , 0 )

    u_width = 2.0 * tan ( hfov / 2.0 ) ;
    px_per_u = px_width / u_width ;
    u_height = px_height / px_per_u ;
  }

  // process incoming 2D image coordinates. image coordinates range
  // from 0 to px_width and 0 to px_height, and the center of
  // pixel ( 0 , 0 ) is at image coordinates ( 0.5 , 0.5 )

  // while codifying the rotation in a quaternion is convenient if the
  // pickup points are given in cartesian coordinates, using the pair
  // of azimuth and elevation is more useful if the pickup points are
  // given as angles on the unit sphere (polar coordinates), in which
  // case we can simply add them to the azimuth and elevation, yielding
  // the polar coordinate for the correctly positioned pickup point.
  // This would be instantly useful for evaluating a spline over a
  // full spherical image, which conveniently uses azimuth and elevation
  // as it's x and y axis. For a rectilinear image we have to move
  // back to cartesian coordinates, then project onto the image plane
  // and proceed in 2D.
  // note that we use a template argument 'argtype' which allows us to
  // use this code to process vectorized data as well.

  template < typename argtype >
  vigra::TinyVector < argtype , 2 > to_spherical
    ( argtype px_x , argtype px_y ) const
  {
    // move to image-centric

    px_x -= px_width / dtype ( 2 ) ;
    px_y -= px_height / dtype ( 2 ) ;

   // scale to unit sphere radii

    argtype u_x = px_x / px_per_u ;
    argtype u_y = px_y / px_per_u ;

    // find azimuth and elevation

    argtype azimuth = atan ( u_x ) ;

    // distance from origin to point projected to horozontal plane

    argtype fl = sqrt ( dtype ( 1 ) + u_x * u_x ) ;

    argtype elevation = - atan ( u_y / fl ) ;

    return { azimuth , elevation } ;
  }

  // this method converts point(s) p in spherical coordinates back to
  // 2D image coordinates

  template < typename argtype >
  vigra::TinyVector < argtype , 2 >
    to_image ( vigra::TinyVector < argtype , 2 > p ) const
  {
    argtype phi = p[0] ;
    argtype theta = p[1] - argtype ( M_PI ) / argtype ( 2 ) ;

//  the complete form is this:
//     dtype r = dtype ( 1 ) ;
//     dtype x = r * sin ( theta ) * cos ( phi ) ;
//     dtype y = r * sin ( theta ) * sin ( phi ) ;
//     dtype z = r * cos ( theta ) ;
    
    // but we use r == 1, hence

    argtype x = sin ( theta ) * cos ( phi ) ;
    argtype y = sin ( theta ) * sin ( phi ) ;
    argtype z = cos ( theta ) ;
    
    // project to image plane

    argtype u_ppx = y / x ;
    argtype u_ppy = z / x ;

    // scale to pixel coordinates

    argtype px_ppx = u_ppx * px_per_u ;
    argtype px_ppy = u_ppy * px_per_u ;

    // now move back from image-centric to UL-based image coordinates

    px_ppx += px_width / dtype ( 2 ) ;
    px_ppy += px_height / dtype ( 2 ) ;

    return { px_ppx , px_ppy } ;
  }
} ;

// the filter is built by functor composition. EV is the
// type of the interpolator producing the pickup values,
// D is the dimension of the arrays holding the filter's
// state

template < typename I , typename O , size_t S ,
           template < typename , typename , size_t > class EV ,
           size_t D >
struct ev_meta
: public vspline::unary_functor < I , O , S >
{
  typedef vspline::unary_functor < I , O , S > base_type ;
  typedef EV < I , O , S > inner_type ;

  using typename base_type::in_type ;
  using typename base_type::in_ele_type ;
  using typename base_type::out_type ;
  using base_type::vsize ;
  using base_type::dim_in ;
  using base_type::dim_out ;

  typedef meta_filter < D , in_type > filter_type ;

  // the filter holds these values:

  const inner_type inner ;
  filter_type filter ;
  const coordinate_type extent ;
  float hfov ;
  image<float> img ;

  // which are initialized in it's constructor:

  ev_meta ( const inner_type & _inner ,
            const filter_type & _filter ,
            coordinate_type _extent ,
            float _hfov )
  : inner ( _inner ) ,
    filter ( _filter ) ,
    extent ( _extent ) ,
    hfov ( _hfov ) ,
    img ( _extent[0] , _extent[1] , _hfov )
  {
    // just to make sure

    assert ( filter.delta.size() == filter.weight.size() ) ;

    // we scale the deltas in the filter so that a value of 1 in the
    // data passed in is converted to the angle on the unit sphere
    // corresponding to a move from the center pixel to the next one

    auto e2 = extent / 2 ;

    auto px_center = img.to_spherical ( e2[0] , e2[1] ) ;
    auto px_next = img.to_spherical ( e2[0] + 1 , e2[1] ) ;
    auto factor = px_next [ 0 ] - px_center [ 0 ] ;

    std::cout << "scaling down by " << factor << std::endl ;

    for ( auto & delta : filter.delta )
      delta *= factor ;
  } ;
  
  // since the code is the same for vectorized and unvectorized
  // operation, we can write an 'eval' template which can be used
  // both for single-value evaluation and vectorized evaluation.
  
  template < class IN , class OUT >
  void eval ( const IN & c ,
                    OUT & result ) const
  {
    // clear 'result'

    result = 0.0f ;

    // iterate over the deltas and weights in lockstep

    auto delta_it = filter.delta.begin() ;

    for ( auto const & weight_it : filter.weight )
    {
      OUT pickup ; // to hold partial result(s) from pickup point(s)

      // the current delta is taken to mean a pair of azimuth and elevation

      auto ae = (*delta_it) ;
      ++delta_it ;

      // convert the incoming coordinate(s) to spherical coordinates

      auto cc = img.to_spherical ( c[0] , c[1] ) ;

      // add the current delta, yielding pickup spherical coordinate(s)
  
      for ( size_t d = 0 ; d < dim_in ; d++ )
        cc [ d ] += ae [ d ] ;

      // now convert the sum back to image coordinates to yield the
      // pickup coordinate(s) in 2D cartesian image coordinates

      auto ci = img.to_image ( cc ) ;

      // there, evaluate the interpolator to yield the pickup value(s)

      inner.eval ( ci , pickup ) ;

      // apply the current weight to the pickup value(s) and sum up,
      // yielding the filter output(s)

      for ( size_t ch = 0 ; ch < dim_out ; ch++ )
        result [ ch ] += weight_it * pickup [ ch ] ;
    }
  } 
  
} ;

int main ( int argc , char * argv[] )
{
  if ( argc < 2 )
  {
    std::cerr << "pass a colour image file as argument" << std::endl ;
    exit( -1 ) ;
  }

  // get the image file name

  vigra::ImageImportInfo imageInfo ( argv[1] ) ;

  // create cubic 2D b-spline object containing the image data

  spline_type bspl ( imageInfo.shape() ) ;

  // load the image data into the b-spline's core.

  vigra::importImage ( imageInfo , bspl.core ) ;
  
  // prefilter the b-spline

  bspl.prefilter() ;

  // create a 'safe' evaluator from the b-spline, so we needn't worry
  // about out-of-range access; the default safe evaluator uses mirror
  // boundary conditions and maps out-of-range coordinates into the range
  // by mirroring on the bounds

  auto ev = vspline::make_safe_evaluator ( bspl ) ;

  // now we set up the meta filter. first the deltas. We pass in values
  // in pixel units which will be converted to corresponding angles in the
  // filter code. The pattern is an X shape.

  coordinate_type delta[]
   {
     { -1.6f , -1.6f } ,
     {  1.6f , -1.6f } ,
     { -0.7f , -0.7f } ,
     {  0.7f , -0.7f } ,
     {  0.0f ,  0.0f } ,
     { -0.7f ,  0.7f } ,
     {  0.7f ,  0.7f } ,
     { -1.6f ,  1.6f } ,
     {  1.6f ,  1.6f }     
   } ;

  // next the weights, all equal for this example

  float weight[]
   { 1.0f / 9.0f ,
     1.0f / 9.0f ,
     1.0f / 9.0f ,
     1.0f / 9.0f ,
     1.0f / 9.0f ,
     1.0f / 9.0f ,
     1.0f / 9.0f ,
     1.0f / 9.0f ,
     1.0f / 9.0f
  } ;

  // both are now packaged in the meta_filter struct

  meta_filter < 1 , coordinate_type > mf ;

  mf.delta = vigra::MultiArrayView < 1 , coordinate_type >
    ( vigra::Shape1 ( 9 ) , delta ) ;

  mf.weight = vigra::MultiArrayView < 1 , float >
    ( vigra::Shape1 ( 9 ) , weight ) ;

  // now we create the actual meta filter function from the
  // interpolator and the filter data. Note that a large hfov is
  // needed to actually see that the result of the filter looks
  // different to the similar operation done planar.

  float hfov = 70.0 * M_PI / 180.0 ; // 70 degree in radians

  auto mev = ev_meta < coordinate_type ,
                       pixel_type ,
                       vspline::vector_traits < float > :: size ,
                       vspline::grok_type ,
                       1 >
               ( ev , mf , imageInfo.shape() , hfov ) ;

  // this is where the result should go:

  target_type target ( imageInfo.shape() ) ;

  vspline::transform ( mev , target ) ;

  // store the result with vigra impex

  vigra::ImageExportInfo eximageInfo ( "meta_filter.tif" );
  
  std::cout << "storing the target image as 'meta_filter.tif'" << std::endl ;
  
  vigra::exportImage ( target ,
                       eximageInfo
                       .setPixelType("UINT8") ) ;
  
  exit ( 0 ) ;
}
