Functions
=========

This section lists all the Lua functions provided by AOFlagger.

.. default-domain:: lua

Summary
^^^^^^^
.. module:: aoflagger

The ``aoflagger`` module provides the following functions:

* Reporting and user interface:
    - :meth:`print_polarization_statistics`
    - :meth:`save_heat_map`
    - :meth:`set_progress`
    - :meth:`set_progress_text`
    - :meth:`visualize`

* Scaling:
    - :meth:`apply_bandpass`
    - :meth:`normalize_subbands`
    
* Filtering and resolution:
    - :meth:`downsample`
    - :meth:`high_pass_filter`
    - :meth:`low_pass_filter`
    - :meth:`upsample`
    
* Thresholding:
    - :meth:`sumthreshold`
    - :meth:`sumthreshold_masked`
    - :meth:`threshold_channel_rms`
    - :meth:`threshold_timestep_rms`

* Morphological:
    - :meth:`scale_invariant_rank_operator`
    - :meth:`scale_invariant_rank_operator_masked`

* Other:
    - :meth:`collect_statistics`
    
Detailed descriptions
^^^^^^^^^^^^^^^^^^^^^

.. function:: apply_bandpass(data, filename)

   Apply a bandpass file to the data. The data is changed in place. Each line
   in the file contains <antenna name> <X/Y polarization>
   <channel index> <gain>, separated by spaces, for example:
   
   | RT2 X 0 0.7022
   | RT2 X 1 0.7371
   | RT2 X 2 0.8092
   | ...

   :param data: Data to which the bandpass is applied. 
   :type data: :class:`Data`
   :param filename: Path to bandpass textfile.
   :type filename: string

.. function:: collect_statistics(after_data, before_data)

   Calculate statistics, such as visibility standard deviation and flag
   percentages. When running the strategy on a measurement set, the statistics
   are stored inside the measurement set after finishing all baselines. These
   can be inspected by the ``aoqplot`` tool.
   
   The function takes the data after and before flagging. Any data that are
   flagged in ``before_data`` will not contribute to the statistics. This
   avoids counting e.g. correlator faults or shadowing as interference.

   :param after_data: Flagged data.
   :type after_data: :class:`Data`
   :param before_data: Unflagged data.
   :type before_data: :class:`Data`

.. function:: downsample(data, xfactor, yfactor, masked)

   Decrease the resolution of the data using simple linear binning. This can
   be effective to increase the speed of data smoothing, for example when using
   :meth:`high_pass_filter`. At the function end of :meth:`execute`,
   the data should have the original size. Therefore, a call to downsample
   should normally be followed by a call to :meth:`upsample` to restore the
   flags and visibilities to their original resolution.
   
   When the input data is not exactly divisable by the downsampling factors,
   fewer samples will be averaged into the last bins.

   :param data: Input data (not modified).
   :type data: :class:`Data`
   :param xfactor: Downsampling factor in time direction.
   :type xfactor: integer
   :param yfactor: Downsampling factor in frequency direction.
   :type yfactor: integer
   :param masked: ``true`` means take flags into account during averaging
   :type masked: boolean
   :return: Downsampled version of input data.
   :rtype: :class:`Data`
   
.. function:: high_pass_filter(data, xsize, ysize, xsigma, ysigma)

   Apply a Gaussian high-pass filter to the data. This removes the
   diffuse 'background' in the data. With appropriate settings, it
   can filter the signal of interest (slow sinusoidal signals), making
   the interference easier to detect.
   
   The function convolves the data with a 2D "1 minus Gaussian" kernel.
   The kernel is clipped at the edges. The sigma parameters
   define the strength (band-limit) of the filter: lower values remove
   more of the diffuse structure.

   :param data: The data (modified in place).
   :type data: :class:`Data`
   :param xsize: Kernel size in time direction
   :type xsize: integer
   :param ysize: Kernel size in frequency direction
   :type ysize: integer
   :param xsigma: Gaussian width in time direction.
   :type xsigma: number
   :param ysigma: Gaussian width in frequency direction.
   :type ysigma: number
   
.. function:: low_pass_filter(data, xsize, ysize, xsigma, ysigma)

   Apply a Gaussian low-pass filter to the data. It convolves the
   data with a Gaussian. See :meth:`high_pass_filter` for further details.
   
   :param data: The data (modified in place).
   :type data: :class:`Data`
   :param xsize: Kernel size in time direction
   :type xsize: integer
   :param ysize: Kernel size in frequency direction
   :type ysize: integer
   :param xsigma: Gaussian width in time direction.
   :type xsigma: number
   :param ysigma: Gaussian width in frequency direction.
   :type ysigma: number

.. function:: normalize_subbands(data, nr_subbands)

   Remove jumps between subbands. A subband is
   in this context a number of adjacent channels, equally spaced over
   the bandwidth. This function therefore assumes that all subbands
   have an equal number of channels. 
   
   Each subband is scaled such that the standard deviation of the visibilities
   in a subband is unity. To avoid influence from interference, a stable method
   is used to estimate the standard deviation (Winsorized standard deviation).
   
   A typical use-case for this function is the MWA phase 1 and 2. The 30 MHz
   bandwidth of the MWA is split in 24 'course channels', each consisting
   of 128 channels. Each course channel has an independent gain, and needs
   normalization before it can be compared with adjacent course channels.

   :param data: The data (modified in place).
   :type data: :class:`Data`
   :param nr_subbands: Number of subbands.
   :type nr_subbands: integer

.. function:: print_polarization_statistics(data)
   :deprecated:
   
   Print RFI percentages per polarization to the command line.
   
   :param data: Input data.
   :type data: :class:`Data`
  
.. function:: save_heat_map(filename, data)

   Save the data as a "heat map" image. The type is determined from the
   extension. Supported extensions are ``.svg``, ``.png`` and ``.pdf``.
   
   :param filename: Path to image to be written.
   :type filename: string
   :param data: Input data.
   :type data: :class:`Data`

.. function:: scale_invariant_rank_operator(data, xlevel, ylevel)

   Extend flags in time and frequency direction in a scale-invariant manner.
   This fills holes in the flag mask and makes flag sequences longer.
   Details are described in
   `Offringa et al. 2012 <https://arxiv.org/abs/1201.3364>`_.
   
   :param data: The data (modified in place).
   :type data: :class:`Data`
   :param xlevel: aggressiveness in time-direction
   :type xlevel: number
   :param ylevel: aggressiveness in frequency-direction
   :type ylevel: number

.. function:: scale_invariant_rank_operator_masked(data, mask_data, xlevel, ylevel)

   Perform the same operation as :meth:`scale_invariant_rank_operator`, but
   with a mask. Data that is flagged in the mask are removed before applying
   the operator.

   :param data: The data (modified in place).
   :type data: :class:`Data`
   :param mask_data: The data that is used as mask.
   :type mask_data: :class:`Data`
   :param xlevel: aggressiveness in time-direction
   :type xlevel: number
   :param ylevel: aggressiveness in frequency-direction
   :type ylevel: number

.. function:: set_progress(progress, max_progress)

   Notify user of the progress of this call. The gui uses this
   information to show a progress bar to the user.
   Example: when the :meth:`execute` function iterates over the
   polarizations, progress can be reported by calling
   ``aoflagger.set_progress(curpol, npol)`` inside the loop.

   :param progress: current progress
   :type progress: integer
   :param max_progress: value of progress when complete
   :type max_progress: integer

.. function:: set_progress_text(task_description)

   Notify user of the current task being done. The description can be anything,
   and can literally be presented to the user.

   :param task_description: Description string.
   :type task_description: string

.. function:: sumthreshold(data, xthreshold_f, ythreshold_f, include_x, include_y)

   Run the SumThreshold algorithm on the data. This algorithm detects sharp,
   line-shaped features in the time-frequency domain that are typical for RFI.
   See `Offringa et al. (2010) <https://arxiv.org/abs/1002.1957>`_ for details
   about the algorithm.
   
   The thresholds are relative to a (stable) estimate of the noise in the
   visibilities. They define the base sensitivity of the algorithm.
   Lower values will detect more features. A reasonable value for the
   thresholds is 1.
   
   The "include" parameters turn
   detection in their particular directions on and off. Note that detection in
   "x" (=time) direction means detection of contiguous high-power samples
   in time, such as transmitters that occupy the same channel continuously.
   The y-direction detection is sensitive to transient, broadband RFI.

   :param data: The data (modified in place).
   :type data: :class:`Data`
   :param xthreshold_f: Threshold factor in time direction
   :type xthreshold_f: number
   :param ythreshold_f: Threshold factor in frequency direction
   :type ythreshold_f: number
   :param include_x: The data that is used as mask.
   :type include_x: :class:`Data`
   :param include_y: The data that is used as mask.
   :type include_y: :class:`Data`
   
.. function:: sumthreshold_masked(data, mask_data, x_threshold_f, y_threshold_f, x_direction, y_direction)

   Same as :meth:`sumthreshold`, but with a mask. Visibilities that are flagged
   in the mask are considered to be visibilities that have not been sampled and
   are removed from the SumThreshold operation. A typical case
   for this is to make sure that correlator faults, shadowing and
   band-edges are correctly treated.

   :param data: The data (modified in place).
   :type data: :class:`Data`
   :param mask_data: The data that is used as mask.
   :type mask_data: :class:`Data`
   :param x_threshold_f: Threshold factor in time direction
   :type x_threshold_f: number
   :param y_threshold_f: Threshold factor in frequency direction
   :type y_threshold_f: number
   :param x_direction: The data that is used as mask.
   :type x_direction: boolean
   :param y_direction: The data that is used as mask.
   :type y_direction: boolean
   
.. function:: threshold_channel_rms(data, threshold, flag_low_outliers)

   Calculate the root-mean-square (RMS) for each channel and flags channels
   that have an outlier RMS. The threshold is a "sigma level". Typical values
   for the threshold are therefore around 3.
   
   :param data: The data (modified in place).
   :type data: :class:`Data`
   :param threshold: Sigma-level of threshold
   :type threshold: number
   :param flag_low_outliers: Flag channels with low RMS
   :type flag_low_outliers: boolean

.. function:: threshold_timestep_rms(data, threshold)

   Like :meth:`threshold_channel_rms`, but thresholds *timesteps* with outlier
   RMS. Both timesteps with high and low RMS values are flagged.

   :param data: The data (modified in place).
   :type data: :class:`Data`
   :param threshold: Sigma-level of threshold
   :type threshold: number

.. function:: upsample(input_data, destination_data, xfactor, yfactor)

   Increase the resolution of the data. This function is to restore the
   resolution of the data after having called :meth:`downsample`.
   ``input_data`` is normally the data that was returned by :meth:`downsample`,
   and ``destination_data`` is the input object that was specified as parameter.
   The upsampling is done by nearest neighbour interpolation.
   
   The x and y factors should be the equal to the values specified in the call
   to `downsample`. The size of the ``destination_data`` is not changed: the
   input data is stretched by the given factors, and trimmed to the destination
   size in case the image dimensions were not exactly divisable by the factors.

   :param input_data: Input low-resolution data (not modified).
   :type input_data: :class:`Data`
   :param destination_data: Where the result will be stored.
   :type destination_data: :class:`Data`
   :param xfactor: Upsampling factor in time direction.
   :type xfactor: integer
   :param yfactor: Upsampling factor in frequency direction.
   :type yfactor: integer
   
.. function:: visualize(data, label, sorting_index)

   Save a visualization of the data for inspection in ``rfigui``. When this
   strategy runs outside of the ``rfigui``, the call is ignored. Can be used
   to e.g. inspect partial results.
   
   :param data: Input data (not modified).
   :type data: :class:`Data`
   :param label: A short description that is displayed to the user.
   :type label: string
   :param sorting_index: Where to place this visualization in the list
       of visualization
   
