ChangeLog of Munipack

# ChangeLog

A comprehensive summary of the recent development.

More detailed list of changes can be found in Munipack development repository.

## Changelog of 0.6.1 (On the new way with errata) release

Release date: 2. October, 2022.

This release includes minor improvements.

### Improvements:

- It can be build with wxWidgets 3.2 (a tiny little minor bug).
- cFITSIO wrappers of Fortran 2008 has added: rows deletion, 2D and 3D image convenience routines, cleanup.
- PSF of point sources for artificial images can be specified by an external file.
- The code for making of elliptical galaxies handles the ellipticity parameter correctly.
- autoconf machinery has a little bit simpler design.

## Changelog of 0.6.0 (On the new way) release

Release date: 6. March, 2022.

This release brings the deep redesign of GUI.

### Principal changes:

I changed my mind; build principles has been revised to create more handy GUI.

- There is a new, highly optimised, multi-threaded, display engine. The engine is designed as a software render maintainable for long time period unlikely to HW ones. Critical parts follows C paradigm rather then slow C++ recipes.
- New algorithms for colour manipulations has been developed.
- View, the image display, has deep revision. By the new way approach, unfinished parts (tools) are (temporary) removed to prevent any user confusion.
- Browser, the file collection browser, has been redesigned. I will follow the way, if the new GUI will be evinced.
- Tune toolbox has been re-implemented from a scratch.

### Improvements:

- New tune options, and functions are implemented. The panel uses PLplot with Cairo driver to made high quality graphs.
- Magnifier and zoom mini-windows are inspired by SAOImage/ds9. The boring implementation via scroll-bars is abandoned.
- FITS files are loaded as a stream and a current state is transparently indicated.
- Kombine has been revised to suppress fringing patterns.
- The development repository of Munipack has changed installation manner to reflect latest security recommendations for GNU/Debian.
- Artificial sky tool supports models of elliptical galaxies.
- Export in View supports shrinking of output files.
- Implemented basic preferences in GUI
- Developed a new coordinate grid.
- Xmunipack has moved config file into `$HOME/.config/` by XDG recommendations. The obsolete config in `$HOME/.xmunipack/` is abandoned, and can be safety removed.

### Possible directions:

- Tools for Image analysis in GUI
- Robust photometry calibration
- GUI wrappers for common functions

## Changelog of 0.5.14 (Bullseye) release

Release date: 4. February, 2021.

A bugfix release of previous release.

### The bugfix

The interface, introduced in latest release, caused run-time crashes under all i386, armhf, armel architectures in Debian nomenclature. An inspection revealed that C_LONG has been associated with C_LONG_LONG, and vice versa, for some routines. It looks like the 32-bit platforms distinguish the types; 64-bit platforms does not. The malfunction has not been detected during previous tests.

The confusion became as result of my blind following of API in the manual (even obsolete), and missing double check against C headers.

### Other improvements:

- Munipack web pages, and CSS, has responsive html/css façade.
- Web pages, and archive, has been secured. All pages are available via both http and https protocols. The tar archive is signed by me.
- Phcorr has included bug-fix for case than corrections are applied sequentially: If the flat correction has been applied on an image with dark or bias already subtracted, the result FITS has no HDU with the corrected frame included, and std.err.frames has been doubled. Now, the first image extension is replaced by the corrected frame, and any std.err. HDU are replaced by a new one. The malfunction has been reported by Majda.
- File system watcher (the utility reloading files on-the-fly) has been made optional. One requires a notification daemon which is missing on GNU Hurd, one from the Debian architectures (revealed by Debian QA).

## Changelog of 0.5.13 (Bullseye) release

Release date: 21. December, 2020.

An interface of cfitsio library for modern Fortran has been developed in this release.

### The improvement:

Autopkgtest (#977559) has reported an issue under arm64 (aarch64) platform for version 3.49 of cfitsio library, whilst amd64 port is unaffected. An incompatibility in passing of integer numerical types between C and Fortran has been revealed for the array argument NAXES of ftgisz() which wraps ffgisz(). The calling of C functions from Fortran, and vice versa, is a complex problem, requiring resourceful C macros and functions, and varying on a specific software platform.

I resolved the issue by building of an interface of cfitsio library for modern Fortran. The interface fully replaces the wrapping parts of cfitsio, and utilises C/C++ interoperability, a part of Fortran standard since 2003 revision.

As the side effect, a speed of I/O operations has significantly increased. I had improved also reading of tables by slicing columns: tables are read in rows-major order, rather than in column-major, according to recommendations in cfitsio manual (13.2 Optimization Strategies, paragraph 3). There are new implementations of the file handling, and the backup.

### Minor changes:

- Added Gaia EDR3 catalogue.
- Reformatted web pages.

## Changelog of 0.5.12 (Bullseye) release

Release date: 30. November, 2020.

This release takes a snapshot of latest improvements for the incoming Debian release.

### Improvements:

- The estimate of aperture photometry errors is implemented in accordance to DAOPHOT.
- The sky level estimation in FIND, APHOT has increased maximum of source pixels on a few millions (only ten thousands pixels has been used previously).
- The sign convention of geographical longitude is changed (now: +east, -west).
- Fluxes has units eV/s/m2 rather then W/m2. Fluxes are used in photometry calibrated images and tables (results of `phcal`). This choice brings more human friendly numbers for quantities (photons-like for optical bands). By default, flam, fnu, stmag and abmag are no longer included into the tables; they are available via --quantity option.
- Elliptical apertures are implemented in APHOT. This improvement looks mostly useless; the accuracy improvement of prolonged stars is disputable.
- Kombine includes of geographic coordinates into FITS headers.
- Main View frame redesigned: the redesign includes complete look change, a new implementation of the display, and a new zoom tool. One adds a caption at bottom (like to figure captions in regular articles). All prints uses a human-like formatting with pre- and suffixes to units.
- The old-school looking help removed; one is replaced by a call of a regular Web browser. wxWebView gives a poor user experience.
- CSS style, layout, html in doc/ are updated to be suitable for small screens, and compatible with latest web standards.
- File properties dialog improved by Gnome GUI guidelines.
- Sesame resolver implemented (CLI, in Cone search GUI).
- Cone search has a new façade.
- GUIs has been developed for FIND, APHOT
- A file system watcher has been implemented: a FITS file opened in View is monitored, and re-loaded immediately following a modification.
- Bug for small icons in the tune window has been fixed; one caused crash due division by zero.
- The backups implementation is simplified, also `--no-clobber` renamed to `--overwrite.`
- The bug leading to the "Malformed input record" error report is solved. "Interrupted system call" warnings are supresses as much as possible. Module implementing sub-processes handling updated.
- FITSIO related bugs has been corrected: in fitsio wrapper (Debian bug #920426), `ftpclj` (programming mistake) replaced by `ftpcll` in `fits_write_col_logical()`; the bug stops compilation under recent gfortran (versions ≥10).
- A support for wxGTK3 toolkit added. Sources are fully compatible with wxGTK2. There was some issues with their parallel presence.
- I fixed use of c_str(). Description of details can be found in Changelog of wx-3.0.x series.

## Changelog of 0.5.11 (Oak Leaf) release

Release date: 22. January, 2019.

This release is the milestone in develop of robust statistical methods.

### Important changes:

- The core part of robust statistical library initiated founding of Oak Leaf library.

### Improvements:

- In both accuracy and reliability of phcal estimates: to hold Poisson distribution, star counts and their errors are re-scaled by the reference; routine for estimates has been rewritten (now part of Oak Leaf),
- In cone: the reliable HTTP client implementation, added magnitude selection limit in convenience parameters, included transformation of UCAC4 Gunn's ri to Johnson RI magnitudes (the external ucac_jmuc.py is unnecessary).
- Redesigned VOTable source code to be more flexible,
- Routine fitsut has been renamed to fits, the command line interface updated, added new keywords, and copy capability,
- In find, the complex code, related to memory buffers, is replaced by easy to use of internal files, sky is estimated by using of significantly larger amount data points
- aphot has corrected potential crash condition for stars near edges: the output star list has an initialisation.
- Major revision of the pre-correction code: phcorr, flat, dark. All the code has been updated to Fortran 2008 (with modern classy look including classes), it has deeply redesigned both control flow and re-arrangement of routines, the flat-fielding part is updated (preserving compatibility, but simpler and with more reliable estimate of the gain), added new masking method by the median.
- Kombine major revision includes: complete redesign in Fortran 2008, also redesign of interpolation core, new interpolation method has been added, command line interface is updated.
- Timeseries major revision: complete redesign in Fortran 2008, listing of keywords, filenames, some command line interface changes, added horizontal coordinates and airmass.
- Colour processing major revision: code re-designed, removed ctrafo, added white balance by photometry calibration, by a white spot, by a star, and by hand. Updated colour transformation tables. The colour processing is energy oriented now, rather than photon-based, to get better correspondence with human perception.
- xmunipack major revision of View window: it has updated look, the display engine is rewrittem, pre-scaling and zooming. Itt has removed some intensity profiles, generally Itt has lost importance in behalf of the pre-scaling. New intensity scaling engine has been developed, including new initial scaling setup. Both colour tuning, and nite view, has been improved, code re-designed: processing in CIE Luv is replaced by Lab. PNG save code updated, tune panel update, coordinates and values has added the optical (energy) intensity in eV. Corrected bugs: zooming synchro of source drawings, Tune panel (reset), looking glass window works correctly on edges, an offset of value and cross-hair has gone. And also, many of bugs has been added.
- Fixed bug which cause of Improper input (for long input lists).
- Altered FITS file backup strategy.
- Minpack library has been removed from source tree.
- Improved command-line processing.
- Updated bootstraping to be compatible with Debian reproducible builds.
- Updated Autotools machinery (configure and Makefiles).
- Initiated to develop of Fortran 95+ interface to cfitsio: it causes of savage simplification of a plenty of source lines, as well as improves quality of code, at least for valgrind.

## Changelog of 0.5.10 (Flat-fielding) release

Release date: 16. January, 2018.

This release is focused on improving of already developed features.

### Improvements:

- Developed new flat-field approach which is mostly like onto photometry calibration than a plain averaging. I believes the implementation is correct, from different points of view, for the first time.
- Astrometry has faster lucky-match option, input stars are sliced by brightness, has enhanced robust algorithms (estimate of scale, stability, reliability) and some bug-fixes.
- Estimation of the photometry calibration is regularised which leads to more precise and reliable results of photometry, flat-fielding, etc.
- Kombine implements averaging of input frames and the result frames might be rotated.
- Improved estimation of std.errors for aperture photometry leading to better photometric precision as well as conversion of some photometric quantities.
- All robust algorithms has updated their estimate of scale.
- Improved artificial features: background in magnitudes per square arcsecond, documentation.
- Bug-fixes in utilities: phcorr, artificial, votable, list and the graphical interface.

## Changelog of 0.5.9 (Artificial Sky) release

Release date: 18. July, 2017.

This release is bugfix release of previous 0.5.8 (Artificial Sky).

## Changelog of 0.5.8 (Artificial Sky) release

Release date: 14. July, 2017.

This release is mostly focused on an artificial sky and related affairs.

### New features:

- The creation of artificial sky as the testing framework.

### Improvements:

- Robust statistical methods are implemented by more reliable algorithms.
- Flat-fielding problem is fully solved.
- Added UCAC5 catalogue. UCAC4 is still default due wider (photometry) completness.

## Changelog of 0.5.7 (Growth Curve) release

Release date: 25. October, 2016.

This release is mostly focused on growth curve photometry.

### New features:

- Growth curve method is implemented for general radial profiles of stars by using a stable and flexible algorithm. Main goal is photometry calibration of total fluxes (one is superior to aperture correction, independent on seeing) for absolute calibration. A subjective choice of aperture radius is no more need and results are more reliable and less noisy.
- Statistical estimates of data scatter are on base of minimise of entropy rather on maximum-likelihood method. Results has the most minimal dispersion possible and limited mostly by photon statistics.

### Improvements:

- More efficient approach and defaults for astrometry.
- New astrometry core, better estimation of limits on base of statistical tests.
- Implemented astrometry for reflected frames.
- Many important gfortran run-time exceptions (zero division, overflow) bugfixed.
- A correct estimation of Hessian (also errors of parameters) in robust optimalisation.
- An inverse matrix implemented also for singular case.
- The output photometry format is changed: implemented new keywords and features, only results are saved by default.

## Changelog of 0.5.6 (Photon Calibration) release

Release date: January 05, 2015.

This release is mostly focused on the precise photometry calibration.

### Improvements:

- Developed new approach for colour transformations and photometry calibration.
- Robust algorithms improved (precise estimation of statistical errors, initial estimates and better convergence).
- Developed a new flat-fielding algorithm (with precision of up to second order, side product is estimation of gain from series of flats).
- Regular system of packaging is preferred (DEB and RPM builders prepared). Bundle installation is obsolete now.
- Changed structure of photometry files. Removed frameconv utility (merged to phcal).
- Bugfixes: phcorr (saturation, null output), timeseries (reports missing values as undefined) and using of compressed FITSes.
- Docs are generated along with full package.
- Plplot replaced by wxMathPlot. Removed dependency on Plplot (which drops dependency on exact version of wxWidgets).

## Changelog of 0.5.5 (Nights Without Convergence) release

Release date: April 1, 2014.

This release is mostly focused on running tests of improved robust algorithms.

### New feature:

- Robust method uses joint estimator of location and scale which reduces scatter.

### Improvements:

- Gain is proper photometry correction like flat-field since now.
- Cone search supports selection of mirrors of VizieR servers and supports network timeout recovering.
- Preliminary implementation of a general cross-match utility.
- Preliminary attempt for PSF.
- Google code site is planed for shutdown. Its content has been merged with homepage to prevent confusing of users.
- A final test of astrometry calibration can be suppressed to allow calibrate with a low-precision (without knowledge of the exact projection for wide-fields).
- Bugfix: DATE-OBS set by environment variables, undefined values in conversion VOTable to FITS, flat-field scaling,
- Default photometry system is renamed from Landolt to Johnson (older designations is incorrect).
- Clear code for: command-line interface, conventions for negative declinations, exit codes of command-line utilities.
- Clean-up of unused and obsolete parts.
- Updated for wxWidgets 3.0. Planed replace of Plplot by wxMathPlot.

## Changelog of 0.5.4 (Photon Rain) release

Release date: July 31, 2013.

This release is mostly focused on the photometric calibration.

### New features:

- Transformation between photometric systems.
- Full photometric calibration.
- Developed format for light-curve storage.

### Improvements:

- In astrometry matching (speed-up, robustness, brightness as next comaprison atributte, closed path).
- In astrometry (proper motion, UCAC4 as default catalogue).
- New approach on photometry pre-corrections (bias, dark, flat-field).
- FITS header edition.
- Default values for long options.
- Changed structure of FITS tables.
- Building supports system-wide configuration.
- GUI updated to match modern desktops.
- Huge expansion of documentation.

## Changelog of 0.5.3 (New Generation) release

Release date: June 29, 2012.

This release is mostly focused on the astronomical engine: photometry, astrometry, listing and kombine.

### New features:

- New astrometry, photometry and cone search dialogs.
- Tutorials for all basic astronomical tasks.
- New implementations: listing (light curves, catalogues, use astrometry calibration) and kombine (spherical projections, geometrical transformations, mosaicking).
- Backup methods uses GNU conventions (suppresed cfitsio's !clobbers)
- Konve and picko has been removed and ones are provided as externals.
- New image display algorithm in View.
- Atom-feed.

### Improvements:

- Docs converted to HTML5.
- Docs has been restructured and extended for description of completed features and tutorials
- Improved astrometry algorithms (new back-tracking core, abs+rob-fitting code, added least squares, complete update of astrometry code, tuning of parameters and methods on data)
- Improvements in robust estimations (core, precision and stability)
- Implemented astrometry of reference and relative frame.
- Locales-friendly changes
- Improved Tuning: new contrast, color band parameters.
- Updated for new compilers g++/gfortran
- tuning of distribution (used -rpath, clean-ups).
- Implemented exposure mask for kombine

## Changelog of 0.5.2 (Astrometry) release

Release date: Oct. 19, 2011.

This release is mostly focused on astrometry.

### New features:

- Developed GUI and fitting engine for astrometry.
- CLI utilities for astrometry, photometry and listing.
- Initial support for VO (cone-search, VOTable).
- Now, objects and photometry are stored as HDU tables in FITS files.
- New match algorithm on base of back-tracking.
- Code for WCS in FITS.
- Implemented splashing and resume panels.
- Binary builders (and distribution packages) for DEB and RPM based distributions.

### Improvements:

- Config files moved to ~/.xmunipack/ directory.
- Updated homepage (added pages for new utilities, design).
- Adapted for latest wxWidgets (important support for event-driven in without GUI - crutial for run of external utilities, main reason for update to latest wxWidgets, support for floats in spin buttons,..).
- Portable implementations for external processes, CLI parameters parsing.
- Reimplementation of interface to launching of external processes (portable support of pipelining including chained comands and I/O data passing, very important improvement).
- Reimplemented original utilities (classic edition) for new architecture.
- Redesigned GUI for display of images (added right panel).
- Replacement for side panel (for HDU selection is used combo-box).
- Improvemnets of icon (file operation) handling in Browser.
- Reimplemented image tunning.
- Removed bottom toolbars.
- Reimplemented color fits.
- FITS header to single window.
- Enhanced code for displaing, reimplemented zooming and scrolling of images.
- Clean code for threading start/stop, conditional variables (more stability).
- Support of different column types in FITS tables.

The reimplementation of Color FITS is extremely important step. The backward compatibility is broken. External utilities (rawtran and fitspng) are required in versions above 0.3.

## Changelog of 0.5.1 (Coloring) release

Release date: Oct, 17, 2010.

This release is mostly focused on color processing.

### Improvements in color processing:

- added coloring utility (create of natural color images from a set of single-band ones)
- added color space transformation utility (XYZ, Luv, BVR, Scotopic)
- added tuning of colors: saturation, hue, white point (Luv is used as the base transformation color space)
- advanced GUIs for coloring

### Improvements in GUI:

- exporting of FITS headers, images and tables in View to another format (png,jpg for images, txt for header and tables)
- rewrote archive and list implementation for speed, efficiency and fashion
- support for clipboard and DnD, a simple XML serialization
- GUI for corrections and averaging
- added Console and Preferences
- GUI class reimplementation for floating windows (detail, tune,..)
- plotting via plplot library
- enhanced all functions in browser (list types, icons, labels and arrange)
- drop shadow and highlighting icons in browser
- started use of updateUI events, custom dialogs and validator (thx to wxWidget bible)
- improved speedup in archive, list and image display
- reimplemented launching of external utilities in GUI
- added GUI for import of RAW pictures
- implemented search in metadata in Browser
- design improvements
- enhancements in display and tune dialogs
- significant improvements in image displaying

### Improvements in command line utilities:

- corrections for dark,flat and bias
- averaging of dark (biases) and flats
- adapted meandark, autoflat, darkbat and flatbat to a new fashion
- founded munipack CLI interface (we're on the way to use only xmunipack and munipack binaries)

### and finally:

- in docpage, forked classic edition, improvements of homepage design
- new doc pages: color FITS specification, color processing, tune
- reimplemented linux installer (very very hard work!)
- bugfixes (see issues)