.. _whatsnew-7.2:

**************************
What's New in Astropy 7.2?
**************************

Overview
========

Astropy 7.2 is a release that adds significant new functionality since
the 7.1 release.

In particular, this release includes:

* :ref:`whatsnew-7.2-faster-ecsv-tab-rdr`
* :ref:`whatsnew-7.2-generic-df-conv`
* :ref:`whatsnew-7.2-table-index`
* :ref:`whatsnew-7.2-cosmology`
* :ref:`whatsnew-7.2-fits-wcs-units`
* :ref:`whatsnew-7.2-cls-concat-stack`

In addition to these major changes, Astropy v7.2 includes a large number of
smaller improvements and bug fixes, which are described in the :ref:`changelog`.
By the numbers:

* 858 commits have been added since 7.1
* 166 issues have been closed since 7.1
* 359 pull requests have been merged since 7.1
* 50 people have contributed since 7.1
* 22 of which are new contributors

.. _whatsnew-7.2-faster-ecsv-tab-rdr:

Faster ECSV table reader
========================

A new :ref:`ECSV <ecsv_format>` table reading module has been added that supports
different backend engines for the CSV data parsing. In addition to the default
"io.ascii" engine, this includes engines that use the `PyArrow
<https://arrow.apache.org/docs/python/csv.html>`_ and `Pandas
<https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html>`_ CSV readers. These
can be up to 16 times faster and are more memory efficient than the native astropy ECSV
reader.

An example is reading the 233 Mb gzipped Gaia ECSV source file
``GaiaSource_000000-003111.csv.gz`` from the `Gaia source archive
<https://cdn.gea.esac.esa.int/Gaia/gdr3/gaia_source/>`_.


.. list-table::
    :width: 50%
    :header-rows: 1

    * - Format
      - Engine
      - Time (s)
    * - ecsv
      - pyarrow
      - 2.04
    * - ecsv
      - pandas
      - 10.5
    * - ecsv
      - io.ascii
      - 33.1
    * - ascii.ecsv
      - --
      - 35.6

This functionality is available using ``Table.read(filename, format="ecsv",
engine=...)``, where ``engine`` is one of ``"io.ascii"``, ``"pyarrow"``, or
``"pandas"``. To get further help with this interface run
``Table.read.help(format="ecsv")``. See also the :ref:`ecsv_format` section.

You can also write an `astropy.table.Table` or `astropy.table.QTable` using
``format="ecsv"``, but this is just a thin wrapper around the ``format="ascii.ecsv"``
writer.

.. _whatsnew-7.2-generic-df-conv:

Generic DataFrame conversion methods
====================================

Astropy 7.2 introduces new generic DataFrame conversion methods that support multiple
DataFrame backends through the `Narwhals <https://narwhals-dev.github.io/narwhals/>`_
library. These new methods complement the existing pandas-specific methods:

* **New generic methods**: :meth:`~astropy.table.Table.to_df` and
  :meth:`~astropy.table.Table.from_df` support multiple DataFrame libraries
* **Existing pandas methods**: :meth:`~astropy.table.Table.to_pandas` and
  :meth:`~astropy.table.Table.from_pandas` remain available for pandas-specific workflows

The generic methods require the ``narwhals`` package to be installed::

    python -m pip install narwhals

Supported DataFrame backends include:

* **pandas** - Full feature support with DataFrame indexing
* **polars** - High-performance DataFrames with multidimensional array support
* **pyarrow** - In-memory columnar format (limited to 1D arrays)
* **modin** - Distributed pandas-compatible DataFrames
* **cudf** - GPU-accelerated DataFrames

.. note::
   Currently, only **pandas**, **polars**, and **pyarrow** are explicitly tested in the
   Astropy test suite. Other narwhals-compatible backends should work in principle but
   may exhibit unexpected behavior.

Example usage::

    from astropy.table import Table
    t = Table({'a': [1, 2, 3], 'b': ['x', 'y', 'z']})

    # Convert to different backends
    df_pandas = t.to_df("pandas")
    df_polars = t.to_df("polars")
    df_pyarrow = t.to_df("pyarrow")

    # You can also specify the backend with a module
    import polars as pl
    df_polars = t.to_df(pl)

    # Convert back from any supported DataFrame
    t2 = Table.from_df(df_polars)

The pandas-specific methods are still maintained for legacy applications.
The generic methods provide the same
feature set while enabling broader DataFrame ecosystem compatibility through a
unified API.

See :ref:`df_narwhals` for detailed documentation and examples.

.. _whatsnew-7.2-table-index:

Table index improvements and deprecation
========================================

A new method has been added for accessing a table index for tables with multiple
indices. You can now select the index with the ``with_index(index_id)`` method of the
``.loc``, ``.iloc``, and ``.loc_indices`` properties. In addition, support has been
added for using these indexed search properties with an index based on two or more key
columns. Previously this raised a ``ValueError``

The example below illustrates both of these new features:

>>> from astropy.table import QTable
>>> t = QTable({"a": ["x", "z", "y"], "b": [2.0, 1.0, 1.5], "c": ["a", "b", "c"]})
>>> t.add_index("a")
>>> t.add_index(["a", "b"])
>>> t.loc.with_index("a", "b")["y", 1.5]  # select index ("a", "b")
<Row index=2>
 a      b     c
str1 float64 str1
---- ------- ----
   y     1.5    c

The previous syntax for selecting the index is now deprecated and planned for removal in
astropy 9.0.  Here the index identifier was the first element of the item, e.g.,
``t.loc["a", "z"]`` to use the index on column ``"a"`` to find rows with
``t["a"] == "z"``.

.. _whatsnew-7.2-cosmology:

Cosmology traits
================

The :mod:`~astropy.cosmology.traits` module provides reusable components, called
:term:`traits <trait type>`, that encapsulate specific cosmological properties or
behaviors. For example, the :class:`~astropy.cosmology.traits.HubbleParameter` trait
provides the Hubble constant (``H0``) and related methods, while
:class:`~astropy.cosmology.traits.ScaleFactor`,
:class:`~astropy.cosmology.traits.TemperatureCMB`,
:class:`~astropy.cosmology.traits.DarkEnergyComponent`
:class:`~astropy.cosmology.traits.DarkMatterComponent`
:class:`~astropy.cosmology.traits.BaryonComponent`,
:class:`~astropy.cosmology.traits.PhotonComponent`,
:class:`~astropy.cosmology.traits.TotalComponent`,
:class:`~astropy.cosmology.traits.MatterComponent`, and
:class:`~astropy.cosmology.traits.CriticalDensity` traits provide the scale factor, the temperature or the CMB, the Dark Energy component, and the Dark Matter component,
respectively.

By combining these traits, you can easily construct custom cosmology classes with
precisely the features you need, without having to reimplement common functionality.


Here is an example of how to use the
:class:`~astropy.cosmology.traits.HubbleParameter`,
:class:`~astropy.cosmology.traits.ScaleFactor`,
:class:`~astropy.cosmology.traits.TemperatureCMB`,
:class:`~astropy.cosmology.traits.DarkEnergyComponent`
:class:`~astropy.cosmology.traits.DarkMatterComponent`
:class:`~astropy.cosmology.traits.BaryonComponent`,
:class:`~astropy.cosmology.traits.PhotonComponent`,
:class:`~astropy.cosmology.traits.TotalComponent`,
:class:`~astropy.cosmology.traits.MatterComponent`, and
:class:`~astropy.cosmology.traits.CriticalDensity` traits in custom cosmology classes:

>>> import numpy as np
>>> from astropy import units as u
>>> from astropy.cosmology import Cosmology
>>> from astropy.cosmology.traits import (
...     HubbleParameter,
...     ScaleFactor,
...     TemperatureCMB,
...     DarkEnergyComponent,
...     DarkMatterComponent,
...     BaryonComponent,
...     MatterComponent,
...     CriticalDensity,
...     PhotonComponent,
...     TotalComponent,
... )
>>> class CustomStandardCosmology(Cosmology, HubbleParameter, ScaleFactor, TemperatureCMB,
...                              DarkEnergyComponent, DarkMatterComponent, BaryonComponent,
...                              MatterComponent, CriticalDensity, PhotonComponent, TotalComponent):
...     """Mimics standard ΛCDM cosmology with Planck 2018 parameters."""
...     def __init__(self, H0=67.66, Om0=0.3111, Ode0=0.6889, Ob0=0.0490, Tcmb0=2.7255, Ogamma0=5.38e-5):
...         self.H0 = H0 << (u.km / u.s / u.Mpc)
...         self.Om0 = Om0
...         self.Ode0 = Ode0
...         self.Ob0 = Ob0
...         self.Odm = Om0 - Ob0  # Dark matter density
...         self.Tcmb0 = u.Quantity(Tcmb0, "K")
...         self.Ogamma0 = Ogamma0  # Photon density parameter
...         super().__init__()
...
...     def w(self, z):
...         """Standard cosmological constant."""
...         return -1.0
...
...     def inv_efunc(self, z):
...         zp1 = np.asarray(z) + 1.0
...         return 1.0 / np.sqrt(self.Om0 * zp1**3 + self.Ogamma0 * zp1**4 + self.Ode0)
...
...     is_flat = True  # Standard ΛCDM is flat

>>> # Create and test standard cosmology
>>> std_cosmo = CustomStandardCosmology()
>>> std_cosmo.H0
<Quantity 67.66 km / (Mpc s)>
>>> std_cosmo.scale_factor(0)
<Quantity 1.>
>>> std_cosmo.Tcmb(1)
<Quantity 5.451 K>
>>> std_cosmo.w(0.5)
-1.0
>>> std_cosmo.Ogamma(0)  # Photon density at z=0
np.float64(5.37...e-05)

.. _whatsnew-7.2-fits-wcs-units:

Preserving units in FITS-WCS
============================

By default, the :class:`~astropy.wcs.WCS` class always converts units into degrees
for angles, and SI units for other physical types::

    >>> from astropy.io import fits
    >>> from astropy.wcs import WCS
    >>> header = """
    ... CTYPE1  = 'GLON-CAR'
    ... CTYPE2  = 'GLAT-CAR'
    ... CTYPE3  = 'FREQ'
    ... CUNIT1  = 'arcsec'
    ... CUNIT2  = 'arcsec'
    ... CUNIT3  = 'GHz'
    ... CRVAL1  = 10
    ... CRVAL2  = 20
    ... CRVAL3  = 50
    ... """.strip()
    >>> wcs = WCS(fits.Header.fromstring(header, sep='\n'))
    >>> wcs  # doctest: +FLOAT_CMP
    WCS Keywords
    <BLANKLINE>
    Number of WCS axes: 3
    CTYPE : 'GLON-CAR' 'GLAT-CAR' 'FREQ'
    CUNIT : 'deg' 'deg' 'Hz'
    CRVAL : 0.002777777777777778 0.005555555555555556 50000000000.0
    ...

However, it is now possible to preserve the original units by specifying
``preserve_units=True`` when initializing the :class:`~astropy.wcs.WCS`
object::

    >>> wcs = WCS(fits.Header.fromstring(header, sep='\n'), preserve_units=True)
    >>> wcs  # doctest: +FLOAT_CMP
    WCS Keywords
    <BLANKLINE>
    Number of WCS axes: 3
    CTYPE : 'GLON-CAR' 'GLAT-CAR' 'FREQ'
    CUNIT : 'arcsec' 'arcsec' 'GHz'
    CRVAL : 10.0 20.0 50.0
    ...

When using this, any input/output world coordinates will now be in these
units, and accessing any of the parameters such as ``wcs.wcs.crval`` will
return values in the original header units.

.. _whatsnew-7.2-cls-concat-stack:

Concatenation and stacking of coordinates and time classes
==========================================================

Support has been added to apply numpy functions such as `~numpy.concatenate`
and `~numpy.stack` to sequences of |SkyCoord|, |Time|, as well as coordinate
representations and frames.

Note that a current limitation is that instance attributes like ``location``
for |Time| and ``obstime`` for |SkyCoord| must be the same (and scalar) for
all instances that are being concatenated or stacked. These constraints are
the same as for setting elements of an existing instance.

For coordinates, this new ability replaces the existing functions
``astropy.coordinates.concatenate_representations`` and
``astropy.coordinates.concatenate``, and hence using these will now lead to a
pending deprecation warning. Note that there are small differences in
behaviour. In general, the new route is more flexible, but an exception is
that the existing functions allowed one to concatenate scalars together with
one-dimensional arrays, while this is not allowed with `~numpy.concatenate`.
Instead, like for arrays, one has to explicitly ensure arrays, e.g., by using
``np.concatenate(np.atleast_1d(coords))``.

Full change log
===============

To see a detailed list of all changes in version v7.2, including changes in
API, please see the :ref:`changelog`.

Contributors to the 7.2 release
===============================

The people who have contributed to the code for this release are:

.. hlist::
  :columns: 4

  -  Adrien Thob  *
  -  Albert Y. Shih
  -  Alex Fox  *
  -  Antonio Bento Pereira  *
  -  Benjamin Scully  *
  -  Brigitta Sipőcz
  -  Caspar van Leeuwen  *
  -  Chiara Marmo
  -  Clément Robert
  -  Eero Vaher
  -  Elise Chavez  *
  -  Emily Hu  *
  -  Evan Jones  *
  -  Everett Schlawin  *
  -  Finn Womack  *
  -  Hans Moritz Günther
  -  Harshada Raut  *
  -  Igor Lemos
  -  Jackson Hayward  *
  -  James Turner
  -  Kim Searle  *
  -  Larry Bradley
  -  Leo Singer
  -  Manon Marchand
  -  Marten van Kerkwijk
  -  Matthieu Bec
  -  Maximilian Linhoff
  -  Michael Belfrage  *
  -  Mihai Cara
  -  Mohsin Mehmood  *
  -  Nathaniel Starkman
  -  P\. L\. Lim
  -  Peter Scicluna  *
  -  Pieter Eendebak  *
  -  Preshanth Jagannathan  *
  -  Raphael Erik Hviding  *
  -  Sam Lee
  -  Shreeharsh Shinde  *
  -  Simon Conseil
  -  Sonu Singh  *
  -  Stelios Voutsinas  *
  -  Stuart Mumford
  -  Thomas Robitaille
  -  Tiago Gomes
  -  Tim Jenness
  -  Tom Aldcroft
  -  Varun Nikam
  -  William Jamieson

Where a * indicates that this release contains their first contribution to astropy.
