Tutorial: Introduction to color segmentation using HSV color scale

Introduction

The HSV scale, which stands for Hue Saturation and Value, provides a numerical readout of your color image that corresponds to the color names contained therein. Hue is measured in degrees from 0 to 360, while Saturation and Value of a color are both analyzed on a scale of 0 to 100 percent.

Hue, Saturation, and Value are the main color properties that allow us to distinguish between different colors. This format is implemented in the vpHSV class as long as you use C++11 or higher. It is a templated class that permits to choose the arithmetic representation of the channels. You can either choose:

• double or float: the range of value will be in the intervall [0; 1]
• unsigned char: the second template parameter useFullScale will determine the range for the Hue channel, while the Saturation and Value channel will be encoded in the range [0; 255]. Please refer to the vpHSV documentation for more information.

In this tutorial, you will learn how to use HSV color scale to segment a specific color in an image.

Note that all the material (source code and images) described in this tutorial is part of ViSP source code (in tutorial/segmentation/color folder) and could be found in https://github.com/lagadic/visp/tree/master/tutorial/segmentation/color.

RGB to HSV color scale conversion

In ViSP, color images can be read and converted to the RGB color scale. The RGB color scale is based on the color theory that all visible colors can be obtained from the additive primary colors red, green and blue. In ViSP, we introduce an additional Alpha channel to add color transparency. The RGB + Alpha channels are therefore implemented in the vpRGBa class. The following snippet shows how to load a color image in ViSP:

#include <visp3/io/vpImageIo.h>
 
int main()
{
  vpImage<vpRGBa> I;
  vpImageIo::read(I, "ballons.jpg");
}

The color conversion between RGBa and HSV images scale is performed in ViSP using vpImageConvert::convert() if you use C++11 or higher and the vpHSV class.

The following snippet shows how you can use vpImageConvert::convert() method to convert vpHSV images from and to vpRGBa images:

#include <visp3/io/vpImageIo.h>
#include <visp3/core/vpImageConvert.h>
 
int main()
{
  vpImage<vpRGBa> I, Iother;
  vpImageIo::read(I, "ballons.jpg");
 
  vpImage<vpHSV<double>> Ihsv;
 
  vpImageConvert::convert(I, Ihsv);
  vpImageConvert::convert(Ihsv, Iother);
}

If you cannot use the vpHSV class, you can perform the color conversion from RGB to HSV or from RGBa to HSV color scale using one of the following functions:

• vpImageConvert::RGBToHSV()
• vpImageConvert::RGBaToHSV()

The following snippet shows how to convert to HSV color scale:

#include <visp3/io/vpImageIo.h>
#include <visp3/core/vpImageConvert.h>
 
int main()
{
  vpImage<vpRGBa> I;
  vpImageIo::read(I, "ballons.jpg");
 
  unsigned int width = I.getWidth();
  unsigned int height = I.getHeight();
 
  vpImage<unsigned char> H(height, width);
  vpImage<unsigned char> S(height, width);
  vpImage<unsigned char> V(height, width);
 
  vpImageConvert::RGBaToHSV(reinterpret_cast<unsigned char *>(I.bitmap),
                            reinterpret_cast<unsigned char *>(H.bitmap),
                            reinterpret_cast<unsigned char *>(S.bitmap),
                            reinterpret_cast<unsigned char *>(V.bitmap), I.getSize());
}

In the previous example, we obtained for each pixel:

• Hue in H image where values are scaled from 0 to 255. here 255 stands for 360 degrees.
• Saturation in S image where values are scaled from 0 to 255. Here 255 stands for 100%.
• Value in V image where values are scaled from 0 to 255. Here 255 stands for 100%.

HSV color segmentation

It's easy to segment a given color if we select the range of hue, saturation and value we're interested in.

In the image ballons.jpg, the pixel at coordinates [93][164] has an RGB value (209, 72, 0) which corresponds to an HSV value (14, 255, 209). We can use these HSV values and an additional offset to determine the low and high values of the HSV ranges used to create a mask corresponding to the segmented color.

First, let's load the image:

  vpImage<vpRGBa> I;
  vpImageIo::read(I, "ballons.jpg");

Then, let's convert the vpRGBa image into a vpHSV image. Here, we encode the HSV channels using unsigned char because we will compute the thresholds using this range, but we could have used double instead by just converting the ranges in the range [0; 1] :

  vpImage<vpHSV<unsigned char, true>> Ihsv;
  vpImageConvert::convert(I, Ihsv);

Then, let's define the HSV ranges we want to use:

  int h = 14, s = 255, v = 209;
  int offset = 30;
  int h_low = std::max<int>(0, h - offset), h_high = std::min<int>(h + offset, 255);
  int s_low = std::max<int>(0, s - offset), s_high = std::min<int>(s + offset, 255);
  int v_low = std::max<int>(0, v - offset), v_high = std::min<int>(v + offset, 255);
  std::vector<int> hsv_range;
  hsv_range.push_back(h_low);
  hsv_range.push_back(h_high);
  hsv_range.push_back(s_low);
  hsv_range.push_back(s_high);
  hsv_range.push_back(v_low);
  hsv_range.push_back(v_high);

Now, we can compute the mask that indicates if a pixel is in the desired HSV range:

  vpImage<unsigned char> maskHSV;
  vpImageConvert::convert(I, Ihsv);
  vpImageTools::inRange(Ihsv, hsv_range, maskHSV);

Finally, we can apply the mask to the original image:

  vpImage<vpRGBa> I_segmented_from_HSV;
  vpImageTools::inMask(I, maskHSV, I_segmented_from_HSV);

Note that all these steps are equivalent to the following lines if you cannot use the vpHSV class:

  unsigned int width = I.getWidth();
  unsigned int height = I.getHeight();
  vpImage<unsigned char> H(height, width);
  vpImage<unsigned char> S(height, width);
  vpImage<unsigned char> V(height, width);
 
  vpImageConvert::RGBaToHSV(reinterpret_cast<unsigned char *>(I.bitmap),
                            reinterpret_cast<unsigned char *>(H.bitmap),
                            reinterpret_cast<unsigned char *>(S.bitmap),
                            reinterpret_cast<unsigned char *>(V.bitmap), I.getSize());
 
  vpImage<unsigned char> mask(height, width);
  vpImageTools::inRange(reinterpret_cast<unsigned char *>(H.bitmap),
                        reinterpret_cast<unsigned char *>(S.bitmap),
                        reinterpret_cast<unsigned char *>(V.bitmap),
                        hsv_range,
                        reinterpret_cast<unsigned char *>(mask.bitmap),
                        mask.getSize());
 
  vpImage<vpRGBa> I_segmented(height, width);
  vpImageTools::inMask(I, mask, I_segmented);

Combining all together, we get the following program also available in tutorial-hsv-segmentation-basic.cpp:

#include <visp3/core/vpConfig.h>
#include <visp3/core/vpHSV.h>
#include <visp3/io/vpImageIo.h>
#include <visp3/core/vpImageConvert.h>
#include <visp3/core/vpImageTools.h>
#include <visp3/gui/vpDisplayFactory.h>
 
int main()
{
#ifdef ENABLE_VISP_NAMESPACE
  using namespace VISP_NAMESPACE_NAME;
#endif

  vpImage<vpRGBa> I;
  vpImageIo::read(I, "ballons.jpg");

  int h = 14, s = 255, v = 209;
  int offset = 30;
  int h_low = std::max<int>(0, h - offset), h_high = std::min<int>(h + offset, 255);
  int s_low = std::max<int>(0, s - offset), s_high = std::min<int>(s + offset, 255);
  int v_low = std::max<int>(0, v - offset), v_high = std::min<int>(v + offset, 255);
  std::vector<int> hsv_range;
  hsv_range.push_back(h_low);
  hsv_range.push_back(h_high);
  hsv_range.push_back(s_low);
  hsv_range.push_back(s_high);
  hsv_range.push_back(v_low);
  hsv_range.push_back(v_high);
 
#if (VISP_CXX_STANDARD >= VISP_CXX_STANDARD_11)
  vpImage<vpHSV<unsigned char, true>> Ihsv;
  vpImageConvert::convert(I, Ihsv);

  vpImage<unsigned char> maskHSV;
  vpImageConvert::convert(I, Ihsv);
  vpImageTools::inRange(Ihsv, hsv_range, maskHSV);

  vpImage<vpRGBa> I_segmented_from_HSV;
  vpImageTools::inMask(I, maskHSV, I_segmented_from_HSV);
#endif

  unsigned int width = I.getWidth();
  unsigned int height = I.getHeight();
  vpImage<unsigned char> H(height, width);
  vpImage<unsigned char> S(height, width);
  vpImage<unsigned char> V(height, width);
 
  vpImageConvert::RGBaToHSV(reinterpret_cast<unsigned char *>(I.bitmap),
                            reinterpret_cast<unsigned char *>(H.bitmap),
                            reinterpret_cast<unsigned char *>(S.bitmap),
                            reinterpret_cast<unsigned char *>(V.bitmap), I.getSize());
 
  vpImage<unsigned char> mask(height, width);
  vpImageTools::inRange(reinterpret_cast<unsigned char *>(H.bitmap),
                        reinterpret_cast<unsigned char *>(S.bitmap),
                        reinterpret_cast<unsigned char *>(V.bitmap),
                        hsv_range,
                        reinterpret_cast<unsigned char *>(mask.bitmap),
                        mask.getSize());
 
  vpImage<vpRGBa> I_segmented(height, width);
  vpImageTools::inMask(I, mask, I_segmented);
 
#if defined(VISP_HAVE_DISPLAY)
#if (VISP_CXX_STANDARD >= VISP_CXX_STANDARD_11)
  std::shared_ptr<vpDisplay> d_I = vpDisplayFactory::createDisplay(I);
  std::shared_ptr<vpDisplay> d_mask = vpDisplayFactory::createDisplay(mask, I.getWidth()+75, 0, "HSV mask");
  std::shared_ptr<vpDisplay> d_I_segmented = vpDisplayFactory::createDisplay(I_segmented, 2*mask.getWidth()+80, 0, "Segmented frame");
  std::shared_ptr<vpDisplay> d_mask_hsv = vpDisplayFactory::createDisplay(maskHSV, I.getWidth()+75, mask.getHeight() + 80, "HSV mask using vpHSV");
  std::shared_ptr<vpDisplay> d_I_segmented_hsv = vpDisplayFactory::createDisplay(I_segmented_from_HSV, 2*mask.getWidth()+80, mask.getHeight() + 80, "Segmented frame using vpHSV");
#else
  vpDisplay *d_I = vpDisplayFactory::allocateDisplay(I, 0, 0, "Current frame");
  vpDisplay *d_mask = vpDisplayFactory::allocateDisplay(mask, I.getWidth()+75, 0, "HSV mask");
  vpDisplay *d_I_segmented = vpDisplayFactory::allocateDisplay(I_segmented, 2*mask.getWidth()+80, 0, "Segmented frame");
#endif
 
  vpDisplay::display(I);
  vpDisplay::display(mask);
  vpDisplay::display(I_segmented);
  vpDisplay::flush(I);
  vpDisplay::flush(mask);
  vpDisplay::flush(I_segmented);
#if (VISP_CXX_STANDARD >= VISP_CXX_STANDARD_11)
  vpDisplay::display(I_segmented_from_HSV);
  vpDisplay::flush(I_segmented_from_HSV);
  vpDisplay::display(maskHSV);
  vpDisplay::flush(maskHSV);
#endif
  vpDisplay::getClick(I);
 
#if (VISP_CXX_STANDARD < VISP_CXX_STANDARD_11)
  if (d_I != nullptr) {
    delete d_I;
  }
 
  if (d_mask != nullptr) {
    delete d_mask;
  }
 
  if (d_I_segmented != nullptr) {
    delete d_I_segmented;
  }
#endif
#endif
  return EXIT_SUCCESS;
}

The end of the previous snippet shows also how to display the following images.

Next tutorial

You are now ready to see how to continue with Tutorial: HSV low/high range tuner tool.