ViSP moving-edges tracker provide real-time tracking capabilities of points normal to the object contours. Such a tracker allows to track a line, an ellipse, a circle or more complex objects using model-based approaches.
Note that all the source code mentioned in this tutorial is part of ViSP source code:
or in example/tracking/ folder if you want to try the tracking on a video or a sequence of successive images.
Moving edges (ME) are points sampled along a visible contour. They are tracked along the contour normal. The following image shows an example where the contour is a line, but could be generalized to any other shape.
In black the previous contour position with sampled MEs. In blue tracked MEs.
The main parameters for moving-edges implemented in vpMe class are as follows:
the range, which is the maximum distance in pixels to be searched along both sides of the contour normal that could be set with vpMe::setRange()
the step, which is the distance in pixels between two successive MEs set with could be set with vpMe::setSampleStep()
the Likelihood normalized threshold, which is the minimum grey level threshold used to differentiate the inner part of the object from the outer part. Values are in range 0-255. To use this normalized threshold use first vpMe::setLikelihoodThresholdType() to set vpMe::NORMALIZED_THRESHOLD type, and then vpMe::setThreshold().
The following sample code shows how to set these parameters:
With ViSP you can track a line using moving edges. The following example code available in tutorial-me-line-tracker.cpp shows how to use ViSP vpMeLine class to track a line in a live stream acquired by a camera.
Note
There is also this other similar example in trackMeLine.cpp that allows to test the line tracker on a recorded video or sequence of successive images.
#include <iostream>
#include <visp3/core/vpConfig.h>
// If openCV available, priority to OpenCV capture, otherwise the user has to modify the code uncommenting/commenting
The source code is build only if one of the grabbers is available. To this end we are checking preprocessor macros defined in visp3/core/vpConfig.h header.
By default, if ViSP is build with OpenCV 3rdparty enabled, we are using OpenCV to grab the live stream from a camera. If you rather want to use a Realsense camera and grab images with vpRealsense2 class, you may edit the code to undef all the grabbers except the one that you want to use.
#undef VISP_HAVE_V4L2
#undef VISP_HAVE_DC1394
#undef VISP_HAVE_CMU1394
#undef VISP_HAVE_FLYCAPTURE
//#undef VISP_HAVE_REALSENSE2 <- If available this is the grabber that will be used
#undef HAVE_OPENCV_HIGHGUI
#undef HAVE_OPENCV_VIDEOIO
Images that are processed could be acquired from various framegrabbing devices. This is allowed by including the frame grabber headers.
#include <opencv2/videoio/videoio.hpp>// for cv::VideoCapture
#endif
To display these images we then include the header of the factory that permit to create a viewer.
#include <visp3/gui/vpDisplayFactory.h>
A graphical library, such as Graphical Display Interface (GDI) on Windows, or X11 on unix-like systems, is required in order to have a functional display.
Finally, to track a line with the moving edges, we include the header of the vpMeLine class.
#include <visp3/me/vpMeLine.h>
Here we create a gray level image container I that will contain the images acquired by our camera.
Then, we create a grabber instance, first for an usb camera under Unix if libv4l (Video 4 Linux) is available, secondly for a firewire camera under Unix if libdc1394 3rd party is available, then for a firewire camera under Windows if CMU1394 3rd party is available, next for an camera working with Flycapture SDK, then for a Realsense camera, and finally with OpenCV if none of the previous 3rd party are available. The Tutorial: Image frame grabbing gives more details concerning the framegrabbing.
int opt_device = 0; // For OpenCV and V4l2 grabber to set the camera device
We then initialize the moving edges parameters used later by the tracker with some parameters:
int opt_me_range = 10;
int opt_me_sample_step = 5;
int opt_me_threshold = 20; // Value in [0 ; 255]
From the previous position of a moving edge, we are tracking its new position along the normal of the contour with a range of 10 pixels on each side of the contour. For each pixel, along the normal we will compute the oriented convolution. The pixel that will be selected by the moving edges algorithm will be the one that has a contrast higher than 20 gray levels. Between two consecutive moving edges on the contour we keep a space of 5 pixels.
We then, create an instance of the vpMeTracker class that will track our line. We initialize the tracker with the previous moving-egdes parameters. We allow also the tracker to display additional information on the viewer overlay. The user has than to initialize the tracker on image I by clicking on two points located on the line to track.
Once the tracker is initialized, we enter in a while loop where we successively acquire a new image, display it, track the line, display the tracking results and finally flush the overlay drawings in the viewer.
Once build, enter tutorial/tracking/moving-edges/ folder and run tutorial-me-line-tracker binary with -h option to see which are the command line options.
$ cd ${VISP_WS}/visp-build/tutorial/tracking/moving-edges
The following video shows an example of results obtained when tracking a line:
This other example shows the tracking of 2 lines:
2.2. Ellipse or arc of ellipse tracking
2.2.1. Source code
With ViSP you can also track an ellipse using moving edges. The following example code available in tutorial-me-ellipse-tracker.cpp shows how to use ViSP vpMeEllipse class to this end.
Note
There is also this other similar example in trackMeEllipse.cpp that allows to test the ellipse tracker on a recorded video or sequence of successive images.
#include <iostream>
#include <visp3/core/vpConfig.h>
// If openCV available, priority to OpenCV capture, otherwise the user has to modify the code uncommenting/commenting
and in the parameters that are given to vpMeEllipse::initTracking(). The parameter opt_track_circle allows to specialize the tracker with the model of a circle. The parameter opt_track_arc allows to consider arc of ellipses or arc of circles.
2.2.2. Use case
Once build, enter tutorial/tracking/moving-edges/ folder and run tutorial-me-ellipse-tracker binary with -h option to see which are the command line options.
$ cd ${VISP_WS}/visp-build/tutorial/tracking/moving-edges
$ ./tutorial-me-ellipse-tracker -h
You can run the tutorial with default options to track an ellipse
$ ./tutorial-me-ellipse-tracker
Note
Here that the user has to initialize the tracker on image I by clicking on five points located on the ellipse to track.
The following video shows an example of results obtained when tracking an ellipse:
If you want rather to track an arc of an ellipse you may use "--track-arc" command line option like:
$ ./tutorial-me-ellipse-tracker --track-arc
The following video shows an example of results obtained when tracking an ellipse arc:
There is also this other video that shows the tracking of an ellipse:
2.3. Circle or arc of circle tracking
2.3.1. Source code
The source code tutorial-me-ellipse-tracker.cpp presented in previous section allows also to track a circle or an arc of a circle.
Warning
Even if your object is a circle, its projection in the image becomes an ellipse. It is only a circle when perspective effects are removed or when the camera plane is parallel to the object plane. That's why in the next videos the object is more or less in a plane parallel to the camera plane and we are mainly moving the object in translation.
Note
There is also this other similar example in trackMeEllipse.cpp that allows to test the circle tracker on a recorded video or sequence of successive images.
2.3.2. Use case
To track a circle, you may use "--track-circle" command line option like:
$ ./tutorial-me-ellipse-tracker --track-circle
Note
Here that the user has to initialize the tracker on image I by clicking on only 3 points located on the circle to track.
The following video shows an example of results obtained when tracking a circle:
If you want to track an arc of a circle you may also use "--track-arc" command line option like:
1.15.0