Matrox Imaging Library has an image registration tool to transform images taken from different vantage points into a unified scene, which would be impractical or impossible to achieve using a single camera. It can also align an image to a reference for subsequent inspection. The tool contends with not only translation, but also with perspective including scale.
Alignment to a reference image or to neighboring images is performed with sub-pixel accuracy and is robust to local changes in contrast and intensity. In addition, the tool can be used for super-resolution where a sharper image is created from a series of images taken from roughly the same vantage point, which is useful for dealing with mechanical vibration.
In vision applications, sometimes the scene requiring inspection is too large for the camera’s field of view (FOV). The scene can be acquired by moving a single camera or with multiple cameras. Either method results in several images – which can be assembled and “stitched” together to create a single image of the entire scene. This process is called “registration”, and is useful when:
- high resolution is needed
- physical constraints prevent the camera from being positioned so that it can capture the entire scene
- the panoramic view of a scene has to be generated
A global coordinate system is usually required for further inspection and analysis of a composed scene, but each image acquired has its own coordinate system. The registration process transforms the different coordinate systems into a single coordinate system. The calculated transformations can then be used to generate a “mosaic” of the scene. (Figure 1)
Images are always registered relative to another image’s coordinate system except one: the image whose coordinate system is set relative to the global coordinate system. Typically, the image registration and the resulting mosaic are performed this way:
1. Set the coordinate system of one of the images relative to the global coordinate system with a call to MregSetLocation(). The target parameter must be set to M_REGISTRATION_GLOBAL.
2. Set the approximate relative location of the other images with successive calls to MregSetLocation(). For each call, the target parameter specifies the reference image.
3. Set additional registration controls with MregControl() (optional). See the section “Camera position is key: good setups for registration”.
4. Call MregCalculate() to perform the registration.
5. Call MregTransformImage() to compose a mosaic.
Need for common distinctive features
The images being registered must share common areas containing meaningful and non-ambiguous pixel data. In other words, the region where the images overlap must contain some unique grayscale information that can easily be used to align the two images correctly. The quality of the content is the key here; uniform grayscale or repetitive patterns must be avoided because they can yield multiple registrations. (Figure 2)
Camera position is key: good setups for registration
Registration can be used in a setup where a single scene is imaged with a moving camera or with multiple cameras. In either case, for registration to work, the scene must be flat (or the camera must be positioned far enough away so that the scene appears to be flat like in aerial photography). Specify the transformation type with MregControl() with M_TRANSFORMATION_TYPE, but use caution:
- select M_TRANSLATION only when camera movements are translations on a plane that is parallel to the scene (this setting offers the fastest and most robust performance)
- select M_TRANSLATION_AND_ROTATION when the camera movements are translations and rotations on a plane parallel to the scene
- select the more general M_PERSPECTIVE when other kinds of camera movements are also present (e.g. tilting, zooming)
A more advanced setup allows the registration of scenes containing objects with 3D features by making use of a telecentric lens that eliminates the parallax effect. In this case, translations and rotations within a single plane are the only camera movements permitted, and the plane must be perpendicular to the optical axis of the camera. Set the M_TRANSFORMATION_TYPE to M_TRANSLATION if the camera does not rotate or M_TRANSLATION_ROTATION if it does. (Figure 3)
The not-so-good camera setup
With a standard optical lens, uneven deformations (from the center of the FOV outwards) will appear due to the parallax effect. In the dual-camera system illustrated, region R1 occupies a larger portion of the field of view of camera A than of camera B. It will therefore appear larger for camera A than for camera B. Inversely, Region R2 will appear larger for camera B than for camera A. There are no global transformations than can correct these local deformations, so it is not possible to generate identical views in the overlap region. In this case registration can not be performed. (Figure 4)
Generating the mosaic
Once the registration has been successfully performed, the image mosaic can be generated. Choose how the mosaic operation will handle the pixel intensities of the overlapping regions with MregControl() with M_MOSAIC_COMPOSITION:
- use the values of either overlapped data
- use the average value of the overlapped data
- use a progressive blend of the overlapped data
When the M_MOSAIC_COMPOSITION parameter is set, the mosaic can be performed by calling the MregTransformImage() MIL function.
In a fixed camera setup, the registration needs to be performed only once. Since the relationship between each image does not change, the same registration result can be used by MregTransformImage()to generate the mosaic of subsequent series of images.
Tips to speed-up the registration process
Set the initial approximate location as close as possible to the estimated perfect match position with MregSetLocation().
Set the maximum allowable displacement to a value that is as small as possible with MregControl() with M_LOCATION_DELTA; larger search regions increase the processing time.
Set the minimum overlap area between the two images with MregControl() with M_MIN_OVERLAP; the match will be optimized in the overlapping region only.
If subpixel accuracy is not required for the application, specify an accuracy setting with MregControl(); set M_ACCURACY to M_LOW.
Link to Matrox Imaging Library Overview:
http://www.rauscher.de/...
VISION 2010
Halle 4 - Stand 4c14
For more information please contact:
RAUSCHER
Johann-G.Gutenberg-Str. 20
D-82140 Olching
Phone +49 81 42 / 4 48 41-0
Fax +49 81 42 / 4 48 41-90
E-Mail: info@rauscher.de
www.rauscher.de
Alignment to a reference image or to neighboring images is performed with sub-pixel accuracy and is robust to local changes in contrast and intensity. In addition, the tool can be used for super-resolution where a sharper image is created from a series of images taken from roughly the same vantage point, which is useful for dealing with mechanical vibration.
In vision applications, sometimes the scene requiring inspection is too large for the camera’s field of view (FOV). The scene can be acquired by moving a single camera or with multiple cameras. Either method results in several images – which can be assembled and “stitched” together to create a single image of the entire scene. This process is called “registration”, and is useful when:
- high resolution is needed
- physical constraints prevent the camera from being positioned so that it can capture the entire scene
- the panoramic view of a scene has to be generated
A global coordinate system is usually required for further inspection and analysis of a composed scene, but each image acquired has its own coordinate system. The registration process transforms the different coordinate systems into a single coordinate system. The calculated transformations can then be used to generate a “mosaic” of the scene. (Figure 1)
Images are always registered relative to another image’s coordinate system except one: the image whose coordinate system is set relative to the global coordinate system. Typically, the image registration and the resulting mosaic are performed this way:
1. Set the coordinate system of one of the images relative to the global coordinate system with a call to MregSetLocation(). The target parameter must be set to M_REGISTRATION_GLOBAL.
2. Set the approximate relative location of the other images with successive calls to MregSetLocation(). For each call, the target parameter specifies the reference image.
3. Set additional registration controls with MregControl() (optional). See the section “Camera position is key: good setups for registration”.
4. Call MregCalculate() to perform the registration.
5. Call MregTransformImage() to compose a mosaic.
Need for common distinctive features
The images being registered must share common areas containing meaningful and non-ambiguous pixel data. In other words, the region where the images overlap must contain some unique grayscale information that can easily be used to align the two images correctly. The quality of the content is the key here; uniform grayscale or repetitive patterns must be avoided because they can yield multiple registrations. (Figure 2)
Camera position is key: good setups for registration
Registration can be used in a setup where a single scene is imaged with a moving camera or with multiple cameras. In either case, for registration to work, the scene must be flat (or the camera must be positioned far enough away so that the scene appears to be flat like in aerial photography). Specify the transformation type with MregControl() with M_TRANSFORMATION_TYPE, but use caution:
- select M_TRANSLATION only when camera movements are translations on a plane that is parallel to the scene (this setting offers the fastest and most robust performance)
- select M_TRANSLATION_AND_ROTATION when the camera movements are translations and rotations on a plane parallel to the scene
- select the more general M_PERSPECTIVE when other kinds of camera movements are also present (e.g. tilting, zooming)
A more advanced setup allows the registration of scenes containing objects with 3D features by making use of a telecentric lens that eliminates the parallax effect. In this case, translations and rotations within a single plane are the only camera movements permitted, and the plane must be perpendicular to the optical axis of the camera. Set the M_TRANSFORMATION_TYPE to M_TRANSLATION if the camera does not rotate or M_TRANSLATION_ROTATION if it does. (Figure 3)
The not-so-good camera setup
With a standard optical lens, uneven deformations (from the center of the FOV outwards) will appear due to the parallax effect. In the dual-camera system illustrated, region R1 occupies a larger portion of the field of view of camera A than of camera B. It will therefore appear larger for camera A than for camera B. Inversely, Region R2 will appear larger for camera B than for camera A. There are no global transformations than can correct these local deformations, so it is not possible to generate identical views in the overlap region. In this case registration can not be performed. (Figure 4)
Generating the mosaic
Once the registration has been successfully performed, the image mosaic can be generated. Choose how the mosaic operation will handle the pixel intensities of the overlapping regions with MregControl() with M_MOSAIC_COMPOSITION:
- use the values of either overlapped data
- use the average value of the overlapped data
- use a progressive blend of the overlapped data
When the M_MOSAIC_COMPOSITION parameter is set, the mosaic can be performed by calling the MregTransformImage() MIL function.
In a fixed camera setup, the registration needs to be performed only once. Since the relationship between each image does not change, the same registration result can be used by MregTransformImage()to generate the mosaic of subsequent series of images.
Tips to speed-up the registration process
Set the initial approximate location as close as possible to the estimated perfect match position with MregSetLocation().
Set the maximum allowable displacement to a value that is as small as possible with MregControl() with M_LOCATION_DELTA; larger search regions increase the processing time.
Set the minimum overlap area between the two images with MregControl() with M_MIN_OVERLAP; the match will be optimized in the overlapping region only.
If subpixel accuracy is not required for the application, specify an accuracy setting with MregControl(); set M_ACCURACY to M_LOW.
Link to Matrox Imaging Library Overview:
http://www.rauscher.de/...
VISION 2010
Halle 4 - Stand 4c14
For more information please contact:
RAUSCHER
Johann-G.Gutenberg-Str. 20
D-82140 Olching
Phone +49 81 42 / 4 48 41-0
Fax +49 81 42 / 4 48 41-90
E-Mail: info@rauscher.de
www.rauscher.de
