About Matrox MIL help
Availability
Available in MIL-Lite

Available in MIL

This chapter presents the help files distributed with MIL and describes how to make the most of their navigation and customizable features. This chapter is also used as a placeholder for information relevant to all the MIL help files.

  • Welcome
  • Using this help
    • Printing MIL Help
    • Customizing your options in MIL Help
    • Blue/red menu bar
    • Gray menu bar
    • Board-specific columns
    • Navigation in MIL Help
    • Combination values
  • Customizing options
  • Copyrights and acknowledgments


Welcome

See Also | Availability | Next
User Guide:
Copyrights and acknowledgments
Customizing options
Introduction
Using this help
Available in MIL-Lite

Available in MIL

Welcome to Matrox MIL help. If you are a first-time user, please refer to the Customizing options section later in this chapter. This will allow you to customize the help so that you view only information that is relevant. For information on how to use this help, see the Using this help section later in this chapter.

The MIL help files have been designed to help you develop your MIL application more efficiently and to answer your MIL questions. Information has been divided among four help files as follows:

  • For general usage of MIL functions, see the MIL User Guide.
  • For information regarding a specific MIL function, see the MIL Reference.
  • For general board-specific information, see the MIL Board-Specific Notes. Note that board-specific information regarding a specific MIL function is integrated into the function's reference topic.
  • For the most-up-to-date information regarding this release, see the MIL Readme help file. This help file is a compilation of the most important readme files that have been included on disk.

To help familiarize yourself with how MIL is actually used, see the MIL examples included on disk.

The following sources should be checked for additional information when it is needed:

  • The on-line documentation.
  • Our website at www.matrox.com/imaging.
  • The Matrox Imaging Developer's Forum at www.matrox.com/imaging/support/dev_forum/.

To view the trademark acknowledgements, see the Copyrights and acknowledgments section later in this chapter.


Using this help

See Also | Availability | Previous | Next
User Guide:
Customizing options

Reference:
MdigControl
MdigInquire
Available in MIL-Lite

Available in MIL

This section will explain how to print, customize your options, and navigate in help.

Printing MIL Help

Before printing tables, you should enable the background printing option of Microsoft Internet Explorer so that the tables are more legible. To enable this option, use the Tools Internet Options menu command of Microsoft Internet Explorer. From the presented Internet Options dialog box, select the Advanced page. In the Printing section, select Print background colors and images.

When you try to print from the Contents tab of the help file and you choose the Print the selected heading and all subtopics option, the help file might stall. If you experience this problem, close the help file and then re-open it.

Customizing your options in MIL Help

To customize the settings for the help, see the Customizing options section later in this chapter. Customizing the settings will narrow the scope of the information so that it is relevant to your needs. It allows you to specify:

  • Whether the tables in the MIL Reference should be opened or collapsed.
  • The product that you are using: MIL or MIL-Lite.
  • The boards for which you are developing.
  • Whether you are developing for a specific computer platform.

You have the ability to change the board and tables settings from within the different topics using the blue menu bar. The other settings have to be changed from the Customizing options section later in this chapter.

Blue/red menu bar

There can be up to four different buttons on the blue menu bar. The left-most button will have the name of the current help file. Clicking on the Customize help button will return you to the Customizing options section later in this chapter. The third and the fourth buttons are the Board and Tables buttons, which will appear on the bar if appropriate.

The Board button allows you to filter the help information so that it is specific to selected boards.

The Tables button allows you to globally change the display of tables in the MIL Reference so that they initially appear collapsed. By default, all the tables are fully expanded. You can then collapse or expand the tables or individual rows of tables locally. On each row, there is a blue arrowhead on the left side ( [image] or [image]). Clicking on the arrowhead will close or expand that particular row. If you click on the arrowhead in the table header, that selection will cascade through the whole table. These local changes are only temporary. If you leave the page and come back, the alterations made to the tables are not remembered and the global settings are applied.

The blue menu bar turns red when the chosen topic is not applicable to one of your global settings. For example, should you choose to investigate a function that is not applicable to the board or product that you have selected, the blue menu bar would go red. In addition, the Availability button would flash red, and then stay red. If you click on the Availability button, it will then show you what is not applicable; after this, the button turns blue.

Gray menu bar

The buttons on the gray menu bar will give you more information on the specific topic. See Also will allow you to jump to topics that are related to the current topic. Availability will let you know when the current topic is applicable. The browse buttons, however labeled, will move you to the next or previous topic.

Board-specific columns

Some parameter settings in the MIL Reference are only available when using certain Matrox imaging boards. In these cases, gray columns appear to the right of the setting descriptions, indicating the boards for which the settings are available. For example, if a certain setting is only available on Matrox Corona-II and Matrox Corona-II is associated with the letter a, then the letter a will appear in the corresponding gray column. When you hover your mouse over a letter, the name of the board to which the letter corresponds is displayed. For tables where settings are available for all boards, no board-specific columns appear.

Although it is stated if an inquire type (M...Inquire() function) is board-specific. It is not stated if a particular return value is board-specific. For board-specific information, refer to the corresponding M...Control() or M...Set...() function if the setting is board specific.

Navigation in MIL Help

You can navigate through the MIL help files using the MIL help browse buttons or using the standard four pages in the left panel of the help viewer (Contents, Index, Search, and Favorites).

The MIL help files allow you to browse through their topics (according to their order in the table of contents) using the arrow buttons on the gray menu bar in each topic.

Particularly useful, when you are uncertain about where a particular topic is discussed, is the Search page in the left panel. You can search for phrases by enclosing the phrase in quotation marks ("). You can also precisely define a search using wild card expressions (* and ?), Boolean operators (AND, OR, NOT, and NEAR), and nested expressions (using parenthesis).

Note that the help will consider a string with a period or dash in it as two consecutive words. For example, to search for X-coordinate, you would have to put it inside quotation marks for the search to find the string.

You might find it useful to search with the Match similar words option enabled, and then refine your query and search through previous results. Sort through the resulting list of topics according to their Title, Location, or Rank by clicking on the corresponding column heading.

Note that the menu command that enables the highlighting of found terms is not very intuitive. When the feature is enabled, the Options Search Highlight Off menu command is available, whereas when disabled, the Options Search Highlight On menu command is available.

MIL help can assist you in finding all information that is relevant to a specific board. Go to the Search page and type in "Fully supported on" or "Not supported on" or "See below for support on", followed by the full name of the board. For example, if you want information relevant to Matrox Corona-II, you can type in "Fully supported on Matrox Corona-II". This should bring up all the topics that are relevant to the board.

In some instances, you will do a search and not find the item on one of the selected topics. This might be because the item is specific to a different scope. For example, if you are searching for the setting M_GRAB_SAMPLING_POSITION and you have your Board option set to Matrox Orion, one of the topics that comes up is the MdigInquire() function. If you look through the topic, you won't find the setting because it is hidden; if you change the Board option to Show All, the setting will show up as a setting supported on Matrox Meteor-II /Multi-Channel. If you cannot find what you are searching for, it is a good idea to set the Board option to Show All.

In some instances, the information that you are looking for might only show up in the grey menu bar. For example, if you search for MdigGrabWait, one of the topics that comes up is the MdigControl() function. In this topic, MdigGrabWait() can only be found under See Also on the gray menu bar.

In some rare cases, found terms are not highlighted when highlighting is enabled. Press <CTRL F> to search for the term in the topic.

For more information on how to use the Microsoft Help Viewer, please click on the following HTML Help link.

Combination values

Combination values are bit-encoded constants that can or must be added to other constants. For example, if you want to allocate an image buffer to hold compressed data, you would specify M_IMAGE and add the value M_COMPRESS:

M_IMAGE + M_COMPRESS

Combination values are either optional or essential. The preamble to a combination values table describes whether or not the combination values are optional or essential. Note that essential combination values cannot be used independently. Generally, combination values are listed in a separate table, below the table in which the value is to be combined.

GRAB SAMPLING POSITION IMAGE COMPRESS IMAGE COMPRESS


Customizing options

User Guide:
Blue/red menu bar
Available in MIL-Lite

Available in MIL

Set the following options so that they reflect your development environment. This will narrow the scope of the information presented, allowing you to use this help file more efficiently. In the help file, the top menu bar will change color depending on whether the topic you are examining is relevant according to the options you have chosen. A blue bar indicates that the topic is pertinent, while a red bar means that it is not.

To save settings, click the button below :

Some of these can be changed from the blue menu bar. For more information, see the Using this help section earlier in this chapter.

The following option is used to determine whether tables are opened or collapsed by default.

Opened
Collapsed

Product:

Matrox MIL-Lite
Matrox MIL

Board:

(a) Matrox Corona-II
(b) Matrox CronosPlus
(c) Matrox Genesis
(d) Matrox Helios XA
(e) Matrox Helios XCL
(f) Matrox Iris
(g) Matrox Meteor-II /1394
(h) Matrox Meteor-II /Camera Link
(i) Matrox Meteor-II /Digital
(j) Matrox Meteor-II /Multi-Channel
(k) Matrox Meteor-II /Standard
(l) Matrox Morphis
(m) Matrox Odyssey XA
(n) Matrox Odyssey XCL
(o) Matrox Odyssey XD
(p) Matrox Orion
(q) Matrox Solios XA
(r) Matrox Solios XCL
Show All boards

Platform:

Matrox 4Sight-II/Matrox 4Sight-M
No specific platform


Copyrights and acknowledgments

Availability | Previous
Available in MIL-Lite

Available in MIL

© Copyright Matrox Electronic Systems Ltd., 2005.

All rights reserved. Limitation of Liabilities: In no event will Matrox or its suppliers be liable for any indirect, special, incidental, economic, cover or consequential damages arising out of the use of or inability to use the product, user documentation or related technical support, including without limitation, damages or costs relating to the loss of profits, business, goodwill, even if advised of the possibility of such damages. In no event will Matrox and its suppliers' liability exceed the amount paid by you, for the product.

Because some jurisdictions do not allow the exclusion or limitation of liability for consequential or incidental damages, the above limitation may not apply to you.

Disclaimer: Matrox Electronic Systems Ltd. reserves the right to make changes in specifications at any time and without notice. The information provided by this document is believed to be accurate and reliable. However, neither Matrox Electronic Systems Ltd. nor its suppliers assume any responsibility for its use; or for any infringements of patents or other rights of third parties resulting from its use. No license is granted under any patents or patent right of Matrox Electronic Systems Ltd.

Matrox® is a registered trademark of Matrox Electronic Systems Ltd.

Microsoft®, and Windows® are registered trademarks of Microsoft Corporation.

Microsoft Visual Studio™ is a trademark of Microsoft Corporation.

PC/104-Plus™ is a trademark of the PC/104 Consortium.

Camera Link® is a registered trademark of the Automated Imaging Association (AIA).

CompactPCI™ is a trademark of PCI Industrial Computer Manufacturers' Group.

Intel®, Pentium®, and MMX™ are registered trademarks of Intel Corporation.

Texas Instruments is a trademark of Texas Instruments Incorporated.

All other nationally and internationally recognized trademarks and tradenames are hereby acknowledged.


Introduction

Chapter 1: Introduction
Availability
Available in MIL-Lite

Available in MIL

This chapter presents the features of the Matrox Imaging Library package. It also explains the installation process and how to run a Matrox Imaging Library application program.

  • What is MIL and MIL-Lite?
  • MIL and MIL-Lite modules
    • Functionality available in both MIL and MIL-Lite
    • Functionality available only in MIL
  • MIL and the Intel MMX/SSE technologies
    • MMX
    • SSE/SSE2
  • System requirements
    • Supported compilers
  • Before you start
  • Installation
    • MilConfig
  • Testing the installation
    • Communicating properly
  • Compiling and linking
  • The MIL user guide
    • Examples


What is MIL and MIL-Lite?

See Also | Availability | Next
User Guide:
MIL and MIL-Lite modules
The MIL Function Developer's Toolkit
Available in MIL-Lite

Available in MIL

The Matrox Imaging Library (MIL) is a hardware-independent, modular 32-bit imaging library.

MIL has been designed for fast application development and ease of use. It has a completely transparent management system and handles physical objects (systems, digitizers, displays, and data buffers) as virtual objects, allowing for platform-independent applications. This means that a MIL application can run on any VESA-compatible VGA board or Matrox imaging board under different environments (that is, Windows 2000/XP). MIL uses the notion of systems to identify boards, and more than one board can be controlled by a single application program. MIL is capable of running solely with the Host CPU, but can take advantage of specialized acceleration Matrox hardware if it is available and is more efficient.

MIL comes in a full version, and a version that includes only core functions. The latter is referred to as MIL-Lite.

MIL-Lite allows you to grab, display, archive, transfer, and annotate images.

The full version of MIL contains all the functionality of MIL-Lite, as well as an extensive set of processing and analysis operations. The image processing operations include point-to-point, statistical, spatial filtering, morphological, geometric (warping and polar-to-rectangular transformations), and Fast Fourier transform operations. The analysis operations include: measurement, blob analysis, calibration, optical character recognition (OCR), correlated and geometric pattern matching, edge finding, and code read/write operations.

MIL can:

  • Grab up to 16-bit grayscale images, or color images.
  • Process 1, 8, 16, and 32-bit integer or floating point images.
  • Process color images depending on the operation. Each band of a color image is processed individually, one after the other. Statistical and analysis operations do not support color processing.
  • Display 1, 8, or 16-bit grayscale or color images (if the platform supports it).

If the available MIL or MIL-Lite operations do not provide the required functionality or do not make use of some board-specific feature, you can use the MIL Developer's Toolkit to directly access your target computer's driver functions through native mode and/or to create your own pseudo-MIL functions. Note, although entering native mode can be useful, you should be aware that the resulting application will not be portable to other Matrox boards supported by the MIL package. For more information on the MIL Developer's Toolkit, see Chapter 25: The MIL Function Developer's Toolkit.

The MIL help file refers to both versions of MIL as simply MIL, unless it is necessary to distinguish between them. Note that the Availability button of each function description indicates whether or not it is supported by MIL-Lite.

For more information about the differences between MIL and MIL-Lite, see the MIL and MIL-Lite modules section later in this chapter.


MIL and MIL-Lite modules

Availability | Previous | Next
Available in MIL-Lite

Available in MIL

The following is a summary of MIL and MIL-Lite functionality.

Functionality available in both MIL and MIL-Lite

MIL and MIL-Lite share core functionality, including application and system control, image acquisition and archiving, and graphic annotations.

Images can be loaded from disk or acquired from the wide range of supported input devices (if hardware permits) and can be stored in your board's storage area. Sequences of images can also be loaded and saved in AVI format. A Bayer filter is also supported to allow you to grab images with cameras using Bayer filters, and then convert the images into 3-band color or single-band monochrome images.

MIL and MIL-Lite allow you to compress and decompress images. MIL supports both lossy and lossless JPEG and JPEG2000 compression algorithms. MIL-Lite supports these algorithms if specialized compression Matrox hardware is available or if you have the compression/decompression runtime license package installed.

You can annotate or alter images using the basic graphics tools in MIL and MIL-Lite. MIL and MIL-Lite have functions to write text, as well as basic graphics functions to draw rectangles, arcs, lines, and dots. MIL also has functions to draw analysis results; these functions are part of the analysis modules themselves.

Functionality available only in MIL

The full version of MIL provides processing and analysis functionality.

The MIL image processing module allows you to smooth, accentuate, qualify, or modify selected features of an image. The module can perform point-to-point, statistical, spatial filtering, morphological, and Fast Fourier transform operations, as well as geometric operations which include warping and polar-to-rectangular transformations.

The MIL blob analysis module allows you to identify and measure connected regions (commonly known as blobs) within an image. The blob analysis module can measure a wide assortment of blob features, such as the blob area, perimeter, Feret diameter at a given angle, minimum bounding box, and compactness. It can also be used to perform some image processing operations, such as reconstructing or eliminating blobs.

The MIL measurement module allows you to find sets of image characteristics or "markers" in an image, based on differences in pixel intensities. Upon finding a marker, the module returns the marker's spatial reference position and measures such features as its width and angle. The module can also take measurements between two markers.

MIL provides two modules with pattern recognition capabilities, for flexible solutions to machine vision problems such as alignment, measurement, and inspection of objects. The pattern is referred to as a model.

The MIL Geometric Model Finder module provides pattern matching using geometric features. It employs an algorithmic approach which finds models using edge-based geometric features instead of a pixel-to-pixel correlation. Its capabilities include:

  • Finding models through a range of angles and scales.
  • Finding of occluded models.
  • Greater tolerance of adverse lighting conditions.
  • Results that provide more detailed information about the nature of the occurrence.

The MIL pattern matching module provides normalized grayscale correlation (NGC) pattern matching. Its capabilities include finding:

  • The number of occurrences of a model in a target image.
  • Occurrences of a model at any angle, in the target image.

The MIL Edge Finder module provides powerful operations which allow you to extract geometric primitives in images, such as object contours or thin curvilinear shapes. It also provides edge analysis capabilities and can measure a wide assortment of edge features, such as edge length and Feret diameter.

MIL provides two modules with character recognition capabilities: one which is template-based, and another which is feature-based.

The MIL optical character recognition (OCR) module is template-based and provides a powerful function set for reading and verifying character strings in images, providing results such as quality scores and validity flags. The module is especially designed to operate on character strings in degraded images, with up to 90° of rotation in the target string as well as on clean images with no rotation. The OCR module supports multiple strings and non-uniform spacing between characters. It is not as robust as the MIL String Reader module, though in special cases it can be faster.

The MIL String Reader module is feature-based and provides a set of powerful functions that allow you to perform robust and fast character recognition. Strings, and their associated fonts, can be quickly defined from either an image, or your system (such as Arial or Times New Roman). The read operation can then be called, and will often yield accurate results without having to alter many of the module's default values. Being feature-based, String Reader is invariant to changes in scale, aspect ratio, and contrast. It also offers considerable tolerance towards variations in perspective and angle. In addition, String Reader supports multiple user-defined grammar rules and multi-font definition in a single context, making applications simple to write and maintain.

The MIL code module allows you to read and write 2D codes, as well as several types of bar codes.

The MIL calibration module consists of a set of functions that allow you to map pixel coordinates to real-world coordinates. This mapping can be used to get and/or calculate results from other MIL modules in real-world units. The mapping can also be used to physically correct an image's distortions. Calibration mappings can compensate for non-uniform aspect ratio, rotation, perspective foreshortening, and other more complex distortions.


MIL and the Intel MMX/SSE technologies

Availability | Previous | Next
Available in MIL-Lite

Available in MIL

MIL's processing operations have been optimized, in assembly language, to take advantage of Intel MMX acceleration and Streaming SIMD Extensions (SSE/SSE2).

MMX

Intel MMX Technology, an extension to the Intel architecture, is designed specifically to accelerate multimedia (and multimedia-like) applications. Intel MMX Technology is built to handle computation-intensive algorithms that perform repetitive operations on small data types (such as 8-bit pixels). The technology covers several areas, such as basic arithmetic operations, logical operations, shift operations, comparison operations, and data transfer instructions. These instructions use a SIMD model that allows the processor to perform a single calculation simultaneously on 2, 4, or 8 data elements by packing multiple operands (8-bit, 16-bit, or 32-bit values) into a single 64-bit register and performing processing functions on them in parallel. On a x86 compatible processor with Intel MMX Technology, MIL operations can execute, typically, 4 times faster than on a regular x86 processor. Some operations benefit even more from the MMX acceleration (for example, a thinning operation can be up to 16 times faster).

SSE/SSE2

Streaming SIMD extensions accelerate performance of floating point operations and include additional integer and cacheability instructions that significantly enhance performance.


System requirements

Availability | Previous | Next
Available in MIL-Lite

Available in MIL

MIL is available as a set of DLLs under Windows 2000/XP.

The following system requirements should be respected to ensure that MIL operates properly.

  • Windows XP or Windows 2000.
  • Minimum of 64 Mbytes of RAM for Windows 2000 and 128 Mbytes of RAM for Windows XP. This does not include non-paged (DMA) memory needed for any of the systems.
  • Minimum of 100 Mbytes of free hard disk space to install the MIL development environment. Minimum of 40 Mbytes of free hard disk space to install the MIL run-time environment.
  • Optionally, a Matrox Imaging frame grabber.
  • Optionally, a graphics board. MIL has been optimized for Matrox graphics boards.

Note that to run MIL help, you will need Microsoft Internet Explorer version 6 or above.

Supported compilers

The MIL CD includes MIL libraries that support the Microsoft Visual C++ .NET 2003 (unmanaged) compiler under Windows 2000/XP. The CD also includes ActiveMIL ActiveX controls for Microsoft Visual Basic .NET 2003 and Microsoft Visual C++ .NET 2003 RAD tools.


Before you start

Reference:
MappAllocDefault
Available in MIL-Lite

Available in MIL

You are probably anxious to start using MIL. However, before you start, we recommend that you follow these steps:

  1. Fill out and mail in your registration card. This ensures that you are on our mailing list and will receive any information on product updates and promotions.
  2. Install MIL on your hard disk using the installation details (described later in this chapter). Upon completion, the readme.txt file, in the \Matrox Imaging\MIL\Doc directory, specifies the location of all MIL files. See the \Matrox Imaging\MIL\Doc directory for additional documentation. Note that a version of these files is available in the Help.
  3. Compile and run our sample program MappStart.exe, in the examples directory, to test the installation.
  4. Review the Default Values tab in MilConfig to make sure that the default setup configuration matches your system configuration.

    5. Note, the defaults are not automatically installed on your system; a call to MappAllocDefault() initializes the system with these defaults. For simplicity, most examples use the default system and default display buffer. Upon installation, the default image buffer is monochrome if the input device is monochrome and color if the input device is color. Note that some examples expect the default image buffer to be monochrome. As you progress in the manual, you are shown how to set up your own buffers and select other system configurations. You can then return to a given example and replace portions of the code to meet your requirements.


Installation

User Guide:
Distribution and licensing
Available in MIL-Lite

Available in MIL

In addition to your MIL CD, you will require a hardware license-key (a two-sided, 25-pin connector) for development of applications. The key allows you to code, debug, and run your applications. MIL supports hardware license-keys for either the parallel or USB ports. To redistribute your MIL applications, see Chapter 24: Distribution and licensing; although MIL-Lite comes with a royalty free run-time license, MIL does not.

To install your MIL software:

  1. Attach the development hardware license-key to the parallel or USB port of your computer, depending on which hardware license key was purchased. If another device is attached to the port, disconnect it, attach the development hardware license-key, and then attach the device's connector to the other end of the hardware license-key. Note, the device need not be turned on.
  2. Place the installation CD in an appropriate drive. The setup.exe program will run automatically.

During installation, you will be asked a number of questions, such as:

  • The drive and directory on which to install the program.
  • Your development tool.
  • Which port you are using for the development hardware license-key: the parallel or the USB port.
  • The type of Matrox hardware installed in your computer (for example, Matrox Corona-II). Note that under Windows 2000/XP, the boards have to be installed before the Matrox frame grabber drivers are installed.
  • Whether to install the MGA drivers. This will only be asked if you have a Matrox Imaging board with a display section or a Matrox graphics board, and the drivers to be installed are newer than the drivers already on your computer.
  • The digitizer and display format to define your default configuration.
  • Whether by default, displayed images should be displayed in a window on the Windows desktop or without a window on an auxiliary screen (a non-Windows desktop screen). If the answer is the latter, you will be asked for the video configuration format (VCF) to use by default. Auxiliary screens require either two graphics controllers or a DualHead graphics controller that integrates two video output controllers.
  • The amount of linear non-paged memory to reserve for MIL grab buffers. The amount of reserved linear non-paged memory also establishes the amount of remaining RAM available to your operating system.

It is important to remember that only one copy of MIL can be present on a computer at a time. When installing MIL or MIL-Lite on a computer with a more recent or equally recent version of MIL or MIL-Lite, the set-up program will not install MIL. The application can use the version of MIL already installed on the computer.

Conversely, if the version of MIL or MIL-Lite on the computer is less recent than the application's required version, a decision must be made. Either the version of MIL or MIL-Lite already on the computer must be removed before installing the newer version, otherwise the current version cannot be installed on that computer.

After installation, read the readme.txt file in the \MIL\Doc directory to determine where MIL files are located and how to compile and run the MIL examples. Note that the installation program also installs Matrox Intellicam (your frame grabber configuration program) and MilConfig (the MIL Configuration utility).

MilConfig

MilConfig (MILConfig.exe), located in your Matrox Imaging\Tools directory, provides licensing, MIL reserved non-paged memory configuration, and system information tools. For example, if you need to change the amount of reserved memory or if you change the amount of physical memory in your computer, you can change the amount of MIL reserved non-paged memory assigned or RAM available to your system at any time by running MilConfig (alternatively, you can adjust the memory by uninstalling and reinstalling MIL). Should you require technical support, use the MilConfig utility's Troubleshooting tab to generate a TXT file that contains all the necessary system information required for basic troubleshooting; this file can then be forwarded to your Matrox technical support representative.

If MIL is run without the hardware license-key, a temporary evaluation license is assigned to your computer, allowing use of MIL for 30 days. Each time you run MIL, a dialog box appears indicating the number of days until the evaluation license expires. Once this time period has elapsed, MIL will not run unless you purchase a license.

Note that MIL's 30-day evaluation license can only be installed once. Any attempt to tamper with the PC's calendar, before the date of expiry, will disable MIL. In that event, MIL can only be re-used once a license is obtained.


Testing the installation

Availability | Previous | Next
Available in MIL-Lite

Available in MIL

The MIL package includes a sample program, MAppStart.c, that allows you to test the installation process and become familiar with running a MIL application. This test program allocates a MIL application, opens communication with the default target system, displays a welcoming message, pauses, and frees the system resources.

MAppStart.c

Subsequent examples will not perform explicit error checking; instead, errors will be returned automatically to the screen.

Communicating properly

During application development, you can use MAppStart.c to ensure that the software is communicating properly with the target system. To make sure your frame grabber is working properly with your camera, use Matrox Intellicam.


Compiling and linking

Availability | Previous | Next
Available in MIL-Lite

Available in MIL

To compile a MIL application program, you must include the mil.h header file, in addition to the required standard C include files. After you have compiled your application program, you will have to link it with the appropriate libraries or import libraries for your operating system, compiler, and target board. The MIL libraries are located in the Matrox Imaging\Mil\LIB directory.

Note that MIL functions run at a slower speed during debug mode. Therefore, once you are done debugging your application, you should compile in release mode to achieve maximum processing speed.

The following is a list of MIL libraries that can be included in your program. Note that you must include mil.lib to compile your application program.

Library | Description
mil.lib | Core library (includes all the functionality available in MIL-Lite).
milblob.lib | Blob Analysis module library.
milcal.lib | Calibration module library.
milcode.lib | Code module library.
miledge.lib | Edge Finder and Analysis module library.
milim.lib | Image Processing module library.
milmeas.lib | Measurement module library.
milmod.lib | Geometric Model Finder module library.
milocr.lib | Character Recognition module library.
milpat.lib | NGC Pattern Matching module library.
milstr.lib | String Reader module library.
milvga.lib | VGA library.

Note that you need to recompile your MIL applications after a major upgrade of MIL. If it is only a minor version upgrade (for example, upgrading from MIL 7.0 to MIL 7.1), it is usually not necessary. Consult the whatsnew.txt file to check for any exceptions to this general rule.


The MIL user guide

See Also | Availability | Previous
User Guide:
Building an application
Measurements
Available in MIL-Lite

Available in MIL

This user guide has been structured to allow you to start processing your images with as little information overload as possible. After reviewing installation, as well as the required MIL header file and libraries, it is expected that you read Chapter 2: Building an application.

The next set of chapters, up to and including the chapter on the measurement module, deal with enhancement, transformation and analysis operations. It is expected that you read only the chapters that you need; for the chapters on image processing, you can read only the required sections.

The remaining chapters go into more detail about setting up and managing your images, displays and digitizers, as well as describing other topics that might be useful but not critical to getting started.

All the MIL documentation uses the words function and command interchangeably, since most of the commands in MIL are C functions. Finally, in general, Host refers to the principal CPU in one's computer, while system refers to your Matrox imaging board and its associated resources.

Descriptions of the individual functions are found in the MIL Reference.

Examples

Throughout this manual, examples have been provided to simplify concepts and get you started quickly. The source listing of most examples can be found on disk in the Matrox Imaging\Mil\Examples directory. To run and edit these examples, you can use the example launcher, ExampleLauncher.exe, located in the \Matrox Imaging\Tools directory.

Note that some examples will display images in grayscale even if the images are grabbed with a color camera. This is due to the fact that those examples perform statistical and/or analysis operations that do not support color processing.

MIL's examples offer many advantages:

  • The MappStart example is very simple, but it allows you to quickly test MIL's installation and become familiar with the basics behind running a MIL application. The MappStart example allocates an application, opens communication with the default system, displays a welcoming message, pauses, and then closes communication with the system. MappStart also demonstrates error handling capabilities.
  • The examples use your default settings which have been specified at installation; therefore, very little time must be spent configuring your programming environment. However, if at some point you want to change the default settings, or are experiencing problems running the examples, your default settings can be changed on the Default Values tab of the MilConfig utility, (MILConfig.exe). This utility is located in the \Matrox Imaging\Tools directory. You can also change the default settings by clicking the Change Defaults button in the example launcher, which will take you directly to the Default Values tab of the MilConfig utility.

Note that some systems cannot run some of the examples because they don't have the hardware capability or enough memory. You should skip these examples or modify them.


Building an application

Chapter 2: Building an application
Availability
Available in MIL-Lite

Available in MIL

This chapter shows you the basics in building an application.

  • Allocating an application and its systems
    • Mandatory allocations for an application
    • Multiple systems
  • Allocating and displaying an image buffer
    • Allocating an image buffer
      • Using the default image buffer
      • Allocating a monochrome image buffer
    • Displaying an image buffer
      • Displaying multiple buffers
  • Grabbing images
    • Continuous grabbing and adjusting your camera
    • An example
  • Error reporting


Allocating an application and its systems

See Also | Availability | Next
Reference:
MappAlloc
MappAllocDefault
MappFree
MappFreeDefault
MbufAllocColor
MbufFree
MdigAlloc
MdigFree
MdispAlloc
MdispFree
MsysAlloc
MsysFree
Available in MIL-Lite

Available in MIL

Mandatory allocations for an application

At the beginning of each application, you must:

  1. Allocate a MIL application. This creates a control and execution environment for your imaging application.
  2. Allocate a MIL system for each Matrox imaging board that you want to use. MIL systems represent and permit access to the specified Matrox Imaging board, the Host CPU and memory, and any available graphics controller. When you allocate a system, MIL opens communication channels and initializes the system (or hardware resources). Once Host communication has been established with a system, you can allocate its memory resources, display, and input capabilities.

    3. Note, systems can have many data buffers, displays, and digitizers. Processing can be done between many systems.

You can use the MappAllocDefault() macro to allocate the default application, system, image buffer, display, and digitizer. Use MappFreeDefault() to free the application, devices, and memory resources that were allocated with MappAllocDefault(), when they are no longer required.

Alternatively, you can use MappAlloc(), MsysAlloc(), MbufAllocColor(), MdispAlloc(), and MdigAlloc() to perform the above-mentioned operations, respectively. In this case, when allocated memory resources, displays, and digitizers are no longer required, free them using MbufFree(), MdispFree(), and MdigFree(), respectively. At the end of each application, free the system using MsysFree(), and then free the application using MappFree().

Note, for information about functionality and hardware limitations specific to your target system, refer to the MIL Board Specific Notes.

Multiple systems

You can allocate more than one system per application and then use their identifiers to access their devices and memory resources. Note that, unless your Matrox Imaging board supports multi-processing, two applications running at the same time cannot share a board. Therefore, in general, only one application can allocate a system for a board at a given time. For boards that support multi-processing however, different applications can allocate a system for the same board.

Any operation involving more than one system will be performed by the most appropriate one. By default, if none of these systems is more appropriate than the Host, the Host is used to perform the operation.


Allocating and displaying an image buffer

User Guide:
Color
Displaying an image
Grabbing images

Reference:
MappAllocDefault
MbufAlloc2d
MbufAllocColor
MdispAlloc
MdispSelect
Available in MIL-Lite

Available in MIL

After you have allocated an application along with any required system(s), you are ready to grab and display an image. This section covers how to allocate and display a monochrome image buffer. For the basics required to start grabbing, see the Grabbing images section later in this chapter.

Allocating an image buffer

Image buffers are storage areas that can hold image data so that it can be displayed, manipulated, grabbed, and/or analyzed. For simple operations, you will find it sufficient to use the default image buffer that can be allocated using the MappAllocDefault() macro. However, for some operations, you will need to allocate another buffer. For example, if you do not want to overwrite the source image data after an image processing operation, you will need two separate image buffers.

Using the default image buffer

You can allocate the default image buffer using the MappAllocDefault() macro. If a monochrome digitizer configuration format (DCF) is specified at installation time, or using MilConfig, then a single-band monochrome buffer is allocated as default image buffer. Else, if a color DCF is specified at installation time, or using MilConfig, then the default image buffer is defined as a multi-band color buffer. Note that all examples will run, whether the default image buffer has been set to monochrome or color. For more details on dealing with color, see Chapter 19: Color.

When allocating the default image buffer and the default display, the image buffer is given a displayable attribute and set to the same size as the allocated display (for windowed displays, the default display size is the same as that of the image capture-size specified in the DCF). This buffer is then cleared and displayed.

Refer to the MilConfig utility to view the MIL default settings.

Allocating a monochrome image buffer

You can allocate a monochrome image buffer, using MbufAlloc2d(). This function requires that you specify:

  • The system on which to allocate the buffer.
  • The image buffer's size in X and Y.
  • The depth of the buffer: 1-, 8-, 16-, or 32-bit.
  • The image buffer's data type. Signed, unsigned, and floating-point buffers are all supported by MIL.
  • The image buffer's intended use. You can allocate an image buffer to have a combination of uses. It can be used as the source or destination buffer for a processing operation (M_PROC), a buffer in which to store acquired data (M_GRAB), and/or a displayable buffer (M_DISP). This type of information determines where the buffer is allocated in physical memory.

Using MbufAllocColor(), you can also allocate a color image buffer. For more information on this type of buffer as well as other buffer attributes, see Chapter 19: Color.

Displaying an image buffer

Especially during application development, it is useful to display the image buffer that you are manipulating. You must first allocate a MIL display on the target system, using MdispAlloc() (or MappAllocDefault()). If you have allocated a displayable buffer (M_DISP), display it in this display, using MdispSelect(); you can use the same function to stop displaying it. Note, however, that the image buffer and the display should be allocated on the same system.

The following example shows you how to allocate and display an image buffer. Upon completion, it leaves the buffer contents on the display so that you can analyze it. You can modify the example and remove it from the display, upon exit, by calling MdispSelect() with M_NULL before freeing the image buffer.

MDispSelect.c

Note, if you allocated the default buffer (MappAllocDefault()), this buffer would be cleared and displayed by default.

Displaying multiple buffers

With MIL, you can also display multiple buffers. For more information, see Chapter 16: Displaying an image.

PROC GRAB DISP DISP


Grabbing images

User Guide:
Auto-focusing
Color
Grabbing with your digitizer

Reference:
MappAllocDefault
MdigAlloc
MdigControl
MdigFocus
MdigGrab
MdigGrabContinuous
MdigHalt
Available in MIL-Lite

Available in MIL

Many applications depend on the ability to grab an image for later analysis or inspection. With MIL, you use an allocated digitizer to grab from a video source (typically a video camera). To allocate a digitizer, use MdigAlloc() or MappAllocDefault(). This configures the required acquisition paths (camera interface) on the frame grabber so it can accept input from the camera. With a call to MdigGrab(), you can then grab into a grab image buffer (M_GRAB).

Allocate the grab image buffer on the same system, and of the same data format, as the digitizer. For color cameras, use color image buffers (see Chapter 19: Color).

By default, when MdigGrab() is issued, it grabs a complete frame of data. Use MdigControl() to control the number of frames or fields grabbed by MdigGrab(). To control the digitizer, see Chapter 18: Grabbing with your digitizer.

Continuous grabbing and adjusting your camera

When adjusting and focusing your camera, grabbing a single frame at a time can be tedious. MIL features a continuous grab function, MdigGrabContinuous(), that grabs image frames into the specified buffer until you issue MdigHalt(). For more information, see Chapter 18: Grabbing with your digitizer.

If your camera supports remote lens adjustment, you can use MdigFocus() to automatically adjust the lens motor of the camera to achieve optimum focus in your images. For more information, see the Auto-focusing section in Chapter 18: Grabbing with your digitizer.

An example

The following example grabs from a camera in continuous mode, and then grabs a single image from the camera.

MDigGrab.c

Later in the manual, we will discuss how to change the current digitizer, how to specify a different digitizer format, and how to allocate different types of image buffer. With that knowledge, you can create more complex applications and can modify examples located in the manual to suit your needs.

GRAB


Error reporting

See Also | Availability | Previous
User Guide:
Testing the installation

Reference:
MappControl
MappGetError
MappHookFunction
Available in MIL-Lite

Available in MIL

You can enable or disable error reporting to the Host screen, using MappControl(). By default, error reporting is enabled. If you disable error reporting, you can still determine the success of a particular function or a sequence of functions, using MappGetError(). In addition, you can assign a user-defined function to handle the event of a MIL error using MappHookFunction().

For an example that uses MappGetError() to determine if the allocation of the default application, default system, default display, and default image buffer is a success, see MAppStart.c in the Testing the installation section in Chapter 1: Introduction.


Image processing

Chapter 3: Image processing
Availability
Not available in MIL-Lite

Available in MIL

This chapter describes the steps to performing a typical application with the MIL image processing module.

  • Image processing overview
    • MIL and image processing
  • Steps to performing a typical application
    • A typical application
    • How to encode these steps
  • Image quality
    • Techniques to improve images
  • Averaging an input sequence
  • Denoising
    • Low-pass spatial linear filters
    • Rank filters
  • Erosion and dilation
    • Basic erosion
    • Basic dilation
    • An example
  • Opening and closing
  • Basic geometric transform
  • Image manipulation in general
  • Image statistics
    • Generating a histogram
    • Calculating general statistics
    • Finding the image extremes
    • Locating events
    • Counting image differences
    • Projecting an image to one dimension
  • Thresholding your images
    • Binarizing
      • Determining threshold value from histogram
    • Clipping
  • Histogram equalization
  • Enhancing and detecting edges
    • Edge enhancement
      • Sharpening edge enhancement
    • Edge detection
      • Horizontal and vertical edge detection
      • Laplacian-based edge detection
      • Gradient-based edge detection
  • Arithmetic with images
    • Combining images
    • Mapping an image
  • Distance transform
    • City Block transform
    • Chessboard transform
    • Chamfer 3-4 transform
  • Labeling


Image processing overview

Availability | Next
Not available in MIL-Lite

Available in MIL

Pictures, or images, are important sources of information for interpretation and analysis. These might be images of a building undergoing renovations, a planet's surface transmitted from a spacecraft, plant cells magnified with a microscope, or electronic circuitry. Human analysis of these images or objects presents inherent difficulties: the visual inspection process is time-consuming and subject to inconsistent interpretations and assessments. Computers, on the other hand, are ideal for performing these tasks.

In order for computers to process images, the images must be numerically represented. This process is known as image digitization.

Once images are represented digitally, computers can reliably automate the extraction of useful information through the use of digital image processing. Digital image processing performs various types of image enhancements, distortion corrections, and measurements.

MIL and image processing

MIL provides a comprehensive set of image processing operations. There are two main types of image processing operations:

  • Those that enhance or transform an image.
  • Those that analyze an image (that is, generate a numeric or graphic report that relates specific image information).

MIL supports such operations as:

  • Point-to-point operations. These operations include constant thresholding, image comparison, image subtraction, and image mapping. They compute each pixel result as a function of the pixel value at a corresponding location in either one or two source images.
  • Statistical operations. These extract statistical information from a given image, such as the minimum or maximum image pixel value or a histogram. They condense a frame of pixels into a smaller, more functional set of values for analysis.
  • Spatial filtering operations. These operations are also known as convolution. They include operations that can smooth images, enhance and detect image edges, and remove 'noise' from an image. Most of these operations compute results based on an underlying neighborhood process: the weighted sum of a pixel value and its neighbors' values.
  • Morphological operations. These operations include erosion, dilation, opening, and closing of images. They compute new values according to geometric relationships and matches to known patterns in the input image.


Steps to performing a typical application

Availability | Previous | Next
Not available in MIL-Lite

Available in MIL

We have described the broad set of operations included in the MIL image processing module. Most applications do not require all of these operations.

Image processing pertains to more than one field and no single application program can solve the problems associated with each one of these fields. Therefore, this section describes what we believe to be a typical problem, where the solution makes use of most of the supported operations. It also outlines the steps to take to implement this solution.

A typical application

In analyzing an image of a tissue sample, you might want to know the number of cell nuclei that are larger than a certain size, indicating an abnormality. This involves the following image processing steps:

  1. Grab or load an image of a magnified tissue sample.
  2. Smooth the image to remove noise produced during the grab.
  3. Binarize the image so that the cell nuclei or particles and the background have different values: represent particles in white and the background in black. This will allow you, later, to label each particle with a unique number.
  4. Perform an opening operation to remove small particles from the image.
  5. Label each particle with a unique consecutive number starting with the label 1.
  6. Calculate and read the extreme value of the image. This value also corresponds to the largest label. Since the image particles are labeled with consecutive unique numbers, the largest valued particle is also labeled with a number that corresponds to the number of particles in the image.

How to encode these steps

The following sample program (MImProcessing.c) shows you how to encode these steps, using an existing image of a tissue sample (Cell.mim).

MImProcessing.c


Image quality

Reference:
MimResize
Not available in MIL-Lite

Available in MIL

Prior to manipulating and extracting information from an image, many applications require that you obtain the best possible digital representation of it. Several factors affect the quality of an image. These include:

  • Random noise. There are two main types of random noise:
    • Gaussian noise. When this type of noise is present, the exact value of any given pixel is different for each grabbed image; this type of noise adds to or subtracts from the actual pixel value.
    • Salt-and-pepper noise (also known as impulse or shot noise). This type of noise introduces pixels of arbitrary values (usually high-frequency values) that are generally noticeable because they are completely unrelated to the neighboring pixels.

    Random noise can be caused, for example, by the camera or digitizer because electronic devices tend to generate a certain amount of noise. If the images were transmitted, the distance between the sending and the receiving devices also magnifies the random noise problem because of interference.

  • Systematic noise. Unlike random noise, this type of noise can be predicted, appearing as a group of pixels that should not be part of the actual image. This can be caused, for example, by the camera or digitizer or by uneven lighting. If the image was magnified, microscopic dust particles, on either the object or a camera lens, can appear to be part of the image.
  • Distortions. Distortions appear as geometric transforms of the actual image. These can be caused, for example, by the position of the camera relative to the object (not perpendicular), the curvature in the optical lenses, or a non-unity aspect ratio of an acquisition device.

Techniques to improve images

Most interference problems cannot be adjusted very easily at the source; therefore, preprocessing will probably be required to improve the image as much as possible, without affecting the information that you are seeking. There are several techniques that you can use to improve your image:

  • Grab the object of interest several times, averaging each image frame with the previous. This technique is generally effective on Gaussian random noise.
  • Apply a low-pass spatial filter to your image to reduce Gaussian random noise and systematic noise with small scale variations. This technique replaces each pixel with a weighted sum of its neighborhood.
  • Apply a median filter to your image to reduce salt-and-pepper noise. This technique replaces each pixel with the median pixel value of its neighborhood.
  • Perform a morphological opening operation to remove small particles and break isthmuses between objects in your image.
  • Perform a morphological closing operation to remove small holes in objects.
  • Make sure that the type of camera you allocate digitizes the image with square pixels (that is, a 1:1 aspect ratio), to reduce object-shape distortions. If this is not possible or does not correct the problem, you can resize the image, using MimResize().


Averaging an input sequence

Reference:
MimArith
MimArithMultiple
Not available in MIL-Lite

Available in MIL

An effective technique to remove random noise is to average a grabbed sequence of the same target image. For instance, Gaussian noise affects the value of any given pixel for each grabbed frame, adding to or subtracting from the actual pixel value. Therefore, over several image acquisitions, this noise averages out to zero. As a rule of thumb, Gaussian noise is generally reduced by the square root of the number of grabbed frames.

With MIL, you can average an input sequence, using either one of the following methods:

  • Adding all input frames and then dividing the result by a specified weight factor. To use this method, use MimArith().
  • Adding weighted input frames to a weighted accumulator buffer ((Iacc = aIin + (1 - a)Iacc) or (Iacc = a(Iin - Iacc) + Iacc)). To use this method, use MimArithMultiple() with M_WEIGHTED_AVERAGE.

The latter approach also acts as a temporal filter if the input is changing. This allows you to filter out moving objects from a constant background.

If you do not want to lose any frames in your sequence, you can use a method called double buffering, a technique whereby which you can grab data into one buffer while another buffer is being processed. For an example on how to implement double buffering, see MDigDoubleBuffering.c file in MIL's example directory.

WEIGHTED AVERAGE


Denoising

Reference:
MimConvolve
MimRank
Not available in MIL-Lite

Available in MIL

Spatial filtering provides an effective method to reduce noise. Spatial filtering operations determine each pixel's value based on its neighborhood values. They allow images to be separated into high-frequency and low-frequency components. There are two main types of spatial filters that can remove noise: low-pass linear filters and rank filters.

Low-pass spatial linear filters

Low-pass spatial linear filters are effective in reducing Gaussian random noise (and high-frequency systematic noise), provided that the noise frequency is not too close to the spatial frequency of significant image data. These filters replace each pixel with a weighted sum of each pixel's neighborhood. Note, these filters have the side-effect of selectively smoothing your image and removing edge information.

You can apply low-pass spatial linear filters using the MimConvolve() function. MIL provides a predefined low-pass linear filter called M_SMOOTH, which you will find satisfies most applications. If you want to control the degree of smoothness (strength of denoising) applied by the neighborhood operation, use MimConvolve() with the M_SHEN_FILTER() or M_DERICHE_FILTER() macro, set its FilterSmoothness macro parameter to the required smoothness value, and set its FilterOperation macro parameter to M_SMOOTH. The M_DERICHE_FILTER() macro applies a Canny-Deriche filter, and the M_SHEN_FILTER() macro applies a Shen-Castan filter. For the Shen-Castan filter, the neighborhoods' influence decreases much faster as the distance from the central pixel inceases, and it tends to produce more enhanced or sharpened edges, compared to the Canny-Deriche filter.

Earlier in the chapter, we looked at a cell-analysis application that determined the number of cell nuclei in a tissue sample. Since Gaussian noise is generally introduced when digitizing, we performed a smoothing operation prior to performing the analysis.

Rank filters

Rank-filter operations are more suitable for removing salt-and-pepper type noise since they replace each pixel with a pixel in its neighborhood rather than a weighted sum of its neighborhood. The weighted sum generally creates a blotchy effect around each noise pixel.

You can perform a rank-filter operation using MimRank(). In most cases, it is best to use a rank that is half of the number of elements in the neighborhood. This effectively replaces each pixel with the median of the neighborhood and is therefore called a median filter. To perform a median filter, use MimRank() with M_MEDIAN. You will find that the median filter will most often suit your application needs.

SMOOTH SHEN FILTER() DERICHE FILTER() SMOOTH DERICHE FILTER() SHEN FILTER() MEDIAN


Erosion and dilation

User Guide:
Custom morphological operations

Reference:
MimArith
MimDilate
MimErode
MimMorphic
Not available in MIL-Lite

Available in MIL

Especially during cell analysis, it can be important to know the growth stages of cell particles. Using the image processing erosion and dilation operations, you can view the possible growth stages of these particles.

  • Erosion operations peel off layers from objects or particles, removing extraneous pixels and small particles from the image.
  • Dilation operations add layers to objects or particles, enlarging any particle. Dilation can return eroded particles to their original size (but not necessarily to their exact original shape).

Erosion and dilation are neighborhood operations that determine each pixel's value according to its geometric relationship with neighborhood pixels, and as such, are part of a group of operations known as morphological operations. This section describes basic erosion and dilation performed using MimErode() and MimDilate(). Both these functions use pre-defined kernels to perform operations. For a description of advanced erosion and dilation, performed by MimMorphic(), see the Custom morphological operations section in Chapter 4: Advanced image processing.

They are also the basic operations used to perform the opening and closing operations discussed in the following section.

Note, zero pixels are considered background, while non-zero pixels are considered foreground and part of objects.

Basic erosion

You can perform a basic erosion operation on 3 by 3 neighborhoods, using MimErode().

  • If the erosion mode is set to M_BINARY, any pixel whose neighborhood is not completely white (any non-zero pixel is considered white) is changed to black (0 is considered black).
  • If the erosion mode is set to M_GRAYSCALE, each pixel is replaced with the minimum value in its neighborhood.

You can use the iteration parameter of MimErode() to perform an erosion on larger neighborhoods. Iterating the erosion is the equivalent to performing an erosion on a (1 + (2* i)) by (1 + (2* i)) neighborhood where i is the number of iterations. For example, two iterations of a 3x3 erosion is equivalent to a 5x5 erosion, and three iterations is equivalent to a 7x7 erosion.

Basic dilation

You can perform a basic dilation operation on 3 by 3 neighborhoods, using MimDilate().

  • If the dilation mode is set to M_BINARY, any pixel that has one or more white pixels (any non-zero pixel is considered white) in its neighborhood is set to white (0xff in an 8-bit image).
  • If the dilation mode is set to M_GRAYSCALE, each pixel is replaced with the maximum value in its neighborhood.

The MimDilate() function is similar to the MimErode() function in that iterating it will effectively cause a dilation on larger neighborhoods. Iterating the dilation is the equivalent to performing a dilation on a (1 + (2* i)) by (1 + (2* i)) neighborhood where i is the number of iterations. For example, two iterations of a 3x3 dilation is equivalent to a 5x5 dilation, and three iterations is equivalent to a 7x7 dilation.

An example

You can use erosion or dilation to find the perimeter of objects. Erode or dilate a binary image and `XOR' the result with the original image, using MimArith().

The following example shows how to obtain the exoskeletons of objects in an image.

MImPerimeter.c
BINARY GRAYSCALE BINARY GRAYSCALE


Opening and closing

User Guide:
Custom morphological operations

Reference:
MimClose
MimDilate
MimErode
MimOpen
Not available in MIL-Lite

Available in MIL

Another way of improving the image might be to remove, for example, small particles that have been introduced by dust, or holes in objects. These tasks can generally be accomplished with an opening or closing operation, respectively.

Opening and closing operations determine each pixel's value according to its geometric relationship with neighborhood pixels, and as such are part of a larger group of operations known as morphological operations.

Besides removing small particles, opening operations also break isthmuses or connections between touching objects. MIL provides the MimOpen() function to perform a basic opening operation on 3 by 3 neighborhoods taking all neighborhood pixels into account.

Closing operations are very useful in filling holes in objects; however in doing so, they also connect close objects, as shown below. MimClose() performs a standard 3 by 3 closing operation taking all neighborhood pixels into account.

Note, opening and closing operations work best on binary images.

Since opening is the result of eroding and then dilating an image, and closing is the result of dilating and then eroding an image, you can also customize an opening or closing operation, using MimErode() and MimDilate(). For more information on erosion and dilation, see the Custom morphological operations section in Chapter 4: Advanced image processing.


Basic geometric transform

User Guide:
Advanced image processing

Reference:
MimFlip
MimPolarTransform
MimResize
MimRotate
MimTranslate
MimWarp
Not available in MIL-Lite

Available in MIL

Image distortions can affect application results. For example, in a medical application that analyzes blood cells, if the camera does not have a one-to-one aspect ratio and no correction is performed, the cells appear distorted and elongated, and incorrect interpretations might result. Rotating such an image causes even more serious object distortion.

To resolve distortion problems, the MIL image processing module offers basic, as well as advanced, geometric functions. Since the advanced geometric functions (MimPolarTransform() and MimWarp()) are slower than the basic geometric functions, they should only be used when the required transform cannot be performed using a basic geometric function. For information on the advanced geometric functions, see Chapter 4: Advanced image processing.

The MimResize() function resizes an image along the horizontal and/or vertical axis. This can help resolve aspect-ratio problems. If both the horizontal and vertical resizing factors are set to the same value, this function can reduce or magnify an image to an appropriate size.

In some instances, the orientation of an image can also cause erroneous conclusions. When an object is rotated from its original position, you can realign it in memory by the required angle, using MimRotate().

MimTranslate() displaces an image by a specified number of pixels in the x and/or y direction, with subpixel accuracy.

MimFlip() flips an image horizontally (left to right) or vertically (top to bottom). Note that flipping horizontally allows you to get a mirror copy of the original image.

Geometric functions are performed according to a specified interpolation mode. For information on interpolation, see Chapter 4: Advanced image processing.


Image manipulation in general

Availability | Previous | Next
Not available in MIL-Lite

Available in MIL

Once you have improved your image as much as possible, you are ready to start manipulating and extracting information from it. The MIL image processing module offers you several image manipulation operations. Depending on your application, you will need to perform one operation before another in order to extract the required information. The following sections will try to help you determine this order.


Image statistics

Reference:
MimAllocResult
MimCountDifference
MimFindExtreme
MimFree
MimGetResult
MimHistogram
MimLabel
MimLocateEvent
MimProject
MimStat
Not available in MIL-Lite

Available in MIL

Many applications need to obtain some type of image statistic to condense a frame of pixels into a smaller, more functional set of values for analysis. The statistic might be required to perform some subsequent operation and/or might be used to summarize the effect of some image operation. The MIL image processing module offers a variety of functions to extract statistical information from an image. These functions allow you, for example, to:

  • Generate the intensity histogram of an image buffer (MimHistogram()).
  • Calculate a number of general statistics for all pixels that satisfy a specified condition (MimStat()).
  • Find the minimum and maximum values of an image buffer (MimFindExtreme()).
  • Find the location of certain pixel values (MimLocateEvent()).
  • Find the number of differences between two image buffers (MimCountDifference()).
  • Perform an image projection from two dimensions to one dimension (MimProject()).

Generating a histogram

A histogram is the intensity distribution of pixel values in an image and is generated by counting the number of times each pixel intensity occurs. This information is very useful for several applications. In particular, it is useful to select a threshold level when binarizing an image (discussed later) and to change the image intensity distribution when trying to increase the image contrast.

You can generate an image histogram, using MimHistogram(). This function takes an image buffer and stores the results in a previously allocated histogram result buffer. You allocate the result buffer, using MimAllocResult(), specifying its type as M_HIST_LIST. Give it enough entries to hold all possible intensities.

You can then read results, using MimGetResult(). Once results have been read from the result structure, you can release the structure, using MimFree().

The following example shows how to load an image, and calculate and draw its intensity histogram.

MImHistogram.c

You can use the MIL graphics functions to plot the histogram results on a graph, as shown below. The graphics functions (Mgra...()) are discussed later in this manual.

Calculating general statistics

You can calculate various statistics for all pixels that satisfy a specified condition in an image, using MimStat(). Setting the condition limits the number of pixels for which the statistics will be calculated. However, the condition can also be set to use all of the pixels in the image.

The types of statistics that can be calculated include the following:

  • Maximum pixel value.
  • Maximum absolute pixel value.
  • Mean pixel value.
  • Minimum pixel value.
  • Minimum absolute pixel value.
  • Number of pixels (that satisfy the condition).
  • Standard deviation value.
  • Sum of pixel values.
  • Sum of absolute pixel values.
  • Sum of squared pixel values.

Note that MimStat() stores results in a result buffer that should have been previously allocated, using MimAllocResult() with M_STAT_LIST.

Finding the image extremes

Besides using MimStat() to find image extremes, you can find the minimum and maximum pixel values of your image with MimFindExtreme(). Perhaps the most common use for finding the minimum and maximum image pixel values is to fine-tune the black and white reference levels of your frame grabber, ensuring full-range digitization. Another use for finding the maximum image pixel value is to find the number of objects in a labeled image. If all objects in an image are labeled with unique consecutive values, using MimLabel() (discussed later in this chapter), the largest label value also corresponds to the number of objects in your image.

MimFindExtreme() stores results in a result buffer that should have been previously allocated, using MimAllocResult() with M_EXTREME_LIST. You can get the resulting values, using MimGetResult(), and free the result buffer, using MimFree().

Locating events

Once you have established certain values of interest in an image, you can find the location of pixels that satisfy conditions based on these values, using MimLocateEvent(). For example, you can use MimLocateEvent() to find the location of all pixels in an image equal to the image's maximum pixel value.

Note that MimLocateEvent() stores results in a result buffer that should have been previously allocated, using MimAllocResult() with M_EVENT_LIST.

Counting image differences

You can find the number of differences between two images, using MimCountDifference(). MimCountDifference() stores results in a result buffer that should have been previously allocated, using MimAllocResult() with M_COUNT_LIST. You can get the resulting values, using MimGetResult(), and free the result buffer, using MimFree().

Projecting an image to one dimension

The MimProject() function projects an image buffer into a one dimensional buffer, generated by adding all pixel values along each diagonal in the image at the specified angle. This projection is referred to as the pixel value density of each diagonal. The 90° projection of the image is known as the row profile, and the 0° projection is known as the column profile.

The MimProject() function can perform both grayscale and binary image projections. On simple binary images, the projection is useful to detect object locations.

You allocate the result buffer, using MimAllocResult() with M_PROJ_LIST. You should define a result buffer with as many locations as there are diagonals in the image at the specified angle. You can then get the resulting values, using MimGetResult(), and free the result structure, using MimFree().

HIST LIST STAT LIST EXTREME LIST EVENT LIST COUNT LIST PROJ LIST


Thresholding your images

User Guide:
Image processing

Reference:
MimBinarize
MimClip
Not available in MIL-Lite

Available in MIL

Thresholding images means reducing each pixel to a certain range of values. Some operations can be performed more efficiently on thresholded images. Images with full grayscale levels are useful for some tasks, but have redundant information for others. The MIL package provides two thresholding methods:

  • Binarizing, using the MimBinarize() function.
  • Clipping, using the MimClip() function.

Binarizing

A binarizing operation reduces an image to two grayscale values. In general, these values are 0 and the maximum value in the image (for example, 255 if the image is 8-bit). However, in the case of a floating-point buffer, these values are 0 and 1.

Binary images are useful when trying to identify geometric patterns and objects in your image since they are not cluttered with shading information. For example, in our cell application in earlier in this chapter, we were concerned with the number of dark particles in the image and not with the actual gray levels of the dark particles. Therefore, we binarized the image to distinguish dark particles from the background.

A binarizing operation is performed by comparing each pixel value in the image against one or two specified threshold values (for example, whether each pixel value is above one of the threshold values, or within the range of the two threshold values). In general, pixels that meet the specified condition are set to the maximum value in the image while other pixels are set to 0. Except in the case of a floating-point buffer, where pixels that meet the specified conditions are set to 1 while others are set to 0.

When using MimBinarize(), it is important to select a threshold value that preserves the required information. For example, in our cell application, an incorrect threshold value might have inappropriately changed image pixels into background pixels, resulting in fewer or more particles than the number that actually exist.

Determining threshold value from histogram

MimBinarize() can automatically determine the threshold value from the source image's characteristics. Specifically, a histogram of the source image is internally generated, then the threshold value is set to the minimum value between the two most statistically important peaks in the histogram, on the assumption that these peaks represent the object and the background. If the histogram contains only two peaks, the threshold value is set to the minimum value between these peaks. If the histogram contains more than two peaks, then the threshold value will typically be set between the two principal peaks, though exceptions exist for pure black and full saturation (0, 255).

Clipping

Clipping changes the image data less dramatically than binarizing. It changes the data to include only the range of pixel values in which you are interested. MimClip() takes a condition with at most two threshold points and replaces only those pixels that meet the condition with given values. Pixels that do not meet the condition are unaffected.

This can be useful to change data from one data type to another. For example, if you have a 16-bit result, but most of the pixels are less than 256, you could clip the result into an 8-bit buffer, and set all the pixels that are too big to the largest possible value, that is, 255.


Histogram equalization

Reference:
MdigLut
MdispLut
MimHistogramEqualize
MimLutMap
Not available in MIL-Lite

Available in MIL

A histogram equalization can be performed to obtain a more uniform distribution of the grayscale values in your image. For example, if the intensity distribution of an image results in a clump in one area of the grayscale range, there might be objects that are not easily distinguished because of their similarity in color. You might want to adjust the image's intensity distribution to solve this problem by giving it a more uniform (M_UNIFORM) distribution, using MimHistogramEqualize().

The MimHistogramEqualize() function first generates a histogram of the source image buffer. The histogram and a selected density function are then used to calculate a transformation LUT. If the destination buffer is an image, the transformation LUT is applied to the source buffer to produce the destination image. If the destination buffer is a LUT, the transformation LUT is copied into the destination LUT; this LUT can then be used to enhance the source image, either permanently (using MimLutMap()) or upon display (using MdispLut()). The transformation LUT can also be applied directly to images as they are being grabbed; to do so, first associate the LUT buffer with the digitizer, using MdigLut() and then grab the image.

UNIFORM


Enhancing and detecting edges

User Guide:
Custom spatial filters
MIL Edge Finder module

Reference:
MimConvolve
MimEdgeDetect
Not available in MIL-Lite

Available in MIL

Many applications perform various edge operations on the image objects to increase the quality of the image or to limit some other operation on the image.

In general, edges can be established from intensity transitions between two or more adjacent pixels in an image. Horizontal edges are created when horizontally connected pixels have values that are different from those immediately above or below them. Vertical edges are created when vertically connected pixels have values that are different from those immediately to the left or right of them. Oblique edges are created from a combination of horizontal and vertical components.

There are three main categories of edge operations:

  • Operations that enhance edges to sharpen the image.
  • Operations that detect edges in the image.
  • Operations that extract edges in the image.

These edge operations are performed using convolutions (or neighborhood operations that replace each pixel with a weighted sum of each pixel's neighborhood). The weights applied to the neighborhood determine the type of operation that is performed. For example, certain weights produce a horizontal edge detection, while others produce a vertical one.

For the first two types of edge operations, the weights are specified using either a predefined or a custom Finite or Infinite Impulse Response (FIR or IIR) filter. For more information on filters, see the Custom spatial filters section in Chapter 4: Advanced image processing. The MimConvolve() function offers predefined FIR and IIR filters for most common edge enhancement and detection operations. Each offers some advantage over the others. You can try these filters to see which best suits your application needs. If you want to control the degree of smoothness (strength of denoising) applied by the neighborhood operations, use a predefined IIR filter (either the M_SHEN_FILTER() or M_DERICHE_FILTER() macro), and set its FilterSmoothness macro parameter to the required smoothness value. The M_DERICHE_FILTER() macro applies a Canny-Deriche filter, and the M_SHEN_FILTER() macro applies a Shen-Castan filter. For the Shen-Castan filter, the neighborhood's influence decreases much faster as the distance from the central pixel increases, and it tends to produce more enhanced or sharpened edges, compared to the Canny-Deriche filter.

For information on edge extraction operations, see the MIL Edge Finder module section in Chapter 9: Edge Finder.

Edge enhancement

Edge enhancement operations amplify edges, accentuating details in the image. Note that these operations might not produce good results for further processing because when you enhance edges, you also enhance noise pixels.

Sharpening edge enhancement

Sharpening edges can be useful to enhance edges in the image while preserving other parts of the image. You can obtain approximately the same result by performing a Laplacian-based edge detection operation on the image and adding the found edges to the original image.

To sharpen edges in an image, you can use MimConvolve() with the M_SHARPEN or M_SHARPEN2 predefined FIR filter. To apply a predefined IIR filter, you can also use MimConvolve() with the M_SHEN_FILTER() or M_DERICHE_FILTER() macro, and set its FilterOperation macro parameter to M_SHARPEN.

When using predefined FIR filters, the M_SHARPEN filter tends to produce more enhanced or sharpened edges than the M_SHARPEN2 filter.

Edge detection

Edge detection operations reveal intensity transitions in the image. The smoother the image, the more gradual the change in intensity, and the weaker the detection will be.

Horizontal and vertical edge detection

Detecting the horizontal and vertical edges in the image can be useful to enhance edges in a certain direction and remove those in another.

To detect the horizontal or vertical edges from an image, you can use MimConvolve() with the M_HORIZ_EDGE or M_VERT_EDGE predefined FIR filter, respectively. To apply a predefined IIR filter, you can also use MimConvolve() with the M_SHEN_FILTER() or M_DERICHE_FILTER() macro, and set its FilterOperation macro parameter to M_HORIZ_EDGE or M_VERT_EDGE.

Laplacian-based edge detection

The Laplacian-based operations place emphasis on the maximum values, or peaks, within the image. The edge representation of the resulting image generally looks very similar to the actual image.

To detect the Laplacian-based edges from an image, you can use MimConvolve() with the M_LAPLACIAN_EDGE or M_LAPLACIAN_EDGE2 predefined FIR filter. To apply a predefined IIR filter, you can also use MimConvolve() with the M_SHEN_FILTER() or M_DERICHE_FILTER() macro, and set its FilterOperation macro parameter to M_LAPLACIAN_EDGE.

When using predefined FIR filters, the M_LAPLACIAN_EDGE2 filter tends to produce more enhanced or sharpened edges than the M_LAPLACIAN_EDGE filter.

Gradient-based edge detection

When performing a gradient-based edge detection operation, edges are determined from the rate of change between pixel values in the image, without regard to the direction of the edges. The resulting image contains only positive values.

To detect the gradient-based edges from an image, you can use MimConvolve() with the M_EDGE_DETECT or M_EDGE_DETECT2 predefined FIR filter. To apply a predefined IIR filter, you can also use MimConvolve() with the M_SHEN_FILTER() or M_DERICHE_FILTER() macro, and set its FilterOperation macro parameter to M_EDGE_DETECT or M_EDGE_DETECT_SQR.

For an advanced gradient-based edge detection operation, you can use MimEdgeDetect(). This function allows you to specify the threshold value and produces a gradient intensity image and/or a gradient angle image.

SHEN FILTER() DERICHE FILTER() DERICHE FILTER() SHEN FILTER() SHARPEN SHARPEN2 SHEN FILTER() DERICHE FILTER() SHARPEN SHARPEN SHARPEN2 HORIZ EDGE VERT EDGE SHEN FILTER() DERICHE FILTER() HORIZ EDGE VERT EDGE LAPLACIAN EDGE LAPLACIAN EDGE2 SHEN FILTER() DERICHE FILTER() LAPLACIAN EDGE LAPLACIAN EDGE2 LAPLACIAN EDGE EDGE DETECT EDGE DETECT2 SHEN FILTER() DERICHE FILTER() EDGE DETECT EDGE DETECT SQR


Arithmetic with images

Reference:
MbufAlloc1d
MbufPut1d
MgenLutFunction
MgenLutRamp
MimArith
MimLutMap
Not available in MIL-Lite

Available in MIL

It is often very useful to perform arithmetic operations on images. These operations apply the specified operator on individual pixel values in a source image or on pixels at corresponding locations in two source images. These operations, whose results do not depend on neighboring values, are known as point-to-point operations.

Besides arithmetic operations, the MIL image processing module includes several other point to point operations: logical, comparative, shifting, or absolute value operations.

Combining images

You can apply most of the above point-to-point operations, using MimArith():

  • You can add, subtract, multiply, divide, AND, NAND, OR, XOR, NOR, or XNOR two images or an image and a constant.
  • You can negate, take the absolute value, NOT, or simply copy the image into the result buffer.
  • You can copy a constant to the entire result buffer.

For example, for a surveillance application, it is more efficient to extract the constant background from the grabbed image and display only changes in the image. The following example shows how this can be done.

MDigSubtract.c

Mapping an image

You can perform complex operations (such as scaling and logarithms) on an image buffer, using MimLutMap(). This function performs the operation simply by mapping the source image buffer through a specified lookup table (LUT) and storing results in the specified destination image buffer.

You allocate a LUT buffer, using MbufAlloc1d(), specifying the buffer attribute as M_LUT. You can assign mapping values to it by copying data from a Host generated buffer (for example, an array) into it, using MbufPut1d(). You can also generate data directly into a LUT buffer according to a specified function, using MgenLutFunction(). If you simply want to invert the image or set the image to a constant, you can alternatively use MgenLutRamp() to generate an inverse ramp.

LUT


Distance transform

Reference:
MimDistance
Not available in MIL-Lite

Available in MIL

You can produce a distance transform using MimDistance(). This function determines the minimum distance from each foreground (non-zero) pixel to a background (zero) pixel, and assigns this distance to the foreground pixel. It produces a type of contour mapping of an image's foreground (object) pixels.

You can calculate the minimum distance using one of three transforms:

  • City Block transform
  • Chessboard transform
  • Chamfer 3-4 transform

City Block transform

The City Block transform (M_CITY_BLOCK) determines the minimum distance using only horizontal or vertical steps. Each step counts as 1.

Chessboard transform

The Chessboard transform (M_CHESSBOARD) determines the minimum distance using horizontal, vertical, or diagonal steps. Each step counts as 1.

Chamfer 3-4 transform

The Chamfer 3-4 transform (M_CHAMFER_3_4), like the Chessboard transform, determines the minimum distance using horizontal, vertical, or diagonal steps. However, horizontal and vertical steps are counted as 3 and diagonal steps as 4. This allows the transform to better approximate the true (Euclidean) distance between two pixels. However, it requires that the destination buffer be large enough to hold a number at least three times the maximum distance from a foreground to a background pixel. For example, an 8-bit buffer (255 max) can be used for a maximum distance of 85 pixels and a 16-bit buffer (65535 max) for a maximum distance of 21845 pixels.

CITY BLOCK CHESSBOARD CHAMFER 3 4


Labeling

See Also | Availability | Previous
Reference:
MimClip
MimLabel
Not available in MIL-Lite

Available in MIL

You can label objects or particles (known as blobs) in an image with MimLabel(). Labeling is useful for several operations:

  • Identifying and distinguishing blobs.
  • Finding the area of a blob. Once a blob is labeled, you find the area by generating a histogram and noting the number of pixels associated with that label value.
  • Counting the number of blobs in the image. The label number assigned to the last blob is also the number of blobs in the image (assuming there are fewer blobs than possible labels).
  • Using the result as a source for a conditional copy to eliminate some blobs (MimClip()).

The MimLabel() function numerically identifies each blob in the specified image. Each non-zero pixel within a blob is given the same numerical value, and blobs within an image are given consecutive values.

You can specify that the operation is performed using one of two types of connectivity modes:

  • M_4_CONNECTED: If two pixels touch on the vertical or horizontal, they are considered part of the same blob.
  • M_8_CONNECTED: If two pixels touch on the vertical, horizontal, or diagonal, they are considered part of the same blob.

To distinguish between touching blobs, separate the blobs by performing an erosion operation before the labeling operation.

4 CONNECTED 8 CONNECTED


Advanced image processing

Chapter 4: Advanced image processing
Availability
Not available in MIL-Lite

Available in MIL

This chapter describes different advanced image processing techniques.

  • Advanced image processing in general
  • Custom spatial filters
    • Finite Impulse Response (FIR) filters
    • Infinite Impulse Response (IIR) filters
    • A summary
    • An example
  • Custom morphological operations
    • Defining your own structuring element
    • Erosion and dilation
      • Using standard erosion and dilation
      • An example
    • Thinning and thickening
      • Matching
      • Searching for hits or misses
  • Connectivity mapping
  • Fast Fourier Transform
    • Magnitude and phase
    • Filtering an image
  • Watershed transformations
    • Using watersheds to separate touching objects
    • Using watersheds to separate objects from their background
    • Minimum grayscale variation of a catchment basin
    • Using marker images
      • Non-labeled marker images
      • Labeled marker images
    • Style of the watershed lines
    • Exact versus straight
    • Skipping the last level
    • Filling the source
  • Polar-to-rectangular and rectangular-to-polar transforms
  • Warping
    • First-order polynomial warpings
    • Using LUTs to perform a warping
      • 3x3 matrix-defined warping
      • Perspective warping
      • First-order polynomial warpings using LUTs
    • Interpolation modes
    • Points outside the source buffer
  • Discrete Cosine Transform


Advanced image processing in general

Availability | Next
Not available in MIL-Lite

Available in MIL

Besides the image processing functions discussed in previous chapters, MIL contains more advanced image processing functions. These advanced functions, among other things, allow you to remove noise, separate objects from their background, and correct image distortions. They include neighborhood operations using custom structuring elements or kernels, frequency transforms, watershed transforms, and warpings.


Custom spatial filters

User Guide:
Buffer overscan region
Customizing the edge extraction settings

Reference:
MbufAlloc1d
MbufAlloc2d
MbufControlNeighborhood
MbufPut
MbufPut2d
MimConvolve
Not available in MIL-Lite

Available in MIL

Spatial filtering operations include operations such as smoothing, denoising, edge enhancement, and edge detection.

Spatial filtering operations compute results based on an underlying neighborhood process: the weighted sum of a pixel value and its neighbors' values. There are two types of spatial filters: Finite Impulse Response (FIR) filters and Infinite Impulse Response (IIR) filters. FIR filters operate on a finite neighborhood, while IIR filters take into account all values in an image. In MIL, you can specify either a predefined or a custom FIR or IIR spatial filter.

Finite Impulse Response (FIR) filters

For FIR filters, the weights are known as the kernel values. These kernel values determine the operation type of the spatial filter. For example, applying the following FIR filter results in a sharpening of the image:

Whereas, applying the following FIR filter smooths an image (it also increases the intensity of the image by a factor of 16, so you will need to normalize the convolution result):

When using FIR filters, the dimensions of the kernel determine the size of the neighborhood that is used in the operation. The result of the operation is stored in the destination buffer at the location corresponding to the kernel's center pixel. When the kernel has an even number of rows and/or columns, the center pixel is considered to be the top-left pixel of the central elements in the neighborhood.

Calculate the X-coordinate of the top-left pixel of the central elements, as follows:

  • If the width (SizeX) of the kernel is an odd number, the X-coordinate is (SizeX-1)/2.
  • If the width (SixeX) of the kernel is an even number, the X-coordinate is (SizeX/2)-1.

To calculate the Y-coordinate of the top-left pixel of the central elements, the same rules apply.

Regardless of the location of the center pixel, there will be some border pixels that have an incomplete neighborhood. To deal with this issue, the image buffer is overscanned. There are several types of overscan. A transparent overscan uses the parent buffer to provide the overscan pixels needed for the border calculation. If the parent buffer is not available, a mirror overscan is performed. A mirror overscan specifies that the overscan pixels will be a mirror copy of the source buffer's border pixels. A replacement overscan allows you to specify a specific value for the overscan pixel values during processing. For more information on overscan, see the Buffer overscan region section in Chapter 14: Specifying and managing your data buffers.

If the predefined FIR filters provided by MimConvolve() do not meet your requirements, you can create your own custom FIR filter. To define your own custom FIR filter:

  1. Allocate a kernel buffer, using MbufAlloc1d() or MbufAlloc2d() with M_KERNEL. The kernel size is specified when allocating the kernel buffer. Note that the kernel size can be constrained by the available resources.
  2. Load the required kernel values into this kernel buffer, using MbufPut() or MbufPut2d().
  3. If required, modify the setting of the operation control types associated with your custom filter, using MbufControlNeighborhood(). The operation control types determine how the convolution operation will be handled. You can control:
    • Whether or not the absolute value of the result is taken (M_ABSOLUTE_VALUE).
    • The division (normalization) factor to apply to the result (M_NORMALIZATION_FACTOR).
    • Whether or not to saturate the result (M_SATURATION).
    • The position of the center pixel (M_OFFSET_CENTER_X and M_OFFSET_CENTER_Y).
    • How the operation handles the borders (overscan) of the source buffer (M_OVERSCAN and M_OVERSCAN_REPLACE_VALUE). If overscan is disabled, the border pixels of the source image are not processed if additional processing time is needed.

To apply your own custom FIR filter, call the MimConvolve() function, specifying the identifier of the required kernel buffer (KernelBufId).

Infinite Impulse Response (IIR) filters

When using IIR filters, the weights are automatically determined by the type of filter, the mode of the filter, the type of operation to perform, and the degree of smoothness (strength of denoising) applied by the filter.

The type of filter determines the distribution of the neighborhoods' influence. MIL supports two types of IIR spatial filters: Canny-Deriche filter and Shen-Castan filter. For the Canny-Deriche filter, the neighborhoods' influence decreases much slower as the distance from the central pixel increases, compared to the Shen-Castan filter. For more information, see the Customizing the edge extraction settings section in Chapter 9: Edge Finder.

There are two modes in which to perform the filter: kernel mode and recursive mode. When performing an IIR filter in recursive mode, the kernel size would be theoretically infinite if this mode actually used a kernel. When performing a custom IIR filter in kernel mode, the filtering is done using a kernel approximation (FIR) of the filter. Kernel mode is not available for a predefined IIR filter. Kernel mode has been provided to take advantage of your system if it is optimized for kernel mode; kernel mode is typically not as efficient as recursive mode unless your system is optimized for kernel mode.

If the predefined IIR filters provided by MimConvolve() do not meet your requirements, you can create your own IIR filter. To define your own IIR filter:

  1. Allocate a kernel buffer, using MbufAlloc1d() or MbufAlloc2d() with M_KERNEL.
  2. Use MbufControlNeighborhood() and specify appropriate M_FILTER_TYPE, M_FILTER_MODE, M_FILTER_OPERATION, and M_FILTER_SMOOTHNESS operation control type settings.

    3. Note that the kernel size specified during allocation is ignored, unless you will be performing the filter in kernel mode. In which case, the specified kernel size can be constrained by the available resources. In addition, regardless of the filter mode, the kernel values specified in the kernel buffer are ignored; they are determined automatically.

  3. If required, modify the setting of your custom filter, using MbufControlNeighborhood(). The operation control types determine how the convolution operation will be handled. You can control:
    • Whether or not the absolute value of the result is taken (M_ABSOLUTE_VALUE).
    • The division (normalization) factor to apply to the result (M_NORMALIZATION_FACTOR).
    • Whether or not to saturate the result (M_SATURATION).

A summary

The following is a summary of spatial filtering options.

An example

The following is an example of a spatial filtering operation using a custom FIR filter with a 3 by 3 kernel.

MImConvolve.c
KERNEL ABSOLUTE VALUE NORMALIZATION FACTOR SATURATION OFFSET CENTER X OFFSET CENTER Y OVERSCAN OVERSCAN REPLACE VALUE KERNEL FILTER TYPE FILTER MODE FILTER OPERATION FILTER SMOOTHNESS ABSOLUTE VALUE NORMALIZATION FACTOR SATURATION


Custom morphological operations

User Guide:
Custom spatial filters

Reference:
MblobReconstruct
MbufAlloc2d
MbufControlNeighborhood
MbufPut
MbufPut2d
MimDilate
MimErode
MimMorphic
MimOpen
MimThick
MimThin
Not available in MIL-Lite

Available in MIL

Morphological operations are neighborhood operations that compute new values according to geometric relationships and matches of known patterns in the input image. The MimMorphic() function supports different types of morphological operations:

  • Erosion
  • Dilation
  • Thinning
  • Thickening
  • Matching
  • Hit of miss transformation

Different geometric relationships for each of these operations are specified, using a structuring element.

Defining your own structuring element

To define your own structuring element:

  1. Allocate a structuring element buffer (M_STRUCT_ELEMENT) , using MbufAlloc2d(). The dimensions of the structuring element determine the size of the neighborhood that is used in the operation. The result of the operation is stored in the destination buffer at the location that corresponds to the structuring element's center pixel. When the structuring element has an even number of rows and/or columns, the center pixel is considered to be the top-left pixel of the central elements in the neighborhood (see the Custom spatial filters section earlier in this chapter).
  2. Load the structuring element values into this buffer, using MbufPut() or MbufPut2d(). Give the structuring element values according to the morphological operation that is to be performed. For binary and some grayscale operations, the structuring element values must be 0, 1, or M_DONT_CARE (the latter means that the corresponding neighbors are not considered in the comparison). For other grayscale operations, any structuring element value can be used, including M_DONT_CARE.

    3. For custom structuring elements, you can use MbufControlNeighborhood() to control how the operation handles the borders (overscan) of the source buffer (see the Custom spatial filters section earlier in this chapter) and the position of the neighborhood's center pixel.

Erosion and dilation

Two fundamental morphological operations are erosion and dilation. These functions allow you to view the possible growth stages of an object in the foreground (non-zero pixels) of an image.

There are two versions of erosion and dilation:

  • Erosion (M_ERODE):
    • Binary erosion. If the structuring element does not match the corresponding neighborhood values exactly, the center pixel is set to zero; otherwise, it remains unchanged. In effect, binary erosion peels off layers of objects.
    • Grayscale erosion. Subtracts each structuring element value from the corresponding pixel value in the neighborhood, and then replaces the center pixel of the neighborhood with the minimum value from the resulting neighborhood values.
  • Dilation (M_DILATE):
    • Binary dilation. If any of the structuring element values match the corresponding neighborhood values, the center pixel is set to the maximum value of the buffer (e.g. 0xff for an 8-bit buffer); otherwise, it remains unchanged. In effect, binary dilation adds layers to the objects.
    • Grayscale dilation. Adds each structuring element value to the corresponding pixel value in the neighborhood, and then replaces the center pixel of the neighborhood with the maximum value from the resulting neighborhood values.

Note, in binary mode, erosion of the white pixels is the same as dilation of the black pixels.

If the processing mode is set to M_BINARY, a binary erosion or dilation is performed and all non-zero pixels are considered as 1's; otherwise, the grayscale version of these operations is performed.

Use MblobReconstruct() to perform a conditional dilation.

Using standard erosion and dilation

MIL also supports MimErode() and MimDilate(), functions specialized in performing the most standard form of erosion and dilation operation. These operations use the following structuring element when performing in binary mode:

And use the following structuring element in grayscale mode:

In other words, these functions execute a simple 3 by 3 minimum or maximum operation without adding or subtracting anything from the pixel.

For example, to perform the most standard dilation operation on a source image buffer, use MimDilate() with the processing mode set to M_BINARY, or use MimMorphic() with a 3 x 3 structuring element of ones and the processing mode set to M_BINARY. Note, in general the standard version is faster.

An example

The following example shows how to define your own structuring element. It demonstrates, on an image with rounded objects, the difference between performing the standard opening operation, MimOpen(), and performing a custom opening with a circular type structuring element. Note, the latter preserves the original shape of the objects better than the square structuring element of the standard erosion.

mimmorphic.c

Thinning and thickening

You can reduce or enlarge objects in the foreground (non-zero pixels) of an image, using operations based on a rigid match of the pixel's neighborhood and the structuring element. Using a thickening operation, you can enlarge the object and perform such operations as a convex hull. Using a thinning operation, you can reduce objects and perform such operations as finding their skeleton.

You can perform a thinning or thickening operation with a specified structuring element, using MimMorphic(). These operations are typically performed several times, using a different structuring element so that the required pattern is sought in different directions.

You can also perform standard thinning or thickening operations with MimThin() or MimThick(), respectively.

There are two versions of thinning and thickening:

  • Thinning (M_THIN).
    • Binary thinning.

      • This operation replaces the center pixel by the value zero if a pixel's neighborhood matches the structuring element exactly. However, if the neighborhood does not match, the pixel value remains unchanged.

    • Grayscale thinning.

      • if (MAX(0) < center pixel <= MIN(1)) then (center pixel = MAX(0)) else (center pixel is unchanged)

      Where MAX(0) is the maximum of all pixels in the neighborhood that correspond to zero in the structuring element, and MIN(1) is the minimum of all pixels in the neighborhood that correspond to one in the structuring element.

  • Thickening (M_THICK).
    • Binary thickening.

      • This operation replaces the center pixel by the maximum possible value of the buffer (for example, 0xff for an 8-bit buffer) if the pixel's neighborhood matches the structuring element exactly. However, if the neighborhood does not match, the pixel value remains unchanged.

    • Grayscale thickening.

      • if (MAX(0) <= center pixel < MIN(1)) then (center pixel = MIN(1)) else (center pixel is unchanged)

      Where MAX(0) is the maximum of all pixels in the neighborhood that correspond to zero in the structuring element, or MIN(1) is the minimum of all pixels in the neighborhood that correspond to one in the structuring element.

Both versions of thinning and thickening take structuring elements containing only 0's, 1's, and M_DONT_CARE values.

If the processing mode is set to M_BINARY, a binary thinning or thickening is performed, otherwise the grayscale version of these operations is performed.

Matching

Matching allows you to determine the degree of similarity between certain areas of the image and a pattern (specified by a structuring element). The operation takes a binary or grayscale source image and produces a corresponding grayscale image, wherein the value of each pixel is equal to the total number of matches between the neighborhood of the source image's corresponding pixel and the structuring element values.

Searching for hits or misses

You can determine which pixels have neighborhoods that match a pattern exactly by performing a 'hit or miss' operation. When the neighborhood of a source image's pixel matches the pattern exactly, the value of the corresponding pixel in the destination image is the maximum value of the buffer (e.g. 0xff for an 8-bit buffer). Except in the case of a float buffer, where, if an exact match is found, the result is 1. When the neighborhood does not match exactly, the pixel value is zero.

STRUCT ELEMENT ERODE DILATE BINARY BINARY BINARY THIN THICK BINARY


Connectivity mapping

Reference:
MimConnectMap
Not available in MIL-Lite

Available in MIL

In some cases, an image must undergo several passes with different structuring elements. This can be very time-consuming. To perform such operations more efficiently, you should consider the connectivity (or cellular) mapping function, MimConnectMap(). This function reduces a serial operation to a parallel operation.

The MimConnectMap() function calculates a connectivity code for each pixel in a binary source image and then maps these codes through the specified LUT buffer.

The connectivity code is obtained by linking the elements of a pixel's 3x3 neighborhood into a string, forming a single 9-bit number. Neighborhood pixels are linked in the following order:

The pixels are connected and mapped as follows:

Program the LUT with values that would result if the required structuring elements were applied. As each connectivity code has 9 bits, you should supply a LUT buffer with at least 512 (2 to the power of 9) entries.


Fast Fourier Transform

Reference:
MimTransform
Not available in MIL-Lite

Available in MIL

A Fast Fourier Transform (FFT) is used to identify any consistent spatial patterns in an image (which can be caused, for example, by systematic noise). MIL can perform one or two dimensional FFTs using MimTransform(). For 1-D transforms, each row or column is treated as a 1-D signal. This method of transform separates a one dimensional signal into a set of sine and cosine waves of different frequencies. For a two-dimensional signal (image), it can be interpreted as the decomposition of an image into a set of 2-D patterns. The composition of these waves make up the original waveform.

The forward Fourier transform is defined as:

Where u and v are coordinates in the frequency domain and x and y are coordinates in the spatial domain. A forward FFT yields a real (R) and an imaginary (I) component of the image in a frequency domain (spectrum).

The reverse Fourier transform is defined as:

where x and y are the coordinates in the spatial domain and u and v are coordinates in the frequency domain.

Magnitude and phase

For a more visual understanding of the FFT results, you can calculate the phase and magnitude, also using the MimTransform() function. The magnitude is calculated as:

where R and I are real and imaginary components of the image, respectively. MimTransform() uses the flag M_MAGNITUDE to obtain this value.

The following figures show single-frequency images and their magnitude. Because single-frequency images contain only one spatial frequency component, their corresponding frequency images appear as a single point of brightness with their associated negative-frequency mirrors. Note that the points in the frequency domain appear in the direction of the pattern. The distance between the points and the center (DC component) represents the frequency of the pattern.

The spatial shift of each pattern in the image (in degrees) is called the phase. It is calculated using the formula atan(I/R). Use the flag M_PHASE to obtain the phase.

Filtering an image

To filter constant spatial patterns using FFTs:

  1. Perform a forward transform (M_FORWARD) calculating the magnitude (M_MAGNITUDE) of the image. Scale the image within displayable range using M_LOG_SCALE (this applies the formula, clog[1+|F(u,v)|]).
  2. Find the frequency components representing the noise and design a mask to remove these components.
  3. Once the mask is designed, perform a simple transform to obtain the real and imaginary components, this time without calculating the magnitude.
  4. Apply the mask to both the real and imaginary components of the image in the frequency domain.
  5. Finally, perform a reverse transform to obtain a filtered image.

If you know the frequency of the noise pattern and have designed the mask, you need only perform steps 3 to 5.

Following is an excerpt from the example MImFFT.c. It performs an FFT on an image with a vertical noise pattern. A forward transform is performed to obtain the real and imaginary components of the image. The values of locations corresponding to the noise pattern are set to 0. Finally, a reverse transform is performed to obtain a spatial image without the noise pattern.

MImFFT.c
MAGNITUDE PHASE FORWARD MAGNITUDE LOG SCALE


Watershed transformations

Reference:
MimArith
MimDistance
MimWatershed
Not available in MIL-Lite

Available in MIL

You can perform watershed transformations using MimWatershed(). A watershed transformation is generally used in conjunction with other processing operations to segment images, that is, to separate objects from their background and/or from each other.

To understand what a watershed transformation is, it is useful to think of an image as a topographic surface with hills and valleys. In other words, the value of each pixel represents a certain height, with the lowest pixel value (the darkest pixel) representing the point of lowest elevation and the highest pixel value (the brightest pixel) representing the point of highest elevation. A minimum in the image is defined as a pixel or a set of connected pixels that is lower in value (or elevation) than all its neighboring pixels. A maximum is a pixel or a set of connected pixels which is higher in value (elevation) than all its neighboring pixels. Pixels are connected if they are vertically, horizontally, or diagonally adjacent. A catchment basin refers to a minimum or maximum's zone of influence. For example, for a minimum, a catchment basin refers to the set of pixels from which, if a drop of water were to fall , it would eventually reach that minimum.

MimWatershed() labels an image's catchment basins and/or builds dividing lines between the catchment basins. These dividing lines are known as the watershed lines of the image. Note that catchment basins can be determined from the image's minima or its maxima.

Using watersheds to separate touching objects

You can use MimWatershed() in conjunction with MimDistance() and MimArith() to separate touching objects in a binary image.

To summarize:

  1. Perform a distance transformation on the image. This will result in a grayscale image with a maximum in each object.
  2. Perform a watershed transformation on the resulting image. Note that:
    • Catchment basins must be determined from the image's maxima rather than its minima since MimDistance() produces a maximum in each object.
    • The transform must show only watershed lines. To save time, you can prevent watershed lines from extending into the background. You can also specify that the watershed lines be straight. (These options are discussed in more detail later.)
    • You must specify the minimum grayscale variation that is required to produce a new catchment basin (this is discussed in more detail later). In general, when separating touching objects in a binary image, a low value (2) is usually sufficient.
  3. Perform an AND operation between the original image and the result of step 2, using MimArith().

Using watersheds to separate objects from their background

MimWatershed() can be used in conjunction with other processing operations to separate objects from their background. For example, if the objects have well-defined edges, an edge detection will produce a maximum along the edges of each object. These maxima will define each object as a catchment basin since they produce a minimum in each object. A watershed transformation will then label the catchment basins, effectively segmenting the image.

To summarize:

  1. Perform an edge detection on the image.
  2. Determine, through some analysis of the resulting image, the minimum grayscale variation that is required to produce a new catchment basin (this is discussed in the next section).
  3. Perform a watershed transformation on the resulting image. You must specify that catchment basins be determined from the image's minima. In addition, the final image should only show labeled catchment basins.

Minimum grayscale variation of a catchment basin

A typical image contains a lot of unwanted extrema, often due to noise. If catchment basins were determined from each extremum, the transform would segment various noise areas, resulting in over-segmentation. To prevent over-segmentation while still separating objects from their background, you can specify the minimum grayscale variation of a catchment basin, using the MinimumVariation parameter of MimWatershed().

The minimum variation of a catchment basin can be more easily understood using the topographical surface analogy. When dealing with minima, the minimum grayscale variation can be thought of as the minimum depth of the valley required to produce a catchment basin. The reverse applies when dealing with maxima.

The following shows a line profile across an object in an image (after an edge detection is performed on the image). In this case, the maximum difference between gray levels in the background (as well as within the object) is about 10, and the minimum difference between the background and the edges is about 50. In this case, therefore, the MinimumVariation parameter should be set to a value somewhere between 10 and 50, for example, 30. Note that, if it is set above 50, the object will not be separated from the background since its extrema will not produce a new catchment basin.

The default value for the MinimumVariation parameter is 1, which means that each extremum produces a catchment basin.

Using marker images

If you are able to approximate the location of your objects in an image (either through some pre-processing or through some previous knowledge of the image), you might want catchment basins determined from a separate image (known as a marker image), instead of from the extrema in the source image. Marker images are useful in preventing over-segmentation since you control not only the location of the extrema but also the number of extrema.

If you use a marker image, you can set the MinimumVariation parameter to M_OFF, M_NULL or 0 , since you mark off the extrema in a separate image.

There are two types of marker images: non-labeled and labeled.

Non-labeled marker images

In a non-labeled marker image, each group of touching pixels with the value zero in the marker image, known as a marker, starts a catchment basin in the corresponding area of the source image. Specifically, each marker in the marker image forces an extremum in the corresponding area of the source image. Pixels in the marker image are considered touching if they are vertically, horizontally, or diagonally adjacent, that is, if they are "8-connected".

Labeled marker images

In a labeled marker image, a marker is a set of pixels that do not have to necessarily touch and that have all the same label value (pixel intensity). Each marker starts a catchment basin in the corresponding area of the source image and each catchment basin of the destination image is assigned the label value of the marker that generated it. Note that, in a labeled marker image, a marker can touch other markers, allowing you to specify adjacent catchment basins. Valid marker label values are 1 to 2n- 2 (where n is the marker image buffer depth). Pixels with the label value of 2n- 1 are considered to be part of a "don't care" mask and are not processed, accelerating the watershed transformation. Finally, pixels with the label value of zero are interpreted as not being part of a marker. To use a labeled marker image, you must add M_LABELED_MARKER to the ControlFlag parameter.

The following is an example that performs a watershed transformation using a labeled marker image. It will separate the pills from the background.

To summarize:

  1. Perform an edge detection on the original image.
  2. Create a marker image that is able to locate your objects in the image.
    • For this example, the labeled marker image is created from the edge detected image using a series of morphology, edge detection, and blob analysis operations.
    • In this example, the almost-white boundary surrounding all the rings is one marker. This marker serves to define the background and is placed in the corresponding area of the background of the original image.
    • All the rings in the marker image are considered to be one marker. This particular marker serves to identify the pills. These rings are smaller than the pills in the original image and are located in the corresponding areas of the pills.
  3. Perform the watershed transformation on the edge detected image. You must specify that the catchment basins be determined from the image's minima. Remark how the resulting catchment basins have the same label value as the marker that generated them.

Note that catchment basins can be determined from markers in the marker image as well as from extrema in the source image. In this case, supply a marker image to MimWatershed() and also specify the minimum grayscale variation in the source image required to produce a new catchment basin.

Style of the watershed lines

Watershed lines can be 8-connected or 4-connected (set the ControlFlag parameter of MimWatershed() to M_4_CONNECTED or M_8_CONNECTED). In addition, they can be traced exactly or forced to be straight (set ControlFlag to M_REGULAR or M_STRAIGHT_WATERSHED).

8-connected watershed lines consist of pixels that are horizontally, vertically, or diagonally touching. 4-connected watershed lines consist of pixels that are just horizontally and/or vertically touching. 8-connected watershed lines can separate 4-connected blobs, that is, blobs whose pixels can touch horizontally or vertically. 4-connected watershed lines are required to separate 8-connected blobs, that is, blobs whose pixels can touch horizontally, vertically, or diagonally.

MIL's blob analysis module allows you to define blobs as either 4- or 8-connected.

Note that 4-connected watershed lines can also separate 4-connected blobs, however they cause over-separation.

Exact versus straight

For visual purposes, watershed lines can be traced exactly or forced to be straight.

Skipping the last level

When you perform MimWatershed(), you can skip the last intensity level of the transformation (by setting the ControlFlag parameter to M_SKIP_LAST_LEVEL). In other words, you can prevent an extremum's zone of influence from extending beyond Lmax - 1 (for a minimum) or Lmin - 1 (for a maximum), where Lmax is the maximum gray level in the image and Lmin is the minimum gray level. In effect, this prevents the background in the image from being processed, resulting in quicker processing times.

This option should be used when separating touching objects since, in this case, watershed lines in the background are unnecessary.

Filling the source

When you perform MimWatershed(), you can fill the catchment basins of unwanted extrema in the source image by adding M_FILL_SOURCE to the ControlFlag parameter. M_FILL_SOURCE fills the catchment basins of the unwanted extrema until the local basins become plateaus. Any subsequent watershed transformation on the source image will no longer process the unwanted extrema because they are no longer extrema.

The following images illustrate how M_FILL_SOURCE transforms the source image:

OFF LABELED MARKER 4 CONNECTED 8 CONNECTED REGULAR STRAIGHT WATERSHED SKIP LAST LEVEL FILL SOURCE FILL SOURCE FILL SOURCE


Polar-to-rectangular and rectangular-to-polar transforms

Reference:
MimPolarTransform
Not available in MIL-Lite

Available in MIL

Polar-to-rectangular and rectangular-to-polar transforms allow conversion of polar coordinates to cartesian coordinates and vice versa. With MIL, you can perform rectangular-to-polar or polar-to-rectangular transforms, using the MimPolarTransform() function.

The following is an example of a rectangular-to-polar transform. The dotted line defines the borders of the zone of interest:

The result will be mapped to the destination buffer as shown below:

For a rectangular-to-polar transform, the borders of the zone of interest are defined by specifying the center, the start and end radius, and the start and end angle in a source buffer. The function scans the specified zone from the start angle to the end angle. In our example, since the start angle is less than the end angle, the direction of the scan is counter clockwise. The increment in angle is determined by the length (in pixels) of the outside arc, calculated as follows:

The valid range of angles is from -360.0 to 360.0° and the maximum span of the angle must not exceed 360.0°. These values are then mapped to a destination buffer.

A polar-to-rectangular transform performs the reverse of the transform described above. It takes a source buffer and maps it to a destination buffer. The center, start angle, end angle, start radius, and end radius parameters are used to specify the position of the contents of the source buffer in the destination buffer.


Warping

User Guide:
Calibration

Reference:
MgenWarpParameter
MimFlip
MimPolarTransform
MimResize
MimRotate
MimTranslate
MimWarp
Not available in MIL-Lite

Available in MIL

In addition to functions which perform specific geometric transforms (MimFlip(), MimResize(), MimRotate(), MimTranslate(), and MimPolarTransform()), MIL includes a more general geometric function, MimWarp(). It can perform any of the specific transforms, as well as complex warpings. Such warpings could be used, for example, to correct geometric distortions.

MimWarp() performs a warping by first associating each pixel position of the destination buffer, (xd, yd), with a specific point (not necessarily a pixel) in the source buffer, (xs, ys). The pixel value at (xd, yd) is then determined from an interpolation around its associated source point. Destination pixels can be associated with source points through a first-order polynomial mapping or through lookup tables (LUTs).

Note that the functions which perform specific transforms are faster than MimWarp(). You should only use MimWarp() when the required transform cannot be otherwise performed.

Geometric distortions can also be resolved using the calibration module. For more information, see Chapter 5: Calibration.

A warping example, MImWarp.c, can be found in your examples directory.

First-order polynomial warpings

A first-order polynomial warping is equivalent to linearly translating, rotating, resizing, and/or shearing an image. First-order polynomial warpings are performed by associating points in the source buffer with pixels in the destination buffer according to the following equations:

xs = a0xd + a1yd + a2

ys = b0xd + b1yd + b2

The coefficients (a0...a2, b0...b2) required to produce a first-order polynomial warping can be automatically generated using MgenWarpParameter() or can be user-supplied. When using MgenWarpParameter(), you specify how you want the warping performed (for example, by how much you want to rotate and resize an image); the function then generates the coefficients required to produce such a warping.

To combine coefficients, you need to use separate calls to MgenWarpParameter(). For example, to generate coefficients for a rotation and translation, you need to call MgenWarpParameter() twice, using the output buffer of the first call as the input buffer of the second call. After all coefficients are generated, pass the coefficient buffer to MimWarp().

Using LUTs to perform a warping

When you perform a warping using LUTs, xs is determined from (xd, yd) through one LUT and ys is determined from (xd, yd) through another LUT. In other words,

xs = LUTx[xd, yd]

ys = LUTy[xd, yd]

Since xs and ys can be arbitrarily mapped, you can perform any type of warping when using LUTs.

The LUTs that you pass to MimWarp() can be user-supplied or, for a 3x3 matrix-defined warping, can be automatically generated using MgenWarpParameter().

3x3 matrix-defined warping

A 3x3 matrix-defined warping is performed by associating each pixel position of the destination buffer, (xd, yd), with a specific point in the source buffer, (xs, ys), according to the following equation:

To perform a 3x3 matrix-defined warping, supply the 3x3 coefficients (a0...a2, b0...b2, c0...c2) to MgenWarpParameter(), which will generate the LUTs required by MimWarp().

Perspective warping

A 3x3 matrix-defined warping can produce perspective transformations that map an arbitrary quadrilateral onto a rectangle or that map a rectangle onto an arbitrary quadrilateral.

To produce such a perspective transformation, specify the coordinates of the above points; MgenWarpParameter() will generate the required 3x3 coefficients (a0...a2, b0...b2, c0...c2). You then call MgenWarpParameter() again, having it generate the LUTs from the 3x3 coefficients. Alternatively, if you do not need to save the 3x3 coefficients, you can have the LUTs generated on the first call to MgenWarpParameter().

After the LUTs are generated, pass them to MimWarp().

First-order polynomial warpings using LUTs

If c0 and c1 are set to 0 in the equation for a 3x3 matrix-defined warping and c2 is set to 1, the equation reduces to a first-order polynomial warping. Therefore, you could perform a first-order polynomial warping by having MgenWarpParameter() generate the LUTs from the (a0...a2, b0...b2, 0 0 1) coefficients, then passing the LUTs to MimWarp(). Depending on your system, this might be faster.

Interpolation modes

When you perform a warping, pixel positions in the destination buffer, (xd, yd), get associated with specific points in the source buffer, (xs, ys). The destination coordinates have integer values but the source coordinates, in general, do not. Therefore, the pixel value at (xd, yd) has to be determined from several source pixels that are near (xs, ys), according to a specified interpolation mode.

The following interpolation modes are available:

  • Nearest-neighbor. This mode determines the nearest value to a point, and copies that value into its associated position.
  • Bilinear. This mode takes a weighted average of the four pixels nearest to the point, and copies that average into its associated position. The pixels closest to the point are given the most weight.
  • Bicubic. This mode takes a weighted average of the sixteen pixels nearest to the point, and copies that average into its associated position. Again, the pixels closest to the point are given the most weight.

In general, nearest-neighbor interpolation is the fastest to perform, and bicubic interpolation is the slowest. However, nearest-neighbor interpolation produces the least accurate results, and bicubic interpolation produces the most accurate. Bilinear interpolation is often the best compromise between speed and accuracy.

Points outside the source buffer

Sometimes, the point associated with a destination pixel will fall outside the source buffer. In such cases, the new value for the destination pixel can be determined in one of the following ways:

  • You can use pixels from the source buffer's ancestor buffer. If the source buffer is not a child buffer or if the point falls outside the ancestor buffer, the destination pixel will be left as is.
  • You can just leave the destination pixel as is.
  • You can set the destination pixel to 0.

In general, you should use pixels from the source buffer's ancestor buffer when the source buffer is a child buffer. This will ensure that the pixels you use are related to the source buffer. If the source buffer is not a child buffer, use one of the other options.

Note that you can set the destination pixel to a value other than 0 by first clearing the destination buffer to that value.


Discrete Cosine Transform

See Also | Availability | Previous
Reference:
MimTransform
Not available in MIL-Lite

Available in MIL

Discrete Cosine Transform (DCT) is mainly used for image JPEG lossy compression. MIL can perform DCT using MimTransform(). For a one dimensional signal, this method separates the signal into a set of cosine waves of different frequency. For a two-dimensional signal (image), it can be interpreted as the decomposition of an image into a set of 2D cosine patterns. The composition of these waves make up the original waveform. The forward DCT is defined as (for an 8x8 matrix):

where u and v are coordinates in the frequency domain,

The reverse DCT is defined as:

where x and y are coordinates in the spatial domain.

Frequency 0, also called the DC component, is plotted in the top-left corner of the spectrum. All other components in the spectrum are called AC components. A DCT concentrates the low frequency components of the image in the first few coefficients (top left-corner) of the spectrum. MIL divides the image into independent blocks of 8x8 pixels and performs the transform on each individual block. Centering of the spectrum is not supported in MIL.


Calibration

Chapter 5: Calibration
Availability
Not available in MIL-Lite

Available in MIL

This chapter describes how to use MIL's calibration module.

  • Calibration - overview
    • Types of distortions
  • Steps to getting results in real-world units
    • Transforming coordinates or results
    • Physically correcting an image
    • Automatically getting results in real-world units
      • Calibrated image
      • Calibrated digitizer
      • Associating a calibration object to image versus digitizer
  • Calibrating your imaging setup
    • Real-world grid
    • List of coordinates
    • Calibration modes
      • Piecewise linear interpolation
      • Perspective transformation
  • Coordinate systems and camera position
    • Image coordinate system
    • Absolute world coordinate system
    • Camera position
    • Relative world coordinate system
  • Multiple fields of view
    • Single camera fixed on a manipulator: Relative camera position example
    • Single camera and moveable object: Relative coordinate system example
    • Several cameras and fixed object: Relative coordinate system example
  • Processing calibrated images


Calibration - overview

Availability | Next
Not available in MIL-Lite

Available in MIL

MIL's calibration module (Mcal...()) consists of a set of functions that allow you to map pixel coordinates to real-world coordinates. This mapping can be used to get results from other MIL modules in real-world units. The mapping can also be used to physically correct an image's distortions.

By getting results in real-world units, you automatically compensate for any distortions in an image. Therefore, you can get accurate results despite an image's distortions.

Defining the pixel-to-world mapping is known as calibration. A calibration object is used to hold the defined mapping, as well as certain control settings.

Once you have created your calibration object, you can:

  • Use it to transform pixel coordinates or results to their real-world equivalents.
  • Use it to physically correct an image.
  • Use it to automatically get results from other MIL modules in real-world units. The modules that can return results in real-world units are:
    • Mblob...()
    • Mcode...()
    • Medge...()
    • Mim...()
    • Mmeas...()
    • Mmod...()
    • Mocr...()
    • Mpat...()

Note that a few results are always returned in pixel units. If a result can be returned in either real-world or pixel units, it will be stated in the function description.

Types of distortions

You can use calibration if you have one or more of the following types of distortion:

  • Non-unity aspect ratio distortion. Present when the X and Y axis have two different scale factors. This is evident, for example, if you know that the object in your image should be round and it appears as an ellipse. This type of distortion is often a side effect of the sampling rate used by some older digitizers.
  • Rotation distortion. Present when the camera is perpendicular to the object grabbed in the image, but not aligned with the object's axes.
  • Perspective distortion. Present when the camera is not perpendicular to the object grabbed in the image. Objects that are further away from the camera appear proportionally smaller than the same size objects closer to the camera.
  • Other spatial distortions. Complex distortions, such as pin cushion and barrel-type distortions, fall in this category. These distortions can be compensated for by using a large number of small sections in the mapping function. If the number of sections used is big enough and the corresponding area covered in each is small enough, the mapping in each area can be approximated with a linear interpolation function.


Steps to getting results in real-world units

User Guide:
Processing calibrated images

Reference:
McalAlloc
McalAssociate
McalControl
McalGrid
McalList
McalTransformCoordinate
McalTransformCoordinateList
McalTransformImage
McalTransformResult
Not available in MIL-Lite

Available in MIL

To get results in real-world units:

  1. Allocate a calibration object, using McalAlloc().
  2. Calibrate your imaging setup, using either McalGrid() or McalList(). These mappings are stored with the calibration object.
  3. Do one of the following:
    • To transform pixel coordinates or results to their real-world equivalents, use McalTransformCoordinate(), McalTransformCoordinateList(), or McalTransformResult().
    • To physically correct an image, use McalTransformImage().
    • To automatically get results from other MIL modules in real-world units, associate the calibration object to an image or digitizer, using McalAssociate().

Transforming coordinates or results

You can use McalTransformCoordinate() to convert coordinates from their pixel to their world values (or vice-versa). You can also use McalTransformCoordinateList() to convert an entire list of coordinates at once.

You can use McalTransformResult() to convert a specific non-positional result (a length, angle, or area) from its pixel to its world value (or vice-versa). Note, however, that this function uses the average pixel size to perform the conversion; results will be more accurate if you first correct the image.

Physically correcting an image

You can use McalTransformImage() to physically correct and remove certain types of distortions in an image. An image that has been physically corrected using the calibration module is known as a corrected image. When a corrected image is used within a MIL module, results can be returned in real-world units. As is expected, the image features in the destination image have the same world coordinates as they had in the source image, despite the fact their pixel coordinates have changed.

For example, after calibrating the source image below, the world coordinates of the bottom-left circle are (5,6). When you correct your image, the world coordinates of the bottom-left circle in the corrected image are also (5,6), even though it is evident that the pixel coordinates are different for the bottom-left circle in the source and corrected images.

Note that images are physically corrected using a geometric warping.

By default, a cache is used to physically correct an image. The first time a calibration object is used to transform an image, this cache fills up with information relevant to the transformation. On subsequent transformations with the calibration object, the information in the cache can significantly accelerate the transform. However, if you need to save memory, you can disable this cache, using the M_TRANSFORM_CACHE setting of McalControl(). (The cache consists of two 32-bit buffers with the same size as the destination buffer of McalTransformImage()).

The information in the cache is flushed whenever the size of the source or destination buffers of McalTransformImage() changes, or whenever the angle of the relative coordinate system of the calibration object changes. Coordinate systems are discussed later in this chapter.

Automatically getting results in real-world units

To automatically get results from other MIL modules in real-world units, associate the calibration object to an image or digitizer, using McalAssociate().

To disassociate a calibration object from an image or digitizer, you also use McalAssociate().

Calibrated image

An image with an associated calibration object is known as a calibrated image. A calibrated image still appears distorted because it has not been physically corrected. However, when it is used within a MIL module, results from this module can be returned in real-world units.

Note that the calibration object gets associated to the image, not to the buffer containing the image. For information on the implications this has on processing, see the Processing calibrated images section later in this chapter.

Also note that, if you save a calibrated image to file, the calibration settings do not get saved.

Calibrated digitizer

A digitizer with an associated calibration object is known as a calibrated digitizer. When you grab an image with a calibrated digitizer, the calibration object currently associated to the digitizer gets associated to the grabbed image. Therefore, the grabbed image becomes a calibrated image.

It is recommended to associate a particular calibration object to only one digitizer and adjust the parameters (if necessary) of each subsequent calibration object so that they are in the same world-coordinate system.

Associating a calibration object to image versus digitizer

When you associate a calibration object to an image (either with a call to McalAssociate() or a grab with a calibrated digitizer), the image receives a copy of the calibration object's current relative coordinate system and current relative camera position and a reference to the calibration object for all other settings. This means that, if you change the relative coordinate system or relative camera position after association, the change will not affect the image. (The relative coordinate system and relative camera position are discussed later). If you change any other setting of the calibration object after association, the change will affect the calibrated image. When you associate a calibration object to a digitizer, the digitizer only receives a reference to the calibration object.

When a calibration object is associated to an image that contains child images, the child images are automatically calibrated. In addition, their offsets to the parent image are taken into account when returning real-world results.

When a calibrated image is used within a MIL module, results can be returned in pixel units or in real-world units. To specify whether results should be in pixel or real-world units, use the M_OUTPUT_COORDINATE_SYSTEM setting of McalControl(). By default, results are in real-world units. While results can be returned in pixel or real-world units, any value which you pass to a MIL function must be in pixel units.

A few results are always returned in pixel units. If a result can be returned in either real-world or pixel units, it will be stated in the function description.

TRANSFORM CACHE OUTPUT COORDINATE SYSTEM


Calibrating your imaging setup

Reference:
McalGrid
McalInquire
McalList
Not available in MIL-Lite

Available in MIL

To calibrate your imaging setup, you can use an image of a user-defined grid of circles or you can use a list of pixel and real-world coordinates. To use a grid, call McalGrid(). To use a list of coordinates, call McalList().

Real-world grid

McalGrid() determines the pixel-to-world mapping from an image of a user-defined grid of circles and the world description of this grid. The world description includes the number of rows and columns, as well as the center-to-center distance between these rows and columns, in real-world units.

McalGrid() can create a pixel-to-world mapping from almost any grid of circles. However, to create an accurate (subpixel) mapping, your physical grid should meet the following guidelines (at the working resolution):

  • The radius of the grid's circles should range between 6 and 10 pixels.
  • The center-to-center distance between the grid's circles should range from 18 to 32 pixels (22 pixels recommended).
  • The minimum distance between the edges of the circles should be 6 pixels.
  • The grid should be large enough to cover the area of the image from which you want real-world results (the working area).
  • The grid image should have high contrast.

By default, the circle in the top-left corner of the grid image is associated to the origin, (0, 0), of the real world coordinate system, the first column of circles is aligned with its Y-axis, and the first row of circles is aligned with its X-axis. This is because, in general, you know where that first circle is in the real-world, so you need results with respect to that position (the top-left pixel, for example, is generally not a known position in the real-world).

If necessary, you can associate the top-left circle of the grid image to a different position within the real-world coordinate system. The origin of the real-world coordinate system does not have to be within the field-of-view. Note that this offset is specified in real-world units.

You can have the positive Y-axis oriented 90° counter-clockwise with respect to the positive X-axis (by default, the calibration module assumes it is oriented 90° clockwise).

After you call McalGrid(), you can inquire about the pixel coordinates and associated real-world coordinates of each circle in the grid using McalInquire() with M_CALIBRATION_IMAGE_POINTS_X or M_CALIBRATION_IMAGE_POINTS_Y, and M_CALIBRATION_WORLD_POINTS_X or M_CALIBRATION_WORLD_POINTS_Y. This will return the coordinates of the center of the grid's circles.

List of coordinates

McalList() uses a list of pixel coordinates and their associated real-world coordinates to define the pixel-to-world mapping. The more coordinates you specify, the more accurate the mapping. McalList() can be used when you explicitly know the real-world coordinates for a given set of pixel coordinates. The specified pixel coordinates should cover the area of the image from which you want real-world coordinates (the working area).

In the case of perspective distortion, knowing the world coordinates of 4 points in the image gives sufficient information to create a mapping function. To create a good mapping for a radial distortion requires a larger number of coordinates (for example, more than 30) distributed over the image.

Calibration modes

When you use McalGrid() or McalList(), you also have to specify the calibration mode. MIL supports the following calibration modes:

  • Piecewise linear interpolation.
  • Perspective transformation.

Piecewise linear interpolation

In general, you should use the piecewise linear interpolation mode. This mode can compensate for any kind of distortion. It is very accurate for points located inside the working area. However, it is less accurate for points outside the working area. The piecewise linear interpolation mode fits a piecewise linear interpolation function to the set of image coordinates and their real-world equivalents.

Perspective transformation

The perspective transformation mode can compensate for rotation, translation, scale, and perspective distortions. For such distortions, the perspective transformation mode is accurate for points inside and outside the working area. This mode cannot compensate for non-linear distortions such as lens distortions. The perspective transformation mode best fits a global perspective transformation function to the set of image coordinates and their real-world equivalents.

CALIBRATION IMAGE POINTS X CALIBRATION IMAGE POINTS Y CALIBRATION WORLD POINTS X CALIBRATION WORLD POINTS Y


Coordinate systems and camera position

User Guide:
Multiple fields of view

Reference:
McalControl
McalRelativeOrigin
McalTransformImage
Not available in MIL-Lite

Available in MIL

By default, after calibration, real-world positional results will be given within the absolute world coordinate system, rather than the image coordinate system. You can specify the position of your camera in the absolute world coordinate system. In addition, you can get results relative to some object by moving and/or orienting the relative world coordinate system.

Image coordinate system

The image coordinate system is the coordinate system used to locate and/or measure objects in an uncalibrated image. Its unit of measure is pixels. Its origin, (0, 0), is the middle of the image's top-left pixel. Its Y-axis is aligned with the first column of pixels and its X-axis is aligned with the first row of pixels.

Absolute world coordinate system

The absolute world coordinate system is the coordinate system used to locate and/or measure objects in the real world. It is implicitly defined from the calibration points when calibrating the imaging setup. Its unit of measure is user-defined (mm, cm, inches, etc.). Calibration relates the image coordinate system to the absolute world coordinate system.

Note that, when using a grid of circles to calibrate your imaging setup, the X-axis of the absolute world coordinate system is aligned with the first row of circles, and the Y-axis is aligned with the first column of circles.

For the sake of simplicity, the absolute world coordinate system will be known as the absolute coordinate system.

Camera position

The camera position can be any arbitrary point that moves with the camera; it does not have to be the actual camera position. For example in the following diagram, the camera position is assigned not to the actual camera, but to the arm of the robotic manipulator.

The relative camera position refers to the position of your camera relative to the absolute coordinate system. Adjusting the relative camera position can be useful when analyzing an object that cannot fit in a single image. For more information, see the Multiple fields of view section later in this chapter.

The relative camera position affects positional results taken from a calibrated image, as shown below.

To adjust the relative camera position, use the M_CAMERA_POSITION_X and M_CAMERA_POSITION_Y controls of McalControl(). Note however, that the Z position of the camera cannot be changed when adjusting the relative camera position, and should be set to M_DEFAULT.

Relative world coordinate system

By default, the relative world coordinate system is aligned with the absolute coordinate system. However, to get results relative to some object, it can be moved anywhere within the absolute coordinate system and rotated by any angle. Its unit of measure is the same as the absolute coordinate system. For the sake of simplicity, the relative world coordinate system will be known as the relative coordinate system.

Note that this image is for illustrative purposes only. In general, the object should not be placed over a grid because if the grid's circles and the object are not differentiated when performing the processing operation, then erroneous results will be returned. To move and/or rotate the relative coordinate system, use McalRelativeOrigin(). Once you change the origin and/or orientation of the relative coordinate system, world coordinates will be returned in this relative coordinate system. Note that, when you physically correct an image (using McalTransformImage()), the image is transformed such that the relative coordinate system is aligned with the image coordinate system.

CAMERA POSITION X CAMERA POSITION Y DEFAULT


Multiple fields of view

Reference:
McalControl
McalGrid
McalRelativeOrigin
Not available in MIL-Lite

Available in MIL

The calibration module makes it possible to measure the length between various points on an object, even when the object is not entirely within the camera's field-of-view. The camera's field-of-view refers to the largest world region visible in an image at a given resolution.

The approach to analyzing a large object involves acquiring images of the various parts of the object, getting real-world coordinates from each image, and then calculating the distance between the coordinates. Three types of applications are considered:

  • A single camera is fixed on a manipulator and the manipulator is moved to different positions to acquire the different images.
  • A single camera is fixed to a location and the object is moved to different positions to acquire the different images.
  • Several cameras are used to acquire the different images.

Single camera fixed on a manipulator: Relative camera position example

When a single camera is fixed on a manipulator, part of the object is grabbed, the camera is moved to a different position, a different part of the object is grabbed, and the process repeats until the entire object has been grabbed. For the coordinates from each image to be in the same absolute coordinate system, the relative camera position has to be updated each time the camera is moved. To change the relative camera position, use the M_CAMERA_POSITION_X and M_CAMERA_POSITION_Y controls of McalControl().

Single camera and moveable object: Relative coordinate system example

When a single camera is fixed to a location, part of the object is grabbed, the object is moved to a different position, another part of the object is grabbed, and the process repeats until the entire object has been grabbed. For the coordinates from each image to be in the same coordinate system, the relative coordinate system must be moved each time the object is moved. To move the relative coordinate system, use McalRelativeOrigin().

Several cameras and fixed object: Relative coordinate system example

For the coordinates from each image to be in the same absolute coordinate system when several cameras are used, the calibration object used to calibrate each camera must use the same absolute coordinate system. When using McalGrid(), you must use an offset to relate the origin of each grid to the same absolute coordinate system. When using McalList(), the real-world coordinates used in each mapping must be from the same absolute coordinate system.

CAMERA POSITION X CAMERA POSITION Y


Processing calibrated images

See Also | Availability | Previous
Reference:
MbufClear
MbufLoad
MimFlip
MimPolarTransform
MimResize
MimRotate
MimTranslate
MimWarp
Not available in MIL-Lite

Available in MIL

The calibration object is associated with the image and not the buffer. This has certain implications on processing operations. Depending on the type of operation performed on the calibrated image, the destination image is associated with the following calibration object:

  • When performing point-to-point or neighborhood processing operations, the destination image is associated with the same calibration as the source image.

    • If the operation uses more than one source image, a calibration object gets associated to the destination image only if all source images have the same calibration object; otherwise, no calibration object gets associated to the destination image.

  • Geometric functions (MimFlip(), MimPolarTransform(), MimResize(), MimRotate(), MimTranslate(), and MimWarp()) always result in an uncalibrated image, even if the source image is calibrated.
  • The function MbufClear() always results in an uncalibrated image.
  • Functions for which the source data is always uncalibrated always produce uncalibrated images. These functions include, for example MbufImport...() and MbufLoad().


Blob analysis

Chapter 6: Blob analysis
Availability
Not available in MIL-Lite

Available in MIL

This chapter describes the basic steps to extract connected regions of pixels (blobs) within an image.

  • Blob analysis overview
    • MIL and blob analysis
  • Steps to performing blob analysis
  • Identifying blobs
    • Segmenting the blob image
    • Preprocessing
  • Adjusting blob analysis processing controls
    • Controlling the image lattice
    • The pixel aspect ratio
    • Setting the Blob identification mode
  • Selecting blobs
  • Making feature extractions
  • The area and perimeter
  • Dimensions
  • Determining the shape
  • Finding the blob location
  • Moments
  • Location, length and number of runs
  • Blob reconstruction
  • A simple blob analysis example


Blob analysis overview

Availability | Next
Not available in MIL-Lite

Available in MIL

Blob analysis allows you to identify connected regions of pixels within an image, then calculate selected features of those regions. The regions are commonly known as blobs. Blobs are areas of touching pixels that are in the same logical pixel state. This pixel state is called the foreground state, while the alternate state is called the background state. Typically, the background has the value zero and the foreground is everything else (although some control is generally provided to reverse the sense).

In many applications, we are interested only in blobs whose features satisfy certain criteria. Since computation is time-consuming, blob analysis is often performed as an elimination process whereby only blobs of interest are considered in further analysis. The steps involved in feature extraction are:

  1. Analyze an image and exclude or delete blobs that don't meet determined criteria.
  2. Analyze the remaining blobs to extract further features and determine their criteria.

Repeat these steps, as necessary, until you have all the blob measurement results you need.

Reducing the raw data to just a few feature measurements generally produces more comprehensible and useful results.

MIL and blob analysis

The MIL package includes a blob analysis module that can extract a wide assortment of blob features, such as the blob area, perimeter, Feret diameter at a given angle, minimum bounding box, and compactness.

MIL uses a user-specified blob identifier image to discriminate between blobs and the background. Controls are provided to allow you to specify how this identifier image is interpreted (which pixels are part of which blob). Blobs are considered to consist of either zero or non-zero pixels, depending on the foreground control setting. The non-zero pixels can either have any value or must be set to the maximum value of the buffer (for example, 0xff for an 8-bit image), depending on the identifier type (grayscale or binary). In addition, MIL provides controls to take into account such blob image information as the pixel aspect ratio.

For binary feature extractions, such as those that pertain to the overall shape of the blob, the blob identifier image is used for both identification and computation. For grayscale extractions (e.g. the mean pixel value in a blob), you must also provide a grayscale image whose pixel values will be used in computation.

With MIL, you can perform blob analysis on 1-bit, 8-bit or 16-bit unsigned buffers.


Steps to performing blob analysis

Reference:
MblobAllocFeatureList
MblobAllocResult
MblobCalculate
MblobControl
MblobGetLabel
MblobGetNumber
MblobGetResult
MblobGetResultSingle
MblobGetRuns
MblobSelect
MblobSelectFeature
MblobSelectFeret
MblobSelectMoment
Not available in MIL-Lite

Available in MIL

Although there are numerous features that can be calculated and used during the elimination or the analysis process, the following is a series of steps that you will typically perform:

  1. Grab or load an image that was captured under the best possible conditions to minimize the amount of preprocessing required.
  2. If necessary, reduce the amount of noise in the image. (Noise makes the next step more difficult.)
  3. Segment the image so that blobs are separated from the background and from each other. Typically, this involves binarizing the image so that the background is in one state (zero or non-zero) and the blob pixels are in the other state. This image is known as the blob identifier image. If you plan to perform grayscale calculations, you will need the original grayscale image as well.
  4. If necessary, preprocess the blob identifier image. If there are too many noise particles, calculation time will be increased. An opening operation (for non-zero blobs) or a closing operation (for zero blobs) will remove most of the noise particles without significantly affecting real blobs. You might also need to separate touching blobs at this stage (or they will be counted as a single blob).
  5. Allocate a buffer for blob analysis results, using MblobAllocResult().
  6. If necessary, adjust default blob analysis controls to fit your application, using MblobControl(). You can control the pixel aspect ratio, when to consider two pixels touching (along horizontal and vertical only or also along the diagonal), which values in the identifier image represent a blob (zero or non-zero), and whether or not non-zero pixels in the identifier image can have any value or must be set to the maximum value of the buffer (grayscale or binary). For example, the maximum value of an 8-bit buffer is 0xff.
  7. Allocate a feature list, using MblobAllocFeatureList(). This list is used to specify the features that should be calculated. By default, this feature list is empty; no features are selected.
  8. Calculate the required features and analyze the results. This involves the following:
    1. Adding the required features into the feature list so that they will be calculated. Typically, you will use MblobSelectFeature() to perform this operation. However, when calculating moments or Feret diameters, you might need to use the more general feature selectors, MblobSelectMoment() or MblobSelectFeret(), respectively.
    2. Specifying certain features as sorting keys. MblobSelectFeature(), MblobSelectMoment(), and MblobSelectFeret() each allow you to specify a selected feature as a sorting key. The calculated results are sorted in accordance with the sorting key selection.
    3. Calculating results for the selected features, using MblobCalculate(). For this function, you will have to specify the blob identifier image that will be used to identify the blobs and calculate binary features, and (optionally) the grayscale image that will be used to calculate grayscale features.
    4. If necessary, excluding or deleting blobs that do not meet the criteria, using MblobSelect(). Results for the excluded or deleted blobs will not be returned. Excluded blobs will be ignored in future calculations, while deleted blobs will be removed from the blob analysis result buffer altogether.
    5. Getting the number of blobs currently included, using MblobGetNumber(), and retrieving the results from the blob result buffer, using MblobGetResult(). Note, MblobGetResultSingle() obtains results for a single blob, while MblobGetLabel() and MblobGetRuns() can be used to obtain more specific results.

You can repeat this step until you obtain all required results for the blobs of interest. Note, the process of excluding or deleting unwanted blobs and then calculating more features is the preferred method if you have many unwanted blobs. If this is not the case, it is often faster to calculate all the required features for all the blobs with a single call to MblobCalculate(), and then exclude or delete unwanted blob results afterwards.


Identifying blobs

Reference:
MimBinarize
MimClip
Not available in MIL-Lite

Available in MIL

The MIL blob analysis capabilities allow you to identify and extract features of connected regions of pixels (commonly known as blobs) within an image. MIL requires a user-specified blob identifier image in order to determine which pixels belong to which blob in the original image. Blob features involving overall shape are extracted directly from the identifier image. Features that use the actual pixel values of the blob also require the original image.

The MIL blob analysis module considers touching foreground pixels in the blob identifier image to be part of the same blob. Consequently, what is easily identifiable by the human eye as several distinct but touching blobs is interpreted by MIL as a single blob. In addition, any part of a blob that is in the background pixel state, because of lighting or reflection, is considered as background during analysis.

To reduce preprocessing, the blob identifier image should be acquired under the best possible circumstances. This means ensuring that blobs do not overlap and, if possible, don't touch. It also means ensuring the best possible lighting and using a background with a gray level that is very distinct from the gray level of the blobs. If noise is a problem, you might also need to filter the image after acquisition (for example, using a median filter or a convolution with M_SMOOTH).

Segmenting the blob image

Once the best possible image is acquired and most noise is filtered out, you must separate the different blobs from the background. Segmentation can be done in two ways:

  • Binarize the image, using MimBinarize() so that background pixels are represented as zero values and blob pixels are represented as another value.
  • Clip all background pixels to zero, while retaining the original values of blob pixels, using MimClip(). This method has the advantage of not needing a separate buffer to hold the binary image, but you will not see the result of the segmentation as clearly. The first method is usually better.

If simple segmentation is not possible due to poor lighting or blobs with the same gray level as parts of the background, you must develop a segmentation algorithm appropriate to your particular image.

Preprocessing

Producing the blob identifier image frequently creates some spurious blobs or holes (for example, due to noise or lighting). Such noise blobs make it harder to interpret blob analysis results. If you have many noise blobs, you should probably preprocess the image before using it as an identifier. An opening operation (for non-zero valued blobs or holes) or a closing operation (for zero valued blobs or holes) will remove most noise without significantly affecting real features.

If blobs are touching, you might try eroding the image a few times to break them apart.

Note, preprocessing the blob identifier image might affect the accuracy of calculations because of the slight change in blob shape. If this is a problem, perform the calculations on all the blobs, including those that are actually introduced by noise, then use the results to filter out the noise. Note, however, that this method increases the memory required and might increase the calculation time.

SMOOTH


Adjusting blob analysis processing controls

Reference:
MblobControl
MimResize
MimWarp
Not available in MIL-Lite

Available in MIL

Before performing any blob analysis calculations, you should ensure the correct interpretation of the blob identifier image. Use MblobControl() to control how certain aspects of the blob identifier image are interpreted, for example:

  • Which pixel values are considered to be in the foreground (M_FOREGROUND_VALUE).
  • Whether two pixels touching at their corners are considered part of the same blob, by appropriately defining the image lattice (M_LATTICE).
  • Whether non-zero pixels can have any value or must be set to the maximum value of the buffer; for example, 0xff for an 8-bit buffer (M_IDENTIFIER_TYPE).
  • The pixel aspect ratio of the image (M_PIXEL_ASPECT_RATIO).
  • Whether to produce separate results for each blob or for groups of blobs (M_BLOB_IDENTIFICATION).
  • How many Feret angles are considered when calculating a Feret feature (M_NUMBER_OF_FERETS). Typically, the default value will be appropriate.

Controlling the image lattice

MIL represents images using a square lattice and considers adjacent pixels along the vertical or horizontal axis as touching. However, you can control whether two diagonally adjacent pixels are considered touching.

Use MblobControl() to specify how the blob identifier image lattice should be interpreted. For example, the following is considered one blob if the lattice is set to M_8_CONNECTED, but two blobs if set to M_4_CONNECTED.

The pixel aspect ratio

When acquiring an image of a scene, each pixel represents some real distance both in width and in height. Ideally, this distance is the same in both directions, producing square pixels and allowing for simple feature calculations. However, after digitization, it is quite common for a pixel to represent a different distance in each direction. The ratio of the pixel's width to its height is called the pixel aspect ratio. For example, a pixel of equal width and height has a pixel aspect ratio of 1.0.

Note that if you have a calibrated image, feature results are returned in calibrated units.

In blob analysis, the pixel aspect ratio directly affects feature extractions. For example, all circular blobs are stretched or squashed if the pixels are not exactly square. In this case, you have two alternatives:

  • You can adjust your image, using MimResize(), and then make the required blob analysis feature extractions.
  • You can have calculations take the actual aspect ratio into consideration without modifying the image, by specifying the ratio, using MblobControl(). However, any feature derived from multiple Feret diameters cannot take the pixel aspect ratio into account accurately, and you will get better results by actually resizing your image. The same is true of the general Feret diameter.

In both cases, the actual aspect ratio can be calculated using a simple procedure. Grab an image of a true circle or square and extract the M_FERET_X and M_FERET_Y features with the default pixel aspect ratio of 1.0. The relationship between these features represents the actual pixel aspect ratio to be used in calculations (M_FERET_Y / M_FERET_X).

Note, if your image has other types of distortions, you can use MimWarp() to adjust the image.

Note, all results are affected by the pixel aspect ratio, including those that are just positions within the image. For example, to mark M_BOX_X_MIN on an image with a graphics function, you must take the aspect ratio into account (in this case by dividing the returned result by the aspect ratio).

Setting the Blob identification mode

Using MblobControl(), you can control how blobs in the blob identifier image are treated during calculations. This depends on the blob identification mode setting:

  • Individually (M_INDIVIDUAL).
  • All blobs grouped together (M_WHOLE_IMAGE).
  • Different blobs with the same label grouped together (M_LABELED).

Note, this mode does not change how blobs are identified (regions of connected foreground pixels); rather, it determines whether results are combined into groups.

When using the blob analysis package, you usually want to make feature calculations on each blob. For example, if you want to find the area of each cell in a tissue image, set the blob identification mode to M_INDIVIDUAL.

Sometimes, however, you need calculations based on the entire image rather than individual blobs. For example, you might want to calculate the area of all the copper in a rock sample image. MIL simplifies your task by allowing you to treat all foreground pixels together by setting the blob identification mode to M_WHOLE_IMAGE. Blobs in an image are treated as one blob and features are calculated for this grouped blob.

Blob identification mode M_LABELED allows you to do joint calculations on blobs with the same label value. When using labeled mode, ensure that each blob in the identifier image has a uniform pixel value. This value is the M_LABEL_VALUE result for that blob, and determines the grouping of the blobs.

FOREGROUND VALUE LATTICE IDENTIFIER TYPE PIXEL ASPECT RATIO BLOB IDENTIFICATION NUMBER OF FERETS 8 CONNECTED 4 CONNECTED FERET X FERET Y FERET Y FERET X BOX X MIN INDIVIDUAL WHOLE IMAGE LABELED INDIVIDUAL WHOLE IMAGE LABELED LABEL VALUE


Selecting blobs

Reference:
MblobCalculate
MblobFill
MblobSelect
Not available in MIL-Lite

Available in MIL

Once all blobs are clearly identifiable by the blob analysis package, you are ready to perform calculations. However, in some cases, you will not want to make time-consuming feature extractions for every single blob in the blob identifier image. For example, you probably do not want to calculate features for blobs that are touching the edges of the image or that are noise artifacts. Often, you cannot preprocess these blobs out of your image without losing too much information.

The MIL blob analysis package has a function, MblobSelect(), for such cases. This allows you to select (on the basis of calculations already made) a subset of blobs for which to make further calculations and get results. This function is generally used in one of two ways:

  • If you don't have too many unwanted blobs, it is usually faster to calculate all required features for all blobs. Then, prior to getting results, use MblobSelect() to exclude or delete results obtained for blobs that do not meet your criteria.
  • If you have many unwanted blobs, you might save time and memory by first calculating, for all blobs, only those features that allow you to distinguish between relevant and unwanted blobs. Exclude from future calculations (or delete altogether from the blob analysis result buffer) blobs that do not meet your criteria, using MblobSelect(). Then, calculate all required features for remaining blobs.

If you cannot exclude or delete many blobs using the second method, use the first.

You can make as many calls as necessary to MblobSelect() and MblobCalculate() in order to arrive at the right set of results. However, you must always give the same identifier and grayscale buffers to MblobCalculate() during this procedure. If you give different buffers or change the existing buffers in any way (for example, if you use MblobFill() to erase blobs from the identifier image), all current results in the result buffer will be discarded the next time you call MblobCalculate(). In addition, all selected features will be re-calculated for all blobs in the new identifier image. This means that you will have to restart the selection procedure. If you intend to calculate grayscale features during your analysis, you must include the grayscale image before starting your calculations.


Making feature extractions

User Guide:
Annotating the displayed image non-destructively
Preparing for graphics

Reference:
MblobAllocFeatureList
MblobCalculate
MblobDraw
MblobFill
MblobGetResult
MblobLabel
MblobSelect
MblobSelectFeature
MblobSelectFeret
MblobSelectMoment
Not available in MIL-Lite

Available in MIL

The MIL blob analysis module can calculate a variety of different blob measurements or features, such as the area, perimeter, Feret diameter, and center of gravity of each selected blob. Although the MblobCalculate() function initiates the actual calculations, it is the specified feature list that determines which calculations will be performed.

When you first allocate the feature list with MblobAllocFeatureList(), no features are selected for calculation. You generally use the MblobSelectFeature() function to add features to this feature list. You can, however, use the more specialized MblobSelectMoment() or MblobSelectFeret() functions to select a specific moment or Feret diameter, respectively.

The blob analysis module supports both binary and grayscale features. When selecting a binary feature, all calculations are performed using only the blob identifier image. When grayscale features are selected, you must also provide the MblobCalculate() function with a grayscale image. The blob identifier image will identify the blobs, and the grayscale image will supply the actual blob pixel values.

When you call MblobCalculate(), the identifier image is scanned to locate blobs, and any selected features are calculated. Even if only a few features are selected, the overhead of scanning the image can be considerable. Therefore, it is usually more efficient to select many features and make one call to MblobCalculate(), rather than to select and calculate one feature at a time. Note, features that have already been calculated for the specified images will not be recalculated if you call MblobCalculate() again, unless any parameters of the calculation have changed.

There are several considerations when selecting features:

  • Before selecting a feature for calculation, you should take the blob shapes into consideration. Some features are more appropriate for certain blob shapes than for others. For example, some should be used for round blobs rather than long, thin ones, and vice versa. The MblobSelectFeature() function provides this information.
  • When trying to distinguish between two similar blobs, selection of certain features, rather than some other features that might also seem appropriate, might reveal a more notable difference.
  • If two features allow you to come to the same conclusion, it is recommended that you select the one that is calculated more quickly. For example, features derived from multiple Feret diameters tend to calculate relatively slowly, and grayscale calculations are considerably longer than binary ones.

Note, for a visual representation of blobs that meet (or don't meet) certain criteria, call MblobFill() or MblobLabel() after calculating some features and calling MblobSelect(). These functions fill blobs with their own label values (MblobLabel()) or with a user-specified value (MblobFill()).

You can also use MblobDraw() to draw specific features of the results in an image buffer. For example, you can draw the blobs contours, holes, position, minimum and maximum Feret diameters, among other features. You can use a previously allocated graphics context (see the Preparing for graphics section in Chapter 17: Generating graphics) to control the drawing color, or use the default graphics context (M_DEFAULT). You can draw directly into the image buffer, or annotate the image non-destructively by drawing into its display overlay buffer. For more information, see the Annotating the displayed image non-destructively section in Chapter 16: Displaying an image.

The results obtained from MblobGetResult() can be sorted in ascending or descending order, by a maximum of three features assigned as sorting keys. To specify a feature as a sorting key, you can add M_SORTn_UP or M_SORTn_DOWN to the features selected with the following blob functions: MblobSelectFeature(), MblobSelectMoment(), MblobSelectFeret(). Assign the numbers 1, 2, or 3 to the n to indicate the sorting precedence of the feature(s).


The area and perimeter

Reference:
MblobControl
Not available in MIL-Lite

Available in MIL

Each pixel in your image represents a real width and height (for example, in millimeters). However, all results from the blob analysis functions (that represent a distance or area) are expressed in raw (uncalibrated) pixel units. You are left with the task of converting these results to actual physical units. This task is made easier if the width and height of the pixels are the same (that is, the pixel aspect ratio (width/height) equals 1.0). In this case, each pixel (P) is represented as follows:

A pixel ratio of 1.0 implies that the area (M_AREA) of a single pixel blob is equal to 1 and the perimeter (M_PERIMETER) is equal to 4. When calculating the area and perimeter of a larger blob, the area would then equal the number of pixels in the blob (excluding holes), and the perimeter would equal the total number of pixel sides along the blob edges (including the edges of holes). Note, an allowance is made for the staircase effect that occurs in a digital image when representing diagonals and curves. For example, in the following blob (where F represents foreground pixels), the area is 10 and the perimeter is 14.242.

You can also calculate an approximation of the convex perimeter (M_CONVEX_PERIMETER) of the blobs. The convex perimeter is the perimeter of the convex hull (see below).

This feature is derived by taking the diameter of the blob (Feret diameter) at different angles. You can adjust the number of Feret diameters used with the MblobControl() function. The greater the number of Feret diameters used, the more accurate the approximation.

When the pixel aspect ratio has been set to anything other than 1.0, using MblobControl(), the aspect ratio is applied to the pixel width during calculations. Each pixel is now represented as:

This affects all calculated features as if you had actually stretched the image (from the top-left corner in the x-direction only), by a factor equal to the pixel aspect ratio. We can no longer say that results are in "pixel" units. In fact, results are really in units of "pixel height" since the height is not affected by the aspect ratio.

AREA PERIMETER CONVEX PERIMETER


Dimensions

Reference:
MblobControl
MblobSelectFeature
MblobSelectFeret
Not available in MIL-Lite

Available in MIL

Besides the area and perimeter, you might need to determine the dimension of the blobs. Since blobs are not typically rectangular in shape, you will probably have to take the length (or diameter) of the blobs at various angles from the horizontal axis. This is actually one of the many definitions of the blob length, called the Feret diameter. Several Feret diameters are illustrated below. Note, the angle at which the Feret diameter is taken (relative to the horizontal axis) is specified as a subscript to the F.

With MIL, you can calculate the Feret diameter at a specified angle (M_GENERAL_FERET) by adding it to the feature list, using MblobSelectFeret(). To add the Feret diameter at 0° (horizontal Feret diameter) and 90° (vertical Feret diameter) to the feature list, you can also use MblobSelectFeature() (M_FERET_X and M_FERET_Y, respectively).

You can automatically determine the minimum, maximum, and average Feret diameters of the blob by adding the M_FERET_MIN_DIAMETER, M_FERET_MAX_DIAMETER, and M_FERET_MEAN_DIAMETER features, respectively, to the feature list, using MblobSelectFeature(). These diameters will be determined by testing the diameter of the blobs at several angles. Increasing the number of angles that are tested increases the accuracy of the results, but also increases processing time.

You can use the MblobControl() function to change the default number of angles (M_NUMBER_OF_FERETS value); these angles will start at 0° and increase in increments of 180°/(number of Feret diameters).

Note, the maximum Feret diameter is not very sensitive to the number of angles; using 8 angles usually produces an accurate result. The minimum diameter, however, can be inaccurate for long thin blobs unless many angles are used.

The angles at which the minimum and maximum Feret diameter were found can be determined by adding the M_FERET_MIN_ANGLE and M_FERET_MAX_ANGLE to the feature list, using MblobSelectFeature().

You can determine the ratio of the maximum to minimum Feret diameter by adding M_FERET_ELONGATION feature to the feature list, using the above function.

Although the Feret diameters provide a good approximation of the blob size, these features are not very good for long, thin blobs, even when using the maximum number of angles (M_MAX_FERETS). For these, the following features, available with MblobSelectFeature(), might provide better results:

  • M_LENGTH: an extraction of the true length of a blob.
  • M_BREADTH: an extraction of the true breadth of a blob.
  • M_ELONGATION: the ratio of the length to the breadth.

These features are derived from the area and perimeter, using the assumption that the blob area is equal to the [length x breadth] and the perimeter is equal to [2(length + breadth)]. These relations only hold if the length and breadth are constant throughout a blob. However, long, thin blobs generally satisfy this assumption, even if they are not straight.

Note, since these features use only the area and perimeter, they are faster to calculate than Feret features.

GENERAL FERET FERET X FERET Y FERET MIN DIAMETER FERET MAX DIAMETER FERET MEAN DIAMETER NUMBER OF FERETS FERET MIN ANGLE FERET MAX ANGLE FERET ELONGATION LENGTH BREADTH ELONGATION


Determining the shape

User Guide:
A simple blob analysis example
Not available in MIL-Lite

Available in MIL

Other useful features during classification are those that give you information about the blob shape. Two blobs can have similar sizes but different shapes because of a different number of holes, curves, or edges.

For example, in the illustration above, the blobs have similar sizes, but can be distinguished by the shape of their holes. If you treat the holes as the actual blobs (set non-zero pixels as foreground pixels and zero pixels as background pixels), you can extract the differences in shape of the holes.

Two features that can qualify the shape of these holes are:

  • Compactness (M_COMPACTNESS)
  • Roughness (M_ROUGHNESS)

The compactness is a measure of how close all particles in the blob are from one another. It is derived from the perimeter and area. A circular blob is most compact and is defined to have a compactness measure of 1.0 (the minimum); more convoluted shapes have larger values.

The roughness is a measure of the unevenness or irregularity of a blob's surface. It is a ratio of the perimeter to the convex perimeter of a blob. Smooth convex blobs have a roughness of 1.0, whereas rough blobs have a higher value because their true perimeter is bigger than their convex perimeter.

Although either of these features can be used in the classification process, compactness is faster to calculate since it is derived using only the area and perimeter.

For a program that, among other things, distinguishes between nuts, bolts and washers by analyzing the compactness of their holes, see the A simple blob analysis example section later in this chapter.

In some cases, you can also distinguish between blobs by determining the number of holes that they have (M_NUMBER_OF_HOLES). For example, you could distinguish between bolts and nuts in the BoltsNutsWashers.mim image by counting blob holes. However, this is not a very robust measure since a single noise pixel in a bolt blob would count as a hole.

COMPACTNESS ROUGHNESS NUMBER OF HOLES


Finding the blob location

Reference:
MblobCalculate
Not available in MIL-Lite

Available in MIL

Finding the location of blobs in an image can sometimes be more useful than finding their shape or size. For example, if a robotic arm needs to pick up several items regardless of their type, it can use their location in an acquired image to determine their actual physical position.

You can also use the blob location to determine if a blob touches the image borders. If there are any such blobs, you might want to adjust the camera's field of view so that all items are completely represented in the image, or you might want to exclude these blobs.

You can determine the following blob points by adding them to the feature list:

The center of gravity can be calculated in binary or grayscale mode. To calculate the latter, you must provide MblobCalculate() with a grayscale image.

You can obtain the coordinates of pixels bordering blobs or delimiting holes in blobs, in a counterclockwise or clockwise direction respectively. These pixels are referred to as chained pixels (M_CHAINS).

You can use the chained pixel coordinates to create a chain code. A chain code is a directional code that records an object's boundary as a discrete set of vectors, where each vector points to the next pixel in the chain.

Chained pixels always form a closed chain. This implies that the starting pixel in the chain is also the closing one. If your blob has regions which are 1 pixel wide, these pixels are chained twice, once in the forward direction and then in the opposite direction.

In the diagram below, the thick lines illustrate pixels which are chained twice.The diagram also illustrates chained pixels of a blob in an 8 and 4-connected lattice, where the solid lines illustrate chained pixels in an 8-connected lattice, and the dotted lines illustrate how chained pixels deviate in a 4-connected lattice. Also, note that the blob's outermost chain is identified as index 1. Chains that delimit holes in blobs are identified by subsequent indexes.

The M_CHAINS feature calculates four separate chain features. This includes M_NUMBER_OF_CHAINED_PIXELS which calculates the total number of chained pixels for each blob or a specified blob; the M_CHAIN_INDEX feature which assigns an index to each chained pixel, for every chain within in a blob; and the M_CHAIN_X and M_CHAIN_Y features which calculate the x and y coordinates of all chained pixels within a blob.

When retrieving results for chain features, you should retrieve results for the number of chained pixels M_NUMBER_OF_CHAINED_PIXELS first. Retrieving results for this feature allows you to allocate an array which is large enough for the other chain results. Thus, to find the results for a single chain, check the M_CHAIN_INDEX array for the appropriate chain indices, and retrieve the x and y results from the corresponding elements in the M_CHAIN_X and M_CHAIN_Y arrays.

For the blob shown on the previous page, the following arrays would result:

CHAINS CHAINS NUMBER OF CHAINED PIXELS CHAIN INDEX CHAIN X CHAIN Y NUMBER OF CHAINED PIXELS CHAIN INDEX CHAIN X CHAIN Y


Moments

Reference:
MblobCalculate
MblobGetLabel
MblobGetResultSingle
MblobGetRuns
MblobSelectFeature
MblobSelectMoment
Not available in MIL-Lite

Available in MIL

Using the blob analysis module, you can also calculate the moments used to find the center of gravity, as well as other grayscale or binary moments. The MblobSelectMoment() function allows you to add any moment to the feature list, whereas MblobSelectFeature() allows you to add only the more common moments.

You can calculate either central or ordinary moments. Central moments use coordinates that are relative to the center of gravity of the blob, and therefore are independent of a blob's position within the image, whereas, ordinary moments are affected by the blob position because they use coordinates relative to the top left corner of the image.

The blob analysis module automatically calculates label values for included blobs when a call to MblobCalculate() is made. You can obtain a label value for a single blob with a call to MblobGetLabel(), by specifying the blob's coordinate. A label value can be useful to obtain calculation results for a single blob with MblobGetResultSingle() or MblobGetRuns().


Location, length and number of runs

Reference:
MblobGetResult
MblobGetResultSingle
MblobGetRuns
Not available in MIL-Lite

Available in MIL

A run is defined as a horizontal string of consecutive foreground pixels. The blob analysis module can be used to obtain the total number of runs (M_NUMBER_OF_RUNS) for each blob. Results are obtained with MblobGetResult() or MblobGetResultSingle().

To obtain the length and coordinate of each run in a specific blob, use the MblobGetRuns() function. The runs that make up each blob can be used to calculate features that are not supported directly by MIL.

NUMBER OF RUNS


Blob reconstruction

Reference:
MblobFill
MblobLabel
MblobReconstruct
MblobSelect
Not available in MIL-Lite

Available in MIL

Although the blob analysis module is used mainly for blob feature calculation purposes, some of the Mblob...() functions can be used to perform blob image reconstruction. An example of this is the MblobReconstruct() function.

Blob reconstruction allows you to remove small particles in an image without distorting the shape of the remaining blobs, which can occur when successive erosion or dilation operations are performed on an image.

Using the different operations of the MblobReconstruct() function, you can:

  • Reconstruct blobs from a seed image. This operation copies to the destination buffer only those blobs that have a corresponding seed pixel in the seed buffer. Blobs that are not seeded are replaced according to the selected processing mode.

    • Note that when performing this operation in grayscale processing mode and with M_FOREGROUND_ZERO set, blobs in the source image which are not seeded are filled with the average grayscale value of the background.

  • Delete blobs that touch a border of the image. This operation copies only those blobs that do not touch the border to the destination buffer.

    • Note that when performing this operation in grayscale processing mode and with M_FOREGROUND_ZERO set, border blobs are filled with the average grayscale value of the background.

  • Fill holes in blobs. This operation copies the source buffer to the destination buffer, filling those blobs that contain holes according to the selected processing mode. Holes that touch the image border are not considered to be holes.

    • Note that when performing this operation in grayscale processing mode, the holes are filled with the average grayscale value of the blob.

  • Extract holes from blobs. This operation copies only the holes within the blobs found in the source buffer. Holes that touch the image border are not considered to be holes.

    • Note that when performing this operation in grayscale processing mode, the copied blob holes are filled with the average grayscale value of the blob in the source buffer. Also, when using the grayscale mode with M_FOREGROUND_ZERO set, the blobs are filled with the average grayscale value of the background.

Finally, you can perform other types of blob reconstruction by calling specific MIL functions, one after the other. For example, to fill all blobs composed of pixels with a required value, use MblobSelect() in conjunction with MblobLabel() and MblobFill().

FOREGROUND ZERO FOREGROUND ZERO FOREGROUND ZERO


A simple blob analysis example

Availability | Previous
Not available in MIL-Lite

Available in MIL

The following illustrates the source image before and after binarization and noise removal operations:

The example below binarizes an image, determines the center of gravity for each blob, and counts the number of bolts, nuts and washers. Note, the binarizing step produces a considerable number of spurious blobs and holes, so some processing is performed to clean up the blob identifier image before doing any calculations.

MBlob.c


Pattern matching

Chapter 7: Pattern matching
Availability
Not available in MIL-Lite

Available in MIL

This chapter explains quick techniques to perform alignment operations, using the MIL pattern matching (recognition)module.

  • Pattern matching - in general
  • Steps to perform a search
  • Defining a model
  • Rotation
    • Setting the angle of search
    • Determining the rotation tolerance of a model
  • Masking the model
  • Search constraints
    • Specifying the number of matches
    • Setting the acceptance level
    • Setting the certainty level
    • Redefining the model's reference position
    • Selecting the search region
    • Positional accuracy
    • Setting the speed parameter
  • Preprocess the search model
  • Speeding up the search
    • Choose the appropriate model
    • Adjust the search speed setting
    • Effectively choose the search region and search angle
    • Searching for multiple models at the same time
  • The pattern matching algorithm (for advanced users)
    • Normalized correlation
    • Hierarchical search
    • Search heuristics
    • Subpixel accuracy
  • Pattern matching examples


Pattern matching - in general

Availability | Next
Not available in MIL-Lite

Available in MIL

The MIL package includes a pattern matching module that uses normalized grayscale correlation (NGC) to help solve machine vision problems such as alignment, measurement, and inspection of objects. The main function of the pattern matching module is to search for occurrences of a pattern in an image. MIL refers to the pattern for which you are searching as the search model and the image from which it is extracted as the model source image.

The image being searched is called the target image.

This chapter describes various pattern matching techniques. It also provides guidelines on how to define a search model, find the occurrences of this model in the target image, and understand the search algorithm.

With MIL, you can only perform pattern matching operations on 8-bit grayscale unsigned buffers.

The pattern matching module also provides some support for calibration. Although the results are calculated in pixels, if the target image has been associated with a calibration object, positional results can be returned in calibrated real-world units.


Steps to perform a search

User Guide:
Defining a model
Masking the model
Pattern matching examples
Preprocess the search model
Rotation
Search constraints

Reference:
MpatAllocResult
MpatDraw
MpatFindModel
MpatFindMultipleModel
MpatFree
MpatGetNumber
MpatGetResult
MpatInquire
MpatPreprocModel
MpatRestore
MpatSave
MpatSetDontCare
Not available in MIL-Lite

Available in MIL

The steps involved in performing a search are as follows:

  1. Load or grab a model source image.
  2. Define the model from the model source image. For information on defining a model, see the Defining a model section later in this chapter.
  3. Optionally, specify a range for angular search. For information on specifying a range, see the Rotation section later in this chapter.
  4. Optionally, mask any irrelevant areas of the model using MpatSetDontCare() to set the model's "don't care" pixels. For more information, see the Masking the model section later in this chapter.
  5. Specify the model's search constraints. For more information, see the Search constraints section later in this chapter.
  6. Preprocess the model, using MpatPreprocModel(). For information on preprocessing the model, see the Preprocess the search model section later in this chapter.
  7. Allocate a result buffer to store the results of your search, using MpatAllocResult().
  8. Grab a target image. Optionally, process it to improve its quality.
  9. Find instances of the model in the target image using MpatFindModel(). The search is performed according to the defined search constraints.

    10. You can also search for several models that are of the same size and that use the same search region in the same image, using MpatFindMultipleModel(). This function finds occurrences of the specified models in the given image, and returns the position of each occurrence for each model or of the best matches from the group of models. Note that in the former case, you have to allocate and specify a result buffer for each model that is being sought. In the latter case, you have to allocate and specify a single result buffer. If you have to search for several different models, this is more efficient.

  10. Read the search results. To read results, use MpatGetNumber() and MpatGetResult() to get the number of model occurrences found in the target, and the required results, respectively. You can use MpatDraw() to draw the occurrences' bounding box and/or a cross at the occurrences's reference position.

    11. The coordinates resulting from a search return the reference position of the model, relative to the top-left corner of the target image. To find the equivalent coordinates in the model source image, use MpatInquire() with M_ORIGINAL_X and M_ORIGINAL_Y.

  11. Free all your allocated objects, using MpatFree().

In general, the first seven steps are performed once, while steps 8 through 11 are repeated as required. Note, in practice, models are usually saved on disk, using MpatSave(); therefore steps 1 through 6 are often replaced by a single step that restores a saved model from disk, using MpatRestore().

For examples on how to perform a search with a user-defined model, see the Pattern matching examples section later in this chapter.

ORIGINAL X ORIGINAL Y


Defining a model

User Guide:
Displaying an image
Generating graphics
Steps to perform a search
Rotation

Reference:
MpatAllocAutoModel
MpatAllocModel
MpatCopy
MpatDraw
MpatSave
Not available in MIL-Lite

Available in MIL

To define a model, you can use MpatAllocModel() to define which portion of the model source image to use as your model, or you can use MpatAllocAutoModel() to have MIL automatically extract a unique model for you.

The model should be unique and not defined from a flat region (consistent pixel values with no edges present). Also, the model source image should be of the best quality possible. If noisy, try to clean the image, using the image processing techniques discussed previously in this manual.

When allocating a model, you must specify its size. Generally, relative to the target image, small models take longer to find than larger ones, although very large models can also be time-consuming.

For information on defining a model that might appear at different angles in the target image, see the Rotation section later in this chapter.

Upon allocation, the model is extracted from the selected region in the model source image buffer and stored in a non-displayable model buffer. The model source image buffer is then no longer needed. To view the portion of the image from which the model was extracted, use MpatCopy() or MpatDraw().

You can use MpatDraw() to draw the various model features. You can use a previously allocated graphics context (see Chapter 17: Generating graphics) to control the drawing color, or use the default graphics context (M_DEFAULT). You can also choose to draw in the display's overlay buffer. By drawing into the display's overlay buffer, you can annotate an image non-destructively. For more information, see Chapter 16: Displaying an image.

Models are usually saved on disk for future use, using MpatSave(). You can also set and save the model's search constraints. For information on setting a models search constraints, see the Steps to perform a search section earlier in this chapter.


Rotation

Reference:
MimRotate
MpatAllocModel
MpatFindModel
MpatPreprocModel
MpatSetAngle
Not available in MIL-Lite

Available in MIL

There are two ways to search for a model that can appear at different angles using the MpatFindModel() function:

  • Search for rotated versions of the model.
  • Search for models taken from the same region in rotated images.

The following describes how to define models to perform these types of searches.

To implement the first type of search, allocate an M_NORMALIZED model with MpatAllocModel(). Then, enable and specify the angular range in which it can appear with MpatSetAngle(). When you call MpatPreprocModel(), it will internally create different models by rotating the original model at the required angles, assigning "don't care" pixels to regions that do not have corresponding data in the original model.

This method should only be used when the pixels surrounding the model follow no predictable pattern (for example, when searching for loose nuts and bolts lying on a conveyor).

To implement the second type of search, first allocate an M_NORMALIZED + M_CIRCULAR_OVERSCAN model with MpatAllocModel(); this will extract the model as well as circular overscan data from the model source image (specifically, MIL extracts the region enclosed by a circle which circumscribes the model). Second, enable and specify the angular range in which the model can appear with MpatSetAngle(). When you call MpatPreprocModel(), it will extract different orientations of the model from the overscanned model.

This type of model should only be used when the model's distinct features lie in the center of the region, so that they are included in all models when rotated. Therefore, it is recommended that an M_NORMALIZED + M_CIRCULAR_OVERSCAN model be as square as possible: the longer the rectangle, the smaller the number of consistent central pixels in every model.

As mentioned, a larger region than the one defined will be fetched from the model source image. Therefore, the model must not be extracted from a region too close to the edge of the model source image.

The pixels surrounding the model should be relevant to the positioning of the pattern (that is, the model should appear in the target image with the same overscan data). An example is the image of an integrated circuit.

Both methods find the position and match score of the model in a target image.

Finally, it should be noted that MIL's implementation of MpatFindModel() with a M_CIRCULAR_OVERSCAN type model is significantly faster than that of a M_NORMALIZED model when performing an angular search.

Setting the angle of search

By default, the angle of search is 0°. However, you can specify a rotational range of up to 360°, as well as the required precision of the resulting angle and the interpolation mode used for the rotated model. These settings can influence the speed of the search significantly. The accuracy of the search can also be influenced.

When an angular range has been specified with MpatSetAngle(), MpatPreprocModel() creates a model for every x degrees within the range, where x is determined by the specified tolerance (M_SEARCH_ANGLE_TOLERANCE). Tolerance defines the full range of degrees within which the pattern in the target image can be rotated from a model at a specific angle and still meet the acceptance level.

After the approximate location is found, MIL fine-tunes the search, according to the specified accuracy (M_SEARCH_ANGLE_ACCURACY). To be effective, you must set the degree of accuracy to a value smaller than that of the rotation tolerance.

When searching within a range of angles, you should use as narrow a range as possible, since the operation can take a long time to perform. Note that the model is rotated according to the interpolation mode (M_SEARCH_ANGLE_INTERPOLATION_MODE).

Determining the rotation tolerance of a model

Every model has its own particular rotation tolerance. This tolerance is dependent on the individual model characteristics and surrounding target image features. To determine the rotation tolerance of a model:

  1. Set the search angle of a model to the same angle as the sought for pattern in a sample target image. However, set the positive and negative delta values to zero since you want to test by how much a pattern in an image can be rotated and still correlate with a model at a specific angle.
  2. Use the MimRotate() function to rotate the image in very small, positive increments (for example, 0.5°), and perform a MpatFindModel() operation at every angle. Make sure that the image's center of rotation is the same as that of the model, otherwise the resulting tolerance will not be accurate. Note, when rotating the image, always set the angle from the image's original position to avoid interpolating the image more than once. Check the results for the greatest angle that produces an acceptable score.
  3. Repeat steps 1 through 3, rotating the image in negative increments.
  4. Take the minimum of the absolute value of these angles. Double this angle and set it as the rotation tolerance for the angular search.
NORMALIZED NORMALIZED CIRCULAR OVERSCAN NORMALIZED CIRCULAR OVERSCAN CIRCULAR OVERSCAN NORMALIZED SEARCH ANGLE TOLERANCE SEARCH ANGLE ACCURACY SEARCH ANGLE INTERPOLATION MODE


Masking the model

Reference:
MpatCopy
MpatDraw
MpatSetDontCare
Not available in MIL-Lite

Available in MIL

Often your search model contains regions that you need MIL to ignore when searching for the model in the target image. These regions might be noise pixels or simply regions that have nothing to do with what you are searching.

For instance, in a machine guidance application, a mechanical device might need to know where mounting holes are located on a circuit board so that screws can be properly inserted. In this case, a mounting hole would be the search model and the circuit board would be the target image.

In the above image, the search model contains too much of the actual board; it might not match holes in different areas of the board.

In such cases, parts of the search model can be masked by setting pixel values in certain regions to "don't care" pixels. MIL then ignores these regions when searching for occurrences of the model.

In our example, you would need to mask the edges of the search model, as follows:

The unmasked region of the search model now more closely resembles the pattern for which to search; it is more circular in shape and contains little of the actual board.

To create a mask:

  1. Allocate an image that is of the same size as the model image. This new image is the mask image.
  2. Clear the mask image to one pixel value. This is the background color of the mask image.
  3. Set the "don't care" pixels in the mask image to a specific value, different from the background. This is the foreground color of the mask image. There are numerous ways of doing this. For example, you can use one of the Mgra...() functions.

    4. To establish which pixels to set to "don't care", you can refer to a copy of the model image obtained using MpatCopy() with M_DEFAULT.

  4. Call MpatSetDontCare(), specifying the mask image along with the foreground color used to draw the "don't care" pixels. This function sets the model's "don't care" pixels.
  5. View the mask using MpatDraw() with M_DRAW_DONT_CARES. Alternatively, you can call MpatCopy() with M_DONT_CARE.

When you change the "don't care" pixels of a model, you should preprocess the search model again.

DEFAULT DRAW DONT CARES DONT CARE


Search constraints

User Guide:
Calibration
Speeding up the search

Reference:
MpatFindModel
MpatGetNumber
MpatGetResult
MpatSetAcceptance
MpatSetAccuracy
MpatSetCenter
MpatSetCertainty
MpatSetNumber
MpatSetPosition
MpatSetSpeed
Not available in MIL-Lite

Available in MIL

When a model is defined (whether manually or automatically), it is assigned a set of default search constraints. You can change the following constraints:

  • The number of occurrences to find.
  • The threshold for acceptance and certainty.
  • The model's reference position.
  • The region to search in the target image.
  • The positional accuracy.
  • The search speed.

Specifying the number of matches

You can specify how many matches to try to find, using MpatSetNumber(). If all you need is one good match, set the required number of occurrences to one (the default value) and avoid unnecessary searches for further matches. If a correlation has a match score above the certainty level, it is automatically considered an occurrence (default 80%), the remaining occurrences will be the best of those above the acceptance level.

When you ask for a specific number of matches (using MpatSetNumber()), the MpatFindModel() function might not find that number; you should always call MpatGetNumber() to see how many occurrences were actually found. When multiple results are found, they are returned in decreasing order of match score (that is, best match first).

Setting the acceptance level

The level at which the correlation (match score) between the model and the pattern in the image is considered a match is called the acceptance level.

You can set the acceptance level for the specified model, using MpatSetAcceptance(). If the correlation between the target image and the model is less than this level, they are not considered a match. A perfect match is 100%, a typical match is 80% or higher (depending on the image), and no correlation is 0%. If your images have considerable noise and/or distortion, you might have to set the level below the default value of 70%. However, keep in mind that such poor-quality images increase the chance of false matches and will probably increase the search time.

Note, perfect matches are generally unobtainable because of noise introduced when grabbing images.

Setting the certainty level

The certainty level is the match score (usually higher than that of the acceptance level) above which the algorithm can assume that it has found a very good match and can stop searching the rest of the image for a better one. The certainty level is very important because it can greatly affect the speed of the search. To understand why, you need to know a little about how the search algorithm works.

Since a brute force correlation of the entire model, at every point of the image, would take several minutes, it is not practical. Therefore, the algorithm has to be intelligent. It first performs a rough but quick search to find likely match candidates, then checks out these candidates in more detail to see which are acceptable.

A significant amount of time can be saved if several candidate matches never have to be examined in detail. This can be done by setting a certainty level that is reasonable for your needs, using MpatSetCertainty(). A good level is slightly lower than the expected score. If you absolutely must have the best match in the image, set the level to 100%. This would be necessary if, for example, you expect the target image to contain other patterns that look similar to your model. Unwanted patterns might have a high score, but this will force the search algorithm to ignore them. Symmetrical models fall into this category. At certain angles symmetrical models might seem like an occurrence in the target image, but if the search was completed, a match with a higher score would be found.

Often, you know that the pattern you want is unique in the image, so anything that reaches the acceptance level must be the match you want; therefore, you can set the certainty and acceptance levels to the same value.

Another common case is a pattern that usually produces very good scores (say above 80%), but occasionally a degraded image produces a much lower score (say 50%). Obviously, you must set the acceptance level to 50%; otherwise you will never get a match in the degraded image. However, you cannot set the certainty level to 50% because you take the risk that it will find a false match (above 50%) in a good image before it finds the real match that scores 90%. A better value is about 80%, meaning that most of the time the search will stop as soon as it sees the real match, but in a degraded image (where nothing reaches the certainty level), it will take the extra time to look for the best match that reaches the acceptance level.

Redefining the model's reference position

The coordinates returned by MpatGetResult(), after a call to MpatFindModel(), are the coordinates of the model's reference position (in pixel or real-world units, depending on whether the imaging setup is calibrated; for more information, see Chapter 5: Calibration). By default, this reference position is defined to be at the geometric center. Note that, when using pixel units, results are returned relative to the top-left corner of the target image.

If there is a particular spot from which you would like results returned, you can change the model's reference position, using MpatSetCenter(). For example, if your model has a hole and you want to find results with respect to this hole, change the reference position of the model accordingly. Note that you can define the reference position to be outside of the model's boundary.

Selecting the search region

Instead of searching the entire region of an image, you can limit the search region with MpatSetPosition(). This function specifies the region in which to find the model's reference position. Therefore, the search region can even be smaller than the model. If you have redefined the model's reference position (with MpatSetCenter()), make sure that the search region defined by MpatSetPosition() covers this new reference position and takes into account the angular search range of the model.

In general, you should not use a child buffer to delimit the search region to a portion of an image; this might cause the search routine to have border or edge effects and be less accurate (the routine does not assume that there is valid data outside of the buffer).

Search time is roughly proportional to the region searched; always set the search region to the minimum required when speed is a consideration.

Positional accuracy

You can set the positional accuracy for your search. Use MpatSetAccuracy() to set the required positional accuracy. It can be set to:

  • M_LOW (typically ± 0.20 pixel)
  • M_MEDIUM (typically ± 0.10 pixel)
  • M_HIGH (typically ± 0.05 pixel)

Note, the actual precision achieved is dependent on the quality of the model and of the image (the tolerances listed above are typical for high-quality, low-noise images).

A less precise positional accuracy will speed up the search. Positional accuracy is also slightly affected by the search speed parameter (MpatSetSpeed()).

Setting the speed parameter

You can specify the algorithm's search speed, using MpatSetSpeed(). When the search speed is set to M_VERY_HIGH or M_HIGH, the search algorithm takes more shortcuts, and the search is performed faster. However, as you increase the speed, the robustness of the search operation (the likelihood of finding a model) can decrease. For more information on search speed, see the Speeding up the search section later in this chapter.

LOW MEDIUM HIGH VERY HIGH HIGH


Preprocess the search model

Reference:
MpatPreprocModel
Not available in MIL-Lite

Available in MIL

When you are ready to search for the allocated model (either manually or automatically), you must preprocess the model. The preprocessing stage uses the known model to decide on the optimal search strategy for subsequent search operations. Preprocessing should be performed after all search constraints have been set. Use the MpatPreprocModel() function to preprocess the model.

MpatPreprocModel() has a parameter that allows you to specify a typical target image. Providing a typical image is optional; you can set this parameter to M_NULL. If you provide this image, it helps MpatPreprocModel() improve the search's robustness and optimize the strategy for subsequent search operations. You should only specify a typical image if the model will always appear on the same type of background.

If you save the model to disk, the model's preprocessing changes are stored with the model. Upon restoration, the model need not be preprocessed.


Speeding up the search

Reference:
MpatFindModel
MpatFindMultipleModel
MpatPreprocModel
MpatSetAngle
MpatSetPosition
MpatSetSpeed
Not available in MIL-Lite

Available in MIL

To ensure the fastest possible search, you should:

  • Choose an appropriate model.
  • Set the search speed to the highest possible setting for your application.
  • Set the search region to the minimum required. Search time is roughly proportional to the region searched, so don't search the whole image if you don't need to.
  • Search the smallest range of angles required.
  • Select the lowest positional accuracy that you need.
  • Set the certainty level to the lowest reasonable value (so that the search can stop as soon as a good match is found).
  • Search for multiple models at the same time, if possible.

Choose the appropriate model

The size of a model affects the search speed. In general, small models take longer to find than larger ones, although very large models can also be time-consuming. In general, the optimal size is approximately 128 x 128 pixels if you are searching a large region (for example, most of the image). Small models are found quickly when the search region is not too large.

Adjust the search speed setting

For any search model, it is possible to set the speed of the search. Increasing the search speed reduces the search time, however, as you increase the speed, the robustness of the search operation (the likelihood of finding a model) can decrease. When you call MpatPreprocModel(), MIL analyzes the pattern in the model, and determines appropriate shortcuts; only shortcuts that are considered safe for a particular model are taken. This also means that higher search speeds might not be any faster for certain models, depending on the particular pattern. Higher search speeds can reduce the positional accuracy very slightly.

You adjust the search speed setting, using MpatSetSpeed(). This function has five settings:

  • M_VERY_HIGH
  • M_HIGH
  • M_MEDIUM
  • M_LOW
  • M_VERY_LOW

As expected, the M_VERY_HIGH and M_HIGH speed settings allow the search to take all possible shortcuts, performing the search as fast as possible. Higher speed settings are recommended when searching on a good quality image or when using a simple model. Note, the search might have a lower tolerance for rotation when using this setting.

The M_MEDIUM speed setting is the default setting and is recommended for medium quality images or more complex models. A search with this setting is capable of withstanding up to approximately 5° of rotation for typical models.

Use the M_LOW or the M_VERY_LOW speed settings only if the image quality is particularly poor and you have encountered problems at higher speeds. The speed settings are discussed further in the algorithm description at the end of this chapter.

Effectively choose the search region and search angle

You can improve performance by not searching the whole image unnecessarily. Search time is roughly proportional to the region searched; set the search region to the minimum required, using MpatSetPosition(). You can also improve performance by selecting the lowest positional accuracy. In addition, for an angular search, select lowest angular accuracy (MpatSetAngle() with M_SEARCH_ANGLE_ACCURACY) and range required, in combination with the highest tolerance possible.

Searching for multiple models at the same time

When searching for multiple models of the same size and search region, it is more efficient to call the MpatFindMultipleModel() function instead of calling MpatFindModel() once for each model.

VERY HIGH HIGH MEDIUM LOW VERY LOW VERY HIGH HIGH MEDIUM LOW VERY LOW SEARCH ANGLE ACCURACY


The pattern matching algorithm (for advanced users)

User Guide:
Custom spatial filters

Reference:
MpatFindModel
MpatFindMultipleModel
MpatGetResult
MpatInquire
MpatPreprocModel
MpatSetAccuracy
MpatSetSearchParameter
MpatSetSpeed
Not available in MIL-Lite

Available in MIL

Normalized grayscale correlation is widely used in industry for pattern matching applications. Although in many cases you do not need to know how the search operation is performed, an understanding of the algorithm can sometimes help you pick an optimal search strategy.

Normalized correlation

The correlation operation can be seen as a form of convolution, where the pattern matching model is analogous to the convolution kernel (see the Custom spatial filters section in Chapter 4: Advanced image processing). Ordinary (un-normalized) correlation is exactly the same as a convolution:

In other words, for each result, the N pixels of the model are multiplied by the N underlying image pixels, and these products are summed. Note, the model does not have to be rectangular because it can contain "don't care" pixels that are completely ignored during the calculation. When the correlation function is evaluated at every pixel in the target image, the locations where the result is largest are those where the surrounding image is most similar to the model. The search algorithm then has to locate these peaks in the correlation result, and return their positions.

Unfortunately, with ordinary correlation, the result increases if the image gets brighter. In fact, the function reaches a maximum when the image is uniformly white, even though at this point it no longer looks like the model. The solution is to use a more complex, normalized version of the correlation function (the subscripts have been removed for clarity, but the summation is still over the N model pixels that are not "don't cares"):

With this expression, the result is unaffected by linear changes (constant gain and offset) in the image or model pixel values. The result reaches its maximum value of 1 where the image and model match exactly, gives 0 where the model and image are uncorrelated, and is negative where the similarity is less than might be expected by chance.

In our case, we are not interested in negative values, so results are clipped to 0. In addition, we use r2 instead of r to avoid the slow square-root operation. Finally, the result is converted to a percentage, where 100% represents a perfect match. So, the match score returned by MpatGetResult() is actually:

Score = max (r, 0)2x 100%

Note, some of the terms in the normalized correlation function depend only on the model, and hence can be evaluated once and for all when the model is defined. The only terms that need to be calculated during the search are:

This amounts to two multiplications and three additions for each model pixel.

On a typical computer, the multiplications alone account for most of the computation time. A typical application might need to find a 128x128-pixel model in a 512x512-pixel image. In such a case, the total number of multiplications needed for an exhaustive search is 2x5122x1282, or over 8 billion. On a typical computer, this would take a few minutes, much more than the 5 msec or so the search actually takes. Clearly, MpatFindModel() does much more than simply evaluate the correlation function at every pixel in the search region and return the location of the peak scores.

Hierarchical search

A reliable method of reducing the number of computations is to perform a so-called hierarchical search. Basically, a series of smaller, lower-resolution versions of both the image and the model are produced, and the search begins on a much-reduced scale. This series of sub-sampled images is sometimes called a resolution pyramid, because of the shape formed when the different resolution levels are stacked on top of each other.

Each level of the pyramid is half the size of the previous one, and is produced by applying a low-pass filter before sub-sampling. If the resolution of an image or model is 512x512 at level 0, then at level 1 it is 256x256, at level 2 it is 128x128, and so on. Therefore, the higher the level in the pyramid, the lower the resolution of the image and model.

The search starts at low resolution to quickly find likely match candidates. It proceeds to higher and higher resolutions to refine the positional accuracy and make sure that the matches found at low resolution actually were occurrences of the model. Because the position is already known from the previous level (to within a pixel or so), the correlation function need be evaluated only at a very small number of locations.

Since each higher level in the pyramid reduces the number of computations by a factor of 16, it is usually desirable to start at as high a level as possible. However, the search algorithm must trade off the reduction in search time against the increased chance of not finding the pattern at very low resolution. Therefore, it chooses a starting level according to the size of the model and the characteristics of the pattern. In the application described earlier (128x128 model and 512x512 image), it might start the search at level 4, which would mean using an 8x8 version of the model and a 32x32 version of the target image.

To determine the default first and last resolution levels, use MpatInquire() with M_PROC_FIRST_LEVEL and M_PROC_LAST_LEVEL, respectively. If required, you can change the first and last resolution levels, using MpatSetSearchParameter() with M_FIRST_LEVEL and M_LAST_LEVEL, respectively.

The logic of a hierarchical search accounts for a seemingly counter-intuitive characteristic of MpatFindModel(): large models tend to be found faster than small ones. This is because a small model cannot be sub-sampled to a large extent without losing all detail. Therefore, the search must begin at fairly high resolution (low level), where the relatively large search region results in a longer search time. Thus, small models can only be found quickly in fairly small search regions.

Note that the pyramidal representation of the buffer is generated each time MpatFindModel() or MpatFindMultipleModel() is called. However, you can save the pyramidal representation of the buffer (generated when MpatFindModel() or MpatFindMultipleModel() is called) in the result buffer, using MpatSetSearchParameter() with M_TARGET_CACHING. This pyramidal representation is re-used by consecutive calls to MpatFindModel() and MpatFindMultipleModel() as long as the same result buffer is used and the image, search region, and model size are not modified.

Search heuristics

Even though performed at very low resolution, the initial search still accounts for most of the computation time if the correlation is performed at every pixel in the search region. On most models, match peaks (pixel locations where the surrounding image is most similar to the model, and correlation results are largest) are several pixels wide. These can be found without evaluating the correlation function everywhere. MpatPreprocModel() analyzes the shape of the match peak produced by the model, and determines if it is safe to try to find peaks faster. If the pattern produces a very narrow match peak, an exhaustive initial search is performed. The search algorithm tends to be conservative; if necessary, force fast peak finding, using MpatSetSearchParameter() with M_FAST_FIND.

Using MpatSetSearchParameter() with M_EXTRA_CANDIDATES, you can set the number of extra peaks to consider. Normally, the search algorithm considers only a limited number of (best) scores as possible candidates to a match when proceeding at the most sub-sampled stage. You can add robustness to the algorithm, by considering more candidates, without compromising too heavily on search speed. In addition, you can use MpatSetSearchParameter() with M_COARSE_SEARCH_ACCEPTANCE to set a minimum match score, valid at all levels except the last level, to be considered as an occurrence of the model. This ensures that possible models are not discarded at lower levels, yet maintains the required certainty during the final level.

At the last (high-resolution) stage of the search, the model is large, so this stage can take a significant amount of time, even though the correlation function is evaluated at only a very few points. To save time, you can select a high search speed, using MpatSetSpeed(). For most models, this has little effect on the score or accuracy, but does increase speed.

Subpixel accuracy

The highest match score occurs at only one point, and drops off around this peak. The exact (subpixel) position of the model can be estimated from the match scores found on either side of the peak. A surface is fitted to the match scores around the peak and, from the equation of the surface, the exact peak position is calculated. The surface is also used to improve the estimate of the match score itself, which should be slightly higher at the true peak position than the actual measured value at the nearest whole pixel.

The actual accuracy that can be obtained depends on several factors, including the noise in the image and the particular pattern of the model. However, if these factors are ignored, the absolute limit on accuracy, imposed by the algorithm itself and by the number of bits and precision used to hold the correlation result, is about 0.025 pixels. This is the worst-case error measured in X or Y when an image is artificially shifted by fractions of a pixel. In a real application, accuracy better than 0.05 pixels is achieved for low-noise images. These numbers apply if you select high-search accuracy, using MpatSetAccuracy(), in which case the search always proceeds to resolution level 0.

If you select medium accuracy (the default), the search might stop at resolution level 1, and hence the accuracy is half of what can be attained at level 0. Selecting low accuracy might cause the search to stop at level 2, so the accuracy is reduced by an additional factor of two (to about 0.2 pixel).

PROC FIRST LEVEL PROC LAST LEVEL FIRST LEVEL LAST LEVEL TARGET CACHING FAST FIND EXTRA CANDIDATES COARSE SEARCH ACCEPTANCE


Pattern matching examples

See Also | Availability | Previous
Reference:
MpatFindModel
Not available in MIL-Lite

Available in MIL

MPat.c contains 3 examples that use the NGC pattern matching module. The first and second examples of MPat.c use the following image as source image:

This first example defines a model and searches for it in a shifted version of the image (without rotation). It also demonstrates the subpixel accuracy of MpatFindModel(). Refer to the SearchModelExample() function for this first example.

The second example of MPat.c uses the following image as target image:

This example defines a model and searches for it in a rotated version of the image. Refer to the SearchRotatedModelExample() function for this second example.

The third example of MPat.c uses the following images:

This example automatically allocates a model in a wafer image and finds its horizontal and vertical displacement. Refer to the AutoAllocationModelExample() function for this third example.

MPat.c


Geometric Model Finder

Chapter 8: Geometric Model Finder
Availability
Not available in MIL-Lite

Available in MIL

This chapter explains how to use the MIL Geometric Model Finder to locate occurrences of models in your target.

  • The MIL Geometric Model Finder module
  • Steps to performing a search
  • Basic concepts
  • Types of Model Finder contexts
    • General geometric contexts
    • Controlled geometric contexts
  • Defining and adding models to your Model Finder context
    • Image-type models
      • Extracting edges
    • Synthetic models
      • Predefined-shape models
      • CAD-file models
      • Model size and units
      • Synthetic models and calibration
    • Models defined from result buffers
      • Models defined from an Edge Finder result buffer
      • Models defined from a Model Finder result buffer
    • Models defined from two other models
    • Preprocessing
    • Model origin
    • Model indices and labels
    • Drawing and inquiring the model’s active edges
  • Guidelines for choosing models
    • Make sure your images have enough contrast
    • Avoid poor geometric models
    • Be aware of ambiguous models
    • Nearly ambiguous models
  • Masking your model
  • Search targets
    • Finding models in an image
    • Finding models in an Edge Finder result buffer
  • Determining what is a match
    • Score and target score
    • Model and target coverage
    • Fit error
    • Interpreting results
  • Position, angle, and scale
    • Enabling calculations specific to searching within a range
    • Search position and position range
    • Angle and angular range
    • Scale and scale range
  • Customizing search settings
    • Acceptance levels
    • Certainty levels
    • Expected number of occurrences
    • Reference axis
      • Reference axis origin
      • Reference axis angle
      • Forward and reverse transformation coefficients
  • Advanced search settings
    • Polarity
    • Separation
    • Shared edges
    • Fit error weighting factor
    • First and last levels
  • Global context settings
    • Setting the search speed
    • Accuracy
    • Timing out your search
  • Speeding up the search
    • Adjust the search speed of the algorithm
    • Disable calculations specific to range search strategies
    • Define several models with different expected angles
    • Limit the position range
    • Limit the search scale
    • Specify the exact number of expected occurrences
    • Clean up your model and your target
    • Changing the search levels
  • Retrieving and analyzing results
    • Possible results
    • Drawing results
  • Calibration
    • Requirements
    • Models and their calibration object
    • Setting the aspect ratio control
    • Using transformation coefficients with calibrated images
  • An example


The MIL Geometric Model Finder module

Availability | Next
Not available in MIL-Lite

Available in MIL

The MIL Geometric Model Finder module is a set of functions to find patterns, or models, based on geometric features. The module finds models using edge-based geometric features instead of a pixel-to-pixel correlation. As such, the Model Finder module offers several advantages over correlation pattern matching, including greater tolerance of lighting variations (including specular reflection), model occlusion, as well as variations in scale and angle.

The Model Finder module allows you to tailor your search to fit the requirements of your application. You can search for any number of different models simultaneously, through a range of angles and scale. You can also search for different kinds of models with Model Finder including models created from images (image-type), models created from Edge Finder result buffers (Edge Finder-type), models created from Model Finder result buffers (Model Finder-type), models merged from two previous models (merge-type), and synthetic models. Synthetic models are either models that have a predefined shape, or models defined from CAD files.

The module provides complete support for calibration. Searches can be performed in the calibrated real-world such that, without physically correcting your images, occurrences can be found even in the presence of complex distortions, and results calculated in real-world units.

The module also allows you to restore a Model Finder context from a file or memory stream, or save a Model Finder context to a file or memory stream.


Steps to performing a search

User Guide:
Calibration

Reference:
MmodAlloc
MmodAllocResult
MmodControl
MmodDefine
MmodDefineFromFile
MmodFind
MmodFree
MmodGetResult
MmodMask
MmodPreprocess
MmodSave
Not available in MIL-Lite

Available in MIL

The following steps provide a basic methodology for using the MIL Geometric Model Finder module:

  1. Allocate your Model Finder context, using MmodAlloc().
  2. Define and add your model(s) to this Model Finder context, using MmodDefine() or MmodDefineFromFile().
  3. If necessary, mask any irrelevant, inconsistent, or featureless areas of your models, using MmodMask().
  4. Specify your required search settings for both the context and the individual model(s), using successive calls to MmodControl().
  5. Preprocess your Model Finder context, using MmodPreprocess().
  6. Allocate a result buffer to hold the results of your search, using MmodAllocResult().
  7. Search the target for occurrences of models in your Model Finder context, using MmodFind().
  8. Retrieve the required results from the result buffer, using MmodGetResult().
  9. If necessary, save your Model Finder context, using MmodSave().
  10. Free all your allocated objects using MmodFree().

For information about using calibrated model source images and targets, see the Calibration section later in this chapter.


Basic concepts

Availability | Previous | Next
Not available in MIL-Lite

Available in MIL

The basic concepts and vocabulary conventions for the Model Finder module are:

  • Active edges. Edges which are used to compose the geometric model without being masked out, and are searched for in the target.
  • Bounding box of the edges. The smallest rectangle that fully encloses all of the edges of the model.
  • Chain. The set of connected edgels that construct an edge.
  • Edge. A curve that delineates a boundary, which can be established from intensity transitions in an image. In Model Finder, an edge is considered to be a chain, and its features.
  • Edgel. Elementary point (or edge element) within an edge.
  • Model. The information that defines the pattern of active edges to find in the target and the search settings with which to do so.
  • Model box. The box that delimits the boundaries of the model. Essentially, the model box is the bounding box of the edges plus a margin. For image-type and Edge Finder-type models, the size of the model box is specified when defining the model. For synthetic models, the margin can be adjusted after model definition.
  • Model Finder context. The container for all models that you want to find. The Model Finder context allows you to set global search settings that apply to the search algorithm.
  • Model image. For an image-type and Edge Finder-type model, the model image is a copy of the region in the model source image from which the model has been defined. Note that the model source image for an Edge Finder-type model is the image source of the result buffer. For a synthetic model, the model image is the image representation (edge map) of the model.
  • Model mask. A binary image used to define irrelevant, inconsistent, or featureless areas in the model, so that only pertinent model details are used for the search.
  • Model origin. The point in the model's coordinate system that is considered to be (0, 0).
  • Model source. The immediate source of the model's edges. For example, for an Edge Finder-type model, the model source is the buffer of the Edge Finder results.
  • Model source image. The image from which you have defined an image-type model, or from which you have extracted the edges for an Edge Finder-type model.
  • Occurrence. An instance of the model found in the target.
  • Synthetic models. Models that have a predefined shape, such as a circle or a square, or are defined from a CAD-type file.
  • Target. The image or result buffer in which to search for occurrences of the model.
  • Target edges. The edges in the target.


Types of Model Finder contexts

Reference:
MmodAlloc
Not available in MIL-Lite

Available in MIL

Using MmodAlloc(), you can allocate one of two types of Model Finder contexts: one that uses a general geometric search algorithm (M_GEOMETRIC) or one that uses a controlled geometric search algorithm (M_GEOMETRIC_CONTROLLED). Both algorithms use edge-based geometric features of the models and the target to establish a match.

General geometric contexts

General geometric Model Finder contexts use a general geometric search algorithm to locate user-specified models. General geometric contexts handle a full range of scale and angles unlike a controlled geometric context.

Controlled geometric contexts

Controlled geometric Model Finder contexts use a controlled geometric search algorithm to locate user-specified models. Using a controlled geometric context is recommended in situations where the image has high geometric complexity and where there are only small differences in angle and scale between the occurrence and the nominal angle and scale of the model (M_ANGLE and M_SCALE, respectively). In such a situation, a controlled geometric search is often faster and more robust than the general geometric search. Model searches use a hierarchical method to reduce the time needed to complete the search. Note that this might cause very thin lines to be missed as candidates.

There are restrictions on when you can use a controlled geometric context. Angle range and scale range searches are disabled when you use this context. As well, if you are retrieving a result relating to the entire context, you cannot get or draw the edges for the entire target; you can only get or draw the target edges in the region of an occurrence.

GEOMETRIC GEOMETRIC CONTROLLED ANGLE SCALE


Defining and adding models to your Model Finder context

User Guide:
Extracting the edges

Reference:
MedgeAllocResult
MedgeCalculate
MedgeControl
MmodAlloc
MmodControl
MmodDefine
MmodDefineFromFile
MmodDraw
MmodFind
MmodInquire
MmodPreprocess
Not available in MIL-Lite

Available in MIL

Once you have allocated a Model Finder context using MmodAlloc(), you can begin to add models to your Model Finder context.

There are four different kinds of models that can be added to a Model Finder context: a model from an image, a synthetic model, a model from an Edge Finder or Model Finder result buffer, or a model created from two other models. All of these models can be mixed in the same Model Finder context, using MmodDefine() or MmodDefineFromFile(). The MmodDefine() function can add image-type models, result-type models, and predefined-shape synthetic models; the MmodDefineFromFile() function can add CAD-file synthetic models.

Image-type models

You can add models defined from an image to a Model Finder context, using MmodDefine() with M_IMAGE. This type of model is referred to as an image-type model.

The specified model source image must be a 1-band, 8-bit unsigned image. You can use the entire image (if you have already cropped the area intended for the model), a child buffer, or you can designate the model region using the offset and size parameters of MmodDefine(). The minimum size of a model is limited to 16.0 X 16.0 pixels, while a maximum size of 1024.0 X 1024.0 pixels is permitted.

Extracting edges

When you call MmodDefine(), it stores the specified region of the model source image with the model. This image is referred to as the model image. The model source image is then no longer needed.

It is actually when you call MmodPreprocess() that edges are extracted from the model image. Specifically, MmodPreprocess() extracts the contour edges from the model image. For more information on contour edges, see the Extracting the edges section in Chapter 9: Edge Finder.

The edge extraction process involves a denoising operation to even out rough edges and remove noise. You can control the degree of smoothness (strength) of the denoising operation used for all models in the context, using MmodControl() with M_SMOOTHNESS. The range of this control type varies from 0.0 to 100.0; a value of 100.0 results in a strong noise reduction effect, while a value of 0.0 has almost no noise reduction effect. The default setting is 50.0.

You can also control the detail level of the edge extraction process for all models in the context, using MmodControl() with M_DETAIL_LEVEL. Basically, this controls how much detail is extracted from the model image of each model in the context. The default setting (M_MEDIUM) offers a robust detection of edges from images with contrast variation, noise, and non-uniform illumination. Nevertheless, in cases where objects of interest have a very low contrast compared to high contrast areas in the image, some low contrast edges can be missed.

The following examples show the use of the M_DETAIL_LEVEL control type

If your images contain low-contrast and high-contrast objects, and your object of interest is low-contrast, a detail level setting of M_HIGH should be used to ensure the detection of the low-contrast edges in the image. The M_VERY_HIGH setting performs an exhaustive edge extraction, including very low contrast edges. However, it should be noted that this setting is very sensitive to noise.

Generally, the default settings for M_SMOOTHNESS and M_DETAIL_LEVEL are sufficient for the majority of images; you should adjust these settings only when dealing with very noisy images, extremely low or multi-contrasted images, or images with very thin, refined features. In such cases, it is recommended that you experiment with different settings to achieve the necessary level of accuracy and speed required by your application.

Note that the settings for M_SMOOTHNESS and M_DETAIL_LEVEL are also applied to the search target when it is an image.

Synthetic models

Synthetic models are models that either have a predefined shape, or have been defined from a CAD DXF file. Synthetic models allow you to search target images and Edge Finder result buffers for shapes that you expect to be in the target.

Predefined-shape models

You can use MmodDefine() to add predefined-shape models to the context. The model types that are available are circle, cross, diamond, ellipse, rectangle, ring, square, and triangle. These model types are described in more detail in the description of MmodDefine().

Use the MmodDefine() parameters to define the attributes of the predefined-shape model, from the foreground color of the model to how it is constructed. The definition of each MmodDefine() parameter depends on the model. For example, for a circle model, MmodDefine() would have a parameter that defines the radius of the circle. For an ellipse model, MmodDefine() would have parameters that define the width and the height.

Predefined-shape models use the center of the shape as the origin (0,0).

CAD-file models

You can add models that are defined from CAD DXF files to a Model Finder context, using MmodDefineFromFile().

Not every entity in a CAD DXF file is supported; unsupported entities are ignored. The supported entities are Line, Polyline, Lwpolyline, Circle, Arc, Ellipse, Block (a collection of entities), and Insert (an entity containing a reference to where the Block should be drawn, and at what location and scale).

When the model is added to the Model Finder context from a CAD DXF file, the model will retain its coordinate sytem (the origin and the axis). Most CAD DXF files are oriented so that the Y-axis is positive going up, whereas the coordinate system MIL uses is oriented so that the Y-axis is positive going down. So if you take the edge coordinates of an object from a CAD DXF file and put them in an image, the imaged object will look flipped when compared to the original object. This is also the case for a model defined from a CAD DXF file. This means that, unless the object is symmetrical, the match will not be made. You can use MmodControl() with M_CAD_Y_AXIS to flip the Y-axis for the model so that it is positive going down.

Model size and units

The dimensions of a synthetic model should be specified in user-defined units. Specify the ratio between model units and pixel units using MmodControl() with M_PIXEL_SCALE.

By default, synthetic models are defined to have a 10% margin around the bounding box of their edges. When finding an occurrence, extra edges found in this area will reduce the target score. You can change the size of the margin using MmodControl() with the M_BOX_MARGIN_... control types.

Synthetic models and calibration

You can search for synthetic models in a calibrated or non-calibrated target. If the target is not calibrated, then M_PIXEL_SCALE is used for the match.

If the target is calibrated, the model units should be the same as the calibrated units. If the target is calibrated, M_PIXEL_SCALE will be used for draw operations, but not for the match nor the returned results.

Note that if you are searching for a synthetic model in a calibrated target, you must also associate the calibration object of the target with the model, using M_ASSOCIATED_CALIBRATION. MIL needs the calibration object for internal purposes at preprocessing time. Note that MIL will not use the calibration object to compensate for incongruencies between the model and the target, nor will MIL map the model's coordinate system to that of the target.

After you have specified your pixel scale and scale settings, you can verify if your synthetic models are valid, using MmodInquire() with M_VALID. Synthetic models can be invalid if their global size in X or Y (size of the model box * M_PIXEL_SCALE * M_SCALE) is greater than 1024.

Models defined from result buffers

You can define models from two different types of result buffers: Edge Finder result buffers and Model Finder result buffers.

In the case of these models, the model source image is the image from which the results were obtained. The model source is the result buffer in which the results are stored. The model image is a copy of the region in the model source image from which the model has been defined.

Models defined from an Edge Finder result buffer

You would define models from an Edge Finder result buffer if you want to use other types of edges, such as crest edges, or if you want to re-use the edge map for another purpose. If you knew that the edges of interest had a minimum size, you could use Edge Finder to include only edges above that size. The included edges of the results are the ones that will be used to define the model. These result-type models are referred to as Edge Finder-type models.

Some things have to be done before you can define a model from an Edge Finder result buffer.

  • The Edge Finder result buffer must have been previously allocated using MedgeAllocResult().
  • The Edge Finder result buffer must be compatible with the Model Finder context. Enable this using MedgeControl() with M_MODEL_FINDER_COMPATIBLE prior to calling MedgeCalculate().
  • MedgeCalculate() must have been already called using this result buffer.

You can add models defined from an Edge Finder result buffer to a Model Finder context using MmodDefine() with M_EDGE_RESULT.

The size of the model can be defined using the offset and size parameters of MmodDefine().

Models defined from a Model Finder result buffer

Adding models defined from a Model Finder result buffer allows you to define new models from previous match occurrences. These models are referred to as Model Finder-type models.

There are many reasons to use a Model Finder-type model. For example, if you have a model that changes slightly over time, you can define a model from the result in order to update it. Updating the model prevents potentially misidentifying the model because it has changed beyond your starting tolerances.

To define a Model Finder-type model, use MmodDefine() with M_MOD_RESULT and pass the identifier of the Model Finder result buffer and the index of the occurrence to use. The model is defined with the edges of the target at the occurrence position. You have to use MmodControl() with M_MOD_DEFINE_COMPATIBLE enabled before you can use this feature; this must be done before the MmodFind() finds the result. The result buffer must be compatible with the Model Finder context.

Models defined from two other models

Models can be defined from two previously existing models and added to the context. Use MmodDefine() with M_MERGE_MODEL and specify the two models from which to create the new model. The model will be formed by taking the first model specified and adding the edges of the second model that are not included in the edges of the first model.

There are no restrictions on the types of models you can merge. They can be of different types, as long as they are part of the same model context.

Preprocessing

After you add models to your context and before performing the search, you have to preprocess the context, using MmodPreprocess(), otherwise an error will occur. It does not matter the type of model, or the type of context. Calling MmodPreprocess() will prepare models for geometric matching. In the case of image-type models, it is also the time when edges are actually extracted from the model image.

Note that the Model Finder context must be preprocessed again if some of the models' search settings are changed. You can check if you need to preprocess after changing a control using MmodInquire() with M_PREPROCESSED.

When you save a Model Finder context, preprocessing information is not saved with the context. You must preprocess the context when the Model Finder context is restored.

Model origin

The model origin is the point in the model that is considered to be (0,0). The location of the model origin depends on the type of model: for image and Edge Finder-type models, the model origin is the center of the upper left pixel of the model image; for predefined-shape models, the center of the shape is the model origin. The model origin for a model defined from a CAD DXF file is the same as point (0,0) in the CAD DXF file.

Model indices and labels

Each model added to a Model Finder context can be accessed by its index. The first model is assigned an index of 0, and for each subsequent model added, the index is increased by one. You can also apply setting changes to all models within a Model Finder context by using M_ALL as the index. When a model is deleted, all model indices greater than the deleted model are shifted down by one. MIL searches for all models within a context in parallel, therefore the model index does not have any bearing in terms of a search order.

Models can also be given a user-defined numeric label, using MmodControl() with M_USER_LABEL. A user label can be used as a means of identifying your model, independently from its index in the Model Finder context. However, user labels cannot be used as a direct replacement for the index; to retrieve the index of a model from the user label, use MmodInquire() with M_INDEX_FROM_LABEL. The user label, once converted to an index value, can be used with any Mmod...() function that takes an index value. All user labels must be unique integers; that is, no two user labels can have the same integer. To remove a label from a model, set the user label to M_NO_LABEL.

Drawing and inquiring the model’s active edges

When testing different model source images for the best model candidate, you can use MmodDraw() to draw the model's active edges. This can be done for any type of model, although for synthetic models it is less useful, since the edges should already be known. The resulting edge map can reveal whether the edges you have chosen are the ones that you want, or if there are problems with the edges. You can use MmodInquire() with the M_ALLOC_SIZE... inquire types to determine the necessary size for the destination image buffer. To make corrections for image-type or Model Finder-type models, you might have to adjust the image processing control types, or mask unwanted edges. For Edge Finder-type models, you can exclude edges that you don't want from the Edge Finder result buffer and redefine the model, or you can mask unwanted edges.

If you have used offsets to define the model region in the model source, you can draw the active edges at the position where the model was defined using MmodDraw() with M_ORIGINAL; otherwise the active edges will be drawn at the top-left corner. This is valid for image-type or Edge Finder-type models.

You can use MmodInquire() with the M_CHAIN_INDEX, M_CHAIN_X, and M_CHAIN_Y controls to inquire the active edges of the model.

Note that if you try to draw a model's edges and its Model Finder context has not been preprocessed, a preliminary edge extraction operation will be performed for the model.

You can also draw features from a zoomed region of the model image, as well as other items. To do so, specify the appropriate values for the MmodControl() M_DRAW_RELATIVE_ORIGIN_X, M_DRAW_RELATIVE_ORIGIN_Y, M_DRAW_SCALE_X, and M_DRAW_SCALE_Y control types. The relative origin values must be specified in pixels, and are relative to the coordinates of the top-left corner of the region in the model image, while the scale values specify the X- and Y-scaling factors used to fill the destination image buffer.

IMAGE SMOOTHNESS DETAIL LEVEL MEDIUM DETAIL LEVEL HIGH VERY HIGH SMOOTHNESS DETAIL LEVEL SMOOTHNESS DETAIL LEVEL CAD Y AXIS PIXEL SCALE PIXEL SCALE PIXEL SCALE ASSOCIATED CALIBRATION VALID MODEL FINDER COMPATIBLE EDGE RESULT MOD RESULT MOD DEFINE COMPATIBLE MERGE MODEL PREPROCESSED ALL USER LABEL INDEX FROM LABEL NO LABEL ORIGINAL CHAIN INDEX CHAIN X CHAIN Y DRAW RELATIVE ORIGIN X DRAW RELATIVE ORIGIN Y DRAW SCALE X DRAW SCALE Y


Guidelines for choosing models

User Guide:
Determining what is a match
Not available in MIL-Lite

Available in MIL

While finding models based on geometric features is a robust, reliable technique, there are a few pitfalls which you should be aware of when allocating your models so that you choose the best possible model.

Make sure your images have enough contrast

Contrast is necessary for identifying edges in your model source and target image with subpixel accuracy. For this reason, it is recommended that you avoid models that contain only slow gradations in grayscale values.

Avoid poor geometric models

Poor geometric models suffer from a lack of clearly defined geometric characteristics, or from geometric characteristics that do not distinguish themselves sufficiently from other image features. These models can produce unreliable results.

Be aware of ambiguous models

Certain types of geometric models provide non-unique, or ambiguous, results in terms of position, angle, or scale.

Models which are ambiguous in position are usually composed of one or more sets of parallel lines only. Such models make it impossible to establish a unique position for them. A large number of matches can be found since the actual number of line segments in any particular line is theoretically limitless.

Models which are ambiguous in scale are usually composed only of straight lines which pass through the same point; some spirals are also ambiguous in scale. Models which consist of small portions of objects should be tested to verify that they are not ambiguous in scale.

For example, a model of an isolated corner is ambiguous in terms of scale because it consists of only two straight lines that pass through the same point.

Symmetric models are often ambiguous in angle due to their similarity in features. For example, circles are completely ambiguous in terms of angle. Other simple symmetric models, such as squares and triangles, are ambiguous with respect to certain angles:

Nearly ambiguous models

When the major part of a model contains ambiguous features, false matches can occur because the percentage of the occurrence's edges involved in the ambiguous features is great enough to be considered a match (see the Determining what is a match section later in this chapter). To avoid this, make sure that your models have enough distinct features to be found among other target features. This will ensure that only correct matches are returned as results. For example, the following model can produce false matches since the greater proportion of active edges in the model is composed of parallel straight lines rather than distinguishing curves.


Masking your model

Reference:
MmodDraw
MmodMask
Not available in MIL-Lite

Available in MIL

Once you have added a model to your Model Finder context, you can then mask any irrelevant, inconsistent, or featureless areas of your model using MmodMask().

You can define three types of masks: a "don't care", a "flat region", and a "weighted region" mask. You can use these types of masks with the same model. Note that "don't care" and "flat region" masks are not supported for synthetic models.

With a "don't care" mask (M_DONT_CARES), MIL ignores the masked regions when searching for occurrences of the model. These regions might be noise edges, unwanted edges, inconsistent features, or simply regions which are irrelevant to your search.

In the example below, a mobile phone is used as the model. However, occurrences of this model have displays containing inconsistent characters from target image to target image. Since these characters are variable, they should be masked in the model:

With a "flat region" mask (M_FLAT_REGIONS), the masked regions are expected to be featureless in the found occurrence, and any edges present in these regions will reduce the target score. For example, in a quality control application, features that are present in a region where none are expected can indicate that an error has occurred in the manufacturing process.

With a "weighted region" mask (M_WEIGHT_REGIONS), the extracted model features have weights according to the values of the "weighted region" mask. The weights of the features are used to calculate the score; although the weights themselves don't affect the target score, edges corresponding to negative weights will be ignored when calculating the target score. The valid range for the weights of the mask is -127 to +127.

Positive weights indicate how significant it is for a given pixel to be part of an edge in the occurrence; the more positive the weight, the greater the influence on the score. The absence of a pixel that corresponds to a very positive weight reduces the score more than the absence of one with a lower weight. The longer the edges with a positive weight, the more significant the edges to the target coverage and to the target score.

Negative weights indicate how significant it is for a given pixel not to be part of an edge of the occurrence; the more negative the weight, the more the influence on the score. The presence of a pixel that corresponds to a very negative weight will reduce the score more than the presence of one that corresponds to a less negative weight.

The effect of positive and negative weights on the score is relative. A "weighted region" mask that has negative weights of -100 and positive weights of 100 will have the same score as the same mask where the negative weight is -1 and the positive weight is 1.

Note that weights that have no correspondence to edges in the model have no effect on the score.

"Weighted regions" are particularly useful when the pattern you are searching for in the target also exists as a subset of other patterns; that is, you cannot otherwise identify the required pattern uniquely. In these cases, you have to create a model that includes the unwanted extra edges and mask the extra edges with negative weights.

In the example below, the square is the shape that is being searched for, while every other shape is unacceptable. The "weighted region" mask will cause the one acceptable shape to be found with a high score, while ensuring that the two other shapes that are similar are found with a low score.

A separate image buffer is used to create each mask and each buffer must be of the same size as the model. Note that if you modify the margin of a synthetic model's bounding box after a mask has been applied, the mask will be discarded since it is no longer the same size as the model. You must then re-apply a mask that is of the same size as the new model box.

For a "don't care" or "flat region" mask, the buffer must be a 1-band 8-bit unsigned buffer, where any non-zero pixel value is considered a masked pixel. For a "weighted region" mask, the buffer must be a 1-band 8-bit signed buffer and the valid range for the weights of the mask is -127 to +127. This buffer can be calibrated; however, even if the model is calibrated, the mask is applied on a pixel basis.

Note that if you expect significant misalignment between the model and the target, you might need to compensate for this by slightly increasing the region of the "don't care" or "flat region" pixels in the mask.

If you want to verify the mask that has been applied to a model, you can use MmodDraw() to draw the model's masked pixels. If necessary, you can clear a model's current mask by setting the mask to M_NULL.

Finally, when you change the masked pixels of a model, you must preprocess the search model again.

DONT CARES FLAT REGIONS WEIGHT REGIONS


Search targets

User Guide:
Extracting edges

Reference:
MedgeAllocResult
MedgeCalculate
MedgeControl
MmodControl
MmodFind
Not available in MIL-Lite

Available in MIL

You can search for instances of a model in an image, or you can search for instances of it in an Edge Finder result buffer. In both cases, use MmodFind() to find the model. This function will write the results of the search in the specified Model Finder result buffer.

Finding models in an image

To search for instances of models in an image, the target image must be a 1-band, 8-bit unsigned image. The minimum and maximum target image sizes are 16x16 and 32768x32768 pixels, respectively. It is important to note that the minimum and maximum target image sizes are MIL limits. To make sure that you have enough memory, you can call MmodFind(); if you don't have enough memory, you will get a MIL error.

The target image must be in the same calibrated state as the models. If the models are calibrated, the target must be; if the models are not calibrated, the target must not be.

You can search for a synthetic model in a calibrated or non-calibrated target image. If the target is calibrated, the model units should be the same as the calibrated units. In addition, you must associate the calibration object of the target image with the model using MmodControl() with M_ASSOCIATED_CALIBRATION. MIL needs the calibration object for internal purposes at preprocessing time. Note that MIL will not use the calibration object to compensate for incongruencies between the model and the target, nor will MIL map the model's coordinate system to that of the target.

Note that the edge extraction process, applied to the target image, uses the same denoising operation and detail level as the one for all image-type models in the Model Finder context. For more information, see the Extracting edges subsection in the Defining and adding models to your Model Finder context section earlier in this chapter.

Finding models in an Edge Finder result buffer

When using an Edge Finder result buffer as the target, Model Finder searches for active edges of the models only in the included edges of the result buffer. Therefore, using the Edge Finder result buffer as a target, instead of an image, allows you to be more specific about the edges in which to search. For example, you could select out unwanted edges in the target, which will speed up the search and further calculations.

Some things have to be done before you can search for instances of a model in an Edge Finder result buffer.

  • The Edge Finder result buffer must have been previously allocated using MedgeAllocResult().
  • The Edge Finder result buffer must be compatible with the Model Finder context. Enable this using MedgeControl() with M_MODEL_FINDER_COMPATIBLE.
  • MedgeCalculate() must have already been called using this result buffer.

If the models of the Model Finder context are calibrated, the results in the Edge Finder result buffer must also be calibrated. If the models are not calibrated, the results must not be calibrated either. In the case of synthetic models, the same rules apply as when using a calibrated image as a target.

ASSOCIATED CALIBRATION MODEL FINDER COMPATIBLE


Determining what is a match

Availability | Previous | Next
Not available in MIL-Lite

Available in MIL

Before customizing your search settings, it is necessary to understand how a match between your model and occurrences in the target is determined. The score (M_SCORE) and the target score (M_SCORE_TARGET) are the primary factors in determining which occurrences are considered matches with the models in your Model Finder context.

Score and target score

The score is a measure of the active edges in the model found in the occurrence, weighted by the deviation in position of these common edges. Active edges in the model not found in the occurrence reduce the score. The target score is a measure of edges found in the occurrence that are not present in the original model (that is, extra edges), weighted by the deviation in position of the common edges. Edges found in the occurrence that are not present in the model will reduce the target score. The model scores are calculated as follows:

  • Score = Model coverage x (1 - (Fit error weighting factor x Normalized Fit Error))
  • Target Score = Target coverage x (1 - (Fit error weighting factor x Normalized Fit Error))

    • Note: the normalized fit error is the fit error converted to a number between 0.0-1.0.

The model coverage, target coverage, and fit error components of the score and target score are explained below.

Model and target coverage

The model coverage and target coverage are defined as follows:

  • Model coverage. The model coverage is the percentage of the total length of the model's active edges found in the occurrence. 100% indicates that for each of the model's active edges, a corresponding edge was found in the occurrence.
  • Target coverage. The target coverage is the percentage of the total length of the model's active edges found in the occurrence, divided by the total length of edges present within the occurrence's bounding box. Thus, a target coverage score of 100% means that no extra edges were found. Lower scores indicate that features or edges found in the target (result occurrence) are not present in the model.

Using a weighted mask with a model can affect the score and target score. In cases where a weighted mask is used, the model coverage would be as below:

The target coverage is as follows:

Fit error

Fit error. The fit error is a measure of how well the edges of the occurrence correspond to those of the model. The fit error is calculated as the average quadratic distance, in pixels or calibrated units, between the edgels in the occurrence and the corresponding active edges in the model:

A perfect fit gives a fit error of 0.0. The fit error weighting factor (between 0.0 - 100.0) determines the importance to place on the fit error when calculating the score and target score.

Interpreting results

The following diagram illustrates how the model coverage, target coverage, and fit error work together to provide details about the nature of a result occurrence.

SCORE SCORE TARGET


Position, angle, and scale

User Guide:
Reference axis

Reference:
MmodControl
MmodGetResult
Not available in MIL-Lite

Available in MIL

You can control the position, angle, and scale at which Model Finder searches for each model in a target.

You can restrict the position, angle, and scale at which model occurrences can be found, to an expected (nominal) position, angle, or scale or to a given range.

Enabling calculations specific to searching within a range

Depending on whether searching for models within a range of positions, angles, and/or scales, Model Finder uses different search strategies to evaluate the edge-based features of the target candidates. Typically, to search for models within a range, calculations specific to the corresponding search strategy should be enabled for the context (MmodControl() with M_SEARCH_POSITION_RANGE, M_SEARCH_ANGLE_RANGE, and/or M_SEARCH_SCALE_RANGE). If you expect that the occurrences sought are close to the specified nominal position, angle, or scale (for example, in a registration application), you can try disabling the calculations specific to the corresponding search strategies to see if Model Finder can still find the required occurrences. Disabling one or more of these calculations might speed up the search depending on the model and the target.

Note that candidates can only be returned as results if found within the ranges specified for their model. Therefore, if you have disabled calculations specific to a search strategy and occurrences at positions, angles, and/or scales outside of a required range are being returned, you can restrict which candidates are returned as occurrences by narrowing the appropriate ranges.

Search position and position range

You can search for each model defined in a Model Finder context at a specific position, or within a position range. The position range limits the region in which the position of a model occurrence can be found; position coordinates which fall outside this region cannot be returned as results (MmodGetResult() with M_POSITION_X and M_POSITION_Y). Note that the position returned for an occurrence is determined by the model's reference axis origin; by default, this position is set to the center of the model, however it can be displaced if necessary (see the Reference axis subsection in the Customizing search settings section later in this chapter).

The region defined by the default position range is the entire image plane (M_INFINITE) in the positive X- and Y-direction, meaning that all X- and Y-coordinates in the positive direction (even outside the target if applicable) can be returned as results. You can use the MmodControl() M_POSITION_DELTA_NEG_X, M_POSITION_DELTA_NEG_Y, M_POSITION_DELTA_POS_X, and M_POSITION_DELTA_POS_Y control types to set the position range relative to the nominal position. You can, if necessary, set a different position range for each model in your context.

Depending on the number of details present in the target, using a small position range generally decreases the search time. Always set the position range to the minimum required when speed is a consideration; M_POSITION_DELTA_NEG_X, M_POSITION_DELTA_NEG_Y, M_POSITION_DELTA_POS_X, and M_POSITION_DELTA_POS_Y can be greater or equal to zero. If they are all set to zero, the occurrence must be at the position specified by the nominal position.

Note that you can specify a position range which defines a region partially, or totally, outside of the target; this might be necessary, depending on the reference axis origin of the model.

By default, calculations specific to position-range search strategies are enabled. If you are searching for a single occurrence of each model in your context and you know the nominal position of each (for example, registration applications), you can disable the calculations specific to position-range search strategies for the context, using MmodControl() with M_SEARCH_POSITION_RANGE set to M_DISABLE. When disabled, you must specify a good nominal position for each model (using MmodControl() with M_POSITION_X and M_POSITION_Y). You can restrict which candidates are returned as occurrences by narrowing the position-range.

Angle and angular range

You can search for each model defined in a Model Finder context at a specific angle, or within an angular range. For each model in your context, you can specify the angle of the search, using MmodControl() with M_ANGLE. By default, the search angle is 0°.

You can also search within the full angular range of 360° from the nominal angle specified with M_ANGLE. Use the MmodControl() M_ANGLE_DELTA_POS and M_ANGLE_DELTA_NEG control types to specify the angular range in the counter-clockwise and clockwise direction from the nominal angle, respectively; the default for both is 180°. The angular range limits the possible angles which can be returned as results for an occurrence. Note that the actual angle of the occurrence does not affect search speed. If you need to search for a model at discrete angles only (for example, at intervals of 90 degrees), it is typically more efficient to define several models with different expected angles, than to search through the full angular range.

By default, calculations specific to angular-range search strategies are enabled. If you expect that the occurrences sought are close to the specified nominal angle, you can disable these calculations using MmodControl() with M_SEARCH_ANGLE_RANGE set to M_DISABLE. When disabled, you must specify a good nominal angle for each model, which is within the model's angular range. You can restrict which candidates are returned as occurrences by narrowing the angular-range.

Scale and scale range

The scale of the model establishes the size of the occurrence that you expect to find in the target. If the expected occurrence is smaller or larger than that of the model, you can set the nominal scale of the occurrence for each individual model, using MmodControl() with M_SCALE. The supported scale factors are 0.5 to 2.0.

When the scale of occurrences can vary around the specified nominal scale, you can enable calculations specific to scale-range search strategies for the context using MmodControl() with M_SEARCH_SCALE_RANGE set to M_ENABLE. To specify the range of scales, use MmodControl() with M_SCALE_MAX_FACTOR (1.0 to 2.0) and M_SCALE_MIN_FACTOR (0.5 to 1.0). The minimum factor and the maximum factor together determine the scale range from the nominal scale (M_SCALE).

The maximum and minimum factors are applied to the M_SCALE setting as follows:

Maximum scale = (M_SCALE) x (M_SCALE_MAX_FACTOR)

Minimum scale = (M_SCALE) x (M_SCALE_MIN_FACTOR)

Note that the range is defined as factors so that if you change the expected scale (M_SCALE), you do not have to modify the range. A search through a range of scales is performed in parallel, meaning that the actual scale of an occurrence has no bearing on which occurrence will be found first.

When calculations specific to scale-range search strategies are enabled, the scale range should be used to cover an expected variance in the scale; you should not use the scale range to cover different expected scales at different positions. In this case, it is typically more efficient to define several models with different expected scales. This is because a large scale range could potentially slow down your operation; as well, you could find unwanted occurrences.

By default, calculations specific to scale-range search strategies are disabled. When disabled, you must specify a good nominal scale for each model, which is within the model's scale range. Note that occurrences can still be found within the scale range specified for their model. You can restrict which candidates are returned as occurrences by narrowing the scale-range.

SEARCH POSITION RANGE SEARCH ANGLE RANGE SEARCH SCALE RANGE POSITION X POSITION Y INFINITE POSITION DELTA NEG X POSITION DELTA NEG Y POSITION DELTA POS X POSITION DELTA POS Y POSITION DELTA NEG X POSITION DELTA NEG Y POSITION DELTA POS X POSITION DELTA POS Y SEARCH POSITION RANGE DISABLE POSITION X POSITION Y ANGLE ANGLE ANGLE DELTA POS ANGLE DELTA NEG SEARCH ANGLE RANGE DISABLE SCALE SEARCH SCALE RANGE ENABLE SCALE MAX FACTOR SCALE MIN FACTOR SCALE SCALE SCALE SCALE MAX FACTOR SCALE SCALE MIN FACTOR SCALE


Customizing search settings

User Guide:
Fit error weighting factor
Search position and position range
Using transformation coefficients with calibrated images

Reference:
MmodControl
MmodFind
MmodInquire
Not available in MIL-Lite

Available in MIL

With successive calls to MmodControl(), you can customize the global search settings of a Model Finder context, the search settings of each individual model in the context, and the general settings of a Model Finder result buffer; specify which settings to modify using the Index parameter (M_CONTEXT, the model's index, or M_GENERAL, respectively). You can also customize a search setting for all the models in a context by setting the Index parameter to M_ALL. However, when using M_ALL, ensure that the specified setting is appropriate for all models. You can inquire about any Model Finder context setting, individual model setting, or Model Finder result buffer setting using MmodInquire().

This section discusses the fundamental model search settings that are usually necessary to adjust (and not previously discussed) and their inter-related global context settings. Other settings are discussed later.

Fundamental model search settings include the acceptance and certainty levels which determine what is considered a match, the expected number of occurrences to find, the reference axis which determines the position returned for a match. Most of the model search settings can affect the speed and robustness of your application. It is recommended that you begin with the default settings, and then, as your application demands, adjust the individual settings as required.

Acceptance levels

For each model, acceptance levels can be set for both the score and target score. The acceptance levels determine the minimum scores required for an occurrence to be considered a match.

MIL will search the target for the required number of occurrences, returning the occurrences with the best scores above the acceptance levels. If the score or target score of an occurrence is less than the acceptance levels, it is not considered a match and no result will be returned.

You can set the acceptance level for the score of a specified model, using MmodControl() with M_ACCEPTANCE. For example, a setting of 100% means that you will only accept occurrences that contain every active edge in the model with a perfect fit. That is, for every active edge in the model, an equivalent edge must be found in the occurrence with a perfect fit. However, perfect matches are generally unobtainable in real images because of noise and distortion introduced when grabbing images. You should use a reasonable acceptance level that is high enough to avoid false matches, but not so high that occurrences are missed. If your images have considerable noise and/or distortion, or if occlusion of occurrences is expected, you might have to set the level below the default value of 60%.

You can set the target score required using M_ACCEPTANCE_TARGET. A setting of 100% for the target score means that you will not tolerate any extra edges, and common edges will have a perfect fit. A setting of 0% (default) allows for any number of extra edges. Essentially, the target score acceptance level allows you to control how tolerant your search is to details not present in the original model.

Note that the contribution of the fit can be controlled using MmodControl() with M_FIT_ERROR_WEIGHTING_FACTOR. For more information, see the Fit error weighting factor subsection in the Advanced search settings section later in this chapter.

Certainty levels

The certainty levels are used to speed up the search when very good matches are expected often. Any occurrence found above the certainty levels is considered a certain match.

The certainty levels determine the score and target score equal to or above which the algorithm can assume that it has found a certain match. Both of these scores must be equal to or above their respective certainty levels for an occurrence to be considered a certain match. If the required number of occurrences has been found equal to or above these certainty levels, MIL stops searching the target for better ones, avoiding exhaustive checking of all possible candidates. If not, MIL will continue to search for the required number of matches equal to or above the acceptance level and with the best score.

You can set the certainty level for the score of a specified model using MmodControl() with M_CERTAINTY (the default setting is 90%). You can set the certainty level for the target score of a specified model using M_CERTAINTY_TARGET (the default value is 0%).

If necessary, when searching for a fixed number of occurrences in a target, you can ensure that MIL always returns those occurrences with the highest match score among those found, by setting the certainty levels of each individual model to 100%. MIL will return occurrences with the best score(s) over the acceptance or those with scores of 100%. Note that doing so will probably increase the search time.

Expected number of occurrences

You can set the expected number of occurrences for the Model Finder context, and for each individual model in the Model Finder context.

For each individual model in the context, you can set the expected number of occurrences of the model in the target. To do so, use MmodControl() with M_NUMBER, specifying the index of the particular model. The default value is 1. To find all occurrences of a model, set M_NUMBER to M_ALL.

The number set for the context specifies the maximum total of all model occurrences (for all models within the context together). To set the number for a Model Finder context, use MmodControl() with M_NUMBER, specifying M_CONTEXT as the index. The default value is M_ALL, which finds the expected number of occurrences specified for each model.

Note that setting the number of occurrences for the context to M_ALL and the number of occurrences for one of the models to M_ALL can significantly slow the MmodFind() operation, depending on the complexity of your model and the target (operation speed will only be slightly affected for simple models on a uniform background). It is recommended that you specify the exact number of expected occurrences whenever possible.

MIL searches for all models within a context in parallel, therefore the model index does not have any bearing in terms of a search order. Once the number of occurrences for the context has been found, with scores above the certainty levels, the search will stop.

For example, in an application involving a number of different objects being examined one at a time on a conveyor belt, the actual object present in the target at any time is unknown. Since only one occurrence is expected at any one time, the Model Finder context's number would be set to one, as would the number for all the models within the context.

MIL will search the target for all three models; the first occurrence of any of these models, found equal to or above the certainty level, will stop the search.

To further illustrate the relationship between the M_NUMBER setting for the context and that of the individual models, the tables below show the maximum possible results which can be returned for different context M_NUMBER settings.

Model Finder Context Element | Setting for M_NUMBER | Maximum possible results
Context | M_ALL | All
Model 0 | 1 | 1
Model 1 | M_ALL | All
Model 2 | 2 | 2

The following table shows that if certain M_NUMBER settings are specified, different combinations of maximum number of results are possible.

Model Finder Context Element | Setting for M_NUMBER | Ex. 1 | Ex. 2 | Ex. 3 | Ex. 4 | Ex. 5 | Ex. 6
Context | 6 | 6 | 6 | 6 | 6 | 6 | 6
Model 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1
Model 1 | M_ALL | 6 | 5 | 5 | 4 | 4 | 3
Model 2 | 2 | 0 | 0 | 1 | 1 | 2 | 2

Reference axis

When an occurrence of a model is found, the model's reference axis determines the coordinates and angle returned as the actual occurrence's found position and angle. Position results return the coordinates of the model's reference axis origin transformed at the model occurrence. Angle results are also returned relative to the reference axis, rather than the model source image axis.

Reference axis origin

By default, the reference axis origin, for an image-type or result-type model, is at the center of the model box. For a synthetic model, the reference axis origin is, by default, at the model origin.

If the default reference axis is not at a practical position for your application, you can change the position of the reference axis origin using MmodControl() with M_REFERENCE_X and M_REFERENCE_Y. Specify coordinates relative to the model origin; position results are returned relative to the origin of the target.

Modifying the reference axis origin can be useful if you cannot define a unique model for an object of interest. In this case, define a model that is unique and at a fixed location relative to your object of interest and move your reference axis.

In the diagram below, the easily found chip is used as the model, and the model's reference axis origin is shifted to the location of the diode.

The reference axis origin does not have to reside within the model, meaning that you can place the reference axis origin outside of the model boundaries.

Note that the found position of an occurrence respects differences in angle and scale, meaning that the found position is relative to the angle and scale of the occurrence.

When changing the reference axis origin, make sure to update the position range if necessary, particularly when searching through a range of angles and/or scales. The position range limits the region in which the position of a model occurrence can be found; position coordinates which fall outside this region cannot be returned as results (see the Search position and position range subsection in the Position, angle, and scale section earlier in this chapter). In the diode example presented above, the position range must include, and can be limited to, a small area including the diode for the chip to be found.

Reference axis angle

Often, when allocating models, the angle of the object in your model source image is not aligned with the horizontal image axis.

When occurrences in your target are successfully located, the angle of the occurrence returned is with respect to the model's reference axis. The actual angle of your object of interest in the model is not taken into consideration. You can align the reference axis with the object of interest using MmodControl() with M_REFERENCE_ANGLE. Angle results will then reflect the actual angle of your object of interest.

In the example above, setting the reference axis angle to 4.0 degrees will ensure that the angle returned for the occurrence will represent the actual angle of the object.

Forward and reverse transformation coefficients

Forward and reverse transformation coefficients are available (as results) for mapping additional points of interest other than the reference axis origin. These coefficients allow you to convert coordinates in the model coordinate system to the corresponding coordinates in the target coordinate system for that occurrence (or vice versa). These coefficients handle variations in scale, translation, and angle, allowing for easier mapping of critical points between model and occurrence.

Use the following equations:

xd = axs + bys + c

yd= -bxs + ays + d

where:

  • a, b, c, d: Transformation coefficients (forward or reverse).
  • xs and ys: Source coordinates (with respect to the origin of the model coordinate system for a forward transformation or target coordinate system for a reverse transformation).
  • xd and yd: Destination coordinates (with respect to the origin of the target coordinate system for a forward transformation or model coordinate system for a reverse transformation).

If the model is calibrated, these coefficients are given for the real world. For more information, see the Using transformation coefficients with calibrated images subsection in the Calibration section later in this chapter.

CONTEXT GENERAL ALL ALL ACCEPTANCE ACCEPTANCE TARGET FIT ERROR WEIGHTING FACTOR CERTAINTY CERTAINTY TARGET NUMBER NUMBER ALL NUMBER CONTEXT ALL ALL ALL NUMBER NUMBER REFERENCE X REFERENCE Y REFERENCE ANGLE


Advanced search settings

User Guide:
Angle and angular range
Determining what is a match

Reference:
MmodControl
Not available in MIL-Lite

Available in MIL

In addition to the fundamental search settings, the Model Finder module also provides advanced settings that allow you to specify whether the polarity of the edges in a model differs from those in occurrences, how much or little separation between occurrences is permitted, as well as other information.

Polarity

The polarity of an edge indicates whether edges occur as transitions from light to dark or vice versa. When the polarity of edges in the target might be different from those of the model (for example, due to shadows), you should specify the expected polarity using MmodControl() with M_POLARITY. This control type allows you to specify whether edges present in the model and in the occurrence have the same, the reverse, either of these, or a mixture of both polarities.

The default polarity setting is M_SAME.

The polarity setting can be useful when dealing with adverse lighting conditions where dark shadows can cause edges to vary in polarity.

Separation

You can specify the minimum amount of separation from other occurrences (of the same model) for an occurrence to be considered distinct (a match). In essence, this determines what amount of overlap by different model occurrences is possible.

You can set the minimum separation for four criteria, which are: the X position, Y position, angle, and scale. These can be set for each individual model in your Model Finder context. For an occurrence to be considered distinct from another, only one of the minimum separation conditions needs to be met. For example, if the minimum separation in terms of angle is met, then the occurrence is considered distinct, regardless of the separation in position or scale. However, each of these separation criteria can be disabled (M_DISABLE) so that it is not considered when determining a valid occurrence.

DISTINCT OCCURRENCE = (Separation in X) OR (Separation in Y) OR (Separation in Angle) OR (Separation in Scale)

The minimum positional separation (MmodControl() with M_MIN_SEPARATION_X and M_MIN_SEPARATION_Y) determines the minimum distance between the found positions of two occurrences of the same model. This separation is specified as a percentage of the model size at the nominal scale (M_SCALE). The minimum value that you can set for M_MIN_SEPARATION_X and M_MIN_SEPARATION_Y is 0, which means that there is no minimum distance needed for occurrences to be distinct; all occurrences will be considered distinct regardless of other separation conditions. You can set very large values for M_MIN_SEPARATION_X and M_MIN_SEPARATION_Y; if they are large enough, for example, if the values are larger than the size of the image, it is equivalent to setting this separation condition to M_DISABLE since this condition will never be met. In this case, whether an occurrence is distinct or not will depend on the other separation conditions.

The minimum angular separation (M_MIN_SEPARATION_ANGLE) determines the minimum difference in angle between occurrences. This value is specified as an absolute angle value. The default value is 10.0o. The minimum value that you can set for M_MIN_SEPARATION_ANGLE is 0, which means that there is no minimum difference in angle needed for occurrences to be distinct; all occurrences will be considered distinct regardless of other separation conditions. The maximum absolute angle difference is 180 degrees. For a diagram of the delta convention used in MIL, see the Angle and angular range subsection in the Position, angle, and scale section earlier in this chapter. If the angle is larger than 180, an error will be returned. At 180 degrees, it is equivalent to setting this separation condition to M_DISABLE. In this case, whether an occurrence is distinct or not will depend on the other separation conditions.

The minimum scale separation (M_MIN_SEPARATION_SCALE) determines the minimum difference in scale between occurrences, as a scale factor. The default value is 1.1. The minimum value that you can set for M_MIN_SEPARATION_SCALE is 1, which means that there is no minimum difference in scale needed for occurrences to be distinct; all occurrences will be considered distinct regardless of other separation conditions. The maximum value that you can set for M_MIN_SEPARATION_SCALE is 4, which is equivalent to setting this separation condition to M_DISABLE. In this case, whether an occurrence is distinct or not will depend on the other separation conditions.

The four criteria are summarized below in equation form.

The following example illustrates the four separation criteria; an arrow represents two occurrences of the same model and the various types of separation possible.

It should be noted that for a model to be found, the number of visible edges in the occurrence must be sufficient to provide a match according to your acceptance levels.

Shared edges

You can choose to allow occurrences to share edges, using MmodControl() with M_SHARED_EDGES set to M_ENABLE. Otherwise, edges that can be part of more than one occurrence are considered part of the occurrence with the greatest score. For example, in the illustration below, two occurrences of two simple models share a common edge. With shared edges enabled, these occurrences would have perfect scores.

However, with shared edges disabled (default), the shared edge would be considered part of occurrence 1, since it has the greater score; the score of occurrence 2 would be subsequently reduced by the loss of the shared edge in the score calculation.

Fit error weighting factor

The fit error weighting factor, MmodControl() with M_FIT_ERROR_WEIGHTING_FACTOR, determines the relative importance of the fit error when calculating the match score and the target score. The higher the factor, the greater the influence that the fit error has on the resulting score and target score for an occurrence (see the Determining what is a match section earlier in this chapter). Setting this factor to 0.0 means that the fit error is not considered in the score calculation, so the score is equal to the coverage score. Setting this factor to 100.0 means that the fit error has a maximum contribution in the score. The default value is 25.0%.

First and last levels

When using an M_GEOMETRIC_CONTROLLED Model Finder context, model searches use a hierarchical method to reduce the time needed to complete the search. Using this method, Model Finder produces a series of smaller, lower-resolution versions of both the target and the models, and the search begins on a much-reduced scale. This series of sub-sampled images is sometimes called a resolution pyramid because of the shape formed when the different resolution levels are stacked on top of each other.

The resolution level for the initial and final stages of the search are typically set automatically. These default levels usually provide the best results for search operations, and you should leave them in most circumstances. If required, you can set the resolution level for the initial and final stages of the search, using MmodControl() with M_FIRST_LEVEL or M_LAST_LEVEL. You can set the first and last levels to any integer value from 0 to 7. Level 0 is the original target size and each higher level is half the size (and resolution) of the previous one. The higher the level, the faster the initial search; however, the search will be less reliable at higher levels.

Very small models and search regions can not be subsampled as many times as large ones. If you specify too high a level, the highest practical level will be used.

The M_FIRST_LEVEL and M_LAST_LEVEL controls are not supported if the Model Finder context uses a general geometric search algorithm.

POLARITY SAME MIN SEPARATION X MIN SEPARATION Y SCALE MIN SEPARATION X MIN SEPARATION Y MIN SEPARATION X MIN SEPARATION Y MIN SEPARATION ANGLE MIN SEPARATION ANGLE DISABLE MIN SEPARATION SCALE MIN SEPARATION SCALE MIN SEPARATION SCALE DISABLE SHARED EDGES ENABLE FIT ERROR WEIGHTING FACTOR GEOMETRIC CONTROLLED FIRST LEVEL LAST LEVEL FIRST LEVEL LAST LEVEL


Global context settings

Reference:
MmodControl
MmodGetResult
Not available in MIL-Lite

Available in MIL

The global search settings of a Model Finder context determine the strategy used to search for all of the context's models in the target. These settings directly affect the speed and robustness of the search. The context settings can be adjusted to fit your individual application's needs, using MmodControl() with the index parameter set to M_CONTEXT. We recommend experimenting with different settings to find the particular settings that provide your application with the necessary robustness.

This section discusses some of the Model Finder context settings that were not previously discussed in this chapter.

Setting the search speed

You can specify the algorithm's search speed, using MmodControl() with M_SPEED. The speed can be set to M_MEDIUM (default), M_HIGH, or M_VERY_HIGH. When you preprocess the context, MIL analyzes the patterns of the models and determines which shortcuts are appropriate for the search speed setting. At higher search speed settings, the search can take all reasonable shortcuts; therefore, when possible, the search is performed faster than at lower speeds. You can typically use higher speed settings when high accuracy is not an issue; this is because increasing the speed might affect the robustness, as well as the accuracy, of the search.

Accuracy

You can set the positional accuracy for your search, using MmodControl() with M_ACCURACY. It can be set to:

  • M_MEDIUM (typically ± 0.03 pixel)
  • M_HIGH (typically ± 0.02 pixel)

Note, the actual precision achieved is dependent on the quality of the model and of the target (the values listed above are typical when using a high-quality, well-contrasted, low-noise image as the target). The default setting is M_MEDIUM.

Note that increasing the accuracy can slightly reduce the search speed.

Timing out your search

In time critical applications, you can set a time limit in milliseconds for Model Finder to find occurrences of the specified models, using MmodControl() with M_TIMEOUT. After the time limit, the search will stop even if the required number of occurrences is not found. Results are still returned for those occurrences found up to the timeout. However, it is not possible to predict which occurrences will be found before the time limit is reached. The default value is 2000 msecs.

You can use MmodGetResult() with M_TIMEOUT_END in order to check whether the timeout limit has been reached.

CONTEXT SPEED MEDIUM HIGH VERY HIGH ACCURACY MEDIUM HIGH MEDIUM TIMEOUT TIMEOUT END


Speeding up the search

Reference:
MmodControl
MmodMask
Not available in MIL-Lite

Available in MIL

There are some things you should do to help ensure that your search runs as quickly as possible:

  • Adjust the search speed of the algorithm.
  • Disable calculations specific to range search strategies, if possible.
  • Define several models with different expected angles.
  • Limit the position range.
  • Limit the search scale.
  • Specify the exact number of expected occurrences.
  • Clean up your model.
  • Change the search levels (for controlled geometric Model Finder contexts only).

Adjust the search speed of the algorithm

You can control the search speed of the algorithm used by your Model Finder context, using MmodControl() with M_SPEED. When you preprocess the context, MIL analyzes the patterns of the models and determines which shortcuts are appropriate for the search speed setting. At higher search speed settings, the search can take all reasonable shortcuts; therefore, when possible, the search is performed faster than at lower speeds. You can typically use higher speed settings when high accuracy is not an issue; this is because increasing the speed might affect the robustness and accuracy.

Disable calculations specific to range search strategies

If you expect that the occurrences sought are close to the specified nominal position, angle, or scale (for example, in a registration application), you can try disabling the calculations specific to the search strategies for the corresponding range (MmodControl() with M_SEARCH_POSITION_RANGE, M_SEARCH_ANGLE_RANGE, and/or M_SEARCH_SCALE_RANGE) to see if Model Finder can still find the required occurrences. Disabling one or more of these calculations might speed up the search depending on the model and the target.

Define several models with different expected angles

The actual angle of the occurrence does not affect search speed. If you need to search for a model at discrete angles only (for example, at intervals of 90 degrees), it is typically more efficient to define several models with different expected angles, than to search through the full angular range.

Limit the position range

Limiting the position range of the model to the minimal required generally decreases the search time. Set the position range of each model to the minimum needed, using MmodControl() with the M_POSITION_DELTA_NEG_X, M_POSITION_DELTA_NEG_Y, M_POSITION_DELTA_POS_X, and M_POSITION_DELTA_POS_Y control types. If the occurrence must be at a specific position, set these control types to zero.

Limit the search scale

It is best to set a fixed scale at which to search for occurrences of a model, instead of searching in a range. Sometimes, you must search for model occurrences that can vary in size; in these cases, enabling MmodControl() with M_SEARCH_SCALE_RANGE might be necessary. In these cases, keep the scale range as small as possible to avoid slowing down your search. You should not use the scale range to cover different expected scales at different positions. In this case, it is typically more efficient to define several models with different expected scales. This is because a large scale range could potentially slow down your operation; as well, you could find unwanted occurrences.

Specify the exact number of expected occurrences

If you know the exact number of occurrences for which to search, specify it using MmodControl() with M_NUMBER. This will typically reduce search time. If you search for more occurrences than required, the search might take longer than necessary.

Clean up your model and your target

Typically, reducing the number of unwanted edges in your models and your target will accelerate your search. There are several different ways to clean up your model and your target:

  • Reduce the noise. Noise in your models can cause inaccurate results. In addition, noise in your models and/or your target can also increase the length of time for a search. The most efficient way to remove noise is to use the MmodControl() M_SMOOTHNESS and M_DETAIL_LEVEL control types. These affect all image-type models in your context, as well as the target when it is an image.
  • Use masking. You can use MmodMask() to mask regions of the model. Using MmodMask(), you can effectively remove areas of the model from consideration. This might help speed up your search.
  • Use the Edge Finder module. You can use the Edge Finder module to extract only the required edges from the image that you want to use as your model source or target, and then use the Edge Finder results as the model source or target instead.

Changing the search levels

For controlled geometric Model Finder contexts, it is possible to speed up the search by setting the resolution levels for the search. This can be done by using MmodControl() with either M_FIRST_LEVEL or M_LAST_LEVEL. This should be done cautiously; the default settings usually provide the best results for search operations.

SPEED SEARCH POSITION RANGE SEARCH ANGLE RANGE SEARCH SCALE RANGE POSITION DELTA NEG X POSITION DELTA NEG Y POSITION DELTA POS X POSITION DELTA POS Y SEARCH SCALE RANGE NUMBER SMOOTHNESS DETAIL LEVEL FIRST LEVEL LAST LEVEL


Retrieving and analyzing results

User Guide:
Annotating the displayed image non-destructively
Generating graphics
Defining and adding models to your Model Finder context

Reference:
MmodControl
MmodDraw
MmodFind
MmodGetResult
Not available in MIL-Lite

Available in MIL

After having successfully located your model occurrences in your target using MmodFind(), you can extract the required results from your result buffer using MmodGetResult().

Possible results

Model Finder provides several types of results which provide considerable information on the nature of the occurrence found. In addition to the score, target score, model coverage, target coverage, fit error, and forward/reverse transformation coefficients discussed previously in this chapter, results can be returned for:

  • Number of occurrences.
  • Index or user label of occurrences.
  • Position X and position Y.
  • Scale.
  • Polarity.
  • Angle.
  • Whether timeout limit has been reached.

Results are returned in descending order of match score, such that the result with the highest score is returned first. Generally, you should first retrieve the total number of occurrences found for all the models in the Model Finder context, to ascertain the size of the result array needed. Note that results can be returned for the entire context; the model index (M_INDEX result type) is used to differentiate results between models. For a complete description of all possible results, refer to the description of MmodGetResult() in the MIL Reference.

Results are indexed as positive integers starting from 0.

Drawing results

The MmodDraw() function provides several operations for drawing results in any specified image buffer. You can also choose to draw in the display's overlay buffer. By drawing into the display's overlay buffer, you can annotate an image non-destructively (see the Annotating the displayed image non-destructively section in Chapter 16: Displaying an image). You can also draw zoomed results. You can draw a model box, or a bounding box around the occurrence, the active edges of the model, the edges of the result occurrence, or draw a cross-like symbol at the model's reference axis origin or occurrence position.

Typically, all drawing operations in MmodDraw() can be combined; you can therefore draw multiple results simultaneously. For example, to draw the edges of the result occurrence and its position, you would specify M_DRAW_EDGES and M_DRAW_POSITION.

When drawing results, you can draw the active edges of the model transformed at the occurrence position, or the edges of the target in the region of the occurrence.

You can also draw a zoomed version of results that were obtained from a specified region in the target. To do so, specify, the appropriate values for the MmodControl() M_DRAW_RELATIVE_ORIGIN_X, M_DRAW_RELATIVE_ORIGIN_Y, M_DRAW_SCALE_X, and M_DRAW_SCALE_Y control types. The relative origin values must be specified in pixels, and are relative to the cooordinates of the top-left corner of the region in the model source, while the scale values specify the X- and Y-scaling factors used to fill the destination buffer. For more information on zooming, see the Defining and adding models to your Model Finder context section earlier in this chapter.

You can use a previously allocated graphics context (see Chapter 17: Generating graphics) to control the drawing color, or use the default graphics context (M_DEFAULT).

INDEX DRAW RELATIVE ORIGIN X DRAW RELATIVE ORIGIN Y DRAW SCALE X DRAW SCALE Y


Calibration

User Guide:
Forward and reverse transformation coefficients

Reference:
McalTransformCoordinate
MmodControl
MmodDraw
MmodGetResult
Not available in MIL-Lite

Available in MIL

The Model Finder module supports calibration. If the target is calibrated, matching is performed and results are calculated in the world coordinate system, compensating for any image distortion. Calibration is performed transparently, without the need for any setting adjustments.

Requirements

To search in a calibrated target with a Model Finder context, all models of the context must also be calibrated. To calibrate image-type, result-type, and merge-type models, use calibrated model sources or associate a calibration object with the models, using MmodControl() with M_ASSOCIATED_CALIBRATION. Note that defining models from calibrated model sources and adjusting individual model settings is done in pixels. Synthetic models can be used with a calibrated target if defined in real-world units and associated with the calibration object of the target.

The models in a Model Finder context can be associated with different calibration objects, however, you should ensure that all calibration objects use the same absolute coordinate system (otherwise, results will be skewed).

Targets must also be calibrated using the same absolute coordinate system as the models in the Model Finder context. Targets can be either physically corrected or not, without affecting the robustness of the search.

Targets can be grabbed from any number of cameras, each with its own calibration object, as long as the same absolute coordinate system has been used. For example, if a model has been extracted from a model source image taken using camera A, the model can be used to locate occurrences in a target image grabbed by camera B or camera C (whether or not the image has been physically transformed) since the same absolute coordinate system has been used to calibrate all images. MIL will transparently convert between calibration objects.

If the target is calibrated, results can be retrieved in real-world or pixel units. Note, however, in the presence of distortion some results are meaningless when converted from real-world to pixel units (for example, angle, scale, and transformation coefficient results). For example, if a model appears warped in the target, but the calibration object of the target compensates for this during the model search, the resulting angle is meaningful in the real-world coordinate system, and meaningless in the image coordinate system.

When using MmodDraw() to draw features from a calibrated model or results from a calibrated target, the destination drawing buffer must also be calibrated, using the same absolute coordinate system, otherwise annotations will be skewed. MIL transparently takes into account the calibration when drawing these annotations.

Models and their calibration object

Calibration objects are not saved with the Model Finder context. When restoring a Model Finder context from disk, you must re-associate your models with their calibration objects (which must also have been saved), using MmodControl() with M_ASSOCIATED_CALIBRATION.

If necessary, you can disassociate a calibration object from a model, using MmodControl() with M_ASSOCIATED_CALIBRATION set to M_NULL. This can be useful if you want to use a calibrated model with an uncalibrated target.

Setting the aspect ratio control

When dealing with aspect ratio distortion, you can use MmodControl() with M_ASPECT_RATIO to compensate for the distortion, if the target is not calibrated. This will allow you to avoid using a calibration object during the search. This will not compensate for aspect ratio distortion in the model, so this method is mostly useful when searching for synthetic models in a target that has aspect ratio distortion.

When performing the search, results will be returned for the corrected target coordinate system. The results will be converted to the original target coordinate system if you draw them using MmodDraw().

Using transformation coefficients with calibrated images

You can use forward and reverse transformation coefficients if you want to map points of interest other than the reference axis origin (see the Forward and reverse transformation coefficients subsection in the Customizing search settings section earlier in this chapter). When using a calibrated model and a calibrated target, forward and reverse transformation coefficients (see MmodGetResult()) are given for the real-world. This means that they can be used to convert any coordinates in the model world coordinate system to the corresponding coordinates in the target world coordinate system for an occurrence (or vice versa).

Forward and reverse transformation coefficients can be used to determine and draw equivalent positions in a calibrated model image and a calibrated target image, for an occurrence. Note, however, that the Mgra...() functions only take pixel coordinates. Therefore, to draw a pixel of interest from a calibrated model to a calibrated target, you must first convert the required pixel coordinates in the model image to real-world coordinates, using McalTransformCoordinate(). Following that, you have to get the forward transformation coefficients using MmodGetResult(). Apply the transformation coefficients to the coordinates; then convert the real-world coordinates back to pixel coordinates, using McalTransformCoordinate() again. These coordinates can then be used to draw in the target image. A reverse transformation is done similarly, except you start from the calibrated target.

See the image below for an example of a forward transformation.

ASSOCIATED CALIBRATION ASSOCIATED CALIBRATION ASSOCIATED CALIBRATION NULL ASPECT RATIO


An example

Availability | Previous
Not available in MIL-Lite

Available in MIL

The Model Finder example, MMod.c illustrates how the model can be used with the following source images:

Specifically, MMod.c shows how to define a Model Finder context with a single model, as well as with several models, and find these models in a target image.

MMod.c


Edge Finder

Chapter 9: Edge Finder
Availability
Not available in MIL-Lite

Available in MIL

This chapter explains how to use the MIL Edge Finder module to extract and analyze edges in your image.

  • MIL Edge Finder module
    • Edges and Edge Finder
  • Steps to performing an edge extraction and analysis
  • Basic concepts
  • Extracting the edges
    • The basics
    • Object contours versus line crests
      • Object contours
      • Line crests
    • How edges are calculated
      • Object contours
      • Line crests
  • Customizing the edge extraction settings
    • Filter type
      • Infinite Impulse Response (IIR) filters
      • Finite Impulse Response (FIR) filters
    • Filter mode
      • Automatically determined kernel size
      • Directly determined kernel size
    • Smoothing
    • Thresholding
    • Edgel accuracy
    • Magnitude type
    • Filling the edge gaps
      • Candidate constraint
      • Region constraint
      • Polarity constraint
      • Continuity constraint
  • Edge features
    • Dimension features
      • Feret
      • Circle fit and line fit
      • Ellipse fit
    • Location features
    • Advanced features
      • Convex perimeter
      • Tortuosity
      • Moment elongation
    • Grouped features
  • Calculating and retrieving results
    • Sorting keys
    • Retrieving the results
    • Selecting the results
      • Edge selection based on edge features
      • Edge selection based on the inter-relationship of edges
      • Edge selection based on the proximity of the edges to a point
    • Internal processing buffers
      • Derivatives
      • Angles
      • Magnitude
      • Source image and mask
    • Post-calculation
      • Restrictions
    • Annotating the results
  • Advanced edge extraction
    • Approximating the edges
    • Masking the edges
    • Cropping the edges
    • Advanced thresholding
    • Providing the image's derivatives
    • Putting data into an Edge Finder result buffer
    • Finding the closest edgels to a list of points
      • Location-based constraints
      • Edgel-based constraints
  • Interfacing with the Geometric Model Finder module
  • An example


MIL Edge Finder module

See Also | Availability | Next
User Guide:
Calibration - overview
The MIL Geometric Model Finder module
Not available in MIL-Lite

Available in MIL

The MIL Edge Finder module is a set of powerful functions that allow you to extract and analyze edges in a source image.

Many applications can be implemented using the Edge Finder module, such as complex measurement operations, defect detection, shape recognition, and shape analysis. You can also perform advanced edge manipulations, such as closing broken edges.

Edge Finder allows you to calculate a large number of edge features that can provide you with useful information, such as the edge's minimum and maximum Feret diameter. You can either calculate all required features for all edges, or perform a post-calculation on a refined subset of selected edges, which can speed up your processing time considerably.

The Edge Finder module can also be used in conjunction with the The MIL Geometric Model Finder module section in Chapter 8: Geometric Model Finder module; that is, you can define models from the result of an edge extraction, or you can find model occurrences in the result of an edge extraction.

If your source image is calibrated, results are calculated in the world coordinate system, otherwise they are calculated in the image coordinate system. Also, if your source image is calibrated, you can retrieve results in either real-world or pixel units. For more information about calibration, see Calibration.

Edges and Edge Finder

Edges are curves that delineate a boundary. These can be established from intensity transitions in an image. Well-defined edges come from sharp transitions in value, typically found in highly-contrasted images. Conversely, weak edges come from gradual transitions in value, typically found in smooth images.

Depending on your settings, Edge Finder will extract one of two edge types: object contours or line crests. An object contour is a type of edge that defines the outline of objects in an image. A line crest is a type of edge that defines the skeleton of objects in an image. Note that line crests should be used to extract edges from objects consisting of thin curvilinear shapes, such as fingerprints.

In either case, edges are extracted from the source image and used to form the image's edge map, which represents how the image is defined as a set of edges.

All Edge Finder operations, such as annotations and feature calculations are performed using the image's edge map.

You can only extract line crests from grayscale images. However, you can extract object contours from both grayscale and color images. When operating on color images, Edge Finder takes into consideration all three bands (RGB) to establish the presence of an edge.


Steps to performing an edge extraction and analysis

Reference:
MedgeAlloc
MedgeAllocResult
MedgeCalculate
MedgeControl
MedgeDraw
MedgeFree
MedgeGetNeighbors
MedgeGetResult
MedgePut
MedgeSave
MedgeSelect
Not available in MIL-Lite

Available in MIL

The following steps provide a basic methodology for using the MIL Edge Finder module:

  1. Allocate an Edge Finder context, using MedgeAlloc().
  2. If necessary, adjust general processing controls to fit your application, using successive calls to MedgeControl().
  3. If necessary, select the required features to calculate, using successive calls to MedgeControl().
  4. Allocate an Edge Finder result buffer to hold the results of the edge extraction, using MedgeAllocResult().
  5. Calculate edges and edge features, using MedgeCalculate().
  6. If necessary, exclude or delete edges that do not meet a required criteria, using MedgeSelect(). Excluded edges will be ignored in future calculations, while deleted edges will be removed from the Edge Finder result buffer.
  7. If necessary, select new features and perform a post-calculation, using MedgeCalculate().
  8. Get the number of edges currently included and retrieve the required results from the Edge Finder result buffer, using MedgeGetResult(). Results for the excluded or deleted edges will not be returned.
  9. If necessary, get the coordinates of edgels from an Edge Finder result buffer that correspond to the closest neighbors from a list of user-specified source point coordinates, using MedgeGetNeighbors().
  10. If necessary, copy data from user-supplied arrays to a specified Edge Finder result buffer, using MedgePut().
  11. If necessary, draw edges and edge features, using MedgeDraw().
  12. If necessary, save your Edge Finder context, using MedgeSave().
  13. Free all your allocated objects, using MedgeFree().

You can repeat steps 6, 7, and 8 until all the required results are obtained. Note that successively excluding or deleting unwanted edges and then calculating more features is the recommended procedure to achieve the required results, especially if you have a large number of unwanted edges.


Basic concepts

Reference:
MedgeCalculate
Not available in MIL-Lite

Available in MIL

The basic concepts and vocabulary conventions for the MIL Edge Finder module are:

  • Edge. A curve that delineates a boundary, which can be established from intensity transitions in an image. In Edge Finder, an edge is considered to be an edge chain and its features.
  • Edge angle. The angle between the horizontal axis and the edge's perpendicular direction at each edgel location.
  • Edge chain. The set of connected edgels that construct an edge.
  • Edge feature. Information describing an edge, such as its length.
  • Edge magnitude. The strength of the edge at each edgel location. For object contours, the magnitude is the norm of the gradient vector at the edgel position. For line crests, the magnitude is equal to the maximum eigenvalue of the Hessian matrix at the edgel position.
  • Edge map. The set of edges extracted from the source image.
  • Edgel. An elementary point (or edge element) within an edge.
  • Line crest edge. A type of edge that defines the skeleton of objects in an image. Typically, objects should be thin curvilinear shapes; otherwise, you might get unexpected results.
  • Object contour edge. A type of edge that defines the outline of objects in an image. Typically, objects should not be thin curvilinear shapes; otherwise, you might get unexpected results.
  • Post-calculation. Any calculation made to refine results after an initial call to MedgeCalculate().
  • Source image. The image from which the edges are extracted.

After edge chains have been extracted, the terms edge chain and edge are used interchangeably.


Extracting the edges

Reference:
MedgeAlloc
MedgeControl
Not available in MIL-Lite

Available in MIL

Before you write even the simplest application, it is important to have a basic understanding of how Edge Finder extracts edges, and what the differences are between the various types of edges (object contours and line crests). Note that if you are writing slightly more advanced applications, it is also worth having a more in depth understanding of how edges are calculated.

The basics

Edge Finder uses operations that are based on differential analysis, where edges (either contours or line crests) are extracted by analyzing intensity transitions in images. Transitions in intensity can come from many physical phenomenon, such as shadows and illumination variations. However, strong transitions are typically caused by the presence of an object contour or a line crest in the image, as illustrated below.

Note that the main intensity transition in the image is observed in the perpendicular direction relative to the object border (for an object contour) or line shape (for a line crest).

Edges are extracted in three general steps. First, a filtering process provides an enhanced image of the edges based on the computation of the image's derivatives. Second, detection and thresholding operations determine, from the image enhancement, all pertinent edge elements, or edgels, and accurately calculates their positions. Third, neighboring edgels are connected to build the edge chains (which results in the image's edge map), and features are calculated for each edge.

To fit the requirements of your application, Edge Finder allows you to customize each step's corresponding processing controls.

Object contours versus line crests

To properly analyze the structures present in an image, it is often useful to consider the image as a topographic surface, where the intensity of each pixel is seen as the elevation value of the surface. By doing so, contours and crests can be more easily understood.

Object contours

An object contour is the inter-pixel boundary that splits into thin layers (delaminates) the separation between the object and the image background, as illustrated below. The edge of the contour is located at the sharpest point within the intensity transition. Moreover, the intensity transition is strongest in the perpendicular direction to the object contour.

Well-defined contours have sharp transitions in value. The smoother the image, the more gradual the change, and the weaker the contour.

To allocate an Edge Finder context for object contours, use MedgeAlloc() with M_CONTOUR.

Line crests

A line crest is the skeleton of a thin curvilinear shape in the image, as illustrated below. The crest is located at the sharpest point of the Gaussian-like profile of the line shape. Moreover, the intensity transition is strongest in the perpendicular direction to the line crest.

Well-defined line crests are thin and have sharp transitions in value. The smoother the image or the larger the lines, the more gradual the change, and the weaker the line crests.

A line crest can either be darker (a valley-like Gaussian profile), lighter (a ridge-like Gaussian profile), or both darker and lighter than the image's background. To specify the color of the line crest to extract, set MedgeControl() with M_FOREGROUND_VALUE to M_FOREGROUND_BLACK, M_FOREGROUND_WHITE, or M_ANY. The default setting is M_FOREGROUND_BLACK.

To allocate an Edge Finder context for line crests, use MedgeAlloc() with M_CREST.

Typically, you should use M_CREST Edge Finder contexts for edges that are established from thin curvilinear shapes; otherwise, you might get unexpected results.

How edges are calculated

For basic applications, you need not know how Edge Finder calculates edges. However, an understanding of this process can help you solve more advanced problems, and adjust some advanced settings.

Object contours

The enhanced image of the object contours is achieved by calculating the gradient magnitude of each pixel in the image. The stronger the intensity transition, the greater the magnitude will be.

The gradient magnitude is calculated at each pixel position from the image's first derivatives. It is defined as:

where Ix and Iy are, respectively, the X and Y derivative values. They define the components of the gradient vector as:

An edgel is located at the maximum value of the gradient magnitude over adjacent pixels, in the direction defined by the gradient vector. The gradient direction is the direction of the steepest ascent at an edgel in the image, while the gradient magnitude is the steepness of that ascent. Note that the gradient direction is also the perpendicular direction to the object contour. Well-defined contours are extracted from strong and sharp intensity transitions. Note that a strong contrast between the objects and their background improves the edge detector's robustness and location accuracy.

Note that Edge Finder also supports 3-band images, when extracting object contours. The extraction process is similar to single band images, except the enhanced image of the object contours is achieved by calculating a generalized gradient magnitude of each pixel in the image, for each color band (RGB).

Line crests

The enhanced image of the line crests is achieved by calculating the Hessian magnitude of each pixel in the image. The stronger the crest, the greater the magnitude will be.

The magnitude is defined as the maximum eigenvalue of the Hessian matrix, calculated at each pixel position from the image's second derivatives. The Hessian matrix is defined as:

where Ixx, Iyy and Ixy are, respectively, the X, Y and cross second derivative values of the image.

An edgel is located at the maximum value of the image magnitude over adjacent pixels, in the direction defined by the associated Eigen vector. Well-defined line crests are extracted from strong and sharp intensity transitions. Note that a strong contrast between the lines and the image's background improves the edge detector's robustness and the location accuracy.

Note that the sign of the magnitude at the crest's edgel locations changes with the color of the line crest. That is, a dark crest has a strong positive magnitude, while a light crest has a strong negative magnitude. The geometric interpretation of the magnitude and its associated orientation is the direction where the crest is the sharpest. Similar to object contours, this direction is also the direction perpendicular to the line crest.

CONTOUR FOREGROUND VALUE FOREGROUND BLACK FOREGROUND WHITE ANY FOREGROUND BLACK CREST CREST


Customizing the edge extraction settings

User Guide:
Advanced edge extraction
Buffer overscan region
Object contours versus line crests
Post-calculation
Smoothing

Reference:
MedgeAlloc
MedgeControl
Not available in MIL-Lite

Available in MIL

Once you have allocated an Edge Finder context with the right context type (using MedgeAlloc()), you can customize the image processing algorithm using MedgeControl(). For example, to fit the requirements of your application, you can select the appropriate:

  • Edge extraction filter (M_FILTER_TYPE).
  • Filter mode (M_FILTER_MODE).
  • Noise reduction factor (M_FILTER_SMOOTHNESS).
  • Sensitivity of the edgel detection (M_THRESHOLD_MODE).
  • Accuracy of the edgel positions (M_ACCURACY).

You can also choose to fill in the edge gaps, which is the process of linking broken edges together based on a set of criteria (M_FILL_GAP_DISTANCE and M_FILL_GAP_ANGLE). In addition, Edge Finder allows you to set an overscan, which determines how to handle an image's bordering pixels. For more information, see the Buffer overscan region section in Chapter 14: Specifying and managing your data buffers.

Generally, the default control settings described in this section are sufficient for the majority of images. You should therefore adjust these settings only when dealing with very noisy images, extremely low or multi-contrast images, or images with very thin, refined features. In all cases, it is recommended that you try the default settings first and, if necessary, experiment with different settings to achieve the required edge map and accuracy.

For more advanced information on extracting edges, see the Advanced edge extraction section later in this chapter.

Note that the edge extraction process involves a denoising operation to even out rough edges and remove noise. The filter type, filter mode, kernel depth, smoothness factor, and threshold mode all work in concert to control this operation, and ultimately determine which edges to extract. Changing any one of these factors will, to varying degrees, alter the edges that are extracted.

Filter type

The edge extraction filter is used to compute the source image's derivatives, which in turn are used to calculate the edge's magnitude (Gradient or Hessian) and angle. You can set the type of filter used when performing the edge extraction using MedgeControl() with M_FILTER_TYPE. You can choose either an Infinite Impulse Response (IIR) edge extraction filter, or a Finite Impulse Response (FIR) edge extraction filter. The default setting is the IIR filter, M_SHEN.

IIR filters allow you to extract both crests and contours, while FIR filters only allow you to extract contours. IIR filters also allow you to take advantage of Edge Finder's smoothing capabilities, while FIR filters ignore your smoothness setting. For more information on smoothing, see the Smoothing subsection in this section.

Infinite Impulse Response (IIR) filters

The M_FILTER_TYPE setting in MedgeControl() establishes the contribution of each neighboring pixel. It provides you with the following IIR filters to extract edges:

  • Shen-Castan (M_SHEN), which is an exponential weighting function, of the general form
  • Canny-Deriche (M_DERICHE), which is an exponential weighting function, of the general form

Note that to perform the edge extraction, IIR filters can either be implemented in recursive or kernel mode. For more information on filter modes, see the Filter mode subsection in this section.

The default filter, Shen-Castan, typically performs an effective edge extraction; it achieves a very good localization of edges, which makes it ideal for the majority of images.

However, Shen-Castan can sometimes be unexpectedly sensitive to noise, or can yield inappropriate results if you are extracting unusually thick crests. If this is the case, you should try Canny-Deriche, which is better suited to handle these situations, since it places more emphasis on an edge's neighbors than Shen-Castan. The following image illustrates how the first derivative distribution of values can affect results for both Shen-Castan and Canny-Deriche:

Note that, as mentioned, IIR filters allow you to extract both crests and contours.

Finite Impulse Response (FIR) filters

The M_FILTER_TYPE setting in MedgeControl() provides you with the following FIR filters to extract edges:

  • Frei Chen (M_FREI_CHEN), which can be represented with the following convolution kernels:
  • Prewitt (M_PREWITT), which can be represented with the following convolution kernels:
  • Sobel (M_SOBEL), which can be represented with the following convolution kernels:

Note that, as mentioned, FIR filters only allow you to extract contours.

Filter mode

Filtering is a neighborhood calculation that is implemented during the edge extraction process. Depending on the filter, this calculation can either be implemented recursively, or using a convolution kernel. Since FIR filters use a specific 3x3 kernel defined by Edge Finder, you cannot calculate them recursively, nor can you alter their kernel size.

On the other hand, IIR filters use an exponential weighting function to perform the filtering. This function can either be implemented recursively, or with a kernel of different sizes. To specify the mode in which to perform the filtering, use MedgeControl() with M_FILTER_MODE set to either M_RECURSIVE or M_KERNEL.

Typically, the default filter mode is M_RECURSIVE. However, if you have dedicated hardware performing the operation, the default is M_KERNEL. Unless you have dedicated hardware, an M_KERNEL convolution should not be set since it can take a long time to process.

If you do not have dedicated hardware, M_RECURSIVE is faster than M_KERNEL, unless you are using very small kernels (for example, 3x3). If you have dedicated hardware, M_KERNEL is typically faster.

If you specify M_RECURSIVE, the filtering is done using a recursive implementation of the IIR filter. If this mode actually used a kernel, the kernel size would be theoretically infinite.

If you specify M_KERNEL, the filtering is done using a kernel approximation of the IIR filter. In this case, you must set the size of the convolution kernel using MedgeControl() with M_KERNEL_WIDTH. The size can either be determined automatically or directly. Note that the size of the kernel can be constrained by the available hardware resources.

When using M_KERNEL, you must also specify the bit depth with which kernel values are calculated. To do so, set M_KERNEL_DEPTH to either 8 or 16. The default is 8-bit, which is faster than 16-bit, although it is less accurate.

Automatically determined kernel size

To specify the size of the kernel automatically, set MedgeControl() with M_KERNEL_WIDTH to M_AUTO. This is the default setting. In this case, Edge Finder establishes the maximum possible kernel size, for a given smoothness factor. You can set the smoothness factor using MedgeControl() with M_FILTER_SMOOTHNESS.

The size of the kernel, for a given smoothness factor, is calculated as the size at which the quantified filter values are zero. If the hardware resources are available, the smoothness factor will always be adhered to. However, if the smoothness factor calls for a kernel size that is beyond the limits of your hardware, the smoothness factor will be sacrificed to achieve the largest possible kernel size.

Note that changing the smoothness factor can also change the convolution kernel size, which can affect processing time. That is, the larger the kernel, the longer the processing will take.

Directly determined kernel size

To specify the size of the kernel directly, set MedgeControl() with M_KERNEL_WIDTH to an odd integer that is greater than or equal to 3. Note that larger kernels result in longer processing times.

If your hardware cannot handle the kernel size, the effective smoothness factor will be saturated to the maximum possible value for the specified kernel size. That is, a filter is always made to fit in the kernel without a pre-imposed truncation; therefore, the best possible approximation is calculated, given the limited kernel size.

Smoothing

The M_FILTER_SMOOTHNESS setting in MedgeControl() allows you to control the degree of smoothness applied in the edge extraction. The smoothing operation evens out rough edges and removes noise; in effect, the smoothness factor affects which edges are extracted.

The range of this control varies from 0 (no smooth) to 100 (a very strong smooth). The default setting is 50.0. Note that if you are using an FIR filter, the smoothness value is ignored.

Increasing the smoothness control value does not affect the processing time. However, a very high smoothing level can result in a loss of important detail and a decrease in precision. Note that, the range of smoothing is not linear; that is, the higher the smoothing factor, the greater the difference between settings. For example, changing the smoothing from 30.0 to 50.0 is less significant than changing the smoothing from 50.0 to 70.0.

Thresholding

Edge Finder uses an hysteresis thresholding to perform the edge extraction. That is, the extracted edge chains are built such that the magnitude values of all connected edgels are stronger than the lower bound threshold value (M_THRESHOLD_LOW) and at least one edgel in each edge chain has a magnitude that is stronger than the upper bound threshold value (M_THRESHOLD_HIGH). In essence, thresholding determines which edges are extracted from the source image based on the magnitude of each edgel. For more information on edgel magnitude, see the Magnitude type subsection in this section.

Thresholding can also be applied during post-calculation. In this case, threshold settings are applied to the Edge Finder result buffer, rather than the source image. Except for filtering operations, the edge extraction process is completely recalculated. As a result, all internal processing buffers (M_SAVE_...) must have been initially saved, using MedgeControl(). For more information on post-calculation, see the Post-calculation subsection in the Calculating and retrieving results section later in this chapter.

Edge Finder provides several predefined threshold modes that automatically select appropriate values for the lower and the upper bound thresholds, based on an internal image evaluation and noise estimation. To automatically determine the threshold settings with which to extract edges, set MedgeControl() with M_THRESHOLD_MODE to one of the following: M_VERY_HIGH, M_HIGH, M_MEDIUM, M_LOW, or M_DISABLE. The default setting is M_HIGH. Note that lower threshold modes result in a more sensitive edgel detection; that is, more edgels are detected. For example, M_VERY_HIGH always results in a lower (or equal) number of edgels than M_HIGH, M_HIGH always results in a lower (or equal) number of edgels than M_MEDIUM, and so on.

The default setting (M_HIGH) is typically sufficient since it offers a robust detection of pertinent edgels, even from images presenting some contrast variations, noise, and non-uniform illumination. However, for multi-contrast images, or for images with a lot of noise or non-uniform illumination, some edges can be missed. In these cases, the setting M_MEDIUM should be used. If all relevant edges are still not being extracted, then use M_LOW, which will get all edges over a minimum noise-based estimated threshold. Finally, to extract all edgels in the image, use M_DISABLE, which will set the lower and upper bound threshold values to 0. Note that M_LOW and M_DISABLE should be used carefully since a large number of unnecessary edgels might be extracted. For images that have strong contrast, little noise, and consistent illumination, the M_VERY_HIGH setting can be used, since it will extract only the strongest edges. Note that M_VERY_HIGH should also be used carefully since pertinent edgels might not be extracted.

In the majority of cases, Edge Finder's predefined threshold modes should provide you with sufficient thresholding control. However, for advanced applications, you might want to explicitly set the threshold bounds. To do so, set M_THRESHOLD_MODE to M_USER_DEFINED, and use the M_THRESHOLD_LOW and M_THRESHOLD_HIGH control types to specify the lower and upper threshold bounds, respectively. The default settings are 0.

If M_THRESHOLD_MODE is not set to M_USER_DEFINED, your M_THRESHOLD_LOW and M_THRESHOLD_HIGH values have no effect.

Note that the threshold bounds must be provided as positive values. However, for line crests, Edge Finder will consider these values to be negative and/or positive, depending on whether the edges were extracted as darker (negative edgel values), lighter (positive edgel values), or both darker and lighter (negative and positive edgel values) than the image's background. For more information on extracting line crests, see the Object contours versus line crests subsection in the Extracting the edges section earlier in this chapter.

Advanced users might want to customize the behavior of the hysteresis thresholding; to do so, see the Advanced thresholding subsection in the Advanced edge extraction section later in this chapter.

Edgel accuracy

The M_ACCURACY setting in MedgeControl() determines the accuracy used to estimate edgel positions. Edgel accuracy can be set to one of the following: M_HIGH, M_VERY_HIGH, or M_DISABLE. The default setting is M_HIGH. When you set the accuracy to high, edgels are calculated with subpixel accuracy, which is typically sufficient. When you set the accuracy to very high, a classical image formation model is used to compensate for any pixel-distortion aberration, resulting in very precise subpixel edge accuracy. If you want to disable subpixel accuracy and calculate edgels with pixel precision, set M_ACCURACY to M_DISABLE.

For the most part, edgel accuracy varies with the image's dynamic range, sharpness, and noise. The best accuracy is achieved in well-contrasted, noise-free images. In perfect situations, Edge Finder can provide a subpixel edgel location accuracy of up to 1/128th of a pixel. However, in real situations, it is reasonable to expect an accuracy from 1/10th of a pixel to 1/40th of a pixel, depending on physical limitations due to the transition's dynamic range and the image's noise.

You can notice the accuracy of the edgels if you zoom a region of the source image that was used to calculate results by specifying the appropriate M_DRAW_RELATIVE_ORIGIN_... and M_DRAW_SCALE_... values with MedgeControl().

Magnitude type

You can specify the type of magnitude used to determine edgel positions using MedgeControl() with M_MAGNITUDE_TYPE. The magnitude type can be set to either M_NORM, where the gradient magnitude is used, or M_SQR_NORM, where the square of the gradient magnitude is used. The default setting is M_SQR_NORM.

Since M_SQR_NORM uses the square of the gradient magnitude, it is faster, though less precise than M_NORM. Typically, M_SQR_NORM is used since it preserves a very good edgel location accuracy. Note that changing M_MAGNITUDE_TYPE can slightly modify the resulting edge map.

Filling the edge gaps

Edge Finder allows you to fill gaps between edges; that is, to automatically link the extremities of non-closed edges together, depending on their relative positions and orientations. Unexpected broken edges can occur when dealing with noisy images, and connecting them can significantly improve the quality of your results. To help you choose which edges can be linked, Edge Finder provides a series of constraints, described below. Note that edges will only be linked if they adhere to each constraint.

Filling edge gaps can also be done in post-calculation; however, there are some restrictions that must be taken into account. For more information, see the Post-calculation subsection in the Calculating and retrieving results section later in this chapter.

Candidate constraint

You can choose whether to join an edge extremity with the other extremity of the same edge, or with an extremity of any edge. To connect an edge's extremity with the extremity of any edge, use MedgeControl() with M_FILL_GAP_CANDIDATE set to M_ANY. This is the default value.

To connect the edge extremities of the same edge, use MedgeControl() with M_FILL_GAP_CANDIDATE set to M_SAME. This is the same as closing an open edge.

Region constraint

The M_FILL_GAP_DISTANCE and M_FILL_GAP_ANGLE settings in MedgeControl() define the region where two edge extremities can be linked. That is, when searching for edge extremity candidates to fill edge gaps, M_FILL_GAP_DISTANCE sets the maximum distance radius, while M_FILL_GAP_ANGLE sets the aperture angle. Note that two edge extremities can only be linked if both meet the M_FILL_GAP_DISTANCE and M_FILL_GAP_ANGLE criteria, and if both are included in the search region of the other.

The gap distance (M_FILL_GAP_DISTANCE) must be specified in pixel units. The gap distance should be set carefully and large values should generally be avoided, otherwise the wrong edges can be connected. Typically, distance values should not exceed 5 pixels; however, if necessary, you can also specify no distance restraint (M_INFINITE). The default M_FILL_GAP_DISTANCE value is 0 (no edge linking) and the default M_FILL_GAP_ANGLE value is 360 degrees.

Polarity constraint

When searching for edge extremity candidates, you can also use the M_FILL_GAP_POLARITY control type to specify that only edges with a certain polarity can be linked. The edge's polarity indicates whether edges were established as a transition from light to dark, or vice versa. By default (M_ANY), all edges that meet the distance and angle restraints are considered candidates, regardless of the edge's polarity. If M_FILL_GAP_POLARITY is set to M_SAME, only edges with the same polarity are considered potential candidates. If M_FILL_GAP_POLARITY is set to M_REVERSE, only edges with reverse polarity are considered potential candidates.

Note that the M_SAME and M_REVERSE polarity settings are not available for M_CREST Edge Finder contexts.

Continuity constraint

It is possible that multiple edges meet the distance (M_FILL_GAP_DISTANCE), angle (M_FILL_GAP_ANGLE), and polarity (M_FILL_GAP_POLARITY) settings. When this occurs, you must decide which edges should be linked. To help choose, Edge Finder provides a continuity constraint.

The continuity constraint allows you to decide whether the candidate chosen to fill the edge gap is selected according to its proximity or its maximum continuity. To set the continuity constraint, use MedgeControl() with M_FILL_GAP_CONTINUITY. The range of this control varies from 0 to 100. When set to 0, the closest edge extremity is chosen to link edges together. When set to 100, the edge that gives the most continuous result (minimum curvature) is chosen. The default value is 50.

FILTER TYPE FILTER MODE FILTER SMOOTHNESS THRESHOLD MODE ACCURACY FILL GAP DISTANCE FILL GAP ANGLE FILTER TYPE SHEN FILTER TYPE SHEN DERICHE FILTER TYPE FREI CHEN PREWITT SOBEL FILTER MODE RECURSIVE KERNEL RECURSIVE KERNEL KERNEL RECURSIVE KERNEL KERNEL RECURSIVE KERNEL KERNEL WIDTH KERNEL KERNEL DEPTH KERNEL WIDTH AUTO FILTER SMOOTHNESS KERNEL WIDTH FILTER SMOOTHNESS THRESHOLD LOW THRESHOLD HIGH THRESHOLD MODE VERY HIGH HIGH MEDIUM LOW DISABLE HIGH VERY HIGH HIGH HIGH MEDIUM HIGH MEDIUM LOW DISABLE LOW DISABLE VERY HIGH VERY HIGH THRESHOLD MODE USER DEFINED THRESHOLD LOW THRESHOLD HIGH THRESHOLD MODE USER DEFINED THRESHOLD LOW THRESHOLD HIGH ACCURACY HIGH VERY HIGH DISABLE HIGH ACCURACY DISABLE MAGNITUDE TYPE NORM SQR NORM SQR NORM SQR NORM NORM SQR NORM MAGNITUDE TYPE FILL GAP CANDIDATE ANY FILL GAP CANDIDATE SAME FILL GAP DISTANCE FILL GAP ANGLE FILL GAP DISTANCE FILL GAP ANGLE FILL GAP DISTANCE FILL GAP ANGLE FILL GAP DISTANCE INFINITE FILL GAP DISTANCE FILL GAP ANGLE FILL GAP POLARITY ANY FILL GAP POLARITY SAME FILL GAP POLARITY REVERSE SAME REVERSE CREST FILL GAP DISTANCE FILL GAP ANGLE FILL GAP POLARITY FILL GAP CONTINUITY


Edge features

User Guide:
Retrieving the results

Reference:
MedgeControl
MedgeGetResult
MedgeInquire
Not available in MIL-Lite

Available in MIL

Edge Finder allows you to calculate many edge features that can provide useful information, such as the edge's length and center of gravity. To enable (M_ENABLE) or disable (M_DISABLE) the calculation of a specific edge feature, use MedgeControl(); you can also select a group of features for calculation in a single call. To know the calculation state of a feature, use MedgeInquire(). To retrieve the results of the feature calculation, use MedgeGetResult(). For more information on retrieving results, see the Retrieving the results subsection in the Calculating and retrieving results section later in this chapter.

Note that the label feature (M_LABEL_VALUE) is the only edge feature that is enabled by default. The label value is a positive integer greater or equal to one; it is assigned to each edge in an image for identification. Each edge in an image has a unique label value. The label allows you to track or choose particular edges for drawing, edge manipulation, or result retrieval purposes. Typically, you should not disable the edge's label value.

Edge features generally fall into one of the following groups:

  • Dimension features (typically very fast to compute).
  • Location features (typically fast to compute).
  • Advanced features (can take some time to compute, due to their complexity).

To compute features as efficiently as possible, you should calculate a few features first (preferably, the fastest), and eliminate as many unnecessary edges as possible. Then, post-calculate expensive features on the remaining edges. For more information on how to select and post-calculate edges, see the Calculating and retrieving results section later in this chapter.

Note that occasionally, enabling a feature for calculation will result in another feature being calculated and available for result retrieval. For example, enabling M_MOMENT_ELONGATION for calculation in MedgeControl() will allow you to retrieve the result of M_CENTER_OF_GRAVITY in MedgeGetResult(), even if you have disabled the calculation of the center of gravity.

The following subsections list all the Edge Finder features that you can enable or disable for calculation using MedgeControl(), according to dimension, location, and advanced application. There is also a subsection on selecting a group of features for calculation in a single call. Note that only the more complicated features are explained in some detail. For a description of each feature, see MedgeControl() in the MIL Reference.

Dimension features

The Edge Finder dimension features are:

  • M_CLOSURE
  • M_CIRCLE_FIT_CENTER_X
  • M_CIRCLE_FIT_CENTER_Y
  • M_CIRCLE_FIT_COVERAGE
  • M_CIRCLE_FIT_ERROR
  • M_CIRCLE_FIT_RADIUS
  • M_ELLIPSE_FIT_ANGLE
  • M_ELLIPSE_FIT_CENTER_X
  • M_ELLIPSE_FIT_CENTER_Y
  • M_ELLIPSE_FIT_ERROR
  • M_ELLIPSE_FIT_MAJOR_AXIS
  • M_ELLIPSE_FIT_MINOR_AXIS
  • M_FAST_LENGTH
  • M_FERET_BOX
  • M_FERET_ELONGATION
  • M_FERET_MAX_ANGLE
  • M_FERET_MAX_DIAMETER
  • M_FERET_MEAN_DIAMETER
  • M_FERET_MIN_ANGLE
  • M_FERET_MIN_DIAMETER
  • M_FERET_X
  • M_FERET_Y
  • M_GENERAL_FERET
  • M_GENERAL_FERET_ANGLE
  • M_LENGTH
  • M_LINE_FIT_A
  • M_LINE_FIT_B
  • M_LINE_FIT_C
  • M_LINE_FIT_ERROR
  • M_SIZE

Feret

The Feret is the diameter of the edge at various angles. Several Feret diameters are illustrated in the diagram below. Note that the angle at which the Feret diameter is taken (relative to the horizontal axis) is specified as a subscript to F.

Some features, such as the edge's maximum Feret diameter, are determined by testing the Feret diameter of the edge at several angles. To set the number of angles, use MedgeControl() with M_NUMBER_OF_FERETS. The default number of angles tested is 8, which is typically sufficient. Note that increasing the number of Ferets increases the accuracy of the results; however, it also increases the processing time.

Circle fit and line fit

The circle fit is calculated as the circle that best fits the edge, while the line fit is calculated as the line that best fits the edge.

To calculate the coordinates of the center of the circle that is the best fit for each edge, use M_CIRCLE_FIT_CENTER_X and M_CIRCLE_FIT_CENTER_Y. To calculate the radius of the circle that is the best fit for each edge, use M_CIRCLE_FIT_RADIUS.

To calculate the A, B, and C variables that define the line that is the best fit for each edge, use M_LINE_FIT_A, M_LINE_FIT_B, and M_LINE_FIT_C. Note that these variables are based on the linear equation, Ax + By + C = 0.

To calculate the coverage of the circle fit, use M_CIRCLE_FIT_COVERAGE. The coverage describes the portion of the circle covered by the edge. The value returned is between 0.0 and 1.0, inclusive, where 0 equals no coverage, 0.5 equals 50 percent coverage, and 1.0 equals 100 percent coverage.

Note that, since a line is theoretically infinite, the coverage for a line fit would always be 100 percent.

You can also calculate the error of either the circle fit or line fit using M_CIRCLE_FIT_ERROR or M_LINE_FIT_ERROR, respectively. These values are calculated as the average quadratic error of the fit.

Ellipse fit

The ellipse fit is calculated as the ellipse that best fits the edge.

To calculate the coordinates of the center of the ellipse that is the best fit for each edge, use M_ELLIPSE_FIT_CENTER_X and M_ELLIPSE_FIT_CENTER_Y. To calculate the major and minor axes of the ellipse that is the best fit for each edge, use M_ELLIPSE_FIT_MAJOR_AXIS and M_ELLIPSE_FIT_MINOR_AXIS, respectively.

To calculate the angle of the ellipse fit, use M_ELLIPSE_FIT_ANGLE. To calculate the error of the ellipse fit, use M_ELLIPSE_FIT_ERROR. This value is calculated as the average quadratic error of the fit.

Location features

The Edge Finder location features are:

  • M_BOX
  • M_BOX_X_MAX
  • M_BOX_X_MIN
  • M_BOX_Y_MAX
  • M_BOX_Y_MIN
  • M_CENTER_OF_GRAVITY
  • M_CENTER_OF_GRAVITY_X
  • M_CENTER_OF_GRAVITY_Y
  • M_CONTACT_POINTS
  • M_FIRST_POINT
  • M_FIRST_POINT_X
  • M_FIRST_POINT_Y
  • M_POSITION_X
  • M_POSITION_Y
  • M_X_MAX_AT_Y_MAX
  • M_Y_MAX_AT_X_MIN
  • M_X_MIN_AT_Y_MIN
  • M_Y_MIN_AT_X_MAX

The following edge map illustrates where each edge location feature is located:

Advanced features

The Edge Finder advanced features are:

  • M_AVERAGE_STRENGTH
  • M_CONVEX_PERIMETER
  • M_MOMENT_ELONGATION
  • M_MOMENT_ELONGATION_ANGLE
  • M_STRENGTH
  • M_TORTUOSITY

Convex perimeter

If you enable the convex perimeter feature (M_CONVEX_PERIMETER), an approximation of the perimeter of each edge's convex hull will be calculated. Abstractly, an edge's convex perimeter is very much like taking a rubber band, and placing it tautly around the edge.

The convex perimeter is derived by taking the diameter of the edge at different angles. The greater the number of Ferets used to calculate the diameter, the more accurate the approximation. Use MedgeControl() with M_NUMBER_OF_FERETS to adjust the number of Ferets. Note that increasing the number of Ferets increases the accuracy of the results; however, it also increases the processing time.

Tortuosity

An edge's tortuosity (M_TORTUOSITY) is the diagonal length of the edge's bounding box (M_BOX), divided by the length of the edge (M_LENGTH). Therefore, a non-tortuous edge (a straight line) will have a tortuosity of 1.0, while a tortuous edge will have its tortuosity decreasing towards zero.

Moment elongation

The moment elongation (M_MOMENT_ELONGATION) is a geometric elongation measure of the edge. It is defined as the ratio of its central moments, which corresponds to the principal directions of the edge shape. The elongation value varies from 0.0 to 1.0. Values closer to 0.0 are considered to have a low elongation, while values closer to 1.0 are considered to have a high elongation.

Note that you can also calculate the angle of the principle axis along each edge's moment elongation, using M_MOMENT_ELONGATION_ANGLE.

Grouped features

The Edge Finder features that allow you to select a group of features for calculation in a single call are:

  • M_ALL_FEATURES
  • M_BOX
  • M_CENTER_OF_GRAVITY
  • M_CIRCLE_FIT
  • M_CONTACT_POINTS
  • M_ELLIPSE_FIT
  • M_FERET_BOX
  • M_FIRST_POINT
  • M_LINE_FIT
  • M_POSITION

Each of these features has multiple features associated with it. Therefore, if you enable one of them for calculation, then all the features associated with it will also be enabled. For example, enabling M_CIRCLE_FIT will result in all the circle fit values of each edge to be calculated. This is equivalent to individually enabling each of the following: M_CIRCLE_FIT_CENTER_X, M_CIRCLE_FIT_CENTER_Y, M_CIRCLE_FIT_RADIUS, M_CIRCLE_FIT_ERROR, and M_CIRCLE_FIT_COVERAGE.

The features that belong to each group is typically self evident. For a description of each, see MedgeControl().

LABEL VALUE MOMENT ELONGATION CENTER OF GRAVITY CLOSURE CIRCLE FIT CENTER X CIRCLE FIT CENTER Y CIRCLE FIT COVERAGE CIRCLE FIT ERROR CIRCLE FIT RADIUS ELLIPSE FIT ANGLE ELLIPSE FIT CENTER X ELLIPSE FIT CENTER Y ELLIPSE FIT ERROR ELLIPSE FIT MAJOR AXIS ELLIPSE FIT MINOR AXIS FAST LENGTH FERET BOX FERET ELONGATION FERET MAX ANGLE FERET MAX DIAMETER FERET MEAN DIAMETER FERET MIN ANGLE FERET MIN DIAMETER FERET X FERET Y GENERAL FERET GENERAL FERET ANGLE LENGTH LINE FIT A LINE FIT B LINE FIT C LINE FIT ERROR SIZE NUMBER OF FERETS CIRCLE FIT CENTER X CIRCLE FIT CENTER Y CIRCLE FIT RADIUS LINE FIT A LINE FIT B LINE FIT C CIRCLE FIT COVERAGE CIRCLE FIT ERROR LINE FIT ERROR ELLIPSE FIT CENTER X ELLIPSE FIT CENTER Y ELLIPSE FIT MAJOR AXIS ELLIPSE FIT MINOR AXIS ELLIPSE FIT ANGLE ELLIPSE FIT ERROR BOX BOX X MAX BOX X MIN BOX Y MAX BOX Y MIN CENTER OF GRAVITY CENTER OF GRAVITY X CENTER OF GRAVITY Y CONTACT POINTS FIRST POINT FIRST POINT X FIRST POINT Y POSITION X POSITION Y X MAX AT Y MAX Y MAX AT X MIN X MIN AT Y MIN Y MIN AT X MAX AVERAGE STRENGTH CONVEX PERIMETER MOMENT ELONGATION MOMENT ELONGATION ANGLE STRENGTH TORTUOSITY CONVEX PERIMETER NUMBER OF FERETS TORTUOSITY BOX LENGTH MOMENT ELONGATION MOMENT ELONGATION ANGLE ALL FEATURES BOX CENTER OF GRAVITY CIRCLE FIT CONTACT POINTS ELLIPSE FIT FERET BOX FIRST POINT LINE FIT POSITION CIRCLE FIT CIRCLE FIT CENTER X CIRCLE FIT CENTER Y CIRCLE FIT RADIUS CIRCLE FIT ERROR CIRCLE FIT COVERAGE


Calculating and retrieving results

User Guide:
Advanced edge extraction
Calibration - overview
Customizing the edge extraction settings
Overview
Edge features
Post-calculation

Reference:
McalControl
MedgeAlloc
MedgeAllocResult
MedgeCalculate
MedgeControl
MedgeDraw
MedgeGetResult
MedgeSelect
Not available in MIL-Lite

Available in MIL

Once you have allocated an Edge Finder context and result buffer (MedgeAlloc() and MedgeAllocResult()), and have calculated the edge features (MedgeCalculate()), you can retrieve the results from your Edge Finder result buffer, using MedgeGetResult().

After the initial calculation, you can select edges that meet a specified criterion with MedgeSelect(), and post-calculate new features for them. By doing so, you can avoid unnecessarily calculating time-consuming edge features for all edges in a source image.

For optimum performance, the source image buffer on which you make calculations should be a 1-band 8-bit unsigned buffer. Other buffer depths and types are generally accepted, but can slightly decrease performance. 3-band source image buffers are also supported, but only when extracting edge contours. When the source image buffer is a 32-bit floating point buffer, Edge Finder uses floating-point precision calculations. In all cases, you can force the edge processing operations to use floating-point precision by enabling M_FLOAT_MODE in MedgeControl(). Note that changing this processing mode can slightly affect the resulting edge map; also, floating-point precision calculations will typically take more time.

Occasionally, an operation can take an unexpectedly long time to calculate. To prevent this, you can use MedgeControl() with M_TIMEOUT to set a maximum edge extraction and calculation time. By default, there is no time limit.

Sorting keys

The results obtained from MedgeGetResult() can be sorted in ascending or descending order, by a maximum of three features assigned as sorting keys. To specify a feature as a sorting key, add M_SORTn_UP or M_SORTn_DOWN to the feature when selecting it for calculation using MedgeControl(). Assign the numbers 1, 2, or 3 to indicate the sorting precedence of the feature(s). Note that features will only be sorted after an MedgeCalculate().

You can also post-sort edges by assigning a sorting key to a previously calculated edge feature. Note that, although the post-calculation changes the order in which the edges are sorted, it does not change their label values, which were assigned at the edges's initial calculation. For more information on post-calculation, see the Post-calculation subsection in this section.

Retrieving the results

Edge Finder offers several types of results that provide considerable information on the nature of the extracted edges, such as magnitude and Feret values. This information can be retrieved for all edges or for a specified edge, from your result buffer.

Typically, before retrieving any edge feature result, you should first retrieve the number of edges found in the image to determine the size of the result array needed to hold the results. To do so, use MedgeGetResult() with M_NUMBER_OF_CHAINS. Similarly, if you need to retrieve edgel results, such as the edgels' coordinates, magnitude, or angle values, you should also determine the size of the result array needed to hold the results. To do so, you must retrieve the number of extracted edgels for each edge using MedgeGetResult() with M_NUMBER_OF_CHAINED_EDGELS.

Edge results are indexed as positive integers starting at 0 and, if applicable, are ordered with respect to the sorting keys. You can also label edges by enabling M_LABEL_VALUE in MedgeControl() (enabled by default). The edge's label value is a positive integer greater than or equal to 1. Each edge in an image is given a unique label value; that is, unlike an edge's index value, an edge's label value will not change, regardless of future operations.

Edge Finder allows you to retrieve results for a particular edge or for all edges, using MedgeGetResult(). To retrieve results for a particular edge, you must specify either the edge's index or label value. However, since an edge's index value can change, it is recommended that you specify the label of the edge, especially if you want to perform selection and post-calculation operations. To retrieve results for all edges, you must specify M_ALL as the edge's index or label value.

Note that before an edge feature can be returned with MedgeGetResult(), you must first enable the feature for calculation, using MedgeControl(), and then call MedgeCalculate(). Every edge feature, except for the label value (M_LABEL_VALUE), is disabled by default. For more information on edge features, see the Edge features section earlier in this chapter.

To verify the availability of an edge feature (if it has been calculated), use MedgeGetResult(), and combine M_AVAILABLE to the specified result type. For a complete description of all possible results, see MedgeGetResult() in the MIL Reference.

In general, results are returned in pixels, and coordinates are relative to the center of the top-left pixel in the source image. If you calculate edges using a calibrated source image, results are automatically returned in the output coordinate system specified by the associated calibration object of the source image. However, you can change the output coordinate system to pixel units or world units by setting McalControl() with M_OUTPUT_COORDINATE_SYSTEM to either M_PIXEL or M_WORLD, respectively. For more information, see Calibration.

Note that if your image is calibrated, results are calculated in the real world; therefore, retrieving results in pixel units instead of world units will typically be slower. Also note that, in the presence of distortion, some results are meaningless when converted from real-world to pixel units (for example, the Feret angles). For example, if an edge appears warped in the source image, but the calibration object of the source image compensates for this during the extraction, the resulting Feret angles are meaningful in the real-world coordinate system, and meaningless in the image coordinate system.

Selecting the results

If you do not have many unwanted edges, it is usually faster to simply calculate all required features for all edges. However, calculating many features on a large number of unwanted edges can be unnecessarily time-consuming. To speed up this process, you can use MedgeSelect() to select a subset of edges for calculation and result retrieval.

Edges that meet the MedgeSelect() selection criteria can be included, excluded, or deleted from the result buffer. Further calculations are subsequently applied to included edges only. Excluded edges can be re-included at any time, while deleted edges are permanently removed from the result buffer.

Typically, if you have many unwanted edges, you calculate (for all edges) only those features that allow you to distinguish between wanted and unwanted edges. Then, you exclude the unwanted edges from further calculations, and calculate all the required features for the remaining included edges. To arrive at the required set of results, you can make as many calls as necessary to MedgeSelect() and MedgeCalculate().

Any calculation made to refine results after an initial call to MedgeCalculate() is considered a post-calculation. Post-calculating edges can be an effective way of speeding up processing time, especially if advanced features are only calculated on a small subset of selected edges. For more information on post-calculation, see the Post-calculation subsection in this section.

Note that, if at any time you calculate edges in a new source image, all current results will be discarded and replaced by the newly calculated edges. In addition, all selected features will be recalculated for all edges in the new source image. This means that you will have to restart the selection procedure.

The MedgeSelect() function can be used to select edges based on one of the following:

  • A calculated edge feature, where the edge selection depends on whether the specified edge feature meets the specified condition.
  • The inter-relationship of edges, where the edge selection depends on whether edges meet the specified box or chain condition of the specified edge or group of edges.
  • The current status of edges, where the edge selection depends on a specific edge, all edges, all included edges, or all excluded edges. The specified edge(s) will be included, included only, excluded, excluded only, or deleted. For example, you can include only (M_INCLUDE_ONLY) the excluded edges (M_EXCLUDED_EDGES), which will essentially swap the previously included and excluded edges.
  • The proximity of an edge or edges to a specified point, where the edge selection depends on the specified radius, and the nearest neighbor condition.

Note that you can also use MedgeSelect() to crop and select a portion of a specified edge. For more information, see the Advanced edge extraction section later in this chapter.

If the edges have been calculated using a calibrated source image, you must specify relevant values in the real world. If the edges have not been calculated using a calibrated source image, you must specify relevant values in pixels. Note that you can change the output coordinate system using McalControl() with M_OUTPUT_COORDINATE_SYSTEM set to M_PIXEL or M_WORLD.

Edge selection based on edge features

You can include, exclude, or delete edges based on whether the specified edge feature meets the specified condition. The edge feature used to select the edges must have been previously calculated for the included edges. After several selection and post-calculation operations, an edge feature might not be available or might only be available for a few edges of the included subset. To verify the availability of an edge feature, use MedgeGetResult(), and combine M_AVAILABLE to the specified feature result type.

The following code shows you how to apply a selection based on edge features:

MedgeSelect(MilResult, M_DELETE, M_MOMENT_ELONGATION, M_LESS, 0.8, M_NULL);

In this example, all edges with a moment elongation that is less than 0.8 are deleted.

Edge selection based on the inter-relationship of edges

You can include, exclude, or delete edges based on whether edges meet the specified box or chain condition. That is, you can include, exclude, or delete all edges that are inside or outside the bounding box of a specified edge or group of edges; similarly, you can include, exclude, or delete all edges that are inside or outside a specified edge or group of edges.

The inside chain and outside chain conditions only take effect when the edge is closed.

The following code shows you how to apply a selection based on the inter-relationship of edges:

MedgeSelect(MilResult, M_EXCLUDE, M_INCLUDED_EDGES, M_INSIDE_CHAIN, M_NULL, M_NULL);

In this example, all edges inside included edges are excluded.

Edge selection based on the proximity of the edges to a point

You can include, exclude, or delete edges based on their proximity to a specified point. You can select either the closest edge to a point, or all the edges in the specified vicinity of a point. Use MedgeControl() with M_NEAREST_NEIGHBOR_RADIUS to specify the maximum radius distance from the point.

The following code shows you how to apply a selection based on the proximity of the edges to a point:

MedgeSelect(MilResult, M_INCLUDE_ONLY, M_NULL, M_ALL_NEAREST_NEIGHBORS, Ax, Ay);

In this example, only the edges closest to the point A (Ax, Ay), and within the radius r, are included.

Internal processing buffers

When enabled, internal processing buffers are saved in the Edge Finder result buffer. These buffers are used when extracting edges. They include image derivative, angle, magnitude, source image, and mask buffers.

Some results can only be post-calculated if the appropriate internal buffer(s) have been previously saved. For example, you can only post-calculate the edge's strength feature if the internal magnitude buffer was initially saved. For more information, see the Post-calculation subsection in this section.

Using MedgeControl(), you can set whether the internal processing buffers will be saved. By default (M_DEFAULT), all internal processing buffers are not saved (M_DISABLE). You must enable saving them in the result buffer, using MedgeControl(), before the first call to MedgeCalculate(). Saving these buffers does not affect processing time.

If required, you can access the internal processing buffers. In general, Edge Finder allows you to optimize your access using MedgeGetResult(). For example, to get the identifier of the internal angle buffer, use MedgeGetResult() with M_ANGLE_ID. Note that the internal angle buffer must have been initially saved.

On certain systems, the internal processing buffer identifier might not be available, and M_NULL is returned instead. In this case, you can access the buffer using conventional MIL techniques. That is, you must:

  1. Use MedgeGetResult() to get the required information from the internal buffer (for example, M_FIRST_DERIVATIVE_X_ID + M_SIZE_BAND + M_TYPE_LONG).
  2. Allocate your own buffer, which should be a clone of the internal buffer you want to access (for example, M_FIRST_DERIVATIVE_X_ID).
  3. Use MedgeDraw() to copy the results into your own buffer.

Internal processing buffers can hold data that varies in dimension, data type, data depth, and attribute. To inquire about the data stored in the buffer, use MedgeGetResult() with one of the following values:

  • M_SIGN, which returns the buffer range.
  • M_SIZE_BAND, which returns the number of buffer color bands.
  • M_SIZE_BIT, which returns the depth per band, in bits.
  • M_SIZE_X, which returns the width of the buffer, in pixels.
  • M_SIZE_Y, which returns the height of the buffer, in pixels.
  • M_TYPE, which returns the buffer data type and depth. Depth is returned in bits.

Derivatives

You can save the internal derivative buffers used when extracting edges, in the Edge Finder result buffer. To save the buffers, use MedgeControl() with M_SAVE_DERIVATIVES.

To access the buffers, use MedgeGetResult() with the following:

  • M_CROSS_DERIVATIVE_ID
  • M_FIRST_DERIVATIVE_X_ID
  • M_FIRST_DERIVATIVE_Y_ID
  • M_FIRST_DERIVATIVES_ID
  • M_SECOND_DERIVATIVE_X_ID
  • M_SECOND_DERIVATIVE_Y_ID
  • M_SECOND_DERIVATIVES_ID

When extracting object contours, the source image's first derivatives in both the X- and Y- directions are used. When extracting line crests, the source image's second derivatives in both the X- and Y-directions, as well as the image's cross derivative, are used.

Typically, you will let Edge Finder manage its own derivative images to perform the edge extraction. However, you can specify your own derivative image buffers, so that the edge extraction is done with them instead. For more information on derivative images, and how they are used to extract edges, see the Providing the image's derivatives subsection in the Advanced edge extraction section later in this chapter.

Angles

You can save the internal angle buffer used when extracting edges, in the Edge Finder result buffer, using MedgeControl() with M_SAVE_ANGLE. To access the buffer, use MedgeGetResult() with M_ANGLE_ID. You can also save the angle value of the edge at each edgel position, in the Edge Finder result buffer, using MedgeControl() with M_SAVE_CHAIN_ANGLE. Angle values refer to the angle between the horizontal axis and the edge's perpendicular direction at each edgel location.

If you save the chain angle, the angle values of the extracted edges are saved. If you save the angle buffer, the angle values of all the edges in the source image are saved; in this case, the only relevant angle values are those whose edgels have a magnitude that is above the lower bound threshold value.

In MIL, angles are returned, counter-clockwise, from 0 degrees to 360 degrees and mapped in the range of 0 to 255.

Note that when extracting edge contours from a color image, the angle of the polarity cannot be obtained. For example, it is impossible to determine if red is darker or lighter than green; therefore, it is impossible to ascertain the direction of the transition.

Magnitude

You can save the internal magnitude buffer used when extracting edges, in the Edge Finder result buffer, using MedgeControl() with M_SAVE_MAGNITUDE. To access the buffer, use MedgeGetResult() with M_MAGNITUDE_ID. You can also save the magnitude value of the edge at each edgel position, in the Edge Finder result buffer, using MedgeControl() with M_SAVE_CHAIN_MAGNITUDE.

For object contours, the magnitude is the norm of the gradient vector at the edgel position. For line crests, the magnitude is equal to the maximum eigenvalue of the Hessian matrix at the edgel position.

If you save the chain magnitude, the magnitude values of the extracted edges are saved. If you save the magnitude buffer, the magnitude values of all the edges in the image are saved.

For more information on magnitude, and how it is used to extract edges, see the Customizing the edge extraction settings section earlier in this chapter.

Source image and mask

You can save both the source image, and the image used to mask the source image, in the Edge Finder result buffer. To do so, use MedgeControl() with M_SAVE_IMAGE and M_SAVE_MASK, respectively. To access the source image buffer, use MedgeGetResult() with M_IMAGE_ID. To access the mask buffer, use MedgeGetResult() with M_MASK_ID. For more information on masking, see the Masking the edges subsection in the Advanced edge extraction section later in this chapter.

Post-calculation

Any calculation made to refine results after an initial call to MedgeCalculate() is considered a post-calculation. As mentioned, to avoid repeatedly calculating time-consuming edge features for all edges in a source image, you can perform post-calculations on included edges of an Edge Finder result buffer. In this case, processing time greatly decreases since you do not have to calculate time-consuming edge features for all edges. The operation of selecting edges and adding new features can be repeated until the required result is calculated.

Post-calculations are done with the result buffer only; that is, you should not provide a source image to MedgeCalculate(). If you do provide a source image, all current results will be discarded and replaced by the newly calculated edges. In addition, all selected features will be recalculated for all edges in the new source image.

Restrictions

Typically, anything that can be calculated can also be post-calculated. However, some post-calculations are not valid unless certain values have been initially saved.

The internal magnitude buffer (MedgeControl() with M_SAVE_MAGNITUDE) must have been initially saved if you want to post-calculate the following:

  • MedgeControl() with M_AVERAGE_STRENGTH.
  • MedgeControl() with M_STRENGTH.
  • MedgeControl() with M_SAVE_CHAIN_MAGNITUDE.

Note that, if any of the values above were initially calculated, and you intend to fill edge gaps during post-calculation, the internal magnitude buffer (M_SAVE_MAGNITUDE) must have been initially saved.

The internal angle buffer (MedgeControl() with M_SAVE_ANGLE) must have been initially saved if you want to post-calculate the following:

  • MedgeControl() with M_SAVE_CHAIN_ANGLE.
  • The M_FILL_GAP_POLARITY settings M_SAME or M_REVERSE

Note that, if any of the values above were initially calculated, and you intend to fill edge gaps during post-calculation, the internal angle buffer (M_SAVE_ANGLE) must have been initially saved.

Thresholding can also be applied during post-calculation. In this case, all internal processing buffers (M_SAVE_...) must have been initially saved, in MedgeControl(). For more information, see the Thresholding subsection in the Customizing the edge extraction settings section earlier in this chapter.

Annotating the results

The MedgeDraw() function offers many drawing operations for annotating your results. You can, for example, draw a zoomed region of the edge map that was used to calculate results by specifying the appropriate values for M_DRAW_RELATIVE_ORIGIN_X, M_DRAW_RELATIVE_ORIGIN_Y, M_DRAW_SCALE_X, and M_DRAW_SCALE_Y, in MedgeControl(). The relative origin values must be specified in pixels, and are relative to the coordinates of the top-left corner of the region in the source image, while the scale values specify the X- and Y-scaling factors used to fill the destination buffer. For example:

You can also perform many other drawing operations with MedgeDraw(), such as draw a bounding box around each edge, a cross at each edge's center of gravity, or draw a cross at each edgel. In addition, you can draw the operation's numerical value by adding M_DRAW_VALUE to the following drawing operations: M_DRAW_CENTER_OF_GRAVITY, M_DRAW_POSITION, M_DRAW_FERET_MIN, M_DRAW_FERET_MAX, and M_DRAW_GENERAL_FERET. For example, if you add M_DRAW_VALUE to M_DRAW_CENTER_OF_GRAVITY, the center of gravity's coordinates are drawn within parenthesis, and centred above the drawing cross.

Typically, all drawing operations in MedgeDraw() can be combined; you can, therefore, draw multiple effects simultaneously. For example, to draw the edge's chains, center of gravity, bounding box, and the source image's internal cross-derivative buffer, you would specify M_DRAW_EDGE + M_DRAW_CENTER_OF_GRAVITY + M_DRAW_BOX + M_DRAW_CROSS_DERIVATIVE.

Note that only one internal drawing buffer can be drawn at a time; for example, you cannot combine M_DRAW_ANGLE and M_DRAW_MAGNITUDE in the same operation.

You can either use a previously allocated graphics context to control the drawing color, or you can use the default (M_DEFAULT) graphic context. You can either draw directly into the selected image buffer, or non-destructively annotate in a display's overlay buffer. For more information on MIL graphics, see the Overview section in Chapter 16: Displaying an image.

Note that if the edges are calculated using a calibrated source image, Edge Finder takes the calibration into account; that is, drawings are done in the real world (using pixel units).

For a complete list and explanation of each drawing operation, see MedgeDraw() in the MIL Reference.

FLOAT MODE TIMEOUT NUMBER OF CHAINS NUMBER OF CHAINED EDGELS LABEL VALUE ALL LABEL VALUE AVAILABLE OUTPUT COORDINATE SYSTEM PIXEL WORLD INCLUDE ONLY EXCLUDED EDGES OUTPUT COORDINATE SYSTEM PIXEL WORLD AVAILABLE NEAREST NEIGHBOR RADIUS ANGLE ID FIRST DERIVATIVE X ID SIZE BAND TYPE LONG FIRST DERIVATIVE X ID SIGN SIZE BAND SIZE BIT SIZE X SIZE Y TYPE SAVE DERIVATIVES CROSS DERIVATIVE ID FIRST DERIVATIVE X ID FIRST DERIVATIVE Y ID FIRST DERIVATIVES ID SECOND DERIVATIVE X ID SECOND DERIVATIVE Y ID SECOND DERIVATIVES ID SAVE ANGLE ANGLE ID SAVE CHAIN ANGLE SAVE MAGNITUDE MAGNITUDE ID SAVE CHAIN MAGNITUDE SAVE IMAGE SAVE MASK IMAGE ID MASK ID SAVE MAGNITUDE AVERAGE STRENGTH STRENGTH SAVE CHAIN MAGNITUDE SAVE MAGNITUDE SAVE ANGLE SAVE CHAIN ANGLE FILL GAP POLARITY SAME REVERSE SAVE ANGLE DRAW RELATIVE ORIGIN X DRAW RELATIVE ORIGIN Y DRAW SCALE X DRAW SCALE Y DRAW VALUE DRAW CENTER OF GRAVITY DRAW POSITION DRAW FERET MIN DRAW FERET MAX DRAW GENERAL FERET DRAW VALUE DRAW CENTER OF GRAVITY DRAW EDGE DRAW CENTER OF GRAVITY DRAW BOX DRAW CROSS DERIVATIVE DRAW ANGLE DRAW MAGNITUDE


Advanced edge extraction

User Guide:
Defining and adding models to your Model Finder context
How edges are calculated
Manipulating and controlling certain data buffer areas
Post-calculation
Thresholding

Reference:
MedgeCalculate
MedgeControl
MedgeGetNeighbors
MedgeGetResult
MedgeMask
MedgePut
MedgeSelect
Not available in MIL-Lite

Available in MIL

In addition to the fundamental edge extraction settings, Edge Finder provides advanced settings that allow you to write customized applications. Edges can be approximated, masked, cropped, and extracted using advanced thresholding settings. You can provide your own derivative images, and you can put data from user-supplied arrays into an Edge Finder result buffer. You can also get edgels, from an Edge Finder result buffer, which are the closest neighbors to a list of user-specified edgel coordinates.

Approximating the edges

If you are interested in a basic, geometric representation of the edge map, you can approximate your edges by specifying a polygonal segmentation. To do so, set MedgeControl() with M_CHAIN_APPROXIMATION to M_LINE. The default setting is M_DISABLE.

When set to M_LINE, each edge is piecewise approximated using a connected series of line segments, commonly called polylines. The polylines are defined by the set of automatically calculated vertex coordinates, which determine the location of the line segment ending points. Use MedgeGetResult() to retrieve the X (M_VERTICES_X) and Y (M_VERTICES_Y) coordinates of the vertices. This information can be retrieved for either a group of edges or a particular edge. You can also get the index of each vertices' corresponding edge or edgel, using MedgeGetResult() with M_VERTICES_CHAIN_INDEX and M_VERTICES_INDEX, respectively.

The resolution of the edge approximation can be adjusted using MedgeControl() with M_APPROXIMATION_TOLERANCE. The range of this control varies from 0 to 100. When set to 0, a very fine approximation of the edges is performed. When set to 100, a coarse approximation of the edges is performed. The default value is 50.

Note that when M_CHAIN_APPROXIMATION is set to M_LINE, both the edge map and the edge approximation is calculated. When M_CHAIN_APPROXIMATION is disabled, only the edge map is calculated.

Masking the edges

The MedgeMask() function allows you to apply a mask to the source image in the Edge Finder context. A mask is a binary image used to define irrelevant, inconsistent, or featureless areas in the source image. As a result, when a mask is set, subsequent calls to MedgeCalculate() will only extract edges in the source image's unmasked regions.

To set a mask, you must specify an image buffer (also known as a mask buffer) that identifies the masked pixels, using MedgeMask(). A masked pixel in the source image corresponds to a non-zero value in the mask buffer. Mask buffers are useful for creating non-rectangular regions of interest (ROIs). For rectangular ROIs, it is more effective to set a mask with child buffers. For more information, see the Manipulating and controlling certain data buffer areas section in Chapter 14: Specifying and managing your data buffers.

If necessary, you can remove the mask by setting the mask buffer identifier to M_NULL.

You can also mask edges during post-calculation. To do so, you must apply the mask after the initial call to MedgeCalculate(). In this case, masked edges are excluded from the result buffer and subsequent calculations. Note that partially masked edges are cropped.

Cropping the edges

Selecting an edge or a group of edges for calculation and result retrieval is typically sufficient for the majority of cases. However, in some advanced applications, you might only be interested in one part of a single edge. In this case, you can use MedgeSelect() with M_CROP_CHAIN.

Cropping an edge essentially means splitting one edge into two, thereby allowing you to choose only one of the edges to be included, excluded, or deleted from the Edge Finder result buffer. The selected portion receives a new label value, while the other portion keeps the original label value.

You can make as many cropping operations as necessary to arrive at the required set of results. Note that all calculated edge features are updated after each cropping operation.

To split an edge into two edges, you must specify the following:

  • The edge to split. This is identified by the edge's label value.
  • The point at which to split the edge. This is identified by the edgel's index value.
  • The portion of the edge to select. This is identified by all edgels in the specified edge with index values that are greater than, less than, greater than or equal to, or less than or equal to, the point at which the edge was split.

The following code shows you how to crop and exclude a portion of the edge labeled 5:

MedgeSelect(MilResult, M_EXCLUDE, M_CROP_CHAIN, M_GREATER, 5, 24);

In this example, the edge labeled 5 is split into two edges at edgel 25. One edge consists of all edgels ranging from the start of edge 5 to the 24th edgel; this edge keeps the label 5 and is not selected for exclusion. The other edge consists of all edgels ranging from the 25th edgel to the end of edge 5; this edge gets a new label value and is selected for exclusion.

Advanced thresholding

Edge Finder uses an hysteresis thresholding method to perform the edge extraction. That is, edge chains are extracted such that the magnitude values of all connected edgels are stronger than the lower bound threshold value, and at least one edgel in each edge chain has a magnitude that is stronger than the upper bound threshold value. For more information on thresholding, and on setting these bounds, see the Thresholding subsection in the Customizing the edge extraction settings section earlier in this chapter.

Edge Finder also allows you to customize the behavior of the hysteresis thresholding using the MedgeControl() with M_THRESHOLD_TYPE. That is, M_THRESHOLD_TYPE allows you to set how the upper and lower bound threshold values will be used (as opposed to setting the bounds themselves).

If you set M_THRESHOLD_TYPE to M_HYSTERESIS, both the lower bound threshold value and the upper bound threshold value will be used. This is the default setting.

If you set M_THRESHOLD_TYPE to M_NO_HYSTERESIS, the lower bound threshold value is ignored, and is considered to be equal to the upper bound threshold value.

If you set M_THRESHOLD_TYPE to M_FULL_HYSTERESIS, only the upper bound threshold value is used; the lower bound threshold value is ignored, and is considered to be 0. In this case, the extracted edges are sets of connected edgels such that the magnitude of at least one edgel in each edge is stronger than the upper bound threshold value. This type of thresholding can be used to extract edges in noise-free highly multi-contrast images.

Providing the image's derivatives

Typically, you will let Edge Finder manage its own derivative images to perform the edge extraction. However, you can specify your own derivative image buffers, so that the edge extraction is done with them instead. In this case, when you call MedgeCalculate(), set the source image to M_NULL, and specify the required derivative image buffers instead. For example:

MedgeCalculate(MilEdgeContext, M_NULL, DerivX, DerivY, M_NULL, MilResult, M_DEFAULT);

Note that, when specifying your own derivative image buffers, the majority of the processing control settings remain in effect, except for filter settings (for example, M_FILTER_SMOOTHNESS).

Derivative image buffers must be 1-band 16-bit signed or 1-band 32-bit floating-point buffers. In addition, all derivative image buffers must be consistent (for example, they must have the same scaling factor), and they must be of the same type and size. When the derivative image buffers are 32-bit floating-point buffers, Edge Finder uses floating-point precision calculations.

When extracting object contours, you must provide the image's first derivatives in both the X- and Y-directions. Note that the image derivatives must be consistent with the image axis; that is, if the intensity transition in the source image increases (from black to white) with the axis directions, the associated derivative values must be positive. Similarly, if the intensity transition in the source image decreases (from white to black) with the axis directions, the associated derivative values must be negative.

When extracting line crests, you must provide the image's second derivatives in both the X- and Y-direction, as well as the image's cross derivative. Note that the image derivatives must be consistent with the image axis; that is, a light line crest in the source image must result in negative second derivative values. Similarly, a dark line crest in the source image must result in positive second derivative values.

Putting data into an Edge Finder result buffer

Typically, you will use the Edge Finder result buffer to hold edge extraction information after an edge selection and calculation. However, Edge Finder also allows you to copy data from user-supplied arrays into an Edge Finder result buffer, using MedgePut(). This can be useful if you want to construct an Edge Finder result buffer that cannot be obtained from one image. You can therefore combine results from multiple Edge Finder result buffers to, for example, build a complete model to add to a Model Finder context. For more information, see the Defining and adding models to your Model Finder context section in Chapter 8: Geometric Model Finder.

Note that once edges are added to the Edge Finder result buffer, their features can be calculated with MedgeCalculate().

If edgels are calculated with pixel accuracy, the distance between consecutive edgels should be one pixel. If edgels are calculated with subpixel accuracy, the distance between consecutive edgels should be one edgel. To change the accuracy edgels are calculated with, use MedgeControl() with M_ACCURACY.

To add data to an Edge Finder result buffer, you must use MedgePut() to specify the following user-supplied arrays:

  • The X-coordinate(s) of the edgel(s).
  • The Y-coordinate(s) of the edgel(s).

You must also specify the number of edgels that you want to add to the Edge Finder result buffer. If necessary, you can also provide arrays containing the index of the edge each edgel belongs to, the orientation of the edgel and the magnitude of the edgel.

The information that you provide about the edges to add to the Edge Finder result buffer must be ordered on an edge by edge basis. For example, the following table illustrates how to supply X- and Y-coordinates for three distinct edges; note that the index provided is that of the edge each edgel belongs to:

Array of X-coordinates | Array of Y-coordinates | Array of edge indices
1 | 2 | 1
2 | 3 | 1
2 | 4 | 1
3 | 3 | 1
1 | 9 | 2
2 | 9 | 2
2 | 8 | 2
4 | 6 | 3
5 | 6 | 3

Finding the closest edgels to a list of points

From an Edge Finder result buffer, you can retrieve the coordinates of edgels that correspond to the closest neighbors from a list of user-specified source point coordinates. You must use MedgeGetNeighbors() to specify your input data, which must include the source points and, if necessary, a corresponding angle for each point.

You can also set a series of constraints to which potential results (edgel candidates) must adhere before being returned. Only edgels that meet all constraints (if any are set) can be returned as the closest neighbors. To set the constraint(s), use MedgeControl().

Note that if no edgels are found, M_NULL is returned.

You can either set simple constraints, advanced constraints, or both. Simple constraints are based on the location of the edgel candidate (the search region), while advanced constraints are typically based on the inner properties of the edgel candidate (the gradient angle).

When applying constraints, you should first set the ones based on location. This will establish the search region in the Edge Finder result buffer, and generate a group of edgel candidates. If you do not apply any edgel-based constraints, then the candidates in this region that are closest to the source points will be returned. However, if you set advanced constraints, then only the edgels in this region that also adhere to these constraints can be returned.

Advanced constraints are typically based on the source edgel's gradient angle. In this case, you must provide an angle value for each source point as part of your input data for MedgeGetNeighbors(). Therefore, the candidate edgel will only be returned if its gradient angle adheres to the constraint applied with the source angle. Note that an edgel's gradient angle is perpendicular to the edge, and is dependent on the edge extraction process. For more information on an edgel's gradient angle, see the How edges are calculated subsection in the Extracting the edges section earlier in this chapter.

Location-based constraints

You can use location-based constraints to limit the area within the Edge Finder result buffer from which edgels matching source points can be found. All edgels that do not fall within this area will be ignored. By default, the search region encompasses all edgels within the Edge Finder result buffer.

The following MedgeControl() control types can be used to specify location-based constraints:

  • M_SEARCH_ANGLE
  • M_SEARCH_ANGLE_SIGN
  • M_SEARCH_ANGLE_TOLERANCE
  • M_SEARCH_RADIUS_MAX
  • M_SEARCH_RADIUS_MIN

The maximum (M_SEARCH_RADIUS_MAX) and minimum (M_SEARCH_RADIUS_MIN) search radius constraints set the maximum and minimum distance radii in the Edge Finder result buffer that will be searched for the closest edgels. These values must be set in pixels. You must use MedgeGetNeighbors() to specify the source point from which the radii are measured.

Together, M_SEARCH_RADIUS_MAX and M_SEARCH_RADIUS_MIN define a ring-like region, in the Edge Finder result buffer, that will be used to find the closest edgels. All edgels that fall outside this ring will be ignored. By default, the minimum distance radius is 0 pixels, and the maximum distance radius is an infinite number of pixels (M_INFINITE); that is, by default, the entire Edge Finder result buffer will be searched. Note that the M_INFINITE setting will retrieve all the closest edgels (one edgel per edge).

The Edge Finder result buffer region that will be searched for the closest edgels can be further constrained to an angular sector within the area defined by the radius constraints (M_SEARCH_RADIUS_...). You can use M_SEARCH_ANGLE to define the search angle, and M_SEARCH_ANGLE_TOLERANCE to define the tolerance. The search angle is relative to the source angle set with MedgeGetNeighbors(), and the tolerance is applied to the resulting angle evenly; that is, half of the tolerance angle is applied in the positive direction, and half is applied in the negative direction. All edgels that are located within this angular sector (tolerance) can be returned. Any value between 0.0° to 360.0° can be set for both the search angle and tolerance. By default, M_SEARCH_ANGLE is set to 0.0° and M_SEARCH_ANGLE_TOLERANCE is set to 360.0 degrees; therefore, all edgels will be considered.

For example, if you set M_SEARCH_ANGLE to 30.0°, M_SEARCH_ANGLE_TOLERANCE to 30.0°, and the source angle to 15°, only edgels that fall between 30.0° and 60.0° will be considered.

You can also set the sign of the angle that will be used to find the closest edgels with M_SEARCH_ANGLE_SIGN. The sign of the angle can be the same as (M_SAME) or the reverse of (M_REVERSE), the source angle set with MedgeGetNeighbors(). The default is M_SAME. The sign can also be set to M_SAME_OR_REVERSE, which means that it can be the same as, or the reverse of, the source angle.

For example, if you set the source angle to 15°, and the search angle and tolerance to 30.0°, then edgels that fall within an angle that has the same sign would be located between 30.0° and 60.0°, edgels that fall within an angle that has a reverse sign would be located between 210° and 240°, and edgels that fall within an angle that has any sign would be located at either between 30.0° and 60.0°, or between 210° and 240°.

Note that, if necessary, you can allow a point between two edgels to be returned, using M_GET_SUBEDGELS in MedgeGetNeighbors(). In this case, an edgel candidate is returned, as long as an edge (and not necessarily an edgel) is located within the search region.

Edgel-based constraints

As mentioned, location-based constraints limit the region in the Edge Finder result buffer that will be searched for the closest edgels. Edgel-based constraints, on the other hand, are applied to the edgels themselves. Only edgels that adhere to these constraints can be returned. Note that, if you have applied location-based constraints, only edgels within the defined search region will be considered.

The following MedgeControl() control types are used to specify edgel-based constraints:

  • M_NEIGHBOR_MAXIMUM_NUMBER
  • M_NEIGHBOR_MINIMUM_SPACING
  • M_NEIGHBOR_ANGLE
  • M_NEIGHBOR_ANGLE_TOLERANCE

The maximum number constraint (M_NEIGHBOR_MAXIMUM_NUMBER) sets the maximum number of closest edgels (that is, the N closest edgels) that can be returned, for each source point. The default maximum number is 1; therefore, the edgel closest to each source point is returned.

The minimum spacing constraint (M_NEIGHBOR_MINIMUM_SPACING) sets the minimum distance separating closest edgel candidates within the same edge. Therefore, edgels that are too close together will not be returned. The spacing is set in number of edgels and must be greater than or equal to 1. The default minimum spacing is infinite (M_INFINITE), which means that there is no minimum distance (number of edgels) that must separate edgel candidates.

The neighbor's angle constraint (M_NEIGHBOR_ANGLE) sets the gradient angle that an edgel must have, before being considered a candidate. M_NEIGHBOR_ANGLE is based on a source angle (for each source point), set with MedgeGetNeighbors(). The gradient angle of the edgel candidate can be the same (M_SAME), the reverse (M_REVERSE), or the same or reverse (M_SAME_OR_REVERSE) as the angle of the source point. The constraint can also be set to any angle (M_ANY); in this case, the source angle and the source point are ignored. The default is M_ANY. Note that M_NEIGHBOR_ANGLE can only be used if the M_SAVE_ANGLE internal buffer was initially saved.

You can also set an angular tolerance for M_NEIGHBOR_ANGLE using M_NEIGHBOR_ANGLE_TOLERANCE. In this case, candidate edgels that have a gradient angle that falls within the specified angular range (tolerance) can be returned. The tolerance is applied to the M_NEIGHBOR_ANGLE value evenly; that is, half of the tolerance angle is applied in the positive direction, and half is applied in the negative direction. Any angle between 0.0° to 360.0° can be set as a tolerance. By default, M_NEIGHBOR_ANGLE_TOLERANCE is set to 180.0°; therefore, only candidate edgels that have a gradient angle that is the same as the source angle will be considered. Note that setting M_NEIGHBOR_ANGLE_TOLERANCE to 360.0° is equivalent to setting the neighbor angle constraint to M_ANY; in either case, edgels with any gradient angle are considered.

For example, if you set the angle of the source point to 45°, M_NEIGHBOR_ANGLE to M_SAME_OR_REVERSE, and M_NEIGHBOR_ANGLE_TOLERANCE to 90°, only edgels that have an angle between 0° and 90° or between 180° and 270° can be returned. In the following example, edgels 1 and 3 are considered the closest edgel candidates.

CHAIN APPROXIMATION LINE DISABLE LINE VERTICES X VERTICES Y VERTICES CHAIN INDEX VERTICES INDEX APPROXIMATION TOLERANCE CHAIN APPROXIMATION LINE CHAIN APPROXIMATION CROP CHAIN THRESHOLD TYPE THRESHOLD TYPE THRESHOLD TYPE HYSTERESIS THRESHOLD TYPE NO HYSTERESIS THRESHOLD TYPE FULL HYSTERESIS FILTER SMOOTHNESS ACCURACY SEARCH ANGLE SEARCH ANGLE SIGN SEARCH ANGLE TOLERANCE SEARCH RADIUS MAX SEARCH RADIUS MIN SEARCH RADIUS MAX SEARCH RADIUS MIN SEARCH RADIUS MAX SEARCH RADIUS MIN INFINITE INFINITE SEARCH ANGLE SEARCH ANGLE TOLERANCE SEARCH ANGLE SEARCH ANGLE TOLERANCE SEARCH ANGLE SEARCH ANGLE TOLERANCE SEARCH ANGLE SIGN SAME REVERSE SAME SAME OR REVERSE GET SUBEDGELS NEIGHBOR MAXIMUM NUMBER NEIGHBOR MINIMUM SPACING NEIGHBOR ANGLE NEIGHBOR ANGLE TOLERANCE NEIGHBOR MAXIMUM NUMBER NEIGHBOR MINIMUM SPACING INFINITE NEIGHBOR ANGLE NEIGHBOR ANGLE SAME REVERSE SAME OR REVERSE ANY ANY NEIGHBOR ANGLE SAVE ANGLE NEIGHBOR ANGLE NEIGHBOR ANGLE TOLERANCE NEIGHBOR ANGLE NEIGHBOR ANGLE TOLERANCE NEIGHBOR ANGLE TOLERANCE ANY NEIGHBOR ANGLE SAME OR REVERSE NEIGHBOR ANGLE TOLERANCE


Interfacing with the Geometric Model Finder module

User Guide:
Geometric Model Finder

Reference:
MedgeCalculate
MedgeControl
MedgeSelect
MmodDefine
Not available in MIL-Lite

Available in MIL

The MIL Geometric Model Finder module allows you to define models from an Edge Finder result buffer or to find model occurrences in an Edge Finder result buffer.

If you want to use Model Finder, and are also familiar with Edge Finder, you can perform specialized edge selections, which can reduce the unnecessary geometric complexity in both the model and/or the target image. For example, if you want to find model occurrences in an Edge Finder result buffer, you can use MedgeSelect() to select which edges to include in the buffer, based on one of several criteria. As a result, Model Finder will have much less information to analyze, thereby potentially speeding up the search for the model(s).

Models and target images should use the same edge types (object contours or line crests). Therefore, if the target edges are object contours, then the model edges must also be object contours. Similarly, if the target edges are line crests, then the model edges must also be line crests.

If you expect to use an Edge Finder result buffer with the MIL Geometric Model Finder module, you must enable M_MODEL_FINDER_COMPATIBLE in MedgeControl() before the initial call to MedgeCalculate(). To add models defined from an Edge Finder result buffer to a Model Finder context, use the M_EDGE_RESULT model type in MmodDefine().

To set the sensitivity with which edgels are detected, Edge Finder users can use MedgeControl() with either M_THRESHOLD_MODE or M_DETAIL_LEVEL. M_DETAIL_LEVEL was provided to help users already familiar with Model Finder and should only be used if you are interfacing with the MIL Geometric Model Finder module. Note that, in MedgeControl(), the M_DETAIL_LEVEL setting overrides the M_THRESHOLD_MODE setting.

The following is an example of a typical Edge Finder/Model Finder application:

For more information on Model Finder, see Chapter 8: Geometric Model Finder.

MODEL FINDER COMPATIBLE EDGE RESULT THRESHOLD MODE DETAIL LEVEL DETAIL LEVEL DETAIL LEVEL THRESHOLD MODE


An example

Availability | Previous
Not available in MIL-Lite

Available in MIL

The Edge Finder example MEdge.c illustrates how the module can be used with the following source image:

Specifically, MEdge.c shows how to define an Edge Finder context to extract and select object contours in the above source image.

MEdge.c


Optical character recognition

Chapter 10: Optical character recognition
Availability
Not available in MIL-Lite

Available in MIL

This chapter presents the features of the optical character recognition (OCR) module.

  • The MIL OCR module
  • Steps to reading or verifying a string in an image
  • Basic concepts
  • Guidelines for choosing context types
    • General OCR font context type
    • Constrained OCR font context type
    • Switching between the two
    • Deciding which OCR font context type to use
  • OCR font
    • User-defined MIL OCR fonts
    • Existing MIL OCR fonts
    • Using a Semi font
    • Quality and scale are important
    • Visualizing
    • Erasing characters
  • Defining the target strings
    • Calibrating your font
      • Automatic calibration
      • Manual calibration
    • Setting appropriate processing controls
      • Blanks
      • Broken characters
      • Morphologicial filtering
      • Thickness and dots
      • Touching characters
    • Specifying other string information
      • Number of strings
      • String lengths
      • Angle
      • Positional variation
    • Constraints
  • Locating your text
  • Determining what is a match
    • Acceptance levels
    • Unrecognized characters
  • Retrieving and analyzing the results
    • A character
    • A string
    • A text
    • Understanding odd results
    • Hooking functions
  • Improving search speed
  • Optical character recognition examples


The MIL OCR module

Availability | Next
Not available in MIL-Lite

Available in MIL

Many types of industries require the analysis of character strings in images. For example, the semiconductor industry requires serial numbers, printed on wafers, to be read for tracking purposes. The pharmaceutical industry requires analysis of medicine bottle labels to ensure, for example, that expiry dates are properly printed.

The MIL optical character recognition (OCR) module provides a powerful and easy to use function set for reading and verifying mechanically generated character strings in 8-bit grayscale images, providing results such as quality (match) scores and validity flags. The module is especially designed to operate on character strings in degraded images, with up to ± 180° of rotation in the target string. The OCR module can be used in conjunction with other MIL functions to develop hardware independent OCR programs for machine vision applications.

The module offers the choice of loading a set of grayscale character representations from a MIL font file, or gathering the needed grayscale character representations from image buffers and/or other files to create a MIL font. Each character in the string to be read (target string) is compared to each character representation in this font. The representation with the closest match is chosen and its ASCII value is returned. You can adjust the value at which this match is considered a success by setting the acceptance level. A read operation yields a string of ASCII characters, a confidence score for each character, and its position in the target string, as well as a confidence score for the whole string. A verify operation also provides a validity flag for each character.

For applications requiring custom font types, the OCR module supports the creation of custom MIL fonts that can be saved to disk and restored as needed. Two predefined MIL font files are provided to read and verify semiconductor wafer serial numbers of standard SEMI font character types.

The module also supports user-specified character constraints, the automatic resizing of a font to better match the target characters (automatic calibration), and the reading of texts that span multiple lines. To ensure recognition accuracy, checksum calculations are performed when analyzing standard SEMI font character strings. For strings of custom font types, user-defined functions can validate the read character string automatically (for example, using custom checksum functions).


Steps to reading or verifying a string in an image

Reference:
MocrAllocFont
MocrAllocResult
MocrCalibrateFont
MocrControl
MocrCopyFont
MocrFree
MocrGetResult
MocrHookFunction
MocrImportFont
MocrInquire
MocrModifyFont
MocrPreprocess
MocrReadString
MocrRestoreFont
MocrSaveFont
MocrSetConstraint
MocrVerifyString
Not available in MIL-Lite

Available in MIL

The following steps provide a basic methodology for using the MIL optical character recognition module:

  1. Load an existing OCR font context from disk using MocrRestoreFont(), or create a custom OCR font context using MocrAllocFont() and then use either MocrCopyFont() or MocrImportFont() to add character representations to the OCR font context.
  2. If necessary, specify the type of characters (alphabetic, numeric, or other) that should appear at specific positions in the string, using MocrSetConstraint().

    3. With custom fonts, you can hook a custom validation check function to the read/verify operations using MocrHookFunction(). After checking character constraints on a found string, the validity function will be executed. If using an OCR font context allocated for a semi font, checksum calculations are performed automatically.

  3. If necessary, adjust general processing controls to fit your application, using successive calls to MocrControl().
  4. Calibrate the OCR font context automatically to match the target image's character size and spacing, using MocrCalibrateFont(). Alternately, you can set these values manually using MocrControl().

    5. To physically modify the polarity and sizing of the font of an OCR font context, use MocrModifyFont().

  5. Preprocess the OCR font context with MocrPreprocess() once all the constraints and processing controls are set.
  6. Allocate a result buffer using MocrAllocResult(). This buffer will be used to store subsequent read or verify result values.
  7. Acquire or load a target image. Optionally, limit the area to be read and/or improve the quality of the image using some other MIL module. Note that OCR can only process unsigned 8-bit buffers.

    8. The cleaner the background of a target image, the better the results of a read/verify operation.

  8. Read or verify the string in the target image, using MocrReadString() or MocrVerifyString(), respectively. These functions are performed according to the defined character constraints and processing controls.
  9. Obtain the OCR results using MocrGetResult().
  10. Free all your allocated OCR objects using MocrFree().

In general, the first five steps are performed once, while steps 6 to 10 are repeated as required. Since you can save the calibration results, character constraints, and processing controls with the character representations (with MocrSaveFont()), steps 1 to 5 can be replaced by a single step which loads an OCR font context from disk.

Before performing a read/verify operation, if any font-specific control or target constraint changes, the OCR font context should be preprocessed. Use MocrInquire() to establish if the OCR font context requires preprocessing.


Basic concepts

Availability | Previous | Next
Not available in MIL-Lite

Available in MIL

The basic concepts and vocabulary conventions for the MIL OCR module are:

  • Acceptance level. The user-defined level against which the match score is compared to determine if the match is valid.
  • Character representations. The grayscale bitmap (image) or ASCII representation (drawing) of a character used by MIL OCR.
  • Entire text. One or many strings contained within the target image.
  • Fixed size font. A series of character representations that each has the same character Y size and character X size. Characters of a fixed size font tend to be all upper-case.
  • Match score. The result of a read/verify operation that shows the level of correlation between the character representations in the OCR font and the characters found within the image, taking into account character constraints.
  • OCR font. The MIL OCR font contains the character representations.
  • OCR font context. The MIL OCR font context is a container for the OCR font information, target image character size and spacing, constraints, and processing controls.
  • Target string(s). The text to be found within the target image. This can be as little as a single character or multiple lines of characters.


Guidelines for choosing context types

Reference:
MocrAllocFont
MocrCalibrateFont
MocrControl
Not available in MIL-Lite

Available in MIL

An OCR font context is a container for the OCR font information, target image character size and spacing, constraints, and processing controls. When allocating your font context (using MocrAllocFont()), you can choose 1 of 2 types:

  • The M_GENERAL OCR font context type.
  • The M_CONSTRAINED OCR font context type.

General OCR font context type

The M_GENERAL OCR font context type requires less information about the target string but works best on clean target images. If you need to specify the location of the string to eliminate erroneous results, use a child buffer to create a smaller region in which to search.

The target image should have a clearly visible threshold (binarization) between the characters and their background. The threshold must preserve the shape of the characters. Broken and/or touching characters can degrade the results.

When using an M_GENERAL OCR font context, the cleaner the background of a target image, the better the results of a search. The backgrounds should not have blobs of similar size and intensity as the characters.

The following MocrControl() control types are available only when using an M_GENERAL OCR font context:

  • Enabling blank character recognition (use M_BLANK_CHARACTERS).
  • Finding the string length automatically (use M_STRING_SIZE with M_ANY).
  • Finding the character width automatically (use M_TARGET_CHAR_SIZE_X with M_SAME) when using a fixed size font.
  • Finding the character height automatically (use M_TARGET_CHAR_SIZE_Y with M_SAME) when using a fixed size font.
  • Finding the inter-character spacing automatically (use M_TARGET_CHAR_SPACING with M_SAME if it is the same between characters or M_ANY if it differs between characters).

An example image best suited to using an M_GENERAL OCR font context is:

Use the M_CONSTRAINED font context type if your image is greatly degraded or if you know a lot about the location of your target string(s).

Constrained OCR font context type

The M_CONSTRAINED OCR font context type works well with degraded target images. This type of context requires more information about the target string, but provides a more robust search. M_CONSTRAINED is best used when all the details are known about the target string. Especially, the more known about the target string's location and the size and spacing of its characters, the better the results of a search. Since more precise information is required to calibrate your font with that of the target string, you can automatically calibrate the font with the target string using MocrCalibrateFont(). Note that the automatic calibration of a font can only be done when using an M_CONSTRAINED OCR font context type.

An example image best suited to using an M_CONSTRAINED OCR font context is:

Use the M_GENERAL OCR font context type if your image contains a string comprised of unevenly spaced characters or a string that has blanks or broken characters.

Switching between the two

Defining a good OCR font context takes time. Modifying an existing OCR font context is faster than creating a new OCR font context from scratch. When dealing with two sets of images, for example, with similar OCR font requirements with regards to character representation, but differing in some other aspect (for example, sizing, spacing, processing controls, and/or quality), it might be faster to switch OCR font context types, reset the required constraints and controls, preprocess, and then read the new images. You can switch the type of context using the MocrControl() function with M_CONTEXT_CONVERT.

For example, after reading a series of images with an evenly spaced font (depicted below), to read a group of images where the font is not equally spaced, requires a switch from M_CONSTRAINED to M_GENERAL.

Note, when you switch between context types, any unsupported setting is reset to its default.

Deciding which OCR font context type to use

The following decision tree should help you decide when you should use an M_GENERAL or an M_CONSTRAINED OCR font context type. Note that, if a question does not apply to your situation, skip to the next question in the list.

  1. Can the target image be thresholded? That is, can a clear distinction be made between the background of the target image and the characters to be read?
    • Yes. Use an M_GENERAL OCR font context.
    • No. Use an M_CONSTRAINED OCR font context.
  2. Does the target image have intensity problems; that is, does it have reflections or uneven lighting, or is it so heavily degraded that bits of the string are difficult to read with the naked eye?
    • Yes. Use an M_CONSTRAINED OCR font context.
    • No. Use an M_GENERAL OCR font context.
  3. Is the target string made up of characters that vary in size? Does the target string contain variable spacing?
    • Yes. Use an M_GENERAL OCR font context.
    • No. Use an M_CONSTRAINED OCR font context.

    Note that variable spacing does not include "jitter", which refers to tiny variations in the position of the target characters relative to the expected character position. Jitter can be corrected by calibrating your OCR font context (with MocrCalibrateFont()) to match the target string.

    Set TargetCharSizeXMin and TargetCharSizeYMin to measure to the clearest edge of the target characters to be read. Set TargetCharSizeXMax and TargetCharSizeYMax to measure to the faintest edge of the target character to be read - just before the shading (if any) matches the background color.

  4. Are any of the characters to be read broken?

    5. When there are broken characters, the best way to determine which OCR font context to use is to try both the M_GENERAL and the M_CONSTRAINED contexts. In a specific situation, one or the other can produce better results.

    If using an M_GENERAL OCR font context, use MocrControl() with M_BROKEN_CHAR set to M_ENABLE.

  5. Are any of the characters to be read touching?

    6. When there are touching characters, the best way to determine which OCR font context to use is to try both the M_GENERAL and the M_CONSTRAINED contexts. In a specific situation, one or the other can produce better results.

    If using an M_GENERAL OCR font context, use MocrControl() with M_TOUCHING_CHAR set to M_ENABLE.

If your answers were a combination of both M_GENERAL and M_CONSTRAINED OCR font contexts, use the OCR font context most often recommended. You will have to further manipulate your target image or your OCR font (mostly using MocrControl()), to improve results.

GENERAL CONSTRAINED GENERAL GENERAL GENERAL BLANK CHARACTERS STRING SIZE ANY TARGET CHAR SIZE X SAME TARGET CHAR SIZE Y SAME TARGET CHAR SPACING SAME ANY GENERAL CONSTRAINED CONSTRAINED CONSTRAINED CONSTRAINED CONSTRAINED GENERAL CONTEXT CONVERT CONSTRAINED GENERAL GENERAL CONSTRAINED GENERAL CONSTRAINED CONSTRAINED GENERAL GENERAL CONSTRAINED GENERAL CONSTRAINED GENERAL BROKEN CHAR ENABLE GENERAL CONSTRAINED GENERAL TOUCHING CHAR ENABLE GENERAL CONSTRAINED


OCR font

User Guide:
Optical character recognition examples

Reference:
MocrAllocFont
MocrCalibrateFont
MocrControl
MocrCopyFont
MocrImportFont
MocrInquire
MocrModifyFont
MocrRestoreFont
MocrSaveFont
Not available in MIL-Lite

Available in MIL

You must specify the MIL OCR font to read/verify the character strings in target images. MIL uses fonts (or typesets) to specify the style and size of characters in the images to be read or verified.

An OCR font contains the following information:

  • The grayscale representations of the characters.
  • Codes identifying each character (ASCII codes for the characters).
  • The number of characters in the OCR font.
  • Character dimensions.

The above information can be calibrated (with MocrCalibrateFont() or MocrControl()), modified (with MocrModifyFont()), and saved (with MocrSaveFont()) for later restoration.

User-defined MIL OCR fonts

Unless using one of the two provided predefined OCR fonts, you must define a custom OCR font. You can create a user-defined OCR font from scratch, or make minor modifications to an existing OCR font, already saved to disk.

To create a custom font:

  1. Allocate an OCR font context, using MocrAllocFont().

    2. When allocating an OCR font context, you must specify the maximum number of characters that can be stored in the font, and the dimensions of the font's character representations and their character cells.

  2. Grab/create the grayscale character representations of the font in a MIL image buffer and then copy them from the image buffer to the OCR font context, using MocrCopyFont(). Alternatively, you can import grayscale character representations from a text file or an image file (for example, a TIFF) into an OCR font context using MocrImportFont().

    3. Note that, for best effect, try using the MIL OCRReader utility, or an interactive tool such as Matrox Inspector, to determine font character widths and heights. Interactive graphic tools are also good for creating new characters to add to an existing font.

    When importing, or copying, character representations to the OCR font context, the OCR font context must have sufficient space to hold the representations of all the specified characters. You can use MocrInquire() to determine the maximum number of characters that can be stored in the font and the size of each character.

  3. Once copied or imported into an OCR font context, the entire OCR font context can be saved on disk using MocrSaveFont(), and then later restored using MocrRestoreFont().

The following is an example of a character in its MIL OCR font character cell and the dimensions that you will have to specify during OCR font context allocation. Values are to be specified in pixels. Each square in the grid represents one pixel.

The MocrAllocFont() CharCellSizeX, CharCellSizeY, CharOffsetX, CharOffsetY, CharSizeX, and CharSizeY parameters must comply with the following restrictions:

  • 2* CharOffsetX + CharSizeX <= CharCellSizeX
  • 2* CharOffsetY + CharSizeY <= CharCellSizeY
  • CharCellSizeY, CharCellSizeX, CharSizeY, CharSizeX must be >= 6 pixels and <= 256 pixels.

When copying the character representations from an image buffer, or importing them from an image file, the characters must have the dimensions specified during OCR font context allocation.

When importing the character representations from an ASCII file, font character representations must be presented as follows.

Note that in this format, 'pixels' are delineated by a blank space. So '00' counts as one pixel.

This information breaks down into the following:

Row | Description
01 | Specifies ASCII file format.
02 | Blank row.
03 | Specifies the start of a new character representation and its associated (generally ASCII) character.
04 to 36 | Specifies the alpha-numerical representation of the character.
37 | Blank row.
38 | Specifies the start of a new character representation and its associated (generally ASCII) character.
39 to 71 | Specifies the alpha-numerical representation of the character.
etc. | This pattern is repeated for every character in the font.

For an example on how to create a user-defined MIL OCR font, see the Optical character recognition examples section later in this chapter.

Existing MIL OCR fonts

Once created, a MIL OCR font can be saved and restored as needed. Restoring this information (with MocrRestoreFont()) rather than creating the MIL OCR font from scratch saves time, especially if the restored font requires no further modifications. Note that the entire OCR font context is restored when restoring the font using MocrRestoreFont().

Using a Semi font

MIL OCR comes with two Semi fonts (M_SEMI_M12_92 or M_SEMI_M13_88). These can be used directly or modified to suit your needs.

To use a Semi font directly, restore it using MocrRestoreFont(). Once restored, it can be modified using the MIL OCR functions.

To create a new font based on a SEMI font:

  1. Create an OCR font context using MocrAllocFont() with:
    • The FontType parameter set to either M_SEMI_M12_92 or M_SEMI_M13_88.
    • The StringLength parameter must be set to 12 when using M_SEMI_M12_92 and 18 when using M_SEMI_M13_88.
    • The CharNumber parameter must be set to 38 (A-Z, 0-9, -, .).
  2. Use either MocrCopyFont() or MocrImportFont() to add character representations from an existing Semi font.

Quality and scale are important

Using high-quality character representations will result in the best results. OCR processing relies on using the cleanest font characters possible.

When using an M_GENERAL OCR font context, broken characters and spaces, even if expected in the target string, should not be defined in the font. Instead, you should enable the ability to read broken characters using MocrControl() with M_BROKEN_CHAR, and/or enable the ability to read spaces using MocrControl() with M_BLANK_CHARACTERS.

When using the MIL OCR font to search for a target string at an angle or if using an M_GENERAL font context type, the threshold between the characters and the background must preserve the shape of the characters and have a clearly-visible point of differentiation (binarization).

If the characters in the target image are brighter than the background (for example, white on black), then the character representations included in your font must also be of characters that are brighter than the background. The foreground is specified at context allocation time (MocrAllocFont()) and can be changed later using MocrModifyFont() with M_INVERT. This changes both the character representations and the setting specified at allocation time.

If the size of the character representations in the font is not the same as those in the target string, you can calibrate the font (discussed later). Alternatively, when the physical size of the character representations of the OCR font differ from those in the target image, changing the size of the character representations of the OCR font could improve the robustness of the search. To change the size, use MocrModifyFont() with M_RESIZE. Changing the size of the font permanently in the OCR font can be faster than resizing the font before each read/verify operation, as is done when the font is calibrated.

Visualizing

It might be necessary, at some point during application development, to display the character representations of your MIL OCR font. To do so, use MocrCopyFont() to copy the character representations to a displayable image buffer.

Erasing characters

To remove a character from the OCR font, use MocrControl() with M_CHAR_ERASE and specify the ASCII code associated with the character representation to remove. An OCR font can contain a limited number of characters; this number is set during OCR font context allocation. Removing unused or erroneously added characters is the easiest way to assure that these characters will not be used when looking for matches in the target string and that there is space for new characters to be added.

SEMI M12 92 SEMI M13 88 SEMI M12 92 SEMI M13 88 SEMI M12 92 SEMI M13 88 GENERAL BROKEN CHAR BLANK CHARACTERS GENERAL INVERT RESIZE CHAR ERASE


Defining the target strings

Reference:
MocrCalibrateFont
MocrControl
MocrInquire
MocrPreprocess
MocrSetConstraint
Not available in MIL-Lite

Available in MIL

Define the target strings in the following ways:

  • Calibrating your font helps to match the characters in the target image to those within the MIL OCR font.
  • Setting the appropriate processing controls and string information helps better define the search criteria for best results.
  • Constraints allow you to limit the search to specific characters within the MIL OCR font.

Calibrating your font

You should either manually or automatically calibrate your MIL OCR font to better match the size of the characters and the inter-character spacing in the target image. The more MIL OCR knows about your target image, the faster it can search for the required string(s) and the more robust the results.

Note that in some cases, it might be more efficient to change the size of the actual font than to calibrate.

Automatic calibration

Automatic calibration is only available when you are using an M_CONSTRAINED OCR font context.

Use MocrCalibrateFont() with a small, given range of sizes in which to search, and it will test all the possible sizes in that range, returning the best match of the width, the height, and the spacing of the characters in your sample target image. The sample target image should be the best possible image with an angle, string length, and number of strings that are representative of the target images.

Automatic calibration is always performed using only the first string with the highest match score found in the sample target image. To calibrate multiple strings, create a child buffer around each. If the string cannot be located in the image, an error is generated. Skipping the string locator step (using MocrControl() with M_SKIP_STRING_LOCATION) might produce undesirable results. This is discussed later in this chapter.

When you are dealing with long strings, the spacing in your target image should be as accurate as possible.

Automatic calibration resets values set during a manual calibration.

Manual calibration

Manual calibration can be done for either M_GENERAL or M_CONSTRAINED OCR font context types. With an M_GENERAL OCR font context type, manual calibration does not require exact numbers.

Use MocrControl() to manually set the width (M_TARGET_CHAR_SIZE_X), the height (M_TARGET_CHAR_SIZE_Y), and spacing (M_TARGET_CHAR_SPACING) of the target image characters, in pixels. If the exact measurements are known, manually setting the size of the target characters is faster than an automatic calibration.

When using an M_GENERAL OCR font context with a fixed size font and reading a string that has the same spacing between characters, but the size of the spacing is unknown, use MocrControl() with M_TARGET_CHAR_SPACING set to M_SAME. If the spacing is unknown and/or not the same between characters, use MocrControl() with M_TARGET_CHAR_SPACING set to M_ANY.

Note that the scale factors between target character sizes and font character sizes must be between 0.25 and 4.0, inclusive. The following restrictions apply:

  • (TargetCharSizeXMin/CharSizeX) >= 0.25
  • (TargetCharSizeYMin/CharSizeY) >= 0.25
  • (TargetCharSizeXMax/CharSizeX) <= 4.0
  • (TargetCharSizeYMax/CharSizeY) <= 4.0

Setting appropriate processing controls

Each of the following controls can improve the robustness of the search. Note, however, that these controls increase the complexity of the operation and reduce its overall speed.

Blanks

By default, MIL OCR ignores blanks in the target image.

When using an M_GENERAL OCR font context type, you can use MocrControl() with M_BLANK_CHARACTERS to enable the ability to read blank characters. Blank characters should not be included in your MIL OCR font. Once enabled, you must modify your string length to include the number of blanks that you expect in the target image. Blanks before the target string are not counted. A blank space will have the same size and inter-character spacing as all other characters in the MIL OCR font, unless M_TARGET_CHAR_SPACING is set to M_ANY.

Note that M_BLANK_CHARACTERS is available only when using an M_GENERAL OCR font context.

Broken characters

If the target image contains characters that contain scratches or breaks, the target image contains a broken character. To force MIL OCR to identify these characters as a possible match of the characters within the MIL OCR font, use MocrControl() with M_BROKEN_CHAR enabled. This will reduce the speed of subsequent read/verify operations but can increase robustness. Broken characters should not be included in a MIL OCR font.

Note that, when using an M_GENERAL OCR font context, M_BROKEN_CHAR must be enabled to read a broken character. In an M_CONSTRAINED OCR font context, MIL OCR will automatically try to identify broken characters.

Morphologicial filtering

When the difference between the foreground and the background is so slight that it causes read or verification errors, increase or decrease the value for morphological filtering, using MocrControl() with M_MORPHOLOGIC_FILTERING, to improve MIL OCR's chances of finding and identifying the characters within the target string. Experimentation is required to determine the exact number that should be passed to this control.

Thickness and dots

You can thicken the target characters using MocrControl() with M_THICKEN_CHAR when reading thin characters made up of dots to make the characters easier to find and identify.

Touching characters

If characters in the target image touch each other, or if they are connected to blobs of a similar intensity, enable touching characters, using MocrControl() with M_TOUCHING_CHAR. This will improve MIL OCR's chances of finding and identifying these characters. This will reduce the speed of subsequent read/verify operations but can increase robustness.

Note that, when using an M_GENERAL OCR font context, M_TOUCHING_CHAR must be enabled to read touching characters. When using an M_CONSTRAINED OCR font context, MIL OCR will automatically try to identify touching characters.

Specifying other string information

In some cases, additional information about the target string is required. For example, to search for multiple strings, or to search at an angle, certain control types should be set. In other cases, additional information will increase the robustness of the search.

Number of strings

You can set the search to find multiple strings using MocrControl() with M_STRING_NUMBER. Multiple strings can only be found if each string resides on a different line within the target image and each line of text does not overlap the previous.

Note that, for best results, all strings in an image should be of similar length, have a consistent inter-line spacing, and should start at a similar location along the X-axis.

Identifying the number of strings to read/verify is important when dealing with a target image containing multiple lines. It does not have to be specified for single-string images since the default value is 1.

String lengths

You should specify the length of the strings to be read, to receive reliable results using MIL OCR. The maximum string length must be set at the time of OCR font context allocation. With an M_GENERAL OCR font context type, the maximum string length can be set to M_ANY. Note, however, that this increases the processing time since the OCR module must then try to calculate the string length based on successful matches between the target image and the MIL OCR font.

The default string length is set to the maximum string length of the OCR font context. When the OCR font context constraints are set (discussed later), specifying a string length can also improve the speed of a following read/verify operation.

Note that if M_SEMI_M12_92 is used, the string length must be 12. If M_SEMI_M13_88 is used, the string length must be 18.

If the entire text of the target image contains one or more strings that differ significantly in length, use an M_GENERAL OCR font context for fast results. For more robust results, allocate separate M_CONSTRAINED OCR font contexts for each line of text to be read/verified. Experimentation with both OCR font context types is the only way to determine which provides the best solution for each case.

Often, when using an M_CONSTRAINED OCR font context, errors in string length result in unpredictable results.

Angle

You can search for the strings within the target image at a specific angle, or through an angular range. The angle of the search is used to locate the string or strings before a read/verify operation. If individual lines in the target image are set at different angles, create a child buffer containing each string at a different angle and read them using a different OCR font context.

These child buffers should not overlap.

For each OCR font context, you can specify the specific angle of the search using MocrControl() with M_STRING_ANGLE. By default, the search angle is 0.0°.

To search through an angular range, use MocrControl() with M_STRING_ANGLE_DELTA_POS to specify the positive range and/or M_STRING_ANGLE_DELTA_NEG to specify the negative range in which to search. A search for a string within the target image through an angular range always starts at the string angle, specified with M_STRING_ANGLE. Note that the range in which to search, either in the positive or negative direction, should never be greater than 180Â°.

If you were searching for a string at approximately 15°, you might want to search between 30° and 0°. To do this, you would set M_STRING_ANGLE to 15°, and M_STRING_ANGLE_DELTA_POS and M_STRING_ANGLE_DELTA_NEG would both be set to 15°. This results in a 30° arc that will be searched for your string.

By default, searching through a range of angles is enabled only if one of the M_STRING_ANGLE_DELTA... control types is set to a value other than 0.0°.

Note that, after setting M_STRING_ANGLE..., you must use MocrPreprocess().

Positional variation

If the inter-character spacing is not even, you can increase the robustness of the search by specifying the maximum variation in position of the characters. The position variation increases the region being searched for characters within the target image (using MocrControl() with M_TARGET_CHAR_POSITION_VARIATION_X and M_TARGET_CHAR_POSITION_VARIATION_Y). These values are relative to the expected position of each character.

Constraints

The read operation compares each character in the target image to each character in the font to find the best match. You might know beforehand that certain characters (or types of characters) should appear at specific positions in the string. If this is the case, you can speed up and increase the robustness of the read operation by restricting the comparison to only those characters in the font. The following types of constraints can be set for each character in the string, using MocrSetConstraint():

  • A digit (M_DIGIT): ASCII codes 48 to 57.
  • A letter (M_LETTER): ASCII codes 65 to 90 and 97 to 122.
  • An uppercase letter (M_LETTER + M_UPPERCASE): ASCII characters 65 to 90.
  • A lowercase letter (M_LETTER + M_LOWERCASE): ASCII characters 97 to 122.
  • A character from a specific list of characters (for example, A,1,b,2), including special characters and punctuations (for example, &, -, ...).

Note that MIL OCR will search for the number of characters specified by the target string length. For best results, the number of characters to be found in your target image should match your string length.

The constraints are stored with the OCR font context as part of its information set and can be inquired, using MocrInquire().

The following is an example of how character constraints are set. For example, the character in the first position should be the letter K and the character in the second position should be any upper or lowercase letter.

/* Set character constraints for each position of the string to read. */
MocrSetConstraint(OcrFont, 0, M_LETTER, "K"); /* Must be K. */
MocrSetConstraint(OcrFont, 1, M_LETTER, M_NULL); /* Any letter. */
MocrSetConstraint(OcrFont, 2, M_DIGIT, M_NULL); /* Any digit. */
MocrSetConstraint(OcrFont, 3, M_DIGIT, "12"); /* Must be 1 or 2. */
MocrSetConstraint(OcrFont, 4, M_DIGIT, M_NULL); /* Any digit. */
MocrSetConstraint(OcrFont, 5, M_DEFAULT,M_NULL); /* Any character. */
CONSTRAINED SKIP STRING LOCATION GENERAL CONSTRAINED GENERAL TARGET CHAR SIZE X TARGET CHAR SIZE Y TARGET CHAR SPACING GENERAL TARGET CHAR SPACING SAME TARGET CHAR SPACING ANY GENERAL BLANK CHARACTERS TARGET CHAR SPACING ANY BLANK CHARACTERS GENERAL BROKEN CHAR GENERAL BROKEN CHAR CONSTRAINED MORPHOLOGIC FILTERING THICKEN CHAR TOUCHING CHAR GENERAL TOUCHING CHAR CONSTRAINED STRING NUMBER GENERAL ANY SEMI M12 92 SEMI M13 88 GENERAL CONSTRAINED CONSTRAINED STRING ANGLE STRING ANGLE DELTA POS STRING ANGLE DELTA NEG STRING ANGLE STRING ANGLE STRING ANGLE DELTA POS STRING ANGLE DELTA NEG TARGET CHAR POSITION VARIATION X TARGET CHAR POSITION VARIATION Y DIGIT LETTER LETTER UPPERCASE LETTER LOWERCASE


Locating your text

User Guide:
Manipulating and controlling certain data buffer areas

Reference:
MocrControl
Not available in MIL-Lite

Available in MIL

MIL OCR tries to locate the strings to read/verify automatically. Both the angle of the string and the string length information are used to perform the location.

The location step can be skipped to improve the speed of a read/verify operation, using MocrControl() with M_SKIP_STRING_LOCATION.

When using an M_CONSTRAINED OCR font context, you can skip the location step if you create a child buffer or mask that contains only the target string and a minimum amount of space around it. Use other modules in MIL to determine the required buffer size. For more information, refer to the Manipulating and controlling certain data buffer areas section in Chapter 14: Specifying and managing your data buffers.

When using an M_GENERAL OCR font context, string location can be skipped if you are using a clean target image.

SKIP STRING LOCATION CONSTRAINED GENERAL


Determining what is a match

Reference:
MocrControl
Not available in MIL-Lite

Available in MIL

Acceptance levels

You can set the acceptance level of a successful read/verify operation:

  • For each character (MocrControl() with M_CHAR_ACCEPTANCE).

    • If the correspondence (also known as the match score) between a character in an image and a character and its constraints in an OCR font context is less than the specified acceptance level, that character is considered invalid and the associated validity flag for that character is set to false.

  • For the entire string of characters and the entire text (MocrControl() with M_STRING_ACCEPTANCE).

    • If the match score for the entire string passes the acceptance level set for the string, the string is considered valid and its validity flag is set to true. The match score for the entire string is determined by taking the average of the match scores for all characters in that string. The match score for the entire text is the average of the match scores for all the read/verified strings.

A perfect match has a match score of 100%, and no correlation has a match score of 0%. If your images have a lot of noise or distortion, set a lower acceptance level. Poor-quality images increase the chance of false readings and will probably increase the time required to read/verify the character.

Perfect matches are highly improbable due to noise obtained during image acquisition.

Unrecognized characters

You can specify the symbol for unrecognized (or invalid) characters (MocrControl() with M_CHAR_INVALID). If a character's match score does not reach the specified acceptance level, you can force a specified symbol to be returned at that position in the string. If no symbol is specified (default), the character with the closest match will be returned. For example:

Target string: | "HELLO"
Invalid character: | "*"
Acceptance threshold: | 30% 30% 30% 30% 30%
Match scores: | 70% 15% 53% 24% 80%
Result: | H*L*O

Since the match scores of the characters in the second and fourth positions are less than the specified acceptance level, these characters are replaced by asterisks in the result.

CHAR ACCEPTANCE STRING ACCEPTANCE CHAR INVALID


Retrieving and analyzing the results

User Guide:
Defining the target strings

Reference:
MocrControl
MocrGetResult
MocrHookFunction
MocrReadString
MocrVerifyString
Not available in MIL-Lite

Available in MIL

After having potentially located the string in your target image using MocrReadString() or MocrVerifyString(), you can extract the required results from your result buffer using MocrGetResult(). All OCR results are based on the concept of a character. A string is a group of characters that reside along the same line along the same principle axis and the same angle. Multiple strings are a group of strings that are to be read together.

Results are returned in the order in which they are read/verified, from left to right. Always check the validity of the string to ensure that the match score is equal to or exceeds the acceptance level, using MocrGetResult() with M_STRING_VALID_FLAG. Additional checks against the actual length read and the intended length could be made to assure accuracy.

Invalid characters in the resulting strings can be replaced with an ASCII character using MocrControl() with M_CHAR_INVALID. Note that this control type must be set before performing the read/verify operation.

A character

Character data is returned for each character in the string or strings. This data includes:

  • Position along the X and Y axis.
  • The height and width of the character.
  • The match score of the character.
  • The spacing of the character.
  • The validity of the character (based on the character's acceptance level).
  • The ASCII version of the character.

A string

String data is returned for each string. String data includes:

  • The contents of the string.
  • The angle of the string.
  • The length of the string.
  • The match score of the string.
  • The validity of the string (based on the string's acceptance level).

A text

When reading/verifying multiple strings, text data is returned for all the strings. The text data includes:

  • All the characters read, even if they are on different lines in the target image.
  • The number of characters read in total.
  • The match score for the entire text.
  • The threshold value used to binarize the target image. Note that this is only available when using an M_GENERAL OCR font context, multiple strings, and/or the string has a few degrees of rotation.

All the results for a single string are also available for each string in the entire text.

Understanding odd results

Unexpected results can come from one of the following scenarios.

  • If characters are vertically overlapping (or touching).

    • Create a child buffer that contains the characters below the point where they overlap. This effectively removes the overlapping portion of the characters.

    The more that characters overlap vertically, the less chance for a valid result.

    To read a target image containing multiple lines of text, MIL OCR requires that there is enough space between lines. If lines are too close together (vertically), the robustness of the search will suffer.

    Use MocrControl() with M_TOUCHING_CHAR to read horizontally touching characters.

  • If the string or strings have non-uniform inter-character spacing.

    • Read/verify the string with an M_GENERAL OCR font context type and use MocrControl() with M_TARGET_CHAR_SPACING set to M_ANY.

    This setting is best suited to read non-uniformly spaced characters.

    Create a child buffer to contain the most extreme spacing examples for best results.

  • If the string or strings in the target image are greatly disparate.

    • Create a child buffer for each different region in position, angle, and/or length. Read each region individually for best results.

    The more similar the strings in angle, position, and length, the faster and more robust the search.

    If the spacing, length or angle differ, use different MIL OCR font contexts for each child buffer, with their operational and processing controls configured appropriately for each situation. For more information, see the Defining the target strings section earlier in this chapter.

  • If the target image contains multiple lines with different lengths.

    • Create a child buffer that contains the area with the shorter strings and read the shorter strings, using a different OCR font context, into a different OCR result buffer.

    Strings of similar length are easiest to locate.

    Make sure that the target image has a consistent inter-line spacing and that all lines start at a similar location along the X-axis, for best results.

  • If the target image contains blanks.

    • Verify that your string length is correct.

    Your string length determines the number of characters sought.

    If the problem persists and the ability to read blank spaces is enabled (use MocrControl() with M_BLANK_CHARACTERS) remember to include them in your string length (use MocrControl() with M_STRING_SIZE).

  • If the target image contains a lot of non-character blobs.

    • Try to improve your target image before trying to read/verify again. For more information, see the Defining the target strings section earlier in this chapter.

    An image that does not include blobs of equal intensity as the foreground will return best results.

    Use MocrControl() with M_MORPHOLOGIC_FILTERING to internally enhance the contrast of the image.

  • If the target image contains a string longer or shorter than what was expected.

    • MIL OCR will try to return the number of characters that you set as existing in the target image using MocrControl() with M_STRING_SIZE.

    If your string length is set to a value larger than the actual length of the string in the target image, and if using M_GENERAL OCR font context, MIL will return the best matches found and stop searching after it has found the specified number of characters. If using an M_CONSTRAINED OCR font context, the string length determines the exact number of characters found, even if this means returning characters with low match scores.

  • If the target string is at an angle.

    • Verify that the principal axis of the entire text is equal to the expected angle (nominal angle), specified using MocrControl() with M_STRING_ANGLE.

    A string at an angle greater or less than 0 takes longer to find.

    Create a target image with a string angle of 0. If the string is properly read, your original application did not follow the expected use of string angle.

Hooking functions

MIL OCR allows you to attach or detach user-defined function to a MIL OCR event when the specified OCR font context is used. This can be used to impose global string constraints, and can be used to implement custom checksum functions or to reject strings that would have otherwise met the character constraints imposed.

Once your user-defined function is created, use MocrHookFunction() to attach it to the validation of a string. It will then execute during the last stage of either MocrReadString() or MocrVerifyString(). When MocrHookFunction() is used in conjunction with an M_SEMI... OCR font context type, the function being hooked will replace the default validation function.

STRING VALID FLAG CHAR INVALID GENERAL TOUCHING CHAR GENERAL TARGET CHAR SPACING ANY BLANK CHARACTERS STRING SIZE MORPHOLOGIC FILTERING STRING SIZE GENERAL CONSTRAINED STRING ANGLE


Improving search speed

Reference:
MocrCalibrateFont
MocrControl
MocrSetConstraint
Not available in MIL-Lite

Available in MIL

To ensure the fastest possible read or verify operation:

  • Use as clean a target image as possible when using an M_GENERAL OCR font context.
  • Calibrate the OCR font context's character size and target spacing, to match the characters in the target images.
  • Preprocess once all the processing controls and constraints are set and before the application's first read/verify operation.
  • Reduce the area to read/verify in the target image by creating a child buffer around the target string using MbufChild...(); the search time is roughly proportional to the area searched.
  • Set character constraints using MocrSetConstraint()
  • Skip the location step, when possible.
  • Set the speed and reliability of the algorithm by setting the robustness factor (MocrControl() with M_SPEED). For instance, when reading larger characters, robustness can be sacrificed for speed.
  • Disable the M_BLANK_CHARACTERS and M_BROKEN_CHAR control types unless absolutely necessary.
  • Adjust your image to minimize the angular search range of the read/verify operation.

The more you know about the target image, the faster it can be read/verified. This includes:

  • Knowing where the string is located within the target image.
  • Knowing the height, width, and the inter-character spacing of the string within the target image.
  • Knowing the exact number of lines to be read from the target image.
  • Knowing the exact string length or at least that all the strings are of similar length.
  • Having good examples of each character to be found in the target image so a high-quality MIL OCR font can be made.

MIL OCR can help determine some of this information, for example, MocrCalibrateFont() can automatically calibrate the OCR font context's character size to match the characters in the sample target image, and the string location step with MocrControl() can find a string within the image, barring any problems.

GENERAL SPEED BLANK CHARACTERS BROKEN CHAR


Optical character recognition examples

Availability | Previous
Not available in MIL-Lite

Available in MIL

MOcr.c consists of two examples that use the OCR module. The first example of MOcr.c uses the following image:

This example demonstrates how to read the serial number in an image of a semiconductor wafer, using the OCR functions in conjunction with other MIL functions. The serial number, printed on the wafer, is a standard SEMI M12 92 font character string, containing a checksum. Refer to the OCRSemiFontReadExample() function for this example.

The second example demonstrates how to create a new custom OCR font from characters in an image, and how to use the font to read a string drawn in an image. Refer to the OCRCustomFontExample() function for this example.

MOcr.c


String Reader

Chapter 11: String Reader
Availability
Not available in MIL-Lite

Available in MIL

This chapter explains how to perform feature-based character recognition with the MIL String Reader module.

  • MIL String Reader module
    • The Basics
    • Feature-based versus template-based
  • Steps to reading a string in an image
  • Basic concepts
  • Creating and customizing the fonts
    • Adding and deleting fonts to the context
    • Adding and deleting characters to the font
      • Character representation
      • Adding multiple strings
      • Adding a single character
      • Source image foreground
    • Normalize characters
    • Baseline
    • Sorting characters
  • Adding and deleting string models to the context
    • Adding and deleting the string models
      • Target image foreground
  • Space
    • Space size
      • Space width
      • Maximum number of consecutive spaces
  • Number of strings to read
  • Degrees of freedom
    • String angle and character angle
    • String scale and character scale
    • String aspect ratio and character aspect ratio
    • Character's maximum baseline deviation
    • Skew angle
  • Rules for character placement
    • Character constraints
    • String size
    • Grammar rules
  • Global context settings
    • General global context settings
      • Minimum contrast
      • Speed
      • Timeout
    • Advanced global context settings
      • Encoding
      • Space character
      • String separator
  • Scores and acceptances
    • String acceptance and score
    • Character acceptance and score
      • Character similarity
      • Character homogeneity
  • Retrieving results and annotation
    • Preprocess and read
    • Annotation
    • Formatted results
  • An example


MIL String Reader module

Availability | Next
Not available in MIL-Lite

Available in MIL

The MIL String Reader module is a set of powerful functions that allow you to perform feature-based character recognition. You can quickly define your strings, and their associated fonts, from either a source image, or from fonts installed on your system, such as Arial or Times New Roman. The read operation can then be called, and will often yield accurate results without having to alter many of the module's default values. With this type of technology at your fingertips, you can easily write robust string identification applications, such as Automatic Number Plate Recognition (ANPR).

The Basics

All applications written with the MIL String Reader module are organized using a String Reader context. Before reading a string, you must first allocate the context, and add at least one font and one string model to that context (by default, the context is empty). You can, in fact, add up to 255 fonts and 255 string models. Fonts and string models are maintained, within the context, as two separate lists, with their own independent index values.

A string model is a container that holds all the parameters and constraints that control how to read one or many specific string(s) in the target image. Each string model is associated to one or more fonts, which are containers that hold the list of characters with their corresponding image representation. Based on this information, strings are either accepted or rejected by the read operation.

The MIL String Reader module reads strings in grayscale images, and provides numerous results, such as acceptance score and character position. String Reader offers many settings that allow you to tailor the read algorithm to meet your specific needs. For example, by setting the character acceptance score to various levels, you can execute precise control over which strings are accepted by the read operation.

Feature-based versus template-based

The MIL OCR module performs character recognition using template-based technology, while the MIL String Reader module offers feature-based technology to perform the character recognition. String Reader is therefore far more robust than OCR, though if you have a lot of information about the string you want to read, such as its exact size and scale, OCR might perform the read operation faster.

However, as soon as some important bit of information is not exactly known, or if the target image is not ideal, String Reader becomes the optimal choice. Being feature-based, String Reader is invariant to changes in scale, aspect ratio, and contrast. It also offers considerable tolerance towards variations in perspective and angle. In addition, String Reader supports multiple user-defined grammar rules and multi-font definition in a single context, making applications far simpler to write and maintain.


Steps to reading a string in an image

Reference:
MstrAlloc
MstrAllocResult
MstrControl
MstrDraw
MstrEditFont
MstrFree
MstrGetResult
MstrPreprocess
MstrRead
MstrSave
MstrSetConstraint
Not available in MIL-Lite

Available in MIL

The following steps provide a basic methodology for using the MIL String Reader module:

  1. Allocate a String Reader context, using MstrAlloc().
  2. Allocate a String Reader result buffer to hold the results of the read operation, using MstrAllocResult().
  3. Add a font to the String Reader context, using MstrControl(). Note, more than one font can be added to the context.
  4. Add characters to the font, using MstrEditFont().
  5. If necessary, draw all the characters that have been added to the font, using MstrDraw(). This allows you to visually verify that the characters have been added correctly.
  6. If necessary, normalize the characters in the font to an appropriate size, using MstrEditFont().
  7. Add a string model to the String Reader context, using MstrControl(). Note, more than one string model can be added to the context.
  8. Set the number of strings to read, using MstrControl().
  9. Set the number of expected characters for all string models, using MstrControl().
  10. If necessary, set character constraints, using MstrSetConstraint().
  11. If necessary, adjust general context controls to fit your application, using MstrControl().
  12. Preprocess the String Reader context, using MstrPreprocess().
  13. Perform the read operation in the specified target image, using MstrRead().
  14. Get the number of strings read and retrieve the required results from the String Reader result buffer, using MstrGetResult().
  15. If necessary, draw the results, using MstrDraw().
  16. If necessary, save your String Reader context, using MstrSave().
  17. Free all your allocated objects, using MstrFree().

Using an interactive graphic tool, such as the MIL StringReader utility or Matrox Inspector, is an excellent way of creating new characters to add to an existing font.


Basic concepts

Availability | Previous | Next
Not available in MIL-Lite

Available in MIL

The basic concepts and vocabulary conventions for the MIL String Reader module are:

  • Character. Any ASCII or Unicode symbol, such as a letter or a digit. A group of characters is used to define a font.
  • Font. A container, within the String Reader context, that holds a list of characters (ASCII or Unicode) with the corresponding character's image representation. In a given font, the ASCII code or Unicode uniquely identifies the character. The String Reader context can contain up to 255 fonts.
  • Source image. The image from which a font can be defined. Note that fonts can also be defined from fonts installed on your system; for example, Times New Roman and Arial.
  • Space. The space delimits multiple strings that are on the same line. Strings on separate lines are automatically considered separate strings. Note that String Reader does not consider space a character.
  • String. A group of regular characters that overlap, in the Y-direction, with every other regular character within that group. Note that if the distance between two successive characters is greater than the maximum number of consecutive spaces, times space size (at the scale of the string), these two characters are considered part of two separate strings.
  • String model. A container, within the String Reader context, that holds all the parameters and constraints that control how to read one or many specific string(s) in the target image. The String Reader context can contain up to 255 string models. By default, each string model is associated to all fonts within the String Reader context.
  • String Reader context. The container for all string models and fonts. The String Reader context also contains global read settings that apply to the character recognition algorithm.
  • String result. A string, read in the target image, that respects all the parameters and constraints of a string model.
  • Target image. The image in which the string is to be read.


Creating and customizing the fonts

User Guide:
Adding and deleting characters to the font
Annotation
Character's maximum baseline deviation
Manipulating and controlling certain data buffer areas
Encoding
Target image foreground

Reference:
MstrAlloc
MstrControl
MstrDraw
MstrEditFont
MstrGetResult
MstrInquire
MstrRead
Not available in MIL-Lite

Available in MIL

Once you have allocated a String Reader context, using MstrAlloc(), you must add at least one font to the font list in that context. By default, the font list is empty. Once a font has been added to it, it is automatically assigned default settings. Typically, these settings are sufficient for many applications. It is recommended that you try the defaults before changing them. However, if the default font settings do not suit your needs, you can:

  • Set the font's character type.
  • Set the font's character representation.
  • Normalize the font's characters.
  • Set the font's character baseline.
  • Sort the font's characters.

Adding and deleting fonts to the context

To add a font to the font list in the String Reader context, use MstrControl() with M_FONT_ADD set to either M_USER_DEFINED or M_DEFAULT. Both settings specify an empty font; that is, there are no characters. To add characters to the font, see the Adding and deleting characters to the font subsection in this section. The maximum number of fonts that you can add to a String Reader context is 255. Note that adding a font is not a font setting; fonts are added on the context level.

When a font has been added to the context, it is assigned an index. Fonts are indexed as positive, consecutive integers starting at zero, and increasing by one.

Indices can be used to access a font. For example, to control a font setting, you must use MstrControl() with M_FONT_INDEX() set to the index of a specific font.

Indices can also be used to delete a font. To do so, use MstrControl() with M_FONT_DELETE. Note that when a font is deleted, its index is reassigned; that is, all font indices are immediately renumbered.

Adding and deleting characters to the font

Once you have added a font to the context, you will typically add characters to that font. Adding a character to a font is not a context setting; characters are added on a font level. That is, the String Reader context contains a list of fonts, each of which, in turn, contains a list of characters. To set the index of the font that you will add characters to, use MstrEditFont() with M_FONT_INDEX(). To add the characters, use MstrEditFont() with M_CHAR_ADD. Note that all operations that apply to the characters of a specific font are done using MstrEditFont().

When a character has been added to a font, it can be accessed with its respective font's index, and its own character value. Although this is not an index value, these two values do, in a sense, represent the character's unique key. For example, to access the character 'A' in font 0, you must use MstrEditFont() with M_FONT_INDEX() set to 0, and the Param2 parameter set to 'A'.

Since the characters are partly identified based on their own value, one font cannot have more than one representation of a particular character. If you try to add a character that already exists in a font, the existing character will be replaced by the newly added one. However, multiple fonts within the context can have the same character, each of which is distinguished by its respective font index, which is always unique. For example, to access the character 'A' in either font 0 or 1, you must use MstrEditFont() with M_FONT_INDEX() set to either 0 or 1, and the Param2 parameter for both fonts set to 'A'.

This unique character access key is needed to delete a character in a font. To do so, use MstrEditFont() with M_CHAR_DELETE, and set M_FONT_INDEX() to the index value of the character's respective font.

When deleting characters, the OperationMode and Param1 parameters must be set to M_DEFAULT, and the Param3 parameter must be set to M_NULL. The Param2 parameter must either be set to M_NULL, which will delete all characters in the font, or to a null-terminated string array of all the characters to delete. For example, to delete the characters 'ABCDE1234' from font 0, you would set M_FONT_INDEX() to 0, and the Param2 parameter to 'ABCDE1234'.

When setting specific characters to delete from a font, the string array you provide must be of the right type for the encoding scheme selected. For more information, see the Encoding subsection in the Global context settings section later in this chapter.

To return the index of the font of each individual character within the string, use MstrGetResult() with M_CHAR_FONT. To return the index of the character in the font for each individual character within the string, use MstrGetResult() with M_CHAR_INDEX. The index returned is that of the character's font, or of the character in its font, at the moment the MstrRead() operation was performed.

Note that when adding a character to a font, its character type (MstrEditFont() with M_CHAR_TYPE) is set to M_REGULAR by default. Currently, there is no other character type available. M_CHAR_TYPE is reserved for future expansion.

Character representation

Characters in the font are represented as either system characters or user-defined characters.

System characters refer to characters that have been added from a given system font, such as Arial. To add characters from a system font, use MstrEditFont() with M_CHAR_ADD, and set the OperationMode parameter to M_SYSTEM_FONT. You must also set the Param1 parameter to the size (in pixels) of the characters to add, the Param2 parameter to a null-terminated string of all the characters to add, and the Param3 parameter to a null-terminated string containing the TrueType file name, such as Arial. Note that you can also add symbols (such as the dash or the apostrophe) to the font. To do so, use Param2 to either provide the symbol itself, or its ASCII code.

User-defined characters refer to characters that have been added from source images. When the font you require is not available on your system, user-defined characters are the ideal solution, since all you need is an image with the character's correct font. To add user-defined characters to the font, use MstrEditFont() with M_CHAR_ADD, and set the OperationMode parameter to M_USER_DEFINED. You must also set the Param1 parameter to the identifier of the source image containing the characters to add, the Param2 parameter to a null-terminated string for all the characters to add, and the Param3 parameter to M_NULL. To add the standard characters of the font (that is, 'A' to 'Z', 'a' to 'z', and '0' to '9'), set Param2 to M_NULL.

The characters in the source image are taken from left to right and from top to bottom and are associated to the corresponding characters given in the character list. That is, if you want to define an 'A' (Param2), then String Reader will look through the source image (Param1) from left to right and from top to bottom, until an 'A' is located. If a suitable character is not located, an error is returned. Note that each potential character in the source image must be entirely connected (except for the accentuated characters) and should not be merged with other characters or other image objects. Also, the characters you want to add from the source image must all be approximately the same size.

When adding user-defined characters, it is best to provide a source image that is of good quality, and that ideally contains only the characters you want to add in the font, though this is not mandatory. As mentioned, if all the characters in the string you want to add cannot be located in the image, an error is returned. Even if the operation succeeds, you should draw the characters of the font to ensure that every character is defined as expected, using MstrDraw(). For more information, see the Annotation subsection in the Retrieving results and annotation section later in this chapter.

Note that locating the specified characters in the source image is typically very robust. You needn't, for example, tell String Reader the specific scale, size, or location of the characters. However, if you are experiencing problems, or if you want to speed up this process, you can create a child buffer around the region that contains the characters you want to add. For more information, see the Manipulating and controlling certain data buffer areas section in Chapter 14: Specifying and managing your data buffers.

When adding characters defined from either system or user-defined fonts, the character cannot be smaller than 6x6 pixels. Also, the string array you provide containing the characters to add must be of the right type for the encoding scheme selected. For more information, see the Encoding subsection in the Global context settings section later in this chapter.

Adding multiple strings

The source image you provide for user-defined fonts can also be used to add multiple strings (groups of characters) to one font simultaneously. To do so, you must insert the space character (' ', or ASCII 32) to separate the strings included in the list of characters you have provided with Param2.

Typically, multiple strings can be used when one or more of the following situations in the source image is encountered: the characters are not all on the same line, the contrast is clearly different between characters, or there is clearly a horizontal space between groups of characters. For example, if you are using the following source image of a license plate, which clearly has a space between groups of characters, you could define the multiple strings, 'QKN 918 389 EYE':

Adding a single character

When adding user-defined characters, String Reader is able to locate the corresponding letters in the source image because it assumes a Latin-based font. For example, if you want to define an 'A', there must be a character in your source image that resembles an 'A'. Typically, this type of intelligence is very useful, since it can, for example, tell the difference between a real character and background noise. However, this type of automatic font definition makes it impossible to define non-Latin-based characters, like Chinese letters, or to define an 'A' as a 'B', or to define special characters (which are not necessarily connected).

You can however choose to bypass String Reader's automatic Latin-based font definition by adding a single user-defined character to the font. To do so, you must combine M_USER_DEFINED with M_SINGLE. In this case, the entire source image that you provide in Param1 is automatically defined as one character. For example, if your source image is the letter 'A', and the character you are adding is listed as the letter 'B' (Param2), then you have just added a 'B' that looks like an 'A'.

Source image foreground

When defining characters from a source image, you can set the foreground for the font in that image. By default, this foreground is black. You can however change it to white. To alter the foreground, you can combine M_USER_DEFINED with either M_FOREGROUND_BLACK or M_FOREGROUND_WHITE.

Note that M_FOREGROUND_BLACK and M_FOREGROUND_WHITE only allow you to set the foreground of the font with which the potential characters in the source image are located. For example, if you want to add an 'A' to the font, and the 'A' in the source image has a white foreground, then you would use M_FOREGROUND_WHITE. However, once the 'A' has been added to the font, it does not have a white foreground; in fact, fonts do not have any foreground at all. Therefore, adding M_FOREGROUND_WHITE when locating the character in the source image has no bearing on the foreground used to perform a read operation in the target image. If, in this scenario, you then wanted to read the 'A' in the target image, where the characters have a white foreground, you must explicitly set the read foreground value to white, otherwise you will get an error. For more information regarding the foreground the read operation ultimately uses, see the Target image foreground subsection in the Adding and deleting string models to the context section later in this chapter.

Normalize characters

Whether you are working with system fonts or user-defined fonts, it is not unusual for the characters to have different dimensions. In fact, fonts added from multiple systems, or multiple images, will often create a rather chaotic group of character sizes, which can make altering some String Reader settings difficult, such as reading within a specific scale range.

You will therefore find it useful to normalize the characters in your font, thereby making them all the same size; this is accomplished by uniformly scaling the specified characters to either a given reference size-X or size-Y. Regardless of the reference size given (size-X or size-Y), the other is always adjusted to maintain the same aspect ratio.

To normalize your characters, you must first decide on the appropriate dimension. This is typically done by obtaining the size of an existing character that you consider appropriate, using MstrInquire() with either M_CHAR_SIZE_X or M_CHAR_SIZE_Y. You can then use either the size-X or size-Y information to normalize the characters in the font, using MstrEditFont() with M_CHAR_NORMALIZE, and setting the OperationMode parameter to either M_SIZE_X or M_SIZE_Y. The Param1 parameter must be set to either the reference size-X or size-Y dimension (in pixels), and the Param2 parameter must be set to a null-terminated string of all the characters to normalize. To normalize all the characters, set Param2 to M_NULL. Param3 must be set to M_NULL.

The reference size-X or size-Y specified with Param2 must be greater than 8 pixels. The string array you provide containing the characters to normalize must be of the right type for the encoding scheme selected. For more information, see the Encoding subsection in the Global context settings section later in this chapter.

Baseline

The baseline is the imaginary horizontal line on which the majority of the characters in the string rest. Specifically, the baseline of a string is the median baseline of each single character in the string:

The baseline is always measured as a value relative to each character; that is, the position of the baseline in a character is defined as a percentage of the height of the character. The bottom of each character is considered to be 0, the top of each character is considered to be 100, and the middle of each character is considered to be 50. Since, by definition, the baseline is located at the bottom of the majority of the characters, the baseline of your characters will typically be 0. All digits, for example, have a baseline of 0. Most uppercase characters, except possibly Q, have a baseline of 0; similarly, the majority of lowercase characters, except a handful (like p, q, and y) also have a baseline of 0.

The bottom of some characters, like p, q, and y, fall below the median baseline. In order for these characters to line up properly with the other characters in the string, the correct baseline (other than 0) must be computed. Notice what the baseline of p, q, and y would be if their baseline was 0:

A suitable baseline for such characters can be determined by calculating an appropriate distance between the bottom of the character and where the median baseline value of the string intersects that character. As you can see, the correct baseline for the following 'p' would be 26.7:

When you define a system font, the baseline information has already been calculated by the font provider (such as Arial or Times New Roman), and stored within the font's file. By default, this information is used to compute the baseline; therefore, all characters defined with a system font are aligned correctly.

When you define a user-defined font, there is no inherent baseline. This is understandable since the font is being taken from an image. By default, String Reader automatically computes an appropriate baseline; therefore, in the vast majority of cases, all characters defined with a user-defined font are aligned correctly.

To have the baseline computed automatically, use MstrEditFont() with M_CHAR_BASELINE, and set the OperationMode to M_DEFAULT, and the Param1 parameter to M_AUTO_COMPUTE. The Param2 parameter must be set to a null-terminated string of all the characters for which you want to automatically compute the baseline, and the Param3 parameter is reserved for future expansion and must be set to M_NULL.

Although it is typically not necessary, you can choose to change the baseline for both system and user-defined fonts. To do so, set the OperationMode parameter to M_DEFAULT, and the Param1 parameter to the baseline value, as a percentage of the character's height. Any value between -1000 and 1000 is valid. The Param2 parameter must be set to a null-terminated string of all the characters for which you want to manually set the baseline, and the Param3 parameter is reserved for future expansion and must be set to M_NULL. Note that the string array that contains the characters you want to change the baseline for (Param2) must be of the right type for the encoding scheme selected. For more information, see the Encoding subsection in the Global context settings section later in this chapter.

You can also decide to give a character no baseline, by setting the Param1 parameter to M_NONE. When you set a character's baseline to M_NONE, String Reader ignores the character's baseline value when reading strings. This is a useful setting because, when characters are being read in the target image, their baseline is one of the conditions that determines whether they will be part of the string. For characters set to M_NONE, this, and only this, condition is ignored. The character must still respect the minimum definition of a string, which means that it must overlap, in the Y-direction, with all other characters in the string, and it must also respect all other possible restrictions that have been set for the string, such as, for example, the character's scale range.

As mentioned, characters that are symbols, such as the dash and the apostrophe, can also be added to the font. In terms of baseline, these symbols are treated like any other character. However, unlike other alphanumeric-type characters, some symbols rest above the median baseline. For such user-defined symbols, the baseline is typically computed to 0. They are therefore usually considered part of the string, provided they meet all of the other string requirements.

However, the baseline of a symbol that has been added from a system font, and that rests above the baseline, is typically saved in the font's file. Unlike letters that fall below the baseline, this distance is measured not from the bottom up, but from the top down (as a percentage of the height of that symbol), and is therefore calculated as a negative value. For example, the baseline of a hyphen ('-') added with the system-font Arial is -250. In other words, the distance between where the hyphen rests and the baseline is -250% the height of the hyphen.

Note that when reading strings, the baseline of the characters that are ultimately considered part of the string must all be aligned within a specified tolerance. The tolerance that is set is relative to each baseline. For more information on the baseline's tolerance, see the Character's maximum baseline deviation subsection in the Degrees of freedom section later in this chapter.

Sorting characters

Characters are not always added to the font in an orderly fashion; particularly user-defined characters. It can therefore be difficult, when displaying these characters, to quickly tell which ones you are missing. Also, you might want to have your characters ordered by sub-lists, such as all the digits, followed by all the letters, followed by all the symbols. To solve these problems, and any others regarding the order of your characters, String Reader provides a sort functionality.

To sort the characters in the font, according to their character values, in either ascending or descending order, use MstrEditFont() with M_CHAR_SORT, and set the OperationMode parameter to either M_ASCENDING or M_DESCENDING, respectively. The Param1 parameter must be set to M_DEFAULT and the Param3 parameter must be set to M_NULL. The Param2 parameter must be set to a null-terminated string of all the characters you want to sort. This string array must be of the right type for the encoding scheme selected. For more information, see the Encoding subsection in the Global context settings section later in this chapter.

FONT ADD USER DEFINED DEFAULT FONT INDEX() FONT DELETE FONT INDEX() CHAR ADD FONT INDEX() FONT INDEX() CHAR DELETE FONT INDEX() FONT INDEX() CHAR FONT CHAR INDEX CHAR TYPE CHAR TYPE CHAR ADD SYSTEM FONT CHAR ADD USER DEFINED USER DEFINED SINGLE USER DEFINED FOREGROUND BLACK FOREGROUND WHITE FOREGROUND BLACK FOREGROUND WHITE FOREGROUND WHITE FOREGROUND WHITE CHAR SIZE X CHAR SIZE Y CHAR NORMALIZE CHAR BASELINE DEFAULT DEFAULT CHAR SORT


Adding and deleting string models to the context

User Guide:
Source image foreground

Reference:
MstrControl
MstrGetResult
Not available in MIL-Lite

Available in MIL

Once you have added a font to the font list in the String Reader context, you must add at least one string model to the string model list in that context. By default, the string model list is empty.

Each string model in the string model list represents a template, a way of locating a string in the target image. This template (string model) has been configured with default settings. Typically, these settings are sufficient for many applications. It is recommended that you try the defaults before changing them. The following sections in this chapter describe, in detail, how this can be accomplished. For example, you can set the minimum and maximum string size, the maximum number of strings you want to locate (for an individual string model and the String Reader context), the acceptance level for the string, and so on.

Note that a string in the target image will only be read if it satisfies every string model criteria, and if it respects the minimum definition of a string, which means that each character in the string must overlap, in the Y-direction, with all other characters in the string.

Adding and deleting the string models

To add a string model to the string model list in the String Reader context, use MstrControl() with M_STRING_ADD set to either M_USER_DEFINED or M_DEFAULT. The maximum number of string models that you can add to a String Reader context is 255. The read operation time increases with the number of string models in the context. Note that adding a string model is not a string model setting; string models are added on the context level.

When a string model has been added to the context, it is assigned an index. String models are indexed as positive, consecutive integers starting at zero, and increasing by one.

Indices can be used to access a string model. For example, to control a string model setting, you must use MstrControl(), with M_STRING_INDEX() set to the index of a specific string model.

Indices can also be used to delete a string model. To do so, use MstrControl() with M_STRING_DELETE. Note that when a string model is deleted, its index is reassigned; that is, all string model indices are immediately renumbered.

To return the index of the string model read, use MstrGetResult() with M_STRING_MODEL_INDEX.

Target image foreground

By default, strings are read with a black foreground. That is, the foreground of the string in the target image is assumed to be black. If it's white, or if it can be either white or black, you must specify this information in the string model. To change the foreground strings are read with, use MstrControl() with M_FOREGROUND_VALUE set to M_FOREGROUND_BLACK, M_FOREGROUND_WHITE, or M_FOREGROUND_BLACK_OR_WHITE.

This functionality allows you to have varying foreground values in your string models. For example: StringModel1 = M_FOREGROUND_BLACK; StringModel2 = M_FOREGROUND_WHITE; StringModel3 = M_FOREGROUND_BLACK_OR_WHITE.

Note that this foreground value differs from the foreground value set when adding a user-defined font from a source image. For more information, see the Source image foreground subsection in the Creating and customizing the fonts section earlier in this chapter.

STRING ADD USER DEFINED DEFAULT STRING INDEX() STRING DELETE STRING MODEL INDEX FOREGROUND VALUE FOREGROUND BLACK FOREGROUND WHITE FOREGROUND BLACK OR WHITE FOREGROUND BLACK FOREGROUND WHITE FOREGROUND BLACK OR WHITE


Space

User Guide:
Adding multiple strings
Formatted results

Reference:
MstrControl
MstrInquire
MstrRead
Not available in MIL-Lite

Available in MIL

When using the MIL String Reader module, the space is not considered a regular character, and you cannot set it as a constraint. For example, if you want to read between a maximum of 6 characters and a minimum of 6 characters, the space will not be included.

In String Reader, the space is used to distinguish individual strings in the target image when performing a read operation (MstrRead()). That is, you can set the maximum distance between two adjacent characters before each character is considered part of two separate strings. This is done by setting the space size, and how many consecutive spaces are allowed in between characters of the same string.

If the space were treated like a regular character, you would have to concern yourself with the space score, distinguishing it from noise, deciding how much space quantifies an actual space, and so on. On the other hand, by defining a space as simply the marker that separates strings, there's less chance for confusion. For example, in the following image, there can be little doubt that there are two groups of strings, 'QKN' and '918':

This is possible, in part, because String Reader treats the Canadian flag in between the strings as noise; it is not a character, and it is therefore ignored. However, if the space was treated like a character, then configuring an application to read these two strings in this licence plate would be tricky. Since most license plates do not have any symbol between the two groups of strings, and the ones that do can't be ignored, multiple types of spaces might have to be considered. That is, you would not only have to account for the space that occurs between multiple strings, but how this space might vary when noise exists. Essentially, it is much simpler to decide whether or not the characters exist, rather than deciding whether or not a space exists.

Note that the space is also used when adding multiple strings to one font simultaneously, and when displaying formatted results. For more information, see the Adding multiple strings subsection in the Creating and customizing the fonts section earlier in this chapter and the Formatted results subsection in the Retrieving results and annotation section later in this chapter.

Like the space, the carriage return also signifies a new string. If a character is on another line, it is automatically part of another string.

Space size

For each font in the font list, you can set the maximum distance between two adjacent characters, in the target image, before each character is considered part of two separate strings. To do so, you must set the width of the space character, and how many consecutive spaces you will allow. For example, if the width of the space character is one pixel, and the maximum number of consecutive spaces allowed is three, then any two adjacent characters, in the target image, that is greater than 3 pixels apart, is considered to be part of two separate strings.

Space width

To set the width of the space character in the font, use MstrControl() with M_SPACE_WIDTH. By default, M_SPACE_WIDTH is set to M_MEAN_CHAR_WIDTH, which is equal to the average size-X (width) of the characters in the font. You can also set M_SPACE_WIDTH to:

  • M_INFINITE, which is equal to an infinite amount of space. That is, all characters on the same line are always part of the same string; also, there will never be a space character in the read string since the distance can never be large enough to constitute a space character.
  • M_MAX_CHAR_WIDTH, which is equal to the maximum size-X character in the font. That is, the space would be set to the biggest possible character.
  • M_MIN_CHAR_WIDTH, which is equal to the minimum size-X character in the font. That is, the space would be set to the smallest possible character.
  • M_QUARTER_MAX_CHAR_WIDTH, which is equal to a quarter of the maximum character width (M_MAX_CHAR_WIDTH) in the font. That is, the space would be set to the typical space size of a standard font (one quarter of em square).
  • Any value > = 1, which allows you to specify, in pixels, the exact width of the space character for the font.

The space value that you set is relative to the font. Therefore, the space value is considered to be at the size in the font, and at the scale and aspect ratio of the string.

To inquire the M_SPACE_WIDTH that you set, use MstrInquire() with M_SPACE_WIDTH. To inquire the actual width of the space width (in pixels), use MstrInquire() with M_SPACE_WIDTH_VALUE.

Maximum number of consecutive spaces

To set the maximum number of consecutive spaces allowed before two adjacent characters are considered part of two separate strings, use MstrControl() with M_SPACE_MAX_CONSECUTIVE. You can set any value greater than or equal to 1. The default value is 3 consecutive spaces.

Note that the M_SPACE_MAX_CONSECUTIVE is a string model setting, unlike M_SPACE_WIDTH, which is a font setting.

The notion of consecutive spaces is important, since non-consecutive spaces do not cause a split in the string. For example, Quebec licence plates can have a maximum of one consecutive space. Therefore, you can have: letter, space, letter, space, letter, space, and so on. But you cannot have letter, letter, letter, space, space, space.

As mentioned, if the number of consecutive spaces between two adjacent characters is above the maximum number of consecutive spaces, the characters are part of two separate strings. If the number of consecutive spaces between two adjacent characters is under the maximum number of consecutive spaces, the characters are part one string, with a space between them.

SPACE WIDTH SPACE WIDTH MEAN CHAR WIDTH SPACE WIDTH INFINITE MAX CHAR WIDTH MIN CHAR WIDTH QUARTER MAX CHAR WIDTH MAX CHAR WIDTH SPACE WIDTH SPACE WIDTH SPACE WIDTH VALUE SPACE MAX CONSECUTIVE SPACE MAX CONSECUTIVE SPACE WIDTH


Number of strings to read

Reference:
MstrControl
MstrGetResult
MstrInquire
MstrRead
Not available in MIL-Lite

Available in MIL

Each string model in the String Reader context represents a template, a way of locating and reading a valid string in the target image. You can set the number of strings that can be read, in the target image, using MstrControl() with M_STRING_NUMBER. The string number can be set for both the String Reader context, and each individual string model in the String Reader context. For both, the default value is 1.

To limit the number of strings that can be read for one specific string, you must use M_STRING_NUMBER, and specify the correct string model index with M_STRING_INDEX(). Any value greater than or equal to 0 can be set. To limit the number of strings that can be read for the entire String Reader context, you must use M_STRING_NUMBER, and specify the context as the index with M_CONTEXT. Any value greater than or equal to 1 can be set. Note that when using M_STRING_NUMBER with M_CONTEXT, the string number limit that you set is the maximum number of strings that can be read for all the string models in the context together.

To read all possible strings (for a specific string model or for the entire context), you can set M_STRING_NUMBER to M_ALL. However, this setting can increase the read time; therefore, always set M_STRING_NUMBER to a specific number whenever possible.

To inquire the number of strings you want to read, use MstrInquire() with M_STRING_NUMBER. To return the number of strings that have actually been read (after an MstrRead()), use MstrGetResult() with M_STRING_NUMBER.

Using M_STRING_NUMBER for both individual string models and their context allows you to set some good restrictions on the number of strings that can be read. For example, you are writing an application that can read three name tags. There are therefore three string models, one for each name. If you set M_STRING_NUMBER for the context to 1, and you set M_STRING_NUMBER for each string model to 1, then as soon as one of the name tags is read, both string number settings will be satisfied. However, if you set M_STRING_NUMBER for the context to 3 and you set M_STRING_NUMBER for each string model to 1, then the string number settings will only be satisfied when one of each name tag has been read.

STRING NUMBER STRING NUMBER STRING INDEX() STRING NUMBER CONTEXT STRING NUMBER CONTEXT STRING NUMBER ALL STRING NUMBER STRING NUMBER STRING NUMBER STRING NUMBER STRING NUMBER STRING NUMBER STRING NUMBER STRING NUMBER


Degrees of freedom

User Guide:
Baseline

Reference:
MstrControl
MstrEditFont
MstrGetResult
MstrInquire
MstrPreprocess
MstrRead
Not available in MIL-Lite

Available in MIL

By default, the String Reader module has been configured to conduct a fast and robust read operation, with a reasonable degree of tolerance for potential strings in the target image. However, to read unusually sized and positioned strings, you might have to change the defaults. Specifically, you can modify the angle, scale, aspect ratio, baseline, and skew settings with which the strings are read.

For example, changing the default tolerance between a character's baseline and the string's baseline is typically unnecessary. As mentioned earlier in this chapter, the baseline refers to the imaginary horizontal line on which most of the characters in the string rest. In the vast majority of cases, characters in the same string do not rest on significantly different baselines. However, this is not always the case for odometers, where each number turns independently, and therefore does not necessarily line up with all other numbers. If you are writing an application that reads an odometer, you might have to allow for a greater baseline deviation between characters of the same string. In the following image, you can see that the character '6' will have a baseline that is significantly different than all the other character baselines in that string. If you do not adjust the baseline tolerance, this valid string might be rejected.

Increasing the angle, scale, aspect ratio, and baseline tolerances with which the string is read can increase the time of the read operation (MstrRead()). Tolerance values are always relative to the angle, scale, aspect ratio, and baseline that have been set. For example, the baseline's tolerance depends on the size of the character; therefore, if a character has been scaled, the tolerance is calculated at that scale.

String angle and character angle

The angle of the string is the angle of the best-fit line that falls beneath all of the characters in that string. For example, the angle of the following string is -45°:

By default, strings are located in the target image at a nominal angle of 0.0°, and with an angular tolerance of +/- 10.0°. This means that, only those strings in the target image that have an angle between +10.0° and -10.0° can actually be read.

To increase the angular tolerance, use MstrControl() with M_STRING_ANGLE_DELTA_POS and M_STRING_ANGLE_DELTA_NEG. Valid values are between 0.0° and 10.0°. These tolerance values are relative to the nominal angle of the string, which you can set using MstrControl() with M_STRING_ANGLE. The only valid value is 0.0°.

Essentially, M_STRING_ANGLE_DELTA_NEG and M_STRING_ANGLE_DELTA_POS set the possible lower (counter-clockwise) and upper (clockwise) limits of the string's angular range (relative to M_STRING_ANGLE). The lower limit can be described as M_STRING_ANGLE - M_STRING_ANGLE_DELTA_NEG, while the upper limit can be described as M_STRING_ANGLE + M_STRING_ANGLE_DELTA_POS. Strings with angles outside this angular range will not be returned as results.

For example, if you know that a string in the target image will typically be located at 0.0°, but you want to allow for an angular tolerance of +/- 10°, you would set M_STRING_ANGLE_DELTA_NEG and M_STRING_ANGLE_DELTA_POS to 10°. This results in a 20° tolerance your string's angle must have for it to be read (10° in the positive direction and 10° in the negative direction).

Each character within the string also has an angle (the same for every character). Typically, this angle value is equivalent to the string's angle value; however, this is not mandatory. The following example illustrates the difference between a string angle and a character angle; here, the angle of the characters in the string is 30°, while the angle of the string itself is 135°.

To retrieve the string angle, use MstrGetResult() with M_STRING_ANGLE. To retrieve the character angle of the string, use MstrGetResult() with M_CHAR_ANGLE. To inquire your angle values, use MstrInquire() with the specified setting.

After setting M_STRING_ANGLE..., you must use MstrPreprocess() for the values to take effect.

Note that the String Reader algorithm is naturally robust to variations in a character's angle. However, if you expect your characters to be located between a wide angular range, it is recommended to perform calculations specific to angular-range search strategies. To do so, use MstrControl() with M_SEARCH_CHAR_ANGLE. Enabling this setting might increase the read operation's processing time.

String scale and character scale

The scale of the string is the median scale, in the X-direction, of each individual character in that string. By default, strings are located in the target image at a nominal scale of 1.0, with a maximum permitted scale of 2.0 and a minimum permitted scale of 0.5. This means that, only those strings in the target image that have a scale between 0.5 and 2.0 can actually be read.

To increase the maximum and minimum permitted string scales, use MstrControl() with M_STRING_SCALE_MAX_FACTOR and M_STRING_SCALE_MIN_FACTOR. Valid values for the maximum scale are between 1.0 and 2.0. Valid values for the minimum scale are between 0.5 and 1.0. These scale values are relative to the nominal scale of the string, which you can set using MstrControl() with M_STRING_SCALE. Valid values are between 0.25 to 4.0.

Essentially, M_STRING_SCALE_MAX_FACTOR and M_STRING_SCALE_MIN_FACTOR set the possible upper (maximum) and lower (minimum) limits of the string's permitted scale (relative to M_STRING_SCALE). The upper limit can be described as M_STRING_SCALE x M_STRING_SCALE_MAX_FACTOR, while the lower limit can be described as M_STRING_SCALE x M_STRING_SCALE_MIN_FACTOR. Strings with scales outside this range will not be returned as results.

By default, the scale of each individual character within the string can vary from the scale of the string by a maximum factor of 1.1, and a minimum factor of 0.9. This means that, only those characters within the string that have a scale between 1.1 and 0.9 can actually be considered part of the string, thereby allowing the string to be read.

To increase the maximum and minimum permitted character scales, use MstrControl() with M_CHAR_SCALE_MAX_FACTOR and M_CHAR_SCALE_MIN_FACTOR. Valid values for the maximum scale are between 1.0 and 2.0. Valid values for the minimum scale are between 0.5 and 1.0. Note that unlike the string scale, you do not set a nominal value for the character scale. The nominal value is the scale of the string in the target image, as calculated by String Reader.

Essentially, M_CHAR_SCALE_MAX_FACTOR and M_CHAR_SCALE_MIN_FACTOR set the possible upper (maximum) and lower (minimum) limits of the character's permitted scale (relative to the calculated string scale). The upper limit can be described as CalculatedStringScale x M_CHAR_SCALE_MAX_FACTOR, while the lower limit can be described as CalculatedStringScale x M_CHAR_SCALE_MIN_FACTOR. Characters with scales outside this range will cause the corresponding string to be rejected. That is, M_CHAR_SCALE_MAX_FACTOR and M_CHAR_SCALE_MIN_FACTOR define how much an individual character's scale can deviate from the scale of the string.

Having control over both the string scale and the character scale can prove an effective way of accepting or rejecting strings for the read operation. For example, you want to write an application that reads the string 'ABCD' within a scale range of 0.5 to 2.0. However, you want to ensure that the scale of each individual character in the string only ranges between 0.9 and 1.1. To do so, you must set M_STRING_SCALE to 1.0, M_STRING_SCALE_MAX_FACTOR and M_STRING_SCALE_MIN_FACTOR to 2.0 and 0.5, and M_CHAR_SCALE_MAX_FACTOR and M_CHAR_SCALE_MIN_FACTOR to 1.1 and 0.9.

To retrieve the string scale, use MstrGetResult() with M_STRING_SCALE. To retrieve the character scale, use MstrGetResult() with M_CHAR_SCALE. To inquire your string and character scale values, use MstrInquire() with the specified setting.

After setting M_STRING_SCALE... or M_CHAR_SCALE..., you must use MstrPreprocess() for the values to take effect.

String aspect ratio and character aspect ratio

The aspect ratio of a string is the median aspect ratio of each individual character in that string. The aspect ratio of a character is the ratio of its scale in the X-direction by its scale in the Y-direction. By default, strings are located in the target image at a nominal aspect ratio of 1.0, with a maximum permitted aspect ratio of 1.25 and a minimum permitted aspect ratio of 0.8. This means that, only those strings in the target image that have an aspect ratio between 1.25 and 0.8 can actually be read.

To increase the maximum and minimum permitted aspect ratios, use MstrControl() with M_STRING_ASPECT_RATIO_MAX_FACTOR and M_STRING_ASPECT_RATIO_MIN_FACTOR. Valid values for the maximum aspect ratio are between 1.0 and 2.0. Valid values for the minimum aspect ratio are between 0.5 and 1.0. These aspect ratio values are relative to the nominal aspect ratio of the string, which you can set using MstrControl() with M_STRING_ASPECT_RATIO. Valid values are between 0.5 to 2.0.

Essentially, M_STRING_ASPECT_RATIO_MAX_FACTOR and M_STRING_ASPECT_RATIO_MIN_FACTOR set the possible upper (maximum) and lower (minimum) limits of the string's permitted aspect ratio (relative to M_STRING_ASPECT_RATIO). The upper limit can be described as M_STRING_ASPECT_RATIO x M_STRING_ASPECT_RATIO_MAX_FACTOR, while the lower limit can be described as M_STRING_ASPECT_RATIO x M_STRING_ASPECT_RATIO_MIN_FACTOR. Strings with an aspect ratio outside this range will not be returned as results.

By default, the aspect ratio of each individual character within the string can vary from the aspect ratio of the string by a maximum factor of 1.1, and a minimum factor of 0.9. This means that, only those characters within the string that have an aspect ratio between 1.1 and 0.9 can actually be considered part of the string, thereby allowing the string to be read.

To increase the maximum and minimum permitted character aspect ratios, use MstrControl() with M_CHAR_ASPECT_RATIO_MAX_FACTOR and M_CHAR_ASPECT_RATIO_MIN_FACTOR. Valid values for the maximum aspect ratio are between 1.0 and 2.0. Valid values for the minimum aspect ratio are between 0.5 and 1.0. Note that unlike the string aspect ratio, you do not set a nominal value for the character aspect ratio. The nominal value is the aspect ratio of the string in the target image, as calculated by String Reader.

Essentially, M_CHAR_ASPECT_RATIO_MAX_FACTOR and M_CHAR_ASPECT_RATIO_MIN_FACTOR set the possible upper (maximum) and lower (minimum) limits of the character's permitted aspect ratio (relative to the calculated string aspect ratio). The upper limit can be described as CalculatedStringAspectRatio x M_CHAR_ASPECT_RATIO_MAX_FACTOR, while the lower limit can be described as CalculatedStringAspectRatio x M_CHAR_ASPECT_RATIO_MIN_FACTOR. Characters with an aspect ratio outside this range will cause the corresponding string to be rejected. That is, M_CHAR_ASPECT_RATIO_MAX_FACTOR and M_CHAR_ASPECT_RATIO_MIN_FACTOR define how much an individual character's aspect ratio can deviate from the aspect ratio of the string.

Having control over both the string aspect ratio and the character aspect ratio can prove an effective way of accepting or rejecting strings for the read operation. For example, you want to write an application that reads the string 'ABCD' within an aspect ratio range of 0.5 to 2.0. However, you want to ensure that the aspect ratio of each individual character in the string only ranges between 0.9 and 1.1. To do so, you must set M_STRING_ASPECT_RATIO to 1.0, M_STRING_ASPECT_RATIO_MAX_FACTOR and M_STRING_ASPECT_RATIO_MIN_FACTOR to 2.0 and 0.5, and M_CHAR_ASPECT_RATIO_MAX_FACTOR and M_CHAR_ASPECT_RATIO_MIN_FACTOR to 1.1 and 0.9.

To retrieve the string aspect ratio, use MstrGetResult() with M_STRING_ASPECT_RATIO. To retrieve the character aspect ratio, use MstrGetResult() with M_CHAR_ASPECT_RATIO. To inquire your string and character aspect ratio values, use MstrInquire() with the specified setting.

After setting M_STRING_ASPECT_RATIO... or M_CHAR_ASPECT_RATIO..., you must use MstrPreprocess() for the values to take effect.

Character's maximum baseline deviation

The baseline is the imaginary horizontal line on which the majority of the characters in that string rest. Specifically, the baseline of a string is the median baseline of each single character in the string. To compute the baseline automatically, or to set a specific baseline value, use MstrEditFont() with M_CHAR_BASELINE. For more information on baseline, and how to set it, refer to the Baseline subsection in the Creating and customizing the fonts section earlier in this chapter.

A character's baseline, provided it has not been set to M_NONE (MstrEditFont() with M_CHAR_BASELINE), is one of the conditions that determine whether it will be part of the string. That is, the baseline of each character in a string must be aligned with every other character in that string, with a certain amount of deviation, or the character will not be considered part of the string, and subsequently, the string might not be considered by the read operation.

The baseline deviation is calculated as the distance between the character's baseline and the string's baseline. As explained earlier in this chapter, this distance is calculated as a percentage of the character's height (size-Y), where the bottom of the character is 0, the top of the character is 100, and the middle of the character is 50. For example, in the following image, the baseline for the characters 'mno' falls at 0, while the baseline for the character 'p' falls at positive 26.7% of 'p'.

If the string baseline were set to 0, and the baseline deviation were set to 20%, the read operation would only consider strings that contain characters that deviate from the baseline by a maximum of 20% their height. In this case, the string 'mnop' would be rejected:

To set the maximum deviation that a character's baseline can have from a string's baseline, use MstrControl() with M_CHAR_MAX_BASELINE_DEVIATION. Valid values are between 0 and 100. The default value is 10.

To inquire the maximum baseline deviation value that you have set, use MstrInquire() with M_CHAR_MAX_BASELINE_DEVIATION.

After setting M_CHAR_MAX_BASELINE_DEVIATION, you must use MstrPreprocess() for the new value to take effect.

Skew angle

The term skew can be defined as a rotational deviation from the correct horizontal and vertical orientation. Typically, when dealing with characters, a skew presents itself as a lateral or horizontal lean; that is, a character is skewed when it is not perpendicular to the baseline of its string.

The String Reader algorithm is naturally robust to variations in a character's skew angle. However, if you expect your characters to be located between a wide angular skew range, it is recommended to perform calculations specific to angular-range skew search strategies. To do so, use MstrControl() with M_SEARCH_SKEW_ANGLE. Enabling this setting might increase the read operation's processing time.

The skew angle for each character must be the same, however it is not necessarily equivalent to the skew angle of the string. To retrieve the character's skew angle, use MstrGetResult() with M_SKEW_ANGLE. To inquire the state of the skew angle control, use MstrInquire() with M_SEARCH_SKEW_ANGLE.

STRING ANGLE DELTA POS STRING ANGLE DELTA NEG STRING ANGLE STRING ANGLE DELTA NEG STRING ANGLE DELTA POS STRING ANGLE STRING ANGLE STRING ANGLE DELTA NEG STRING ANGLE STRING ANGLE DELTA POS STRING ANGLE DELTA NEG STRING ANGLE DELTA POS STRING ANGLE CHAR ANGLE SEARCH CHAR ANGLE STRING SCALE MAX FACTOR STRING SCALE MIN FACTOR STRING SCALE STRING SCALE MAX FACTOR STRING SCALE MIN FACTOR STRING SCALE STRING SCALE STRING SCALE MAX FACTOR STRING SCALE STRING SCALE MIN FACTOR CHAR SCALE MAX FACTOR CHAR SCALE MIN FACTOR CHAR SCALE MAX FACTOR CHAR SCALE MIN FACTOR CHAR SCALE MAX FACTOR CHAR SCALE MIN FACTOR CHAR SCALE MAX FACTOR CHAR SCALE MIN FACTOR STRING SCALE STRING SCALE MAX FACTOR STRING SCALE MIN FACTOR CHAR SCALE MAX FACTOR CHAR SCALE MIN FACTOR STRING SCALE CHAR SCALE STRING ASPECT RATIO MAX FACTOR STRING ASPECT RATIO MIN FACTOR STRING ASPECT RATIO STRING ASPECT RATIO MAX FACTOR STRING ASPECT RATIO MIN FACTOR STRING ASPECT RATIO STRING ASPECT RATIO STRING ASPECT RATIO MAX FACTOR STRING ASPECT RATIO STRING ASPECT RATIO MIN FACTOR CHAR ASPECT RATIO MAX FACTOR CHAR ASPECT RATIO MIN FACTOR CHAR ASPECT RATIO MAX FACTOR CHAR ASPECT RATIO MIN FACTOR CHAR ASPECT RATIO MAX FACTOR CHAR ASPECT RATIO MIN FACTOR CHAR ASPECT RATIO MAX FACTOR CHAR ASPECT RATIO MIN FACTOR STRING ASPECT RATIO STRING ASPECT RATIO MAX FACTOR STRING ASPECT RATIO MIN FACTOR CHAR ASPECT RATIO MAX FACTOR CHAR ASPECT RATIO MIN FACTOR STRING ASPECT RATIO CHAR ASPECT RATIO CHAR BASELINE CHAR BASELINE CHAR MAX BASELINE DEVIATION CHAR MAX BASELINE DEVIATION CHAR MAX BASELINE DEVIATION SEARCH SKEW ANGLE SKEW ANGLE SEARCH SKEW ANGLE


Rules for character placement

User Guide:
Space
String size

Reference:
MstrInquire
MstrRead
MstrSetConstraint
Not available in MIL-Lite

Available in MIL

The read operation (MstrRead()) compares each character in the target image to each character in the font and locates the best one that satisfies all restrictions. When writing an application, you will typically want to ensure that certain characters (or types of characters) appear at specific positions in the string, or that the string itself is of a certain maximum and/or minimum length. By setting up simple character-based constraints, which can be referred to as grammar rules, you can restrict the character comparison process, thereby speeding up the read operation, and increasing its robustness.

Every grammar rule that you set for a string model is referred to as the string model's grammar while every grammar rule that you set for every string model in the String Reader context is referred to as the context's grammar. For example, if you want to write an application that reads telephone numbers with the area code 514, the grammar rules for both the string model and the context would state:

  • Every character in the string must be a digit, except for characters at position 3 and 7, which must be a hyphen ('-').
  • The first three characters of every string must be '514'.
  • The string must have a maximum and minimum length of 12 characters.

However, if you wanted to read two types of telephone numbers, one that starts with 514, and another that starts with 450, you would require two string models; one for each area code. The grammar rules for both string models would be exactly the same, except for the constraint placed on the first three characters. The grammar rules for the first string model would state that the first three characters must be 514, while the grammar rules for the second string model would state that the first three characters must be 450. However, the grammar rules for the String Reader context would include both area code constraints, as well as all other constraints.

Note that you do not set the grammar rules for the context. The context's grammar is simply assumed to be all the grammar rules that you have set for all the string models in that context.

The space character cannot be set as a constraint and is not counted in the string size. For more information, see the Space section earlier in this chapter.

Character constraints

To ensure that only certain characters appear at specific positions in the string, you can apply a character constraint rule using:

  • A type of character.
  • A character.
  • A character list.

To set these constraints, you must use MstrSetConstraint() with one of the following:

  • A digit (M_DIGIT): '0' to '9'.
  • A letter (M_LETTER): 'a' to 'z' and 'A' to 'Z'.
  • An uppercase letter (M_LETTER + M_UPPERCASE): 'A' to 'Z'.
  • A lowercase letter (M_LETTER + M_LOWERCASE): 'a' to 'z'.
  • A character from a specific list of characters (CharList parameter), including special characters and punctuations (for example, 'a', '1', 'aA1', '&', '-', '...').

If you set a type of character as a constraint, then only those characters that are of that type can be considered part of the string. For example, if you set the constraint type for the character at position 1 to M_DIGIT, then only if the character at position 1 is between '0' and '9' can it be considered part of the string. If needed, you can add the digit, letter, and case constraint types together. To accept any character, use M_ANY as the constraint type.

If you are providing a specific character, or a specific list of characters, only those characters will be considered part of the string. For example, setting 'A' as the character constraint will cause only the letter 'A' to be considered part of the string. When providing a specific list of characters, the constraint type settings serve as a double check. For example, if the specific character is '1', and the constraint type is M_LETTER, an error will be generated. Also, you cannot repeat letters in your character list; that is, each letter must be unique. For example, 'AABB' is not allowed. The string array that contains the character list must be of the right type for the encoding scheme selected (MstrInquire() with M_ENCODING). At preprocessing time, each character in an explicit character list must exist in at least one of the fonts of the constraint, or an error will be generated.

The constraint can either be applied to a specific string in the context, or to all strings in the context. To apply the constraint to a specific string, set M_STRING_INDEX() to the index value of the string. To apply the constraint to all strings, set M_STRING_INDEX() to M_ALL.

The constraint is automatically taken from all fonts in the context. To restrict the constraint to a specific font, set M_FONT_INDEX() to the index value of the individual font, and add it to the constraint. For example, M_DIGIT + M_FONT_INDEX(4). This allows you to have a great deal of control in a single String Reader context. For example, you are writing an ANPR application that reads license plates that have one type of font face for pre-1990 plates, and another type of font face for post-1990 plates. Your application can then specify an "old plates" string model with Font1 and a "new plates" string model with Font2. Note that the index you provide to the M_FONT_INDEX() macro can be for a font that does not currently exist. However, if the font still does not exist at preprocessing time, an error will be generated.

To specify the character position in the string for which the constraint is set, use the CharPos parameter.

If it is appropriate, you can set a default (M_DEFAULT) constraint for the string. This is useful if the majority of the characters must be of one constraint type. For example, you can have an application that reads 26 characters, all of which are digits, except for the character at position three, which is a letter. Rather than having 26 lines of code with almost identical MstrSetConstraint() calls (one for each character position), you can set your character constraints in two lines of code:

  • Set the default constraint to M_DIGIT for all string models.
  • Set the constraint for the character at position three to M_LETTER.

Note that a constraint can be set for any position, even a position greater than the maximum number of characters in the string. Constraints set at a position greater than the minimum number of characters for the string are ignored (when the string read does not contain a character at that position).

String Reader will read the number of characters specified by the string size. For best results, the number of characters to be read in your target image should match your string length. For more information on string length, see the String size subsection in this section.

The constraints are stored with the String Reader context as part of its information set and can be inquired, using MstrInquire().

String size

You can set the maximum and minimum number of characters a string must have for it to be read by using M_STRING_SIZE_MAX and M_STRING_SIZE_MIN. For both the minimum and maximum string size, any value greater than or equal to one can be set. For the maximum string size, you can set an infinite number of strings by setting M_STRING_SIZE_MAX to M_INFINITE.

For example, you have an application that reads name tags and it requires the following constraints:

In this case, if the minimum string size is set to 6, and the maximum string size is set to 12, the following table lists some strings that would be either accepted or rejected for the read operation:

String | Read state
Mr. White | Accepted
Mr. Orange | Accepted
Mr. Blonde | Accepted
Mr. Pink | Accepted
Joe Cabot | Rejected
Mr. Roger Avary | Rejected
Mr. X | Rejected

Note that the space character cannot be set as a constraint and is not counted in the string size. For more information, see the Space section earlier in this chapter.

Grammar rules

As previously mentioned, grammar rules refer to the combination of character constraints and string size constraints. Each string model can have rules specific to that model, while all the rules for all the string models make up the grammar rules of the context.

Character constraints and string size constraints are typically used together in concert. For example, if you want to define grammar rules that will ensure that the string 'WIN' will be read, you would specify the minimum and maximum number of characters as 3, and position 0, 1, 2 as 'W', 'I', 'N'. You can then expand these rules to read either the string 'WIN' or 'WINS' by stating that the minimum number of characters is 3, the maximum number of characters is 4, and that position 0, 1, 2, 3, must be filled by 'W', "I", 'N', 'S'.

The ability to write an application with grammar rules, combined with the ability to have multiple string models, gives String Reader a lot of versatility. For example, Quebec license plates have two types of strings: three letters followed by three digits, or three digits followed by three letters.

If you want to write an application that reads these license plates, you would need two string models. Each would have a maximum and minimum size of 6. One model would read letter, letter, letter, digit, digit, digit, and another model would read digit, digit, digit, letter, letter, letter.

Such an application, developed with the MIL OCR module, would be more complex to write, and misleading. Since the MIL OCR module does not allow for grammar rules, the state of all characters cannot be inferred by the state of some. For example, you know that for Quebec license plates, strings that start with a digit must have three digits followed by three letters. Similarly, strings that start with a letter must have three letters followed by three digits. While String Reader can make this assumption, OCR cannot. Therefore, with OCR, you would have to specify, for each character position, digit or letter, digit or letter, digit or letter, digit or letter, digit or letter, digit or letter. Although this would read the license plates, it might also read some strings that are not license plates. That is, if the character at position 0 is a letter, it is simply not true that the character at position 1 can be either a letter or a digit; it must be a letter. Having the ability to instill grammar rules helps you avoid such inaccuracies.

DIGIT LETTER LETTER UPPERCASE LETTER LOWERCASE DIGIT ANY LETTER ENCODING STRING INDEX() STRING INDEX() FONT INDEX() DIGIT FONT INDEX(4) FONT INDEX() DEFAULT DIGIT LETTER STRING SIZE MAX STRING SIZE MIN STRING SIZE MAX INFINITE


Global context settings

User Guide:
Formatted results
Number of strings to read

Reference:
MstrControl
MstrEditFont
MstrGetResult
MstrInquire
MstrRead
Not available in MIL-Lite

Available in MIL

The global settings of a String Reader context determine the strategy used to read all of the context's string models, which in turn, directly affects the speed and robustness of the read operation (MstrRead()). To adjust the global context settings to fit your individual application's needs, use MstrControl() with the Index parameter set to M_CONTEXT.

There are two categories of global context settings: general and advanced. The default value for each setting in either category is typically good. However, if you want to find the optimum balance between speed and robustness for a particular application, it is recommended that you experiment with different global context setting values.

General global context settings

The general global context settings include:

  • Minimum contrast.
  • Speed.
  • Timeout.

Note that the general global context settings also include string number. Unlike other global context settings, string number can be applied to both the String Reader context, and to each string model within the context. For more information, see the Number of strings to read section in Chapter 11: String Reader.

Minimum contrast

String Reader is contrast invariant. Therefore, if a valid character exists in the target image, it will be read, regardless of whether the contrast between that character and its background is high or low.

However, the speed with which the character is read is typically affected by the contrast, especially in the presence of noise, where every little variation in pixel intensity must be evaluated. If you know for certain that there is a definite minimum contrast between the characters and their background, you can potentially increase the speed of the read operation by setting a high minimum contrast value. Any pixel value that does not have a contrast greater than the minimum value set will be ignored by String Reader. Therefore, the higher the minimum contrast, the quicker the read will be.

To set the minimum contrast, use MstrControl() with M_MINIMUM_CONTRAST. Valid values are between 1 and 255. The default value is 15. To inquire the minimum contrast that you set, use MstrInquire() with M_MINIMUM_CONTRAST.

Setting the minimum contrast will not make an unreadable character readable; however, it might make a readable character unreadable. For example, if the contrast between a valid character and its background is 25, and you set the minimum contrast to 50, that character will not be read. It is therefore important to make sure the minimum contrast value is not causing the read operation to ignore acceptable characters.

Speed

You can specify the algorithm's read speed, using MstrControl() with M_SPEED. At this time, the speed must be set to M_MEDIUM, which is the default value.

When you preprocess the context, MIL analyzes the patterns in the target and determines which shortcuts are appropriate for the read speed setting. At higher read speeds, the read can take all reasonable shortcuts; therefore, when possible, the read is performed faster than at lower speeds. You can typically use higher read settings when high accuracy is not an issue; this is because increasing the read might affect the robustness, as well as the accuracy, of the search.

Timeout

In time critical applications, you can set a time limit, in milliseconds, for String Reader to read the specified string models. To do so, use MstrControl() with M_TIMEOUT.

After the time limit has been reached, the read operation will terminate, even if the required number of strings has not been read. Results are returned for all strings read up to the timeout. It is not possible to predict which strings will be read beforehand.

Any value, greater than or equal to 0, can be set. The default value is 2000 msecs. Due to certain application-dependent calculations, the actual maximum read time might vary slightly from the timeout specified. You can specify an infinite amount of read time by disabling the timeout setting (M_DISABLE).

To check whether the timeout limit has been reached, use MstrGetResult() with M_TIMEOUT_END. To inquire the timeout that you set, use MstrInquire() with M_TIMEOUT.

Advanced global context settings

The advanced global context settings include:

  • Encoding.
  • Space character.
  • String separator.

Encoding

You can set the data type of the character encoding used by the String Reader context. To do so, use MstrControl() with M_ENCODING.

The encoding scheme that you select will affect the data type of the information that you pass to String Reader. For example, when adding characters to the font, using MstrEditFont() with M_CHAR_ADD, the data type of the string array that contains the characters you want to add (Param2) is based on the M_ENCODING setting.

At this time, the encoding must be set to M_ASCII, which is the default value. M_ASCII specifies an 8-bit ASCII standard character type; this corresponds to the data type char.

To inquire the encoding that you set, use MstrInquire() with M_ENCODING.

Space character

You can choose to display your results in formatted text (MstrGetResult() with M_FORMATTED_STRING). When you do so, String Reader will insert a space every time the distance between 2 characters exceeds the space width (MstrControl() with M_SPACE_WIDTH), at the scale of the string read. To set the character value of this inserted space, use MstrControl() with M_SPACE_CHARACTER. To inquire the inserted space character that you set, use MstrInquire() with M_SPACE_CHARACTER.

By default, the space character is set to ASCII 32, which can be represented by a space (' '). You can specify any character value as the space character, using its ASCII code. For example, to specify a new line as the space character, set M_SPACE_CHARACTER to 10. You can also specify no space character by setting M_SPACE_CHARACTER to M_NONE.

The space character that you want inserted for formatted results must be set before the text is read. For more information, see the Formatted results subsection in the Retrieving results and annotation section in Chapter 11: String Reader.

String separator

You can choose to display your results in formatted text (MstrGetResult() with M_FORMATTED_STRING). When you do so, String Reader will insert a separator character between strings. To set the character value of this string separator, use MstrControl() with M_STRING_SEPARATOR. To inquire the separator character that you set, use MstrInquire() with M_STRING_SEPARATOR.

By default, the string separator character is set to ASCII 10, which is a new line. You can specify any character value as the string separator, using its ASCII code. For example, to specify a space as the separator character, set M_STRING_SEPARATOR to 32. You can also specify no separator character by setting M_STRING_SEPARATOR to M_NONE.

The separator character that you want inserted for formatted results must be set before the text is read. For more information, see the Formatted results subsection in the Retrieving results and annotation section in Chapter 11: String Reader.

CONTEXT MINIMUM CONTRAST MINIMUM CONTRAST SPEED MEDIUM TIMEOUT DISABLE TIMEOUT END TIMEOUT ENCODING CHAR ADD ENCODING ASCII ASCII ENCODING FORMATTED STRING SPACE WIDTH SPACE CHARACTER SPACE CHARACTER SPACE CHARACTER SPACE CHARACTER NONE FORMATTED STRING STRING SEPARATOR STRING SEPARATOR STRING SEPARATOR STRING SEPARATOR NONE


Scores and acceptances

Reference:
MstrControl
MstrGetResult
MstrRead
Not available in MIL-Lite

Available in MIL

When String Reader performs a read operation, it analyzes the target image and localizes potential character candidates for each string model. Every character candidate, and the string candidate it belongs to, is given various scores between 0 and 100. These scores quantify the similarity between:

  • The character in the target image and the corresponding character in the font.
  • The character in the target image and the other characters of the string in the target.

String Reader allows you to set various acceptance values for character and string candidates. If the candidate's score is equal to or above these acceptance values, the candidate will be accepted; otherwise, it will be rejected. For example, if you set the string acceptance to 80, and the actual string score is 75, then that string will not be accepted by the read operation.

The following acceptance values can be set:

  • String acceptance.
  • Character acceptance.
    • Character similarity acceptance.
    • Character homogeneity acceptance.

The default acceptance values are typically sufficient. However, to fit the specific needs of your application, it is recommended that you try experimenting with the acceptance values. Although each acceptance value represents a threshold for a specific score, when used in concert they provide a great deal of versatility for accepting and rejecting strings as a whole.

String acceptance and score

The string score is the average score of all the characters in the string. The string acceptance value sets the threshold score that the string must meet in order for it to be read. That is, MstrRead() will only read the strings that have a score greater than or equal to the string acceptance value that you set.

To set the string acceptance value, use MstrControl() with M_STRING_ACCEPTANCE. To get the actual score of the string, use MstrGetResult() with M_STRING_SCORE.

The following ANPR example illustrates an image used to define a font, and a target image where the string 'ZEY 877' must be read. In this example, the string has a score of 91.9%. Therefore, if M_STRING_ACCEPTANCE is set at 91.9 or below, the string 'ZEY 877' will be accepted by the read operation.

Character acceptance and score

The character score is calculated as the average value of the character's similarity and homogeneity scores. The character acceptance value sets the threshold score that a character must meet in order for it to be accepted as part of the string. All characters with character scores below the character acceptance value will not be considered part of the string, and therefore the string will not be read.

To set the character acceptance value, use MstrControl() with M_CHAR_ACCEPTANCE. To get the actual score of the character, use MstrGetResult() with M_CHAR_SCORE.

Character similarity

The character's similarity score quantifies the similarity between the character in the target image and the character in the font. To calculate the character's similarity score, many character traits are taken into account, such as variations in contrast, size, aspect ratio, scale, and baseline. If each of these characteristics (and others not mentioned) is the same, between the character's representation in the target image and the font, the resulting similarity score for that character would be 100. Of course, this is seldom the case.

The character's similarity acceptance value sets the threshold similarity score that a character must meet for it to be accepted as part of the string. All characters with similarity scores below the character similarity acceptance value will not be considered part of the string, and therefore the string will not be read.

To set the character's similarity acceptance value, use MstrControl() with M_CHAR_SIMILARITY_ACCEPTANCE. To get the actual similarity score of the character, use MstrGetResult() with M_CHAR_SIMILARITY_SCORE.

Note that if a character does not meet the character similarity acceptance, its score is ignored when calculating the character score (M_CHAR_SCORE).

Character homogeneity

The character's homogeneity score quantifies the similarity between the character in the target image and the other characters of the string in the target image. To calculate the character's homogeneity score, many character traits are taken into account, such as variations in contrast, size, aspect ratio, scale, and baseline. If each of these characteristics (and others not mentioned) is the same, between the character and all other characters of the string in the target image, the resulting homogeneity score for that character would be 100. Of course, this is seldom the case.

The character's homogeneity acceptance value sets the threshold homogeneity score that a character must meet for it to be accepted as part of the string. All characters with homogeneity scores below the character homogeneity acceptance value will not be considered part of the string, and therefore the string will not be read.

To set the character's homogeneity acceptance value, use MstrControl() with M_CHAR_HOMOGENEITY_ACCEPTANCE. To get the actual homogeneity score of the character, use MstrGetResult() with M_CHAR_HOMOGENEITY_SCORE.

Since the character's homogeneity is calculated by how well the character compares to the other characters in the string, rather than how well it compares to the font, it can solve problems that cannot be resolved solely by similarity. For example, in a target image, you might have characters that have high similarity scores and that, individually, are all acceptable. However, if their characteristics cause them to be out of context with the other characters in the string, you might want to reject them. You can use the homogeneity acceptance to reject characters that are otherwise acceptable.

Note that if a character does not meet the character homogeneity acceptance, its score is ignored when calculating the character score (M_CHAR_SCORE).

STRING ACCEPTANCE STRING SCORE STRING ACCEPTANCE CHAR ACCEPTANCE CHAR SCORE CHAR SIMILARITY ACCEPTANCE CHAR SIMILARITY SCORE CHAR SCORE CHAR HOMOGENEITY ACCEPTANCE CHAR HOMOGENEITY SCORE CHAR SCORE


Retrieving results and annotation

User Guide:
Sorting characters
Space

Reference:
MstrControl
MstrDraw
MstrGetResult
MstrInquire
MstrPreprocess
MstrRead
MstrRestore
MstrSave
Not available in MIL-Lite

Available in MIL

After adding at least one string model and one font to the String Reader context, and performing a successful preprocess and read operation, you would typically retrieve and annotate your results. To do so, use MstrGetResult() and MstrDraw().

These functions offer many different types of results and draw operations. For example, you can retrieve the aspect ratio of the string, using M_CHAR_ASPECT_RATIO, or draw a cross at the center of the bounding box of each string's character, using M_DRAW_STRING_CHAR_POSITION.

String Reader also provides you with forward and reverse transformation coefficients. These results allow you to transform any point from the character image to its corresponding position in the target image (forward), or from the read position in the target image to the corresponding position in the character image (reverse). With this information, any point of any character can be drawn, thereby allowing you to have full annotation control, if needed.

This section, as well as other sections in this chapter, discusses drawing capabilities and results. For a full list of what you can retrieve and what you can annotate, refer to MstrGetResult() and MstrDraw().

When retrieving results, you will need to know the type and size of the array in which to store the results. Typically, the default data type is double. For character results (such as M_TEXT, M_FORMATTED_STRING, and M_STRING), the default type of the array corresponds to the encoding scheme that you have selected, using MstrControl() with M_ENCODING. Of course, you can specify any data type by explicitly adding it to the result. The size of the array depends on the result you are getting. Typically, it is either M_STRING_NUMBER, M_STRING_SIZE, or M_CHAR_NUMBER. For more information, see MstrGetResult().

Preprocess and read

After you add string models and fonts to your String Reader context and before reading a string from the target image, the String Reader context must be preprocessed, otherwise an error will occur. It is during this preprocessing stage that String Reader decides what calculation strategies it will employ to read the target image most effectively and efficiently. To preprocess the String Reader context, use MstrPreprocess(). To read a string from the target image, use MstrRead().

Note that you might have to preprocess the String Reader context again if you change some of its control settings, such as the nominal angle of the string (M_STRING_ANGLE). To check if you need to preprocess the String Reader context, use MstrInquire() with M_PREPROCESSED.

When you save a String Reader context, its preprocessing information is not saved. You must therefore preprocess the context when it is restored, even if you haven't changed any of its settings. To save a String Reader context to a file, use MstrSave(). To restore the context, use MstrRestore().

Annotation

The String Reader module allows you to draw specific features of the String Reader context or String Reader results, using MstrDraw(). For example, you can draw your string results to ensure that the read operation has been executed correctly. This will establish that the correct characters have been read. Note that even if the read operation has been executed error-free, it is always recommended to visually verify your string results.

After each drawing operation, the optimal image size that was needed for that operation is updated. This information can be inquired using MstrInquire() with M_DRAW_LAST_SIZE_X and M_DRAW_LAST_SIZE_Y. If you set M_NULL as the destination image in MstrDraw(), these size constants are updated, even though the drawing operation is not performed. This can be useful if you, for example, only want to get the optimal image buffer size for a future drawing operation.

All characters are drawn according to their draw box margin, which you can set with M_DRAW_BOX_MARGIN_X and M_DRAW_BOX_MARGIN_Y in MstrControl(). These margins determine the amount of white space between each character that is drawn. Any value greater than or equal to zero can be set. The default value is 3. Note that a character has no inherent margin; that is, if these margin values are set to 0, the characters will be drawn, one after the other and one on top of the other, with no white space.

For a String Reader context, you can draw a character representation of the font in the destination image. This is useful to ensure that the characters added to the font are correct. To do so, you must specify the explicit list of characters to draw and verify. Characters are drawn from left to right, and from top to bottom. Characters that fall outside the destination image are clipped. To draw the characters on multiple lines, a space character must be inserted to separate the strings included in the null-terminated string.

It might be useful to sort the characters before you draw them, thereby making it easier to visually ascertain if all the characters you want are there. For more information, see the Sorting characters subsection in the Creating and customizing the fonts section earlier in this chapter.

For String Reader results, you can perform the operations listed below. These operations draw various features of the String Reader results and can be added together to draw multiple features simultaneously. Note that characters are always drawn at the location read in the target with the correct angle, scale and aspect ratio.

  • M_DRAW_STRING: draws the characters of the string at the location read in the target.
  • M_DRAW_STRING_BOX: draws a box around the string that was read in the target. The bounding box of the string is the bounding box of all the string's characters bounding box, including the draw margin (set with M_DRAW_BOX_MARGIN_X and M_DRAW_BOX_MARGIN_Y in MstrControl()).
  • M_DRAW_STRING_CHAR_BOX: draws a box around the characters of the string that was read in the target. The character box drawn is the bounding box of the character plus the draw margin (set with M_DRAW_BOX_MARGIN_X and M_DRAW_BOX_MARGIN_Y in MstrControl()).
  • M_DRAW_STRING_CHAR_POSITION: draws a cross at the center of the bounding box of each string's character.
  • M_DRAW_STRING_CONTOUR: draws the contour of all the characters of the result(s) of the string(s) read. A contour is a type of edge that defines the outline of the characters in an image.

The following image shows how drawing the character position of a string (M_DRAW_STRING_CHAR_POSITION) for a typical Quebec license place would be displayed:

Formatted results

Unlike String Reader, humans have a hard time processing a long string of characters uninterrupted by any space. To make characters easier to read, you can display the string with spaces and separators. This is referred to as a formatted result.

As mentioned earlier in this chapter, the space is not considered a character, and is not a part of any string that has been read. String Reader can however still display a space between strings because, although String Reader does not read a space, it does mark its existence. When you ask for a formatted result, a space is then inserted at the correct mark.

A space character is inserted in the formatted string each time the distance between 2 adjacent characters exceeds the space width. Obviously, the default character used as the space character is ASCII 32, which is a space. You can however set a different character, using MstrControl() with M_SPACE_CHARACTER. Any character's ASCII numerical code can be used. You can also set M_SPACE_CHARACTER to M_NONE, which specifies no space character. Note that you can set the space width using MstrControl() with M_SPACE_WIDTH. For more information, see the Space section earlier in this chapter.

When getting formatted results, a separator character is inserted between each string. By default, this character is ASCII 10, which is a new line. You can however control the string separator character, using MstrControl() with M_STRING_SEPARATOR. Any character's ASCII numerical code can be used. You can also set M_STRING_SEPARATOR to M_NONE, which specifies no separator character.

The space character and the string separator must be set before the string is read from the target image.

To get formatted results, use MstrGetResult() with M_FORMATTED_STRING. This result includes all string characters, the inserted space, and the null-termination character. To get un-formatted results, use M_STRING, which doesn't include the space. To get the size of the formatted string or the un-formatted string, use M_FORMATTED_STRING_SIZE and M_STRING_SIZE, respectively. The null-termination character is not counted in the either string size. The inserted space is counted in the formatted string size, and not counted in the un-formatted string size.

CHAR ASPECT RATIO DRAW STRING CHAR POSITION TEXT FORMATTED STRING STRING ENCODING STRING NUMBER STRING SIZE CHAR NUMBER STRING ANGLE PREPROCESSED DRAW LAST SIZE X DRAW LAST SIZE Y DRAW BOX MARGIN X DRAW BOX MARGIN Y DRAW STRING DRAW STRING BOX DRAW BOX MARGIN X DRAW BOX MARGIN Y DRAW STRING CHAR BOX DRAW BOX MARGIN X DRAW BOX MARGIN Y DRAW STRING CHAR POSITION DRAW STRING CONTOUR DRAW STRING CHAR POSITION SPACE CHARACTER SPACE CHARACTER NONE SPACE WIDTH STRING SEPARATOR STRING SEPARATOR NONE FORMATTED STRING STRING FORMATTED STRING SIZE STRING SIZE


An example

Availability | Previous
Not available in MIL-Lite

Available in MIL

The String Reader example MStr.c illustrates how to define a font from an image with a mosaic of sample Quebec licence plates.

Two string models are then defined and its controls set to read two valid types of Quebec licence plates: three letters followed by three numbers, or three numbers followed by three letters. A valid licence plate reading is then performed in a target image containing a car on the road.

The example also displays the read time, and the score of the string that was read.

MStr.c


Codes

Chapter 12: Codes
Availability
Not available in MIL-Lite

Available in MIL

This chapter describes how to read, write, and verify various codes.

  • Overview of codes
  • Using MIL to read and write codes
  • Supported code types and noteworthy characteristics
    • Anatomy of 1D and 2D codes
    • Technical code information
  • Controlling read and write operations in MIL
    • Issues pertaining to both read and write operations
    • Read operations - issues to consider
    • Write operations - issues to consider
  • Retrieving results
  • Verifying your code


Overview of codes

See Also | Availability | Next
User Guide:
Supported code types and noteworthy characteristics
Not available in MIL-Lite

Available in MIL

Many industries label products using symbols from symbologies such as bar codes and Data Matrix codes. This is done for identification purposes during different stages of production and distribution. Each symbology, known as a code type in MIL, follows a different set of rules to encode the data into light and dark patterns; data is encoded into units called cells. There are many different standards for the rules on encoding data. To find out more about the standards, see standards organizations such as the Association for Automatic Identification and Mobility or EAN International.

MIL can both read symbols from and write symbols to images. To read a symbol, MIL searches for a specified code type in an image and decodes it. The decoded string can then be used to identify the object in the image. To write a symbol, MIL encodes a string into a symbol using the specified code type and encoding scheme. The resulting image of the symbol can then be rotated, combined with text on a logo, and then printed. In MIL, symbols are known as codes.

MIL also allows you to verify the quality of the code you want to read, or have written. To verify a code, MIL computes the quality-grade of the code in the specified source image. Verification is often used to check the conditions under which you are reading codes and to check if the codes you are reading or writing meet your quality specifications.

MIL supports 1D, 2D, and composite code types. One-dimensional (1D) code types are linear bar codes, and are used for small amounts of data, for example, the identification number for a grocery store item, or a stock room part number (data is represented by bars and spaces).

Two-dimensional (2D) code types store information along the height, as well as the width, of the code. 2D codes can store more information than 1D codes, and are typically used for shipment information or medical records.

MIL also supports composite codes, which are a combination of specific 1D and 2D code types. The 1D component encodes the item's primary identification, while the 2D component encodes additional data (such as batch number or expiration date, for example).

For examples of different codes, see the Supported code types and noteworthy characteristics section later in this chapter.

Some code types support several methods of encoding, known as encoding schemes. This means a code type might support an encoding scheme of alpha characters only, numeric characters only, or both alpha and numeric characters. In addition, some code types can support any number of characters, while others require a fixed number.

If the image is degraded, codes can still be read if an error correction scheme was used when the code was generated. Error correction is essentially redundant data included in the encoding scheme of some code types. Some error correction schemes are used only for error detection, while others are used for error detection and recovery. For more information, see the Supported code types and noteworthy characteristics section later in this chapter.

The code module only supports 8-bit unsigned one-band buffers. Buffers in other formats produce an error.


Using MIL to read and write codes

User Guide:
Write operations - issues to consider

Reference:
McodeAlloc
McodeControl
McodeFree
McodeGetResult
McodeRead
McodeWrite
Not available in MIL-Lite

Available in MIL

The steps below describe, in general, how to perform read and write code operations using MIL. Note that you must take into consideration any particularities of the chosen code type. More detailed and code type-dependent information is described in subsequent sections in this chapter, as well as the MIL Reference.

To perform a read or write operation:

  1. If reading a code, grab or load an image that contains the code into an allocated image buffer. If the image is large, or if there are multiple codes in the image, create a child buffer to isolate the code from the rest of the image. If your image contains other important information besides the code, you might want to use the pre-search feature using McodeControl(), which will help MIL to locate the code.

    2. If writing a code, allocate the image buffer in which to generate the code. For more information on how to determine the required size of the image buffer, see the Write operations - issues to consider subsection in the Controlling read and write operations in MIL section later in this chapter.

  2. Allocate a code object, using McodeAlloc(). A code object specifies the type of code to read or write and how to perform the operation.
  3. If necessary, change control settings of the code object, using McodeControl().
  4. To perform a read operation, use McodeRead(). To perform a write operation, use McodeWrite().
  5. Retrieve results, using McodeGetResult().
  6. Free the memory allocated to the code object, using McodeFree().


Supported code types and noteworthy characteristics

Reference:
McodeAlloc
McodeControl
McodeRead
MgraRectFill
Not available in MIL-Lite

Available in MIL

Whether you plan on reading or writing codes, you must specify the code type when allocating the code object with McodeAlloc(). Below is a list of the supported code types and their MIL predefined constants:

Following are the 2D code types supported.

The following is an example of a composite code.

Anatomy of 1D and 2D codes

To perform more sophisticated read and write operations using MIL, it will be useful to understand the following terms that are illustrated in the example Code 128 code below.

The following are terms commonly used in code reading and writing operations:

  • Cell. The narrowest dark or white space into which data is encoded. Cells have different shapes, depending on the symbology: cells in 1D code types are bars, some 2D codes have cells that are approximately-square blocks, while Maxicode cells are hexagonal. In 1D codes, thicker bars (and therefore spaces) occupy several contiguous cells.

    • Industry uses slightly different terminology for PDF417 codes. For example, industry uses the term modules to refer to cells. There are 17 modules (cells) in each column of a PDF417 code, and each column has four bars and four spaces. Industry also refers to codewords; a codeword is a column within one row.

  • Cell size (also known as x-width). Width, in pixels, of the cell, the code's narrowest unit.

    • The cell size of a Maxicode is defined as the distance between the center points of two hexagons in the same row.

  • Encoding scheme. The method used to encode a string into a code. Not all code types support all encoding schemes, so you must be aware of the characteristics of your code type. The encoding schemes supported for a given code type are listed under McodeControl() in the MIL Reference.
  • Error correction. Error correction schemes prevent read operations from returning incorrect results. During read operations, their purpose is for error detection and error recovery. Error detection discovers bit errors (when a 1 is read as a 0 or vice versa). Error recovery both detects and corrects bit errors; depending on the error correction scheme used, questionable data can be corrected based on the remaining data.
  • Checksum number. Number included as part of a 1D or 2D code, which is calculated based on the other characters in the code; it is a simple form of error detection. Typically during a read operation, the characters that make up the string will be put into a checksum calculation whose results are compared against the checksum number. If a discrepancy exists between the calculated and stored checksum, the read operation will yield no results. Not all code types use a checksum.
  • Scanline. A scanline is the line where McodeRead() tries to read the code information. Multiple scanlines are needed to read the code; the first line is usually taken across the middle of the image. Scanlines are used with codes where the information can be read in a straight line, that is, codes with 1D components.
  • Start and stop characters. Characters which inform the reader or scanner of the beginning and end of a code, analogous to the start and stop bits which are specified when transferring data across a modem. Some code types, such as Code 39, use identical stop and start characters, while others use different stop and start characters. The PDF417 code type refers to these characters as start and stop patterns.
  • Quiet zone. Quiet zones are areas with no marks and are used to aid the scanning process. For 1D codes, the quiet zone immediately precedes the start character and immediately follows the stop character. The required quiet zone for a 1D bar code is generally the width of the widest bar. For 2D codes, the quiet zone must be present on all four sides of the code. The required quiet zone for a 2D bar code is generally the cell size. The quiet zone must contain no marks; even hand-written scribbles in the quiet zone will impede the read operation significantly.
  • Bearer bars. Bars that run along the tops and bottoms of a code, which are illustrated in the diagram below.

    • If you are reading a code with bearer bars, you must specify the search angle at which to read the code. In addition, MIL does not generate bar codes with bearer bars. If you want to generate a code with bearer bars, generate the required code and then draw a rectangle at the top and bottom of the code using MgraRectFill(); the minimum width of the bearer bar (height of the rectangle) must respect the specifications of your code type.

Technical code information

For technical information about Data Matrix, Maxicode, or PDF417, see AIM International Symbology Specification.

For technical information about UPC-A, UPC-E or bar codes, see The Bar Code Book, Roger C. Palmer, Helmers Publishing, United States, 1995.

For technical information about Pharmacode, see The Pharmacode Guide by Laetus.

For technical information about Planet code, go to the United States Postal Service site, Rapid Information Bulletin Board System (www.ribbs.usps.gov). Look under File Libraries for a file named Confirm®. This will call a file titled Customer Guide to Confirm using PLANET Codes.

For technical information about Postnet, go to the United States Postal Service site, (www.usps.com) and find the POSTNET Barcode Certification page. This should be under Certification Programs, in the Address Management Services section under Address Quality.

For technical information about composite codes, MicroPDF417, or RSS codes, see AIM Global Network Standards


Controlling read and write operations in MIL

Reference:
McodeAlloc
McodeControl
McodeGetResult
McodeWrite
MimFlip
Not available in MIL-Lite

Available in MIL

This section provides information to consider and describes settings that you might have to change for both read and write operations.

Issues pertaining to both read and write operations

You must know the code type when performing a read or write operation. If your read operation is unsuccessful, ensure that you have allocated the code object with McodeAlloc() using the correct code type.

MIL can automatically detect the type of encoding scheme for read operations for most code types, but it must be specified for write operations; use the M_ENCODING control type of the McodeControl() function. The list of supported encoding schemes is found under McodeControl() in the MIL Reference.

Note that you must specify the encoding scheme when reading a Code 39.

In most cases, MIL can automatically detect the error correction scheme for read operations. Similarly, MIL can automatically choose the best error correction scheme for most write operations. If your code type supports more than one error correction scheme and does not support M_ANY, you must specify the error correction scheme. Specify error correction using the M_ERROR_CORRECTION control type of the McodeControl() function. See McodeControl() in the MIL Reference to determine supported error correction schemes for your code type. If you specify a scheme that is not supported by your code type, you will get a MIL error.

During a read or write operation, you must specify the foreground color of the code; use the M_FOREGROUND_VALUE control type of the McodeControl() function.

Note that specifying M_ANY for a PDF417 code, when using M_ENCODING or M_ERROR_CORRECTION, does not exact a performance penalty.

Read operations - issues to consider

It is strongly recommended to use child buffers for read operations, especially if your image contains multiple codes of the same type and you want to read more than one, or if your image contains other information besides the code. MIL's code module is designed to read one code at a time, therefore if you do not use child buffers and your image contains multiple codes, results will be unpredictable.

A quiet zone is part of each code type's specification. During read operations, therefore, your image buffer must be large enough for your code to have a quiet zone. Generally, the required quiet zone for a 1D barcode is the width of the widest bar. For a 2D barcode, the required width is generally the size of the cell.

MIL can, in some cases, successfully read a code that does not meet the minimum requirements for the quiet zone; however, it is strongly recommended that codes have one in order to perform robust operations and return accurate results.

There are some situations where you are not able to create a child buffer for your read operation. Your image could be very complex and contain other information besides the code. In this case, you could use the code module's pre-search algorithm, which helps MIL to locate the code in the image prior to the read operation. The pre-search algorithm is only supported with 2D codes, and is disabled by default. This algorithm can also be used if your image contains several codes of different types. Note that if your image contains more than one code of the same type, MIL will randomly select only one to decode. Set the pre-search option by using the M_USE_PRESEARCH control type of the McodeControl() function.

It is highly recommended that when using this control type, you also specify the best cell size range possible, especially if the cell size is greater than 10 or smaller than 6.

In most cases, you should not have to specify the cell size when reading codes; the default setting (M_DEFAULT) is sufficient. However, for some 2D code types, such as Data Matrix and Maxicode, read operations can be more robust if you specify the cell size. Specify the cell size as a range using the M_CELL_SIZE_MIN and M_CELL_SIZE_MAX control types of the McodeControl() function. MIL will search for codes with cells that fall within this range. If the cell size is not within the specified range, the code will not be found. Note that MIL might have difficulty reading 1D codes if the cell size is less than 2 pixels, and 2D codes if the cell size is less than 3 pixels, even if the size is specified.

During read operations, MIL can automatically detect the number of cells, and will search for the specified 2D code type with any number of cells in the X and Y direction; the X direction representing the number of columns, and the Y direction representing the number of rows. You can specify the number of cells in the X and Y directions to increase the speed and robustness of the operation for 2D code types only. To specify the number of cells, use both the M_CELL_NUMBER_X and M_CELL_NUMBER_Y control types of the McodeControl() function.

Note the following when using these control types and some code types:

  • Specifying the number of cells is mostly useful when reading PDF417 and Data Matrix code types; the Maxicode code type has a fixed number of cells.
  • For the PDF417 code type, M_CELL_NUMBER_X = 17c + 35, where c is the number of columns, and 35 represents the number of cells required for the start and stop patterns.

By default, MIL can read codes if they fall within the angular range of 0 ± 5°. It is strongly recommended that you change the search angle if the code appears rotated in the image, and you should increase the angular range if you are unsure of the code's exact orientation. Note, however, that when searching for 1D codes, the operation speed is slower for angular ranges greater than ± 5°. The angular range is the range of angles defined by the M_SEARCH_ANGLE- M_SEARCH_ANGLE_DELTA_NEG and M_SEARCH_ANGLE + M_SEARCH_ANGLE_DELTA_POS control types of the McodeControl() function.

Note the following when using these control types and some code types:

  • If the code has bearer bars, you must specify the search angle.
  • For the PDF417, Maxicode, and Data Matrix codes, the angular range does not affect the speed of the operation.

In some cases, the image of the Data Matrix code may need to be flipped vertically or horizontally (MimFlip()) to be read correctly. In these cases, the image is essentially a mirror image of the correct code.

You can specify the speed at which to perform a read operation; the faster the speed, the less robust the operation. In general, the larger and more clearly defined the code, the better chance it has of being found at a speed higher than the default speed (M_MEDIUM). Specify the search speed using the M_SPEED control type of the McodeControl() function. If you are having problems finding the code, you might want to search at a speed lower than the default.

In a read operation, the source image is internally binarized so as to separate the code from the background. By default, the threshold value is automatically chosen and is suitable in most cases. However, if you think that a different value would result in a better separation (and therefore in a more efficient operation), you can manually adjust the threshold level by setting the M_THRESHOLD control type of the McodeControl() function to another value. In the case of non-uniform lighting, (see image below for an example) setting M_THRESHOLD to M_ADAPTIVE, instead of using the default, is recommended.

Sometimes 2D codes are composed of dots, such as when a code is punched into an object. Often in these cases, the dots have some spacing between them, which can make the code more difficult to read. In a case like this, it will be necessary to set M_DOT_SPACING. This should be set to half the pixel distance between dots. An example of a code that might need dot spacing to be set is given below.

By default, the operation searches for a code that encodes a string of any size. However, if you know the exact size of the string, you might want to specify this value to increase the robustness of the operation using the M_STRING_SIZE control type of the McodeControl() function.

Note that you must specify the string size when reading the BC412 code type.

Write operations - issues to consider

It is strongly recommended that you define your codes as clearly as possible, as it will facilitate reading them later on. When writing a code ensure that you are familiar with the requirements and restrictions of your chosen code type, otherwise the code will not be generated; a MIL error will result.

The destination image of the write operation should be large enough to hold the encoded string. For a given code type, cell size, and string, you can inquire about the minimum buffer size required by first calling McodeWrite() with its image buffer parameter set to M_NULL and then using the M_WRITE_SIZE_X and M_WRITE_SIZE_Y result types of McodeGetResult().

Note that some code types have specific requirements for the encoded string:

  • The encoded string for a Codabar code type must be numeric, but can contain the following ASCII marks: -, $, :, /, ., and +. In addition, the string must start and end with any of the following characters: a, b, c, or d.
  • The Interleaved 2 of 5 code type requires a string that has an even number of characters.
  • The encoded string for a Maxicode code type with the M_ENC_MODE2 or M_ENC_MODE3 control type of the McodeControl() function, must respect the structured carrier message format (the portion of the string that contains postal code, country code, and class of service information).

If you need to encode non-printing characters such as a carriage return or tab, use a code type that supports them. If you set the McodeWrite() control flag to M_ESCAPE_SEQUENCE, you can use the ASCII codes to encode all non-printing characters. In the string to encode, the non-printing character's ASCII code must be in hexadecimal format and be preceded by the \x, for example, \x0D for ASCII 13, which is the carriage return.

If you want to encode the "\" character in escape sequence mode, type "\\".

In most cases, you should not have to specify the cell size when writing codes; the default setting (M_CELL_SIZE_MIN set to M_DEFAULT for the McodeControl() function) is sufficient. In this case, the code is resized so as to just fit into the destination image of the operation, when possible. During a write operation, you can use the M_CELL_SIZE_MIN control type of the McodeControl() function to force the cell size; the size is used to determine the size of the generated code. If you specify the required cell size with McodeControl(), you should ensure that the destination image has an appropriate size. See the discussion earlier in this subsection.

The Planet code type requires an 11-digit string which begins with 21 or 22.

The Postnet code type requires a 5, 9, or 11-digit string.

The M_CELL_SIZE_MAX control type is not used in write operations.

During a write operation, the number of cells will be automatically chosen to minimize the code written. To specify the number of cells, use both the M_CELL_NUMBER_X and M_CELL_NUMBER_Y control types of the McodeControl() function. If you specify more cells than are necessary to generate the code, fill characters will automatically be added. If you specify fewer cells than are necessary to generate the code, you will get a MIL error.

Note the following when using these control types and some code types:

  • Specifying the number of cells is mostly useful when writing PDF417 and Data Matrix code types; the Maxicode code type has a fixed number of cells.
  • Note that for the PDF417 code type, M_CELL_NUMBER_X = 17c + 35, where c is the number of columns, and 35 represents the number of cells required for the start and stop patterns.
ENCODING ANY ERROR CORRECTION FOREGROUND VALUE ANY ENCODING ERROR CORRECTION USE PRESEARCH CELL SIZE MIN CELL SIZE MAX CELL NUMBER X CELL NUMBER Y CELL NUMBER X SEARCH ANGLE SEARCH ANGLE DELTA NEG SEARCH ANGLE SEARCH ANGLE DELTA POS MEDIUM SPEED THRESHOLD THRESHOLD ADAPTIVE DOT SPACING STRING SIZE NULL WRITE SIZE X WRITE SIZE Y ENC MODE2 ENC MODE3 ESCAPE SEQUENCE CELL SIZE MIN DEFAULT CELL SIZE MIN CELL SIZE MAX CELL NUMBER X CELL NUMBER Y CELL NUMBER X


Retrieving results

Reference:
McodeGetResult
Not available in MIL-Lite

Available in MIL

You can retrieve results by calling McodeGetResult() with the control type for the result that you want returned.

You can retrieve the string or the length of the string in escape sequence format. Using the M_ESCAPE_SEQUENCE flag with M_STRING will return the string showing the ASCII codes for non-printing characters. Using the M_ESCAPE_SEQUENCE flag with M_STRING_SIZE will return the length of the encoded string, including all ASCII characters used to encode all non-printing characters in the count.

When retrieving a string result for an UPC-E code type, the full, uncompressed 12-digit product code is returned.

The ASCII codes are returned in hexadecimal format.

ESCAPE SEQUENCE STRING ESCAPE SEQUENCE STRING SIZE


Verifying your code

See Also | Availability | Previous
Reference:
McodeControl
McodeGetResult
McodeVerify
Not available in MIL-Lite

Available in MIL

Verification is used to check the conditions under which a code is read, to check if codes you are reading meet your quality specifications, and to see if the codes you are writing meet your quality specifications. When you are reading or writing codes, it is important to know that the quality of the code is sufficient for your purposes.

You can use McodeVerify() to verify the quality of 1D, 2D, or composite codes. You can pass McodeVerify() the string of the code to be graded. If the string is a composite code, the following format for the string must be used: 1D string|2D string, where | is used as the separator.

Note that McodeVerify() is not supported for Pharmacode, Postnet, and Planet code types.

You can read the results of the verify operation with McodeGetResult().

There are specific results of the verify operation that you can get for your code, including the attributes of the scan reflectance profile. The scan reflectance profile of a code is a record of values measured along a line across the width of the code (line profile). There are nine attributes of a scan reflectance profile that are graded; not all of these are applicable for all codes. See McodeGetResult() to see which results are possible for which codes.

  • Decodability. This attribute is used to verify how near the scan reflectance profile is to a failing decode grade.

    • This is applicable for all 1D barcodes, composite codes, RSS codes and the PDF417 code types.

  • Decode. This attribute is used to verify whether the symbol can be decoded.
  • Defects. This attribute is used to verify the uniformity of the reflectance of the code. The more spots or voids, the lower the grade.

    • This is applicable for all 1D barcodes, composite codes, RSS codes and the PDF417 code types.

  • Edge determination. This attribute is used to verify whether the code has an appropriate number of bars and spaces.

    • This is applicable for all 1D barcodes, composite codes, RSS codes and the PDF417 code types.

  • Minimum edge contrast. This attribute is used to verify whether the minimum contrast between bars and their adjacent spaces is acceptable.

    • This is applicable for all 1D barcodes, composite codes, RSS codes and the PDF417 code types.

  • Minimum reflectance. This attribute is used to verify the reflectance of the darkest bar. Reflectance is a measure of the light reflected off the code. If the darkest bar has a reflectance less than half of the background, this attribute gets a failing grade.

    • This is applicable for all 1D barcodes, composite codes, RSS codes and the PDF417 code types.

  • Modulation. This attribute is used to verify the intensity of the edge contrast as a proportion of symbol contrast.

    • This is applicable for all 1D barcodes, composite codes, RSS codes and the PDF417 code types.

  • Start/stop pattern. This attribute is used to verify the quality of the set pattern that marks the end points of the code.

    • This is applicable for composite code and the PDF417 code types.

  • Symbol contrast. This attribute is used to verify the code contrast. The higher the contrast, the better it is for scanning purposes and the better the grade.

    • This is applicable for all codes except the MicroPDF417 code types.

Other aspects of the code that you can verify are given below; not all of these are applicable for all codes. See McodeGetResult() to see which results are possible for which codes.

  • Axial nonuniformity. This is a measure of how the spacing between sampling points along the width differs compared to height.

    • This is applicable for the DataMatrix and Maxicode code types.

  • Overall symbol grade. This is a measure of the worst grade for any of the results for the code.
  • Print growth. This is a measure of the extent in which the boundaries of the black and white markings of the code are within their module's boundaries.

    • This is applicable for the DataMatrix and Maxicode code types.

  • Row Address Pattern (RAP). This is a measure of how close a scanned code's RAP stands to its ideal pattern.

    • This is applicable for composite codes and the MicroPDF417 code type.

  • Symbol-based grade. This is a measure of how close a scanned character's pattern stands to its ideal pattern.

    • This is applicable for the composite code, MicroPDF417, and PDF417 code types.

  • Unused Error Correction. This is a measure of the extent to which regional or spot damage in the code has eroded the reading safety margin that error correction provides.

    • This is applicable for the DataMatrix and Maxicode code types.

Note that there are some controls in McodeControl() that might have to be set before you verify a code. These are listed in the table below.

Control | Notes
M_FOREGROUND_VALUE | Specifies the foreground value of the image. This control is essential and the code will not be graded if the foreground value is not correctly set.
M_ENCODING | Specifies the type of encoding scheme for the code. This control has to be set for the code types where M_ANY is not supported.
M_ERROR_CORRECTION | Specifies the type of error correction for the code. This control has to be set for the code types where M_ANY is not supported.
M_STRING_SIZE | Specifies the size of the string for which to search. This control has to be specified for the BC412 code type.
M_SEARCH_ANGLE... | Specifies the nominal search angle and angular search range. These controls have to be specified if the code to read is not within the angular range of -5 to 5 degrees.


Measurements

Chapter 13: Measurements
Availability
Not available in MIL-Lite

Available in MIL

This chapter describes the MIL measurement module and the steps to follow to take measurements.

  • The measurement module
  • Markers
    • A multiple marker
  • Steps to finding and obtaining measurements of markers
    • Allocating or restoring a marker
    • Setting the marker's measurement box
    • Viewing the marker
    • Setting the marker's characteristics and processing area
    • Specifying the measurement control settings
    • Acquiring and pre-processing a target image
    • Finding the marker and taking measurements
    • Reading results
    • Annotating results
    • Measurement examples
  • Measurement box
    • Obtaining valid results
    • Orientation
    • Setting the measurement box's angle
    • Multiple-angle search for a marker
    • Determining the rotation tolerance of a marker
  • Search algorithm
    • Identifying edges
    • First derivative filter
  • Marker characteristics
  • Edge markers: fundamental characteristics
    • Polarity
    • Position and position variation
    • Contrast and contrast variation
    • Length
    • Line equation
  • Edge markers: advanced characteristics
    • Edge width
    • Minimum/maximum position
    • Edge strength
    • Edge threshold
    • Determining the strength of the required edge
    • Marker reference
    • Weight factors
  • Stripe markers: fundamental characteristics
    • Polarity
    • Contrast and contrast variation
    • Width and width variation
    • Position
    • Length
    • Line equation
  • Stripe markers: advanced characteristics
    • Minimum and maximum position
    • Inside edge and inside-edge variation
    • Position inside stripe
  • Multiple marker characteristics
  • Measurements between two markers
    • Steps to taking measurements between two markers
    • Calculating with multiple markers
    • Angle
    • Line equation and distance
  • Measurement examples


The measurement module

See Also | Availability | Next
User Guide:
Calibration
Not available in MIL-Lite

Available in MIL

The MIL measurement module allows you to find sets of image characteristics or "markers" in an image, based on differences in pixel intensities. Upon finding a marker, the module returns the marker's spatial reference position and measures such features as its width and angle. The module can also take measurements between two markers.

The measurement module can be used, for example, to measure the width of pins protruding from chips or printed circuit boards (PCBs) and to measure the distance between each pin.

The measurement module relies on a one-dimensional analysis. As such, it has several advantages over the pattern matching module when locating relatively simple image characteristics; it is independent of lighting, more tolerant of slight differences, and much faster.

You specify the approximate location and other characteristics of a marker to help locate the marker in an image. The more precisely the marker characteristics are defined, the more likely it is to be distinguished from similar aspects of the image.

The measurement module can operate on 8-bit or 16-bit unsigned grayscale buffers. Measurements are made with subpixel accuracy and results can be returned in pixels or real-world units. For more information, see Chapter 5: Calibration.

This chapter discusses finding and obtaining measurements of markers, how to define and set marker characteristics, and then the steps that are generally followed when taking measurements between markers.


Markers

Reference:
MmeasSetMarker
Not available in MIL-Lite

Available in MIL

To take any type of measurement with the MIL Measurement module, you must first define your markers, using MmeasSetMarker(). The marker contains the image characteristics to search for in the target image.

There are three types of markers that can be used in measurement operations:

  • Point marker. A marker consisting of a single point or multiple points and generally used as a reference position in calculations involving two markers. Point markers cannot be searched for but can be placed manually at the required location as a reference marker. Positional results from a previous pattern matching or blob analysis operation on the image can also be declared as point markers.
  • Edge marker. A marker consisting of an edge or multiple edges. In an image, edges are curves that delineate a boundary, which can be established from intensity transitions.
  • Stripe marker. A marker consisting of a stripe (two edges) or multiple stripes. The edges of a stripe marker do not have to be parallel.

A multiple marker

The measurement module allows you to define a multiple edge, stripe, or point marker, so that you can search for multiple instances of the same image characteristics. You can specify the number of edges or stripes to locate, using MmeasSetMarker() with M_NUMBER. Using a multiple marker in a measurement operation makes it possible to take global measurements, then compare them for conformity. For example, a multiple marker could be used to verify if a series of presumed-identical pins are actually of equal width or if the spacing between the pins is within established limits. Note that a multiple marker is considered to be only one marker that has a specified number of instances of the same characteristics.

NUMBER


Steps to finding and obtaining measurements of markers

User Guide:
Annotating the displayed image non-destructively
Measurement examples
Preparing for graphics

Reference:
MmeasAllocContext
MmeasAllocMarker
MmeasControl
MmeasDraw
MmeasFindMarker
MmeasFree
MmeasGetResult
MmeasInquire
MmeasRestoreMarker
MmeasSaveMarker
MmeasSetMarker
Not available in MIL-Lite

Available in MIL

Although there are many types of measurements that can be taken with the MIL measurement module, the series of steps outlined below are usually followed to find and obtain measurements of markers:

  1. Allocate an edge or stripe marker. For a multiple marker, set the number of occurrences of the edge or stripe.
  2. Set the processing area, generally referred to as the measurement box.
  3. Set the marker's characteristics. Optionally, set the measurement control settings.
  4. Grab or load a target image. Optionally, preprocess it to improve its quality.
  5. Find the marker in the target image and calculate measurements.
  6. Read the results.

In general, the first four steps are performed once, while steps 5 and 6 are repeated as required. Note, you can avoid step 1 to 4 by saving an initialized marker on disk and restoring it when needed.

Allocating or restoring a marker

Allocate a new marker, using MmeasAllocMarker() or restore a previously saved marker from disk, using MmeasRestoreMarker(). You can allocate an edge, stripe, or point marker, depending on your application needs; if a multiple marker is needed, specify the number of occurrences of the edge or stripe. When a marker is no longer required, you should free the memory associated with it, using MmeasFree(). Store a marker to disk using MmeasSaveMarker(). The save function stores all of the marker's associated characteristic settings, such as the marker's typical position, width, and contrast.

Setting the marker's measurement box

Before you can search for a marker, you must define the area in which to perform the search. This area is known as the measurement box. The measurement box is stored as a characteristic of the marker and is set with MmeasSetMarker(). Subsequent search operations for this marker will only be performed in the area defined by the box.

Viewing the marker

You can use MmeasDraw() to draw specific features of your marker as a means of verifying your marker settings. For example, you can draw the measurement box, the position, possible position variation, width variation, and edge of the expected marker. For stripe markers, you can draw both the first and second edges. For multiple markers, you also draw the expected spacing. You can use a previously allocated graphics context to control the drawing color, or use the default graphics context (see the Preparing for graphics section in Chapter 17: Generating graphics). You can draw directly into the image buffer, or annotate the image non-destructively by drawing into its display overlay buffer. For more information, see the Annotating the displayed image non-destructively section in Chapter 16: Displaying an image.

Setting the marker's characteristics and processing area

The marker's characteristics, such as its approximate width, orientation, and position, must also be defined using MmeasSetMarker().

You can inquire about the current settings of a marker's characteristics, using MmeasInquire().

Specifying the measurement control settings

When performing an MmeasFindMarker() operation, you must specify a measurement context, either the default one or one allocated using MmeasAllocContext(). Measurement context settings, such as the pixel aspect ratio, control the behavior of measurement operations. You can modify these settings using MmeasControl().

When a measurement context is no longer required, it should be freed, using MmeasFree().

Acquiring and pre-processing a target image

The target image can be either loaded from disk or acquired from an input device and placed into an image buffer. You can preprocess the target image to remove noise and improve the image prior to performing the measurement operation. Note, however, that preprocessing operations such as filtering often result in a slight shifting of the location position of edges. Therefore, if precise edge location and measurement are crucial to your application, preprocessing operations should be kept to a minimum.

Finding the marker and taking measurements

Use MmeasFindMarker() to find a marker. The MmeasFindMarker() function can measure all calculable edge and stripe characteristics, such as the width, orientation, or contrast. The function's parameters allow you to specify which measurements to take (the default setting performs all measurements).

Reading results

You can obtain results using MmeasGetResult(). Results from a MmeasFindMarker() operation are stored directly with the marker rather than in a result buffer. All positional results are relative to the top-left pixel in the target image or the origin of the coordinate system of a calibrated image (recall that the pixel reference position is its center). Note that results do not overwrite specified marker characteristics.

For a multiple marker, the MmeasFindMarker() function takes the required measurements for all the located edges or stripes. The number of edges or stripes found can be obtained using M_NUMBER as the result type with the MmeasGetResult() function. Global results, such as the maximum, minimum, mean, or standard deviation of any characteristic of the result group can be returned.

Annotating results

In addition to the Mgra...() functions available for annotating your results, the MmeasDraw() function provides several additional operations for drawing features of the marker occurrences. For example, you can draw the edge profile of the occurrence, a cross at the found position, the width of the marker, among other features.

Measurement examples

For examples on finding and obtaining measurements of markers, see the Measurement examples section later in this chapter.

NUMBER


Measurement box

Reference:
MimRotate
MmeasFindMarker
MmeasSetMarker
Not available in MIL-Lite

Available in MIL

The marker's measurement box indicates the area of the target image in which to search for the marker. Proper placement of the measurement box is essential to the success of any MmeasFindMarker() search. The default setting of the measurement box is the whole image. The measurement box should be limited to as small an area containing the marker as possible, in order to ensure the success of the operation, especially when using a highly detailed or complex target image. In addition, by limiting the processing region, you can accelerate the find marker operation.

There are two ways to specify the measurement box. To specify the box's position, you can set either the origin (M_BOX_ORIGIN) or the center (M_BOX_CENTER) coordinates. You must also set the box's width and height (M_BOX_SIZE).

Note that the origin is always the top-left corner of the unrotated box and all width and height values are positive.

Obtaining valid results

To obtain valid results, the edge or stripe must enter and leave by opposite sides of the box. The illustration below is an example of valid and invalid measurement box definitions.

Orientation

The orientation (M_ORIENTATION) specifies the angle of the edge or stripe relative to that of the measurement box and can be set to either M_VERTICAL or M_HORIZONTAL. More importantly, the orientation determines the direction in which the search will proceed. If set to M_ANY, both orientations will be searched.

These settings can tolerate a certain amount of rotation. The amount is determined by the target image, placement of the measurement box, and the marker characteristic settings. The greater the degree of rotation, the greater the chance of not finding the marker and miscalculation of characteristics, such as edge strength and width. If the rotation is too great and you cannot find the marker, the marker's measurement box should be defined with an angle.

Setting the measurement box's angle

You can set the measurement box angle (M_BOX_ANGLE) to approximately the same angle as the marker. The angle is in a counter-clockwise direction relative to the positive X-axis and can be any value from 0 to 360°. When an angle is specified, the center of rotation used is the center of the measurement box. To modify this default center of rotation, use MmeasSetMarker() with M_BOX_ANGLE_REFERENCE.

Multiple-angle search for a marker

You can also rotate the measurement box to search within a specified range of angles. To perform a multiple-angle search for a marker, enable multiple-angle search, using MmeasSetMarker() with M_BOX_ANGLE_MODE. Then, specify the range of angles to be searched (M_BOX_ANGLE_DELTA_NEG and M_BOX_ANGLE_DELTA_POS), the degree of rotation tolerance at any given angle (M_BOX_ANGLE_TOLERANCE), the interpolation method (M_BOX_ANGLE_INTERPOLATION_MODE), and the required degree of accuracy for the resulting marker (M_BOX_ANGLE_ACCURACY).

The approximate location of the edge or stripe marker is first found by searching at every n degrees within the angular range, where n is specified by the rotation tolerance. The marker's rotation tolerance is the full range of degrees within which a marker can be rotated from a measurement box that is at a specific angle and still be found. Once the approximate location of the edge is found, the degree of accuracy controls the number of fine-tuned searches that are performed. To be effective, you must set the degree of accuracy to a value smaller than that of the rotation tolerance.

If searching within a range of angles causes the measurement box to be rotated beyond the edges of the target image, you must reposition or resize the measurement box.

Determining the rotation tolerance of a marker

Every marker has its own particular rotation tolerance (that is, by how much the marker can be offset from the measurement box and still be found). This tolerance is dependent on the individual marker characteristics and surrounding image features. To determine the rotation tolerance of a marker, simulate the rotation of the marker by rotating the target image while maintaining the search region at a fixed position and angle. That is:

  1. Make sure that the marker is at its expected (nominal) angle, and that the position and angle of the measurement box is set so that the marker is located within the measurement box at close to the same angle. The measurement box must remain at a set position and angle. Therefore, multiple-angle search (M_BOX_ANGLE_MODE) must be disabled.
  2. Use the MimRotate() function to rotate the image in very small increments (for example, 0.5°), in the positive direction, and perform a MmeasFindMarker() operation on each rotated image. Make sure that the image's center of rotation is the same as that of the measurement box, otherwise the resulting tolerance will not be accurate. Note, when rotating the image, always set the angle from the image's original position to avoid interpolating the image more than once.
  3. Check the results for the greatest angle that produces an acceptable score.
  4. Repeat steps 2 through 3, rotating the image in the negative direction.
  5. Take the minimum of the absolute value of these angles. Double this angle and set it as the rotation tolerance for the angular search.

Alternatively, you can set the measurement box angle to M_ANY to allow MIL to analyze the contents of the measurement box and determine the angle. However, it should be noted that the processing time will be greatly increased when using this technique.

BOX ORIGIN BOX CENTER BOX SIZE ORIENTATION VERTICAL HORIZONTAL ANY BOX ANGLE BOX ANGLE REFERENCE BOX ANGLE MODE BOX ANGLE DELTA NEG BOX ANGLE DELTA POS BOX ANGLE TOLERANCE BOX ANGLE INTERPOLATION MODE BOX ANGLE ACCURACY BOX ANGLE MODE ANY


Search algorithm

User Guide:
Edge markers: advanced characteristics

Reference:
MmeasSetMarker
Not available in MIL-Lite

Available in MIL

This section describes the search algorithm used to identify edges in the Measurement module.

Identifying edges

The MIL Measurement module performs the following search algorithm to identify edges:

  1. The MIL measurement search algorithm projects the pixels bounded by the specified two-dimensional measurement box into a one-dimensional pixel summation, storing the result in an internal projection buffer (that is, it takes the box's profile). The pixel summation is performed horizontally or vertically, depending on the measurement box's origin (M_BOX_ORIGIN) and the orientation of the marker (M_ORIENTATION). Each sum represents the intensity of all the pixels in that column.
  2. The search algorithm applies a first derivative filter to the internal projection buffer (pixel summation). The filter finds the edge value of each projection value. The greater the difference in neighboring projection values, the larger the edge value. Edge values are represented as a normalized percentage of the maximum pixel value possible for the specific image buffer. Filters are discussed later in more detail.
  3. The search algorithm then identifies possible edges by their edge strength (M_EDGE_STRENGTH), that is their maximum/minimum edge value (depending on the polarity of the edge represented by the sign of the edge value). It rejects as possible marker occurrences any edge with an edge strength below the edge threshold value (M_EDGE_THRESHOLD).
  4. It then scores each possible edge based on geometric constraints that you specify. You can assign a specific weight (degree of importance) to each specified characteristic using MmeasSetMarker() with the M_WEIGHT_FACTOR combination constant. By default, the edge strength (M_EDGE_STRENGTH) receives the most importance. For more information on weights, see the Edge markers: advanced characteristics section later in this chapter. The edge(s) with the highest score is returned as an occurrence of the marker.

First derivative filter

The first derviative filter extracts the difference in neighboring projection values. The edge value calculated for each projection value of the measurement box is dependent on the type of the first derivative filter used. In the MIL measurement module, there are two types of first derivative filters which can perform the edge extraction: Finite Impulse Response (FIR) filters and Infinite Impulse Response (IIR) filters. FIR filters are implemented using a non-recursive convolution operation with a predefined kernel of the specified size. To use a FIR filter, set MmeasSetMarker() M_FILTER_TYPE to M_EULER or M_PREWITT. IIR filters are implemented using a recursive convolution operation, and the kernel size would be theoretically infinite if this filter actually used a kernel. To use an IIR filter, set MmeasSetMarker() M_FILTER_TYPE to M_SHEN, and specify the degree of smoothness (strength of denoising) applied to the internal projection buffer of the measurement box, using MmeasSetMarker() with M_FILTER_SMOOTHNESS.

You can try these filters to see which best suits your application needs. The following is the general characteristics of each filter:

  • [-1,1] (M_EULER). This is a FIR filter with a convolution kernel of size 2. This filter is faster but more sensitive to noise, compared to the M_PREWITT filter.
  • [-1,0,1] (M_PREWITT). This is a FIR filter with a convolution kernel of size 3. This filter is slower but less sensitive to noise, compared to the M_EULER filter.
  • An IIR filter (M_SHEN). This is a Shen-Castan Infinite Support Exponential filter. This is an exponential weighting function, of the general form:

    • This IIR filter is slower than FIR filters (M_EULER or M_PREWITT). However, this filter is less sensitive to noise and provides more accurate results. You can also control the strength of denoising.

BOX ORIGIN ORIENTATION EDGE STRENGTH EDGE THRESHOLD WEIGHT FACTOR EDGE STRENGTH FILTER TYPE EULER PREWITT FILTER TYPE SHEN FILTER SMOOTHNESS PREWITT EULER EULER PREWITT


Marker characteristics

Reference:
MmeasAllocMarker
MmeasFindMarker
MmeasSetMarker
Not available in MIL-Lite

Available in MIL

Associated with a marker is a set of parameters specifying the characteristics of the marker. The MmeasFindMarker() function searches through the target image to find the edge or stripe that best matches the characteristics (parameter settings) of the specified marker. The more precisely defined your marker characteristics, the more likely the find routine will have success in distinguishing it from similar aspects of the image.

Marker characteristics are set to their default values upon allocation of the marker (see the MmeasAllocMarker() function description for the default values). These values can be modified at any time, using MmeasSetMarker().

The important characteristics to set when searching for a marker are the measurement box, the polarity, the contrast, and for stripe markers, the width. Fundamental and advanced characteristics are discussed for edge, stripe, and multiple markers in the following sections. Note, all characteristics can be set to M_ANY if the value is unknown or not a criteria, unless otherwise specified in the MIL Reference.


Edge markers: fundamental characteristics

User Guide:
Measurement box

Reference:
MmeasCalculate
MmeasFindMarker
MmeasGetResult
MmeasSetMarker
Not available in MIL-Lite

Available in MIL

This section describes the fundamental characteristics of edge markers which are set using MmeasSetMarker(), or obtained as measurement results using MmeasFindMarker().

Polarity

The polarity (M_POLARITY) of an edge describes whether an edge is rising or falling. A rising edge denotes a rise in grayscale values and a positive (M_POSITIVE) polarity. A falling edge denotes a decrease in grayscale values and a negative (M_NEGATIVE) polarity. When setting the polarity of a marker it is important to keep in mind the direction of the search, which is performed horizontally or vertically, depending on the measurement box's origin and the orientation of the marker. For more information, see the Measurement box section earlier in this chapter.

Position and position variation

The marker's position is defined as the X and Y coordinates of the marker's center pixel (the center of the portion of the marker located within the measurement box). These coordinates are relative to the top-left pixel of the image and are used as the default reference position when using MmeasCalculate() to calculate measurements between two markers.

When several edges have similar characteristics within the same measurement box, then you can use the position characteristic to specify the approximate X and/or Y coordinates (M_POSITION, M_POSITION_X, M_POSITION_Y) at which to find the required marker's center.

You can also specify a tolerance for these coordinates (M_POSITION_VARIATION).

The position must be located within the measurement box (taking into account the measurement box's angle or angular range), otherwise an error is generated.

Contrast and contrast variation

You can indicate the typical difference in average grayscale values between the start and end of the intensity transition (edge width) from which an edge is established. (M_CONTRAST). Contrast is useful in distinguishing between several different edges, particularly when the required edge does not have the largest edge strength (described later), or when the edge is at an angle. When the edge strength is not very high, you should specify the contrast. In general, it is recommended to set the contrast rather than to specify an edge strength, since the edge strength is very dependent on lighting.

You can also specify a tolerance for the contrast (M_CONTRAST_VARIATION).

Length

You can measure the length (M_LENGTH) of an edge marker. The length being measured is restricted to the portion of the marker contained within the measurement box. Note, the length of an edge marker cannot be set, but can be returned with MmeasGetResult().

Line equation

You can calculate the line equation (M_LINE_EQUATION) of the mean line following an edge marker: Y = MX + B, where M denotes the slope of the line and B denotes the Y-intercept.

To determine the equation of the line (slope) for each edge, MIL performs additional internal search operations in the upper and lower thirds of the measurement box. If an image has substantial specks that appear in either of these two areas, the calculated lines might be at incorrect angles, as illustrated in the following images.

Notice the edge results on the left compared to the results on the right. Often, raising the edge threshold (using MmeasSetMarker() with M_EDGE_THRESHOLD) will correct this type of error. Also, you should always verify the edge values using MmeasSetMarker() with M_BOX_EDGE_VALUES.

Note, the line equation of an edge marker cannot be set, but can be returned with MmeasGetResult(). The slope of the line equation (M_LINE_EQUATION_SLOPE) and the Y-intercept of the line equation (M_LINE_EQUATION_INTERCEPT) can also be returned separately.

POLARITY POSITIVE NEGATIVE POSITION POSITION X POSITION Y POSITION VARIATION CONTRAST CONTRAST VARIATION LENGTH LINE EQUATION EDGE THRESHOLD BOX EDGE VALUES LINE EQUATION SLOPE LINE EQUATION INTERCEPT


Edge markers: advanced characteristics

User Guide:
Search algorithm

Reference:
MmeasCalculate
MmeasFindMarker
MmeasGetResult
MmeasSetMarker
Not available in MIL-Lite

Available in MIL

This section describes the advanced characteristics of edge markers that are set using MmeasSetMarker(), or obtained as measurement results using MmeasFindMarker().

Edge width

In an image, edges are curves that delineate a boundary, which can be established from intensity transitions. The smoother the image, the more gradual the transition. The edge width can be seen as a measure (in pixels) of this gradual transition in grayscale values. The diagram below illustrates a profile of gradual transition from dark to light.

The measurement module calculates the edge's position to be in the middle of this edge width. The position variation is equivalent to half the edge width. Note, the more an edge is at an angle, the greater the stretching or distortion of its edge width.

The edge width of a marker cannot be set, but can be found with MmeasFindMarker().

Minimum/maximum position

The minimum and maximum positions (M_POSITION_MIN and M_POSITION_MAX) indicate the minimum and maximum positions of an edge in an edge marker.

The X-coordinate of these positions are the minimum/maximum position within the edge width from which an edge is extracted, respectively. The Y-coordinate is the Y-coordinate of the measurement box center. The minimum position is always on the side of the measurement box adjacent to the origin, and the maximum position is always on the side of the measurement box furthest from the origin, regardless of how the measurement box is rotated.

The minimum and maximum positions of an edge cannot be set but can be found with, MmeasFindMarker().

Edge strength

A marker's edge strength (M_EDGE_STRENGTH) is either the minimum or maximum edge value along the edge width, depending on the polarity of the edge. Edge values are the output of the first derivative filter (that is applied to the internal projection buffer) represented as a normalized percentage of the maximum pixel value possible for the specific image buffer. The sign of the edge value represents the polarity of the edge. For example, for a measurement box with a vertical orientation, the equation for an edge value is:

You can also specify a tolerance for the edge strength (M_EDGE_STRENGTH_VARIATION).

For the following diagrams, edge values are calculated using an Euler FIR filter, [-1,1]. Using different filters will result in different edge values. For more information on identifying edges using first derivative filters, see the Search algorithm section earlier in this chapter.

The edge in the diagram below has an edge strength of 100%, the maximum edge value possible since the edge is completely vertical and has an edge width of one pixel.

However, if the edge is at the following angle, the edge profiles are distributed over ten profiles, resulting in a much lower edge strength of 10%:

When the edge strength is not very high, you should specify the contrast (M_CONTRAST). By using the contrast characteristic, you can indicate the typical difference in average grayscale values between the start and end of the intensity transition (edge width) from which an edge is established, allowing the marker to be located. Contrast is useful in distinguishing between several different edges, particularly when the required edge does not have the largest edge strength, or when the edge is at an angle. In general, it is recommended to set the contrast rather than to specify an edge strength, since the edge strength is very dependent on lighting.

Edge threshold

The edge threshold is the edge value beneath which a grayscale variation is not considered an edge and is set with M_EDGE_THRESHOLD.

Determining the strength of the required edge

To determine the edge strength of the required marker, use MmeasGetResult() with M_BOX_EDGE_VALUES. This returns the calculated edge value for every projection value of the measurement box. This allows you to determine what is likely to be considered an edge and specify an appropriate edge threshold value.

The illustration below shows a specific edge strength measurement, a graphical representation of edge values across the measurement box, and a sampling of these results.

Notice that the measurement box is 400 pixels wide so there are 400 edge values that are returned.

Marker reference

The marker reference position (M_MARKER_REFERENCE) defines the position from which measurements between two markers are taken. By default, the reference position is set to the center position of the marker. You can, however, move the reference position by specifying X- and Y-offsets relative to the center of the marker. The marker reference position is only used with the MmeasCalculate() function; MmeasFindMarker() always returns the marker's actual center. For example, the reference position for the edge marker in the diagram below is set to the center left of the marker.

Weight factors

When searching for a marker, the relative importance (weight) assigned to each of the marker characteristics is crucial to the robustness of the operation. By default, 50% of the search weight is assigned to the edge strength; the remaining 50% is equally divided among all characteristics that can have a weight factor and that are set to a value other than M_ANY (the value used to flag an "ignore" state). This makes the edge strength by far the most important characteristic in the search.

You can override this default by adding M_WEIGHT_FACTOR to certain of the marker characteristics (see MmeasSetMarker() in the MIL Reference manual for the list of applicable characteristics). However, to better control the search, it is recommended that when specifying weight factors, that you assign a weight factor to all the enabled characteristics which support weight factors to a total of 100%.

For example, in a case where you must distinguish between two edge markers of different contrast, you can specify the typical contrast of the marker to be found. If you specify only this characteristic, the default search algorithm will assign a 50% weight to the edge strength and the remaining 50% to the contrast. However, if the edge you want to ignore has the higher edge strength, the desired edge might not be found. In this case, specifying the weights as 30% for the edge strength and 70% for the contrast will give precedence to the edge with the best match to the specified contrast.

POSITION MIN POSITION MAX EDGE STRENGTH EDGE STRENGTH VARIATION CONTRAST EDGE THRESHOLD BOX EDGE VALUES MARKER REFERENCE WEIGHT FACTOR


Stripe markers: fundamental characteristics

Reference:
MmeasFindMarker
MmeasGetResult
MmeasSetMarker
Not available in MIL-Lite

Available in MIL

A stripe marker is simply a marker with two edges, therefore the discussion of edge characteristics applies to each of the stripe marker's edges. However, certain characteristics have special attributes applicable only to stripe markers.

As with edge markers, the characteristics of stripe markers are set using MmeasSetMarker(), or obtained as measurement results using MmeasFindMarker().

Polarity

The polarity of an edge describes whether an edge is rising or falling. The edges of a stripe marker can have opposite (M_OPPOSITE) polarities or can have similar (M_SAME) polarities.

Contrast and contrast variation

The contrast for a stripe marker requires two values, one for each of the stripe's edges. The contrast of the second edge can be set to M_SAME if both edges of the stripe have approximately the same contrast. Both values can be set to M_ANY if the contrast is unknown (default). The contrast variation for a stripe marker should be the maximum of the contrast variations of both its edges.

Width and width variation

To help find a stripe marker, you can specify the typical distance between both of its edges (M_WIDTH) in pixels.You can also specify by how many pixels the width of a stripe marker might vary (M_WIDTH_VARIATION). This value should be equivalent to the maximum amount of variation.

The width of a stripe marker is the average distance in pixels between its edges. Note, if the marker's measurement box (processing region) is at an angle, the width is measured according to the orientation of the box, as shown below.

Position

The position (M_POSITION) of a stripe marker is considered either the center between the two edges of the stripe or a position outside of the stripe marker, depending on the M_POSITION_INSIDE_STRIPE setting.

In addition, the stripe marker's center is the default reference position. The reference position is specified relative to the marker's center. You can move the reference position of a marker using MmeasSetMarker() with M_MARKER_REFERENCE.

Length

The length of a stripe marker is measured along the mean line between its exterior edges. The length of either of its edges can also be measured. The length being measured is restricted to the portion of the marker contained within the measurement box. Note, the length of a stripe marker cannot be set, but can be returned with MmeasGetResult().

Line equation

You can calculate the equation of the mean line following a stripe marker: Y= MX + B, where M denotes the slope of the line and B denotes the Y-intercept.

Line equations can be calculated for a stripe, or each edge of a stripe marker. The line equation of a stripe marker is the mean of the line equations of its edges.

Note, the line equation of a stripe marker cannot be set, but can be returned with MmeasGetResult(). The slope of the line equation (M_LINE_EQUATION_SLOPE) and the Y-intercept of the line equation (M_LINE_EQUATION_INTERCEPT) can also be returned separately.

OPPOSITE SAME SAME ANY WIDTH WIDTH VARIATION POSITION POSITION INSIDE STRIPE MARKER REFERENCE LINE EQUATION SLOPE LINE EQUATION INTERCEPT


Stripe markers: advanced characteristics

User Guide:
Edge markers: advanced characteristics

Reference:
MmeasFindMarker
MmeasGetResult
Not available in MIL-Lite

Available in MIL

This section deals with advanced characteristics with stripe markers.

Minimum and maximum position

The minimum and maximum positions (M_POSITION_MIN and M_POSITION_MAX) indicate the minimum and maximum positions of edges in a stripe marker. For more information on the minimum and maximum position of an edge, see the Edge markers: advanced characteristics section earlier in this chapter.

The minimum and maximum positions of edges cannot be set but can be found with MmeasFindMarker().

The average value of the two edges of a stripe marker is returned unless otherwise specified (using MmeasGetResult() with M_EDGE_FIRST or M_EDGE_SECOND).

Inside edge and inside-edge variation

To help find a stripe marker, you can specify the typical number of edges located between the external edges of the stripe marker you are defining. For example, in the following illustration, the two stripes share the same position since their centers coincide.

To find the larger stripe without having to determine and specify its width, specify 2 as the number of inside edges of the stripe marker (M_EDGE_INSIDE). To find the smaller stripe, specify 0 (the default, M_ANY, ignores the possibility of any inside edges). The identification of inside edges is based only on the edge threshold setting (M_EDGE_THRESHOLD). The number of such edges found can also be returned using M_EDGE_INSIDE as a result type.

You can also specify the tolerance in the number of inside edges of a stripe (M_EDGE_INSIDE_VARIATION). Note, this tolerance should be in increments of two if stripes are contained within stripes, since two edges are recognized for each stripe.

Position inside stripe

If necessary when defining a stripe marker, you can specify if the X and Y coordinates of M_POSITION must be located inside or outside of the stripe (M_POSITION_INSIDE_STRIPE). If the position is defined as being within the stripe, the search algorithm is modified so that the search proceeds outwards in both directions from that point, making the operation faster and more robust. M_POSITION_INSIDE_STRIPE is useful when the required stripe does not have the greatest edge strength or when it is difficult to set the measurement box without clipping other similar image characteristics. If defined as outside, no stripe including the position is considered.

Since other aspects of the image can have similar features or stronger edges, the required stripe might not be found. In order to successfully locate the stripe, specify its M_POSITION and then set M_POSITION_INSIDE_STRIPE to M_YES so that the search begins from the marker's approximate position, allowing the marker to be found.

POSITION MIN POSITION MAX EDGE INSIDE ANY EDGE THRESHOLD EDGE INSIDE EDGE INSIDE VARIATION POSITION POSITION INSIDE STRIPE POSITION INSIDE STRIPE POSITION POSITION INSIDE STRIPE YES


Multiple marker characteristics

Reference:
MmeasSetMarker
Not available in MIL-Lite

Available in MIL

To use a multiple edge or stripe marker, the marker's edge or stripe characteristics are set just as with any marker; only a few additional MmeasSetMarker() characteristics need to be set. In addition, the M_POSITION control type and its weight factor are ignored.

Specify the number of edges or stripes (M_NUMBER) to be found (default is 1) and the typical spacing between them if a regular pattern is expected (M_SPACING). Unless a minimum number (M_NUMBER_MIN) is specified, no results will be returned if the number of edges or stripes found falls below M_NUMBER. If the exact number of edges or stripes is unknown then M_NUMBER can be set to M_ALL.

When the M_SPACING setting is enabled, an initial search is performed to find all edges or stripes which best conform to the specified marker characteristics. This group of edges or stripes are then inspected to ensure that the spacing constraints are met. The marker's M_SPACING can be set to M_SAME, which takes the average spacing of all located edges or stripes and then applies this spacing as a criteria for determining the marker's actual edges or stripes.

M_WIDTH can also be set to M_SAME, meaning that the average width is applied as a constraint to all located stripes.

Note that the M_SAME setting is recommended only when several occurrences are expected; if the number of occurrences is too few, matches for width or spacing might not correspond to those of the required stripes.

A multiple point marker can also be specified for performing calculations between markers. A multiple point marker must have its initial position set using MmeasSetMarker() with M_POSITION. The spacing between these points must also be set using MmeasSetMarker() with M_SPACING. A multiple point marker can only be defined to use the same spacing between points; if irregularly spaced points are needed, you must specify each one individually as a point marker.

POSITION NUMBER SPACING NUMBER MIN NUMBER NUMBER ALL SPACING SPACING SAME WIDTH SAME SAME POSITION SPACING


Measurements between two markers

Reference:
MmeasAllocResult
MmeasCalculate
MmeasFindMarker
MmeasGetResult
MmeasGetResultSingle
Not available in MIL-Lite

Available in MIL

The MmeasCalculate() function performs calculations between two markers' reference positions. The default setting of the MmeasCalculate() function performs all measurements; distance, angle, and line equation. For a stripe marker the center position is used as the reference position from which to take measurements.

Steps to taking measurements between two markers

The series of steps outlined below are usually followed to take measurements between two markers:

  1. Allocate a result buffer, using MmeasAllocResult().
  2. Allocate, define, and find each marker with the MmeasFindMarker() function (see the steps previously outlined in the Steps to finding and obtaining measurements of markers section).
  3. Call MmeasCalculate(), specifying which markers are to be used as the first and second reference positions and which measurements to take.
  4. Read the results, using MmeasGetResult().

Calculating with multiple markers

If both markers are multiple markers, then calculations are made using the edges or stripes of the first marker and the corresponding edges or stripes in the second marker. The number of calculations is limited to the smallest number of results held in either marker (that is, if a marker contains only one edge or stripe, then only one calculation is performed, regardless of the number of edges or stripes contained in the other marker).

With a multiple marker, results for each calculation will be held in an array. Note that the array which you pass to MmeasGetResult() must be large enough to hold the result for each edge or stripe. If necessary, MmeasGetResultSingle() can be used to retrieve a single result from the array.

Angle

You can calculate the angle of a line joining two markers as shown below.

The angle is measured in a counter-clockwise direction relative to the positive x-axis, and can be a value from 0 to 360°.

Line equation and distance

The line equation and distance can be calculated for the line joining two markers (see diagram below).

The distance between two markers is the distance between both of their reference positions, as illustrated in the diagram below.

The horizontal or vertical distance between two markers can also be calculated by using M_DISTANCE_X or M_DISTANCE_Y as the result type with MmeasGetResult().

DISTANCE X DISTANCE Y


Measurement examples

Availability | Previous
Not available in MIL-Lite

Available in MIL

MMeas.c consists of two examples that use the measurement module to calculate the position, width, and angle of objects in an image. The first example of MMeas.c uses the following image:

This example demonstrates the steps to take to find a stripe in an image, and measure its position, width, and angle. Refer to the SingleMeasurementExample() function for this first example.

The second example of MMeas.c uses the following image:

This example demonstrates how to use the measurement module to find the average position, average width, and average angle of a row of pins on a chip. Refer to the MultipleMeasurementExample() function for this second example.

MMeas.c


Specifying and managing your data buffers

Chapter 14: Specifying and managing your data buffers
Availability
Available in MIL-Lite

Available in MIL

This chapter discusses data buffers in detail. It shows you how to allocate and manage data buffers, and how to restrict an operation to a portion of a data buffer by using child buffers. It shows you how YUV buffers are stored, how to create a user-defined buffer, and how MIL defines the pixel reference position. It shows you how to grab images with a Bayer camera and restore the color information.

  • Data buffers
  • Target system
  • Specifying the dimensions of a data buffer
  • Data type and depth
  • Attribute
  • Manipulating and controlling certain data buffer areas
    • Child buffers
    • Copying specific buffer areas
    • Processing a non-rectangular region of interest
  • Managing data buffers
    • Loading a data buffer
    • Saving a data buffer
    • Loading and saving a sequence of data buffers
  • Controlling how color image buffers are stored
  • RGB buffers
    • RGB data formats
  • Binary buffers
  • YUV buffers
    • YUV16 Packed
    • YUV9 Planar
    • YUV12 Planar
    • YUV16 Planar
    • YUV24 Planar
    • Child YUV buffers
  • Accessing a MIL buffer directly
  • Mapping a data buffer to user-allocated memory
  • Pixel conventions
  • Using buffers with the Bayer color filter
    • Using MIL to convert the image
    • How the Bayer image gets converted
      • Bilinear interpolation algorithm
      • Adaptive algorithm
    • Performing color correction
    • Correcting the white balance of your Bayer images
    • Performing gamma correction
  • Buffer overscan region


Data buffers

See Also | Availability | Next
Reference:
MbufAlloc1d
MbufAlloc2d
MbufAllocColor
Available in MIL-Lite

Available in MIL

In this help file, the term data buffer is used loosely to refer to the most general type of data buffer (storage area) that is allocated by the MIL package and operated on by most MIL functions. For example, a data buffer can be a buffer for image data or one for lookup table (LUT) data. Besides data buffers, there are also other buffers (for example, result buffers), which are specific to a particular group of functions. These types of buffers are discussed in the chapters describing their related functions.

All data buffers must be allocated before a function can access them. You can allocate a monochrome buffer using MbufAlloc1d(), MbufAlloc2d(), or MbufAllocColor(). You allocate a color buffer using MbufAllocColor().

When allocating a data buffer, you must specify its:

  • Target system.
  • Dimensions.
  • Data type and depth.
  • Attribute.

You can manipulate or control specific parts of data buffers by allocating and using child buffers. A child buffer is a subset of the parent buffer (a specific area of the parent buffer). Although any change made to the child buffer data affects the parent buffer, the buffer is considered a data buffer in its own right; wherever the parent buffer can be used, you can use the child buffer instead to affect only a part of the buffer. All results are returned relative to the child buffer coordinates rather than the parent buffer.


Target system

Availability | Previous | Next
Available in MIL-Lite

Available in MIL

A data buffer is allocated on the specified system. If the M_DEFAULT_HOST system is specified, the default Host system of the current MIL application will be used.

In addition, any operation involving one or more buffers will be performed by the most appropriate system that is associated with one of the buffers. By default, if none of these systems is more appropriate than the Host, the Host is used to perform the operation.

DEFAULT HOST


Specifying the dimensions of a data buffer

User Guide:
Color

Reference:
MbufFree
Available in MIL-Lite

Available in MIL

Data buffers can have up to three dimensions: an x, y, and color band dimension. Most data buffers have an x dimension (for example, LUT buffers) or an x and y dimension (for example, monochrome image buffers). The color-band dimension has been provided to allow you to store data for each color component used to represent an image; when allocating color buffers, each band will be of the same data depth and type.

Once you finish using a data buffer, you should release its memory space, using MbufFree().

Certain MIL functions support manipulating multi-band image buffers. For details on handling color image buffers, see Chapter 19: Color.


Data type and depth

Availability | Previous | Next
Available in MIL-Lite

Available in MIL

The data depth of a buffer indicates the number of bits per band in the buffer (1, 8, 16, 32). The data type of a buffer indicates how its data is internally represented (that is, whether the data is considered signed, unsigned, or floating-point). Supported combinations are: 1-bit packed binary; 8-, 16-, and 32-bit integer (signed and unsigned); and 32-bit floating-point. If a function can only operate on data buffers of certain depths, this is explicitly stated in the function's description, otherwise the function can be used with any combination of data buffers.

The packed binary data format represents each pixel by a single bit, in a state of 0 or 1. Therefore, 8 pixels can be packed in a single byte (known as an 8-bit data unit); that is, in a format eight times smaller than an 8-bit image.

Processing done directly on a packed binary buffer is very fast and efficient. Many MIL functions support accelerated processing using packed binary buffers. General processing functions which do not support packed binary buffers directly, automatically convert the data into a suitable data type buffer, perform the operation, and re-convert the resulting buffer to packed binary.

For efficiency, when possible, you should store binary data in packed binary buffers (rather than, for example, 8-bit integer buffers with only the values 0 and 0xFF). General processing functions that are optimized for packed binary buffers are noted as such in the MIL Reference.

In general, the fewer bits per pixel in a buffer, the faster an operation can be performed on the buffer. Packed binary buffers are the fastest to process. When you need to use integer buffers, use 8 bits per pixel when possible, 16 bits if necessary, and 32 bits as a last resort. When you need non-integer values, extra precision, or a greater dynamic range, you can use floating-point data buffers.


Attribute

Reference:
MbufAllocColor
MdispSelect
Not available in MIL-Lite

Available in MIL

The data buffer attribute indicates the buffer type and its intended usage. MIL uses this information to determine the most appropriate location in physical memory in which to allocate the buffer, and how to handle the buffer. A data buffer can be one of the following types:

  • M_IMAGE (image buffer).
  • M_LUT (lookup table buffer).
  • M_KERNEL (kernel buffer to define the filter for convolution functions).
  • M_STRUCT_ELEMENT (structuring element buffer for morphology functions).
  • M_ARRAY (array buffer).

When allocating an image buffer (M_IMAGE), you must give more information about its intended usage. An image buffer can be any combination of the following:

  • A buffer that can be displayed (M_DISP).
  • A buffer that can be processed (M_PROC).
  • A buffer in which data can be grabbed (M_GRAB).
  • A buffer in which data is stored in a compressed format (M_COMPRESS).

For example, to allocate an image buffer that can be displayed and used for processing, its attribute should be given as:

M_IMAGE + M_DISP + M_PROC + M_GRAB

When using MIL-Lite, the M_KERNEL and M_STRUCT_ELEMENT attributes are not available. In addition, the M_PROC image attribute is not available.

In general, buffers are allocated in Host memory instead of on-board memory by default. This is because on-board memory is limited in size and Host memory can be accessed much faster than on-board memory. However, if the system has an on-board processor, the buffer is allocated on-board by default. These defaults can be overridden using MbufAlloc...() with the M_ON_BOARD and M_OFF_BOARD attributes.

Buffers with the M_GRAB attribute are allocated in MIL non-paged (DMA) memory, which is physically contiguous and always present. An advantage of non-paged memory is that a bus mastering device can write to it without the help of the Host CPU. If a system does not support grab buffers (for example, M_SYSTEM_HOST), you could still allocate a buffer on such a system in physically contiguous and always present memory by giving it an M_NON_PAGED attribute instead.

When a displayable buffer is allocated and selected for display (MbufAlloc...() with M_DISP, and then MdispSelect()), two buffers are maintained internally: one in Host memory for processing purposes, and the other in a frame buffer (maintained directly or through a DIB) for display purposes (not necessarily the same size). When the Host buffer is modified, its associated buffer in the frame buffer is automatically updated. When displaying a buffer, both the buffer and the display should be allocated on the same system.

When grabbing a single frame into a displayable buffer, MIL grabs into the Host memory version of the buffer and then updates the display of the buffer. When grabbing continuously, the grab is made directly to the frame buffer, if possible, and then at the end of the grab, the Host buffer is updated.

Overriding the default allocation sequence is useful when allocating a displayable buffer for any auxiliary display. If you are not using the displayable buffer for processing or are only using it as a destination, storing the buffer on-board will avoid the extra copy operation to the display without the penalty of slowing down processing.

Even if it is not in the displayed area of the frame buffer, the image buffer depth and display depth must be the same.

It is also possible to force the internal representation of a data buffer using internal storage format specifiers, such as M_PACKED or M_PLANAR, which force the data buffer to be in a packed or planar format, respectively. Refer to MbufAllocColor() for a complete list of internal format specifiers.

If there is insufficient memory of the appropriate type to allocate a buffer with the specified attributes, the function generates an error and does not allocate the buffer.

If you try to use a data buffer in a situation that is not appropriate for its allocated attributes, an error message is generated and the operation is not performed. For example, if you try to display a buffer without an M_DISP attribute with MdispSelect(), an error message will be generated.

IMAGE LUT KERNEL STRUCT ELEMENT ARRAY IMAGE DISP PROC GRAB COMPRESS IMAGE DISP PROC GRAB KERNEL STRUCT ELEMENT PROC ON BOARD OFF BOARD GRAB SYSTEM HOST NON PAGED DISP PACKED PLANAR DISP


Manipulating and controlling certain data buffer areas

User Guide:
Using GDI annotations

Reference:
MbufChild1d
MbufChild2d
MbufChildColor
MbufChildColor2d
MbufCopy
MbufCopyClip
MbufCopyColor
MbufCopyColor2d
MbufCopyCond
MbufCopyMask
MbufFree
MdispSelect
Available in MIL-Lite

Available in MIL

You can manipulate or control specific parts of a data buffer by creating a child buffer within it or by copying specific parts of it to another buffer.

Child buffers

A child buffer is a subset (or region of interest) of a given data buffer (known as the parent buffer). Child buffers occupy a specific area of the parent buffer. Since this area is part of the same physical space as the parent buffer, changes made to the child buffer affect the parent buffer and vice versa.

The child buffer is considered a data buffer in its own right. Like its parent buffer, a child buffer must be allocated so that it can be associated with an identifier and recognized as an entity by the MIL package. Allocate a monochrome child buffer using MbufChild1d() or MbufChild2d(). To allocate a child buffer consisting of only one of the color bands of a multi-band image buffer, use MbufChildColor() or MbufChildColor2d(). Note, as a subset of the parent buffer, a child buffer cannot exceed the bounds of its parent in any dimension. For example, a color buffer cannot be created from a monochrome parent buffer.

A child buffer takes on the same attributes and type as the parent buffer. In general, any operation that can be performed on the parent buffer can also be performed on the child buffer.

Allocate a child buffer by specifying its size and offset with respect to each of the parent buffer dimensions. After, when using the child buffer, any specified or returned coordinates are relative to the child's top-left corner.

Once you have finished using a child buffer, you must delete it using MbufFree(), before freeing the parent buffer.

One major benefit of the child buffer is being able to handle several buffers simultaneously, in contexts where normally only one buffer can be handled. For example, for auxiliary displays, you can only display one buffer at a time. However, you might want to display the source and destination buffer of an operation simultaneously. You can get around this situation by allocating a displayable image buffer as large as the display, then allocating two child buffers from this buffer. You can then use one as the source data buffer and one as the destination. When the parent buffer is selected on the display (MdispSelect()), both the source and the destination child buffers can be seen.

Copying specific buffer areas

As an alternative to using a child buffer, you can restrict operations to specific areas or bits of a buffer (child or parent) by copying the required portions to another buffer. You can copy data from any type of data buffer to another using any of the following functions. For example:

  • Copy an image buffer to another buffer at the specified offset, using MbufCopyClip(). Data that falls outside of the destination buffer will be automatically clipped.
  • Copy specific non-sequential areas to another buffer based on a condition buffer, using MbufCopyCond(). Source buffer data is copied to the destination buffer if corresponding data in the specified condition buffer satisfies the copy condition. Other data in the destination buffer is left unaffected.
  • Copy specific non-consecutive bits to another buffer based on a mask, using MbufCopyMask(). Only destination bits that correspond to non-zero bits in the mask are modified with source bits.
  • Copy a single band of a multi-color band buffer to or from a single-band buffer, using MbufCopyColor() or MbufCopyColor2d(). This allows you to operate on a single color band of a buffer.

If the source buffer depth is greater than that of the destination, the most significant bits are truncated when the data is copied into the destination. If the source is signed and the destination depth is greater than the source, the source data is sign-extended when it is copied into the destination.

MbufCopy() copies the entire buffer into another buffer, while the other functions copy only portions of a buffer.

Processing a non-rectangular region of interest

To process a non-rectangular region of interest, you can apply a mask to the source image. For example:

  1. Allocate an image buffer, using MbufAlloc...(). The buffer should be at least the same size as the region of interest.
  2. Draw a filled shape in the mask buffer corresponding to the pixels to be modified, using the graphics functions (Mgra...()) or the drawing functions (M...draw()). You can also annotate the image with Windows GDI annotations. For more information, see the Using GDI annotations subsection in the Annotating the displayed image non-destructively section in Chapter 16: Displaying an image.
  3. Allocate a temporary destination buffer, using MbufAlloc...(). The temporary destination buffer should be at least the same size as the region of interest.
  4. Perform the operation from the source buffer to the temporary destination buffer.
  5. Copy the temporary destination buffer to the source buffer, using MbufCopyCond(), with the mask buffer as the condition buffer.


Managing data buffers

Reference:
MbufExport
MbufExportSequence
MbufGet
MbufGet1d
MbufGet2d
MbufGetColor
MbufGetColor2d
MbufImport
MbufImportSequence
MbufInquire
MbufLoad
MbufPut
MbufPut1d
MbufPut2d
MbufPutColor
MbufPutColor2d
MbufRestore
MbufSave
Available in MIL-Lite

Available in MIL

Besides the copy functions discussed in the previous section, MIL provides several other data buffer management functions. These allow you to transfer data between an array and a buffer, load data into a buffer (or a sequence of buffers), and save a buffer (or a sequence of buffers) to disk.

You can put data from an array into a data buffer, using MbufPut(), MbufPut1d(), MbufPut2d(), MbufPutColor(), or MbufPutColor2d(). MbufPut() puts data in the entire buffer, while MbufPutColor() or MbufPutColor2d() put data into one or all color bands of a multi-band buffer. The other two functions allow you to put data in a selected area of a monochrome buffer, respectively.

In addition, you can retrieve data from a data buffer and place it into an array, using MbufGet(), MbufGet1d(), MbufGet2d(), MbufGetColor(), or MbufGetColor2d(). MbufGet() gets data from the entire buffer, while MbufGetColor() or MbufGetColor2d() get data from one or all bands of a multi-band buffer. The other two functions, like their 'put in buffer' counterparts, allow you to get data from a selected area of a monochrome, respectively.

Note that you can also access the contents of a MIL buffer from an array by using MbufInquire(). Inquire the Host address of the buffer, and then using a pointer access the buffer as an array. This is discussed in more detail later.

Loading a data buffer

You can load data, using one of two methods:

  • Load data into an automatically allocated MIL data buffer, using MbufImport() with M_RESTORE, or using MbufRestore().
  • Load data into a previously allocated MIL data buffer, using MbufImport() with M_LOAD or using MbufLoad().

These functions internally handle the opening and closing of the file. With MbufImport(), you can specify the file's format. MbufLoad() and MbufRestore() will read the data in the file to determine the format, therefore they might take more time to return a result.

Saving a data buffer

You can save a data buffer to disk, using MbufExport() or MbufSave(). MbufExport() is the most general of these functions and can save data in any MIL-supported file format. MbufSave() can only save data in an M_MIL file format.

These functions internally handle opening and closing the file. If the given file name already exists, the file will be overwritten.

Loading and saving a sequence of data buffers

You can import or export a sequence of image buffers to a file using MbufImportSequence() or MbufExportSequence(), respectively. The available file formats are: standard AVI DIB format, MJPEG format, and proprietary AVI MIL format.

RESTORE LOAD MIL


Controlling how color image buffers are stored

Reference:
MbufAllocColor
Available in MIL-Lite

Available in MIL

A color image buffer's internal representation can be either in a planar or packed format. When allocating the buffer, if its attribute is also set to M_PLANAR, the pixels are stored in planes (for example, RRR GGG BBB). When allocating the buffer, if its attribute is set to M_PACKED, each pixel is stored as one unit containing all its components (for example, RGB RGB RGB).

MIL automatically selects the most appropriate format, according to the specified intended usage attribute. If an image buffer is allocated in one format, and a general processing function requiring another format is called, the function will automatically convert the data to the required format and re-convert it back to its original format upon completion. To change a buffer's default internal storage format, change the internal storage part of the attribute parameter for MbufAllocColor(). Note that it might be slower to process buffers with M_PACKED attributes.

In general, packed formats are mostly used for display purposes; when selecting a buffer's attribute as M_DISP, the default internal representation is usually packed. This configuration allows for faster transfers to display sections that handle packed data (for example, VGA). However, if the display section of your board has dedicated red, green, and blue frame buffer planes, the buffer is allocated in planar format.

Planar formats are generally preferred for processing. Here, the buffer stores each pixel as three component planes (for example, RRR, GGG, BBB). Processing is done on each of the components separately.

When allocating an image buffer with more than one attribute, for example, M_DISP and M_PROC, the buffer's internal storage requirements for the display will take precedence over other attributes.

See the MbufAllocColor() reference to determine which formats are supported on your board.

PLANAR PACKED PACKED DISP DISP PROC


RGB buffers

Availability | Previous | Next
Available in MIL-Lite

Available in MIL

By default, MIL allocates color image buffers in an RGB color format. The pixels are internally stored in little-endian order, that is, they are stored in memory from their least-significant to the most significant bytes. The definitions of the RGB formats that are available are shown here. The corresponding MIL constant is shown in brackets beside the common format name.

RGB data formats

BGR24 packed (M_BGR24 + M_PACKED) is a format whereby each pixel is internally stored as three consecutive bytes in little-endian order, that is:

BGR32 packed (M_BGR32 + M_PACKED) is a format whereby each pixel is internally stored as four consecutive bytes, in little-endian order. The most-significant byte is a "don't care" byte, as shown below:

RGB15 packed (M_RGB15 + M_PACKED) is a format whereby each pixel is internally stored as a 16-bit word with a 5-bit blue value (least significant), a 5-bit green value, a 5-bit red value, and a "don't care" bit (most significant), in little-endian order, as shown below. Note that when accessing an M_RGB15 + M_PACKED buffer as a 3-band 8-bit buffer, the least significant bits of each band are set to 0.

RGB16 packed (M_RGB16 + M_PACKED) is a format whereby each pixel is internally stored as a 16-bit word with a 5-bit blue value (least significant), a 6-bit green value, and a 5-bit red value (most significant), in little-endian order, as shown below. Note that when accessing an M_RGB16 + M_PACKED buffer as a 3-band 8-bit buffer, the least significant bits of each band are set to 0.

RGB planar are formats whereby the color components of all the pixels are stored contiguously: (RRR...., BBB..., GGG...).

BGR24 PACKED BGR32 PACKED RGB15 PACKED RGB15 PACKED RGB16 PACKED RGB16 PACKED


Binary buffers

Availability | Previous | Next
Available in MIL-Lite

Available in MIL

Binary buffers have a different internal storage format than other types of buffers: eight pixels are stored in one byte. The leftmost pixel of an image is the least significant bit that is stored in memory.


YUV buffers

Reference:
MbufAllocColor
Available in MIL-Lite

Available in MIL

YUV is a format in which Y is the grayscale component (luminance) and U and V are the color components. MIL supports grabbing, loading, or saving images in a YUV color format.

Although any general processing operation can be performed on YUV buffers, allocating them for processing purposes is not recommended because MIL is configured to process RGB color data only. However, MIL will automatically convert YUV buffer data to RGB for all general processing operations (including conversion for display), and re-convert it to YUV upon completion.

All YUV formats are supported even on the Host system. However, only some systems support grabbing into YUV buffers. See the MbufAllocColor() reference to determine if grabbing into YUV buffers is supported on your system.

YUV buffers must be allocated as 3-band 8-bit buffers, however, the actual number of bits per pixel will differ depending on the YUV format selected.

The supported YUV formats are:

  • YUV16 Packed.
  • YUV9 Planar.
  • YUV12 Planar.
  • YUV16 Planar.

YUV16 Packed

YUV16 Packed or YUV 4:2:2 (M_YUV16 + M_PACKED) is an interleaved data format. Although each pixel has a corresponding one byte Y (luminance component), each pair of pixels share the same one byte U (chrominance U) and the same one byte V (chrominance V). Since a pair (two pixels) is represented by 4 bytes, each pixel has an average of 16 bits per pixel.

The YUV16 packed data format has two available formats: YUYV and UYVY. The only difference between these two YUV formats is the ordering of data in the buffer. Certain digitizer boards grab data in exclusively YUYV or UYVY packed data format. Note that, for display, certain operations are optimized to handle the YUYV format; for example, displaying a decompressed buffer.

When you allocate an M_YUV16 + M_PACKED buffer, MIL allocates the buffer in the format that is most suitable for the selected system and the specified buffer attributes. You can, however, force a format using the M_YUV16_YUYV or M_YUV16_UYVY control types. When the buffer has an M_GRAB attribute, forcing an inappropriate format generates an error. When the buffer has an M_DISP attribute, if you force the buffer in the other YUV format, then CPU intervention is required to perform the automatic conversion. See the MbufAllocColor() reference for supported data formats.

The following image is an example of a YUYV buffer:

The following image is an example of a UYVY buffer:

YUV9 Planar

YUV9 Planar (M_YUV9 + M_PLANAR) is a planar format whose components have a depth of one byte but are not of the same size. Although each pixel has a corresponding 1 byte Y (luminance) component, each block of 16 pixels share the same one byte of U (chrominance U) and the same one byte of V (chrominance V). Since the 16 pixels are represented by 18 bytes, each pixel has an average 9 bits. For example, a block of 16 pixels has the following: 16 bytes Y and 1 byte each of U and V.

YUV12 Planar

YUV12 Planar (M_YUV12 + M_PLANAR) is a planar format whose components have a depth of one byte but are not of the same size. Although each pixel has a corresponding 1 byte Y (luminance) component, each block of 4 pixels share the same one byte of U (chrominance U) and the same one byte of V (chrominance V). Since the 16 pixels are represented by 24 bytes, each pixel has an average of 12 bits. For example, a block of 16 pixels has the following: 16 bytes Y and 4 bytes each of U and V.

YUV16 Planar

YUV16 Planar (M_YUV16 + M_PLANAR) is a planar format whose components have a depth of one byte but are not of the same size. Although each pixel has a corresponding 1 byte Y (luminance) component, each block of 2 pixels share the same 1 byte of U (chrominance U) and the same 1 byte of V (chrominance V). Since the 16 pixels are represented by 32 bytes, each pixel has an average 16 bits. For example, a block of 16 pixels has the following: 16 bytes Y and 8 bytes each of U and V.

YUV24 Planar

YUV24 Planar (M_YUV24 + M_PLANAR) is an uncompressed planar format whose components have a depth of one byte and are of equal size. Each pixel has a corresponding 1 byte Y (luminance) component, 1 byte U component (chrominance U), and 1 byte V component (chrominance V). Since the 16 pixels are represented by 48 bytes, each pixel has an average 24 bits. For example, a block of 16 pixels has the following: 16 bytes Y and 16 bytes each of U and V.

Child YUV buffers

You can create child buffers from YUV buffers in the same way as RGB child buffers. When creating YUV child buffers, MIL will keep the proportions of the U and V bands with respect to the Y band. For example, if your YUV9 Planar Y band is a size of 256 x 256 pixels, the U and V bands will be 1/4 the size of the Y band in each dimension (width and height): 64 x 64 pixels, which is 1/16 the size of the Y band. If a child buffer is 16 x 16 pixels, then the U and V bands will be 4 x 4 pixels. In other words, the 4 x 4 U and V bands (16 pixels) is 1/16 the size of the Y band (256 pixels).

YUV16 PACKED YUV16 PACKED YUV16 YUYV YUV16 UYVY GRAB DISP YUV9 PLANAR YUV12 PLANAR YUV16 PLANAR YUV24 PLANAR


Accessing a MIL buffer directly

Reference:
MbufGet
MbufInquire
MbufPut
Available in MIL-Lite

Available in MIL

If needed, a MIL buffer's contents can be accessed directly. For instance, if you want to calculate the average value of the pixels of your image, you could create a custom algorithm. The algorithm could be applied directly to the buffer without having to copy the contents of the MIL buffer into a user-allocated array (MbufAlloc...()) by using MbufGet() and MbufPut(). To do so would be more efficient and might improve the performance of the custom algorithm.

In order to access the MIL buffer directly, the buffer's address and pitch must be known. Once you know this, you will be able to access them directly for optimum performance.

The address of a parent or child buffer can be returned using MbufInquire(). Selecting M_HOST_ADDRESS will return a logical address, while M_PHYSICAL_ADDRESS will return a physical address. In either case, the first address of the buffer you are specifying will be the top left-most pixel in the image. Knowing the pitch and the depth of the buffer will tell you the address of the following row.

The pitch of a buffer is the number of units between the beginnings of any two adjacent lines of the buffer's data and can be measured in pixels or bytes. Note that in some instances, the pitch in bytes will be more accurate than in pixels. If the last pixel falls outside of a 32-bit boundary (required by Windows), the start of the next row will be located at the beginning of the next 32-bit boundary; this process is called internal padding. When measuring the pitch in pixels, the padding can be counted as "extra" pixels, depending on the depth of the pixels. This will result in an inaccurate pitch.

HOST ADDRESS PHYSICAL ADDRESS


Mapping a data buffer to user-allocated memory

User Guide:
Data manipulation with multiple systems

Reference:
MbufControl
MbufCreate2d
MbufCreateColor
MbufInquire
Available in MIL-Lite

Available in MIL

Instead of allocating new memory to a buffer using MbufAlloc...(), you can create a buffer from the memory at a specified location, using MbufCreate2d() to create a monochrome data buffer and MbufCreateColor() to create a color data buffer. In these cases, MIL does not allocate any memory; it uses the memory that you provide.

When creating a buffer with MbufCreateColor(), you can pass an array of data pointers. For packed color buffers, you can pass an array of one pointer; for planar buffers, you can pass an array with the same number of pointers as the number of bands in the buffer. When creating a buffer with MbufCreate2d(), you can pass the address of the data. The address(es) can be either logical or physical. If you want to use the buffer for grabbing, the address(es) must be physical (grab buffers must be allocated in physically contiguous and always present memory, that is, non-paged).

The MbufCreate...() functions must be used with caution because, when using physical addresses, these functions provide direct access to any of your computer's memory mapped devices; when using logical addresses, memory protection errors could result.

You can use MbufInquire() with the M_HOST_ADDRESS or M_PHYSICAL_ADDRESS inquire type to determine the Host's logical address or the physical address of a buffer's data, respectively. Note that the physical address is not necessarily an address in Host memory. It could be an address in on-board memory. If an on-board buffer is mapped to the Host, you can use the MbufInquire() function with the M_HOST_ADDRESS inquire type to determine the Host address to which it is mapped.

Both MbufCreate2d() and MbufCreateColor() can also be used to create a buffer that maps to an already existing buffer. When mapping to an existing buffer, pass the MIL identifier of the buffer to MbufCreate2d(), or a pointer to the MIL identifier of the buffer to MbufCreateColor(). Note that on some Matrox frame grabbers, on-board buffers are allocated in unshared memory by default. In this case, MbufCreate2d() and MbufCreateColor() are not supported, unless creating the buffer on the same system. See the Board Specific Notes for your frame grabber.

There are several instances when memory mapping is useful. A particularly useful instance is when processing and displaying an interlaced grab in a time critical application. In this case, you could use a displayable buffer to store and display the grabbed data. Then, to process each field as it is grabbed, you could use a buffer that is mapped to the odd field of the displayable buffer (Buffer 1) and a buffer that is mapped to the even field (Buffer 2).

Note that the following example does not work with a created buffer with an M_DIB storage format because, if the pitch is in pixels (MbufCreate...() with M_PITCH), the pitch must equal to the width. If the pitch is in bytes (MbufCreate...() with M_PITCH_BYTE), the pitch value must be the next multiple of 4 that is larger or equal to (SizeX * bytes per pixels).

Create Buffers 1 and 2 as follows:

Buffer 1 (Odd field):

  • Size = 640 x 240 (i.e., half height)
  • Pitch = 1280 (i.e., to skip to the next field)
  • Address = Address A (i.e., first pixel of the first row)

Buffer 2 (Even field):

  • Size = 640 x 240 (i.e., half height)
  • Pitch = 1280 (i.e., to skip to the next field)
  • Address = Address B (i.e., first pixel of the second row)

In general, MIL automatically issues a display update after a displayed buffer has been modified. However, if a buffer selected on the display is modified using a mapped buffer, its display is not updated until you notify it of the change using the M_MODIFIED setting in MbufControl().

For another instance where creating buffers is useful, see Chapter 21: Data manipulation with multiple systems.

HOST ADDRESS PHYSICAL ADDRESS HOST ADDRESS DIB PITCH PITCH BYTE MODIFIED


Pixel conventions

Availability | Previous | Next
Available in MIL-Lite

Available in MIL

The center of a pixel is important for all MIL functions which return positional results with subpixel accuracy. The reference position of a pixel is its center, and the resulting subpixel coordinates are with respect to the pixel's center.

With this in mind, the coordinates of the center of an image can always be found using the following formula:

For example, the following image contains 4 pixels. If the formula is applied, the center of the image is found at (1.5, 0).


Using buffers with the Bayer color filter

User Guide:
Correcting the white balance of your Bayer images

Reference:
MbufAlloc1d
MbufAlloc2d
MbufBayer
MbufPut1d
MbufPut2d
MdigControl
Available in MIL-Lite

Available in MIL

Cameras that feature a Bayer color filter can be used with MIL to provide a cost-effective method for grabbing color images: the camera grabs a single-band color-encoded image, and then you can convert it to a multi-band color image, using the MbufBayer() function. Bayer images are distinct from standard single-band images because of the color information contained in their pixels, which is extracted by the MbufBayer() function.

When grabbing from these cameras, each pixel quantifies only one of the color components of the image in the camera's field of view at the corresponding location. Within a group of 2x2 pixels, there are two pixels containing color information for the green component, and one pixel for each the red and blue components; Bayer images contain more green pixels because the human eye is more sensitive to this color. The pixels are arranged in the following pattern: green pixels are always diagonal to each other, as are the red and blue pixels.

The MbufBayer() function can also perform color correction, white balance correction, and gamma correction on the Bayer image during conversion. Color correction adjusts an image for color artifacts that are introduced during the demosaicing operation and that appear on the transition edges of the image. White balance correction adjusts an image for color variations that were introduced by the lighting conditions when the image was grabbed; MbufBayer() converts pixels that represent white so they appear as close to white as possible, and adjusts other pixels accordingly. Gamma correction changes the overall brightness and color saturation of an image so that the midtones appear more realistic on your display device; gamma correction effectively adjusts an image to compensate for the non-linear relationship between a pixel's value and its displayed intensity on your display device. Color correction, white balance, and gamma correction are discussed in greater detail later in this section.

Note that if the scale of the image is changed (using MdigControl() with M_GRAB_SCALE...) to a value other than 1 prior to grabbing a Bayer image, the Bayer image will not be converted property; some of the Bayer pattern is lost during the scaling process, rendering color recovery impossible.

Using MIL to convert the image

The steps below describe, in general, how to convert a Bayer image using MIL:

  1. Determine the white balance correction coefficients (optional). For information on how to calculate the coefficients, see the Correcting the white balance of your Bayer images subsection in this section.
  2. Determine the gamma correction exponents (optional). For information on how to calculate the exponents, see the Performing gamma correction subsection in this section.
  3. Grab or load a Bayer image into your source buffer.
  4. Apply the MIL Bayer filter on the image using MbufBayer(), including the white balance correction coefficients and/or gamma correction exponents, if they are being used. To use the adaptive demosaicing algorithm, you must also add M_ADAPTIVE to the ControlFlag parameter.

Below is an example of how to grab a Bayer image and convert it to a 3-band color image. This example also shows how to correct the white balance, which will be discussed later in this section.

MBufBayerColorFilter.c

How the Bayer image gets converted

Bayer images are arranged in groups of 2x2 pixels. Each group contains one blue pixel, one red pixel, and two green pixels; the values of which are used in calculating the corresponding bands of the destination pixels. Your camera will grab an image with one of the following four patterns:

You must specify the pattern that is used by your camera when calling MbufBayer(). Since the green pixels are always diagonal to each other, specifying the starting two pixels of the pattern defines the pattern uniquely. Consult your camera's documentation or contact the manufacturer if you are unsure; the Bayer image will not be converted properly if you specify the wrong pattern. If you cannot obtain information regarding the pattern of your camera, try all of the MbufBayer() function's supported patterns to find the correct one.

Once you have specified the Bayer pattern of your camera, you must consider which demosaicing algorithm to use to convert a single-band Bayer color-encoded image to a multi-band color image. There are currently two from which you can choose from: the bilinear interpolation demosaicing algorithm and the adaptive demosaicing algorithm.

Bilinear interpolation algorithm

When using the bilinear interpolation algorithm, the value of a source pixel is used in the corresponding band of its destination pixel. The two remaining color bands use the average value of the source pixel's corresponding neighbors. If the source pixel is green, then the average value for the remaining two components (red and blue) is based on two neighboring pixels.

If the source pixel is either red or blue, the average value for the remaining two components is based on four neighboring pixels.

Note that if the source pixel is on an edge of the image, MIL will use as many neighbors as possible when determining the average pixel value for the remaining components.

Regardless of the destination buffer format, MIL converts the Bayer image first to RGB, and then to the format of the destination buffer.

This is the default demosaicing algorithm used by MbufBayer().

Adaptive algorithm

The bilinear demosaicing algorithm is simple, fast, and does an effective job of demosaicing the Bayer images. However, since it uses only the neighborhood averages to determine the pixel values, details are sometimes lost and sporadic artifacts are introduced. To minimize image artifacts, such as the zipper effect and false colors, you can use the adaptive demosaicing algorithm to convert your images.

The adaptive demosaicing algorithm analyzes the surrounding neighborhood of the source pixel and assigns weights to its neighbors. MIL then uses these weights in the calculation of the destination pixel value, depending on the neighbor's relevancy to the source pixel.

To use the adaptive demosaicing algorithm, add M_ADAPTIVE to the ControlFlag parameter.

Performing color correction

Sometimes grabbed images appear with a color fringe on transition edges. To minimize these color artifacts on the edges, you can perform a color correction on the image during the Bayer conversion.

Color correction can only be used in conjunction with the adaptive demosaicing algorithm and is designed to produce very good looking vertical and horizontal edges. For best visual results, you should combine M_ADAPTIVE and M_COLOR_CORRECTION together.

The following compares the results of the two demosaicing algorithms.

Correcting the white balance of your Bayer images

Sometimes grabbed images appear with "the wrong colors". This is due primarily to color distortions introduced by the light source or lighting conditions. Such distortions can be corrected by performing a white balance correction on the image.

On the premise that white pixels should contain no chrominance, white balance correction applies a coefficient to each band of the image so that "white" pixels contain no chrominance. For RGB images, this means that a white pixel's value for all 3 bands is equal. After white balance correction, pixels that are white appear white (or a shade of gray), and the other pixels also appear with the correct colors. The result is an image that more accurately reflects the colors of the object that was grabbed.

The steps below show how to white balance Bayer images using the MbufBayer() function:

  1. Grab an image that should be a uniform shade of gray. To do so, hold a flat, smooth, non-reflective object that is a uniform shade of gray in front of a camera so that the object occupies the camera's entire field of view. The shade of gray should not be too close to white (too saturated) because if one or more color bands is actually being clipped to maximum saturation, the unbalance between the bands cannot be observed.

    2. Ensure that the image is grabbed under the same lighting conditions as subsequent source images. Note that it is unlikely that you will be able to grab an image whose pixels have the same red, green, and blue value.

  2. Allocate a 3 x 1 (or 6 x 1 if gamma is used) MIL array of type M_FLOAT using MbufAlloc1d() or MbufAlloc2d().
  3. Call MbufBayer(), using the gray image as the source image, and adding M_WHITE_BALANCE_CALCULATE to the control flag. This call calculates the three coefficients required to white balance the specified source image and writes them to the array.

    4. Alternatively, you could fill an array with custom values, and then load these values (in the order described below) into the MIL array using MbufPut1d() or MbufPut2d().

  4. Grab the image required for the Bayer conversion.
  5. Call MbufBayer(), using the new source image and the MIL array with the white balance coefficients.

When MbufBayer() calculates the coefficients, the three white balance correction coefficients, a, b, and c, are calculated and written to the array as the first, second, and third values, respectively. These coefficients are for a given lighting condition, and calculated such that given an image of a flat gray surface in that lighting:

where R, G, and B with macrons (¯) are the average values of the intermediate red, green, and blue color components, respectively. When source images are converted, the intermediate color components of each pixel are multiplied by their corresponding coefficient. Regardless of the destination buffer format, white balance correction is performed on the intermediate RGB values recovered by the demosaicing process (before being converted into the proper destination format).

Performing gamma correction

Sometimes the colors of grabbed images can appear to be over-saturated or too dark due to the display device used. Most display devices receive your image as an analog video signal. The voltage range of the active portion of the video signal is divided linearly among the possible pixel value range. However, the intensity to voltage response ratio of the display device is roughly a power function (intensity = voltage gamma), whereby the exponent is rarely 1 (for many display devices, it is 2.5). This means that there is typically a non-linear relationship between a pixel's value and its displayed intensity on your display device. Therefore, images can look either bleached out or too dark.

The exponent of the power function of each display device is referred to as the gamma of the display device. You can compensate for the gamma of your display device by performing gamma correction on your images. That is, you can apply a power function that uses a 1/gamma exponent, on each band of your images. The exponents must be positive, are typically in the range of 0.0 to 1.0, and are most commonly 1/2.5 = 0.4.

If you pass the required exponents to the MbufBayer() function, the function can perform gamma correction on the intermediate RGB values recovered by the demosaicing process (just after performing the white balance correction operation and before converting the image into the proper destination format). Pass the exponents in the forth, fifth, and sixth elements of the MIL array buffer.

ADAPTIVE ADAPTIVE ADAPTIVE COLOR CORRECTION FLOAT WHITE BALANCE CALCULATE


Buffer overscan region

See Also | Availability | Previous
Reference:
MbufControlNeighborhood
MbufInquire
MimConvolve
MimEdgeDetect
MimErode
MsysControl
MsysInquire
Available in MIL-Lite

Available in MIL

Buffer overscan pixels are pixels added around an image buffer. MIL neighborhood operation, such as MimConvolve(), uses these pixels to perform operations that go beyond the limits of the buffer. For example, in the image below, if MimConvolve() is called with the M_HORIZ_EDGE FIR filter, which operates on 3 by 3 neighborhoods, the neighborhoods of the image border pixels would be incomplete. In such cases, overscan pixels might be used. MIL recursive neighborhood operations, such as MimConvolve() called with the M_SHEN_FILTER IIR filter, ignore the values of overscan pixels.

To set the values of overscan pixels, use the MbufControlNeighborhood() function. However, some MIL neighborhood operations with predefined kernels or structuring elements do not require that you set the values of overscan pixels. For example, if you call MimErode(), the overscan pixels are automatically set to the highest possible buffer value, which will produce the most accurate possible results for the image border pixels.

The number of overscan pixels added around an image buffer during allocation is set using MsysControl(). The set size represents the number of 1-pixel high/wide rows/columns added to each side of the image buffer. For example, in the image above, the size of the overscan is 2. Each buffer allocated on a particular system will have the same overscan size. You can inquire the size of the overscan using either the MbufInquire() or MsysInquire() function.

When the results of operations performed on image border pixels are not important, and you want to speed up your application, you can use the MbufControlNeighborhood() function to disable the use of overscan pixels. Also, both MimConvolve() and MimEdgeDetect() allow you to disable the use of overscan pixels for pre-defined kernels.

HORIZ EDGE


Lookup tables

Chapter 15: Lookup tables
Availability
Available in MIL-Lite

Available in MIL

This chapter describes lookup tables (LUTs). It shows you how to generate and modify them and briefly discusses how to use them.

  • Lookup tables in general
  • LUTs and data buffers
  • Loading and generating data into LUTs
    • Generating data directly into the LUT buffer
    • Loading LUTs with precalculated data
  • Using LUTs


Lookup tables in general

See Also | Availability | Next
Reference:
MimLutMap
Available in MIL-Lite

Available in MIL

Lookup tables (LUTs) are collections of memory locations that are used to map data to pre-calculated values. They can easily reduce a multi-step or complex operation to a single-step LUT mapping.

You can map an image buffer through a LUT, using MimLutMap(). If the hardware system permits, you can also use LUTs to precondition input data at acquisition time, before it is stored in an image buffer. LUTs can also be used (hardware system permitting) to adjust the color contrast and intensity of an image upon display, without affecting the actual data.


LUTs and data buffers

Reference:
MbufAlloc1d
MbufAllocColor
MbufFree
Available in MIL-Lite

Available in MIL

The MIL package represents LUTs as LUT data buffers. As with any other data buffer, LUT buffers must be allocated before they are used. A LUT buffer can be loaded, stored, or copied to another buffer (not necessarily to another LUT buffer) or to disk. You can also allocate child LUT buffers. When a LUT buffer is no longer required, you should free its memory space, using MbufFree().

LUT buffers are typically one-dimensional data buffers created with MbufAlloc1d() (single row). However, you can allocate a color RGB LUT, using MbufAllocColor(). In this case, set the number of bands to 3 (for RGB), the Y-dimension to 1, and the X-dimension to have enough entries to represent the full range of possible values of the image buffer.


Loading and generating data into LUTs

Reference:
MbufCopy
MbufCopyColor
MbufLoad
MbufPut
MbufPut1d
MbufPutColor
MbufRestore
MgenLutFunction
MgenLutRamp
MimArith
MimHistogramEqualize
Available in MIL-Lite

Available in MIL

With MIL, you can generate data directly into a LUT buffer or calculate the data and then load it in a LUT buffer.

Generating data directly into the LUT buffer

You can generate general data directly into a LUT buffer, using MgenLutRamp() or MgenLutFunction().

The MgenLutRamp() function generates a value for each LUT index within the specified index range. The difference between the start value and the end value divided by the number of entries specified by the index range produces the increment. The increment is then used to generate the remaining entries of the index range.

If the increment is positive, MgenLutRamp() generates a ramp. If the increment is negative, the function generates an inverse ramp. If the increment is equal to zero, it loads the entire LUT range with the given start value.

The MgenLutFunction() function generates a value for each LUT index within the specified index range according to a specified mathematical function. The functions available are: M_LOG, M_EXP, M_SIN, M_COS, M_TAN, and M_QUAD. The specified start value is used as the initial X value in the equation. The remaining entries of the index range are generated by incrementing the value of X by 1 for each index.

The MimHistogramEqualize() function can be used to create a LUT for intensity correction.

When generating data in a color LUT buffer, the same data is written to all bands.

To load each color band with different data, you would have to generate the data into three separate one-dimensional LUT buffers, then copy each buffer to the appropriate color band of the color LUT buffer, using MbufCopyColor().

Alternatively, you can allocate three separate one-dimensional child buffers into which the values for each color band will be generated. The use of child buffers will cause the values for each color band in the LUT buffer to be automatically updated and no copying is necessary.

Loading LUTs with precalculated data

There are several ways to generate more complex LUTs. Most of these, however, involve pre-calculating the data, then loading it into the LUT buffer:

  • Calculate data, using your Host system, and then load it into the LUT, using MbufPut(), MbufPut1d(), or MbufPutColor().
  • Generate data into another data buffer, using MIL functions other than MgenLutRamp() (for example, using the MimArith() function and perhaps the histogram of the image), then copy the data to the LUT buffer, using MbufCopy() or MbufCopyColor().
  • Load previously saved LUT data from disk to the LUT buffer (MbufLoad()). Note, when loading data from disk, there should be enough data for each dimension of the LUT buffer.
  • Restore a previously saved LUT, using MbufRestore(). Note, this function actually performs the LUT buffer allocation.
LOG EXP SIN COS TAN QUAD


Using LUTs

Availability | Previous
Available in MIL-Lite

Available in MIL

In MIL, LUTs can be used in different circumstances:

  • When performing certain processing operations.
  • When displaying data (if supported by hardware).
  • When acquiring data from a digitizer (if supported by hardware).

For details, refer to the chapters that discuss image processing, displays, and acquiring data with your digitizer.


Displaying an image

Chapter 16: Displaying an image
Availability
Available in MIL-Lite

Available in MIL

This chapter discusses the display of image buffers, in detail. It shows you how to display several images simultaneously, and discusses some of the special effects that can be applied to a displayable image buffer.

  • Overview
  • Types of displays
    • Windowed display
    • Auxiliary display
    • Display size and depth
    • Displaying buffers of different data depths
      • M_BIT_SHIFT
      • M_AUTO_SCALE
      • M_MULTI_BYTES
      • M_TRANSPARENT
  • Removing a buffer from the display
  • Displaying multiple buffers
    • Display number
  • Screen tearing
  • Panning and zooming
  • Annotating the displayed image non-destructively
    • Annotating images using the overlay-display mechanism
    • Overlay buffer behavior
    • CPU-assisted overlay
    • Transparency (Keying)
    • Using GDI annotations
  • Displaying an image in a user-defined window
    • Using MdispSelectWindow()
  • Palettes and output LUTs for windowed display (256-color)
    • Reference material: Windows palettes and physical output LUTs
      • Color images
      • Palettes and physical output LUTs
      • Monochrome images
    • Default palette settings
      • Why change the default LUT values
    • Changing the default LUT values
      • Viewing each gray intensity in a different color
      • A 1-band custom LUT buffer
      • A 3-band custom LUT buffer
    • When using physical output LUTs
  • CPU-assisted display
    • Overlay flickering
    • Grabbing continuously
    • Avoiding CPU-assisted displays


Overview

See Also | Availability | Next
Reference:
MappAllocDefault
MdispAlloc
MdispSelect
Not available in MIL-Lite

Available in MIL

Whether or not your imaging board has a display section, MIL can display images. It will use the most appropriate graphics controller in your computer for display purposes. If your imaging board has a display section, and it is available, MIL will typically use it for display purposes.

To display an image buffer, the buffer must have been allocated with a displayable attribute (M_DISP). In addition, a display must have been allocated using MdispAlloc() or MappAllocDefault(). Note that the buffer and the display should be allocated on the same system.

Once a buffer and a display have been allocated, use MdispSelect() to select the image buffer to display. The buffer is displayed in a dedicated window or at the top-left corner of an auxiliary screen. If the specified image buffer is smaller in size than the display, the border outside the buffer is blacked out. If the specified image buffer is larger in size than the display, the right and bottom part of the buffer, the part that exceeds the display size, is not displayed.

Note that a buffer, or any of its child regions, can be selected on more than one display.

This manual uses the term frame buffer to refer to physical display (graphics controller) memory (not a buffer, per se).

DISP


Types of displays

Reference:
MdispAlloc
MdispControl
Not available in MIL-Lite

Available in MIL

You can allocate a display that, when an image is selected to this display, it is either displayed:

  • With a windowed border, which is called a windowed display (M_WINDOWED).
  • Without a windowed border on a dedicated screen, which is called an auxiliary display (M_AUXILIARY).

You must specify either one of these two types of displays upon allocating the display, with MdispAlloc().

Typically, the default display type (M_DEFAULT) is M_WINDOWED. However, a Matrox imaging board might be dedicated for MIL auxiliary display, which can make the default M_AUXILIARY. For more information, see the board's installation and hardware reference manual.

Windowed display

An image selected to a windowed display is displayed with a windowed border on the Windows desktop screen(s). To choose a windowed display, set the initialization flag parameter of MdispAlloc() to M_WINDOWED.

A windowed display is not affected by whether or not your desktop is displayed using one screen or multiple screens. We refer to these screens, either one or many, as the Windows desktop screen(s). Under Windows 2000/XP your desktop can be extended over screens of different resolutions.

All windowed displays (M_WINDOWED) are displayed in their own MIL default window (or, as will be seen later, in a user-allocated window). This window is transparently tracked and updated with the image buffer selected to the display; that is, if the window moves or is occluded, the window is automatically updated with the image buffer accordingly.

Multiple windowed displays can be allocated and selected for display; the display's device number should always be set to M_DEFAULT.

For windowed displays, MIL does not typically communicate directly with the graphics controller, but uses the normal Windows mechanisms (Windows API functions and extensions) to display images. Upon selecting a windowed display, MIL allocates a second image buffer in a Windows Device Independent BITMAP (DIB) or DirectDraw format, passes Windows the address of this buffer, and copies the content of the selected buffer to this displayed buffer. MIL also loads display LUT buffers into Windows' logical palettes (only in 256 color display resolution). Refer to the Microsoft SDK Programming Guide for information on Windows' DIBs, DirectDraw, and logical palettes.

Auxiliary display

An image selected to an auxiliary display is displayed without a windowed border or frame, at the top-left corner of a screen that is not used to display the Windows desktop. This screen is referred to as an auxiliary screen.

Note that in this context, the term screen refers to any device that supports video output data, such as a high-resolution monitor, TV, or VCR.

Auxiliary displays are supported with a minimum of one of the following resources:

  • Two graphics controllers (one to drive your Windows desktop screen and one to drive your auxiliary screen).
  • One Matrox graphics controller (for example, Matrox G400, G450, or G550) that integrates two video output controllers, one to drive your Windows desktop screen and one to drive your auxiliary screen.

    • Note that the graphics controller can be on, or apart from, the Matrox imaging board. Also, some boards might have special features or limitations regarding auxiliary displays; see the MdispControl() reference for details.

You can only allocate one auxiliary display at a time on a given auxiliary screen. Moreover, you are responsible for moving and tracking an auxiliary display, if required. To choose this type of display, set the initialization flag parameter of MdispAlloc() to M_AUXILIARY.

When using an imaging board with a display section (for example, Matrox Orion), it might be necessary to set a Dip switch to display on an auxiliary screen. For more information, see the board's installation and hardware reference manual.

When allocating an auxiliary display, MIL imposes restrictions on the video output format of the auxiliary screen depending on the presence of an encoder in the on-board display section of the Matrox imaging board.

When allocating an auxiliary display, MIL does not impose any restrictions on the video output format of the auxiliary screen.

  • If the auxiliary display does not use an encoder, the display format is set to a high resolution format (for example, 1024x768x32@70hz).
  • If the auxiliary display does use an encoder in the on-board display section of the Matrox imaging board, the display format is set to an encoded video format (for example, NTSC/PAL).

The video output format of the auxiliary screen is set with the display format parameter of MdispAlloc(). For example:

MdispAlloc(MilSystem, M_DEFAULT, "M_NTSC", M_AUXILIARY, &MilDisplay1);
MdispAlloc(MilSystem, M_DEFAULT, "1024x768x32@70", M_AUXILIARY, &MilDisplay2);

For all the supported formats, see the description of the DispFormat parameter of MdispAlloc().

The maximum number of auxiliary displays that can be allocated is determined by the number of graphics controllers that support the specified format. For example, two auxiliary displays with high-resolution formats can only be allocated if there are two available graphics controllers that support high-resolution formats.

When allocating a display, MIL checks for an appropriate graphics controller, and not an appropriate device attached to it. It is therefore possible to allocate an auxiliary display without an auxiliary screen connected to the video output controller.

To use the second video output controller of Matrox Millennium G400, G450, or G550 for MIL auxiliary display, your display driver's DualHead mode must be disabled; otherwise both the display driver and the MIL driver will attempt to access the second video output controller, and the auxiliary display will not work. In addition, the Matrox Millennium G400, G450, and G550's second video output controller does not support encoded video format.

For auxiliary displays, the display device number should always be set to M_DEFAULT.

Display size and depth

For windowed displays, the display format of the on-screen portion of the display memory (frame buffers) is set using the selected Windows display resolution. In this case, the display format parameter of MdispAlloc() is ignored. When you select a buffer to a windowed display, Windows will create a display of the same size as the buffer, unless such a display cannot fit in the Windows desktop. If the image is too large, there will be scroll-bars to view other parts of the image, and the initial view of the image will be the upper-left corner. If the image is too small, it will be centered in the display, and the surrounding area will be blacked out.

For auxiliary displays, you set the display format with the display format parameter of MdispAlloc().

Displaying buffers of different data depths

Displayable image buffers usually have a depth of 8 bits (or 8 bits per band, in the case of color images). You can also display images of other depths (for example, 1-bit or 16-bit images). Using MdispControl() with M_VIEW_MODE, you can control the way such buffers are actually displayed.

The M_VIEW_MODE control type provides different modes of displaying non 8-bit images:

  • M_BIT_SHIFT
  • M_AUTO_SCALE
  • M_MULTI_BYTES
  • M_TRANSPARENT
  • M_DEFAULT

M_BIT_SHIFT

The M_BIT_SHIFT setting will bit-shift the pixel values of the image by the specified number of bits upon updating the display.

M_AUTO_SCALE

The M_AUTO_SCALE setting remaps the pixel values to the display range such that the minimum and maximum values in the image (not the full range of the buffer) are set to 0 and 255, respectively.

If the image buffer contains a single value, its corresponding displayed value is determined by linearly re-mapping the full range of the buffer; for example, (0 to 64K) to (0 to 255).

M_MULTI_BYTES

The M_MULTI_BYTES setting is primarily useful when grabbing from a multi-tap camera. This setting displays each byte of the image in separate display pixels. For instance, each pixel of a 16-bit image will occupy two consecutive display pixels; each pixel of a 32-bit image will occupy four consecutive display pixels. This mode is only supported for 16-bit and 32-bit 1-band image.

M_TRANSPARENT

The M_TRANSPARENT setting will display only the 8 least-significant bits of the image. No pixel remapping will be performed. This is the default setting.

WINDOWED AUXILIARY DEFAULT WINDOWED AUXILIARY WINDOWED WINDOWED DEFAULT AUXILIARY DEFAULT VIEW MODE VIEW MODE BIT SHIFT AUTO SCALE MULTI BYTES TRANSPARENT DEFAULT BIT SHIFT AUTO SCALE MULTI BYTES TRANSPARENT


Removing a buffer from the display

Reference:
MdispFree
MdispSelect
Available in MIL-Lite

Available in MIL

After displaying an image buffer (with MdispSelect()), you can remove it from the display and close the associated window (for windowed displays), or leave the display blank (for auxiliary displays), using MdispSelect() with M_NULL. To display a different image buffer, you are not required to remove the current buffer from the display; selecting another buffer for display automatically updates the display with the new buffer.

Once you have finished using a display, you should free it, using MdispFree(). If a displayed buffer is freed, the buffer is automatically removed from the display for windowed displays and is left blank for auxiliary displays.


Displaying multiple buffers

Reference:
MbufAlloc2d
MbufAllocColor
MbufChild2d
MbufChildColor
MdispAlloc
MdispSelect
Available in MIL-Lite

Available in MIL

MdispSelect() allows you to view one buffer at a time in one display. You can, however, use many windowed displays and therefore view more than one buffer at the same time on the Windows desktop screen(s).

This is not the case for auxiliary displays, where you can only display one display at a time on a given auxiliary screen. However, you can still view more than one buffer at a time using child buffers. For example, you can display the source and destination buffers of an operation, using the following steps:

  1. Allocate a large displayable buffer using MbufAlloc2d() or MbufAllocColor(). This buffer will be known as the parent buffer.
  2. Select the parent buffer for display using MdispSelect().
  3. Allocate two non-overlapping child buffers within it, using MbufChild2d() or MbufChildColor().
  4. Use one of the child buffers as the source image buffer and the other as a destination image buffer of the operation.

The following example shows how to display multiple buffers in a single display. The source bird image, Bird.mim, is loaded into a child of a displayable buffer and then used as the source of an image processing operation (increasing the image luminance). The result is stored in another child of the same displayable buffer.

MBufColor.c

For MIL-Lite, the luminance operation is not performed; the image is merely copied from the left child buffer to the right.

Display number

The display number parameter of MdispAlloc() should always be set to M_DEFAULT. Based on the specified format, MIL will find the best device to use when displaying an image. If your imaging board has a display section, and it is available, MIL will typically use it for display purposes.

DEFAULT


Screen tearing

Reference:
MdigHalt
MdispControl
MdispInquire
Available in MIL-Lite

Available in MIL

Screen tearing occurs when display memory (also called frame buffer memory) and the screen are not updated synchronously. The updates to display memory and the screen might not be synchronized because the rate at which display memory is updated is either slower or faster than the rate at which the screen is refreshed. Alternatively, the updates might occur at the same rate but are not in phase with each other. The non-synchronous update causes two images to temporarily appear simultaneously. For example:

MIL supports displays with no tearing. Use MdispControl() with M_NO_TEARING to set the no-tearing mode. This mode requires special hardware, such as a Matrox Millennium G400, G450, or G550 graphics controller. Note that if the M_NO_TEARING control type is used and is not supported, MIL will not generate an error; instead, it will compensate by using the only mode available which is the mode in which tearing can occur. To determine if the graphics controller is currently implementing a specified display with no tearing, use MdispInquire() with M_NO_TEARING_ACTIVE. To determine the no-tearing mode specified using MdispControl() with M_NO_TEARING, use MdispInquire() with M_NO_TEARING.

To specify that all updates made to the display are done with a no-tearing mechanism, set the M_NO_TEARING control type to M_BASIC.

To use the no-tearing mechanism only when displaying a continuous grab, set the M_NO_TEARING control type to M_GRAB_CONTINUOUS_ONLY. In this case, the graphics controller does not use the no-tearing mechanism for all other display updates (that is, updates that are necessary due to modifications made to the display's overlay buffer or the buffer selected to the display). Note that in this case, MdispInquire() with M_NO_TEARING_ACTIVE returns M_YES only during a continuous grab, indicating that the graphics controller is updating the screen with a no-tearing mechanism. After MdigHalt() is called, MdispInquire() with M_NO_TEARING_ACTIVE returns M_NO.

Note that M_BASIC and M_GRAB_CONTINUOUS_ONLY compensate only when the rate at which display memory is updated is equal to or slower than the rate at which the screen is refreshed. Therefore, screen tearing might still occur when MdispInquire() with M_NO_TEARING_ACTIVE returns M_YES.

M_NO_TEARING with M_ADVANCED is similiar to M_NO_TEARING with M_BASIC in that all updates made to the display are done with a no-tearing mechanism. However, M_ADVANCED ensures that screen tearing does not occur even when display memory is updated faster than the rate at which the screen is refreshed. In this case, when more than one update to display memory occurs between two vertical synchronization pulses of the screen, M_ADVANCED uses multiple buffers to queue the updates. Note that during a vertical synchronization pulse, only one update is copied to the screen. By default, every update in the queue is copied to the screen; this results in a lag in the time it takes for the updates to be displayed. To avoid this lag, MIL can skip the oldest updates in the queue and copy only the newest update to the screen during each vertical synchronization pulse. To do so, add M_SKIP_OLDEST to the M_ADVANCED setting. Alternatively, you can have MIL skip the newest updates in the queue and copy the oldest update to the screen during each vertical synchronization pulse. To do so, add M_SKIP_NEWEST to the M_ADVANCED setting.

If the M_NO_TEARING control type is set to M_ENABLE, MIL will choose the no-tearing mode which is optimal for the current situation (varies depending on such factors as the board used and CPU availability).

When an encoder is displaying a continuous grab on an auxiliary display, the default setting for the M_NO_TEARING control type is M_ENABLE; otherwise, the default setting is M_DISABLE which specifies not to use a no-tearing mechanism to update the display.

NO TEARING NO TEARING NO TEARING ACTIVE NO TEARING NO TEARING NO TEARING NO TEARING NO TEARING ACTIVE NO TEARING ACTIVE NO TEARING ACTIVE NO TEARING NO TEARING NO TEARING NO TEARING


Panning and zooming

Reference:
MdispControl
MdispPan
MdispZoom
Available in MIL-Lite

Available in MIL

At times, your image buffer might be larger than the display, or have details that are too fine or too small to see. Display effects can be associated with the display to view specific parts of the image. These effects are panning and zooming.

Panning and scrolling displace an image horizontally or vertically, respectively, on the display. You can pan and scroll your image to display the appropriate location at the top-left corner of the window (for windowed displays) or of the screen (for auxiliary displays), using MdispPan().

For auxiliary displays, to view the image at another location on the display, you must create a large displayable image buffer, display it, and then allocate and use a child buffer at the required location on the display. For windowed displays, just move the window to the required location on the screen.

Zooming is the horizontal and/or vertical replication of each pixel by a given factor. You can zoom the display, with floating point precision, using MdispZoom(). For an example, refer to MImConvolve.c, where the display is zoomed by a factor of 2 to better demonstrate the result of the image processing operation (spatial filtering operation).

Note that zooming by a large factor might cause a "blocky" effect. Also, you can reduce the size of an image on the display by passing a negative zoom factor to MdispZoom().

Zooming uses an interpolation mode that can be specified using MdispControl() with M_INTERPOLATION_MODE. By default, zooming uses the fastest interpolation mode.

INTERPOLATION MODE


Annotating the displayed image non-destructively

User Guide:
Transparency (Keying)

Reference:
MbufClear
MbufControl
MbufInquire
MdispAlloc
MdispControl
MdispInquire
MgraText
Available in MIL-Lite

Available in MIL

Annotating images using the overlay-display mechanism

For either windowed displays or auxiliary displays, you can annotate the displayed image non-destructively using MIL's overlay-display mechanism. To make use of this functionality, do the following:

  1. Enable the overlay-display mechanism using the following function call:
    2. MdispControl(DisplayID, M_OVERLAY, M_ENABLE);
  2. Select a buffer to the display:
    3. MdispSelect(DisplayID, ImageBufId);

    Since the overlay-display mechanism is enabled, this will not only display the selected image, but it will also associate a temporary overlay buffer with the display. This buffer is referred to as the display's overlay buffer. The overlay buffer can be used to annotate the underlying image with an effect called transparency, or keying. For more information, see Transparency (Keying).

  3. To access the display's overlay buffer, use MdispInquire() with M_OVERLAY_ID to determine the MIL identifier of the buffer:
    4. MdispInquire(DisplayID, M_OVERLAY_ID, &OverlayBufferID);
  4. Annotate the display's overlay buffer as required. For example, to write text in the overlay buffer, you can use MgraText(). Note that since this temporary overlay buffer is a real buffer, any function (except grabbing) can be used.

    5. You can also annotate the displayed image buffer with Windows GDI annotations. For information, see the Using GDI annotations subsection in this section.

Typically, for a non 8-bit display resolution (15-bit, 16-bit, 24-bit, or 32-bit color resolution), the overlay buffer will have 3 bands and will be the same size as the buffer selected to the display (not the size of the display). For an 8-bit display resolution, the overlay buffer will be 1 band.

Overlay buffer behavior

When an image is selected to a display that has an associated overlay buffer, and you select another image to that display, one of the following occurs:

  • If the new image has the same format and size as the image currently selected to that display, the current overlay buffer is not freed. Any annotations will, therefore, remain until you clear the overlay buffer, with MbufClear().
  • If the new image has a different format or size than the image currently selected to that display, the current overlay buffer is freed, and another overlay buffer is created. The new overlay buffer has the same size as the image selected to the display, and the annotations of the old overlay buffer are copied into the new one.

    • The identifier of the old overlay buffer is now obsolete. To inquire the identifier of this newly created overlay buffer, use MdispInquire() with the M_OVERLAY_ID inquire type.

CPU-assisted overlay

The ability to annotate the displayed image non-destructively using MIL's overlay-display mechanism is always available, and is typically accomplished by your hardware (that is, your graphics controller). However, if your hardware limits have been reached, MIL produces a simulated version of the overlay effect using the CPU; the display is therefore said to be CPU-assisted.

Transparency (Keying)

The display's overlay buffer annotates the image selected to the display using transparency. Transparency, also known as keying, is a mechanism that replaces pixels of an image that are of the specified transparency (keying) color with the underlying areas of another image. Using transparency, annotations made to the overlay buffer in a color other than the transparency color will annotate the image selected to the display.

When allocating a display (MdispAlloc()), the transparency color is automatically set to a default color, which is generally appropriate. Use MdispInquire() with M_TRANSPARENT_COLOR to determine the transparency color. If required, select another transparency color using MdispControl() with M_TRANSPARENT_COLOR.

If you are using an 8-bit display resolution (256 colors), set the transparency color to a value between 0 and 255. If you are using a non 8-bit display resolution (15-bit, 16-bit, 24-bit, or 32-bit color resolution), call the macro M_RGB888 and specify the RGB value. For example:

MdispControl(DisplayID, M_TRANSPARENT_COLOR, M_RGB888(20,32,24));

When the display's overlay buffer is created, it is cleared to the effective transparency color. If the transparency color is changed after the overlay buffer is created, the buffer will not be cleared to the new color.

Using GDI annotations

Once the overlay-display mechanism is enabled and an image buffer is selected to the display, you can also annotate the image buffer selected to the display with Windows GDI annotations, if either the selected buffer or the overlay buffer is GDI compatible (using MbufAlloc...() with M_GDI or MdispAlloc() with M_GDI_OVERLAY, respectively). This can be done using one of the techniques described below.

The first technique is:

  1. Allocate a device context (DC) for either the image buffer or the overlay buffer of the display using MbufControl() with M_DC_ALLOC. Windows GDI annotation functions require a DC to draw. If you allocate a DC for the image buffer and then draw using this DC, drawing will be destructive (that is, the data of the image buffer is actually changed).

    2. Note that the DC can only be allocated for a buffer that is internally stored in M_GDI format, otherwise, an error is reported. The display's overlay buffer can be forced to be compatible with GDI operations if the display is allocated using MdispAlloc() with M_GDI_OVERLAY.

  2. Inquire the identifier of the device context (DC) created in the previous step using MbufInquire() with the M_DC_HANDLE inquire type.
  3. Annotate the displayed image buffer using Windows GDI annotation functions.
  4. Free the device context (DC), using MbufControl() with the M_DC_FREE control type.
  5. Notify MIL that the display needs to be updated with the GDI annotations, using MbufControl() with M_MODIFIED.

The following portion of MIL code shows the creation of the device context for the overlay buffer, the inquiring of the device context, and the drawing and writing in the overlay buffer (see also, MDispOverlay.c).

HDC hCustomDC;
HGDIOBJ hpen, hpenOld;
char chText[80];
/* Create a device context for drawing. */
MbufControl(MilOverlayImage, M_DC_ALLOC, M_DEFAULT);
/* Inquire the handle of the device context. */
hCustomDC = ((HDC)MbufInquire(MilOverlayImage, M_DC_HANDLE, M_NULL));
if (hCustomDC)
{
/* Create a blue pen. */
hpen=CreatePen(PS_SOLID, 1, RGB(0, 0, 255));
hpenOld = SelectObject(hCustomDC,hpen);

/* Draw a cross in the overlay buffer. */
MoveToEx(hCustomDC,0,ImageHeight/2,NULL);
LineTo(hCustomDC,ImageWidth,ImageHeight/2);
MoveToEx(hCustomDC,ImageWidth/2,0,NULL);
LineTo(hCustomDC,ImageWidth/2,ImageHeight);

/* Write text in the overlay buffer. */
strcpy(chText, "GDI Overlay Text ");
SetTextColor(hCustomDC,RGB(0, 0, 255));
TextOut(hCustomDC,ImageWidth*3/18,ImageHeight*4/6, chText,
strlen(chText));
SetTextColor(hCustomDC,RGB(255, 0, 0));
TextOut(hCustomDC,ImageWidth*12/18,ImageHeight*4/6, chText,
strlen(chText));
/* Deselect and destroy the blue pen. */
SelectObject(hCustomDC,hpenOld);
DeleteObject(hpen);
}

/* Free the created device context. */
MbufControl(MilOverlayImage, M_DC_FREE, M_DEFAULT);

/* Signal MIL that the overlay buffer was modified. */
MbufControl(MilOverlayImage, M_MODIFIED, M_DEFAULT);

The second technique is:

  1. Inquire the display's Windows handle using MdispInquire() with the M_WINDOW_HANDLE inquire type.
  2. Pass the window handle to the Windows GetDC() function to get a Windows display device context (DC).
  3. Then, paint the annotations with GDI functions each time Windows receives a paint message (for example, WM_PAINT).
OVERLAY ID OVERLAY ID TRANSPARENT COLOR TRANSPARENT COLOR GDI OVERLAY DC ALLOC GDI GDI OVERLAY DC HANDLE DC FREE MODIFIED WINDOW HANDLE


Displaying an image in a user-defined window

Reference:
MdispSelect
MdispSelectWindow
Available in MIL-Lite

Available in MIL

For windowed displays, images are automatically displayed in a default window created by MIL, using MdispSelect(). This function dynamically creates a window on the Windows desktop for the specified display, if the display is not already selected. The created window respects any window control that has been associated with the display using an Mdisp...() function.

However, for windowed displays, you can choose to display a specific image buffer in a user-defined window, using MdispSelectWindow(). Note that typically, the display need not have the same resolution as the image buffer. If the defined window is of a different dimension than the image buffer, any excess window area will be left untouched or any excess image area will be cropped.

Using MdispSelectWindow()

The MdispSelectWindow() function is similar to MdispSelect(), except that it allows you to specify the handle of the user-defined window or child window to use for display, rather than displaying into a MIL created window. This user-defined window is automatically refreshed when the display is modified (for example, when the image data is modified). You can use MdispSelect() with M_NULL to remove the image from the display.

Note that the user-defined window must have been created with Windows API functions. In addition, if the handle parameter of MdispSelectWindow() is set to zero, this function behaves like MdispSelect().

The following example shows how to display an image in a user-defined window, grab into such a window, and remove the image from the display.

MDispWindow.c


Palettes and output LUTs for windowed display (256-color)

Reference:
MbufAlloc1d
MbufControl
MbufPut
MdispInquire
MdispLut
MgenLutRamp
Available in MIL-Lite

Available in MIL

For windowed displays, when displaying in a 256-color Windows display resolution, images are mapped through physical output LUTs on display. Windows uses these LUTs to achieve color; in addition, Windows uses the concept of a palette to load these LUTs.

MIL provides Windows with good default logical palettes for the realization of the physical output LUTs in a 256-color display resolution. However, since not all colors are available at this resolution, the default might not always be optimal for certain atypical cases. Therefore, the physical output LUTs are always available and programmable, by a user, to achieve the best display effect for your images.

Reference material: Windows palettes and physical output LUTs

Before describing the default and explaining how and why to change it, a basic explanation of palettes and output LUTs is discussed. For more detailed information, consult the following reference: Halibard, Moishe. Windows Developer's Journal. "Palettes and 256 Colors." July, 2000.

Color images

For windowed displays, in a 256-color display resolution, the following example shows how a color image is consequently displayed as the best color image possible:

  1. The color image goes through Windows translation functions. Windows searches the physical output LUTs for the entry that best matches the color of the image's pixels. Depending on what colors are available, Windows translates the image to an 8-bit index image.
  2. The physical output LUTs subsequently translate that index image and display it as the best color image possible.

Palettes and physical output LUTs

Each of the color images' palette is used to specify the required colors. Windows uses these palettes to realize a system palette. The system palette is ultimately loaded into the physical output LUTs and is searched by the translation functions when generating an 8-bit (index) image.

Note that the palette of the active window has priority over available system palette entries and is therefore loaded in the system palette first. Also, Windows tries to map colors from the logical palette into the currently realized system palette to reduce the number of requested new entries. This reduces the chance of a color occurring more than once in the system palette.

Monochrome images

Since LUTs are available, you can use them to create special effects for 8-bit images. However, unlike color images, grayscale values map to values at the corresponding index in their associated palette.

Default palette settings

By default, windowed displays use the default MIL palette. MIL provides good default logical palettes for the realization of the physical output LUTs. To accomplish this, the MIL default takes into consideration the number of bands of the image, and produces the best performance versus visual quality compromise possible. To use the default LUT, use the M_DEFAULT setting of the MdispLut() control type.

Why change the default LUT values

If the default LUT values are not appropriate for your application, you can change the LUT values to control the displayed colors or gray levels of an image. Some situations that might require special display effects are:

  • When displaying monochrome images, you might want to view the images with each gray intensity in a different color. For example, you can associate specific colors to ranges of temperatures obtained by an infrared camera.
  • When displaying monochrome images, you might want to invert the image values. For example, when grabbing a film negative, you can negate the video and display the film as it will be printed.
  • When displaying color images in a 256-color Windows display resolution, you might want to reduce the loss of color resolution. Generally, the default LUT values will not optimize and distinguish between subtle differences in one color. For example, when displaying a color image with many shades of red, you might want to select a LUT so that all shades of red in that particular image are being represented.

Changing the default LUT values

Change the default LUT values by associating a LUT to either the display, using MdispLut(), or to the image buffer, using MbufControl(), with M_ASSOCIATED_LUT. Changing the default LUT values by associating a LUT to the display affects all images displayed in that display. On the other hand, changing the default LUT values by associating a LUT to the buffer affects only the display of that buffer. In this case, when the buffer is saved, the LUT is saved with it.

Viewing each gray intensity in a different color

To view an 8-bit image buffer with each gray intensity in a different color, associate the default pseudo-color LUT buffer (M_PSEUDO) with the display of the image. In this case, the data is loaded in each component of the logical palette.

A 1-band custom LUT buffer

To invert the values of an 8-bit image on display, you would need physical output LUTs that map each value to the maximum pixel value minus the current pixel value. To do so:

  1. Allocate a 1-band 256-entry LUT buffer using MbufAlloc1d().
  2. Generate the data into the buffer using MgenLutRamp(), or load the data into it, using MbufPut(). The depth of the LUT buffer data must be 8 bits.
  3. Associate the LUT buffer with the required display using MdispLut(), or to a particular image using MbufControl() with M_ASSOCIATED_LUT.

If you associate a 1-band LUT buffer with the display or buffer, the same data is loaded into each component of the logical palette.

A 3-band custom LUT buffer

To reduce the loss of color resolution when displaying an image with a specific range of colors, you would need physical output LUTs that contain all the required colors so that, when Windows creates a translation table for the image, most colors are mapped to their exact values. To do so, allocate a 3-band 256-entry LUT buffer, and:

  1. Take a histogram of the image.
  2. Order and load frequently used colors in the LUT buffer according to popularity (the most popular color first).

    3. Be careful not to remove infrequent colors that illustrate critical information.

  3. Associate the LUT buffer with the required display using MdispLut(), or to a particular image using MbufControl() with M_ASSOCIATED_LUT.

When you associate the 3-band LUT buffer (RGB) with a display and then select the display, each band of the LUT buffer is loaded into its corresponding component of the logical palette.

A 3-band LUT buffer can also be used to create a custom pseudo-color LUT for 8-bit images.

When using physical output LUTs

When using physical output LUTs, you must keep the following points in mind:

  • If the contents of a LUT buffer changes while the image is selected on the display, the changes will not take effect until calling MdispLut() again.
  • When displaying in a non-256-color display resolution, MIL can simulate a display LUT in software only for 8-bit 1-band images.
  • The LUT buffer must have one or three bands. The number of LUT buffer entries must be the same as the maximum number of intensities that can be represented in the displayed image buffer. In other words, if you want to invert an 8-bit grayscale image (that is, an image that can have 256 intensities), your LUT must also have 256 entries.

You can use MdispInquire() to obtain information about the physical output LUTs of a display.

DEFAULT ASSOCIATED LUT PSEUDO ASSOCIATED LUT ASSOCIATED LUT


CPU-assisted display

Availability | Previous
Available in MIL-Lite

Available in MIL

All displays are typically accelerated by the graphics controller, which means that CPU usage is low since the graphics controller is handling the display. However, if your graphics controller's limits have been reached, MIL compensates by making the display CPU-assisted. This results in higher than expected CPU usage.

In addition to higher than expected CPU usage, you might experience the following behaviors when your display is CPU-assisted:

  • Overlay flickering.
  • Slower updates of the display during continuous grabbing.

Overlay flickering

When your graphics controller's limits have been reached, MIL uses your CPU to implement the overlay-display mechanism. Annotations in this overlay buffer might flicker, though they are still non-destructive.

Grabbing continuously

When your overlay buffer is CPU-assisted, the display of a continuous grab can be slower. This is due to an additional operation needed to combine the grabbed image with the display's overlay buffer in an intermediate buffer. Note that the actual image buffer selected on the display is not overwritten by the contents of the overlay buffer.

Avoiding CPU-assisted displays

To avoid a CPU-assisted display, you can attempt to:

  • Lower your screen resolution or refresh rate.
  • Free all on-board buffers or displays that are no longer being used.
  • Avoid forcing a specific internal format when allocating the image buffer, unless it is required.


Generating graphics

Chapter 17: Generating graphics
Availability
Available in MIL-Lite

Available in MIL

This chapter describes the graphics functions that are available with MIL. These consist of drawing and text-writing functions.

  • MIL and graphics in general
  • Preparing for graphics
  • Drawing graphics
  • Writing text


MIL and graphics in general

Availability | Next
Available in MIL-Lite

Available in MIL

The MIL package supports basic drawing and text functions that are useful in typical image processing or machine vision applications. These functions could be used, for example, to create a condition buffer or to annotate an image.


Preparing for graphics

Reference:
MappAlloc
MappAllocDefault
MbufClear
MgraAlloc
MgraBackColor
MgraClear
MgraColor
MgraFree
Available in MIL-Lite

Available in MIL

There are two requirements for graphics operations:

  • An image buffer in which to perform the operation.
  • A set of graphics parameters, referred to as a graphics context, with which to perform the operation.

Allocate a graphics context, using MgraAlloc(). Upon allocation, each of the graphics parameters of the graphics context is set to the default (refer to the MgraAlloc() function description for the defaults). You can change these parameter settings according to your needs.

Different graphics contexts can coexist. Use their identifier to specify which to use or change.

Once a graphics context is no longer required, it should be freed, using MgraFree().

When a MIL application is created, using MappAlloc() or MappAllocDefault(), a default graphics context is automatically created. It can be used as a normal graphics context by specifying M_DEFAULT as the graphics context identifier. Since M_DEFAULT is simply another graphics context, you can change its parameter settings according to your needs.

There are two basic parameters that apply to graphic objects:

  1. Background color. This determines the background color of textual graphic objects. The default background color value is zero (typically corresponds to black). You can change this color, using MgraBackColor().
  2. Foreground color. This determines the color in which graphic objects are drawn or written. The default foreground color value is the highest positive buffer value (typically corresponds to white). You can change this color, using MgraColor().

A grayscale value can be any integer or floating-point number. If the given value exceeds the range of the possible values that can be stored in each band of the destination buffer, the least significant bits of the value are used.

Once you are satisfied with the graphics parameters, you should determine whether you need to clear the graphics image buffer prior to drawing or writing to it. You can use MgraClear() or MbufClear() to clear the buffer to a specific color.


Drawing graphics

Reference:
McalTransformCoordinate
McalTransformCoordinateList
MgraArc
MgraArcFill
MgraDot
MgraDots
MgraFill
MgraLine
MgraLines
MgraRect
MgraRectFill
Available in MIL-Lite

Available in MIL

With the MIL package, you can draw:

  • Lines (MgraLine(), MgraLines()).
  • Rectangles (MgraRect() and MgraRectFill()).
  • Arcs, circles, and ellipses (MgraArc() and MgraArcFill()).
  • Dots (MgraDot(), MgraDots()).

Using MgraLine(), MgraRect(), MgraArc(), or MgraDot(), you can draw the outline of most required shapes. The outlines are drawn one pixel wide. You can also call MgraLines() and MgraDots() to draw multiple lines and dots.

In addition, the MIL package includes MgraRectFill() and MgraArcFill() so you can draw solid rectangles and arcs.

If you need complex filled-in shapes, draw the outline of the shape and use MgraFill() to fill it.

MgraFill() performs a boundary-type seed fill. It fills an area of the target buffer with the current foreground color, starting from the specified seed position. Filling occurs on adjacent pixels of the same value as the original seed pixel.

Note, any drawing is clipped outside the boundaries of the buffer.

Note, the Mgra...() functions use the image coordinate system (pixel coordinates). To draw in a calibrated image using real-world coordinates, you must first convert the coordinates from their real-world value to their pixel value using McalTransformCoordinate() or McalTransformCoordinateList().


Writing text

See Also | Availability | Previous
Reference:
MgraFont
MgraFontScale
MgraText
Available in MIL-Lite

Available in MIL

You can also write text in the drawing area, using MgraText(). This function writes a null-terminated (\0) ASCII string at the specified position in a given buffer, using the foreground and background color and current font of the specified graphics context.

When specifying the location at which to write the string, give the top-left corner coordinates of the first character in the string.

Although the graphics context specifies a default character font and size, you can change the font and size of this context, using MgraFont() and MgraFontScale(), respectively. MgraFont() provides a set of predefined fonts from which to choose.


Grabbing with your digitizer

Chapter 18: Grabbing with your digitizer
Availability
Available in MIL-Lite

Available in MIL

This chapter discusses how to grab images using a digitizer allocated on your frame grabber and the features available when grabbing.

  • Cameras and video sources
  • Basic concepts
  • Data format
  • Device number
  • Multiple cameras
    • Simultaneous acquisition
    • Data input channels of acquisition paths
    • Switching between cameras of the same type
    • Switching between cameras of different types
    • Ultra-fast channel switching
    • Channel locking and unlocking
  • Grabbing a single field
  • Line-scan cameras
  • Grabbing to the display
  • Reference levels, lookup tables, and scaling
    • Black and white reference levels
    • Color image reference levels
    • Mapping grabbed data through a LUT
    • Scaling
  • Optimizing application performance when grabbing
    • Grab mode
    • Double buffering
    • Multiple buffering
  • Grabbing a sequence of frames in real-time
  • Grabbing with triggers and exposures
    • Asynchronous reset mode and next valid frame mode
    • Setting the exposure
    • Software triggers
  • Auto-focusing
    • Search strategies
      • Bisection strategy
      • Refocus strategy
      • Scan-All strategy
      • Smart-scan strategy
      • Evaluate the focus indicator


Cameras and video sources

See Also | Availability | Next
Reference:
MappAllocDefault
MdigAlloc
MdigFree
MdigGrab
MdigGrabContinuous
MdigProcess
MsysAlloc
Available in MIL-Lite

Available in MIL

MIL supports input from any type of video source supported by the imaging board (frame grabber). Each frame grabber has one or more available acquisition paths. MIL refers to the acquisition paths used to grab from one video source as a digitizer.

Note, since most video sources are cameras, they will hereafter be referred to as such.

For a digitizer to be recognized by MIL, it must be allocated on the target system, using MdigAlloc() (or MappAllocDefault()). The allocation sets up the digitizer to match your camera's data format and to access the active data input channel. Once you have finished using a digitizer, you should free it, using MdigFree().

To make a more portable application, you can allocate and use the default system and digitizer set up when installing MIL or using MILConfig. To allocate the default system and digitizer, use MappAllocDefault(). Alternatively, you can use MsysAlloc() with M_SYSTEM_DEFAULT to allocate the default system. Use the returned system identifier with MdigAlloc() and set the device number and data format both to M_DEFAULT to allocate the default digitizer.

Use MdigGrab() or MdigGrabContinuous() to grab data from a camera through a digitizer. The data is stored in an image buffer. The image buffer should have as many bands as the grabbed data has color components. If you need to grab and process concurrently, you can use MdigProcess().

Both the image buffer and digitizer must be allocated on the same system.

When developing an application, it is recommended that you use a simple camera. Once the application is working, switch to a more sophisticated camera, if necessary. This approach makes debugging much easier.

SYSTEM DEFAULT DEFAULT


Basic concepts

Availability | Previous | Next
Not available in MIL-Lite

Available in MIL

The basic concepts and vocabulary conventions for the MIL digitizers are:

  • Acquisition path. A path that has the components to digitize or capture a video input signal.
  • Camera. A generic term to refer to any physical device used for creating image/video data and that can be connected to a frame grabber.
  • Frame grabber. A type of imaging board which can acquire video data from a camera.
  • Digitizer. The acquisition path(s) with which to grab from one camera of the specified type. When several MIL digitizers are allocated, their device number, along with their DCF, identify if they represent the same path(s) (but perhaps for a different input format) or independent path(s) for simultaneous acquisition.
  • Data input channel (channel). Identifies which camera to use when several of its type can be connected to the same acquisition path(s) (for example, grab from channel 0 or channel 1 of digitizer 0).
  • Independent acquisition path. An acquisition path that can, if required, acquire data from an input source independently from another such path on the same frame grabber. Each independent acquisition path has its own digitizing components to manage all video timing and synchronization for the path.
  • Imaging board. A generic term to refer to all boards that manipulate video data, including: frame grabbers, vision processors, compression boards, and other specialized boards.
  • Timing control unit. A hardware unit (for example, a Programmable Synchronization Generator or PSG) that is responsible for managing all video timing and synchronization signals.
  • Vision processor. A type of imaging board which can process and analyze image/video data. A vision processor includes a CPU and additional memory.


Data format

Reference:
MdigAlloc
MdigControl
MdigInquire
Available in MIL-Lite

Available in MIL

MdigAlloc() needs the digitizer configuration format (DCF) that corresponds to your camera to perform the digitizer allocation. The DCF defines such settings as the input frequency and resolution, and will determine limits when grabbing an image. Once a digitizer has been allocated, many of the settings are established based on the selected DCF. You can change these settings to values other than the default using MdigControl() and other Mdig...() functions. You can use MdigInquire() to inquire about a digitizer's settings.

MIL provides a number of predefined DCFs for the basic cameras supported by your frame grabber. Refer to the MIL Reference with regards to the predefined DCFs available for your particular board. MIL also provides some DCF files that you can load if the predefined DCFs do not suit your needs.

If you find a DCF file that is appropriate for your camera, but you need to adjust some of the more common settings, you can do so directly, without adjusting the file, using the Mdig...() functions. For more specialized adjustments, you can adjust the DCF file itself, using Matrox Intellicam.

If you cannot find an appropriate DCF file because, perhaps, you have a non-standard camera or other video source, you can create your own DCF file, using Matrox Intellicam. For more information on Matrox Intellicam, refer to the Matrox Intellicam User Guide manual.

If you cannot develop the required DCF using Matrox Intellicam, you should provide the camera specifications to your Matrox Technical Support Engineer. A suitable customized DCF file can then be developed, if your frame grabber supports the camera.


Device number

Reference:
MdigAlloc
Available in MIL-Lite

Available in MIL

In addition to the data format, MdigAlloc() requires that you specify the device number of the first acquisition path of the digitizer. Its digitizer configuration format (DCF) establishes the total number of acquisition paths used. MIL associates a timing control unit (for example, a PSG) on your frame grabber, with these acquisition paths when grabbing. If a timing control unit or the specified aquisition paths are not available when a grab is requested, the grab request must wait until they become available.

When the Matrox frame grabber has only one timing control unit, you must specify M_DEV0 or M_DEFAULT as the device number for the digitizer, regardless of the number of acquisition paths available.

If a digitizer can grab while another digitizer on the same frame grabber is simultaneously grabbing, it is said to be independent. If it cannot, it is dependent. The maximum number of digitizers from which you can grab simultaneously equals the number of timing control units on your Matrox frame grabber.

In order for the two digitizers depicted below to grab simultaneously, there must be at least two timing control units on your Matrox frame grabber. In the following image, the first camera is a component RGB camera and the second one is a monochrome camera. They are connected to their associated acquisition paths on a frame grabber.

For more information regarding the number of available timing control units, refer to the Installation and Hardware Reference manual and the MIL Board-specific notes for your Matrox frame grabber.

DEFAULT


Multiple cameras

User Guide:
Device number

Reference:
MdigAlloc
MdigChannel
MdigControl
MdigFree
MdigGrab
MdigGrabContinuous
MdigHookFunction
MdigInquire
MdigProcess
MdigReference
MthrWait
Available in MIL-Lite

Available in MIL

MIL supports applications that require input from different cameras. With most Matrox frame grabbers, you can connect multiple cameras. If each camera connects to an independent acquisition path, then you can allocate a series of independent MIL digitizers and grab from these cameras simultaneously.

Simultaneous acquisition

Simultaneous acquisition is available on Matrox frame grabbers that support multiple independent acquisition paths. To grab from several sources simultaneously, you must allocate multiple digitizers with MdigAlloc() and then call MdigGrab(), MdigGrabContinuous(), or MdigProcess() with the identifiers of the required digitizers. When allocating a digitizer, you are committing one or more acquisition paths to a specific camera, depending on the format of the camera (for example, an RGB color camera uses three acquisition paths). To perform simultaneous acquisition, your digitizers must use different acquisition paths. For more information, see the Device number section earlier in this chapter.

Refer to the Board Specific Notes for more information regarding the number of cameras from which you can acquire data simultaneously using your Matrox frame grabber.

The following example shows you how to allocate two digitizers on a system and use them to perform two continuous grabs simultaneously.

MDigGrabMultiple.c

Data input channels of acquisition paths

Most acquisition paths have several multiplexed data input channels. This means that they have several data input channels but can only grab from one channel at a time. In this case, the MIL digitizer representing the acquisition path(s) has multiple data input channels. If you have a camera that is connected to a channel other than the first of its digitizer, you must specify the channel, using MdigChannel().

Switching between cameras of the same type

When connecting several cameras of the same data format to different data input channels of a digitizer, allocate a single digitizer with the appropriate DCF for the first camera and use MdigChannel() to switch between the others of the same type.

Switching between cameras of different types

When connecting cameras of different data formats to different data input channels of a digitizer, you must use a different DCF for each camera. You could allocate a digitizer, grab the required frame, free the digitizer, and then allocate the digitizer again with the second DCF; this can increase the time required for the operation. MIL can circumvent this problem by using a fast DCF-switching technique which is outlined in the steps below:

  1. Make as many calls to MdigAlloc() as you have cameras, with different formats, from which you want to grab.
  2. Specify the required digitizer settings using MdigControl(), MdigReference(), and MdigHookFunction() for each allocated digitizer.
  3. Specify the channel to use for the grab using MdigChannel().
  4. Call MdigGrab() with an allocated digitizer identifier. Note that if this call uses a digitizer identifier different from the previous call, a DCF switch will occur.

    5. If there is a grab in progress with one digitizer, calling any of the following functions with any other digitizer will result in an error: MdigGrab(), MdigGrabContinuous(), MdigInquire(), MdigProcess(), MdigAlloc(), and MdigFree(). To synchronize your grabs with different digitizers, use MthrWait().

Ultra-fast channel switching

If you intend to switch channels after grabbing a single field or frame, use the ultra-fast channel switching mode to minimize time between grabs; set the M_CAMERA_LOCK control type to M_ENABLE + M_FAST. This mode is optimized for applications that use several cameras of the same type and it does impose certain restrictions:

  • Some digitizer hook events might only be generated a few frames after the channel switch: M_FRAME_START, M_FIELD_START, M_FIELD_START_ODD, and M_FIELD_START_EVEN.
  • Trigger inputs are unavailable.
  • In frame mode, the following hook events are unreliable: M_GRAB_FIELD_END, M_GRAB_FIELD_END_ODD, and M_GRAB_FIELD_END_EVEN. Note these hook events are reliable in field mode.

Channel locking and unlocking

If the synchronization between a digitizer and a camera is uncertain, enable MdigControl() with M_CAMERA_LOCK to lock the digitizer to the synchronization signal of the camera after a channel-switch. This provides additional stability. In this case, the digitizer is unlocked from the first camera and then locked to the next camera after a channel-switch occurs.

Typically, MIL finds the best balance between the fastest lock and the most reliable lock possible, depending on your camera. On some Matrox frame grabbers, you can control channel locking sensitivity using MdigControl() with M_CAMERA_LOCK_SENSITIVITY and M_CAMERA_UNLOCK_SENSITIVITY. Note that the speed and reliability of a camera lock directly affects the subsequent unlock and vice versa (for example, a fast lock might create an unreliable unlock, or a reliable lock might cause a slow unlock).

CAMERA LOCK ENABLE FAST FRAME START FIELD START FIELD START ODD FIELD START EVEN GRAB FIELD END GRAB FIELD END ODD GRAB FIELD END EVEN CAMERA LOCK CAMERA LOCK SENSITIVITY CAMERA UNLOCK SENSITIVITY


Grabbing a single field

Reference:
MdigControl
MdigGrab
Available in MIL-Lite

Available in MIL

With interlaced scanning cameras, 2 fields are grabbed by default; therefore one call to MdigGrab() will grab both the odd and even fields. You can change the number of fields to 1 and have MIL treat each field as one frame using MdigControl() with M_GRAB_FIELD_NUM, grabbing every second row and storing them in sequential rows. Therefore, the grab time is reduced by half. This control type can only be set to 1 or 2, and should only be used for interlaced video. When set to 1, each field is treated like a frame and the following digitizer hooks are aligned with the field: M_GRAB_FRAME_START, M_GRAB_END, and M_GRAB_FRAME_END. To achieve 60 fps in NTSC or 50 fps in PAL, the control type M_GRAB_START_MODE must be set to M_FIELD_START.

GRAB FIELD NUM GRAB FRAME START GRAB END GRAB FRAME END GRAB START MODE FIELD START


Line-scan cameras

Availability | Previous | Next
Available in MIL-Lite

Available in MIL

If your target digitizer supports it, you can grab from a line-scan camera as you would, for example, an RS-170 type camera. However, you should be aware of how data from these cameras is stored.

When acquiring data from a line-scan camera, each line (row) of each destination buffer band is filled from top to bottom. The operation will only end once the entire buffer has been filled.


Grabbing to the display

Reference:
MdigGrab
MdigProcess
MdispSelect
Available in MIL-Lite

Available in MIL

To grab to the display, a displayable buffer must first be allocated and selected for display (MbufAlloc...() with M_DISP, and then MdispSelect()). When MdispSelect() is called, internal version(s) of the selected buffer are created in the frame buffer (also called display memory) for display purposes. These buffers are maintained and synchronized internally.

The frequency with which the selected buffer and the display synchronize is dependent upon whether performing a monoshot or continuous grab. A monoshot grab updates the selected buffer first and then the display. A continuous grab updates the display immediately after each frame is grabbed; only the last frame is stored in the selected buffer for processing. If you need to save/process each grabbed frame, you should perform the grab using MdigGrab() (that is, perform a monoshot grab); alternatively, you can use MdigProcess() for its multiple buffering capabilities.

When grabbing, the frame grabber (for example, Matrox Meteor-II) always acts as the bus master. When grabbing to an on-board display section or when grabbing to a graphics controller that supports fast linear-memory accesses to its display memory, the frame grabber can generally transfer all grabbed data directly to display memory. In the latter case, the display memory also has to be physically accessible from the frame grabber. This is the case for most AGP display boards and PCI display boards plugged into the same PCI-X bus segment as the frame grabber.

DISP


Reference levels, lookup tables, and scaling

Reference:
MdigControl
MdigLut
MdigReference
Available in MIL-Lite

Available in MIL

MIL provides functions to improve the appearance of a grabbed image on input (if your hardware allows it). You can adjust the brightness and contrast of the images, as well as the hue and saturation for color grabs, by fine-tuning the controls of the analog-to-digital converters in your system. You can also correct and precondition the input data prior to storing it, through scaling, or by mapping it through an input LUT.

Black and white reference levels

When digitizing images, the black and white reference levels determine the zero and full-scale levels, respectively, of the input voltage range. The analog-to-digital converters convert any voltage above the white reference level to the maximum pixel value, and any voltage below the black reference level to a zero pixel value.

Matrox digitizers support fine-tuning of these reference levels. By reducing or increasing either or both the black and white reference levels, you affect the brightness of the image. By reducing one reference level and increasing the other, you affect the contrast of the image.

MIL linearly represents the distance between the minimum and maximum voltages, in which the black reference level can be adjusted (hardware-specific), as units between M_MIN_LEVEL and M_MAX_LEVEL. The same is done for the white reference level adjustment range. These units are the values by which you can adjust the specified reference level, using MdigReference().

To calculate the value to pass to MdigReference(), use the following equation with the appropriate voltages specified for your particular board.

The smallest voltage increment supported by your board can differ such that consecutive reference-level settings might produce the same result.

Note, the new reference level might not take effect until the next grab, at which point, a certain amount of delay might be incurred as the hardware adjusts to the reference-level changes.

Color image reference levels

When grabbing composite color images, MdigReference() provides specific control parameters to adjust the levels of contrast, brightness, hue, and saturation. These levels can be set to values from 0 to 255; use values appropriate for your particular board.

Mapping grabbed data through a LUT

If supported by your digitizer, grabbed data is mapped through a physical lookup table (LUT). By default, the physical LUT is transparent; that is, it is loaded with data so that each input pixel is mapped to its original value. You can load the physical LUT with your own custom LUT data if, for example, you want to correct or precondition grabbed data. To do so, copy a LUT buffer to the digitizer's physical input LUT, using MdigLut(). MIL uses the data format (DCF) of the digitizer to determine whether a physical LUT is supported. If it is not, an error is generated.

The characteristics of the LUT buffer and the digitizer's DCF establishes how the physical LUT is actually configured.

Configuration of the digitizer's LUT | Determining factor
Number of entries in the digitizer's physical LUT | Data being grabbed (DCF)
Depth (8- or 16-bit) | LUT buffer
Number of bands | DCF & LUT buffer

The digitizer's physical LUT is typically configured to have the same number of components (bands) as either the LUT buffer or the data to be grabbed (determined by the digitizer's DCF), depending on which has more bands. The digitizer's physical LUT is also configured to have the same number of entries as the maximum possible value per band of the data to grab. The depth of a digitizer's physical LUT is configured to be either 8- or 16-bits per band, depending on if the LUT buffer depth is 8-bit or 16-bit, respectively. Cameras, however, are not so limited. When dealing with a 10- or 12-bit camera, use a 16-bit destination grab buffer and load the digitizer's physical LUT with the difference zero padded.

To copy the data from a LUT buffer to the digitizer's physical LUT, the number of entries in the LUT buffer must match those of the digitizer's physical LUT. In addition, if the digitizer's physical LUT cannot support the depth of the LUT buffer, an error will occur.

LUT buffer data is loaded into the digitizer's physical LUT, as follows:

LUT buffer | Digitizer's physical LUT | Result
1 band | 1 band | The LUT buffer is copied directly into the digitizer's physical LUT.
1 band | 3 band | The LUT buffer is copied into each component of the digitizer's physical LUT.
3 band | 1 band | The first band of the LUT buffer is copied into the digitizer's physical LUT.
3 band | 3 band | Each of the LUT buffer's bands are copied into the corresponding component of the digitizer's physical LUT.

If the destination grab buffer depth is larger than that of the digitizer's physical LUT, the destination grab buffer's least significant bits are set to zero when the data is grabbed. If the digitizer's physical LUT depth is greater than that of the destination grab buffer, the most-significant bits of the data (the non-zero values) are used when the data is grabbed.

To revert back to using a transparent LUT, you must copy the default LUT (M_DEFAULT) to the digitizer's physical input LUT.

Scaling

The MdigControl() function allows you to scale grabbed data horizontally and vertically. If you scale grabbed data, the stored image size is different from the original image by the specified factors in the X and/or Y direction. The scaled image is written in contiguous locations in the image buffer, starting from the top-left corner. For example, if you set both the X and Y scaling factors to 0.5, only one column and one row out of two are written to the image buffer (starting with the first row and column).

The X and Y scaling factors are independent. Note, depending on the Matrox frame grabber and camera used, some scaling factors might not be available.

To disable scaling, set the scaling factors to 1.

MIN LEVEL MAX LEVEL


Optimizing application performance when grabbing

User Guide:
Multiple buffering

Reference:
MdigControl
MdigGrab
MdigGrabContinuous
MdigGrabWait
MdigHalt
MdigProcess
Available in MIL-Lite

Available in MIL

To optimize application performance when grabbing, you can:

  • Set the grab mode.
  • Perform double buffering.
  • Perform multiple buffering.

Grab mode

When grabbing data with MdigGrab(), you can control the synchronization by setting the MdigControl() M_GRAB_MODE control type to a value of M_SYNCHRONOUS, M_ASYNCHRONOUS, or M_ASYNCHRONOUS_QUEUED (if supported).

  • If the grab mode is set to M_SYNCHRONOUS, your application will be synchronized with the end of a grab operation. In other words, your application will wait until the grab has finished before executing the next function.
  • If the grab mode is set to M_ASYNCHRONOUS, your application will not be synchronized with the end of a grab operation. This option allows other functions to execute while you are still grabbing. This is a useful option when performing double buffering, a technique whereby you can grab data into one buffer while processing the previously grabbed buffer (discussed below). Note, another call to MdigGrab() before the current grab has finished will cause your application to wait until the current grab has finished.
  • If your frame grabber supports queuing, you can set the grab mode to M_ASYNCHRONOUS_QUEUED; if another grab is issued before the first one is finished, the grab will be queued on-board, allowing you to perform other processes while waiting for the next MdigGrab() to be executed. Note, you can still force your application to wait until the end of a grab before executing an operation, by calling MdigGrabWait().

Note that MdigGrabContinuous() is by definition asynchronous since you must use MdigHalt() to stop the grab.

Double buffering

Double buffering involves grabbing into one image while processing the previously grabbed image. Double buffering allows you to grab and process concurrently. You must switch the destination of the grab between the two image buffers. In addition, you need to synchronize the grabbing and processing so that:

  • You do not process an image until an entire frame has been grabbed into the buffer.
  • You do not grab into a buffer until the previous frame in that buffer has been processed.

For an example of a double buffering implementation, see MDigDoubleBuffering.

Note that the double buffering method is not recommended for real-time processing when the CPU usage is high.

For a more robust real-time behaviour, use the multiple-buffering technique in the Multiple buffering subsection in this section.

Multiple buffering

When a frame takes longer to process than the time required to grab the subsequent frame, you can use a multiple buffering technique to ensure that all processing is completed without losing any frames.

To perform multiple buffering in MIL, you can use MdigProcess(). It allows you to use a list of previously-allocated buffers to hold a series of sequentially grabbed images and process them as they are being grabbed. These grabs can either continue until stopped (using MdigProcess() with M_START to start and then M_STOP to stop) or continue until all buffers in the list are filled (using MdigProcess() with M_SEQUENCE). In the former case, MdigProcess() grabs round-robin through the list of buffers, however care must still be taken to assure that the average time it takes to process a frame is not greater than the frame rate of a camera, so that frames will not be missed.

MdigProcess() hooks a user-defined function to the modification of any buffer in the specified list. So every time a new frame is grabbed in a buffer in the list, the user-defined function is called. If the hook-handler function modifies the buffer that called it, the hook-handler function for that buffer will not be called again until that buffer is modified by a new grab. This is to avoid infinite recursive calls being generated. Note that, if the buffer is modified by some function other than the one hooked to it, the hooked function will be called. In addition, if the hook-handler function modifies another buffer which has another function hooked, that function will be called.

Note, processing is generally faster if the buffer is not on the display.

The following example shows you how to perform multiple buffering.

MDigProcessMultiple.c
GRAB MODE SYNCHRONOUS ASYNCHRONOUS ASYNCHRONOUS QUEUED SYNCHRONOUS ASYNCHRONOUS ASYNCHRONOUS QUEUED START STOP SEQUENCE


Grabbing a sequence of frames in real-time

User Guide:
Multiple buffering

Reference:
MbufExportSequence
MbufImportSequence
MdigGrab
MdigProcess
Available in MIL-Lite

Available in MIL

To grab a sequence of frames in real-time, simply use successive, asynchronous calls to MdigGrab():

/* Put digitizer in asynchronous mode */
MdigControl(MilDigitizer, M_GRAB_MODE, M_ASYNCHRONOUS);
/* Grab the sequence. */
for (n=0; n<NbFrames; n++)
{
/* Grab one buffer at a time. */
MdigGrab(MilDigitizer, MilImage[n]);

Alternatively, you can make a single call to MdigProcess(). For more information, see the Multiple buffering subsection in the Optimizing application performance when grabbing section earlier in this chapter.

In either case, you must allocate buffers to store the frames of the sequence. After you have grabbed a sequence, you can use the MbufExportSequence() function to export the sequence of image buffers (compressed or uncompressed 8-bit) to an AVI file. When exporting, you must specify the number of buffers and the frame rate (number of images/second) of the sequence. Note, the MIL identifiers of the image buffers to export must be kept in an array.

Use the MbufImportSequence() to import a sequence of images from an AVI file into separate image buffers. You can import images in all format supported by MIL. You can also choose to import the sequence into automatically allocated buffers or previously allocated buffers.


Grabbing with triggers and exposures

Reference:
MdigControl
MdigGrab
MdigGrabContinuous
MdigHalt
Available in MIL-Lite

Available in MIL

If your Matrox frame grabber supports trigger input, you can set up your digitizer to perform a triggered grab, that is, to grab a frame upon the occurrence of an event. In this case, nothing is grabbed when you call MdigGrab() or MdigGrabContinuous(), until a specified event occurs. When grabbing continuously, the digitizer waits for a trigger before grabbing each frame; you must still call MdigHalt() after grabbing all required frames.

The digitizer configuration format (DCF) can initialize the digitizer to perform a triggered grab and can indicate exactly how it should be carried out. For example, if the DCF specifies that an exposure signal should be generated (for the camera) upon the grab trigger event, the actual grab would only be triggered once the active exposure time was over.

You can use the MdigControl() M_GRAB_TRIGGER... control types to override the DCF trigger settings. For example, you can enable/disable whether MdigGrab() and MdigGrabContinuous() perform a triggered grab, using MdigControl() with M_GRAB_TRIGGER. You can also specify the source and activation mode of the event upon which to grab, using MdigControl() with M_GRAB_TRIGGER_SOURCE and M_GRAB_TRIGGER_MODE, respectively.

Asynchronous reset mode and next valid frame mode

There are two possible grab activation modes: asynchronous reset mode and next valid frame mode. Note that the grab activation mode is specified in the DCF file.

If your frame grabber and your camera support asynchronous reset mode, you can set up the digitizer to reset the camera and have it start exposing a new frame when the trigger signal is received.

Otherwise, the digitizer waits for the next valid frame (or field) before starting to grab.

Setting the exposure

If your frame grabber and your camera support it, you can set up your digitizer to transmit an exposure signal to your camera, to specify when to start exposing a new frame and the duration for exposure. To do so, you must enable and set-up the required exposure timers and grab trigger connections as illustrated below.

Software triggers

In general, the digitizer's grab trigger receiver and exposure timers can also be triggered by software (use MdigControl() with M_GRAB_TRIGGER_SOURCE set to M_SOFTWARE). In this case, following a grab call, nothing is grabbed until you call a specific function. To issue a software trigger for the grab trigger receiver, call MdigControl() with M_GRAB_TRIGGER and M_ACTIVATE. To issue a software trigger for one of the exposure timers, call MdigControl() with M_GRAB_EXPOSURE + M_TIMERn and M_ACTIVATE. Note that in this case, the grab call must be asynchronous (that is, you must issue the grab with MdigGrab() in asynchronous mode or with MdigGrabContinuous()), or the grab call must be called on a separate thread.

Note, for a frame grabber without an exposure timer, the exposure time is considered to be zero.

GRAB TRIGGER GRAB TRIGGER SOURCE GRAB TRIGGER MODE GRAB TRIGGER SOURCE SOFTWARE GRAB TRIGGER ACTIVATE GRAB EXPOSURE ACTIVATE


Auto-focusing

See Also | Availability | Previous
Reference:
MdigFocus
Not available in MIL-Lite

Available in MIL

Note that this section is not supported with MIL-Lite.

You can use MdigFocus() to automatically adjust the lens motor of your camera to a position that produces optimum focus in your images. This function is primarily useful when the quality of the focus in your images is unacceptable and physically moving the grabbed object is not possible.

MdigFocus() determines the optimum focus position by grabbing an image at an initial lens position, analyzing the focus quality of the grabbed image, calling a user-defined function that changes the position of the lens motor, and then grabbing and analyzing another image. The process repeats until the optimum focus position is found.

The focus quality of an image (known as its focus indicator) is measured by analyzing its edges. An image with good focus quality (a high focus indicator) has well-defined edges, that is, has a sharp difference in gray-levels between its object edges and its background.

By default, MdigFocus() subsamples and filters each grabbed image before analyzing it. This makes it easier to analyze the image. If necessary, you can specify that the subsampling and/or filtering operations be skipped. Skipping these operations will result in a more accurate analysis of the image's focus quality. It is primarily useful to skip these operations when your images contain fine details since subsampling or filtering can remove these details. Note that subsampling the grabbed images increases the speed of MdigFocus(); filtering the grabbed images slows down MdigFocus().

If necessary, you can specify that only a sub-region of the image be analyzed, by passing a child buffer to the function. This is primarily useful if there are objects at different distances within the camera's field of view. In such a case, each object will have a different optimum focus position, so you need to use a child buffer to specify the object on which to focus.

Search strategies

When you perform MdigFocus(), you have to specify the minimum, maximum, and starting position of the lens motor. Given these parameters, different strategies can be used to find the optimum focus position. These strategies determine how the position is updated (in which direction and by how much) between grabs. They can affect the speed and accuracy of the operation. The default search strategy used by MIL is the smart-scan strategy.

Bisection strategy

The bisection strategy breaks down the given positional range, step-by-step, until it finds the optimum focus position.

In general, the bisection strategy processes the fewest amount of images. However, it is most sensitive to noise and requires that the lens motor travel the greatest distance.

Refocus strategy

The refocus strategy scans upward or downward until it finds the optimum focus position or until it reaches the minimum or maximum position. While scanning in one direction, if the focus indicator decreases continuously (indicating an out-of-focus condition), the focus position is returned to its starting point and scanning is started in the opposite direction. By default, if a peak in focus indicator values is found, the next two positions are scanned to make sure the peak is truly the optimum. If necessary, you can change the number of positions used to verify a peak.

The refocus strategy is the best strategy to use when the current focus position is close to optimum.

Scan-All strategy

The scan-all strategy scans, by 1, all positions between the minimum and maximum and returns the position which produced the highest focus indicator.

The scan-all strategy is the slowest but most accurate.

Smart-scan strategy

The smart-scan strategy performs three refocus searches, each with a smaller positional increment. You specify the initial positional increment; the subsequent increments are factors of the initial one. As with the refocus strategy, the default number of positions used to verify a peak is 2 but can be changed.

The smart-scan strategy is a compromise between the speed of a bisection and the accuracy of a scan-all.

Evaluate the focus indicator

Rather than determine the optimum focus position, MdigFocus() can be used to simply return the focus indicator value for a given image or for the image grabbed at the current lens position.


Color

Chapter 19: Color
Availability
Available in MIL-Lite

Available in MIL

This chapter discusses how to handle objects in color with MIL.

  • Dealing with color
  • Grabbing
    • Mapping grabbed data through a LUT
  • Displaying
  • Managing color images
  • Processing
  • Performing color conversions


Dealing with color

See Also | Availability | Next
Reference:
MbufAllocColor
Available in MIL-Lite

Available in MIL

MIL supports grabbing, displaying, accessing, and processing color images.

MIL-Lite does not support processing.

MIL can represent an object in color with a single color buffer, allocated with MbufAllocColor().


Grabbing

Reference:
MappAllocDefault
MdigAlloc
MdigFree
MdigGrab
MdigGrabContinuous
MdigLut
MdigReference
Available in MIL-Lite

Available in MIL

You grab from a color video source (typically a camera) into a color image buffer, as you would into a grayscale image buffer, by calling MdigGrab() or MdigGrabContinuous().

Before performing a color grab, you must allocate a digitizer, using MdigAlloc() (or MappAllocDefault()), specifying a digitizer configuration format (DCF) appropriate for your color camera. In addition, the digitizer and the image buffer must be allocated on the same system and have compatible dimensions. Once you have finished using the digitizer, you should free it, using MdigFree().

When grabbing from a color digitizer, each color component is transmitted simultaneously. The destination buffer must have the same number of color bands as the digitizer. The data is simultaneously stored in the appropriate component of the image buffer. When grabbing RGB, the red component is stored in the first color band, the green component is stored in the second color band, while the blue component is stored in the third color band.

If the hardware permits, you can control the digitization reference level of each acquisition path of the digitizer, using MdigReference().

Note, upon installation, if you specified a color camera, the default image buffer, allocated with MappAllocDefault(), will be a three-band color image buffer. If you didn't specify a color camera, but would now prefer to use one, you can update your defaults using MilConfig.

Mapping grabbed data through a LUT

You can also correct or precondition input data by mapping it through a LUT upon acquisition (if the hardware permits). This requires that you associate a LUT buffer with the digitizer, using MdigLut().

You can associate either a single-band LUT buffer or a LUT buffer that has the same number of color bands as the digitizer. If you associate a single-band LUT buffer with the digitizer, each component of the digitizer LUT is loaded with the same data. If you associate a multi-band LUT buffer with the digitizer, each component of the digitizer LUT is loaded with the data from corresponding color band of the LUT buffer.

To disassociate the LUT buffer from the digitizer, you need to re-associate the default LUT with the digitizer, using MdigLut() with M_DEFAULT.


Displaying

Reference:
MappAllocDefault
MdispAlloc
MdispLut
MdispSelect
Available in MIL-Lite

Available in MIL

You display a color-image buffer as you would a two-dimensional grayscale image buffer. You must first allocate the image buffer with a displayable attribute (M_DISP), and then select it for display, using MdispSelect(). To stop displaying the image buffer and leave the display blank, use MdispSelect() with M_NULL.

Before you can display a buffer, the display must be allocated, using MdispAlloc() (or MappAllocDefault()). The image buffer and the display should be allocated on the same system.

When you display a color image buffer (usually RGB), the first band is routed to the first output channel (usually red), the second band is routed to the second output channel (usually green), while the third band is routed to the third output channel (usually blue).

When a display is allocated, a default pass-through LUT (transparent LUT) is loaded into the output LUTs (if any). You can change the displayed colors of an image by associating a lookup table (LUT) to the display, using MdispLut().

DISP


Managing color images

User Guide:
Processing

Reference:
MbufExport
MbufImport
MbufLoad
MbufRestore
MbufSave
Available in MIL-Lite

Available in MIL

MIL supports the saving and loading of color images from disk in different file formats. See the MbufSave(), MbufLoad(), MbufRestore(), MbufImport(), and MbufExport() function reference descriptions for more details.

Note, all the MIL data allocation, access, and generation (Mbuf...() and MgenLut...()) functions can handle color image buffers.

For an example on how to manage color buffers, see MBufColor.c in the Processing section later in this chapter.


Processing

User Guide:
Performing color conversions

Reference:
MbufChildColor
MbufCopyColor
MimConvert
Not available in MIL-Lite

Available in MIL

Note that this section is not supported with MIL-Lite.

MIL can process color (multi-band) image buffers by processing each band of an image individually. However, MIL cannot perform statistical analysis, blob analysis or pattern recognition operations on color image buffers.

To process a single band of a color image, you can extract one band, using MbufCopyColor() or access it directly, using MbufChildColor(). In either case, processing can then be performed on the two-dimensional single band buffer. In the case of MbufCopyColor(), after each color band has been extracted and processed, it can be re-inserted into the buffer, using MbufCopyColor(). If MbufChildColor() was used, the parent buffer (in this case a multi-band buffer) is automatically updated after processing since its child buffer occupies the same physical space in memory.

Using the MIL function MimConvert(), you can perform color conversions, such as converting an RGB image into a HLS (Hue, Luminance, and Saturation) image and vice versa. You can also extract the luminance (intensity) from an RGB image or copy the luminance component of an image into a three-band buffer to create a monochromatic (gray) RGB buffer. For more information, see the Performing color conversions section later in this chapter.

The following is an example of color image manipulation and conversion.

MBufColor.c


Performing color conversions

See Also | Availability | Previous
Reference:
MimConvert
Not available in MIL-Lite

Available in MIL

You can convert between color spaces using MimConvert(). For example, you can convert RGB to HLS values. For efficiency when converting from a 3-band (RGB) buffer, you can calculate just the hue component of the HLS color space into a 1-band buffer (M_RGB_TO_H). You can also use this function to extract the luminance (intensity) from an RGB image (M_RGB_TO_L), or to copy the luminance component of an image into a 3-band buffer (M_L_TO_RGB), to create a monochromatic (gray) RGB buffer.

In the HLS color space (also known as HSL), color is represented as a combination of hue, luminance, and saturation. The hue corresponds to the wavelength of the main color. It is represented as an angular position on a circular color disk. The luminance corresponds to the brightness of the color, while the saturation can be thought of as the measure of color purity or concentration.

The HLS color space is similar to the human way of describing colors. Each color has its own hue value (such as red, orange, or green). Once the hue value is chosen, changes to the luminance or the saturation alter only the color quality, not the basic color.

When converting, MimConvert() stores the hue component (H) in band 0, the luminance component (L) in band 2, and the saturation component (S) in band 1.

The following code is the algorithm that MimConvert() uses when converting between the RGB and HLS color spaces.

#define min(a, b) (((a) < (b)) ? (a) : (b))
#define max(a, b) (((a) > (b)) ? (a) : (b))

// Input and output between 0 and 1.
void RGBToHSL(float R, float G, float B, float* H, float* S, float* L)
   {
   float MinVal, MaxVal, Delta;

   // Min and max values.
   MaxVal = max(R, max(G, B));
   MinVal = min(R, min(G, B));

   Delta = MaxVal - MinVal;

   // Compute the luminance.
   *L = 0.5f * (MaxVal + MinVal);

   if (Delta == 0.0f)
      {
      *S = *H = 0.0f;
      }
   else
      {
      // Compute the saturation.
      if (*L <= 0.5f)
         {
         *S = Delta / (MaxVal + MinVal);
         }
      else
         {
         *S = Delta / (2.0f - MaxVal - MinVal);
         }

      // Compute the hue.
      if (MaxVal == R)
         {
         *H = 60.0f * ((G - B) / Delta);
         }
      else if (MaxVal == G)
         {
         *H = 60.0f * (2.0f + ((B - R) / Delta));
         }
      else
         {
         *H = 60.0f * (4.0f + ((R - G) / Delta));
         }

      if (*H < 0.0f)
         {
         *H += 360.0f;
         }
      }

   // Remap the angle between 0 and 1.
   *H = *H / 360.0f;
   }

float HueToRGB(float Temp1, float Temp2, float Hue)
   {
   if (Hue > 360.0f)
      {
      Hue = Hue - 360.0f;
      }
   else if (Hue < 0.0f)
      {
      Hue = Hue + 360.0f;
      }

   if (Hue < 60.0f)
      {
      return (Temp1 + (Temp2 - Temp1) * Hue / 60.0f);
      }
   else if (Hue < 180.0f)
      {
      return Temp2;
      }
   else if (Hue < 240.0f)
      {
      return (Temp1 + (Temp2 - Temp1) * (240.0f - Hue) / 60.0f);
      }

   return Temp1;
   }

// Input and output between 0 and 1.
void HSLToRGB(float H, float S, float L, float* R, float* G, float* B)
   {
   float Temp1, Temp2;

   // Remap the angle between 0 and 360.
   H = H * 360.0f;

   // Achromatic case.
   if (S == 0.0f)
      {
      *R = *G = *B = L;
      }
   else
      {
      if (L <= 0.5f)
         {
         Temp2 = L * (1.0f + S);
         }
      else
         {
         Temp2 = L + S - (L * S);
         }

      Temp1 = 2.0f * L - Temp2;

      *R = HueToRGB(Temp1, Temp2, H + 120.0f);
      *G = HueToRGB(Temp1, Temp2, H);
      *B = HueToRGB(Temp1, Temp2, H - 120.0f);
      }
   }

For more technical information, refer to Computer Graphics: Principles and Practice in C, James D. Foley et al., Addison-Wesley Publishing Company, United States, 1995.

RGB TO H RGB TO L L TO RGB


JPEG and JPEG2000 compression

Chapter 20: JPEG and JPEG2000 compression
Availability
Available in MIL-Lite with restrictions

Available in MIL

This chapter describes how to compress and decompress images.

  • Introduction
    • JPEG lossless
    • JPEG lossy
    • Interlaced JPEG
    • JPEG2000 lossless and lossy
    • Control options
  • General steps
    • Compression
    • Decompression
    • Sequences
    • Multi-band buffers, color formats, and control settings - JPEG
    • Multi-band buffers, color formats, and control settings - JPEG2000
    • Application-specific markers
  • Controlling a JPEG compression
    • JPEG lossless
      • Predictive coding
      • Huffman encoding
    • JPEG lossy
    • Restart markers
  • JPEG2000
    • Preparation of source image
    • Discrete wavelet transform (DWT)
    • Quantization
    • Bit-plane decomposition
    • Arithmetic encoding
    • Post-processing
  • Improving results
  • Working with tables
    • Inquiring values in default tables
    • Using your own table


Introduction

See Also | Availability | Next
User Guide:
JPEG2000
Available in MIL-Lite with restrictions

Available in MIL

MIL allows you to compress and decompress images and sequences. Compression allows you to store more images in memory than would normally be possible. In addition, compression allows images to be transferred more quickly, since it reduces the amount of data that must be transferred. MIL supports both lossy and lossless JPEG and JPEG2000 (and JP2) compression algorithms.

MIL-Lite compression support is reliant upon the presence of Matrox compression acceleration hardware. The compression algorithm supported depends on the hardware.

JPEG lossless

The JPEG lossless algorithm compresses images without any loss of information. Typically, the algorithm compresses images by a factor of 2:1, although a factor of 4:1 can sometimes be achieved. The JPEG lossless algorithm can compress 8- or 16-bit buffers with 1 or 3 bands.

JPEG lossy

The JPEG lossy algorithm compresses images by a variable factor but introduces some loss of information. The higher the compression factor, the more the compression, but the lower the image quality. The JPEG lossy algorithm can compress 8-bit buffers with 1 or 3 bands. To be compatible with most image-viewing software, MIL allows you to store compressed color images in YUV format.

Interlaced JPEG

MIL can perform a JPEG compression such that the image data is stored in separate fields. This is referred to as an interlaced JPEG compression. Unless otherwise stated, everything that applies to a JPEG compression also applies to an interlaced JPEG compression.

JPEG2000 lossless and lossy

JPEG2000 is another standard for image compression. When compared with regular JPEG lossy compression at the same compression ratio, JPEG2000 lossy compression provides better image quality. JPEG2000 lossless compression permits a smaller buffer size while retaining the same image quality as regular JPEG lossless compression. The JPEG2000 lossy and lossless algorithms can compress 8- or 16-bit buffers with 1 or 3 bands (RGB or YUV). For a more detailed description of supported features, see the JPEG2000 section later in this chapter.

In addition to saving files in the regular JPEG2000 lossless or lossy file format, it is also possible to save files in the JPEG2000 lossless format or lossy format using JP2 standard. While the JPEG2000 lossless and lossy file formats are only a dump of the compressed buffer's data, the JP2 lossless and lossy file formats contain the dump of the compressed buffer's data, as well as some additional information. For example, the additional information could be intellectual property rights, optional color palette, color space description, or application-specific data. Note that MIL adds the JP2 header when the buffer is exported to a file for the first time. The most important information in the JP2 header is the color specification because it allows you to view an image in color even if you are not using MIL. If the image was previously loaded from a JP2 file, any additional JP2 information is resaved with the image.

Control options

MIL allows you to control certain aspects of a compression. For example, you can use your own compression tables, although the default tables are suitable for most applications.


General steps

Reference:
MbufControl
MbufCopy
MbufExport
MbufExportSequence
MbufImport
MbufImportSequence
MdigGrab
Available in MIL-Lite with restrictions

Available in MIL

This section discusses general steps in compressing and decompressing images and sequences, as well as other aspects of compression.

Compression

To compress an image and save it in an image buffer:

  1. Allocate a buffer in which to hold the compressed image. Use MbufAlloc...() with M_COMPRESS combined with a compression attribute value.

    2. Compressed buffers that are created using the MbufCreate...() functions should not be used as the destination buffer of a MIL-Lite function. However, if a buffer with an M_COMPRESS attribute is used as a source buffer for an operation, the data will be decompressed depending on the attributes of the destination buffer.

  2. If necessary, change the control settings of the buffer, using MbufControl().

    3. For example, for a JPEG or JPEG2000 lossy compression, you might want to change the quantization factor (M_Q_FACTOR), which is one of the factors that determine the amount of compression. The default value of the quantization factor is 50; setting a lower value will produce marginal improvement in image quality and will result in a larger file size; setting a higher value will produce a smaller file, and therefore a poorer quality image.

  3. If the image to compress is stored in a buffer, use MbufCopy() to compress it into the buffer allocated in step 1. If it is stored in a file, use MbufImport().

    4. You can also automatically compress your grabbed images. To do so, use MdigGrab() with a destination buffer that has an M_GRAB + M_COMPRESS + CompressionType attribute. Note that due to the computational complexity of JPEG2000 compression, grabbing into such a buffer presents a risk of missing frames.

    Compression operations are optimized when the uncompressed source buffer and the compressed destination buffer are in the same format. Typically, buffers in YUV16 format produce the best compromise for quality and speed.

Note that, if you want the compressed image stored on file rather than in a buffer, use MbufExport() instead of MbufCopy(). In this case, there is no need to allocate a destination buffer.

Decompression

To decompress an image, use MbufCopy(), MbufImport(), or MbufExport(), depending on where the source image is stored (in a buffer or on file) and where you want results written (to a buffer or file).

Before the decompression, you should not change any control settings in the source image; the same controls must be used for decompression, otherwise the image data will be lost. The only exception to this rule is for JPEG2000 lossy compression, where you can change the target size of the image (the M_TARGET_SIZE control type).

Decompression operations are optimized when the compressed source and uncompressed destination buffers are in the same format. Typically, buffers in YUV16 format produce the best compromise for quality and speed.

Decompressing a JPEG buffer into a YUV16 packed (YUYV) buffer might accelerate transfer to the display.

Sequences

When compressing sequences, you can use MbufImportSequence() to import a sequence of images from an audio video interleave (AVI) file into separate compressed buffers. You can use MbufExportSequence() to export a sequence of compressed image buffers to an AVI file.

Multi-band buffers, color formats, and control settings - JPEG

When you allocate a multi-band buffer for a JPEG lossy compression, you can specify that the compressed image be stored in an RGB or YUV format. YUV is convenient because most image-viewing software support compressed color images in YUV16 format.

If you are performing a JPEG lossy compression on a YUV image, you can use the control types that specify luminance (such as M_QUANTIZATION_LUMINANCE) to control the Y band; to control the U and V bands, use the control types that specify chrominance (such as M_HUFFMAN_DC_CHROMINANCE). The control types without these suffixes control all bands. See the MIL Reference for the list of YUV-specific control types.

When the specified compressed buffer format differs from that of the source image, MIL will internally convert the source image to the specified format before performing the compression.

Multi-band buffers, color formats, and control settings - JPEG2000

When you allocate a multi-band buffer for a JPEG2000 compression, you can specify that the compressed image be stored in an RGB or YUV format. If you are compressing a multi-band buffer in JPEG2000, you can specify different control settings for each band. To do so, add M_RED, M_BLUE, or M_GREEN to your control type (for a YUV buffer, add M_Y, M_U, or M_V). Using the control type alone or with M_ALL_BANDS will control all bands.

Application-specific markers

During a compression, MIL adds some application-specific markers to the resulting image. Most other packages will ignore these markers and therefore be able to decompress the file. MIL itself ignores unrecognized markers when it decompresses files.

COMPRESS COMPRESS Q FACTOR GRAB COMPRESS TARGET SIZE QUANTIZATION LUMINANCE HUFFMAN DC CHROMINANCE RED BLUE GREEN Y U V ALL BANDS


Controlling a JPEG compression

User Guide:
Improving results
Working with tables

Reference:
MbufControl
Available in MIL-Lite with restrictions

Available in MIL

This section provides a brief overview of the JPEG lossless and lossy algorithms and of the controls you have over these algorithms. In general, you should only change these controls if you are familiar with the algorithm you are using. For detailed information about the JPEG lossless and lossy algorithms, see Information technology -- Digital compression and coding of continuous-tone still images: Requirements and guidelines, which is available from the International Standards Organization (www.iso.ch). For techniques to use to improve compression operations, see the Improving results section later in this chapter.

JPEG lossless

The JPEG lossless algorithm is basically a two-step process. First, predictive coding is performed on the image. Then, the result is Huffman encoded.

Predictive coding

Predictive coding is based on the fact that adjacent pixels in an image generally have similar values. Therefore, the value of a pixel can be "predicted" from the values of its neighbor(s). The difference between the original value of the pixel and the predicted value requires fewer bits to store than the original pixel value.

MIL supports three types of predictive coding: predictor #0 (no predictor), predictor #1 (the "pixel-to-the-left" predictor), and predictor #2 (the "pixel-above" predictor). By default, MIL-Lite uses the pixel-to-the-left to predict values, which is suitable for most images. In some applications, you might prefer to use the pixel-above predictor. You can also specify no predictor (predictor #0), but note that in this case, the values after predictive coding will be the same as the original values. This predictor can be useful if you have developed your own algorithm to take the place of predictive coding and only need your images Huffman encoded. Note that you must implement your own algorithm to use one of the other "predictors" supported by the JPEG lossless algorithm. You can specify the predictor with the MbufControl() M_PREDICTOR control type.

Huffman encoding

After an image has been predictive coded, Huffman encoding assigns a variable-length "code word" to each value. This code is based on the number of bits by which adjacent values differ. Values are assigned code words according to a DC Huffman table. You can use the default DC Huffman table or you can create your own table. If you want to use your own table, see the Working with tables section later in this chapter.

JPEG lossy

The JPEG lossy algorithm is outlined below. First the source image must be in the correct format before it can be compressed. Although the JPEG algorithm requires signed source data, the algorithm accepts both signed and unsigned data. Initially the algorithm internally treats all data as unsigned. Then a computational shift is performed to set all the values to signed.

After the computational shift, a color conversion is performed if the source and destination buffers are in different formats, for example an RGB source buffer and a YUV destination buffer. Note that conversion to YUV introduces some loss.

Afterwards, each 8x8 block of the image is represented in its frequency domain through a discrete cosine transform, resulting in 1 DC and 63 AC values. Each block is then quantized and Huffman encoded.

Quantization divides each of the 64 values in a block by a specified value, according to a quantization table. After each block is quantized, Huffman encoding assigns a variable-length "code word" to each value. Each DC value in a block is assigned a code word according to a DC Huffman table. The AC values are assigned a code word according to an AC Huffman table. You can control a JPEG lossy compression by using your own quantization and/or Huffman tables.

Restart markers

When an image is compressed, MIL adds restart markers to the bit-stream of the compressed image. A restart marker is a special code that signifies that the encoded bit-stream has been padded to the next byte boundary before the encoding process was restarted. Restart markers can be useful if you are transmitting the compressed image over a medium that is susceptible to errors. If an error does occur and there are no restart markers, the error will propagate and affect subsequent data. However, if there are restart markers, the error will be confined to the data between markers.

By default, MIL places restart markers after a certain number of rows of data have been encoded (for lossless compressions) or after a certain number of 8x8 blocks of data have been encoded (for lossy compressions). If necessary, you can use the MbufControl() M_RESTART_INTERVAL control type to change the number of rows or blocks between restart markers.

For a lossy compression with a high compression ratio, too many restart markers can significantly increase the size of the compressed image. In this case, you might want to increase the number of blocks between restart markers, especially if you are not transmitting the image over a noisy medium. In fact, if you are sure that the transmission medium is not noisy, you might want to set the restart interval to 0, that is, not use restart markers. This will increase the compression ratio, as well as reduce the time required to decompress the image.

PREDICTOR RESTART INTERVAL


JPEG2000

User Guide:
Improving results
Working with tables

Reference:
MbufControl
MbufInquire
Available in MIL-Lite with restrictions

Available in MIL

This section provides a brief overview of the JPEG2000 lossy and lossless algorithms and the control you have over these algorithms. Everything applicable to lossy compression is applicable to lossless compression, unless otherwise stated. In general, you should only change compression controls if you are familiar with the algorithm (except for the quantization factor, which you can change without in-depth knowledge, using MbufControl() with M_Q_FACTOR). For techniques to use to improve compression operations, see the Improving results section later in this chapter. For more detailed information about the JPEG2000 algorithm, see www.jpeg.org.

There are two fundamental differences between the JPEG2000 and regular JPEG compression algorithms. The first is the use of a discrete wavelet transform (DWT), instead of a discrete cosine transform. The second is the use of arithmetic encoding instead of Huffman encoding as the entropy encoding technique; arithmetic encoding reduces the number of bits required to encode data, based on how frequently that data occurs in the image.

When compared to regular JPEG compression, JPEG2000 supports much higher ratios of compression without compromising image quality. Note however, that JPEG2000 is best suited for archiving purposes because of the processing time required to compress and decompress images; therefore, JPEG2000 can be used for grabbing, but it presents a risk of missing frames.

When compressing images with the JPEG2000 algorithm, MIL supports:

  • Custom quantization tables (using MbufControl() with M_QUANTIZATION).
  • Custom wavelet settings (setting the number of decomposition levels using MbufControl() with M_DECOMPOSITION_LEVEL).
  • Saving files either in regular JPEG2000 format or in JPEG2000 format with JP2 standard, as well as saving AVI sequences in JPEG2000 format.
  • Band-specific compression settings.
  • Easy control over image size and quality (setting the quantization factor using MbufControl() with M_Q_FACTOR).
  • Specifying a target size for the compressed image when performing a JPEG2000 lossy compression (using MbufControl() with M_TARGET_SIZE).

Note that MIL does not support the following JPEG2000 features:

  • Progressive encoding/decoding (layering).
  • Random bit-stream access (tiling and/or ROI coding).
  • The JP2 file format optional features.
  • Error resilience (not typically used in imaging applications).

The JPEG2000 compression consists of the following steps:

  1. Preparation of source image.
  2. Discrete wavelet transform.
  3. Quantization (lossy only).
  4. Bit-plane decomposition.
  5. Arithmetic encoding.
  6. Post-processing (lossy only).

Preparation of source image

Although the JPEG2000 algorithm requires signed source data, the algorithm accepts both signed and unsigned data. Initially the algorithm internally treats all data as unsigned. Then a computational shift is performed to set all the values to signed.

After the computational shift, a color conversion is performed if the source and destination buffers are in different formats, for example an RGB source buffer and a YUV destination buffer. Note that conversion from RGB to YUV naturally introduces some loss.

Discrete wavelet transform (DWT)

After preparation of the source image, a lossy or lossless discrete wavelet transform (DWT) is applied to the image. The discrete wavelet transform both subsamples and spatially filters the image. Depending on the type of compression, the DWT uses one of two sets of filters: for lossy compression, a Daubechies 9-tap/7-tap filter is used, while a 5-tap/3-tap filter is used for lossless compression. The DWT subsamples and then separates the data into high frequency areas and low frequency areas; these areas are referred to as sub-bands. Each iteration, which consists of one pass of both the high and low frequency filters in both the horizontal and vertical directions, results in four sub-bands. Subsequent iterations of the transform are always applied on the top-left (low frequency) sub-band. MIL refers to each iteration of the DWT as a decomposition level.

The default number of decomposition levels is 5, unless the width or the height of the smallest sub-band is less than 16 pixels; in which case, the number of decomposition levels is chosen so that the width or the height of the 4 smallest sub-bands of the image is 1 pixel. You can override the default using MbufControl() with M_DECOMPOSITION_LEVEL. For example, if your original image is 1024 x 768, by default, the number of decomposition levels is 5, and the size of the 4 smallest sub-bands is 32 x 24. If you set the M_DECOMPOSITION_LEVEL control type to 6, then the size of the smallest sub-bands would be 16 x 12. The largest number of decomposition levels is the one that yields a width or a height of 1 pixel for the 4 smallest sub-bands. After allocating the image buffer, you can inquire the number of decomposition levels that will be used, using MbufInquire() with M_DECOMPOSITION_LEVEL.

After the wavelet transform, each pixel in the original image becomes a wavelet coefficient. There will always be as many wavelet coefficients as there were pixels in the original image.

After the wavelet transform is applied, MIL no longer considers the data as an image.

Sub-bands that are larger than 64 x 64 are further segmented into as many 64 x 64 regions as possible. These regions are referred to as code-blocks. Smaller sub-bands are not segmented, and each contains one code-block. In the diagram below, the sub-band that is 128 x 128 contains four code-blocks, while the sub-bands that are 64 x 64, 32 x 32, and 16 x 16 each contain one code-block.

Segmentation optimizes compression by grouping areas with similar values; when the values to compress are similar, the result is a better compression ratio. Segmentation also optimizes access to Host memory, thereby improving the performance of the quantization (for JPEG2000 lossy compression), bit-plane decomposition, and arithmetic encoding steps in the algorithm.

Quantization

During JPEG2000 lossy compression, the wavelet coefficients contained in each code-block are quantized based on both the sub-band in which they fall, and an entry in the quantization table. Quantization multiplies each wavelet coefficient by its sub-band's corresponding entry, the quantization coefficient, in the table. The sub-band's rank in significance, which is illustrated below, determines which entry in the quantization table is used. Note that the top-left sub-band is the most significant.

JPEG2000 lossless compression does not use the quantization step.

MIL automatically determines the values for the quantization table based on the number of sub-bands resulting from the DWT transform; that is, all images of the same depth that have five decomposition levels will use the same quantization table. The first entry (for sub-band 0) always has the largest quantization coefficient.

When you adjust the quantization factor (MbufControl() M_Q_FACTOR control type), MIL scales all the quantization coefficients in the quantization table according to the specified factor. If you want more control, you can pass your own custom quantization table. To establish good values for a custom quantization table, you can inquire the default quantization coefficients. For more information, see the Working with tables section later in this chapter.

Bit-plane decomposition

After the wavelet coefficients are multiplied by the quantization coefficients, the code-blocks are decomposed into bit-planes. A bit-plane contains the nth bit of all the multiplied wavelet coefficients that occupy a specific code-block. Decomposition into bit-planes is necessary for arithmetic encoding and post-processing.

A specific code-block might not have the same number of bit-planes as another code-block. Decomposition starts from the most-significant non-zero bit-plane of a code-block, and all subsequent bit-planes are decomposed, even eventual zero bit-planes. For example, the code-block in the diagram below contains coefficients that are 3 bits, and therefore will be decomposed into 3 bit-planes.

Arithmetic encoding

The bit-planes are then passed to the arithmetic encoder. The arithmetic encoder analyzes the bit-plane data to find redundancies, in other words, patterns of bits that occur frequently in the bit-planes. Using this information, the arithmetic encoder replaces repeating bit patterns, also called symbols, by shorter bit sequences. The more often a symbol occurs in the bit-plane data, the fewer bits into which that symbol will be coded.

During lossless compression, the arithmetic encoder processes each bit-plane to produce a continuous bit-stream. Since arithmetic encoding is the final stage of compression, this bit-stream is actually the compressed image.

During lossy compression, the arithmetic encoder produces several independent bit-streams; the post-processing step determines which bit-streams are concatenated in the final image.

Post-processing

Once the most-significant bit-plane of a code-block is encoded, its size in bytes is stored in memory, and MIL calculates how much error would be present in the decompressed image if the remaining bit-planes, for that code-block, were not arithmetically encoded.

Once the second most-significant bit-plane of the code-block is arithmetically encoded, MIL calculates the error introduced in the decompressed image if only the first two bit-planes were compressed. As each remaining bit-plane is encoded, MIL repeats the same calculation until all the bit-planes have been processed.

As an example only, the values in the following table represent the amount of distortion (error) with respect to the size in bytes of the encoded bit-planes of one code-block.

Bit-plane (in order of significance) | Compressed size in bytes | Distortion
1 | 5 | 58817925.11
2 | 54 | 34495584.39
3 | 159 | 11160557.81
4 | 296 | 2633195.13
5 | 436 | 594555.28
6 | 574 | 94112.98
7 | 717 | 21139.81
8 | 863 | 4080.74
9 | 980 | 0.00

Then, the best compromise between decompressed image quality and requested size in bytes (MbufControl() M_TARGET_SIZE) is determined. Any encoded bit-planes that are not required are discarded after the best compressed image is determined for the specified size. The remaining encoded bit-planes are then concatenated in the final image.

Determining an appropriate target size will require some trial and error, but a logical guess can be calculated by dividing the image's original size (in bytes) by the required compression ratio. For example, if you want to compress a 256-byte image by a factor of 16:1, you would specify a target size of 16 bytes.

Post-processing is not performed in JPEG2000 lossless, and none of the encoded bit-planes are discarded; therefore, you cannot specify a target buffer size when performing a lossless compression.

Q FACTOR QUANTIZATION DECOMPOSITION LEVEL Q FACTOR TARGET SIZE DECOMPOSITION LEVEL DECOMPOSITION LEVEL DECOMPOSITION LEVEL Q FACTOR TARGET SIZE


Improving results

User Guide:
Working with tables

Reference:
MbufControl
Available in MIL-Lite with restrictions

Available in MIL

If the defaults do not meet your application requirements, you can try to improve your compression ratio using the following techniques. We recommend trying these techniques in the order they appear.

Regardless of the type of your compression operation, you should first remove extraneous noise from the image (if possible) using MIL processing functions.

For JPEG lossy compression:

  • Allocate a YUV buffer for compression.
  • Increase the quantization factor with the MbufControl() M_Q_FACTOR control type.
  • Decrease the restart interval.
  • Change the quantization table (see the Working with tables section later in this chapter).
  • Change the Huffman table (see the Working with tables section later in this chapter).

For JPEG lossless compression:

  • Try the other supported predictors with the MbufControl() M_PREDICTOR control type.
  • Decrease the restart interval.

For JPEG2000 lossy compression:

  • Allocate a YUV buffer for compression.
  • Increase the quantization factor with the MbufControl() M_Q_FACTOR control type.
  • Decrease the target size with the MbufControl() M_TARGET_SIZE control type.
  • Change the number of decomposition levels of the DWT with the MbufControl() M_DECOMPOSITION_LEVEL control type.
  • Pass another quantization table (see the Working with tables section later in this chapter).

For JPEG2000 lossless compression:

  • Change the number of decomposition levels of the DWT with the MbufControl() M_DECOMPOSITION_LEVEL control type.
Q FACTOR PREDICTOR Q FACTOR TARGET SIZE DECOMPOSITION LEVEL DECOMPOSITION LEVEL


Working with tables

See Also | Availability | Previous
Reference:
MbufControl
MbufInquire
MbufPut1d
MbufPut2d
Available in MIL-Lite with restrictions

Available in MIL

In some applications, the default quantization or Huffman tables might not be suitable. MIL allows you to create your own. You can inquire the default table values to help you determine appropriate values. You might have to select values by trial and error to determine the best ones for your application.

For JPEG2000 compression, quantization multiplies coefficients, while in JPEG compression, quantization divides values.

Whether you are inquiring the default tables or customizing your own, you must allocate arrays that are large enough to contain the data. The table below lists the tables that you can manipulate and their required array size for each compression algorithm.

Compression algorithm | Table type | Buffer type, size, and attribute
JPEG lossless | DC Huffman | 1-dimensional, 8+ M_UNSIGNED, 28 entries, M_ARRAY
JPEG lossy | DC Huffman | 1-dimensional, 8+ M_UNSIGNED, 28 entries, M_ARRAY
 | AC Huffman | 1-dimensional, 8+ M_UNSIGNED, 178 entries, M_ARRAY
 | Quantization | 2-dimensional, 8+ M_UNSIGNED, 8 x 8 entries, M_ARRAY
JPEG2000 lossy | Quantization | 1-dimensional, 32+ M_FLOAT, 3N + 1 entries, where N is the number of decomposition levels, M_ARRAY

Inquiring values in default tables

Inquiring the default values of a table is useful to determine values for your custom tables. The steps below outline this procedure.

  1. First, inquire the MIL identifier of the default table using MbufInquire(). Then, inquire the size of the table using the same function.
  2. Allocate a user array of the appropriate size for storing the default table values.
  3. Get the values from the inquired table in Step 1 and store them in the user array using MbufGet...().

Using your own table

To use your own table:

  1. Allocate a buffer with an M_ARRAY attribute and of the same data type as the default table.
  2. Transfer the custom table values from the user array to the array buffer, using MbufPut1d() or MbufPut2d(), depending on the type of table.
  3. Associate the M_ARRAY buffer to the required M_COMPRESS image buffer, using the MbufControl() control types specific to your table (for example, use the M_HUFFMAN_AC control type for an AC Huffman table). Specifying these control types as-is, or combined with M_ALL_BANDS, controls all bands for JPEG2000. This is the default setting.

    4. For a JPEG2000 compression, you can associate a different table with each band of a multi-band buffer. To do so, add M_RED, M_BLUE, or M_GREEN to your control type for an RGB buffer, whereas for a YUV buffer, add M_Y, M_U, or M_V.

    For JPEG lossy compressions of YUV buffers, use the luminance and chrominance control types. The control types without these suffixes control all bands.

If you set the M_Q_FACTOR control type after specifying a custom table, the custom table will be scaled.

UNSIGNED ARRAY UNSIGNED ARRAY UNSIGNED ARRAY UNSIGNED ARRAY FLOAT ARRAY ARRAY ARRAY COMPRESS HUFFMAN AC ALL BANDS RED BLUE GREEN Y U V Q FACTOR


Data manipulation with multiple systems

Chapter 21: Data manipulation with multiple systems
Availability
Available in MIL-Lite

Available in MIL

This chapter describes how to manipulate data with different systems.

  • Data manipulation with multiple systems


Data manipulation with multiple systems

See Also | Availability |
User Guide:
Specifying and managing your data buffers

Reference:
MbufControl
MbufCopy
Available in MIL-Lite

Available in MIL

To use multiple Matrox imaging boards, you have to allocate a MIL system for each board.

To perform a processing operation, your source and destination buffers can be on different systems; MIL will transparently copy buffers to the most efficient of these system, if necessary.

To exchange data between systems, you can physically copy the data from one system to another. The copy is always performed by the most suitable system. If both systems are of the same type, the copy is always performed by the destination system.

Instead of performing a physical copy using MbufCopy(), you can allocate a buffer on one system and use MbufCreate...() to access this buffer from another system. MbufCreate...() creates a buffer that maps to allocated memory (for example, on the Host or any MIL system); no memory is actually allocated to this newly created buffer.

The second method can be used, for example, to update a buffer (or part of it) with data grabbed from different systems. Note that after writing to the created buffer, you should notify the real buffer that its contents have been changed, by calling MbufControl() with M_MODIFIED. For more information about creating data buffers, see Chapter 14: Specifying and managing your data buffers.

To grab, the digitizer and the destination buffer must be allocated on the same MIL system. Similarly, to display a buffer, the display and the buffer must be allocated on the same MIL system.

Systems without an on-board display section use the VGA for display. Therefore, under Windows, such systems will automatically display together on the same screen.

MODIFIED


Using MIL with multi-processing and under multi-thread systems

Chapter 22: Using MIL with multi-processing and under multi-thread systems
Availability
Available in MIL-Lite

Available in MIL

This chapter describes how MIL handles multi-processing and multi-threading.

  • Multi-processing
  • Multi-threading
    • MIL and multi-threading
      • Creating threads using MIL
      • Thread execution
      • Synchronization
      • Thread control
      • Error reporting


Multi-processing

Availability | Next
Available in MIL-Lite

Available in MIL

Multi-processing is the ability to execute various processes (applications) simultaneously.

MIL applications are autonomous processes (or executables) designed to execute a complete operation or series of operations. Therefore, they can profit from multi-processing by executing independently, without interference from each other.

Not all systems support multi-processing. For example, a simple imaging board with only acquisition capability (such as Matrox Meteor-II) cannot ensure either the response time to a function or the independence of a process necessary for multi-processing. On such boards, two different processes cannot use the same system simultaneously. To use a non-multi-processing system within a multi-processing environment, all processes that need to communicate with the system must do so by sending their requests through a single dedicated process.

Systems that support multi-processing, such as Host or Matrox Odyssey, have on-board resources (such as, processors) that can be shared by different processes. However, multiple processes running at the same time have to share the available processing time, and cannot share data, since the memory used by one process is protected from all other processes.


Multi-threading

See Also | Availability | Previous
User Guide:
Executing the slave function on a system with an on-board processor

Reference:
MappAlloc
MappGetError
MbufGet
MdigControl
MdigGrab
MdigHookFunction
MthrAlloc
MthrControl
MthrFree
MthrInquire
MthrWait
Available in MIL-Lite

Available in MIL

MIL also supports multi-threading. Multi-threading is the ability to perform multiple operations simultaneously in the same process. This is done by creating different threads (execution queues) to ensure sequential execution of operations within the same thread, while allowing simultaneous yet independent execution of operations in other threads.

Threads within a process share the same data. Individual threads can communicate with each other and exchange data such as MIL identifiers.

Multi-threading is most appropriate for applications where independent tasks can be done simultaneously but need to share data or to be controlled and synchronized within a main task.

Multi-threading does not always result in an increase of speed and efficiency. Threads running simultaneously on the same CPU share the same system resources (such as memory). When using a machine with multiple CPUs under Windows, the threads generally run on separate CPUs and provide more processing power. However, since they share the same memory, operations that are I/O intensive and require only simple processing might not be accelerated.

Most applications do not require the use of multiple threads since there are other ways of multi-tasking. Mechanisms such as asynchronous grab and call-back functions can be used (see MdigControl() and MdigHookFunction()). Applications resolved by alternative means are often simpler to implement and easier to maintain than multi-threaded applications.

MIL and multi-threading

When your application contains several independent processing tasks that can be performed in parallel, you can design it so that each part is controlled by a separate thread (or task).

Creating threads using MIL

Under multi-thread operating systems, you can create as many threads as you require. You can create threads using commands provided by the operating system, or using the MthrAlloc() function provided by MIL. The MthrAlloc() function is portable, which means that it can be called from user-defined MIL functions that are executed on systems with on-board processors (for example, Matrox Odyssey). For information on executing functions on systems with on-board processors, see the Executing the slave function on a system with an on-board processor section in Chapter 25: The MIL Function Developer's Toolkit.

There are two methods for creating threads using the MthrAlloc() function:

  • With the first method (M_THREAD), the MthrAlloc() function creates a MIL thread context for the new thread, and allows you to specify a pointer to a function that will be executed by the thread. This user-created function must include all operations, and call all of the functions that you consider as being part of one thread. When a thread contains a function call whose target processor is an on-board processor that supports multi-threading, MIL automatically creates a corresponding thread on that system's on-board processor. The functions of the thread module allow you to synchronize threads running on the Host and/or various MIL systems.
  • With the second method, MthrAlloc() creates a selectable thread (M_SELECTABLE_THREAD). Selectable threads are threads executed on an on-board processor that supports multi-threading and can be controlled from a single corresponding thread on the Host. Use MthrControl() with M_THREAD_SELECT to send MIL functions to be executed by a selectable thread.

For each new thread calling MIL functions, MIL creates a new default MIL application context and initializes it to the state of the main MIL application (the first application allocated with MappAlloc()). The application context allows the new thread to behave as an independent application, respecting all of MIL environment controls, such as error reporting.

To set the thread's application context to its own initial state (not necessarily the same as the main MIL application), allocate a new application using MappAlloc(). Note that the call to MappAlloc() must be the first MIL function call in the thread.

Thread execution

MIL functions in any thread are executed as follows:

  • If the target processor is the Host CPU, processing in each thread is determined by the operating system.
  • If the target processor is an on-board processor of a system that supports multi-threading (such as Matrox Odyssey), MIL automatically creates, and eventually terminates, an on-board thread for each thread that sends commands to the board.

Important
Since the creation of on-board threads is done automatically, you do not have to specify the system on which to create a specific thread. However, you can do so by creating selectable threads on a particular system.

Synchronization

Thread synchronization is generally done using the Host synchronization services (such as Windows event objects). However, when using a system with an on-board processor, the activities of this processor cannot be synchronized by the Host.

This means that threads continue execution without waiting for the execution of the on-board functions to complete. In most cases, this behavior is acceptable, since it leaves the Host available for other tasks. However, for operations that require sequential execution of functions to return valid results (for example, MbufGet() after an MdigGrab()), MIL automatically synchronizes the threads, forcing the Host to wait for completion of the earlier function(s).

Explicit synchronization of threads is necessary if functions sharing a common resource might conflict with each other. For example, if two threads sharing the same image buffer are not synchronized, and each thread tries to clear the buffer to a different value, these functions might execute at the same time and the buffer could be cleared to either value or even to a combination of both. Use the synchronization features of the MthrControl() and MthrWait() functions to control the flow of such commands.

Thread control

Windows operating systems are multi-process and multi-thread operating systems. They provide various thread control services, including events (used to synchronize threads).

The MthrAlloc() function serves as a link between MIL and the operating system; the function allows you to create an operating-system-independent MIL version of these services. Threads and events created using MthrAlloc() can be used in addition to, or instead of, the events created using commands of the operating system.

MthrControl() controls and coordinates both threads and events. The MthrWait() function synchronizes thread processing by forcing a "wait" state. The MthrInquire() function inquires about both the settings of a MIL thread context and the state of an event allocated using MthrAlloc(). MthrFree() frees the allocated MIL thread context or event.

The MthrAlloc() function allows you to specify a particular system on which to allocate MIL thread contexts or events. This permits you to synchronize the execution of functions on a specific MIL system. Also, you can create events whose identifiers will be recognized by multiple systems with on-board processors, using MthrAlloc() with M_EVENT_CREATE.

Error reporting

Some functions in MIL are asynchronous, that is, they queue their command to the hardware and then immediately return control to the Host. Errors are logged once the function is executed on the processor of the specified system, and are reported immediately after being logged.

To check for errors, use the MappGetError() function. In multi-thread environments, a call to MappGetError() returns the last error that occurred in the current thread or, if none, checks for errors in other threads running MIL. To return only errors occurring in the current thread, add M_THREAD_CURRENT to the ErrorType parameter (M_CURRENT + M_THREAD_CURRENT).

THREAD SELECTABLE THREAD THREAD SELECT EVENT CREATE THREAD CURRENT CURRENT THREAD CURRENT


Using MIL with Native Mode Functions

Chapter 23: Using MIL with Native Mode Functions
Availability
Available in MIL-Lite

Available in MIL

This chapter covers the use of Native Mode functions with MIL.

  • Integrating native functions with MIL code
    • Portability
    • Signaling MIL about Native Mode use
  • A native mode example


Integrating native functions with MIL code

See Also | Availability | Next
Reference:
MbufControl
MsysControl
Available in MIL-Lite

Available in MIL

MIL allows you to mix board-specific code (from the native library function set) with its own code. This is useful when you need to access some board-specific functionality that is not supported directly by the MIL function set or to optimize a time-critical piece of code.

When programming in native mode through MIL, you use the same board driver and programmer's kit that are used by regular native mode programmers. The only difference is the need to use certain rules and functions to ensure proper communication between MIL and the native functions. These rules and functions allow you to enter and leave native mode from MIL and access MIL for information, such as the object native handle, concerning data objects on the target board.

Portability

You should note that applications containing native mode functions are not portable to other present or future Matrox platforms supported by MIL.

Signaling MIL about Native Mode use

MIL must be signaled when entering and leaving native mode and when MIL objects have been modified while in native mode, using MsysControl(). For buffer modification, MbufControl() can also be used to signal MIL.

On entering native mode, MIL does not affect the current state of either the board or the environment.

The M...Inquire() functions can be used to determine the buffer, digitizer, or display native identifier (handle) required to use the system's native library.

On leaving native mode, MIL assumes that the board is in the same state as when entering. Therefore, you must ensure that you return the board to the proper state before returning control to MIL. Inquiries about the board state must be made using the board's native library inquiry functions.


A native mode example

Availability | Previous
Available in MIL-Lite

Available in MIL

In this example, we use MIL mixed with Genesis native library code to grab and warp an image.

MNatGEN.c


Distribution and licensing

Chapter 24: Distribution and licensing
Availability
Available in MIL-Lite

Available in MIL

This chapter presents how to distribute MIL-Lite and license MIL applications.

  • Distribution of MIL applications
  • Redistributing MIL or MIL-Lite run-time DLL files and device drivers with your application
    • Redistributing directly from the MIL or MIL-Lite CD
    • Redistributing using your own setup program
  • Normal redistribution using your custom CD
  • Silent redistribution
    • Important notes when using Windows 2000/XP
  • Uninstalling
  • MIL and MIL-Lite licenses
    • Development license
    • Temporary license
    • Run-time license
  • Hardware license-key
  • Software license-key
  • Hiding the MIL license
    • Generating the lock code
    • Getting a software license-key
    • Registering the software license-key
  • Gencode utility
    • Synopsis
    • Prototype
    • Description
    • Parameters
  • Protecting your own MIL software application using a Matrox hardware fingerprint


Distribution of MIL applications

Availability | Next
Available in MIL-Lite

Available in MIL

There are some details that you must consider before you can distribute a MIL or MIL-Lite application, either for your customers' use or for your own. When doing this distribution, you must redistribute MIL-Lite run-time DLLs and device drivers. In addition, when using a MIL processing module, you must purchase an appropriate run-time license; otherwise, your MIL application will stop functioning after 30 days. This chapter deals with the different ways to distribute your application, purchase a license, and handle licensing concerns for your customers.

Note that MIL-Lite applications don't require a run-time license.


Redistributing MIL or MIL-Lite run-time DLL files and device drivers with your application

Availability | Previous | Next
Available in MIL-Lite

Available in MIL

To distribute your MIL or MIL-Lite application, you will have to redistribute MIL or MIL-Lite run-time DLL files and the necessary device drivers with your application.

It is important to remember that only one copy of MIL or MIL-Lite can be present on a computer at a time. When installing an application that uses MIL or MIL-Lite on a computer with the same version already installed, the application's set-up program must not reinstall MIL or MIL-Lite. The application should use the version of MIL or MIL-Lite already installed on the computer.

Conversely, if the version of MIL or MIL-Lite on the computer is different from the application's required version, a decision must be made. The latest version of MIL or MIL-Lite should be kept or installed, otherwise the application cannot be installed on that computer. Note that the application might need to be recompiled with the installed version of MIL or MIL-Lite.

Redistributing directly from the MIL or MIL-Lite CD

If the target computer (on which you want to install the MIL or MIL-Lite run-time DLLs and device drivers) is immediately accessible, you can install the run-time DLLs directly from the MIL or MIL-Lite CD. To do so, run the MIL or MIL-Lite setup program and choose the MIL run-time option.

Redistributing using your own setup program

Alternatively, to redistribute your MIL or MIL-Lite run-time DLLs and device drivers, you can have your application's setup program call MIL or MIL-Lite's redistribution setup program in one of two ways:

  • Normal redistribution. Prompts your customer for setup information.
  • Silent redistribution. Uses a custom response file, instead of prompting your customer for information.

These are described in detail in the next sections.


Normal redistribution using your custom CD

Availability | Previous | Next
Available in MIL-Lite

Available in MIL

To redistribute the MIL or MIL-Lite run-time DLLs and device drivers, you can have your application's setup program call MIL or MIL-Lite's redistribution setup program using the normal redistribution mechanism. In this case, your customer will be prompted for information during the setup. There are two ways to implement this mechanism.

To redistribute all the run-time DLLs and device drivers and use the default installation options, do as follows:

  1. Copy the contents of the MIL or MIL-Lite CD to your installation directory.
  2. Have your redistribution program call the setup.exe file, located in the root of the CD.
  3. The setup.exe program will prompt your customer to choose which features to install. Your customer should only select the Mil Runtime option. The Mil Runtime option installs the required run-time MIL or MIL-Lite DLL files and device drivers on your client's computer.

To limit the contents of the MIL or MIL-Lite CD to include in your installation directory and restrict the installation options, you can use the Redist.exe program. To use this program, do as follows:

  1. Launch the Redist.exe program located in the Redist directory of your MIL or MIL-Lite CD. The Matrox Imaging Library Redistribution dialog box is presented.
  2. Specify the location of the source files to copy to your installation directory. To do so, click on the Browse button next to the Source edit field. From the presented Open dialog box, select the setup.exe file located in the root of your MIL or MIL-Lite CD, and then click on the Open button. The Matrox Imaging Library Redistribution dialog box re-appears, displaying the selected location of the source files.
  3. Specify the location of your installation directory. To do so, click on the Browse button next to the Destination edit field. From the presented Choose folder dialog box, select your installation directory, and then click on the Select button. The Matrox Imaging Library Redistribution dialog box re-appears, displaying the selected location of your installation directory.
  4. Select the redistribution packages (MIL and/or Intellicam), the systems, and the drivers that you want to make available to your customer during installation. Note that if your customer is not using a hardware license-key or a hardware ID key, you can deselect the SuperPro (Hardware Key) option to avoid copying the SuperPro driver to your installation directory. In addition, if you customer does not have an on-board display section, you can deselect the Matrox video card option to avoid copying the MGA drivers to your installation directory.
  5. Click on the Generate button to begin copying the limited installation version of MIL or MIL-Lite into your installation directory.
  6. Have your redistribution program call the setup.exe file, located in the root of the specified installation directory.
  7. The setup.exe program will prompt your customer to choose which features to install from the features that you have made available.


Silent redistribution

User Guide:
Normal redistribution using your custom CD
Available in MIL-Lite

Available in MIL

A silent redistribution does not prompt your user for any MIL or MIL-Lite information; instead, it uses a response file to provide the necessary setup parameters for the intended computer. You would use a silent redistribution when you are including MIL or MIL-Lite within your application and you do not want to have any Matrox Imaging setup dialog boxes appear. You could also use a silent redistribution if you wanted to control the setup parameters for your client. There are two ways to use the silent redistribution.

It is not possible to create a response file for only a few of the setup parameters and ask the customer for the rest of the setup information. If you are going to use a response file, you must answer all of the setup questions in the response file.

To redistribute the MIL or MIL-Lite run-time DLLs and device drivers using a silent redistribution:

  1. Copy the contents of MIL or MIL-Lite CD to your installation directory. Alternatively, you can use the Redist.exe program to select the redistribution packages (MIL and/or Intellicam) and drivers to copy in your installation directory. For more details on how to use this program, see the Normal redistribution using your custom CD section earlier in this chapter.
  2. Record your setup options. To do so, start the setup.exe program from the command line with the /r switch and follow the steps for a normal MIL or MIL-Lite redistribution. Any selected setting is recorded and stored in the setup.iss file.
  3. Have your redistribution program call the setup.exe program with the /s switch. The setup will silently install MIL or MIL-Lite with the options that were selected during the recording phase and stored in the setup.iss file.

Important notes when using Windows 2000/XP

The following is a list of points to consider if using Windows 2000/XP:

  • Ensure that the operating system has a standard VGA display driver installed prior to installing the MGA drivers.
  • The MGA drivers are only installed for Matrox boards that have an on-board display section.


Uninstalling

Availability | Previous | Next
Available in MIL-Lite

Available in MIL

To uninstall previous versions of MIL or MIL-Lite, and all other Matrox Imaging Products, call the Matrox Imaging Products Uninstall program. To make uninstalling silent, use the silent uninstall program located in \SilentUninstall on the MIL or MIL-Lite CD.

For example, if you want to uninstall MIL and ActiveMIL in silent mode without rebooting your computer, call:

 uninstall.exe

or to reboot the computer after uninstalling, call:

 uninstall.exe reboot


MIL and MIL-Lite licenses

User Guide:
Gencode utility
Available in MIL-Lite

Available in MIL

MIL and MIL-Lite have different licensing terms and mechanisms.

When you purchase MIL-Lite, you receive a development license and a registration number. MIL-Lite also comes with a royalty-free run-time license, which allows you to redistribute applications based on MIL-Lite without paying additional royalty fees.

With MIL, you can purchase two types of permanent licenses: a development license and/or a run-time license. Licenses are always verified when a MIL application is allocated, as well as when the application is running; this verification has negligible overhead. You can use a temporary license while waiting for a permanent license.

Please refer to the Matrox Software License Agreement for the legal provisions of using/redistributing MIL or MIL-Lite.

The rest of this chapter, with the exception of the last section, deals exclusively with MIL and its licensing mechanisms. The last section pertains to both MIL and MIL-Lite.

Development license

A development license permits you to run and debug MIL applications with Microsoft Visual Studio. For the license to take effect, a development hardware license-key must be attached to your computer's parallel or USB port. A development hardware license-key is included with the MIL starter or additional developer package.

Temporary license

A temporary license is a 30-day development license. This type of license is most useful for developers who are performing a development or redistribution installation while waiting for a permanent license.

The temporary license is used when MIL is first executed, unless a development license is already present. However, a temporary license only lasts 30 days from the first launch of MIL on the computer. After allocating an application, if the temporary license is used, a dialog box appears indicating the number of days until the temporary license expires.

If a development license is removed, a temporary license dialog box will appear indicating the number of days until the license expires, if it is less than 30 days from the first launch of MIL on the computer. If it is more than 30 days, a dialog box will appear asking you if you want to purchase a run-time license. If you do not purchase a run-time license, you won't be able to use the application since MIL will have been disabled.

Once the temporary license has expired, it is not possible to prolong or restart the 30-day trial period. Attempting to alter the computer's calendar before the temporary license expires will immediately disable MIL. In that event, MIL can only be re-used once an appropriate license is installed.

Run-time license

A run-time license is required for each computer that will run a MIL application. It does not allow for development (debugging) of MIL applications.

Recall that MIL-Lite applications do not require a run-time license. They can be distributed royalty free.

MIL run-time licenses are purchased according to the requirements of the application. A MIL run-time license consists of different MIL modules, grouped in packages, for specific uses. Consult the MIL datasheet or contact Matrox Imaging sales or the local representative regarding the available run-time packages.

A MIL run-time license can be in one of two formats:

  • A hardware license-key.
  • A software license-key.

If you want to, you can hide the MIL license from your customer using the Gencode utility, a modified version of the MIL License Manager. Gencode is located in the same directory as the MIL License Manager. See the Gencode utility section later in this chapter.


Hardware license-key

Availability | Previous | Next
Not available in MIL-Lite

Available in MIL

A hardware license-key is a type of run-time license. For the license to take effect, the key must be attached to the parallel or USB port of your computer. This license can be bought by calling Matrox Imaging or your local representative.

If you decide that another computer needs to run a MIL application, you can shift the hardware license-key. This will allow the second computer to use MIL, but the first computer will have the temporary license dialog box appearing again. If this is done for more than 30 days after MIL was first used on the computer, MIL will be disabled.

To have MIL run-time DLLs recognize the hardware license-key, you must install the SuperPro driver. To install this hardware license-key driver, if you did not install it when you were installing MIL, call the \Matrox Imaging\Drivers\SuperProDriver\wdm\RainbowSSD.exe file and select the Install Sentinel Driver menu command.

For more information about Sentinel driver installation, refer to the DriverInfo.htm file located in the \Matrox Imaging\Drivers\SuperProDriver directory. This file also includes information on how to install the Sentinel driver silently.


Software license-key

User Guide:
Introduction
Not available in MIL-Lite

Available in MIL

A software license-key is another type of run-time license. The software license-key uses a fingerprint, generated from one of several components on the target computer, to produce the license.

You can obtain a license as follows:

  1. Install your MIL application on the target computer as described in Chapter 1: Introduction. A temporary license is assigned to your computer, allowing use of MIL for 30 days. Each time your application runs MIL, a dialog box appears indicating the number of days until the temporary license expires and a setting to obtain a permanent run-time license is displayed. If you click "Yes" to get the license, the MIL License Manager program will be launched. The License Manager program will also be launched if you attempt to use a MIL function that is not available in the purchased package.
  2. In the License Manager program, choose any combination of the different MIL packages for purchase, by enabling the required package's check box.

    3. Since the MIL Image Analysis package is included in the MIL Machine Vision package, these two packages are mutually exclusive.

  3. Select the hardware component, from which to generate a fingerprint, from the System Fingerprint drop-down list box. After choosing the hardware component, click the Generate button. This will create a lock code.

    4. If you replace the hardware component that you have chosen, you will need a new license.

    For portability or if your computer does not have any of the hardware components listed, you can purchase a hardware ID-key and use it as your hardware component instead. This gets attached to your computer's parallel or USB port, and it will allow you to generate a lock code that you can use to create a software license. If you attach the hardware ID-key to a different computer, the run-time license will only operate on the computer on which the hardware ID-key is installed. The hardware ID-key allows you to quickly shift the MIL license to another computer without removing a hardware component. It also allows you to include other MIL packages; a hardware ID-key allows you to choose the package you want, unlike a hardware license-key.

    You also have to copy over the lservrc file, which is located in the Windows\system32 directory under Windows 2000/XP on the target computer.

  4. After you have generated a lock code, the software license-key can be obtained at www.Shopmatrox.com or by contacting Matrox Imaging sales or your local representative.
  5. Return to the License Manager program, type or paste the software license-key into the Software License Key edit field, and click on the OK button. The licensing process is finished.


Hiding the MIL license

User Guide:
Gencode utility
Not available in MIL-Lite

Available in MIL

It is possible to make the presence of MIL licensing non-invasive so that your customer can install your MIL application and not have to deal with MIL licensing issues.

Perhaps the simplest way to hide the MIL licensing is to purchase and redistribute hardware license-keys. You will have to purchase a hardware license-key for each application that you are distributing.

If this is not possible or preferable, there is a utility, Gencode, that allows you to get a software license-key while hiding the MIL licensing process. It is essentially the silent command-line version of the MIL License Manager. Gencode can generate the lock code needed to get a MIL software license-key, as well as register the license.

Generating the lock code

There are many different ways of getting the information that the utility needs. You could create an interactive interface, or have a series of prompts during the setup of your application. If you know the hardware and the packages your client will be using, you could build a batch file which would call Gencode with the predetermined information. For example, if you know that your client will be using a Matrox Corona-II board, you could hardcode Gencode's command-line parameter SystemFingerprint to the constant that means Matrox Corona-II.

However implemented, the setup program of your application must internally call Gencode as follows to generate the lock code:

 Gencode /G Filename SystemFingerprint Packages

See the Gencode utility section later in this chapter for details.

Getting a software license-key

Once the code is actually created, there are some steps that must be followed.

  1. Your customer must send you the lock code, by whichever means you designate.
  2. Take the lock code and obtain the software license-key at Shopmatrox.com or by contacting Matrox Imaging Sales or your local representative.
  3. Your customer must input the software license-key into whatever registration interface you have created.

Registering the software license-key

Your registration interface must again call Gencode, this time as follows:

 Gencode /L LicenseCode

and input the license code. This will activate your MIL license.

Gencode's command-line parameters are described in the Gencode utility section later in this chapter.


Gencode utility

Availability | Previous | Next
Not available in MIL-Lite

Available in MIL

This section describes the Gencode utility.

Synopsis

Generates a lock code for licensing, and activates a software license-key for MIL.

Prototype

GENCODE /H or

GENCODE /G <Filename> <SystemFingerprint> <Packages> or

GENCODE /L <SoftwareKey> or

GENCODE /I

Description

The Gencode utility does one of the following:

  • With the /H switch, it displays a help screen.
  • With the /G switch, it generates a lock code that is necessary for getting a software license-key from the Matrox web site.
  • With the /L switch, it registers the license-key for running MIL on the current computer.
  • With the /I switch, it prints the numeric values of available settings of the SystemFingerprint parameter.

Parameters

The following is a list of the parameters available:

  • The Filename parameter is the name of the file that will be created to store the lock code. You can specify any file name; if the file name already exits, the lock code information will be appended to the existing file. The lock code can also be retrieved from the registry under:

    • HKEY_LOCAL_MACHINE\SOFTWARE\Matrox\Matrox Imaging Library\Runtime\Redist\ Licenses\ManagerOutput

  • The SystemFingerprint parameter designates the hardware component upon which the system fingerprint is based. This parameter can be set to one of the following numeric values:
    • SystemFingerprint settings | Description
7 | Ethernet Controller
8 | Hardware ID-key
20 | Matrox Solios
21 | Matrox Meteor-II /1394
22 | Matrox Orion
23 | Matrox Corona-II
26 | Matrox Odyssey
27 | Matrox CronosPlus
28 | Matrox Helios
29 | Matrox Morphis
31 | Matrox board

    If the specified board is not on the target computer, a 0 is returned.

  • The Packages parameter identifies which MIL packages are going to be purchased. This is something you would likely set for your customer, since you know which MIL modules your application uses.

    • The software license-key packages correspond to the following numeric values. If you need to buy more than one package, add their corresponding numeric values together. For example, to purchase the Machine Vision and Identification packages, set the Packages parameter to 6.

    Packages setting | Description
1 | Image Analysis
2 | Machine Vision
4 | Identification
8 | Compression/Decompression
16 | Geometric Model Finder
256 | Edge Finder
512 | Serial bus driver for third-party IEEE 1394 hardware
1024 | String Reader

    The one exception is that the Image Analysis and Machine Vision packages can not both be included in the value of the Packages parameter. Refer to the Matrox Imaging website for more information on these run-time license packages.

  • The SoftwareKey parameter is the software license-key code that you receive from Matrox or your local representative to finalize registration.


Protecting your own MIL software application using a Matrox hardware fingerprint

See Also | Availability | Previous
Reference:
MappInquire
Available in MIL-Lite

Available in MIL

You might want to protect your own MIL software application using a third-party licensing package that requires a hardware fingerprint. In this case, you can use the hardware fingerprint of the Matrox Imaging board that your application uses. The MIL hardware fingerprint is a numerical value, unique to the Matrox hardware component.

You can inquire about your hardware fingerprint using MappInquire() with one of the following:

  • M_CORONA_II_FINGERPRINT
  • M_CRONOSPLUS_FINGERPRINT
  • M_ETHERNET_FINGERPRINT
  • M_HARD_ID_KEY_FINGERPRINT
  • M_HELIOS_FINGERPRINT
  • M_MATROX_BOARD_FINGERPRINT
  • M_METEOR_II_1394_FINGERPRINT
  • M_MORPHIS_FINGERPRINT
  • M_ODYSSEY_FINGERPRINT
  • M_ORION_FINGERPRINT
  • M_SOLIOS_FINGERPRINT

If the inquired Matrox hardware component is present, the requested hardware fingerprint is returned. If it is not present, 0 is returned.

CORONA II FINGERPRINT CRONOSPLUS FINGERPRINT ETHERNET FINGERPRINT HARD ID KEY FINGERPRINT HELIOS FINGERPRINT MATROX BOARD FINGERPRINT METEOR II 1394 FINGERPRINT MORPHIS FINGERPRINT ODYSSEY FINGERPRINT ORION FINGERPRINT SOLIOS FINGERPRINT


The MIL Function Developer's Toolkit

Chapter 25: The MIL Function Developer's Toolkit
Availability
Available in MIL-Lite

Available in MIL

This chapter covers the purpose and contents of the toolkit that provides a privileged interface with MIL.

  • The MIL Function Developer's Toolkit
  • A user-defined MIL function
    • Steps to create a user-defined MIL function
  • Executing the slave function on a system with an on-board processor
  • Associating a MIL identifier with a user-defined object
    • Allocating a user-defined MIL object on a system with an on-board processor


The MIL Function Developer's Toolkit

See Also | Availability | Next
User Guide:
Associating a MIL identifier with a user-defined object
Available in MIL-Lite

Available in MIL

The MIL Function Developer's Toolkit allows programmers to define functions to extend MIL's functionality. Using this toolkit, you can implement functions and integrate them directly into the MIL library, where they behave like standard MIL functions (for example, respecting error handling and tracing). This is useful to create high-level packages on top of MIL, to extend the MIL library function set (for example, by adding new functions with specialized algorithms), or to create functions that will run entirely on an on-board processor. The toolkit provides a series of functions (Mfunc...()) designed to facilitate the creation of user-defined MIL functions.

You can also use the MIL Function Developer's Toolkit to associate MIL identifiers with user-created objects even if your application does not call any user-defined MIL function. For more information, see the Associating a MIL identifier with a user-defined object section later in this chapter.


A user-defined MIL function

Reference:
MfuncAlloc
MfuncAllocId
MfuncCall
MfuncFree
MfuncParamValue
Available in MIL-Lite

Available in MIL

To create a function that behaves like a MIL function, use the Mfunc...() functions of the MIL Function Developer's Toolkit. The individual functions of this toolkit are described in greater detail in the on-line reference.

Each user-defined MIL function consists of a master and a slave function. The master function provides the user interface and sets up the MIL function context for the new function. In the master function, you must allocate the function context for the new function, register the parameters declared by the master function as parameters of the user-defined MIL function, call the slave function, and, in the end, free the allocated MIL identifier of the function.

The slave function is a separate function that actually performs the required operations when the user-defined MIL function is called. The slave function must be implemented as a separate function because it can be executed remotely on a system with an on-board processor, such as Matrox Odyssey. The only parameter that is passed to the slave function is the MIL identifier of the user-defined MIL function. The slave function uses this identifier to access the MIL function context. You can also make calls to other MIL functions from the slave function.

When the user-defined MIL function is called in a user application, the master function registers the passed parameters in the function context, and calls the slave function. The slave function recuperates the parameters from the context using the MfuncParamValue() function, and performs the required operations.

Note that to increase efficiency, parameter checking, automatically performed for all other MIL functions, is not performed for the functions of the MIL Function Developer's Toolkit. Also, the validity of identifiers returned by MfuncAlloc() and MfuncAllocId() is not verified by MIL.

Steps to create a user-defined MIL function

To create a user-defined MIL function, follow the ensuing steps:

  1. Create a master function. The name that you will use to call the user-defined function from an application is the same as the name of the master function.
  2. Allocate a function context for the new function using MfuncAlloc(). The MfuncAlloc() function should be the first MIL function called in the master function. The allocated function context provides a MIL identifier for your user-defined MIL function, and allows this function to be treated as a regular MIL function (for example, error checking and tracing will be performed).
  3. Register the parameters of the master function in the function context using the appropriate MfuncParam...() functions. Registering the parameters gives the slave function access to the parameter values. You should register every parameter of the master function and ensure that their data types match.
  4. Call the slave function from the master function, using MfuncCall(). The slave function must be created as a separate function.
  5. Free the function context. At the end of the master function, the created function context must be freed, using the MfuncFree() function. This should be the last MIL function called in the master function.
  6. Create the slave function. This function must accept the MIL identifier of the user-defined MIL function as its only parameter. See MfuncCall() for a prototype of a slave function. In the slave function, you must recuperate the values of the parameters registered in the master function, using MfuncParamValue(), and develop the operations that will produce the required results. You can make calls to other MIL functions from the slave function.

Once you have performed the steps listed above, the user-defined MIL function is created, and you can use it in an application.

The following example shows how to mix MIL code and custom code to create a user-defined MIL function.

MFunc.c


Executing the slave function on a system with an on-board processor

User Guide:
Associating a MIL identifier with a user-defined object

Reference:
MdigGrab
MfuncParamValue
Available in MIL-Lite

Available in MIL

Typically, MIL automatically determines whether to execute the slave function on the Host or on a system with an on-board processor. To determine where to execute the slave function, MIL looks to the values of the user-defined MIL function parameters, registered in the master function, and retrieved in the slave function (MfuncParamValue()). When the retrieved values are objects allocated on a system with an on-board processor, the slave function is executed by the on-board processor. If the master function has multiple parameters, and the objects passed to these parameters are allocated on different systems, there are no guarantees about where the slave function will be executed. However, the following behavior can be anticipated:

  • If the value of a parameter is an object allocated on a system with an on-board processor, and the values of other parameters are objects that are portable (presently only image objects are portable), the slave function will be executed by the on-board processor.
  • If the values of two or more parameters are objects that are not portable, and are allocated on different systems, an error message is returned.

When the user-defined MIL function has a user-defined MIL object as one of its parameters, and you want the slave function to be executed by an on-board processor, you must allocate this object on a system with an on-board processor. For more information, see the Associating a MIL identifier with a user-defined object section later in this chapter.

Note that when the slave function is executed on a system with an on-board processor, operations that use other systems cannot be executed. For example, when the slave function is executed on Matrox Odyssey, and there is a call in this slave function to an MdigGrab() function that performs a grab on Matrox Meteor-II, this grab operation will cause an error.

For more information on executing functions on Matrox Odyssey, refer to Odyssey Native Library Developer's Toolkit Programming Guide and Matrox Imaging Library Developer's Toolkit Programming Guide for Matrox Odyssey.


Associating a MIL identifier with a user-defined object

See Also | Availability | Previous
User Guide:
A user-defined MIL function
Executing the slave function on a system with an on-board processor

Reference:
MfuncAlloc
MfuncAllocId
MfuncFreeId
MfuncInquire
MfuncParamValue
Available in MIL-Lite

Available in MIL

The MIL Function Developer's Toolkit allows you to associate a MIL identifier with an object so that this object can be treated as a standard MIL object. Use the MfuncAllocId() function to associate a MIL identifier with any array or custom data structure. Once you have associated the object with a MIL identifier, it is known as a user-defined MIL object, and it will be subject to error checking and tracing as any other MIL object.

Typically, you would create user-defined MIL objects in a user-defined MIL function (see the A user-defined MIL function section earlier in this chapter). You can also create user-defined MIL objects in your application, even if it does not call any user-defined MIL functions. To refer to these objects, use their MIL identifiers. To refer to the actual data grouped into this user-defined object, use the pointer to the object. You can retrieve the pointer to the object using the MfuncInquire() function.

When creating a user-defined MIL object, you must specify the MIL type of your object. MIL provides two object type groups, permitting you to distinguish between custom created objects, especially when the created objects are to be used in different MIL modules. Use bit-wise encoding to distinguish between the different object types within each group. For example, specify M_USER_OBJECT_1 + 0x0001 as the type of your user-defined MIL object.

To inquire about the type of a user-defined MIL object, or about the address of the object's data in memory, use the MfuncInquire() function.

Once you have finished using a user-defined MIL object, you must free the MIL identifier associated with this object, using the MfuncFreeId() function. Note that you need not free the MIL identifier in the same user-defined MIL function.

Allocating a user-defined MIL object on a system with an on-board processor

In some cases, you will want to assure that a user-defined object is allocated on a system with an on-board processor, and not on the Host. This is the case when passing the object to a user-defined MIL function whose slave function you want executed by an on-board processor. For more information, see the Executing the slave function on a system with an on-board processor section earlier in this chapter.

To allocate the user-defined object on a system with an on-board processor, create a user-defined MIL allocation function, using MfuncAlloc() with M_ALLOC. The allocation function should be created as follows:

  1. In the master function, define parameters whose values would indicate that the slave function must be executed on a system with an on-board processor. When the values of these parameters are retrieved in the slave function (MfuncParamValue()), the slave function is executed by the on-board processor.
  2. In the slave function, create the object and then associate it with a MIL identifier using MfuncAllocId(). Because the slave function is executed by the on-board processor, any object created in the slave function will be allocated on the system with that on-board processor. Note that user objects stored in the memory of an on-board processor are not accessible from Host, and vice versa, unless they are stored in shared memory. See the MIL Board Specific Notes for the default memory type of your board.

When you pass this user-defined MIL object as a parameter to another user-defined MIL function, and the conditions described in the Executing the slave function on a system with an on-board processor section earlier in this chapter are met, the slave function will be executed on the required target system.

To free the MIL identifier associated with the object, create a free-type user-defined MIL function using MfuncAlloc() with M_FREE.

For more information, see the Executing the slave function on a system with an on-board processor section earlier in this chapter.

USER OBJECT 1 ALLOC FREE


Example: mappstart.c

/******************************************************************************/
/*
 * File name: MAppStart.c
 *
 * Synopsis:  This program allocates a MIL application and system, then displays
 *            a welcoming message using graphics functions. It also shows how
 *            to check for errors.
 */
#include <conio.h>
#include <mil.h>

void main(void)
{
   MIL_ID MilApplication,  /* Application identifier.  */
          MilSystem,       /* System identifier.       */
          MilDisplay,      /* Display identifier.      */
          MilImage;        /* Image buffer identifier. */

   /* Allocate a default MIL application, system, display and image. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay, M_NULL, &MilImage);

   /* If no allocation errors. */
   if (!MappGetError(M_GLOBAL, M_NULL))
      {
      /* Perform graphic operations in the display image. */
      MgraColor(M_DEFAULT, 0xF0);
      MgraFont(M_DEFAULT, M_FONT_DEFAULT_LARGE);
      MgraText(M_DEFAULT, MilImage, 160L, 230L, MIL_TEXT(" Welcome to MIL !!! "));
      MgraColor(M_DEFAULT, 0xC0);
      MgraRect(M_DEFAULT, MilImage, 100L, 150L, 530L, 340L);
      MgraRect(M_DEFAULT, MilImage, 120L, 170L, 510L, 320L);
      MgraRect(M_DEFAULT, MilImage, 140L, 190L, 490L, 300L);

      /* Print a message. */
      printf("\nSYSTEM ALLOCATION:\n");
      printf("------------------\n\n");
      printf("System allocation successful.\n\n");
      printf("     \"Welcome to MIL !!!\"\n\n");
      }
   else
      printf("System allocation error !\n\n");

   /* Wait for a key press. */
   printf("Press <Enter> to end.\n");
   getch();

   /* Free defaults. */
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, MilImage);
}


Example: mblob.c

/*******************************************************************************/
/*
 * File name: MBlob.c
 *
 * Synopsis:  This program loads an image of some nuts, bolts and washers,
 *            determines the number of each of these, finds and marks
 *            their center of gravity using the Blob analysis module.
 */
#include <conio.h>
#include <mil.h>
#include <malloc.h>

/* Target MIL image file specifications. */
#define IMAGE_FILE            M_IMAGE_PATH MIL_TEXT("BoltsNutsWashers.mim")
#define IMAGE_THRESHOLD_VALUE 26L

/* Minimum and maximum area of blobs. */
#define MIN_BLOB_AREA         50L
#define MAX_BLOB_AREA         50000L

/* Radius of the smallest particles to keep. */
#define MIN_BLOB_RADIUS       3L

/* Minimum hole compactness corresponding to a washer. */
#define MIN_COMPACTNESS       1.5

void main(void)
{
   MIL_ID MilApplication,                 /* Application identifier.            */
          MilSystem,                      /* System identifier.                 */
          MilDisplay,                     /* Display identifier.                */
          MilImage,                       /* Image buffer identifier.           */
          MilOverlayImage,                /* Overlay image.                     */
          MilBinImage,                    /* Binary image buffer identifier.    */
          MilBlobResult,                  /* Blob result buffer identifier.     */
          MilBlobFeatureList;             /* Feature list identifier.           */
   long   TotalBlobs,                     /* Total number of blobs.             */
          BlobsWithHoles,                 /* Number of blobs with holes.        */
          BlobsWithRoughHoles,            /* Number of blobs with rough holes.  */
          n;                              /* Counter.                           */
   double *CogX,                          /* X coordinate of center of gravity. */
          *CogY;                          /* Y coordinate of center of gravity. */

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay, M_NULL, M_NULL);

   /* Restore source image into image buffer. */
   MbufRestore(IMAGE_FILE, MilSystem, &MilImage);

   /* Display the buffer and prepare for overlay annotations. */
   MdispSelect(MilDisplay, MilImage);
   MdispControl(MilDisplay, M_OVERLAY, M_ENABLE);
   MdispInquire(MilDisplay, M_OVERLAY_ID, &MilOverlayImage);

   /* Allocate a binary image buffer for fast processing. */
   MbufAlloc2d(MilSystem,
               MbufInquire(MilImage, M_SIZE_X, M_NULL),
               MbufInquire(MilImage, M_SIZE_Y, M_NULL),
               1+M_UNSIGNED, M_IMAGE+M_PROC, &MilBinImage);

   /* Pause to show the original image. */
   printf("\nBLOB ANALYSIS:\n");
   printf("--------------\n\n");
   printf("This program determines the number of bolts, nuts and washers\n");
   printf("in the image and finds their center of gravity.\n");
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Binarize image. */
   MimBinarize(MilImage, MilBinImage, M_GREATER_OR_EQUAL,
                                      IMAGE_THRESHOLD_VALUE, M_NULL);

   /* Remove small particles and then remove small holes. */
   MimOpen(MilBinImage, MilBinImage, MIN_BLOB_RADIUS, M_BINARY);
   MimClose(MilBinImage, MilBinImage, MIN_BLOB_RADIUS, M_BINARY);

   /* Allocate a feature list. */
   MblobAllocFeatureList(MilSystem, &MilBlobFeatureList);

   /* Enable the Area and Center Of Gravity feature calculation. */
   MblobSelectFeature(MilBlobFeatureList, M_AREA);
   MblobSelectFeature(MilBlobFeatureList, M_CENTER_OF_GRAVITY);

   /* Allocate a blob result buffer. */
   MblobAllocResult(MilSystem, &MilBlobResult);

   /* Calculate selected features for each blob. */
   MblobCalculate(MilBinImage, M_NULL, MilBlobFeatureList, MilBlobResult);

   /* Exclude blobs whose area is too small. */
   MblobSelect(MilBlobResult, M_EXCLUDE, M_AREA, M_LESS_OR_EQUAL,
                                                 MIN_BLOB_AREA, M_NULL);

   /* Get the total number of selected blobs. */
   MblobGetNumber(MilBlobResult, &TotalBlobs);
   printf("There are %ld objects ", TotalBlobs);

   /* Read and print the blob's center of gravity. */
   if ((CogX = (double *)malloc(TotalBlobs*sizeof(double))) &&
       (CogY = (double *)malloc(TotalBlobs*sizeof(double)))
      )
      {
      /* Get the results. */
      MblobGetResult(MilBlobResult, M_CENTER_OF_GRAVITY_X, CogX);
      MblobGetResult(MilBlobResult, M_CENTER_OF_GRAVITY_Y, CogY);

      /* Print the center of gravity of each blob. */
      printf("and their center of gravity are:\n");
      for(n=0; n < TotalBlobs; n++)
         printf("Blob #%ld: X=%5.1f, Y=%5.1f\n", n, CogX[n], CogY[n]);
      free(CogX);
      free(CogY);
      }
   else
      printf("\nError: Not enough memory.\n");

   /* Draw a cross at the center of gravity of each blob. */
   MgraColor(M_DEFAULT, M_COLOR_RED);
   MblobDraw(M_DEFAULT, MilBlobResult, MilOverlayImage, M_DRAW_CENTER_OF_GRAVITY,
                                                        M_INCLUDED_BLOBS, M_DEFAULT);

   /* Reverse what is considered to be the background so that
    * holes are seen as being blobs.
    */
   MblobControl(MilBlobResult, M_FOREGROUND_VALUE, M_ZERO);

   /* Add a feature to distinguish between types of holes. Since area
    * has already been added to the feature list, and the processing
    * mode has been changed, all blobs will be re-included and the area
    * of holes will be calculated automatically.
    */
   MblobSelectFeature(MilBlobFeatureList, M_COMPACTNESS);

   /* Calculate selected features for each blob. */
   MblobCalculate(MilBinImage, M_NULL, MilBlobFeatureList, MilBlobResult);

   /* Exclude small holes and large (the area around objects) holes. */
   MblobSelect(MilBlobResult, M_EXCLUDE, M_AREA, M_OUT_RANGE,
                                         MIN_BLOB_AREA, MAX_BLOB_AREA);

   /* Get the number of blobs with holes. */
   MblobGetNumber(MilBlobResult, &BlobsWithHoles);

   /* Exclude blobs whose holes are compact (i.e. nuts). */
   MblobSelect(MilBlobResult, M_EXCLUDE, M_COMPACTNESS, M_LESS_OR_EQUAL,
                                                        MIN_COMPACTNESS, M_NULL);

   /* Get the number of blobs with holes that are NOT compact. */
   MblobGetNumber(MilBlobResult, &BlobsWithRoughHoles);

   /* Print results. */
   printf("\nIdentified objects:\n");
   printf("%ld bolts\n", TotalBlobs-BlobsWithHoles);
   printf("%ld nuts\n", BlobsWithHoles - BlobsWithRoughHoles);
   printf("%ld washers\n\n", BlobsWithRoughHoles);
   printf("Press <Enter> to end.\n\n");
   getch();

   /* Free all allocations. */
   MblobFree(MilBlobResult);
   MblobFree(MilBlobFeatureList);
   MbufFree(MilBinImage);
   MbufFree(MilImage);
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
}


Example: mbufbayercolorfilter.c

/************************************************************************************/
/*
 * File name: MBufBayerColorFilter.c
 *
 * Synopsis:  This program shows how to perform Bayer-to-Color conversion.
 */

#include "mil.h"
#include "conio.h"
void main(void)
{
MIL_ID MilApplication,
MilSystem,
MilDigitizer,
MilDisplay,
MilWBCoefficients,
MilImageDisp,
MilImageGrab;

/* User array for white balance coefficients. */
float WBCoefficients[3];
/* Specify the Bayer pattern of your camera. */
long ConversionType = M_BAYER_GR;
/* Buffer characteristics. */
long XSize;
long YSize;

/* Allocate an application. */
MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay,
&MilDigitizer, M_NULL);
XSize = MdigInquire(MilDigitizer, M_SIZE_X, M_NULL);
YSize = MdigInquire(MilDigitizer, M_SIZE_Y, M_NULL);
/* Allocate a display buffer. */
MbufAllocColor(MilSystem, 3, XSize, YSize, 8L+M_UNSIGNED, M_PROC+M_IMAGE,
&MilImageDisp);
/* Allocate a grab buffer. */
MbufAllocColor(MilSystem, 1, XSize, YSize, 8L+M_UNSIGNED,
M_IMAGE+M_DISP+M_GRAB+M_PROC, &MilImageGrab);
/* Allocate an array for the white balance coefficients. */
MbufAlloc1d(MilSystem, 3, 32+M_FLOAT, M_ARRAY, &MilWBCoefficients);

/* Display the image. */
MbufClear(MilImageDisp, M_RGB888(0, 0, 0));
MdispSelect(MilDisplay, MilImageDisp);
/* Ask the user for a white image for white balance. */
printf("Place a white paper in front of the camera and " \
"press <ENTER> when ready.\n");
do
{
/* Grab a white Bayer image. */
MdigGrab(MilDigitizer, MilImageGrab);
/* Convert the white Bayer image to color without white balance. */
MbufBayer(MilImageGrab, MilImageDisp, M_DEFAULT, ConversionType);
}
while (!kbhit());
getch();
/* Determine the white balance coefficients. */
MbufBayer(MilImageGrab, MilImageDisp, MilWBCoefficients,
ConversionType+M_WHITE_BALANCE_CALCULATE);
/* Print the computed coefficients. */
MbufGet(MilWBCoefficients, (void *) &WBCoefficients[0]);
printf("\nWhite balance correction coefficients : %f, %f, %f\n\n",
WBCoefficients[0], WBCoefficients[1], WBCoefficients[2]);
/* Grab a new Bayer image with white balance correction. */
printf("Press <ENTER> to grab an image\n");
getchar();
do
{
/* Grab a Bayer image. */
MdigGrab(MilDigitizer, MilImageGrab);
/* Convert the Bayer image to color. */
MbufBayer(MilImageGrab, MilImageDisp, MilWBCoefficients, ConversionType);
}
while (!kbhit());
getch();
printf("Press <ENTER> to end\n");
getchar();

/* Terminate and free everything. */
MbufFree(MilImageGrab);
MbufFree(MilImageDisp);
MbufFree(MilWBCoefficients);
MappFreeDefault(MilApplication, MilSystem, MilDisplay, MilDigitizer, M_NULL);


Example: mbufcolor.c

/**********************************************************************************/
/*
 * File name: MBufColor.c
 *
 * Synopsis:  This program demonstrates color buffer manipulations. It allocates
 *            a displayable color image buffer, displays it, and loads a color
 *            image into its left part. After that, color annotations are done
 *            in each band of the color buffer. If image processing is available,
 *            it converts the image to Hue, Luminance and Saturation (HLS), adds
 *            a certain offset to the luminance component and converts
 *            the image back to Red, Green, Blue (RGB) to display the result.
 *
 *            The example also demonstrates how to display multiple images
 *            in a single display using child buffers.
 */
#include <mil.h>
#include <conio.h>

/* Source MIL image file specifications. */
#define IMAGE_FILE              M_IMAGE_PATH MIL_TEXT("Bird.mim")

/* Luminance offset to add to the image. */
#define IMAGE_LUMINANCE_OFFSET  40L

/* Main function. */
void main(void)
{
   MIL_ID MilApplication,       /* Application identifier.  */
          MilSystem,            /* System identifier.       */
          MilDisplay,           /* Display identifier.      */
          MilImage,             /* Image buffer identifier. */
          MilLeftSubImage,      /* Sub-image buffer identifier for original image. */
          MilRightSubImage,     /* Sub-image buffer identifier for processed image.*/
          MilLumSubImage=0,     /* Sub-image buffer identifier for luminance.      */
          MilRedBandSubImage,   /* Sub-image buffer identifier for red component.  */
          MilGreenBandSubImage, /* Sub-image buffer identifier for green component.*/
          MilBlueBandSubImage;  /* Sub-image buffer identifier for blue component. */
   long   SizeX, SizeY, SizeBand, Type;

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay, M_NULL, M_NULL);

   /* Allocate a color display buffer twice the size of the source image and display it. */
   MbufAllocColor(MilSystem,
                  MbufDiskInquire(IMAGE_FILE, M_SIZE_BAND, &SizeBand),
                  MbufDiskInquire(IMAGE_FILE, M_SIZE_X, &SizeX) * 2,
                  MbufDiskInquire(IMAGE_FILE, M_SIZE_Y, &SizeY),
                  MbufDiskInquire(IMAGE_FILE, M_TYPE,   &Type),
                  M_IMAGE+M_DISP+M_PROC, &MilImage);
   MbufClear(MilImage, 0L);
   MdispSelect(MilDisplay, MilImage);

   /* Define 2 child buffers that maps to the left and right part of the display
      buffer, to put the source and destination color images.
    */
   MbufChild2d(MilImage, 0L, 0L, SizeX, SizeY, &MilLeftSubImage);
   MbufChild2d(MilImage, SizeX, 0L, SizeX, SizeY, &MilRightSubImage);

   /* Load the color source image on the left. */
   MbufLoad(IMAGE_FILE, MilLeftSubImage);

   /* Define child buffers that map to the red, green and blue components
      of the source image.
    */
   MbufChildColor(MilLeftSubImage, M_RED,   &MilRedBandSubImage);
   MbufChildColor(MilLeftSubImage, M_GREEN, &MilGreenBandSubImage);
   MbufChildColor(MilLeftSubImage, M_BLUE,  &MilBlueBandSubImage);

   /* Write color text annotations to show access in each individual band of the image.

      Note that this is typically simplified by using:
      MgraColor(M_DEFAULT, M_RGB(0xFF,0x90,0x00));
      MgraText(M_DEFAULT, MilLeftSubImage, ...);
    */
   MgraColor(M_DEFAULT, 0xFF);
   MgraText(M_DEFAULT, MilRedBandSubImage,   SizeX/16, SizeY/8, MIL_TEXT(" TOUCAN "));
   MgraColor(M_DEFAULT, 0x90);
   MgraText(M_DEFAULT, MilGreenBandSubImage, SizeX/16, SizeY/8, MIL_TEXT(" TOUCAN "));
   MgraColor(M_DEFAULT, 0x00);
   MgraText(M_DEFAULT, MilBlueBandSubImage,  SizeX/16, SizeY/8, MIL_TEXT(" TOUCAN "));

   /* Print a message. */
   printf("\nCOLOR OPERATIONS:\n");
   printf("-----------------\n\n");
   printf("A color source image was loaded on the left and color text\n");
   printf("annotations were written in it.\n");
   printf("Press <Enter> to continue.\n\n");
   getch();

   /*  If Image Processing module is available, augment the image luminance. */
   #if (!M_MIL_LITE)
      {
      /* Convert image to Hue, Luminance, Saturation color space (HLS). */
      MimConvert(MilLeftSubImage, MilRightSubImage, M_RGB_TO_HLS);

      /* Create a child buffer that maps to the luminance component. */
      MbufChildColor(MilRightSubImage, M_LUMINANCE, &MilLumSubImage);

      /* Add an offset to the luminance component. */
      MimArith(MilLumSubImage, IMAGE_LUMINANCE_OFFSET, MilLumSubImage, M_ADD_CONST+M_SATURATION);

      /* Convert image back to Red, Green, Blue color space (RGB) for display. */
      MimConvert(MilRightSubImage, MilRightSubImage, M_HLS_TO_RGB);

      /* Free the luminance band child buffer. */
      MbufFree(MilLumSubImage);

      /* Print a message. */
      printf("Luminance was increased using color image processing.\n");
      }
   #else
      {
      /* Copy the color source image. */
      MbufCopy(MilLeftSubImage, MilRightSubImage);

      /* Print a message. */
      printf("The image was copied to the right.\n");
      }
   #endif

   /* Print a message. */
   printf("Press <Enter> to end.\n");
   getch();

   /* Release sub-images and color image buffer. */
   MbufFree(MilRedBandSubImage);
   MbufFree(MilGreenBandSubImage);
   MbufFree(MilBlueBandSubImage);
   MbufFree(MilRightSubImage);
   MbufFree(MilLeftSubImage);
   MbufFree(MilImage);

   /* Release defaults. */
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
}


Example: mdiggrab.c

/********************************************************************************/
/*
 * File name: MDigGrab.c
 *
 * Synopsis:  This program demonstrates how to grab from a camera in
 *            continuous and monoshot mode.
 */
#include <mil.h>
#include <conio.h>

void main(void)
{
   MIL_ID MilApplication,  /* Application identifier.  */
          MilSystem,       /* System identifier.       */
          MilDisplay,      /* Display identifier.      */
          MilDigitizer,    /* Digitizer identifier.    */
          MilImage;        /* Image buffer identifier. */

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem,
                             &MilDisplay, &MilDigitizer, &MilImage);

   /* Grab continuously. */
   MdigGrabContinuous(MilDigitizer, MilImage);

   /* When a key is pressed, halt. */
   printf("\nDIGITIZER ACQUISITION:\n");
   printf("----------------------\n\n");
   printf("Continuous image grab in progress.\n");
   printf("Press <Enter> to stop.\n\n");
   getch();

   /* Stop continuous grab. */
   MdigHalt(MilDigitizer);

   /* Pause to show the result. */
   printf("Continuous grab stopped.\n\n");
   printf("Press <Enter> to do a single image grab.\n\n");
   getch();

   /* Monoshot grab. */
   MdigGrab(MilDigitizer, MilImage);

   /* Pause to show the result. */
   printf("Displaying the grabbed image.\n");
   printf("Press <Enter> to end.\n\n");
   getch();

   /* Free defaults. */
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, MilDigitizer, MilImage);
}


Example: mdiggrabmultiple.c

/**************************************************************************/
/*
 * File name: MDigGrabMultiple.c
 *
 * Synopsis: This example performs 2 continuous grabs using the 2 digitizers
 *           of a system.
 */
/* Headers. */
#include <mil.h>
#include <conio.h>

/* Grab scale. */
#define GRAB_SCALE  1.0

/* Main function. */
void main(void)
{
   MIL_ID   MilApplication;
   MIL_ID   MilSystem     ;
   MIL_ID   MilDigitizer[2];
   MIL_ID   MilDisplay[2] ;
   MIL_ID   MilImageDisp[2];

   /* Allocations. */
   MappAlloc(M_DEFAULT, &MilApplication);
   MsysAlloc("M_DEFAULT", M_DEFAULT, M_DEFAULT, &MilSystem);
   MdigAlloc(MilSystem,  M_DEV0, "M_DEFAULT", M_DEFAULT, &MilDigitizer[0]);
   MdigAlloc(MilSystem,  M_DEV1, "M_DEFAULT", M_DEFAULT, &MilDigitizer[1]);
   MdispAlloc(MilSystem, M_DEV0, "M_DEFAULT", M_DEFAULT, &MilDisplay[0]);
   MdispAlloc(MilSystem, M_DEV1, "M_DEFAULT", M_DEFAULT, &MilDisplay[1]);

   /* Allocate 2 display buffers and clear them. */
   MbufAlloc2d(MilSystem,
               (long)(MdigInquire(MilDigitizer[0], M_SIZE_X, M_NULL)*GRAB_SCALE),
               (long)(MdigInquire(MilDigitizer[0], M_SIZE_Y, M_NULL)*GRAB_SCALE),
               8L+M_UNSIGNED,
               M_IMAGE+M_GRAB+M_PROC+M_DISP, &MilImageDisp[0]);
   MbufClear(MilImageDisp[0], 0x0);
   MbufAlloc2d(MilSystem,
               (long)(MdigInquire(MilDigitizer[1], M_SIZE_X, M_NULL)*GRAB_SCALE),
               (long)(MdigInquire(MilDigitizer[1], M_SIZE_Y, M_NULL)*GRAB_SCALE),
               8L+M_UNSIGNED,
               M_IMAGE+M_GRAB+M_PROC+M_DISP, &MilImageDisp[1]);
   MbufClear(MilImageDisp[1], 0x80);

   /* Display the buffers. */
   MdispSelect(MilDisplay[0], MilImageDisp[0]);
   MdispSelect(MilDisplay[1], MilImageDisp[1]);

   /* Grab continuously on displays at the specified scale. */
   MdigControl(MilDigitizer[0], M_GRAB_SCALE, GRAB_SCALE);
   MdigGrabContinuous(MilDigitizer[0],MilImageDisp[0]);
   MdigControl(MilDigitizer[1], M_GRAB_SCALE, GRAB_SCALE);
   MdigGrabContinuous(MilDigitizer[1],MilImageDisp[1]);

   /* Print a message. */
   printf("Press <Enter> to stop continuous grab.\n");
   getchar();

   /* Halt continuous grab. */
   MdigHalt(MilDigitizer[0]);
   MdigHalt(MilDigitizer[1]);

   /* Print a message. */
   printf("Press <Enter> to end.\n");
   getchar();

   /* Free allocations. */
   MbufFree(MilImageDisp[0]);
   MbufFree(MilImageDisp[1]);
   MdispFree(MilDisplay[0]);
   MdispFree(MilDisplay[1]);
   MdigFree(MilDigitizer[0]);
   MdigFree(MilDigitizer[1]);
   MsysFree(MilSystem);
   MappFree(MilApplication);
}


Example: mdigprocessmultiple.c

/***********************************************************************************/
/*
 * File name: MDigProcessMultiple.c
 *
 * Synopsis:  This program shows how to use multiple digitizers to do processing.
 *            The main starts an independent processing job for each digitizer
 *            (one per camera) and then waits for a key to be pressed to stop them.
 *
 *     Note:  To do PA processing when using the Matrox Helios, +M_ON_BOARD
 *            attribute is added to the buffer allocations.
 *
 */
#include <mil.h>
#include <conio.h>
#include <stdlib.h>

/* Number of digitizer to use. */
#define DIGITIZER_NUM   2

/* Maximum number of images in the buffering grab queue of each digitizer.
   Generally, increasing the number of buffers prevents missing frames.
 */
#define BUFFERING_SIZE_MAX  30

/* Draw annotation in the Grab image buffers. This can increase CPU
   usage significantly. This is especially true for on-board buffers.
 */
#define DRAW_ANNOTATION     M_NO

/* User's processing function prototype and hook parameter structure. */
long MFTYPE ProcessingFunction(long HookType, MIL_ID HookId, void MPTYPE *HookDataPtr);
typedef struct
   {
   MIL_ID  MilImageDisp;
   long    ProcessedImageCount;
   } HookDataStruct;

/* Main function: */
/* -------------- */
void main(void)
   {
   MIL_ID MilApplication;
   MIL_ID MilSystem;
   MIL_ID MilDigitizer[DIGITIZER_NUM];
   MIL_ID MilDisplay[DIGITIZER_NUM];
   MIL_ID MilImageDisp[DIGITIZER_NUM];
   MIL_ID MilGrabBufferList[DIGITIZER_NUM][BUFFERING_SIZE_MAX];
   long   n, m, BufferLocation = M_NULL;
   long   LastAllocatedN = M_NULL;
   long   LastAllocatedM = M_NULL;
   long   MilGrabBufferListSize = M_NULL;
   long   ProcessFrameCount;
   double ProcessFrameRate;
   int    BufferAllocationSuccess = 1;
   HookDataStruct UserHookData[DIGITIZER_NUM];

   /* Allocate application and system. */
   MappAlloc(M_DEFAULT, &MilApplication);
   MsysAlloc(M_DEF_SYSTEM_TYPE, M_DEFAULT, M_DEFAULT, &MilSystem);

   /* Allocate digitizers using the default DCF. */
   for(n=0; n<DIGITIZER_NUM; n++)
       MdigAlloc(MilSystem, n, "M_DEFAULT", M_DEFAULT, &MilDigitizer[n]);

   /* Allocate displays. */
   for(n=0; n<DIGITIZER_NUM; n++)
       MdispAlloc(MilSystem, M_DEV0, "M_DEFAULT", M_DEFAULT, &MilDisplay[n]);

   /* If the system is a Matrox Helios, allocate the buffer on-board to benefit from Oasis PA processing. */
   if (MsysInquire(MilSystem, M_SYSTEM_TYPE, M_NULL) == M_SYSTEM_HELIOS_TYPE)
      BufferLocation = M_ON_BOARD;

   /* Allocate display buffers and clear them. */
   for(n=0; n<DIGITIZER_NUM; n++)
       {
       MbufAlloc2d(MilSystem,
                   MdigInquire(MilDigitizer[n], M_SIZE_X, M_NULL),
                   MdigInquire(MilDigitizer[n], M_SIZE_Y, M_NULL),
                   8+M_UNSIGNED, M_IMAGE+M_GRAB+M_PROC+M_DISP+BufferLocation,
                   &MilImageDisp[n]);
       MbufClear(MilImageDisp[n], 0);
       }

   /* Allocate the grab buffers for each digitizer and clear them. */
    MappControl(M_ERROR, M_PRINT_DISABLE);

    /* Init table. */
    for(n = 0; n < DIGITIZER_NUM; n++)
       for(m = 0; m < BUFFERING_SIZE_MAX; m++)
          MilGrabBufferList[n][m] = M_NULL;

    /* Try to allocate as many buffers as possible */
    for(m = M_NULL; m < BUFFERING_SIZE_MAX; m++)
      {
      for(n = M_NULL; n < DIGITIZER_NUM; n++)
         {
         /* Try to allocate a buffer. */
         MbufAlloc2d(MilSystem,
                     MdigInquire(MilDigitizer[n], M_SIZE_X, M_NULL),
                     MdigInquire(MilDigitizer[n], M_SIZE_Y, M_NULL),
                     MdigInquire(MilDigitizer[n], M_TYPE, M_NULL),
                     M_IMAGE+M_GRAB+M_PROC+BufferLocation,
                     &MilGrabBufferList[n][m]);

         /* If allocation is successful, clear the buffer. */
         if (MilGrabBufferList[n][m])
            {
            MbufClear(MilGrabBufferList[n][m], 0xFF);

            /* Keep index in table, in case it is the last allocated buffer. */
            LastAllocatedN = n;
            LastAllocatedM = m;
          }

         /* Allocation failed, stop immediately. */
         else
            {
            BufferAllocationSuccess = 0;
            break;
            }

         }

      /* If allocation worked on all digitizers, increase the number of buffers per digitizer. */
      if(BufferAllocationSuccess)
         MilGrabBufferListSize++;

      /* Allocation failed, stop immediately. */
      else
         break;
      }
   MappControl(M_ERROR, M_PRINT_ENABLE);

   /* Free the last allocated buffer to leave space for possible temporary buffer. */
   MbufFree(MilGrabBufferList[LastAllocatedN][LastAllocatedM]);
   MilGrabBufferList[LastAllocatedN][LastAllocatedM] = M_NULL;

   /* If other digitizers have a buffer allocated at same index, delete them for symmetry. */
   for(n = 0; n < DIGITIZER_NUM; n++)
      {
      if(MilGrabBufferList[n][LastAllocatedM])
         {
         MbufFree(MilGrabBufferList[n][LastAllocatedM]);
         MilGrabBufferList[n][LastAllocatedM] = M_NULL;
         }
      }
    MilGrabBufferListSize--;

    /* Start processing jobs. */
   for(n = 0; n < DIGITIZER_NUM; n++)
      {
      /* Display the target image. */
       MdispSelect(MilDisplay[n], MilImageDisp[n]);

       /* Grab continuously on the display and wait for a key press. */
       MdigGrabContinuous(MilDigitizer[n], MilImageDisp[n]);

       /* Print a message. */
       printf("\nPROCESSING #%ld: Multiple buffered processing (%ld buffers).\n", n+1, MilGrabBufferListSize);
       printf("---------------------------------------------------------\n\n");
       printf("Press <Enter> to start.\r");
       getch();
       printf("Processing started...  \n\n");

       /* Halt continuous grab. */
       MdigHalt(MilDigitizer[n]);

       /* Initialize the User processing function data structure. */
       UserHookData[n].MilImageDisp        = MilImageDisp[n];
       UserHookData[n].ProcessedImageCount = 0;

       /* Start the processing. The processing function is called for every grabbed frame. */
       MdigProcess(MilDigitizer[n], MilGrabBufferList[n], MilGrabBufferListSize,
                                 M_START, M_DEFAULT, ProcessingFunction, &UserHookData[n]);
       }

   /* NOTE: Now the main is free to do other tasks while the processing is executing. */
   /* ------------------------------------------------------------------------------- */

   /* Report that the threads are started and wait for a key press. */
   printf("\nMAIN: Processing task(s) running...\n");
   printf("-----------------------------------\n\n");
   printf("Press <Enter> to stop.\n\n");
   getch();

   /* Stop the processing jobs. */
   for(n=0; n<DIGITIZER_NUM; n++)
       {
       /* Stop the processing. */
       MdigProcess(MilDigitizer[n], MilGrabBufferList[n], MilGrabBufferListSize,
                   M_STOP, M_DEFAULT, ProcessingFunction, &UserHookData[n]);

       /* Print statistics. */
       MdigInquire(MilDigitizer[n], M_PROCESS_FRAME_COUNT,  &ProcessFrameCount);
       MdigInquire(MilDigitizer[n], M_PROCESS_FRAME_RATE,   &ProcessFrameRate);
       printf("Processing #%ld: %ld frames grabbed at %.1f frames/sec (%.1f ms/frame).\n",
                          n+1, ProcessFrameCount, ProcessFrameRate, 1000.0/ProcessFrameRate);
       }

    /* Free the grab buffers and MIL objects. */
    MappControl(M_ERROR, M_PRINT_DISABLE);
    for (n=0 ;n<DIGITIZER_NUM; n++)
       {
       for (m = 0; m < BUFFERING_SIZE_MAX; m++)
          {
          if(MilGrabBufferList[n][m])
             {
             MbufFree(MilGrabBufferList[n][m]);
             MilGrabBufferList[n][m] = 0;
             }
          }
       MbufFree(MilImageDisp[n]);
       MdispFree(MilDisplay[n]);
       MdigFree(MilDigitizer[n]);
       }

    MappControl(M_ERROR, M_PRINT_ENABLE);

   /* Wait for a key press to exit. */
   printf("\nPress <Enter> to end.\n\n");
   getch();

   /* Free buffers. */
   MsysFree(MilSystem);
   MappFree(MilApplication);
   }

/* Processing thread(s) function: */
/* ---------------------------------------------------------------- */

/* User's processing function called every time a grab buffer is modified. */
/* ------------------------------------------------------------------------*/

/* Local defines. */
#define STRING_LENGTH_MAX  20
#define STRING_POS_X       20
#define STRING_POS_Y       20

long MFTYPE ProcessingFunction(long HookType, MIL_ID HookId, void MPTYPE *HookDataPtr)
   {
   HookDataStruct *UserHookDataPtr = (HookDataStruct *)HookDataPtr;
   MIL_ID ModifiedBufferId;
   char   Text[STRING_LENGTH_MAX]= {'\0',};

   /* Retrieve the MIL_ID of the grabbed buffer. */
   MdigGetHookInfo(HookId, M_MODIFIED_BUFFER+M_BUFFER_ID, &ModifiedBufferId);

   /* Increase the frame count. */
   UserHookDataPtr->ProcessedImageCount++;

   /* Draw the frame count (if enabled). */
   if (DRAW_ANNOTATION)
      {
      MOs_ltoa(UserHookDataPtr->ProcessedImageCount, Text, 10);
      MgraText(M_DEFAULT, ModifiedBufferId, STRING_POS_X, STRING_POS_Y, Text);
      }

   /* Perform the processing and update the display. */
   #if (!M_MIL_LITE)
      MimArith(ModifiedBufferId, M_NULL, UserHookDataPtr->MilImageDisp, M_NOT);
   #else
      MbufCopy(ModifiedBufferId, UserHookDataPtr->MilImageDisp);
   #endif

   return 0;
   }


Example: mdigsubtract.c

/************************************************************************************/
/*
 * File name: MDigSubtract.c
 *
 * Synopsis:  This program grabs an image which is expected to be the constant dark
 *            background, and then subtracts this background from subsequent grabbed
 *            images.
 */
#include <mil.h>
#include <conio.h>
#include <stdlib.h>

void main(void)
{
   MIL_ID MilApplication,  /* Application identifier.             */
          MilSystem,       /* System identifier.                  */
          MilDisplay,      /* Display identifier.                 */
          MilCamera,       /* Camera identifier.                  */
          MilImage,        /* Image buffer identifier.            */
          GrabImage,       /* Grab image buffer identifier.       */
          BackgroundImage; /* Background image buffer identifier. */
   long   CamSizeX,        /* Camera width variable.              */
          CamSizeY;        /* Camera height variable.             */

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem,
                    &MilDisplay, &MilCamera, M_NULL);

   /* Reads camera X, Y and depth dimensions. */
   MdigInquire(MilCamera, M_SIZE_X, &CamSizeX);
   MdigInquire(MilCamera, M_SIZE_Y, &CamSizeY);

   /* Allocate a first image buffer to store and display the result image. */
   MbufAlloc2d(MilSystem, CamSizeX, CamSizeY, 8+M_UNSIGNED,
               M_IMAGE+M_PROC+M_GRAB+M_DISP, &MilImage);

   /* Allocate a second image buffer to store the background image. */
   MbufAlloc2d(M_DEFAULT, CamSizeX, CamSizeY, 8+M_UNSIGNED,
               M_IMAGE+M_PROC, &BackgroundImage);

   /* Allocate a third image buffer to grab the changing image. */
   MbufAlloc2d(MilSystem, CamSizeX, CamSizeY, 8+M_UNSIGNED,
               M_IMAGE+M_PROC+M_GRAB, &GrabImage);

   /* Select the image on the display. */
   MbufClear(MilImage, 0);
   MdispSelect(MilDisplay, MilImage);

   /* Grab the background image in the display buffer. */
   MdigGrabContinuous(MilCamera, MilImage);

   /* When a key is pressed, halt. */
   printf("Point your camera at a constant dark background and adjust the focus.\n");
   printf("Press <Enter> to continue.\n");
   getchar();
   MdigHalt(MilCamera);

   /* Copy the displayed buffer into the background buffer. */
   MbufCopy(MilImage, BackgroundImage);

   /* When a key is pressed, halt. */
   printf("Continuous subtraction in progress...\n\n");
   printf("Keeping your camera in the same position, create motion with a bright\n");
   printf("object in front of the background.\n");
   printf("Press <Enter> to end.\n");

   /* Grab and subtract background in loop. */
   while (!kbhit())
      {
      MdigGrab(MilCamera, GrabImage);
      MimArith(GrabImage, BackgroundImage, MilImage, M_SUB_ABS);
      }
   getch();

   /* Release defaults and image. */
   MbufFree(GrabImage);
   MbufFree(BackgroundImage);
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, MilCamera, MilImage);
}


Example: mdispselect.c

/************************************************************************************/
/*
 * File name: MDispSelect.c
 *
 * Synopsis:  This program allocates a displayable image buffer, clears its contents,
 *            draws a filled circle and some text and then displays it.
 */
#include <mil.h>
#include <conio.h>
#include <stdlib.h>

int main(void)
{
   MIL_ID MilApplication,  /* Application identifier.  */
          MilSystem,       /* System identifier.       */
          MilDisplay,      /* Display identifier.      */
          MilImage;        /* Image buffer identifier. */

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay, M_NULL, M_NULL);

   /* Allocate an image buffer with the default dimensions to do graphics. */
   MbufAlloc2d(MilSystem, M_DEF_IMAGE_SIZE_X_MIN,M_DEF_IMAGE_SIZE_Y_MIN,
                          M_DEF_IMAGE_TYPE, M_IMAGE+M_DISP, &MilImage);

   /* Draw and display a circle */
   MbufClear(MilImage, 0L);
   MgraColor(M_DEFAULT, 255L);
   MgraArcFill(M_DEFAULT, MilImage, 256L, 240L, 100L, 100L, 0.0, 360.0);
   MgraText(M_DEFAULT, MilImage, 238L, 234L, " MIL ");

   /* Display the image buffer. */
   MdispSelect(MilDisplay, MilImage);

   /* Print a message. */
   printf("A circle was drawn in the displayed image buffer.\n");
   printf("Press <Enter> to end.\n");
   getchar();

   /* Release image buffer. */
   MbufFree(MilImage);

   /* Release defaults. */
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
   return 0;
}


Example: mdispwindow.c

/*****************************************************************************/
/*
 * File name: MDispWindow.c
 *
 * Synopsis:  This program displays a welcoming message in a user-defined
 *            window and grabs (if supported) in it. It uses
 *            the MIL system and the MdispSelectWindow() function
 *            to display the MIL buffer in a user-created client window.
 *
 */
#include <windows.h> /* windows.h must precede mil.h. */
#include <mil.h>

/* Window title. */
#define MIL_APPLICATION_NAME      "MIL Application"
#define MIL_APPLICATION_NAME_SIZE  128

/* Default image dimensions. */
#define DEFAULT_IMAGE_SIZE_X       640
#define DEFAULT_IMAGE_SIZE_Y       480
#define DEFAULT_IMAGE_SIZE_BAND      1

/* Background color of the window client area. */
#define BACKCOLORRED               160
#define BACKCOLORGREEN             160
#define BACKCOLORBLUE              164

/* Defines for menu. */
#define IDM_START_MAIN             100

/* Function prototypes. */
void MilApplication(HWND UserWindowHandle);
void MilApplicationPaint(HWND UserWindowHandle);

/***************************************************************************/
/*/
* Name:         MilApplication()
*
* Synopsis:     This function is the core of the MIL application that
*               is executed when the "Start" menu item of this
*               Windows program is selected. See WinMain() below
*               for the program entry point.
*
*               It uses MIL to display a welcoming message in the
*               specified user window and to grab in it (if it is supported)
*               using the target system.
*
*/

void MilApplication(HWND UserWindowHandle)
{
   /* MIL variables */
   MIL_ID MilApplication,  /* MIL Application identifier.  */
          MilSystem,       /* MIL System identifier.       */
          MilDisplay,      /* MIL Display identifier.      */
          MilDigitizer,    /* MIL Digitizer identifier.    */
          MilImage;        /* MIL Image buffer identifier. */

   long BufSizeX;
   long BufSizeY;
   long BufSizeBand;

   /* Allocate a MIL application. */
   MappAlloc(M_DEFAULT, &MilApplication);

   /* Allocate a MIL system. */
   MsysAlloc("M_DEFAULT", M_DEFAULT, M_DEFAULT, &MilSystem);

   /* Allocate a MIL display. */
   MdispAlloc(MilSystem, M_DEFAULT, "M_DEFAULT", M_WINDOWED, &MilDisplay);

   /* Allocate a MIL digitizer, if supported, and set the target image size. */
   if (MsysInquire(MilSystem, M_DIGITIZER_NUM, M_NULL) > 0)
      {
      MdigAlloc(MilSystem, M_DEFAULT, "M_DEFAULT", M_DEFAULT, &MilDigitizer);
      MdigInquire(MilDigitizer, M_SIZE_X,    &BufSizeX);
      MdigInquire(MilDigitizer, M_SIZE_Y,    &BufSizeY);
      MdigInquire(MilDigitizer, M_SIZE_BAND, &BufSizeBand);
      }
   else
      {
      MilDigitizer = M_NULL;
      BufSizeX     = DEFAULT_IMAGE_SIZE_X;
      BufSizeY     = DEFAULT_IMAGE_SIZE_Y;
      BufSizeBand  = DEFAULT_IMAGE_SIZE_BAND;
      }

   /* Do not allow example to run in auxiliary mode. */
   if(MdispInquire(MilDisplay, M_DISPLAY_MODE, M_NULL) != M_WINDOWED)
      {
      MessageBox(0,"""This example does no run in auxiliary mode.",
                   "MIL application example",
                   MB_APPLMODAL | MB_ICONEXCLAMATION );
      goto end;
      }

   /* Allocate a MIL buffer. */
   MbufAllocColor(MilSystem, BufSizeBand, BufSizeX, BufSizeY, 8+M_UNSIGNED,
                  (MilDigitizer ? M_IMAGE+M_DISP+M_GRAB : M_IMAGE+M_DISP), &MilImage);

   /* Clear the buffer. */
   MbufClear(MilImage,0);

   /* Select the MIL buffer to be displayed in the user-specified window. */
   MdispSelectWindow(MilDisplay, MilImage, UserWindowHandle);

   /* Print a string in the image buffer using MIL.
      Note: When a MIL buffer is modified using a MIL command, the display
      automatically updates the window passed to MdispSelectWindow().
   */
      MgraFont(M_DEFAULT, M_FONT_DEFAULT_LARGE);
      MgraText(M_DEFAULT, MilImage, (BufSizeX/8)*2, BufSizeY/2,
                                    MIL_TEXT(" Welcome to MIL !!! "));
      MgraRect(M_DEFAULT, MilImage, ((BufSizeX/8)*2)-60, (BufSizeY/2)-80,
                                    ((BufSizeX/8)*2)+370, (BufSizeY/2)+100);
      MgraRect(M_DEFAULT, MilImage, ((BufSizeX/8)*2)-40, (BufSizeY/2)-60,
                                    ((BufSizeX/8)*2)+350, (BufSizeY/2)+80);
      MgraRect(M_DEFAULT, MilImage, ((BufSizeX/8)*2)-20, (BufSizeY/2)-40,
                                    ((BufSizeX/8)*2)+330, (BufSizeY/2)+60);

   /* Open a message box to wait for a key press. */
   MessageBox(0,"""Welcome to MIL !!!"" was printed", "MIL application example",
                                              MB_APPLMODAL | MB_ICONEXCLAMATION);

   /* Grab in the user window if supported. */
   if (MilDigitizer)
      {
      /* Grab continuously. */
      MdigGrabContinuous(MilDigitizer, MilImage);

      /* Open a message box to wait for a key press. */
      MessageBox(0,"Continuous grab in progress", "MIL application example",
                                         MB_APPLMODAL | MB_ICONEXCLAMATION );

      /* Stop continuous grab. */
      MdigHalt(MilDigitizer);
      }

   /* Remove the MIL buffer from the display. */
   MdispSelect(MilDisplay, M_NULL);

   /* Free allocated objects. */
   MbufFree(MilImage);

end:

   if (MilDigitizer)
      MdigFree(MilDigitizer);
   MdispFree(MilDisplay);
   MsysFree(MilSystem);
   MappFree(MilApplication);
}

/***************************************************************************/
/*/
* Name:       MilApplicationPaint()
*
* synopsis:  This function can be used to complete the work done by
*            the MIL paint manager when the client window receives
*            a paint message.
*
*            This is called every time WindowProc receives a
*            WM_PAINT message
*/

void MilApplicationPaint(HWND UserWindowHandle)
{
   /* Nothing is done here because nothing gets drawn on top of
    * or around our displayed MIL buffer in the window.
    */
   UserWindowHandle = UserWindowHandle; /* line to remove warning */
}

/***************************************************************************/
/*
 * Synopsis:  The next functions lets the user to quickly create a Windows
 *            program which will call the MilApplication() function each
 *            time the Start menu item is selected. The user may also
 *            use the MilApplicationPaint() function that is called
 *            each time the window receives a paint message to
 *            complete the work already done by the MIL paint manager.
 */

/* Prototypes for Windows application */
int WINAPI WinMain(HINSTANCE, HINSTANCE, LPSTR, int);
long WINAPI MainWndProc(HWND, UINT, WPARAM, LPARAM);
BOOL InitApplication(HINSTANCE, int);

/***************************************************************************/
/*
 *   Name:     WinMain()
 *
 *   Synopsis: Calls initialization function, processes message loop.
 */

int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance,
                                        LPSTR lpCmdLine, int nCmdShow)
{
   MSG msg;

   /* Avoid warning for the unused parameter. */
   lpCmdLine = lpCmdLine;

   /* Allow only one instance of program. */
   if (hPrevInstance)
      {
      MessageBox(0, "Sample MIL Host Windows Application already active.\n"
                    "This version sample application does not allow multiple instances.",
                    "Sample MIL Host Application - Error",
                     MB_OK | MB_ICONSTOP);
      return (FALSE);
      }

   /* Exit if unable to initialize. */
   if (!InitApplication(hInstance,nCmdShow))
      return (FALSE);

   /* Get and dispatch messages until a WM_QUIT message is received. */
   while (GetMessage(&msg,(HWND)NULL,(UINT)NULL,(UINT)NULL))
      {
      TranslateMessage(&msg);   /* Translates virtual key codes. */
      DispatchMessage(&msg);    /* Dispatches message to window. */
      }

   return (msg.wParam);    /* Returns the value from PostQuitMessage. */
}

/***************************************************************************/
/*
 *   Name:      MainWndProc()
 *
 *   Synopsis:  Processes messages:
 *              WM_COMMAND     - application menu.
 *              WM_ENDSESSION  - destroy window.
 *              WM_CLOSE       - destroy window.
 *              WM_DESTROY     - destroy window.
 */

long WINAPI MainWndProc(HWND   hWnd, UINT message, WPARAM wParam, LPARAM lParam)
{
   HMENU hAppMenu;

   switch (message)
      {
      case WM_COMMAND:     /* message: command from application menu. */
         switch(wParam)
            {
            case IDM_START_MAIN:
               hAppMenu = GetMenu(hWnd);
               EnableMenuItem(hAppMenu,MF_BYCOMMAND | IDM_START_MAIN,MF_GRAYED);
               DrawMenuBar(hWnd);
               MilApplication(hWnd);
               if (!GetWindowLong(hWnd,sizeof(LPVOID)))
                  {
                  PostQuitMessage(0);
                  }
               else
                  {
                  EnableMenuItem(hAppMenu,MF_BYCOMMAND | IDM_START_MAIN,MF_ENABLED);
                  DrawMenuBar(hWnd);
                  }
               break;
            default:
               return (DefWindowProc(hWnd, message, wParam, lParam));
            }
         break;
      case WM_PAINT:
         MilApplicationPaint(hWnd);
         return (DefWindowProc(hWnd, message, wParam, lParam));
      case WM_ENDSESSION:
      case WM_CLOSE:
         PostMessage(hWnd, WM_DESTROY, (WORD)0, (LONG)0);
         break;
      case WM_DESTROY:
         /* Free the reserved extra window LPSTR to szAppName. */
         if (GetWindowLong(hWnd,sizeof(LPVOID)))
            {
            free((char *)GetWindowLong(hWnd,sizeof(LPVOID)));
            SetWindowLong(hWnd,sizeof(LPVOID),0L);
            }
         PostQuitMessage(0);

      default:
         return (DefWindowProc(hWnd, message, wParam, lParam));
      }
   return(0L);
}

/***************************************************************************/
/*
 *   Name:     InitApplication()
 *
 *   Synopsis: Initializes window data, registers window class and
 *             creates main window.
 */

BOOL InitApplication(HINSTANCE hInstance, int nCmdShow)
{
   WNDCLASS        wc;
   HWND            hwnd;
   char            TmpName[MIL_APPLICATION_NAME_SIZE];
   char *          szAppName;

   /* Allocation and storage of application name. */
   szAppName = (LPSTR)malloc((MIL_APPLICATION_NAME_SIZE+40) * sizeof(char));
   strcpy(szAppName,MIL_APPLICATION_NAME);

   wc.style          = CS_VREDRAW | CS_HREDRAW;
   wc.lpfnWndProc    = (WNDPROC)MainWndProc;
   wc.cbClsExtra     = 0;
   wc.cbWndExtra     = sizeof(LPVOID)+sizeof(LPSTR);
   wc.hInstance      = hInstance;
   wc.hIcon          = LoadIcon(0,IDI_APPLICATION);
   wc.hCursor        = LoadCursor(0, IDC_ARROW);
   wc.hbrBackground  = CreateSolidBrush(RGB(BACKCOLORRED,
                                            BACKCOLORGREEN,
                                            BACKCOLORBLUE));
   wc.lpszMenuName   = "MILAPPLMENU";
   wc.lpszClassName  = szAppName;

   /* Register the class. */
   if (!RegisterClass(&wc))
      return (FALSE);

   /* Create the window. */
   hwnd = CreateWindow (szAppName,
                        szAppName,
                        WS_OVERLAPPEDWINDOW,
                        CW_USEDEFAULT, CW_USEDEFAULT,
                        CW_USEDEFAULT, CW_USEDEFAULT,
                        0, 0, hInstance, 0) ;
   if (!hwnd)
      return (FALSE);

   lstrcat(szAppName," - ");
   GetModuleFileName(hInstance,TmpName,MIL_APPLICATION_NAME_SIZE);
   lstrcat(szAppName,TmpName);
   SetWindowText(hwnd,szAppName);

   /* Set the reserved extra window LPVOID to NULL. */
   SetWindowLong(hwnd,0,0L);

   /* Set the reserved extra window LPSTR to szAppName. */
   SetWindowLong(hwnd,sizeof(LPVOID),(DWORD)szAppName);

   /* Display the window. */
   ShowWindow (hwnd, nCmdShow) ;
   UpdateWindow (hwnd) ;

   return (TRUE);
}


Example: medge.c

/*****************************************************************************/
/*
 * File name: MEdge.c
 *
 * Synopsis:  This program uses the MIL Edge Finder module to find and measure
 *            the outer diameter of good seals in a target image.
 */
#include <mil.h>
#include <conio.h>

/* Source MIL image file specifications. */
#define CONTOUR_IMAGE               M_IMAGE_PATH MIL_TEXT("Seals.mim")
#define CONTOUR_MAX_RESULTS         100
#define CONTOUR_MAXIMUM_ELONGATION  0.8
#define CONTOUR_DRAW_COLOR          M_COLOR_GREEN
#define CONTOUR_LABEL_COLOR         M_COLOR_RED

 /* Main function. */
void main(void)
   {
   MIL_ID MilApplication,                         /* Application identifier.   */
          MilSystem,                              /* System Identifier.        */
          MilDisplay,                             /* Display identifier.       */
          MilImage,                               /* Image buffer identifier.  */
          MilOverlayImage,                        /* Overlay image.            */
          MilEdgeContext,                         /* Edge context.             */
          MilEdgeResult;                          /* Edge result identifier.   */
   long   EdgeDrawColor = CONTOUR_DRAW_COLOR,     /* Edge draw color.          */
          LabelDrawColor = CONTOUR_LABEL_COLOR,   /* Text draw color.          */
          NumEdgeFound  = 0L,                     /* Number of edges found.    */
          NumResults    = 0L,                     /* Number of results found.  */
          i;                                      /* Loop variable.            */
   double MeanFeretDiameter[CONTOUR_MAX_RESULTS]; /* Edge mean Feret diameter. */

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay, M_NULL, M_NULL);

   /* Restore the image and display it. */
   MbufRestore(CONTOUR_IMAGE, MilSystem, &MilImage);
   MdispSelect(MilDisplay, MilImage);

   /* Prepare for overlay annotations. */
   MdispControl(MilDisplay, M_OVERLAY, M_ENABLE);
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);
   MdispInquire(MilDisplay, M_OVERLAY_ID, &MilOverlayImage);

   /* Pause to show the original image. */
   printf("\nEDGE MODULE:\n");
   printf("------------\n\n");
   printf("This program determines the outer seal diameters in the displayed image \n");
   printf("by detecting and analyzing contours with the Edge Finder module.\n");
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Allocate a Edge Finder context. */
   MedgeAlloc(MilSystem, M_CONTOUR, M_DEFAULT, &MilEdgeContext);

   /* Allocate a result buffer. */
   MedgeAllocResult(MilSystem, M_DEFAULT, &MilEdgeResult);

   /* Enable features to compute. */
   MedgeControl(MilEdgeContext, M_MOMENT_ELONGATION,                M_ENABLE);
   MedgeControl(MilEdgeContext, M_FERET_MEAN_DIAMETER+M_SORT1_DOWN, M_ENABLE);

   /* Calculate edges and features. */
   MedgeCalculate(MilEdgeContext, MilImage, M_NULL, M_NULL, M_NULL, MilEdgeResult, M_DEFAULT);

   /* Get the number of edges found. */
   MedgeGetResult(MilEdgeResult, M_DEFAULT, M_NUMBER_OF_CHAINS+M_TYPE_LONG, &NumEdgeFound, M_NULL);

   /* Draw edges in the source image to show the result. */
   MgraColor(M_DEFAULT, EdgeDrawColor);
   MedgeDraw(M_DEFAULT, MilEdgeResult, MilOverlayImage, M_DRAW_EDGES, M_DEFAULT, M_DEFAULT);

   /* Pause to show the edges. */
   printf("%d edges were found in the image.\n", NumEdgeFound);
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Exclude elongated edges. */
   MedgeSelect(MilEdgeResult, M_EXCLUDE, M_MOMENT_ELONGATION, M_LESS, CONTOUR_MAXIMUM_ELONGATION, M_NULL);

   /* Exclude inner chains. */
   MedgeSelect(MilEdgeResult, M_EXCLUDE, M_INCLUDED_EDGES, M_INSIDE_BOX, M_NULL, M_NULL);

   /* Draw remaining edges and their index to show the result. */
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);
   MgraColor(M_DEFAULT, EdgeDrawColor);
   MedgeDraw(M_DEFAULT, MilEdgeResult, MilOverlayImage, M_DRAW_EDGES, M_DEFAULT, M_DEFAULT);

   /* Pause to show the results. */
   printf("Elongated edges and inner edges of each seal were removed.\n");
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Get the number of edges found. */
   MedgeGetResult(MilEdgeResult, M_DEFAULT, M_NUMBER_OF_CHAINS+M_TYPE_LONG, &NumResults, M_NULL);

   /* If the right number of edges were found. */
   if ( (NumResults >= 1) && (NumResults <= CONTOUR_MAX_RESULTS) )
      {
      /* Draw the index of each edge. */
      MgraColor(M_DEFAULT, LabelDrawColor);
      MedgeDraw(M_DEFAULT, MilEdgeResult, MilOverlayImage, M_DRAW_INDEX, M_DEFAULT, M_DEFAULT);

      /* Get the mean Feret diameters. */
      MedgeGetResult(MilEdgeResult, M_DEFAULT, M_FERET_MEAN_DIAMETER, &MeanFeretDiameter, M_NULL);

      /* Print the results. */
      printf("Mean diameter of the %ld outer edges are:\n\n", NumResults);
      printf("Index   Mean diameter \n");
      for (i=0; i<NumResults; i++)
         {
         printf("%-11d%-13.2f%\n", i, MeanFeretDiameter[i]);
         }
      }
   else
      {
      printf("Edges have not been found or the number of found edges is greater than\n");
      printf("the specified maximum number of edges !\n\n");
      }

   /* Wait for a key press. */
   printf("\nPress <Enter> to end.\n");
   getch();

   /* Free MIL objects. */
   MbufFree(MilImage);
   MedgeFree(MilEdgeContext);
   MedgeFree(MilEdgeResult);

   /* Free defaults. */
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
   }


Example: mfunc.c

/*****************************************************************************************/
/*
 * File name: MFunc.c
 *
 * Synopsis:  This example shows the use of the MIL Function Developer Tool Kit and how
 *            MIL and custom code can be mixed to create a custom MIL function that
 *            accesses the data pointer of a MIL buffer directly in order to process it.
 *
 *            The example creates a Master MIL function that registers all the parameters
 *            to MIL and calls the Slave function. The Slave function retrieves all the
 *            parameters, gets the pointers to the MIL image buffers, uses them to access
 *            the data directly, and adds a constant.
 *
 *            Note: The images must be 8-bit unsigned.
 */
#include <mil.h>
#include <conio.h>

/* MIL function specifications. */
#define FUNCTION_NB_PARAM                 3
#define FUNCTION_OPCODE_ADD_CONSTANT      1
#define FUNCTION_PARAMETER_ERROR_CODE     1
#define FUNCTION_SUPPORTED_IMAGE_DEPTH    8

/* Target image file name. */
#define IMAGE_FILE   M_IMAGE_PATH MIL_TEXT("BoltsNutsWashers.mim")

/* Master and Slave MIL functions declarations. */
void AddConstant(MIL_ID SrcImageId, MIL_ID DstImageId, unsigned long ConstantToAdd);
void MFTYPE SlaveAddConstant(MIL_ID Func);

/* Master MIL Function definition. */
/* ------------------------------- */

void AddConstant(MIL_ID SrcImageId, MIL_ID DstImageId, unsigned long ConstantToAdd)
{
   MIL_ID   Func;

   /* Allocate a MIL function context that will be used to call a target
      Slave function locally on the Host to do the processing.
   */
   MfuncAlloc(MIL_TEXT("AddConstant"),
              FUNCTION_NB_PARAM,
              SlaveAddConstant, M_NULL, M_NULL,
              M_USER_MODULE_1+FUNCTION_OPCODE_ADD_CONSTANT,
              M_ASYNCHRONOUS_FUNCTION+M_LOCAL,
              &Func
             );

   /* Register the parameters. */
   MfuncParamId(  Func, 1, SrcImageId, M_IMAGE, M_IN);
   MfuncParamId(  Func, 2, DstImageId, M_IMAGE, M_OUT);
   MfuncParamLong(Func, 3, ConstantToAdd);

   /* Call the target Slave function. */
   MfuncCall(Func);

   /* Free the MIL function context. */
   MfuncFree(Func);
}

/* MIL Slave function definition. */
/* ------------------------------ */

void MFTYPE SlaveAddConstant(MIL_ID Func)
{
  MIL_ID SrcImageId, DstImageId;
  unsigned long ConstantToAdd;
  unsigned char *SrcImageDataPtr, *DstImageDataPtr;
  long   SrcImageSizeX, SrcImageSizeY, SrcImageSizeBit, SrcImagePitchByte;
  long   DstImageSizeX, DstImageSizeY, DstImageSizeBit, DstImagePitchByte;
  long   x, y, TempValue;

  /* Read the parameters. */
  MfuncParamValue(Func, 1, &SrcImageId);
  MfuncParamValue(Func, 2, &DstImageId);
  MfuncParamValue(Func, 3, &ConstantToAdd);

  /* Read image information. */
  MbufInquire(SrcImageId, M_HOST_ADDRESS, &SrcImageDataPtr);
  MbufInquire(SrcImageId, M_SIZE_X,       &SrcImageSizeX);
  MbufInquire(SrcImageId, M_SIZE_Y,       &SrcImageSizeY);
  MbufInquire(SrcImageId, M_SIZE_BIT,     &SrcImageSizeBit);
  MbufInquire(SrcImageId, M_PITCH_BYTE,   &SrcImagePitchByte);
  MbufInquire(DstImageId, M_HOST_ADDRESS, &DstImageDataPtr);
  MbufInquire(DstImageId, M_SIZE_X,       &DstImageSizeX);
  MbufInquire(DstImageId, M_SIZE_Y,       &DstImageSizeY);
  MbufInquire(SrcImageId, M_SIZE_BIT,     &DstImageSizeBit);
  MbufInquire(DstImageId, M_PITCH_BYTE,   &DstImagePitchByte);

  /* Reduce the destination area to process if necessary. */
  if (SrcImageSizeX < DstImageSizeX)   DstImageSizeX = SrcImageSizeX;
  if (SrcImageSizeY < DstImageSizeY)   DstImageSizeY = SrcImageSizeY;

  /* If images have the proper depth, perform the operation using a custom C code. */
  if ((SrcImageSizeBit == DstImageSizeBit) && (SrcImageSizeBit == FUNCTION_SUPPORTED_IMAGE_DEPTH))
     {
     /* Add the constant to the image. */
     for (y= 0; y < DstImageSizeY; y++)
        {
        for (x= 0; x < DstImageSizeX; x++)
           {
           /* Calculate the value to write. */
           TempValue = (long)SrcImageDataPtr[x] + (long)ConstantToAdd;

           /* Write the value if no overflow, else saturate. */
           if (TempValue <= 0xff)
              DstImageDataPtr[x] = (unsigned char)TempValue;
           else
              DstImageDataPtr[x] = 0xff;
           }

         /* Move pointer to the next line taking into account the image's pitch. */
         (unsigned char *)SrcImageDataPtr += SrcImagePitchByte;
         (unsigned char *)DstImageDataPtr += SrcImagePitchByte;
        }
     }
  else
     {
     /* Report a MIL error. */
     MfuncErrorReport(Func,M_FUNC_ERROR+FUNCTION_PARAMETER_ERROR_CODE,
                      MIL_TEXT("Invalid parameter."),
                      MIL_TEXT("Image type not supported"),
                      MIL_TEXT("Image depth must be 8-bit."),
                      M_NULL
                     );
     }

  /* Signal to MIL that the buffers are now in cache (required by Odyssey). */
  MbufControl(SrcImageId, M_CACHE_CONTROL, M_IN_CACHE);
  MbufControl(DstImageId, M_CACHE_CONTROL, M_IN_CACHE);

  /* Signal to MIL that the destination buffer was modified. */
  MbufControl(DstImageId, M_MODIFIED, M_DEFAULT);
}

/* Main to test the custom function. */
/* --------------------------------- */

void main(void)
{
   MIL_ID MilApplication,    /* Application identifier.  */
          MilSystem,         /* System identifier.       */
          MilDisplay,        /* Display identifier.      */
          MilImage;          /* Image buffer identifier. */

   /* Allocate default application, system, display and image. */
   MappAllocDefault(M_COMPLETE, &MilApplication, &MilSystem,
                                &MilDisplay, M_NULL, M_NULL);

   /* Load source image into a Host memory image buffer. */
   MbufAlloc2d(MilSystem,
               MbufDiskInquire(IMAGE_FILE, M_SIZE_X, M_NULL),
               MbufDiskInquire(IMAGE_FILE, M_SIZE_Y, M_NULL),
               8+M_UNSIGNED,
               M_IMAGE+M_DISP+M_HOST_MEMORY,
               &MilImage);
   MbufLoad(IMAGE_FILE, MilImage);

   /* Display the image. */
   MdispSelect(MilDisplay, MilImage);

   /* Pause. */
   printf("\nMIL FUNCTION DEVELOPER'S TOOLKIT:\n");
   printf("----------------------------------\n\n");
   printf("This example creates a custom MIL function that processes\n");
   printf("an image using its data pointer directly.\n\n");
   printf("Target image was loaded.\n");
   printf("Press a key to continue.\n\n");
   getch();

   /* Process the image directly with the custom MIL function. */
   AddConstant(MilImage, MilImage, 0x40);

   /* Pause */
   printf("The white level of the image was augmented.\n");
   printf("Press a key to terminate.\n\n");
   getch();

   /* Free all allocations. */
   MbufFree(MilImage);
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
}


Example: mimconvolve.c

/*****************************************************************************/
/*
 * File name: MImConvolve.c
 *
 * Synopsis:  This program performs a 3x3 convolution using a custom kernel
 *            and calculates the convolution time.
 */
#include <conio.h>
#include <mil.h>

/* Target MIL image specifications. */
#define IMAGE_FILE       M_IMAGE_PATH MIL_TEXT("BaboonMono.mim")
#define ZOOM_VALUE       2

/* Kernel data definition. */
#define KERNEL_WIDTH     3L
#define KERNEL_HEIGHT    3L
#define KERNEL_DEPTH     8L
unsigned char  KernelData[KERNEL_HEIGHT][KERNEL_WIDTH] =
               { {1, 2, 1},
                 {2, 4, 2},
                 {1, 2, 1}
               };

/* Timing loop iterations. */
#define NB_LOOP 100

void main(void)
{
   MIL_ID MilApplication,  /* Application identifier.     */
          MilSystem,       /* System identifier.          */
          MilDisplay,      /* Display identifier.         */
          MilDisplayImage, /* Image buffer identifier.    */
          MilImage,        /* Image buffer identifier.    */
          MilKernel;       /* Custom kernel identifier.   */
   long   n;
   double Time;

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem,
                    &MilDisplay, M_NULL, M_NULL);

   /* Restore source image into an automatically allocated image buffers. */
   MbufRestore(IMAGE_FILE, MilSystem, &MilImage);
   MbufRestore(IMAGE_FILE, MilSystem, &MilDisplayImage);

   /* Zoom display to see the result of image processing better. */
   MdispZoom(MilDisplay, ZOOM_VALUE, ZOOM_VALUE);

   /* Display the image buffer. */
   MdispSelect(MilDisplay, MilDisplayImage);

   /* Pause to show the original image. */
   printf("\nIMAGE PROCESSING:\n");
   printf("-----------------\n\n");
   printf("This program performs a convolution on the displayed image.\n");
   printf("It uses a custom smoothing kernel.\n");
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Allocate a MIL kernel. */
   MbufAlloc2d(MilSystem, KERNEL_WIDTH, KERNEL_HEIGHT,
               KERNEL_DEPTH+M_UNSIGNED, M_KERNEL, &MilKernel);

   /* Put the custom data in it. */
   MbufPut(MilKernel,KernelData);

   /* Set a normalization (divide) factor to have a kernel with
    * a sum equal to one.
    */
   MbufControlNeighborhood(MilKernel,M_NORMALIZATION_FACTOR,16L);

   /* Convolve the image using the kernel. */
   MimConvolve(MilImage, MilDisplayImage, MilKernel);

   /* Now time the convolution (MimConvolve()):
      Overscan calculation is disabled and a destination image that
      is not displayed is used to have the real convolution time. Also the
      function must be called once before the timing loop for more accurate
      time (dll load, ...).
   */
   MbufControlNeighborhood(MilKernel, M_OVERSCAN, M_DISABLE);
   MimConvolve(MilDisplayImage, MilImage, MilKernel);
   MappTimer(M_TIMER_RESET+M_SYNCHRONOUS, M_NULL);
   for (n= 0; n < NB_LOOP; n++)
         MimConvolve(MilDisplayImage, MilImage, MilKernel);
   MappTimer(M_TIMER_READ+M_SYNCHRONOUS, &Time);

   /* Pause to show the result. */
   printf("Convolve time: %.3f ms.\n", Time*1000/NB_LOOP);
   printf("Press <Enter> to terminate.\n");
   getch();

   /* Free all allocations. */
   MbufFree(MilKernel);
   MbufFree(MilImage);
   MbufFree(MilDisplayImage);
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
}


Example: mimfft.c

/************************************************************************************/
/*
 * File name: MImFFT.c
 *
 * Synopsis:  This program uses the Fast Fourier Transform to filter an image.
 */
#include <mil.h>
#include <math.h>
#include <conio.h>

/* Target image specifications. */
#define NOISY_IMAGE                    M_IMAGE_PATH MIL_TEXT("noise.mim")
#define IMAGE_WIDTH                    256
#define IMAGE_HEIGHT                   256
#define X_NEGATIVE_FREQUENCY_POSITION   63
#define X_POSITIVE_FREQUENCY_POSITION  191
#define Y_FREQUENCY_POSITION           127
#define CIRCLE_WIDTH                     9

void main(void)
{
   MIL_ID MilApplication,   /* Application identifier.                  */
          MilSystem,        /* System identifier.                       */
          MilDisplay,       /* Display identifier.                      */
          MilImage,         /* Image buffer identifier.                 */
          MilOverlayImage,  /* Overlay image buffer identifier.         */
          MilSubImage00,    /* Child buffer identifier.                 */
          MilSubImage01,    /* Child buffer identifier.                 */
          MilSubImage10,    /* Child buffer identifier.                 */
          MilSubImage11,    /* Child buffer identifier.                 */
          MilTransformReal, /* Real part of the transformed image.      */
          MilTransformIm;   /* Imaginary part of the transformed image. */

   float  ZeroVal = 0.0;

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay,M_NULL, M_NULL);

   /* Allocate a display buffer and clear it. */
   MbufAlloc2d(MilSystem, IMAGE_WIDTH*2, IMAGE_HEIGHT*2,
               8+M_UNSIGNED, M_IMAGE+M_PROC+M_DISP, &MilImage);
   MbufClear(MilImage, 0L);

   /* Display the image buffer and prepare for overlay annotations. */
   MdispSelect(MilDisplay, MilImage);
   MdispControl(MilDisplay, M_OVERLAY, M_ENABLE);
   MdispInquire(MilDisplay, M_OVERLAY_ID, &MilOverlayImage);

   /* Allocate child buffers in the 4 quadrants of the display image. */
   MbufChild2d(MilImage, 0L, 0L, IMAGE_WIDTH, IMAGE_HEIGHT, &MilSubImage00);
   MbufChild2d(MilImage, IMAGE_WIDTH, 0L,IMAGE_WIDTH, IMAGE_HEIGHT, &MilSubImage01);
   MbufChild2d(MilImage, 0L, IMAGE_HEIGHT, IMAGE_WIDTH, IMAGE_HEIGHT, &MilSubImage10);
   MbufChild2d(MilImage, IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_WIDTH, IMAGE_HEIGHT, &MilSubImage11);

   /* Allocate processing buffers. */
   MbufAlloc2d(MilSystem, IMAGE_WIDTH, IMAGE_HEIGHT, 32+M_FLOAT, M_IMAGE+M_PROC, &MilTransformReal);
   MbufAlloc2d(MilSystem, IMAGE_WIDTH, IMAGE_HEIGHT, 32+M_FLOAT, M_IMAGE+M_PROC, &MilTransformIm);

   /* Load a noisy image. */
   MbufLoad(NOISY_IMAGE, MilSubImage00);

   /* Print a message on the screen. */
   printf("\nFFT:\n");
   printf("----\n\n");
   printf("The frequency spectrum of a noisy image will be computed to remove the periodic noise.\n");
   printf("Press <Enter> to continue.\n\n");
   getchar();

   /* The image is Fourier transformed to obtain the magnitude of the
      spectrum. This result will be used to design the filter.   */
   MimTransform(MilSubImage00, M_NULL, MilTransformReal, MilTransformIm,
                M_FFT, M_FORWARD+M_CENTER+M_MAGNITUDE+M_LOG_SCALE);
   MbufCopy(MilTransformReal, MilSubImage10);

   /* Draw circles in the overlay around the points of interest. */
   MbufCopy(MilTransformReal, MilSubImage11);
   MgraColor(M_DEFAULT, M_COLOR_YELLOW);
   MgraArc(M_DEFAULT, MilOverlayImage, X_NEGATIVE_FREQUENCY_POSITION,
           Y_FREQUENCY_POSITION+IMAGE_HEIGHT, CIRCLE_WIDTH, CIRCLE_WIDTH, 0, 360);
   MgraArc(M_DEFAULT, MilOverlayImage, X_POSITIVE_FREQUENCY_POSITION,
           Y_FREQUENCY_POSITION+IMAGE_HEIGHT, CIRCLE_WIDTH, CIRCLE_WIDTH, 0, 360);
   MgraArc(M_DEFAULT, MilOverlayImage, X_NEGATIVE_FREQUENCY_POSITION+IMAGE_WIDTH,
           Y_FREQUENCY_POSITION+IMAGE_HEIGHT, CIRCLE_WIDTH, CIRCLE_WIDTH, 0, 360);
   MgraArc(M_DEFAULT, MilOverlayImage, X_POSITIVE_FREQUENCY_POSITION+IMAGE_WIDTH,
           Y_FREQUENCY_POSITION+IMAGE_HEIGHT, CIRCLE_WIDTH, CIRCLE_WIDTH, 0, 360);

   /* Put zero in the spectrum where the noise is located. */
   MbufPut2d(MilSubImage11,   X_NEGATIVE_FREQUENCY_POSITION,
                              Y_FREQUENCY_POSITION, 1, 1, &ZeroVal);
   MbufPut2d(MilSubImage11,   X_POSITIVE_FREQUENCY_POSITION,
                              Y_FREQUENCY_POSITION, 1, 1, &ZeroVal);

   /* Compute the Fast Fourier Transform of the image. */
   MimTransform(MilSubImage00, M_NULL, MilTransformReal,
                MilTransformIm, M_FFT, M_FORWARD+M_CENTER);

   /* Filter the image in the frequency domain. */
   MbufPut2d(MilTransformReal, X_NEGATIVE_FREQUENCY_POSITION,
                               Y_FREQUENCY_POSITION, 1, 1, &ZeroVal);
   MbufPut2d(MilTransformReal, X_POSITIVE_FREQUENCY_POSITION,
                               Y_FREQUENCY_POSITION, 1, 1, &ZeroVal);
   MbufPut2d(MilTransformIm, X_NEGATIVE_FREQUENCY_POSITION,
                             Y_FREQUENCY_POSITION, 1, 1, &ZeroVal);
   MbufPut2d(MilTransformIm, X_POSITIVE_FREQUENCY_POSITION,
                             Y_FREQUENCY_POSITION, 1, 1, &ZeroVal);

   /* Recover the image in the spatial domain. */
   MimTransform(MilTransformReal, MilTransformIm,
                MilSubImage01, M_NULL, M_FFT, M_REVERSE+M_CENTER);

   /* Print a message. */
   printf("The frequency components of the noise are located in the center of the circles.\n");
   printf("The noise was removed by setting these frequency components to zero.\n");
   printf("Press <Enter> to end.\n\n");
   getchar();

   /* Free buffers. */
   MbufFree(MilSubImage00);
   MbufFree(MilSubImage01);
   MbufFree(MilSubImage10);
   MbufFree(MilSubImage11);
   MbufFree(MilImage);
   MbufFree(MilTransformReal);
   MbufFree(MilTransformIm);

   /* Free defaults. */
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
}


Example: mimhistogram.c

/***********************************************************************************/
/*
 * File name: MImHistogram.c
 *
 * Synopsis:  This program loads an image of a tissue sample, calculates its intensity
 *            histogram and draws it.
 */
#include <conio.h>
#include <mil.h>

/* Target MIL image file specifications. */
#define IMAGE_FILE  M_IMAGE_PATH MIL_TEXT("Cell.mim")

/* Number of possible pixel intensities. */
#define HIST_NUM_INTENSITIES  256
#define HIST_SCALE_FACTOR     8
#define HIST_X_POSITION       250
#define HIST_Y_POSITION       450

/* Main function. */
void main(void)
   {
   MIL_ID MilApplication,                   /* Application identifier       */
          MilSystem,                        /* System identifier.           */
          MilDisplay,                       /* Display identifier.          */
          MilImage,                         /* Image buffer identifier.     */
          MilOverlayImage,                  /* Overlay buffer identifier.   */
          HistResult;                       /* Histogram buffer identifier. */
   long   HistValues[HIST_NUM_INTENSITIES]; /* Histogram values.            */
   long   XStart[HIST_NUM_INTENSITIES], YStart[HIST_NUM_INTENSITIES],
          XEnd[HIST_NUM_INTENSITIES],   YEnd[HIST_NUM_INTENSITIES];
   long   AnnotationColor = M_COLOR_RED, i;

   /* Allocate the default system and image buffer. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay, M_NULL, M_NULL);

   /* Restore source image into an automatically allocated image buffer. */
   MbufRestore(IMAGE_FILE, MilSystem, &MilImage);

   /* Display the image buffer and prepare for overlay annotations. */
   MdispSelect(MilDisplay, MilImage);
   MdispControl(MilDisplay, M_OVERLAY, M_ENABLE);
   MdispInquire(MilDisplay, M_OVERLAY_ID, &MilOverlayImage);

   /* Allocate a histogram result buffer. */
   MimAllocResult(MilSystem, HIST_NUM_INTENSITIES, M_HIST_LIST, &HistResult);

   /* Calculate the histogram. */
   MimHistogram(MilImage, HistResult);

   /* Get the results. */
   MimGetResult(HistResult, M_VALUE, HistValues);

   /* Draw the histogram in the overlay. */
   MgraColor(M_DEFAULT, AnnotationColor);
   for(i=0; i<HIST_NUM_INTENSITIES; i++)
      {
      XStart[i] = i+HIST_X_POSITION+1,
      YStart[i] = HIST_Y_POSITION;
      XEnd[i]   = i+HIST_X_POSITION+1,
      YEnd[i] = HIST_Y_POSITION-(HistValues[i]/HIST_SCALE_FACTOR);
      }
   MgraLines(M_DEFAULT, MilOverlayImage, HIST_NUM_INTENSITIES,
                        XStart, YStart, XEnd, YEnd, M_DEFAULT);

   /* Print a message. */
   printf("\nHISTOGRAM:\n");
   printf("----------\n\n");
   printf("The histogram of the image was calculated and drawn.\n");
   printf("Press <Enter> to end.\n\n");
   getch();

   /* Free all allocations. */
   MimFree(HistResult);
   MbufFree(MilImage);
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
}


Example: mimmorphic.c

/*******************************************************************************/
/* File name: MImMorphic.c
 *
 * Synopsis: This program loads an image of some tissue and then do opening on
 *           it using two methods to show the differences.
 */
#include <mil.h>
#include <conio.h>

/* Target MIL image file specifications. */
#define IMAGE_FILE            M_IMAGE_PATH MIL_TEXT("Cell.mim")
#define IMAGE_WIDTH           240L
#define IMAGE_HEIGHT          240L
#define IMAGE_DEPTH           8L
#define IMAGE_THRESHOLD_VALUE 128L

/* Structuring element information. */
#define STRUCT_ELEM_WIDTH     5L
#define STRUCT_ELEM_HEIGHT    5L
#define STRUCT_ELEM_DEPTH     32L

/* Small particle radius (in pixels). */
#define SMALL_PARTICLE_RADIUS 2L

main(void)
{
   MIL_ID MilApplication, /* Application identifier. */
          MilSystem,    /* System identifier.        */
          MilDisplay,   /* Display identifier.       */
          MilImage,     /* Image buffer identifier.  */
          BinImage,     /* Binary Image buffer identifier.                 */
          MilSubImage0, /* Sub-image buffer identifier for original image. */
          MilSubImage1, /* Sub-image buffer identifier for binarization.   */
          MilSubImage2, /* Sub-image buffer identifier for common open.    */
          MilSubImage3; /* Sub-image buffer identifier for customized Open.*/
   MIL_ID StructElem;   /* Structing element buffer. */

   /* Structuring element data definition. */
   long  StructArray[STRUCT_ELEM_HEIGHT][STRUCT_ELEM_WIDTH]  =
         { {M_DONT_CARE, M_DONT_CARE, 1, M_DONT_CARE, M_DONT_CARE},
           {M_DONT_CARE,           1, 1,           1, M_DONT_CARE},
           {          1,           1, 1,           1,           1},
           {M_DONT_CARE,           1, 1,           1, M_DONT_CARE},
           {M_DONT_CARE, M_DONT_CARE, 1, M_DONT_CARE, M_DONT_CARE}
         };

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay, M_NULL, M_NULL);

   /* Allocate an image buffer, load and displays it. */
   MbufAlloc2d(M_DEFAULT, IMAGE_WIDTH*2, IMAGE_HEIGHT*2, 8+M_UNSIGNED, M_IMAGE+M_PROC+M_DISP, &MilImage);
   MbufClear(MilImage, 0);
   MdispSelect(MilDisplay, MilImage);

   /* Allocate a binary image buffer for fast processing. */
   MbufAlloc2d(M_DEFAULT, IMAGE_WIDTH, IMAGE_HEIGHT, 1+M_UNSIGNED, M_IMAGE+M_PROC, &BinImage);

   /* Define 4 processing buffers in the display buffer, restricting the
    * regions to be processed to the top left corner of the original image.
    */
   MbufChild2d(MilImage, 0L, 0L, IMAGE_WIDTH, IMAGE_HEIGHT, &MilSubImage0);
   MbufChild2d(MilImage, IMAGE_WIDTH, 0L, IMAGE_WIDTH, IMAGE_HEIGHT, &MilSubImage1);
   MbufChild2d(MilImage, 0L, IMAGE_HEIGHT, IMAGE_WIDTH, IMAGE_HEIGHT, &MilSubImage2);
   MbufChild2d(MilImage, IMAGE_WIDTH, IMAGE_HEIGHT, IMAGE_WIDTH, IMAGE_HEIGHT, &MilSubImage3);

   /* Load source image into image buffer for original image. */
   MbufLoad(IMAGE_FILE, MilSubImage0);

   /* Allocate a structuring element. */
   MbufAlloc2d(MilSystem, STRUCT_ELEM_WIDTH, STRUCT_ELEM_HEIGHT,
               STRUCT_ELEM_DEPTH + M_SIGNED, M_STRUCT_ELEMENT, &StructElem);

   /* Load buffer with data. */
   MbufPut2d(StructElem, 0L, 0L, STRUCT_ELEM_WIDTH, STRUCT_ELEM_HEIGHT, StructArray);

   /* Pause to show the original image. */
   printf("\nThis program does the opening of an image using two different\n");
   printf("structuring elements.\nPress <Enter> to continue.\n\n");
   getch();

   /* Smooth the image to remove noise. */
   MimConvolve(MilSubImage0, MilSubImage0, M_SMOOTH);

   /* Binarize the image so that particles are represented in white and
    * the background in black placing result in subimage1 on the display.
    */
   MimBinarize(MilSubImage0, MilSubImage1, M_LESS_OR_EQUAL, IMAGE_THRESHOLD_VALUE, M_NULL);

   /* Copy the binarized image to a binary buffer for fast processing */
   MbufCopy(MilSubImage1, BinImage);

   /* Opening using common method, placing result in subimage2 on the display. */
   MimOpen(BinImage, MilSubImage2, SMALL_PARTICLE_RADIUS, M_BINARY);

   /* Opening (Erode and Dilate) using customized method, placing
    * result in subimage3 on the display.
    */
   MimMorphic(BinImage, BinImage, StructElem, M_ERODE, SMALL_PARTICLE_RADIUS/2, M_BINARY);
   MimMorphic(BinImage, MilSubImage3, StructElem, M_DILATE,  SMALL_PARTICLE_RADIUS/2, M_BINARY);

   /* Pause to show the opened particle(s). */
   printf("The top right image is the binarized source image. When \n");
   printf("opening it using the standard method, the bottom left image \n");
   printf("results, whereas when opening it using the customized method,\n");
   printf("the bottom right image results and best preserves the original\n");
   printf("shape of the objects.\nPress <Enter> to end.\n\n");
   getch();

   /* Free structuring element buffer. */
   MbufFree(StructElem);

   /* Free the allocated buffers. */
   MbufFree(MilSubImage0);
   MbufFree(MilSubImage1);
   MbufFree(MilSubImage2);
   MbufFree(MilSubImage3);
   MbufFree(BinImage);

   /* Release defaults. */
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, MilImage);
}


Example: mimperimeter.c

/*******************************************************************************/
/*
 * File name: MImPerimeter.c
 *
 * Synopsis:  This program finds the exoskeletons of the  perimeters of
 *            dark objects in an image.
 */
#include <mil.h>
#include <conio.h>

/* Source MIL image file specifications. */
#define IMAGE_FILE             M_IMAGE_PATH MIL_TEXT("cell.mim")
#define IMAGE_WIDTH            512L
#define IMAGE_HEIGHT           480L
#define IMAGE_THRESHOLD_VALUE  128L

/* Small particle radius (in pixels). */
#define SMALL_PARTICLE_RADIUS    2L

main(void)
{
   MIL_ID MilApplication,  /* Application identifier.  */
          MilSystem,       /* System identifier.       */
          MilDisplay,      /* Display identifier.      */
          MilImage,        /* Image buffer identifier. */
          BinImage,        /* Binary image buffer identifier. */
          DilBinImage;     /* Dilated binary image buffer identifier. */

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay, M_NULL, M_NULL);

   /* Restore source image into an image buffer and displays it. */
   MbufRestore(IMAGE_FILE, MilSystem, &MilImage);
   MdispSelect(MilDisplay, MilImage);

   /* Allocate 2 binary image buffers for fast processing. */
   MbufAlloc2d(M_DEFAULT, IMAGE_WIDTH, IMAGE_HEIGHT, 1+M_UNSIGNED, M_IMAGE+M_PROC, &BinImage);
   MbufAlloc2d(M_DEFAULT, IMAGE_WIDTH, IMAGE_HEIGHT, 1+M_UNSIGNED, M_IMAGE+M_PROC, &DilBinImage);

   /* Pause to show the original image. */
   printf("\nThis program finds the exoskeletons of the particles in the displayed image.\n");
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Binarize the image. */
   MimBinarize(MilImage, BinImage, M_LESS_OR_EQUAL, IMAGE_THRESHOLD_VALUE, M_NULL);

   /* Remove small particles. */
   MimOpen(BinImage, BinImage, SMALL_PARTICLE_RADIUS, M_BINARY);

   /* Dilate image (adds one pixel around all objects). */
   MimDilate(BinImage, DilBinImage, 1L, M_BINARY);

   /* XOR the dilated image with the original image. */
   MimArith(BinImage, DilBinImage, BinImage, M_XOR);

   /* Convert the binary image to a visible grayscale image (0-0xFF). */
   MimBinarize(BinImage, MilImage, M_GREATER, 0, M_NULL);

   /* Pause to show the resulting image. */
   printf("Exoskeletons of the object's perimeters are being calculated.\n");
   printf("Press <Enter> to end.\n");
   getch();

   /* Free all allocations. */
   MbufFree(BinImage);
   MbufFree(DilBinImage);
   MbufFree(MilImage);
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
}


Example: mimprocessing.c

/*****************************************************************************/
/*
 * File name: MImProcessing.c
 *
 * Synopsis:  This program uses different image processing primitives to
 *            determine the number of cell nuclei that are larger than
 *            a certain size in an image of a tissue sample.
 */
#include <mil.h>
#include <conio.h>

/* Target MIL image file specifications. */
#define IMAGE_FILE                     M_IMAGE_PATH MIL_TEXT("Cell.mim")
#define IMAGE_THRESHOLD_VALUE          128L
#define IMAGE_SMALL_PARTICLE_RADIUS    2L

void main(void)
{
   MIL_ID MilApplication,   /* Application identifier.  */
          MilSystem,        /* System identifier.       */
          MilDisplay,       /* Display identifier.      */
          MilImage,         /* Image buffer identifier. */
          MilExtremeResult; /* Extreme result buffer identifier. */
   long   MaxLabelNumber;   /* Highest label value. */

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem,
                             &MilDisplay, M_NULL, M_NULL);

   /* Restore source image and display it. */
   MbufRestore(IMAGE_FILE, MilSystem, &MilImage);
   MdispSelect(MilDisplay, MilImage);

   /* Pause to show the original image. */
   printf("\nIMAGE PROCESSING:\n");
   printf("-----------------\n\n");
   printf("This program counts the number of large particles in the image.\n");
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Smooth the image to remove noise. */
   MimConvolve(MilImage, MilImage, M_SMOOTH);

   /* Binarize the image so that particles are represented in white and
      the background in black.
    */
   MimBinarize(MilImage, MilImage, M_LESS_OR_EQUAL, IMAGE_THRESHOLD_VALUE, M_NULL);

   /* Remove small particles. */
   MimOpen(MilImage, MilImage, IMAGE_SMALL_PARTICLE_RADIUS, M_BINARY);

   /* Pause to show the remaining particle(s). */
   printf("These particles have been extracted from the original image.\n");
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Label the image. */
   MimLabel(MilImage, MilImage, M_DEFAULT);

   /* The largest label value corresponds to the number of particles in the image. */
   MimAllocResult(MilSystem, 1L, M_EXTREME_LIST, &MilExtremeResult);
   MimFindExtreme(MilImage, MilExtremeResult, M_MAX_VALUE);
   MimGetResult(MilExtremeResult, M_VALUE, &MaxLabelNumber);

   /* Multiply the labeling result to augment the gray level of the particles. */
   MimArith(MilImage, 256L/MaxLabelNumber, MilImage, M_MULT_CONST);

   /* Display the result in pseudo-color. */
   MdispLut(MilDisplay, M_PSEUDO);

   /* Print results. */
   printf("There were %ld large particles in the original image.\n", MaxLabelNumber);
   printf("Press <Enter> to end.\n\n");
   getch();

   /* Free all allocations. */
   MimFree(MilExtremeResult);
   MbufFree(MilImage);
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
}


Example: mmeas.c

/*****************************************************************************/
/*
 * File name: MMeas.c
 *
 * Synopsis:  This program consist of 2 examples that use the Measurement module
 *            to calculate the position, width and angle of objects in an image.
 *            The first one measures the position, width and angle of a stripe
 *            in an image, and marks its center and edges. The second one measures
 *            the average position, width and angle of a row of pins on a chip.
 */
#include <conio.h>
#include <mil.h>

/* Example selection. */
#define RUN_SINGLE_MEASUREMENT_EXAMPLE      M_YES
#define RUN_MULTIPLE_MEASUREMENT_EXAMPLE    M_YES

/* Example functions declarations. */
void SingleMeasurementExample(MIL_ID MilSystem, MIL_ID MilDisplay);
void MultipleMeasurementExample(MIL_ID MilSystem, MIL_ID MilDisplay);

/*****************************************************************************
 Main.
 *****************************************************************************/
void main(void)
   {
   MIL_ID MilApplication,     /* Application identifier. */
          MilSystem,          /* System Identifier.      */
          MilDisplay;         /* Display identifier.     */

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay, M_NULL, M_NULL);

   /* Print module name. */
   printf("\nMEASUREMENT MODULE:\n");
   printf("-------------------\n\n");

   #if (RUN_SINGLE_MEASUREMENT_EXAMPLE)
   SingleMeasurementExample(MilSystem, MilDisplay);
   #endif

   #if (RUN_MULTIPLE_MEASUREMENT_EXAMPLE)
   MultipleMeasurementExample(MilSystem, MilDisplay);
   #endif

   /* Free defaults. */
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
   }

/*****************************************************************************
 Single measurement example.
 *****************************************************************************/

/* Source MIL image file specification. */
#define MEAS_IMAGE_FILE        M_IMAGE_PATH MIL_TEXT("Lead.mim")

/* Processing region specification. */
#define MEAS_BOX_WIDTH         128
#define MEAS_BOX_HEIGHT        100
#define MEAS_BOX_POS_X         166
#define MEAS_BOX_POS_Y         130

/* Target stripe typical specifications. */
#define STRIPE_POLARITY_LEFT   M_POSITIVE
#define STRIPE_POLARITY_RIGHT  M_NEGATIVE
#define STRIPE_WIDTH           45L
#define STRIPE_WIDTH_VARIATION 10L

void SingleMeasurementExample(MIL_ID MilSystem, MIL_ID MilDisplay)
   {
   MIL_ID MilImage,                     /* Image buffer identifier. */
          MilOverlayImage,              /* Overlay image.           */
          StripeMarker;                 /* Stripe marker identifier.*/
   double StripeCenterX,                /* Stripe X center position.*/
          StripeCenterY,                /* Stripe Y center position.*/
          StripeWidth,                  /* Stripe width.            */
          StripeAngle;                  /* Stripe angle.            */
   long   CrossColor = M_COLOR_YELLOW,  /* Cross drawing color.        */
          BoxColor   = M_COLOR_RED;     /* Box drawing color.       */

   /* Restore and display the source image. */
   MbufRestore(MEAS_IMAGE_FILE, MilSystem, &MilImage);
   MdispSelect(MilDisplay, MilImage);

   /* Prepare for overlay annotation. */
   MdispControl(MilDisplay, M_OVERLAY, M_ENABLE);
   MdispInquire(MilDisplay, M_OVERLAY_ID, &MilOverlayImage);

   /* Allocate a stripe marker. */
   MmeasAllocMarker(MilSystem, M_STRIPE, M_DEFAULT, &StripeMarker);

   /* Specify the stripe characteristics. */
   MmeasSetMarker(StripeMarker, M_POLARITY, STRIPE_POLARITY_LEFT, STRIPE_POLARITY_RIGHT);
   MmeasSetMarker(StripeMarker, M_WIDTH, STRIPE_WIDTH, M_NULL);
   MmeasSetMarker(StripeMarker, M_WIDTH_VARIATION, STRIPE_WIDTH_VARIATION, M_NULL);
   MmeasSetMarker(StripeMarker, M_BOX_ANGLE_MODE, M_ENABLE, M_NULL);

   /* Specify the search box size and position. */
   MmeasSetMarker(StripeMarker, M_BOX_ORIGIN, MEAS_BOX_POS_X, MEAS_BOX_POS_Y);
   MmeasSetMarker(StripeMarker, M_BOX_SIZE,   MEAS_BOX_WIDTH, MEAS_BOX_HEIGHT);

   /* Draw the contour of the measurement box. */
   MgraColor(M_DEFAULT, BoxColor);
   MmeasDraw(M_DEFAULT, StripeMarker, MilOverlayImage, M_DRAW_BOX, M_DEFAULT, M_MARKER);

   /* Pause to show the original image. */
   printf("Position, width and angle of the stripe in the highlighted box\n");
   printf("will be calculated and the center and edges will be marked.\n");
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Clear the overlay image. */
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);

   /* Find the stripe and measure its width and angle. */
   MmeasFindMarker(M_DEFAULT, MilImage , StripeMarker, M_DEFAULT);

   /* Get the stripe position, width and angle. */
   MmeasGetResult(StripeMarker, M_POSITION, &StripeCenterX, &StripeCenterY);
   MmeasGetResult(StripeMarker, M_WIDTH,    &StripeWidth, M_NULL);
   MmeasGetResult(StripeMarker, M_ANGLE,    &StripeAngle, M_NULL);

   /* Draw the contour of the found measurement box. */
   MgraColor(M_DEFAULT, BoxColor);
   MmeasDraw(M_DEFAULT, StripeMarker, MilOverlayImage, M_DRAW_BOX, M_DEFAULT, M_RESULT);

   /* Draw a cross on the center, left edge and right edge of the found stripe. */
   MgraColor(M_DEFAULT, CrossColor);
   MmeasDraw(M_DEFAULT, StripeMarker, MilOverlayImage, M_DRAW_POSITION, M_DEFAULT, M_RESULT);
   MmeasDraw(M_DEFAULT, StripeMarker, MilOverlayImage,
             M_DRAW_POSITION+M_EDGE_FIRST+M_EDGE_SECOND, M_DEFAULT, M_RESULT);

   /* Print the result. */
   printf("The stripe in the image is at position %.2f,%.2f and\n",
                                         StripeCenterX, StripeCenterY);
   printf("is %.2f pixels wide with an angle of %.2f degrees.\n",
                                         StripeWidth, StripeAngle);
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Clear the overlay image. */
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);

   /* Free all allocations. */
   MmeasFree(StripeMarker);
   MbufFree(MilImage);
   }

/*****************************************************************************
 Multiple measurement example.
 *****************************************************************************/

/* Source MIL image file specification. */
#define MULT_MEAS_IMAGE_FILE     M_IMAGE_PATH MIL_TEXT("Chip.mim")

/* Processing region specification. */
#define MULT_MEAS_BOX_WIDTH      230
#define MULT_MEAS_BOX_HEIGHT     7
#define MULT_MEAS_BOX_POS_X      220
#define MULT_MEAS_BOX_POS_Y      171

/* Target stripe specifications. */
#define MULT_STRIPES_ORIENTATION      M_VERTICAL
#define MULT_STRIPES_POLARITY_LEFT    M_POSITIVE
#define MULT_STRIPES_POLARITY_RIGHT   M_NEGATIVE
#define MULT_STRIPES_NUMBER           12

void MultipleMeasurementExample(MIL_ID MilSystem, MIL_ID MilDisplay)
{
   MIL_ID MilImage,                    /* Image buffer identifier.  */
          MilOverlayImage,             /* Overlay image.            */
          StripeMarker;                /* Stripe marker identifier. */
   double MeanAngle,                   /* Stripe mean angle.        */
          MeanWidth,                   /* Stripe mean width.        */
          MeanSpacing;                 /* Stripe mean spacing.      */
   long   CrossColor = M_COLOR_YELLOW, /* Cross drawing color.      */
          BoxColor   = M_COLOR_RED;    /* Box drawing color.        */

   /* Restore and display the source image. */
   MbufRestore(MULT_MEAS_IMAGE_FILE, MilSystem, &MilImage);
   MdispSelect(MilDisplay, MilImage);

   /* Prepare for overlay annotation. */
   MdispControl(MilDisplay, M_OVERLAY, M_ENABLE);
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);
   MdispInquire(MilDisplay, M_OVERLAY_ID, &MilOverlayImage);

   /* Allocate a stripe marker. */
   MmeasAllocMarker(MilSystem, M_STRIPE, M_DEFAULT, &StripeMarker);

   /* Specify the stripe characteristics. */
   MmeasSetMarker(StripeMarker, M_NUMBER, MULT_STRIPES_NUMBER, M_NULL);
   MmeasSetMarker(StripeMarker, M_POLARITY, MULT_STRIPES_POLARITY_LEFT,
                                            MULT_STRIPES_POLARITY_RIGHT);
   MmeasSetMarker(StripeMarker, M_ORIENTATION, MULT_STRIPES_ORIENTATION, M_NULL);

   /* Specify the measurement box size and position. */
   MmeasSetMarker(StripeMarker, M_BOX_ORIGIN, MULT_MEAS_BOX_POS_X, MULT_MEAS_BOX_POS_Y);
   MmeasSetMarker(StripeMarker, M_BOX_SIZE,   MULT_MEAS_BOX_WIDTH, MULT_MEAS_BOX_HEIGHT);

   /* Draw the contour of the measurement box. */
   MgraColor(M_DEFAULT, BoxColor);
   MmeasDraw(M_DEFAULT, StripeMarker, MilOverlayImage, M_DRAW_BOX, M_DEFAULT, M_MARKER);

   /* Pause to show the original image. */
   printf("The position and angle of a row of pins on a chip will be calculated.\n");
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Find the stripe and measure its width and angle. */
   MmeasFindMarker(M_DEFAULT, MilImage, StripeMarker, M_POSITION + M_ANGLE + M_WIDTH);

   /* Draw the contour of the measurement box. */
   MgraColor(M_DEFAULT, BoxColor);
   MmeasDraw(M_DEFAULT, StripeMarker, MilOverlayImage, M_DRAW_BOX, M_DEFAULT, M_RESULT);

   /* Draw a cross at the center of each stripe found. */
   MgraColor(M_DEFAULT, CrossColor);
   MmeasDraw(M_DEFAULT, StripeMarker, MilOverlayImage, M_DRAW_POSITION, M_ALL, M_RESULT);

   /* Get the stripe's width, angle and spacing. */
   MmeasGetResult(StripeMarker, M_ANGLE +   M_MEAN, &MeanAngle,   M_NULL);
   MmeasGetResult(StripeMarker, M_WIDTH +   M_MEAN, &MeanWidth,   M_NULL);
   MmeasGetResult(StripeMarker, M_SPACING + M_MEAN, &MeanSpacing, M_NULL);

   /* Print the results. */
   printf("The center and angle of each pin have been marked.\n\n");
   printf("The statistics for the pins are:\n");
   printf("Average angle   : %5.2f\n", MeanAngle);
   printf("Average width   : %5.2f\n", MeanWidth);
   printf("Average spacing : %5.2f\n\n", MeanSpacing);
   printf("Press <Enter> to end.\n");
   getch();

   /* Clear the overlay image. */
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);

   /* Free all allocations. */
   MmeasFree(StripeMarker);
   MbufFree(MilImage);


Example: mmod.c

/*************************************************************************************/
/*
 * File name: MMod.c
 *
 * Synopsis:  This program uses the Geometric Model Finder module to define geometric
 *            models and searches for these models in a target image. A simple single model
 *            example (1 model, 1 occurrence, good search conditions) is presented first,
 *            followed by a more complex example (multiple models, multiple occurrences in a
 *            complex scene with bad search conditions).
 */
#include <mil.h>
#include <conio.h>

/* Example functions declarations. */
void SingleModelExample(MIL_ID MilSystem, MIL_ID MilDisplay);
void MultipleModelsExample(MIL_ID MilSystem, MIL_ID MilDisplay);

/*****************************************************************************/
/* Main.
******************************************************************************/
void main(void)
   {
   MIL_ID MilApplication,     /* Application identifier. */
          MilSystem,          /* System Identifier.      */
          MilDisplay;         /* Display identifier.     */

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay, M_NULL, M_NULL);

   /* Run single model example. */
   SingleModelExample(MilSystem, MilDisplay);

   /* Run multiple model example. */
   MultipleModelsExample(MilSystem, MilDisplay);

   /* Free defaults. */
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
   }

/*****************************************************************************/
/* Single model example. */

/* Source MIL image file specifications. */
#define SINGLE_MODEL_IMAGE          M_IMAGE_PATH MIL_TEXT("SingleModel.mim")

/* Target MIL image file specifications. */
#define SINGLE_MODEL_TARGET_IMAGE   M_IMAGE_PATH MIL_TEXT("SingleTarget.mim")

/* Search speed: M_VERY_HIGH for faster search, M_MEDIUM for precision and robustness. */
#define SINGLE_MODEL_SEARCH_SPEED   M_VERY_HIGH

/* Model specifications. */
#define MODEL_OFFSETX               176L
#define MODEL_OFFSETY               136L
#define MODEL_SIZEX                 128L
#define MODEL_SIZEY                 128L
#define MODEL_MAX_OCCURRENCES       16L

void SingleModelExample(MIL_ID MilSystem, MIL_ID MilDisplay)
   {
   MIL_ID MilImage,                         /* Image buffer identifier.*/
          MilOverlayImage;                  /* Overlay image.          */
   MIL_ID MilSearchContext,                 /* Search context          */
          MilResult;                        /* Result identifier.      */
   long   Model[MODEL_MAX_OCCURRENCES],     /* Model index.            */
          ModelDrawColor = M_COLOR_RED;     /* Model draw color.       */
   long   NumResults  = 0L;                 /* Number of results found.*/
   double Score[MODEL_MAX_OCCURRENCES],     /* Model correlation score.*/
          XPosition[MODEL_MAX_OCCURRENCES], /* Model X position.       */
          YPosition[MODEL_MAX_OCCURRENCES], /* Model Y position.       */
          Angle[MODEL_MAX_OCCURRENCES],     /* Model occurrence angle. */
          Scale[MODEL_MAX_OCCURRENCES],     /* Model occurrence scale. */
          Time = 0.0;                       /* Bench variable.         */
   int    i;                                /* Loop variable.          */

   /* Restore the model image and display it */
   MbufRestore(SINGLE_MODEL_IMAGE, MilSystem, &MilImage);
   MdispSelect(MilDisplay, MilImage);

   /* Prepare for overlay annotation. */
   MdispControl(MilDisplay, M_OVERLAY, M_ENABLE);
   MdispInquire(MilDisplay, M_OVERLAY_ID, &MilOverlayImage);

   /* Allocate a Geometric Model Finder context. */
   MmodAlloc(MilSystem, M_GEOMETRIC, M_DEFAULT, &MilSearchContext);

   /* Allocate a result buffer. */
   MmodAllocResult(MilSystem, M_DEFAULT, &MilResult);

   /* Define the model. */
   MmodDefine(MilSearchContext, M_IMAGE, MilImage,
              MODEL_OFFSETX, MODEL_OFFSETY, MODEL_SIZEX, MODEL_SIZEY);

   /* Set the search speed. */
   MmodControl(MilSearchContext, M_CONTEXT, M_SPEED, SINGLE_MODEL_SEARCH_SPEED);

   /* Preprocess the search context. */
   MmodPreprocess(MilSearchContext, M_DEFAULT);

   /* Draw box and position it in the source image to show the model. */
   MgraColor(M_DEFAULT, ModelDrawColor);
   MmodDraw(M_DEFAULT, MilSearchContext, MilOverlayImage,
            M_DRAW_BOX+M_DRAW_POSITION, 0, M_ORIGINAL);

   /* Pause to show the model. */
   printf("\nGEOMETRIC MODEL FINDER:\n");
   printf("-----------------------\n\n");
   printf("A model context was defined with the model in the displayed image.\n");
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Clear the overlay image. */
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);

   /* Load the single model target image. */
   MbufLoad(SINGLE_MODEL_TARGET_IMAGE, MilImage);

   /* Dummy first find operation for better function timing accuracy (model cache effet,...). */
   MmodFind(MilSearchContext, MilImage, MilResult);

   /* Reset the timer. */
   MappTimer(M_TIMER_RESET+M_SYNCHRONOUS, M_NULL);

   /* Find the model. */
   MmodFind(MilSearchContext, MilImage, MilResult);

   /* Read the find time. */
   MappTimer(M_TIMER_READ+M_SYNCHRONOUS, &Time);

   /* Get the number of models found. */
   MmodGetResult(MilResult, M_DEFAULT, M_NUMBER+M_TYPE_LONG, &NumResults);

   /* If a model was found above the acceptance threshold. */
   if ( (NumResults >= 1) && (NumResults <= MODEL_MAX_OCCURRENCES) )
      {
      /* Get the results of the single model. */
      MmodGetResult(MilResult, M_DEFAULT, M_INDEX+M_TYPE_LONG, Model);
      MmodGetResult(MilResult, M_DEFAULT, M_POSITION_X, XPosition);
      MmodGetResult(MilResult, M_DEFAULT, M_POSITION_Y, YPosition);
      MmodGetResult(MilResult, M_DEFAULT, M_ANGLE, Angle);
      MmodGetResult(MilResult, M_DEFAULT, M_SCALE, Scale);
      MmodGetResult(MilResult, M_DEFAULT, M_SCORE, Score);

      /* Print the results for each model found. */
      printf("The model was found in the target image:\n\n");
      printf("Result   Model   X Position   Y Position   Angle   Scale   Score\n\n");
      for (i=0; i<NumResults; i++)
         {
         printf("%-9d%-8d%-13.2f%-13.2f%-8.2f%-8.2f%-5.2f%%\n",
               i, Model[i], XPosition[i], YPosition[i], Angle[i], Scale[i], Score[i]);
         }
      printf("\nThe search time is %.1f ms\n\n", Time*1000.0);

      /* Draw edges, position and box over the occurrences that were found. */
      for (i=0; i<NumResults; i++)
         {
         MgraColor(M_DEFAULT, ModelDrawColor);
         MmodDraw(M_DEFAULT,  MilResult, MilOverlayImage,
                  M_DRAW_EDGES+M_DRAW_BOX+M_DRAW_POSITION, i, M_DEFAULT);
         }
      }
   else
      {
      printf("The model was not found or the number of models found is greater than\n");
      printf("the specified maximum number of occurrence !\n\n");
      }

   /* Wait for a key to be pressed. */
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Free MIL objects. */
   MbufFree(MilImage);
   MmodFree(MilSearchContext);
   MmodFree(MilResult);
   }

/*****************************************************************************
/* Multiple models example. */

/* Source MIL image file specifications. */
#define MULTI_MODELS_IMAGE          M_IMAGE_PATH MIL_TEXT("MultipleModel.mim")

/* Target MIL image file specifications. */
#define MULTI_MODELS_TARGET_IMAGE   M_IMAGE_PATH MIL_TEXT("MultipleTarget.mim")

/* Search speed:  M_VERY_HIGH for faster search, M_MEDIUM for precision and robustness. */
#define MULTI_MODELS_SEARCH_SPEED   M_VERY_HIGH

/* Number of models. */
#define NUMBER_OF_MODELS            3L
#define MODELS_MAX_OCCURRENCES     16L

/* Model 1 specifications. */
#define MODEL0_OFFSETX              34L
#define MODEL0_OFFSETY              93L
#define MODEL0_SIZEX                214L
#define MODEL0_SIZEY                76L
#define MODEL0_DRAW_COLOR           M_COLOR_RED

/* Model 2 specifications. */
#define MODEL1_OFFSETX              73L
#define MODEL1_OFFSETY              232L
#define MODEL1_SIZEX                150L
#define MODEL1_SIZEY                154L
#define MODEL1_REFERENCEX           23L
#define MODEL1_REFERENCEY           127L
#define MODEL1_DRAW_COLOR           M_COLOR_GREEN

/* Model 3 specifications. */
#define MODEL2_OFFSETX              308L
#define MODEL2_OFFSETY              39L
#define MODEL2_SIZEX                175L
#define MODEL2_SIZEY                357L
#define MODEL2_REFERENCEX           62L
#define MODEL2_REFERENCEY           150L
#define MODEL2_DRAW_COLOR           M_COLOR_BLUE

/* Models array specifications. */
#define MODELS_ARRAY_SIZE           3L
#define MODELS_OFFSETX              {MODEL0_OFFSETX, MODEL1_OFFSETX, MODEL2_OFFSETX}
#define MODELS_OFFSETY              {MODEL0_OFFSETY, MODEL1_OFFSETY, MODEL2_OFFSETY}
#define MODELS_SIZEX                {MODEL0_SIZEX, MODEL1_SIZEX, MODEL2_SIZEX}
#define MODELS_SIZEY                {MODEL0_SIZEY, MODEL1_SIZEY, MODEL2_SIZEY}
#define MODELS_DRAW_COLOR           {MODEL0_DRAW_COLOR, MODEL1_DRAW_COLOR, MODEL2_DRAW_COLOR}

void MultipleModelsExample(MIL_ID MilSystem, MIL_ID MilDisplay)
   {
   MIL_ID MilImage,                                             /* Image buffer identifier. */
          MilOverlayImage;                                      /* Overlay image.           */
   MIL_ID MilSearchContext,                                     /* Search context           */
          MilResult;                                            /* Result identifier.       */
   long   Models[MODELS_MAX_OCCURRENCES],                       /* Model indices.           */
          ModelsOffsetX[MODELS_ARRAY_SIZE] = MODELS_OFFSETX,    /* Model X offsets array.   */
          ModelsOffsetY[MODELS_ARRAY_SIZE] = MODELS_OFFSETY,    /* Model Y offsets array.   */
          ModelsSizeX[MODELS_ARRAY_SIZE]   = MODELS_SIZEX,      /* Model X sizes array.     */
          ModelsSizeY[MODELS_ARRAY_SIZE]   = MODELS_SIZEY;      /* Model Y sizes array.     */
   long   ModelsDrawColor[MODELS_ARRAY_SIZE]=MODELS_DRAW_COLOR, /* Model drawing colors.    */
          NumResults  = 0L;                                     /* Number of results found. */
   double Score[MODELS_MAX_OCCURRENCES],                        /* Model correlation scores.*/
          XPosition[MODELS_MAX_OCCURRENCES],                    /* Model X positions.       */
          YPosition[MODELS_MAX_OCCURRENCES],                    /* Model Y positions.       */
          Angle[MODELS_MAX_OCCURRENCES],                        /* Model occurrence angles. */
          Scale[MODELS_MAX_OCCURRENCES],                        /* Model occurrence scales. */
          Time = 0.0;                                           /* Time variable.           */
   int    i;                                                    /* Loop variable            */

   /* Restore the model image and display it. */
   MbufRestore(MULTI_MODELS_IMAGE, MilSystem, &MilImage);
   MdispSelect(MilDisplay, MilImage);

   /* Prepare for overlay annotation. */
   MdispControl(MilDisplay, M_OVERLAY, M_ENABLE);
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);
   MdispInquire(MilDisplay, M_OVERLAY_ID, &MilOverlayImage);

   /* Allocate a geometric model finder. */
   MmodAlloc(MilSystem, M_GEOMETRIC, M_DEFAULT, &MilSearchContext);

   /* Allocate a result buffer. */
   MmodAllocResult(MilSystem, M_DEFAULT, &MilResult);

   /* Define the models. */
   for (i=0; i<NUMBER_OF_MODELS; i++)
      {
      MmodDefine(MilSearchContext, M_IMAGE, MilImage,
                 ModelsOffsetX[i], ModelsOffsetY[i], ModelsSizeX[i], ModelsSizeY[i]);
      }

   /* Set the desired search speed. */
   MmodControl(MilSearchContext, M_CONTEXT, M_SPEED, MULTI_MODELS_SEARCH_SPEED);

   /* Increase the smoothness for the edge extraction in the search context. */
   MmodControl(MilSearchContext, M_CONTEXT, M_SMOOTHNESS, 75);

   /* Modify the acceptance and the certainty for all the models that were defined. */
   MmodControl(MilSearchContext, M_DEFAULT, M_ACCEPTANCE, 40);
   MmodControl(MilSearchContext, M_DEFAULT, M_CERTAINTY,  60);

   /* Set the number of occurrences to 2 for all the models that were defined. */
   MmodControl(MilSearchContext, M_DEFAULT, M_NUMBER, 2);

   #if (NUMBER_OF_MODELS>1)
   /* Change the reference point of the second model. */
   MmodControl(MilSearchContext, 1, M_REFERENCE_X, MODEL1_REFERENCEX);
   MmodControl(MilSearchContext, 1, M_REFERENCE_Y, MODEL1_REFERENCEY);

   #if (NUMBER_OF_MODELS>2)
   /* Change the reference point of the third model. */
   MmodControl(MilSearchContext, 2, M_REFERENCE_X, MODEL2_REFERENCEX);
   MmodControl(MilSearchContext, 2, M_REFERENCE_Y, MODEL2_REFERENCEY);
   #endif
   #endif

   /* Preprocess the search context. */
   MmodPreprocess(MilSearchContext, M_DEFAULT);

   /* Draw boxes and positions in the source image to identify the models. */
   for (i=0; i<NUMBER_OF_MODELS; i++)
      {
      MgraColor(M_DEFAULT, ModelsDrawColor[i]);
      MmodDraw( M_DEFAULT, MilSearchContext, MilOverlayImage,
                M_DRAW_BOX+M_DRAW_POSITION, i, M_ORIGINAL);
      }

   /* Pause to show the models. */
   printf("A model context was defined with the models in the displayed image.\n");
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Clear the overlay image. */
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);

   /* Load the complex target image. */
   MbufLoad(MULTI_MODELS_TARGET_IMAGE, MilImage);

   /* Dummy first find operation for better function timing accuracy (model cache effet,...). */
   MmodFind(MilSearchContext, MilImage, MilResult);

   /* Reset the timer. */
   MappTimer(M_TIMER_RESET+M_SYNCHRONOUS, M_NULL);

   /* Find the models. */
   MmodFind(MilSearchContext, MilImage, MilResult);

   /* Read the find time. */
   MappTimer(M_TIMER_READ+M_SYNCHRONOUS, &Time);

   /* Get the number of models found. */
   MmodGetResult(MilResult, M_DEFAULT, M_NUMBER+M_TYPE_LONG, &NumResults);

   /* If the models were found above the acceptance threshold. */
   if( (NumResults >= 1) && (NumResults <= MODELS_MAX_OCCURRENCES) )
      {
      /* Get the results for each model. */
      MmodGetResult(MilResult, M_DEFAULT, M_INDEX+M_TYPE_LONG, Models);
      MmodGetResult(MilResult, M_DEFAULT, M_POSITION_X, XPosition);
      MmodGetResult(MilResult, M_DEFAULT, M_POSITION_Y, YPosition);
      MmodGetResult(MilResult, M_DEFAULT, M_ANGLE, Angle);
      MmodGetResult(MilResult, M_DEFAULT, M_SCALE, Scale);
      MmodGetResult(MilResult, M_DEFAULT, M_SCORE, Score);

      /* Print information about the target image. */
      printf("The models were found in the target image although their is:\n");
      printf("   Full rotation\n   Small scale change\n   Contrast variation\n");
      printf("   Specular reflection\n   Occlusion\n   Multiple models\n");
      printf("   Multiple occurrences\n\n");

      /* Print the results for the found models. */
      printf("Result   Model   X Position   Y Position   Angle   Scale   Score\n\n");
      for (i=0; i<NumResults; i++)
         {
         printf("%-9d%-8d%-13.2f%-13.2f%-8.2f%-8.2f%-5.2f%%\n",
               i, Models[i], XPosition[i], YPosition[i], Angle[i], Scale[i], Score[i]);
         }
      printf("\nThe search time is %.1f ms\n\n", Time*1000.0);

      /* Draw edges and positions over the occurrences that were found. */
      for (i=0; i < NumResults; i++)
         {
         MgraColor(M_DEFAULT, ModelsDrawColor[Models[i]]);
         MmodDraw(M_DEFAULT, MilResult, MilOverlayImage,
                  M_DRAW_EDGES+M_DRAW_POSITION, i, M_DEFAULT);
         }
      }
   else
      {
      printf("The models were not found or the number of models found is greater than\n");
      printf("the defined value of maximum occurrences !\n\n");
      }

   /* Wait for a key to be pressed. */
   printf("Press <Enter> to end.\n\n");
   getch();

   /* Free MIL objects. */
   MbufFree(MilImage);
   MmodFree(MilSearchContext);
   MmodFree(MilResult);
   }


Example: mnatgen.c

/* File name: mnatgen.c
 * Synopsis:  This program show how to use GENESIS native library mixed
 *            with MIL calls.
 */

/* general includes */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <mil.h>
#include <imapi.h>

/* Operation control defines */
#define ALLOCATE 1
#define PROCESS  2
#define FREE     3

/* Native functions to grab  and warp an image. */
void GrabAndWarp(MIL_ID MilSystem, MIL_ID MilDisplay, MIL_ID MilCamera,
                                   MIL_ID MilImage, long Operation);

/* Main function: */
/* -------------- */
void main(void)
{
   MIL_ID MilApplication,  /* Application identifier.  */
          MilSystem,       /* System identifier.       */
          MilDisplay,      /* Display identifier.      */
          MilCamera,       /* Camera identifier.       */
          MilImage;        /* Image buffer identifier. */

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem,
                             &MilDisplay, &MilCamera, &MilImage);
   /* Make sure that the system allocated is a Genesis */
   if ((MsysInquire(MilSystem, M_SYSTEM_TYPE, M_NULL)!= M_SYSTEM_GENESIS_TYPE) ||
       (MsysInquire(MilSystem, M_BOARD_TYPE, M_NULL)!= M_GENESIS))
      {
      printf("This example only runs on a Matrox Genesis main board system.\n");
      printf("Press <Enter> to end...");
      getchar();
      goto end;
      }

   /* Allocate and initialialize work buffers */
   GrabAndWarp(MilSystem, MilDisplay, MilCamera, MilImage, ALLOCATE);

   /* Print a message on the host screen. */
   printf("Native function called in a loop...\n");
   printf("Press <Enter> to end...");

   /* Grab and warp grabbed image in a loop */
   while (!kbhit())
      {
      GrabAndWarp(MilSystem, MilDisplay, MilCamera, MilImage, PROCESS);
      }
   getchar();

   /* Free work buffers */
   GrabAndWarp(MilSystem, MilDisplay, MilCamera, MilImage, FREE);

end:

   MappFreeDefault(MilApplication, MilSystem, MilDisplay, MilCamera, MilImage);
}

/* Native function: */
/* ---------------- */
void GrabAndWarp(MIL_ID MilSystem, MIL_ID MilDisplay, MIL_ID MilCamera,
                                   MIL_ID MilImage, long Operation)
{
   /* Warp coefficients and LUT ID variables.
    * (kept in static to avoid warp coeficients caculation at each call.)
    */
   static long NativeWarpBufId       = M_NULL;
   static long NativeWarpLutXBufId   = M_NULL;
   static long NativeWarpLutYBufId   = M_NULL;
   static long NativeGrabBufId       = M_NULL;
   static long NativeWarpResultBufId = M_NULL;

   /* Inquire useful MIL informations. */
   long SizeX    = MdigInquire(MilCamera, M_SIZE_X   , M_NULL);
   long SizeY    = MdigInquire(MilCamera, M_SIZE_Y   , M_NULL);
   long SizeBand = MdigInquire(MilCamera, M_SIZE_BAND, M_NULL);

   /* Miscelanous local variables */
   double CornerX1  = 0.0;
   double CornerY1  = 0.0;
   double CornerX2  = SizeX - 1.0;
   double CornerY2  = 0.0;
   double CornerX3  = 2.0 * SizeX;
   double CornerY3  = SizeY - 1.0;
   double CornerX4  = -1.0 * SizeX;
   double CornerY4  = SizeY - 1.0;
   long   SrcXStart = 0L;
   long   SrcYStart = 0L;
   long   SrcXEnd   = SizeX - 1L;
   long   SrcYEnd   = SizeY - 1L;

   /* Inquire Genesis native Id's */
   long NativeSysThreadId = MsysInquire(MilSystem, M_NATIVE_THREAD_ID,  M_NULL);
   long NativeDigCameraId = MdigInquire(MilCamera, M_NATIVE_CAMERA_ID , M_NULL);
   long NativeDigControlId= MdigInquire(MilCamera, M_NATIVE_CONTROL_ID, M_NULL);
   long NativeDigId       = MdigInquire(MilCamera, M_NATIVE_ID        , M_NULL);
   long NativeBufId       = MbufInquire(MilImage,  M_NATIVE_ID        , M_NULL);

   /* Notify MIL that we are entering native mode. */
   MsysControl(MilSystem, M_NATIVE_MODE_ENTER, M_NULL);

   /* Do the selected operation.*/
   switch (Operation)
   {
   /* Preallocate grab and warp buffers (done once for speed). */
   case ALLOCATE:
      {
      imBufAlloc(NativeSysThreadId, SizeX, SizeY, SizeBand, IM_UBYTE, IM_PROC,
                                                      &NativeGrabBufId);
      imBufAlloc(NativeSysThreadId, SizeX, SizeY, SizeBand, IM_UBYTE, IM_PROC,
                                                      &NativeWarpResultBufId);
      imBufAlloc(NativeSysThreadId, 3L, 3L, 1L, IM_FLOAT, IM_PROC,
                                                      &NativeWarpBufId);
      imBufAlloc(NativeSysThreadId, SizeX, SizeY, 1L, IM_SHORT, IM_PROC,
                                                      &NativeWarpLutXBufId);
      imBufAlloc(NativeSysThreadId, SizeX, SizeY, 1L, IM_SHORT, IM_PROC,
                                                      &NativeWarpLutYBufId);
      if (NativeGrabBufId     && NativeWarpResultBufId && NativeWarpBufId &&
          NativeWarpLutXBufId && NativeWarpLutYBufId)
         {
         /* Calculate warp coeficients */
         imGenWarp4Corner(NativeSysThreadId, NativeWarpBufId, CornerX1,
                          CornerY1, CornerX2, CornerY2, CornerX3, CornerY3,
                          CornerX4, CornerY4, SrcXStart, SrcYStart, SrcXEnd,
                          SrcYEnd, IM_DEFAULT, 0L);
         imGenWarpLutMatrix(NativeSysThreadId, NativeWarpLutXBufId,
                            NativeWarpLutYBufId, NativeWarpBufId, 0L, 0L);
         }
      else
         {
         printf("Error allocating resources...\n");
         }

      if (SizeBand > 1)
         {
         printf("Error: only monochrome cameras are supported...\n");
         }

      break;
      }

   /* Grab and Warp buffer. */
   case PROCESS:
      {
      /* Process if allocations were sucessful */
      if ((SizeBand == 1)     &&
          NativeGrabBufId     && NativeWarpResultBufId && NativeWarpBufId &&
          NativeWarpLutXBufId && NativeWarpLutYBufId)
         {
         /* Grab the image */
         imDigGrab(NativeSysThreadId, NativeDigId, NativeDigCameraId,
                   NativeGrabBufId, 1L, NativeDigControlId, 0L);

         /* Warp the grabbed image. */
         imIntWarpLut(NativeSysThreadId, NativeGrabBufId, NativeWarpResultBufId,
                      NativeWarpLutXBufId, NativeWarpLutYBufId, 0L, 0L);

         /* Copy the result into the display buffer */
         imBufCopy(NativeSysThreadId, NativeWarpResultBufId, NativeBufId, 0L, 0L);
         }
      break;
      }

   /* Free grab and warp buffers. */
   case FREE:
      {
      if (NativeGrabBufId)
         imBufFree(NativeSysThreadId, NativeGrabBufId);
      if (NativeWarpResultBufId)
         imBufFree(NativeSysThreadId, NativeWarpResultBufId);
      if (NativeWarpBufId)
         imBufFree(NativeSysThreadId, NativeWarpBufId);
      if (NativeWarpLutXBufId)
         imBufFree(NativeSysThreadId, NativeWarpLutXBufId);
      if (NativeWarpLutYBufId)
         imBufFree(NativeSysThreadId, NativeWarpLutYBufId);
      break;
      }
   }

   /* Notify MIL that we leave native mode. */
   MsysControl(MilSystem, M_NATIVE_MODE_LEAVE, M_NULL);

   /* Notify MIL that the buffer was modified. */
   MbufControl(MilImage, M_MODIFIED, M_DEFAULT);
}


Example: mocr.c

/*****************************************************************************/
/*
 * File name: MOcr.c
 *
 * Synopsis:  This program uses the OCR module to read a SEMI font string:
 *            the example calibrates a predefined OCR font (semi font)
 *            and uses it to read a string in the image. The string read
 *            is then printed to the screen and the calibrated font is
 *            saved to disk.
 */
#include <mil.h>
#include <conio.h>

/*****************************************************************************
 OCR SEMI font read example.
 *****************************************************************************/

/* Target image character specifications. */
#define CHAR_IMAGE_FILE        M_IMAGE_PATH MIL_TEXT("OcrSemi1292.mim")
#define CHAR_SIZE_X_MIN        22.0
#define CHAR_SIZE_X_MAX        23.0
#define CHAR_SIZE_X_STEP       0.50
#define CHAR_SIZE_Y_MIN        43.0
#define CHAR_SIZE_Y_MAX        44.0
#define CHAR_SIZE_Y_STEP       0.50

/* Target reading specifications. */
#define READ_REGION_POS_X      30L
#define READ_REGION_POS_Y      40L
#define READ_REGION_WIDTH      420L
#define READ_REGION_HEIGHT     70L
#define READ_SCORE_MIN         50.0

/* Font file names. */
#define FONT_FILE_IN           M_IMAGE_PATH MIL_TEXT("Semi1292.mfo")
#define FONT_FILE_OUT          M_IMAGE_PATH MIL_TEXT("Semi1292Calibrated.mfo")

/* Length of the string to read (null terminated) */
#define STRING_LENGTH          13L
#define STRING_CALIBRATION     MIL_TEXT("M940902-MXD5")

void main(void)
   {
   MIL_ID MilApplication,               /* Application identifier.       */
          MilSystem,                    /* System identifier.            */
          MilDisplay,                   /* Display identifier.           */
          MilImage,                     /* Image buffer identifier.      */
          MilSubImage,                  /* Sub-image buffer identifier.  */
          MilFontSubImage,              /* Font display sub image.       */
          MilOverlayImage,              /* Overlay image.                */
          OcrFont,                      /* OCR font identifier.          */
          OcrResult;                    /* OCR result buffer identifier. */
   MIL_TEXT_CHAR String[STRING_LENGTH]; /* Array of characters to read.  */
   double Score;                        /* Reading score.                */
   long   SizeX, SizeY, Type;           /* Source image dimensions.      */

   printf("\nOCR MODULE (SEMI font reading):\n");
   printf("-------------------------------\n\n");

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay, M_NULL, M_NULL);

   /* Load and display the source image into a new image buffer. */
   MbufAlloc2d(MilSystem,
               MbufDiskInquire(CHAR_IMAGE_FILE, M_SIZE_X, &SizeX),
               MbufDiskInquire(CHAR_IMAGE_FILE, M_SIZE_Y, &SizeY)*3/2,
               MbufDiskInquire(CHAR_IMAGE_FILE, M_TYPE,   &Type),
               M_IMAGE+M_PROC+M_DISP,
               &MilImage);
   MbufClear(MilImage, 0);
   MbufLoad(CHAR_IMAGE_FILE, MilImage);
   MdispSelect(MilDisplay, MilImage);

   /* Restrict the region of the image where to read the string. */
   MbufChild2d(MilImage, READ_REGION_POS_X, READ_REGION_POS_Y,
               READ_REGION_WIDTH, READ_REGION_HEIGHT, &MilSubImage);

   /* Define the bottom of the image as the region where to copy the font representation.*/
   MbufChild2d(MilImage, 50, SizeY+10, SizeX-100, (SizeY/3)-10, &MilFontSubImage);

   /* Restore the OCR character font from disk. */
   MocrRestoreFont(FONT_FILE_IN, M_RESTORE, MilSystem, &OcrFont);

   /* Show the font representation. */
   MocrCopyFont(MilFontSubImage, OcrFont, M_COPY_FROM_FONT+M_ALL_CHAR, M_NULL );

   /* Pause to show the original image. */
   printf("The SEMI string at the top will be read using the font displayed at the bottom.\n\n");
   printf("Calibrating SEMI font...\n\n");

   /* Calibrate the OCR font. */
   MocrCalibrateFont(MilSubImage, OcrFont, STRING_CALIBRATION,
                     CHAR_SIZE_X_MIN, CHAR_SIZE_X_MAX, CHAR_SIZE_X_STEP,
                     CHAR_SIZE_Y_MIN, CHAR_SIZE_Y_MAX, CHAR_SIZE_Y_STEP,
                     M_DEFAULT);

   /* Set the user-specific character constraints for each string position.  */
   MocrSetConstraint(OcrFont, 0,  M_LETTER, M_NULL);        /* A to Z only   */
   MocrSetConstraint(OcrFont, 1,  M_DIGIT,  MIL_TEXT("9")); /* 9      only   */
   MocrSetConstraint(OcrFont, 2,  M_DIGIT,  M_NULL);        /* 0 to 9 only   */
   MocrSetConstraint(OcrFont, 3,  M_DIGIT,  M_NULL);        /* 0 to 9 only   */
   MocrSetConstraint(OcrFont, 4,  M_DIGIT,  M_NULL);        /* 0 to 9 only   */
   MocrSetConstraint(OcrFont, 5,  M_DIGIT,  M_NULL);        /* 0 to 9 only   */
   MocrSetConstraint(OcrFont, 6,  M_DIGIT,  M_NULL);        /* 0 to 9 only   */
   MocrSetConstraint(OcrFont, 7,  M_DEFAULT,MIL_TEXT("-")); /* -      only   */
   MocrSetConstraint(OcrFont, 8,  M_LETTER, MIL_TEXT("M")); /* M      only   */
   MocrSetConstraint(OcrFont, 9,  M_LETTER, MIL_TEXT("X")); /* X      only   */
   MocrSetConstraint(OcrFont, 10, M_LETTER, MIL_TEXT("ABCDEFGH")); /* SEMI checksum */
   MocrSetConstraint(OcrFont, 11, M_DIGIT,  MIL_TEXT("01234567")); /* SEMI checksum */

   /* Pause before the read operation. */
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Allocate an OCR result buffer. */
   MocrAllocResult(MilSystem, M_DEFAULT, &OcrResult);

   /* Read the string. */
   MocrReadString(MilSubImage, OcrFont, OcrResult);

   /* Get the string and its reading score. */
   MocrGetResult(OcrResult, M_STRING, String);
   MocrGetResult(OcrResult, M_STRING_SCORE,  &Score);

   /* Print the result. */
   MOs_printf(MIL_TEXT("\nThe string read is: \"%s\" (score: %.1f%%).\n\n"), String, Score);

   /* Draw the string in the overlay under the reading region. */
   MdispControl(MilDisplay, M_OVERLAY, M_ENABLE);
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);
   MdispInquire(MilDisplay, M_OVERLAY_ID, &MilOverlayImage);
   MgraFont(M_DEFAULT, M_FONT_DEFAULT_LARGE);
   MgraColor(M_DEFAULT, M_COLOR_YELLOW);
   MgraText(M_DEFAULT, MilOverlayImage, READ_REGION_POS_X+(READ_REGION_WIDTH/4),
                                        READ_REGION_POS_Y+READ_REGION_HEIGHT+50, String);

   /* Save the calibrated font if the reading score was sufficiently high. */
   if (Score > READ_SCORE_MIN)
      {
      MocrSaveFont(FONT_FILE_OUT, M_SAVE, OcrFont);
      printf("Read successful, calibrated OCR font was saved to disk.\n");
      }
   else
      {
      printf("Error: Read score too low, calibrated OCR font not saved.\n");
      }
   printf("Press <Enter> to continue.\n\n\n");
   getch();

   /* Clear the overlay. */
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);

   /* Free all allocations. */
   MocrFree(OcrFont);
   MocrFree(OcrResult);
   MbufFree(MilSubImage);
   MbufFree(MilFontSubImage);
   MbufFree(MilImage);
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
}


Example: mpat.c

/***************************************************************************************/
/*
 * File name: MPat.c
 *
 * Synopsis:  This program contains 3 examples of the pattern matching module:
 *
 *            The first example defines a model and then searches for it in a shifted
 *            version of the image (without rotation).
 *
 *            The second example defines a model and then searches for it in a
 *            rotated version of the image.
 *
 *            The third example automatically allocates a model in a wafer image and finds
 *            its horizontal and vertical displacement.
 */
#include <mil.h>
#include <conio.h>
#include <math.h>

/* Example functions declarations. */
void SearchModelExample(MIL_ID MilSystem, MIL_ID MilDisplay);
void SearchRotatedModelExample(MIL_ID MilSystem, MIL_ID MilDisplay);
void AutoAllocationModelExample(MIL_ID MilSystem, MIL_ID MilDisplay);

/*****************************************************************************
 Main.
 *****************************************************************************/
void main(void)
   {
   MIL_ID MilApplication,     /* Application identifier. */
          MilSystem,          /* System identifier.      */
          MilDisplay;         /* Display identifier.     */

   printf("\nGRAYSCALE PATTERN MATCHING:\n");
   printf("---------------------------\n\n");

   /* Allocate defaults. */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay, M_NULL, M_NULL);

   /* Run the search at 0 degree example. */
   SearchModelExample(MilSystem, MilDisplay);

   /* Run the search over 360 degrees example. */
   SearchRotatedModelExample(MilSystem, MilDisplay);

   /* Run the automatic model allocation example. */
   AutoAllocationModelExample(MilSystem, MilDisplay);

   /* Free defaults. */
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
   }

/*****************************************************************************/
/* Find model in shifted version of the image example. */

/* Source image file name. */
#define FIND_IMAGE_FILE    M_IMAGE_PATH MIL_TEXT("CircuitsBoard.mim")

/* Model position and size. */
#define FIND_MODEL_X_POS   153L
#define FIND_MODEL_Y_POS   132L
#define FIND_MODEL_WIDTH   128L
#define FIND_MODEL_HEIGHT  128L
#define FIND_MODEL_X_CENTER   (FIND_MODEL_X_POS+(FIND_MODEL_WIDTH -1)/2.0)
#define FIND_MODEL_Y_CENTER   (FIND_MODEL_Y_POS+(FIND_MODEL_HEIGHT-1)/2.0)

/* Target image shifting values. */
#define FIND_SHIFT_X        4.5
#define FIND_SHIFT_Y        7.5

/* Minimum match score to determine acceptability of model (default). */
#define FIND_MODEL_MIN_MATCH_SCORE 70.0

/* Minimum accuracy for the search. */
#define FIND_MODEL_MIN_ACCURACY    0.1

/* Absolute value macro. */
#define Absolute(x) (((x) < 0.0) ? -(x) : (x))

void SearchModelExample(MIL_ID MilSystem, MIL_ID MilDisplay)
{
   MIL_ID MilImage,                /* Image buffer identifier.  */
          MilOverlayImage,         /* Overlay image.            */
          Model,                   /* Model identifier.         */
          Result;                  /* Result identifier.        */
   double XOrg  = 0.0, YOrg = 0.0; /* Original model position.  */
   double x     = 0.0, y    = 0.0; /* Model position.           */
   double ErrX  = 0.0, ErrY = 0.0; /* Model error position.     */
   double Score = 0.0;             /* Model correlation score.  */
   double Time  = 0.0;             /* Model search time.        */
   long   AnnotationColor = M_COLOR_RED; /* Drawing color.      */

   /* Restore source image into an automatically allocated image buffer. */
   MbufRestore(FIND_IMAGE_FILE, MilSystem, &MilImage);

   /* Display the image buffer. */
   MdispSelect(MilDisplay, MilImage);

   /* Prepare for overlay annotations. */
   MdispControl(MilDisplay, M_OVERLAY, M_ENABLE);
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);
   MdispInquire(MilDisplay, M_OVERLAY_ID, &MilOverlayImage);

   /* Allocate a normalized grayscale model. */
   MpatAllocModel(MilSystem, MilImage, FIND_MODEL_X_POS, FIND_MODEL_Y_POS,
                  FIND_MODEL_WIDTH, FIND_MODEL_HEIGHT, M_NORMALIZED, &Model);

   /* Set the search accuracy to high. */
   MpatSetAccuracy(Model, M_HIGH);

   /* Set the search model speed to high. */
   MpatSetSpeed(Model, M_HIGH);

   /* Preprocess the model. */
   MpatPreprocModel(MilImage, Model, M_DEFAULT);

   /* Draw a box around the model in the model image. */
   MgraColor(M_DEFAULT, AnnotationColor);
   MpatDraw(M_DEFAULT, Model, MilOverlayImage, M_DRAW_BOX+M_DRAW_POSITION,
                                                          M_DEFAULT, M_ORIGINAL);

   /* Pause to show the original image and model position. */
   printf("\nA %ldx%ld model was defined in the source image.\n",
                                 FIND_MODEL_WIDTH, FIND_MODEL_HEIGHT);
   printf("It will be found in an image shifted by %.2f in X and %.2f in Y.\n",
                                                         FIND_SHIFT_X, FIND_SHIFT_Y);
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Clear the overlay image. */
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);

   /* Translate the image on a subpixel level. */
   MimTranslate(MilImage, MilImage, FIND_SHIFT_X, FIND_SHIFT_Y, M_DEFAULT);

   /* Allocate result buffer. */
   MpatAllocResult(MilSystem, 1L, &Result);

   /* First, perform a dummy find operation for better function timing accuracy
      (model cache effect,...) and reset the timer.
    */
   MpatFindModel(MilImage, Model, Result);
   MappTimer(M_TIMER_RESET+M_SYNCHRONOUS, M_NULL);

   /* Find the model in the target buffer. */
   MpatFindModel(MilImage, Model, Result);

   /* Read the time spent in MpatFindModel. */
   MappTimer(M_TIMER_READ+M_SYNCHRONOUS, &Time);

   /* If one model was found above the acceptance threshold. */
   if (MpatGetNumber(Result, M_NULL) == 1L)
      {
      /* Read results and draw a box around the model occurrence. */
      MpatGetResult(Result, M_POSITION_X, &x);
      MpatGetResult(Result, M_POSITION_Y, &y);
      MpatGetResult(Result, M_SCORE, &Score);
      MgraColor(M_DEFAULT, AnnotationColor);
      MpatDraw(M_DEFAULT, Result, MilOverlayImage, M_DRAW_BOX+M_DRAW_POSITION,
                                                              M_DEFAULT, M_DEFAULT);

      /* Calculate the position errors in X and Y and inquire original model position. */
      ErrX = fabs((FIND_MODEL_X_CENTER+FIND_SHIFT_X) - x);
      ErrY = fabs((FIND_MODEL_Y_CENTER+FIND_SHIFT_Y) - y);
      MpatInquire(Model, M_ORIGINAL_X, &XOrg);
      MpatInquire(Model, M_ORIGINAL_Y, &YOrg);

      /* Print out the search result of the model in the original image. */
      printf("Search results:\n");
      printf("---------------------------------------------------\n");
      printf("The model was found to have been shifted by\tX:%.2f, Y:%.2f.\n", x-XOrg, y-YOrg);
      printf("The model position error is \t\tX:%.2f, Y:%.2f\n", ErrX, ErrY);
      printf("The model match score is \t\t%.1f\n", Score);
      printf("The search time is \t\t\t%.3f ms\n\n", Time*1000.0);

      /* Verify the results. */
      if (
          (Absolute((x - XOrg) - FIND_SHIFT_X) > FIND_MODEL_MIN_ACCURACY) ||
          (Absolute((y - YOrg) - FIND_SHIFT_Y) > FIND_MODEL_MIN_ACCURACY) ||
          (Score                              < FIND_MODEL_MIN_MATCH_SCORE)
         )
         printf("Results verification error !\n");
      }
   else
      printf("Model not found !\n");

   /* Wait for a key to be pressed. */
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Clear the overlay image. */
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);

   /* Free all allocations. */
   MpatFree(Result);
   MpatFree(Model);
   MbufFree(MilImage);
}

/*****************************************************************************/
/* Find rotated model example. */

/* Source image file name. */
#define ROTATED_FIND_IMAGE_FILE      M_IMAGE_PATH MIL_TEXT("CircuitsBoard.mim")

/* Image rotation values. */
#define ROTATED_FIND_ROTATION_DELTA_ANGLE  10
#define ROTATED_FIND_ROTATION_ANGLE_STEP   1
#define ROTATED_FIND_RAD_PER_DEG           0.01745329251

/* Model position and size. */
#define ROTATED_FIND_MODEL_X_POS           153L
#define ROTATED_FIND_MODEL_Y_POS           132L
#define ROTATED_FIND_MODEL_WIDTH           128L
#define ROTATED_FIND_MODEL_HEIGHT          128L

#define ROTATED_FIND_MODEL_X_CENTER           ROTATED_FIND_MODEL_X_POS+ \
                                             (ROTATED_FIND_MODEL_WIDTH -1)/2.0
#define ROTATED_FIND_MODEL_Y_CENTER           ROTATED_FIND_MODEL_Y_POS+ \
                                             (ROTATED_FIND_MODEL_HEIGHT-1)/2.0

/* Minimum accuracy for the search position. */
#define ROTATED_FIND_MIN_POSITION_ACCURACY     0.10

/* Minimum accuracy for the search angle. */
#define ROTATED_FIND_MIN_ANGLE_ACCURACY        0.25

/* Angle range to search. */
#define ROTATED_FIND_ANGLE_DELTA_POS           ROTATED_FIND_ROTATION_DELTA_ANGLE
#define ROTATED_FIND_ANGLE_DELTA_NEG           ROTATED_FIND_ROTATION_DELTA_ANGLE

/* Prototypes of utility functions. */
void RotateModelCenter(MIL_ID Buffer,
                       double *X,
                       double *Y,
                       double Angle);

double CalculateAngleDist(double Angle1,
                          double Angle2);

void SearchRotatedModelExample(MIL_ID MilSystem, MIL_ID MilDisplay)
   {
   MIL_ID MilSourceImage,              /* Model  image buffer identifier.   */
          MilTargetImage,              /* Target image buffer identifier.   */
          MilDisplayImage,             /* Target image buffer identifier.   */
          MilOverlayImage,             /* Overlay image.                    */
          MilModel,                    /* Model identifier.                 */
          MilResult;                   /* Result identifier.                */
   double RealX       = 0.0,           /* Model real position in x.         */
          RealY       = 0.0,           /* Model real position in y.         */
          RealAngle   = 0.0,           /* Model real angle.                 */
          X           = 0.0,           /* Model position in x found.        */
          Y           = 0.0,           /* Model position in y found.        */
          Angle       = 0.0,           /* Model angle found.                */
          Score       = 0.0,           /* Model correlation score.          */
          Time        = 0.0,           /* Model search time.                */
          ErrX        = 0.0,           /* Model error position in x.        */
          ErrY        = 0.0,           /* Model error position in y.        */
          ErrAngle    = 0.0,           /* Model error angle.                */
          SumErrX     = 0.0,           /* Model total error position in x.  */
          SumErrY     = 0.0,           /* Model total error position in y.  */
          SumErrAngle = 0.0,           /* Model total error angle.          */
          SumTime     = 0.0;           /* Model total search time.          */
   long   NbFound     = 0,             /* Number of models found.           */
          AnnotationColor = M_COLOR_RED;  /* Drawing color.                 */

   /* Load target image into image buffers and display it. */
   MbufRestore(ROTATED_FIND_IMAGE_FILE, MilSystem, &MilSourceImage);
   MbufRestore(ROTATED_FIND_IMAGE_FILE, MilSystem, &MilTargetImage);
   MbufRestore(ROTATED_FIND_IMAGE_FILE, MilSystem, &MilDisplayImage);
   MdispSelect(MilDisplay, MilDisplayImage);

   /* Prepare for overlay annotations. */
   MdispControl(MilDisplay, M_OVERLAY, M_ENABLE);
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);
   MdispInquire(MilDisplay, M_OVERLAY_ID, &MilOverlayImage);

   /* Allocate a normalized grayscale model. */
   MpatAllocModel(MilSystem, MilSourceImage,
                  ROTATED_FIND_MODEL_X_POS, ROTATED_FIND_MODEL_Y_POS,
                  ROTATED_FIND_MODEL_WIDTH, ROTATED_FIND_MODEL_HEIGHT,
                  M_NORMALIZED+M_CIRCULAR_OVERSCAN, &MilModel);

   /* Set the search model speed. */
   MpatSetSpeed(MilModel, M_MEDIUM);

   /* Set the position search accuracy. */
   MpatSetAccuracy(MilModel, M_HIGH);

   /* Activate the search model angle mode. */
   MpatSetAngle(MilModel, M_SEARCH_ANGLE_MODE, M_ENABLE);

   /* Set the search model range angle. */
   MpatSetAngle(MilModel, M_SEARCH_ANGLE_DELTA_NEG, ROTATED_FIND_ANGLE_DELTA_NEG);
   MpatSetAngle(MilModel, M_SEARCH_ANGLE_DELTA_POS, ROTATED_FIND_ANGLE_DELTA_POS);

   /* Set the search model angle accuracy. */
   MpatSetAngle(MilModel, M_SEARCH_ANGLE_ACCURACY, ROTATED_FIND_MIN_ANGLE_ACCURACY);

   /* Set the search model angle interpolation mode to bilinear. */
   MpatSetAngle(MilModel, M_SEARCH_ANGLE_INTERPOLATION_MODE, M_BILINEAR);

   /* Preprocess the model. */
   MpatPreprocModel(MilSourceImage, MilModel, M_DEFAULT);

   /* Allocate a result buffer. */
   MpatAllocResult(MilSystem, 1L, &MilResult);

   /* Draw the original model position */
   MpatDraw(M_DEFAULT, MilModel, MilOverlayImage, M_DRAW_BOX+M_DRAW_POSITION,
                                                       M_DEFAULT, M_ORIGINAL);

   /* Pause to show the original image and model position. */
   printf("\nA %ldx%ld model was defined in the source image.\n",
                          ROTATED_FIND_MODEL_WIDTH, ROTATED_FIND_MODEL_HEIGHT);
   printf("It will be searched in images rotated from %ld degree to %ld degree.\n",
                    -ROTATED_FIND_ROTATION_DELTA_ANGLE, ROTATED_FIND_ROTATION_DELTA_ANGLE);
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* First, perform a dummy find operation to verify the model for better function timing accuracy. */
   MpatFindModel(MilSourceImage, MilModel, MilResult);

   /* If the model was found above the acceptance threshold. */
   if (MpatGetNumber(MilResult, M_NULL) == 1L)
      {
      /* Search for the model in images at different angles. */
      RealAngle = ROTATED_FIND_ROTATION_DELTA_ANGLE;
      while (RealAngle >= -ROTATED_FIND_ROTATION_DELTA_ANGLE)
         {
         /* Rotate the image from the model image to target image. */
         MimRotate(MilSourceImage, MilTargetImage, RealAngle, M_DEFAULT,
                   M_DEFAULT, M_DEFAULT, M_DEFAULT, M_BILINEAR+M_OVERSCAN_CLEAR);

         /* Reset the timer. */
         MappTimer(M_TIMER_RESET+M_SYNCHRONOUS, M_NULL);

         /* Find the model in the target image. */
         MpatFindModel(MilTargetImage, MilModel, MilResult);

         /* Read the time spent in MpatFindModel(). */
         MappTimer(M_TIMER_READ+M_SYNCHRONOUS, &Time);

         /* Clear the overlay image. */
         MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);

         /* If one model was found above the acceptance threshold. */
         if (MpatGetNumber(MilResult, M_NULL) == 1L)
            {
            /* Read results and draw a box around model occurrence. */
            MpatGetResult(MilResult, M_POSITION_X, &X);
            MpatGetResult(MilResult, M_POSITION_Y, &Y);
            MpatGetResult(MilResult, M_ANGLE, &Angle);
            MpatGetResult(MilResult, M_SCORE, &Score);

            MgraColor(M_DEFAULT, AnnotationColor);
            MpatDraw(M_DEFAULT, MilResult, MilOverlayImage,
                     M_DRAW_BOX+M_DRAW_POSITION, M_DEFAULT, M_DEFAULT);

            MbufCopy(MilTargetImage, MilDisplayImage);

            /* Calculate the angle error and the position errors for statistics. */
            ErrAngle = CalculateAngleDist(Angle, RealAngle);

            RotateModelCenter(MilSourceImage, &RealX, &RealY, RealAngle);
            ErrX = fabs(X - RealX);
            ErrY = fabs(Y - RealY);

            SumErrAngle += ErrAngle;
            SumErrX += ErrX;
            SumErrY += ErrY;
            SumTime += Time;
            NbFound++;

            /* Verify the precision for the position and the angle. */
            if ((ErrX > ROTATED_FIND_MIN_POSITION_ACCURACY) ||
                (ErrY > ROTATED_FIND_MIN_POSITION_ACCURACY) ||
                (ErrAngle > ROTATED_FIND_MIN_ANGLE_ACCURACY))
               {
               printf("Model accuracy error at angle %.1f !\n\n", RealAngle);
               printf("Errors are X:%.3f, Y:%.3f and Angle:%.2f\n\n", ErrX, ErrY, ErrAngle);
               printf("Press <Enter> to continue.\n\n");
               getch();
               }
            }
         else
            {
            printf("Model was not found at angle %.1f !\n\n", RealAngle);
            printf("Press <Enter> to continue.\n\n");
            getch();
            }

         RealAngle -= ROTATED_FIND_ROTATION_ANGLE_STEP;
         }

      /* Print out the search result statistics */
      /* of the models found in rotated images. */
      printf("\nSearch statistics for the model found in the rotated images.\n");
      printf("------------------------------------------------------------\n");
      printf("The average position error is \t\tX:%.3f, Y:%.3f\n", SumErrX/NbFound, SumErrY/NbFound);
      printf("The average angle error is \t\t%.3f\n", SumErrAngle/NbFound);
      printf("The average search time is \t\t%.3f ms\n\n", SumTime*1000.0/NbFound);
      }
   else
      {
      printf("Model was not found!\n\n");
      }

   /* Wait for a key to be pressed. */
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Clear the overlay image. */
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);

   /* Free all allocations. */
   MpatFree(MilResult);
   MpatFree(MilModel);
   MbufFree(MilTargetImage);
   MbufFree(MilSourceImage);
   MbufFree(MilDisplayImage);
   }

/* Calculate the rotated center of the model to compare the accuracy with
 *  the center of the occurrence found during pattern matching.
 */
void RotateModelCenter(MIL_ID Buffer,
                       double *X,
                       double *Y,
                       double Angle)
   {
   long   BufSizeX = MbufInquire(Buffer, M_SIZE_X, M_NULL);
   long   BufSizeY = MbufInquire(Buffer, M_SIZE_Y, M_NULL);
   double RadAngle = Angle * ROTATED_FIND_RAD_PER_DEG;
   double CosAngle = cos(RadAngle);
   double SinAngle = sin(RadAngle);

   double OffSetX = (BufSizeX-1)/2.0F;
   double OffSetY = (BufSizeY-1)/2.0F;

   *X = (ROTATED_FIND_MODEL_X_CENTER-OffSetX)*CosAngle + (ROTATED_FIND_MODEL_Y_CENTER-OffSetY)*SinAngle + OffSetX;
   *Y = (ROTATED_FIND_MODEL_Y_CENTER-OffSetY)*CosAngle - (ROTATED_FIND_MODEL_X_CENTER-OffSetX)*SinAngle + OffSetY;
   }

/* Calculate the absolute difference between the real angle
 * and the angle found.
 */
double CalculateAngleDist(double Angle1, double Angle2)
   {
   double dist = fabs(Angle1 - Angle2);

   while(dist >= 360.0)
      dist -= 360.0;

   if(dist > 180.0)
      dist = 360.0 - dist;

   return dist;
   }

/*****************************************************************************/
/* Automatic model allocation example. */

/* Source and target images file specifications. */
#define AUTO_MODEL_IMAGE_FILE         M_IMAGE_PATH MIL_TEXT("Wafer.mim")
#define AUTO_MODEL_TARGET_IMAGE_FILE  M_IMAGE_PATH MIL_TEXT("WaferShifted.mim")
//#define AUTO_MODEL_IMAGE_FILE         M_IMAGE_PATH MIL_TEXT("CircuitsBoard.mim")
//#define AUTO_MODEL_TARGET_IMAGE_FILE  M_IMAGE_PATH MIL_TEXT("CircuitsBoard.mim")

/* Model width and height */
#define AUTO_MODEL_WIDTH   64L
#define AUTO_MODEL_HEIGHT  64L

void AutoAllocationModelExample(MIL_ID MilSystem, MIL_ID MilDisplay)
{
   MIL_ID MilImage,                /* Image buffer identifier.             */
          MilOverlayImage,         /* Overlay image.                       */
          MilSubImage,             /* Sub-image buffer identifier.         */
          MilOverlaySubImage,      /* Overlay sub-image buffer identifier. */
          Model,                   /* Model identifier.                    */
          Result;                  /* Result buffer identifier.            */
   long   AllocError;              /* Allocation error variable.           */
   long   ImageWidth, ImageHeight; /* Target image dimensions              */
   double OrgX=0.0, OrgY=0.0;      /* Original center of model.            */
   double x=0.0, y=0.0, Score=0.0; /* Result variables.                    */
   long   AnnotationColor = M_COLOR_RED; /* Drawing color.                 */

   /* Load model image into an image buffer. */
   MbufRestore(AUTO_MODEL_IMAGE_FILE, MilSystem, &MilImage);

   /* Display the image and prepare for overlay annotations. */
   MdispSelect(MilDisplay, MilImage);
   MdispControl(MilDisplay, M_OVERLAY, M_ENABLE);
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);
   MdispInquire(MilDisplay, M_OVERLAY_ID, &MilOverlayImage);

   /* Restrict the region to be processed to the bottom right corner of the image. */
   MbufInquire(MilImage, M_SIZE_X, &ImageWidth);
   MbufInquire(MilImage, M_SIZE_Y, &ImageHeight);
   MbufChild2d(MilImage, ImageWidth/2, ImageHeight/2,
                         ImageWidth/2, ImageHeight/2, &MilSubImage);
   MbufChild2d(MilOverlayImage, ImageWidth/2, ImageHeight/2,
                         ImageWidth/2, ImageHeight/2, &MilOverlaySubImage);

   /* Automatically allocate a normalized grayscale type model. */
   MpatAllocAutoModel(MilSystem, MilSubImage, AUTO_MODEL_WIDTH, AUTO_MODEL_HEIGHT,
                      M_DEFAULT, M_DEFAULT, M_NORMALIZED, M_DEFAULT, &Model);

   /* Set the search accuracy to high. */
   MpatSetAccuracy(Model, M_HIGH);

   /* Check for that model allocation was successful. */
   MappGetError(M_CURRENT, &AllocError);
   if (!AllocError)
      {
      /* Draw a box around the model. */
      MgraColor(M_DEFAULT, AnnotationColor);
      MpatDraw(M_DEFAULT, Model, MilOverlaySubImage, M_DRAW_BOX+M_DRAW_POSITION,
                                                                M_DEFAULT, M_ORIGINAL);
      printf("A model was automatically defined in the image.\n");
      printf("Press <Enter> to continue.\n\n");
      getch();

      /* Clear the overlay image. */
      MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);

      /* Load target image into an image buffer. */
      MbufLoad(AUTO_MODEL_TARGET_IMAGE_FILE, MilImage);

      /* Allocate result. */
      MpatAllocResult(MilSystem, 1L, &Result);

      /* Find model. */
      MpatFindModel(MilSubImage, Model, Result);

      /* If one model was found above the acceptance threshold set. */
      if (MpatGetNumber(Result, M_NULL) == 1L)
         {
         /* Get results. */
         MpatGetResult(Result, M_POSITION_X, &x);
         MpatGetResult(Result, M_POSITION_Y, &y);
         MpatGetResult(Result, M_SCORE, &Score);

         /* Draw a box around the occurrence. */
         MgraColor(M_DEFAULT, AnnotationColor);
         MpatDraw(M_DEFAULT, Result, MilOverlaySubImage, M_DRAW_BOX+M_DRAW_POSITION,
                                                                    M_DEFAULT, M_DEFAULT);

         /* Analyze and print results. */
         MpatInquire(Model,M_ORIGINAL_X,&OrgX);
         MpatInquire(Model,M_ORIGINAL_Y,&OrgY);
         printf("An image misaligned by 50 pixels in X and 20 pixels in Y was loaded.\n\n");
         printf("The image was found to be shifted by %.2f in X, and %.2f in Y.\n",x-OrgX, y-OrgY);
         printf("Model match score is %.1f percent.\n", Score);
         printf("Press <Enter> to end.\n\n");
         getch();
         }
      else
         {
         printf("Error: Pattern not found properly.\n");
         printf("Press <Enter> to end.\n\n");
         getch();
         }

      /* Free result buffer and model. */
      MpatFree(Result);
      MpatFree(Model);
      }
   else
      {
      printf("Error: Automatic model definition failed.\n");
      printf("Press <Enter> to end.\n\n");
      getch();
      }

   /* Clear the overlay image. */
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);

   /* Free child buffer and defaults. */
   MbufFree(MilSubImage);
   MbufFree(MilOverlaySubImage);
   MbufFree(MilImage);
}


Example: mstr.c

/*******************************************************************************/
/*
* File name: MStr.c
*
* Synopsis:  This program uses the String Reader module to define a font
*            from an image containing a mosaic of Quebec license plates.
*            Two String Models are then defined and parameterized to read
*            only some valid Quebec license plates.
*            A license plate reading is then performed in a target image
*            of a car on the road.
*/
#include <mil.h>
#include <conio.h>

/* MIL image file specifications. */
#define IMAGE_FILE_DEFINITION M_IMAGE_PATH MIL_TEXT("QcPlates.mim")
#define IMAGE_FILE_TO_READ    M_IMAGE_PATH MIL_TEXT("LicPlate.mim")

/* String containing all characters used for font definition. */
#define TEXT_DEFINITION       "AVS300CVK781JNK278 EBX380QKN918HCC839 YRH900ZQR756977AMQ GPK742389EYE569ESQ"

/* Font normalization size Y. */
#define NORMALIZATION_SIZE_Y    20L

/* Max size of plate string. */
#define STRING_MAX_SIZE         32L

/*****************************************************************************/
/* Main. */
void main(void)
   {
   MIL_ID MilApplication,                          /* Application identifier.          */
          MilSystem,                               /* System identifier.               */
          MilDisplay,                              /* Display identifier.              */
          MilImage,                                /* Image buffer identifier.         */
          MilOverlayImage,                         /* Overlay image.                   */
          MilStrContext,                           /* String context identifier.       */
          MilStrResult;                            /* String result buffer identifier. */
   long   NumberOfStringRead;                      /* Total number of strings to read. */
   double Score;                                   /* String score.                    */
   char*  StringResult[STRING_MAX_SIZE+1];         /* String of characters read.       */
   double Time = 0.0;                              /* Time variable.                   */

   /* Print module name. */
   printf("\nSTRING READER MODULE:\n");
   printf("---------------------\n\n");

   /* Allocate defaults */
   MappAllocDefault(M_SETUP, &MilApplication, &MilSystem, &MilDisplay, M_NULL, M_NULL);

   /* Restore the font definition image */
   MbufRestore(IMAGE_FILE_DEFINITION, MilSystem, &MilImage);

   /* Display the image and prepare for overlay annotations. */
   MdispSelect(MilDisplay, MilImage);
   MdispControl(MilDisplay, M_OVERLAY, M_ENABLE);
   MdispInquire(MilDisplay, M_OVERLAY_ID, &MilOverlayImage);

   /* Allocate a new empty String Reader context. */
   MstrAlloc( MilSystem, M_FEATURE_BASED, M_DEFAULT, &MilStrContext);

   /* Allocate a new empty String Reader result buffer. */
   MstrAllocResult(MilSystem, M_DEFAULT, &MilStrResult);

   /* Add a new empty user defined font to the context. */
   MstrControl(MilStrContext, M_CONTEXT, M_FONT_ADD, M_USER_DEFINED);

   /* Add user defined characters from the license plate mosaic image. */
   MstrEditFont(MilStrContext, M_FONT_INDEX(0), M_CHAR_ADD,
                M_USER_DEFINED + M_FOREGROUND_BLACK,
                MilImage, TEXT_DEFINITION, M_NULL);

   /* Draw all the characters in the font in the overlay image. */
   MgraColor(M_DEFAULT, M_COLOR_GREEN);
   MstrDraw(M_DEFAULT, MilStrContext, MilOverlayImage, M_DRAW_CHAR,
            M_FONT_INDEX(0), M_NULL, M_ORIGINAL);

   /* Normalize the characters of the font to an appropriate size. */
   MstrEditFont(MilStrContext, M_FONT_INDEX(0), M_CHAR_NORMALIZE,
                M_SIZE_Y, NORMALIZATION_SIZE_Y, M_NULL, M_NULL);

   /* Add 2 new empty strings models to the context for the 2 valid types of
      plates (3 letters followed by 3 numbers or 3 numbers followed by 3 letters)
      Note that the read time increases with the number of strings in the context.
   */
   MstrControl(MilStrContext, M_CONTEXT, M_STRING_ADD, M_USER_DEFINED);
   MstrControl(MilStrContext, M_CONTEXT, M_STRING_ADD, M_USER_DEFINED);

   /* Set number of strings to read. */
   MstrControl(MilStrContext, M_CONTEXT, M_STRING_NUMBER, 1);

   /* Set number of expected characters for all string models to 6 characters. */
   MstrControl(MilStrContext, M_STRING_INDEX(M_ALL), M_STRING_SIZE_MIN, 6);
   MstrControl(MilStrContext, M_STRING_INDEX(M_ALL), M_STRING_SIZE_MAX, 6);

   /* Set default constraint to uppercase letter for all string models */
   MstrSetConstraint(MilStrContext, M_STRING_INDEX(0), M_DEFAULT, M_LETTER + M_UPPERCASE, M_NULL );
   MstrSetConstraint(MilStrContext, M_STRING_INDEX(1), M_DEFAULT, M_LETTER + M_UPPERCASE, M_NULL );

   /* Set constraint of 3 last characters to digit for the first string model */
   MstrSetConstraint(MilStrContext, M_STRING_INDEX(0), 3, M_DIGIT, M_NULL );
   MstrSetConstraint(MilStrContext, M_STRING_INDEX(0), 4, M_DIGIT, M_NULL );
   MstrSetConstraint(MilStrContext, M_STRING_INDEX(0), 5, M_DIGIT, M_NULL );

   /* Set constraint of 3 first characters to digit for the second string model */
   MstrSetConstraint(MilStrContext, M_STRING_INDEX(1), 0, M_DIGIT, M_NULL );
   MstrSetConstraint(MilStrContext, M_STRING_INDEX(1), 1, M_DIGIT, M_NULL );
   MstrSetConstraint(MilStrContext, M_STRING_INDEX(1), 2, M_DIGIT, M_NULL );

   /* Pause to show the font definition. */
   printf("This program has defined a font with this Quebec plates mosaic image.\n");
   printf("Press <Enter> to continue.\n\n");
   getch();

   /* Clear the display overlay. */
   MdispControl(MilDisplay, M_OVERLAY_CLEAR, M_DEFAULT);

   /* Load image to read into image buffer. */
   MbufLoad(IMAGE_FILE_TO_READ, MilImage);

   /* Preprocess the String Reader context. */
   MstrPreprocess(MilStrContext, M_DEFAULT);

   /* First, perform a dummy read for better function timing accuracy (model cache effet,...). */
   MstrRead(MilStrContext, MilImage, MilStrResult);

   /* Reset the timer. */
   MappTimer(M_TIMER_RESET+M_SYNCHRONOUS, M_NULL);

   /* Perform the read operation on the specified target image. */
   MstrRead(MilStrContext, MilImage, MilStrResult);

   /* Read the time. */
   MappTimer(M_TIMER_READ+M_SYNCHRONOUS, &Time);

   /* Get number of strings read and show the result. */
   MstrGetResult(MilStrResult, M_GENERAL, M_STRING_NUMBER + M_TYPE_LONG, &NumberOfStringRead);
   if( NumberOfStringRead >= 1)
      {
      printf("The license plate was read successfully (%.2lf ms)\n\n", Time*1000 );

      /* Draw read result. */
      MgraColor(M_DEFAULT, M_COLOR_BLUE);
      MstrDraw(M_DEFAULT, MilStrResult, MilOverlayImage, M_DRAW_STRING, M_ALL, M_NULL, M_DEFAULT);
      MgraColor(M_DEFAULT, M_COLOR_GREEN);
      MstrDraw(M_DEFAULT, MilStrResult, MilOverlayImage, M_DRAW_STRING_BOX, M_ALL, M_NULL, M_DEFAULT);

      /* Print the read result. */
      printf(" String                   Score\n" );
      printf(" ---------------------------------------\n" );
      MstrGetResult(MilStrResult, 0, M_STRING, StringResult);
      MstrGetResult(MilStrResult, 0, M_STRING_SCORE, &Score);
      printf(" %s                  %.1lf\n", StringResult, Score );
      }
   else
      {
      printf("Error: Plate was not read.\n");
      }

   /* Pause to show results. */
   printf("\nPress <Enter> to end.\n\n");
   getch();

   /* Free all allocations. */
   MstrFree(MilStrContext);
   MstrFree(MilStrResult);
   MbufFree(MilImage);

   /* Free defaults. */
   MappFreeDefault(MilApplication, MilSystem, MilDisplay, M_NULL, M_NULL);
   }