About Matrox ActiveMIL help

This chapter presents the help files distributed with ActiveMIL and describes how to make the most of their navigation and customizable features. This chapter is also used as a placeholder for information relevant to all the ActiveMIL help files.

  • Welcome
  • Using this help
    • Printing ActiveMIL Help
    • Customizing your options in ActiveMIL Help
    • Blue/Red menu bar
    • Gray menu bar
    • Navigation in ActiveMIL Help
  • ActiveMIL reference structure
  • Customizing options
  • Copyrights and acknowledgments


Welcome

Next

Welcome to Matrox ActiveMIL help. If you are a first-time user, please refer to the Customizing options topic. This will allow you to customize the help so that you view only the information that is relevant. For information on how to use this help, see Using this help.

The ActiveMIL help files have been designed to help you develop your ActiveMIL application more efficiently and to answer your ActiveMIL questions. Information has been divided among four help files as follows:

  • For general usage of ActiveMIL controls, see the rest of the ActiveMIL User Guide.
  • For information regarding a specific property, method, or event of an ActiveMIL control, see the ActiveMIL Reference.
  • For general board-specific information, see the ActiveMIL Board-Specific Notes. Note that board-specific information regarding a specific property is integrated into the property's reference topic. The same applies for a specific method or event.
  • For a description of some of the terms used in the ActiveMIL help files, see the ActiveMIL Glossary.

Get the latest up-to-date information for this release by referring to the ActiveMIL Readme help file, located in the Matrox Imaging\ActiveMIL\Help folder.

To help familiarize yourself with how ActiveMIL is actually used, see the ActiveMIL examples, which are located in the \Matrox Imaging\ActiveMIL\examples folder. You can also launch or edit these examples using the ActiveMIL example launcher, ExampleLauncher.exe, which is located in the \Matrox Imaging\Tools folder. The launcher also provides a quick description, key words, and lists controls used for each example.

There are also demos available in the Matrox Imaging\ActiveMIL\Demos folder. These can be launched by going to the folder and clicking twice on the file you want to run, or you can run them from the Start menu.

The following sources should be checked for additional information when it is needed:

  • The on-line documentation.
  • Our website at www.matrox.com/imaging .
  • The Matrox Imaging Developer's Forum at www.matrox.com/imaging/support/dev_forum/ .

To view the trademark acknowledgements, see Copyrights and acknowledgements.


Using this help

See Also | Previous | Next
User Guide:
Customizing options
The ActiveMIL user guide

This section will explain how to print, customize your options, and navigate in help.

Printing ActiveMIL Help

Before printing tables, you should enable the background printing option of Microsoft Internet Explorer so that the tables are more legible. To enable this option, use the Tools Internet Options menu command of Microsoft Internet Explorer. From the presented Internet Options dialog box, select the Advanced page. In the Printing section, select Print background colors and images.

When you try to print from the Contents tab of the help file and you choose the Print the selected heading and all subtopics option, the help file might stall. If you experience this problem, close the help file and then re-open it.

Customizing your options in ActiveMIL Help

The panel Customizing options is a very useful topic in help because it allows you to customize the settings for the help. It will narrow the scope of the information so that it is relevant to your needs. It allows you to specify:

  • The product that you are using: ActiveMIL or ActiveMIL-Lite.
  • Your programming language: Visual Basic .NET 2003 or Visual C++ .NET 2003.
  • The boards for which you are developing.
  • Whether you are developing for a specific computer platform.

Two useful tools are the blue and grey menu bars. You have the ability to change the language and board settings from within the different topics on the blue menu bar. The other settings have to be changed from the Customizing options topic. It should be noted that buttons will appear on the blue or gray menu bars only if they are relevant to the topic.

Blue/Red menu bar

There can be up to three different buttons on the blue menu bar. The left-most button will have the name of the current help file. Clicking on it will return you to the Customizing options topic. The other two buttons are the Language and the Board menu buttons, which will appear on the bar when appropriate. These two buttons allow you to make global changes to the scope of the help file without going back to the Customizing options topic.

The Language button allows you to filter the presented information to reflect your programming language. For example, if you are looking at the syntax of a property, you can see what the syntax would be for Visual Basic .NET, Visual C++ .NET, or both, depending on what you set as the programming language.

The Board button allows you to filter the help information so that it is specific to a particular board, or to all boards.

The blue menu bar turns red when the chosen topic is not applicable to one of your global settings. For example, should you choose to investigate a property that is not applicable to the board you have selected, the blue menu bar would go red. In addition, the Availability button would flash red, and then stay red. If you click on Availability, it will then show you what is not applicable; after this, the button turns blue.

Gray menu bar

The buttons on the gray menu bar will give you more information on the specific topic. See Also will allow you to jump to topics that are related to the current topic. Availability will let you know when the current topic is applicable. MIL Equivalent will let you see if the topic has an equivalent in MIL, and if so, what it is. The browse buttons, however labeled, will move you to the next or previous topic.

Navigation in ActiveMIL Help

You can navigate through the ActiveMIL help files using the ActiveMIL help browse buttons or using the standard four tabs in the left panel of the help viewer (Contents, Index, Search, and Favorites).

The ActiveMIL help files allow you to browse through their topics (according to their order in the table of contents) using the arrow buttons on the gray menu bar in their topic panels.

Particularly useful when you are uncertain about where a particular topic is discussed is the Search tab in the left panel. You can search for phrases by enclosing the phrase in quotation marks ("). You can also precisely define a search using wild card expressions (* and ?), Boolean operators (AND, OR, NOT, and NEAR), and nested expressions (using parenthesis).

Note that the help will consider a string with a period in it as two consecutive words. For example, to search for the Digitizer.SignalChannel property, you would have to put it inside quotation marks for the search to find the property.

You might find it useful to search with the Match similar words option enabled, and then refine your query and search through previous results. Sort through the resulting list of topics according to their Title, Location, or Rank by clicking on the corresponding column heading.

Note that the menu command that enables the highlighting of found terms is not very intuitive. When the feature is enabled, the Options menu command Search Highlight Off is available, whereas when disabled, Search Highlight On is available.

ActiveMIL help can assist you in finding all information that is relevant to a specific board. Go to the Search tab and type in "Fully supported on" or "Not supported on" or "See remarks for support on", followed by the full name of the board you are interested in. For example, if you want information relevant to the Matrox Corona-II, you can type in "Fully supported on Matrox Corona-II". This should bring up all the topics that are relevant to the board.

In some instances, you will do a search and not find the item on one of the selected topics. This might be because the item is specific to a different scope. For example, if you are searching for the setting digGrabSamplingPosition and you have your Board option set to Matrox Orion, one of the topics that comes up is the Inquire method of the Digitizer control. If you look through the topic, you won't find the setting you are searching for. That is because it is hidden; if you change the Board option to Show All, the setting will show up under the remarks for Matrox Meteor-II/Multi-Channel. If you can not find what you are searching for, it is a good idea to Show All for the Language and Board options.

In some instances, the information you are looking for might only show up in the grey menu bar. For example, if you search for MbufAllocColor, one of the topics that comes up is the SizeX property of the Image Control. In this topic, MbufAllocColor can only be found under MIL Equivalents on the gray menu bar.

In some rare cases, found terms are not highlighted when highlighting is enabled. Press CTRL F to search for the term in the topic.

For more information on how to use the Microsoft Help Viewer, please click on the following HTML Help link.


ActiveMIL reference structure

Previous | Next

The main topic for each control presents an illustration of that control's object model. Alternatively, you can determine the complete object model of each control by referring to the table of contents of the ActiveMIL Reference.

When using the table of contents, you can expand the view to show the properties, methods, and events of a control by clicking on the "+" sign next to its name. Headings separate these into sub-categories, that is, the "--Properties--" heading is followed by the list of all of the control's properties, next, the "--Methods--" heading is followed by the list of methods, and finally, the events are listed in the same fashion. Any property represented by a book icon in the table of contents is of type object; clicking on the book icon expands the table of contents to show its properties and methods.

For example, in the diagram below, the Application control contains various properties including AvailableSystems, MemoryCompensation, Name, ProcessingCompensation and Timer. The Timer property of the Application control has its own properties and methods, including the Resolution property and the Read method. The AvailableSystems property can also be expanded to show its properties and/or methods.

Each topic describing a property, method, or event includes an illustration showing its position in the control's object model. Each property of type object contains a grey box which lists its properties and methods. Properties in bold text are of type object themselves, and therefore contain their own properties and methods. For example, the AvailableSystems property of the Application control contains the following diagram, illustrating its place in the object model.


Customizing options

Previous | Next

Set the following options so that they reflect your development environment. This will narrow the scope of the information presented, allowing you to use this help file more efficiently. In the help file, the top menu bar will change color depending on whether the topic you are examining is relevant according to the options you have chosen. A blue bar indicates that the topic is pertinent, while a red bar means that it is not.

Some of these can be changed from the Blue menu bar. For more information, see Using this help.

Product:

Matrox ActiveMIL-Lite
Matrox ActiveMIL

Programming language:

Visual Basic .NET 2003
Visual C++ .NET 2003
Show all languages

Board:

Matrox Corona-II
Matrox CronosPlus
Matrox Genesis
Matrox Helios XA
Matrox Helios XCL
Matrox Meteor-II /Standard
Matrox Meteor-II /1394
Matrox Meteor-II /Camera Link
Matrox Meteor-II /Digital
Matrox Meteor-II /Multi-Channel
Matrox Morphis
Matrox Odyssey XA
Matrox Odyssey XCL
Matrox Orion
Matrox Solios XA
Matrox Solios XCL
Show All boards

Platform:

Matrox 4Sight-II/Matrox 4Sight-M
No specific platform


Copyrights and acknowledgments

Previous

© Copyright Matrox Electronic Systems Ltd., 2005.

All rights reserved. Limitation of Liabilities: In no event will Matrox or its suppliers be liable for any indirect, special, incidental, economic, cover or consequential damages arising out of the use of or inability to use the product, user documentation or related technical support, including without limitation, damages or costs relating to the loss of profits, business, goodwill, even if advised of the possibility of such damages. In no event will Matrox and its suppliers' liability exceed the amount paid by you, for the product.

Because some jurisdictions do not allow the exclusion or limitation of liability for consequential or incidental damages, the above limitation may not apply to you.

Disclaimer: Matrox Electronic Systems Ltd. reserves the right to make changes in specifications at any time and without notice. The information provided by this document is believed to be accurate and reliable. However, neither Matrox Electronic Systems Ltd. nor its suppliers assume any responsibility for its use; or for any infringements of patents or other rights of third parties resulting from its use. No license is granted under any patents or patent right of Matrox Electronic Systems Ltd.

Matrox® is a registered trademark of Matrox Electronic Systems Ltd.

Microsoft®, and Windows® are registered trademarks of Microsoft Corporation.

Microsoft Visual Studio™ is a trademark of Microsoft Corporation.

MSDN® is a registered trademark of the Microsoft Developers Network.

PC/104-Plus™ is a trademark of the PC/104 Consortium.

Camera Link® is a registered trademark of the Automated Imaging Association (AIA).

CompactPCI™ is a trademark of PCI Industrial Computer Manufacturers' Group.

Intel®, Pentium®, and MMX™ are registered trademarks of Intel Corporation.

Texas Instruments is a trademark of Texas Instruments Incorporated.

All other nationally and internationally recognized trademarks and tradenames are hereby acknowledged.


Introduction

Chapter 1: Introduction

This chapter presents the features of the ActiveMIL package. It also explains the installation process and how to set up your development environment.

  • ActiveMIL user guide
  • What is ActiveMIL and ActiveMIL-Lite?
  • ActiveMIL and ActiveMIL-Lite controls
  • Migrating from MIL to ActiveMIL
  • ActiveMIL and Intel MMX Technology
    • MMX
    • SSE/SSE2
  • Supported development environments
  • System requirements
    • Supported compilers
  • Installation
    • MilConfig
  • Creating an application (Visual Basic .NET 2003)
    • ActiveMIL Application template
  • Creating an application and adding ActiveMIL to it (Visual C++)
  • Using the ActiveMIL MFC Application template to create an application (Visual C++)
  • Using the MFC Application template to create an application (Visual C++)
  • Using the Win32 Console Application (Visual C++)
  • Adding ActiveMIL/MIL add-ins, include path, and library path (Visual C++)
    • ActiveMIL MFC Application template
    • MFC Application template and console project template
    • Enable ActiveMIL's add-in settings in MSDEV
    • Specifying the location of ActiveMIL's include (*.h) and OCX (*.ocx) files for all projects
    • Specifying the location of ActiveMIL's include (*.h) and OCX (*.ocx) files for one single project


ActiveMIL user guide

See Also | Next
User Guide:
Building an application

This user guide has been structured to allow you to start processing your images with as little information overload as possible. After setting up your development environment as described in this chapter, it is expected that you read the next chapter Building an application. This second chapter will give you enough information to grab an image using the default imaging board and camera, and display the image.

The next set of chapters, up to and including Chapter 12, deal with enhancement, transformation, and then analysis operations. It is expected that you read only the chapters that you need; for Chapter 3, you can read only the required sections.

The remaining chapters go into more detail about setting up and managing your images, displays, and digitizers, as well as describing other topics that might be useful but not critical to getting started.


What is ActiveMIL and ActiveMIL-Lite?

User Guide:
ActiveMIL and ActiveMIL Lite controls
Supported development environments

ActiveMIL is a set of ActiveX controls that are based on the Matrox Imaging Library (MIL). ActiveMIL was designed for rapid application development (RAD) tools, such as Microsoft's Visual Basic .NET and Visual C++. ActiveMIL is hardware-independent, that is, it can run on any graphics controller supported by Windows or on any Matrox imaging board. In general, ActiveMIL's operations are performed by the Host CPU unless specialized Matrox hardware is available and is more efficient.

ActiveMIL comes in a full and lite version. ActiveMIL-Lite allows you to grab, display, and annotate images.

The full version of ActiveMIL contains all the functionality of ActiveMIL-Lite, as well as an extensive set of processing and analysis operations. The analysis operations include: measurement, blob analysis, calibration, optical character recognition (OCR), correlated and geometric pattern matching, Edge Finder, and code read/write operations. The image processing operations include point-to-point, statistical, spatial filtering, morphological, Fast Fourier transform, and geometric operations (warping and polar-to-rectangular transformations).

In general, ActiveMIL can:

  • Grab up to 16-bit grayscale images, or color images.
  • Process 1, 8, 16, and 32-bit integer or floating-point images.
  • Process color images depending on the operation. Each band of a color image is processed individually, one after the other. Statistical and analysis operations do not support color images.
  • Display 1, 8, or 16-bit grayscale or color images.

ActiveMIL documentation refers to both versions of ActiveMIL simply as ActiveMIL, unless it is necessary to distinguish between them. Note that the Availability button of each property, method, or event reference description indicates whether the property, method, or event is supported by ActiveMIL-Lite.

Please review the ActiveMIL Readme file for late breaking information before using ActiveMIL. It is in the Matrox Imaging\ActiveMIL directory.


ActiveMIL and ActiveMIL-Lite controls

User Guide:
Blob Analysis
Building an application
Calibration
Code
Digitizer
Display
Geometric Model Finder
Graphics
Image Processing
Images
Measurement
Optical character recognition
Pattern Matching

The following is a summary of ActiveMIL and ActiveMIL-Lite controls.

Application control. Allows you to control application-wide attributes and functionality. For example, it allows you to accurately measure the execution time of an operation.
System control. Allows you to set up the system on which to run an application. It also allows you to inquire about system-specific attributes, such as the number of digitizers supported by the system.
Image control. Allows you to allocate and operate on images. These operations include loading an image from a file and transferring image data. The Image control also allows you to compress images using the JPEG and JPEG2000 algorithms in either lossless or lossy.
Digitizer control. Allows you to allocate and use a digitizer on your frame grabber to acquire images. Note that a digitizer is the set of acquisition paths, on a frame grabber, required to grab from one camera of a specified type.
Display control. Allows you to display images. You can manipulate the displayed image through various display properties.
GraphicContext control. Allows you to annotate or alter images with text, as well as basic graphics, such as rectangles, arcs, lines, and dots.

In addition, sequences of images can also be loaded from and saved in an AVI file format using ActiveMIL's AVIFileHandler object.

The following controls require the full version of ActiveMIL.

BlobAnalysis control. Allows you to identify connected regions of pixels (blobs) within an image, and then calculate features of these blobs. The BlobAnalysis control can measure a wide assortment of blob features, such as the blob area, perimeter, Feret diameter at a given angle, minimum bounding box, and compactness. It can also be used to perform some image processing operations, such as reconstructing or eliminating blobs.
Calibration control. Allows you to calibrate your imaging setup to correct certain types of image distortion and return results from other ActiveMIL controls in real-world units. The mappings can also be used to physically correct an image's distortions. ActiveMIL's calibration mappings can compensate for non-uniform aspect ratio, rotation, perspective foreshortening, and other more complex distortions.
CharacterRecognition control. Allows you to read or verify mechanically generated, character strings in grayscale images, providing results such as quality scores and validity flags.
Code control. Allows you to read and write 2D codes, as well as several types of bar codes.
EdgeFinder control. Allows you to extract and analyze geometric primitives in images, such as object contours or thin curvilinear shapes.
ImageProcessing control. Allows you to smooth, accentuate, qualify, or modify selected features of an image using ActiveMIL's processing capabilities. These capabilities include point-to-point, statistical, spatial filtering, morphological, Fast Fourier transform, as well as geometric operations which include warping and polar-to-rectangular transformations.
Measurement control. Allows you to find sets of image characteristics or "markers" in an image, based on differences in pixel intensities. Upon finding a marker, ActiveMIL returns the marker's spatial reference position and measures such features as its width and angle. The control can also take measurements between two markers.
ModelFinder control. Allows pattern matching based on edge-based geometric features instead of a pixel-to-pixel correlation. The ModelFinder control can:
  • Search for different models simultaneously.
  • Find models through a range of angles and scales.
  • Has a greater tolerance of adverse lighting conditions.
  • Provides complete support for calibration.
The ModelFinder control also provides more detailed information with its results.
PatternMatching control. Allows normalized grayscale correlation pattern matching. These capabilities include finding:
  • The coordinates of a pattern (referred to as a model) in a target image.
  • The number of occurrences of a model in a target image.
  • Occurrences of a model at any angle, in the target image.

Note that an ActiveMIL application can use many systems. In addition, a system can have many images, digitizers, displays, and processing controls; the exact number depends on the system and how you have allocated your memory.


Migrating from MIL to ActiveMIL

User Guide:
Blue /Red menu bar
Integrating MIL functions into your ActiveMIL code

ActiveMIL contains all the functionality of MIL. However, since it provides an ActiveX interface to that functionality, it sometimes deviates from the organization in MIL. To help your transition from MIL to ActiveMIL, each on-line reference topic states its MIL equivalent. MIL equivalents are a good approximation of the functions and constants to use to achieve the same operation in MIL. A few ActiveMIL concepts do not have any equivalent in MIL. For example:

  • Item properties (except when the collection exists in MIL).
  • Name properties.
  • IsAllocated and AutomaticAllocation properties.
  • Region of interest and its sub-properties (except when the ROI exists in MIL, like Image.ChildRegion).
  • Modified methods and ResultsModified events.
  • PropertyValidation properties.
  • Properties that enable events (for example, the ContentModifiedEvent property).

For each MIL module, there is an equivalent ActiveMIL control, except for the buffer module. The buffer module has been divided. Image buffers have their own control. All other types of buffers have been placed, as object-type properties, in the controls that need them.

Typically, properties that correspond to parameters of allocation functions in MIL (for example, MbufAlloc1d()) cannot be set at run-time if the control is in the allocated state.

The following table shows the MIL equivalents of some main concepts:

MIL | ActiveMIL
image buffer | Image control, often generalized in discussion just as image
LUT buffer | LUT of an image, digitizer, or display, or image processing LUTs
result buffers | the Results properties of different controls
calibration object | Calibration control
graphics context | graphic context
measurement box | measurement search region
Model Finder context | ModelFinder control
Edge Finder context | EdgeFinder control

For information on integrating MIL functions into ActiveMIL code, see Integrating MIL functions into your ActiveMIL code


ActiveMIL and Intel MMX Technology

User Guide:
System requirements

ActiveMIL's processing operations have been optimized, in assembly language, to take advantage of Intel MMX acceleration and Streaming SIMD Extensions (SSE/SSE2).

MMX

Intel MMX Technology, an extension to the Intel architecture, is designed specifically to accelerate multimedia (and multimedia-like) applications. Intel MMX Technology is built to handle computation-intensive algorithms that perform repetitive operations on small data types (such as 8-bit pixels). The technology covers several areas, such as basic arithmetic operations, logical operations, shift operations, comparison operations, and data transfer instructions. These instructions use a SIMD model that allows the processor to perform a single calculation simultaneously on 2, 4, or 8 data elements by packing multiple operands (8-bit, 16-bit, or 32-bit values) into a single 64-bit register and performing processing functions on them in parallel. On a x86 compatible processor with Intel MMX Technology, ActiveMIL operations can execute, typically, 4 times faster than on a regular x86 processor. Some operations benefit even more from the MMX acceleration (for example, a thinning operation can be up to 16 times faster).

SSE/SSE2

Streaming SIMD extensions accelerate performance of floating point operations and include additional integer and caching instructions that significantly enhance performance.


Supported development environments

Previous | Next

ActiveMIL supports the following environments:

  • Microsoft Visual Basic .NET 2003
  • Microsoft Visual C++ .NET 2003 (unmanaged)

The information presented in the ActiveMIL User Guide applies to both environments, unless otherwise stated.


System requirements

User Guide:
Installation

The following system requirements should be respected to ensure that ActiveMIL operates properly:

  • Windows XP or Windows 2000.
  • Minimum of 64 Mbytes of RAM for Windows 2000 and 128 Mbytes of RAM for Windows XP. This does not include non-paged (DMA) memory needed for any of the systems.
  • Minimum of 180 Mbytes of free hard disk space to install the ActiveMIL development environment. Minimum of 50 Mbytes of free hard disk space to install the ActiveMIL run-time environment.
  • Optionally, a Matrox Imaging frame grabber.
  • Optionally, a Matrox graphics board. ActiveMIL has been optimized for Matrox graphics boards.

Note that to run ActiveMIL help, you will need Microsoft Internet Explorer version 6 or above.

Supported compilers

ActiveMIL ActiveX controls are supported for Microsoft Visual Basic .NET 2003 and Microsoft Visual C++ .NET 2003 (unmanaged) RAD tools. These are the actual development tools used for testing.


Installation

User Guide:
System requirements
Uninstalling

In addition to your MIL CD, you will require a development hardware license-key (a two-sided, 25-pin connector) to develop applications. The key allows you to code, debug, and run your applications. ActiveMIL supports development hardware license-keys for either the parallel or USB ports. To redistribute your ActiveMIL applications, see the Distribution and licensing chapter; although ActiveMIL-Lite comes with a royalty free run-time license, ActiveMIL does not.

To install your ActiveMIL software:

  1. Attach the development hardware license-key to the parallel or USB port of your computer, depending on which hardware license key was purchased. If another device is attached to the port, disconnect it, attach the development hardware license-key, and then attach the device's connector to the other end of the development hardware license-key. Note, the device need not be turned on.
  2. Place the installation CD in an appropriate drive. The setup.exe program will run automatically.

During installation, you will be asked a number of questions, such as:

  • The drive and directory on which to install the program.
  • Your development tool.
  • The port used for the development hardware license-key: the parallel or the USB port. You will not be asked this question if you are installing ActiveMIL-Lite.
  • The type of Matrox hardware installed in your computer (for example, Matrox Corona-II). Note that under Windows 2000/XP, the boards should be installed before the Matrox frame grabber drivers are installed.
  • Whether to install the Matrox display drivers. This will only be asked if you have a Matrox Imaging board with a display section or a Matrox graphics board, and the drivers to be installed are newer than the drivers already on your computer.
  • The digitizer and display format to define your default configuration. These will be used when a Digitizer or a Display control is created and to run the examples.
  • Whether by default, displayed images should be displayed in a window on the Windows desktop or without a window on an auxiliary screen (a non-Windows desktop screen). If the latter, you will be asked to specify the required default video configuration format (VCF). Auxiliary screens require either two graphics controllers or a graphics controller that integrates two video output controllers.
  • The amount of linear non-paged memory to reserve for MIL for grabbing. The amount of reserved linear non-paged memory also establishes the amount of remaining RAM available for your operating system.

It is important to note that only one copy of ActiveMIL can be present on a computer at a time. When installing ActiveMIL or ActiveMIL-Lite on a computer with a more recent or equally recent version of ActiveMIL or ActiveMIL-Lite, the setup program will not install ActiveMIL. The application can use the version of ActiveMIL already installed on the computer.

Conversely, if the version of ActiveMIL or ActiveMIL-Lite on the computer is less recent than the application's required version, a decision must be made. Either the version of ActiveMIL or ActiveMIL-Lite already on the computer must be removed before installing the newer version, otherwise the current version cannot be installed on that computer.

After installation, read the ActiveMIL Readme file in the Matrox Imaging\ActiveMIL directory to determine where ActiveMIL files are located and how to run the ActiveMIL examples. Note that the installation program also installs Matrox Intellicam (your frame grabber configuration program) and the MilConfig utility.

After you have finished installation, we suggest that you run the examples available.

MilConfig

MilConfig (Matrox Imaging\Tools\MILConfig.exe), provides licensing, MIL reserved non-paged memory configuration, and system information tools. For example, if you need to change the amount of reserved memory or if you change the amount of physical memory in your computer, you can change the amount of MIL reserved non-paged memory assigned or RAM available to your system at any time by running MilConfig (alternatively, you can adjust the memory by uninstalling and reinstalling ActiveMIL). If you require technical support, use MilConfig's Troubleshooting tab to generate either a TXT file that contains all the necessary system information required for basic troubleshooting; this file can then be forwarded to your Matrox technical support representative.


Creating an application (Visual Basic .NET 2003)

User Guide:
Setting up controls

As of version 8.0, ActiveMIL no longer supports any version of Visual Basic prior to Visual Basic .NET 2003. Applications written in Visual Basic .NET 2003 require the .NET Framework and run within a managed environment. For more information on writing ActiveMIL Visual Basic .NET 2003 applications, see the Using ActiveMIL with .NET chapter.

When you create a Visual Basic .NET project, ActiveMIL's controls are added to the ActiveMIL tab in the toolbox. In addition, entries are added to an ActiveMIL toolbar. These entries permit you to:

  • Change your default settings initially set-up at installation time.
  • Access ActiveMIL's help.

For more information on ActiveMIL's controls, see the Setting up controls section in the Building an application chapter.

ActiveMIL Application template

To use the ActiveMIL Application template:

  1. Select New Project from the File menu command. The New Project dialog box is presented.
  2. Select Visual Basic Projects.
  3. From the list of available templates, select ActiveMIL Application.
  4. Type a project name in the Name edit field (for example, ActiveMILApplication).
  5. Specify the location in which to save your project, in the Location edit field.
  6. Choose one of the following (available only when a solution is open in the IDE):
    • Close Solution (closes the open solution and creates a new one for the new project.)
    • Add to Solution (adds the new project to the open solution.)
  7. Click Ok. Your workspace now contains your Windows Form.

    8. Notice that your Windows Form now contains the basic ActiveMIL controls:

    • Application Control
    • System Control
    • Image Control
    • GraphicContext Control
    • Digitizer Control

    Note that if your computer does not support one of the basic ActiveMIL controls listed above (for example, if you do not have a digitizer), then that control will not be added.

    You can add an ActiveMIL control to your project from the toolbox. To do so, drag and drop it onto your Windows Form.

  8. You can set and verify any control's property settings interactively (that is, at design-time) by choosing Properties from the control's context menu. You can also manipulate any control by altering the code itself (so that the control's property settings are set and verified at run-time).


Creating an application and adding ActiveMIL to it (Visual C++)

User Guide:
Adding ActiveMIL/MIL add-ins, include path, and library path (Visual C++)
Building an application
Using the ActiveMIL MFC Application template
Using the MFC Application template
Using the Win32 Console Application (with or without MFC)
What is Native COM

Depending on your needs, choose one of the following techniques to create an ActiveMIL application, and/or add ActiveMIL to an application:

  • Use the ActiveMIL MFC Application template. This custom-made template creates the foundation for a dialog-based application, and automatically adds ActiveMIL to that application. In addition, the following five basic ActiveMIL controls are automatically added to the dialog: Application, Image, System, GraphicContext, and Digitizer.

    • With this template, the paths for ActiveMIL's include (*.h) and OCX (*.ocx) files are automatically added to your application. For instructions on using this template, refer to the Using the ActiveMIL MFC Application template to create an application (Visual C++) section later in this chapter.

  • Use the MFC Application template. This template creates the foundation for a dialog-based application; however, the paths for ActiveMIL's include (*.h) and OCX (*.ocx) files as well as the ActiveMIL controls are not automatically added to your application. As a result, you can more easily customize how the application accesses ActiveMIL's functionality; for example, you can decide to instantiate specific ActiveMIL controls at design-time, while instantiating others at run-time. For instructions on using this template, refer to the Using the MFC Application template to create an application (Visual C++) section later in this chapter.
  • Using the Win32 Console Project template (with or without MFC). Unlike the other two templates, this one does not provide the automatic foundation for a dialog-based application; instead, it is used to create an application at run-time (a dialog can be added later). Note that the paths for ActiveMIL's include (*.h) and OCX (*.ocx) files are not automatically added to your application. For instructions on using this template, refer to the Using the Win32 Console Application (Visual C++) section later in this chapter.
  • Add ActiveMIL to your current application. If you want to add ActiveMIL to your current application, the paths for ActiveMIL's include (*.h) and OCX (*.ocx) files must be added to that application, refer to the Adding ActiveMIL/MIL add-ins, include path, library path (Visual C++) section later in this chapter.

For more information about designing an ActiveMIL application (once it has been created), see the Setting up controls section in the Building an application chapter.


Using the ActiveMIL MFC Application template to create an application (Visual C++)

User Guide:
Adding ActiveMIL/MIL add-ins, include path, library path
Creating an application (Visual C++)
Run-time object declaration
What is Native COM

To use the ActiveMIL MFC Application template:

  1. Select New Project from the File menu command. The New Project dialog box is presented.
  2. Select Visual C++ Projects.
  3. From the list of available templates, select ActiveMIL MFC Application.
  4. Type a project name in the Name edit field (for example, ActiveMILApplication).
  5. Specify the location in which to save your project, in the Location edit field.
  6. Choose one of the following (available only when a solution is open in the IDE):
    • Close Solution (closes the open solution and creates a new one for the new project).
    • Add to Solution (adds the new project to the open solution).
  7. Click Ok. Your project's Resource View tab now contains your resources, ActiveMILApplication.rc. To access the dialog: click on the Dialog sub-folder of the ActiveMILApplication resource folder; then, double-click on the dialog entry, IDD_ACTIVEMILAPPLICATION_DIALOG.

    8. Notice that your dialog now contains the basic ActiveMIL controls, and a member variable for each of these basic controls:

    • Application (m_Application1)
    • System (m_System1)
    • Image (m_Image1)
    • GraphicContext (m_GraphicContext1)
    • Digitizer (m_Digitizer1)

    Note that if your computer does not support one of the basic ActiveMIL controls listed above (for example, if you do not have a digitizer), then that control will not be added.

  8. You can add an ActiveMIL control to your project from the ActiveMIL tab in the toolbox. To do so drag and drop it onto your dialog. Notice that the path for ActiveMIL's include (*.h) and OCX (*.ocx) files (as specified at ActiveMIL's installation) have been added to your project.
  9. You can set and verify any control's property settings interactively (that is, at design-time) by accessing the control's Property Pages through the Property Pages button in the Properties Window. You can also manipulate any control by altering the code itself (so that the control's property settings are set and verified at run-time).

To take advantage of creating ActiveMIL controls in your project at run-time, refer to the section Run-time object declaration in the Using ActiveMIL with Microsoft Visual C++ (Native COM) chapter.


Using the MFC Application template to create an application (Visual C++)

User Guide:
ActiveMIL/MIL Add-ins, Include path, and Library path
Creating an application (Visual C++)
Run-time object declaration
What is Native COM

To use the MFC Application template:

  1. Select New Project from the File menu command. The New Project dialog box is presented.
  2. Select Visual C++ Projects.
  3. From the list of available templates, select MFC Application.
  4. Type a project name in the Name edit field (for example, ActiveMILApplication).
  5. Specify the location in which to save your project, in the Location edit field.
  6. Choose one of the following (available only when a solution is open in the IDE):
    • Close Solution (closes the open solution and creates a new one for the new project).
    • Add to Solution (adds the new project to the open solution).
  7. Click Ok. A basic dialog is created; however, the ActiveMIL controls are not added. Your project's Resource View tab now contains your resources, ActiveMILApplication.rc. To access the dialog; click on the Dialog sub-folder of the ActiveMILApplication resource folder. Then, double-click on the dialog entry, IDD_ACTIVEMILAPPLICATION_DIALOG.

    8. Notice that your dialog does not contain any ActiveMIL controls. To add a control from the control Toolbox to your project, drag and drop it onto your dialog.

  8. You can add an ActiveMIL control to your project from the ActiveMIL tab in the toolbox. To do so drag and drop it onto your dialog. Notice that the path for ActiveMIL's include (*.h) and OCX (*.ocx) files (as specified at ActiveMIL's installation) have been added to your project. However, the path for ActiveMIL's include and OCX files are not automatically added to your application. Refer to the Adding ActiveMIL/MIL add-ins, include path, and library path (Visual C++) section later in this chapter. Moreover, to create a variable for your ActiveMIL control, you should use the classes included in the header files instead of creating new MFC wrapper classes (for example, CSystem for the ActiveMIL System).
  9. You can set and verify any control's property settings interactively (that is, at design-time) by accessing the control's Property Pages through the Property Pages button in the Properties Window. You can also manipulate any control by altering the code itself (so that the control's property settings are set and verified at run-time).

To take advantage of creating ActiveMIL controls in your project at run-time, refer to the section Run-time object declaration.


Using the Win32 Console Application (Visual C++)

User Guide:
Adding ActiveMIL/MIL add-ins, include path, and library path
Creating an application (Visual C++)
Run-time object declaration
What is Native COM

To create a console project template (with or without MFC):

  1. Select New Project from the File menu command. The New Project dialog box is presented.
  2. Select Visual C++ Projects.
  3. From the list of available templates, select Win32 Console Project.
  4. Type a project name in the Name edit field (for example, ActiveMILApplication).
  5. Specify the location in which to save your project, in the Location edit field.
  6. Choose one of the following (available only when a solution is open in the IDE):
    • Close Solution (closes the open solution and creates a new one for the new project).
    • Add to Solution (adds the new project to the open solution).
  7. Click Ok. You are now presented with a window that asks what type of application you would like to create.
  8. Choose your application settings, and then click Finish.
  9. Click Ok. A basic console application is created; however, the ActiveMIL controls have not been added.

    10. To take advantage of adding ActiveMIL controls to your project at run-time, refer to the Run-time object declaration section in the Using ActiveMIL with Microsoft Visual C++ (Native COM) chapter. Note that if you want to take advantage of accessing ActiveMIL's controls at design-time, you must add an MFC dialog to your project.


Adding ActiveMIL/MIL add-ins, include path, and library path (Visual C++)

See Also | Previous
User Guide:
Using the ActiveMIL MFC Application template to create an application (Visual C++)
Using the MFC Application template to create an application (Visual C++)
Using the Win32 Console Application (Visual C++)
What is Native COM

A Visual C++ project can use ActiveMIL's controls only if the correct path for ActiveMIL's include (*.h) and OCX (*.ocx) files are added to that project. How these paths are added depends partly on the technique used to create that project:

  • ActiveMIL MFC Application template.
  • MFC Application template.
  • Console project template.

ActiveMIL MFC Application template

If a project was created using the ActiveMIL MFC Application template, the paths for ActiveMIL's include and OCX files (as specified at ActiveMIL's installation) are automatically added.

MFC Application template and console project template

If a project was created using the MFC Application template, or using the console project template (with or without MFC), the path for ActiveMIL's include and OCX files can be added in one of the following ways:

  • Specify the location of ActiveMIL's files for all projects, as explained in the section, Specifying the location of ActiveMIL's include (*.h) and OCX (*.ocx) files for all projects.
  • Specify the location of ActiveMIL's files for one single project, as explained in the section, Specifying the location of ActiveMIL's include (*.h) and OCX (*.ocx) files for one single project.

Enable ActiveMIL's add-in settings in MSDEV

You need to enable the ActiveMIL add-ins to preserve the Toolbox 's ActiveMIL tab and have fast access to ActiveMIL help.

To enable the add-ins, follow these steps:

  1. Select Add-in Manager from the Tools menu command. The Add-in Manager dialog box is presented.
  2. Enable the ActiveMIL Add-ins Available Add-ins and ActiveMIL Add-ins Startup options.

Specifying the location of ActiveMIL's include (*.h) and OCX (*.ocx) files for all projects

The following steps describe how to access ActiveMIL's include and OCX files when creating console projects template (with or without using MFC). Note that these steps are not typically applicable to projects created with the ActiveMIL MFC Application template.

To specify the correct include-path (*.h files) for all projects:

  1. Select Options... from the Tools menu command. The Options dialog box is presented.
  2. Select the Projects VC++ Directories item from the Options dialog tree structured list.
  3. In the Show directories for drop-down list-box, select Include Files.
  4. Add the path to the Directories list as follows:
    1. Click on the folder icon to add a new entry to the Directories list.
    2. Click on the ... button to browse your directories, or enter the correct path, for example: C:\Program Files\Matrox Imaging\ActiveMIL\Containers\VStudio.NET\VC++\Include.

To specify the correct library-path (*.ocx files) for all projects:

  1. Select Options... from the Tools menu command. The Options dialog box is presented.
  2. Select the Projects VC++ Directories item from the Options dialog tree structured list.
  3. In the Show directories for drop-down list-box, select Library Files.
  4. Add the path to the Directories list as follows:
    1. Click on the folder icon to add a new entry to the Directories list.
    2. Click on the ... button to browse your directories, or enter the correct path, for example: C:\Program Files\Matrox Imaging\ActiveMIL\Controls.

Specifying the location of ActiveMIL's include (*.h) and OCX (*.ocx) files for one single project

The following steps describe how to access ActiveMIL's include and OCX files when creating console projects (with or without using MFC). Note that these steps are not typically applicable to projects created with the ActiveMIL MFC Application template.

To add the ActiveMIL include and OCX files to one single project, follow these steps:

  1. Open the project from which to access the ActiveMIL include and OCX files.
  2. Select ActiveMILApplication Properties from the Project menu command. The ActiveMILApplication Property Pages dialog box is presented.
  3. Select the Configuration Properties C/C++ General items.
  4. In the Additional Include Directories edit field, add the path of your include and OCX files, for example:
    • C:\Program Files\Matrox Imaging\ActiveMIL\Containers\VStudio.NET\VC++\Include
    • C:\Program Files\Matrox Imaging\ActiveMIL\Controls


Building an application

Chapter 2: Building an application

This chapter describes the steps required to build an application using ActiveMIL.

  • Setting up controls
    • Default settings
    • Controls and resources
    • Allocating and freeing resources
  • Property pages
  • Examples
  • Displaying an image - overview
  • Grabbing an image - overview
  • Error reporting (Visual Basic .NET and Visual C++ .NET)
    • Design-time
    • Run-time
  • Generating an executable program
  • Integrating MIL functions into your ActiveMIL code
    • Visual Basic .NET 2003
    • Visual C++ .NET 2003 (unmanaged)
  • Multiple systems


Setting up controls

See Also | Next
User Guide:
Creating an application (Visual C++)
Multiple systems

Reference:
Allocate
AutomaticAllocation
Free
OwnerSystem
PropertyValidation

When designing an ActiveMIL application, you will need:

  1. An Application control. This control is optional, but you are likely to need it.
  2. A System control.
  3. An Image control.
  4. Once you have the above controls, you can create the other controls (for example, the Digitizer, the Display, or the ImageProcessing control) in any order you like.

By default, when a control is dropped onto a form, it will attempt to link to other ActiveMIL controls (when appropriate). For example, when dropping an Image control onto a form, its OwnerSystem property will attempt to link the Image control to an existing System control on the form.

For more information, see also the multiple systems section.

Default settings

Controls are created such that their initial properties match your default settings, which are initially specified during installation. For example, if your default camera is color, images are created with three bands. Before you start to design your application, you might want to verify and/or change your default settings. To do so, use the Default Values tab of the MilConfig utility, (MILConfig.exe). This utility can be accessed from the Windows Start menu under the Matrox Imaging Tools folder or the \Matrox Imaging\Tools directory.

Note that if you change your default settings, this change does not affect the properties of controls that have already been created.

Controls and resources

ActiveMIL controls represent either a physical entity (for example, a digitizer) or an image processing concept (for example, a graphic context). Only when an ActiveMIL control allocates its resources does it make a link with the entity or concept that it represents, ensuring that the entity exists and is properly initialized; otherwise, a control is independent of what it represents. To illustrate this concept, consider ActiveMIL's System control. The System control represents a Matrox imaging board that is installed in your computer. Assume that instance A of the System control is configured to represent a Matrox Odyssey imaging board. Whether a Matrox Odyssey board exists in your system and is initialized is independent of the existence of instance A. Only when instance A allocates its resources will the Matrox Odyssey board be found and properly initialized. Only at this point will instance A be made aware of the presence (or absence) of the Matrox Odyssey board.

Allocating and freeing resources

By default, the resources associated with a control are automatically allocated when the application begins and automatically freed when the application terminates. To override automatic allocation of a control's resources, set its AutomaticAllocation property to False. In this case, you must explicitly allocate the control's resources using the Allocate method. The control's resources will still be automatically freed when all the instances are released (for example, when the form is closed), but can be freed beforehand, if required, using the Free method.

Note that, at design-time, the resources associated with a control are automatically allocated (if possible) when the control is dropped onto the form. This is done primarily to allow methods to be called at design-time, but also to allow live validation of the controls. The resources are automatically freed when the form is closed or when the application is run. To disable design-time validation, use the control's PropertyValidation property.


Property pages

Previous | Next

Most properties of a control can be set from its property pages. To access a control's property pages at design-time, choose Properties from the control's context menu.

To access a control's property pages at run-time, call the ShowPropertyPages method, which is included in each ActiveMIL control. This method will disable property page elements corresponding to properties that cannot be changed due to the current state of the control. For example, if a property cannot be changed while its control is in the allocated state, and the control is allocated, the corresponding element in the property page will be disabled. To change this property, free the control using the Free method first.

Property pages offer design-time validation and allow you to create graphical models, define search regions, and much more. Note that in most cases, you cannot access ActiveMIL control properties from the Visual Studio Properties Window. Therefore, it is recommended that you rely on the property pages to customize your ActiveMIL controls.


Examples

Previous | Next

To simplify concepts and help you get started quickly, various ActiveMIL examples have been provided. To run and edit these examples, you can use the example launcher, ExampleLauncher.exe, located in the \Matrox Imaging\Tools directory. Note that the example launcher gives you the option of running and editing its examples in either Visual Basic .NET 2003 or Visual C++ .NET 2003.

To better illustrate the numerous examples, the Language option in the online ActiveMIL documentation allows you to select the environment for which to show the examples (Visual Basic .NET 2003, Visual C++ .NET 2003, or both).

ActiveMIL's examples offer many advantages:

  • The MappStart example is very simple, but it allows you to quickly test ActiveMIL's installation and become familiar with the basics behind running an ActiveMIL application. The MAppStart example allocates an application, opens communication with the default system, displays a welcoming message, pauses, and then closes communication with the system. MappStart also demonstrates error handling capabilities.
  • The examples use your default settings which have been specified at installation; therefore, very little time must be spent configuring your programming environment. However, if at some point you want to change the default settings, or are experiencing problems running the examples, your default settings can be changed on the Default Values tab of the MilConfig utility, (MILConfig.exe). This utility is located in the \Matrox Imaging\Tools directory. You can also change the default settings by clicking the Change Defaults button in the example launcher, which will take you directly to the Default Values tab of the MilConfig utility.

ActiveMIL's examples serve as an ideal starting point for creating applications; it is therefore highly recommended that you become familiar with them. More often than not, a seemingly complex application becomes simplified when the appropriate example is used as the starting point. However, if you load an example and change a property value, the property value will be saved and the default value will be ignored in the future. It might therefore be wise to rename the examples before customizing them, thereby keeping the original intact.

Note that some systems might not be able to run some of the examples because they do not have the hardware capability or enough memory. These examples should be ignored or modified to suit your particular configuration.


Displaying an image - overview

User Guide:
Displaying non 8-bit images
The display window

Reference:
Display.BackColor
Display.Image
Display.OwnerSystem
Image.CanDisplay
Image.OwnerSystem

Whether or not your imaging board has a display section, ActiveMIL can display images. ActiveMIL will use the most appropriate graphics controller in your computer for display purposes. If your imaging board has a display section, and it is available, ActiveMIL will typically use it for display purposes.

You can display images with a windowed border (or on a form) on the Windows desktop screen(s); this is called a windowed display. Alternatively, you can display images without a windowed border on an auxiliary screen, hardware permitting; this is called an auxiliary display.

To display an image, you need a Display control, an Image control, and a System control. Specify the image to display using the Image property of the Display control. For an image to be displayed, its CanDisplay property must be set to True, which is the default setting. In addition, both the image and the display should be on the same system; that is, the OwnerSystem property of the Image control and the OwnerSystem property of the Display control should match.

The image is typically displayed from the top-left corner of the display. When the image is too large to be displayed at once in a windowed display, you can use scroll bars to view parts of the image that are not visible. When this is the case in an auxiliary display, use the Pan method.

In general, you can display any type of image in a display. However, when displaying non 8-bit images in an 8-bit display resolution, processing speed might be slower.

An image, or any of its child images, can be displayed on more than one display.

ActiveMIL documentation uses the term frame buffer to refer to physical display (graphics controller) memory (not a buffer, per se).


Grabbing an image - overview

Reference:
Digitizer.DeviceNumber
Digitizer.Format
Digitizer.Grab
Digitizer.GrabContinuous
Digitizer.Image
Digitizer.MultipleBuffering.Process
OwnerSystem

ActiveMIL supports input from any type of video source supported by the imaging board (frame grabber). Each frame grabber has one or more available acquisition paths. ActiveMIL refers to the acquisition paths used to grab from one video source as a digitizer.

Note, since most video sources are cameras, they will hereafter be referred to as such.

To grab an image (monochrome or color), you need a Digitizer control, a System control, and an Image control. The Image and Digitizer controls are created such that their initial properties match those of your default data format and device number settings. The data format and the device number of the digitizer define the acquisition paths that the digitizer represents and from which it will grab.

To grab a specific number of frames or fields, use the digitizer's Grab method. To grab continuously, use the digitizer's GrabContinuous method. The data will be grabbed into the image specified by the digitizer's Image property. The image should have as many bands as the data has color components. If you need to grab and process concurrently, you can use the MultipleBuffering.Process method of the Digitizer control.

The Image and Digitizer controls must be on the same system, that is, their OwnerSystem properties must match.

When you are done with your Digitizer control, System control, and/or Image control, you should free each control using the free method of the control.

When developing an application, it is recommended that you use a simple camera. Once the application is working, switch to a more sophisticated camera, if necessary. This approach makes debugging much easier.

For an example of grabbing an image, see MDigGrab.


Error reporting (Visual Basic .NET and Visual C++ .NET)

User Guide:
Error management (Visual C++ .NET)

ActiveMIL uses two error mechanisms, one at design-time and one at run-time.

Design-time

At design-time, the properties of each control are validated. Any errors appear in the Errors page of a control's property pages. If a control contains any errors, a red frame appears around the control.

To disable interactive property validation for a control, set its PropertyValidation property to False.

Run-time

At run-time, ActiveMIL uses trappable errors, or exceptions. Under both Visual Basic .NET and Visual C++ .NET, you would use a Try block to handle errors. The exception returned in a Visual Basic .NET Catch block is of type System.Runtime.InteropServices.COMException. While the exception returned in a Visual C++ .Net Catch block is of type COleDispatchException. Note that, if you do not use these mechanisms, any run-time error causes execution to stop and an error message to be displayed.

For an explanation on managing errors with Visual C++ (Native COM), see Error management using Visual C++.

For an example of how error reporting can be integrated into an application, see MAppStart.


Generating an executable program

User Guide:
Distribution of ActiveMIL applications

To generate an executable program in the Visual Studio .NET environment (for Visual Basic .NET 2003 and Visual C++ .NET 2003 (unmanaged - Native COM)), perform the following directive:

  • Select Build Solution from the Build menu command.

Note that once you are done debugging your application, you should compile in release mode to achieve maximum processing speed. You can switch between release and debug modes using the Standard toolbar.


Integrating MIL functions into your ActiveMIL code

User Guide:
Adding ActiveMIL/MIL add-ins, include path, and library path

Reference:
Control
Inquire
MilID
Modified

Since ActiveMIL contains all of the functionality of MIL, it should never be necessary to incorporate MIL functions in your ActiveMIL code. ActiveMIL does, however, allow this interaction for the rare circumstance where it might be useful. For example, in a few cases, some time-critical piece of code can be made to run faster using MIL functions. Another reason might be to use new MIL functions that you receive from Matrox Imaging in a quick fix DLL that is not yet included in ActiveMIL. This section explains how to integrate MIL functions into your ActiveMIL code, if required.

In order for MIL to access an ActiveMIL resource (for example, an ActiveMIL Image or GraphicContext control), it needs the resource's MIL identifier. To inquire about a resource's MIL identifier, use its MilID property. Note that some controls have more than one MilID property because the control contains more than one MIL object (for example, every marker of the Measurement control has its own MilID property).

After an ActiveMIL control has been modified by MIL, you must notify ActiveMIL, so that it can update the control's properties. To do so, use the control's Modified method.

For new constant settings available in a MIL module's M...Control() or M...Inquire() function, but not found in ActiveMIL, use the Control and Inquire methods of the module's corresponding ActiveMIL control, with the numerical value of the MIL constant found in mil.h.

If you start your application using ActiveMIL controls, you will not need to call MappAlloc() before using MIL functions; this is done automatically when the first ActiveMIL control is created. Then, when you free your last ActiveMIL control, the application is freed (the equivalent of calling MappFree()). This means that you can no longer call MIL functions.

You can also use ActiveMIL controls in your MIL application. In this case, you must free all of the ActiveMIL controls before freeing the MIL application (using the MappFree() function).

Note that within MIL code, you can mix board-specific code if necessary (if your Matrox imaging board has its own library). For example, if you have a Matrox Odyssey board, you can use functions from the Odyssey Native Library within your MIL code. To mix board-specific code within your MIL code, MIL must be signaled before and after the board-specific portion of the code. For more information about mixing board-specific code within your MIL code, see your MIL documentation.

Described below are some environment-specific details required to mix MIL code with ActiveMIL code.

Visual Basic .NET 2003

To integrate MIL code into ActiveMIL code when using Visual Basic .NET, you must first declare the required MIL functions using the DllImport attribute, as you would to access an API function. See your Visual Basic documentation for more information.

Note that you can still use the Declare statement but pay attention to data type correspondence. For example, an IntegerShortInteger (Short for Visual Basic .Net) in Visual Basic 6.0 is a Short in Visual Basic .NET.

Visual C++ .NET 2003 (unmanaged)

When using Visual C++, you must include the mil.h header file and you must add the \Matrox Imaging\ActiveMIL\NativeMIL\Include directory to your "include directories". In addition, you need to link with the mil.lib library and to any processing-specific libraries (for example, the milpat.lib library if you need functions from the pattern matching module). Library files are located under the \Matrox Imaging\ActiveMIL\NativeMIL\Libraries directory. For more information, see Adding ActiveMIL/MIL add-ins, include path, and library path.


Multiple systems

See Also | Previous
Reference:
Application.AvailableSystems

There is no limit to the number of systems an application can access; you are limited only by the physical devices present in your computer.

Any operation involving more than one system will be performed by the most appropriate one. By default, if none of these systems is more appropriate than the Host system, then the Host system is used to perform the operation.

If necessary, the AvailableSystems property of the Application control can be used to select the system to use at run-time. An example of this is outlined below.

[]
' Determine the system to use at run-time.
System1.SystemDescriptor = _
        Application1.AvailableSystems.Item(1).SystemDescriptor
System1.DeviceNumber = _
        Application1.AvailableSystems.Item(1).DeviceNumber
System1.Allocate()
[]
// Determine the system to use at run-time.
_variant_t Index(1L);
m_System1->SystemDescriptor =
   m_Application1->AvailableSystems->Item(Index)->SystemDescriptor;

m_System1->DeviceNumber =
   m_Application1->AvailableSystems->Item(Index)->DeviceNumber;

m_System1->Allocate();


Image processing

Chapter 3: Image processing

This chapter describes the steps to performing a typical application with the ActiveMIL ImageProcessing control.

  • Overview of image processing
  • Image processing operations
  • Steps to performing a typical application
    • A typical application
  • Image quality
    • Techniques to improve images
  • Averaging an input sequence
    • Frame averaging with ActiveMIL
  • Denoising
    • Low-pass spatial linear filters
    • Rank filters
  • Opening and closing
    • Removing small particles
    • Filling holes
  • Basic geometric transforms
    • Resizing an image
    • Rotating an image
    • Translating an image
    • Flipping an image
    • Interpolation
  • Image manipulation
  • Image statistics
    • Generating a histogram
    • Calculating general statistics
    • Finding the image extremes
    • Locating events
    • Counting image differences
    • Projecting an image to one dimension
  • Thresholding your image
    • Binarizing
      • Determining a threshold value from a histogram
    • Clipping
  • Histogram equalization
  • Enhancing and detecting edges
    • Edge enhancement
      • Sharpening edge enhancement
    • Edge detection
      • Horizontal and vertical edge detection
      • Laplacian-based edge detection
      • Gradient-based edge detection
  • Arithmetic with images
    • Combining images
    • Mapping an image
  • Performing color conversions
  • Distance transform
  • City block transform
  • Chessboard transform
  • Chamfer 3-4 transform
  • Labeling


Overview of image processing

See Also | Next
User Guide:
Images - overview

Reference:
ImageProcessing

To perform image processing on an image, you need an ImageProcessing control, a System control, and at least one Image control.

The ImageProcessing control's source properties specify the image(s) on which an image processing method is performed. The destination properties specify where to store the resulting image(s). Any statistical result is placed in the collection of Results.

The ImageProcessing control includes methods that allow you to perform point-to-point, statistical, spatial filtering, and morphological operations. The ImageProcessing control also includes methods and properties that allow you to create and set custom LUTs, kernels, and structuring elements.

Note that the ImageProcessing control and the source and destination images should be on the same system, that is, their OwnerSystem properties should match. However, if the source and destination images are not on the same owner system, ActiveMIL will compensate for this by copying one of the images onto the most appropriate owner system, and then performing the processing operation on the copied image.


Image processing operations

User Guide:
Advanced image manipulation
Basic geometric transforms
Connectivity mapping
Denoising
Enhancing and detecting edges
Erosion and dilation
Histogram equalization
Image quality
Image statistics
Matching
Opening and closing
Thresholding your image

Reference:
BlobAnalysis

The ImageProcessing control supports such operations as:

  • Point-to-point operations. These operations include constant thresholding, image comparison, image subtraction, and image mapping. They compute each pixel result as a function of the pixel value at a corresponding location in either one or two source images.
  • Statistical operations. These extract statistical information from a given image, such as the minimum or maximum image pixel value or a histogram. They condense a frame of pixels into a smaller, more functional set of values for analysis. The image processing control supports basic statistical operations. ActiveMIL has other controls for specialized analysis (for example, the BlobAnalysis control)
  • Spatial filtering operations. These operations are also known as convolutions. They include operations that can smooth images, enhance and detect image edges, and remove 'noise' from an image. Most of these operations compute results based on an underlying neighborhood process: the weighted sum of a pixel value and its neighbors' values.
  • Morphological operations. These operations include erosion, dilation, opening, and closing of images. They compute new values according to geometric relationships and matches to known patterns in the source image.


Steps to performing a typical application

User Guide:
Advanced image manipulation
Basic geometric transforms
Borders of an image
Image manipulation
Image processing operations
Image quality
Image statistics
Matching
Searching for hits and misses

Reference:
ImageProcessing.FindExtremes
ImageProcessing.Label
ImageProcessing.Open
ImageProcessing.Smooth

Image processing pertains to more than one field and no single application program can solve the problems associated with each one of these fields. Therefore, this section describes what we believe to be a typical problem, where the solution makes use of most of the supported operations. It also outlines the steps to take to implement this solution.

A typical application

In analyzing an image of a tissue sample, you might want to know the number of cell nuclei that are larger than a certain size, indicating an abnormality. This involves the following image processing steps:

  1. Grab or load an image of a magnified tissue sample.
  2. Smooth (using the Smooth method) the image to remove noise produced during the grab.
  3. Binarize (using the Smooth method) the image so that the cell nuclei or particles and the background have different values: represent particles in white and the background in black. This will allow you, later, to label each particle with a unique number.
  4. Perform an opening operation (using the Open method) to remove small particles from the image.
  5. Label (using the Label method each particle with a unique consecutive number starting with the label 1.
  6. Calculate and read the extreme value of the image (using the FindExtremes method). This value also corresponds to the largest label. Since the image particles are labeled with consecutive unique numbers, the largest valued particle is also labeled with a number that corresponds to the number of particles in the image.

The example MImageProcessing shows you how to encode these steps using an existing image of a tissue sample (cell.mim).


Image quality

User Guide:
Advanced image manipulation
Averaging an input sequence
Basic geometric transforms
Custom spatial filters
Denoising
Enhancing and detecting edges
Image manipulation
Opening and closing
Thresholding your image

Reference:
ImageProcessing.Resize

Prior to manipulating and extracting information from an image, many applications require that you obtain the best possible digital representation of it. Several factors affect the quality of an image. These include:

  • Random noise. There are two main types of random noise:
    • Gaussian noise. When this type of noise is present, the exact value of any given pixel is different for each grabbed image; this type of noise adds to or subtracts from the actual pixel value.
    • Salt-and-pepper noise (also known as impulse or shot noise). This type of noise introduces pixels of arbitrary values (usually high-frequency values) that are generally noticeable because they are completely unrelated to the neighboring pixels.

    Random noise can be caused, for example, by the camera or digitizer because electronic devices tend to generate a certain amount of noise. If the images were transmitted, the distance between the sending and the receiving devices also magnifies the random noise problem because of interference.

  • Systematic noise. Unlike random noise, this type of noise can be predicted, appearing as a group of pixels that should not be part of the actual image. This can be caused, for example, by the camera or digitizer or by uneven lighting. If the image was magnified, microscopic dust particles, on either the blob or a camera lens, can appear to be part of the image.
  • Distortions. Distortions appear as geometric transforms of the actual image. These can be caused, for example, by the position of the camera relative to the item (not perpendicular), the curvature in the optical lenses, or a non-unity aspect ratio of an acquisition device.

Techniques to improve images

Most interference problems cannot be adjusted very easily at the source; therefore, preprocessing will probably be required to improve the image as much as possible, without affecting the information that you are seeking. There are several techniques that you can use to improve your image:

  • Grab the object of interest several times, averaging each image frame with the previous. This technique is generally effective on Gaussian random noise.
  • Apply a low-pass spatial linear filter (Denoising) to your image to reduce Gaussian random noise and systematic noise with small scale variations. This technique replaces each pixel with a weighted sum of its neighborhood.
  • Apply a Rank filter (Denoising) to your image to reduce salt-and-pepper noise. This technique replaces each pixel with the median pixel value of its neighborhood.
  • Perform an opening operation to remove small particles and break isthmuses between blobs in your image.
  • Perform a closing operation to remove small holes in blobs.
  • Make sure that the type of camera you use digitizes the image with square pixels (that is, a 1:1 aspect ratio), to reduce blob-shape distortions. If this is not possible or does not correct the problem, you can resize the image, using the Resize method.

There are also several other more advanced methods, which are discussed in the advanced section, that you can use to improve image quality.


Averaging an input sequence

User Guide:
Advanced image manipulation
Arithmetic with images
Denoising
Double buffering
Image quality
Image statistics

Reference:
ImageProcessing.Add
ImageProcessing.Divide
ImageProcessing.WeightedAverage

An effective technique to remove random noise is to average a grabbed sequence of the same target image. For instance, Gaussian noise affects the value of any given pixel for each grabbed frame, adding to or subtracting from the actual pixel value. Therefore, over several image acquisitions, this noise averages out to zero. As a rule of thumb, Gaussian noise is generally reduced by the square root of the number of grabbed frames.

Frame averaging with ActiveMIL

With ActiveMIL, you can average an input sequence, using either one of the following methods:

  • Add all input frames and then divide the result by a specified weight factor (using the Add and Divide methods).
  • Add weighted input frames to a weighted accumulator image using the WeightedAverage method:

    • Iacc = aIin + (1 - a)Iacc or Iacc = a(Iin - Iacc) + Iacc

The latter approach also acts as a temporal filter if the input is changing. This allows you to filter out moving objects from a constant background.

If you do not want to lose any frames in your sequence, you can use a method called double buffering, a technique whereby you grab data into one image while another image is being processed.


Denoising

User Guide:
Averaging an input sequence
Custom spatial filters
Image quality

Reference:
ImageProcessing.Convolve
ImageProcessing.Rank
ImageProcessing.Smooth

Spatial filtering provides an effective method to reduce noise. Spatial filtering operations determine each pixel's value based on its neighborhood values. They allow images to be separated into high-frequency and low-frequency components. There are two main types of spatial filters that can remove noise: low-pass linear filters and rank filters.

Low-pass spatial linear filters

Low-pass spatial linear filters are effective in reducing Gaussian random noise (and high-frequency systematic noise), provided that the noise frequency is not too close to the spatial frequency of significant image data. These filters replace each pixel with a weighted sum of each pixel's neighborhood. Note, these filters have the side-effect of selectively smoothing your image and removing edge information.

ActiveMIL provides a method that applies a predefined low-pass linear filter (ImageProcessing.Smooth method) , which you will find satisfies most applications. If you want to control the degree of smoothness (strength of denoising) applied by the neighborhood operation, use the Deriche or Shen method, set its FilterSmoothness parameter to the required smoothness value, and set its FilterOperation parameter to impSmooth. The Deriche mehtod applies a Canny-Deriche filter, and the Shen method applies a Shen-Castan filter. For the Shen-Castan filter, the neighborhoods' influence decreases much faster as the distance from the central pixel increases, and it tends to produce more enhanced or sharpened edges, compared to the Canny-Deriche filter.

Earlier in the chapter, we looked at a cell-analysis application that determined the number of cell nuclei in a tissue sample. Since Gaussian noise is generally introduced when digitizing, we performed a smoothing operation prior to performing the analysis.

Rank filters

Rank-filter operations are more suitable for removing salt-and-pepper type noise since they replace each pixel with a pixel in its neighborhood rather than a weighted sum of its neighborhood. The weighted sum generally creates a blotchy effect around each noise pixel.

You can perform a rank-filter operation, using the Rank method. In most cases, it is best to use a rank that is half of the number of elements in the neighborhood. This effectively replaces each pixel with the median of the neighborhood and is therefore called a median filter. To perform a median filter, use the Rank method with impMedian. You will find that the median filter will most often suit your application needs.


Opening and closing

User Guide:
Advanced image manipulation
Custom morphological operations
Custom spatial filters
Denoising
Enhancing and detecting edges
Erosion and dilation
Erosion and dilation
Image quality
Thinning and thickening
Warping

Reference:
ImageProcessing.Close
ImageProcessing.Dilate
ImageProcessing.Erode
ImageProcessing.Open

Another way of improving the image might be, for example, to remove small particles that have been introduced by dust, or to remove holes in blobs. These tasks can generally be accomplished using the Open or Close methods.

Opening and closing operations determine each pixel's value according to its geometric relationship with neighborhood pixels, and as such are part of a larger group of operations known as morphological operations.

Removing small particles

Besides removing small particles, opening operations also break isthmuses or connections between touching blobs. The Open method performs a basic opening operation on 3 by 3 neighborhoods taking all neighborhood pixels into account.

Filling holes

Closing operations are very useful in filling holes in blobs; however in doing so, they also connect close blobs, as shown below:

The Close method performs a standard 3 by 3 closing operation taking all neighborhood pixels into account.

Note, opening and closing operations work best on binary images.

Since opening is the result of eroding and then dilating an image, and closing is the result of dilating and then eroding an image, you can also customize an opening or closing operation, by using customized structuring elements with the Erode and Dilate methods.


Basic geometric transforms

User Guide:
Advanced image manipulation
Image manipulation
Steps to performing a typical application
Thinning and thickening
Thresholding your image
Warping - Interpolation modes

Reference:
ImageProcessing.Flip
ImageProcessing.Resize
ImageProcessing.Rotate
ImageProcessing.Translate

Image distortions can affect application results. For example, in a medical application that analyzes blood cells, if the camera does not have a one-to-one aspect ratio and no correction is performed, the cells appear distorted and elongated, and incorrect interpretations might result. Rotating such an image causes even more serious blob distortion.

To resolve distortion problems, ActiveMIL offers basic, as well as advanced, geometric transforms. Since the advanced geometric transforms (polar-to-rectangular operations and warping) are slower than the basic geometric transforms, they should only be used when the required transform cannot be performed using a basic geometric method.

Resizing an image

The Resize method resizes an image along the horizontal and/or vertical axis. This can help resolve aspect-ratio problems. If both the horizontal and vertical resizing factors are set to the same value, this method can reduce or magnify an image to an appropriate size.

Rotating an image

In some instances, the orientation of an image can also cause erroneous conclusions. When a blob is rotated from its original position, you can realign it by the required angle, using the Rotate method.

Translating an image

The Translate method displaces an image by a specified number of pixels in the X- and/or Y-direction, with subpixel accuracy.

Flipping an image

The Flip method flips an image horizontally (left to right) or vertically (top to bottom). Note that flipping horizontally allows you to get a mirror copy of the original image.

Interpolation

Geometric operations are performed according to a specified interpolation mode.


Image manipulation

User Guide:
Advanced image manipulation
Arithmetic with images
Basic geometric transforms
Custom morphological operations
Distance transform
Enhancing and detecting edges
Histogram equalization
Image statistics
Thresholding your image

Once you have improved your image as much as possible, you are ready to start manipulating and extracting information from it. The image processing control offers you several image manipulation operations. Depending on your application, you will need to perform one operation before another in order to extract the required information.


Image statistics

User Guide:
Advanced image manipulation
Histogram equalization
Image manipulation
Thresholding your image

Reference:
ImageProcessing.CountDifferences
ImageProcessing.FindExtremes
ImageProcessing.Histogram
ImageProcessing.LocateEvents
ImageProcessing.Project
ImageProcessing.Results

Many applications need to obtain some type of image statistic to condense a frame of pixels into a smaller, more functional set of values for analysis. The statistic might be required to perform some subsequent operation and/or might be used to summarize the effect of some image operation. The ImageProcessing control offers a variety of methods to extract statistical information from an image. These methods allow you, for example, to:

  • Generate the intensity histogram of an image (Histogram method).
  • Calculate a number of general statistics for all pixels that satisfy a specified condition (CalculateStats method).
  • Find the minimum and maximum values of an image (FindExtremes method).
  • Find the location of certain pixel values (LocateEvents method).
  • Find the number of differences between two images (CountDifferences method).
  • Perform an image projection from two dimensions to one dimension (Project method).

You can access results for these methods using the ImageProcessing control's Results property.

Generating a histogram

A histogram is the intensity distribution of pixel values in an image and is generated by counting the number of times each pixel intensity occurs. This information is very useful for several applications. In particular, it is useful to select a threshold level when binarizing an image and to change the image intensity distribution when trying to increase the image contrast.

You can generate an image histogram, using the Histogram method. The MImageHistogram example demonstrates how to obtain the histogram of an image (see the result of this example).

Calculating general statistics

You can calculate various statistics for all pixels in an image that satisfy a specified condition, using the CalculateStats method. Setting the condition limits the number of pixels for which the statistics will be calculated. However, the condition can also be set to use all of the pixels in the image.

The types of statistics that can be calculated include the following:

  • Maximum pixel value.
  • Maximum absolute pixel value.
  • Mean pixel value.
  • Minimum pixel value.
  • Minimum absolute pixel value.
  • Number of pixels (that satisfy the condition).
  • Standard deviation value.
  • Sum of pixel values.
  • Sum of absolute pixel values.
  • Sum of squared pixel values.

Note that you must access the results of the CalculateStats method by using the ImageProcessing control's Results.Stats property.

Finding the image extremes

Besides using the CalculateStats method to find image extremes, you can find the minimum and maximum pixel values of your image with the FindExtremes method. Perhaps the most common use for finding the minimum and maximum image pixel values is to fine-tune the black and white reference levels of your digitizer, ensuring full-range digitization. Another use for finding the maximum image pixel values is to find the number of objects in a labeled image. If all objects in an image are labeled with unique consecutive values, using the Label method, the largest label value also corresponds to the number of objects in your image.

Locating events

Once you have established certain values of interest in an image, you can find the location of pixels that satisfy conditions based on these values, using the LocateEvents method. For example, you can use LocateEvents to find the location of all pixels in an image equal to the image's maximum pixel value.

Counting image differences

You can find the number of differences between two images, using the CountDifferences method.

Projecting an image to one dimension

The Project method projects an image into a one dimensional image, generated by adding all pixel values along each diagonal in the image at the specified angle. This projection is referred to as the pixel value density of each diagonal. The 90° projection of the image is known as the row profile, and the 0° projection is known as the column profile.

The Project method can perform both grayscale and binary image projections. On simple binary images, the projection is useful to detect blob locations


Thresholding your image

User Guide:
Advanced image manipulation
Histogram equalization
Image manipulation
Image quality

Reference:
ImageProcessing.Binarize
ImageProcessing.Clip

Thresholding images means reducing each pixel to a certain range of values. Some operations can be performed more efficiently on thresholded images. Images with full grayscale levels are useful for some tasks, but have redundant information for others. The ImageProcessing control provides two thresholding methods:

  • The Binarize method.
  • The Clip method.

Binarizing

A binarizing operation reduces an image to two grayscale values: 0 and the maximum value in the image (for example, 255 if the image is 8-bit). Binary images are useful when trying to identify geometric patterns and blobs in your image since they are not cluttered with shading information.

A binarizing operation is performed by comparing each pixel value in the image against one or two specified threshold values (for example, whether each pixel value is above one of the threshold values, or within the range of the two threshold values). Pixels that meet the specified condition are set to the maximum value in the image while other pixels are set to 0.

When using the Binarize method, it is important to select a threshold value that preserves the required information. For example, in the above image, an inappropriate threshold value might have changed fewer or more image pixels into background pixels, resulting in fewer or more particles than actually exist.

Determining a threshold value from a histogram

The Binarize method can automatically determine the threshold value from the source image's characteristics. Specifically, a histogram of the source image is internally generated, then the threshold value is set to the minimum value between the two most statistically important peaks in the histogram, on the assumption that these peaks represent the blob and the background. If the histogram contains only two peaks, the threshold value is set to the minimum value between these peaks. If the histogram contains more than two peaks, then the threshold value will typically be set between the two principal peaks, though exceptions exist for pure black and full saturation (0,255).

Clipping

Clipping changes the image data less dramatically than binarizing. It changes the data to include only the range of pixel values in which you are interested. The Clip method takes a condition with at most two threshold points and replaces only those pixels that meet the condition with given values. Pixels that do not meet the condition are unaffected.

This can be useful to change data from one data type to another. For example, if you have a 16-bit image, but most of the pixels are less than 256, you could clip the image into an 8-bit image, and set all the pixels that are too big to the largest possible value, that is, 255.


Histogram equalization

User Guide:
Advanced image manipulation
Image quality
Image statistics

Reference:
ImageProcessing.HistogramEqualize
ImageProcessing.LUTMap

A histogram equalization can be performed to obtain a more uniform distribution of the grayscale values in your image. For example, if the intensity distribution of an image results in a clump in one area of the grayscale range, there might be blobs that are not easily distinguished because of their similarity in color. You might want to adjust the image's intensity distribution to solve this problem by giving it a more uniform distribution, using the HistogramEqualize method.

The HistogramEqualize method first generates a histogram of the source image. The histogram and a selected density function are then used to calculate a transformation LUT.

The transformation LUT is usually applied to the source image, and the result is placed in the destination image. Alternatively, the specified transformation LUT can also be generated and placed into the collection of LUTs available to the ImageProcessing control. This LUT can then be used to equalize several images, using the LUTMap method.

The transformation LUT can also be applied directly to images as they are being grabbed; to do so, get the value of the LUT (using the LUTs.Item(LutIndex).Get method) and put the values in the digitizer LUT, using the Digitizer.LUT.Put method.


Enhancing and detecting edges

User Guide:
Borders of an image
Histogram equalization
Image statistics

Reference:
ImageProcessing.Convolve
ImageProcessing.EdgeDetect1
ImageProcessing.EdgeDetect2
ImageProcessing.HorizontalEdge
ImageProcessing.LaplacianEdge1
ImageProcessing.LaplacianEdge2
ImageProcessing.Sharpen1
ImageProcessing.Sharpen2
ImageProcessing.VerticalEdge

Many applications perform various edge operations on the images to increase the quality of the image or to limit some other operation on the image.

In general, edges can be established from intensity transitions between two or more adjacent pixels in an image. Horizontal edges are created when horizontally connected pixels have values that are different from those immediately above or below them. Vertical edges are created when vertically connected pixels have values that are different from those immediately to the left or right of them. Oblique edges are created from a combination of horizontal and vertical components.

There are three main categories of edge operations:

  • Operations that enhance edges to sharpen the image.
  • Operations that detect edges in the image.
  • Operations that extract edges in the image.

These edge operations are performed using convolutions (or neighborhood operations that replace each pixel with a weighted sum of each pixel's neighborhood). The weights applied to the neighborhood determine the type of operation that is performed. For example, certain weights produce a horizontal edge detection, while others produce a vertical one.

For the first two types of edge operations, the weights are specified using either a predefined or a custom FIR or IIR filter. For more information on creating custom filters, see the Custom spatial filters section, in the Advanced image processing chapter. ActiveMIL also offers predefined FIR filters (see sub-sections below) for most common edge enhancement and detection operations. Each offers some advantage over the others. You can try these filters to see which best suits your application needs. If you want to control the degree of smoothness (strength of denosing) applied by the neighborhood operations, use a predefined IIR filter (either the Deriche or Shen method of the ImageProcessing control), and set its FilterSmoothness parameter to the required smoothness value. The Deriche method applies a Canny-Deriche filter, and the Shen method applies a Shen-Castan filter. For the Shen-Castan filter, the neighborhood's influence decreases much faster as the distance from the central pixel increases, and it tends to produce more enhanced or sharpened edges, compared to the Canny-Deriche filter.

For information on edge extraction operations, see the Edge Finder chapter.

Edge enhancement

Edge enhancement operations amplify edges, accentuating details in the image. Note that these operations might not produce good results for further processing because when you enhance edges, you also enhance noise pixels.

Sharpening edge enhancement

Sharpening edges can be useful to enhance edges in the image while preserving other parts of the image. You can obtain approximately the same result by performing a Laplacian-based edge detection operation on the image and adding the found edges to the original image.

To sharpen edges in an image, you can apply a predefined FIR filter using the Sharpen1 or Sharpen2 method. To apply a predefined IIR filter, you can use the Deriche or Shen method, and set its FilterOperation parameter to impSharpen.

When using predefined FIR filters, the Sharpen1 method tends to produce more enhanced or sharpened edges than the Sharpen2 mehtod.

Edge detection

Edge detection operations reveal intensity transitions in the image. The smoother the image, the more gradual the change in intensity, and the weaker the detection will be.

Horizontal and vertical edge detection

Detecting the horizontal and vertical edges in the image can be useful to enhance edges in a certain direction and remove those in another.

To detect the horizontal or vertical edges in an image, you can apply a predefined FIR filter with the HorizontalEdge or VerticalEdge method, respectively. To apply a predefined IIR filter, you can use the Deriche or Shen method, and set its FilterOperation parameter to impHorizontalEdge or impVerticalEdge.

Laplacian-based edge detection

The Laplacian-based operations place emphasis on the maximum values, or peaks, within the image. The edge representation of the resulting image generally looks very similar to the actual image.

To detect the Laplacian-based edges from an image, you can apply a predefined FIR filter with the LaplacianEdge1 or LaplacianEdge2 method. To apply a predefined IIR filter, you can use the Deriche or Shen method, and set its FilterOperation parameter to impLaplacianEdge.

When using predefined FIR filters, the LaplacianEdge2 method tends to produce more enhanced or sharpened edges than the LaplacianEdge1 method.

Gradient-based edge detection

When performing a gradient-based edge detection operation, edges are determined from the rate of change between pixel values in the image, without regard to the direction of the edges. The resulting image contains only positive values.

To detect the gradient-based edges from an image, you can apply a predefined FIR filter with the EdgeDetect1 or EdgeDetect2 method. To apply a predefined IIR filter, you can use the Deriche or Shen method, and set its FilterOperation parameter to impEdgeDetect or impEdgeDetectSquare.

For an advanced gradient-based edge detection operation, you can use the EdgeDetect method. This method allows you to specify the threshold value and produces a gradient intensity image and/or a gradient angle image.


Arithmetic with images

User Guide:
Averaging an input sequence
Image manipulation

Reference:
ImageProcessing.Add
ImageProcessing.And
ImageProcessing.Divide
ImageProcessing.Multiply
ImageProcessing.Nand
ImageProcessing.Subtract

It is often very useful to perform arithmetic operations on images. These operations apply the specified operator on individual pixel values in a source image or on pixels at corresponding locations in two source images. These operations are known as point-to-point operations because results do not depend on neighboring values.

Besides arithmetic operations, the ImageProcessing control includes several other point-to-point operations: logical, comparative, shifting, and absolute value operations.

Combining images

You can apply most of the above point-to-point operations, using the following arithmetic operator methods:

  • You can add, subtract, multiply, divide, AND, NAND, OR, XOR, NOR, or XNOR two images or an image and a constant, using the Add, Subtract, Multiply, Divide, And, Nand, Or, Xor, Nor, and Xnor methods, respectively.
  • You can NOT, negate, take the absolute value of an image, using the Not, Negate, AbsoluteValue methods, respectively
  • You can copy a constant to the entire image.

Mapping an image

You can perform complex operations (such as scaling and logarithms) on an image, using the LUTMap method. This method maps the source image through a specified lookup table (LUT) and stores results in the specified destination image.


Performing color conversions

Reference:
ImageProcessing.Convert

You can convert between color spaces using the Convert method of the Image Prcoessing control. For example, you can convert RGB to HLS values. For efficiency when converting from a 3-band (RGB) image, you can calculate just the hue component of the HLS color space into a 1-band image. You can also use this method to extract the luminance (intensity) from an RGB image, or to copy the luminance component of an image into a 3-band image, to create a monochromatic (gray) RGB image.

In the HLS color space (also known as HSL), color is represented as a combination of hue, luminance, and saturation. The hue corresponds to the wavelength of the main color. It is represented as an angular position on a circular color disk. The luminance corresponds to the brightness of the color, while the saturation can be thought of as the measure of color purity or concentration.

The HLS color space is similar to the human way of describing colors. Each color has its own hue value (such as red, orange, or green). Once the hue value is chosen, changes to the luminance or the saturation alter only the color quality, not the basic color.

When converting, the Convert method stores the hue component (H) in band 0, the luminance component (L) in band 2, and the saturation component (S) in band 1.

The following code is the algorithm that the Convert method uses when converting between the RGB and HLS color spaces.

#define min(a, b) (((a) < (b)) ? (a) : (b))
#define max(a, b) (((a) > (b)) ? (a) : (b))

// Input and output between 0 and 1.
void RGBToHSL(float R, float G, float B, float* H, float* S, float* L)
   {
   float MinVal, MaxVal, Delta;

   // Min and max values.
   MaxVal = max(R, max(G, B));
   MinVal = min(R, min(G, B));

   Delta = MaxVal - MinVal;

   // Compute the luminance.
   *L = 0.5f * (MaxVal + MinVal);

   if (Delta == 0.0f)
      {
      *S = *H = 0.0f;
      }
   else
      {
      // Compute the saturation.
      if (*L <= 0.5f)
         {
         *S = Delta / (MaxVal + MinVal);
         }
      else
         {
         *S = Delta / (2.0f - MaxVal - MinVal);
         }

      // Compute the hue.
      if (MaxVal == R)
         {
         *H = 60.0f * ((G - B) / Delta);
         }
      else if (MaxVal == G)
         {
         *H = 60.0f * (2.0f + ((B - R) / Delta));
         }
      else
         {
         *H = 60.0f * (4.0f + ((R - G) / Delta));
         }

      if (*H < 0.0f)
         {
         *H += 360.0f;
         }
      }

   // Remap the angle between 0 and 1.
   *H = *H / 360.0f;
   }

float HueToRGB(float Temp1, float Temp2, float Hue)
   {
   if (Hue > 360.0f)
      {
      Hue = Hue - 360.0f;
      }
   else if (Hue < 0.0f)
      {
      Hue = Hue + 360.0f;
      }

   if (Hue < 60.0f)
      {
      return (Temp1 + (Temp2 - Temp1) * Hue / 60.0f);
      }
   else if (Hue < 180.0f)
      {
      return Temp2;
      }
   else if (Hue < 240.0f)
      {
      return (Temp1 + (Temp2 - Temp1) * (240.0f - Hue) / 60.0f);
      }

   return Temp1;
   }

// Input and output between 0 and 1.
void HSLToRGB(float H, float S, float L, float* R, float* G, float* B)
   {
   float Temp1, Temp2;

   // Remap the angle between 0 and 360.
   H = H * 360.0f;

   // Achromatic case.
   if (S == 0.0f)
      {
      *R = *G = *B = L;
      }
   else
      {
      if (L <= 0.5f)
         {
         Temp2 = L * (1.0f + S);
         }
      else
         {
         Temp2 = L + S - (L * S);
         }

      Temp1 = 2.0f * L - Temp2;

      *R = HueToRGB(Temp1, Temp2, H + 120.0f);
      *G = HueToRGB(Temp1, Temp2, H);
      *B = HueToRGB(Temp1, Temp2, H - 120.0f);
      }
   }

For more technical information, refer to Computer Graphics: Principles and Practice in C, James D. Foley et al., Addison-Wesley Publishing Company, United States, 1995.


Distance transform

User Guide:
Chamfer 3-4 transform
Chessboard transform
City block transform

You can produce a distance transform using the Distance method of the ImageProcessing control. This method determines the minimum distance from each foreground (non-zero) pixel to a background (zero) pixel, and assigns this distance to the foreground pixel. It produces a type of contour mapping of an image's foreground (object) pixels.

You can use the Distance method to calculate the minimum distance using one of three distance transforms (the ActiveMIL distance transform constant appears in parentheses):

  • The City Block transform (impCityBlock)
  • The Chessboard transform (impChessboard)
  • The Chamfer 3-4 transform (impChamfer34)


City block transform

User Guide:
Chamfer 3-4 transform
Chessboard transform
Distance transform

The City Block transform (impCityBlock) determines the minimum distance using only horizontal or vertical steps. Each step counts as 1.

This distance transform is available when using the Distance method of the ImageProcessing control.


Chessboard transform

User Guide:
Chamfer 3-4 transform
City block transform
Distance transform

The Chessboard transform (impChessboard) determines the minimum distance using horizontal, vertical, or diagonal steps. Each step counts as 1.

This distance transform is available when using the Distance method of the ImageProcessing control.


Chamfer 3-4 transform

User Guide:
Chessboard transform
City block transform
Distance transform

The Chamfer 3-4 transform (impChamfer34), like the Chessboard transform, determines the minimum distance using horizontal, vertical, or diagonal steps. However, horizontal and vertical steps are counted as 3 and diagonal steps as 4. This allows the transform to better approximate the true (Euclidean) distance between two pixels. However, it requires that the destination image be large enough to hold a number at least three times the maximum distance from a foreground to a background pixel. For example, an 8-bit image (255 max) can be used for a maximum distance of 85 pixels and a 16-bit image (65535 max) for a maximum distance of 21845 pixels.

This distance transform is available when using the Distance method of the ImageProcessing control.


Labeling

See Also | Previous
User Guide:
Connectivity mapping
Custom morphological operations
Distance transform
Histogram equalization
Thresholding your image
Watershed transformations

Reference:
ImageProcessing.Clip
ImageProcessing.Label
ImageProcessing.Watershed

You can label blobs in an image with the ImageProcessing control's Label method. Labeling is useful for several operations:

  • Identifying and distinguishing blobs.
  • Finding the area of a blob. Once a blob is labeled, you find the area by generating a histogram and noting the number of pixels associated with that label value.
  • Counting the number of blobs in the image. The label number assigned to the last blob is also the number of blobs in the image (assuming there are fewer blobs than possible labels).
  • Using the result as a source for a conditional copy to eliminate some blobs (Clip method).

The Label method numerically identifies each blob in the specified image. Each non-zero pixel within a blob is given the same numerical value, and blobs within an image are given consecutive values.

You can specify that the operation is performed using one of two types of connectivity modes:

  • imp4Connected: If two pixels touch on the vertical or horizontal, they are considered part of the same blob.
  • imp8Connected: If two pixels touch on the vertical, horizontal, or diagonal, they are considered part of the same blob.

To distinguish between touching blobs, separate the blobs using the Watershed method before the labeling operation.


Advanced Image processing

Chapter 4: Advanced Image processing

This chapter describes the steps to performing advanced operations with the ActiveMIL ImageProcessing control.

  • Advanced image manipulation
  • Custom spatial filters
    • Finite Impulse Response (FIR) filters
    • Infinite Impulse Response (IIR) filters
    • A summary
    • An example
  • Custom morphological operations
  • Borders of an image
  • Erosion and dilation
  • Thinning and thickening
  • Matching
  • Searching for hits and misses
  • Connectivity mapping
  • Fast Fourier transform
    • Magnitude and phase
    • Filtering an image
  • Watershed transformations
    • Using watersheds to separate touching blobs
    • Using watersheds to separate blobs from their background
    • Minimum grayscale variation of a catchment basin
    • Using marker images
      • Non-labeled marker images
      • Labeled marker images
    • Style of the watershed lines
    • Exact versus straight
    • Skipping the last level
    • Filling the source
  • Polar-to-rectangular and rectangular-to-polar transforms
  • Warping
    • Using coefficients
      • First-order polynomial warpings
      • Perspective warping
      • Clearing coefficients
    • Using LUTs to perform a warping
    • Interpolation modes
    • Points outside the source image
  • Discrete cosine transform


Advanced image manipulation

See Also | Next
User Guide:
Custom morphological operations
Fast Fourier transform
Polar-to-rectangular and rectangular-to-polar transform
Steps to performing a typical application
Thinning and thickening
Warping
Watershed transformations

Besides the methods discussed in the Image processing chapter, ActiveMIL contains more advanced image processing methods. The advanced methods are descibed in this chapter, among other things, allow you to remove noise, separate objects from their background, and correct image distortions. They include neighborhood operations using custom structuring elements or kernels, frequency transforms (Fast Fourier transform, Polar-to-rectangular) transforms, watershed, and warping.


Custom spatial filters

User Guide:
Advanced image manipulation
Arithmetic with images
Borders of an image
Connectivity mapping
Custom spatial filters
Denoising
Image manipulation
Image statistics
Thinning and thickening
Thresholding your image

Reference:
ImageProcessing.Convolve
ImageProcessing.Kernels
ImageProcessing.Kernels.Item.AbsoluteValue
ImageProcessing.Kernels.Item.CenterX
ImageProcessing.Kernels.Item.CenterY
ImageProcessing.Kernels.Item.DataDepth
ImageProcessing.Kernels.Item.DataType
ImageProcessing.Kernels.Item.NormalizationFactor
ImageProcessing.Kernels.Item.OverscanType
ImageProcessing.Kernels.Item.OverscanValue
ImageProcessing.Kernels.Item.Saturation
ImageProcessing.Kernels.Item.SizeX
ImageProcessing.Kernels.Item.SizeY

Spatial filtering operations include operations such as smoothing, denoising, edge enhancement, and edge detection.

Spatial filtering operations compute results based on an underlying neighborhood process: the weighted sum of a pixel value and its neighbors' values. There are two types of spatial filters: Finite Impulse Response (FIR) filters and Infinite Impuse Response (IIR) filters. FIR filters operate on a finite neighborhood, while IIR filters take into account all values in an image. In ActiveMIL, you can specify either a predefined or a custom FIR or IIR spatial filter.

Finite Impulse Response (FIR) filters

For FIR filters, the weights are known as the kernel values. These kernel values determine the operation type of the spatial filter. For example, applying the following FIR filter results in a sharpening of the image:

Whereas, applying the following FIR filter smooths an image (it also increases the intensity of the image by a factor of 16, so you will need to normalize the convolution result):

When using FIR filters, the dimensions of the kernel determine the size of the neighborhood that is used in the operation. This operation is performed on the image specified by the Source1 property of the ImageProcessing control, and the result of the operation is stored in the image specified by the Destination1 property at the location corresponding to the kernel's center pixel. When the kernel has an even number of rows and/or columns, the center pixel is considered to be the top-left pixel of the central elements in the neighborhood.

Calculate the X-coordinate of the top-left pixel of the central elements, as follows:

  • If the width (SizeX) of the kernel is an odd number, the X-coordinate is (SizeX-1)/2.
  • If the width (SixeX) of the kernel is an even number, the X-coordinate is (SizeX/2)-1.

To calculate the Y-coordinate of the top-left pixel of the central elements, the same rules apply.

Regardless of the location of the center pixel, there will be some border pixels that have an incomplete neighborhood. To deal with this issue, the image is overscanned. There are several types of overscan. A transparent overscan uses the parent image to provide the overscan pixels needed for the border calculation. If the parent image is not available, a mirror overscan is performed. A mirror overscan specifies that the overscan pixels will be a mirror copy of the source image's border pixels. A replacement overscan allows you to specify a specific value for the overscan pixel values during processing. For more information on overscan, see the Buffer Overscan Region section in the Chapter 13: Images chapter.

If the predefined filters provided by the ImageProcessing control's methods do not meet your requirements, you can create your own custom FIR filters using the properties of the Kernels property. To define your own custom FIR filter:

  1. Add a new element to the collection of kernels (filters), using the Kernels.Add method. The dimensions (SizeX and SizeY properties), data type (DataType), and depth (DataDepth) of the kernel are specified when adding the kernel. Note that the dimensions of the kernel can be constrained by the available resources.
  2. Copy the required data from a user-supplied array into the kernel, using the Kernels.Item(KernelIndex).Put method.
  3. If required, modify the attributes of the kernel associated with your custom filter, using the Kernels.Item property and its subproperties. These properties determine how the convolution operation will be handled. You can control:
    • Whether or not the absolute value of the result is taken (AbsoluteValue property).
    • The division (normalization) factor to apply to the result (NormalizationFactor property).
    • Whether or not to saturate the result (Saturation property).
    • The position of the center pixel (CenterX and CenterY properties).
    • The data type and depth (DataType and DataDepth properties).
    • The dimensions of the kernel (SizeX and SizeY properties).
    • How the operation handles the borders (overscan) of the source iamge (OverscanType and OverscanValue properties). If overscan is disabled, the border pixels of the source image are not processed if additional processing time is needed.

To apply your custom FIR filter, call the Convolve method, specifying the index that corresponds to the required kernel.

Infinite Impulse Response (IIR) filters

When using IIR filters, the weights are automatically determined by the type of filter, the mode of the filter, the type of operation to perform, and the degree of smoothness (strength of denoising) applied by the filter.

The type of filter determines the distribution of the neighborhoods' influence. ActiveMIL supports two types of IIR spatial filters: Canny-Deriche filter and Shen-Castan filter. For the Canny-Deriche filter, the neighborhoods' influence decreases much slower as the distance from the central pixel increases, compared to the Shen-Castan filter. See the Customizing the extraction settings section in Chapter 9: Edge Finder for more details.

There are two modes in which to perform the filter: kernel mode and recursive mode. When performing an IIR filter in recursive mode, the kernel size would be theoretically infinite if this mode actually used a kernel. When performing a custom IIR filter in kernel mode, the filtering is done using a kernel approximation (FIR) of the filter. Kernel mode is not available for a predefined IIR filter. Kernel mode has been provided to take advantage of your system if it is optimized for kernel mode; kernel mode is typically not as efficient as recursive mode unless your system is optimized for kernel mode.

If the predefined IIR filters provided by the Deriche and Shen methods do not meet your requirements, you can create your own IIR filter. To define your own IIR filter:

  1. Add a new element to the collection of kernels (filters), using the Kernels.Add method.
  2. Use the Kernels.Item(KernelIndex).FilterType, FilterSmoothness, and FilterOperation properties to specify appropriate filter settings.

    3. Note that the size specified when adding the element is ignored, unless you will be performing the filter in kernel mode. In which case, the specified kernel size can be constrained by the available resources. In addition, regardless of the filter mode, the kernel values specified in the kernel (filter) are ignored; they are determined automatically.

  3. If required, modify the attributes of your custom filter, using the Kernels.Item property and its sub-properties. These properties determine how the convolution operation will be handled. You can control:
    • Whether or not the absolute value of the result is taken (AbsoluteValue property).
    • The division (normalization) factor to apply to the result (NormalizationFactor property).
    • Whether or not to saturate the result (Saturation property).

To use your custom IIR filter, call the Convolve method, specifying the index that corresponds to the kernel you want to use.

A summary

The following is a summary of spatial filtering options.

An example

For an example of a custom spatial filtering operation that uses a custom 3 by 3 kernel, see MImageConvolve.


Custom morphological operations

User Guide:
Borders of an image
Erosion and dilation
Matching
Searching for hits and misses
Thinning and thickening

Reference:
ImageProcessing.Dilate
ImageProcessing.Erode
ImageProcessing.HitOrMiss
ImageProcessing.Kernels.Item.CenterX
ImageProcessing.Kernels.Item.CenterY
ImageProcessing.Kernels.Item.DataDepth
ImageProcessing.Kernels.Item.DataType
ImageProcessing.Kernels.Item.OverscanType
ImageProcessing.Kernels.Item.OverscanValue
ImageProcessing.Kernels.Item.SizeX
ImageProcessing.Kernels.Item.SizeY
ImageProcessing.Match
ImageProcessing.StructuringElements
ImageProcessing.StructuringElements.Item.Data
ImageProcessing.Thick
ImageProcessing.Thin

Morphological operations are neighborhood operations that compute new values according to geometric relationships and matches of known patterns in the input image. The following ImageProcessing control methods perform morphological operations using a custom structuring element or a predefined structuring element:

  • Erode method.
  • Dilate method.
  • Thin method.
  • Thick method.
  • Match method.
  • HitOrMiss method.

If the predefined structuring elements used by these ImageProcessing control methods do not meet your requirements, you can create custom structuring elements using the properties of the StructuringElements property. These properties allow you to set the following attributes for your structuring element:

  • How to process the borders of an image (OverscanType and OverscanValue properties).
  • The position of the center pixel (CenterX and CenterY properties).
  • The data type and depth (DataType and DataDepth properties).
  • The dimensions of the structuring element (SizeX and SizeY properties).
  • The values to store at the given position of the structuring element (Data(X,Y) property)

    • For binary and some grayscale operations (Thin or Thick method), the structuring element values must be 0,1, or a capital 'D' (impDontCare). A capital 'D' means that the corresponding neighbors are not considered in the comparison. For other grayscale operations, any structuring element value can be used, including a 'D'.

After setting these properties, you can call any of the above mentioned methods specifying the index that corresponds to the required structuring element.


Borders of an image

User Guide:
Connectivity mapping
Custom morphological operations
Erosion and dilation
Opening and closing
Thinning and thickening

Reference:
ImageProcessing.Kernels.Item.OverscanType
ImageProcessing.Kernels.Item.OverscanValue

When you create custom structuring elements or kernels, you can specify how the borders of an image are processed. If the structuring element's or kernel's OverscanType property is set to impDisableOverscanType, the bordering pixels of the source image are not processed if additional processing time is needed. For example, if you are using a 3 by 3 kernel with a default center pixel, and overscan is disabled, the pixels on the borders of the source image are not processed if processing time can be saved.

To process the bordering pixels, specify the type of overscan to use with the OverscanType property. A transparent overscan (impTransparent) uses the parent image to provide the overscan pixels needed for the border calculation. Note, if the parent image is not available, a mirror overscan (impMirror) is performed. A mirror overscan specifies that the overscan pixels will be a mirror copy of the source image's bordering pixels. A replacement (impReplace) overscan allows you to specify a specific value (using the OverscanValue property) to use for the overscan pixel values during processing.


Erosion and dilation

User Guide:
Borders of an image
Connectivity mapping

Reference:
BlobAnalysis.Reconstruct
ImageProcessing.Dilate
ImageProcessing.Erode
ImageProcessing.StructuringElements

Especially during cell analysis, it might be important to know the growth stages of cell particles. Using the erosion and dilation operations, you can view the possible growth stages of these particles.

  • Erosion operations peel off layers from blobs, removing extraneous pixels and small particles from the image.
  • Dilation operations add layers to blobs, enlarging any particle. Dilation can return eroded particles to their original size (but not necessarily to their exact original shape).

Erosion and dilation are neighborhood operations that determine each pixel's value according to its geometric relationship with neighborhood pixels, and as such, are part of a group of operations known as morphological operations. You can create your own erosion or dilation operation by customizing the structuring element (using the StructuringElements property) or you can use ActiveMIL's predefined structuring elements.

There are two versions of erosion (Erode method):

  • Binary erosion (impBinary): If the structuring element does not match the corresponding neighborhood values exactly, the center pixel is set to zero; otherwise, it remains unchanged. In effect, binary erosion peels off layers of blobs.
  • Grayscale erosion (impGrayscale): Subtracts each structuring element value from the corresponding pixel value in the neighborhood, and then replaces the center pixel of the neighborhood with the minimum value from the resulting neighborhood values.

There are two versions of dilation (Dilate method):

  • Binary dilation (impBinary): If any of the structuring element values match the corresponding neighborhood values, the center pixel is set to the maximum value of the image (e.g. 255 for an 8-bit image); otherwise, it remains unchanged. In effect, binary dilation adds layers to the blobs.
  • Grayscale dilation (impGrayscale): Adds each structuring element value to the corresponding pixel value in the neighborhood, and then replaces the center pixel of the neighborhood with the maximum value from the resulting neighborhood values.

Note, in binary mode, erosion of the white pixels is the same as dilation of the black pixels.

If the processing mode is set to impBinary, a binary erosion or dilation is performed and all non-zero pixels are considered as 1's; otherwise, the grayscale version of these operations is performed.

You can also use the predefined structuring elements to perform the most standard erosion and dilation operations. These operations use the following structuring element when operating in binary mode:

They use the following structuring element in grayscale mode:

In other words, these commands execute a simple 3 by 3 minimum or maximum operation without adding or subtracting anything from the pixel.

You can set the number of iterations parameter of the Erode and Dilate methods to perform an erosion or dilation, respectively, on larger neighborhoods. Iterating the operation is equivalent to performing the operation on a (1 + (2*i)) by (1 + (2*i)) neighborhood where i is the number of iterations. For example, two iterations of a 3x3 erosion is equivalent to a 5x5 erosion, and three iterations is equivalent to a 7x7 erosion.

To perform conditional dilation you can use the Reconstruct method of the BlobAnalysis control.


Thinning and thickening

User Guide:
Borders of an image
Connectivity mapping

Reference:
ImageProcessing.Thick
ImageProcessing.Thin

You can reduce or enlarge blobs in the foreground (non-zero pixels) of an image, using operations based on a rigid match of the pixel's neighborhood and a structuring element. Using a thickening operation, you can enlarge the blob and perform such operations as a convex hull. Using a thinning operation, you can reduce blobs and perform such operations as finding their skeleton.

You can perform a thinning or thickening operation with a custom structuring element using the Thick or Thin methods, respectively. These operations are typically performed several times, using a different structuring element so that the required pattern is sought in different directions. You can also perform standard thinning or thickening operations with these methods.

There are two versions of thinning (Thin method):

  • Binary thinning (impBinary). This operation replaces the center pixel by the value zero if a pixel's neighborhood matches the structuring element exactly. However, if the neighborhood does not match, the pixel value remains unchanged.
  • Grayscale thinning (impGrayscale). This operation uses two values in the pixel's neighborhood to determine the value of the center pixel. It uses the maximum of all pixel values that correspond to zero in the structuring element, MAX(0), and the minimum of all pixel values that correspond to one in the structuring element, MIN(1). If the value of the center pixel is greater than MAX(0) and less than or equal to MIN(1), then the value of the center pixel is set to MAX(0), otherwise the value of the center pixel remains unchanged.

There are two versions of thickening (Thick method):

  • Binary thickening (impBinary). This operation replaces the center pixel by the maximum possible value of the image (for example, 0xff for an 8-bit image) if the pixel's neighborhood matches the structuring element exactly. However, if the neighborhood does not match, the pixel value remains unchanged.
  • Grayscale thickening (impGrayscale). This operation uses two values in the pixel's neighborhood to determine the value of the center pixel. It uses the maximum of all pixel values that correspond to zero in the structuring element, MAX(0), and the minimum of all pixels in the neighborhood that correspond to one in the structuring element, MIN(1). If the value of the center pixel is greater than MAX(0) or less than or equal to MIN(1), then the value of the center pixel is set to MIN(1), otherwise the value of the center pixel remains unchanged.

Both versions of thinning and thickening take structuring elements containing only 0's, 1's, and captial 'D's (impDontCare) values.

If the processing mode is set to impBinary, a binary thinning or thickening is performed, otherwise the grayscale version of these operations is performed.


Matching

User Guide:
Borders of an image
Connectivity mapping
Searching for hits and misses

Reference:
ImageProcessing.Match

Matching allows you to determine the degree of similarity between certain areas of the image and a pattern (specified by a structuring element). The Match method takes a binary or grayscale source image and produces a corresponding grayscale image, wherein the value of each pixel is equal to the total number of matches between the neighborhood of the source image's corresponding pixel and the structuring element values.


Searching for hits and misses

User Guide:
Borders of an image
Connectivity mapping
Matching

Reference:
ImageProcessing.HitOrMiss

You can determine which pixels have neighborhoods that match a pattern exactly by using the HitOrMiss method. In general, when the neighborhood of a source image's pixel matches the pattern exactly, the value of the corresponding pixel in the destination image is the maximum value of the image (e.g. 255 for an 8-bit image). Except in the case of a float buffer, where, if an exact match is found, the result is one. When the neighborhood does not match exactly, the pixel value is zero.


Connectivity mapping

User Guide:
Borders of an image
Custom morphological operations

Reference:
ImageProcessing.ConnectMap

In some cases, an image must undergo several passes with different structuring elements. This can be very time-consuming. To perform such operations more efficiently, you should consider the connectivity (or cellular) mapping method (ConnectMap method). This method reduces a serial operation to a parallel operation.

The ConnectMap method calculates a connectivity code for each pixel in a binary source image and then maps these codes through the specified custom LUT.

The connectivity code is obtained by linking the elements of a pixel's 3x3 neighborhood into a string, forming a single 9-bit number. Neighborhood pixels are linked in the following order:

The pixels are connected and mapped as follows:

Result = LUTMAP (connectivity code)

Program the LUT with values that would result if the required structuring elements were applied. Since each connectivity code has 9 bits, you should supply a LUT with at least 512 (2 to the power of 9) entries.


Fast Fourier transform

User Guide:
Advanced image manipulation
Denoising
Image quality

Reference:
ImageProcessing.FastFourierTransform

A Fast Fourier transform (FFT) is used to identify any consistent spatial patterns in an image (which can be caused, for example, by systematic noise). ActiveMIL can perform one- or two-dimensional FFTs using the FastFourierTransform method. For 1D transforms, each row or column is treated as a 1D signal. This method of transform separates a one-dimensional signal into a set of sine and cosine waves of different frequencies. For a two-dimensional signal (image), it can be interpreted as the decomposition of an image into a set of 2D patterns. The composition of these waves make up the original waveform.

The forward Fourier transform is defined as:

Where u and v are coordinates in the frequency domain and x and y are coordinates in the spatial domain. A forward FFT yields a real (R) and an imaginary (I) component of the image in a frequency domain (spectrum).

The reverse Fourier transform is defined as:

Where x and y are the coordinates in the spatial domain and u and v are coordinates in the frequency domain.

Magnitude and phase

For a more visual understanding of the FFT results, you can calculate the phase and magnitude, also using the FastFourierTransform method. The magnitude is calculated as:

where R and I are real and imaginary components of the image, respectively. The following figures show single-frequency images and their magnitude. Because single-frequency images contain only one spatial frequency component, their corresponding frequency images appear as a single point of brightness with their associated negative-frequency mirrors. Note that the points in the frequency domain appear in the direction of the pattern. The distance between the points and the center (DC component) represents the frequency of the pattern.

The spatial shift of each pattern in the image (in degrees) is called the phase. It is calculated using the formula atan(I/R). The image can be scaled within displayable range using the formula, clog [1=|F(u,v)|]. The FastFourierTransform method uses the flags impMagnitudePhase or impScaledMagnitudePhase to obtain these values.

Filtering an image

To filter constant spatial patterns using FFTs:

  1. Perform a forward transform using the FastFourierTransform method with ForwardTransform set to True. Calculate the phase and magnitude, and scale the image within the displayable range using the flag, impScaledMagnitudePhase.
  2. Find the frequency components representing the noise and design a mask to remove these components.
  3. Once the mask is designed, call the FastFourierTransform method to obtain the real and imaginary components, this time without calculating the phase and the magnitude, using the flag impRealImaginary.
  4. Apply the mask to both the real and imaginary components of the image in the frequency domain.
  5. Finally, perform a reverse transform (call the FastFourierTransform method with ForwardTransform set to False) to obtain a filtered image.

The MImageFFT example performs an FFT on an image with a vertical noise pattern. The magnitude is obtained in the frequency domain. The frequency components corresponding to the pattern are circled. Another forward transform is performed to obtain the real and imaginary components of the image. The values of locations corresponding to the noise pattern are set to 0. Finally, a reverse transform is performed to obtain a spatial image without the noise pattern.


Watershed transformations

User Guide:
Distance transform
Thresholding your image

Reference:
ImageProcessing.And
ImageProcessing.Distance
ImageProcessing.EdgeDetect
ImageProcessing.Source2
ImageProcessing.Watershed

You can perform watershed transformations using the Watershed method. A watershed transformation is generally used in conjunction with other processing operations to segment images, that is, to separate blobs from their background and/or from each other.

To understand watershed transformations, it is useful to think of images as topographic surfaces with hills and valleys. In other words, the value of each pixel represents a certain height, with the lowest pixel value (the darkest pixel) representing the point of lowest elevation and the highest pixel value (the brightest pixel) representing the point of highest elevation. A minimum in the image is defined as a pixel or a set of connected pixels that is lower in value (or elevation) than all its neighboring pixels. A maximum is a pixel or a set of connected pixels that is higher in value (elevation) than all its neighboring pixels. Pixels are connected if they are vertically, horizontally, or diagonally adjacent. A catchment basin refers to a minimum or maximum's zone of influence. For example, for a minimum, a catchment basin refers to the set of pixels from which, if a drop of water were to fall, it would eventually reach that minimum.

The Watershed method labels an image's catchment basins and/or builds dividing lines between the catchment basins. These dividing lines are known as the watershed lines of the image. Note that catchment basins can be determined from the image's minima or its maxima.

Using watersheds to separate touching blobs

You can use the Watershed method in conjunction with the Distance and the And methods to separate touching blobs in a binary image.

To summarize:

  1. Perform a distance transformation on the image. This will result in a grayscale image with a maximum in each blob.
  2. Perform a watershed transformation on the resulting image. Note that:
    • Catchment basins must be determined from the image's maxima rather than its minima since the Distance method produces a maximum in each blob.
    • The transform must show only watershed lines. To save time, you can prevent watershed lines from extending into the background. You can also specify that the watershed lines be straight. (These options are discussed in more detail below.)
    • You must specify the minimum grayscale variation that is required to produce a new catchment basin (this is discussed in more detail below). In general, when separating touching blobs in a binary image, a low value (2) is usually sufficient.
  3. Use the And method to AND the original image and the result of step 2.

Using watersheds to separate blobs from their background

The Watershed method can be used in conjunction with other processing operations to separate blobs from their background. For example, if the blobs have well-defined edges, an edge detection will produce a maximum along the edges of each blob. These maxima will define each blob as a catchment basin since they produce a minimum in each blob. A watershed transformation will then label the catchment basins, effectively segmenting the image.

To summarize:

  1. Perform an edge detection on the image (using the EdgeDetect method).
  2. Determine, through some analysis of the resulting image, the minimum grayscale variation that is required to produce a new catchment basin (this is discussed in the next section).
  3. Perform a watershed transformation on the resulting image. You must specify that catchment basins be determined from the image's minima. In addition, the transform should only show labeled catchment basins.

For an example that uses the watershed and the edge detect functions to remove the background of an image, see MImageSegmentation.

Minimum grayscale variation of a catchment basin

A typical image contains a lot of unwanted extrema, often due to noise. If catchment basins were determined from each extremum, the transform would segment various noise areas, resulting in over-segmentation. To prevent over-segmentation while still separating blobs from their background, you can specify the minimum grayscale variation of a catchment basin, using the MinimumVariation parameter of the Watershed method.

The minimum variation of a catchment basin can be more easily understood using the topographical surface analogy. When dealing with minima, the minimum grayscale variation can be thought of as the minimum depth of the valley required to produce a catchment basin. The reverse applies when dealing with maxima.

The following shows a line profile across an blob in an image (after an edge detection is performed on the image). In this case, the maximum difference between gray levels in the background (as well as within the blob) is about 10, and the minimum difference between the background and the edges is about 50. In this case, therefore, the specified minimum variation should be set to a value somewhere between 10 and 50, for example, 30. Note that, if it is set above 50, the blob will not be separated from the background since its extrema will not produce a new catchment basin.

By default, the minimum variation is 1, which means that each extremum produces a catchment basin.

Using marker images

If you are able to approximate the location of your blobs in an image (either through some pre-processing or through some previous knowledge of the image), you might want catchment basins determined from a separate image (known as a marker image), instead of from the extrema in the source image. Marker images, specified by the Source2 property, are useful in preventing over-segmentation since you control not only the location of the extrema but also the number of extrema.

If you use a marker image, there is no need to set the minimum grayscale variation, since you mark off the extrema in a separate image.

There are two types of marker images: non-labeled and labeled.

Non-labeled marker images

In a non-labeled marker image, each group of touching pixels with the value zero in the marker image is known as a marker and produces a catchment basin in the corresponding area of the source image. Specifically, each marker in the marker image forces an extremum in the corresponding area of the source image. Pixels in the marker image are considered touching if they are vertically, horizontally, or diagonally adjacent, that is, if they are "8-connected".

Labeled marker images

In a labeled marker image, a marker is a set of pixels that do not have to necessarily touch and that have all the same label value (pixel intensity). Each marker produces catchment basin(s) in the corresponding area(s) of the source image and each catchment basin of the destination image is assigned the label value of the marker that generated it. Note that, in a labeled marker image, a marker can touch other markers, allowing you to specify adjacent catchment basins. Valid marker label values are 1 to 2n-2 (where n is the marker image buffer depth). Pixels with the label value of 2n-1 are considered to be part of a "don't care" mask and are not processed, accelerating the watershed transformation. Finally, pixels with the label value of zero are interpreted as not being part of a marker. To use a labeled marker image, you must set the LabeledMarker parameter to True.

The following is an example that performs a watershed transformation using a labeled marker image. It will separate the pills from the background.

To summarize:

  1. Perform an edge detection on the original image (using the EdgeDetect method).
  2. Create a marker image that is able to locate your blobs in the image.
    • For this example, the labeled marker image is created from the edge detected image using a series of morphology, edge detection, and blob analysis operations.
    • In this example, the almost-white boundary surrounding all the rings is one marker. This marker serves to define the background and is placed in the corresponding area of the background of the original image.
    • All the rings in the marker image are considered to be one marker. This particular marker serves to identify the pills. These rings are smaller than the pills in the original image and are located in the corresponding areas of the pills.
  3. Perform the watershed transformation on the edge detected image. You must specify that the catchment basins be determined from the image's minima. Remark how the resulting catchment basins have the same label value as the marker that generated them.

Note that catchment basins can be determined from markers in the marker image as well as from extrema in the source image. In this case, supply a marker image (Source2 property) and also specify the minimum grayscale variation in the source image required to produce a new catchment basin.

Style of the watershed lines

Watershed lines can be 8-connected or 4-connected, specified by the Lattice parameter. In addition, they can be traced exactly or forced to be straight, specified by the StraightLines parameter.

8-connected watershed lines consist of pixels that are horizontally, vertically, or diagonally touching. 4-connected watershed lines consist of pixels that are just horizontally and/or vertically touching. 8-connected watershed lines can separate 4-connected blobs, that is, blobs whose pixels can touch horizontally or vertically. 4-connected watershed lines are required to separate 8-connected blobs, that is, blobs whose pixels can touch horizontally, vertically, or diagonally.

ActiveMIL's Blob control allows you to define blobs as either 4- or 8-connected.

Note that 4-connected watershed lines can also separate 4-connected blobs, however they cause over-separation.

Exact versus straight

For visual purposes, watershed lines can be traced exactly or forced to be straight.

Skipping the last level

The Watershed method allows you to skip the last intensity level of the transformation. This prevents an extremum's zone of influence from extending beyond Lmax - 1 (for a minimum) or Lmin - 1 (for a maximum), where Lmax is the maximum gray level in the image and Lmin is the minimum gray level. In effect, this prevents the background in the image from being processed, resulting in quicker processing times.

This option should be used when separating touching blobs since, in this case, watershed lines in the background are unnecessary.

Filling the source

The Watershed method allows you to fill the catchment basins of unwanted extrema in the source image by setting the FillSource parameter to True. Setting the FillSource parameter fills the catchment basins of the unwanted extrema until the local basins become plateaus. Any subsequent watershed transformation on the source image will no longer process the unwanted extrema because they are no longer extrema.

The following images illustrate how the filling of the source transforms the source image:


Polar-to-rectangular and rectangular-to-polar transforms

User Guide:
Advanced image manipulation

Reference:
ImageProcessing.PolarParameters
ImageProcessing.PolarParameters.EndAngle
ImageProcessing.PolarParameters.EndRadius
ImageProcessing.PolarParameters.StartAngle
ImageProcessing.PolarParameters.StartRadius
ImageProcessing.PolarToRectangular
ImageProcessing.RectangularToPolar
ImageProcessing.Source1

Polar-to-rectangular and rectangular-to-polar transforms allow conversion of polar coordinates to Cartesian coordinates and vice versa. With ActiveMIL, you can perform rectangular-to-polar or polar-to-rectangular transforms, using the RectangularToPolar and PolarToRectangular methods.

The following is an example of a rectangular-to-polar transform. The dotted line defines the borders of the zone of interest:

The result will be mapped to the destination image as shown below:

For a rectangular-to-polar transform, the borders of the zone of interest are defined by specifying the center, the start and end radius, and the start and end angle in a source image. The method scans the specified zone from the start angle to the end angle. In our example, since the start angle is less than the end angle, the direction of the scan is counter clockwise. The increment in angle is determined by the length (in pixels) of the outside arc, calculated as follows:

The valid range of angles is from -360.0 to 360.0° and the maximum span of the angle must not exceed 360.0°. These values are then mapped to a destination image.

A polar-to-rectangular transform performs the reverse of the transform described above. It takes a source image (Source1 property) and maps it to a destination image (Destination1 property). To specify the position of the contents of the source image in the destination image, use the CenterX, CenterY, StartAngle, EndAngle, StartRadius, and EndRadius properties of the PolarParameters property.

For an example that shows polar-to-rectangular transformations, see MImagePolar.


Warping

User Guide:
Advanced image manipulation
Custom morphological operations

Reference:
ImageProcessing.Flip
ImageProcessing.Resize
ImageProcessing.Rotate
ImageProcessing.Translate
ImageProcessing.Warp
ImageProcessing.WarpParameters.Coefficients
ImageProcessing.WarpParameters.RotateCoefficients
ImageProcessing.WarpParameters.ScaleCoefficients
ImageProcessing.WarpParameters.ShearCoefficients
ImageProcessing.WarpParameters.TranslateCoefficients

In addition to methods which perform specific geometric transforms (Resize, Translate, Rotate, and Flip), ActiveMIL includes the Warp method, which can perform any of the specific transforms as well as complex warpings. Such warpings could be used, for example, to correct geometric distortions.

The Warp method performs a warping by associating each pixel position of the destination image, (Xd, Yd), with a specific point (not necessarily a pixel) in the source image, (Xs, Ys). The pixel value at (Xd, Yd) is then determined from an interpolation around its associated source point. Destination pixels can be associated with source points through a 3x3 matrix of coefficients or through look-up tables (LUTs). Use the WarpParameters property to set up the coefficients or LUTs.

Note that the specific geometric methods are faster than the Warp method. You should only use the Warp method when the required transform cannot be otherwise performed.

For an example that uses warp transformations, see MImageWarp.

Using coefficients

A 3x3 matrix-defined warping associates source points to destination pixels according to the following equation:

The coefficients (a0... a2, b0... b2, c0... c2) required to produce a 3x3 matrix-defined warping are specified by the WarpParameters.Coefficients property. You can use your own coefficients or, for first-order polynomial warpings and perspective warpings that map an arbitrary quadrilateral onto a rectangle or that map a rectangle onto an arbitrary quadrilateral, you can have the coefficients automatically generated.

First-order polynomial warpings

A first-order polynomial warping is equivalent to linearly translating, rotating, resizing, and/or shearing an image. It is a type of 3x3 matrix-defined warping in which c0 and c1 are set to 0 and c2 is set to 1, that is,

xs = a0xd + a1yd + a2

ys = b0xd + b1yd + b2

To automatically generate coefficients for a first-order polynomial warping, use the RotateCoefficients, ScaleCoefficients, ShearCoefficients, and/or TranslateCoefficients methods of the WarpParameters property.

Perspective warping

To generate coefficients for a perspective warping that maps an arbitrary quadrilateral onto a rectangle or that maps a rectangle onto an arbitrary quadrilateral, first specify the coordinates of the quadrilateral and rectangle using the WarpParameters.Quadrilateral and WarpParameters.Rectangle properties, respectively. Then, use the WarpParameters.MapQuadrilateralToRectangle or WarpParameters.MapRectangleToQuadrilateral methods to generate the coefficients.

Clearing coefficients

Note that the values of the 3x3 matrix are not reset each time one of the methods used to generate coefficients are used. This allows you to combine transforms. For example, to combine coefficients for a rotation and translation, simply call the RotateCoefficients and TranslateCoefficients methods. After all coefficients are generated, call the Warp method.

If you need to reset coefficients, use the WarpParameters.ResetCoefficients method.

Using LUTs to perform a warping

When you perform a warping using LUTs, xs is determined from (xd, yd) through one LUT and ys is determined from (xd, yd) through another LUT. Since xs and ys can be arbitrarily mapped, you can perform any type of warping when using LUTs. To set up the LUTs, use the WarpParameters.WarpLUT property. You can transfer your own data to the LUTs, using the WarpLUT.Put method, or for a 3x3 matrix-defined warping, you can have the LUTs automatically generated using the WarpParameters.GenerateLUT method. The GenerateLUT method generates LUTs to perform the 3x3 warping specified by the Coefficients(X, Y) property. Depending on your system, it might be faster to perform a 3x3 warping using LUTs.

Interpolation modes

In a warping, the destination coordinates have integer values but the source coordinates, in general, do not. Therefore, the pixel value at (xd, yd) has to be determined from several source pixels that are near (xs, ys), according to an interpolation mode.

When you use the Warp method, you can choose from the following interpolation modes.

  • Nearest-neighbor. This mode determines the nearest pixel to a point, and copies that pixel value into its associated position.
  • Bilinear. This mode takes a weighted average of the four pixels nearest to the point, and copies that average into its associated position. The pixels closest to the point are given the most weight.
  • Bicubic. This mode takes a weighted average of the sixteen pixels nearest to the point, and copies that average into its associated position. Again, the pixels closest to the point are given the most weight.

In general, nearest-neighbor interpolation is the fastest to perform, and bicubic interpolation is the slowest. However, nearest-neighbor interpolation produces the least accurate results, and bicubic interpolation produces the most accurate. Bilinear interpolation is often the best compromise between speed and accuracy.

Points outside the source image

Sometimes, the point associated with a destination pixel will fall outside the source image. In such cases, the Warp method can determine the new value for the destination pixel by:

  • Using pixels from the source image's ancestor. If the source image is not a child image or if the point falls outside its ancestor, the destination pixel will be left as is.
  • Just leaving the destination pixel as is.
  • Setting the destination pixel to 0.

In general, you should use pixels from the source image's ancestor when the source image is a child image. This will ensure that the pixels you use are related to the source image. If the source image is not a child image, use one of the other options.

Note that you can set the destination pixel to a value other than 0 by first clearing the destination image to that value.


Discrete cosine transform

See Also | Previous
User Guide:
Controlling a JPEG lossy compression
Fast Fourier transform

Reference:
ImageProcessing.DiscreteCosineTransform

Discrete Cosine Transform (DCT) is mainly used for image JPEG lossy compression. ActiveMIL can perform a DCT using the DiscreteCosineTransform method. For a one dimensional signal, this method separates the signal into a set of cosine waves of different frequency. For a two-dimensional signal (image), it can be interpreted as the decomposition of an image into a set of 2D cosine patterns. The composition of these waves make up the original waveform.

The forward DCT is defined as:

where u and v are coordinates in the frequency domain,

The reverse DCT is defined as:

where x and y are coordinates in the spatial domain.

Frequency 0, also called the DC component, is plotted in the top-left corner of the spectrum. All other components in the spectrum are called AC components. A DCT concentrates the low frequency components of the image in the first few coefficients (top-left-corner) of the spectrum. ActiveMIL divides the image into independent blocks of 8x8 pixels and performs the transform on each individual block. Centering of the spectrum is not supported in ActiveMIL.


Calibration

Chapter 5: Calibration

This chapter describes how to useActiveMIL's Calibration control.

  • Calibration control overview
  • Types of distortions
  • Steps to getting results in real-world units
  • Physically correcting an image
  • Automatically getting results in real-world units
    • Associating
    • Calibrated image
    • Calibrated digitizer
    • Associating to image versus digitizer
    • Child images
    • Returned results
  • Calibrating with a list of points
  • Calibrating with a real-world grid
    • Real-world grid
    • General rules for constructing a grid
    • Default axes
    • Offset and Y-axis
  • Calibration modes
    • Piecewise linear interpolation
    • Perspective transformation
  • Coordinate systems and camera position
    • Image coordinate system
    • Absolute world coordinate system
    • Camera position
    • Relative camera position
    • Relative world coordinate system
  • Multiple fields of view
    • Single camera fixed on a manipulator : Relative camera position example
    • Single camera and moveable object : Relative coordinate system example
    • Several cameras and fixed object
  • Processing calibrated images


Calibration control overview

See Also | Next
Reference:
BlobAnalysis
Calibration
CharacterRecognition
Code
ImageProcessing
Measurement
ModelFinder
PatternMatching

ActiveMIL's Calibration control consists of a set of properties, methods, and events that allow you to map pixel coordinates to real-world coordinates. This mapping can be used to get results from other ActiveMIL controls in real-world units. The mapping can also be used to physically correct an image's distortions.

By getting results in real-world units, you automatically compensate for any distortions in an image. Therefore, you can get accurate results despite an image's distortions.

Defining the pixel-to-world mapping is known as calibration. The Calibration control is used to hold the defined mapping, as well as certain control settings.

Once you have created your Calibration control, you can:

  • Use it to transform pixel coordinates or results to their real-world equivalents.
  • Use it to physically correct an image.
  • Use it to automatically get results from other ActiveMIL controls in real-world units. The controls that can return results in real-world units are:
    • BlobAnalysis
    • CharacterRecognition
    • Code
    • ImageProcessing
    • Measurement
    • PatternMatching
    • ModelFinder

Note that a few results are always returned in pixel units.

For an example that uses the Calibration control, see MCalibration.


Types of distortions

User Guide:
Basic geometric transforms

Reference:
Calibration

You can use calibration if you have one or more of the following types of distortion:

  • Non-unity aspect ratio distortion Present when the X- and Y-axis have two different scale factors. This is evident, for example, if you know that an object in your image should be round and it appears as an ellipse. This type of distortion is often a side effect of the sampling rate used by some older digitizers.
  • Rotation distortion Present when the camera is perpendicular to the object grabbed in the image, but not aligned with the object's axes.
  • Perspective distortion Present when the camera is not perpendicular to the object grabbed in the image. Objects that are further away from the camera appear proportionally smaller than the same size objects closer to the camera.
  • Other spatial distortions Complex distortions, such as pin cushion and barrel-type distortions, fall in this category. These distortions can be compensated for by using a large number of small sections in the mapping function. If the number of sections used is big enough and the corresponding area covered in each is small enough, the mapping in each area can be approximated with a piecewise linear interpolation function.


Steps to getting results in real-world units

User Guide:
Automatically getting results in real-world units
Physically correcting an image

Reference:
Calibration.CalibrateGrid
Calibration.CalibratePoints
Calibration.CalibrationMode
Calibration.CalibrationPoints
Calibration.ConvertCoordinatePixelToWorld
Calibration.ConvertCoordinateWorldToPixel
Calibration.ConvertResultPixelToWorld
Calibration.ConvertResultWorldToPixel
Calibration.CorrectImage
Calibration.Grid
Digitizer.Calibration
Image.Calibration

To get results in real-world units:

  1. Set the CalibrationMode property of a Calibration control.
  2. Set the properties of the Grid or CalibrationPoints property, depending on the method of calibration you choose in the following step.
  3. Calibrate your imaging setup, using either the CalibrateGrid or CalibratePoints methods.
  4. Do one of the following:
    • To transform pixel coordinates to their real-world equivalents or vice-versa, use the ConvertCoordinatePixelToWorld or ConvertCoordinateWorldToPixel method, respectively.
    • To convert a specific non-positional result (a length, angle, or area) from its pixel value to its world value or vice-versa, use the ConvertResultPixelToWorld or ConvertResultWorldToPixel method, respectively. Note, however, that these methods use the average pixel size to perform the conversion; results will be more accurate if you first correct the image.
    • To physically correct an image, use the CorrectImage method.
    • To automatically get results from other ActiveMIL controls in real-world units, associate the Calibration control to an image or digitizer, using the Calibration property of the Digitizer control or the Calibration property of the Image control.


Physically correcting an image

User Guide:
Warping

Reference:
Calibration.CorrectImage
Calibration.TransformCacheEnabled

You can use the CorrectImage method to physically correct and remove any distortions in an image. An image that has been physically corrected using the Calibration control is known as a corrected image. When analysis results are extracted from a corrected image, these results will be returned in real-world units.

For example, after calibrating the source image below, the world coordinate of the bottom-left circle is (5,6). When you correct your image, the world coordinate of the bottom-left circle in the corrected image is also (5,6), even though it is evident that the pixel coordinates are different for the bottom-left circle in the source and corrected images.

Note that images are physically corrected using a geometric warping.

By default, a cache is used to physically correct an image. The first time a Calibration control is used to transform an image, this cache fills up with information relevant to the transformation. On subsequent transformations with the Calibration control, the information in the cache can significantly accelerate the transform. However, if you need to save memory, you can disable this cache, using the TransformCacheEnabled property. (The cache consists of two 32-bit images that have with the same size as the destination image).

The information in the cache is flushed whenever the size of the source or destination image changes, or whenever the angle of the relative coordinate system of the Calibration control changes.


Automatically getting results in real-world units

User Guide:
Child Images
Coordinate systems and camera position
Physically correcting an image
Processing calibrated images

Reference:
Calibration.OutputCoordinateSystem
Digitizer.Calibration
Digitizer.Calibration
Image.Calibration
Image.Calibration

Associating

To automatically get results from other ActiveMIL controls in real-world units, associate the Calibration control to an image or digitizer, using either the Calibration property of the Image control or Calibration property of the Digitizer control.

To dissociate a calibration control from an image or digitizer, you can also use these properties.

Calibrated image

An image with an associated Calibration control is known as a calibrated image. A calibrated image still appears distorted because it has not been physically corrected. However, when it is used with an ActiveMIL control, results from this control can be returned in real-world units.

Note that the Calibration control gets associated to the image data, not to the Image control containing the image. This implies that the destination image of an operation receives the same Calibration control as the source image (if any). To understand the implications this has on processing refer to Processing calibrated images.

Also note that, if you save a calibrated image to file, the calibration settings do not get saved.

Calibrated digitizer

A digitizer with an associated Calibration control is known as a calibrated digitizer. When you grab an image with a calibrated digitizer, the Calibration control currently associated to the digitizer gets associated to the grabbed image. Therefore, the grabbed image becomes a calibrated image.

Associating to image versus digitizer

When you associate a Calibration control to an image (using the Calibration property of the Image control or by grabbing it with a calibrated digitizer), the image receives a copy of the Calibration control's current relative coordinate system and current relative camera position and a reference to the Calibration control for all other settings. This means that, if you change the relative coordinate system or relative camera position after association, the change will not affect the image (See Coordinate system and camera position). If you change any other setting of the Calibration control after association, the change will affect the calibrated image. When you associate a Calibration control to a digitizer, the digitizer only receives a reference to the Calibration control.

Child images

When a Calibration control is associated to an image that contains child images, the child images are automatically calibrated. In addition, their offsets to the parent image are taken into account when returning real-world results.

Returned results

When a calibrated image is used within an ActiveMIL control, results can be returned in pixel units or in real-world units. To specify whether results should be in pixel or real-world units, use the OutputCoordinateSystem property of the Calibration control. By default, results are in real-world units.

While results can be returned in pixel or real-world units, any value which you pass to an ActiveMIL method or property must be in pixel units.

A few results are always returned in pixel units. If a result can be returned in either real-world or pixel units, it will be stated in the method or property description.


Calibrating with a list of points

Reference:
Calibration.CalibratePoints
Calibration.CalibrationPoints

To use a list of coordinates, first set the properties of the CalibrationPoints property, then call the CalibratePoints method. The CalibratePoints method uses a list of pixel coordinates and their associated real-world coordinates to define the pixel-to-world mapping. The more coordinates you specify, the more accurate the mapping. The CalibrationPoints property can be used when you explicitly know the real-world coordinates for a given set of pixel coordinates. The specified pixel coordinates should cover the area of the image from which you want real-world coordinates (the working area).

In the case of perspective distortions, knowing the world coordinates of 4 points in the image gives sufficient information to create a mapping function. To create a good mapping for a radial distortion requires a larger number of coordinates (for example, more than 30) distributed over the image.


Calibrating with a real-world grid

User Guide:
Calibrating with a list of points

Reference:
Calibration.CalibrateGrid
Calibration.CalibrationPoints
Calibration.Grid

To calibrate your imaging setup, you can use an image of a user-defined grid of circles (or you can use a list of pixel and real-world coordinates). To use a grid, first set the properties of the Grid property, then call the CalibrateGrid method.

Real-world grid

Calibration from a grid is often used when a number of images will be grabbed using the same camera position and lens configuration.

The Grid property determines the pixel-to-world mapping from an image of a user-defined grid of circles and the world description of this grid. The world description includes the number of rows and columns, as well as the center-to-center distance between these rows and columns, in real-world units.

General rules for constructing a grid

The CalibrateGrid method can create a pixel-to-world mapping from almost any grid of circles. However, to create an accurate (subpixel) mapping, your physical grid should meet the following guidelines (at the working resolution):

  • The radius of the grid's circles should range between 6 and 10 pixels.
  • The center-to-center distance between the grid's circles should range from 18 to 32 pixels (22 pixels recommended).
  • The minimum distance between the edges of the circles should be 6 pixels.
  • The grid should be large enough to cover the area of the image from which you want real-world results (the working area).
  • The grid image should have high contrast.

Default axes

By default, the circle in the top-left corner of the grid image is associated to the origin, (0, 0), of the real world coordinate system, the first column of circles is aligned with its Y-axis, and the first row of circles is aligned with its X-axis. This is because, in general, you know where that first circle is in the real-world, so you need results with respect to that position (the top-left pixel, for example, is generally not a known position in the real-world).

Offset and Y-axis

If necessary, you can associate the top-left circle of the grid image to a different position within the real-world coordinate system. The origin of the real-world coordinate system does not have to be within the field-of-view.

You can have the positive Y-axis oriented 90º counter-clockwise with respect to the positive X-axis (by default, the Calibration control assumes it is oriented 90º clockwise).

Once the properties of the Grid property been set, call the CalibrateGrid method to calibrate your image. When the Calibration control is calibrated, all the points of the grid are accessible by the CalibrationPoints property. This will return the coordinates of the center of the grid's circles.


Calibration modes

Reference:
Calibration.CalibrateGrid
Calibration.CalibratePoints

Before you use the CalibrateGrid or CalibratePoints method, you should also specify the calibration mode. ActiveMIL supports the following calibration modes:

  • Piecewise linear interpolation.
  • Perspective transformation.

Piecewise linear interpolation

In general, if you have sufficient points defined, you should use the piecewise linear interpolation mode. This mode can compensate for any kind of distortion. It is very accurate for points located inside the working area. However, it is less accurate for points outside the working area. The piecewise linear interpolation mode fits a piecewise linear interpolation function to the set of image coordinates and their real-world equivalents.

Perspective transformation

The perspective transformation mode can compensate for rotation, translation, scale, and perspective distortions. For such distortions, the perspective transformation mode is accurate for points inside and outside the working area. This mode cannot compensate for non-linear distortions such as lens distortions. The perspective transformation mode best fits a global perspective transformation function to the set of image coordinates and their real-world equivalents.


Coordinate systems and camera position

User Guide:
Multiple fields of view

Reference:
Calibration.CameraPosition
Calibration.CorrectImage
Calibration.RelativeOrigin

By default, after calibration, real-world positional results will be given within the absolute world coordinate system, rather than the image coordinate system. You can specify the position of your camera in the absolute world coordinate system. In addition, you can get results relative to some object by moving and/or orienting the relative world coordinate system.

Image coordinate system

The image coordinate system is the coordinate system used to locate and/or measure objects in an uncalibrated image. Its unit of measure is pixels. Its origin, (0, 0), is the middle of the image's top-left pixel. Its Y-axis is aligned with the first column of pixels and its X-axis is aligned with the first row of pixels.

Absolute world coordinate system

The absolute world coordinate system is the coordinate system used to locate and/or measure objects in the real world. It is implicitly defined from the calibration points when calibrating the imaging setup. Its unit of measure is user-defined (mm, cm, inches, etc.). Calibration relates the image coordinate system to the absolute world coordinate system.

Note that, when using a grid of circles to calibrate your imaging setup, the X-axis of the absolute world coordinate system is aligned with the first row of circles, and the Y-axis is aligned with the first column of circles.

For the sake of simplicity, the absolute world coordinate system will be known as the absolute coordinate system.

Camera position

The camera position can be any arbitrary point that moves with the camera; it does not have to be the actual camera position. For example in the following diagram, the camera position is assigned not to the actual camera, but to the arm of the robotic manipulator.

Relative camera position

The relative camera position refers to the position of your camera relative to the absolute coordinate system. Adjusting the relative camera position can be useful when analyzing an object that cannot fit in a single image (see the Multiple fields of view section for details).

The relative camera position affects positional results taken from a calibrated image, as shown below.

To adjust the relative camera position, use the properties of the CameraPosition property.

Relative world coordinate system

A relative coordinate system is a Cartesian system defined within the world coordinate system. By default, the relative world coordinate system is aligned with the absolute coordinate system. However, to get results relative to some object, it can be moved anywhere within the absolute coordinate system and rotated by any angle. Its unit of measure is the same as the absolute coordinate system. For the sake of simplicity, the relative world coordinate system will be known as the relative coordinate system.

The relative coordinate system is generally used to allow measurements to be made relative to a particular object in the field of view, especially when there are several such objects in the field of view. For example, the following is an image that was calibrated from a grid using the default settings which have been specified at installation. By default, the relative and world coordinate system's top-left circle in this image is 0,0. However, to get results relative to the origin of the object, the relative coordinate system is at (2, 1).

Note that this image is for illustrative purposes only. In general, the object should not be placed over a grid because if the grid of circles and the object are not differentiated when performing the processing operation, then erroneous results will be returned.

To move and/or rotate the relative coordinate system, use the properties of the RelativeOrigin property. Once you change the origin and/or orientation of the relative coordinate system, world coordinates will be returned in this relative coordinate system.

Note that, when you physically correct an image (using the CorrectImage method), the image is transformed such that the relative coordinate system is aligned with the image coordinate system.


Multiple fields of view

User Guide:
Coordinate systems and camera position

Reference:
Calibration.CalibrateGrid
Calibration.CalibrationPoints
Calibration.CameraPosition
Calibration.RelativeOrigin

The Calibration control makes it possible to measure the length between various points on an object, even when the object is not entirely within the camera's field-of-view. The camera's field-of-view refers to the largest world region visible in an image at a given resolution.

The approach to analyzing a large object involves acquiring images of the various parts of the object, getting real-world coordinates from each image, and then calculating the distance between the coordinates.

Three types of applications are considered:

  • A single camera is fixed on a manipulator and the manipulator is moved to different positions to acquire the different images.
  • A single camera is fixed to a location and the object is moved to different positions to acquire the different images.
  • Several cameras are used to acquire the different images.

Single camera fixed on a manipulator : Relative camera position example

When a single camera is fixed on a manipulator, part of the object is grabbed, the camera is moved to a different position, a different part of the object is grabbed, and the process repeats until the entire object has been grabbed. For the coordinates from each image to be in the same absolute coordinate system, the relative camera position has to be updated each time the camera is moved. To change the relative camera position, use the properties of the CameraPosition property.

Single camera and moveable object : Relative coordinate system example

When a single camera is fixed to a location, part of the item is grabbed, the object is moved to a different position, another part of the object is grabbed, and the process repeats until the entire object has been grabbed. For the coordinates from each image to be in the same coordinate system, the relative coordinate system must be moved each time the object is moved. To move the relative coordinate system, use the properties of the RelativeOrigin property.

Several cameras and fixed object

For the coordinates from each image to be in the same absolute coordinate system when several cameras are used, the Calibration control used to calibrate each camera must use the same absolute coordinate system. When using the CalibrateGrid method, you must use an offset to relate the origin of each grid to the same absolute coordinate system. When using the CalibrationPoints method, the real-world coordinates used in each mapping must be from the same absolute coordinate system.


Processing calibrated images

See Also | Previous
User Guide:
Automatically getting results in real-world units
Physically correcting an image

Reference:
Calibration.CorrectImage
Calibration.CorrectImage
Image.Clear
Image.Load
ImageProcessing.Flip
ImageProcessing.PolarToRectangular
ImageProcessing.RectangularToPolar
ImageProcessing.Resize
ImageProcessing.Rotate
ImageProcessing.Translate
ImageProcessing.Warp

Depending on the type of operation performed on the calibrated image, the destination image is associated with the following Calibration control:

  • When performing point-to-point or neighborhood processing operations into destination images, the destination image is associated with the same Calibration control as the source image.

    • If the operation uses more than one source image, a Calibration control gets associated to the destination image only if all source images have the same Calibration control; otherwise, no Calibration control gets associated to the destination image.

  • When performing geometric operations, the destination image uses the default Calibration control, which might not be the Calibration control that the source image uses.
  • When you perform a CorrectImage method operation on a calibrated image, the new image coordinates will still be returned in world units. As is expected, the image features in the destination image have the same world coordinates as they had in the source image, despite the fact their pixel coordinates have changed.

The above rule implies that methods for which the source data is always uncalibrated always produce uncalibrated images. These include, for example, the Load method.

The above rule has a few exceptions. Specifically, the following methods of the ImageProcessing control always result in an uncalibrated image, even if the source image is calibrated:

  • Flip
  • PolarToRectangular
  • RectangularToPolar
  • Resize
  • Rotate
  • Translate
  • Warp

The Clear method also always results in an uncalibrated image.

Note that, if you save a calibrated image to file, the calibration settings do not get saved.


Blob Analysis

Chapter 6: Blob Analysis

This chapter describes the basic steps to extract connected regions of pixels (blobs) within an image.

  • BlobAnalysis control overview
  • Identifying blobs
    • Blobs
    • Blob identifier image
    • Segmenting the blob identifier image
    • Supported images
  • Interpreting the identifier image
    • Controlling the image lattice
    • Pixel aspect ratio
      • Adjusting the aspect ratio
      • Positions and the pixel aspect ratio
    • Setting the blob identifier type
      • Results for each blob
      • Results for all blobs as one
      • Results for blobs grouped by label value
  • Selecting blobs
  • Making feature extractions
    • Calculating features
      • Binary and grayscale features
      • Selecting features
      • Which features to calculate
    • Drawing results
    • Sorting results
  • The area and perimeter
    • The pixel
    • The area and perimeter
    • The convex perimeter
    • The aspect ratio
  • Calculating dimensions
    • The Feret diameter
    • Calculating different Feret diameters
    • Dimensions of long thin blobs
  • Determining the shape
    • Compactness and roughness
  • Finding blob location
    • Blob points
    • Chained pixels
    • Moments
  • Location, length and number of runs
  • Blob reconstruction
  • Steps to performing blob analysis


BlobAnalysis control overview

See Also | Next
Reference:
BlobAnalysis
BlobAnalysis.ApplyFilter
BlobAnalysis.Calculate
BlobAnalysis.FeatureList
BlobAnalysis.GrayscaleImage
BlobAnalysis.Image
BlobAnalysis.Results
BlobAnalysis.SortingKeys

Blob analysis allows you to identify and extract features of connected regions of pixels (commonly known as blobs) within an image. To perform blob analysis, you need a BlobAnalysis control, a System control, and at least one Image control. You can perform blob analysis on 1-bit, 8-bit, or 16-bit unsigned blob identifier images.

The BlobAnalysis control uses a blob identifier image (Image property) to determine which pixels are part of which blob. Before performing any blob analysis calculations, you should ensure correct interpretation of the blob identifier image by setting the following characteristics: foreground pixel value, blob identification mode, non-zero pixel values, image lattice, and pixel aspect ratio.

After you have produced the best possible image, you can select the features that you would like calculated, from the FeatureList property. Features that use the actual pixel values of the blob require a grayscale image as well as the blob identifier image. In this case, the blob identifier image determines which pixels in the grayscale image (GrayscaleImage property) belong to which blob.

A call to the Calculate method finds the blobs in the image and calculates the selected features. You can set up sorting keys to get the results in a sorted manner (using the SortingKeys property). The blob results can be obtained using the Results property. Unwanted blobs can be discarded from future calculations and result retrieval by applying a filter (ApplyFilter method) from the collection of filters. The BlobAnalysis control also provides methods that allow you to reconstruct blobs, fill holes, extract holes, and erase border blobs.

For an example that uses the BlobAnalysis control, see MBlobAnalysis.


Identifying blobs

User Guide:
Interpreting the identifier image
Selecting blobs
Steps to performing blob analysis

Reference:
BlobAnalysis.ForegroundPixelValue
BlobAnalysis.GrayscaleImage
BlobAnalysis.IdentifierPixelType
BlobAnalysis.Image
ImageProcessing.Binarize
ImageProcessing.Clip
ImageProcessing.Rank
ImageProcessing.Smooth
ImageProcessing.Watershed

Blobs

Blobs are areas of touching pixels that are in the same logical pixel state; this is called the foreground state, while the alternate state is called the background state. Blobs are considered to consist of either zero or non-zero pixels, depending on the foreground setting (ForegroundPixelValue property).

Blob identifier image

The BlobAnalysis control uses a user-specified blob identifier image (Image property) to determine which pixels are part of which blob. Blob features involving overall shape are extracted directly from the identifier image. Features that use the actual pixel values of the blob require the grayscale image as well. In this case, the blob identifier image determines which pixels in the grayscale image (GrayscaleImage property) belong to which blob.

You can set the pixel type of the blob identifier image using the IdentifierPixelType property. This property allows you to specify if the image has grayscale (blobGrayscale) or binary (blobBinary)pixel values. Setting the blob identifier image to blobGrayscale is useful when you want to calculate grayscale features. This allows you to assign the same image to the Image and GrayscaleImage properties. Note, however, setting the IdentifierPixelType to blobBinary performs calculations faster than if set to blobGrayscale mode.

Because the BlobAnalysis control considers touching foreground pixels in the blob identifier image to be part of the same blob, what is easily identifiable by the human eye as several distinct but touching blobs is interpreted by ActiveMIL as a single blob. In addition, any part of a blob that is in the background pixel state, because of lighting or reflection, is considered as background during analysis.

To reduce preprocessing, the blob identifier image should be acquired under the best possible circumstances. This means ensuring that blobs do not overlap and, if possible, do not touch. It also means ensuring the best possible lighting and using a background with a gray level that is very distinct from the gray level of the blobs.

Segmenting the blob identifier image

If noise is a problem, you might also need to filter the blob identifier image after acquisition (for example, using a median filter or a convolution with the Rank and Smooth methods, respectively). Also producing the blob identifier image frequently creates some spurious blobs or holes (for example, due to noise or lighting). Such noise blobs make it harder to interpret blob analysis results. If you have many noise blobs, you should probably perform an opening operation (for non-zero valued blobs or holes) or a closing operation (for zero valued blobs or holes) to remove most noise.

Note, performing opening or closing operations on the blob identifier image might affect the accuracy of calculations because of the slight change in blob shape. If this is a problem, perform the calculations on all the blobs, including those that are actually introduced by noise, then use the results to filter out the noise. Note, however, that this method increases the memory required and might increase the calculation time.

Once the best possible image is acquired and most of the noise is filtered out, you must separate the different blobs from the background. Simple segmentation can be done using the following methods:

  • Binarize the image, using the Binarize method of the ImageProcessing control so that background pixels are represented as zero values and blob pixels are represented as another value.
  • Clip all background pixels to zero, while retaining the original values of blob pixels, using the ImageProcessing control's Clip method. This method has the advantage of not needing a separate image to hold the binary image, but you will not see the result of the segmentation as clearly. The first method is usually better.

If simple segmentation is not possible due to poor lighting or blobs which have the same gray level as parts of the background, you must use a segmentation algorithm appropriate for your particular image.

If blobs are touching, you might try eroding the image a few times to break them apart. You can also use the Watershed method to separate blobs.

Supported images

You can perform blob analysis on 1-bit, 8-bit or 16-bit unsigned blob identifier images.


Interpreting the identifier image

User Guide:
Identifying blobs
Steps to performing blob analysis
The area and perimeter

Reference:
BlobAnalysis.FeatureList.Box.FeretX
BlobAnalysis.FeatureList.Box.FeretY
BlobAnalysis.FeatureList.Box.XMinimum
BlobAnalysis.ForegroundPixelValue
BlobAnalysis.IdentifierBlobType
BlobAnalysis.IdentifierPixelType
BlobAnalysis.Lattice
BlobAnalysis.NumberOfFeretAngles
BlobAnalysis.PixelAspectRatio
BlobAnalysis.Results.Item.LabelValue
ImageProcessing.Resize

Before performing any blob analysis calculations, you should ensure the correct interpretation of the blob identifier image. This can be done by setting the following properties:

  • The ForegroundPixelValue property allows you to determine which pixel values are considered to be in the foreground.
  • The Lattice property allows you to determine whether two pixels touching at their corners are considered part of the same blob, by appropriately defining the image lattice.
  • The IdentifierPixelType property allows you to determine whether non-zero pixels can have any value or must be set to the maximum value of the image; for example, 255 for an 8-bit image.
  • The PixelAspectRatio property allows you to determine the pixel aspect ratio of the image.
  • The IdentifierBlobType property allows you to specify whether to produce separate results for each blob or for groups of blobs.
  • The NumberOfFeretAngles property allows you determine how many Feret angles are considered when calculating a Feret feature. Typically, the default value will be appropriate.

Controlling the image lattice

ActiveMIL represents images using a square lattice and considers adjacent pixels along the vertical or horizontal axis as touching. However, you can control whether two diagonally adjacent pixels are considered touching.

Use the Lattice property to specify how the blob identifier image lattice should be interpreted. For example, the following is considered one blob if the lattice is set to blob8Connected, but two blobs if set to blob4Connected.

Pixel aspect ratio

When acquiring an image of a scene, each pixel represents some real distance both in width and in height. Ideally, this distance is the same in both directions, producing square pixels and allowing for simple feature calculations. However, after digitization, it is quite common for a pixel to represent a different distance in each direction. The ratio of the pixel's width to its height is called the PixelAspectRatio. For example, a pixel of equal width and height has a pixel aspect ratio of 1.0.

Adjusting the aspect ratio

In blob analysis, the pixel aspect ratio directly affects feature extractions. For example, all circular blobs are stretched or squashed if the pixels are not exactly square. In this case, you have two alternatives:

  • You can adjust your image, using the Resize method of the ImageProcessing control, and then make the required blob analysis feature extractions.
  • You can have calculations take the actual aspect ratio into consideration without modifying the image, by specifying the ratio, using the PixelAspectRatio property. However, any feature derived from multiple Feret diameters cannot take the pixel aspect ratio into account accurately, and you will get better results by actually resizing your image. The same is true of the general Feret diameter.

In both cases, the actual aspect ratio can be calculated using a simple procedure. Grab an image of a true circle or square and extract the FeretX and the FeretY features with the default pixel aspect ratio of 1.0. The relationship between these features represents the actual pixel aspect ratio used in calculations.

Positions and the pixel aspect ratio

Note all results are affected by the pixel aspect ratio, including those that are just positions within the image. For example, to mark the XMinimum property on an image using the GraphicsContext control, you must take the aspect ratio into account (in this case by dividing the returned result by the aspect ratio).

Setting the blob identifier type

To determine whether results are grouped, you can set IdentifierBlobType property to one of the following:

  • All blobs are measured individually (blobIndividual)
  • All blobs grouped together (blobEntireImage)
  • Different blobs with the same label grouped together (blobLabeled)

Note, this mode does not change how blobs are identified (regions of connected foreground pixels).

Results for each blob

When using the BlobAnalysis control, you usually want to make feature calculations on each blob. For example, if you want to find the area of each cell in an image of tissue, set the IdentifierBlobType to blobIndividual for the image.

Results for all blobs as one

Sometimes, however, you need calculations based on the entire image rather than individual blobs. For example, you might want to calculate, in an image, the area of all the copper in a rock sample. ActiveMIL simplifies your task by allowing you to treat all foreground pixels together by setting the blob identification mode to blobEntireImage. In this mode all blobs are grouped together and treated as one blob, features for this blob are calculated.

Results for blobs grouped by label value

Setting the IdentifierBlobType property to blobLabeled allows you to do joint calculations on blobs with the same label value. When using labeled mode, ensure that each blob in the blob identifier image that you want to give the same label to has the same pixel value. The LabelValue property allows you to obtain the pixel value of a blob.


Selecting blobs

User Guide:
Making feature extractions
Steps to performing blob analysis

Reference:
BlobAnalysis.ApplyFilter
BlobAnalysis.Calculate
BlobAnalysis.Filters
BlobAnalysis.GrayscaleImage
BlobAnalysis.Image

Once all blobs are clearly identifiable, you are ready to perform calculations. However, in some cases, you will not want to make time-consuming feature extractions for every single blob in the blob identifier image. For example, you probably do not want to calculate features for blobs that are touching the edges of the image or that are noise artifacts. Often, you cannot remove these blobs from your image without losing too much information.

You can apply a filter using the Filters property for such cases. This allows you to select (on the basis of calculations already made) a subset of blobs for which to make further calculations and get results. The filters are generally used in one of two ways:

  • If you do not have many unwanted blobs, it is usually faster to calculate all required features for all blobs. Then, prior to getting results, use the ApplyFilter method to apply the filter to exclude or delete results obtained for blobs that do not meet your criteria.
  • If you have many unwanted blobs, you can save time and memory by first calculating, for all blobs, only those features that allow you to distinguish between relevant and unwanted blobs. Exclude from future calculations (or delete altogether from the blob analysis result) blobs that do not meet your criteria, using a filter. Then, calculate all required features for remaining blobs.

If you cannot exclude or delete many blobs using the second method, use the first.

You can also re-include some or all excluded blobs by applying a filter, which uses a feature that was calculated for the excluded blobs. However, you cannot re-include deleted blobs.

You can apply a filter on your image as many times as necessary using the ApplyFilter method. However, you must always give the same identifier image (using the Image property) and grayscale image (using the GrayscaleImage property) to the Calculate method during this procedure. If you give different images or change the existing images in any way, all current results will be discarded the next time you use the Calculate method. In addition, all selected features will be re-calculated for all blobs in the new identifier image. This means that you will have to restart the selection procedure. If you intend to calculate grayscale features during your analysis, you must include the grayscale image before starting your calculations.


Making feature extractions

User Guide:
Calculating dimensions
Determining the shape
Finding blob location
Interpreting the identifier image
Steps to performing blob analysis
The area and perimeter

Reference:
BlobAnalysis.Calculate
BlobAnalysis.FeatureList
BlobAnalysis.GrayscaleImage
BlobAnalysis.IdentifierBlobType
BlobAnalysis.Image
BlobAnalysis.LabelImage
BlobAnalysis.Results
BlobAnalysis.Results.Draw
BlobAnalysis.SortingKeys.DecreasePriority
BlobAnalysis.SortingKeys.IncreasePriority
BlobAnalysis.SortingKeys.Item.Feature
BlobAnalysis.SortingKeys.Item.Ordering

Calculating features

The BlobAnalysis control can calculate a variety of different blob measurements or features, such as the area, perimeter, Feret diameter, and center of gravity of each selected blob. Although the Calculate method initiates the actual calculations, it is the feature list that determines which calculations to perform. Initially, no features are selected for calculation in the FeatureList property.

Binary and grayscale features

The BlobAnalysis control supports calculation of both binary and grayscale features. When selecting a binary feature, all calculations are performed using only the blob identifier image specified with the Image property. When grayscale features are selected, you must also provide a grayscale image (specified with the GrayscaleImage property). In the latter case, the blob identifier image determines which pixels in the grayscale image belong to which blob.

Selecting features

When you use the Calculate method, the blob identifier image is scanned to locate blobs, and any selected features are calculated. Even if only a few features are selected, the overhead of scanning the image can be considerable. Therefore, it is usually more efficient to select many features and make one call to the Calculate method, rather than to select and calculate one feature at a time. If several calls are made to the Calculate method with the same blob identifier and grayscale images, features calculated in one call are not recalculated in subsequent calls. However, if you then use different images or if you change the processing mode (IdentifierBlobType, Lattice, PixelAspectRatio, NumberOfFeretAngles, ForegroundPixelValue and IdentifierPixelType properties), any result already in the Results property becomes invalid and will be discarded. Therefore, it is more efficient to use the BlobAnalysis control exclusively in one processing mode with the same blob identifier and grayscale images.

Which features to calculate

There are several considerations when selecting features:

  • Before selecting a feature for calculation, you should take the blob shapes into consideration. Some features are more appropriate for certain blob shapes than for others. For example, some should be used for round blobs rather than long, thin ones, and vice versa.
  • When trying to distinguish between two similar blobs, selection of certain features, rather than some other features that might also seem appropriate, might reveal a more notable difference.
  • If two features allow you to come to the same conclusion, it is recommended that you select the one that is calculated more quickly. For example, features derived from multiple Feret diameters tend to calculate relatively slowly, and grayscale calculations are considerably longer than binary ones.

Note, if you want a visual representation of blobs that meet (or do not meet) certain criteria, then after calculating some features, you can call the Draw or LabelImage methods to draw or label these blobs.

The LabelImage method will label blobs depending on the IdentifierBlobType property's settings. If the IdentifierBlobType property is set to blobLabeled mode then the blobs will be labeled with their own labels, however if the IdentifierBlobType property is set to blobIndividual or blobEntireImage, the index value of the blob(s) in the results collection is used to label the blob(s).

Drawing results

You can use the Results.Draw method to draw specific features of the results in an image buffer. For example, you can draw the blobs contours, holes, position, minimum and maximum Feret diameters, among other features. You can use a previously allocated graphic context (see the Graphics chapter) to control the drawing color, or use the default graphic context. You can draw directly into the image, or annotate the image non-destructively by drawing into its display's overlay image (see the Annotating the displayed image non-destructively section).

Sorting results

The results from the Results property can be sorted in increasing or decreasing order (using the Ordering property). A maximum of three features (specified by Feature property) can be assigned as sorting keys. Sorting keys are automatically assigned a priority according to the order in which they are selected. You can increase or decrease the priority of a sorting key using the IncreasePriority and DecreasePriority methods. For example, if you want to increase the priority of a priority 2 sorting key, then the priority one and the priority two sorting keys will switch priorities.


The area and perimeter

User Guide:
Calculating dimensions
Interpreting the identifier image
Making feature extractions
Steps to performing blob analysis

Reference:
BlobAnalysis.FeatureList.Area
BlobAnalysis.FeatureList.ConvexPerimeter
BlobAnalysis.FeatureList.Perimeter
BlobAnalysis.NumberOfFeretAngles
BlobAnalysis.PixelAspectRatio

The pixel

Each pixel in your image represents a real width and height (for example, in millimeters). However, by default all results from the blob analysis methods (that represent a distance or area) are expressed in raw (uncalibrated) pixel units. You are left with the task of converting these results to actual physical units. This task is made easier if the width and height of the pixels are the same (that is, the pixel aspect ratio (width/height) equals 1.0). In this case, each pixel (P) is represented as follows:

The area and perimeter

A pixel aspect ratio of 1.0 implies that the area of a single pixel blob is equal to 1 and the perimeter is equal to 4. When calculating the area (Area property) and perimeter (Perimeter property) of a larger blob, the area would then equal the number of pixels in the blob (excluding holes), and the perimeter would equal the total number of pixel sides along the blob edges (including the edges of holes). Note an allowance is made for the staircase effect that occurs in a digital image when representing diagonals and curves. For example, in the following blob (where F represents foreground pixels), the area is 10 and the perimeter is 14.242.

The convex perimeter

You can also calculate an approximation of the convex perimeter (ConvexPerimeter property) of the blobs. The convex perimeter is the perimeter of the convex hull (see below):

This feature is derived by taking the diameter of the blob at different angles (known as the Feret diameter). You can adjust the number of Feret diameters using NumberOfFeretAngles property. The greater the number of Feret diameters used, the more accurate the approximation.

The aspect ratio

When the pixel aspect ratio (PixelAspectRatio property) has been set to anything other than 1.0, the aspect ratio is applied to the pixel width during calculations. Each pixel is now represented as:

This affects all calculated features as if you had actually stretched the image (from the top-left corner in the X-direction only), by a factor equal to the pixel aspect ratio. We can no longer say that results are in "pixel" units. In fact, results are really in units of "pixel height" since the height is not affected by the aspect ratio.


Calculating dimensions

User Guide:
Making feature extractions
Steps to performing blob analysis
The area and perimeter

Reference:
BlobAnalysis.FeatureList.Box.FeretX
BlobAnalysis.FeatureList.Box.FeretY
BlobAnalysis.FeatureList.Breadth
BlobAnalysis.FeatureList.Elongation
BlobAnalysis.FeatureList.FeretElongation
BlobAnalysis.FeatureList.GeneralFeret
BlobAnalysis.FeatureList.Length
BlobAnalysis.FeatureList.MaximumFeretAngle
BlobAnalysis.FeatureList.MaximumFeretDiameter
BlobAnalysis.FeatureList.MeanFeretDiameter
BlobAnalysis.FeatureList.MinimumFeretAngle
BlobAnalysis.FeatureList.MinimumFeretDiameter
BlobAnalysis.NumberOfFeretAngles

The Feret diameter

Besides the area and perimeter, you might need to determine the dimension of the blobs. Since blobs are not typically rectangular in shape, you will probably have to take the length (or diameter) of the blobs at various angles from the horizontal axis. This is actually one of the many definitions of the blob length, called the Feret diameter. Several Feret diameters are illustrated below. Note, the angle at which the Feret diameter is taken (relative to the horizontal axis) is specified as a subscript to the F.

Calculating different Feret diameters

With ActiveMIL, you can calculate the Feret diameter at a specified angle by selecting the GeneralFeret property in the feature list. To select the Feret diameter at 0° (horizontal Feret diameter) and 90° (vertical Feret diameter) in the feature list, you can use the FeretX and the FeretY properties.

You can determine the minimum, maximum, and average Feret diameters of the blob using the MinimumFeretDiameter, the MaximumFeretDiameter or the MeanFeretDiameter properties, respectively. These diameters will be determined by testing the diameter of the blobs at several angles.

You can use the NumberOfFeretAngles property to change the default number of angles; these angles will start at 0° and increase in increments of 180°/(number of Feret diameters). Increasing the number of angles that are tested increases the accuracy of the results, but also increases processing time.

Note, the maximum Feret diameter is not very sensitive to the number of angles; using 8 angles usually produces an accurate result. The minimum diameter, however, can be inaccurate for long thin blobs unless many angles are used.

The angles at which the minimum and maximum Feret diameter were found can be determined using the MinimumFeretAngle and MaximumFeretAngle properties, respectively.

You can determine the ratio of the maximum to minimum Feret diameter by using the FeretElongation property.

Dimensions of long thin blobs

Although the Feret diameters provide a good approximation of the blob size, these features are not very good for long, thin blobs, even when using the maximum number of angles (NumberOfFeretAngles property). For these, the following features, might provide better results:

  • Length: an extraction of the true length of a blob (FeatureList.Length property).
  • Breadth: an extraction of the true breadth of a blob (FeatureList.Breadth property).
  • Elongation: the ratio of the length to the breadth (FeatureList.Elongation property).

These features are derived from the area and the perimeter, using the assumption that the blob area is equal to the [length x breadth] and the perimeter is equal to [2(length + breadth)]. These relations only hold if the length and breadth are constant throughout a blob. However, long, thin blobs generally satisfy this assumption, even if they are not straight.

Note, since these features use only the area and perimeter, they are faster to calculate than Feret features.


Determining the shape

User Guide:
Making feature extractions
Steps to performing blob analysis

Reference:
BlobAnalysis.FeatureList.Compactness
BlobAnalysis.FeatureList.Roughness

Other useful features during classification are those that give you information about the blob shape. Two blobs can have similar sizes but different shapes because of a different number of holes, curves, or edges:

For example, in the illustration above, the blobs have similar sizes, but can be distinguished by the shape of their holes. If you treat the holes as the actual blobs (set non-zero pixels as foreground pixels and zero pixels as background pixels), you can extract the differences in shape of the holes.

Compactness and roughness

Two features that can qualify the shape of these holes are:

  • Compactness property
  • Roughness property

Compactness is a measure of how close all particles in the blob are from one another. It is derived from the perimeter and area. A circular blob is most compact and is defined to have a compactness measure of 1.0 (the minimum); more convoluted shapes have larger values.

The roughness is a measure of the unevenness or irregularity of a blob's surface. It is a ratio of the perimeter to the convex perimeter of a blob. Smooth convex blobs have a roughness of 1.0, whereas rough blobs have a higher value because their true perimeter is bigger than their convex perimeter.

Although either of these features can be used in the classification process, compactness is faster to calculate since it is derived using only the area and perimeter.


Finding blob location

User Guide:
Making feature extractions
Steps to performing blob analysis

Reference:
BlobAnalysis.FeatureList
BlobAnalysis.FeatureList.Box
BlobAnalysis.FeatureList.CenterOfGravity
BlobAnalysis.FeatureList.Chains
BlobAnalysis.FeatureList.ContactPoints
BlobAnalysis.FeatureList.GeneralMoment
BlobAnalysis.Results.Item.Chains
BlobAnalysis.Results.Item.Chains.Count
BlobAnalysis.Results.Item.Chains.Count
BlobAnalysis.Results.Item.Chains.Item
BlobAnalysis.Results.Item.Chains.Item.Pixels
BlobAnalysis.Results.Item.Chains.Item.Pixels.Item.X
BlobAnalysis.Results.Item.Chains.Item.Pixels.Item.Y

Finding the location of blobs in an image can sometimes be more useful than finding their shape or size. For example, if a robotic arm needs to pick up several items regardless of their type, it can use their location in an acquired image to determine their actual physical position.

You can also use the blob location to determine if a blob touches the image borders. If there are any such blobs, you might want to adjust the camera's field of view so that all items are completely represented in the image, or you might want to exclude these blobs.

Blob points

You can determine the blob points shown in the diagram below by selecting the following features in the feature list: the Box, ContactPoints and CenterOfGravity properties:

Chained pixels

You can obtain the coordinates of pixels bordering blobs or delimiting holes in blobs, in a counterclockwise or clockwise direction, respectively. These pixels are referred to as chained pixels (Chains property).

You can use the chained pixel coordinates to create a chain code. A chain code is a directional code that records an object's boundary as a discrete set of vectors, where each vector points to the next pixel in the chain.

Chained pixels always form a closed chain. This implies that the starting pixel in the chain is also the closing one. If your blob has regions which are 1 pixel wide, these pixels are chained twice, once in the forward direction and then in the opposite direction.

In the diagram that follows, the thick lines illustrate pixels that are chained twice. The diagram also illustrates chained pixels of a blob in an 8 and 4-connected lattice, where the solid lines illustrate chained pixels in an 8-connected lattice, and the dotted lines illustrate how chained pixels deviate in a 4-connected lattice. Also, note that the blob's outermost chain is identified as index 1. Chains that delimit holes in blobs are identified by subsequent indexes.

You can obtain the number of chains and a particular chain of a blob using the Count and Item properties of the Chains property, respectively. You can also obtain the number of chained pixels and the X- and Y-coordinates of the chained pixels using the Count, X and Y properties of the Pixels property, respectively.

Moments

Using the BlobAnalysis control, you can also calculate the moments used to find the center of gravity, as well as other grayscale or binary moments. The GeneralMoment property allows you to select any moment in the feature list. The FeatureList property also contains some of the more common central and ordinary moments.

Central moments use coordinates that are relative to the center of gravity of the blob, and therefore are independent of a blob's position within the image, whereas, ordinary moments are affected by the blob position because they use coordinates relative to the top-left corner of the image.


Location, length and number of runs

User Guide:
Making feature extractions
Steps to performing blob analysis

Reference:
BlobAnalysis.Results.Item
BlobAnalysis.Results.Item.Runs.Count
BlobAnalysis.Results.Item.Runs.Item.Length

A run is defined as a horizontal string of consecutive foreground pixels. The BlobAnalysis control can be used to obtain the total number of runs (using the Count property) for each blob. Results are obtained using the Results(BlobIndex) property for each blob.

To obtain the length of a run in a specific blob, use the Length property of the Results(BlobIndex) property. The runs that make up each blob can be used to calculate features that are not supported directly by ActiveMIL.


Blob reconstruction

Reference:
BlobAnalysis.EraseBorderBlobs
BlobAnalysis.ExtractHoles
BlobAnalysis.FillHoles
BlobAnalysis.LabelImage
BlobAnalysis.Reconstruct
BlobAnalysis.Results.Draw

Although the BlobAnalysis control is used mainly for blob feature calculation purposes, some of the blob methods can be used to perform blob image reconstruction. Blob reconstruction allows you to remove small particles in an image without distorting the shape of the remaining blobs.

Using the different blob reconstruction methods, you can:

  • Reconstruct blobs from a seed image (Reconstruct method). This method copies to the destination image only those blobs that have a corresponding seed pixel in the seed image. Blobs that are not seeded are replaced according to the selected processing mode.

    • Note that when using this method with ProcessingMode set to blobGrayscale and ForegroundPixelValue set to blobZero, blobs in the source image which are not seeded are filled with the average grayscale value of the background.

  • Remove blobs that touch the border of the image (EraseBorderBlobs method). This method copies to the destination image only those blobs that do not touch the border.

    • Note that when using this method with ProcessingMode set to blobGrayscale and ForegroundPixelValue set to blobZero, border blobs are filled with the average grayscale value of the background.

  • Fill holes in blobs (FillHoles method). This method copies the source image to the destination image, filling those blobs that contain holes according to the selected processing mode. Holes that touch the image border are not considered to be holes.

    • Note that when using this method with ProcessingMode set to blobGrayscale, the holes are filled with the average grayscale value of the blob.

  • Extract holes from blobs (ExtractHoles method). This method copies to the destination image only the holes within the blobs found in the source image. Holes that touch the image border are not considered to be holes.

    • Note that when using this method with ProcessingMode set to blobGrayscale, the copied blob holes are filled with the average grayscale value of the blob in the source image. Also, when ProcessingMode is set to blobGrayscale and ForegroundPixelValue is set to blobZero, the blobs are filled with the average grayscale value of the background.

Finally, you can perform other types of blob reconstruction by using specific ActiveMIL methods, one after the other. For example, to copy only blobs that exceed a particular size from a source image to a destination image, use the Calculate method in conjuction with LabelImage and Draw methods.


Steps to performing blob analysis

See Also | Previous
Reference:
BlobAnalysis.ApplyFilter
BlobAnalysis.Calculate
BlobAnalysis.FeatureList
BlobAnalysis.Results.Item
BlobAnalysis.Results.Item.Runs
BlobAnalysis.Results.Item.Runs.Count
ImageProcessing.Close
ImageProcessing.Open

Although there is a multitude of features that can be calculated and used during the elimination or the analysis process, the following is a typical series of steps to perform:

  1. Grab or load an image that was captured under the best possible conditions.
  2. If necessary, reduce the amount of noise in the image. (Noise makes the next step more difficult.)
  3. Segment the image so that blobs are separated from the background and from each other. Typically, this involves binarizing the image so that the background is in one state (zero or non-zero) and the blob pixels are in the other state. This image is known as the blob identifier image. If you plan to perform grayscale calculations, you will need the original grayscale image as well.
  4. If there are too many noise particles, calculation time will be increased, or the number of blobs might exceed the maximum number that can be handled by ActiveMIL. An opening operation (Open method) for non-zero blobs or a closing operation (Close method) for zero blobs will remove most of the noise particles without affecting real blobs significantly. You might also need to separate touching blobs at this stage (or they will be counted as a single blob).
  5. If necessary, adjust the pixel aspect ratio, to set when to consider two pixels touching (along horizontal and vertical only or also along the diagonal), and if values in the blob identifier image can have any value or must be set to the maximum value of the image (grayscale or binary). For example, the maximum value of an 8-bit image is 255.
  6. Select the feature(s) to calculate, using the FeatureList property. This list is used to specify the features that should be calculated. By default, the feature list is empty; no features are selected.
  7. Calculate the required features and analyze the results. This involves the following:
    • Calculate results for the selected features, using the Calculate method.
    • If necessary, exclude or delete blobs that do not meet the criteria, using the ApplyFilter method. Results for the excluded or deleted blobs will not be returned. Excluded blobs will be ignored in future calculations, while deleted blobs will be removed from the blob analysis results altogether.
    • Obtain results for a particular blob using the Results.Item property. You can get the number of runs in the blob using Count property of the Runs property.

    You can repeat this step until you obtain all required results for the blobs of interest. Note the process of excluding or deleting unwanted blobs and then calculating more features is the preferred method if you have many unwanted blobs. If this is not the case, it is often faster to calculate all the required features for all the blobs with a single call to the Calculate method, and then exclude or delete unwanted blob results afterwards.


Pattern Matching

Chapter 7: Pattern Matching

This chapter explains quick techniques to perform alignment operations, using the ActiveMIL PatternMatching control.

  • Pattern matching - overview
  • Simple vertical and horizontal alignment techniques
    • A wafer alignment example
  • Performing a search
    • Steps to perform a general search with a model
  • Define the model
  • Rotation
    • Creating rotated versions of the model
    • Extracting models at different rotations
    • Setting the angle of search
    • Determining the rotation tolerance of a model
  • Masking the model
  • Search constraints
    • Specifying the number of matches
    • Setting the acceptance level
    • Setting the certainty level
    • Redefining the model's reference position
    • Selecting the search region
    • Positional accuracy
    • Setting the speed
  • Preprocessing the search model
  • Finding the model
  • Speeding up the search
    • Choose the appropriate model
    • Adjust the search speed
    • Effectively choose the search region and search angle
    • Searching for multiple models at the same time
  • The pattern matching algorithm (for advanced users)
    • Normalized correlation
    • Hierarchical search
    • Search heuristics
    • Subpixel accuracy


Pattern matching - overview

See Also | Next
Reference:
PatternMatching

ActiveMIL's PatternMatching control uses normalized grayscale correlation to help solve machine vision problems such as alignment, measurement, and inspection of objects. The control also provides quick techniques to find horizontal, vertical, and angular displacement of most images.

The main function of the pattern matching control is to search for occurrences of a pattern in an image. ActiveMIL refers to the pattern for which you are searching as the search model and the image from which it is extracted as the model's source image.

To perform pattern matching, you need a PatternMatching control, a System control, one Image control from which to extract the model, and one Image control to use as the target image to search for occurrences of the model(s). You can perform pattern matching on 1-band, 8-bit unsigned images only.

The image being searched is called the target image.

For an example that uses the PatternMatching control, see MModelTracking.


Simple vertical and horizontal alignment techniques

User Guide:
Define the model
Finding the model
Images - overview
Preprocessing the search model

Reference:
PatternMatching.FindModel
PatternMatching.Image
PatternMatching.Models.AddAutomatic
PatternMatching.Models.Item.OriginalPositionX
PatternMatching.Models.Item.OriginalPositionY
PatternMatching.Models.Item.Preprocess
PatternMatching.Results.Item.PositionX
PatternMatching.Results.Item.PositionY

ActiveMIL can find the vertical and horizontal displacement of a target image by comparing the location of a unique model, taken from an aligned source image, with its actual location in the target image. A unique model can be chosen from any location in the aligned image as long as the model is known to appear in a shifted target image.

This can be done by following these steps:

  1. Create a model from a source image automatically, using the Models.AddAutomatic method.

    2. This method creates the best unique search model for a given source image.

  2. Once the model is defined, use the Models.Item(ModelIndex).Preprocess method to train the system to find the model in the most efficient manner. This method analyzes the model and determines which shortcuts can be safely used during the search.
  3. Set the Image property to your target image.
  4. Now, you are ready to find the model in the target image. To do so, call the FindModel method.
  5. Compare the model's coordinates in the model's source image with those in the target image, to obtain the vertical and horizontal displacement of the target image.
  6. The coordinates resulting from a search return the reference position (Results.Item(ResultIndex).PositionX and Results.Item(ResultIndex).PositionY properties) of the model, relative to the top-left corner of the target image. To find the equivalent coordinates in the model's source image, use Models.Item(ModelIndex).OriginalPositionX and Models.Item(ModelIndex).OriginalPositionY properties.

Note, once the model is defined, you can perform the search operation on an unlimited number of target images.

A wafer alignment example

For an example of how to find the vertical and horizontal displacement of a wafer image, see MPatternMatching. By executing MPatternMatching, you will find that shfwafer.mim is shifted by approximately 50 pixels horizontally and 20 pixels vertically.


Performing a search

User Guide:
Define the model
Finding the model
Masking the model
Preprocessing the search model
Search constraints
Setting the angle of search
Simple vertical and horizontal alignment techniques

Reference:
PatternMatching.FindModel
PatternMatching.Models.Item.Save
PatternMatching.Models.Load

In the Simple vertical and horizontal alignment techniques section, we discussed some quick techniques to determine the alignment of a target image or an object in a target image. This section looks at defining a search model and finding the occurrences of this model in the target image to help solve machine vision problems such as ones listed below:

  • In machine guidance applications, mechanical devices need to be informed of the location of parts to be picked. Therefore, the search model must be specific to the part in question; it cannot be for an arbitrary location.
  • When performing an alignment using gauging techniques, the location of two or more points of reference (fiducial marks) is required. To perform this process, you must define your own model to uniquely identify the reference points.

Steps to perform a general search with a model

The steps involved in performing a search with a model are as follows:

  1. Define the model from the model's source image.
  2. Optionally, specify a range for angular search.
  3. Optionally, set the model's "don't care" pixels to exclude certain pixels from the search process.
  4. Specify the model's search constraints.
  5. Preprocess the model.
  6. Grab a target image. Optionally, process it to improve its quality.
  7. Find the model in the target image.
  8. Read the search results.

In general, the first five steps are performed once, while steps six through eight are repeated as required. Note, in practice, models and their settings are usually saved on disk, using the Models.Item(ModelIndex).Save method; therefore by using the Models.Load method, you can replace steps 1 through 5 by restoring a saved model from disk.

For an example that defines a model and then searches for it, see MPatternMatching.


Define the model

User Guide:
Masking the model
Overview of image processing
Speeding up the search

Reference:
ImageProcessing
PatternMatching.Models.Add
PatternMatching.Models.AddAutomatic

At design-time, you can define a model as follows:

  1. Select the Models property page.
  2. Click on Add. The Add Model dialog box is presented.
  3. Select the model source image by clicking on the Browse button.
  4. Select the Model Position button (under the displayed source image).
  5. In the source image, double click, and drag the mouse to define your model.
  6. Select the model type from the Model type list box.
  7. Click on Add.

Alternatively, at run-time, use the Models.Add method to define which portion of the model's source image is to be used as your model, or use the Models.AddAutomatic method to have ActiveMIL automatically select the model for you. This image must be of the best quality possible. If your images tend to be noisy, try to clean the image, using methods of the ImageProcessing control. Upon creation, the model is extracted from the selected region in the model's source image. The model's source image is then no longer needed. When creating a model, you must specify its size. Generally, relative to the target image, small models take longer to find than larger ones, although very large models can also be time-consuming.

To view the portion of the image from which the model was extracted, use the Models..Item(ModelIndex).Draw method with patDrawImage.

You can use the Models..Item(ModelIndex).Draw method to draw the various model features. You can use a previously allocated graphic context (see the Graphics chapter) to control the drawing color, or use the default graphic context. You can draw directly into the image image, or annotate the image non-destructively by drawing into its display's overlay image (see the Annotating the displayed image non-destructively section).


Rotation

User Guide:
Define the model
Preprocessing the search model
Search constraints
Speeding up the search

Reference:
ImageProcessing.Rotate
PatternMatching.FindModel
PatternMatching.Models.Item.Preprocess
PatternMatching.Models.Item.SearchAngle

There are two ways to search for a model that can appear at different angles using the FindModel method:

  • Search for rotated versions of the model.
  • Search for models that would appear upright if the target image was rotated.

The following describes how to create the models to perform these types of searches.

Creating rotated versions of the model

To implement the first type of search, create a patNormalized type model (see Defining a model). Then, enable and specify the angular range in which it can appear (see Setting the angle of search). When you preprocess the model (see Preprocessing the search model), it will internally create different models by rotating the original model at the required angles, assigning don't care pixels to regions that do not have corresponding data in the original model (see figure below).

A patNormalized type model is used usually when the pixels surrounding the model follow no predictable pattern, such as an image of loose nuts and bolts lying on a metal sheet.

Extracting models at different rotations

To implement the second type of search, create a patNormalized + patCircularOverscan type model (see Defining a model); this will extract the model as well as circular overscan data from the model's source image. Specifically, ActiveMIL extracts the region enclosed by a circle which circumscribes the model. Then, enable and specify the angular range in which the model can appear (see Setting the angle of search). When you preprocess the model (see Preprocessing the search model), it will internally extract different orientations of the model from the overscanned model.

It is recommended that the model be as square as possible: the longer the rectangle, the smaller the number of consistent central pixels in every model. Therefore, this type of model should only be used when the model's distinct features lie in the center of the region, so that they are included in all models when rotated.

As mentioned, a larger region than the one defined will be fetched from the model's source image. Therefore, the model must not be extracted from a region too close to the edge of the model's source image.

The pixels surrounding the model should be relevant to the positioning of the pattern (that is, the model should appear in the target image with the same overscan data). An example is the image of an integrated circuit.

Both types of models find the position and match score of the model in a target image.

Finally, it should be noted that ActiveMIL's implementation of a FindModel method with a patCircularOverscan type model is significantly faster than that of a patNormalized model when performing an angular search.

Setting the angle of search

By default, the angle of search is 0 degrees. However, you can specify a rotational range of up to 360 degrees, as well as the required precision of the resulting angle and the interpolation mode used for the rotated model.

To perform a search at an angle, you must enable the angular search mode and specify the constraints of the angular search. At design-time, you can enable angular search mode and specify the limits of the search, by clicking on the Category View button (identified by the ">>") in the Models property page, selecting the Search Angle view, and setting the angular search values. At run-time, enable angular search by setting the Models.Item(ModelIndex).SearchAngle.Enabled property to True and then set the angular search limits for the specified model, using the properties of the Models.Item(ModelIndex).SearchAngle property.

These settings can influence the speed of the search significantly. The accuracy of the search can also be influenced.

If an angular search has been specified, the Models.Item(ModelIndex).Preprocess method creates a model for every x degrees within the range, where x is determined by the specified angular tolerance (Models.Item(ModelIndex).SearchAngle.Tolerance property). Tolerance defines the full range of degrees within which the pattern in the target image can be rotated from a model at a specific angle and still meet the acceptance level.

After the approximate location is found, ActiveMIL fine-tunes the search, according to the specified accuracy (Models.Item(ModelIndex).SearchAngle.Accuracy property). To be effective, you must set the degree of accuracy to a value smaller than that of the rotation tolerance.

When searching within a range of angles, you should use as narrow a range as possible, since the operation can take a long time to perform. Note that the model is rotated according to the interpolation mode (Models.Item(ModelIndex).SearchAngle.InterpolationMode property).

Determining the rotation tolerance of a model

Every model has its own particular rotation tolerance. This tolerance is dependent on the individual model characteristics and surrounding target image features. To determine the rotation tolerance of a model:

  1. Set the search angle of a model to the same angle as the sought for pattern in a sample target image. However, set the positive and negative delta values to zero since you want to test by how much a pattern in an image can be rotated and still correlate with a model at a specific angle.
  2. Rotate the image in very small increments, for example 0.5 degrees, in the positive direction, using the Rotate method of the ImageProcessing control and perform a find model operation (FindModel method) at every angle. Make sure that the image's center of rotation is the same as that of the model, otherwise the resulting tolerance will not be accurate. Note, when rotating the image, always set the angle from the image's original position to avoid interpolating the image more than once.
  3. Check the results for the greatest angle that produces an acceptable score.
  4. Repeat steps 1 through 3, rotating the image in the negative direction.
  5. Take the minimum of the absolute value of these angles. Double this angle and set it as the rotation tolerance for the angular search.


Masking the model

User Guide:
Define the model

Reference:
GraphicContext
PatternMatching.Models.Item.ImportDontCareImage

Often your search model contains regions that you need ActiveMIL to ignore when searching for a model in the target image. These regions might be noise pixels or simply regions that have nothing to do with what you are searching for. They might also be edge transitions.

For instance, in a machine guidance application, a mechanical device might need to know where mounting holes are located on a circuit board so that screws can be properly inserted. In this case, a mounting hole would be the search model and the circuit board would be the target image.

In the above image, the search model contains too much of the actual board; it might not match holes in different areas of the board.

In such cases, parts of the search model can be masked by setting pixel values in certain regions to "don't cares". ActiveMIL then ignores these regions when searching for occurrences of the model. You can set pixels in the model to the "don't care" state by copying the image representation into an image and then setting the "don't care" pixels. These pixels will not be considered when finding occurrences of the model in a target image.

In our example, you would need to mask the edges of the search model, as follows:

The unmasked region of the search model now more closely resembles the pattern for which to search; it is more circular in shape and contains little of the actual board.

To create a mask at design-time:

  1. Define a model
  2. Now your model is displayed in the window. Select the Don't Cares button (under the displayed source image).
  3. Set the pen width to draw don't care pixels using the Pen Width slider.
  4. Draw don't care pixels in the displayed model.

To create a mask at run-time:

  1. View the portion of the image from which the model was extracted. To do so, call the Models..Item(ModelIndex).Draw method with patDrawImage.
  2. Clear the pixels that should be set to "don't care", using the GraphicContext control.
  3. Import the don't care pixels into the model using the Models..Item(ModelIndex).ImportDontCareImage method, specifying the image along with the foreground color used to draw the "don't care" pixels.
  4. To view the mask, call Models..Item(ModelIndex).Draw method with patDrawDontCares.

When you change the "don't care" pixels of a model, you should preprocess the search model again.


Search constraints

User Guide:
Performing a search
Speeding up the search
Speeding up the search

Reference:
PatternMatching.FindModel
PatternMatching.Models.Item.AcceptanceThreshold
PatternMatching.Models.Item.CertaintyThreshold
PatternMatching.Models.Item.NumberOfOccurrences
PatternMatching.Models.Item.PositionAccuracy
PatternMatching.Models.Item.ReferenceX
PatternMatching.Models.Item.ReferenceY
PatternMatching.Models.Item.SearchRegion
PatternMatching.Models.Item.Speed
PatternMatching.Results.Count
PatternMatching.Results.Item.PositionX
PatternMatching.Results.Item.PositionY

When search models are created (whether manually or automatically), they are assigned a set of default search constraints. Some of these constraints can be changed. You can change the following constraints:

  • The number of occurrences to find.
  • The threshold for acceptance and certainty.
  • The model's reference position.
  • The search to a specific region of the target image.
  • The positional accuracy.
  • The search speed.

Specifying the number of matches

You can specify how many matches to try to find. At design-time, click on the Category View button (identified by the ">>") in the Models property page, select the Attributes view, and set the Number of occurrences edit field. At run-time, to specify the number of matches to search for, use the Models.Item(ModelIndex).NumberOfOccurrences property.

If all you need is one good match, set it to one (the default value) and avoid unnecessary searches for further matches.

If a correlation has a match score above the certainty level, it is automatically considered an occurrence (default 80%), the remaining occurrences will be the best of those above the acceptance level.

Setting the acceptance level

The level at which the correlation (match score) between the model and the pattern in the image is considered a match is called the acceptance level.

At design-time, click on the Category View button in the Models property page, select the Attributes view, and set the Acceptance threshold edit field. To set the acceptance threshold at run-time, use the Models.Item(ModelIndex).AcceptanceThreshold property.

If the correlation between the occurrence in the target image and the model (also referred to as the match score) is less than this level, they are not considered a match. A perfect match is 100%, a typical match is 80% or higher (depending on the image), and no correlation is 0%. If your images have considerable noise and/or distortion, you might have to set the level below the default value of 70%. However, keep in mind that such poor-quality images increase the chance of false matches and will probably increase the search time.

Note, perfect matches are generally unobtainable because of noise obtained during a grab.

When you ask for a specific number of matches (using the Models.Item(ModelIndex).NumberOfOccurrences property), the FindModel method might not find that number because the acceptance level affects the number of results returned; only those matches with a score greater than or equal to the acceptance level will be counted. You should always use the Results.Count property to see how many occurrences were actually found. When multiple occurrences are found, the results are returned in decreasing order of match score (that is, best match first).

Setting the certainty level

The certainty level is the match score (usually higher than that of the acceptance level) above which the algorithm can assume that it has found a very good match and can stop searching the rest of the image for a better one. The certainty level is very important because it can greatly affect the speed of the search. To understand why, you need to know a little about how the search algorithm works.

Since a brute force correlation of the entire model, at every point of the image, would take several minutes, it is not practical. Therefore, the algorithm has to be intelligent. It first performs a rough but quick search to find likely match candidates, then checks out these candidates in more detail to see which are acceptable.

A significant amount of time can be saved if several candidate matches never have to be examined in detail. This can be done by setting a certainty level that is reasonable for your needs. A good level is slightly lower than the expected score. If you absolutely must have the best match in the image, set the level to 100%. This would be necessary if, for example, you expect the target image to contain other patterns that look similar to your model. Unwanted patterns might have a high score, but this will force the search algorithm to ignore them. Symmetrical models fall into this category. At certain angles symmetrical models might seem like an occurrence in the target image, but if the search was completed, a match with a higher score would be found.

At design-time, you can set the certainty level for the specified model, by clicking on the Category View button in the Models property page, selecting the Attributes view, and setting the Certainty threshold edit field. To set the certainty level at run-time, use the Models.Item(ModelIndex).CertaintyThreshold property.

Often, you know that the pattern you want is unique in the image, so anything that reaches the acceptance level must be the match you want; therefore, you can set the certainty and acceptance levels to the same value.

Another common case is a pattern that usually produces very good scores (say above 80%), but occasionally a degraded image produces a much lower score (say 50%). Obviously, you must set the acceptance level to 50% or you will never get a match in the degraded image. But what value is appropriate for the certainty level? If you set it to 50%, you take the risk that it will find a false match (above 50%) in a good image before it finds the real match that scores 90%. A better value is about 80%, meaning that most of the time the search will stop as soon as it sees the real match, but in a degraded image (where nothing reaches the certainty level), it will take the extra time to look for the best match that reaches the acceptance level.

Redefining the model's reference position

The coordinates returned by the Results.Item(ResultIndex).PositionX and Results.Item(ResultIndex).PositionY properties, after a call to the FindModel method, are the coordinates of the reference position of the model's occurrence, relative to the top-left corner of the target image. By default, this reference point is defined to be at the geometric center of the model.

If there is a particular spot from which you would like results returned, you can change the model's reference position. At design-time, click on the Category View button in the Models property page, select the General view and click the Ref. Point button under the displayed model. Click once anywhere in the model to specify the reference position. At run-time, use the Models.Item(ModelIndex).ReferenceX and Models.Item(ModelIndex).ReferenceY properties.

For example, if your model has a hole and you want to find results with respect to this hole, change the reference position of the model to this position. Note that you can define the reference position to be outside of the model's boundary.

Selecting the search region

Instead of searching the entire region of an image, you can limit the search region. At design-time, click on the Category View button in the Models property page, select the Search Region view, click on the Interactive button, and load your target image. In the displayed target image, double click, hold the mouse button down and drag the mouse to define your search region. At run-time, use the Models.Item(ModelIndex).SearchRegion property.

The search region is the region in which to find the model's reference position. Therefore, the search region can even be smaller than the model. If you have redefined the model's reference position (with the Models.Item(ModelIndex).ReferenceX and Models.Item(ModelIndex).ReferenceY properties), make sure that the search region defined by the Models.Item(ModelIndex).SearchRegion property covers this new reference position and takes into account the angular search range of the model.

In general, you should not use a child image to delimit the search region to a portion of an image; this might cause the search routine to have border or edge effects and be less accurate (the routine does not assume that there is valid data outside of the image).

Search time is roughly proportional to the region searched; always set the search region to the minimum required when speed is a consideration.

Positional accuracy

You can set the positional accuracy for your search. At design-time, click on the Category View button in the Models property page, select the Attributes view, and choose from the Position accuracy list box. At run-time, use the Models.Item(ModelIndex).PositionAccuracy property.

Positional accuracy can be set to:

  • patLow (typically ± 0.20 pixel)
  • patMedium (typically ± 0.10 pixel)
  • patHigh (typically ± 0.05 pixel)

Note that the required positional accuracy depends on the quality of the model and the target image.

A less precise positional accuracy will speed up the search. Positional accuracy is also slightly affected by the search speed (Models.Item(ModelIndex).Speed property).

Setting the speed

You can also select the algorithm's search speed. At design-time, click on the Category View button in the Models property page, select the Attributes view, and choose from the Speed list box. At run-time, use the Models.Item(ModelIndex).Speed property.

As you increase the speed, the robustness of the search operation (the likelihood of finding a model) decreases. Search speed is discussed at greater length in the section, Speeding up the search.


Preprocessing the search model

User Guide:
Define the model
Search constraints

Reference:
PatternMatching.Models.Item.Preprocess

Once you are ready to search for the created model (manually or automatically), you must preprocess the model. The preprocessing stage uses the known model to decide on the optimal search strategy for subsequent search operations. Preprocessing should be performed after all search constraints have been set.

At design-time, click on the Category View button (identified by the ">>") in the Models property page, select the Attributes view, and click the Preprocess button to preprocess the selected model (the model index is displayed in the Index edit box). At run-time use the Models.Item(ModelIndex).Preprocess method.

If you save the model to disk, the model's preprocessing changes are stored with the model. Upon restoration, the model need not be preprocessed.


Finding the model

User Guide:
Define the model
Search constraints

Reference:
PatternMatching.FindModel
PatternMatching.MultipleModelMode
PatternMatching.Results.Count
PatternMatching.Results.Item.Angle
PatternMatching.Results.Item.ModelIndex
PatternMatching.Results.Item.PositionX
PatternMatching.Results.Item.PositionY

Once the model is defined and the model's search constraints meet your application needs, the target image should be loaded from disk, or acquired from the input device into an image.

You can now search in the target image for model occurrences, using the FindModel method. The search is performed according to the defined model constraints.

You can also search for several models of the same size and search region in the same image, using the FindModel method with the MultipleModelMode property set to True. This mode is used to determine the position of each occurrence for each model. If you have to search for only one model, it is more efficient to set MultipleModelMode property to False before performing the find operation.

To read results, use the Results.Count property to get the number of model occurrences found in the target image. You can obtain the angle, position, and match score for each occurrence using the Results.Item(ResultIndex).Angle property, Results.Item(ResultIndex).PositionX and PositionY properties, and the Results.Item(ResultIndex).Score property, respectively. After a search for multiple number of models, use the Results.Item(ResultIndex).ModelIndex property to determine the model index for each result.

You can use the Results.Item(ResultIndex).Draw method to draw the occurrences' bounding box and/or a cross at the occurrences's reference position.


Speeding up the search

User Guide:
The pattern matching algorithm (for advanced users)

Reference:
PatternMatching.FindModel
PatternMatching.Models.Item.PositionAccuracy
PatternMatching.Models.Item.Preprocess
PatternMatching.Models.Item.SearchAngle.Accuracy
PatternMatching.Models.Item.SearchRegion
PatternMatching.Models.Item.Speed
PatternMatching.MultipleModelMode

To ensure the fastest possible search, you should:

  • Choose an appropriate model.
  • Set the search speed constraint to the highest possible setting for your application.
  • Set the search region to the minimum required. Search time is roughly proportional to the region searched, so don't search the whole image if you don't need to.
  • Search the smallest range of angles.
  • Select the lowest positional accuracy that you need.
  • Set the certainty level to the lowest reasonable value (so that the search can stop as soon as a good match is found).
  • Search for multiple models at the same time, if possible.

Choose the appropriate model

The size of a model affects the search speed. In general, small models take longer to find than larger ones, although very large models can also be time-consuming. In general, the optimal size is approximately 128 x 128 pixels if you are searching a large region (for example, most of the image). Small models are found quickly when the search region is not too large.

Adjust the search speed

For any search model, it is possible to set the speed of the search, using the Models.Item(ModelIndex).Speed property. Increasing the search speed reduces the search time; however, as you increase the speed, the robustness of the search operation (the likelihood of finding a model) can decrease. When you call the Models.Item(ModelIndex).Preprocess method, ActiveMIL analyzes the pattern in the model, and determines what shortcuts are appropriate; only shortcuts that are considered safe for a particular model are taken. This also means that higher search speeds might not be any faster for certain models, depending on the particular pattern. Higher search speeds can reduce the positional accuracy very slightly.

The Models.Item(ModelIndex).Speed property has five settings:

  • patVeryHigh
  • patHigh
  • patMedium
  • patLow
  • patVeryLow

As expected, the patVeryHigh and patHigh speed settings allow the search to take all possible shortcuts, performing the search as fast as possible. Higher speed settings are recommended when searching on a good quality image or when using a simple model. Note, the search might have a lower tolerance for rotation when using these settings.

The patMedium speed setting is the default setting and is recommended for medium quality images or more complex models. A search with this setting is capable of withstanding up to approximately 5 degrees of rotation for typical models.

Use the patLow or the patVeryLow speed settings only if the image quality is particularly poor and you have encountered problems at higher speeds. The speed constraint is discussed further in The pattern matching algorithm section.

Effectively choose the search region and search angle

You can improve performance by not searching the whole image unnecessarily. Search time is roughly proportional to the region searched; set the search region to the minimum required, using the Models.Item(ModelIndex).SearchRegion property. You can also improve performance by selecting the lowest positional accuracy (Models.Item(ModelIndex).PositionAccuracy).

In addition, for an angular search, select the lowest angular accuracy (Models.Item(ModelIndex).SearchAngle.Accuracy property) and range required, in combination with the highest tolerance possible.

Searching for multiple models at the same time

When searching for multiple models of the same size and search region, it is more efficient to set the MultipleModelMode property to true and then call the FindModel method instead of calling the FindModel method once for each model.


The pattern matching algorithm (for advanced users)

See Also | Previous
User Guide:
Image manipulation
Preprocessing the search model
Speeding up the search

Reference:
PatternMatching.FindModel
PatternMatching.Models.Item.Control
PatternMatching.Models.Item.PositionAccuracy
PatternMatching.Models.Item.Preprocess
PatternMatching.Models.Item.SearchAlgorithm.FindMode
PatternMatching.Models.Item.SearchAlgorithm.FirstLevel
PatternMatching.Models.Item.SearchAlgorithm.LastLevel
PatternMatching.Models.Item.Speed
PatternMatching.Results.Item.Score

Normalized grayscale correlation is widely used in industry for pattern matching applications. Although in many cases you do not need to know how the search operation is performed, an understanding of the algorithm can sometimes help you pick an optimal search strategy.

Normalized correlation

The correlation operation can be seen as a form of convolution, where the pattern matching model is analogous to the convolution kernel (see Image manipulation). In fact, ordinary (un-normalized) correlation is exactly the same as a convolution:

In other words, for each result, the N pixels of the model are multiplied by the N underlying image pixels, and these products are summed. Note, the model does not have to be rectangular because it can contain "don't care" pixels that are completely ignored during the calculation. When the correlation function is evaluated at every pixel in the target image, the locations where the result is largest are those where the surrounding image is most similar to the model. The search algorithm then has to locate these peaks in the correlation result, and return their positions.

Unfortunately, with ordinary correlation, the result increases if the image gets brighter. In fact, the function reaches a maximum when the image is uniformly white, even though at this point it no longer looks like the model. The solution is to use a more complex, normalized version of the correlation function (the subscripts have been removed for clarity, but the summation is still over the N model pixels that are not "don't cares"):

With this expression, the result is unaffected by linear changes (constant gain and offset) in the image or model pixel values. The result reaches its maximum value of 1 where the image and model match exactly, gives 0 where the model and image are uncorrelated, and is negative where the similarity is less than might be expected by chance.

In our case, we are not interested in negative values, so results are clipped to 0. In addition, we use r2 instead of r to avoid the slow square-root operation. Finally, the result is converted to a percentage, where 100% represents a perfect match. So, the match score returned by the Results.Item(ResultIndex).Score property is actually:

Note, some of the terms in the normalized correlation function depend only on the model, and hence can be evaluated once when the model is defined. The only terms that need to be calculated during the search are:

This amounts to two multiplications and three additions for each model pixel.

On a typical computer, the multiplications alone account for most of the computation time. A typical application might need to find a 128x128-pixel model in a 512x512-pixel image. In such a case, the total number of multiplications needed for an exhaustive search is 2x5122x1282, or over 8 billion. On a typical computer, this would take a few minutes, much more than the 5 msec or so the search actually takes. Clearly, the FindModel method does much more than simply evaluate the correlation function at every pixel in the search region and return the location of the peak scores.

Hierarchical search

A reliable method of reducing the number of computations is to perform a so-called hierarchical search. Basically, a series of smaller, lower-resolution versions of both the image and the model are produced, and the search begins on a much-reduced scale. This series of sub-sampled images is sometimes called a resolution pyramid, because of the shape formed when the different resolution levels are stacked on top of each other.

Each level of the pyramid is half the size of the previous one, and is produced by applying a low-pass filter before sub-sampling. If the resolution of an image or model is 512x512 at level 0, then at level 1 it is 256x256, at level 2 it is 128x128, and so on. Therefore, the higher the level in the pyramid, the lower the resolution of the image and model.

The search starts at low resolution to quickly find likely match candidates. It proceeds to higher and higher resolutions to refine the positional accuracy and make sure that the matches found at low resolution actually were occurrences of the model. Because the position is already known from the previous level (to within a pixel or so), the correlation function need be evaluated only at a very small number of locations.

Since each higher level in the pyramid reduces the number of computations by a factor of 16, it is usually desirable to start at as high a level as possible. However, the search algorithm must trade off the reduction in search time against the increased chance of not finding the pattern at very low resolution. Therefore, it chooses a starting level according to the size of the model and the characteristics of the pattern. In the application described earlier (128x128 model and 512x512 image), it might start the search at level 4, which would mean using an 8x8 version of the model and a 32x32 version of the target image. You can, if required, force a specific starting level, using the Models.Item(ModelIndex).SearchAlgorithm.FirstLevel property.

The last (lowest) level used is usually determined by the specified positional accuracy; however, you can set this explicitly, using the Models.Item(ModelIndex).SearchAlgorithm.LastLevel property.

The logic of a hierarchical search accounts for a seemingly counter-intuitive characteristic of the FindModel method: large models tend to be found faster than small ones. This is because a small model cannot be sub-sampled to a large extent without losing all detail. Therefore, the search must begin at fairly high resolution (low level), where the relatively large search region results in a longer search time. Thus, small models can only be found quickly in fairly small search regions.

Search heuristics

Even though performed at a very low resolution, the initial search still accounts for most of the computation time if the correlation is performed at every pixel in the search region. On most models, match peaks (pixel locations where the surrounding image is most similar to the model, and correlation results are largest) are several pixels wide. These can be found without evaluating the correlation function everywhere. The Models.Item(ModelIndex).Preprocess method analyzes the shape of the match peak produced by the model, and determines if it is safe to try to find peaks faster. If the pattern produces a very narrow match peak, an exhaustive initial search is performed. The search algorithm tends to be conservative; if necessary, force fast peak finding, using the Models.Item(ModelIndex).SearchAlgorithm.FindMode property with patForceFastPeak.

Using Models.Item(ModelIndex).Control method with patExtraCandidates, you can set the number of extra peaks to consider. Normally, the search algorithm considers only a limited number of (best) scores as possible candidates to a match when proceeding at the most sub-sampled stage. You can add robustness to the algorithm, by considering more candidates, without compromising too heavily on search speed. In addition, you can use the Models.Item(ModelIndex).Control method with patCoarseSearchAcceptance to set a coarse acceptance level. This is the minimum match score, valid at all levels except the last level, to be considered as an occurrence of the model. This ensures that possible models are not discarded at lower levels, yet maintains the required certainty during the final level.

At the last (high-resolution) stage of the search, the model is large, so this stage can take a significant amount of time, even though the correlation function is evaluated at only a very few points. To save time, you can select a high search speed, using the Models.Item(ModelIndex).Speed property. For most models, this has little effect on the score or accuracy, but does increase speed.

Subpixel accuracy

The highest match score occurs at only one pixel position, and drops off around this peak. The exact (subpixel) position of the model can be estimated from the match scores found on either side of the peak. A surface is fitted to the match scores around the peak and, from the equation of the surface, the exact peak position is calculated. The surface is also used to improve the estimate of the match score itself, which should be slightly higher at the true peak position than the actual measured value at the nearest whole pixel.

The actual accuracy that can be obtained depends on several factors, including the noise in the image and the particular pattern of the model. However, if these factors are ignored, the absolute limit on accuracy, imposed by the algorithm itself and by the number of bits and precision used to hold the correlation result, is about 0.025 pixels. This is the worst-case error measured in X or Y when an image is artificially shifted by fractions of a pixel. In a real application, accuracy better than 0.05 pixels is achieved for low-noise images. These numbers apply if you select high-search accuracy, using the Models.Item(ModelIndex).PositionAccuracy property, in which case the search always proceeds to resolution level 0.

If you select medium accuracy (the default), the search might stop at resolution level 1, and hence the accuracy is half of what can be attained at level 0. Selecting low accuracy might cause the search to stop at level 2, so the accuracy is reduced by an additional factor of two (to about 0.2 pixel).


Geometric Model Finder

Chapter 8: Geometric Model Finder

This chapter describes the ActiveMIL ModelFinder control and the steps to follow to find models in target images.

  • Overview of the ModelFinder control
  • Steps to performing a search
  • Basic concepts
  • Types of search algorithms used by the ModelFinder control
    • General geometric search algorithm
    • Controlled geometric search algorithm
  • Defining and adding models to your ModelFinder control
    • Image-type models
      • Run-time
      • Design-time
      • Extracting edges
    • Synthetic models
      • Predefined-shape models
      • CAD-file models
      • Defining and adding predefined-shape models at design-time
      • Model size and units
      • Synthetic models and calibration
    • Models defined from the Results properties
      • Models defined from an EdgeFinder control result
      • Models defined from a ModelFinder control result
    • Models defined from two other models
    • Preprocessing
    • Model origin
    • Model indices and labels
    • Drawing and inquiring the model's active edges
  • Guidelines for choosing models
    • Make sure your images have enough contrast
    • Avoid poor geometric models
    • Be aware of ambiguous models
    • Nearly ambiguous models
  • Masking your model
  • Search targets
    • Finding models in an image
    • Finding models in the results of an EdgeFinder control
  • Determining what is a match
    • Score and target score
    • Model and target coverage
    • Fit error
    • Interpreting results
  • Position, angle, and scale
    • Enabling calculations specific to searching within a range
    • Search position and position range
    • Angle and angular range
    • Scale and scale range
  • Customizing search settings
    • Acceptance levels
    • Certainty levels
    • Expected number of occurrences
    • Reference axis
      • Reference axis origin
      • Reference angle position
      • Forward and reverse transformation coefficients
  • Advanced search settings
    • Polarity
    • Separation
    • Shared edges
    • Fit error weighting factor
    • First and last levels
  • Global ModelFinder control search settings
    • Setting the search speed
    • Accuracy
    • Timing out your search
  • Speeding up the search
    • Adjust the search speed of the algorithm
    • Disable calculations specific to range search strategies
    • Define several models with different expected angles
    • Limit the position range
    • Limit the search scale
    • Specify the exact number of expected occurrences
    • Clean up your model and your target
    • Changing the search levels
  • Retrieving and analyzing results
    • Possible results
    • Drawing results
  • Calibration
    • Requirements
    • Models and their Calibration control
    • Setting the aspect ratio control
    • Using transformation coefficients with calibrated images


Overview of the ModelFinder control

Next

The ActiveMIL ModelFinder control finds patterns, or models, in target images or in the results of an EdgeFinder control, based on geometric features. Its algorithm finds models using edge-based geometric features instead of a pixel-to-pixel correlation. As such, the ModelFinder control offers several advantages over correlation pattern matching, including greater tolerance of lighting variations (including specular reflection), model occlusion, as well as variations in scale and angle.

The ActiveMIL ModelFinder control allows you to tailor your search to fit the requirements of your application. You can search for any number of different models simultaneously, through a range of angles and scales. These models can be models created from images(image-type), models created from the result properties of an EdgeFinder control (Edge Finder-type), models created from the result properties of a ModelFinder control (Model Finder-type), models created from a combination of models (merge-type), and synthetic models. Synthetic models are either models that have a predefined shape, or are models taken from CAD-type files.

Searches can be performed in the calibrated real-world, using each model's associated Calibration control, such that, even without physically correcting your images, occurrences can be found, and results returned in real-world units. This is true even in the presence of complex distortions.

To find geometric models in your target, you need a ModelFinder control and a System control. Moreover, depending on the model-type and target, you will also need additional controls; for example, if you want to add an Edge Finder-type model, then you need an EdgeFinder control. Also, you can use an EdgeFinder control or an Image control as the target in which to search for occurrences of the model(s).

For an example that uses the ModelFinder control, see MModelTracking.


Steps to performing a search

User Guide:
Advanced search settings
Calibration
Customizing search settings
Defining and adding models to your ModelFinder control
Determining what is a match
Global ModelFinder control search settings
Guidelines for choosing models
Masking your model
Retrieving and analyzing results

Reference:
ModelFinder.Find
ModelFinder.Models.Item.Draw
ModelFinder.Models.Item.Mask
ModelFinder.Preprocess
ModelFinder.Results.Item
ModelFinder.Save

Steps outlined below provide a basic methodology for using the ActiveMIL ModelFinder control.

  1. First, you must add your model(s) to your ModelFinder control.
  2. If necessary, mask any irrelevant, inconsistent, or featureless areas of your model(s).
  3. Specify the global search settings for the ModelFinder control (with the immediate properties of the ModelFinder control and with the Models property), and the individual model(s) search settings (Models.Item(ModelIndex) property).

    4. These search settings are discussed in detail in the Customizing search settings, Advanced search settings, and ModelFinder control's global search settings sections.

  4. Preprocess your ModelFinder control.
  5. Search the target for the occurrence(s) of the model(s) in your ModelFinder control.
  6. Retrieve the required results.
  7. If necessary, save your ModelFinder control.

If you are using a calibrated model source and calibrated target, refer to the Calibration section at the end of this chapter.

See the MModelFinder example.


Basic concepts

Previous | Next

The basic concepts and vocabulary conventions while using the ActiveMIL Geometric ModelFinder control are:

  • Active edges. Edges which are used to compose the geometric model without being masked out, and are searched for in the target.
  • Bounding box of the edges. The smallest rectangle that fully encloses all of the edges of the model.
  • Chain. The set of connected edgels that construct an edge.
  • Edge. A curve that delineates a boundary, which can be established from intensity transitions in an image. When discussing a ModelFinder control, an edge is considered to be a chain and its features.
  • Edgel. Elementary point (or edge element) within an edge.
  • Model. The information that defines the pattern of active edges to find in the target and the search settings with which to do so.
  • Model box. The box that delimits the boundaries of the model. Essentially, the model box is the bounding box of the edges plus a margin. For image-type and Edge Finder-type models, the size of the model box is specified when defining the model. For synthetic models, the margin can be adjusted after model definition.
  • Model image. For an image-type and Edge Finder-type model, the model image is a copy of the region of the model source image from which the model has been defined. Note that the model source image for an Edge Finder-type model is the image source of the results of the EdgeFinder control. For a synthetic model, the model image is the image representation (edge map) of the model.
  • Model mask. A binary image used to define irrelevant, inconsistent, or featureless areas in the model, so that only the pertinent model details are used for the search. Non-zero values indicate masked regions.
  • Model origin. The point in the model's coordinate system that is considered to be (0, 0).
  • Model source. The immediate source of the model's edges. For example, for an Edge Finder-type model, the model source would be the results of the EdgeFinder control.
  • Model source image. The image from which you have defined an image-type model, or from which you have extracted the edges for an Edge Finder-type model.
  • Occurrence. An instance of the model found in the target.
  • Synthetic models. Models that have a predefined shape, such as a circle or a square, or are defined from a CAD-type file.
  • Target. The image, or results of the EdgeFinder control, in which to search for occurrences of the model.
  • Target edges. The edges in the target.


Types of search algorithms used by the ModelFinder control

Previous | Next

There are two types of search algorithms used by the ModelFinder control: a general geometric search algorithm and a controlled geometric search algorithm. Both algorithms use edge-based geometric features of the models and the target to establish a match.

General geometric search algorithm

A general geometric search algorithm handles a full range of scales and angles, unlike the controlled geometric search algorithm. To specify a general geometric search algorithm, set the ModelFinderType property of the ModelFinfer control to modGeometric.

Controlled geometric search algorithm

A controlled geometric search algorithm is recommended in situations where the image has high geometric complexity and where there are only small differences in angle and scale between the occurrence and the nominal angle and scale of the model, respectively. In such situations, a search using a controlled geometric search algorithm is often faster and more robust than using a general geometric search algorithm. Model searches use a hierarchical method to reduce the time needed to complete the search. Note that this might cause very thin lines to be missed as candidates. To specify a controlled geometric search algorithm, set the ModelFinderType property to modGeometricControlled.

There are restrictions on when you can use a controlled geometric search algorithm. Angle range and scale range searches are disabled when you use this algorithm. As well, if you are retrieving a result relating to the entire control, you cannot get or draw the edges for the entire target image; you can only get or draw the target edges in the region of an occurrence.


Defining and adding models to your ModelFinder control

User Guide:
Customizing search settings
Extracting the edges
Guidelines for choosing models
Masking your model

Reference:
EdgeFinder
EdgeFinder.Calculate
EdgeFinder.ModelFinderCompatible
ModelFinder
ModelFinder.Allocate
ModelFinder.IsPreprocessed
ModelFinder.ModelFinderType
ModelFinder.Models.AddDxfModel
ModelFinder.Models.AddEdgeModel
ModelFinder.Models.AddFromMerge
ModelFinder.Models.AddFromResult
ModelFinder.Models.AddImageModel
ModelFinder.Models.DetailLevel
ModelFinder.Models.Item
ModelFinder.Models.Item.Draw
ModelFinder.Models.Item.Draw
ModelFinder.Models.Item.DrawingParameters.RelativeOriginX
ModelFinder.Models.Item.DrawingParameters.RelativeOriginY
ModelFinder.Models.Item.DrawingParameters.ScaleX
ModelFinder.Models.Item.DrawingParameters.ScaleY
ModelFinder.Models.Item.SyntheticDefinition
ModelFinder.Models.Item.SyntheticDefinition
ModelFinder.Models.Item.SyntheticDefinition.PixelScale
ModelFinder.Models.Item.SyntheticDefinition.YAxisFlipped
ModelFinder.Models.Item.UserLabel
ModelFinder.Models.ItemAtLabel
ModelFinder.Models.SmoothnessLevel
ModelFinder.Preprocess

Once you have created a ModelFinder control, you can begin to add models to it.

There are four different kinds of models that can be added to a ModelFinder control: a model from an image, a synthetic model, a model from the results of an EdgeFinder or ModelFinder control, or a model created from two other models. All of these models can be mixed in the same Modelfinder control. Using the Models.Item(ModelIndex).ModelType property of the ModelFinder control, you can get the type.

Image-type models

You can add models defined from an image to a ModelFinder control. This type of model is referred to as an image-type model.

Image-type models can be added to your ModelFinder control from different 1-band, 8-bit unsigned images. The minimum size of a model is limited to 16.0x16.0 pixels, while a maximum size of 1024.0x1024.0 pixels is permitted. You can add image-type models at run-time or design-time.

Run-time

At run-time, to add image-type models to your ModelFinder control, use the Models.AddImageModel method. The Models.AddImageModel method requires that you specify the model source image.

You can use the entire image (if you have already cropped the area intended for the model), a child image, or you can designate the model region using the offset and size parameters of the Models.AddImageModel method. To view the portion of the image from which the model was extracted, copy the model region to an Image control using the Models.Item(ModelIndex).Draw method.

After adding the model, you can use the Models.Item(ModelIndex).ImageDefinition property to get the attributes of the model.

Design-time

At design-time, you can define and add image-type model(s) to the ModelFinder control as follows:

  1. Select the Models property page from the ModelFinder control's property pages.
  2. Click on Add. The Add Model dialog box is presented.
  3. From the Model type drop-down list box, select Image.
  4. Select the model source image by clicking on the Browse button.
  5. Click on the Model Position button (under the displayed source image).
  6. In the model source image, double click and drag the mouse to define the region of your model, or resize the existing red bounding box, using the circular handles. You can zoom in and out of the displayed model source image by clicking on the Zoom button and right or left clicking your mouse buttons, respectively.
  7. Click on Add. (Other fields on this property page are discussed later in the Customizing search settings section.)
  8. Repeat steps 1 through 7 to add other models, as required by your application.

Extracting edges

When you call the Models.AddImageModel method, it stores the specified region of the model source image with the model. This image is referred to as the model image. The model source image is then no longer needed.

It is actually when you call the Preprocess method of the ModelFinder control that edges are extracted from the model image. Specifically, the Preprocess method extracts the contour edges from the model image. For more information on contour edges, see the Extracting the edges section in Chapter 09: Edge Finder.

The edge extraction process involves a denoising operation to even out rough edges and remove noise. You can control the degree of smoothness (strength) of the denoising operation used for all models in the control, using the Models.SmoothnessLevel property. The range of this property varies from 0.0 to 100.0; a value of 100.0 results in a strong noise reduction effect, while a value of 0.0 has almost no noise reduction effect. The default setting is 50.0.

At design-time, you can use the Smoothness level drop-down list box on the Models Global property page to control the denoising operation.

You can also control the detail level of the edge extraction process for all models in the control. At run-time, use the Models.DetailLevel property. Basically, this controls how much detail is extracted from the model image of each model in the control. The default setting (modMedium) offers a robust detection of edges from images with contrast variation, noise, and non-uniform illumination. Nevertheless, in cases where objects of interest have a very low contrast compared to high contrast areas in the image, some low contrast edges can be missed.

At design-time, you can use the Detail level drop-down list box on the Models Global property page to control the detail level of the edge extraction process.

The following examples show the use of the Models.DetailLevel property:

If your images contain low-contrast and high-contrast objects, and your object of interest is low-contrast, a detail level setting of modHigh should be used to ensure the detection of the low-contrast edges in the image. The modVeryHigh setting performs an exhaustive edge extraction, including very low contrast edges. However, it should be noted that this setting is very sensitive to noise.

Generally, the default settings for the Models.SmoothnessLevel and Models.DetailLevel properties are sufficient for the majority of images; you should adjust these settings only when dealing with very noisy images, extremely low or multi-contrasted images, or images with very thin, refined features. In such cases, it is recommended that you experiment with different settings to achieve the necessary level of accuracy and speed required by your application.

Note that the settings for the Models.SmoothnessLevel and the Models.DetailLevel properties are also applied to the search target when it is an image.

Synthetic models

Synthetic models are models that either have a predefined shape, or have been defined from a CAD DXF file. Synthetic models allow you to search the targets for shapes that you expect to be in the target.

Predefined-shape models

You can use the Models.Add...Model methods to add predefined-shape models to the ModelFinder control. The model types that are available are circle, cross, diamond, ellipse, rectangle, ring, square, and triangle. These model types are described in more detail in their respective Models.Add...Model method.

After adding the model, use the Models.Item(ModelIndex).SyntheticDefinition property to get the attributes of the predefined-shape model.

Predefined-shape models use the center of the shape as the origin (0,0).

CAD-file models

You can add models that are defined from CAD-files to a ModelFinder control, using the Models.AddDxfModel method.

Not every entity in a CAD DXF file is supported; unsupported entities are ignored. The supported entities are Line, Polyline, Lwpolyline, Circle, Arc, Ellipse, Block (a collection of entities), and Insert (an entity containing a reference to where the Block should be drawn, and at what location and scale).

When the model is added to the ModelFinder control from a CAD DXF file, the model will retain its coordinate system (the origin and the axis). Most CAD DXF files are oriented so that the Y-axis is positive going up, whereas the coordinate system ActiveMIL uses is oriented with the Y-axis positive going down. So if you take the edge coordinates of an object from a CAD DXF file and put them in an image, the imaged object will look flipped when compared to the original object. This is also the case for a model defined from a CAD DXF file. This means that, unless the object is symmetrical, the match will not be made. You can use the Models.Item(ModelIndex).SyntheticDefinition.YAxisFlipped property to flip the Y-axis for the model so that it is positive going down.

Defining and adding predefined-shape models at design-time

At design-time, you can define and add predefined-shape model(s) to the ModelFinder control as follows:

  1. Select the Models property page from the ModelFinder control's property pages.
  2. Click on Add. The Add Model dialog is presented.
  3. From the Model type drop-down list box, located in the Model definition section, select the type of predefined-shape model needed. You can define the specifications of the chosen model.
  4. The Margin box is now enabled. From here, you can choose the margin size of the model box, if you disable the Default Margin checkbox . From the Margin box, you can also select the pixel scale for the model.
  5. Click on Add. (Other fields on this property page are discussed later in the Customizing search settings section.)
  6. Repeat steps 1 through 5 to add other models, as required by your application.

Upon creation, the model is extracted from the selected region in the model's source image. The model source image is then no longer needed.

CAD DXF models can only be added at run-time.

Model size and units

The dimensions of a synthetic model should be specified in user-defined units. Specify the ratio between model units and pixel units using the Models.Item(ModelIndex).SyntheticDefinition.PixelScale property.

By default, synthetic models are defined to have a 10% margin around the bounding box of their active edges. When finding an occurrence, extra edges found in this area will reduce the target score. You can change the size of the margin using the BoxMargin... properties of the Models.Item.SyntheticDefinition property.

Synthetic models and calibration

You can search for synthetic models in calibrated or non-calibrated targets. If the target is not calibrated, then the Models.Item(ModelIndex).SyntheticDefinition.PixelScale property of the ModelFinder control is used for the match.

If the target is calibrated, the model units should be the same as the calibrated units. If the target is calibrated, the Models.Item(ModelIndex).SyntheticDefinition.PixelScale property will be used for draw operations, but not for the match nor for the returned results.

Note that if you are searching for a synthetic model in a calibrated target, you must also associate the Calibration control of the target with the model using the Models.Item(ModelIndex).Calibration property. ActiveMIL needs the calibration object for internal purposes at preprocessing time. Note that ActiveMIL will not use the calibration object to compensate for incongruencies between the model and the target, nor will ActiveMIL map the model's coordinate system to that of the target.

Note that synthetic models can be invalid if their global size in X or Y (size of the model box * PixelScale * Scale) is greater than 1024.

Models defined from the Results properties

You can define models from two different types of results: EdgeFinder control results and ModelFinder control results.

In the case of these models, the model source image is the image from which the results were obtained. The model source is the control in which the results are stored. The model image is a copy of the region in the model source image from which the model has been defined.

Models defined from an EdgeFinder control result

You would define models from an EdgeFinder control result if you want to use other types of edges, such as crest edges, or if you want to re-use the edge map for another purpose. If you knew that the edges of interest had a minimum size, you could use the EdgeFinder control to include only edges above that size. The included edges of the results are the ones that will be used to define the model. These result-type models are referred to as Edge Finder-type models.

Some things have to be done before you can define a model from the Results property of an EdgeFinder control.

  • The EdgeFinder control must be compatible with the ModelFinder control. Enable this using the ModelFinderCompatible property of the EdgeFinder control.
  • The Calculate method of the EdgeFinder control must have been called after making it compatible.

You can add models defined from the results of the EdgeFinder control to a ModelFinder control using the Models.AddEdgeModel method of the ModelFinder control.

The size of the model can be defined using the offset and size parameters of the Models.AddEdgeModel method.

It is not possible to add Edge Finder-type models at design-time.

Models defined from a ModelFinder control result

Adding models defined from a ModelFinder control result allows you to define new models from previous match occurrences. These models are referred to as Model Finder-type models.

There are many reasons to use a Model Finder-type model. For example, if you have a model that changes slightly over time, you can define a model from the result in order to update it. Updating the model prevents potentially misidentifying the model because it has changed beyond your starting tolerances.

To define a Model Finder-type model, use the Models.AddFromResult method and pass the index of the occurrence to use. The model is defined with the edges of the target at the occurrence position.

The result must be model-addition compatible. You have to enable the Results.ModelAdditionCompatible property before you can use this feature; this must be done before the find operation that found the result.

Models defined from two other models

Models can be defined from two previously existing models and added to the control. Use the Models.AddFromMerge method and specify the two models from which to create the new model. The model will be formed by taking the first model specified and adding the edges of the second model that are not included in the edges of the first model.

There are no restrictions on the types of models you can merge. They can be of different types, as long as they are part of the same control.

Preprocessing

After you add models to your control and before performing the search, you have to preprocess the models, using the Preprocess method, otherwise an error will occur. It does not matter the type of model, or the type of algorithm used by the control. Calling the Preprocess method will prepare models for geometric matching. In the case of image-type models, it is also the time when edges are actually extracted from the model.

Note that the ModelFinder control must be preprocessed again if some of the models' search settings are changed. You can check if you need to preprocess after changing a property setting by checking the IsPreprocessed property.

When you save a ModelFinder control, preprocessing information is not saved with the control. You must preprocess the control again when the ModelFinder control has been restored.

Model origin

The model origin is the point in the model that is considered to be (0,0). The location of the model origin depends on the type of model: for image and Edge Finder-type models, the model origin is the center of the upper left pixel of the model image; for predefined-shape models, the center of the shape is the model origin. The model origin for a model defined from a CAD DXF file is the same as the point (0,0) in the CAD DXF file.

Model indices and labels

Each model added to a ModelFinder control can be accessed by the Models.Item property, using its index. At design-time, select the index of the required model using the Index list box in the Models property page. The first model is assigned an index of 1, and for each subsequent model added, the index is increased by one. When a model is deleted, all model indices greater than the deleted model are shifted down one. ActiveMIL searches for all models within a ModelFinder control in parallel, therefore the model index does not have any bearing in terms of a search order.

Models can also be given a user-defined numeric label. At run-time, use the Models.Item(ModelIndex).UserLabel method. At design-time, click on the General button in the Models property page (identified by the ">>"), select the Attributes view, and set the user label. A user label can be used as a means of identifying your model, independently from its index in the ModelFinder control. User labels can be used as a direct replacement for the index; to retrieve the element in the collection of Model Finder models with the specified user label, use the Models.ItemAtLabel property instead of the Models.Item property. All user labels must be unique integers; that is, no two user labels can have the same integer. To remove a label from a model, set the user label to modNoLabel.

Drawing and inquiring the model's active edges

When testing different models for the best model candidate, you can use the Models.Item(ModelIndex).Draw method to draw the model's active edges. This can be done for any type of model, although for synthetic models it is less useful, since the edges should already be known. The resulting edge map can reveal whether the edges you have chosen are the ones you want, or if there are problems with the edges. To make corrections for image-type models, you might have to adjust the edge extraction properties (for example, the Models.SmoothnessLevel property) or mask unwanted edges. For Edge Finder-type models, you should exclude edges that you don't want from the EdgeFinder control and redefine the model or you can mask unwanted edges.

You can use the Models.Item(ModelIndex).ImageDefinition or Models.Item(ModelIndex).SyntheticDefinition properties to determine the necessary size for the destination image buffer.

If you have used offsets to define the model region in the model source, you can draw the active edges at the position where the model was defined by adding modOriginalPosition to the operation parameter of the Models.Item(ModelIndex).Draw method; otherwise the active edges will be drawn at the top-left corner of the specified destination image. This is valid for image-type or Edge Finder-type models.

At design-time, to draw the model's active edges, enable the Edges checkbox in the General view of the Models property page.

You can use the Models.Item(ModelIndex).Chains property to return the collection of active edge chains of a model.

Note that if you try to draw a model's edges and its ModelFinder control has not been preprocessed, a preliminary edge extraction operation will be performed for the model.

You can also draw features from a zoomed region of the model image, as well as other items. To do so, specify the appropriate values for the RelativeOriginX, RelativeOriginY, ScaleX, and ScaleY properties of the Models.Item(ModelIndex).DrawingParameters property. The relative origin values must be specified in pixels, and are relative to the coordinates of the top-left corner of the model image, while the scale values specify the X- and Y-scaling factors used to fill the destination image buffer.


Guidelines for choosing models

User Guide:
Defining and adding models to your ModelFinder control
Masking your model

While finding models based on geometric features is a robust, reliable technique, there are a few pitfalls which you should be aware of when allocating your models so that you choose the best possible model.

Make sure your images have enough contrast

Contrast is necessary for identifying edges in your model source and target with subpixel accuracy. For this reason, it is recommended that you avoid models that contain only slow gradations in grayscale values.

Avoid poor geometric models

Poor geometric models suffer from a lack of clearly defined geometric characteristics, or from geometric characteristics that do not distinguish themselves sufficiently from other image features. These models can produce unreliable results.

Be aware of ambiguous models

Certain types of geometric models provide nonunique, or ambiguous, results in terms of position, angle, or scale.

Models which are ambiguous in position are usually composed of one or more sets of parallel lines only. Such models make it impossible to establish a unique position for them. A large number of matches can be found since the actual number of line segments in any particular line is theoretically limitless.

Models which are ambiguous in scale are usually composed of straight lines which pass through the same point; some spirals are also ambiguous in scale. Models which consist of small portions of objects should be tested to verify that they are not ambiguous in scale.

For example, a model of an isolated corner is ambiguous in scale because it consists of only two straight lines that pass through the same point.

Symmetric models are often ambiguous in angle due to their similarity in features. For example, circles are completely ambiguous in terms of angle. Other simple symmetric models, such as squares and triangles, are ambiguous with respect to certain angles:

Nearly ambiguous models

When the major part of a model contains ambiguous features, false matches can occur because the percentage of the occurrence's edges involved in the ambiguous features is great enough to be considered a match (see the Determining what is a match section). To avoid this, make sure that your models have enough distinct features to be found among other target features. This will ensure that only correct matches are returned as results. For example, the following model can produce false matches since the greater proportion of active edges in the model is composed of parallel straight lines rather than distinguishing curves.


Masking your model

User Guide:
Defining and adding models to your ModelFinder control
Determining what is a match
Masking your model
Steps to performing a search

Reference:
Image.Clear
ModelFinder.Models.Item.Draw
ModelFinder.Models.Item.Mask
ModelFinder.Preprocess

Once you have added a model to your ModelFinder control, you can then mask any irrelevant, inconsistent, or featureless areas of your model using the Models.Item(ModelIndex).Mask method.

You can define three types of mask: a "don't care", a "flat region", and a "weighted region" mask. You can use these types of masks with the same model. Note that "don't care" and "flat region" masks are not supported for synthetic models.

With a "don't care" mask, ActiveMIL ignores the masked regions when searching for occurrences of the model. These regions might be noise edges, unwanted edges, inconsistent features, or simply regions which are irrelevant to your search.

In the example below, a mobile phone is used as the model. However, occurrences of this model have displays containing inconsistent characters from target image to target image. Since these characters are variable, they should be masked in the model:

With a "flat region" mask, the masked regions (non-zero pixels) are expected to be featureless in the found occurrence, and any edges present in these regions will reduce the target score. For example, in a quality control application, features that are present in a region where none are expected can indicate that an error has occurred in the manufacturing process.

With a "weighted region" mask, the extracted model features have weights according to the values of the weighted region mask. The weights of the features are used to calculate the score; although the weights themselves don't affect the target score, edges corresponding to negative weights will be ignored when calculating the target score. The valid range for the weights of the mask is -127 to +127.

Positive weights indicate how significant it is for a given pixel to be part of an edge in the occurrence; the more positive the weight, the greater the influence on the score. The absence of a pixel that corresponds to a very positive weight reduces the score more than the absence of one with a lower weight. The longer the edges with a positive weight, the more significant the edges to the target coverage and to the target score.

Negative weights indicate how significant it is for a given pixel not to be part of an edge of the occurrence; the more negative the weight, the more the influence on the score. The presence of a pixel that corresponds to a very negative weight will reduce the score more than the presence of one that corresponds to a less negative weight.

The effect of positive and negative weights on the score is relative. A "weighted region" mask that has negative weights of -100 and positive weights of 100 will have the same score as the same mask where the negative weight is -1 and the positive weight is 1.

Note that weights that have no correspondence to edges in the model have no effect on the score.

Weighted regions are particularly useful when the pattern you are searching for in the target also exists as a subset of other patterns; that is, you cannot otherwise identify the required pattern uniquely. In these cases, you have to create a model that includes the unwanted extra edges and mask the extra edges with negative weights.

In the example below, the square is the shape that is being searched for, while every other shape is unacceptable. The weighted region mask will cause the one acceptable shape to be found with a high score, while ensuring that the two other shapes that are similar are found with a low score.

A separate image is used to create each mask and it must be of the same size as the model. Note that if you modify the margin of a synthetic model's bounding box after a mask has been applied, the mask will be discarded since it is no longer the same size as the model. You must then re-apply a mask that is of the same size as the new model box.

For "flat region" and "don't care" masks, the image must be a 1-band 8-bit unsigned image , where any non-zero pixel value is considered a masked pixel. For a "weighted region" mask, the image must be a 1-band 8-bit signed image and the valid range for the weights of the mask is -127 to +127. This image can be associated with a Calibration control; however, even if the model is calibrated, the mask is applied on a pixel basis.

Note that if you expect significant misalignment between the model and the target, you might need to compensate for this by slightly increasing the region of the "don't care" or "flat region" pixels in the mask.

At run-time, you can mask your model(s) by calling the Models.Item(ModelIndex).Mask method with the MaskType parameter set to modDontCare, modFlatRegions or modWeightRegions, depending on the type of mask to apply. If you want to verify the mask which has been applied to a model, you can use the Models.Item(ModelIndex).Draw method to draw the model's masked pixels. If necessary, you can clear a model's current mask image using the Clear method of the Image control. Finally, when you change the masked pixels of a model, you must preprocess the search model again, using the Preprocess method.

At design-time, you can mask your model(s), from the General view of the Models property page. You can choose the type of mask by selecting either the Don't Cares or Flat Regions option. To mask the model, click the Mask button to enable the mask tool; then, click and drag the mouse over the model region(s) to mask. Holding the right mouse button down undoes the masking. You can control the pen size of the mask tool by adjusting the Pen Width slider that appears above the image. You can view the current "don't care" or "flat region" mask by selecting the corresponding option. Finally, when you change the masked pixels of a model, you must preprocess the search model again by clicking on the Preprocess button found on the General property page.

Note that a "weighted region" mask can only be applied during run-time.


Search targets

Previous | Next

You can search for instances of a model in an image, or you can search for instances of it in the results of an EdgeFinder control. To search for a model in an image, you would use the Find method of the ModelFinder control; to search in the results of an EdgeFinder control, use the FindInEdgeFinderResults method.

Finding models in an image

To search for models in an image, the target image must be a 1-band, 8-bit unsigned image. The minimum and maximum target image sizes are 16x16 and 32768x32768 pixels, respectively. It is important to note that the minimum and maximum target image sizes are ActiveMIL limits. To make sure that you have enough memory, you can call the Find method; if you don't have enough memory, you will get an ActiveMIL error.

The target image must be in the same calibrated state as the models. If the models are calibrated, the target must be; if the models are not calibrated, the target must not be.

You can search for a synthetic model in a calibrated or non-calibrated target image. If the target is calibrated, the model units should be the same as the calibrated units. In addition, you must associate the Calibration control of the target image with the model, using the Models.Item(ModelIndex).Calibration property. ActiveMIL needs the Calibration control for internal purposes at preprocessing time. Note that ActiveMIL will not use the Calibration control to compensate for incongruencies between the model and the target, nor will ActiveMIL map the model's coordinate system to that of the target.

Note that the edge extraction process, applied to the target image, uses the same denoising operation and detail level as the one for all image-type models in the ModelFinder control. For more information, see the Extracting edges subsection of the Defining and adding models to your ModelFinder control section.

Finding models in the results of an EdgeFinder control

When using the results of an EdgeFinder control as the target, Model Finder searches for active edges of the models only in the included edges of the results. Therefore, using the results of the EdgeFinder control as a target, instead of an image, allows you to be more specific about the edges in which to search. For example, you could select out unwanted edges in the target, which will speed up the search and further calculations.

Some things have to be done before you can search for instances of a model in the results of an EdgeFinder control.

  • The EdgeFinder control must be compatible with the ModelFinder control. Enable this by setting the ModelFinderCompatible property of the EdgeFinder control to True.
  • The Calculate method of the EdgeFinder control must have already been called.

If the models of the ModelFinder control are calibrated, the results in the EdgeFinder control must also be calibrated. If the models are not calibrated, the results must not be calibrated either. In the case of synthetic models, the same rules apply as when using a calibrated image as a target.


Determining what is a match

User Guide:
Customizing search settings
Defining and adding models to your ModelFinder control
Guidelines for choosing models
Masking your model

Reference:
ModelFinder.Results.Item
ModelFinder.Results.Item.FitError
ModelFinder.Results.Item.ModelCoverage
ModelFinder.Results.Item.TargetCoverage

Before customizing your search settings, it is necessary to understand how a match between your model and occurrences in the target is determined. The score and the target score are the primary factors in determining which occurrences are considered matches with the models in your ModelFinder control.

Score and target score

The score is a measure of the active edges in the model found in the occurrence, weighted by the deviation in position of these common edges. Active edges in the model not found in the occurrence reduce the score. The target score is a measure of edges found in the occurrence that are not present in the original model (that is, extra edges), weighted by the deviation in position of the common edges. Edges found in the occurrence that are not present in the model will reduce the target score.

The scores are calculated as follows:

  • Score = Model coverage x (1 - (Fit error weighting factor x Normalized Fit Error))
  • Target score = Target coverage x (1 - (Fit error weighting factor x Normalized Fit Error))

    • Note: the normalized fit error is the fit error converted to a number between 0.0-1.0.

The model coverage, target coverage, and fit error components of the score and target score are explained below.

Model and target coverage

The model coverage and target coverage are defined as follows:

  • Model Coverage. The model coverage is the percentage of the total length of the model's active edges found in the occurrence. 100% indicates that for every edge in the model, a corresponding edge was found in the occurrence.
  • Target coverage. The target coverage is the percentage of the total length of the model's active edges found in the occurrence, divided by the total length of edges present within the occurrence's bounding box. Thus, a target coverage score of 100.0 means that no extra edges were found. Lower scores indicate that features or edges found in the target (result occurrence) are not present in the model.

Using a weighted mask with a model can affect the score and target score. In cases where a weighted mask is used, the model coverage would be as below:

The target coverage is as follows:

Fit error

The fit error is a measure of how well the edges of the occurrence correspond to those of the model. The fit error is calculated as the average quadratic distance, in pixels or calibrated units, between the edgels in the occurrence and the corresponding active edges in the model:

A perfect fit gives a fit error of 0.0. The fit error weighting factor (between 0.0 - 100.0) determines the importance to place on the fit error when calculating the score and target score.

Interpreting results

The following diagram illustrates how the ModelCoverage, TargetCoverage, and FitError properties of the Results.Item(OccurrenceIndex) property work together to provide details about the nature of a result occurrence.


Position, angle, and scale

User Guide:
Reference axis

Reference:
ModelFinder.Models.Item.Angle.NegativeDelta
ModelFinder.Models.Item.Angle.PositiveDelta
ModelFinder.Models.Item.Angle.Value
ModelFinder.Models.Item.MinimumSeparationAngle
ModelFinder.Models.Item.MinimumSeparationScale
ModelFinder.Models.Item.Position
ModelFinder.Models.Item.Position.NegativeDeltaX
ModelFinder.Models.Item.Position.NegativeDeltaY
ModelFinder.Models.Item.Position.PositiveDeltaX
ModelFinder.Models.Item.Position.PositiveDeltaY
ModelFinder.Models.Item.Position.X
ModelFinder.Models.Item.Position.Y
ModelFinder.Models.Item.Scale.Maximum
ModelFinder.Models.Item.Scale.Minimum
ModelFinder.Models.Item.Scale.Value
ModelFinder.Models.Item.Scale.Value
ModelFinder.Models.SearchAngleEnabled
ModelFinder.Models.SearchPositionEnabled
ModelFinder.Models.SearchScaleEnabled
ModelFinder.Results.Item.PositionX
ModelFinder.Results.Item.PositionY

You can control the position, angle, and scale at which the ModelFinder control searches for each model in a target.

You can restrict the position, angle, and scale at which model occurrences can be found, to an expected (nominal) position, angle, or scale or to a given range.

Enabling calculations specific to searching within a range

Depending on whether searching for models within a range of positions, angles, and/or scales, the ModelFinder control uses different search strategies to evaluate the edge-based features of the target candidates. Typically, to search for models within a range, calculations specific to the corresponding search strategy should be enabled for the control (the Models.SearchAngleEnabled, Models.SearchPositionEnabled, and/or Models.SearchScaleEnabled properties set to True). If you expect that the occurrences sought are close to the specified nominal position, angle, or scale (for example, in a registration application), you can try disabling the calculations specific to the corresponding search strategies to see if the ModelFinder control can still find the required occurrences. Disabling one or more of these calculations might speed up the search depending on the model and the target.

Note that candidates can only be returned as occurrences if found within the ranges specified for their model. Therefore, if you have disabled calculations specific to a search strategy and occurrences at positions, angles, and/or scales outside of a required range are being returned, you can restrict which candidates are returned as occurrences by narrowing the appropriate ranges.

Search position and position range

You can search for each model, defined in a ModelFinder control, at a specific position or within a position range. The position range limits the area in which the position of a model occurrence can be found; coordinates which fall outside this area cannot be returned as results (the Results.Item(ResultIndex).PositionX and Results.Item(ResultIndex).PositionY properties). Note that the position returned for an occurrence is determined by the model's reference axis origin. By default, this position is set to the center of the model; however, it can be displaced if necessary (see the Reference axis subsection in the Customizing search settings section later in this chapter).

The default region is the entire image plane in the positive X- and Y-direction, meaning that all X- and Y-coordinates in the positive direction (even outside the target if applicable) can be returned as results. You can, if necessary, set a different position range within the target for each of the models within your control. At run-time, use the NegativeDeltaX, NegativeDeltaY, PositiveDeltaX, PositiveDeltaY properties of the Models.Item(ModelIndex).Position property to set the position range relative to the nominal position.

To define the position range at design-time, use the drop-down lists on the Position view of the Models property page to specify the appropriate values.

Depending on the number of details present in the target, using a small region generally decreases the search time. Always set the region to the minimum required when speed is a consideration. If you set the NegativeDeltaX, NegativeDeltaY, PositiveDeltaX, PositiveDeltaY properties of the Models.Item(ModelIndex).Position property equal to zero, the occurrence must be at the nominal position.

Note that you can specify a position range which defines a region partially, or totally, outside of the target; this might be necessary, depending on the reference axis origin of the model.

By default, calculations specific to position-range search strategies are enabled. If you are searching for a single occurrence of each model in your control and you know the nominal position of each (for example, registration applications), you can disable the calculations specific to position-range search strategies for the control at run-time by setting the Models.SearchPositionEnabled property to False. At design-time, disable the Search position check box in the Models Global property page. When disabled, you must specify a good nominal position for each model (using the Models.Item(ModelIndex).Position.X and Models.Item(ModelIndex).Position.Y properties). You can restrict which candidates are returned as occurrences by narrowing the position range.

Angle and angular range

You can search for each model, defined in a ModelFinder control, at a specific angle or within an angular range. For each model in your control, you can specify the angle of the search, using the Models.Item(ModelIndex).Angle.Value property. By default, the search angle is 0o.

You can also search within the full angular range of 360o from the nominal angle specified with Models.Item(ModelIndex).Angle.Value. Use the Models.Item(ModelIndex).Angle.PositiveDelta and Models.Item(ModelIndex).Angle.NegativeDelta properties to specify the angular range in the counter-clockwise and clockwise direction from the nominal angle, respectively; the default for both is 180o. The angular range limits the possible angles which can be returned as results for an occurrence. Note that the actual angle of the occurrence does not affect search speed. If you need to search for a model at discrete angles only (for example, at intervals of 90 degrees), it is typically more efficient to define several models with different expected angles, than to search through the full angular range.

At design-time, you can specify the angular search range using the Value, Negative delta, and Positive delta edit fields in the Degrees Of Freedom view of the Models property page.

By default, calculations specific to angular-range search strategies are enabled. If you expect that the occurrences sought are close to the specified nominal angle, you can disable these calculations at run-time by setting the Models.SearchAngleEnabled property to False. At design-time, disable the Search angle check box in the Models Global property page. When disabled, you must specify a good nominal angle for each model, which is within the model's angular range. You can restrict which candidates are returned as occurrences by narrowing the angular range.

Scale and scale range

The scale of the model establishes the size of the occurrence that you expect to find in the target. If the expected occurrence is smaller or larger than that of the model, you can set the nominal scale of the occurrence for each individual model, using the Model.Item(ModelIndex).Scale.Value property. The supported scale factors are 0.5 to 2.0.

When the scale of occurrences can vary around the specified nominal scale, you can enable calculations specific to scale-range search strategies for the control. At run-time, to enable these calculations, set the Models.SearchScaleEnabled property to True. To specify the range of scales, use the Models.Item(ModelIndex).Scale.Maximum and Models.Item(ModelIndex).Scale.Minimum properties (0.5 to 1.0). The minimum factor and the maximum factor together determine the scale range from the nominal scale (the Models.Item(ModelIndex).Scale.Value property).

At design-time, enable the Search scale check box in the Models Global property page. You can specify the search scale range using the Value, Minimum, and Maximum edit fields in the Degrees Of Freedom view of the Models property page.

The minimum and maximum factors are applied to the nominal scale as follows:

  • Minimum scale = Value property * Minimum property
  • Maximum scale = Value property * Maximum property

Note that the range is defined as factors so that if you change the nominal scale (Value property setting), you do not have to modify the range.

When calculations specific to scale-range search strategies are enabled, the scale range should be used to cover an expected variance in the scale; you should not use the scale range to cover different expected scales at different positions. In this case, it is typically more efficient to define several models with different expected scales. This is because a large scale range could potentially slow down your operation; as well, you could find unwanted occurrences.

By default, calculations specific to scale-range search strategies are disabled. When disabled, you must specify a good nominal scale for each model, which is within the model's scale range. Note that occurrences can still be found within the scale range specified for their model. You can restrict which candidates are returned as occurrences by narrowing the scale-range.


Customizing search settings

User Guide:
Advanced search settings
Determining what is a match
Global ModelFinder control search settings
Using transformation coefficients with calibrated images

Reference:
ModelFinder.Models
ModelFinder.Models.Item
ModelFinder.Models.Item.Acceptance
ModelFinder.Models.Item.AcceptanceTarget
ModelFinder.Models.Item.Certainty
ModelFinder.Models.Item.CertaintyTarget
ModelFinder.Models.Item.FitErrorWeightingFactor
ModelFinder.Models.Item.NumberOfOccurrences
ModelFinder.Models.Item.Position
ModelFinder.Models.Item.ReferenceAngle
ModelFinder.Models.Item.ReferenceX
ModelFinder.Models.Item.ReferenceY
ModelFinder.Models.TotalNumberOfOccurrences
ModelFinder.Results.Item
ModelFinder.Results.Item.ForwardA
ModelFinder.Results.Item.ForwardB
ModelFinder.Results.Item.ForwardC
ModelFinder.Results.Item.ForwardD
ModelFinder.Results.Item.ReverseA
ModelFinder.Results.Item.ReverseB
ModelFinder.Results.Item.ReverseC
ModelFinder.Results.Item.ReverseD

You can customize the ModelFinder control's global search settings which apply to all models, as well as individual model settings.

At run-time, to set global search settings, use the immediate properties of the control or of the Models property. Individual model settings are specified using the properties of the Models.Item property.

At design-time, to adjust the ModelFinder control's global search settings, use the check boxes and edit fields found on the Models Global property page. To adjust individual model(s) settings, use the Models property page and select the index of the required model. For each model, the Models property page has been subdivided into different views based on the different categories of properties it makes available. To select a view, click on the Category View button in the Models property page (identified by the ">>"), and select the General, Attributes, Image, Advanced, Degree Of Freedom, or Search Region view.

The following section discusses the fundamental search settings that are usually necessary to adjust. The advanced search settings, which usually only require adjusting for more sophisticated applications, are discussed later, in the Advanced search settings section.

Fundamental search settings include the acceptance and certainty levels which determine what is considered a match, the expected number of occurrences to find, the reference axis which determines the position returned for a match, and the search region in the target. All of the model search settings can affect the speed and robustness of your application. It is recommended that you begin with the default settings, and then, as your application demands, adjust the individual settings as required

Acceptance levels

For each model, acceptance levels can be set for both the score and target score. The acceptance levels determine the minimum scores required for an occurrence to be considered a match.

ActiveMIL will search the target for the required number of occurrences, returning the occurrences with the best scores above the acceptance levels. If the score or target score of an occurrence is less than the acceptance levels, it is not considered a match and no result will be returned.

You can set the acceptance level for the score of the specified model, using the Models.Item(ModelIndex).Acceptance property; at design-time, use the Acceptance edit field, in the Attributes view of the Models property page. A setting of 100% means that you will only accept occurrences that contain every active edge in the model, with a perfect fit. That is, for every active edge in the model, an equivalent edge must be found in the occurrence with a perfect fit. However, perfect matches are generally unobtainable in real images because of noise and distortion introduced when grabbing images. You should use a reasonable acceptance level that is high enough to avoid false matches, but not so high that occurrences are missed. If your images have considerable noise and/or distortion, or if occlusion of occurrences is expected, you might have to set the level below the default value of 60%.

You can set the target score required using the Models.Item(ModelIndex).AcceptanceTarget property; at design-time, use the Acceptance target edit field, in the Attributes view of the Models property page. A setting of 100% for the target score means that you will not tolerate any extra edges, and common edges will have a perfect fit. A setting of 0% (default) allows for any number of extra edges. Essentially, the target score acceptance level allows you to control how tolerant your search is to details not present in the original model.

Note that the contribution of the fit can be controlled using the Models.Item(ModelIndex).FitErrorWeightingFactor property. For more information, see the Fit error weighting factor sub-section in the Advanced search settings section later in this chapter.

Certainty levels

The certainty levels are used to speed up the search when very good matches are expected often. Any occurrence found above the certainty levels is considered a certain match.

The certainty levels determine the score and target score equal to or above which the algorithm can assume that it has found a match. Both of these scores must be equal to or above their respective certainty levels for an occurrence to be considered a certain match. If the required number of occurrences has been found equal to or above these certainty levels, ActiveMIL stops searching the target for better ones, avoiding exhaustive checking of all possible candidates. If not, ActiveMIL will continue to search for the required number of matches equal to or above the acceptance level and with the best score.

At run-time, you can set the certainty level for the score of a specified model using the Models.Item(ModelIndex).Certainty property (the default setting is 90.0%), and the certainty level for the target score using the Models.Item(ModelIndex).CertaintyTarget property (the default settings is 0.0%). At design-time, use the Certainty and Certainty target edit fields, in the Attributes view of the Models property page.

If necessary, when searching for a fixed number of occurrences in a target, you can ensure that ActiveMIL always returns those occurrences with the highest match score among those found, by setting the certainty levels of each individual model to 100%. ActiveMIL will return occurrences with the best score(s) over the acceptance or those with scores of 100%. Note that doing so will probably increase the search time.

Expected number of occurrences

You can set the expected number of occurrences to find in the target for the ModelFinder control, and for each individual model in the ModelFinder control. The number set for the control specifies the maximum total of all model occurrences (for all models within the control together).

At run-time, use the Models.Item(ModelIndex).NumberOfOccurrences property to set the expected number of occurrences of a model, specifying the index of the particular model. To find all occurrences of a model, set this property to modAll. At design-time, you can set the expected number of occurrences of the model using the Number of occ. edit field, in the Attributes view of the Models property page.

At run-time, to set the number for a ModelFinder control, use the Models.TotalNumberOfOccurrences property. The default value is modAll, which finds the expected number of occurrences specified for each model of the control. At design-time, you can set the number for a ModelFinder control, using the Total occurrence edit field, in the Models Global property page.

Note that setting the number of occurrences for the control to modAll and the number of occurrences for one of the models to modAll can significantly slow the Find and FindInEdgeFinderResults methods, depending on the complexity of your model and the target (operation speed will only be slightly affected for simple models on a uniform background). It is recommended that you specify the exact number of expected occurrences whenever possible.

ActiveMIL searches for all models within a control in parallel, therefore the model index does not have any bearing in terms of a search order. Once the number of occurrences for the control has been found, with scores equal to or above the certainty levels, the search will stop.

For example, in an application involving a number of different objects being examined one at a time on a conveyor belt, the actual object present in the target at any time is unknown. Since only one occurrence is expected at any one time, the ModelFinder control's number would be set to one, as would the number for all the models within the control.

ActiveMIL will search the target for all three models; the first occurrence of any of these models, found above the certainty level, will stop the search.

To further illustrate the relationship between the TotalNumberOfOccurrences property setting for the control and the NumberOfOccurrences property setting of the individual models, the tables below shows the maximum possible results which can be returned for different settings of these properties.

Example 1 |
Property settings | Possible Results
ModelFinder1.Models.TotalNumberOfOccurrences = modAll | All
ModelFinder1.Models.Item(1).NumberOfOccurrences = 1 | 1
ModelFinder1.Models.Item(2).NumberOfOccurrences = modAll | All
ModelFinder1.Models.Item(3).NumberOfOccurrences = 2 | 2

In this table, the last 6 columns present the maximum number of possible results for model 2, given the maximum number of possible results for the control and the different number of possible results for models 1 and 3.

Example 2 |
Property settings | Possible Results
ModelFinder1.Models.TotalNumberOfOccurrences = 6 | 6 |6 |6 |6 |6 |6
ModelFinder1.Models.Item(1).NumberOfOccurrences = 1 | 0 |1 |0 |1 |0 |1
ModelFinder1.Models.Item(2).NumberOfOccurrences = modAll | 6 |5 |5 |4 |4 |3
ModelFinder1.Models.Item(3).NumberOfOccurrences = 2 | 0 |0 |1 |1 |2 |2

Reference axis

When an occurrence of a model is found, the model's reference axis determines the coordinates and angle returned as the actual occurrence's found position and angle. Position results return the coordinates of the model's reference axis origin transformed at the model occurrence. Angle results are also returned relative to the reference axis, rather than the model source image axis.

Reference axis origin

By default, the reference axis origin, for an image-type or Edge Finder-type model, is at the center of the model box. For a synthetic model, the reference axis origin is at the model origin.

You can change the position of the reference axis origin. Modifying the position can be useful if you cannot define a unique model for an object of interest. In this case, define a model that is unique and at a fixed location relative to your object of interest and move your reference axis.

For example, in the diagram below, the easily found chip is used as the model, and the model's reference axis origin is shifted to the location of the diode.

At run-time, you can change the position of the reference axis origin using the Models.Item(ModelIndex).ReferenceX and Models.Item(ModelIndex).ReferenceY properties, specifying coordinates relative to the model origin. Position results are returned relative to the origin of the target.

At design-time, you can adjust the reference axis origin by clicking on the Ref. Axis button (under the displayed model source image) in the General view of the Models property page, placing the cross at the required position in the model source image using the mouse, and clicking. Alternatively, when adding a model, you can set the reference axis origin by clicking on the Ref. Axis button in the Add Model dialog box, placing the cross at the required position with the mouse, and clicking.

The reference axis origin specified does not have to reside within the model, meaning that you can place the reference axis origin outside of the model boundaries. At design-time, to place the reference axis origin outside the model source image, you must specify the coordinates using the X and Y edit fields, in the General view of the Models property page.

Note that the found position of an occurrence respects differences in angle and scale, meaning that the found position is relative to the angle and scale of the occurrence.

When changing the reference axis origin, make sure to update the position range if necessary, particularly when searching through a range of angles and/or scales. The position range limits the region in which the position of a model occurrence can be found; position coordinates which fall outside this region cannot be returned as results (see the Search position and search region subsection in the Position, angle, and scale section earlier in this chapter). In the diode example presented above, the position range must include, and can be limited to, a small area including the diode for the chip to be found.

Reference angle position

Often, when allocating models, the angle of the object in your model source image is not aligned with the horizontal image axis.

When occurrences in your target are successfully located, the angle of the occurrence returned is with respect to model's reference axis. The actual angle of the object of interest in the model is not taken into consideration. To correct this at run-time, you can set the reference angle of the model's reference axis using the Models.Item(ModelIndex).ReferenceAngle property. At design-time, use the Angle edit field, in the General view of the Models property page. Angle results will then reflect the actual angle of the object of interest.

In the above image, setting the reference axis angle to 4.0 degrees will ensure that the angle returned for the occurrence will represent the actual angle of the object.

Forward and reverse transformation coefficients

Forward and reverse transformation coefficients are available (as results, using the ForwardA, ForwardB, ForwardC, ForwardD, ReverseA, ReverseB, ReverseC, ReverseD properties of the Results.Item property) for mapping additional points of interest other than the reference axis origin. These coefficients allow you to convert coordinates in the model coordinate system to the corresponding coordinates in the target coordinate system for that occurrence (or vice versa). These coefficients handle variations in scale, translation, and angle, allowing for easier mapping of critical points between model and occurrence.

Use the following equations:

xd = axs + bys + c

yd= -bxs + ays + d

where:

  • a, b, c, d: Transformation coefficients (forward or reverse).
  • x s and y s: Source coordinates (with respect to origin of the model coordinate system for a forward transformation or target coordinate system for a reverse transformation).
  • x d and y d: Destination coordinates (with respect to the origin of the target coordinate system for a forward transformation or model coordinate system for a reverse transformation).

If the model is calibrated, these coefficients are given for the real world. For more information, see the Using transformation coefficients with calibrated images subsection of the Calibration section, later in this chapter.


Advanced search settings

User Guide:
Advanced search settings
Calibration
Defining and adding models to your ModelFinder control
Determining what is a match
Global ModelFinder control search settings

Reference:
ModelFinder.Models.Item.FitErrorWeightingFactor
ModelFinder.Models.Item.MinimumSeparationAngle
ModelFinder.Models.Item.MinimumSeparationScale
ModelFinder.Models.Item.MinimumSeparationX
ModelFinder.Models.Item.MinimumSeparationY
ModelFinder.Models.Item.Polarity
ModelFinder.Models.SharedEdges

In addition to the fundamental search settings, the ModelFinder control also provides advanced settings that allow you to specify whether the polarity of the edges in a model differs from those in occurrences, how much or how little separation between occurrences is permitted, as well as other information.

Polarity

The polarity of an edge indicates whether edges occur as transitions from light to dark or vice versa. When the polarity of edges in the target might be different from those of the model (for example, due to shadows), you should specify the expected polarity.

At run-time, use the Models.Item(ModelIndex).Polarity property. This property allows you to specify whether edges present in the model and in the occurrence have the same, the reverse, either of these, or a mixture of both polarities. The default polarity setting is modSame.

At design-time, you can specify the polarity using the Polarity edit fields in the Advanced view of the Models property page.

The polarity setting can be useful when dealing with adverse lighting conditions where dark shadows can cause edges to vary in polarity.

Separation

You can specify the minimum amount of separation from other occurrences (of the same model) for an occurrence to be considered distinct (a match). In essence, this determines what amount of overlap by the same model occurrence is possible.

You can set the minimum separation for four criteria, which are: the X position, Y position, angle, and scale. These can be set for each individual model in your ModelFinder control. For an occurrence to be considered distinct from another, only one of the minimum separation conditions needs to be met. For example, if the minimum separation in terms of angle is met, then the occurrence is considered distinct, regardless of the separation in position or scale. However, each of these separation criteria can be disabled (modDisable) so that it is not considered when determining a valid occurrence.

DISTINCT OCCURRENCE = (Separation in X) OR (Separation in Y) OR (Separation in Angle) OR (Separation in Scale)

The minimal positional separation (the Models.Item(ModelIndex).MinimumSeparationX and Models.Item(ModelIndex).MinimumSeparationY properties) determines the minimum distance between the found positions of two occurrences of the same model. This separation is specified as a percentage of the model size at the nominal scale (Models.Item(ModelIndex).Scale). The minimum value that you can set for Models.Item(ModelIndex).MinimumSeparationX and Models.Item(ModelIndex).MinimumSeparationY is 0.0, which means that there is no minimum distance needed for occurrences to be distinct; all occurrences will be considered distinct regardless of other separation conditions. You can set very large values for Models.Item(ModelIndex).MinimumSeparationX and Models.Item(ModelIndex).MinimumSeparationY; if they are large enough, for example, if the values are larger than the size of the image, it is equivalent to setting this separation condition to modDisable since this condition will never be met. In this case, whether an occurrence is distinct or not will depend on the other separation conditions.

The minimum angular separation (the Models.Item(ModelIndex).MinimumSeparationAngle property) determines the minimum difference in angle between occurrences. This value is specified as an absolute angle value. The default value is 10.0 degrees. The minimum value that you can set for Models.Item(ModelIndex).MinimumSeparationAngle is 0, which means that there is no minimum difference in angle needed for occurrences to be distinct; all occurrences will be considered distinct regardless of other separation conditions. The maximum absolute angle difference is 180 degrees. For a diagram of the angle and delta conventions used in ActiveMIL, see the Angle and angular range subsection of the Position, angle, and scale section earlier in this chapter. If the angle is larger than 180 degrees, an error will be returned. At 180 degrees, it is equivalent to setting this separation condition to modDisable. In this case, whether an occurrence is distinct or not will depend on the other separation conditions.

The minimum scale separation (Models.Item(ModelIndex).MinimumSeparationScale) determines the minimum difference in scale between occurrences, as a scale factor. The default value is 1.1. The minimum value that you can set for Models.Item(ModelIndex).MinimumSeparationScale is 1, which means that there is no minimum difference in scale needed for occurrences to be distinct; all occurrences will be considered distinct regardless of other separation conditions. The maximum value that you can set for Models.Item(ModelIndex).MinimumSeparationScale is 4, which is equivalent to setting this separation condition to modDisable. In this case, whether an occurrence is distinct or not will depend on the other separation conditions.

At design-time, you can specify the minimum separation using the X, Y, Angle, and Scale edit fields in the Advanced view of the Models property page.

The four criteria are summarized below in equation form.

The following example illustrates the four separation criteria; an arrow represents two occurrences of the same model and the various types of separation possible.

It should be noted that for a model to be found, the number of visible edges in the occurrence must be sufficient to provide a match according to your acceptance levels.

Shared edges

You can choose to allow occurrences to share edges. If edges are not shared, edges that can be part of more than one occurrence are considered part of the occurrence with the greatest score.

At run-time, use the Models.SharedEdges property to enable or disable sharing of edges between occurrences. At design-time, use the Shared Edges check box in the Models Global property page.

For example, in the illustration below, two occurrences of two simple models share a common edge. With shared edges enabled, these occurrences would have perfect scores:

However, with shared edges disabled (default), the shared edge would be considered part of occurrence 1, since it has the greatest score; the score of occurrence 2 would be subsequently reduced by the loss of the shared edge in the score calculation.

Fit error weighting factor

The fit error weighting factor determines the relative importance of the fit error when calculating the match score and the target score. The higher the factor, the greater the influence that the fit error has on the resulting score and target score for an occurrence (see the earlier section, Determining what is a match, for more information).

At run-time, you can set the fit error weighting factor using the Models.Item(ModelIndex).FitErrorWeightingFactor property. At design-time, use the Fit error factor edit field, in the Advanced view of the Models property page.

Setting this factor to 0.0 means that the fit error is not considered in the score calculation, so the score is equal to the coverage score. Setting this factor to 100.0 means that the fit error has a maximum contribution in the score. The default value is 25.0%.

First and last levels

When using a ModelFinder control whose ModelFinderType property is set to modGeometricControlled, model searches use a hierarchical procedure to reduce the time needed to complete the search. Using this procedure, the ModelFinder control produces a series of smaller, lower-resolution versions of both the target and the models, and the search begins on a much-reduced scale. This series of sub-sampled images is sometimes called a resolution pyramid because of the shape formed when the different resolution levels are stacked on top of each other.

The resolution level for the initial and final stages of the search are typically set automatically. These default levels usually provide the best results for search operations, and you should leave them in most circumstances. If required, you can set the resolution level for the initial and final stages of the search, using the Models.FirstLevel and Models.LastLevel properties of the ModelFinder control. You can set the first and last levels to any integer value from 0 to 7. Level 0 is the original target size and each higher level is half the size (and resolution) of the previous one. The higher the level, the faster the initial search; however, the search will be less reliable at higher levels.

Very small models and search regions can not be subsampled as many times as large ones. If you specify too high a level, the highest practical level will be used.

The Models.FirstLevel and Models.LastLevel properties are not supported if the ModelFinderType property is set to modGeometric.


Global ModelFinder control search settings

User Guide:
Advanced search settings
Customizing search settings
Determining what is a match

Reference:
ModelFinder.Models.Accuracy
ModelFinder.Models.Speed
ModelFinder.Models.Timeout

The global search settings of a ModelFinder control determine the strategy used to search for all of the control's models in the target. These settings directly affect the speed and robustness of the search. The control settings can be adjusted to fit your individual application's needs, using the immediate properties of the control or of the Models property. We recommend experimenting with different settings to find the particular settings that provide your application with the necessary robustness.

This section discusses some of the ModelFinder control settings that were not previously discussed in this chapter.

Setting the search speed

You can specify the algorithm's search speed, using the Models.Speed property. The speed can be set to modMedium, modHigh, modVeryHigh. When you preprocess the control, ActiveMIL analyzes the patterns of the models and determines which shortcuts are appropriate for the search speed setting. At higher search speed settings, the search can take all reasonable shortcuts; therefore, when possible, the search is performed faster than at lower speeds. You can typically use higher speed settings when high accuracy is not an issue; this is because increasing the speed might affect the robustness, as well as the accuracy, of the search.

At design-time, use the Speed edit field in the Models Global property page.

Accuracy

You can set the positional accuracy for your search, using the Models.Accuracy property. It can be set to:

  • modMedium (typically ± 0.03 pixel)
  • modHigh (typically ± 0.02 pixel)

Note that the actual precision achieved is dependent on the quality of the model and of the target (the values listed above are typical when using a high-quality, well-contrasted, low-noise image as the target). The default setting is modMedium.

At design-time, use the Accuracy edit field in the Models Global property page.

Note that increasing the accuracy can slightly reduce the search speed.

Timing out your search

In time critical applications, you can set a time limit in msec for the ModelFinder control to find occurrences of the specified models. After the time limit, the search will stop even if the required number of occurrences is not found. Results are still returned for those occurrences found up to the timeout. However, it is not possible to predict which occurrences will be found before the time limit is reached. The default value is 2000 msec.

At run-time, use the Models.Timeout property to disable or specify the time limit. At design-time, use the Timeout edit field in the Models Global property page.


Speeding up the search

Previous | Next

There are some things you should do to help ensure that your search runs as quickly as possible:

  • Adjust the search speed of the algorithm.
  • Disable calculations specific to range search strategies, if possible.
  • Define several models with different expected angles.
  • Limit the position range.
  • Limit the search scale.
  • Specify the exact number of expected occurrences.
  • Clean up your model.
  • Change the search levels (only when using the controlled geometric search algorithm).

Adjust the search speed of the algorithm

At run-time, you can control the search speed of the algorithm used by your ModelFinder control, using the Models.Speed property of the ModelFinder control. When you preprocess the control, ActiveMIL analyzes the patterns of the models and determines which shortcuts are appropriate for the search speed setting. At higher search speed settings, the search can take all reasonable shortcuts; therefore, when possible, the search is performed faster than at lower speeds. You can typically use higher speed settings when high accuracy is not an issue; this is because increasing the speed might affect the robustness and accuracy.

At design-time, you can control the search speed using the Speed drop-down list of the Models Global property page.

Disable calculations specific to range search strategies

If you expect that the occurrences sought are close to the specified nominal position, angle, or scale (for example, in a registration application), you can try disabling the calculations specific to the search strategies for the corresponding range (setting the Models.SearchAngleEnabled, Models.SearchPositionEnabled, and/or Models.SearchScaleEnabled properties to False) to see if Model Finder can still find the required occurrences. Disabling one or more of these calculations might speed up the search depending on the model and the target.

Define several models with different expected angles

The actual angle of the occurrence does not affect search speed. If you need to search for a model at discrete angles only (for example, at intervals of 90 degrees), it is typically more efficient to define several models with different expected angles than to search through the full angular range.

Limit the position range

Limiting the position range of the model to the minimal required generally decreases the search time. Set the position range of each model to the minimal needed, using the NegativeDeltaX, NegativeDeltaY, PositiveDeltaX, PositiveDeltaY properties of the Models.Item(ModelIndex).Position property, at run-time. If the occurrence must be at a specific position, set these properties to zero.

At design-time, you can set the position range using the drop-down lists on the Position view of the Models property page.

Limit the search scale

It is best to set a fixed scale at which to search for occurrences of a model, instead of specifying a range. Sometimes you must search for model occurrences that can vary in size; in these cases, enabling calculations specific to scale-range search strategies might be necessary. To do so at run-time, you can enable the Models.SearchScaleEnabled property of the ModelFinder control. Keep the scale range as small as possible to avoid slowing down your search. You should not use the scale range to cover different expected scales at different positions. In this case, it is typically more efficient to define several models with different expected scales. This is because a large scale range could potentially slow down your operation; as well, you could find unwanted occurrences.

At design-time, you can enable calculations specific to scale-range search strategies by enabling the Search scale checkbox in the Models Global property page.

Specify the exact number of expected occurrences

If you know the exact number of occurrences for which to search, you should specify it. At run-time, use the Models.TotalNumberOfOccurrences and Models.Item(ModelIndex).NumberOfOccurrences properties of the ModelFinder control. This will typically reduce search time. If you search for more occurrences than required, the search might take longer than necessary.

At design-time, you can set the number of occurrences using the Total occurrence drop-down list box on the Models Global property page.

Clean up your model and your target

Typically, reducing the number of unwanted edges in your models and your target will accelerate your search. There are several different ways to clean up your model and your target:

  • Reduce the noise. Noise in your models can cause inaccurate results. In addition, noise in your models and/or your target can increase the length of time for a search. The most efficient way to remove noise at run-time is to use the Models.SmoothnessLevel and Models.DetailLevel properties of the ModelFinder control. These affect all image-type models in your control, as well as the target when it is an image. At design-time, you can use the Smoothness level and Detail level drop-down lists on the Models Global property page.
  • Use masking. At run-time, you can use the Models.Item(ModelIndex).Mask method to mask regions of the model. Using the Models.Item(ModelIndex).Mask method, you can effectively remove areas of the model from consideration. This might help speed up your search. At design-time, you can use the Mask button on the General view of the Models property page.
  • Use the EdgeFinder control. You can use an EdgeFinder control to extract only the required edges from the image that you want to use as your model source or target, and then use the Edge Finder results as the model source or target instead.

Changing the search levels

When using the controlled geometric search algorithm, it is possible to speed up the search by setting the resolution levels for the search. At run-time, this can be done using either the Models.FirstLevel or the Models.LastLevel properties of the ModelFinder control. This should be done cautiously; the default settings usually provide the best results for search operations.

At design-time, you can use the First Level and Last Level drop-down list boxes on the Models Global property page.


Retrieving and analyzing results

User Guide:
Calibration
Steps to performing a search

Reference:
GraphicContext
ModelFinder.Find
ModelFinder.Results
ModelFinder.Results.Count
ModelFinder.Results.Item.Angle
ModelFinder.Results.Item.Draw
ModelFinder.Results.Item.FitError
ModelFinder.Results.Item.ForwardA
ModelFinder.Results.Item.ModelCoverage
ModelFinder.Results.Item.ModelIndex
ModelFinder.Results.Item.ReverseA
ModelFinder.Results.Item.Score
ModelFinder.Results.Item.ScoreTarget
ModelFinder.Results.Item.TargetCoverage
ModelFinder.Results.Item.UserLabel

After having successfully located your model occurrences in your target using the Find or FindInEdgeFinderResults method, you can extract the required results using the properties of the Results property.

Possible results

The ModelFinder control provides several types of results which provide considerable information on the nature of the occurrence found. In addition to the score, target score, model coverage, target coverage, fit error, and forward / reverse transformation coefficients discussed previously in this chapter, results can be returned for:

  • Number of occurrences.
  • Model index or user label of occurrences.
  • X and Y coordinates of the occurence position.
  • Scale.
  • Polarity.
  • Angle.
  • Whether timeout limit has been reached.

Results are returned in descending order of match score, such that the highest score is returned first. Generally, you should first retrieve the total number of occurrences found for all the models in the ModelFinder control, to ascertain the size of the result array needed. Note that results can be returned for the entire control; the model index (Results.Item(OccurrenceIndex).ModelIndex property) or user label (Results.Item(OccurrenceIndex).UserLabel property) is used to differentiate results between models. For a complete description of all possible results, refer to the descriptions of the properties of the Results property in the ActiveMIL Reference.

Results are indexed as positive integers starting from 1.

Drawing results

The Results.Draw and Results.Item(OccurrenceIndex).Draw methods provide several operations for drawing results in any specified image. You can also choose to draw in the display's overlay image. By drawing into the display's overlay image, you can annotate an image non-destructively (see the Annotating the displayed image non-destructively section of the Display chapter).

You can draw the active edges of the model transformed at the position of the occurrence, the edges of the result occurrence, the bounding box around the occurrence, or a cross-like symbol at the model's reference axis origin transformed at the position of the occurrence. Typically, drawing operations can be combined; you can therefore draw multiple results simultaneously. For example, to draw the active edges of a result occurrence and its position, you would set the Operation parameter to modDrawResultsModelEdges + modDrawResultsPosition.

You can also draw a zoomed version of results that were obtained from a specified region in the target. To do so, set the properties of the Results.DrawingParameters property to an appropriate value. Results.DrawingParameters.RelativeOriginX and Results.DrawingParameters.RelativeOriginY specify, in pixels, the coordinates of the top-left corner of the region in the target while Results.DrawingParameters.ScaleX and Results.DrawingParameters.ScaleY specify the X- and Y-scaling factors used to fill the destination image. Note that if the feature of the specified occurrence to draw with the Results.Item(OccurrenceIndex).Draw method does not fall within the specified region, then nothing is drawn. For more information on zooming, see the Defining and adding models to your ModelFinder control section.

You can use a previously allocated GraphicContext control (see the Graphics chapter) to set the drawing color, or use the default graphic context.


Calibration

See Also | Previous
User Guide:
Calibration
Forward and reverse transformation coefficients
Retrieving and analyzing results

Reference:
Calibration
GraphicContext
ModelFinder.Models.Item.Calibration
ModelFinder.Models.Item.CalibrationName
ModelFinder.Models.Item.Draw
ModelFinder.Results.Item.Draw
ModelFinder.Results.Item.Draw
ModelFinder.Results.Item.ForwardA
ModelFinder.Results.Item.ForwardB
ModelFinder.Results.Item.ForwardC
ModelFinder.Results.Item.ForwardD
ModelFinder.Results.Item.ReverseA
ModelFinder.Results.Item.ReverseB
ModelFinder.Results.Item.ReverseC
ModelFinder.Results.Item.ReverseD

The ActiveMIL ModelFinder control supports calibration. If the target is calibrated, matching is performed and results are calculated in the world coordinate system, compensating for any image distortion. Calibration is performed transparently, without the need for any setting adjustments.

Requirements

To search in a calibrated target with a ModelFinder control, all models of the control must also be calibrated. To calibrate image-type, result-type, and merge-type models, use calibrated model sources or associate a Calibration control with the models, using the Models.Item(ModelIndex).Calibration property. Note that defining models from calibrated model sources and adjusting individual model settings is done in pixels. Synthetic models can be used with a calibrated target if defined in real-world units and associated with the Calibration control of the target.

The models in a ModelFinder control can be associated with different Calibration controls; however, you should ensure that all Calibration controls use the same absolute coordinate system (otherwise, results will be skewed).

Targets must also be calibrated using the same absolute coordinate system as the models of the ModelFinder control. Targets can be either physically corrected or not, without affecting the robustness of the search.

Targets can be grabbed from any number of cameras, each with its own Calibration control, as long as the same absolute coordinate system has been used. For example, if a model has been extracted from a model source image taken using camera A, the model can be used to locate occurrences in a target image grabbed by camera B or camera C (whether or not the image has been physically transformed) since the same absolute coordinate system has been used to calibrate all images. ActiveMIL will transparently convert between the Calibration controls.

If the target is calibrated, results can be retrieved in real-world or pixel units. Note, however, in the presence of distortion some results are meaningless when converted from real-world to pixel units (for example, angle, scale, and transformation coefficient results). For example, if a model appears warped in the target, but the Calibration control of the target compensates for this during the model search, the resulting angle is meaningful in the real-world coordinate system, and meaningless in the image coordinate system.

When using the Models.Item(ModelIndex).Draw or Results.Item(OccurrenceIndex).Draw method to draw features from a calibrated model or results from a calibrated target, the destination drawing image must also be calibrated, using the same absolute coordinate system, otherwise annotations will be skewed. ActiveMIL transparently takes into account the calibration when drawing these annotations.

Models and their Calibration control

Calibration control associations are not saved with the ModelFinder control when using the Save or SaveStream method. When restoring a ModelFinder control, you must re-associate your models with their Calibration controls (which must also have been saved), using the Models.Item(ModelIndex).Calibration property.

If necessary, you can disassociate a Calibration control from a model, by setting either the Calibration property to Nothing (Visual Basic) or NULL (Visual C++) or the CalibrationName property to an empty string (""). This can be useful if you want to use a calibrated model with an uncalibrated target.

Setting the aspect ratio control

When dealing with aspect ratio distortion, you can use the AspectRatio property of the ModelFinder control to compensate for the distortion, if the target is not calibrated. This will allow you to avoid using a Calibration control during the search. This will not compensate for aspect ratio distortion in the model, so this method is mostly useful when searching for synthetic models in a target that has aspect ratio distortion.

When performing the search, results will be returned for the corrected target coordinate system. The results will be converted to the original target coordinate system if you draw them using the Results.Draw or Results.Item(OccurrenceIndex).Draw method.

Using transformation coefficients with calibrated images

You can use forward and reverse transformation coefficients if you want to map points of interest other than the reference axis origin (see Forward and reverse transformation coefficients). When using a calibrated model and calibrated target, forward or reverse transformation coefficients (the ForwardA, ForwardB, ForwardC, ForwardD, ReverseA, ReverseB, ReverseC, ReverseD properties of the Results.Item property) are given for the real-world. This means that they can be used to convert any coordinates in the model world coordinate system to the corresponding coordinates in the target world coordinate system for an occurrence (or vice versa).

Forward and reverse transformation coefficients can be used to determine and draw at equivalent positions in a calibrated model image and a calibrated target image, for an occurrence. To draw these positions, you will need a GraphicContext control. Note, however, that the GraphicContext control's methods and properties only take pixel coordinates. Therefore, to draw a pixel of interest from a calibrated model image to a calibrated target image, you must first convert the required pixel coordinates in the model image to real-world coordinates, using the ConvertCoordinatePixelToWorld method of the Calibration control. Following that, you have to get the forward transformation coefficients. Apply the transformation coeffiecients to the coordinates; then convert the real-world coordinates back to pixel coordinates, using the ConvertCoordinateWorldToPixel method of the Calibration control. A reverse transformation is done similarly, except that you start from the calibrated target.


Edge Finder

Chapter 9: Edge Finder

This chapter explains how to use the ActiveMIL EdgeFinder control to extract and analyze edges in your image.

  • ActiveMIL EdgeFinder control
    • Edges and the EdgeFinder control
  • Steps to performing an edge extraction and analysis
  • Basic concepts
  • Extracting the edges
    • The basics
    • Object contours versus line crests
      • Object contours
      • Line crests
    • How edges are calculated
      • Object contours
      • Line crests
  • Customizing the edge extraction settings
    • Filter type
      • Infinite Impulse Response (IIR) filters
      • Finite Impulse Response (FIR) filters
    • Filter mode
      • Automatically determined kernel size
      • Directly determined kernel size
    • Smoothing
    • Thresholding
    • Edgel accuracy
    • Magnitude type
    • Filling the edge gaps
      • Candidate constraint
      • Region constraint
      • Polarity constraint
      • Continuity constraint
  • Edge features
    • Dimension features
      • Feret
      • Circle fit and line fit
      • Ellipse fit
    • Location features
    • Advanced features
      • Convex perimeter
      • Tortuosity
      • Moment elongation
    • Grouped features
  • Calculating and retrieving results
    • Sorting keys
    • Retrieving the results
    • Selecting the results
      • Edge selection based on edge features
      • Edge selection based on the inter-relationship of edges
      • Edge selection based on the proximity of the edges to a point
    • Internal processing images
      • Derivatives
      • Angles
      • Magnitude
      • Source image and mask
    • Post-calculation
      • Restrictions
    • Annotating the results
  • Advanced edge extraction
    • Approximating the edges
    • Masking the edges
    • Cropping the edges
    • Advanced thresholding
    • Providing the image's derivatives
    • Putting data into the EdgeFinder results
    • Finding the closest edgels to a list of points
      • Location-based constraints
      • Edgel-based constraints
  • Interfacing with the Geometric Model Finder module


ActiveMIL EdgeFinder control

See Also | Next
User Guide:
Calibration
ModelFinder

Reference:
EdgeFinder
ModelFinder

The ActiveMIL EdgeFinder control allows you to extract and analyze edges in a source image.

Many applications can be implemented using the EdgeFinder control, such as complex measurement operations, defect detection, shape recognition, and shape analysis. You can also perform advanced edge manipulations, such as closing broken edges.

The EdgeFinder control allows you to calculate a large number of edge features that can provide you with useful information, such as the edge's minimum and maximum Feret diameter. You can either calculate all required features for all edges, or perform a post-calculation on a refined subset of selected edges, which can speed up your processing time considerably.

The EdgeFinder control can also be used in conjunction with the ModelFinder control; that is, you can define models from the result of an edge extraction, or you can find model occurrences in the result of an edge extraction.

If your source image is calibrated, results are calculated in the world coordinate system, otherwise they are calculated in the image coordinate system. Also, if your source image is calibrated, you can retrieve results in either real-world or pixel units. For more information about calibration, see the Calibration chapter.

Edges and the EdgeFinder control

Edges are curves that delineate a boundary. These can be established from intensity transitions in an image. Well-defined edges come from sharp transitions in value, typically found in highly-contrasted images. Conversely, weak edges come from gradual transitions in value, typically found in smooth images.

Depending on your settings, the EdgeFinder control will extract one of two edge types: object contours or line crests. An object contour is a type of edge that defines the outline of objects in an image. A line crest is a type of edge that defines the skeleton of objects in an image. Note that line crests should be used to extract edges from objects consisting of thin curvilinear shapes, such as fingerprints.

In either case, edges are extracted from the source image and used to form the image's edge map, which represents how the image is defined as a set of edges.

All EdgeFinder control operations, such as annotations and feature calculations are performed using the image's edge map.

You can only extract line crests from grayscale images. However, you can extract object contours from both grayscale and color images. When operating on color images, Edge Finder takes into consideration all three bands (RGB) to establish the presence of an edge.

For an example that uses the EdgeFinder control, see MEdgeFinder.


Steps to performing an edge extraction and analysis

Reference:
EdgeFinder
EdgeFinder.ApplyFilter
EdgeFinder.AutomaticAllocation
EdgeFinder.Calculate
EdgeFinder.CalculateFromResult
EdgeFinder.Draw
EdgeFinder.Filters.Item
EdgeFinder.MagnitudeType
EdgeFinder.NearestNeighbors
EdgeFinder.Results
EdgeFinder.Results.Draw
EdgeFinder.Results.Save
EdgeFinder.Save
EdgeFinder.Threshold

The following steps provide a basic methodology for using the ActiveMIL EdgeFinder control:

  1. If necessary, set general processing properties, such as Threshold and MagnitudeType.
  2. If necessary, select the required features to calculate with the FeatureList property.
  3. Extract the edges from an image and calculate selected features with the Calculate method.
  4. If necessary, exclude or delete edges that do not meet a required criteria with the ApplyFilter method.
  5. If necessary, select new features and perform a post-calculation with the CalculateFromResult method.
  6. Retrieve the required results with the properties of the Results property. Results for the excluded or deleted edges will not be returned.
  7. If necessary, get the coordinates of edgels from the EdgeFinder results that correspond to the closest neighbors from a list of user-specified source point coordinates, using the NearestNeighbors property.
  8. If necessary, copy data from user-supplied arrays to the EdgeFinder results, using the PutEdgeResults method.
  9. If necessary, draw edges and edge features with the Results.Draw method.
  10. If necessary, save the settings of the EdgeFinder control to disk with the Save method.

You can repeat steps 4, 5, and 6 until all the required results are obtained. Note that successively excluding or deleting unwanted edges and then calculating more features is the recommended procedure to achieve the required results, especially if you have a large number of unwanted edges.


Basic concepts

Reference:
EdgeFinder
EdgeFinder.CalculateFromResult

The basic concepts and vocabulary conventions for the ActiveMIL EdgeFinder control are:

  • Edge. A curve that delineates a boundary, which can be established from intensity transitions in an image. The EdgeFinder control considers an edge to be an edge chain, and its features.
  • Edge angle. The angle between the horizontal axis and the edge's perpendicular direction at each edgel location.
  • Edge chain. The set of connected edgels that construct an edge.
  • Edge feature. Information describing an edge, such as its length.
  • Edge magnitude. The strength of the edge at each edgel location. For object contours, the magnitude is the norm of the gradient vector at the edgel position. For line crests, the magnitude is equal to the maximum eigenvalue of the Hessian matrix at the edgel position.
  • Edge map. The set of edges extracted from the source image.
  • Edgel. An elementary point (or edge element) within an edge.
  • Line crest edge. A type of edge that defines the skeleton of objects in an image. Typically, objects should be thin curvilinear shapes; otherwise, you might get unexpected results.
  • Object contour edge. A type of edge that defines the outline of objects in an image. Typically, objects should not be thin curvilinear shapes; otherwise, you might get unexpected results.
  • Post-calculation. Any calculation made to refine results after an initial calculation. To perform a post-calculation, you must use the CalculateFromResult method.
  • Source image. The image from which the edges are extracted.

After edge chains have been extracted, the terms edge chain and edge are used interchangeably.


Extracting the edges

Reference:
EdgeFinder
EdgeFinder.EdgeType
EdgeFinder.ForegroundValue
EdgeFinder.MagnitudeType
EdgeFinder.Threshold

Before you write even the simplest application, it is important to have a basic understanding of how the EdgeFinder control extracts edges, and what the differences are between the various types of edges (object contours and line crests). Note that if you are writing slightly more advanced applications, it is also worth having a more in depth understanding of how edges are calculated.

The basics

The EdgeFinder control uses operations that are based on differential analysis, where edges (either contours or line crests) are extracted by analyzing intensity transitions in images. Transitions in intensity can come from many physical phenomenon, such as shadows and illumination variations. However, strong transitions are typically caused by the presence of an object contour or a line crest in the image, as illustrated below.

Note that the main intensity transition in the image is observed in the perpendicular direction relative to the object border (for an object contour) or line shape (for a line crest).

Edges are extracted in three general steps. First, a filtering process provides an enhanced image of the edges based on the computation of the image's derivatives. Second, detection and thresholding operations determine, from the image enhancement, all pertinent edge elements, or edgels, and accurately calculates their positions. Third, neighboring edgels are connected to build the edge chains (which results in the image's edge map), and features are calculated for each edge.

To fit the requirements of your application, the EdgeFinder control allows you to customize each step's corresponding processing properties, such as Threshold and MagnitudeType.

Object contours versus line crests

To properly analyze the structures present in an image, it is often useful to consider the image as a topographic surface, where the intensity of each pixel is seen as the elevation value of the surface. By doing so, contours and crests can be more easily understood.

Object contours

An object contour is the inter-pixel boundary that splits into thin layers (delaminates) the separation between the object and the image background, as illustrated below. The edge of the contour is located at the sharpest point within the intensity transition. Moreover, the intensity transition is strongest in the perpendicular direction to the object contour.

Well-defined contours have sharp transitions in value. The smoother the image, the more gradual the change, and the weaker the contour.

To specify a contour context type, set the EdgeType property to edgeContour.

Line crests

A line crest is the skeleton of a thin curvilinear shape in the image, as illustrated below. The crest is located at the sharpest point of the Gaussian-like profile of the line shape. Moreover, the intensity transition is strongest in the perpendicular direction to the line crest.

Well-defined line crests are thin and have sharp transitions in value. The smoother the image or the larger the lines, the more gradual the change, and the weaker the line crests.

A line crest can either be darker (a valley-like Gaussian profile), lighter (a ridge-like Gaussian profile), or both darker and lighter than the image's background. To specify the color of the line crest to extract, set the ForegroundValue property to edgeForegroundBlack, edgeForegroundWhite, or edgeForegroundAny. The default setting is edgeForegroundBlack.

To specify a line crest context type, set the EdgeType property to edgeCrest.

Typically, you should use edgeCrest contexts for edges that are established from thin curvilinear shapes; otherwise, you might get unexpected results.

How edges are calculated

For basic applications, you need not know how the EdgeFinder control calculates edges. However, an understanding of this process can help you solve more advanced problems, and adjust some advanced settings.

Object contours

The enhanced image of the object contours is achieved by calculating the gradient magnitude of each pixel in the image. The stronger the intensity transition, the greater the magnitude will be.

The gradient magnitude is calculated at each pixel position from the image's first derivatives. It is defined as:

where Ix and Iy are, respectively, the X and Y derivative values. They define the components of the gradient vector as:

An edgel is located at the maximum value of the gradient magnitude over adjacent pixels, in the direction defined by the gradient vector. The gradient direction is the direction of the steepest ascent at an edgel in the image, while the gradient magnitude is the steepness of that ascent. Note that the gradient direction is also the perpendicular direction to the object contour. Well defined contours are extracted from strong and sharp intensity transitions. Note that a strong contrast between the objects and their background improves the edge detector's robustness and location accuracy.

Note that the EdgeFinder control also supports 3-band images, when extracting object contours. The extraction process is similar to single band images, except the enhanced image of the object contours is achieved by calculating a generalized gradient magnitude of each pixel in the image, for each color band (RGB).

Line crests

The enhanced image of the line crests is achieved by calculating the Hessian magnitude of each pixel in the image. The stronger the crest, the greater the magnitude will be.

The magnitude is defined as the maximum eigenvalue of the Hessian matrix, calculated at each pixel position from the image's second derivatives. The Hessian matrix is defined as:

where Ixx, Iyy and Ixy are, respectively, the X, Y and cross second derivative values of the image.

An edgel is located at the maximum value of the image magnitude over adjacent pixels, in the direction defined by the associated Eigen vector. Well defined line crests are extracted from strong and sharp intensity transitions. Note that a strong contrast between the lines and the image's background improves the edge detector's robustness and the location accuracy.

Note that the sign of the magnitude at the crest's edgel locations changes with the color of the line crest. That is, a dark crest has a strong positive magnitude, while a light crest has a strong negative magnitude. The geometric interpretation of the magnitude and its associated orientation is the direction where the crest is the sharpest. Similar to object contours, this direction is also the direction perpendicular to the line crest.


Customizing the edge extraction settings

User Guide:
Advanced edge extraction
Advanced thresholding
Buffer overscan
Calculating and retrieving results
Extracting the Edges
Images
Object contours versus line crests
Post-calculation

Reference:
EdgeFinder
EdgeFinder.Accuracy
EdgeFinder.DrawingParameters
EdgeFinder.EdgeType
EdgeFinder.ExtractionFilter
EdgeFinder.ExtractionFilter.FilterType
EdgeFinder.ExtractionFilter.Kernel.Depth
EdgeFinder.ExtractionFilter.Kernel.Width
EdgeFinder.ExtractionFilter.Mode
EdgeFinder.ExtractionFilter.Smoothness
EdgeFinder.FillGapParameters
EdgeFinder.FillGapParameters.Angle
EdgeFinder.FillGapParameters.Candidate
EdgeFinder.FillGapParameters.Continuity
EdgeFinder.FillGapParameters.Distance
EdgeFinder.FillGapParameters.Polarity
EdgeFinder.ImageFeature
EdgeFinder.MagnitudeType
EdgeFinder.Threshold
EdgeFinder.Threshold.HighValue
EdgeFinder.Threshold.LowValue
EdgeFinder.Threshold.Mode

Once you have set the right context type for the EdgeFinder control (with the EdgeType property), you can customize the image processing algorithm with the appropriate processing properties. For example, to fit the requirements of your application, you can set the appropriate:

  • Edge extraction filter (ExtractionFilter.FilterType).
  • Filter mode (ExtractionFilter.Mode).
  • Noise reduction factor (ExtractionFilter.Smoothness).
  • Sensitivity of the edgel detection (Threshold.Mode).
  • Accuracy of the edgel positions (Accuracy).

You can also choose to fill in the edge gaps, which is the process of linking broken edges together based on a set of criteria (FillGapParameters.Distance and FillGapParameters.Angle). In addition, Edge Finder allows you to set an overscan, which determines how to handle an image's bordering pixels. For more information, see the Buffer overscan section in the Images chapter.

Generally, the default settings described below are sufficient for the majority of images. You should therefore adjust these settings only when dealing with very noisy images, extremely low or multi-contrast images, or images with very thin, refined features. In all cases, it is recommended that you try the default setting first and, if necessary, experiment with different settings to achieve the required edge map and accuracy.

For more advanced information on extracting edges, see the Advanced edge extraction section.

Note that the edge extraction process involves a denoising operation to even out rough edges and remove noise. The filter type, filter mode, kernel depth, smoothness factor, and threshold mode all work in concert to control this operation, and ultimately determine which edges to extract. Changing any one of these factors will, to varying degrees, alter the edges that are extracted.

Filter type

The edge extraction filter is used to compute the source image's derivatives, which in turn are used to calculate the edge's magnitude (gradient or Hessian) and angle. You can set the type of filter used when performing the edge extraction with the ExtractionFilter.FilterType property. You can choose either an Infinite Impulse Response (IIR) edge extraction filter, or a Finite Impulse Response (FIR) edge extraction filter. The default setting is the IIR filter, edgeShen.

IIR filters allow you to extract both crests and contours, while FIR filters only allow you to extract contours. IIR filters also allow you to take advantage of Edge Finder's smoothing capabilities, while FIR filters ignore your smoothness setting. For more information on smoothing, see the Smoothing subsection, later in this section.

Infinite Impulse Response (IIR) filters

The ExtractionFilter.FilterType property establishes the contribution of each neighboring pixel. This property provides you with the following IIR filters to extract edges:

  • Shen-Castan (edgeShen), which is an exponential weighting function, of the general form
  • Canny-Deriche (edgeDeriche), which is an exponential weighting function, of the general form

Note that to perform the edge extraction, IIR filters can either be implemented in recursive or kernel mode. For more information on filter modes, see the Filter mode subsection, later in this section.

The default filter, Shen-Castan, typically performs an effective edge extraction; it achieves a very good localization of edges, which makes it ideal for the majority of images.

However, Shen-Castan can sometimes be unexpectedly sensitive to noise, or can yield inappropriate results if you are extracting unusually thick crests. If this is the case, you should try Canny-Deriche, which is better suited to handle these situations, since it places more emphasis on an edge's neighbors than Shen-Castan. The following image illustrates how the first derivative distribution of values can affect results for both Shen-Castan and Canny-Deriche:

Finite Impulse Response (FIR) filters

The ExtractionFilter.FilterType property provides you with the following FIR filters to extract edges.

  • edgeFreiChen, which can be represented with the following convolution kernels:
  • edgePrewitt, which can be represented with the following convolution kernels:
  • edgeSobel, which can be represented with the following convolution kernels:

Filter mode

Filtering is a neighborhood calculation that is implemented during the edge extraction process. Depending on the filter, this calculation can either be implemented recursively, or using a convolution kernel. Since FIR filters use a specific 3x3 kernel defined by the EdgeFinder control, you cannot calculate them recursively, nor can you alter their kernel size.

On the other hand, IIR filters use an exponential weighting function to perform the filtering. This function can either be implemented recursively, or with a kernel of different sizes. To specify the mode in which to perform the filtering, set the ExtractionFilter.Mode property to either edgeRecursive or edgeKernel.

Typically, the default filter mode is edgeRecursive. However, if you have dedicated hardware performing the operation, the default is edgeKernel. Unless you have dedicated hardware, an edgeKernel convolution should not be set since it can take a long time to process.

If you do not have dedicated hardware, edgeRecursive is faster than edgeKernel, unless you are using very small kernels (for example, 3x3). If you have dedicated hardware, edgeKernel is typically faster.

If you specify edgeRecursive, the filtering is done using a recursive implementation of the IIR filter. If this mode actually used a kernel, the kernel size would be theoretically infinite.

If you specify edgeKernel, the filtering is done using a kernel approximation of the IIR filter. In this case, you must set the size of the convolution kernel with the ExtractionFilter.Kernel.Width property. The size can either be determined automatically or directly. Note that the size of the kernel can be constrained by the available hardware resources.

When using edgeKernel, you must also specify the bit depth with which kernel values are calculated. To do so, set the ExtractionFilter.Kernel.Depth property to either 8 or 16. The default is 8-bit, which is faster than 16-bit, although it is less accurate.

Automatically determined kernel size

To specify the size of the kernel automatically, set the ExtractionFilter.Kernel.Width property to edgeAutomatic. This is the default setting. In this case, the EdgeFinder control establishes the maximum possible kernel size, for a given smoothness factor. You can set the smoothness factor with the ExtractionFilter.Smoothness property.

The size of the kernel, for a given smoothness factor, is calculated as the size at which the quantified filter values are zero. If the hardware resources are available, the smoothness factor will always be adhered to. However, if the smoothness factor calls for a kernel size that is beyond the limits of your hardware, the smoothness factor will be sacrificed to achieve the largest possible kernel size.

Note that changing the smoothness factor can also change the convolution kernel size, which can affect processing time. That is, the larger the kernel, the longer the processing will take.

Directly determined kernel size

To specify the size of the kernel directly, set the ExtractionFilter.Kernel.Width property to an odd integer that is greater than or equal to 3. Note that larger kernels result in longer processing times.

If your hardware cannot handle the kernel size, the effective smoothness factor will be saturated to the maximum possible value for the specified kernel size. That is, a filter is always made to fit in the kernel without a pre-imposed truncature, other than quantification; essentially, after quantification, the values outside the kernel limits are all zero.

Smoothing

The ExtractionFilter.Smoothness property allows you to set the degree of smoothness applied in the edge extraction. The smoothing operation evens out rough edges and removes noise; in effect, the smoothness factor affects which edges are extracted.

The range of smoothness varies from 0.0 (no smooth) to 100.0 (a very strong smooth). The default setting is 50.0. Note that if you are using an FIR filter, the smoothness value is ignored.

Increasing the smoothness value does not affect the processing time. However, a very high smoothing level can result in a loss of important detail and a decrease in precision. Note that, the range of smoothing is not linear; that is, the higher the smoothing factor, the greater the difference between settings. For example, changing the smoothing from 30.0 to 50.0 is less significant than changing the smoothing from 50.0 to 70.0.

Thresholding

The EdgeFinder control uses an hysteresis thresholding to perform the edge extraction. That is, the extracted edge chains are built such that the magnitude values of all connected edgels are stronger than the lower bound threshold value (set with the Threshold.LowValue property) and at least one edgel in each edge chain has a magnitude that is stronger than the upper bound threshold value (set with the Threshold.HighValue property). In essence, thresholding determines which edges are extracted from the source image based on the magnitude of each edgel. For more information on edgel magnitude, see the Magnitude type subsection below.

Thresholding can also be applied during post-calculation. In this case, threshold settings are applied to the EdgeFinder results, rather than the source image. Except for filtering operations, the edge extraction process is completely recalculated. As a result, all internal processing images must have been initially saved, with the properties of the ImageFeature property. For more information on post-calculation, see the Post-calculation subsection in the Calculating and retrieving results section.

Edge Finder provides several predefined threshold modes that automatically select appropriate values for the lower and the upper bound thresholds, based on an internal image evaluation and noise estimation. To automatically determine the threshold settings with which to extract edges, set the Threshold.Mode property to one of the following: edgeThresholdVeryHigh, edgeThresholdHigh, edgeThresholdMedium, edgeThresholdLow, or edgeThresholdDisable. The default setting is edgeThresholdHigh. Note that lower threshold modes result in a more sensitive edgel detection; that is, more edgels are typically detected. For example, edgeThresholdVeryHigh always results in a lower (or equal) number of edgels than edgeThresholdHigh, edgeThresholdHigh always results in a lower (or equal) number of edgels than edgeThresholdMedium, and so on.

The default setting (edgeThresholdHigh) is typically sufficient since it offers a robust detection of pertinent edgels, even from images presenting some contrast variations, noise, and non-uniform illumination. However, for multi-contrast images, or for images with a lot of noise or non-uniform illumination, some edges can be missed. In these cases, the setting edgeThresholdMedium should be used. If all relevant edges are still not being extracted, then use edgeThresholdLow, which will get all edges over a minimum noise-based estimated threshold. Finally, to extract all edgels in the image, use edgeThresholdDisable, which will set the lower and upper bound threshold values to 0. Note that edgeThresholdLow and edgeThresholdDisable should be used carefully since a large number of unnecessary edgels might be extracted. For images that have strong contrast, little noise, and consistent illumination, the edgeThresholdVeryHigh setting can be used, since it will extract only the strongest edges. Note that edgeThresholdVeryHigh should also be used carefully since pertinent edgels might not be extracted.

In the majority of cases, the EdgeFinder control's predefined threshold modes should provide you with sufficient thresholding control. However, for advanced applications, you might want to explicitly set the threshold bounds. To do so, set the Threshold.Mode property to edgeThresholdUserDefined, and use the Threshold.LowValue and Threshold.HighValue properties to set the lower and upper threshold bounds, respectively. The default settings are 0.

If the Threshold.Mode property is not set to edgeThresholdUserDefined, your Threshold.LowValue and Threshold.HighValue property settings have no effect.

Note that the threshold bounds must be provided as positive values. However, for line crests, the EdgeFinder control will consider these values to be negative and/or positive, depending on whether the edges were extracted as darker (negative edgel values), lighter (positive edgel values), or both darker and lighter (negative and positive edgel values) than the image's background. For more information on extracting line crests, see the subsection Object contours versus line crests, in the Extracting the Edges section.

Advanced users might want to customize the behavior of the hysteresis thresholding; to do so, see the subsection Advanced thresholding, in the Advanced edge extraction section.

Edgel accuracy

The Accuracy property determines the accuracy used to estimate edgel positions. Edgel accuracy can be set to one of the following: edgeAccuracyHigh, edgeAccuracyVeryHigh, or edgeAccuracyDisable. The default setting is edgeAccuracyHigh. When you set the accuracy to high, edgels are calculated with subpixel accuracy, which is typically sufficient. When you set the accuracy to very high, a classical image formation model is used to compensate for any pixel-distortion aberration, resulting in very precise subpixel edge accuracy. If you want to disable subpixel accuracy and calculate edgels with pixel precision, set the Accuracy property to edgeAccuracyDisable.

For the most part, edgel accuracy varies with the image's dynamic range, sharpness, and noise. The best accuracy is achieved in well-contrasted, noise-free images. In perfect situations, the EdgeFinder control can provide a subpixel edgel location accuracy up to 1/128th of a pixel. However, in real situations, it is reasonable to expect an accuracy from 1/10th of a pixel to 1/40th of a pixel, depending on physical limitations due to the transition's dynamic range and the image's noise.

You can notice the accuracy of the edgels if you zoom a region of the source image that was used to calculate results by specifying the appropriate relative origin and scale properties of the DrawingParameters property.

Magnitude type

You can set the type of magnitude used to determine edgel positions using the MagnitudeType property. The magnitude type can be set to either edgeNorm, where the gradient magnitude is used, or edgeSquareOfNorm, where the square of the gradient magnitude is used. The default setting is edgeSquareOfNorm.

Since edgeSquareOfNorm uses the square of the gradient magnitude, it is faster, though less precise than edgeNorm. Typically, edgeSquareOfNorm is used since it preserves a very good edgel location accuracy. Note that changing the magnitude type can slightly modify the resulting edge map.

Filling the edge gaps

The EdgeFinder control allows you to fill gaps between edges; that is, to automatically link the extremities of non-closed edges together, depending on their relative positions and orientations. Unexpected broken edges can occur when dealing with noisy images, and connecting them can significantly improve the quality of your results. To help you choose which edges can be linked, the EdgeFinder control provides a series of constraints, described below. Note that edges will only be linked if they adhere to each constraint.

Filling edge gaps can also be done in post-calculation; however, there are some restrictions that must be taken into account. For more information, see the Post-calculation subsection in the Calculating and retrieving results section.

Candidate constraint

You can choose whether to fill an edge extremity with the other extremity of the same edge, or with an extremity of any edge. To connect an edge's extremity with the extremity of any edge, set the FillGapParameters.Candidate property to edgeCandidateAny.

To connect the edge extremities of the same edge, set the FillGapParameters.Candidate property to edgeCandidateSame. This is the default value; it is the same as closing an open edge.

Region constraint

The FillGapParameters.Distance and FillGapParameters.Angle properties define the region where two edge extremities can be linked. That is, when searching for edge extremity candidates to fill edge gaps, FillGapParameters.Distance sets the maximum distance radius, while FillGapParameters.Angle sets the aperture angle. Note that two edge extremities can only be linked if both meet the fill gap distance and fill gap angle criteria, and if both are included in the search region of the other.

The fill gap distance must be specified in pixel units. The gap distance should be set carefully and large values should generally be avoided, otherwise the wrong edges can be connected. Typically, distance values should not exceed 5 pixels; however, if necessary, you can also specify no distance restraint by setting the FillGapParameters.Distance property to edgeInfinite. The default fill gap distance value is 0.0 (no edge linking) and the default fill gap angle value is 360.0 degrees.

Polarity constraint

When searching for edge extremity candidates, you can also use the FillGapParameters.Polarity property to specify that only edges with a certain polarity can be linked. The edge's polarity indicates whether edges were established as a transition from light to dark, or vice versa. The default setting is edgePolarityAny, where all edges that meet the distance and angle restraints are considered candidates, regardless of the edge's polarity. If the FillGapParameters.Polarity property is set to edgeSame, only edges with the same polarity are considered potential candidates. If FillGapParameters.Polarity is set to edgeReverse, only edges with reverse polarity are considered potential candidates.

Note that the edgeSame and edgeReverse polarity settings are not available if the EdgeType property is set to edgeCrest.

Continuity constraint

It is possible that multiple edges meet the fill gap distance (set with the FillGapParameters.Distance property), fill gap angle (set with the FillGapParameters.Angle property), and fill gap polarity (set with the FillGapParameters.Polarity property) settings. When this occurs, you must decide which edges should be linked. To help choose, the EdgeFinder control provides a continuity constraint.

The continuity constraint allows you to decide whether the candidate chosen to fill the edge gap is selected according to its proximity or its maximum continuity. To set the continuity constraint, use the FillGapParameters.Continuity property. The range of the continuity varies from 0.0 to 100.0. When set to 0.0, the closest edge extremity is chosen to link edges together. When set to 100.0, the edge that gives the most continuous result (minimum curvature) is chosen. The default value is 50.0.


Edge features

User Guide:
Calculating and retrieving results

Reference:
EdgeFinder
EdgeFinder.FeatureList
EdgeFinder.FeatureList
EdgeFinder.FeatureList.AverageStrength
EdgeFinder.FeatureList.Box.DeselectAll
EdgeFinder.FeatureList.Box.FeretX
EdgeFinder.FeatureList.Box.FeretY
EdgeFinder.FeatureList.Box.SelectAll
EdgeFinder.FeatureList.Box.XMaximum
EdgeFinder.FeatureList.Box.XMinimum
EdgeFinder.FeatureList.Box.YMaximum
EdgeFinder.FeatureList.Box.YMinimum
EdgeFinder.FeatureList.CenterOfGravity.DeselectAll
EdgeFinder.FeatureList.CenterOfGravity.SelectAll
EdgeFinder.FeatureList.CenterOfGravity.X
EdgeFinder.FeatureList.CenterOfGravity.Y
EdgeFinder.FeatureList.CircleFit.CenterX
EdgeFinder.FeatureList.CircleFit.CenterY
EdgeFinder.FeatureList.CircleFit.Coverage
EdgeFinder.FeatureList.CircleFit.DeselectAll
EdgeFinder.FeatureList.CircleFit.Error
EdgeFinder.FeatureList.CircleFit.Radius
EdgeFinder.FeatureList.CircleFit.SelectAll
EdgeFinder.FeatureList.Closure
EdgeFinder.FeatureList.ContactPoints.DeselectAll
EdgeFinder.FeatureList.ContactPoints.SelectAll
EdgeFinder.FeatureList.ContactPoints.XMaximumAtYMaximum
EdgeFinder.FeatureList.ContactPoints.XMinimumAtYMinimum
EdgeFinder.FeatureList.ContactPoints.YMaximumAtXMinimum
EdgeFinder.FeatureList.ContactPoints.YMinimumAtXMaximum
EdgeFinder.FeatureList.ConvexPerimeter
EdgeFinder.FeatureList.DeselectAll
EdgeFinder.FeatureList.EllipseFit.Angle
EdgeFinder.FeatureList.EllipseFit.CenterX
EdgeFinder.FeatureList.EllipseFit.CenterY
EdgeFinder.FeatureList.EllipseFit.Coverage
EdgeFinder.FeatureList.EllipseFit.DeselectAll
EdgeFinder.FeatureList.EllipseFit.Error
EdgeFinder.FeatureList.EllipseFit.MajorAxis
EdgeFinder.FeatureList.EllipseFit.MinorAxis
EdgeFinder.FeatureList.EllipseFit.SelectAll
EdgeFinder.FeatureList.FastLength
EdgeFinder.FeatureList.FeretElongation
EdgeFinder.FeatureList.FirstPoint.DeselectAll
EdgeFinder.FeatureList.FirstPoint.SelectAll
EdgeFinder.FeatureList.FirstPoint.X
EdgeFinder.FeatureList.FirstPoint.Y
EdgeFinder.FeatureList.GeneralFeret
EdgeFinder.FeatureList.Length
EdgeFinder.FeatureList.LineFit.A
EdgeFinder.FeatureList.LineFit.B
EdgeFinder.FeatureList.LineFit.C
EdgeFinder.FeatureList.LineFit.DeselectAll
EdgeFinder.FeatureList.LineFit.Error
EdgeFinder.FeatureList.LineFit.SelectAll
EdgeFinder.FeatureList.MaximumFeretAngle
EdgeFinder.FeatureList.MaximumFeretDiameter
EdgeFinder.FeatureList.MeanFeretDiameter
EdgeFinder.FeatureList.MinimumFeretAngle
EdgeFinder.FeatureList.MinimumFeretDiameter
EdgeFinder.FeatureList.MomentElongation
EdgeFinder.FeatureList.Position.DeselectAll
EdgeFinder.FeatureList.Position.SelectAll
EdgeFinder.FeatureList.Position.X
EdgeFinder.FeatureList.Position.Y
EdgeFinder.FeatureList.SelectAll
EdgeFinder.FeatureList.Size
EdgeFinder.FeatureList.Strength
EdgeFinder.FeatureList.Tortuosity
EdgeFinder.NumberOfFeretAngles
EdgeFinder.Results.Item

The EdgeFinder control allows you to calculate many edge features that can provide useful information, such as the edge's length and center of gravity. To select features for calculation, set the appropriate property of the FeatureList property to True, otherwise set it to False. You can also select all features for calculation with a single call to the FeatureList.SelectAll method, and you can select a group of features for calculation with a single call to the SelectAll method of the appropriate property of the FeatureList property. To retrieve the results of the feature calculation, use the relevant property of the Results.Item property. For more information on retrieving results, see the Calculating and retrieving results section.

Note that the FeatureList.LabelValue property is the only edge feature that is selected for calculation by default. The label value is a positive integer greater or equal to one. Each edge in an image has a unique label value. The label allows you to track or choose particular edges for drawing, edge manipulation, or result retrieval purposes. Typically, you should not disable the edge's label value.

Edge features generally fall into one of the following groups:

  • Dimension features (typically very fast to compute).
  • Location features (typically fast to compute).
  • Advanced features (can take some time to compute, due to their complexity).

To compute features as efficiently as possible, you should calculate a few features first (preferably, the fastest), and eliminate as many unnecessary edges as possible. Then, post-calculate expensive features on the remaining edges. For more information on how to select and post-calculate edges, see the Calculating and retrieving results section.

Note that occasionally, enabling a feature for calculation will result in another feature being calculated and available for result retrieval. For example, setting the FeatureList.MomentElongation property to True will allow you to retrieve the result of each edge's center of gravity.

The following subsections list all the EdgeFinder control features that you can select for calculation, according to dimension, location, and advanced application. There is also a subsection on selecting a group of features for calculation in a single call. Note that only the more complicated features are explained in some detail. For a description of each feature, refer to the properties of the FeatureList property.

Dimension features

The EdgeFinder control dimension feature properties are:

  • FeatureList.Box.FeretX
  • FeatureList.Box.FeretY
  • FeatureList.CircleFit.CenterX
  • FeatureList.CircleFit.CenterY
  • FeatureList.CircleFit.Radius
  • FeatureList.CircleFit.Error
  • FeatureList.CircleFit.Coverage
  • FeatureList.Closure
  • FeatureList.EllipseFit.Angle
  • FeatureList.EllipseFit.CenterX
  • FeatureList.EllipseFit.CenterY
  • FeatureList.EllipseFit.Coverage
  • FeatureList.EllipseFit.Error
  • FeatureList.EllipseFit.MajorAxis
  • FeatureList.EllipseFit.MinorAxis
  • FeatureList.FastLength
  • FeatureList.FeretElongation
  • FeatureList.GeneralFeret
  • FeatureList.Length
  • FeatureList.LineFit.A
  • FeatureList.LineFit.B
  • FeatureList.LineFit.C
  • FeatureList.LineFit.Error
  • FeatureList.MaximumFeretAngle
  • FeatureList.MaximumFeretDiameter
  • FeatureList.MeanFeretDiameter
  • FeatureList.MinimumFeretAngle
  • FeatureList.MinimumFeretDiameter
  • FeatureList.Size

Feret

The Feret is the diameter of the edge at various angles. Several Feret diameters are illustrated in the diagram below. Note that the angle at which the Feret diameter is taken (relative to the horizontal axis) is specified as a subscript to F.

Some features, such as the edge's maximum Feret diameter, are determined by testing the Feret diameter of the edge at several angles. To set the number of angles, use the NumberOfFeretAngles property. The default number of angles tested is 8, which is typically sufficient. Note that increasing the number of Ferets increases the accuracy of the results; however, it also increases the processing time.

Circle fit and line fit

The circle fit is calculated as the circle that best fits the edge, while the line fit is calculated as the line that best fits the edge.

To calculate the coordinates of the center of the circle that is the best fit for each edge, set the FeatureList.CircleFit.CenterX and FeatureList.CircleFit.CenterY properties to True. To calculate the radius of the circle that is the best fit for each edge, set the FeatureList.CircleFit.Radius property to True.

To calculate the A, B, and C variables that define the line that is the best fit for each edge, set the FeatureList.LineFit.A, FeatureList.LineFit.B, and FeatureList.LineFit.C properties to True. Note that these variables are based on the linear equation, Ax + By + C = 0.

To calculate the coverage of the circle fit, set the FeatureList.CircleFit.Coverage property to True. The coverage describes the portion of the circle covered by the edge. The value returned is between 0.0 and 1.0, inclusive, where 0.0 equals no coverage, 0.5 equals 50 percent coverage, and 1.0 equals 100 percent coverage.

Note that, since a line is theoretically infinite, the coverage for a line fit would always be 100 percent.

You can also calculate the error of the circle fit by setting the FeatureList.CircleFit.Error property to True, and the error of the line fit by setting the FeatureList.LineFit.Error property to True. These values are calculated as the average quadratic error of the fit.

Ellipse fit

The ellipse fit is calculated as the ellipse that best fits the edge.

To calculate the coordinates of the center of the ellipse that is the best fit for each edge, set the FeatureList.EllipseFit.CenterX and FeatureList.EllipseFit.CenterY properties to True. To calculate the major and minor axes of the ellipse that is the best fit for each edge, set the FeatureList.EllipseFit.MajorAxis and FeatureList.EllipseFit.MinorAxis properties to True.

To calculate the angle of the ellipse fit, set the FeatureList.EllipseFit.Angle property to True. To calculate the error of the ellipse fit, set the FeatureList.EllipseFit.Error property to True. The fit error is calculated as the average quadratic error of the fit.

Location features

The EdgeFinder control location feature properties are:

  • FeatureList.Box.XMaximum
  • FeatureList.Box.XMinimum
  • FeatureList.Box.YMaximum
  • FeatureList.Box.YMinimum
  • FeatureList.CenterOfGravity.X
  • FeatureList.CenterOfGravity.Y
  • FeatureList.ContactPoints.XMaximumAtYMaximum
  • FeatureList.ContactPoints.YMaximumAtXMinimum
  • FeatureList.ContactPoints.XMinimumAtYMinimum
  • FeatureList.ContactPoints.YMinimumAtXMaximum
  • FeatureList.FirstPoint.X
  • FeatureList.FirstPoint.Y
  • FeatureList.Position.X
  • FeatureList.Position.Y

The following edge map illustrates where each edge location feature is located:

Advanced features

The EdgeFinder control advanced feature properties are:

  • FeatureList.AverageStrength
  • FeatureList.ConvexPerimeter
  • FeatureList.MomentElongation
  • FeatureList.Strength
  • FeatureList.Tortuosity

Convex perimeter

If you select the convex elongation feature for calculation, an approximation of the perimeter of each edge's convex hull will be calculated. Abstractly, an edge's convex perimeter is very much like taking a rubber band, and placing it tautly around the edge. To calculate the convex perimeter, set the FeatureList.ConvexPerimeter property to True.

The convex perimeter is derived by taking the diameter of the edge at different angles. The greater the number of Ferets used to calculate the diameter, the more accurate the approximation. Use the NumberOfFeretAngles property to adjust the number of Ferets. Note that increasing the number of Ferets increases the accuracy of the results; however, it also increases the processing time.

Tortuosity

An edge's tortuosity is the diagonal length of the edge's bounding box (FeatureList.Box.XMaximum, FeatureList.Box.XMinimum, FeatureList.Box.YMaximum, FeatureList.Box.YMinimum), divided by the length of the edge (FeatureList.Length). Therefore, a non-tortuous edge (a straight line) will have a tortuosity of 1.0, while a tortuous edge will have its tortuosity decreasing toward zero. To calculate the tortuosity, set the FeatureList.Tortuosity property to True.

Moment elongation

The moment elongation is a geometric elongation measure of the edge. It is defined as the ratio of the principle values of the edge's inertial matrix, which corresponds to the principal directions of the edge shape. The elongation value varies from 0.0 to 1.0. Values closer to 0.0 are considered to have a low elongation, while values closer to 1.0 are considered to have a high elongation. To calculate the moment elongation, set the FeatureList.MomentElongation property to True.

Grouped features

The EdgeFinder methods that allow you to select a group of features for calculation in a single call are:

  • FeatureList.SelectAll, FeatureList.DeselectAll
  • FeatureList.Box.SelectAll, FeatureList.Box.DeselectAll
  • FeatureList.CenterOfGravity.SelectAll, FeatureList.CenterOfGravity.DeselectAll
  • FeatureList.CircleFit.SelectAll, FeatureList.CircleFit.DeselectAll
  • FeatureList.ContactPoints.SelectAll, FeatureList.ContactPoints.DeselectAll
  • FeatureList.EllipseFit.SelectAll, FeatureList.EllipseFit.DeselectAll
  • FeatureList.FirstPoint.SelectAll, FeatureList.FirstPoint.DeselectAll
  • FeatureList.LineFit.SelectAll, FeatureList.LineFit.DeselectAll
  • FeatureList.Position.SelectAll, FeatureList.Position.DeselectAll

Each of these methods has multiple features associated with it. Therefore, if you call one of them, all the features associated with it will be appropriately selected or deselected. For example, calling the FeatureList.CenterOfGravity.SelectAll method will result in the center of gravity of each edge to be calculated. This is equivalent to setting the FeatureList.CenterOfGravity.X and FeatureList.CenterOfGravity.Y properties to True.

The features that belong to each group is typically self evident. For a description of each, see the relevant properties of the FeatureList property.


Calculating and retrieving results

User Guide:
Advanced edge extraction
Calibration
Customizing the edge extraction settings
Edge features
Filter type
Masking the edges
Thresholding

Reference:
Calibration
Calibration.OutputCoordinateSystem
Calibration.OutputCoordinateSystem
EdgeFinder
EdgeFinder.ApplyFilter
EdgeFinder.Calculate
EdgeFinder.CalculateFromResult
EdgeFinder.Draw
EdgeFinder.DrawingParameters.RelativeOriginX
EdgeFinder.DrawingParameters.RelativeOriginY
EdgeFinder.DrawingParameters.ScaleX
EdgeFinder.DrawingParameters.ScaleY
EdgeFinder.EdgeType
EdgeFinder.FeatureList
EdgeFinder.FeatureList.AverageStrength
EdgeFinder.FeatureList.EdgelAngle
EdgeFinder.FeatureList.EdgelMagnitude
EdgeFinder.FeatureList.LabelValue
EdgeFinder.FeatureList.Strength
EdgeFinder.FillGapParameters.Polarity
EdgeFinder.Filters
EdgeFinder.Filters.NearestNeighborRadius
EdgeFinder.FloatMode
EdgeFinder.ImageFeature
EdgeFinder.ImageFeature.Angle
EdgeFinder.ImageFeature.Derivatives
EdgeFinder.ImageFeature.Magnitude
EdgeFinder.ImageFeature.Mask
EdgeFinder.ImageFeature.Source
EdgeFinder.Results
EdgeFinder.Results.AngleImage
EdgeFinder.Results.Count
EdgeFinder.Results.CrossDerivativeImage
EdgeFinder.Results.Draw
EdgeFinder.Results.FirstDerivativeXImage
EdgeFinder.Results.FirstDerivativeYImage
EdgeFinder.Results.Get
EdgeFinder.Results.GetMultiple
EdgeFinder.Results.Item.Draw
EdgeFinder.Results.ItemAtLabel
EdgeFinder.Results.MagnitudeImage
EdgeFinder.Results.SecondDerivativeXImage
EdgeFinder.Results.SecondDerivativeYImage
EdgeFinder.Results.TotalNumberOfEdgels
EdgeFinder.SortingKeys.DecreasePriority
EdgeFinder.SortingKeys.IncreasePriority
EdgeFinder.SortingKeys.Item.Feature
EdgeFinder.Timeout
Image.DataType

Once you have set the type of EdgeFinder control with the EdgeType property, and have calculated the edge features with the Calculate method, you can retrieve the results with the appropriate Results property.

After the initial calculation, you can select edges that meet a specified criterion using the appropriate Filters properties, and post-calculate new features for them, using the CalculateFromResult method. By doing so, you can avoid unnecessarily calculating time-consuming edge features for all edges in a source image.

For optimum performance, the source image on which you make calculations should be 1-band 8-bit unsigned. Other depths and types are generally accepted, but can slightly decrease performance. 3-band source images are also supported, but only when extracting edge contours. When the source image is 32-bit floating point, the EdgeFinder control uses floating-point precision calculations. In all cases, you can force the edge processing operations to use floating-point precision by setting the FloatMode property to True. Note that changing this processing mode can slightly affect the resulting edge map; also, floating-point precision calculations will typically take more time.

Occasionally, an operation can take an unexpectedly long time to calculate. To prevent this, you can use the Timeout property to set a maximum edge extraction and calculation time. By default, there is no time limit.

Sorting keys

The edge results can be sorted in ascending or descending order, by a maximum of three features assigned as sorting keys. To specify the sorting key features, use the SortingKeys.Item.Feature property. You can increase or decrease the priority of a sorting key using the SortingKeys.IncreasePriority and SortingKeys.DecreasePriority methods. Note that features will only be sorted after the edge results are calculated.

You can also post-sort edges by assigning a sorting key to a previously calculated edge feature. Note that, although the post-calculation changes the order in which the edges are sorted, it does not change their label values, which were assigned at the edges's initial calculation. For more information on post-calculation, see the Post-calculation subsection, later in this section.

Retrieving the results

The EdgeFinder control offers several types of results that provide considerable information on the nature of the extracted edges, such as magnitude and Feret values. This information can either be returned for all edges or for a specified edge.

Typically, before retrieving any edge feature result, you should first retrieve the number of edges found in the image to determine the size of the result array needed to hold the results; to do so, use the Results.Count property. Similarly, if you need to retrieve edgel results, such as the edgels' coordinates, magnitude, or angle values, you should also determine the size of the result array needed to hold the result; to do so, use the Results.TotalNumberOfEdgels property.

Edge results are indexed as positive integers typically starting at 1 and, if applicable, are ordered with respect to the sorting keys. You can also label edges by setting the FeatureList.LabelValue property to True (the default setting). The edge's label value is a positive integer greater or equal to 1. Each edge in an image is given a unique label value; that is, unlike an edge's index value, an edge's label value will not change, regardless of future operations.

To accelerate the retrieval of data, the following results in the Results.Get and Results.GetMultiple methods are indexed as positive integers starting at 0: edgeChainIndex, edgeVerticesIndex, edgeVerticesChainIndex, edgeGeneralFeretFirstEdgelIndex, edgeGeneralFeretSecondEdgelIndex, edgeMinimumFeretFirstEdgelIndex, edgeMinimumFeretSecondEdgelIndex, edgeMaximumFeretFirstEdgelIndex, edgeMaximumFeretSecondEdgelIndex. Therefore, index values for these results fall within the range, 0 to (the total number of elements in the collection – 1); to access a specific element in the collection, you must add one to it.

The EdgeFinder control allows you to retrieve results for a particular edge or for all edges. To retrieve results for all edges, use the Results.Get method. You can also use this method to retrieve results for a particular edge by specifying the edge's index. If you want to retrieve results for a particular edge by specifying the edge's label value, use the Results.ItemAtLabel method. Since an edge's index value can change, it is recommended that you specify the label value if you want to retrieve results for a particular edge, especially if you want to perform selection and post-calculation operations.

Note that before an edge feature can be returned, you must first set that feature's property to True, using the appropriate property of the FeatureList property. Every edge feature, except for the label value (set with FeatureList.LabelValue), is set to False by default. For more information on edge features, see the Edge features section.

In general, results are returned in pixels, and coordinates are relative to the center of the top-left pixel in the source image. If you calculate edges using a calibrated source image, results are automatically returned in the output coordinate system specified by the associated calibration control of the source image. However, you can change the output coordinate system to pixel units or world units by setting the OutputCoordinateSystem property of the Calibration control to either calPixel or calWorld to return all results in pixel units or world units, respectively. For more information, see the Calibration chapter.

Note that if your image is calibrated, results are calculated in the real world; therefore, retrieving results in pixel units instead of world units will typically be slower. Also note that, in the presence of distortion, some results are meaningless when converted from real-world to pixel units (for example, the Feret angles). For example, if an edge appears warped in the source image, but the calibration object of the source image compensates for this during the extraction, the resulting Feret angles are meaningful in the real-world coordinate system, and meaningless in the image coordinate system.

Selecting the results

If you do not have many unwanted edges, it is usually faster to simply calculate all required features for all edges. However, calculating many features on a large number of unwanted edges can be unnecessarily time-consuming. To speed up this process, you can use the Filters property to set the required filter, and then use the ApplyFilter to apply the filter; that is, to select the subset of edges for calculation and result retrieval.

Edges that meet the selection criteria can be included, excluded, or deleted permanently from future edge selection operations. Further calculations are subsequently applied to included edges only. Excluded edges can be re-included at any time, while deleted edges are permanently discarded.

Typically, if you have many unwanted edges, you calculate (for all edges) only those features that allow you to distinguish between wanted and unwanted edges. Then, you exclude the unwanted edges from further calculations, and calculate all the required features for the remaining included edges. To arrive at the required set of results, you can make as many calls as necessary to the ApplyFilter and CalculateFromResult methods.

The CalculateFromResult method can only be called after an initial call to the Calculate method. The CalculateFromResult method is used to refine results and is considered a post-calculation. Post-calculating edges can be an effective way of speeding up processing time, especially if advanced features are only calculated on a small subset of selected edges. For more information on post-calculation, see the Post-calculation subsection, later in this section.

Note that, if at any time you calculate edges in a new source image (that is, use the Calculate method), all current results will be discarded and replaced by the newly calculated edges. In addition, all selected features will be recalculated for all edges in the new source image. This means that you will have to restart the selection procedure.

The Filters property can be used to select edges based on one of the following:

  • A calculated edge feature, where the edge selection depends on whether the specified edge feature meets the specified condition.
  • The inter-relationship of edges, where the edge selection depends on whether edges meet the specified box or chain condition of the specified edge or group of edges.
  • The current status of edges, where the edge selection depends on a specific edge, all edges, all included edges, or all excluded edges. The specified edge(s) will be included, included only, excluded, excluded only, or deleted. For example, you can include only the excluded edges, which will essentially swap the previously included and excluded edges.
  • The proximity of an edge or edges to a specified point, where the edge selection depends on the specified radius, and the nearest neighbor condition.

Note that you can also use the Filters property to crop and select a portion of a specified edge. For more information, see the Advanced edge extraction section.

If the edges have been calculated using a calibrated source image, you must specify relevant values in the real world. If the edges have not been calculated using a calibrated source image, you must specify relevant values in pixels. Note that you can change the output coordinate system by setting the Calibration.OutputCoordinateSystem property to calPixel or calWorld.

Edge selection based on edge features

You can include, exclude, or delete edges based on whether the specified edge feature meets the specified condition. The edge feature used to select the edges must have been previously calculated for the included edges.

In this example, all edges with a moment elongation that is less than 0.8 are deleted.

Edge selection based on the inter-relationship of edges

You can include, exclude, or delete edges based on whether edges meet the specified box or chain condition. That is, you can include, exclude, or delete all edges that are inside or outside the bounding box of a specified edge or group of edges; similarly, you can include, exclude, or delete all edges that are inside or outside a specified edge or group of edges.

The inside chain and outside chain conditions only take effect when the edge is closed.

In this example, all edges inside included edges are excluded.

Edge selection based on the proximity of the edges to a point

You can include, exclude, or delete edges based on their proximity to a specified point. You can select either the closest edge to a point, or all the edges in the specified vicinity of a point. Use the Filters.NearestNeighborRadius property to set the maximum radius distance from the point.

In this example, only the edges closest to the point A (Ax, Ay), and within the radius r, are included.

Internal processing images

When saved, the internal processing images are available for result retrieval. These images are used when extracting edges; they include image derivative, angle, magnitude, source and mask images.

Some results can only be post-calculated if the appropriate internal image(s) have been previously saved. For example, you can only post-calculate the edge's strength feature if the internal magnitude image was initially saved. For more information, see the Post-calculation subsection, later in this section.

By using the properties of the ImageFeature property, you can set whether the internal processing images will be saved. By default, all the properties of the ImageFeature property are set to False and therefore, all internal processing images are not saved. You must enable saving them before the first call to the Calculate method. To do so, set the appropriate internal processing image property to True. Saving these images does not affect processing time.

If required, you can access the internal processing images. In general, the EdgeFinder control allows you to optimize your access using the appropriate Results property. For example, to get the identifier of the internal angle image, use the Results.AngleImage property. Note that the internal angle image must have been initially saved.

On certain systems, the internal processing image identifier might not be available, and Nothing (Visual Basic) or NULL (Visual C++) is returned instead. In this case, you can access the image using conventional ActiveMIL techniques.

Internal processing images can hold data that varies in dimension, data type, data depth, and attribute. To get the data stored in the image, call the Results.Get method with a result type in one of the following categories.

  • For the range of the image, use:
    • edgeAngleImageSign, edgeMagnitudeImageSign, edgeFirstDerivativeXImageSign, edgeSecondDerivativeXImageSign, edgeSecondDerivativeYImageSign, edgeCrossDerivativeImageSign.
  • For the number of color bands of the image, use:
    • edgeAngleImageNumberOfBands, edgeMagnitudeImageNumberOfBands, edgeFirstDerivativeXImageNumberOfBands, edgeFirstDerivativeYImageNumberOfBands, edgeSecondDerivativeXImageNumberOfBands, edgeSecondDerivativeYImageNumberOfBands, edgeCrossDerivativeImageNumberOfBands.
  • For the depth (in bits) per band of the image, use:
    • edgeAngleImageSizeBit, edgeMagnitudeImageSizeBit, edgeFirstDerivativeXImageSizeBit, edgeFirstDerivativeYImageSizeBit, edgeSecondDerivativeXImageSizeBit, edgeSecondDerivativeYImageSizeBit, edgeCrossDerivativeImageSizeBit.
  • For the width of the image (in pixels), use:
    • edgeAngleImageSizeX, edgeMagnitudeImageSizeX, edgeFirstDerivativeXImageSizeX, edgeFirstDerivativeYImageSizeX, edgeSecondDerivativeXImageSizeX, edgeSecondDerivativeYImageSizeX, edgeCrossDerivativeImageSizeX.
  • For the height of the image (in pixels), use:
    • edgeAngleImageSizeY, edgeMagnitudeImageSizeY, edgeFirstDerivativeXImageSizeY, edgeFirstDerivativeYImageSizeY, edgeSecondDerivativeXImageSizeY, edgeSecondDerivativeYImageSizeY, edgeCrossDerivativeImageSizeY.
  • For the data type and depth of the image, use:
    • edgeAngleImageType, edgeMagnitudeImageType, edgeFirstDerivativeXImageType, edgeFirstDerivativeYImageType, edgeSecondDerivativeXImageType, edgeSecondDerivativeYImageType, edgeCrossDerivativeImageType.

Derivatives

You can save the internal derivative images used when extracting edges. To save the images, set the ImageFeature.Derivatives property to True.

To access the images, use the following properties:

  • Results.CrossDerivativeImage
  • Results.FirstDerivativeXImage
  • Results.FirstDerivativeYImage
  • Results.SecondDerivativeXImage
  • Results.SecondDerivativeYImage

When extracting object contours, the source image's first derivatives in both the X- and Y- directions are used. When extracting line crests, the source image's second derivatives in both the X- and Y-directions, as well as the image's cross derivative, are used.

Typically, you will let the EdgeFinder control manage its own derivative images to perform the edge extraction. However, you can specify your own derivative images, so that the edge extraction is done with them instead. For more information on derivative images, and how they are used to extract edges, see the Providing the image's derivatives subsection in the Advanced edge extraction section.

Angles

You can save the internal angle image used when extracting edges with the ImageFeature.Angle property. To access the angle image, use the Results.AngleImage property. You can also save the angle value of the edge at each edgel position with the FeatureList.EdgelAngle property. Angle values refer to the angle between the horizontal axis and the edge's perpendicular direction at each edgel location.

If you save the edgel angle, the angle values of the extracted edges are saved. If you save the angle image, the angle values of all the edges in the source image are saved; in this case, the only relevant angle values are those whose edgels have a magnitude that is above the lower bound threshold value.

In ActiveMIL, angles are returned, counter-clockwise, from 0 degrees to 360 degrees and mapped in the range of 0 to 255.

Note that when extracting edge contours from a color image, the angle of the polarity cannot be obtained. For example, it is impossible to determine if red is darker or lighter than green; therefore, it is impossible to ascertain the direction of the transition.

Magnitude

You can save the internal magnitude image used when extracting edges with the ImageFeature.Magnitude property. To access the magnitude image, use the Results.MagnitudeImage property. You can also save the magnitude value of the edge at each edgel position with the FeatureList.EdgelMagnitude property.

For object contours, the magnitude is the norm of the gradient vector at the edgel position. For line crests, the magnitude is equal to the maximum eigenvalue of the Hessian matrix at the edgel position.

If you save the edgel magnitude, the magnitude values of the extracted edges are saved. If you save the magnitude image, the magnitude values of all the edges in the image are saved.

For more information on magnitude and how it is used to extract edges, see the Filter type subsection in the Customizing the edge extraction settings section.

Source image and mask

You can save both the source image, and the image used to mask the source image. To do so, use the ImageFeature.Source and ImageFeature.Mask properties, respectively. For more information on masking, see the Masking the edges subsection in the Advanced edge extraction section.

Post-calculation

Any calculation made to refine results with the CalculateFromResult method is considered a post-calculation. Note that this method should only be called after you make an initial calculation with the Calculate method. As mentioned, to avoid repeatedly calculating time-consuming edge features for all edges in a source image, you can perform post-calculations on included edges of an edge selection filtering. In this case, processing time greatly decreases since you do not have to calculate time-consuming edge features for all edges. The operation of selecting edges and adding new features can be repeated until the required result is calculated.

The CalculateFromResult method performs post-calculations on edge results, while the Calculate method performs calculations on a source image. Whenever you use the Calculate method, all current results will be discarded and replaced by the results of the newly calculated edges. In addition, all selected features will be recalculated for all edges in the new source image.

Restrictions

Typically, anything that can be calculated can also be post-calculated. However, some post-calculations are not valid unless certain values have been initially saved.

The ImageFeature.Magnitude property must have been initially set to True if you want to post-calculate the following:

  • The average strength feature, which you can select for calculation with the FeatureList.AverageStrength property.
  • The strength feature, which you can select for calculation with the FeatureList.Strength property.
  • The magnitude value of the edge at each edgel position, which you can set for saving with the FeatureList.EdgelMagnitude property.

Note that, if any of the values above were calculated, and you intend to fill edge gaps during post-calculation, the ImageFeature.Magnitude property must have been initially set to True.

The ImageFeature.Angle property must have been initially set to True if you want to post-calculate the following:

  • The angle value of the edge at each edgel position, which you can set for saving with the FeatureList.EdgelAngle property.
  • The edgeSame and edgeReverse polarity settings, which you set with the FeatureList.FillGapParameters.Polarity property.

Note that, if any of the values above were calculated, and you intend to fill edge gaps during post-calculation, the ImageFeature.Angle property must have been initially set to True.

Thresholding can also be applied during post-calculation. In this case, all internal processing images must have been initially saved, by setting the properties of the ImageFeature property to True. For more information, see the Thresholding subsection, in the Customizing the edge extraction settings section.

Annotating the results

The EdgeFinder control offers many drawing operations for annotating your results, using the Results.Draw or Results.Item.Draw methods. You can, for example, draw a zoomed region of the edge map that was used to calculate results by setting the appropriate values for the DrawingParameters.RelativeOriginX, DrawingParameters.RelativeOriginY, DrawingParameters.ScaleX, DrawingParameters.ScaleY properties. The relative origin values must be specified in pixels, and are relative to the coordinates of the top-left corner of the region in the source image, while the scale values specify the X- and Y-scaling factors used to fill the destination image. For example:

You can also draw a bounding box around each edge, a cross at each edge's center of gravity, or draw a cross at each edgel. In addition, you can draw the drawing operation's numerical value with the following settings:

  • edgeDrawCenterOfGravityWithValue
  • edgeDrawPositionWithValue
  • edgeDrawMaximumFeretWithValue
  • edgeDrawMinimumFeretWithValue
  • edgeDrawGeneralFeretWithValue

For example, if you use edgeDrawCenterOfGravityWithValue, the center of gravity's coordinates are drawn within parenthesis, and centred above the drawing cross.

You can either use a previously allocated graphics context to control the drawing color, or you can use the default graphics context. You can either draw directly into the image, or annotate the image non-destructively by drawing into its display's overlay image. For more information on ActiveMIL graphics, see the Graphics chapter. For more information on non-destructive annotation, see the Annotating the displayed image non-destructively section in the Display chapter.

Note that if the edges are calculated using a calibrated source image, the EdgeFinder control takes the calibration into account; that is, drawings are done in the real world (using pixel units).

For a complete list and explanation of each drawing operation, refer to the Draw, Results.Draw or Results.Item.Draw methods. in the ActiveMIL Reference.


Advanced edge extraction

User Guide:
Customizing the extraction settings
Defining and adding models to your Model Finder control
How edges are calculated
Operating on a region of interest
Thresholding

Reference:
EdgeFinder.ApplyFilter
EdgeFinder.ApproximationTolerance
EdgeFinder.Calculate
EdgeFinder.CalculateFromResult
EdgeFinder.ChainApproximation
EdgeFinder.ExtractionFilter.Smoothness
EdgeFinder.Filters.Item
EdgeFinder.Filters.Item.Condition
EdgeFinder.Filters.Item.Feature
EdgeFinder.Filters.Item.FirstValue
EdgeFinder.Filters.Item.SecondValue
EdgeFinder.ImageFeature.Angle
EdgeFinder.Mask
EdgeFinder.NearestNeighbors
EdgeFinder.NearestNeighbors.Angle
EdgeFinder.NearestNeighbors.AngleTolerance
EdgeFinder.NearestNeighbors.EdgelType
EdgeFinder.NearestNeighbors.GetNeighbors
EdgeFinder.NearestNeighbors.GetNeighborsInUserArrays
EdgeFinder.NearestNeighbors.MaximumNumberOfEdgels
EdgeFinder.NearestNeighbors.MaximumSearchRadius
EdgeFinder.NearestNeighbors.MinimumSearchRadius
EdgeFinder.NearestNeighbors.MinimumSpacing
EdgeFinder.NearestNeighbors.SearchAngle
EdgeFinder.NearestNeighbors.SearchAngleSign
EdgeFinder.NearestNeighbors.SearchAngleTolerance
EdgeFinder.NearestNeighbors.SourcePoints.Add
EdgeFinder.NearestNeighbors.SourcePoints.AddMultiple
EdgeFinder.Results.Item.Vertices.Item.ChainIndex
EdgeFinder.Results.Item.Vertices.Item.EdgelIndex
EdgeFinder.Results.Item.Vertices.Item.X
EdgeFinder.Results.Item.Vertices.Item.Y
EdgeFinder.SourceDerivativeImages
EdgeFinder.Threshold.ThresholdType
Image
Image.Clear

In addition to the fundamental edge extraction settings, the EdgeFinder control also provides advanced settings that allow you to write customized applications. Edges can be approximated, masked, cropped, and extracted using advanced thresholding properties. You can provide your own derivative images, and you can put data from user-supplied arrays into the EdgeFinder control results. You can also get edgels, from the EdgeFinder control results, which are the closest neighbors to a list of user-specified edgel coordinates.

Approximating the edges

If you are interested in a basic, geometric representation of the edge map, you can approximate your edges by specifying a polygonal segmentation. To do so, set the ChainApproximation property to edgeLine. The default setting is edgeDisable.

When the ChainApproximation property is set to edgeLine, each edge is piecewise approximated using a connected series of line segments, commonly called polylines. The polylines are defined by the set of automatically calculated vertex coordinates, which determine the location of the line segment ending points. You can use the Results.Item.Vertices.Item.X and Results.Item.Vertices.Item.Y properties to retrieve the X and Y vertex coordinates for a group of edges or for a particular edge. You can also get the index of each vertices' corresponding edge or edgel, using the Results.Item.Vertices.Item.ChainIndex and Results.Item.Vertices.Item.EdgelIndex properties, respectively.

The resolution of the edge approximation can be adjusted by setting the ApproximationTolerance property. The range of this property varies from 0.0 to 100.0. When set to 0.0, a very fine approximation of the edges is performed. When set to 100.0, a coarse approximation of the edges is performed. The default value is 50.0.

Note that when the ChainApproximation property is set to edgeLine, both the edge map and the edge approximation is calculated. When the ChainApproximation property is set to edgeDisable, only the edge map is calculated.

Masking the edges

The Mask method allows you to apply a mask to the source image in the EdgeFinder control context. A mask is a binary image used to define irrelevant, inconsistent, or featureless areas in the source image. As a result, when a mask is set, subsequent calculations will only extract edges in the source image's unmasked regions.

You must use the Image control to specify the image to use as the mask. A masked pixel in the source image corresponds to a non-zero value in the mask image. Mask images are useful for creating non-rectangular regions of interest (ROIs). For rectangular ROIs, it is more effective to set a mask with child buffers. For more information, see the Operating on a region of interest section in the Images chapter.

The Mask method allows you to set a mask for an EdgeFinder control context so that edges are extracted only from the unmasked regions of the source image. This allows you to prevent any irrelevant, inconsistent, or featureless areas of your image from being extracted.

A masked pixel corresponds to a non-zero value in the mask image. Mask images are useful to automatically generate non-rectangular regions of interest (ROIs). For rectangular ROIs, using child images is the preferred method.

You must use the Image control to specify the image to use as the mask. If necessary, you can clear the mask image using the Clear method of the Image control.

Cropping the edges

Selecting an edge or a group of edges for calculation and result retrieval is typically sufficient for the majority of cases. However, in some advanced applications, you might only be interested in one part of a single edge. In this case, you can use some properties of the Filters.Item property.

Cropping an edge essentially means splitting one edge into two, thereby allowing you to choose only one of these edges to be included, excluded, or deleted from the edge results. The selected portion receives a new label value, while the other portion keeps the original label value.

You can make as many cropping operations as necessary to arrive at the required set of results. Note that all calculated edge features are updated after each cropping operation.

To split an edge into two edges, you must specify the following:

  • The edge to split. This is identified by the edge's label value. To do so, set the Filters.Item.Feature property to edgeCropChainAtLabel.
  • The point at which to split the edge. This is identified by the edgel's index value. To do so, use the Filters.Item.SecondValue property.
  • The portion of the edge to select. This is identified by all edgels in the specified edge with index values that are greater than, less than, greater than or equal to, or less than or equal to, the point at which the edge was split. To do so, use the Filters.Item.Condition property.

    • To specify the new edge's label, use the Filters.Item.FirstValue property.

In this example, the edge labeled 5 is split into two edges at edgel 25. One edge consists of all edgels ranging from the start of edge 5 to the 24th edgel; this edge keeps the label 5 and is not selected for exclusion. The other edge consists of all edgels ranging from the 25th edgel to the end of edge 5; this edge gets a new label value and is selected for exclusion.

Advanced thresholding

The EdgeFinder control uses an hysteresis thresholding operation to perform the edge extraction. That is, edge chains are extracted such that the magnitude values of all connected edgels are stronger than the lower bound threshold value, and at least one edgel in each edge chain has a magnitude that is stronger than the upper bound threshold value. For more information on thresholding, and on setting these bounds, see the Thresholding subsection, in the Customizing the extraction settings section.

The EdgeFinder control also allows you to customize the behavior of the hysteresis thresholding using the Threshold.ThresholdType property. That is, this property allows you to set how the upper and lower bound threshold values will be used (as opposed to setting the bounds themselves).

If you set the Threshold.ThresholdType property to edgeHysteresis, both the lower bound threshold value and the upper bound threshold value will be used. This is the default setting.

If you set the Threshold.ThresholdType property to edgeNoHysteresis, the lower bound threshold value is ignored, and considered to be equal to the upper bound threshold value.

If you set the Threshold.ThresholdType property to edgeFullHysteresis, only the upper bound threshold value is used; the lower bound threshold value is ignored, and considered to be 0. In this case, the extracted edges are sets of connected edgels such that the magnitude of at least one edgel in each edge is stronger than the upper bound threshold value. This type of thresholding can be used to extract edges in noise-free highly multi-contrast images.

Providing the image's derivatives

Typically, you will let the EdgeFinder control manage its own derivative images to perform the edge extraction. However, you can specify your own derivative images, so that the edge extraction is done with them instead. In this case, before calling the Calculate method, use the properties of the SourceDerivativeImages property to specify the derivative images.

Note that, when specifying your own derivative images, the majority of the processing settings remain in effect, except for filter settings (for example, the ExtractionFilter.Smoothness property).

Derivative images must be 1-band 16-bit signed or 1-band 32-bit floating-point. In addition, all derivative images must be consistent (for example, they must have the same scaling factor), and they must be of the same type and size. When the derivative images are 32-bit floating-point, the EdgeFinder control uses floating-point precision calculations.

When extracting object contours, you must provide the image's first derivatives in both the X- and Y-directions. Note that the image derivatives must be consistent with the image axis; that is, if the intensity transition in the source image increases (from black to white) with the axis directions, the associated derivative values must be positive. Similarly, if the intensity transition in the source image decreases (from white to black) with the axis directions, the associated derivative values must be negative.

When extracting line crests, you must provide the image's second derivatives in both the X- and Y-direction, as well as the image's cross derivative. Note that the image derivatives must be consistent with the image axis; that is, a light line crest in the source image must result in negative second derivative values. Similarly, a dark line crest in the source image must result in positive second derivative values.

Putting data into the EdgeFinder results

Typically, the EdgeFinder control results will hold edge extraction information after an edge selection and calculation. However, you can also copy data from user-supplied arrays into the EdgeFinder control results, using the PutEdgeResults method. This can be useful if you want to construct an EdgeFinder control result that cannot be obtained from one image. You can therefore combine results from multiple EdgeFinder control results to, for example, build a complete model to add to a ModelFinder control. For more information, see the Defining and adding models to your ModelFinder control section, in the Geometric Model Finder chapter.

Note that once edges are added to the EdgeFinder control results, their features can be calculated with the CalculateFromResult method.

To add data to the EdgeFinder control results, you must use the PutEdgeResults method to specify the following user-supplied arrays:

  • The edgel indices.
  • The X-coordinate(s) of the edgel(s).
  • The Y-coordinate(s) of the edgel(s).

If necessary, you can also provide arrays containing the orientation of the edgel(s) and the magnitude of the edgel(s).

The information that you provide about the edges to add to the EdgeFinder control results must be ordered on an edge by edge basis. For example, the following table illustrates how to supply X- and Y-coordinates for two distinct edges:

X-coordinate array | Y-coordinate array | Index array
1 | 1 | 1
11 | 11 | 1
21 | 21 | 1
2 | 2 | 2
12 | 12 | 2
22 | 22 | 2

Finding the closest edgels to a list of points

From the EdgeFinder control results, you can retrieve the coordinates of edgels that correspond to the closest neighbors from a list of user-specified source point coordinates. You must use either the NearestNeighbors.SourcePoints.Add or NearestNeighbors.SourcePoints.AddMultiple methods to specify your input data, which must include the source points and, if necessary, a corresponding angle for each point. Note that the Add method allows you to add one new source point to the collection of source points while the AddMultiple method allows you to add many at one time.

To find the closest edgels, you must use either the NearestNeighbors.GetNeighbors or NearestNeighbors.GetNeighborsInUserArrays methods. The NearestNeighbors.GetNeighbors method copies the data to the collection of nearest neighbors. To retrieve the closest edgels, you must use the NearestNeighbors.SourcePoints.Item.Neighbors.Item property. The NearestNeighbors.GetNeighborsInUserArrays method copies the data to the specified user arrays. When using this method, the closest edgels are not accessible with the NearestNeighbors.SourcePoints.Item.Neighbors.Item property; they are only accessible with the user arrays.

If you change the EdgeFinder control results by adding or removing source points (for example, by calling the Calculate or ApplyFilter method), the nearest neighbors contained in the NearestNeighbors.SourcePoints.Item.Neighbors collection are no longer synchronized with those results. You must therefore call the NearestNeighbors.GetNeighbors or NearestNeighbors.GetNeighborsInUserArrays method to find the nearest neighbors in the current set of results. Note that the NearestNeighbors.IsNeighborsPresent property returns whether the current neighbors in the collection are synchronized with the current results.

You can also set a series of constraints to which potential results (edgel candidates) must adhere before being returned. Only edgels that meet all constraints (if any are set) can be returned as the closest neighbors. To set the constraint(s), use the properties of the NearestNeighbors property.

You can either set simple constraints, advanced constraints, or both. Simple constraints are based on the location of the edgel candidate (the ROI), while advanced constraints are typically based on the inner qualities of the edgel candidate (the gradient angle).

When applying constraints, you should first set the ones based on location. This will establish the EdgeFinder control results, and generate a group of edgel candidates. If you do not apply any edgel-based constraints, then the candidates in this region that are closest to the source points will be returned. However, if you set advanced constraints, then only the edgels in this region that also adhere to these constraints can be returned.

Advanced constraints are typically based on the source edgel's gradient angle. In this case, you must provide an angle value for each source point as part of your input data for either the Add or AddMultiple methods. Therefore, the candidate edgel will only be returned if its gradient angle adheres to the constraint applied with the source angle. Note that an edgel's gradient angle is perpendicular to the edge, and is dependent on the edge extraction process. For more information on an edgel's gradient angle, see the How edges are calculated subsection in the Extracting the edges section.

Location-based constraints

You can use location-based constraints to limit the area within the EdgeFinder control results from which edgels matching source points can be found. All edgels that do not fall within this area will be ignored. By default, the search region encompasses all edgels within the EdgeFinder control results.

The following NearestNeighbors properties can be used to specify location-based constraints:

  • SearchAngle
  • SearchAngleSign
  • SearchAngleTolerance
  • MaximumSearchRadius
  • MinimumSearchRadius

The MaximumSearchRadius and MinimumSearchRadius properties set the maximum and minimum distance radii in the EdgeFinder control results that will be searched for the closest edgels. These values must be set in pixels. You must use either the Add or AddMultiple methods to specify the source point from which the radii are measured.

Together, the MaximumSearchRadius and MinimumSearchRadius properties define a ring-like region, in the EdgeFinder control results, that will be used to find the closest edgels. All edgels that fall outside this ring will be ignored. By default, the minimum distance radius is 0 pixels, and the maximum distance radius is an infinite number of pixels (edgeInfinite); that is, by default, the entire contents of the EdgeFinder control results will be searched. Note that setting the MaximumSearchRadius property to edgeInfinite will retrieve all the closest edgels (one edgel per edge).

The EdgeFinder control result region that will be searched for the closest edgels can be further constrained to an angular sector within the area defined by the MaximumSearchRadius and MinimumSearchRadius properties. You can use the SearchAngle property to define the search angle, and the SearchAngleTolerance property to define the tolerance. The search angle is relative to the source angle set with either the Add or AddMultiple methods, and the tolerance is applied to the resulting angle evenly; that is, half of the tolerance angle is applied in the positive direction, and half is applied in the negative direction. All edgels that are located within this angular sector (tolerance) can be returned. Any value between 0.0° to 360.0° can be set for both the search angle and tolerance. By default, the SearchAngle property is set to 0.0° and the SearchAngleTolerance property is set to 360.0 degrees; therefore, all edgels will be considered.

For example, if you set the SearchAngle property to 30.0°, the SearchAngleTolerance property to 30.0°, and the source angle to 15°, only edgels that fall between 30.0° and 60.0° will be considered.

You can also set the sign of the angle that will be used to find the closest edgels with the SearchAngleSign property. The sign of the angle can be the same as (edgeSearchAngleSignSame) or the reverse of (edgeSearchAngleSignReverse), the source angle set with either the Add or AddMultiple methods. The default setting for the SearchAngleSign property is edgeSearchAngleSignSame. The SearchAngleSign property can also be set to edgeSearchAngleSignSameOrReverse, which means that the sign of the search angle can be the same as, or the reverse of, the source angle.

For example, if you set the source angle to 15°, and the search angle and tolerance to 30.0°, then edgels that fall within an angle that has the same sign would be located between 30.0° and 60.0°, edgels that fall within an angle that has a reverse sign would be located between 210° and 240°, and edgels that fall within an angle that has any sign would be located at either between 30.0° and 60.0°, or between 210° and 240°.

Note that, if necessary, you can allow a point between two edgels to be returned. To do so, set the NearestNeighbors.EdgelType property to edgeSubEdgels. In this case, an edgel candidate is returned, as long as an edge (and not necessarily an edgel) is located within the search region.

Edgel-based constraints

As mentioned, location-based constraints limit the region in the EdgeFinder control results that will be searched for the closest edgels. Edgel-based constraints, on the other hand, are applied to the edgels themselves. Only edgels that adhere to these constraints can be returned. Note that, if you have applied location-based constraints, only edgels within the defined search region will be considered.

The following NearestNeighbors properties can be used to specify edgel-based constraints:

  • MaximumNumberOfEdgels
  • MinimumSpacing
  • Angle
  • AngleTolerance

The MaximumNumberOfEdgels property sets the maximum number of closest edgels (that is, the N closest edgels) that can be returned, for each source point. The default maximum number is 1; therefore, the edgel closest to each source point is returned.

The MinimumSpacing property sets the minimum distance separating closest edgel candidates within the same edge. Therefore, edgels that are too close together will not be returned. The spacing is set in number of edgels and must be greater than or equal to 1. The default minimum spacing setting is infinite (edgeInfinite), which means that there is no minimum distance (number of edgels) that must separate edgel candidates.

The Angle property sets the gradient angle that an edgel must have, before being considered a candidate. The Angle property setting is based on a source angle (for each source point), set with either the Add or AddMultiple methods. The gradient angle of the edgel candidate can be the same (edgeAngleSame), the reverse (edgeAngleReverse), or the same or reverse (edgeAngleSameOrReverse) as the angle of the source point. The Angle property can also be set to any angle (edgeAngleAny); in this case, the source angle and the source point are ignored. The default is edgeAngleAny. Note that the Angle property is only valid if the ImageFeature.Angle property was set to True.

You can also set an angular tolerance for the Angle property setting with the AngleTolerance property. In this case, candidate edgels that have a gradient angle that falls within the specified angular range (tolerance) can be returned. The tolerance is applied to the Angle property setting evenly; that is, half of the tolerance angle is applied in the positive direction, and half is applied in the negative direction. Any angle between 0.0° to 360.0° can be set as a tolerance. By default, the AngleTolerance property is set to 180.0°; therefore, only candidate edgels that have a gradient angle that is the same as the source angle will be considered. Note that setting the AngleTolerance property to 360.0° is equivalent to setting the Angle property to edgeAngleAny; in either case, edgels with any gradient angle are considered.

For example, if you set the angle of the source point to 45°, the Angle property to edgeAngleSameOrReverse, and the AngleTolerance property to 90°, only edgels that have an angle between 0° and 90° or between 180° and 270° can be returned. In the following example, edgels 1 and 3 are considered the closest edgel candidates.


Interfacing with the Geometric Model Finder module

See Also | Previous
User Guide:
MIL Geometric Model Finder

Reference:
EdgeFinder
EdgeFinder.Calculate
EdgeFinder.DetailLevel
EdgeFinder.Filters.Item
EdgeFinder.ModelFinderCompatible
EdgeFinder.Threshold.Mode
ModelFinder
ModelFinder.ModelFinderType
ModelFinder.Models.AddEdgeModel

An EdgeFinder control can also be used with a ModelFinder control; that is, you can define models from the result of an edge extraction, or you can find model occurrences in the result of an edge extraction.

If you want to use the ModelFinder control, and are also familiar with the EdgeFinder control, you can perform specialized edge selections, which can reduce the unnecessary geometric complexity in both the model and/or the target image. For example, if you want to find model occurrences in the result of an edge extraction, you can use the properties of the Filters.Item property to select which edges to include in the results, based on one of several criteria. As a result, the ModelFinder control will have much less information to analyze, thereby potentially speeding up the search for the model(s).

Models and target images should use the same geometric primitives (object contours or line crests). Therefore, if the target edges are object contours, then the model edges must also be object contours. Similarly, if the target edges are line crests, then the model edges must also be line crests.

If you expect to use the results of an edge extraction with the ModelFinder control, you must set the ModelFinderCompatible property of the EdgeFinder control to True before the initial call to the Calculate method. To add models defined from the result of an edge extraction to the collection of models, use the Models.AddEdgeModel method of the ModelFinder control.

To set the sensitivity with which edgels are detected, you can use either the Threshold.Mode or DetailLevel property of the EdgeFinder control. The DetailLevel property was provided to help users already familiar with the ModelFinder control and should only be used if you are interfacing with it. Note that a DetailLevel property setting overrides a Threshold.Mode property setting.

The following is an example of a typical EdgeFinder control/ ModelFinder control application:

For more information on the ModelFinder control, see the MIL Geometric Model Finder chapter.


Optical character recognition

Chapter 10: Optical character recognition

This chapter presents the features of the optical CharacterRecognition control.

  • Optical character recognition - overview
  • Steps to reading or verifying a string in an image
  • Basic concepts
  • Guidelines for choosing the appropriate search algorithm
    • General search algorithm
    • Constrained search algorithm
    • Switching between the two
    • Deciding which search algorithm to use
  • Fonts
    • User-defined character recognition fonts
      • Run-time
      • Design-time
    • Existing character recognition fonts
    • Using a Semi font
    • Quality and scale are important
    • Visualizing
    • Erasing characters
  • Defining the target strings
    • Calibrating your font
      • Automatic calibration
      • Manual calibration
    • Setting appropriate processing properties
      • Blanks
      • Broken characters
      • Morphologicial filtering
      • Thickness and dots
      • Touching characters
    • Specifying other string information
      • Number of strings
      • String lengths
      • Angle
      • Positional variation
    • Constraints
  • Locating your text
  • Determining what is a match
    • Acceptance levels
    • Unrecognized characters
  • Retrieving and analyzing the results
    • A character
    • A string
    • A text
    • Retrieving and analyzing the results
    • Validating String Events
  • Improving search speed
  • A typical application


Optical character recognition - overview

See Also | Next
User Guide:
Images - overview

Reference:
CharacterRecognition
CharacterRecognition.StringType
Image
PatternMatching

Many types of industries require the analysis of character strings in images. For example, the semiconductor industry requires serial numbers printed on wafers to be read for tracking purposes. The pharmaceutical industry requires analysis of medicine bottle labels to ensure, for example, that expiry dates are properly printed.

The ActiveMIL CharacterRecognition control provides a powerful and easy to use set of methods for reading and verifying character strings in 8-bit unsigned grayscale images, providing results such as quality (match) scores and validity flags. The CharacterRecognition control can read and verify mechanically generated, character strings of known lengths. This control is especially designed to operate on character strings in degraded images, with up to ± 90° of rotation in the target string. The CharacterRecognition control can also be used in conjunction with other ActiveMIL controls to develop hardware independent optical character recognition (OCR) programs for machine vision applications.

The CharacterRecognition control offers the choice of loading a set of grayscale character representations from a MIL font file, or gathering the needed grayscale character representations from image buffers and/or other files to create a character recognition font. Each character in the string to be read (target string) is compared to each character representation in this font. The representation with the closest match is chosen and its ASCII value is returned. You can adjust the value at which this match is considered a success by setting the acceptance level. A read operation yields a string of ASCII characters, a confidence score for each character, and its position in the target string, as well as a confidence score for the whole string. A verify operation also provides a validity flag for each character.

For applications requiring custom font types, the CharacterRecognition control supports the creation of custom fonts that can be saved to disk and loaded as needed. Two predefined character recognition font files are provided to read and verify semiconductor wafer serial numbers of standard SEMI font character types.

To perform optical character recognition, you need a CharacterRecognition control, a System control, and an Image control, to use as the target image to search for the string.

The control also supports user-specified character constraints, processing controls, automatic font calibration and reading texts that span multiple lines. To ensure recognition accuracy, checksum calculations are performed when analyzing standard SEMI font character strings. When using a custom font type, user-defined functions can validate the read character string automatically (for example, using custom checksum functions).

For an example that uses the CharacterRecognition control, see MOcr.


Steps to reading or verifying a string in an image

User Guide:
Image quality

Reference:
CharacterRecognition.Constraints
CharacterRecognition.Font.ImportGrayscale
CharacterRecognition.Load
CharacterRecognition.ReadString
CharacterRecognition.Speed
CharacterRecognition.TargetCharacter
CharacterRecognition.TargetCharacter.AcceptanceThreshold
CharacterRecognition.VerifyString

The basic steps to read or verify a string in an image are:

  1. Load an existing CharacterRecognition control from disk using the Load method or create a custom font using the Font.ImportGrayscale method to add character representations to the CharacterRecognition control.
  2. If necessary, specify the type of characters (alphabetic, numeric, or other) that should appear at specific positions in the string, using the constraints property.

    3. With custom fonts, you can use the ValidateString event. After checking character constraints on a found string, the user-defined function in the ValidateString event.will be executed. If using a CharacterRecognition control for a semi font, checksum calculations are performed automatically.

  3. If necessary, adjust various settings to optimize your application, such as the speed of the search (using the Speed property) and the acceptance level required for a valid match (using the TargetCharacter.AcceptanceThreshold property)
  4. Calibrate the CharacterRecognition control automatically to match the target image's character size and spacing using the CalibrateFont method. Alternatively, you can modify these values using the TargetCharacter properties.

    5. To physically modify the polarity and sizing of the font of a CharacterRecognition control, use the Invert or the Resize methods.

  5. Read or verify the string in the target image using the ReadString or the VerifyString methods, respectively. These methods are performed according to the defined character constraints and set CharacterRecognition properties.
  6. Obtain the CharacterRecognition results using the Results properties.

In general, the first five steps are performed once, while steps 6 to 9 are repeated as required. Since you can save the calibration results, character constraints, and other property settings with the character representations (using the Save method) steps 1 to 5 can be replaced by a single step which loads a CharacterRecognition control from disk.

Before performing a read or verify, if any font property or constraint changes, the CharacterRecognition control should be preprocessed (using the Preprocess method). Use the IsPreprocessed property to determine if the CharacterRecongition control requires preprocessing.


Basic concepts

Previous | Next

The basic concepts and vocabulary conventions for the CharacterRecognition contol are:

  • Acceptance level. The user-defined level against which the match score is compared to determine if the match is valid.
  • Character representations. The grayscale bitmap (image) or ASCII representation (drawing) of a character used by the CharacterRecognition control.
  • Entire text. One or many strings contained within the target image.
  • Fixed size font. A series of character representations that each has the same character y size and character x size. Characters of a fixed size font tend to be all upper-case.
  • Match score. The result of a read/verify operation that shows the level of correlation between the character representations in the character recognition font and the characters found within the image, taking into account character constraints.
  • Character recognition font. The character recognition font contains the character representations.
  • CharacterRecognition control. The CharacterRecognition control is a container for the character recognition font information, target image character size and spacing, constraints, and properties.
  • Target string(s). The text to be found within the target image. This can be as little as a single character or multiple lines of characters.


Guidelines for choosing the appropriate search algorithm

Previous | Next

A CharacterRecognition control specifies character font information, target image character size and spacing, constraints, and processing information. There are two different search algorithms that the CharacterRecognition control can use: general and constrained.

General search algorithm

The general search algorithm requires less information about the target string but works best on clean target images. If you need to specify the location of the string to eliminate erroneous results, use a child image to create a smaller region in which to search.

The target image should have a clearly visible threshold (binarization) between the characters and their background. The threshold must preserve the shape of the characters. Broken and/or touching characters can degrade the results.

When using the general search algorithm, the cleaner the background of a target image, the better the results of a search. The backgrounds should not have blobs of similar size and intensity as the characters.

The settings of the following properties are only meaningful when using the general search algorithm:

  • Enabling blank character recognition (using the EnableBlankCharacters property).
  • Finding the string length automatically (by setting the StringLengthToRead parameter of the ReadString method to ocrAny).
  • Finding the character width automatically (by setting the TargetCharacter.SizeX property to ocrSame) when using a fixed size font.
  • Finding the character height automatically (by setting the TargetCharacter.SizeX property to ocrSame) when using a fixed size font.
  • Finding the inter-character spacing automatically (by setting the TargetCharacter.Spacing property to ocrSame when the inter-character spacing is the same between characters or to ocrAny if it differs between characters).

An example image best suited to using the general search algorithm is:

Use the constrained search algorithm if your image is greatly degraded or if you know a lot about the location of your target string(s).

Constrained search algorithm

The constrained search algorithm works well with degraded target images. This type of context requires more information about the target string, but provides a more robust search. The constrained search algorithm is best used when all the details are knows about the target string. Especially, the more known about the target string's location and the size and spacing of its characters, the better the results of a search. Since more precise information is required to calibrate your font with that of the target string, you can automatically calibrate the font with the target string using the CalibrateFont method. Note that the automatic calibration of a font can only be done when using the constrained search algorithm.

An example image best suited to using the constrained search algorithm is:

Use the general search algorithm if your image contains a string comprised of unevenly spaced characters or a string that has blanks or broken characters.

Switching between the two

Determining good CharacterRecognition control settings takes time. Modifying an existing CharacterRecognition control is faster than creating a new CharacterRecognition control from scratch. When dealing with two sets of images, for example, with similar character recognition font requirements with regards to character representation, but differing in some other aspect (for example, sizing, spacing, and/or quality), it might be faster to switch search algorithms, make the necessary settings adjustments, and then read the new images. You can switch the type of search algorithm using the ConvertOCRType method.

For example, after reading a series of images with an evenly spaced font (depicted below), to read a group of images where the font is not equally spaced, requires a switch from the constrained to the general search algorithm.

Note, when you switch between search algorithms, any unsupported setting is reset to its default.

Deciding which search algorithm to use

The following decision tree should help you decide when you should use the general or the constrained search algorithm. Note that, if a question does not apply to your situation, skip to the next question in the list.

  1. Can the target image be thresholded? That is, can a clear distinction be made between the background of the target image and the characters to be read?
    • Yes. Use the general search algorithm.
    • No. Use the constrained search algorithm.
  2. Does the target image have intensity problems; that is, does it have reflections, or uneven lighting, or is it so heavily degraded that bits of the string are difficult to read with the naked eye?
    • Yes. Use the constrained search algorithm.
    • No. Use the general search algorithm.
  3. Is the target string made up of characters that vary in size? Does the target string contain variable spacing?
    • Yes. Use the general search algorithm.
    • No. Use the constrained search algorithm.

      • Note that variable spacing does not include "jitter", which refers to tiny variations in the position of the target characters relative to the expected character position. Jitter can be corrected by calibrating your character recognition font (with the CalibrateFont method) to match the target string.

      Set MinimumSizeX and MinimumSizeY to measure to the clearest edge of the target characters to be read. Set MaximumSizeX and MaximumSizeY to measure to the faintest edge of the target character to be read - just before the shading (if any) matches the background color.

  4. Are any of the characters to be read broken?

    5. When there are broken characters, the best way to determine which search algorithm to use is to try both the general and the constrained search algorithms. In a specific situation, one or the other can produce better results.

    If you are using the general search algorithm, the EnableBrokenCharacters property must be set to True.

  5. Are any of the characters to be read touching?

    6. When there are touching characters, the best way to determine which search algorithm to use is to try both the general and the constrained search algorithms. In a specific situation, one or the other can produce better results.

    If you are using the general search algorithm, the EnableTouchingCharacters property must be set to True.

If your answers were a combination of both the general and the constrained search algorithm, use the search algorithm the most often recommended. You will have to further manipulate your target image or your character recognition font to improve results.


Fonts

Reference:
CharacterRecognition.CalibrateFont
CharacterRecognition.Constraints
CharacterRecognition.Font
CharacterRecognition.Font.Character
CharacterRecognition.Font.ImportGrayscale
CharacterRecognition.Font.NumberOfCharacters
CharacterRecognition.Load
CharacterRecognition.MaximumStringLength
CharacterRecognition.Save
CharacterRecognition.StringType
CharacterRecognition.TargetCharacter

You must specify the character recognition font to read/verify the character strings in target images. Fonts (or typesets) are used to specify the style and size of characters in the images to be read or verified.

A character recognition font contains the following information:

  • The grayscale representations of the characters.
  • Codes identifying each character (ASCII codes for the characters).
  • The number of characters in the character recognition font.
  • Character dimensions.

The above information can be calibrated (using the CalibrateFont method or the properties of the TargetCharacter property), modified (using the Invert and the Resize methods of the Font property), and saved (using the Save method) for later restoration.

User-defined character recognition fonts

Unless using one of the two provided predefined Semi character recognition fonts, you must define a custom character recognition font. You can create a user-defined character recognition font from scratch, or make minor modifications to an existing character recognition font, already saved to disk.

Note that, for best effect, try using the OCRReader utility, or an interactive tool such as Matrox Inspector, to determine font character widths and heights. Interactive graphic tools are also good for creating new characters to add to an existing font.

The following is an example of a character in its character cell and the dimensions that you will have to specify when creating a custom font. Values are to be specified in pixels. Each square in the grid represents one pixel.

The SizeX, SizeY, OffsetX, OffsetY, and Thickness properties of the Font.Character property must comply with the following restrictions:

  • 2 * OffsetX + SizeX <= CellSizeX
  • 2 * OffsetY + SizeY <= CellSizeY
  • CellSizeY, CellSizeX, SizeX, SizeY must be >= 6 pixels and <= 256 pixels.

When copying or importing the character representations from an image, the characters must have the dimensions specified using the properties of the Font.Character property.

When importing the character representations from an ASCII file, font character representations must be presented as follows.

Note that in this format, 'pixels' are delineated by a blank space. So '00' counts as one pixel.

This information breaks down into the following:

Row | Description
01 | Specifies ASCII file format.
02 | Blank row.
03 | Specifies the start of a new character representation and its associated (generally ASCII) character.
04 to 36 | Specifies the alpha-numerical representation of the character.
37 | Blank row.
38 | Specifies the start of a new character representation and its associated (generally ASCII) character.
39 to 71 | Specifies the alpha-numerical representation of the character.
etc. | This pattern is repeated for every character in the font.

Run-time

At run time, to create a custom font:

  1. Grab/create the grayscale character representations of the font in an image and then copy them from the image to the CharacterRecognition control, using the Font.ImportGrayscale method. Alternatively, you can import grayscale character representations from a text file or an image file (for example, a TIFF) into an CharacterRecogntion control using the Load method.

    2. When importing character representations into the CharacterRecognition control, the control must have sufficient space to hold the representations of all the specified characters. You can use the MaximumStringLength property to determine the maximum number of characters that can be stored in the font.

  2. Once copied or imported into a CharacterRecognition control, all of the control settings can be saved on disk using the Save method, and then later restored using the Load method.

For an example that creates a new custom character recognition font, see MOcr.

Design-time

At design-time, you can create a new font to be used by the CharacterRecognition control as follows:

  1. If you are using an existing character recognition font (*.mfo), use the Load button on the General properties page to load it into your CharacterRecognition control, and skip to step 6.
  2. Obtain a good quality 8-bit single-band image containing all of the characters to add to your font. You might need to combine different character representations into one image. This image should have little or no other content than the characters to be read.
  3. Set the various properties on the Attributes and Font property pages according to your requirements.

    4. When importing character representations, the CharacterRecognition control must have sufficient space to hold the representations of all the specified characters. The maximum number of characters that can be stored in the font is set in the Maximum string length field on the Attributes property page.

  4. Click on the Import button. The Open dialog box is presented. Find the 8-bit grayscale image containing your character representations, and click on the Open button.

    5. The characters in your image appear enclosed in character boxes.

    If an error occurs, verify that the file exists, that it isn't of the wrong type (too many bands), and it isn't too large or two small for the various properties set on both the Attributes and the Font property pages.

  5. On the Font property page, add the ASCII codes corresponding to the character representations in your image to the Character list field.
  6. Make all necessary adjustments on the Advanced, Constraints, Target Character, and Regions property pages of the CharacterRecognition control.
  7. When your font is complete, you can save it by clicking on the Save button on the General properties page. Then, later it can be restored using the Load button.

Existing character recognition fonts

Once created, a character recognition font can be saved and restored as needed. Restoring this information (using the Load method) rather than creating the character recognition font from scratch saves time, especially if the restored font requires no further modifications. Note that all the settings the CharacterRecognition control are also restored when using this method.

Using a Semi font

There are two Semi fonts included in your ActiveMIL package: ocrSemiM12_92 and ocrSemiM13_88. These can be used directly or modified to suit your needs.

To use a Semi font directly, load it using the Load method. Once loaded, the font can be modified. For instance, you can import additional characters using the Font.ImportGrayscale method.

To create a new font based on a SEMI font:

  1. Before allocating your CharacterRecognition control:
    • Set the StringType property to either ocrSemiM12_92 or ocrSemiM13_88.
    • Set the MaximumStringLength property to 12 when using ocrSemiM12_92 and 18 when using ocrSemiM13_88.
    • Set the Font.NumberOfCharacters property to 38 (A-Z, 0-9, -, .).
  2. Use Font.ImportGrayscale method to add character representations from an existing Semi font.

Quality and scale are important

Using high-quality character representations will result in the best results. Character recognition processing relies on using the cleanest font characters possible.

When using a general search algorithm, broken characters and spaces, even if expected in the target string, should not be defined in the font. Instead, you should enable the ability to read broken characters using the EnableBrokenCharacters property, and/or enable the ability to read spaces using the EnableTouchingCharacters property.

When using the character recognition font to search for a target string at an angle or if using a general search algorithm, the threshold between the characters and the background must preserve the shape of the characters and have a clearly-visible point of differentiation (binarization).

If the characters in the target image are brighter than the background (for example, white on black), then the character representations included in your font must also be of characters that are brighter than the background. The foreground is specified at the font's creation (Allocate method) and can be changed later using the Font.Invert method. This changes both the character representations and the setting of the Allocate method within the CharacterRecognition control.

If the size of the character representations in the font is not the same as those in the target string, you can calibrate the font (discussed later). Alternatively, when the physical size of the character representations differ from those in the target image, changing the size of the character representations could improve the robustness of the search. To change the size, use the Font.Resize method. Permanently changing the size of the font can be faster than resizing the font before each read/verify operation, as is done if the font is calibrated.

Visualizing

It might be necessary, at some point during application development, to display the character representations of your character recognition font. To do so, use Font.ExportGrayscale method to copy the character representations to a displayable image. The following example shows you how to inquire about a character recognition font and how to display its character representations: MOcr.

Erasing characters

To remove a character from the character recognition font, use the Font.EraseCharacter method and specify the ASCII code associated with the character representation to remove. A character recognition font can contain a limited number of characters; this number is set using the Font.NumberOfCharacters property prior to the creation of the font (Allocate method). Removing unused or erroneously added characters is the easiest way to assure that these characters will not be used when looking for matches in the target string and that there is space for new characters to be added.


Defining the target strings

Previous | Next

Define the target strings in the following ways:

  • Calibrating your font helps to match the characters in the target image to those within the character recognition font.
  • Setting the appropriate properties and providing information about the target string helps to better define the search criteria and provides best results.
  • Constraints allow you to limit the search to specific characters within the character recognition font.

Calibrating your font

You should either manually or automatically calibrate your character recognition font to better match the size of the characters and the inter-character spacing in the target image. The more the CharacterRecognition control knows about your target image, the faster it can search for the required string(s) and the more robust the results.

Note that in some cases, it might be more efficient to change the size of the actual font than to calibrate.

Automatic calibration

Automatic calibration is only available when you are using a constrained search algorithm.

Use the CalibrateFont method, and specify a sample target image and a small range of sizes in which to search for the target string. The method will test all the possible sizes in that range, returning the best match of the width, the height, and the spacing of the characters in your sample target image.

Automatic calibration is always performed using the first string found with the highest match score in the sample target image. If the string cannot be located in the image, an error is generated. Skipping the string locator step (by setting the StringLocation property to False) might produce undesirable results. This is discussed later in this chapter.

When dealing with long strings, the spacing in your target image should be as accurate as possible.

Automatic calibration resets values set during a manual calibration.

Manual calibration

Manual calibration can be done when using either the general or the constrained search algorithm. With a general search algorithm, manual calibration does not require exact numbers.

Use properties of the TargetCharacter property to manually set the width (using the SizeX property), the height (using the SizeY property), and spacing (using the Spacing property) of the target image characters, in pixels. If the exact measurements are known, manually setting the size of the target characters is faster than an automatic calibration.

When using a general search algorithm with a fixed size font and reading a string that has the same spacing between characters, but the size of the spacing is unknown, set the Spacing property to ocrSame. If the spacing is unknown and/or not the same between characters, set the Spacing property to ocrAny.

Note that the scale factors between target character sizes and font characters must be between 0.25 and 4.0, inclusive. The following restrictions apply:

  • (TargetCharacter.SizeX/Font.Character.SizeX)>=0.25
  • (TargetCharacter.SizeY/Font.Character.SizeY)>=0.25
  • (TargetCharacter.SixeX/Font.Character.SizeX)<=4.0
  • (TargetCharacter.SizeY/Font.Character.SizeY)<=4.0

Setting appropriate processing properties

Setting each of the following properties can improve the robustness of the search. Note, however, that setting these properties does increase the complexity of the operation and reduces its overall speed.

Blanks

By default, the CharacterRecognition control ignores blanks in the target image.

When using a general search algorithm, you can use the EnableBlankCharacters property to enable the ability to read blank characters. Blank characters should not be included in your character recognition font. Once enabled, you must modify your string length to include the number of blanks that you expect in the target image when calling the ReadString method. Blanks before the target string are not counted. A blank space will have the same size and inter-character spacing as all other characters in the character recognition font, unless the Spacing property is set to ocrAny.

Note that EnableBlankCharacters is available only when using a general search algorithm.

Broken characters

If the target image contains characters that contain scratches or breaks, the target image contains a broken character. To force the CharacterRecognition control to identify these characters as a possible match of the characters within the character recognition font, set the EnableBrokenCharacters property to True. This will reduce the speed of subsequent read/verify operations but can increase robustness. Broken characters should not be included in a character recognition font.

Note that, when using a general search algorithm, the EnableBrokenCharacters property must be enabled to read a broken character. When using a constrained search algorithm, the CharacterRecognition control will automatically try to identify broken characters.

Morphologicial filtering

When the difference between the foreground and the background is so slight that it causes read or verification errors, increase or decrease the value for morphological filtering, using the MorphologicalFilteringIterations property, to improve the CharacterRecognition control's chances of finding and identifying the characters within the target string. This property will enhance the contrast between the foreground and the background. Experimentation is required to determine the exact number that should be passed to this property.

Thickness and dots

You can thicken the characters using the ThickeningIterations property when reading thin characters made up of dots, to make the characters easier to find and identify.

Touching characters

If characters in the target image touch each other, or if they are connected to blobs of a similar intensity, enable touching characters, using the EnableTouchingCharacters property. This will improve the CharacterRecognition control's chances of finding and identifying these characters. This will reduce the speed of subsequent read/verify operations but can increase robustness.

Note that, when using a general search algorithm, the EnableTouchingCharacters property must be enabled to read touching characters. When using a constrained search algorithm, the CharacterRecognition control will automatically try to identify touching characters.

Specifying other string information

In some cases, additional information about the target string is required. For example, to search for multiple strings, or to search at an angle, certain properties should be set. In other cases, additional information will increase the robustness of the search.

Number of strings

You can set the search to find multiple strings using the NumberOfStrings property. Multiple strings can only be found if each string resides on a different line within the target image and each line of text does not overlap the previous.

Note that, for best results, all strings in an image should be of similar length, have a consistent inter-line spacing, and should start at a similar location along the X-axis.

Identifying the number of strings to read/verify is important when dealing with a target image containing multiple lines. It does not have to be specified for single-string images since the default value is 1.

String lengths

You should specify the length of the strings to be read, to receive reliable results using the CharacterRecognition control. The maximum string length must be set before allocating the CharacterRecognition control using the MaximumStringLength property. With a general search algorithm, the MaximumStringLength property can be set to ocrAny. Note, however, that this increases the processing time since the CharacterRecognition control must then try to calculate the string length based on successful matches between the target image and the character recognition font.

The default string length is set to the maximum string length of the CharacterRecognition control. When the CharacterRecognition control constraints are set (discussed later), specifying a string length can also improve the speed of a following read/verify operation.

Note that if the StringType property is set to ocrSemiM12_92, the string length must be 12. If the StringType property is set to ocrSemiM13_88, the string length must be 18.

If the entire text of the target image contains one or more strings that differ significantly in length, use a general search algorithm for fast results. For more robust results, create separate CharacterRecognition controls that use the constrained search algorithm for each line of text to be read/verified. Experimentation with both search algorithms is the only way to determine which provides the best solution for each case.

Often, when using a general search algorithm, errors in string length result in unpredictable results.

Angle

You can search for the strings within the target image at a specific angle, or through an angular range. The angle of the search is used to locate the string or strings before a read/verify operation. If individual lines in the target image are set at different angles, create a child image containing each string at a different angle and read them using a different CharacterRecognition control.

These child images should not overlap.

For each CharacterRecognition control, you can specify the search angle using the SearchAngle property. By default, the search angle is 0.0º.

To search through an angular range, use the DeltaPositive property to specify the positive range and/or the DeltaNegative property to specify the negative range in which to search. A search for a string within the target image through an angular range always starts at the string angle, as specified using the SearchAngle property. Note that the range in which to search, either in the positive or negative direction, should never be greater than 180°.

For example, if you were searching for a string at approximately 15°, you might want to search between 30° and 0°. To do this, you would set the Value property of the SearchAngle property to 15°, and DeltaPositive and DeltaNegative properties to 15°. This results in a 30° arc that will be searched for your string.

By default, searching through a range of angles is enabled only if either the DeltaPositive or the DeltaNegative property, or both, are set to a value other than 0.0º.

Note that, after setting any of the properties of the SearchAngle property, you must use the Preprocess method.

Positional variation

If the inter-character spacing is not even, you can increase the robustness of the search by specifying the maximum variation in position of the characters. The position variation increases the region being searched for characters within the target image (using the PositionVariationX and the PositionVariationY properties of the TargetCharacter property. These values are relative to the expected position of each character.

Constraints

The read operation compares each character in the target image to each character in the font to find the best match. You might know beforehand that certain characters (or types of characters) should appear at specific positions in the string. If this is the case, you can speed up and increase the robustness of the read operation by restricting the comparison to only those characters in the font. The following types of constraints can be set for each character in the string, using the Constraints.Item(ConstraintIndex).ConstraintType property:

  • A digit (ocrDigits): ASCII codes 48 to 57.
  • A letter (ocrLetters): ASCII codes 65 to 90 and 97 to 122.
  • An uppercase letter (ocrUppercaseLetters): ASCII characters 65 to 90.
  • A lowercase letter (ocrLowercaseLetters): ASCII characters 97 to 122.
  • A character from a specific list of characters (ocrUserDefinedConstraintType, for example, A,1,b,2), including special characters and punctuations (for example, &, -, ...).

Note that the CharacterRecognition control will search for the number of characters specified by the target string length. For best results, the number of characters to be found in your target image should match your string length.


Locating your text

Reference:
CharacterRecognition.Font.ImportGrayscale
CharacterRecognition.Load
CharacterRecognition.Save

The CharacterRecognition control tries to locate the strings to read/verify automatically. Both the angle of the string and the string length information are used to perform the location.

The location step can be skipped to improve the speed of a read/verify operation, by setting the StringLocation property to False.

When using a constrained search algorithm, you can skip the location step if you create a new image or mask that contains only the target string and a minimum amount of space around it. Use other controls in ActiveMIL to determine the required buffer size. For more information, refer to the Accessing and mapping image data section in the Images chapter.

When using a general search algorithm, string location can be skipped if you are using a clean target image.


Determining what is a match

Previous | Next

Acceptance levels

You can set the acceptance level of a successful read/verify operation:

  • For each character (use the TargetCharacter.AcceptanceThreshold property).

    • If the correspondence (also known as the match score) between a character in an image and a character and its constraints in an CharacterRecognition control is less than the specified acceptance level, that character is considered invalid and the associated validity flag for that character is set to false.

  • For the entire string of characters and the entire text (use the AcceptanceThreshold property).

    • If the match score for the entire string passes the acceptance level set for the string, the string is considered valid and its validity flag is set to true. The match score for the entire string is determined by taking the average of the match scores for all characters in that string. The match score for the entire text is the average of the match scores for all the read/verified strings.

A perfect match has a match score of 100%, and no correlation has a match score of 0%. If your images have a lot of noise or distortion, set a lower acceptance level. Poor-quality images increase the chance of false readings and will probably increase the time required to read/verify the character.

Perfect matches are highly improbable due to noise obtained during image acquisition.

Unrecognized characters

You can specify the symbol for unrecognized (or invalid) characters (use the TargetCharacter.InvalidSymbol property). If a character's match score does not reach the specified acceptance level, you can force a specified symbol to be returned at that position in the string. If no symbol is specified (default), the character with the closest match will be returned. For example:

Target string: | "HELLO"
Invalid character: | "*"
Acceptance threshold: | 30% 30% 30% 30% 30%
Match scores: | 70% 15% 53% 24% 80%
Result: | H*L*O

Since the match scores of the characters in the second and fourth positions are less than the specified acceptance level, these characters are replaced by asterisks in the result.


Retrieving and analyzing the results

Previous | Next

After having potentially located the string in your target image using the ReadString or VerifyString method, you can extract the required results from your result buffer using Results properties. All CharacterRecognition results are based on the concept of a character. A string is a group of characters that reside along the same line along the same principle axis and the same angle. Multiple strings are a group of strings that are to be read together.

Results are returned in the order in which they are read/verified, from left to right. Always check the validity of the string to ensure that the match score is equal or exceeds the acceptance level, using the Results.Valid properties. Additional checks against the actual length read and the intended length could be made to assure accuracy.

Invalid characters in the resulting strings can be replaced with an ASCII character using the TargetCharacter.InvalidSymbol property. Note that this property must be set before performing the read/verify operation.

A character

Character data is returned for each character in the string or strings. This data includes:

  • Position along the X and Y axis.
  • The height and width of the character.
  • The match score of the character.
  • The spacing of the character.
  • The validity of the character (based on the character's acceptance level).
  • The ASCII version of the character.

A string

String data is returned for each string. String data includes:

  • The contents of the string.
  • The angle of the string.
  • The length of the string.
  • The match score of the string.
  • The validity of the string (based on the string's acceptance level).

A text

When reading/verifying multiple strings, text data is returned for all the strings. The text data includes:

  • All the characters read, even if they are on different lines in the target image.
  • The number of characters read in total.
  • The match score for the entire text.
  • The threshold value used to binarize the target image. Note that this is only available when using a general search algorithm, multiple strings, and/or the string has a few degrees of rotation.

All the results for a single string are also available for each string in the entire text.

Retrieving and analyzing the results

Unexpected results can come from one of the following scenarios.

  • If characters are vertically overlapping (or touching).

    • Create a child image that contains the characters below the point where they overlap. This effectively removes the overlapping portion of the characters.

    The more that characters overlap vertically, the less chance for a valid result.

    To read a target image containing multiple lines of text, the CharacterRecognition control requires that there is enough space between lines. If lines are too close together (vertically), the robustness of the search will suffer.

    Use the EnableTouchingCharacters property to read horizontally touching characters.

  • If the string or strings have non-uniform inter-character spacing.

    • Read/verify the string with a general search algorithm and use the TargetCharacter.Spacing property set to ocrAny.

    This setting is best suited to read non-uniformly spaced characters.

    Create a child image to contain the most extreme spacing examples for best results.

  • If the string or strings in the target image are greatly disparate.

    • Create a child image for each different region in position, angle, and/or length. Read each region individually for best results.

    The more similar the strings in angle, position, and length, the faster and more robust the search.

    If the spacing, length or angle differ, use different CharacterRecognition controls for each child image, with their operational and processing controls configured appropriately for each situation. For more information, refer to the Defining_the_target_strings section previously in this chapter.

  • If the target image contains multiple lines with different lengths.

    • Create a child image that contains the area with the shorter strings and read the shorter strings, using a different CharacterRecognition control, into a different result buffer.

    Strings of similar length are easiest to locate.

    Make sure that the target image has a consistent inter-line spacing and that all lines start at a similar location along the X-axis, for best results.

  • If the target image contains blanks.

    • Verify that your string length is correct.

    Your string length determines the number of characters sought.

    If the problem persists and the ability to read blank spaces is enabled (use the EnableBlankCharacters property) remember to include them in your string length (use the MaximumStringLength property).

  • If the target image contains a lot of non-character blobs.

    • Try to improve your target image before trying to read/verify again. For more information, refer to the Defining the target strings section, previously in this chapter.

    An image that does not include blobs of equal intensity as the foreground will return best results.

    Use the MorphologicalFilteringIterations property to internally enhance the contrast of the image.

  • If the target image contains a string longer or shorter than what was expected.

    • The CharacterRecognition control will try to return the number of characters that you set as existing in the target image using the MaximumStringLength property.

    If your string length is set to a value larger than the actual length of the string in the target image, and if using a general search algorithm, the CharacterRecognition control will return the best matches found and stop searching after it has found the specified number of characters. If using a constrained search algorithm, the string length determines the exact number of characters found, even if this means returning characters with low match scores.

  • If the target string is at an angle.

    • Verify that the principal axis of the entire text is equal to the expected angle (nominal angle), specified using the SearchAngle property.

    A string at an angle greater or less than 0 takes longer to find.

    Create a target image with a string angle of 0. If the string is properly read, your original application did not follow the expected use of string angle.

Validating String Events

The CharacterRecognition control allows you to attach or detach user-defined function to a character recognition event immediately following a read/verify operation. The ValidateString event contains the user-defined function. This user-defined function can be used to impose global string constraints, and can be used to implement custom checksum functions or to reject strings that would have otherwise met the character constraints imposed.

Once your user-defined function is created, use the ValidateStringEvent property set to True. It will then execute during the last stage of either the ReadString or VerifyString method. When the user-defined function is used in conjunction with the StringType property set to ocrSemi..., the user-defined function will replace the default validation function.


Improving search speed

Previous | Next

To ensure the fastest possible read or verify operation:

  • Use as clean a target image as possible when using the general search algorithm.
  • Calibrate the character recognition font's character size and target spacing, to match the characters in the target images.
  • Preprocess once all the processing controls and constraints are set and before the application's first read/verify operation.
  • Reduce the area to read/verify in the target image by creating a child image around the target string using the Image.ChildRegion properties; the search time is roughly proportional to the area searched.
  • Set character constraints using the Constraints property.
  • Skip the location step (using the StringLocation property set to False), when possible.
  • Set the speed and reliability of the algorithm by setting the robustness factor (using the Speed property). For instance, when reading larger characters, robustness can be sacrificed for speed.
  • Disable the EnableBlankCharacters and the EnableBrokenCharacters properties, unless absolutely necessary.
  • Adjust your image to minimize the angular search range of the read/verify operation.

The more you know about the target image, the faster it can be read/verified. This includes:

  • Knowing where the string is located within the target image.
  • Knowing the height, width, and the inter-character spacing of the string within the target image.
  • Knowing the exact number of lines to be read from the target image.
  • Knowing the exact string length or at least that all the strings are of similar length.
  • Having good examples of each character to be found in the target image so a high-quality character recognition font can be made.

The CharacterRecognition control can help determine some of this information, for example, the CalibrateFont method can automatically calibrate the character recognition font's character size to match the characters in the sample target image, and the string location step with the StringLocation property can find a string within the image, barring any problems.


A typical application

Previous

This image is used in the following example to demonstrate a typical character recognition application.

The following example demonstrates how to read the serial number in an image of a semiconductor wafer, using the CharacterRecogntion control in conjunction with other ActiveMIL controls. The serial number, printed on the wafer, is a standard SEMI font character string, using SEMI M12 92, containing a checksum: mocrread.c


Measurement

Chapter 11: Measurement

This chapter describes the ActiveMIL Measurement control and the steps to follow to take measurements.

  • Overview of the Measurement control
  • Markers
    • A multiple marker
  • Steps to finding and obtaining measurements of markers
    • Add a marker or load a marker from disk
    • Setting the marker's search region
    • Setting the marker's characteristics
    • Viewing the marker
    • Specifying the measurement control settings
    • Specifying which measurements to take
    • Acquiring and pre-processing a target image
    • Finding the marker and taking measurements
    • Reading and saving results
    • Annotating results
    • A measurement example
  • Search region
    • Obtaining valid results
    • Orientation
    • Setting the search region's angle
    • Multiple-angle search for a marker
    • Determining the rotation tolerance of a marker
  • Search algorithm
    • Identifying edges
    • First derivative filter
  • Marker characteristics
  • Edge markers: fundamental characteristics
    • Polarity
    • Position and position variation
    • Contrast and contrast variation
    • Length
    • Line equation
  • Edge markers: advanced characteristics
    • Edge width
    • Minimum and maximum position
    • Edge strength
    • Edge threshold
    • Determining the strength of the required edge
    • Marker reference position
    • Weight factors
  • Stripe markers: fundamental characteristics
    • Polarity
    • Contrast and contrast variation
    • Width and width variation
    • Position
    • Length
    • Line equation
  • Stripe markers: advanced characteristics
    • Minimum and maximum position
    • Inside edge and inside edge variation
    • Position inside stripe
  • Multiple marker characteristics
  • Measurements between two markers
    • Steps to taking measurements between two markers
    • Calculating with multiple markers
    • Angle
    • Line equation and distance


Overview of the Measurement control

See Also | Next
Reference:
Measurement

The ActiveMIL Measurement control allows you to find sets of image characteristics or "markers" in an image, based on differences in pixel intensities. Upon finding a marker, the control returns the marker's spatial reference position and measures such features as its width and angle. The control can also take measurements between two markers.

The Measurement control can be used, for example, to measure the width of pins protruding from chips or printed circuit boards (PCBs) and to measure the distance between each pin.

The Measurement control relies on a one-dimensional analysis. As such, it has several advantages over the pattern matching control when locating relatively simple image characteristics; it is independent of lighting, more tolerant of slight differences, and much faster.

You specify the approximate location and other characteristics of a marker to help locate the marker in an image. The more precisely the marker characteristics are defined, the more likely it is to be distinguished from similar aspects of the image.

The Measurement control can operate on 8-bit or 16-bit unsigned grayscale images. Measurements are made and results are returned with subpixel accuracy.

This chapter discusses finding and obtaining measurements of markers, how to define and set marker characteristics, and then the steps that are generally followed when taking measurements between markers.


Markers

User Guide:
Edge markers: fundamental characteristics
Marker characteristics
Measurements between two markers
Multiple marker characteristics
Steps to finding and obtaining measurements of markers
Stripe markers: fundamental characteristics

Reference:
Measurement.Markers.Add
Measurement.Markers.Item.NumberOfOccurrences

To take any type of measurement with the ActiveMIL Measurement control, you must first add a marker, using the Markers.Item(MarkerIndex).Add method. The marker contains the image characteristics for which to search in the target image.

There are three basic types of markers that can be used in measurement operations:

  • Point marker. A marker consisting of a single point or multiple points and generally used as a reference position in calculations involving two markers. Point markers are placed manually at the required location(s) as reference markers; they are not searched for in the target image. Positional results from a previous pattern matching or blob analysis operation on the image can also be declared as point markers.
  • Edge marker. A marker consisting of an edge or multiple edges. In an image, edges are curves that delineate a boundary, which can be established from intensity transitions.
  • Stripe marker. A marker consisting of a stripe (two edges) or multiple stripes. Edges of a stripe marker do not have to be parallel.

A multiple marker

The Measurement control allows you to define a multiple edge or stripe marker, so that you can search for multiple instances of the same image characteristics. The Markers.Item(MarkerIndex).NumberOfOccurrences property specifies the number of edges or stripes to locate. Using a multiple marker in a measurement operation makes it possible to take global measurements, then compare them for conformity.

For example, a multiple marker could be used to verify if a series of presumed-identical pins are actually of equal length and width or if the spacing between the pins is within established limits. Note that a multiple marker is considered to be only one marker that has a specified number of instances of the same characteristics.


Steps to finding and obtaining measurements of markers

User Guide:
Marker characteristics
Markers
Measurements between two markers
Search Region

Reference:
Digitizer
Image
Measurement.FindMarker
Measurement.Free
Measurement.Image
Measurement.Markers
Measurement.MeasurementList
Measurement.PixelAspectRatio
Measurement.Results

Although there are many types of measurements that can be taken with the ActiveMIL measurement control, the series of steps outlined below are usually followed to find and obtain measurements of markers:

  1. Add a marker or load a marker from disk.
  2. Set the processing area, generally known as the search region.
  3. Set the marker's characteristics.
  4. Optionally, set the measurement control settings.
  5. Optionally, using the MeasurementList property, specify which measurements to take.
  6. Grab or load a target image. Optionally, preprocess it to improve its quality.
  7. Find the marker in the target image and calculate measurements.
  8. Read and save the results.

In general, the first five steps are performed once, while steps 6 to 8 are repeated as required. Note, you can avoid step 2 through 5 by saving an initialized marker on disk and recalling it when needed.

Add a marker or load a marker from disk

You can add an edge, stripe, or point marker, depending on your application needs, with the Markers.Add method; if a multiple marker is needed, specify the number of occurrences of the edge or stripe with the Markers.Item(MarkerIndex).NumberOfOccurrences property. When a marker is no longer required, you should free the memory associated with it, using the Free method. Store a marker to disk using the Markers.Item(MarkerIndex).Save method. The save method stores all of the marker's associated characteristic settings, such as the marker's typical position, width, and contrast. To retrieve a marker from disk, use the Markers.Load method.

Setting the marker's search region

Before you can search for a marker, you must define the area in which to perform the search. This area is known as the search region. The search region is stored as a characteristic of the marker and is set with the Markers.Item(MarkerIndex).SearchRegion property. Subsequent search operations for this marker will only be performed in the area defined by the region.

Setting the marker's characteristics

The marker's characteristics, such as its approximate width, orientation, and position, can also be defined using the properties of the Markers property.

Viewing the marker

You can use the Markers.Item(MarkerIndex).Draw method to draw specific features of your marker as a means of verifying your marker settings. For example, you can draw the measurement search region, the position, possible position variation, width variation, and edge of the expected marker. For stripe markers, you can draw both the first and second edges. Note that when drawing a multiple marker, drawing is done at the initial specified position only (other instances are not drawn).

You can use a previously allocated graphic context (see the Graphics chapter) to control the drawing color, or use the default graphic context. You can draw directly into the image, or annotate the image non-destructively by drawing into its display's overlay image (see the Annotating the displayed image non-destructively section).

Specifying the measurement control settings

The properties of the PixelAspectRatio property control the behavior of measurement operation and should be specified.

Specifying which measurements to take

The MeasurementList property allows you to specify which measurements to take (the default setting performs all measurements).

Acquiring and pre-processing a target image

The Image property of the Measurement control specifies the target image. The target image data can be either loaded from disk or acquired from an input device and placed into an image (using the Digitizer and Image controls). You can preprocess the target image to remove noise and improve the image prior to performing the measurement operation. Note, however, that preprocessing operations, such as filtering, often result in a slight shifting of the position of edges. Therefore, if precise edge location and measurement are crucial to your application, preprocessing operations should be kept to a minimum.

Finding the marker and taking measurements

Use the FindMarker method to find a marker and measure its characteristics.

After making a call to the FindMarker method, you should use the Markers.Item(MarkerIndex).IsFound property of the Measurement control to check if the marker has been successfully located before obtaining results. If the marker is not found, you should modify your characteristic settings and check the placement of the search region.

Reading and saving results

You can obtain results using the Results property. All coordinates are relative to the top-left pixel of the target image. Results do not overwrite specified marker characteristics.

Save the required results in variables before finding another marker since results are erased upon the next call to the FindMarker method. Alternatively, you can call the FindMarker method with the same marker index to retrieve the required result at a later time, provided the image and the marker characteristics have not changed. Note that the second call to the FindMarker method will retrieve the internal copy of the results rather than perform the operation again.

Note that an error is generated if results are requested for a marker which has not been found.

For a multiple marker, results are returned for all the edges or stripes found. The number of edges or stripes found can be obtained using the Results.Count property. Global statistic results, such as the maximum, minimum, mean, or standard deviation of any characteristic of the result group can be returned with Results.CalculateMaximum, Results.CalculateMean, Results.CalculateMinimum, and Results.CalculateStandardDeviation methods.

Annotating results

In addition to the GraphicContext control methods available for annotating your results, the Results.Item(ResultsIndex).Draw, Results.Item(ResultIndex).Edge1.Draw, and Results.Item(ResultIndex).Edge2.Draw methods provide several additional operations for drawing features of the marker occurrences. For example, you can draw the edge profile of the occurrence, a cross at the found position, the width of the marker, among other features.

A measurement example

The MMeasurement example demonstrates how to use the ActiveMIL measurement control to find the positions, widths, and angles of the pins on a chip.


Search region

User Guide:
Basic geometric transforms
Search algorithm

Reference:
ImageProcessing.Rotate
Measurement.FindMarker
Measurement.Markers.Item.Orientation
Measurement.Markers.Item.SearchRegion

The marker's search region (Markers.Item(MarkerIndex).SearchRegion property) indicates the area of the target image in which to search for the marker. Proper placement of the search region is essential to the success of any FindMarker search. The default setting of the search region is the whole image (SearchRegion.FullSize = True). The search region should be limited to as small an area containing the marker as possible, in order to ensure the success of the operation, especially when using a highly detailed or complex target image. In addition, by limiting the search region, you can accelerate the find marker operation.

There are two ways to specify the search region. To specify the region's position, you can set either the coordinates of the origin with the StartX and StartY properties or the center with the CenterX and CenterY properties. You must also set the region's width and height with the SizeX and SizeY properties.

Note that the origin is always the top-left corner of the unrotated region and all width and height values are positive.

Obtaining valid results

To obtain valid results the edge or stripe must enter and leave by opposite sides of the search region. The illustration below is an example of valid and invalid search region definitions.

Orientation

The Markers.Item(MarkerIndex).Orientation property specifies the angle of the edge or stripe relative to that of the search region and can be set to measVertical, measHorizontal, or measAny. More importantly, the orientation determines the direction in which the search will proceed. If set to measAny, both orientations will be searched.

These settings can tolerate a certain amount of rotation. The amount is determined by the target image, placement of the search region, and the marker characteristic settings. The greater the degree of rotation, the greater the chance of not finding the marker and miscalculation of characteristics, such as edge strength and width. If the rotation is too great and you cannot find the marker, the marker's search region should be defined with an angle.

Setting the search region's angle

You can set the search region's angle, using the Markers.Item(MarkerIndex).SearchRegion.Angle.Value property, to approximately the same angle as the marker. The angle is in a counter-clockwise direction relative to the positive X-axis and can be any value from 0 to 360 degrees. When an angle is specified, the center of rotation used is the center of the search region. To modify this default center of rotation, use the CenterOfRotation property of the Angle property.

Multiple-angle search for a marker

You can also rotate the search region to search within a specified range of angles. To perform a multiple-angle search for a marker, enable multiple-angle search, using the Markers.Item(MarkerIndex).SearchRegion.Angle.Enabled property. Then, specify the range of angles to be searched (NegativeDelta and PositiveDelta properties), the degree of rotation tolerance (Tolerance property), the interpolation method (InterpolationMode property), and the required degree of accuracy for the resulting marker (Accuracy property).

The approximate location of the edge or stripe marker is first found by searching at every n degrees within the angular range, where n is specified by the rotation tolerance (Tolerance property). The marker's rotation tolerance is the full range of degrees within which a marker can be rotated from a search region that is at a specific angle and still be found. Once the approximate location of the edge is found, the degree of accuracy controls the number of fine-tuned searches that are performed. To be effective, you must set the degree of accuracy to a value smaller than that of the rotation tolerance.

If searching within a range of angles causes the search region to be rotated beyond the edges of the target image, you must reposition and resize the search region.

Determining the rotation tolerance of a marker

Every marker has its own particular rotation tolerance (that is, by how much the marker can be offset from the search region and still be found). This tolerance is dependent on the individual marker characteristics and surrounding target image features. To determine the rotation tolerance of a marker, simulate the rotation of the marker by rotating the image while maintaining the search region at a fixed position and angle. That is:

  1. Make sure that the marker is at its expected (nominal) angle, and that the position and angle of the search region is set so that the marker is located within the search region at close to the same angle. The search region must remain in a set position. Therefore, multiple-angle search (Markers.Item(MarkerIndex).SearchRegion.Angle.Enabled property) must be disabled.
  2. Use the Rotate method of the ImageProcessing control to rotate the image in very small increments (for example, 0.5°), in the positive direction, and call the FindMarker method on each rotated image. Make sure that the image's center of rotation is the same as that of the search region, otherwise the resulting tolerance will not be accurate. Note, when rotating the image, always set the angle from the image's original position to avoid interpolating the image more than once.
  3. Check the results for the greatest angle that produces an acceptable score.
  4. Repeat steps 2 through 3, rotating the image in the negative direction.
  5. Take the minimum of the absolute value of these angles. Double this angle and set it as the rotation tolerance for an angular search.

Alternatively, you can set the search region angle to measAny to allow ActiveMIL to analyze the contents of the search region and determine the angle. However, it should be noted that the processing time will be greatly increased when using this technique.


Search algorithm

User Guide:
Search region

Reference:
Measurement

This section describes the search algorithm used to identify edges by the Measurement control.

Identifying edges

The ActiveMIL Measurement control performs the following search algorithm to identify edges:

  1. The ActiveMIL Measurement search algorithm projects the pixels bounded by the specified two-dimensional search region into a one-dimensional pixel summation, storing the result in an internal projection buffer (that is, it takes the search region's profile). The pixel summation is peformed horizontally or vertically, depending on the search region's origin (Markers.Item(MarkerIndex).SearchRegion.StartX and StartY properties) and the orientation of the marker (Markers.Item(MarkerIndex).Orientation property). Each sum represents the intensity of all the pixels in that column.
  2. The search algorithm applies a first derivative filter to the internal projection buffer (pixel summation). The filter finds the edge value of each projection value. The greater the difference in neighboring projection values, the larger the edge value. Edge values are represented as a normalized percentage of the maximum pixel value possible for the specific image buffer. Filters are discussed later in more detail.
  3. The search algorithm then identifies possible edges by their edge strength (EdgeStrength property), that is their maximum/minimum edge value (depending on the polarity of the edge represented by the sign of the edge value). It rejects as possible marker occurrences any edge with an edge strength below the edge threshold value (EdgeThreshold property).
  4. It then scores each possible edge based on geometric constraints that you specify. You can assign a specific weight (degree of importance) to each specified characterstic using the WeightFactor properties. By default, the edge strength (EdgeStrength property) receives the most importance. For more information on weights, see Edge markers: advanced characteristics. The edge(s) with the highest score is returned as an occurrence of the marker.

First derivative filter

The first derivative filter extracts the difference in neighboring projection values. The edge value calculated for each projection value of the search region is dependent on the type of the first derivative filter used. In the ActiveMIL Measurement control supports two types of first derivative filters which can perform the edge extraction: Finite Impulse Response (FIR) filters and Infinite Impulse Response (IIR) filters. FIR filters are implemented using a non-recursive convolution operation with a predefined kernel of the specified size. To use a FIR filter, set the Markers.Item(MarkerIndex).FilterType property to measEuler or measPrewitt. IIR filters are implemented using a recursive convolution operation, and the kernel size would be theoretically infinite if this filter actually used a kernel. To use an IIR filter, set the FilterType property to measShen, and specify the degree of smoothness (strength of denoising) applied to the internal projection buffer of the search region, using the FilterSmoothness property.

You can try these filters to see which best suits your application needs. The following are the general characteristics of each filter:

  • [-1,1] (measEuler). This is a FIR filter with a convolution kernel of size 2. This filter is faster but more sensitive to noise, compared to the measPrewitt filter.
  • [-1,0,1] (measPrewitt). This is a FIR filter with a convolution kernel of size 3. This filter is slower but less sensitive to noise, compared to the measEuler filter.
  • An IIR filter (measShen). This is a Shen-Castan Infinite Support Exponential filter. This is an exponential weighting function, of the general form:

    • This IIR filter is slower than FIR filters (measEuler or measPrewitt). However, this filter is less sensitive to noise and provides more accurate results. You can also control the strength of denoising.


Marker characteristics

User Guide:
Edge markers: fundamental characteristics
Markers
Search region
Stripe markers: fundamental characteristics

Reference:
Measurement.FindMarker
Measurement.Markers
Measurement.Markers.Item.Contrast
Measurement.Markers.Item.Polarity
Measurement.Markers.Item.SearchRegion
Measurement.Markers.Item.Width

Associated with a marker is a set of parameters specifying the characteristics of the marker. The FindMarker method searches through the target image to find the edge or stripe that best matches the characteristics of the specified marker. The more precisely defined your marker characteristics, the more likely the FindMarker method will succeed in distinguishing the marker from similar aspects of the image.

Marker characteristics are set to their default values upon allocation of the marker. These values can be modified at any time, using the properties of the Markers property.

The important characteristics to set when searching for a marker are the search region, polarity, contrast, and for stripe markers, the width.


Edge markers: fundamental characteristics

User Guide:
Edge markers: advanced characteristics
Measurements between two markers
Search region
Stripe markers: fundamental characteristics

Reference:
Measurement.Calculate
Measurement.FindMarker
Measurement.Markers
Measurement.Results

This section describes the fundamental characteristics of edge markers which are set using the properties of the Markers property or obtained by the FindMarker method as measurement results.

Polarity

The polarity of an edge describes whether an edge is rising or falling and is set with the Markers.Item(MarkerIndex).Polarity.Edge1 property. A rising edge denotes a rise in grayscale values and a positive (measPositive) polarity. A falling edge denotes a decrease in grayscale values and a negative (measNegative) polarity. When setting the polarity of a marker it is important to keep in mind the direction of the search, which is performed horizontally or vertically, depending on the search region's origin and the orientation of the marker (see the Search Region section).

Position and position variation

The marker's position is defined as the X and Y coordinates of the marker's center pixel (the center of the portion of the marker located within the search region). These coordinates are relative to the top-left pixel of the image and are used as the default reference position when using the Calculate method to calculate measurements between two markers.

When several edges have similar characteristics within the same search region, you can use the properties of the Markers.Item(MarkerIndex).Position property to specify the approximate X- and/or Y-coordinates (X and Y property) at which to find the required marker's center. You can also specify a tolerance for these coordinates (Variation property).

After calling the FindMarker method, results are accessible with the Results.Item(OccurrenceIndex).PositionX, Results.Item(OccurrenceIndex).PositionY, and Results.Item(OccurrenceIndex).PositionVariation properties.

Note, the position must be located within the search region (taking into account the search region's angle or angular range), otherwise an error is generated.

Contrast and contrast variation

You can indicate the typical difference in average grayscale values between the start and end of the intensity transition (edge width) from which an edge is established, using the Markers.Item(MarkerIndex).Contrast.Edge1 property. Contrast is useful in distinguishing between several different edges, particularly when the required edge does not have the largest edge strength, or when the edge is at an angle. When the edge strength is not very high, you should specify the contrast. In general, it is recommended to set the contrast rather than to specify an edge strength, since the edge strength is very dependent on lighting.

You can also specify a tolerance for the contrast with the Markers.Item(MarkerIndex).Contrast.Variation property.

After calling the FindMarker method, results are accessible with the Results.Item(OccurrenceIndex).Contrast property.

Length

You can measure the length of an edge marker. The length being measured is restricted to the portion of the marker contained within the search region. Note, the length of an edge marker cannot be set, but can be returned with Results.Item(OccurrenceIndex).Length property after calling the FindMarker method.

Line equation

You can calculate the line equation of the mean line following an edge marker: Y = MX + B, where M denotes the slope of the line and B denotes the Y-intercept.

To determine the equation of the line (slope) for each edge, ActiveMIL performs additional internal search operations in the upper and lower thirds of the search region. If an image has substantial specks that appear in either of these two areas, the calculated lines might be at incorrect angles, as illustrated in the following images.

Notice the edge results on the left compared to the results on the right. Often, raising the edge threshold (Markers.Item(MarkerIndex).EdgeThreshold property) will correct this type of error. Also, you should always verify the edge values using the Results.EdgeValues.Item(OccurenceIndex) property.

Note, the line equation of an edge marker cannot be set, but can be returned with the Results property after calling the FindMarker method. The slope of the line (Results.Item(OccurrenceIndex).LineEquationSlope property) and the Y-intercept of the line equation (Results.Item(OccurrenceIndex).LineEquationIntercept property) are returned separately.


Edge markers: advanced characteristics

User Guide:
Edge markers: fundamental characteristics

Reference:
Measurement.Calculate
Measurement.FindMarker
Measurement.Markers
Measurement.Results

This section describes the advanced characteristics of edge markers that are set using the properties of the Markers property or obtained by the FindMarker method as measurement results.

Edge width

In an image, edges are curves that delineate a boundary, which can be established from intensity transitions. The smoother the image, the more gradual the transition. The edge width can be seen as a measure (in pixels) of this gradual transition in grayscale values. The diagram below illustrates a profile of gradual transition from dark to light.

The ActiveMIL Measurement control calculates the edge's position to be in the middle of this edge width. Note, the more an edge is at an angle, the greater the stretching or distortion of its edge width.

The edge width cannot be set, but can be found with the FindMarker method. Results are returned with the Results.Item(OccurrenceIndex).Width property.

Minimum and maximum position

The minimum and maximum positions indicate the minimum and maximum positions of an edge in an edge marker.

The X-coordinate of these positions are the minimum and maximum position within the edge width from which an edge is extracted, respectively. The Y-coordinate is the Y-coordinate of the search region center. The minimum position is always on the side of the search region adjacent to the origin, and the maximum position is always on the side of the search region furthest from the origin, regardless of how the search region is rotated.

The minimum and maximum positions of an edge cannot be set but can be found with the FindMarker method. Results are returned with the Results.Item(OccurrenceIndex).MaximumPositionX, Results.Item(OccurrenceIndex).MaximumPositionY, Results.Item(OccurrenceIndex).MinimumPositionX, and Results.Item(OccurrenceIndex).MinimumPositionY properties.

Edge strength

A marker's edge strength (Markers.Item(MarkerIndex).EdgeStrength property) is either the minimum or maximum edge value along the edge width, depending on the polarity of the edge. Edge values are the output of the first derivative filter (that is applied to the internal projection buffer) represented as a normalized percentage of the maximum pixel value possible for the specific image. The sign of the edge value represents the polarity of the edge. For example, for a search region with a vertical orientation, the equation for an edge value is:

You can also specify a tolerance for the edge strength (Markers.Item(MarkerIndex).EdgeStrength.Variation property).

For the following diagrams, edge values are caculated using an Euler FIR filter, [-1,1]. Using different filters will result in different edge values. For more information on identifying edges using first derivative filters, see the Search algorihm

The edge in the diagram below has an edge strength of 100%, the maximum edge value possible since the edge is completely vertical and has an edge width of one pixel.

However, if the edge is at the following angle, the edge values are distributed over ten profile values, resulting in a much lower edge strength of 10%:

When the edge strength is not very high, you should also specify the contrast (Markers.Item(MarkerIndex).Contrast). By using the contrast characteristic, you can indicate the typical difference in average grayscale values between the start and end of the intensity transition (edge width) from which an edge is established, allowing the marker to be located. Contrast is useful in distinguishing between several different edges, particularly when the required edge does not have the largest edge strength, or when the edge is at an angle. In general, it is recommended to set the contrast rather than to specify an edge strength, since the edge strength is very dependent on lighting.

After calling the FindMarker method, edge strength results are returned with the Results.Item(OccurrenceIndex).EdgeStrength property.

Edge threshold

The edge threshold is the edge value beneath which a grayscale variation is not considered an edge. It can be set with the Markers.Item(MarkerIndex).EdgeThreshold property.

Determining the strength of the required edge

The Results.EdgeValues property determines the edge values for every profile value of the search region. This returns the calculated edge value for every projection value of the search region after calling the FindMarker method. This allows you to determine what is likely to be considered an edge and specify an appropriate edge threshold value.

The illustration below shows a specific edge strength measurement, a graphical representation of edge values across the search region, and a sampling of these results.

Notice that the search region is 400 pixels wide so there are 400 edge values that are returned.

Marker reference position

The marker reference position Markers.Item(MarkerIndex).ReferenceX and Markers.Item(MarkerIndex).ReferenceY properties) defines the position from which measurements between two markers are taken. By default, the reference position is set to the center position of the marker. You can, however, move the reference position by specifying X- and Y-offsets relative to the center of the marker. The marker reference position is only used with the Calculate method; the FindMarker method always returns the marker's actual center. For example, the reference position for the edge marker in the diagram below is set to the center left of the marker.

Weight factors

When searching for a marker, the relative importance (weight) assigned to each of the marker characteristics is crucial to the robustness of the operation. By default, 50% of the search weight is assigned to the edge strength; the remaining 50% is equally divided among all characteristics that support weight factors and that are set to a value other than measAny (the value used to flag an "ignore" state). This makes the edge strength by far the most important characteristic in the search.

You can override this default by using the WeightFactor property of the marker characteristic (note, not all marker characteristics have weight factors). However, to better control the search, it is recommended when specifying weight factors that you assign a weight factor to all the enabled characteristics which support weight factors to a total of 100%.

For example, in a case where you must distinguish between two edge markers of different contrast, you can specify the typical contrast of the marker to be found. If you specify only this characteristic, the default search algorithm will assign a 50% weight to the edge strength and the remaining 50% to the contrast. However, if the edge you want to ignore has the higher edge strength, the required edge might not be found. In this case, specifying the weights as 30% for the edge strength and 70% for the contrast will give precedence to the edge with the best match to the specified contrast.


Stripe markers: fundamental characteristics

User Guide:
Edge markers: advanced characteristics
Edge markers: fundamental characteristics
Stripe markers: advanced characteristics

Reference:
Measurement.FindMarker
Measurement.Markers
Measurement.MeasurementList.Length
Measurement.Results

A stripe marker is simply a marker with two edges, therefore the discussion of edge characteristics applies to each of the stripe marker's edges. However, certain characteristics have special attributes applicable only to stripe markers.

As with edge markers, the characteristics of stripe markers are set using the properties of the Markers property or obtained by the FindMarker method as measurement results.

Polarity

The polarity of an edge describes whether an edge is rising or falling. The polarity of a stripe marker's edges is set with the Markers.Item(MarkerIndex).Polarity.Edge1 and Markers.Item(MarkerIndex).Polarity.Edge2 properties. The edges of a stripe marker can have opposite (measOpposite) polarities or can have similar (measSame) polarities.

After calling the FindMarker method, results are accessible with the Results.Item(OccurrenceIndex).Edge1.Polarity and Results.Item(OccurrenceIndex).Edge2.Polarity properties.

Contrast and contrast variation

The contrast for a stripe marker requires two values, one for each of the stripe's edges and is set with the Markers.Item(MarkerIndex).Contrast.Edge1 and Markers.Item(MarkerIndex).Contrast.Edge2 properties. The contrast of the second edge can be set to measSame if both edges of the stripe have approximately the same contrast. Both values can be set to measAny if the contrast is unknown (default). The contrast variation for a stripe marker should be the maximum of the contrast variations of both its edges.

After calling the FindMarker method, results are accessible with the Results.Item(OccurrenceIndex).Edge1.Contrast and Results.Item(OccurrenceIndex).Edge2.Contrast properties.

Width and width variation

To help find a stripe marker, you can specify the typical distance between both of its edges (Markers.Item(MarkerIndex).Width.Value) in pixels. You can also specify by how many pixels the width of a stripe marker might vary (Markers.Item(MarkerIndex).WidthVariation). This value should be equivalent to the maximum amount of variation.

The width of a stripe marker is the average distance in pixels between its edges. Note, if the marker search region is at an angle, the width is measured according to the orientation of the region, as shown below.

The width of a stripe marker can be found with the FindMarker method and results are returned with the Results.Item(OccurrenceIndex).Width property.

Position

The position (Markers.Item(MarkerIndex).Position property) of a stripe marker is considered either the center between the two edges of the stripe or a position outside of the stripe marker, depending on the Markers.Item(MarkerIndex).Position.StripeLocation property.

In addition, the stripe marker's center is the default reference position. The reference position is specified relative to the marker's center. You can move the reference position of a marker using the Markers.Item(MarkerIndex).ReferenceX and Markers.Item(MarkerIndex).ReferenceY properties.

After calling the FindMarker method, the position of the stripe (the center position between its two edges) is returned with the Results.Item(OccurrenceIndex).PositionX and Results.Item(OccurrenceIndex).PositionY properties. The position of the first edge of a stripe is accessible with the Results.Item(OccurrenceIndex).Edge1.PositionX and Results.Item(OccurrenceIndex).Edge1.PositionY properties and for the second edge with the Results.Item(OccurrenceIndex).Edge2.PositionX and Results.Item(OccurrenceIndex).Edge2.PositionY properties.

Length

The length of a stripe marker can be measured along the mean line between its exterior edges (MeasurementList.Length property). The length of either of its edges can also be measured. The length being measured is restricted to the portion of the marker contained within the search region. Note, the length of a stripe marker cannot be set, but can be returned with the Results.Item(OccurrenceIndex).Length property.

Line equation

You can calculate the equation of the mean line following a stripe marker: Y = MX + B, where M denotes the slope of the line and B denotes the Y-intercept.

Line equations can be calculated for a stripe, or each edge of a stripe marker. The line equation of a stripe marker is the mean of the line equations of its edges.

Note, the line equation of a stripe marker cannot be set, but can be returned with the Results property. For the mean line of a stripe marker, the slope of the line equation (Results.Item(OccurrenceIndex).LineEquationSlope property) and the Y-intercept of the line equation (Results.Item(OccurrenceIndex).LineEquationIntercept property) are returned separately. For the first edge of a stripe, results are obtained with the Results.Item(OccurrenceIndex).Edge1.LineEquationSlope and Results.Item(OccurrenceIndex).Edge1.LineEquationIntercept properties; for the second edge, specify Edge2.


Stripe markers: advanced characteristics

User Guide:
Edge markers: advanced characteristics
Stripe markers: fundamental characteristics

Reference:
Measurement.Markers.Item.EdgeThreshold
Measurement.Markers.Item.NumberOfInsideEdges
Measurement.Markers.Item.Position
Measurement.Results.Item.EdgeInside

Minimum and maximum position

The minimum and maximum positions indicate the minimum and maximum positions of edges in a stripe marker. For more information on the minimum/maximum position of an edge, see the Edge markers: advanced characteristics.

The minimum and maximum positions of an edge cannot be set but can be found with the FindMarker method. The average value of the two edges of a striper marker is returned with Results.Item(OccurrenceIndex).MaximumPositionX, Results.Item(OccurrenceIndex).MaximumPositionY, Results.Item(OccurrenceIndex).MinimumPositionX, and Results.Item(OccurrenceIndex).MinimumPositionY properties.

The minimum and maximum positions of the first edge in a stripe marker are returned with the Results.Item(OccurrenceIndex).Edge1.MaximumPositionX, Results.Item(OccurrenceIndex).Edge1.MaximumPositionY, Results.Item(OccurrenceIndex).Edge1.MinimumPositionX, and Results.Item(OccurrenceIndex).Edge1.MinimumPositionY properties; for the second edge, specify Edge2.

Inside edge and inside edge variation

To help find a stripe marker, you can specify the typical number of edges located between the external edges of the stripe marker you are defining. For example, in the following illustration, the two stripes share the same position since their centers coincide.

To find the larger stripe without having to determine and specify its width, specify 2 as the number of inside edges of the stripe marker (Markers.Item(MarkerIndex).NumberOfInsideEdges.Value property). To find the smaller stripe, specify 0 (the default, measAny, ignores the possibility of any inside edges). The identification of inside edges is based only on the edge threshold setting (Markers.Item(MarkerIndex).EdgeThreshold property. The number of such edges is found with the FindMarker method and are returned with the Results.Item(OccurrenceIndex).EdgeInside property.

You can also specify the tolerance in the number of inside edges of a stripe (Markers.Item(MarkerIndex).NumberOfInsideEdges property). Note, the value and tolerance should be in increments of two if stripes are contained within stripes, since two edges are recognized for each stripe.

Position inside stripe

If necessary, when defining a stripe marker, you can specify if the X- and Y-coordinates must be located inside or outside of the stripe (Markers.Item(MarkerIndex).Position.StripeLocation property). If the position is defined as being within the stripe, the search algorithm is modified so that the search proceeds outwards in both directions from that point, making the operation faster and more robust. The StripeLocation property is useful when the required stripe does not have the greatest edge strength or when it is difficult to set the search region without clipping other similar image characteristics. If defined as outside, no stripe including the position is considered.

Since other aspects of the image can have similar features or stronger edges, the required stripe might not be found. In order to successfully locate the stripe, specify its position with the X and Y properties and then set the StripeLocation property to measInsideOf so that the search begins from the marker's approximate position, allowing the marker to be found.


Multiple marker characteristics

User Guide:
Edge markers: advanced characteristics
Edge markers: fundamental characteristics
Stripe markers: advanced characteristics
Stripe markers: fundamental characteristics

Reference:
Measurement.Markers.Item.MinimumNumberOfOccurrences
Measurement.Markers.Item.NumberOfOccurrences
Measurement.Markers.Item.Position
Measurement.Markers.Item.Position.WeightFactor
Measurement.Markers.Item.Spacing.Value
Measurement.Markers.Item.Width.Value

To use a multiple marker, the marker's edge or stripe characteristics are set just as with any marker; only a few additional characteristics need to be specified. In addition, the Markers.Item(MarkerIndex).Position property and its WeightFactor property are ignored.

Specify the number of edges or stripes (Markers.Item(MarkerIndex).NumberOfOccurrences property) to be found (default is 1) and the typical spacing between them if a regular pattern is expected (Markers.Item(MarkerIndex).Spacing.Value property). Unless a minimum number (Markers.Item(MarkerIndex).MinimumNumberOfOccurrences property) is specified, no results will be returned if the number of edges or stripes found falls below the specified number of occurrences. If the exact number of edges or stripes is unknown, then the NumberOfOccurrences property can be set to measAny.

When the Markers.Item(MarkerIndex).Spacing.Value property is set to a value other than measAny, an initial search is performed to find all edges or stripes which best conform to the specified marker characteristics. This group of edges or stripes is then inspected to ensure that the spacing constraints are met.

The marker's Spacing.Value property can be set to measSame, which takes the average spacing of all located edges or stripes and then applies this spacing as a criteria for determining the marker's actual edges or stripes.

The Markers.Item(MarkerIndex).Width.Value property can also be set to measSame, meaning that the average width is applied as a constraint to all located stripes.

Note, the measSame setting is recommended for use only when several occurrences are expected: if the number of occurrences is too few, matches for width or spacing might not correspond to those of the required stripes.

A multiple point marker can also be specified for performing calculations between markers. A multiple point marker must have its initial position set using the properties of the Markers.Item(MarkerIndex).Position property. The number of points of the marker is determined by the Markers.Item(MarkerIndex).NumberOfOccurrences property. The spacing between these points must also be set using the Markers.Item(MarkerIndex).Spacing.Value property (a multiple point marker can only be defined to use the same spacing between points; if irregularly spaced points are needed, you must specify each one individually as a point marker).


Measurements between two markers

See Also | Previous
User Guide:
Edge markers: fundamental characteristics
Markers
Steps to finding and obtaining measurements of markers
Stripe markers: fundamental characteristics

Reference:
Measurement.Calculate
Measurement.FindMarker
Measurement.Markers.Item.IsFound
Measurement.MeasurementList
Measurement.Results

The Calculate method performs calculations between two markers' reference positions. The MeasurementList property specifies which measurements to take. The default setting of the MeasurementList property performs all measurements: distance, angle, and line equation. For a stripe marker the center position is used as the reference position from which to take measurements.

Steps to taking measurements between two markers

The series of steps outlined below are usually followed to take measurements between two markers:

  1. Add, define, and find each marker (see the steps previously outlined in the Steps to finding and obtaining measurements of markers section). Note that although the results of the first marker are erased upon finding the second marker, they are still kept internally.
  2. Use the Markers.Item(MarkerIndex).IsFound property to ensure that the markers are found before calling the Calculate method.
  3. Call the Calculate method, specifying which markers are to be used as the first and second reference positions and which measurements to take.
  4. Read the results, using the Results property. Note that these results are erased upon calling either the FindMarker method or Calculate method.

Calculating with multiple markers

If both markers are multiple markers, then calculations are made using the edges or stripes of the first marker and the corresponding edges or stripes of the second marker, for the entire first marker's number of edges or stripes. The number of calculations performed is limited to the smallest number of occurrences held in either marker (that is, if a marker contains only one edge or stripe, then only one calculation is performed, regardless of the number of edges or stripes contained in the other marker).

Angle

You can calculate the angle of a line joining two markers with the Calculate method.

The angle is measured in a counter-clockwise direction relative to the positive X-axis, and can be a value from 0 to 360 degrees.

Results are returned with the Results.Item(OccurrenceIndex).Angle property.

Line equation and distance

The line equation and distance can be determined, with the Calculate method, for the line joining two markers. Results are accessible with the Results.Item(OccurrenceIndex).LineEquationSlope and Results.Item(OccurrenceIndex).LineEquationIntercept properties.

The distance between two markers is the distance between both of their reference positions, as illustrated in the diagram below.

The horizontal or vertical distance between two markers can be calculated and are returned with the Results.Item(OccurrenceIndex).DistanceX and Results.Item(OccurrenceIndex).DistanceY properties.

The MMeasurement example illustrates the steps to find a stripe in an image, and measure certain marker features, such as its position and width. It also demonstrates how to perform calculations using a point marker as a reference point.


Code

Chapter 12: Code

This chapter describes how to read and write 2D and bar codes.

  • Overview of codes
  • Using ActiveMIL to read and write codes
  • Supported code types and noteworthy characteristics
    • Anatomy of 1D and 2D codes
    • More information
  • Controlling read operations
    • Code type
    • Encoding scheme
    • Error correction
    • Foreground color
    • Cell size
    • Number of cells
    • Search angle
    • Threshold value
    • Search speed
    • String length
  • Controlling write operations
    • Size of destination image
    • String requirements
    • Code type
    • Encoding scheme
    • Error correction
    • Foreground color
    • Cell size
    • Number of cells
    • Encoding non-printable characters
  • Retrieving results
  • Verifying your code


Overview of codes

See Also | Next
Reference:
Code.ReadCode
Code.WriteCode

Many industries label products using symbols from symbologies such as bar codes and Data Matrix codes. This is done for identification purposes during different stages of production and distribution. Each symbology, known as a code type in ActiveMIL, follows a different set of rules to encode the data into light and dark patterns; data is encoded into units called cells. There are many different standards for the rules on encoding data. To find out more about the standards, see standards organizations such as the Association for Automatic Identification and Mobility or EAN International.

The Code control can both read symbols from and write symbols to images using the control's ReadCode and WriteCode methods, respectively. To read a symbol, the ReadCode method searches for a code type in the image specified by the Image property of the control and decodes it. The decoded string can then be used to identify the object in the image. To write a symbol, the WriteCode method encodes the specified string into a symbol using the specified code type and encoding scheme, and writes the symbol in the image passed to the method. The resulting image of the symbol can then be rotated, combined with text on a logo, and then printed. In ActiveMIL, symbols are known as codes.

ActiveMIL also allows you to verify the quality of the code you want to read, or have written, with the VerifyCode method.. To verify a code, ActiveMIL computes the quality-grade of the code in the specified source image. Verification is often used to check the conditions under which you are reading codes and to check if the codes you are reading or writing meet your quality specifications.

The Code control supports 1D, 2D, and composite code types. One-dimensional (1D) code types are linear bar codes, and are used for small amounts of data, for example, the identification number for a grocery store item, or a stock room part number (data is represented by bars and spaces).

Two-dimensional (2D) code types store information along the height, as well as the width, of the code. 2D codes can store more information than 1D codes, and are typically used for shipment information or medical records.

The Code control also supports composite codes, which are a combination of specific 1D and 2D code types. The 1D component encodes the item's primary identification, while the 2D component encodes additional data (such as batch number or expiration date, for example).

For examples of different codes, see Supported code types and noteworthy characteristics.

Some code types support several methods of encoding, known as encoding schemes. This means a code type might support an encoding scheme of alpha characters only, numeric characters only, or both alpha and numeric characters. In addition, some code types can support any number of characters, while others require a fixed number.

If the image is degraded, codes can still be read if an error correction scheme was used when the code was generated. Error correction is essentially redundant data included in the encoding scheme of some code types. Some error correction schemes are used only for error detection, while others are used for error detection and recovery. These terms are discussed in further detail in the topic, Supported code types and noteworthy characteristics.

To perform code operations, you need a Code control, a System control, and at least one Image control. You can perform code operations on 8-bit unsigned code images. Performing code operations on any other image type will result in an error.

To retrieve the results of a read or write operation, use the Results property.

For an example that uses the Code control, see MCode.


Using ActiveMIL to read and write codes

User Guide:
Controlling read operations
Controlling write operations
Retrieving results

Reference:
Code.CalculateCodeSize
Code.CodeType
Code.EncodingType
Code.ErrorCorrectionType
Code.ReadCode
Code.Results
Code.SearchRegion
Code.UsePresearch
Code.WriteCode

The steps below describe, in general, how to perform read and write operations with a Code control. Note that you must take into consideration any particularities of the chosen code type. More detailed and code type-dependent information is described in subsequent sections in this chapter, as well as the ActiveMIL Reference.

To perform a read or write operation:

  1. Set the code type using the CodeType property.
  2. Based on the selected code type, set all required properties of the Code control, for example, the EncodingType or ErrorCorrectionType properties.
  3. If reading a code, grab or load the image that contains the code into the image specified by the Image property. If the image is large, or if there are multiple codes in the image, create a search region using the SearchRegion property of the Code control to isolate the code from the rest of the image. If you can't set a search region because of the complexity of the image, for example, and your image contains other important information besides the code, you might want to set the UsePresearch property to True, which will help ActiveMIL locate the code.
  4. If writing a code, call the CalculateCodeSize method to determine the required size of the image.
  5. To perform a read operation, use the ReadCode method. To perform a write operation, use the WriteCode method.
  6. Retrieve results, using the properties of the Results property.


Supported code types and noteworthy characteristics

Reference:
Code.CodeType
Code.EncodingType
Code.ErrorCorrectionType
Code.TargetGrid.MinimumCellSize
Code.TargetGrid.NumberOfCellsX
Code.TargetGrid.NumberOfCellsY
Code.TargetGrid.SearchAngle

Whether you plan on reading or writing codes with a Code control, you must specify the code type with its CodeType property.

Below is a list of the supported code types.

The following are the 2D code types supported.

The following is an example of a composite code.

Anatomy of 1D and 2D codes

To perform more sophisticated read and write operations using ActiveMIL, it is useful to understand the diagram below.

The following are terms commonly used in code reading and writing operations:

  • Cell. The narrowest dark or white space into which data is encoded. Cells have different shapes, depending on the symbology: cells in 1D code types are bars, some 2D codes have cells that are approximately-square blocks, while Maxicode cells are hexagonal. In 1D codes, thicker bars (and therefore spaces) occupy several contiguous cells.

    • Industry uses slightly different terminology for PDF417 codes. For example, industry uses the term modules to refer to cells. There are 17 modules (cells) in each column of a PDF417 code, and each column has four bars and four spaces. Industry also refers to columns as codewords; a codeword is a column within one row.

  • Cell size (also known as the x-width). Width, in pixels, of the cell, the code's narrowest unit.

    • The cell size of a Maxicode is defined as the distance between the center points of two adjacent hexagons in the same row.

  • Encoding scheme. The method used to encode a string into a code. Not all code types support all encoding schemes, so you must be aware of the characteristics of your code type.
  • Error correction. Error correction schemes prevent read operations from returning incorrect results. During read operations, their purpose is for error detection or error detection and recovery. Error detection discovers bit errors (when a 1 is read as a 0 or vice versa). Error detection and recovery first detects and then corrects bit errors; depending on the error correction scheme used, questionable data can be corrected based on the remaining data. To read a code, you must specify the same error correction scheme that was used to write the code.
  • Checksum number. Number included as part of a 1D or 2D code, which is calculated based on the other characters in the code; it is a simple form of error detection. Typically during a read operation, the characters that make up the string will be put into a checksum calculation whose results are compared against the checksum number. If a discrepancy exists between the calculated and stored checksum, the read operation will yield no results. Not all code types use a checksum.
  • Scanline. A scanline is the line where the Code control tries to read the code information. Multiple scanlines are needed to read the code; the first line is usually taken across the middle of the image. Scanlines are used with codes where the information can be read in a straight line, that is, codes with 1D components.
  • Start and stop characters. Characters which inform the reader or scanner of the beginning and end of a code, analogous to the start and stop bits which are specified when transferring data across a modem. Some code types, such as Code 39, use identical stop and start characters, while others use different stop and start characters. The PDF417 code type refers to these characters as start and stop patterns.
  • Quiet zone. Quiet zones are areas with no marks and are used to aid the scanning process. For 1D codes, the quiet zone immediately precedes the start character and immediately follows the stop character. For 2D codes, the quiet zone must be present on all four sides of the code. The quiet zone must contain no marks; even hand-written scribbles in the quiet zone will impede the read operation significantly.
  • Bearer bars. Bars that run along the tops and bottoms of a code, which are illustrated in the diagram below:

    • If you are reading a code with bearer bars, you must specify the search angle at which to read the code. ActiveMIL does not generate bar codes with bearer bars. If you want to generate a code with bearer bars, generate the required code and then draw a rectangle at the top and bottom of the code; the minimum width of the bearer bar must respect the specifications of your code type.

More information

For technical information about composite codes, MicroPDF417, or RSS codes, see AIM Global Network Standards

For technical information about Data Matrix, Maxicode, or PDF417, see AIM International Symbology Specification.

For technical information about Pharmacode, see The Pharmacode Guide by Laetus.

For technical information about Planet code, go to the United States Postal Service site, Rapid Information Bulletin Board System (www.ribbs.usps.gov). Look under File Libraries for a file named Confirm®. This will call a file titled Customer Guide to Confirm using PLANET Codes.

For technical information about Postnet, go to the United States Postal Service site, (www.usps.com) and find the POSTNET Barcode Certification page. This should be under Certification Programs, in the Address Management Services section under Address Quality.

For technical information about UPC-A, UPC-E or bar codes, see The Bar Code Book, Roger C. Palmer, Helmers Publishing, United States, 1995.


Controlling read operations

User Guide:
Retrieving results

Reference:
Code.CodeType
Code.EncodingType
Code.ErrorCorrectionType
Code.ForegroundPixelValue
Code.ImageThreshold
Code.ReadCode
Code.Results.NumberOfCellsX
Code.Results.NumberOfCellsY
Code.Speed
Code.TargetGrid
Code.TargetGrid.MaximumCellSize
Code.TargetGrid.MinimumCellSize
Code.TargetGrid.SearchAngle
Code.TargetGrid.SearchAngleDeltaNegative
Code.TargetGrid.SearchAngleDeltaPositive
Code.TargetStringLength
Code.UsePresearch

This section provides information to consider and describes settings that you might have to change for read operations when using a Code control.

It is advisable to specify a search region for read operations, especially if your image contains multiple codes of the same type and you want to read more than one, or if your image contains other information besides the code. There are, however, some situations where you are not able to specify the search region for your read operation. Your image could be very complex and the code could be difficult to locate. In this case, you could use the Code control's pre-search algorithm, which helps ActiveMIL to locate the code in the image prior to the read operation. The pre-search algorithm is only supported with 2D codes, and is disabled by default. This algorithm can also be used if your image contains several codes of different types. To use the pre-search algorithm, set the UsePresearch property to True.

ActiveMIL's Code control is designed to read one code at a time; therefore, if your image contains multiple codes and you cannot use a search region, and the pre-search algorithm cannot locate the code, results will be unpredictable.

It is highly recommended that when using the pre-search algorithm, you also specify the best cell size range possible, especially if the cell size is greater than 10 or smaller than 6.

ActiveMIL requires that all code types have a quiet zone for read operations; therefore, your image must be large enough for your code to have a quiet zone. ActiveMIL can, in some cases, successfully read a code that does not meet the minimum requirements for the quiet zone; however, it is strongly recommended that codes have one in order to perform robust operations and return accurate results.

You might want to, or be required to, set positional information of the search with the properties of the TargetGrid property. These properties determine the cell size, the number of cells in the X- and Y-direction, and the search angle.

Code type

You must specify the code type with the CodeType property when performing a read operation. If your read operation is unsuccessful, check this property's setting.

Encoding scheme

The Code control can automatically detect the encoding scheme for read operations. If required, you can specify the encoding scheme with the EncodingType property.

Note that you must set the EncodingType property when reading a Code 39 type of code.

Error correction

In most cases, the Code control can automatically detect the error correction scheme for read operations. If your code type supports more than one error correction scheme and does not support automatic detection of error correction scheme, you must specify the error correction scheme, using the ErrorCorrectionType property. If you specify a scheme that is not supported by your code type, you will get an ActiveMIL error.

Foreground color

During read operations, you must specify the foreground color of the code. Specify the foreground color with the ForegroundPixelValue property.

Cell size

In most cases, you should not have to specify the cell size when reading codes; the default setting is sufficient. However, for some 2D code types, read operations can be more robust if you specify the cell size. Specify the cell size as a range, setting the MinimumCellSize and MaximumCellSize properties. The Code control will search for codes with cells that fall within this range. If the cell size is not within the specified range, the code will not be found. Note that the Code control might have difficulty reading codes if the cell size is less than 2 pixels, even if the size is specified.

Note that Data Matrix and Maxicode read operations are more robust if you specify the cell size.

Number of cells

During read operations, the Code control can automatically detect the number of cells and will search for the specified 2D code type with any number of cells in the X- and Y-directions; the X-direction representing the number of columns, and the Y-direction representing the number of rows. You can specify the number of cells in the X- and Y-directions to increase the speed and robustness of the operation for 2D code types only. To specify the number of cells, set the NumberOfCellsX and/or NumberOfCellsY properties.

Note the following when using these properties and some code types:

  • Specifying the number of cells is mostly useful when reading PDF417 and Data Matrix code types; the Maxicode code type has a fixed number of cells.
  • For the PDF417 code type, the NumberOfCellsX property must equal 17c + 35, where c is the number of columns, and 35 represents the number of cells required for the start and stop patterns.

Search angle

By default, the Code control can read codes if they fall within the angular range of 0 ± 5°. It is strongly recommended that you change the search angle if the code appears rotated in the image, and you should increase the angular range if you are unsure of the code's exact orientation. Note, however, that when searching for 1D codes, the operation speed is slower for angular ranges greater than ± 5°. The angular range is the range of angles defined by:

(SearchAngle- SearchAngleDeltaNegative) to (SearchAngle + SearchAngleDeltaPositive)

In some cases, the image of the Data Matrix code might need to be flipped vertically or horizontally (ImageProcessing.Flip) to be read correctly. In these cases, the image is essentially a mirror image of the correct code.

Note the following when setting these properties for some code types:

  • If the code has bearer bars, you must specify the search angle.
  • For the PDF417, Maxicode, and Data Matrix codes, the angular range does not affect the speed of the operation.

Threshold value

In a read operation, the source image is internally binarized so as to separate the code from the background. By default, the threshold value is automatically chosen and is suitable in most cases. However, if you think that a different value would result in a better separation (and therefore in a more efficient operation), you can manually adjust the threshold level by setting the ImageThreshold property to another value.

Note that, if the background is both darker and lighter than the code, a simple binarization will not separate the code from the background. In this case, you should process the image before performing the read operation so that the background is either darker or lighter than the code.

Search speed

You can specify the speed at which to perform a read operation; the faster the speed, the less robust the operation. In general, the larger and more clearly defined the code, the better chance it has of being found at a speed higher than the default speed. Specify the search speed with the Speed property.

If you are having problems finding the code, you might want to search at a speed lower than the default.

String length

By default, the operation searches for a code that encodes a string of any size. However, if you know the exact size of the string, you might want to specify this value to increase the robustness of the operation using the TargetStringLength property.

Note that you must specify the string size when reading the BC412 code type. For a Postnet code, the string size must be 5, 9, or 11. For a Planet code, the string size must be 11.


Controlling write operations

User Guide:
Retrieving results

Reference:
Code.CalculateCodeSize
Code.CodeType
Code.EncodingType
Code.ErrorCorrectionType
Code.EscapeSequence
Code.ForegroundPixelValue
Code.TargetGrid.MinimumCellSize
Code.TargetGrid.NumberOfCellsX
Code.TargetGrid.NumberOfCellsY
Code.WriteCode

This section describes settings that you might have to change for write operations when using a Code control.

It is strongly recommended that you define your codes as clearly as possible, since it will facilitate any subsequent read operation. When writing a code, ensure that you are familiar with the requirements and restrictions of your chosen code type, otherwise the code will not be generated and an ActiveMIL error will result.

Size of destination image

The destination image of the write operation should be large enough to hold the encoded string. If you specify the code type with the CodeType property, and pass a string to the CalculateCodeSize method, the required minimum image size (in pixels) will be returned as one of the properties in the Results.VerificationGrade property.

String requirements

Some code types have specific requirements for the string to encode:

  • If the code type is a Codabar code, the string must be numeric but can contain the following ASCII marks: -, :, /, ., and +. In addition, the string must start and end with any of the following characters: a, b, c, or d.
  • If the code type is an Interleaved 2 of 5 code, the string must have an even number of characters.
  • If the code type is a Maxicode code, with the EncodingType property set to codeMode2 or codeMode3, the string must respect the structured carrier message format (the portion of the string that contains postal code, country code, and class of service information).
  • If the code type is a composite code, the string must be passed in the following format: 1D string|2D string, where | is used as the separator. In addition, the FNC1 separator is represented by the unprintable character with ASCII value 29.

Code type

You must specify the code type when performing a write operation. Specify the code type with the CodeType property.

Encoding scheme

You must specify the type of encoding scheme for write operations. Specify the encoding scheme with the EncodingType property. For a full list of supported encoding schemes, see the property description.

Error correction

You should specify the error correction scheme only when there is a particular scheme that you want to use. To do so, use the ErrorCorrectionType property. If you specify a scheme that is not supported by your code type, you will get an ActiveMIL error.

For the code types that support codeAnyErrorCorrection, the Code control can automatically select the best error correction scheme. For a full list of supported error correction types, see the property description.

Foreground color

For write operations, you must specify the foreground color of the code using the ForegroundPixelValue property.

Cell size

In most cases, you should not have to specify the cell size when writing codes; the default setting of the MinimumCellSize property is sufficient. In this case, the code is resized so as to just fit into the destination image of the operation, when possible. During a write operation, you can use the aforementioned property to force the cell size; the size is used to determine the size of the generated code. If you specify the required cell size with this property, you should ensure that the size of the destination image takes this into account.

The MaximumCellSize property is not used in write operations.

Number of cells

During write operations, the Code control can automatically detect the number of cells and will search for the specified 2D code type with any number of cells in the X- and Y-directions; the X-direction representing the number of columns, and the Y-direction representing the number of rows. To specify the number of cells, set both the TargetGrid.NumberOfCellsX and TargetGrid.NumberOfCellsY properties. If set to codeAny, the minimum number of cells possible will be used to perform the write operation. If you specify more cells than are necessary to generate the code, fill characters will automatically be added. If you specify fewer cells than are necessary to generate the code, you will get an ActiveMIL error.

Note the following when using these properties and some code types:

  • Specifying the number of cells is mostly useful when writing PDF417 and Data Matrix code types; the Maxicode code type has a fixed number of cells.
  • For the PDF417 code type, the TargetGrid.NumberOfCellsX property = 17c + 35, where c is the number of columns, and 35 represents the number of cells required for the start and stop patterns.

Encoding non-printable characters

If you need to encode non-printable characters such as a carriage return or tab for a write operation, use a code type that supports them, and set the EscapeSequence property to True. Then, when specifying the string to encode with the WriteCode method, specify the character in the string using its ASCII code; the non-printable character's ASCII code must be in hexadecimal format and be preceded by \x (for example, \x0D for ASCII 13, which is the carriage return).

If you want to encode the "\" character in escape sequence mode, type "\\".


Retrieving results

User Guide:
Controlling read operations
Controlling write operations

Reference:
Code.EscapeSequence
Code.Results
Code.Results.DecodedString
Code.Results.DecodedStringSize

The code results can be obtained using the properties of the Results property.

You can retrieve the string or the length of the string in escape sequence format. If the EscapeSequence property is set to True, and you use the Results.DecodedString property, the string will be returned with the non-printable characters in hexadecimal notation, preceded by \x. Similarly, if the EscapeSequence property is set to True, and you use the Results.DecodedStringSize property, the length of the decoded string will include all non-printable characters.


Verifying your code

See Also | Previous
User Guide:
Verifying your code

Verification is used to check the conditions under which a code is read, to check if codes you are reading meet your quality specifications, and to see if the codes you are writing meet your quality specifications. When you are reading or writing codes, it is important to know that the quality of the code is sufficient for your purposes.

You can use the VerifyCode method to verify the quality of 1D, 2D, or composite codes. You can pass the VerifyCode method the string of the code to be graded. If verifying a composite code, the following format for the string must be used: 1D string|2D string, where | is used as the separator.

Note that the VerifyCode method is not supported for Pharmacode, Postnet, and Planet code types.

You can find the results of the verify operation with the Results property.

There are specific results of the verify operation that you can get for your code, including the attributes of the scan reflectance profile. The scan reflectance profile of a code is a record of values measured along a line across the width of the code (line profile). There are nine attributes of a scan reflectance profile that are graded.

  • Decodability. This result is used to verify how near the scan reflectance profile is to a failing decode grade. This result is only available for all 1D, composite, RSS, and PDF417 code types.
  • Decode. This result is used to verify whether the code can be decoded.
  • Defects. This result is used to verify the uniformity of the reflectance of the code. The more spots or voids, the lower the grade. This result is only available for all 1D, composite, RSS, and PDF417 code types.
  • Edge determination. This result is used to verify whether the code has an appropriate number of bars and spaces. This result is only available for all 1D, composite, RSS, and PDF417 code types.
  • Minimum edge contrast. This result is used to verify whether the minimum contrast between bars and their adjacent spaces is acceptable. This result is only available for all 1D, composite, RSS, and PDF417 code types.
  • Minimum reflectance. This result is used to verify the reflectance of the darkest bar. Reflectance is a measure of the light reflected off the code. If the darkest bar has a reflectance less than half of the background, this attribute gets a failing grade. This result is only available for all 1D, composite, RSS, and PDF417 code types.
  • Modulation. This result is used to verify the intensity of the edge contrast as a proportion of code contrast. This result is only available for all 1D, composite, RSS, and PDF417 code types.
  • Start/stop pattern. This result is used to verify the quality of the set pattern that marks the end points of the code. This result is only available for the composite and PDF417 code types.
  • Symbol contrast. This result is used to verify how close a scanned character's pattern stands to its ideal pattern. This result is available for all codes except the MicroPDF417 code type.

Other aspects of the code that you can verify are:

  • Axial nonuniformity. This is a measure of how the spacing between sampling points along the width differs compared to height. This result is only available for the DataMatrix and Maxicode code types.
  • Overall symbol grade. This is a measure of the worst grade for any of the results for the code.
  • Print growth. This is a measure of the extent in which the boundaries of the black and white markings of the code are within their module's boundaries. This result is only available for the DataMatrix and Maxicode code types.
  • Row Address Pattern (RAP). This is a measure of how close a scanned code's RAP stands to its ideal pattern. This result is only available for the composite and MicroPDF417 code types.
  • Symbol-based grade. This is a measure of how close a scanned character's pattern stands to its ideal pattern. This result is only available for the composite, MicroPDF417, and PDF417 code types.
  • Unused Error Correction. This is a measure of the extent to which regional or spot damage in the code has eroded the reading safety margin that error correction provides. This result is only available for the DataMatrix and Maxicode code types.

Note that there are some properties that might have to be set before you verify a code. These are listed in the table below.

Control Type | Notes
ForegroundPixelValue | Specifies the foreground value of the image. This property is essential and the code will not be graded if the foreground value is not correctly set.
EncodingType | Specifies the type of encoding scheme for the code. This property has to be set for the code types where the encoding scheme cannot be set.
ErrorCorrectionType | Specifies the type of error correction for the code. This property has to be set for the code types where the encoding scheme cannot be set.
TargetStringLength | Specifies the size of the string for which to search. This property has to be specified for the BC412 code type.
TargetGrid.SearchAngle... | Specifies the nominal search angle and angular search range. These properties have to be specified if the code to read is not within the angular range of -5 to 5 degrees.


Images

Chapter 13: Images

This chapter describes images in detail. It shows you how to allocate and manage images, and how to restrict an operation to a portion of an image by using child images. It shows you how child images are stored, how to create a user-defined image, and how ActiveMIL defines the pixel reference position.

  • Images - overview
  • Properties of images
  • Owner system
  • Dimensions
  • Data type and depth
  • Controlling color image storage
    • Color images-storage format
    • RGB image formats
      • BGR24 packed
      • BGR32 packed
      • RGB15 packed
      • RGB16 packed
      • RGB planar
    • YUV image formats
      • YUV16 packed
      • YUV9 planar
      • YUV12 planar
      • YUV16 planar
      • YUV24Planar uncompressed image format
      • Child YUV images
  • Intended usage
  • Operating on a region of interest
    • Child images
    • Copying data between images
  • Accessing and mapping image data
    • Accessing image data directly
      • Address
      • Pitch
    • Mapping an image to user-allocated memory
    • Updating the display of a modified image
  • Loading and saving images
  • Using images with the Bayer color filter
    • Using ActiveMIL to convert the image
    • How the Bayer image gets converted
    • Correcting the white balance of your Bayer images
    • Performing gamma correction
  • Image overscan region


Images - overview

See Also | Next
User Guide:
Compressing images - overview
Displaying an image - overview
Grabbing an image - overview
Overview of image processing

The Image control represents an image. Images can hold image data to be displayed, manipulated, grabbed, and/or analyzed.

Image controls can have different data depths (1, 8, 16, or 32 bits per pixel). In addition, they can have up to three dimensions: an X-, Y-, and color band dimension. The X- and Y-dimensions specify the surface of the image. The color band dimension specifies the number of such surfaces required to represent color image data.

You can transfer data between an image and a user array, between images, and between an image and a file. You can also compress images in ActiveMIL. (ActiveMIL-Lite does not support JPEG2000 or JPEG compression. This is not a restriction under ActiveMIL.)

You can manipulate or control specific parts of an image by creating child images within it or by copying specific parts of it to another image.


Properties of images

User Guide:
Data type and depth
Dimensions
Intended usage
Owner system
Setting up controls

Reference:
Image.Allocate
Image.Free

Image controls have the following types of properties:

  • Owner system
  • Dimensions
  • Data type and depth
  • Intended usage

Image controls are created such that their initial properties match your default settings which have been specified at installation. For example, if your default camera is color, Image controls are created with three bands.

To change the default settings of an Image control, you should follow these steps:

  1. Use the ActiveMIL Property Defaults utility dialog box to specify default settings. An ActiveMIL Property Defaults dialog box is presented.
  2. Click on the Image property page.
  3. Specify whether the image's default properties will be monochrome or color, by specifying the number of bands: 1 for monochrome; 3 for color.
  4. Set the image resolution (width and height) by typing in the SizeX and SizeY edit fields.

Other image properties can be set by using specific image properties.

Note, many Image control properties cannot be set at run-time if the image is in the allocated state. In order to set such properties at run-time, you need to first free the resources of the Image control, using the Free method. You can then set the required properties. Note that Image controls lose their data (LUT data is not affected) when the Free method is called. Once you are finished setting properties, you must re-allocate the Image control, using the Allocate method.


Owner system

User Guide:
Properties of images

Reference:
Image.CanGrab
Image.OwnerSystem

The owner system of an image refers to the system on which its memory is allocated (or the system to which its memory is associated). The owner system of an image is specified by its OwnerSystem property.

The owner system of an Image control largely dictates which image properties are valid. For example, since a Corona-II system permits grabbing, an Image control on a Corona-II system can have its CanGrab property set to True. Since a Host system does not permit grabbing, an Image control on a Host system can only have its CanGrab property set to False.

Any operation involving more than one Image control will be performed by the most appropriate system that is associated with one of these Image controls. By default, if none of these systems is more appropriate than the Host system, then the Host system is used to perform the operation.


Dimensions

User Guide:
Data type and depth
Operating on a region of interest

Reference:
Image.NumberOfBands
Image.SizeX
Image.SizeY

Images can have up to three dimensions: an X-, Y-, and color band dimension. By default, grayscale images have an X- and Y-dimension. If your default camera is color, the image also has a color band dimension. The default settings for the camera have been specified at installation.

To change an image's X-dimension (width) or Y-dimension (height), set its SizeX or SizeY property, respectively.

To change the number of bands in an Image control, set its NumberOfBands property.


Data type and depth

User Guide:
Controlling color image storage
Copying data between images
Dimensions
Intended usage
Properties of images
Using images with the Bayer color filter

Reference:
Image.DataDepth
Image.DataType

The data depth of an image indicates the number of bits per pixel in the image (1, 8, 16, or 32). The data type of an image indicates how its data is internally represented (that is, whether it is considered signed, unsigned, or floating-point). Supported combinations are: 1-bit packed binary; 8-, 16-, and 32-bit integer (signed and unsigned); and 32-bit floating-point.

By default, images are allocated as 8-bit unsigned. To change an image's data depth, use its DataDepth property. To change an image's data type, use its DataType property.

Note the following:

  • In general, the fewer bytes per pixel in the image(s) of an operation, the faster the operation. Therefore, when using integer images, use 8 bits per pixel when possible, 16 bits if necessary, and 32 bits as a last resort. When you need non-integer values, extra precision, or a greater dynamic range, you can use floating-point images.
  • The packed binary data format represents each pixel by a single bit, in a state of 0 or 1. Therefore, 8 pixels can be packed in a single byte, that is, in a format eight times smaller than an 8-bit image.
  • Each band of a multi-band (color) image is of the same data type and data depth.
  • If a method only supports images of certain types or depths, this is explicitly stated in its description.
  • For auxiliary displays, only 8-bit images (with up to three bands) can be displayed.


Controlling color image storage

User Guide:
Accessing and mapping image data
Child images
Data type and depth
Loading and saving images
Using images with the Bayer color filter

Reference:
Image.CanDisplay
Image.CanGrab
Image.CanProcess
Image.Format

Color images-storage format

A color image's internal representation can be either in a planar or packed format. When allocating an image, if its Format property is set to a planar format, the pixels are stored in planes (for example, RRR GGG BBB); if its Format property is set to a packed format, each pixel is stored as one unit containing all its components (for example, RGB RGB RGB).

ActiveMIL automatically selects the most appropriate format, according to the specified intended usage attribute. If an image is allocated in one format, and a general processing method requiring another format is called, the method will automatically convert the data to the required format and re-convert it back to its original format upon completion. To change an image's default internal storage format (imFormatDefault), set the image's Format property to the required internal storage format constant. Note that it might be slower to process images with a packed format attribute.

In general, packed formats are mostly used for display purposes; when the CanDisplay property is set to True, the default internal representation is usually packed. This configuration allows for faster transfers to display sections that handle packed data (for example, VGA). However, if the display section of your board has dedicated red, green, and blue surfaces, the image is allocated in planar format.

Planar formats are generally preferred for processing. Here, each pixel is stored as three component planes (for example, RRR, GGG, BBB). Processing is done on each of the components separately.

When allocating an image with more than one property, for example, CanDisplay and CanProcess, the image's internal storage requirements for the display will take precedence over other properties.

See the remarks in the Image control's Format property description, to determine which formats are supported by your particular Matrox board.

RGB image formats

By default, ActiveMIL allocates color images in an RGB color format. The pixels are internally stored in little-endian order, that is, they are stored in memory from their least- to most-significant bytes. The definitions of the RGB color data formats that are available are listed below. The corresponding ActiveMIL constant is shown in brackets beside the common format name.

  • BGR24 packed (imBGR24Packed)
  • BGR32 packed (imBGR32Packed)
  • RGB15 packed (imRGB15Packed)
  • RGB16 packed (imRGB16Packed)
  • RGB planar (for example, imRGB24Planar)

BGR24 packed

BGR24 packed (imBGR24Packed) is a format whereby each pixel is internally stored as three consecutive bytes in little-endian order, that is:

BGR32 packed

BGR32 packed (imBGR32Packed) is a format whereby each pixel is internally stored as four consecutive bytes, in little-endian order. The most-significant byte is a "don't care" byte, as shown below:

RGB15 packed

RGB15 packed (imRGB15Packed) is a format whereby each pixel is internally stored as a 16-bit word with a 5-bit blue value (least significant), a 5-bit green value, a 5-bit red value, and a "don't care" bit (most significant), in little-endian order, as shown below. Note that when accessing an imRGB15Packed image as a 3-band 8-bit image, the least significant bits of each band are set to 0.

RGB16 packed

RGB16 packed (imRGB16Packed) is a format whereby each pixel is internally stored as a 16-bit word with a 5-bit blue value (least significant), a 6-bit green value, and a 5-bit red value (most significant), in little-endian order, as shown below. Note that when accessing an imRGB16Packed image as a 3-band 8-bit image, the least significant bits of each band are set to 0.

RGB planar

RGB planar (for example, imRGB24Planar) are formats whereby the color components of all the pixels are stored contiguously: (RRR..., BBB..., GGG...).

YUV image formats

It is possible to set the Image control's Format property to a YUV color format. YUV is a format in which Y is the grayscale component (luminance) and U and V are the color components. ActiveMIL supports grabbing, loading, or saving images in a YUV color format.

Although any general processing method can be performed on images in YUV format, allocating them for processing purposes is not recommended because ActiveMIL is configured to process RGB color data only. ActiveMIL will automatically convert YUV image data to RGB for all general processing methods (including conversion for display), and re-convert it to YUV upon completion.

All YUV formats are supported even by the Host system. However, only some systems support grabbing into YUV formatted images. See the remarks of the Format method of the Image control to determine if grabbing to YUV images is supported by your system.

YUV images must be allocated as 3-band 8-bit images, however, the actual number of bits per pixel will differ depending on the YUV format selected.

The supported YUV formats are:

  • YUV16 Packed
  • YUV9 Planar
  • YUV12 Planar
  • YUV16 Planar

YUV16 packed

YUV16 Packed or YUV 4:2:2 (imYUV16Packed) is an interleaved data format. Although each pixel has a corresponding one byte Y (luminance component), each pair of pixels share the same one byte U (chrominance U) and the same one byte V (chrominance V). Since a pair (two pixels) is represented by 4 bytes, each pixel has an average of 16 bits per pixel.

The YUV16 packed data format has two available formats: YUYV and UYVY. The only difference between these two YUV formats is the ordering of data in the image. Certain digitizer boards grab data in exclusively YUYV or UYVY packed data format. Note that, for display, certain operations are optimized to handle the YUYV format; for example, displaying a decompressed buffer

When you allocate an imYUV16Packed image, ActiveMIL allocates the image in the format that is most suitable for the selected platform and the specified image attributes. You can, however, force a format using the imYUV16YUYV or imYUV16UYVY ImFormatConstants. When the image's CanGrab property is set to True, forcing an inappropriate format generates an error. When the image's CanDisplay property is set to True, if you force the image in the other YUV format, then CPU intervention is required to perform the automatic conversion.

See the remarks for the required Image control's Format property description for data formats supported by your board.

YUYV
UYVY

YUV9 planar

YUV9 Planar (imYUV9Planar) is a planar format whose components have a depth of one byte but are not of the same size. Although each pixel has a corresponding 1 byte Y (luminance) component, each block of 16 pixels share the same one byte of U (chrominance U) and the same one byte of V (chrominance V). Since the 16 pixels are represented by 18 bytes, each pixel has an average 9 bits. For example, a block of 16 pixels has the following: 16 bytes Y and 1 byte each of U and V.

YUV12 planar

YUV12 Planar (imYUV12Planar) is a planar format whose components have a depth of one byte but are not of the same size. Although each pixel has a corresponding 1 byte Y (luminance) component, each block of 4 pixels share the same one byte of U (chrominance U) and the same one byte of V (chrominance V). Since the 16 pixels are represented by 24 bytes, each pixel has an average of 12 bits. For example, a block of 16 pixels has the following: 16 bytes Y and 4 bytes each of U and V.

YUV16 planar

YUV16 Planar (imYUV16Planar) is a planar format whose components have a depth of one byte but are not of the same size. Although each pixel has a corresponding 1 byte Y (luminance) component, each block of 2 pixels share the same 1 byte of U (chrominance U) and the same 1 byte of V (chrominance V). Since the 16 pixels are represented by 32 bytes, each pixel has an average 16 bits. For example, a block of 16 pixels has the following: 16 bytes Y and 8 bytes each of U and V.

YUV24Planar uncompressed image format

YUV24 planar (imYUV24Planar) is an uncompressed planar format whose components have a depth of one byte and are of equal size. Each pixel has a corresponding 1 byte Y (luminance) component, 1 byte U component (chrominance U), and 1 byte V component (chrominance V). Since the 16 pixels are represented by 48 bytes, each pixel has an average of 24 bits. For example, a block of 16 pixels has the following: 16 bytes Y and 16 bytes each of U and V.

Child YUV images

You can create child images from YUV images in the same way as RGB child images. When creating YUV child images, ActiveMIL will keep the proportions of the U and V bands with respect to the Y band. For example, if your YUV9 Planar Y band is a size of 256 x 256 pixels, the U and V bands will be 1/4 the size of the Y band in each dimension (width and height): 64 x 64 pixels. A 64 x 64 pixel U or V band is 1/16 the size of the Y band. If a child image is 16 x 16 pixels, then the U and V bands will be 4 x 4 pixels. In other words, the 4 x 4 U and V bands (16 pixels) are 1/16 the size of the Y band (256 pixels).


Intended usage

User Guide:
Displaying an image - overview
Grabbing an image - overview
Integrating MIL functions into your ActiveMIL code
Overview of image processing

Reference:
Image.CanDisplay
Image.CanGrab
Image.CanProcess

An Image control can be used to hold image data and can be grabbed into, displayed, and processed.

  • To be able to grab into an image, its CanGrab property must be set to True. If your system supports acquisition, an image's CanGrab property is set to True by default.
  • To be able to display an image, its CanDisplay property must be set to True. An image's CanDisplay property is set to True by default.
  • To be able to process an image, its CanProcess property must be set to True. An Image control's CanProcess property is set to True by default. With ActiveMIL-Lite, the required processing functionality must be added by integrating the appropriate MIL processing functions into your ActiveMIL-Lite code, that is, by entering native mode.

In general, you should only set the required Can... properties of an image to True. This is because, depending on what your system supports, ActiveMIL optimizes images for speed and/or memory usage.

Note that, if you try to use a image in a situation that is not appropriate to its usage, an error message will be generated and the operation will not be performed.


Operating on a region of interest

User Guide:
Accessing and mapping image data
Data type and depth
Dimensions
Displaying multiple images
Operating on a region of interest
Types of displays

Reference:
Image.Band
Image.ChildRegion
Image.ChildRegion.OffsetX
Image.ChildRegion.OffsetY
Image.ChildRegion.SizeX
Image.ChildRegion.SizeY
Image.Copy
Image.CopyClip
Image.CopyConditional
Image.CopyMask
Image.CopyRegion
Image.ParentImage

You can manipulate or control specific parts of an image by creating a child image within it or by copying specific parts of it to another image.

Child images

A child image is a subset (or region of interest) of a given image (known as the parent image). A child image can be used in the same way as any other image. Since a child image occupies a specific area of the parent image, changes made to the child image directly affect the parent image and vice versa. Any returned coordinates from an operation that uses a child image are relative to those of the child, not the parent.

Child images can be used to handle several images simultaneously, in contexts where normally only one image can be handled. For example, only one image can be displayed in a window at a time. However, by using a large image with several child images, you can view all the child images by viewing the parent image. Therefore, you could, for example, display the source and destination image of an operation by creating two non-overlapping child images within a large display image, then using one child image as your source and the other as your destination. For an example of using child images to view several images simultaneously, see MImageColor. Note that, for windowed displays, an easier way to display multiple images is to associate the images with different Display controls. However, for auxiliary displays, you can have only one Display control, so you must use child images in order to display multiple images.

When the parent image is multi-band, the child image can occupy one or all bands, as illustrated below.

To create a child image:

  1. Create a new Image control.
  2. Set its ParentImage property to the name of its parent image.
  3. If you want to create a child image from a single band of a multi-band image, set its Band property to the required band. By default, a child image has the same number of bands as its parent.
  4. Specify the width and height of the child image, as well as the horizontal and vertical offset from the parent's top-left corner to the child's top-left corner. The ChildRegion property contains the properties (SizeX, SizeY, OffsetX, and OffsetY) required to do so. In order to set these properties, use the General page of the image's property pages. You can access this page by right-clicking on the Image control and then selecting Properties from the presented menu or by using the property browser. Since a child image cannot exceed the bounds of its parent in either the X- or Y-direction, an error message will be generated if these properties are not correctly set.

    5. You can also specify the size and position of the child image interactively. To do so, click on Positions from the General page; this brings up the Interactive Preview window. The box in this window is used to define the size and position of the child image; drag and resize the box as required.

    To see the parent image in the background while selecting a child image, you can load the image into the Interactive Preview window, using the Load image button. You can also zoom in and zoom out on the window, using the Zoom In and Zoom Out buttons, respectively. If necessary, you can change the color that is used to display the box; use the drop-down list box next to the zoom buttons.

Copying data between images

You can copy an entire image to another image, using the Copy method (omit the method's Band parameter).

You can also copy specific areas or bits of an image to another image. Specifically, you can:

  • Copy a single band of a multi-band image to another image, using the Copy method (use the method's Band parameter to specify the band). This allows you to operate on a single band of a multi-band image.
  • Copy an image to another image at the specified offset, using the CopyClip method. Data that falls outside of the destination image will be automatically clipped.
  • Copy a rectangular region of an image to another image, using the CopyRegion method. If the source image is single-band and the destination image is multi-band, you must specify which band of the destination image to overwrite. If the source image is multi-band and the destination image is single-band, you must specify which band of the source image to copy. If both images are multi-band, you can copy from one band to another band, or you can copy all bands.
  • Copy an image to another image based on a condition image, using the CopyConditional method. Source pixels are copied to the destination image only if corresponding pixels in the condition image satisfy a specified condition. This allows you to copy non-rectangular regions of an image to another image.
  • Copy specific non-consecutive bits to another image based on a mask, using the CopyMask method. Source bits overwrite only destination bits that correspond to non-zero bits in the mask.

If the source image depth is greater than that of the destination, the most significant bits are truncated when the data is copied to the destination. If the source and destination images are signed and the destination depth is greater than that of the source, the source data is sign-extended when it is copied to the destination.


Accessing and mapping image data

User Guide:
Child images
Operating on a region of interest

Reference:
Image.AssignMemory
Image.CanDisplay
Image.CanGrab
Image.Get
Image.HostAddress
Image.Modified
Image.PhysicalAddress
Image.PitchByte

Accessing image data directly

If needed, you can access the data of an ActiveMIL image directly. This can be used to increase efficiency and improve performance of a custom algorithm. For instance, if you want to calculate the average value of the pixels of your image, you could create a custom algorithm. The algorithm could be applied directly to the image without first having to copy the contents of the ActiveMIL image into a user-allocated array using the Image control's Get method.

To access image data directly, you must know its address and pitch.

Address

The address of a parent or child image can be obtained using the Image control's HostAddress and PhysicalAddress properties. The HostAddress property will return the logical address of the image; while the PhysicalAddress property will return the physical address of the image. In either case, the address will be the top-left-most pixel in the image. To determine the address of the following row, you need to know the pitch and depth of the image.

Pitch

The pitch of an image is the number of units between the beginnings of any two adjacent lines of an image and can be measured in pixels or bytes. Note that in some instances, the pitch in bytes will be more accurate than in pixels. If the last pixel falls outside of a 32-bit boundary (required by Windows), the start of the next row will be located at the beginning of the next 32-bit boundary; this process is called internal padding. When measuring the pitch in pixels, the padding can be counted as "extra" pixels, depending on the depth of the pixels. This will result in an inaccurate pitch. In ActiveMIL, you can determine an image's pitch in bytes using its PitchByte property; no property is provided for the pitch in pixels.

Note that with an image with a DIB storage format, the pitch must be equal to the width. If the pitch is in bytes, the pitch must be the next multiple of 4 that is larger or equal to: (width * the number of bytes per pixel).

Mapping an image to user-allocated memory

You can create a monochrome or color image from the memory at a specified location by calling the Image control's AssignMemory method. In this case, ActiveMIL does not allocate any memory; it uses the memory that you provide.

When creating an image with the Image control's AssignMemory method, the memory that will store the image data must be passed as a parameter to the method. For monochrome or packed color images, a pointer to the memory or a user-defined array should be provided. For planar color images, an array containing pointers to the memory of each band, or a user-defined array is required. Refer to the remarks section of the AssignMemory method for more information.

The address(es) can be either logical or physical. If you want to use the image for grabbing, the address(es) must be physical (an image with its CanGrab property set to True must be allocated in physically contiguous and always present memory, that is, non-paged). The Image control's AssignMemory method must be used with caution because, when using physical addresses, this method provides direct manipulation of any of your PC's memory mapped devices; when using logical addresses, memory protection errors could result.

You can use an Image control's HostAddress and PhysicalAddress properties to determine its Host's logical address or physical address, respectively. Note that the physical address is not necessarily an address in Host memory. It could be an address in on-board memory.

There are several instances when memory mapping is useful. A particularly useful instance is when processing and displaying an interlaced grab in a time-critical application. In this case, you could use an Image control with its CanDisplay property set to True to store and display the grabbed data. Then, to process each field as it is grabbed, you could use an image that is mapped to the odd field of the displayable image (Image 1) and an image that is mapped to the even field (Image 2).

Note that the following example does not work with an image with a DIB storage format because, if the pitch is in pixels, the pitch must be equal to the width. If the pitch is in bytes, the pitch must be the next multiple of 4 that is larger or equal to: (width * the number of bytes per pixel).

Create images 1 and 2 as follows:

Image 1: (Odd field)

Size = 640 x 240 (i.e., half height)

Pitch = 1280 (i.e., to skip to the next field)

Address = Address A (i.e., first pixel of the first row)

Image 2: (Even field)

Size = 640 x 240 (i.e., half height)

Pitch = 1280 (i.e., to skip to the next field)

Address = Address B (i.e., first pixel of the second row)

Updating the display of a modified image

In general, ActiveMIL automatically issues a display update after a displayed image has been modified. However, if an image selected on the display is modified using a mapped image, its display is not updated until you notify it of the change using the Image control's Modified method.


Loading and saving images

User Guide:
Decompression

Reference:
AVIFileHandler.Close
AVIFileHandler.Open
Image.FileFormat
Image.Load
Image.Save

To transfer data from a file to an image, use the Load method. To transfer data from an image to a file, use the Save method.

When transferring from a file to an image:

  • Supported file formats include TIFF, BMP, MIL, JPEG and RAW.
  • You can have the image's data type and dimensions adjusted to match those of the image on file, by setting the AdjustImage parameter to True. If the AdjustImage parameter is set to False, the image must be deep enough and big enough to hold the data; otherwise, some data will be clipped.
  • The image can be already allocated or not. If it is already allocated, any data in the image will be overwritten.

When transferring from an image to a file, the FileFormat property determines the way in which the data is saved. When saving a color image, you can save the image in packed or planar format. In packed format, the color components of each pixel are interleaved. In planar format, the color components of each pixel are stored in separate bands. When saving in a packed or planar format, you can use the MIL or TIFF format. These formats are the same, except that the MIL format contains extra information in the comment field of the file's header.

When saving an image, you can save the image in a lossy, lossy interlaced, lossless, or lossless interlaced compressed JPEG file format, or lossy or lossless JPEG2000 format.


Using images with the Bayer color filter

User Guide:
Performing gamma correction
White balancing your Bayer images

Reference:
Digitizer.Grab
Image.Bayer
Image.Load
Image.WhiteBalanceCoefficients

Cameras that feature a Bayer color filter can be used with ActiveMIL to provide a cost-effective method for grabbing color images: the camera grabs a single-band color-encoded image, and then you can convert it to a multi-band color image, using the Bayer method of the Image control. Bayer images are distinct from standard single-band images because of the color information contained in their pixels, which is extracted by the Bayer method.

When grabbing from these cameras, each pixel quantifies only one of the color components of the image in the camera's field of view at the corresponding location. Within a group of 2x2 pixels, there are two pixels containing color information for the green component, and one pixel for each the red and blue components; Bayer images contain more green pixels because the human eye is more sensitive to this color. The pixels are arranged in the following pattern: green pixels are always diagonal to each other, as are the red pixels to blue pixels.

The Bayer method can also perform white balance correction and gamma correction on the Bayer image during conversion. White balance correction adjusts an image for color variations that were introduced by the lighting conditions when the image was grabbed; the Bayer method converts pixels that represent white so they appear as close to white as possible, and adjusts other pixels accordingly. Gamma correction changes the overall brightness and color saturation of an image so that the midtones appear more realistic on your display device; gamma correction effectively adjusts an image to compensate for the non-linear relationship between a pixel's value and its displayed intensity on your display device. White balance and gamma correction are discussed in greater detail later in this section.

Note that if the scale of the image is changed (using the digitizer's ScaleX and ScaleY properties) to a value other than 1 prior to grabbing a Bayer image, the Bayer image will not be converted property; some of the Bayer pattern is lost during the scaling process, rendering color recovery impossible.

Using ActiveMIL to convert the image

To convert a Bayer image, you need two Image controls: one to contain the Bayer image (the source image) and one to contain the converted image (the target image). The steps below describe, in general, how to convert a Bayer image using ActiveMIL:

  1. Determine the white balance correction coefficients (optional). For information on how to calculate the white balance coefficients, see the subsection, Correcting the white balance of your Bayer images.
  2. Determine the gamma correction exponents (optional). For information on how to calculate the exponents, see the subsection, Performing gamma correction.
  3. Grab or load a Bayer image into your source image.
  4. Call the Bayer method of your target image, passing the source image from the previous step.

How the Bayer image gets converted

Bayer images are arranged in groups of 2x2 pixels. Each group contains one blue pixel, one red pixel, and two green pixels, the values of which are used in calculating the corresponding bands of the destination pixels. Your camera will grab an image with one of the following four patterns:

You must specify the pattern that is used by your camera when calling the Bayer method. Since the green pixels are always diagonal to each other, specifying the starting two pixels of the pattern defines the pattern uniquely. Consult your camera's documentation or contact the manufacturer if you are unsure; the Bayer image will not be converted properly if you specify the wrong pattern. If you cannot obtain information regarding the pattern of your camera, try all of the Bayer method's supported patterns to find the correct one.

The value of a source pixel is used in the corresponding band of its destination pixel. The two remaining color bands use the average value of the source pixel's corresponding neighbors. If the source pixel is green, then the average value for the remaining two bands (red and blue) is based on two neighboring pixels.

If the source pixel is either red or blue, the average value for the remaining two bands is based on four neighboring pixels.

Note that if the source pixel is on an edge of the image, ActiveMIL will use as many neighbors as possible when determining the average pixel value for the remaining bands.

Regardless of the target image format, ActiveMIL converts the Bayer image first to RGB, and then to the format of the target image.

Correcting the white balance of your Bayer images

Sometimes grabbed images appear with "the wrong colors". This is due primarily to color distortions introduced by the light source or lighting conditions. Such distortions can be corrected by performing a white balance correction on the image.

On the premise that white pixels should contain no chrominance, white balance correction applies a coefficient to each band of the image so that "white" pixels contain no chrominance. For RGB images, this means that a white pixel's value for all 3 bands is equal. After white balance correction, pixels that are white appear white (or a shade of gray), and the other pixels also appear with the correct colors. The result is an image that more accurately reflects the colors of the object that was grabbed.

The steps below show how to white balance Bayer images using the Bayer method:

  1. Grab an image that should be a uniform shade of gray. To do so, hold a flat, smooth, non-reflective object that is a uniform shade of gray in front of a camera so that the object occupies the camera's entire field of view. The shade of gray should not be too close to white (too saturated) because if one or more color bands is actually being clipped to maximum saturation, the unbalance between the bands cannot be observed.

    2. Ensure that the image is grabbed under the same lighting conditions as subsequent source images. Note, it is unlikely that you will be able to grab an image whose pixels have the same red, green, and blue value.

  2. Call the Bayer method of the target Image control, using the gray image as the source image, and setting the WhiteBalance parameter to imCalculate. This call calculates the three coefficients required to correct the white balance of the specified source image. These coefficients are saved in the WhiteBalanceCoefficients property of the target Image control.

    3. Alternatively, you could set the white balance coefficient for each band of your image with a custom values, using the WhiteBalanceCoefficients property of the target Image control.

  3. Grab the image required for the Bayer conversion into your source Image control.
  4. Call the Bayer method again, using the new source image and set the WhiteBalance parameter to imUse.

When the Bayer method calculates the white balance correction coefficients, the three coefficients, a, b, and c, are calculated and saved in the WhiteBalanceCoefficients property. These coefficients are for a given lighting condition, and calculated such that given an image of a flat gray surface in that lighting:

where R, G, and B with macrons (¯) are the average values of the intermediate red, green, and blue color components, respectively. When source images are converted, the intermediate color components of each pixel are multiplied by their corresponding coefficient. Regardless of the destination buffer format, white balance correction is performed on the intermediate RGB values recovered by the filtering process (before being converted into the proper destination format).

Performing gamma correction

Sometimes the colors of grabbed images can appear to be over-saturated or too dark due to the display device used. Most display devices receive your image as an analog video signal. The voltage range of the active portion of the video signal is divided linearly among the possible pixel value range. However, the intensity to voltage response ratio of the display device is roughly a power function (intensity = voltagegamma), whereby the exponent is rarely 1 (for many display devices, it is 2.5). This means that there is typically a non-linear relationship between a pixel's value and its displayed intensity on your display device. Therefore, images can look either bleached out or too dark.

The exponent of the power function of each display device is referred to as the gamma of the display device. You can compensate for the gamma of your display device by performing gamma correction on your images. That is, you can apply a power function that uses a 1/gamma exponent, on each band of your images. The exponents must be positive, are typically in the range of 0.0 to 1.0, and are most commonly 1/2.5 = 0.4.

When you call the Bayer method, set the WhiteBalance parameter to imUseWithGammaCorrection to perform gamma correction on the intermediate RGB values recovered by the filtering process (just after performing the white balance correction operation and before converting the image into the proper destination format). Specify the exponents to use for the gamma correction, using the GammaCorrectionExponents property.


Image overscan region

See Also | Previous
User Guide:
Custom spatial filters

Reference:
ImageProcessing.Convert
ImageProcessing.Convolve
ImageProcessing.Destination1
ImageProcessing.EdgeDetect
ImageProcessing.EdgeDetect
ImageProcessing.EdgeDetect1
ImageProcessing.EdgeDetect2
ImageProcessing.HorizontalEdge
ImageProcessing.Kernels
ImageProcessing.LaplacianEdge1
ImageProcessing.LaplacianEdge2
ImageProcessing.Rank
ImageProcessing.Sharpen1
ImageProcessing.Sharpen2
ImageProcessing.Smooth
ImageProcessing.VerticalEdge

Image overscan pixels are pixels added around an image. ActiveMIL neighborhood operations, such as the ImageProcessing.Convolve method, use these pixels to perform operations that go beyond the limits of the image. For example, if the HorizontalEdge method is called, the FIR filter is applied on 3 by 3 neighborhoods. In such a case, the neighborhoods of the image border pixels would be incomplete, and overscan pixels might be used. ActiveMIL recursive neighborhood operations, such as Deriche and Shen methods, apply IIR filters which ignore the values of overscan pixels.

To set the value of overscan pixels, use the StructuringElements.Item(StructuringElementIndex).OverscanValue or Kernels.Item(KernelIndex).OverscanValue property. However, some ActiveMIL neighborhood operations with predefined kernels or structuring elements do not require that you set the value of overscan pixels. For example, if you call the Erode method, the overscan pixels are automatically set to the highest possible image value, which will produce the most accurate possible results for the image border pixels.

The number of overscan pixels added around an image during allocation is set using the AllocationOverscanSize property of the System control. The set size represents the number of 1-pixel high/wide rows/columns added to each side of the image. For example, in the image above, the size of the overscan is 2. Each image allocated on a particular system will have the same overscan size. You can inquire the size of the overscan using either the AllocationOverscanSize property of the Image control or AllocationOverscanSize property of the System control..

When the results of operations performed on image border pixels are not important, and you want to speed up your application, you can set the StructuringElements.Item(StructuringElementIndex).OverscanType or Kernels.Item(KernelIndex).OverscanType property to impDisableOverscanType to disable the use of overscan pixels. Also, both the Convolve method and the EdgeDetect method allow you to disable the use of overscan pixels for predefined filters.


Look-up tables

Chapter 14: Look-up tables

This chapter describes lookup tables (LUTs). It shows you how to generate and modify them and briefly discusses how to use them.

  • Overview of LUTs
  • Setting up your LUT
  • Generating data into LUTs
    • Generating data directly into the LUT


Overview of LUTs

See Also | Next
User Guide:
LUTs and the digitizer
LUTs and the display

Reference:
Digitizer.LUT
Digitizer.LUTEnabled
Display.LUT
Display.LUTEnabled

Lookup tables (LUTs) are memory locations that are used to map data to pre-calculated values.

If supported by your digitizer, grabbed data is mapped through a physical LUT. By default, the physical LUT is transparent; that is, it is loaded with data so that each input pixel is mapped to its original value. You can load the physical LUT with your own custom LUT data if, for example, you want to correct or precondition grabbed data. To do so, use the digitizer's LUT property to set up the LUT data and then enable the use of this data using the digitizer's LUTEnabled property. For more information, see LUTs and the digitizer.

If supported by your display, displayed data is mapped through a LUT. By default, a transparent LUT is used, that is, each input pixel is mapped to its original value. You can use your own LUT if, for example, you want to adjust the color contrast of the displayed image. To do so, you can either set the properties of the Display control's LUT property and enable the display's LUTEnabled property, or set the properties of the Image control's LUT property and then enable the control's LUTEnabled property. For more information, see LUTs and the display.

When a LUT is specified for an image, the LUT is sometimes called a palette. If the image is saved to a file, the LUT data is saved with the image.

On an ImageProcessing control, you can perform complex operations (such as scaling and logarithms) on an image, using the LUTMap method. This method maps the source image through a specified lookup table (LUT) and stores results in the specified destination image.


Setting up your LUT

User Guide:
Child images
Generating data into LUTs
LUTs and the digitizer
LUTs and the display

Reference:
Get
NumberOfBands
Size

The number of entries in the LUT should be at least equal to the number of possible pixel values in the data being mapped. For example, if 8-bit data is being mapped, the LUT should have at least 2 to the power of 8 = 256 entries. The first entry in the LUT is used to map the lowest pixel value (0 for 8-bit unsigned data, -128 for 8-bit signed data); the next entry is used to map the next highest value. To change the number of entries in a LUT, use the Size property.

A LUT can have one band or multiple bands.

  • If the LUT has one band and is used to map a multi-band image, the same LUT data will be used to map all bands of the image.
  • If the LUT has multiple bands and is used to map a multi-band image, each band will be used to map the corresponding band of the image.

To change the number of bands in a LUT, use the NumberOfBands property.

You can transfer data between a LUT and a user array, between a LUT and another LUT, and between a LUT and a file. Use the LUT.Put and LUT.Get methods.


Generating data into LUTs

See Also | Previous
User Guide:
LUTs and the digitizer
LUTs and the display
Setting up your LUT

Reference:
GenerateColor
GenerateFunction
GenerateRamp
Put

After you set up a LUT for your control, you need to generate data into it. You can do so in one of two ways:

  • Generate the data and then transfer it to the LUT using the LUT.Put method.
  • Generate the data directly into the LUT.

Generating data directly into the LUT

You can generate data directly into the LUT using the LUT.GenerateFunction, LUT.GenerateRamp, or LUT.GenerateColor method. In addition, for a LUT of the ImageProcessing control, you can also use the HistogramEqualize method of the control to generate data into the LUT for intensity correction.

The LUT.GenerateFunction method uses a specified function to generate data into the LUT. The StartIndex and EndIndex parameters of this method specify the first and last entries of the LUT in which to generate the data. The a, b, c, and StartXValue parameters specify the constants required by the function.

As an example of how the LUT.GenerateFunction method fills the LUT, assume that you want to use the lutExp function, ab x + c, to map 8-bit data. The LUT would need 2 to the power of 8 = 256 entries, so StartIndex and EndIndex would be 0 and 255, respectively. The LUT will then be filled in this way:

LUT(0) = ab(StartXValue) + c

LUT(1) = ab(StartXValue + 1) + c

...

LUT(255) = ab(StartXValue + 255) + c

The LUT.GenerateRamp method generates "ramp" or "inverse ramp" data into the LUT. In other words, it generates data of the sort:

StartValue, StartValue + k, StartValue+ 2k, StartValue+ 3k, ..., EndValue

where k = (EndValue- StartValue) / (EndIndex- StartIndex). StartIndex and EndIndex specify the first and last entries of the LUT in which to generate the data. If k is positive (that is, EndValue is greater than StartValue), the method is said to generate "ramp" data. If k is negative (that is, EndValue is less than StartValue), the method is said to generate "inverse ramp" data. If k = 0 (that is, StartValue is equal to EndValue), the method will fill the LUT with the given StartValue.

As an example of how the LUT.GenerateRamp method fills the LUT, assume that you want to map 8-bit data using an inverse ramp. StartIndex and EndIndex would be 0 and 255, respectively. Assume that StartValue = 255 and EndValue = 0. The input data will then be mapped in this way:

Whenever the data generated by the LUT.GenerateRamp method is plotted, the result is a straight line.

Note that the LUT.GenerateColor method allows you to set a range of LUT entries to a specified color.


Display

Chapter 15: Display

This chapter discusses the display of images, in detail. It shows you how to display several images simultaneously, and discusses some of the special effects that can be applied to a displayable image.

  • Displaying an image - overview
  • Types of displays
    • Windowed display
    • Auxiliary display
      • Video output format
      • Matrox Millennium G400, G450, G550
  • The display window (for windowed displays)
    • Displaying in the Display control's window
    • Displaying in a user-defined window
    • Displaying in an external window
  • Attaching scroll bars to the display window
    • The Display control's window and the user-defined window
    • The external window
  • Removing an image from the display
  • Displaying multiple images
    • Display number
  • Screen tearing
  • Panning and zooming
    • Panning
    • Zooming
  • Displaying non 8-bit images
    • Bit-shift mode
    • Auto-scale mode
    • Multi-bytes mode
    • Transparent mode
  • Annotating the displayed image non-destructively
    • Overlay-display mechanism
      • Number of bands
      • Overlay image behavior
      • CPU-assisted overlay
      • Transparency (Keying)
    • Using GDI annotations (only for Visual C++)
    • Visual Basic annotations
      • Overlay flickering
  • LUTs and the display
    • Palettes and output LUTs for windowed display (256-color)
    • Reference material: Windows palettes and physical output LUTs
      • Color images
      • Palettes and physical output LUTs
      • Monochrome images
    • Default palette settings
      • Why change the default LUT values
    • Changing the default LUT values
      • A 1-band custom LUT
      • A 3-band custom LUT
      • The behavior of physical output LUTs
  • CPU-assisted display
    • Overlay flickering
    • Grabbing continuously
    • Avoiding CPU-assisted displays


Displaying an image - overview

See Also | Next
User Guide:
Displaying non 8-bit images
The display window

Reference:
Display.BackColor
Display.Image
Display.OwnerSystem
Image.CanDisplay
Image.OwnerSystem

Whether or not your imaging board has a display section, ActiveMIL can display images. ActiveMIL will use the most appropriate graphics controller in your computer for display purposes. If your imaging board has a display section, and it is available, ActiveMIL will typically use it for display purposes.

You can display images with a windowed border (or on a form) on the Windows desktop screen(s); this is called a windowed display. Alternatively, you can display images without a windowed border on an auxiliary screen, hardware permitting; this is called an auxiliary display.

To display an image, you need a Display control, an Image control and a System control. Specify the image to display using the Image property of the Display control. For an image to be displayed, its CanDisplay property must be set to True, which is the default setting. In addition, both the image and the display should be on the same system; that is, the OwnerSystem property of the Image control and the OwnerSystem property of the Display control should match.

The image is typically displayed from the top-left corner of the display. When the image is too large to be displayed at once in a windowed display, you can use scroll bars to view parts of the image that are not visible. When this is the case in an auxiliary display, use the Pan method.

In general, you can display any type of image in a display. However, when displaying non 8-bit images in an 8-bit display resolution, processing speed might be slower.

An image, or any of its child images, can be displayed on more than one display.

The ActiveMIL documentation uses the term frame buffer to refer to physical display (graphics controller) memory (not a buffer, per se).


Types of displays

User Guide:
The display window (for windowed displays)

Reference:
Display.DeviceNumber
Display.DisplayType
Display.Format
Image.Location

By default, images are displayed in the Display control's window. This, however, is only one way that ActiveMIL can display images.

With ActiveMIL, you can display images:

  • With a windowed border, which is called a windowed display.
  • Without a windowed border on a dedicated screen, which is called an auxiliary display.

Windowed display

An image displayed in a windowed display is displayed on the Windows desktop screen(s); it is displayed in one of the following windows:

  • The Display control's window (default).
  • A user-defined window.
  • An external window.

To display an image in one of these windows, see the section, The display window (for windowed displays).

All windowed displays are transparently tracked and updated; that is, if the window moves or is occluded, the window is automatically updated with the image.

A windowed display is not affected by whether or not your desktop is displayed using one screen or multiple screens. We refer to these screens, either one or many, as the Windows desktop screen(s). Under Windows 2000/XP, your desktop can be extended over screens of different resolutions.

Multiple windowed displays can be on the Windows desktop screen(s). The display number should always be the ActiveMIL default; that is, the DeviceNumber property of the Display control should always be set to amilDefault.

For windowed displays, the display format of the on-screen portion of the frame buffer is set using the selected Windows display resolution.

Auxiliary display

An image displayed in an auxiliary display is displayed without a windowed border or frame, at the top-left corner of a screen that is not used to display the Windows desktop. This screen is referred to as an auxiliary screen.

In this context, the term screen refers to any device that supports video output data, such as a high-resolution monitor, TV, or VCR.

Auxiliary displays are supported if one of the following resources is available:

  • Two graphics controllers.
  • One DualHead graphics controller that integrates two video output controllers (for example, Matrox G400, G450, or G550).

    • The graphics controller can be on, or apart from, the Matrox imaging board. Also, some boards might have special features or limitations regarding auxiliary displays; for more information, see the ActiveMIL Board-Specific Notes.

You can only display one auxiliary display at a time on a given auxiliary screen. Moreover, you are responsible for moving and tracking an auxiliary display, if required. To choose this type of display, set the Display control's DisplayType property to dispAuxiliary.

When using an imaging board with a display section (for example, Matrox Orion), it might be necessary to set a Dip switch to display on an auxiliary screen. For more information, see the board's installation and hardware reference manual.

For auxiliary displays, the display number should always be the ActiveMIL default; that is, the DeviceNumber property of the Display control should always be set to amilDefault.

Video output format

ActiveMIL does not impose any restrictions on the video output format of the auxiliary screen. The format can be:

  • A high-resolution format (for example, 1024x768x32@70 Hz).
  • An encoded video format (for example, NTSC/PAL).

    • For all the supported formats, see the ActiveMIL Board-Specific Notes.

The video output format of the auxiliary screen is set with the Display control's Format property.

The maximum number of auxiliary displays that can be allocated is determined by the number of video output controllers that support the specified format. For example, two auxiliary displays with high-resolution formats can only be allocated if there are two available video output controllers that support high-resolution formats.

When allocating a display, ActiveMIL checks for an appropriate video output controller, and not an appropriate device attached to it. It is therefore possible to allocate an auxiliary display without an auxiliary screen connected to the video output controller.

Matrox Millennium G400, G450, G550

To use the second video output controller of Matrox Millennium G400, G450, or G550 for ActiveMIL auxiliary display, your display driver's DualHead mode must be disabled; otherwise both the display driver and the MIL driver will attempt to access the second video output controller. In addition, the G400's second video output controller does not support encoded video formats.


The display window (for windowed displays)

User Guide:
Attaching scroll bars to the display window
Types of displays

Reference:
Display.DisplayType
Display.Free
Display.UserWindow

An image displayed in a windowed display is displayed on the Windows desktop screen(s); in addition, it is displayed in one of the following windows:

  • The Display control's window (default)
  • A user-defined window
  • An external window

Typically, the display need not have the same size as the image. If the defined window is of a different dimension than the image, any excess window area will be left untouched or any excess image area will be cropped.

Displaying in the Display control's window

To display in the Display control's window, make sure the DisplayType property of the Display control is set to dispDefaultWindow or dispActiveMILWindow. When the display type is set to M_DEFAULT in the MILConfig utility, and the DisplayType property is set to dispDefaultWindow, the display type is actually set to dispActiveMILWindow.

Displaying in a user-defined window

To display in a user-defined window, set the DisplayType property of the Display control to dispUserWindow. The window can be created using Windows' API functions or can be an existing window. To associate the window with the display, pass the window's handle to the UserWindow property of the Display control.

The UserWindow property is only valid when the DisplayType property is set to dispUserWindow. If this property is not set, then nothing will be displayed.

This user-defined window is automatically refreshed when the display is modified (for example, when the image is modified).

Displaying in an external window

To display in an external window, set the display's DisplayType property to dispExternalWindow. Various properties of the external window can be changed, such as its initial position, its title, and whether to disable its zoom buttons. These properties can be changed using the External Window dialog box or using the property browser. To access the External Window dialog box, click on Properties from the Window page of the display's property pages.


Attaching scroll bars to the display window

User Guide:
Auxiliary displays
Panning and zooming
The display window

Reference:
Display.ExternalWindow.Scrollbar
Display.PanX
Display.PanY
Display.ScrollHorizontal
Display.ScrollVertical

For windowed displays, you can interface your environment's scroll bars to the display control.

Scroll bars are not available on auxiliary displays.

The Display control's window and the user-defined window

If you are using either the Display control's window or a user-defined window, you can interface your environment's scroll bars to the ActiveMIL Display control by following these steps:

  1. Create a vertical and horizontal scroll bar with the environment's scroll bar controls. Then, add procedures for the ScrollHorizontal and ScrollVertical events of the Display control.
  2. In each event procedure, assign to the scroll bar's Value property the Value parameter that is passed to the event procedure of the Display control. Then, assign to the scroll bar's Min and Max properties the MinValue and MaxValue parameters that are passed to the event procedure of the Display control. This way, whenever the display is panned or zoomed, the scroll bars will automatically follow.
  3. Create a procedure for the Change event of each scroll bar.
    • In the horizontal scroll bar's Change event procedure, set the PanX property of the Display control to the current Value property of the scroll bar.
    • In the vertical scroll bar's Change event procedure, set the PanY property of the Display control to the current Value property of the scroll bar.

The external window

ActiveMIL will create an external window of the same size as the image, unless such an external window cannot fit in the Windows desktop. If the image is too large, there will automatically be scroll bars to view other parts of the image, and the initial view of the image will be the top-left corner. If the image is too small, it will be centered in the window, and the surrounding area will be blacked out. If the external window is resized, causing it not to fit in the Windows desktop, then scroll bars will appear.

If you do not want scroll bars to appear, set the ExternalWindow.Scrollbar property of the Display control to False.


Removing an image from the display

Reference:
Display.BackColor
Display.Image

To remove an image from the display, set the Image property of the Display control to:

  • Nothing for Visual Basic.
  • NULL for Visual C++.

This leaves the display blank, filled with the color specified by the BackColor property of the Display control.

To display a different image, it is not required to remove the current image from the display; selecting another image for display (that is, setting the Image property of the Display control to the name of another image) automatically removes the current image from the display. Also, if an image is freed while on the display, the display will be automatically filled with the color specified by the BackColor property of the Display control.


Displaying multiple images

User Guide:
Auxiliary displays
Operating on a region of interest
Windowed display

Reference:
Display.DeviceNumber
Display.OwnerSystem
Image.ChildRegion
Image.ParentImage

Only one image can be viewed in one windowed display; however many windowed displays can be viewed on the Windows desktop screen(s). To display multiple windowed displays, associate the images with different Display controls. The Display controls should be on the same system (have the same OwnerSystem properties).

For auxiliary displays, you can have as many auxiliary displays as you have auxiliary screens, using the same technique as described above; however, you can have only one auxiliary display at a time on a given auxiliary screen. To view more than one image at a time in one display, use child images. For example, you can display the source and destination images of an operation, using the following steps:

  1. Create a large image (big enough to contain the source and destination images in the following steps). This large image will be called the parent image.
  2. Create two non-overlapping child images within the large displayable parent image.
  3. Use one child image as the source and the other as the destination of your operation.
  4. Display the parent image.

For an example of using child images to view several images simultaneously, see MImageColor.

For windowed displays, you could also display multiple images using child images. However, it is generally easier to manage multiple Display controls than multiple child images.

Display number

For either windowed displays or auxiliary displays, the Display control's DeviceNumber property must be set to amilDefault. Based on the Display control's Format property, ActiveMIL finds the best device to use when displaying an image.


Screen tearing

User Guide:
Grab mode

Reference:
Display.NoTearing

Screen tearing occurs when the grab and the display are not updated synchronously, for example:

The non-synchronous update between the grab and the display causes two images to temporarily appear simultaneously, with one of the images drifting down the screen line by line. When the lag between the grab and the display is high, the drift is fast; when the lag is slow, so is the drift.

Your monitor's vertical frequency must be a multiple of your camera's vertical frequency. In North America, this does not present a problem since both monitors and cameras usually function at 60Hz, although a phase shift can still cause tearing. However in Europe, monitors function at 60Hz, while cameras operate at 50Hz, therefore creating the disturbing visual effect.

To eliminate tearing for a selected display, set the NoTearing property of the Display control to True. This property requires special hardware, such as a Matrox Millennium G400, G450, or G550 graphics controller. Note that if the NoTearing property is set and is not supported, an error will be produced.


Panning and zooming

User Guide:
Auxiliary displays
Child images
Displaying multiple images
Windowed displays

Reference:
Display.Pan
Display.PanX
Display.PanY
Display.Zoom
Display.ZoomX
Display.ZoomY
Image.ChildRegion
Image.ParentImage

At times, your image might be larger than the display, or have details that are too fine or too small to see. Display effects can be associated with the display to view specific parts of the image. These effects are panning and zooming.

Panning

Panning displaces an image horizontally or vertically. This can be useful, for example, when the image is larger than the display and, therefore, cannot be viewed all at once. You can pan your image to display the appropriate location at the top-left corner of the window (for windowed displays) or screen (for auxiliary displays). To pan the image, use the Pan method or the PanX and PanY properties of the Display control.

Note that, to pan, you can also interface your environment's scroll-bars to an ActiveMIL Display control.

For auxiliary displays, to view an image at another location on the display, you must create a large displayable image, display it, and then use a child image at the required location on the display. For windowed displays, just move the window to the required location on the screen.

Zooming

Zooming is the horizontal and/or vertical replication of each pixel by a given factor. This can be useful, for example, when part of the image needs to be seen in more detail. You can zoom the display, with floating point precision, using the Zoom method or the ZoomX and ZoomY properties of the Display control.

You can specify that the zoom factors be added to the current zoom factors by calling the Zoom method of the Display control, and setting the Incremental parameter to True. In this case, the zoom factors are relative to the current zoom factors; otherwise, the zoom factors are applied directly to the display.

Note that zooming might cause the image to be larger than the display. If necessary, you can pan the displayed image to bring the required area into view.

Zooming by a large factor might cause a "blocky" effect. Also, you can reduce the size of an image on the display by using negative zoom factors.


Displaying non 8-bit images

User Guide:
Auxiliary displays
Data type and depth
Frame buffers
Windowed displays

Reference:
Display.Format
Display.ViewBitShift
Display.ViewMode
Image.CanDisplay

Images that have their CanDisplay property set to True, usually have a depth of 8 bits (8-bits per band in the case of color images). You can display images of other depths (for example, 1-bit or 16-bit images). Using the ViewMode and ViewBitShift properties, you can control the way such images are actually displayed.

The ViewMode property provides different modes of displaying non 8-bit images:

  • Bit-shift mode.
  • Auto-scale mode.
  • Multi-bytes mode.
  • Transparent mode.
  • Default view mode.

Auto-scale mode is not available in ActiveMIL-Lite.

Bit-shift mode

The bit-shift mode will bit-shift the pixel values of the image by the specified number of bits upon updating the display. To use this mode, set the ViewMode property of the Display control to dispBitShift.

Auto-scale mode

The auto-scale mode remaps the pixel values to the display range such that the minimum and maximum values in the image (not the full range of the image) are set to 0 and 255, respectively. To use this mode, set the ViewMode property of the Display control to dispAutoScale.

If the image contains a single value, its corresponding displayed value is determined by linearly re-mapping the full range of the image; for example, (0 to 64K) to (0 to 255).

Multi-bytes mode

The multi-bytes mode is primarily useful when grabbing from a multi-tap camera. This setting displays each byte of the image in separate display pixels. For instance, each pixel of a 16-bit image will occupy two consecutive display pixels; each pixel of a 32-bit image will occupy four consecutive display pixels. This mode is only supported for 16-bit and 32-bit 1-band images. To use this mode, set the ViewMode property of the Display control to dispMultiBytes.

Transparent mode

The tranparent mode will display only the 8-least significant bits of the image. No pixel remapping will be performed. To use this mode, set the ViewMode property of the Display control to dispTransparent. This is the default view mode.


Annotating the displayed image non-destructively

User Guide:
CPU-assisted display
Creating an application (Visual Basic)
Creating an application (Visual C++)
Non 8-bit display resolution
Screen tearing
Types of displays

Reference:
Display.ExternalWindow.hWnd
Display.Image
Display.NoTearing
Display.OverlayImage
Display.OverlayKeyColor
Display.OverlayWriting
GraphicContext.Text
Image.Clear
Image.Control
Image.Inquire
Image.Modified

Overlay-display mechanism

For windowed displays or auxiliary displays, you can annotate the displayed image non-destructively using ActiveMIL's overlay-display mechanism. To make use of this functionality, do the following:

  1. Enable the overlay mechanism, by setting the OverlayWriting property of the Display control to True.
  2. Set the Image property to the image you want to display.

    3. Since the overlay-display mechanism is enabled, this will not only display the image, but it will also associate a temporary overlay image with the display. This image is referred to as the display's overlay image. This overlay image can be used to annotate the underlying image with an effect called keying, described in greater detail in the Transparency (Keying) subsection.

  3. To access the display's overlay image, use the OverlayImage property of the Display control.
  4. Draw into the display's overlay image. For example, set the Image property of the GraphicContext control to the OverlayImage property of the Display control. Then call the Text method of the GraphicContext control.

    5. You can also annotate the displayed image with Windows GDI annotations, or with Visual Basic annotations, which are discussed later.

Number of bands

Typically, the overlay image will have the same number of bands and will be the same size as its associated image (not the size of the display). However, if you are using a non 8-bit display resolution (15-bit, 16-bit, 24-bit, or 32-bit color resolution), and the image selected to the display is 1-band, then the overlay image is 3-band.

Note that if the graphics controller does not have non-destructive overlay capabilities, and you are using a non 8-bit display resolution (15-bit, 16-bit, 24-bit, or 32-bit color resolution), a 1-band image will have a 1-band overlay image.

Overlay image behavior

When an image is selected to a display that has an overlay image associated with it, and you set the display's Image property to another image, which:

  • Has the same dimensions as the image currently being displayed, the overlay image remains the same and the current display's OverlayWriting property is still set to True. Any annotations will, therefore, remain until you clear the overlay image by calling its Clear method and specifying the keying color (see the Transparency (Keying) subsection to determine the keying color).
  • Has different dimensions than the image currently being displayed, the current overlay is freed and a new one is created. In addition, the annotations of the old overlay image are copied into the new one. The current display's OverlayWriting property is still set to True.

When the display's overlay image is created, it is cleared to the effective transparency (keying) color. If the transparency (keying) color is changed after the display's OverlayWriting property is set to True, the overlay image will not be cleared.

CPU-assisted overlay

The ability to annotate the displayed image non-destructively using ActiveMIL's overlay-display mechanism is always available, and is typically accomplished by your hardware (that is, your graphics controller). However, if your hardware limits have been reached, ActiveMIL produces a simulated version of the overlay effect using the CPU; the display is therefore said to be CPU-assisted.

Transparency (Keying)

The display's overlay image annotates the image related to the display using transparency. Transparency, more commonly referred to as keying, allows you to annotate your images, by replacing pixels of one image that are of the specified transparent (keying) color with the underlying areas of another image. The annotations drawn in the overlay buffer in a color other than the transparent color will annotate the image selected to the display.

The portions of the overlay that are made transparent depend on the keying color, which is specified by the display's OverlayKeyColor property. When the overlay-display mechanism is enabled, keying is automatically enabled, and the keying color is automatically set to a default color, which is generally appropriate.

If you are using an 8-bit display resolution (256 colors), set the keying color to a value between 0 and 255. If you are using a non 8-bit display resolution (15-bit, 16-bit, 24-bit, or 32-bit color resolution), use your environment's RGB() function to explicitly set a value for each of the color bands (red, green, and blue).

For an example of annotating the displayed image non-destructively, see MDisplayOverlay.

Using GDI annotations (only for Visual C++)

Under Visual C++, there are two techniques that you can use to annotate the displayed image with Windows GDI annotations.

The first technique is:

  1. Allocate a device context (DC), calling the Control method of the Image control, and set its ControlType parameter to imDCAlloc. Windows GDI annotations require a DC to draw in the displayed image.

    2. You can create a DC for either the image or the overlay image of the display. Note that if you create a DC for the image and then draw using this DC, drawing will be destructive (that is, the data of the image is actually changed).

  2. Inquire the identifier of this context using the Inquire method of the Image control, and set its ParamToInquire parameter to imDCHandle.
  3. Annotate the DC using Windows GDI annotation functions.

    4. The image which you are annotating should be internally stored in imDIB, imGDI, or imDirectX format, and cannot be a child image.

  4. Free the device context (DC) by calling the Control method of the Image control, and setting its ControlType parameter to imDCFree.
  5. Notify ActiveMIL that the display needs to be updated by calling the Modified method of the Image control and setting the ModificationType parameter to imContents.

    6. If your image format is specified as imDirectX, you must free the DC before signaling ActiveMIL.

The second technique is:

  1. Inquire the display's window handle.
    • If using the Display control's window, call the hWnd property of the Display control.
    • If using the user-defined window, call the UserWindow property of the Display control.
    • If using an external window, call the ExternalWindow.hWnd property of the Display Control.
  2. Pass the window handle to the Windows GetDC() function to get a Windows display device context (DC).
  3. Paint the annotations with GDI functions each time a paint message appears in the window (WM_PAINT).

For an example of using GDI annotations, see MDisplayOverlay (only for Visual C++).

Visual Basic annotations

Besides using a GraphicContext control, there are two techniques that you can use to annotate the displayed image with Visual Basic:

  • If you are using the Display control's window or a user-defined window, drag a Visual Basic graphical control (for example, Shape or Line) onto its display.
  • If you are using a user-defined window, you can also use the Visual Basic graphics methods of the user window (if available), such as Circle, Line, or PSet (in addition to using Visual Basic graphical controls).

When using either technique, if the drawing color is the same as set by the display's OverlayKeyColor property, the underlying image will appear; that is, graphical annotations will not be visible.

The user window's Visual Basic Circle, Line, or PSet methods must be called from the Paint event of the Display control. When annotating the Display control's window with Visual Basic's graphical controls, you must add a comment in Visual Basic's Paint event of the Display control, otherwise the Visual Basic control will not appear.

When using Visual Basic for graphical annotations, the OverlayWriting property of the Display control need not be enabled.

Overlay flickering

For MDI (Multiple Document Interface) applications, Visual Basic's overlay annotations might flicker when performing a continuous grab to the display. To avoid this type of screen tearing, set the NoTearing property of the Display control to True.

Your overlay might also flicker if your display is CPU-assisted. For more information, see the section on CPU-assisted displays.


LUTs and the display

User Guide:
Generating data into LUTs
Setting up your LUT
Windowed displays

Reference:
Display.LUT
Display.LUT.GenerateColor
Display.LUT.Get

Palettes and output LUTs for windowed display (256-color)

For windowed displays, when displaying in a 256-color Windows display resolution, images are mapped through physical output LUTs on display. Windows uses these LUTs to achieve color; in addition, Windows uses the concept of a palette to load these LUTs.

ActiveMIL provides Windows with good default logical palettes for the realization of the physical output LUTs in a 256-color display resolution. However, since not all colors are available at this resolution, the default might not always be optimal for certain atypical cases. Therefore, the physical output LUTs are always available and programmable, by a user, to achieve the best display effect for your images.

Reference material: Windows palettes and physical output LUTs

Before describing the default and explaining how and why to change it, a basic explanation of palettes and output LUTs is discussed. For more detailed information, consult the following reference:

Halibard, Moishe. Windows Developer's Journal. "Palettes and 256 Colors." July, 2000.

Color images

For windowed displays, in a 256-color display resolution, the following example shows how a color image is consequently displayed as the best color image possible.

To display the best color image possible:

  1. The color image goes through Windows translation functions. Windows searches the physical output LUTs for the entry that best matches the color of the image's pixels. Depending on what colors are available, Windows translates the image to an 8-bit index image.
  2. The physical output LUTs subsequently translate that index image and display it as the best color image possible.

Palettes and physical output LUTs

Each of the color images' palette is used to specify the required colors. Windows uses these palettes to realize a system palette. The system palette is ultimately loaded into the physical output LUTs and is searched by the translation functions when generating an 8-bit (index) image.

The palette of the active window has priority over available system palette entries and is therefore loaded in the system palette first. Also, Windows tries to map colors from the logical palette into the currently realized system palette to reduce the number of requested new entries. This reduces the chance of a color occurring more than once in the system palette.

Monochrome images

Since LUTs are available, you can use them to create special effects for 8-bit images. However, unlike color images, grayscale values map to values at the corresponding index in their associated palette.

Default palette settings

By default, windowed displays use the default ActiveMIL palette. ActiveMIL provides good default logical palettes for the realization of the physical output LUTs. To use the default, set the LUTEnabled property of the Image control to False. The ActiveMIL default takes into consideration the number of bands of the image, and produces the best performance versus visual quality compromise possible.

Why change the default LUT values

If the default LUT values are not appropriate for your application, you can change the LUT values to control the displayed colors or gray levels of an image. Some situations that might require special display effects are:

  • When displaying monochrome images, you might want to view the images with each gray intensity in a different color. For example, you can associate specific colors to ranges of temperatures obtained by an infrared camera.
  • When displaying monochrome images, you might want to invert the image values. For example, when grabbing a film negative, you can negate the video and display the film as it will be printed.
  • When displaying color images in a 256-color Windows display resolution, you might want to reduce the loss of color resolution. Generally, the default LUT values will not optimize and distinguish between subtle differences in one color. For example, when displaying a color image with many shades of red, you might want to select a LUT so that all shades of red in that particular image are being represented.

Changing the default LUT values

Change the default LUT values by associating a LUT to:

  • The display. Specify a custom LUT with the LUT property of the Display control. Then, set the LUTEnabled property of the Display control to True. This allows the same LUT to be used for all displayed images; consequently, when the image is saved, the LUT is not saved with it.
  • The image. Specify a custom LUT with the LUT property of the Image control. Then, set the LUTEnabled property of the Image control to True. This allows each displayed image to have its own LUT; consequently, when the image is saved, the LUT is saved with it.

A 1-band custom LUT

To invert the values of an 8-bit image on display, you would need physical output LUTs that map each value to the maximum pixel value minus the current pixel value. To do so:

  1. Set up a 1-band 256-entry LUT as described above.
  2. Generate the data into the LUT using the LUT.GenerateRamp method of the Display control, or load the data into it, using the LUT.Put method of the Display control. The depth of the LUT buffer data must be 8-bits.

    3. When you associate a 1-band LUT with the display or image, the same data is loaded into each component of the logical palette.

A 3-band custom LUT

To reduce the loss of color resolution when displaying an image with a specific range of colors, you would need physical output LUTs that contain all the required colors so that, when Windows creates a translation table for the image, most colors are mapped to their exact values. To do so:

  1. Set up a 1-band 256-entry LUT as described above.
  2. Take a histogram of the image.
  3. Order and load frequently used colors in the LUT according to popularity (the most popular color first).

    4. Be careful not to remove infrequent colors that illustrate critical information. Also, when you associate the 3-band LUT (RGB) with a display, each band of the LUT is loaded into its corresponding component of the logical palette.

A 3-band LUT can also be used to create a custom pseudo-color LUT for 8-bit images.

The behavior of physical output LUTs

When using physical output LUTs, you must keep the following points in mind:

  • If the content of a LUT changes while the image is on the display, the changes will not take effect until you enable the LUTEnabled property of the Display control again.
  • When displaying in a non-256-color display resolution, ActiveMIL can simulate a display LUT in software only for 8-bit 1-band images.
  • The LUT must have one or three bands. The number of LUT entries must be the same as the maximum number of intensities that can be represented in the displayed image. In other words, if you want to invert an 8-bit grayscale image (that is, an image that can have 256 intensities), your LUT must also have 256 entries.

The properties of the LUT property, and the Get method of the LUT property can be used to obtain information about the LUTs of a display.


CPU-assisted display

See Also | Previous
User Guide:
Grabbing to the display
Overlay image

All displays are typically accelerated by the graphics controller, which means that CPU usage is low since the graphics controller is handling the display. However, if your graphics controller's limits have been reached, ActiveMIL compensates by making the display CPU-assisted. This results in higher than expected CPU usage.

In addition to higher than expected CPU usage, you might experience the following behaviors when your display is CPU-assisted:

  • Overlay flickering.
  • Slower updates of the display during continuous grabbing.

Overlay flickering

When your graphics controller's limits have been reached, ActiveMIL uses your CPU to implement the overlay-display mechanism. Annotations in this overlay image might flicker, particularly those done with Visual Basic, though they are still non-destructive.

Grabbing continuously

When your overlay image is CPU-assisted, the display of a continuous grab can be slower. This is due to an additional operation needed to combine the grabbed image with the display's overlay image in an intermediate image. Note that the actual image on the display is not overwritten by the content of the overlay image.

Avoiding CPU-assisted displays

To avoid a CPU-assisted display, you can attempt to:

  • Lower your screen resolution or refresh rate.
  • Free all on-board images or displays that are no longer being used.
  • Avoid forcing a specific internal format when allocating the image buffer, unless it is required.


Graphics

Chapter 16: Graphics

This chapter describes the graphics commands that are available with ActiveMIL. These consists of drawing and text-writing commands.

  • Overview of graphics
  • Preparing for graphics
  • Filling an object
  • Setting positional properties
    • Interactively setting properties
  • Drawing an arc
  • Drawing a cross
  • Writing text
  • Using non-ActiveMIL graphics


Overview of graphics

See Also | Next
Reference:
GraphicContext
GraphicContext.Arc
GraphicContext.Cross
GraphicContext.Dot
GraphicContext.Dots
GraphicContext.Fill
GraphicContext.LineSegment
GraphicContext.LineSegments
GraphicContext.Rectangle
GraphicContext.Text

ActiveMIL's GraphicContext control can be used to draw into an image or into the overlay image of your display. The GraphicContext control allows you to:

  • Draw lines, using the LineSegment or LineSegments method.
  • Draw rectangles (filled or unfilled), using the Rectangle method.
  • Draw arcs, circles, and ellipses (filled or unfilled), using the Arc method.
  • Draw dots, using the Dot or Dots method.
  • Draw crosses, using the Cross method.
  • Write text, using the Text method.

You can also fill an object (or the intersection of two objects), using the Fill method.

To use the above methods, you need an Image control, a System control, and a GraphicContext control. For more information, see Preparing for graphics.

In addition to ActiveMIL's graphic capabilities, you can use your environment's graphic capabilities for overlay annotation.

Note that ActiveMIL graphics are only drawn at run-time.

For an example that uses the GraphicContext control, see MAppStart.


Preparing for graphics

User Guide:
Setting positional properties

Reference:
GraphicContext.Clear
GraphicContext.DrawingRegion
GraphicContext.ForegroundColor
GraphicContext.ForegroundShade
GraphicContext.Image

In order to draw graphics or write text, you need an Image control, a System control and a GraphicContext control. The graphic or text is drawn into the image specified by the graphic context's Image property. Before drawing into this image, you might want to clear it, using the Clear method.

The position of the graphic or text must be specified using pixel coordinates. To draw in a calibrated image using real-world coordinates, you must first convert the coordinates from their real-world value to their pixel value using the ConvertCoordinateWorldToPixel method or ConvertCoordinateListWorldToPixel method. With the exception of the Dots and LineSegments methods, specify the positional properties of the graphic or text using the DrawingRegion property.

The graphic or text should fit within the boundaries of the specified image; if not, it will be clipped.

By default, the graphic or text is drawn in the color specified by the ForegroundColor property. By default, this color is white. You can select a different foreground color, using the property's list box, or you can use a shade of gray as your foreground color, by setting the ForegroundShade property to the required value. If you set the ForegroundShade property, you can use any 32-bit value. However, if the given value exceeds the range of the destination image, only the least significant bits of the value are used.


Filling an object

User Guide:
Preparing for graphics
Setting positional properties

Reference:
GraphicContext.DrawingRegion
GraphicContext.DrawingRegion.StartX
GraphicContext.DrawingRegion.StartY
GraphicContext.Fill
GraphicContext.ForegroundColor
GraphicContext.ForegroundShade

The Fill method fills the connected region around a specified pixel (called the seed pixel) with a specified color. A connected region is an area of touching pixels that have the same value (horizontally and vertically adjacent pixels are considered touching; diagonally adjacent pixels are not).

You specify the coordinates of the seed pixel using the StartX and StartY properties of the DrawingRegion property (this property is described in Setting positional properties). The filling color is specified by the ForegroundColor or ForegroundShade property (the difference between these properties is described in Preparing for graphics).


Setting positional properties

Reference:
GraphicContext.Dot
GraphicContext.DrawingRegion
GraphicContext.DrawingRegion.CenterX
GraphicContext.DrawingRegion.CenterY
GraphicContext.DrawingRegion.EndX
GraphicContext.DrawingRegion.EndY
GraphicContext.DrawingRegion.SizeX
GraphicContext.DrawingRegion.SizeY
GraphicContext.DrawingRegion.StartX
GraphicContext.DrawingRegion.StartY
GraphicContext.ForegroundColor
GraphicContext.ForegroundColor

With the exception of the Dots and LineSegments methods, specify the required positional properties (shown below) of the graphic or text using the DrawingRegion property.

To set the above properties, use the Regions page of the graphic context's property pages. You can access this page by right-clicking on the GraphicContext control and then selecting Properties from the presented menu or by using the property browser. The required properties can be set using the edit fields of the Regions page or can be set interactively.

Note the following:

  • Most graphic methods do not use all properties. For example, the Dot method uses only the StartX and StartY properties to define the coordinates of the dot.
  • Coordinates are relative to the top-left corner of the image.
  • SizeY can be calculated from StartY and EndY, SizeX from StartX and EndX, CenterX from StartX and EndX, and CenterY from StartY and EndY. This allows you to set positional properties by specifying the size or the coordinates of the graphic or text, depending on what is known. Note that, when you adjust one property, its corresponding property or properties are adjusted automatically.

Interactively setting properties

To interactively set properties, click on the Interactive button from the Regions page; this brings up the Interactive Preview window. The box in the top-left corner of this window is used to define the required properties; drag and resize the box as required.

To visualize how the graphic or text will appear in the required image, you can load an image into the window, using the Load image button. You can also zoom in and zoom out on the window, using the Zoom In and Zoom Out buttons, respectively.

To see how the box defines the graphic or text, click on the appropriate graphic or text button (these buttons are at the right of the window). For text, type the required text in the edit field next to the Text button.

You can change the color that is used to display the box. To do so, use the drop-down list box next to the zoom buttons. Note that this color has no affect on the ForegroundColor or ForegroundShade property.

You cannot use the Interactive feature to set the properties of the Dots and LineSegments methods.


Drawing an arc

User Guide:
Filling an object
Preparing for graphics
Setting positional properties

Reference:
GraphicContext.Arc
GraphicContext.DrawingRegion
GraphicContext.DrawingRegion.CenterX
GraphicContext.DrawingRegion.CenterY
GraphicContext.DrawingRegion.EndX
GraphicContext.DrawingRegion.EndY
GraphicContext.DrawingRegion.SizeX
GraphicContext.DrawingRegion.SizeY
GraphicContext.DrawingRegion.StartX
GraphicContext.DrawingRegion.StartY
GraphicContext.ForegroundColor
GraphicContext.ForegroundShade

When you use the Arc method, you define the arc using:

  • The SizeX, SizeY, CenterX, and CenterY properties of the DrawingRegion property, along with the StartAngle and EndAngle parameters of the Arc method.
  • The StartX, StartY, EndX, and EndY properties of the DrawingRegion property, along with the StartAngle and EndAngle parameters of the Arc method.

The DrawingRegion property is described in Setting positional properties.

The arc is drawn in the color specified by the ForegroundColor or ForegroundShade property (the difference between these properties is described in Preparing for graphics). If you choose to fill the arc, it is filled with the specified color or shade.


Drawing a cross

User Guide:
Preparing for graphics
Setting positional properties

Reference:
GraphicContext.Cross
GraphicContext.DrawingRegion
GraphicContext.DrawingRegion.CenterX
GraphicContext.DrawingRegion.CenterY
GraphicContext.DrawingRegion.SizeX
GraphicContext.DrawingRegion.SizeY
GraphicContext.ForegroundColor
GraphicContext.ForegroundShade

When you use the Cross method, you define the cross using:

  • The SizeX, SizeY, CenterX, and CenterY properties of the DrawingRegion property.

The DrawingRegion property is described in Setting positional properties.

The cross is drawn in the color specified by the ForegroundColor or ForegroundShade property (the difference between these properties is described in Preparing for graphics).


Writing text

User Guide:
Preparing for graphics
Setting positional properties

Reference:
GraphicContext.BackgroundColor
GraphicContext.BackgroundMode
GraphicContext.BackgroundShade
GraphicContext.DrawingRegion
GraphicContext.DrawingRegion.SizeX
GraphicContext.DrawingRegion.SizeY
GraphicContext.DrawingRegion.StartX
GraphicContext.DrawingRegion.StartY
GraphicContext.Font
GraphicContext.FontScaleX
GraphicContext.FontScaleY
GraphicContext.ForegroundColor
GraphicContext.ForegroundShade
GraphicContext.Text

When you use the Text method, the text is drawn in the color specified by the ForegroundColor or ForegroundShade property (the difference between these properties is described in Preparing for graphics). The size of the text is determined from the Font, FontScaleX and FontScaleY properties. The StartX and StartY parameters of the DrawingRegion property determine the coordinates at which to start writing the text (the first character is written from the top-left corner of these coordinates). The text must fit within the boundaries defined by the SizeX and SizeY parameters of the DrawingRegion property; if not, it will be clipped. (The DrawingRegion property is described in Setting positional properties).

By default, the text is drawn against a black background. You can use a background of a different color or shade, or you can use a transparent background. To use a different color or shade, set the BackgroundMode property to graOpaque, then set the BackgroundColor or BackgroundShade property accordingly. To use a transparent background, set the BackgroundMode property to graTransparent. In this case, the BackgroundColor and BackgroundShade properties have no effect.


Using non-ActiveMIL graphics

See Also | Previous
User Guide:
Preparing for graphics
Setting positional properties

Reference:
GraphicContext.LineSegment

In addition to ActiveMIL's graphic capabilities, you can use your environment's graphic capabilities for overlay annotation. When you use your environment's graphic capabilities, the graphic or text is drawn directly in the Windows desktop; the overlay image of your display is not overwritten.

Note the following:

  • Your environment's graphic capabilities can only be used for overlay annotation. An ActiveMIL graphic can be drawn into any image.
  • Some of your environment's graphic controls can be used to draw on forms at design-time. ActiveMIL graphics are only drawn on image at run-time.
  • Your environment's graphics can appear on top of the display window only when your environment's graphics control is created as a child to the display.
  • In most cases, your environment's graphic capabilities offer greater flexibility than those of ActiveMIL. For example, Visual Basic's Line object can draw lines that are several pixels wide, unlike ActiveMIL's LineSegment method.

For an example of non-destructively annotating the display image using ActiveMIL and non-ActiveMIL controls, see MDisplayOverlay.


Digitizer

Chapter 17: Digitizer

This chapter discusses how to grab images using a digitizer allocated on your frame grabber and the features available when grabbing.

  • Camera and video sources
  • Basic concepts
  • Data format
  • Device number
  • Multiple cameras
    • Simultaneous acquisition
    • Data input channels of aquisition paths
    • Switching between cameras of the same type
    • Switching between cameras of different types
    • Ultra-fast channel switching
    • Channel locking and unlocking
  • Digitizer events
  • Grabbing a single field
  • Line-scan cameras
  • Grabbing to the display
  • Reference levels
    • Black and white reference levels
    • Color image reference levels
  • LUTs and the digitizer
  • Scaling grabbed data
  • Optimizing application performance when grabbing
    • Grab mode
    • Double buffering
    • Multiple buffering
  • Grabbing a sequence of frames in real-time
  • Triggered grabs
    • Asynchronous reset mode and next valid frame mode
    • Setting the exposure
    • Software triggers
  • Auto-focusing
    • Search strategies
      • Bisection strategy
      • Refocus strategy
      • Scan-All strategy
      • Smart-Scan strategy
      • Evaluate the focus indicator


Camera and video sources

See Also | Next
Reference:
Digitizer.DeviceNumber
Digitizer.Format
Digitizer.Grab
Digitizer.GrabContinuous
Digitizer.Image
Digitizer.MultipleBuffering.Process
Digitizer.OwnerSystem

ActiveMIL supports input from any type of video source supported by the imaging board (frame grabber). Each frame grabber has one or more available acquisition paths. ActiveMIL refers to the acquisition paths used to grab from one video source as a digitizer.

Note, since most video sources are cameras, they will hereafter be referred to as such.

To grab an image (monochrome or color), you need a Digitizer control, a System control, and an Image control. The Image and Digitizer controls are created such that their initial properties match those of your default data format and device number settings. The data format and the device number of the digitizer define the acquisition paths that the digitizer represents and from which it will grab.

To grab a specific number of frames or fields, use the digitizer's Grab method. To grab continuously, use the digitizer's GrabContinuous method. The data will be grabbed into the image specified by the digitizer's Image property. The image should have as many bands as the data has color components. If you need to grab and process concurrently, you can use the MultipleBuffering.Process method of the Digitizer control.

The Image and Digitizer controls must be on the same system, that is, their OwnerSystem properties must match.

When you are done with your Digitizer control, System control, and/or Image control, you should free each control using the free method of the control.

When developing an application, it is recommended that you use a simple camera. Once the application is working, switch to a more sophisticated camera, if necessary. This approach makes debugging much easier.

For an example of grabbing an image, see MDigGrab.


Basic concepts

Previous | Next

The basic concepts and vocabulary conventions for the ActiveMIL digitizers are:

  • Acquisition path. A path that has the components to digitize or capture a video input signal.
  • Camera. A generic term to refer to any physical device used for creating image/video data and that can be connected to a frame grabber.
  • Frame grabber. A type of imaging board which can acquire video data from a camera.
  • Digitizer. The acquisition path(s) with which to grab from one camera of the specified type. When several ActiveMIL digitizers are allocated, their device number, along with their DCF, identify if they represent the same path(s) (but perhaps for a different input format) or independent path(s) for simultaneous acquisition.
  • Data input channel (channel). Identifies which camera to use when several of its type can be connected to the same acquisition path(s) (for example, grab from channel 0 or channel 1 of digitizer 0).
  • Independent acquisition path. An acquisition path that can, if required, acquire data from an input source independently from another such path on the same frame grabber. Each independent acquisition path has its own digitizing components to manage all video timing and synchronization for the path.
  • Imaging board. A generic term to refer to all boards that manipulate video data, including: frame grabbers, vision processors, compression boards, and other specialized boards.
  • Timing control unit. A hardware unit (for example, a programmable synchronization generator (PSG) or video capture controller) that is responsible for managing all video timing and synchronization signals.
  • Vision processor. A type of imaging board which can process and analyze image/video data. A vision processor includes a CPU and additional memory.


Data format

User Guide:
Reference levels

Reference:
Digitizer.Format

Before you can grab from a digitizer, you must set the Format property of its Digitizer control to a digitizer configuration format (DCF) that corresponds to your camera. The DCF defines such settings as the input frequency and resolution, and will determine limits when grabbing an image. Once the resources of a Digitizer control are allocated, many of its properties are initialized to the settings specified in the DCF, unless you have set these properties to values other than the default.

A number of predefined DCFs are provided for the basic cameras supported by your digitizer. Refer to the ActiveMIL Reference with regards to the predefined DCFs available for your particular board. ActiveMIL also provides some DCF files that you can load if the predefined DCFs do not suit your needs. At run-time, set the Format property to the predefined DCF or the DCF file to be used. At design-time, only the predefined DCFs are listed in the drop-down list box of the Format property. To add a DCF to the list box, select Add DCF from the General page of a digitizer's property pages. In the presented dialog box, select a DCF file that matches your camera's data format most closely. This adds the DCF to the list box.

If you find a DCF file that is appropriate for your camera, but you need to adjust some of the more common settings, you can do so directly, without adjusting the file. To do this, change the properties of the Digitizer control corresponding to the settings that need to be modified. For more specialized adjustments, you can adjust the DCF file itself, using Matrox Intellicam.

If you cannot find an appropriate DCF file because, perhaps, you have a non-standard camera or other video source, you can create your own DCF file, using Matrox Intellicam. For more information on Matrox Intellicam, refer to the Matrox Intellicam User Guide manual.

If you cannot develop the required DCF using Matrox Intellicam, you should provide the camera specifications to your Matrox Technical Support Engineer. A suitable customized DCF file can then be developed, if your frame grabber supports the camera.


Device number

Reference:
Digitizer.DeviceNumber

In addition to the data format, you must specify the device number of the first acquisition path of the digitizer. Its digitizer configuration format (DCF) establishes the total number of acquisition paths used. ActiveMIL associates a timing control unit (for example, a PSG) on your frame grabber, with these acquisition paths when grabbing. If a timing control unit or the specified aquisition paths are not available when a grab is requested, the grab request must wait until they become available.

When the Matrox frame grabber has only one timing control unit, you must set the DeviceNumber property of the Digitizer control to 0, regardless of the number of acquisition paths available.

If a digitizer can grab while another digitizer on the same frame grabber is simultaneously grabbing, it is said to be independent. If it cannot, it is dependent. The maximum number of digitizers from which you can grab simultaneously equals the number of timing control units on your Matrox frame grabber.

In order for the two digitizers depicted below to grab simultaneously, there must be at least two timing control units on your Matrox frame grabber. In the following image, the first camera is a component RGB camera and the second one is a monochrome camera. They are connected to their associated acquisition paths on a frame grabber.

For more information regarding the number of available timing control units, refer to the Installation and Hardware Reference manual and the ActiveMIL Board-specific notes for your Matrox frame grabber.


Multiple cameras

Reference:
Digitizer.Allocate
Digitizer.CameraLockEnabled
Digitizer.CameraLockSensitivity
Digitizer.CameraUnlockSensitivity
Digitizer.Channel
Digitizer.Control
Digitizer.FieldStart
Digitizer.FieldStartEven
Digitizer.FieldStartOdd
Digitizer.Format
Digitizer.FrameStart
Digitizer.Free
Digitizer.Grab
Digitizer.GrabContinuous
Digitizer.GrabFieldEnd
Digitizer.GrabFieldEndEven
Digitizer.GrabFieldEndOdd
Digitizer.Inquire
Digitizer.MultipleBuffering.Process
Threading.Wait

ActiveMIL supports applications that require input from different cameras. With most Matrox frame grabbers, you can connect multiple cameras. If each camera connects to an independent acquisition path, then you can allocate a series of independent ActiveMIL Digitizer controls and grab from these cameras simultaneously.

Simultaneous acquisition

Simultaneous acquisition is available on Matrox frame grabbers that support multiple independent acquisition paths. To grab from several sources simultaneously, you must create multiple Digitizer controls and then call the Grab, GrabContinuous, or MultipleBuffering.Process method using the Digitizer controls. When allocating the resources of a Digitizer control, you are committing one or more acquisition paths to a specific camera, depending on the format of the camera (for example, an RGB color camera uses three acquisition paths); these acquisition paths are referred to as a digitizer. To perform simultaneous acquisition, your digitizers must use different acquisition paths. For more information, see the Device Number section earlier in this chapter.

Refer to the Board Specific Notes for more information regarding the number of cameras from which you can acquire data simultaneously using your Matrox frame grabber.

Data input channels of aquisition paths

Most acquisition paths have several multiplexed data input channels. This means that they have several data input channels but can only grab from one channel at a time. In this case, the ActiveMIL Digitizer control representing the acquisition path(s) has multiple data input channels. If you have a camera that is connected to a channel other than the first of its digitizer, you must specify the channel using the Channel property of the Digitizer control.

Switching between cameras of the same type

When connecting several cameras of the same data format to different data input channels of a digitizer, create a single Digitizer control and set its Format property to a DCF that is appropriate for the cameras. Then, use the Channel property of the Digitizer control to switch between the cameras.

Switching between cameras of different types

When connecting cameras of different data formats to different data input channels of a digitizer, you must use a different DCF for each camera. You could create a Digitizer control, grab the required frame, free the Digitizer control, and then create the Digitizer control again with the second DCF; this can increase the time required for the operation. ActiveMIL can circumvent this problem by using a fast DCF-switching technique which is outlined in the steps below:

  1. Create as many Digitizer controls as you have cameras with different formats, from which you want to grab.
  2. Specify the required digitizer settings by setting the properties and events of each Digitizer control, including the Channel property of the Digitizer control.
  3. Call the Grab method of a Digitizer control. Note that when a different Digitizer control is used to grab, a DCF switch occurs.

    4. If there is a grab in progress with one Digitizer control, calling any of the following methods of any other Digitizer control will result in an error: Grab, GrabContinuous, MultipleBuffering.Process, Inquire, Allocate, and Free. To synchronize your grabs with different digitizers, use the Wait method of the Threading control.

Ultra-fast channel switching

If you intend to switch channels after grabbing a single field or frame, use the ultra-fast channel switching mode to minimize time between grabs; enable the digFastCameraLock control type of the Digitizer.Control method. This mode is optimized for applications that use several cameras of the same type, and it does impose certain restrictions:

  • Some events of the Digitizer control might be generated only a few frames after the channel switch: FrameStart, FieldStart, FieldStartOdd, FieldStartEven.
  • Trigger inputs are unavailable.
  • In frame mode, the following events of the Digitizer control are unreliable: GrabFieldEnd, GrabFieldEndOdd, GrabFieldEndEven. Note that these events are reliable in field mode.

Channel locking and unlocking

If the synchronization between a digitizer and a camera is uncertain, lock the digitizer to the synchronization signal of the camera after a channel-switch by setting the CameraLockEnabled property of the Digitizer control to True. This provides additional stability. In this case, the digitizer is unlocked from the first camera and then locked to the next camera after a channel-switch occurs.

Typically, ActiveMIL finds the best balance between the fastest lock and the most reliable lock possible, depending on your camera. On some Matrox frame grabbers, you can control channel locking sensitivity using the CameraLockSensitivity and CameraUnlockSensitivity properties of the Digitizer control. Note that the speed and reliability of a camera lock directly affects the subsequent unlock and vice versa (for example, a fast lock might create an unreliable unlock, or a reliable lock might cause a slow unlock).


Digitizer events

User Guide:
Grab mode
Triggered grabs

Reference:
Digitizer.FrameStart
Digitizer.FrameStartEvent
Digitizer.GrabFrameEnd

The Digitizer control contains events that allow you to keep track of the various stages of a grab operation. For example, the FrameStart and GrabFrameEnd events can indicate the start and end of a grabbed frame, respectively. To enable an event, its corresponding property must be set to True. For example, to enable the FrameStart event, the FrameStartEvent property must be set to True.

For an example that uses digitizer events, see MDigGrabEvents.


Grabbing a single field

User Guide:
Digitizer events
Optimizing application performance when grabbing
The data format
Triggered grabs

Reference:
Digitizer.Grab
Digitizer.GrabEnd
Digitizer.GrabFieldNumber
Digitizer.GrabFrameEnd
Digitizer.GrabFrameStart
Digitizer.GrabStartMode

With interlaced scanning cameras, 2 fields are grabbed by default; therefore one call to the Grab method will grab both the odd and even fields. You can change the number of fields to 1 and have ActiveMIL treat each field as one frame by setting the GrabFieldNumber property, grabbing every second row and storing them in sequential rows. Therefore, the grab time is reduced by half. This property can only be set to 1 or 2, and should only be used for interlaced video. When set to 1, each field is treated like a frame and the following digitizer events are aligned with the field: GrabFrameStart, GrabEnd, and GrabFrameEnd. To achieve 60 fps in NTSC or 50 fps in PAL, the GrabStartMode property must be set to digFieldStart.


Line-scan cameras

User Guide:
The data format

If your target digitizer supports it, you can grab from a line-scan camera as you would, for example, an RS-170 type camera. However, you should be aware of how data from these cameras is stored. When acquiring data from a line-scan camera, each line (row) of each destination image band is filled from top to bottom. The operation will only end once the entire image has been filled.


Grabbing to the display

User Guide:
Displaying an image - overview

Reference:
Digitizer.Grab
Digitizer.GrabContinuous
Digitizer.MultipleBuffering.Process
Display.Image
Display.ImageName
Image.CanDisplay

To grab to the display, a displayable image must first be allocated and selected for display (create an Image control, set its CanDisplay property to true and then select the image to the display using the Image property of the Display control). When the Image property of the Display control is set to display the selected image, internal version(s) of the selected image are created in the frame buffer (also called display memory) for display purposes. These images are maintained and synchronized internally.

The frequency with which the selected image and the display synchronize is dependent upon whether performing a monoshot or continuous grab. A monoshot grab updates the selected image first and then the display. A continuous grab updates the display immediately after each frame is grabbed; only the last frame is stored in the selected image for processing. If you need to save/process each grabbed frame, you should perform the grab using the Grab method of the Digitizer control (that is, perform a monoshot grab); alternatively, you can use the MultipleBuffering.Process method of the Digitizer control for its multiple buffering capabilities.

When grabbing, the frame grabber (for example, Matrox Meteor-II) always acts as the bus master. When grabbing to an on-board display section or when grabbing to a graphics controller that supports fast linear-memory accesses to its display memory, the frame grabber can generally transfer all grabbed data directly to display memory. In the latter case, the display memory also has to be physically accessible from the frame grabber. This is the case for most AGP display boards and PCI display boards plugged into the same PCI-X bus segment as the frame grabber.


Reference levels

User Guide:
Data format

Reference:
Digitizer.BlackReference
Digitizer.Brightness
Digitizer.Contrast
Digitizer.Hue
Digitizer.ReferenceChannel
Digitizer.Saturation
Digitizer.WhiteReference

When grabbing monochrome images or RGB images, you can adjust the brightness and contrast of the image by adjusting the black and white reference levels of your digitizer. For composite color image grabs, the brightness, contrast, hue, and saturation of the image can be adjusted using the brightness, contrast, hue, and saturation reference levels, respectively.

Black and white reference levels

When digitizing images, the black and white reference levels determine the zero and full-scale levels, respectively, of the input voltage range. The analog-to-digital converters convert any voltage above the corresponding white reference level to the maximum pixel value, and any voltage below the corresponding black reference level to a zero pixel value.

ActiveMIL linearly represents the distance between the minimum and maximum voltages, in which the black reference level can be adjusted (hardware-specific), as units between 0 and 255. The same is done for the white reference level adjustment range. These units are the values by which you can adjust the specified reference level, using the BlackReference and WhiteReference properties of the Digitizer control.

To calculate the value to pass to these properties, use the following equation with the appropriate voltages for your particular board:

By reducing or increasing either or both the black and white reference levels, you affect the brightness of the image. By reducing one reference level and increasing the other, you affect the contrast of the image. The new reference level might not take effect until the next grab, at which point, a certain amount of delay might be incurred as the hardware adjusts to the reference-level changes.

The smallest voltage increment supported by your board can differ such that consecutive reference-level settings might produce the same result. Also note that the distance between the minimum and maximum voltages of each range is hardware-specific. Consult your board's installation manual or Board-specific notes for more information.

For RGB images, the new levels apply to all channels by default. To adjust the reference level of a specific channel, set the ReferenceChannel property to the desired channel.

Color image reference levels

When grabbing composite color images, you can adjust the brightness, contrast, hue, and saturation of the image. Use the Brightness, Contrast, Hue, and Saturation properties of the Digitizer control, respectively. These can be set to values from 0 to 255; use values appropriate for your particular board.

Note that there might be some delay before the new levels take effect.


LUTs and the digitizer

User Guide:
Generating data into LUTs
The data format

Reference:
Digitizer.LUT
Digitizer.LUTEnabled

If supported by your digitizer, grabbed data is mapped through a physical lookup table (LUT). By default, the physical LUT is transparent; that is, it is loaded with data so that each input pixel is mapped to its original value. You can load the physical LUT with your own custom LUT data if, for example, you want to correct or precondition grabbed data. To do so, use the digitizer's LUT property to set up the LUT data and then enable the use of this data using the digitizer's LUTEnabled property. ActiveMIL uses the data format (DCF) of the digitizer to determine whether a physical LUT is supported. If it is not, an error is generated.

The information in the LUT property and the digitizer's DCF establishes how the physical LUT is actually configured.

Configuration of the digitizer's LUT | Determining factor
Number of entries in the digitizer's physical LUT | Data being grabbed (DCF)
Depth (8- or 16-bit) | The LUT.DataDepth property
Number of bands | The DCF and the LUT.NumberOfBands property

The digitizer's physical LUT is typically configured to have the same number of components (bands) as either the LUT.NumberOfBands property or the data to be grabbed (determined by the digitizer's DCF), depending on which has more bands. The digitizer's physical LUT is also configured to have the same number of entries as the maximum possible value per band of the data to grab. The depth of the digitizer's physical LUT is configured to be either 8- or 16-bits per band, depending on if the LUT.DataDepth property is 8-bit or 16-bit, respectively. Cameras, however, are not so limited. When dealing with a 10- or 12-bit camera, use a 16-bit destination grab image and load the digitizer's physical LUT with the difference zero padded.

To copy the data from the LUT property to the digitizer's physical LUT, the number of entries of the LUT must match the digitizer's input data range. In addition, if the digitizer's physical LUT cannot support the depth of the LUT data, an error will occur.

LUT data is loaded into the digitizer's physical LUT, as follows:

LUT | Digitizer's physical LUT | Result
1 band | 1 band | The LUT is copied directly into the digitizer's physical LUT.
1 band | 3 band | The LUT is copied into each component of the digitizer's physical LUT.
3 band | 1 band | The first band of the LUT is copied into the digitizer's physical LUT.
3 band | 1 band | Each of the LUT's bands are copied into the corresponding component of the digitizer's physical LUT.

If the destination grab image depth is larger than that of the digitizer's physical LUT, the destination grab image's least significant bits are set to zero when the data is grabbed. If the digitizer's physical LUT depth is greater than that of the destination grab image, the most-significant bits of the data (the non-zero values) are used when the data is grabbed.

For more information about LUTs, see Generating data into LUTs.

To revert back to using a transparent LUT, set the LUTEnabled property back to False.


Scaling grabbed data

Reference:
Digitizer.ScaleX
Digitizer.ScaleY

You can scale grabbed data in the horizontal and/or vertical directions, using the digitizer's ScaleX and/or ScaleY properties, respectively. If you scale grabbed data, the stored image size is different from the original image by the specified factors in the X and/or Y direction. For example, if you set both the X- and Y-scaling factors to 0.5, only one column and one row out of two are written to the destination image (starting with the first row and column).

Note the following:

  • The X- and Y-scaling factors are generally independent.
  • Depending on your Matrox frame grabber and camera, some scaling factors might not be available.
  • To disable scaling, set the scaling factors to 1.


Optimizing application performance when grabbing

User Guide:
Double buffering
Grab mode
Grabbing a single field
Triggered grabs

Reference:
AVIFileHandler.Read
AVIFileHandler.Write
Digitizer.Grab
Digitizer.GrabContinuous
Digitizer.GrabMode
Digitizer.GrabWait
Digitizer.Halt

To optimize application performance when grabbing, you can:

  • Set the grab mode.
  • Perform double buffering.
  • Perform multiple buffering.

Grab mode

When grabbing data with the Grab method of the Digitizer control, you can control the synchronization by setting the GrabMode property to either digSynchronous, digAsynchronous, or digAsynchronousQueued (if supported by your frame grabber).

  • If the GrabMode property is set to digSynchronous, your application will be synchronized with the end of a grab operation. In other words, your application will wait until the grab has finished before executing the next command.
  • If the GrabMode property is set to digAsynchronous, your application will not be synchronized with the end of a grab operation. This allows subsequent commands to execute while you are still grabbing. This is a useful option when performing double buffering. Note that another call to the Grab method before the current grab has finished will cause your application to wait until the current grab has finished.
  • If your frame grabber supports queuing, you can set the GrabMode property to digAsynchronousQueued; if another grab is issued before the first one is finished, the grab will be queued on-board, allowing you to perform other processes while waiting for the next grab to execute. Note that you can still force your application to wait until the end of a grab before executing an operation, by calling the GrabWait method.

The GrabMode property does not apply to the GrabContinuous method because the GrabContinuous method is, by definition, asynchronous. You must call the Halt method to stop the grab. (If the GrabContinuous method was synchronous, the call to the Halt method would never be executed to stop the continuous grab.)

Double buffering

Double buffering involves grabbing into one image while processing the previously grabbed image. Double buffering allows you to grab and process concurrently. You will need one Digitizer control and two Image controls. You must switch the destination of the grab between the two images. In addition, you need to synchronize the grabbing and processing so that:

  • You do not process an image until an entire frame has been grabbed into its buffer.
  • You do not grab into the buffer of the image until the previous frame in that buffer has been processed.

For an example of a double buffering implementation, see MDigDoubleBuffering.

Note that double buffering is not recommended for real-time processing when the CPU usage is high.

For a more robust real-time behaviour, use the multiple-buffering technique in the Multiple Buffering subsection later in this section.

Multiple buffering

When a frame takes longer to process than the time required to grab the subsequent frame, you can use a multiple buffering technique to ensure that all processing is completed without losing any frames.

To perform multiple buffering in ActiveMIL, you can use the MultipleBuffering.Process method of the Digitizer control. It allows you to use a list of previously-allocated Image controls to hold a series of sequentially grabbed images and process them as they are being grabbed. These grabs can either continue until stopped (using the MultipleBuffering.Process method with digStart to start and then the MultipleBuffering.Halt method to stop) or continue until all Image controls in the list are filled (using MultipleBuffering.Process with digSequence). In the former case, the MultipleBuffering.Process method grabs round-robin through the list of Image controls; however, care must still be taken to assure that the average time it takes to process a frame is not greater than the frame rate of the camera, so that frames will not be missed.

The MultipleBuffering.Process method enables the ProcessModifiedImage event of the Digitizer control so that an event procedure can be executed every time there is a modification to the content of any image in the specified list. So, every time a new frame is grabbed in an image in the list, the event procedure is executed. If the event procedure modifies the image that triggered it, the event will not be called again for that image until that image is modified by a new grab. This is to avoid infinite recursive calls being generated. Note that, if the buffer is modified by some method other than the event procedure, the event will be called. In addition, if the event procedure modifies another image in the list, the ProcessModifiedImage event will be triggered for that image.

Note that processing is generally faster if the image is not on the display.


Grabbing a sequence of frames in real-time

Previous | Next

To grab a sequence of frames in real-time, use successive, asynchronous calls to the Grab method, specifying a different image for each frame of the sequence. Alternatively, you can make a single call to the MultipleBuffering.Process method of the Digitizer control. After you have grabbed a sequence, you can use the Write method of the AVIFileHandler object to export the sequence of images (compressed of un-compressed 8-bit) to an *.avi file. When exporting, you must specify the number of images and the frame rate (number of images/second) of the sequence. Note, the images to export must be kept in an array.

To reload a sequence, use the Read method of the AVIFileHandler object to import a sequence of images from an *.avi file into separate images. You can import compressed (MJPEG) or un-compressed 8-bit images.


Triggered grabs

User Guide:
Data format
Digitizer events
Double buffering
Grab mode

Reference:
Digitizer.Exposure1Mode
Digitizer.Exposure1TimeDelay
Digitizer.Exposure2Mode
Digitizer.Exposure2TimeDelay
Digitizer.Grab
Digitizer.GrabContinuous
Digitizer.SendTrigger
Digitizer.TriggerEnabled
Digitizer.TriggerMode
Digitizer.TriggerSource

If your Matrox frame grabber supports trigger input, you can set up your digitizer to perform a triggered grab, that is, to grab a frame upon the occurrence of an event. In this case, nothing is grabbed when you call the Grab or GrabContinuous method, until the specified event occurs. When grabbing continuously, the digitizer waits for a trigger before grabbing each frame; you must still call the Halt method of the Digitizer control after grabbing all required frames.

The digitizer configuration format (DCF) can initialize the digitizer to perform a triggered grab and exactly how it should be carried out. For example, if the DCF specifies that an exposure signal should be generated (for the camera) upon the grab trigger event, the actual grab would only be triggered once the active exposure time was over.

You can explicitly set the properties of the Digitizer control to override the DCF trigger settings. For example, you can enable/disable whether the Grab and GrabContinuous methods perform a triggered grab using the TriggerEnabled property. You can also specify the source and activation mode of the event upon which to grab, using the TriggerSource and TriggerMode properties of the Digitizer control, respectively.

Asynchronous reset mode and next valid frame mode

There are two possible grab activation modes: asynchronous reset mode and next valid frame mode. Note that the grab activation mode is specified in the DCF.

If your frame grabber and your camera support asynchronous reset mode, you can set up the digitizer to reset the camera to begin a new frame when the trigger signal is received.

Otherwise, the digitizer waits for the next valid frame (or field) before starting to grab.

Setting the exposure

If your frame grabber and your camera support it, you can set up your digitizer to transmit an exposure signal to your camera, to specify when to start exposing a new frame and the duration for exposure. To do so, you must enable and set-up the required exposure timers and grab trigger connections as illustrated below.

Software triggers

In general, the digitizer's grab trigger receiver and exposure timers can also be triggered by software if the TriggerSource property is set to digSoftware. In this case, following a grab call, nothing is grabbed until you call a specific method. To issue a software trigger for the grab trigger receiver, call the SendTrigger method with digGrabTrigger. To issue a software trigger for one of the exposure timers, call the SendTrigger method with digTimer1Trigger or digTimer2Trigger to activate the grab exposure Timer1 or Timer2, respectively. Note that when using a software trigger, the grab call must be asynchronous (that is, you must issue the grab with the Grab method in asynchronous mode or with GrabContinuous), or the grab call must be called on a separate thread.

Note, for a frame grabber without an exposure timer, the exposure time is considered to be zero.


Auto-focusing

See Also | Previous
Reference:
Digitizer.Focus
Digitizer.FocusParameters
Digitizer.FocusParameters.FocusRegion
Digitizer.Image
Digitizer.MoveLens

You can use the Focus method of the Digitizer control to automatically adjust the lens motor of your camera to a position that produces optimum focus in your images. This method is primarily useful when the quality of the focus in your images is unacceptable and physically moving the grabbed object is not possible.

The Focus method determines the optimum focus position by grabbing an image at an initial lens position, analyzing the focus quality of the grabbed image, and then grabbing and analyzing an image at a different lens position. The process repeats until the optimum focus position is found. You must add a MoveLens event procedure that physically changes the position of the lens motor between grabs.

The focus quality of an image (known as its focus indicator) is measured by analyzing its edges. An image with good focus quality (a high focus indicator) has well-defined edges, that is, has a sharp difference in gray-levels between its object edges and its background.

By default, the Focus method subsamples and filters each grabbed image before analyzing it. This makes it easier to analyze the image. If necessary, you can specify that the subsampling and/or filtering operations be skipped. Skipping these operations will result in a more accurate analysis of the image's focus quality. It is primarily useful to skip these operations when your images contain fine details since subsampling or filtering can remove these details. Note that subsampling the grabbed images increases the speed of the focus operation; filtering the grabbed images slows down the focus operation.

If necessary, you can specify that only a sub-region of the image be analyzed, using the FocusParameters.FocusRegion property. Analyzing only a sub-region is primarily useful if there are objects at different distances within the camera's field of view. In such a case, each object will have a different optimum focus position, so you need to specify the region on which to focus.

For an example of auto-focusing, see MDigAutoFocus.

Search strategies

The properties of the FocusParameters property specify the minimum, maximum, and starting position of the lens motor. Given these positions, different strategies can be used to find the optimum focus position. These strategies determine how the position is updated (in which direction and by how much) between grabs. They can affect the speed and accuracy of the operation. The default search strategy used by ActiveMIL is the smart-scan strategy.

Bisection strategy

The bisection strategy breaks down the given positional range, step-by-step, until it finds the optimum focus position.

In general, the bisection strategy processes the fewest amount of images. However, it is most sensitive to noise and requires that the lens motor travel the greatest distance.

Refocus strategy

The refocus strategy scans upward or downward until it finds the optimum focus position or until it reaches the minimum or maximum position. While scanning in one direction, if the focus indicator decreases continuously (indicating an out-of-focus condition), the focus position is returned to its starting point and scanning is started in the opposite direction. By default, if a peak in focus indicator values is found, the next two positions are scanned to make sure the peak is truly the optimum. If necessary, you can change the number of positions used to verify a peak.

The refocus strategy is the best strategy to use when the current focus position is close to optimum.

Scan-All strategy

The scan-all strategy scans, by 1, all positions between the minimum and maximum and returns the position which produced the highest focus indicator.

The scan-all strategy is the slowest but most accurate.

Smart-Scan strategy

The smart-scan strategy performs three refocus searches, each with a smaller positional increment. You specify the initial positional increment; the subsequent increments are factors of the initial one. As with the refocus strategy, the default number of positions used to verify a peak is 2 but can be changed.

The smart-scan strategy is a compromise between the speed of a bisection and the accuracy of a scan-all.

Evaluate the focus indicator

Rather than determine the optimum focus position, the Focus method can be used to simply return the focus indicator value for the image specified by the Image property. This can be a grabbed or loaded image.


Compression

Chapter 18: Compression

This chapter describes how to compress and decompress images.

  • Compressing images - overview
    • JPEG lossless
    • JPEG lossy
    • Interlaced JPEG
    • JPEG2000 lossless and lossy
    • Decompression
    • Control options
  • General steps to compress or decompress images
    • Compression
    • Decompression
    • Sequences
    • Multi-band images, color formats, and control settings - JPEG compression
    • Multi-band images, color formats, and control settings - JPEG2000 compression
    • Application-specific markers
  • Controlling a JPEG lossless compression
    • Predictive coding
    • Huffman encoding
  • Controlling a JPEG lossy compression
  • Restart markers
  • JPEG2000
    • Preparation of source image
    • Discrete wavelet transform (DWT)
    • Quantization
    • Bit-plane decomposition
    • Arithmetic encoding
    • Post-processing
  • Improving results
  • Working with tables
    • Inquiring values in default tables
    • Using your own table


Compressing images - overview

See Also | Next
Reference:
Image.CompressionType
Image.JPEG2000Algorithm
Image.JPEGAlgorithm

ActiveMIL allows you to compress and decompress images and sequences. Compression allows you to store more images in memory than would normally be possible. In addition, compression allows images to be transferred more quickly, since it reduces the amount of data that must be transferred. ActiveMIL supports both lossy and lossless JPEG and JPEG2000 algorithms.

Once you have set an image's CompressionType property, any data transferred to the image (using the Copy, Grab, or Load methods), will be automatically compressed. Similarly, any compressed data transferred to a non-compressed image (using the Copy or Load method) will be automatically decompressed. Saving an image to file (using the Save method) can also result in compression or decompression, depending on the compression types of the image and file.

ActiveMIL-Lite compression support is reliant upon the presence of Matrox compression acceleration hardware. The compression algorithm supported depends on the hardware.

JPEG lossless

The JPEG lossless algorithm compresses images without any loss of information. Typically, the algorithm compresses images by a factor of 2:1, although a factor of 4:1 can sometimes be achieved. The JPEG lossless algorithm can compress 8- or 16-bit images with 1 or 3 bands.

JPEG lossy

The JPEG lossy algorithm compresses images by a variable factor but introduces some loss of information. The higher the compression factor, the more the compression, but the lower the image quality. The JPEG lossy algorithm can compress 8-bit images with 1 or 3 bands. To be compatible with most image-viewing software, ActiveMIL allows you to store compressed color images in YUV format.

Interlaced JPEG

ActiveMIL can perform a JPEG compression such that the image data is stored in separate fields. This is referred to as an interlaced JPEG compression. Unless otherwise stated, everything that applies to a JPEG compression also applies to an interlaced JPEG compression.

JPEG2000 lossless and lossy

JPEG2000 is another standard for image compression. When compared with regular JPEG lossy compression at the same compression ratio, JPEG2000 lossy compression provides better image quality. JPEG2000 lossless compression permits a smaller image size while retaining the same image quality as regular JPEG lossless compression. The JPEG2000 lossy and lossless algorithms can compress 8- or 16-bit images with 1 or 3 bands (RGB or YUV). For a more detailed description of supported features, see the section, JPEG2000.

Decompression

Note that you should not change any compression settings of an image before decompressing it. This is because, in order for the reconstructed image to match the original, the same settings must be used to decompress. If you do change a setting, the image data becomes irrelevant and is therefore erased.

Control options

ActiveMIL allows you to control certain aspects of a compression. For example, you can use your own compression tables, although the default tables are suitable for most applications.


General steps to compress or decompress images

User Guide:
Controlling a JPEG lossless compression
Controlling a JPEG lossy compression
JPEG2000

Reference:
Image.AssignMemory
Image.CompressionType
Image.FileFormat
Image.JPEG2000Algorithm
Image.JPEG2000Algorithm.TargetSize
Image.JPEGAlgorithm

Compression

To compress an image:

  1. Create an Image control to contain the compressed image. Set its CompressionType property to the required compression setting.

    2. Images with a compressed attribute that are created with the AssignMemory method of the Image control, should not be used as the destination of an ActiveMIL operation.

  2. Set the properties of the JPEGAlgorithm property or the JPEG2000Algorithm property to control the compression operation. For example, for JPEG2000 compression, you might want to change the quantization factor, which is one of the factors that determine the amount of compression. The default value of the quantization factor is 50; setting a lower value will produce marginal improvement in image quality and will result in a larger file size; setting a higher value will produce a smaller file, and therefore a poorer quality image.
  3. To compress an image, use the Copy method of the Image control created in step 1, and specify the image to compress as the source image. If the image to compress is stored in a file, use the Load method. If you want the compressed image stored on file rather than in an image, set the FileFormat property of the image to compress to the appropriate compression format and then call the Save method of that image.

    4. You can also automatically compress your grabbed images by calling the Grab method of the Digitizer control, using the Image control created in step 1 as the destination image. Note that due to the computational complexity of JPEG2000 compression, grabbing into an image with a JPEG2000 compression type presents a risk of missing frames.

    Compression operations are optimized when both the source and destination Image controls have the same data format. Typically, images in YUV16 format produce the best compromise for quality and speed.

Decompression

To decompress an image, use the Copy, Load, or Save method, depending on whether the compressed image is in an Image control or on file, and where you want the image written (to an Image control or a file). Note that if a compressed image is used as the source image for an operation, the data will be decompressed depending on the attributes of the destination image.

Before the decompression, you should not change any control settings in the source image; the same controls must be used for decompression, otherwise the image data will be lost. The only exception to this rule is for JPEG2000 lossy decompression, where you can change the target size of the image (the JPEG2000Algorithm.TargetSize property).

Decompression operations are optimized when both the source and destination Image controls have the same format. Typically, images in YUV16 format produce the best compromise for quality and speed.

Decompressing a JPEG image into a YUV16 packed (YUYV) image might accelerate transfer to the display.

Sequences

When compressing sequences, call the Open method of the AVIFileHandler object to import a sequence of images from an audio video interleave (AVI) file into separate compressed images. Use the Write method of the AVIFileHandler object to export a sequence of compressed images to an AVI file.

Multi-band images, color formats, and control settings - JPEG compression

When you create a multi-band Image control for a JPEG lossy compression, you can specify that the compressed image be stored in an RGB or YUV format. YUV is convenient because most image-viewing software support compressed color images in YUV16 format.

If you are compressing a multi-band image, you can specify different compression settings for each band. To do so, create a child image from each band, then set each child image's compression settings.

Multi-band images, color formats, and control settings - JPEG2000 compression

When you create a multi-band Image control for a JPEG2000 compression, you can specify that the compressed image be stored in an RGB or YUV format. In addition, you can specify different control settings for each band. To do so, specify the appropriate band index when setting any of the properties of the JPEG2000Algorithm property.

Application-specific markers

During a compression, ActiveMIL adds some application-specific markers to the resulting image. Most other packages will ignore these markers and therefore be able to decompress the file. ActiveMIL itself ignores unrecognized markers when it decompresses files.


Controlling a JPEG lossless compression

User Guide:
Compressing images - overview
Controlling a JPEG lossy compression
Improving results
Restart markers
Working with tables

Reference:
Image.JPEGAlgorithm
Image.JPEGAlgorithm.HuffmanDC

This topic provides a brief overview of the JPEG lossless algorithm and of the control that you have over the algorithm. In general, you should only change a control if you are familiar with the algorithm. The section, Improving results, summarizes techniques to use to improve compression operations. For detailed information about the JPEG lossless algorithm, see Information technology -- Digital compression and coding of continuous-tone images: Requirements and guidelines, which is available from the International Standards Organization (www.iso.org).

The JPEG lossless algorithm is basically a two-step process. First, predictive coding is performed on the image. Then, the result is Huffman encoded.

Predictive coding

Predictive coding is based on the fact that adjacent pixels in an image generally have similar values. Therefore, the value of a pixel can be "predicted" from the values of its neighbor(s). The difference between the original value of the pixel and the predicted value requires fewer bits to store than the original pixel value.

ActiveMIL supports three types of predictive coding: predictor #0 (no predictor), predictor #1 (the "pixel-to-the-left" predictor), and predictor #2 (the "pixel-above" predictor). By default, ActiveMIL uses the pixel to the left to predict values. This is suitable for most images. In some applications, you might prefer to use the pixel-above predictor. You can also specify no predictor (predictor #0), but note that in this case, the values after predictive coding will be the same as the original values. This predictor can be useful if you have developed your own algorithm to take the place of predictive coding and only need your images Huffman encoded. Note that you must implement your own algorithm to use one of the other "predictors" supported by the JPEG lossless algorithm.

You can specify the predictor with the JPEGAlgorithm.Predictor property of the Image control.

Huffman encoding

After an image has undergone predictive coding, Huffman encoding assigns a variable-length "code word" to each value. This code is based on the number of bits by which the difference between adjacent values differ. By storing the code word, rather than the actual difference value, further compression can be achieved. Values are assigned code words according to a DC Huffman table. You can use the default DC Huffman table or you can use your own table. To use your own table, set the JPEGAlgorithm.HuffmanDC property. If you want to use your own table, refer to the section, Working with tables.


Controlling a JPEG lossy compression

User Guide:
Compressing images - overview
Controlling a JPEG lossless compression
Improving results
Restart markers
Working with tables

Reference:
Image.JPEGAlgorithm
Image.JPEGAlgorithm.HuffmanAC
Image.JPEGAlgorithm.HuffmanDC
Image.JPEGAlgorithm.Quantization

This topic provides a brief overview of the JPEG lossy algorithm and of the control that you have over the algorithm. In general, you should only change a control if you are familiar with the algorithm. The section, Improving results, summarizes techniques to use to improve compression operations. For detailed information about the JPEG lossy algorithm, see the Information technology -- Digitial compression and coding of continuous-tone images: Requirements and guidelines, which is available from the International Standards Organization (www.iso.org).

The JPEG lossy algorithm is outlined below. First the source image must be in the correct format before it can be compressed. Although the JPEG algorithm requires signed source data, the algorithm accepts both signed and unsigned data. Initially the algorithm internally treats all data as unsigned. Then a computational shift is performed to set all the values to signed.

After the computational shift, a color conversion is performed if both the source and destination Image controls have different data formats, for example an RGB source and a YUV destination. Note that conversion to YUV introduces some loss.

Afterwards, each 8x8 block of the image is represented in its frequency domain through a discrete cosine transform, resulting in 1 DC and 63 AC values. Each block is then quantized and Huffman encoded.

Quantization divides each of the 64 values in a block by a specified value, according to a quantization table. After each block is quantized, Huffman encoding assigns a variable-length "code word" to each value. Each DC value in a block is assigned a code word according to a DC Huffman table. The AC values are assigned a code word according to an AC Huffman table. You can control a JPEG lossy compression by using your own quantization and/or Huffman tables. To do so, set the Quantization, HuffmanAC, and/or HuffmanDC properties of the JPEGAlgorithm property of the Image control.


Restart markers

User Guide:
Compressing images - overview
Controlling a JPEG lossless compression
Controlling a JPEG lossy compression
Improving results

Reference:
Image.JPEGAlgorithm.RestartInterval

When an image is compressed, ActiveMIL adds restart markers to the bit stream of the compressed image. A restart marker is a special code that signifies that the encoded bit stream was padded to the next byte boundary before the encoding process was restarted. Restart markers can be useful if you are transmitting the compressed image over a medium that is susceptible to errors. If an error does occur and there are no restart markers, the error will propagate and affect subsequent data. However, if there are restart markers, the error will be confined to the data between markers.

By default, ActiveMIL places restart markers after a certain number of rows of data have been encoded (for lossless compressions) or after a certain number of 8x8 blocks of data have been encoded (for lossy compressions). If necessary, you can use the JPEGAlgorithm.RestartInterval property of the Image control to change the number of rows or blocks between restart markers.

Note that, for a lossy compression with a high compression ratio, too many restart markers can significantly increase the size of the compressed image. In this case, you might want to increase the number of rows or blocks between restart markers, especially if you are not transmitting the image over a noisy medium. In fact, if you are sure that the transmission medium is not noisy, you might want to set the restart interval to 0, that is, not use restart markers. This will increase the compression ratio, as well as reduce the time required to decompress the image.


JPEG2000

User Guide:
Compressing images - overview
Improving results
Working with tables

Reference:
Image.JPEG2000Algorithm.DecompositionLevel
Image.JPEG2000Algorithm.QFactor
Image.JPEG2000Algorithm.TargetSize

This section provides a brief overview of the JPEG2000 lossy and lossless algorithms and the control that you have over these algorithms. Everything applicable to lossy compression is applicable to lossless compression, unless otherwise stated. In general, you should only change compression controls if you are familiar with the algorithm (except for the quantization factor, which you can change without in-depth knowledge, using the JPEG2000Algorithm.QFactor property of the Image control). The section, Improving results, summarizes techniques to use to improve compression operations. For more detailed information about the JPEG2000 algorithm, see www.jpeg.org .

There are two fundamental differences between the JPEG2000 and regular JPEG compression algorithms. The first is the use of a discrete wavelet transform (DWT), instead of a discrete cosine transform. The second is the use of arithmetic encoding instead of Huffman encoding as the entropy encoding technique; arithmetic encoding reduces the number of bits required to encode data, based on how frequently that data occurs in the image.

When compared to regular JPEG compression, JPEG2000 supports much higher ratios of compression without compromising image quality. Note however, that JPEG2000 is best suited for archiving purposes because of the processing time required to compress and decompress images; therefore, JPEG2000 can be used for grabbing, but it presents a risk of missing frames.

When compressing images with the JPEG2000 algorithm, ActiveMIL supports:

  • Custom quantization tables (using the JPEG2000Algorithm.Quantization property of the Image control).
  • Custom wavelet settings (setting the number of decomposition levels using the JPEG2000Algorithm.DecompositionLevel property of the Image control).
  • Saving files either in regular JPEG2000 format or in JPEG2000 format with JP2 standard, as well as saving AVI sequences in JPEG2000 format.
  • Band-specific compression settings.
  • Easy control over image size and quality (setting the quantization factor using the JPEG2000Algorithm.QFactor property of the Image control).
  • Specifying a target size for the compressed image when performing a JPEG2000 lossy compression (using the JPEG2000Algorithm.TargetSize property of the Image control).

Note that ActiveMIL does not support the following JPEG2000 features:

  • Progressive encoding/decoding (layering).
  • Random bit-stream access (tiling and/or ROI coding).
  • The JP2 file format optional features.
  • Error resilience (not typically used in imaging applications).

The JPEG2000 compression consists of the following steps:

  1. Preparation of source image.
  2. Discrete wavelet transform.
  3. Quantization (lossy only).
  4. Bit-plane decomposition.
  5. Arithmetic encoding.
  6. Post-processing (lossy only).

Preparation of source image

Although the JPEG2000 algorithm requires signed source data, the algorithm accepts both signed and unsigned data. Initially the algorithm internally treats all data as unsigned. Then a computational shift is performed to set all the values to signed.

After the computational shift, a color conversion is performed if both the source and destination Image controls have different data formats, for example an RGB source and a YUV destination. Note that conversion from RGB to YUV naturally introduces some loss.

Discrete wavelet transform (DWT)

After preparation of the source image, a lossy or lossless discrete wavelet transform (DWT) is applied to the image. The discrete wavelet transform both subsamples and spatially filters the image. Depending on the type of compression, the DWT uses one of two sets of filters: for lossy compression, a Daubechies 9-tap/7-tap filter is used, while a 5-tap/3-tap filter is used for lossless compression. The DWT subsamples and then separates the data into high frequency areas and low frequency areas; these areas are referred to as sub-bands. Each iteration, which consists of one pass of both the high and low frequency filters in both the horizontal and vertical directions, results in four sub-bands. Subsequent iterations of the transform are always applied on the top-left (low frequency) sub-band. ActiveMIL refers to each iteration of the DWT as a decomposition level.

The default number of decomposition levels is 5, unless the width or the height of the smallest sub-band is less than 16 pixels; in which case, the number of decomposition levels is chosen so that the width or the height of the 4 smallest sub-bands of the image is 1 pixel. You can override the default using the JPEG2000Algorithm.DecompositionLevel property of the Image control. For example, if your original image is 1024 x 768, by default, the number of decomposition levels is 5, and the size of the 4 smallest sub-bands is 32 x 24. If you set the JPEG2000Algorithm.DecompositionLevel property to 6, then the size of the smallest sub-bands would be 16 x 12. The largest number of decomposition levels is the one that yields a width or a height of 1 pixel for the 4 smallest sub-bands. After creating an Image control, you can inquire the number of decomposition levels that will be used, using the JPEG2000Algorithm.DecompositionLevel property of the Image control.

After the wavelet transform, each pixel in the original image becomes a wavelet coefficient. There will always be as many wavelet coefficients as there were pixels in the original image.

After the wavelet transform is applied, ActiveMIL no longer considers the data as an image.

Sub-bands that are larger than 64 x 64 are further segmented into as many 64 x 64 regions as possible. These regions are referred to as code-blocks. Smaller sub-bands are not segmented, and each contains one code-block. In the diagram below, the sub-band that is 128 x 128 contains four code-blocks, while the sub-bands that are 64 x 64, 32 x 32, and 16 x 16 each contain one code-block.

Segmentation optimizes compression by grouping areas with similar values; when the values to compress are similar, the result is a better compression ratio. Segmentation also optimizes access to Host memory, thereby improving the performance of the quantization (for JPEG2000 lossy compression), bit-plane decomposition, and arithmetic encoding steps in the algorithm.

Quantization

During JPEG2000 lossy compression, the wavelet coefficients contained in each code-block are quantized based on both the sub-band in which they fall, and an entry in the quantization table. Quantization multiplies each wavelet coefficient by its sub-band's corresponding entry, the quantization coefficient, in the table. The sub-band's rank in significance, which is illustrated below, determines which entry in the quantization table is used. Note that the top-left sub-band is the most significant.

JPEG2000 lossless compression does not use the quantization step.

ActiveMIL automatically determines the values for the quantization table based on the number of sub-bands resulting from the DWT transform; that is, all images of the same depth that have five decomposition levels will use the same quantization table. The first entry (for sub-band 0) always has the largest quantization coefficient.

When you adjust the JPEG2000Algorithm.QFactor property, ActiveMIL scales all the quantization coefficients in the quantization table according to the specified factor. If you want more control, you can pass your own custom quantization table. To establish good values for a custom quantization table, you can inquire the default quantization coefficients. See the section Working with tables for more detailed information.

Bit-plane decomposition

After the wavelet coefficients are multiplied by the quantization coefficients, the code-blocks are decomposed into bit-planes. A bit-plane contains the nth bit of all the multiplied wavelet coefficients that occupy a specific code-block. Decomposition into bit-planes is necessary for arithmetic encoding and post-processing.

A specific code-block might not have the same number of bit-planes as another code-block. Decomposition starts from the most-significant non-zero bit-plane of a code-block, and all subsequent bit-planes are decomposed, even eventual zero bit-planes. For example, the code-block in the diagram below contains coefficients that are 3 bits, and therefore will be decomposed into 3 bit-planes.

Arithmetic encoding

The bit-planes are then passed to the arithmetic encoder. The arithmetic encoder analyzes the bit-plane data to find redundancies, in other words, patterns of bits that occur frequently in the bit-planes. Using this information, the arithmetic encoder replaces repeating bit patterns, also called symbols, by shorter bit sequences. The more often a symbol occurs in the bit-plane data, the fewer bits into which that symbol will be coded.

During lossless compression, the arithmetic encoder processes each bit-plane to produce a continuous bit-stream. Since arithmetic encoding is the final stage of compression, this bit-stream is actually the compressed image.

During lossy compression, the arithmetic encoder produces several independent bit-streams; the post-processing step determines which bit-streams are concatenated in the final image.

Post-processing

Once the most-significant bit-plane of a code-block is encoded, its size in bytes is stored in memory, and ActiveMIL calculates how much error would be present in the decompressed image if the remaining bit-planes, for that code-block, were not arithmetically encoded.

Once the second most-significant bit-plane of the code-block is arithmetically encoded, ActiveMIL calculates the error introduced in the decompressed image if only the first two bit-planes were compressed. As each remaining bit-plane is encoded, ActiveMIL repeats the same calculation until all the bit-planes have been processed.

As an example only, the values in the following table represent the amount of distortion (error) with respect to the size in bytes of the encoded bit-planes of one code-block.

Bit-plane (in order of significance) | Compressed size in bytes | Distortion
1 | 5 | 58817925.11
2 | 54 | 34495584.39
3 | 159 | 11160557.81
4 | 296 | 2633195.13
5 | 436 | 594555.28
6 | 574 | 94112.98
7 | 717 | 21139.81
8 | 863 | 4080.74
9 | 980 | 0.00

Then, the best compromise between decompressed image quality and requested size in bytes (specified by the JPEG2000Algorithm.TargetSize property) is determined. Any encoded bit-planes that are not required are discarded after the best compressed image is determined for the specified size. The remaining encoded bit-planes are then concatenated in the final image.

Determining an appropriate target size will require some trial and error, but a logical guess can be calculated by dividing the image's original size (in bytes) by the required compression ratio. For example, if you want to compress a 256-byte image by a factor of 16:1, you would specify a target size of 16 bytes.

Post-processing is not performed in JPEG2000 lossless, and none of the encoded bit-planes are discarded; therefore, you cannot specify a target image size when performing a lossless compression.


Improving results

User Guide:
Compressing images - overview
JPEG2000
Working with tables

If the defaults do not meet your application requirements, you can try to improve your compression ratio using the following techniques. We recommend trying these techniques in the order they appear.

Regardless of the type of your compression operation, you should first remove extraneous noise from the image (if possible) using an ActiveMIL ImageProcessing control.

For JPEG lossy compression:

  • Use an Image control that has a YUV data format to store the compressed image.
  • Increase the quantization factor with the JPEGAlgorithm.QFactor property of the Image control.
  • Decrease the restart interval with the JPEGAlgorithm.RestartInterval property.
  • Change the quantization table. See the section, Working with tables.
  • Change the Huffman table. See the section, Working with tables.

For JPEG lossless compression:

  • Try the other supported predictors by changing the setting of the JPEGAlgorithm.Predictor property.
  • Decrease the restart interval with the JPEGAlgorithm.RestartInterval property.

For JPEG2000 lossy compression:

  • Use an Image control that has a YUV data format to store the compressed image.
  • Increase the quantization factor with the JPEG2000Algorithm.QFactor property.
  • Decrease the target size with the JPEG2000Algorithm.TargetSize property.
  • Change the number of decomposition levels of the DWT with JPEG2000Algorithm.DecompositionLevel property.
  • Pass another quantization table. See the section, Working with tables.

For JPEG2000 lossless compression:

  • Change the number of decomposition levels of the DWT with JPEG2000Algorithm.DecompositionLevel property.


Working with tables

See Also | Previous
Reference:
Image.JPEG2000Algorithm.QFactor
Image.JPEG2000Algorithm.Quantization
Image.JPEGAlgorithm.HuffmanAC
Image.JPEGAlgorithm.HuffmanDC
Image.JPEGAlgorithm.QFactor
Image.JPEGAlgorithm.Quantization

In some applications, the default quantization or Huffman table values might not be suitable. ActiveMIL allows you to inquire the current values and to specify your own with the HuffmanAC, HuffmanDC, and Quantization properties. Inquiring the default values of the tables can help you determine appropriate values, but you can only inquire all values in the table; you cannot inquire a specific entry. To determine the best values for your application, you might have to set values by trial and error .

For JPEG2000 compression, quantization multiplies coefficients, while in JPEG compression, quantization divides values.

Whether you are inquiring the default table values or specifying your own, you must allocate arrays that are large enough to contain all the table values. The following lists the compression tables that you can manipulate and the required array size for each compression type.

Compression type | Table type | Array type
JPEG lossless | DC Huffman | 1-dimensional, 28-entry array of type unsigned Byte (unsigned BYTE).
JPEG lossy | DC Huffman | 1-dimensional, 28-entry array of type unsigned Byte (unsigned BYTE).
 | AC Huffman | 1-dimensional, 178-entry array of type unsigned Byte (unsigned BYTE).
 | Quantization | 2-dimensional, 8x8-entry array of type unsigned Byte (unsigned BYTE).
JPEG2000 lossy | Quantization | 1-dimensional array of type Single (float) with 3N+1 entries, where N is the number of decomposition levels.

If you set the QFactor property after specifying a custom quantization table, this table will be scaled accordingly.

Inquiring values in default tables

Inquiring the default values of a table is useful to determine values for your custom tables. The steps below outline this procedure.

  1. First, inquire the identifier of the default table using Image.HostAddress. Then, inquire the size of the table using the Image.DataSize property.
  2. Allocate a user array of the appropriate size for storing the default table values.
  3. Get the values from the inquired table in Step 1 and store them in the user array using the Image.Get method.

Using your own table

To use your own table:

  1. Create a buffer with the same data type as the default table.
  2. Transfer the custom table values from the user array to the array buffer, using the Image.Put method.
  3. Associate the buffer to the required compressed image buffer, using the compression control types specific to your table (for example, use the HuffmanAC control type for an AC Huffman table). Specifying these control types as-is, or with the QFactor property set to imAllBands, controls all bands for JPEG2000. This is the default setting.

    4. For a JPEG2000 compression, you can associate a different table with each band of a multi-band buffer. To do so, add the NumberOfSubband property set to imRed, imBlue or imGreen to your control type for an RGB buffer, whereas for a YUV buffer, add imY, imU, or imV.

    For JPEG lossy compressions of YUV buffers, use the luminance and chrominance control types. The control types without these suffixes control all bands.

If you set the QFactor property after specifying a custom table, the custom table will be scaled.


Distribution and licensing

Chapter 19: Distribution and licensing

This chapter presents how to distribute and license ActiveMIL applications.

  • Distribution of ActiveMIL applications
  • Redistributing ActiveMIL or ActiveMIL-Lite run-time environment files with your application
    • Redistributing directly from the MIL or MIL-Lite CD
    • Redistributing using your own setup program
  • Normal redistribution
  • Silent redistribution
    • Important notes when using Windows 2000/XP
  • Uninstalling
  • ActiveMIL and ActiveMIL-Lite licenses
    • Development license
    • Temporary license
    • Run-time license
  • Hardware license-key
  • Software license-key
  • Hiding the ActiveMIL license
    • Generating the lock code
    • Getting a software license-key
    • Registering the software license-key
  • Gencode utility
    • Synopsis
    • Protoype
    • Description
    • Parameters
  • Protecting your own ActiveMIL software application using Matrox hardware fingerprint


Distribution of ActiveMIL applications

See Also | Next
User Guide:
Installation
Normal redistribution
Redistributing ActiveMIL or ActiveMIL-Lite run-time environment files with your application
Silent redistribution

There are some details that you must consider before you can distribute an ActiveMIL or ActiveMIL-Lite application, either for your customers' use or for your own. When doing this distribution, you must redistribute ActiveMIL or ActiveMIL-Lite run-time environment files (for example, OCXs and DLLs) and device drivers. In addition, when using an ActiveMIL processing control, you must purchase an appropriate run-time license; otherwise, your ActiveMIL application will stop functioning after 30 days. This chapter deals with the different ways to distribute your application, purchase a license, and handle licensing concerns for your customers.

Note that ActiveMIL-Lite applications don't require a run-time license.


Redistributing ActiveMIL or ActiveMIL-Lite run-time environment files with your application

User Guide:
Normal redistribution
Silent redistribution

To distribute your ActiveMIL or ActiveMIL-Lite application, you will have to redistribute the ActiveMIL or ActiveMIL-Lite run-time environment files and the necessary device drivers with your application.

It is important to remember that only one copy of ActiveMIL or ActiveMIL-Lite can be present on a computer at a time. When installing an application that uses ActiveMIL or ActiveMIL-Lite on a computer with the same version already installed, the application's set-up program must not reinstall ActiveMIL or ActiveMIL-Lite. The application should use the version of ActiveMIL or ActiceMIL-Lite already installed on the computer.

Conversely, if the version of ActiveMIL or ActiveMIL-Lite on the computer is different from the application's required version, a decision must be made. The latest version of ActiveMIL or ActiveMIL-Lite should be kept or installed, otherwise the application cannot be installed on that computer. Note that the application might need to be recompiled with the installed version of ActiveMIL or ActiveMIL-Lite.

Redistributing directly from the MIL or MIL-Lite CD

If the target computer (on which you want to install the ActiveMIL or ActiveMIL-Lite run-time environment files and device drivers) is immediately accessible, you can install the run-time environment files directly from the MIL or MIL-Lite CD. To do so, run the MIL or MIL-Lite setup program and choose the MIL run-time option.

Redistributing using your own setup program

Alternatively, to redistribute your ActiveMIL or ActiveMIL-Lite run-time environment files and device drivers, you can have your application's setup program call MIL or MIL-Lite's redistribution setup program in one of two ways:

  • Normal redistribution. Prompts your customer for setup information.
  • Silent redistribution. Uses a custom response file, instead of prompting your customer for information.

These are described in detail in the next sections.


Normal redistribution

User Guide:
Silent redistribution

To redistribute the ActiveMIL or ActiveMIL run-time environment files and device drivers, you can have your application's setup program call MIL or MIL-Lite's redistribution setup program using the normal redistribution mechanism. In this case, your customer will be prompted for information during the setup. There are two ways to implement this mechanism.

To redistribute all the run-time environment files and device drivers and use the default installation options, do as follows:

  1. Copy the contents of MIL or MIL-Lite CD to your installation directory.
  2. Have your redistribution program call the setup.exe file, located in the root of your installation directory.
  3. The setup.exe program will prompt your customer to choose which features to install. Your customer should only select the ActiveMil Runtime option. The ActiveMil Runtime option installs the required run-time ActiveMIL or ActiveMIL-Lite environment files and device drivers on your client's computer.

To limit the contents of the MIL or MIL-Lite CD to include in your installation directory and restrict the installation options, you can use the Redist.exe program. To use this program, do as follows:

  1. Launch the Redist.exe program located in the root of your MIL or MIL-Lite CD. The Matrox Imaging Library Redistribution dialog box is presented.
  2. Specify the location of the source files to copy to your installation directory. To do so, click on the Browse button next to the Source edit field. From the presented Open dialog box, select the setup.exe file located in the root of your MIL or MIL-Lite CD, and then click on the Open button. The Matrox Imaging Library Redistribution dialog box re-appears, displaying the selected location of the source files.
  3. Specify the location of your installation directory. To do so, click on the Browse button next to the Destination edit field. From the presented Choose folder dialog box, select your installation directory, and then click on the Select button. The Matrox Imaging Library Redistribution dialog box re-appears, displaying the selected location of your installation directory.
  4. Select the redistribution packages (ActiveMIL and/or Intellicam) and the drivers that you want to make available to your customer during installation.
  5. Click on the Generate button to begin copying the limited installation version of MIL or MIL-Lite into your installation directory.
  6. Have your redistribution program call the setup.exe file, located in the root of the specified installation directory.
  7. The setup.exe program will prompt your customer to choose which features to install from the features that you have made available.


Silent redistribution

User Guide:
Distribution of ActiveMIL applications
Hiding the ActiveMIL license

A silent redistribution does not prompt your user for any ActiveMil or ActiveMIL-Lite information; instead, it uses a response file to provide the necessary setup parameters for the intended computer. You would use a silent redistribution when you are including ActiveMIL or ActiveMIL-Lite within your application and you do not want to have any Matrox Imaging setup dialog boxes appear. You could also use a silent redistribution if you wanted to control the setup parameters for your client. There are two ways to use the silent redistribution.

It is not possible to create a response file for only a few of the setup parameters and ask the customer for the rest of the setup information. If you are going to use a response file, you must answer all of the setup questions in the response file.

To redistribute the ActiveMIL or ActiveMIL-Lite run-time environment files and device drivers using a silent redistribution:

  1. Copy the contents of MIL or MIL-Lite CD to your installation directory. Alternatively, you can use the Redist.exe program to select the redistribution packages (ActiveMIL and/or Intellicam) and drivers to copy in your installation directory. For more details on how to use this program, see the Normal redistribution section in this chapter.
  2. Record your setup options. To do so, start the setup.exe program from the command line with the /r switch and follow the steps for a normal ActiveMIL or ActiveMIL-Lite redistribution. Any selected setting is recorded and stored in the setup.iss file.
  3. Have your redistribution program call the setup.exe program with the /s switch. The setup will silently install ActiveMIL or ActiveMIL-Lite with the options that were selected during the recording phase and stored in the setup.iss file.

Important notes when using Windows 2000/XP

The following is a list of points to consider if using Windows 2000/XP:

  • Ensure that the operating system has a standard VGA display driver installed prior to installing the MGA drivers.
  • The MGA drivers are only installed for Matrox boards that have an on-board graphics controller (for example, Matrox Corona-II, Matrox Genesis, and Matrox Orion).


Uninstalling

User Guide:
Installation
Silent redistribution

To uninstall previous versions of ActiveMIL or ActiveMIL-Lite, and all other Matrox Imaging Products, call the Matrox Imaging Products Uninstall program. To make uninstalling silent, use the silent uninstall program located in \SilentUninstall on the MIL or MIL-Lite CD.

For example, if you want to uninstall MIL and ActiveMIL in silent mode without rebooting your system, call:

 uninstall.exe

or to reboot the computer after uninstalling, call:

 uninstall.exe reboot


ActiveMIL and ActiveMIL-Lite licenses

User Guide:
Hardware license-key
Software license-key

ActiveMIL and ActiveMIL-Lite have different licensing terms and mechanisms.

When you purchase ActiveMIL-Lite, you receive a development license and a registration number. ActiveMIL-Lite also comes with a royalty-free run-time license, which allows you to redistribute applications based on ActiveMIL-Lite without paying additional royalty fees.

With ActiveMIL, you can purchase two types of permanent licenses: a development license and/or a run-time license. Licenses are always verified when an ActiveMIL application is allocated, as well as when the application is running; this verification has negligible overhead. You can use a temporary license while waiting for a permanent license.

Please refer to the Matrox Software License Agreement for the legal provisions of using/redistributing ActiveMIL or ActiveMIL-Lite.

The rest of this chapter, with the exception of the last section, deals exclusively with ActiveMIL and its licensing mechanisms. The last section pertains to both ActiveMIL and ActiveMIL-Lite.

Development license

A development license permits you to run and debug ActiveMIL applications with Microsoft Visual Studio. For the license to take effect, a development hardware license-key must be attached to your computer's parallel or USB port. A development hardware license-key is included with the ActiveMIL starter or additional developer package.

Temporary license

A temporary license is a 30-day development license. This type of license is most useful for developers who are performing a development or redistribution installation while waiting for a permanent license.

The temporary license is used when ActiveMIL is first executed, unless a development license is already present. However, a temporary license only lasts 30 days from the first launch of ActiveMIL on the computer. After allocating an application, if the temporary license is used, a dialog box appears indicating the number of days until the temporary license expires.

If a development license is removed, a temporary license dialog box will appear indicating the number of days until the license expires, if it is less than 30 days from the first launch of ActiveMIL on the computer. If it is more than 30 days, a dialog box will appear asking you if you want to purchase a run-time license. If you do not purchase a run-time license, you won't be able to use the application since ActiveMIL will have been disabled.

Once the temporary license has expired, it is not possible to prolong or restart the 30-day trial period. Attempting to alter the computer's calendar before the temporary license expires will immediately disable ActiveMIL. In that event, ActiveMIL can only be re-used once an appropriate license is installed.

Run-time license

A run-time license is required for each computer that will run an ActiveMIL application. It does not allow for development (debugging) of ActiveMIL applications.

Recall that ActiveMIL-Lite applications do not require a run-time license. They can be distributed royalty-free.

ActiveMIL run-time licenses are purchased according to the requirements of the application. An ActiveMIL run-time license consists of different ActiveMIL controls, grouped in packages, for specific uses. Consult the MIL datasheet or contact Matrox Imaging sales or the local representative regarding the available run-time packages.

An ActiveMIL run-time license can be in one of two formats:

  • A hardware license-key.
  • A software license-key.

If you want to, you can hide the ActiveMIL license from your customer using the Gencode utility, a modified version of the MIL License Manager. Gencode is located in the same directory as the MIL License Manager. See the Gencode utility section.


Hardware license-key

User Guide:
ActiveMIL and ActiveMIL-Lite licenses
Software license-key

A hardware license-key is a type of run-time license. For the license to take effect, the key must be attached to the parallel or USB port of your computer. This license can be bought by calling Matrox Imaging or your local representative.

If you decide that another computer needs to run an ActiveMIL application, you can shift the hardware license-key. This will allow the second computer to use ActiveMIL, but the first computer will have the temporary license dialog box appearing again. If this is done for more than 30 days after ActiveMIL was first used on the computer, ActiveMIL will be disabled.

To have the ActiveMIL run-time environment files recognize the hardware license-key, you must install the Superpro driver. To install this hardware license-key driver, if you did not install it when you were installing ActiveMIL, call the \Matrox Imaging\Drivers\SuperProDriver\wdm\RainbowSSD.exe file and select the Install Sentinel Driver menu command.

For more information about Sentinel driver installation, refer to the DriverInfo.htm file located in the \Matrox Imaging\Drivers\SuperProDriver directory. This file also includes information on how to install the Sentinel driver silently.


Software license-key

User Guide:
ActiveMIL and ActiveMIL-Lite licenses
Gencode utility
Hardware license-key

A software license-key is another type of run-time license. The software license-key uses a fingerprint, generated from one of several components on the target computer, to produce the license.

You can obtain a license as follows:

  1. Install your ActiveMIL application on the target computer as described in Chapter 1. A temporary license is assigned to your computer, allowing use of ActiveMIL for 30 days. Each time your application runs ActiveMIL, a dialog box appears indicating the number of days until the temporary license expires and a setting to obtain a permanent run-time license is displayed. If you click "Yes" to get the license, the MIL License Manager program will be launched. The License Manager program will also be launched if you attempt to use an ActiveMIL control that is not available in the purchased package.
  2. In the License Manager program, choose any combination of the different packages for purchase, by enabling the required package's check box.

    3. Since the Image Analysis package is included in the Machine Vision package, these two packages are mutually exclusive.

  3. Select the hardware component, from which to generate a fingerprint, from the System Fingerprint drop-down list box. After choosing the hardware component, click the Generate button. This will create a lock code.

    4. If you replace the hardware component that you have chosen, you will need a new license.

    For portability or if your computer does not have any of the hardware components listed, you can purchase a hardware ID-key and use it as your hardware component instead. This gets attached to your computer's parallel or USB port, and it will allow you to generate a lock code that you can use to create a software license. If you attach the hardware ID-key to a different computer, the run-time license will only operate on the computer on which the hardware ID-key is installed. The hardware ID-key allows you to quickly shift the MIL license to another computer without removing a hardware component. It also allows you to include other ActiveMIL packages; a hardware ID-key allows you flexibility to choose the packages you want, unlike a hardware license-key.

    You also have to copy over the lservrc file, which is located in the Windows\System32 directory under Windows 2000/XP on the target computer.

  4. After you have generated a lock code, the software license-key can be obtained at www.Shopmatrox.com or by contacting Matrox Imaging sales or your local representative.
  5. Return to the License Manager program, type or paste the software license-key into the Software License Key edit field, and click on the OK button. The licensing process is finished.


Hiding the ActiveMIL license

User Guide:
Gencode utility
Silent Redistribution
Software license-key

It is possible to make the presence of ActiveMIL licensing non-invasive so that your customer can install your ActiveMIL application and not have to deal with ActiveMIL licensing issues.

Perhaps the simplest way to hide the ActiveMIL licensing is to purchase and redistribute hardware license-keys. You will have to purchase a hardware license-key for each application that you are distributing.

If this is not possible or preferable, there is a utility, Gencode, that allows you to get a software license-key while hiding the ActiveMIL licensing process. It is essentially the silent command-line version of the MIL License Manager. Gencode can generate the lock code needed to get an ActiveMIL software license-key, as well as register the license.

Generating the lock code

There are many different ways of getting the information that the utility needs. You could create an interactive interface, or have a series of prompts during the setup of your application. If you know the hardware and the packages your client will be using, you could build a batch file which would call Gencode with the predetermined information. For example, if you know that your client will be using a Matrox Corona-II board, you could hardcode Gencode's command-line parameter SystemFingerprint to the constant that means Matrox Corona-II.

However implemented, the setup program of your application must internally call Gencode as follows to generate the lock code:

 Gencode /G Filename SystemFingerprint Packages

See the Gencode utility section for details.

Getting a software license-key

Once the code is actually created, there are some steps that must be followed.

  1. Your customer must send you the lock code, by whichever means you designate.
  2. Take the lock code and obtain the software license-key at Shopmatrox.com or by contacting Matrox Imaging Sales or your local representative.
  3. Your customer must input the software license-key into whatever registration interface you have created.

Registering the software license-key

Your registration interface must again call Gencode, this time as follows:

 Gencode /L LicenseCode

and input the license code. This will activate your ActiveMIL license.

Gencode's command-line parameters are described in the Gencode utility section.


Gencode utility

User Guide:
Hiding the ActiveMIL license
Silent redistribution

This section describes the Gencode utility.

Synopsis

Generates a lock code for licensing, and activates a software license-key for ActiveMIL.

Protoype

GENCODE /H or

GENCODE /G <Filename> <SystemFingerprint> <Packages> or

GENCODE /L <SoftwareKey> or

GENCODE /I

Description

The Gencode utility does one of the following:

  • With the /H switch, it displays a help screen.
  • With the /G switch, it generates a lock code that is necessary for getting a software license-key from the Matrox web site.
  • With the /L switch, it registers the license-key for running ActiveMIL on the current computer.
  • With the /I switch, it prints the numeric values of available settings of the SystemFingerprint parameter.

Parameters

The following is a list of parameters available:

  • The Filename parameter is the name of the file that will be created to store the lock code. You can specify any file name; if the file name already exists, the lock code information will be appended to the existing file. The lock code can also be retrieved from the registry under:

    • HKEY_LOCAL_MACHINE\SOFTWARE\Matrox\Matrox Imaging Library\Runtime\Redist\Licenses\ManagerOutput

  • The SystemFingerprint parameter designates the hardware component upon which the system fingerprint is based. This parameter can be set to one of the following numeric values:
    • SystemFingerprint settings | Description
7 | Ethernet Controller
8 | Hardware ID-key
20 | Matrox Solios
21 | Matrox Meteor-II /1394
22 | Matrox Orion
23 | Matrox Corona-II
26 | Matrox Odyssey
27 | Matrox CronosPlus
28 | Matrox Helios
29 | Matrox Morphis
31 | Matrox board

    If the specified board is not on the target computer, a 0 is returned.

  • The Packages parameter identifies which ActiveMIL packages are going to be purchased. This is something you would likely set for your customer, since you know which ActiveMIL controls your application uses.

    • The software license-key packages correspond to the following numeric values. If you need to purchase more than one package, add their corresponding numeric values together. For example, to purchase the Machine Vision and Identification packages, set the Packages parameter to 6.

    Packages setting | Description
1 | Image Analysis
2 | Machine Vision
4 | Identification
8 | Compression/Decompression
16 | Geometric Model Finder
256 | Edge Finder
512 | Serial bus driver for third-party IEEE 1394 hardware
1024 | String Reader

    The one exception is that the Image Analysis and Machine Vision packages can not both be included in the value of the Packages parameter. Refer to the Matrox Imagin website for more information on these run-time license packages.

  • The SoftwareKey parameter is the software license-key code that you receive from Matrox or your local representative to finalize registration.


Protecting your own ActiveMIL software application using Matrox hardware fingerprint

See Also | Previous
User Guide:
Hiding the ActiveMIL license
Silent redistribution

You might want to protect your own ActiveMIL software application using a third-party licensing package that requires a hardware fingerprint. In this case, you can use the hardware fingerprint of the Matrox Imaging board that your application uses. The ActiveMIL hardware fingerprint is a numerical value, unique to the Matrox hardware component.

You can inquire about your hardware fingerprint using the Inquire method of the Application control with one of the following:

  • appCoronaIIFingerPrint
  • appCronosPlusFingerPrint
  • appEthernetFingerPrint
  • appHardIdKeyFingerPrint
  • appHeliosFingerPrint
  • appMatroxBoardFingerPrint
  • appMeteorII_1394FingerPrint
  • appMorphisFingerPrint
  • appOdysseyFingerPrint
  • appOrionFingerPrint
  • appSoliosFingerPrint

If the inquired Matrox hardware component is present, the requested hardware fingerprint is returned. If it is not present, 0 is returned.


Using ActiveMIL with Microsoft Visual C++ (Native COM)

Chapter 20: Using ActiveMIL with Microsoft Visual C++ (Native COM)

This chapter explains how to access ActiveMIL controls using Native COM support in Microsoft Visual C++. It begins with a basic introduction to Native COM, illustrating briefly why this interface is superior to wrapper classes. This chapter also discusses how to convert a project that is currently using wrapper classes to use Native COM support.

  • Overview of using ActiveMIL with Microsoft Visual C++
  • What is Native COM?
    • Accessing ActiveMIL controls
    • Dual Interface
  • Smart pointers
  • Methods and properties
    • Access controls and objects by reference
    • Default values for methods
    • Get/Set prefix
    • Access a collection item
    • CString variables
    • Enumerations
    • Comparison conditions
    • Properties that take parameters
  • Using variants and arrays
    • Variants
    • Arrays
      • COleSafeArray
      • SAFEARRAY
      • CActiveMILObjectsArray
  • Using strings
  • Run-time object declaration
  • Error management (Visual C++)
  • Convert an existing project using wrapper classes to use Native COM
    • Access controls and objects by reference
    • Default values for methods
    • Get/Set prefix
    • Access a collection item
    • Inquire methods
    • CString variables
    • Enumerations
    • Comparison conditions
    • Properties that take parameters
    • Preprocessor definitions


Overview of using ActiveMIL with Microsoft Visual C++

See Also | Next
User Guide:
Creating an application and adding ActiveMIL to it (Visual C++)
Error management (Visual C++)
Methods and properties
Run-time object declaration
Smart pointers
Using strings
Using variants and arrays

When programming an ActiveMIL application in Visual C++, you must use Native COM support. This chapter explains what Native COM is and why it is superior to wrapper classes. This chapter also explains how to convert a project that is currently using wrapper classes to use Native COM.

To take full advantage of using Native COM, you must follow certain programming rules that relate to smart pointers, methods and properties, variants and arrays, strings, declaring your ActiveMIL objects at run-time, and error management.

Discussions and examples on using Native COM in this chapter apply to both MFC and the Windows API, unless otherwise stated.

To create a Visual C++ application that uses Native COM support, see the section, Creating an application and adding ActiveMIL to it (Visual C++), in the Introduction chapter.

For more information regarding Microsoft specific concepts, refer to the Microsoft Developer Network documentation (MSDN).


What is Native COM?

User Guide:
Convert an existing project using wrapper classes to use Native COM
Creating an application and adding ActiveMIL to it (Visual C++)

Reference:
Image.SizeX

In the past, there were two ways to access ActiveMIL controls from Visual C++. You could use wrapper classes (which implied the use of MFC), or Microsoft's Native COM (MFC and Windows API). With ActiveMIL version 7.5 or later, and with Microsoft Developer Studio version 6 or later, you can only use Native COM (with or without MFC). This means ActiveMIL's controls can only be accessed directly. This type of access makes applications easier to write, easier to understand, and much faster to run than they would be with wrapper classes.

Accessing ActiveMIL controls

Using Native COM, you are interfacing with ActiveMIL controls by pointing directly to them, for example:

In this example, the SizeX property of ActiveMIL's Image control would be accessed as follows:

Image1->SizeX = 640;

However, if you would have accessed ActiveMIL controls with wrapper classes, you would not be interfacing with the control itself; rather, you would be referencing, through dot notation, an object that references that control. This object would be an instance of the control's wrapper class. The following diagram shows how accessing ActiveMIL controls with wrapper classes would actually be a twofold process.

With wrapper classes, you would call a method of the wrapper class to access the ActiveMIL control. For example, to set or get the value of an ActiveMIL property, you would call its control's wrapper class method that had the same name as the property, except prefixed with "Set" or "Get", respectively. As illustrated above, the SizeX property of ActiveMIL's Image control would be accessed with the following line:

Image1.SetSizeX(640);

Dual Interface

ActiveMIL controls support "dual interfaces". This means that ActiveMIL simultaneously supports both IDispatch (also known as "late binding"), and virtual function tables (also known as "early binding"). When accessing ActiveMIL controls using Native COM, early binding is used. Late binding is only used internally by ActiveMIL wrapper classes, which is no longer supported by ActiveMIL 7.5 and newer.

When using early binding, an object is called directly via a virtual function table indirection, which is implemented internally. This process is the same as C++ virtual methods; the calls are therefore faster, and the compiler can perform more compile-time type checking, which corresponds more with a strongly typed language like C++. Using Native COM support lets you transparently use early binding to access ActiveMIL controls.

When using late binding, an object is internally called with the name of the specified method or property to get the "ID" of that method or property; the object is then called again with that "ID" to actually invoke the method or property. This process is best suited for scripting clients such as JScript and Visual Basic for Application, since these environments cannot perform compile-time type checking. However, late binding involves a significant overhead.


Smart pointers

User Guide:
Using strings
Using variants and arrays
What is Native COM

Reference:
Application.CreateObject

Although regular pointers can be used to access COM objects, it is strongly recommended that smart pointers be used instead. You use a smart pointer like a regular pointer; however, smart pointers prevent memory leaks since the release of COM objects takes place automatically when the smart pointer goes out of scope.

In Native COM, smart pointers are implemented using a class that is built from Microsoft's _com_ptr_t class template; these smart pointer classes contain a pointer to a COM object. Each time a smart pointer points to, or stops pointing to, a COM object, that object's AddRef and Release methods are appropriately called by the smart pointer. The COM object will automatically destroy itself when its reference count goes to zero.

The AddRef and Release methods are inherited from Microsoft's IUnknown interface. All COM objects must inherit from IUnknown and therefore all COM objects implement AddRef and Release.

Each ActiveMIL control has two equivalent smart pointer classes, typically named I<Control>Ptr and C<Control>, while each ActiveMIL object has only one equivalent smart pointer class, typically named I<Object>Ptr. In general, to declare a pointer to access an ActiveMIL control or object, use the I<Control>Ptr or I<Object>Ptr class, respectively. However, to declare a pointer to access an ActiveMIL control that is on an MFC dialog, use the C<Control> class instead. This is because, when you have a control on an MFC dialog, the control must have a window and must therefore use CWnd, which is the MFC class that provides the base functionality for all window classes. Each ActiveMIL C<Control> class is therefore derived from both the CWnd class and from the respective control smart pointer class (I<Control>Ptr); for example, CSystem is derived from both CWnd and ISystemPtr. Consequently, the C<Control> class allows you to use the functionality of both the CWnd class and the control smart pointer class.

Refer to the Visual C++ remarks in the CreateObject method of the Application control for a list of all ActiveMIL I<Control>Ptr smart pointer classes. Refer to the Visual C++ syntax of properties of type object to determine their object's smart pointer class.

The following example illustrates how to declare and use a variable to access an ActiveMIL Image control:

IImagePtr DrawingImagePtr;
DrawingImagePtr = m_Display1->Image;
m_Measurement->Markers->Item(1)->Draw(DrawingImagePtr, measDrawPosition);

The following example illustrates how to declare and use a variable to access an ActiveMIL RegionOfInterest object:

int SizeX = 50;
int SizeY = 65;
int OffsetX = 10;
int OffsetY = 10;
IRegionOfInterestPtr SearchRegionPtr;

SearchRegionPtr = m_Measurement->Markers->Item(1)->SearchRegion;
SearchRegionPtr->Mode = roiOffsetSize;
SearchRegionPtr->OffsetX = OffsetX;
SearchRegionPtr->OffsetY = OffsetY;
SearchRegionPtr->SizeX = SizeX;
SearchRegionPtr->SizeY = SizeY;

As mentioned, you use a smart pointer like a regular pointer; therefore, you can use a smart pointer wherever a normal pointer is needed. For example, the Draw method of ActiveMIL controls requires a pointer to a control (an IDispatch *) for its first parameter; in this case, a smart pointer could be used instead. In the following example, the smart pointer DrawingImagePtr is used:

IImagePtr DrawingImagePtr;
DrawingImagePtr = m_Display1->Image;
m_Measurement->Markers->Item(1)->Draw(DrawingImagePtr, measDrawPosition);

For more information on smart pointers and, specifically, the _com_ptr_t template, refer to MSDN.


Methods and properties

User Guide:
Convert an existing MFC wrapper project to use Native COM
Using strings
Using variants and arrays
What is Native COM

Reference:
Image.Load

This section illustrates programming rules that must be followed when using Native COM; specifically, it illustrates rules in relation to using methods and properties. Note that this section is similar to the one entitled, Convert an existing MFC wrapper project to use Native COM. This is the case since, when converting a project that uses MFC wrapper classes to use Native COM, the same programming rules apply.

Access controls and objects by reference

All controls and objects are accessible by reference; for example:

IImagePtr ChildImage;
ChildImage = m_Application1->CreateObject("MIL.Image", FALSE);
ChildImage->ParentImage = m_Image1;
ChildImage->ChildRegion->SizeX = 100;

ChildImage->Allocate();

Default values for methods

Methods do not have default values for any of their parameters; therefore, all values must be set explicitly. For example, the Load method of the Image control has an optional parameter, AdjustImage, which must be set when using Native COM:

m_Image1->Load("X:\\images\\MyImage.bmp", TRUE);

Get/Set prefix

Properties do not use the Get/Set prefix, for example:

m_Image1->SizeX = 640;
long SizeX = m_Image1->SizeX;

Access a collection item

A collection item must be accessed explicitly (using the Item property). For example:

const FIRST_OCCURRENCE = 1;
double PositionY;
PositionY = m_PatternMatching1->Results->Item(FIRST_OCCURRENCE)->PositionY;

CString variables

Your CString variables either have to be cast to the null-terminated string LPCTSTR or the length-prefixed string _bstr_t, or you can use _bstr_t directly.

  • Example using casting to _bstr_t:
    • CString AnImagePath = "X:\\images\\MyImage.bmp";
int MimImageSizeX =
   m_Image1->FileInquire(static_cast<_bstr_t>(AnImagePath), imSizeX);
  • Example using _bstr_t directly:
    • _bstr_t AnotherImagePath = "X:\\images\\MyImage.bmp";
MimImageSizeX = m_Image1->FileInquire(AnotherImagePath, imSizeX);

Enumerations

As usual, method parameters, properties, and property index parameters that are of type enumeration are bound to these enumerations. That is, a value passed to a method parameter, property, or property index parameter of type enumeration must be of the same type, otherwise the value must be cast. When passing the sum of an enumeration value with a longintegerlong (integer for Visual Basic .Net) or another enumeration value, the expression must be cast; these expressions result in type longintegerlong (integer for Visual Basic .Net), so they must be cast back to the correct enumeration type. For example:

const FOCUS_SENSITIVITY = 1;
Position = m_Digitizer1->Focus(
   static_cast<DigFocusModeConstants>(digSmartScan + FOCUS_SENSITIVITY),
   FALSE, TRUE, TRUE);

Comparison conditions

The Count property must be used with a comparison operator if it is used in a comparison condition, for example:

if (m_PatternMatching1->Results->Count == 1) { /* ... */ }

Properties that take parameters

For properties that take parameters (except for the Item property), use the conventional array access notation (for example, []):

double KernelArray[3][3];
for (int i = 0; i<3; i++)
   for (int j = 0; j<3; j++)
      KernelArray[i][j] = m_ImageProcessing1->Kernels->Item(1)->Data[i][j];


Using variants and arrays

User Guide:
Smart pointers
Using strings

Reference:
AVIFileHandler.Write

ActiveMIL uses the variant data type in two places. First, when a method parameter or property takes more than one data type. Second, when a method parameter takes an array. This section discusses variants and arrays and how they are used with ActiveMIL.

Variants

When using ActiveMIL with Native COM support, you can use the following types to handle method parameters and properties of type variant:

  • The VARIANT data type.
  • The COleVariant class (only available with MFC).
  • The _variant_t class.

VARIANT is a structure containing a C/C++ union and a type tag. The union part can contain data of any data type; the type tag specifies the data type of the data that the union currently contains. When using the VARIANT structure, you must handle the initialization and deallocation of the resources that the structure contains (or references) by using the Windows API VariantInit() and VariantClear() functions, respectively. For more information, see "VariantInit" and "VariantClear" in MSDN.

COleVariant and _variant_t are classes that encapsulate the VARIANT data type. These classes offer you a significant amount of advantages; for example, they automatically handle initialization and resource deallocation. It is recommended that you use _variant_t, as it was specifically designed for Native COM users.

You can pass a COleVariant or _variant_t data type wherever a VARIANT data type is expected. For example, you can pass a reference to a _variant_t to method parameters and properties of type VARIANT *:

_variant_t v;
m_Code1->Results->Get(codeDecodedStringSize, &v);
long DecodedStringSize = v;

Note that, when using variants, VT_ is used to describe the particular type of data (for example, VT_I4 is a longintegerlong (integer for Visual Basic .Net) type), while V_ is used to access that data (for example, V_I4 accesses a longintegerlong (integer for Visual Basic .Net) type). For more information, see "The Ultimate Data Type" and "VARIANT and VARIANTARG" in MSDN.

The following is an example of using ActiveMIL with _variant_t:

// Get the window's HDC
HDC hdc = (HDC) V_I4(&m_Display1->OverlayImage->Inquire(imDCHandle));

// Call the Display->Control() method using _variant_t
_variant_t WindowUpdate(static_cast<long>(dispEnable), VT_I4);

m_Display1->Control(dispWindowUpdateOnPaint, WindowUpdate);

The following is an example of using ActiveMIL with COleVariant:

// Get the window's DC
HDC hdc = (HDC) V_I4(&m_Display1->OverlayImage->Inquire(imDCHandle));

// Set how to update the display using MFC variant
COleVariant WindowUpdate((long)dispEnable, VT_I4);
m_Display1->Control(dispWindowUpdateOnPaint, WindowUpdate);

The following is an example of using ActiveMIL with VARIANT:

// Get the window's HDC
HDC hdc = (HDC) V_I4(&m_Display1->OverlayImage->Inquire(imDCHandle));

// Call the Display->Control() method using VARIANT
VARIANT WindowUpdate;

VariantInit(&WindowUpdate);
V_VT(&WindowUpdate) = VT_I4;
V_I4(&WindowUpdate) = dispEnable;

m_Display1->Control(dispWindowUpdateOnPaint, WindowUpdate);

VariantClear(&WindowUpdate);

Arrays

With ActiveMIL, method parameters that take arrays are always of type variant. When using Native COM support, the following types are typically used to handle these parameters:

  • The COleSafeArray class (only available with MFC).
  • The SAFEARRAY structure.
  • The CActiveMILObjectsArray class (only available with MFC).

COleSafeArray

COleSafeArray is an MFC class, derived from the VARIANT structure, that was built for working with arrays of different types and dimensions. COleSafeArray encapsulates the VARIANT structure and offers you a significant amount of advantages; for example, it automatically handles initialization and resource deallocation. When using MFC, it is recommended that you use COleSafeArray.

ActiveMIL's Get methods take a VARIANT * as one of their parameters. However, COleSafeArray can be used directly in the parameter, since COleSafeArray objects are derived from the VARIANT structure and have an LPVARIANT() operator.

The following is an example of using the COleSafeArray class:

// This example gets a 3 by 3 array of pixels from an image using
// the MFC COleSafeArray class.

// Initialize the size and offset of data to get.
int SizeX = 3;
int SizeY = 3;
int OffsetX = 2;
int OffsetY = 2;

// Initialize the array information.
DWORD numElements[] = {SizeX, SizeY};
COleSafeArray SafeArray;

// Our image has a data depth of 8 bits, and an unsigned data type,
// therefore create a two-dimensional array of type VT_UI1.
SafeArray.Create(VT_UI1, 2, numElements);

// Get the data of the desired image region into our array.
//
// ActiveMIL's Get methods accept a 'VARIANT *' as their first parameter,
// therefore we can pass COleSafeArray objects directly,
// as they derive from VARIANT and have an LPVARIANT() operator.
m_LoadedImage->Get(SafeArray, imPlanar, imAllBands,
                   OffsetX, OffsetY, SizeX, SizeY);

// Print the data that was just received using the Get method.
for(long  i = 0; i < SizeX; i++)
   {
   for(long j = 0; j < SizeY; j++)
      {
      long rgIndices[2];
      unsigned char PixelValue;
      rgIndices[0] = i;
      rgIndices[1] = j;
      SafeArray.GetElement(rgIndices, &PixelValue);
      printf("Pixel at (%d, %d) : %d\n", i, j, PixelValue);
      }
   }

// COleSafeArray will release the array resources at its destruction.

SAFEARRAY

SAFEARRAY is a structure that holds the array data and the bounds for each dimension of the array. Upon defining the SAFEARRAY dimensions and data type with SafeArrayCreate(), you must also pass a vector of SAFEARRAYBOUND, which contains the bounds of each dimension to allocate for the array. Note that the SAFEARRAY structure does not offer you any advantages; for example, unlike COleSafeArray, you must handle initialization and resource deallocation. For more information, see "SAFEARRAY Structure" and "SAFEARRAYBOUND Structure" in MSDN.

Note that before passing a SAFEARRAY to an ActiveMIL method parameter, you must wrap it into a variant.

The following is an example of using the SAFEARRAY structure:

// This example gets a 3 by 3 array of pixels from an image using
// the SAFEARRAY structure.

// Initialize the size and offset of data to get.
int SizeX = 3;
int SizeY = 3;
int OffsetX = 2;
int OffsetY = 2;

// Initialize the array informations.
SAFEARRAY FAR* psa = NULL;
SAFEARRAYBOUND rgsabound[2];
rgsabound[0].lLbound = 0;
rgsabound[0].cElements = SizeX;
rgsabound[1].lLbound = 0;
rgsabound[1].cElements = SizeY;

// Our image has a data depth of 8 bits, and an unsigned data type,
// therefore create a two-dimensional array of type VT_UI1.
psa = SafeArrayCreate(VT_UI1, 2, rgsabound);

if (psa != NULL)
   {
   // The SAFEARRAY object will be wrapped into a VARIANT because the Get method
   // needs the array to be passed as a 'VARIANT *'.

   // Create the VARIANT containing the array.
   VARIANT ArrayVariant;
   VariantInit(&ArrayVariant);
   V_VT(&ArrayVariant) = VT_ARRAY|VT_UI1;
   V_ARRAY(&ArrayVariant) = psa;

   // From now on the SAFEARRAY will be managed by the VARIANT
   psa = NULL;

   // Get the data of the specified image region into our array.
   //
   m_LoadedImage->Get(&ArrayVariant, imPlanar, imAllBands,
                      OffsetX, OffsetY, SizeX, SizeY);

   // Print the data that was just received.
   for(long  i = 0; i < SizeX; i++)
      {
      for(long j = 0; j < SizeY; j++)
         {
         long rgIndices[2];
         unsigned char PixelValue;
         rgIndices[0] = i;
         rgIndices[1] = j;
         SafeArrayGetElement(V_ARRAY(&ArrayVariant), rgIndices, &PixelValue);
         printf("Pixel at (%d, %d) : %d\n", i, j, PixelValue);
         }
      }

   // Clear the VARIANT.
   // This will destroy all the data in the array.
   // Note that if we did not wrap the SAFEARRAY into a VARIANT, we should
   // have called 'SafeArrayDestroy(psa);'
   VariantClear(&ArrayVariant);
   }

CActiveMILObjectsArray

The Write and Read methods for the ActiveMIL object AVIFileHandler, and the Process method for the Digitizer.MultipleBuffering object, are the only methods that take an array of ActiveMIL objects as one of their parameters. Specifically, the array that their Images parameter takes should be passed as a CActiveMILObjectsArray object.

To create a CActiveMILObjectsArray object, you must first define an array using Microsoft's CTypedPtrArray class template. CTypedPtrArray is a class template that provides a type-safe wrapper for object references in an array. When using CTypedPtrArray, C++ type-checks the array, helping to eliminate errors caused by mismatched pointer types. CTypedPtrArray takes two template parameters, BASE_CLASS and TYPE. The BASE_CLASS parameter specifies the base class of the typed pointer array class; in this case, use the Microsoft class, CPtrArray. The TYPE parameter specifies the type of the elements stored in the base class array; in this case, use an ActiveMIL control smart pointer type (for example, IImagePtr).

Once you have defined a CTypedPtrArray class, use it to create a CActiveMILObjectsArray object by passing the following two parameters to the CActiveMILObjectsArray constructor: the data of CTypedPtrArray and the number of elements in CTypedPtrArray. Then, as mentioned, pass the CActiveMILObjectsArray object to the Write method parameter. For more information, see "CTypedPtrArray" and "CPtrArray" in MSDN.

The following is an example of using the CActiveMILObjectsArray class:

 CTypedPtrArray<CPtrArray, IImagePtr> SequenceImagesArray;

 // The size of the array can be predefined.
 const long MAXIMUM_NUMBER_OF_OBJECTS = 5;
 SequenceImagesArray.SetSize(MAXIMUM_NUMBER_OF_OBJECTS);

 // Fill the SequenceImagesArray with ActiveMIL Image controls.
 // See the Application.CreateObject() method
 for (long i=0; i<MAXIMUM_NUMBER_OF_OBJECTS; i++)
    {
    IImagePtr NewImage = m_Application1->CreateObject("MIL.Image", FALSE);
    NewImage->SizeX = 100;
    NewImage->SizeY = 100;
    NewImage->Allocate();
    SequenceImagesArray[i] = NewImage;
    }

 IAVIFileHandlerPtr AVIHandler;
 AVIHandler.CreateInstance("MIL.AVIFileHandler");
 AVIHandler->Open("MySequence.avi", aviFileFormatDefault, aviCreate);

 // Build a CActiveMILObjectsArray.
 CActiveMILObjectsArray<IImagePtr> ActiveMILObjectsArray(
    SequenceImagesArray.GetData(), SequenceImagesArray.GetSize());

 // Use it with the AVIFileHandler.Write method.
 AVIHandler->Write(ActiveMILObjectsArray, m_NumberOfFrames / m_Time);
 AVIHandler->Close();

 // Destroy the ActiveMIL Image controls by setting their pointers to NULL.
 // CPtrArray will only get rid of the pointers without destroying their
 // contents.
 for (i=0; i<MAXIMUM_NUMBER_OF_OBJECTS; i++)
    {
    SequenceImagesArray[i] = NULL;
    }


Using strings

User Guide:
Using variants and arrays

In ActiveMIL, there are properties and methods that take values of type string. When using Native COM support, the following string types are typically used with ActiveMIL:

  • LPTSTR and LPCTSTR. LPTSTR is a long pointer to a string, while LPCTSTR is a long pointer to a constant string. Note that these are both C/C++ null-terminated strings.
  • CString. CString is a class that encapsulates a variable-length sequence of characters. The CString class provides methods for manipulating strings. In addition, the CString class has its own LPCTSTR operator; this operator operates on the null-terminated string contained in CString. CString is only available with MFC.
  • BSTR. BSTR (basic string) is a structured data type that contains a character string and the string's length. This type does not offer any advantages; for example, resource allocation and deallocation are not handled automatically.
  • _bstr_t. _bstr_t is a class that encapsulates and manages the BSTR data type. The _bstr_t class offers a significant amount of advantages; for example, _bstr_t handles resource allocation and deallocation automatically.

It is recommended that you use _bstr_t, as it was specifically designed for Native COM support. For more information on these string types, see MSDN.

To use strings of type _bstr_t with Native COM, everything is direct and no casting is needed. For example:

// Set the SystemDescriptor using _bstr_t
_bstr_t SystemDescriptor = "VGA";
m_System1->SystemDescriptor = SystemDescriptor;

// Get the SystemDescriptor using _bstr_t
SystemDescriptor = m_System1->SystemDescriptor;

To use strings of type BSTR, everything is direct and no casting is needed; however, as mentioned, resource allocation and deallocation are not handled automatically. You must do so using the Windows API functions, SysAllocString() and SysFreeString(). For example:

// Set the SystemDescriptor using BSTR
BSTR SystemDescriptor = SysAllocString(L"VGA");
m_System1->SystemDescriptor = SystemDescriptor;
SysFreeString(SystemDescriptor);

// Get the SystemDescriptor using BSTR
SystemDescriptor = SysAllocString(m_System1->SystemDescriptor);
SysFreeString(SystemDescriptor);

To use strings of type LPTSTR or LPCTSTR, no casting is needed. However, if you want to get a local copy of the string, you must allocate memory for it, copy the string, and then delete the memory that was allocated for the string once you have finished using it. Note that if you want to call a function that has a prototype which accepts an LPTSTR or LPCTSTR, a _bstr_t can be given to that function directly. For example:

// Set the SystemDescriptor using an LPCTSTR
LPCTSTR lpcszSystemDescriptor = "VGA";
m_System1->SystemDescriptor = lpcszSystemDescriptor;

// or, with a string literal
m_System1->SystemDescriptor = "VGA";

// Keep the SystemDescriptor into an LPTSTR
LPTSTR lpszSystemDescriptor = new TCHAR[m_System1->SystemDescriptor.length()+1];
lstrcpy(lpszSystemDescriptor, m_System1->SystemDescriptor);
// ...
delete [] lpszSystemDescriptor;

// Call a function that takes an LPCTSTR parameter
MyFunction(m_System1->SystemDescriptor);

void MyFunction(LPCTSTR SystemDescriptor)
{
   // ...
}

To use strings of type CString, the string must either be cast to LPCTSTR, or cast to _bstr_t directly. For example:

// Set the SystemDescriptor using MFC CString
CString SystemDescriptor = "VGA";

// These two lines are equivalent
m_System1->SystemDescriptor = static_cast<LPCTSTR>(SystemDescriptor);
m_System1->SystemDescriptor = static_cast<_bstr_t>(SystemDescriptor);

// Get the SystemDescriptor using MFC CString
// A cast to LPCTSTR is necessary when assigning a CString with a _bstr_t
SystemDescriptor = static_cast<LPCTSTR>(m_System1->SystemDescriptor);


Run-time object declaration

User Guide:
Creating an application and adding ActiveMIL to it (Visual C++)

Reference:
Application.CreateObject

ActiveMIL controls can be created at run-time rather than dropping the controls onto your dialog at design-time. Creating ActiveMIL controls at run-time has the following advantages:

  • Saves memory because the controls are created as needed.
  • Controls can be created on demand.

Creating ActiveMIL controls at run-time has the following disadvantages:

  • Events are not available.
  • Properties cannot be initialized at design-time; you must set them explicitly at run-time.

To create ActiveMIL controls at run-time:

  1. Create ActiveMIL's Application control first, using the Windows API CoCreateInstance function, or the _com_ptr_t::CreateInstance function.
  2. Create all other ActiveMIL controls, using the CreateObject method of the Application control.

The following example illustrates how to create ActiveMIL controls at run-time:

#include "Application.h"
#include "System.h"
#include "Digitizer.h"

// Initialize the COM library
CoInitialize(NULL);

// Create the Application control using the CreateInstance method of
// the _com_ptr_t smart pointer.
//
// NOTE: Only the ActiveMIL Application control should be created this way;
// all other ActiveMIL controls should be created using the CreateObject
// method of the Application control.
//
IApplicationPtr Application;
Application.CreateInstance("MIL.Application");

// Create the ActiveMIL System control and the ActiveMIL Digitizer
// control using the CreateObject method of the ActiveMIL Application control.
ISystemPtr System;
IDigitizerPtr Digitizer;
System = Application->CreateObject("MIL.System", TRUE);
Digitizer = Application->CreateObject("MIL.Digitizer", TRUE);

Note that you can create some ActiveMIL controls at design-time, while creating others at run-time. To do so, drop the Application control onto the dialog, which creates an instance of this control in the application, then use the CreateObject method of this object to create subsequent ActiveMIL controls.


Error management (Visual C++)

User Guide:
Creating an application and adding ActiveMIL to it (Visual C++)
Error reporting (Visual Basic and Visual C++)

Error management is the proper handling of various predictable irregular situations (also known as exceptions) that can occur during the program's execution. The program should anticipate possible errors and properly communicate them to code that is outside the normal flow of the program's execution. Error management equips the program to recover from such errors, if and when they occur.

By default, Native COM objects generate native _com_error objects on error. However, MFC does not catch native _com_error objects when Native COM is used in conjunction with MFC; as a result, the errors will propagate through MFC's automatic exception mechanisms and simply appear as "uncaught" exceptions. In this case, special precautions must be taken, such as setting traps for errors on every call to your COM objects with either MFC's TRY/CATCH macros, or with the function, _com_raise_error().

The _com_raise_error() function is an error handling mechanism offered by Native COM. Whenever a COM error occurs, a call to _com_raise_error() is made if this function is included in your header files. If not, a _com_error object is thrown.

You can implement a _com_raise_error() function when creating a new project to which ActiveMIL was added, as explained in the section, Creating an application and adding ActiveMIL to it (Visual C++), in the Introduction chapter. This _com_raise_error() function should throw MFC's COleDispatchException objects instead of the _com_error objects; in this case, MFC's exception handling mechanism will recognize and manage the exceptions correctly.

The following example implements a _com_raise_error() function. This code is found in all ActiveMIL examples. You must include this code in your application's main file.

/////////////////////////////////////////////////////////////////////////////
//  ActiveMIL procedure :   This function causes the compiler to throw
//                          COleDispatchExceptions instead of _com_error.
//                          This means that MFC catch statements will work.
//

void __stdcall _com_raise_error(HRESULT hr, IErrorInfo* perrinfo)
{
   if (perrinfo != NULL)
      {
      COleDispatchException* pException =
         new COleDispatchException(NULL, 0, (WORD)HRESULT_CODE(hr));

      BSTR bstrSource;
      perrinfo->GetSource(&bstrSource);
      if (bstrSource != NULL)
         {
         pException->m_strSource = bstrSource;
         SysFreeString(bstrSource);
         }

      BSTR bstrDescription;
      perrinfo->GetDescription(&bstrDescription);
      if (bstrDescription != NULL)
         {
         pException->m_strDescription = bstrDescription;
         SysFreeString(bstrDescription);
         }

      BSTR bstrHelpFile;
      perrinfo->GetHelpFile(&bstrHelpFile);
      if (bstrHelpFile != NULL)
         {
         pException->m_strHelpFile = bstrHelpFile;
         SysFreeString(bstrHelpFile);
         }

      perrinfo->GetHelpContext(&pException->m_dwHelpContext);
      pException->m_scError = hr;

      // then throw the exception
      throw pException;
      }
   else
      {
      // non-exception error code
      AfxThrowOleException(hr);
      }
}


Convert an existing project using wrapper classes to use Native COM

See Also | Previous
User Guide:
Methods and properties
Smart pointers
Using strings
Using variants and arrays
What is Native COM

Reference:
Image.Load

To use Native COM support with an existing project that was coded with wrapper classes, you must incorporate into your code all the changes listed below. If, however, you are starting a new project, and you do not want a discussion of wrapper classes, you must still adhere to a similar set of standards, as explained in the section, Methods and properties.

Remember that, when using Native COM support, the use of smart pointers is strongly recommended.

Access controls and objects by reference

When using Native COM, all controls and objects are accessible by reference.

  • Wrapper classes example:
    • IImage ChildImage;
ChildImage = m_Application1.CreateObject("MIL.Image", FALSE);
ChildImage.SetParentImage(m_Image1);
ChildImage.GetChildRegion().SetSizeX(100);

ChildImage.Allocate();
  • Native COM example:
    • IImagePtr ChildImage;
ChildImage = m_Application1->CreateObject("MIL.Image", FALSE);
ChildImage->ParentImage = m_Image1;
ChildImage->ChildRegion->SizeX = 100;

ChildImage->Allocate();

Default values for methods

Methods do not have default values for any of their parameters; therefore, all values must be set explicitly. For example, the Load method of the Image control takes the following two parameters, FileName and AdjustImage. If you were using wrapper classes, the AdjustImage parameter was optional since its default value is TRUE. However, when using Native COM, this parameter must be set explicitly.

  • Wrapper classes example:
    • Image.Load("X:\\Images\\MyImage.bmp");
  • Native COM example:
    • m_Image1->Load("X:\\images\\MyImage.bmp", TRUE);

Get/Set prefix

When using Native COM, properties do not use the Get/Set prefix.

  • Wrapper classes example:
    • Image.SetSizeX(640);
long SizeX = Image.GetSizeX();
  • Native COM example:
    • m_Image1->SizeX = 640;
long SizeX = m_Image1->SizeX;

Access a collection item

A collection item must be accessed explicitly (using the Item property). For example:

  • Wrapper classes example:
    • const FIRST_OCCURRENCE = 1;
double PositionY;
PositionY = m_PatternMatching1.GetResults(FIRST_OCCURRENCE).GetPositionY();
  • Native COM example
    • const FIRST_OCCURRENCE = 1;
double PositionY;
PositionY = m_PatternMatching1->Results->Item(FIRST_OCCURRENCE)->PositionY;

Inquire methods

When using Native COM, Inquire methods return a value of type _variant_t; when using ActiveMIL wrapper classes, Inquire methods return a value of type VARIANT.

  • Wrapper classes example:
    • HDC hdc = (HDC) m_Display1.GetOverlayImage().Inquire(imWindowDC);
  • Native COM example:
    • _variant_t v = m_Display1->OverlayImage->Inquire(imDCHandle);
HDC hdc = reinterpret_cast<HDC>(V_I4(&v));

CString variables

When using Native COM, your CString variables either have to be cast to the null-terminated string LPCTSTR or the length-prefixed string _bstr_t, or you can use _bstr_t directly.

  • Wrapper classes example:
    • CString ImagePath = "X:\\images\\MyImage.bmp";
int MimImageSizeX = m_Image1.FileInquire(ImagePath, imSizeX);
  • Native COM example using casting to _bstr_t:
    • CString AnImagePath = "X:\\images\\MyImage.bmp";
int MimImageSizeX =
   m_Image1->FileInquire(static_cast<_bstr_t>(AnImagePath), imSizeX);
  • Native COM example using _bstr_t directly:
    • _bstr_t AnotherImagePath = "X:\\images\\MyImage.bmp";
MimImageSizeX = m_Image1->FileInquire(AnotherImagePath, imSizeX);

Enumerations

With ActiveMIL wrapper classes, method parameters, properties, and property index parameters that take enumerations are actually of type long and therefore casting is unnecessary. However, when using ActiveMIL with Native COM support, method parameters, properties, and property index parameters that take enumerations are of type enumeration and are bound to these enumerations. That is, a value passed to a method parameter, property, or property index parameter of type enumeration must be of the same type, otherwise the value must be cast. When passing the sum of an enumeration value with a long or another enumeration value, the expression must be cast; these expressions result in type long, so they must be cast back to the correct enumeration type. For example:

  • Wrapper classes example:
    • const FOCUS_SENSITIVITY = 1;
double Position = m_Digitizer1.Focus(digSmartScan + FOCUS_SENSITIVITY,
                                     FALSE, TRUE, TRUE);
  • Native COM example:
    • const FOCUS_SENSITIVITY = 1;
Position = m_Digitizer1->Focus(
   static_cast<DigFocusModeConstants>(digSmartScan + FOCUS_SENSITIVITY),
   FALSE, TRUE, TRUE);

Comparison conditions

When using Native COM, the Count property must be used with a comparison operator if it is used in a comparison condition.

  • Wrapper classes example:
    • if (m_PatternMatching1.GetResults().GetCount()) {}
  • Native COM example:
    • if (m_PatternMatching1->Results->Count == 1) { /* ... */ }

Properties that take parameters

When using Native COM, properties that take parameters (except for the Item property), use the conventional array access notation (for example, []):

  • Wrapper classes example:
    • double KernelArray[3][3];
for (int i = 0; i<3; i++)
   for (int j = 0; j<3; j++)
      KernelArray[i][j] = m_ImageProcessing1.GetKernels(1).GetData(i,j);
  • Native COM example:
    • double KernelArray[3][3];
for (int i = 0; i<3; i++)
   for (int j = 0; j<3; j++)
      KernelArray[i][j] = m_ImageProcessing1->Kernels->Item(1)->Data[i][j];

Preprocessor definitions

You must ensure that the following predefined constants are not present in the preprocessor definitions:

  • ACTIVEMIL_IN_PROJECT
  • ACTIVEMIL_USE_COM_CLASS
  • ACTIVEMIL_WIN32_ONLY

To remove the predefined constants:

  1. Select Properties from the Project menu command. The Property Pages dialog box is presented.
  2. Select the C/C++ folder.
  3. Select the Preprocessor page.
  4. In the Preprocessor definitions text box, remove, if present, the predefined constants stated above.


Using ActiveMIL with .NET

Chapter 21: Using ActiveMIL with .NET

This chapter explains how to use ActiveMIL in .NET applications, as well as some .NET features to keep in mind while using ActiveMIL.

  • Overview of using ActiveMIL with .NET
  • Namespace
    • Matrox.* and AxMatrox.* namespaces
    • Namespace for ActiveMIL-Lite controls
    • Namespaces for other ActiveMIL controls
    • Accessing the Windows Forms properties of ActiveMIL controls
    • How to use the Imports statement (VB.NET)
      • The Imports statement (VB.NET)
    • Generated DLL files (advanced)
      • Ax<name of ocx>.dll (Windows Form)
      • <name of ocx>.dll (Runtime)
  • Adding ActiveMIL controls to a .NET application
    • Adding project references
    • Adding controls at design-time
      • Using the ActiveMIL Toolbox tab
      • Hiding controls
    • Adding controls at run-time
  • .NET ActiveMIL data types
  • Peculiarities of running ActiveMIL in managed code
    • Exceptions in event handler
    • Using arrays more efficiently
  • Garbage collector
    • Free versus destructors
      • Out of scope ActiveMIL controls
      • Free method is skipped due to a thrown error
    • Invoking the garbage collector
      • Freeing an ActiveMIL control created under the AxMatrox.ActiveMIL.* namespace
      • Freeing an ActiveMIL control declared in a Try block


Overview of using ActiveMIL with .NET

See Also | Next
User Guide:
Creating an application and adding ActiveMIL to it (.NET)

As of Version 8.0, ActiveMIL supports the Visual Studio .NET development environment. This includes support for Visual C++ .NET (unmanaged) and Visual Basic .NET.

Applications written in Visual Basic .NET require the .NET framework and run within a managed environment. ActiveMIL, however, is a set of ActiveX controls created using unmanaged C++. To handle calls between managed and unmanaged code, ActiveMIL uses automatically generated wrappers (Runtime Callable Wrapper).

This chapter is designed to help you add and use ActiveMIL in a managed .NET application. Since this chapter focuses on creating managed ActiveMIL applications, you do not need to read this chapter if you are using Visual C++ .NET.


Namespace

Previous | Next

In the .NET Framework, objects are included in assemblies. Each of these assemblies represents a namespace. Namespaces are a hierarchical system of organizing objects.

Matrox.* and AxMatrox.* namespaces

The ActiveMIL controls and objects are organized under the Matrox.ActiveMIL.* and the AxMatrox.ActiveMIL.* namespaces.

The ActiveMIL-Lite controls are available directly through the Matrox.ActiveMIL namespace. Each ActiveMIL processing or analysis control has its own namespace within the Matrox.ActiveMIL.* namespace. See the Namespace for ActiveMIL-Lite controls and the Namespaces for other ActiveMIL controls subsections later in this section.

The Matrox.ActiveMIL.* namespace is added to your project once you reference an ActiveMIL control within your project. Adding an ActiveMIL control at design-time through the Toolbox automatically references that control's OCX within your project.

The AxMatrox.* namespace is automatically created by Visual Studio .NET when you add an ActiveMIL control on a Windows Form at design-time. This is done through the ActiveX Control Importer, which converts an ActiveX control into a Windows Forms control. ActiveMIL controls under the AxMatrox.* namespace are derived from the AxHost class, and are compiled into a .NET Dynamic Link Library file (Primary Interop Assembly). The AxHost class exposes the ActiveMIL controls as Windows Form controls, adding to them all the properties and methods expected from any .NET Windows Form control.

All ActiveMIL controls are available under the AxMatrox.ActiveMIL namespace. At design-time, you are forced to use the AxMatrox.ActiveMIL namespace, and at run-time, you have a choice to use either the Matrox.ActiveMIL or the AxMatrox.ActiveMIL namespace to create ActiveMIL controls. To expose all properties and methods inherited from the Windows Forms control in ActiveMIL controls, use the AxMatrox.ActiveMIL namespace; see Accessing the Windows Forms properties of ActiveMIL controls subsection later in this section.

All ActiveMIL controls within the Matrox.ActiveMIL.* namespace are prefixed with the letter M (such as MDisplay), while all ActiveMIL controls within the AxMatrox.ActiveMIL.* namespace are prefixed with the letters AxM (such as AxMDisplay).

If you want to capture the MouseMove event of an ActiveMIL Display control created at run-time, use the ActiveMIL Display control available under the AxMatrox.* namespace. This also applies to all .NET events used with ActiveMIL controls.

Namespace for ActiveMIL-Lite controls

All the ActiveMIL-Lite controls are available in the ActiveMIL Base Control (MIL.ocx) and belong to the Matrox.ActiveMIL namespace. The ActiveMIL Base Control must be referenced in the project before any of the ActiveMIL-Lite controls can be used, see the Adding project references subsection in the Adding ActiveMIL controls to a .NET application section. These controls are:

  • Application control
  • System control
  • Image control
  • Digitizer control
  • Display control
  • GraphicContext control
  • Threading control

For more information about all basic controls, see the ActiveMIL and ActiveMIL-Lite controls section in the Introduction chapter.

Note that all ActiveMIL-Lite controls are also available under the AxMatrox.ActiveMIL namespace.

Namespaces for other ActiveMIL controls

Each ActiveMIL processing or analysis control has its own namespace and must be referenced in the project before your application can have access to it. These controls and their namespaces are:

Control | OCX File Name | Namespace
BlobAnalysis control | MILBlobAnalysis.ocx | Matrox.ActiveMIL.MBlobAnalysis
Calibration control | MILCalibration.ocx | Matrox.ActiveMIL.MCalibration
CharacterRecognition control | MILCharacterRecognition.ocx | Matrox.ActiveMIL.MCharacterRecognition
Code control | MILCode.ocx | Matrox.ActiveMIL.MCode
EdgeFinder | MILEdgeFinder.ocx | Matrox.ActiveMIL.MEdgeFinder
ImageProcessing control | MILImageProcessing.ocx | Matrox.ActiveMIL.MImageProcessing
Measurement control | MILMeasurement.ocx | Matrox.ActiveMIL.MMeasurement
ModelFinder control | MILModelFinder.ocx | Matrox.ActiveMIL.MModelFinder
PatternMatching control | MILPatternMatching.ocx | Matrox.ActiveMIL.MPatternMatching

For more information about these controls, see the ActiveMIL and ActiveMIL-Lite controls section in the Introduction chapter.

Note that all ActiveMIL processing or analysis controls are also available under the AxMatrox.ActiveMIL.* namespace.

Accessing the Windows Forms properties of ActiveMIL controls

To access the Windows Forms Control properties of an ActiveMIL control, you must add the ActiveMIL control found in the AxMatrox namespace to your project. For example, if you create a Display control at run-time using the Matrox namespace, you will not be able to access its location and size properties. Windows Forms Control properties will only be exposed through ActiveMIL controls residing within the AxMatrox namespace.

How to use the Imports statement (VB.NET)

The Imports statement (VB.NET)

It is recommended that you use the Visual Basic .NET Imports statement to shorten the names of the ActiveMIL controls. Add:

Imports Matrox.ActiveMIL
Dim OwnerSystem As MSystem

The Imports statement must be placed after any option statement but before any type declaration.

You can also do the same with the AxMatrox.ActiveMIL namespace.

You should not confuse the Imports statement with adding a reference to your controls. To add references to your controls, refer to the Adding ActiveMIL controls to a .NET application section later in this chapter.

Generated DLL files (advanced)

For every ActiveMIL OCX imported to the .NET Windows Forms Control format, two DLL files are created: Ax<name of ocx>.dll and <name of ocx>.dll.

Ax<name of ocx>.dll (Windows Form)

The Ax<name of ocx>.dll is a wrapper DLL generated by the ActiveX Control Importer (Aximp.exe). ActiveX controls must be wrapped in a class that derives from the System.Windows.Forms.AxHost class before they can be used on a Windows Form. The Ax<name of ocx>.dll:

  • Wraps ActiveX controls and exposes them as fully featured Windows Form controls.
  • Derives from the System.Windows.Forms.AxHost class.
  • Contains the definitions for all properties, methods, and events of the control's root.
  • Does not contain the definition for sub-objects.
  • Delegates all calls to the Primary Interop Assembly (PIA), even those defined in the control's root. The Ax<name of ocx>.dll assembly is not needed if all objects are created at run-time.

The Ax<name of ocx>.dll is automatically generated by Visual Studio.

For more information about the AxHost and the ActiveX Control Importer, refer to the .NET documentation available through the MSDN library.

<name of ocx>.dll (Runtime)

COM type definitions usually reside in a type library. In contrast, Common Language Specification (CLS) compliant compilers produce type metadata in an assembly. The PIA, a wrapper DLL that allows the Common Language Runtime (CLR) to communicate with legacy COM components, is generated by the Type Library Importer (tlbimp.exe) when you add a reference to a COM object to you project. The <name of ocx>.dll:

  • Implements all interfaces of the ActiveX controls.
  • Delegates calls to the ActiveX controls.

The <name of ocx>.dll is automatically generated by Visual Studio.

For more information about the Primary Interop Assembly and Type Library Importer, refer to the .NET documentation available through the MSDN library.


Adding ActiveMIL controls to a .NET application

Previous | Next

Adding project references

Before using the ActiveMIL controls in your .NET application, you must add references to them in your project. If you insert the ActiveMIL controls using the Visual Studio .NET Toolbox, control referencing is automatic. If you are adding your ActiveMIL controls only at run-time, you will have to manually reference them. To manually reference the ActiveMIL controls, follow these steps:

  1. Select Add Reference... from the Project menu command or from the Solution Explorer context menu. The Add Reference dialog box is presented.
  2. Select the COM tab.
  3. Select all the ActiveMIL controls available or only those required for your application and click on Select.

    4. All the selected controls are added to the Selected Components list.

  4. Verify that these are the required controls and then click on OK.

You will now see all the controls that you selected for referencing in the Solution Explorer's References list.

Adding controls at design-time

Generally, you would add your ActiveMIL controls at design-time. To do so, you can use the Visual Studio control Toolbox.

Using the ActiveMIL Toolbox tab

To insert a control in your application using the ActiveMIL Toolbox tab:

  1. Open your Windows Form in design mode.
  2. Select the ActiveMIL Toolbox tab.

    3. The ActiveMIL Toolbox tab is only available when you are working in a Windows Form designer.

  3. Select the control to add to your form. The cursor changes to represent the selected control. Click on your form to create the control.

For more information on using controls once they are inserted on the form, see ActiveMIL and ActiveMIL-Lite controls, in the Introduction chapter.

Hiding controls

To hide the icon placed on a Windows Form by an ActiveMIL control, change the control's Visible property to false. The icon will be hidden at run-time.

Adding controls at run-time

To create an initial object instance of the Application control at run-time, use the New keyword. To create all other controls at run-time, use the CreateObject method of the Application control because it creates a valid (ready to use) ActiveMIL control.

' The Application control is the only ActiveMIL control
' you can declare with the New keyword
' Ensure to use Application.CreateObject for other ActiveMIL controls
Dim Application1 As New Matrox.ActiveMIL.MApplication

Dim System1 As Matrox.ActiveMIL.MSystem
Dim Image1 As Matrox.ActiveMIL.MImage
Dim Display1 As Matrox.ActiveMIL.MDisplay

' Dim your object WithEvents if you want to handle events
Dim WithEvents Digitizer1 As Matrox.ActiveMIL.MDigitizer

System1 = Application1.CreateObject("MIL.System", True)

Image1 = Application1.CreateObject("MIL.Image", False)
Image1.CanDisplay = True
Image1.CanGrab = True
Image1.Allocate()

Display1 = Application1.CreateObject("MIL.Display", True)
Digitizer1 = Application1.CreateObject("MIL.Digitizer", True)
Digitizer1.Image = Image1

If applicable, you should free a control's resources before it is out of scope or once it is no longer needed. Refer to the MSDN Library for information on the life cycle of .NET objects. For more information on releasing resources and garbage collector peculiarities, see Garbage collector.


.NET ActiveMIL data types

Previous | Next

The names of some Visual Basic 6.0 data types have changed in Visual Basic .NET. These data types are:

Data type | Integer Size | Visual Basic 6.0 | Visual Basic .NET | Common language run-time type
Integer | 8 bits, signed | (none) | (none) | System.SByte
 | 16 bits, signed | Integer | Short | System.Int16
 | 32 bits, signed | Long | Integer | System.Int32
 | 64 bits, signed | (none) | Long | System.Int64
Object |  | Variant | Object | System.Object

Some other data types have changed in the .NET Framework:

  • The OLE_COLOR data type is equivalent to a System.Drawing.Color. To convert a System.Drawing.Color to an integer value, use the System.Drawing.ColorTranslator 's ToOle method.


Peculiarities of running ActiveMIL in managed code

Previous | Next

Managed code is any code compiled for the .NET run-time environment. Such code runs in the Common Language Runtime (CLR) and is contained within the boundaries of the .NET Framework, providing a great deal of optimization and system safety.

ActiveMIL controls are not managed by the CLR. To run within CLR applications, ActiveMIL controls use the Runtime Callable Wrapper (RCW) to marshal calls between managed and unmanaged code. For more information on managed code, refer to the MSDN library.

Exceptions in event handler

In Visual Studio .NET, a new event handling scheme has been implemented where a delegate, a type-safe and object-oriented function pointer, is used to call event handlers when an event is raised. Even if you do not explicitly use delegates, .NET will analyze the context of event-handling keywords like Handles, AddHandler, WithEvents and internally convert the code to use delegates.

Since ActiveMIL is created using unmanaged C++, ActiveMIL events must use the RCW to interact with the .NET delegates. However, when handling events that come from ActiveMIL controls, exceptions from the event procedures (handlers) are swallowed by the RCW without your knowledge. To be notified of these exceptions, you can use a Try Catch block and, in the Catch block, report the exception using a message box. For more information on event handling and the RCW, see the MSDN library.

Using arrays more efficiently

The marshaling of arrays between the Microsoft .NET managed world and the ActiveMIL controls can be time consuming. This is because arrays are not passed directly to the ActiveMIL controls. Instead, arrays are copied to unmanaged memory, the call to ActiveMIL is made, and then the arrays are copied back to managed memory. To avoid all this copying, ActiveMIL provides the class MArrayRef. Arrays wrapped in an object of this type are passed directly. The following table shows approximate timings:

 |  | Normal | MArrayRef | Ratio
One band | Image.Get | 18 ms | 1 ms | 18 times better
 | Image.Put | 13 ms | 1 ms | 13 times better
All bands | Image.Get | 168 ms | 2 ms | 84 times better
 | Image.Put | 94 ms | 2 ms | 47 times better

Note that timings were obtained on a Pentium IV 3 GHz with 1 GB of RAM.

The MArrayRef class is included in the Matrox.ActiveMIL.Utility.dll assembly, which can be found in the "Matrox Imaging\ActiveMIL\Containers\VStudio.NET" folder on the Matrox Imaging installation CD.

This example shows how to get the data of a 3-band image into 3 arrays using the MArrayRef class:

Dim RedArray(200, 100) As Byte
Dim GreenArray(200, 100) As Byte
Dim BlueArray(200, 100) As Byte

Dim RedArrayRef As Utility.MArrayRef = New Utility.MArrayRef(RedArray)
Dim GreenArrayRef As Utility.MArrayRef = New Utility.MArrayRef(GreenArray)
Dim BlueArrayRef As Utility.MArrayRef = New Utility.MArrayRef(BlueArray)

' Get the data of each band of the loaded image into our arrays.
LoadedImage.Get(RedArrayRef.GetSafeArray(), imPlanar, imRed, 0, 0, 200, 100)
LoadedImage.Get(GreenArrayRef.GetSafeArray(), imPlanar, imGreen, 0, 0, 200, 100)
LoadedImage.Get(BlueArrayRef.GetSafeArray(), imPlanar, imBlue, 0, 0, 200, 100)


Garbage collector

Previous

The .NET garbage collector is responsible for the allocation and release of objects in managed memory. Whenever an object is initialized using the New operator, the garbage collector allocates memory to that object and tracks its usage within your application. To free memory, the garbage collector sweeps through managed memory and frees the memory allocated to all objects that are no longer in use by your application. The timing of a garbage collection is unpredictable; it is possible, however, to force a garbage collection using the .NET System.GC.Collect method.

Free versus destructors

Objects that rely on a garbage collection scheme in the .NET environment have nondeterministic destructors. These destructors are called when the garbage collector decides to delete the object and not when the object is out of scope. Hence you cannot rely on having your resources freed in the object's destructor since you do not know when the destructor will be called. Note that the Free method does not delete a control from memory; it merely frees the physical resources reserved by the control.

Out of scope ActiveMIL controls

If you omit a call to the Free method, you will have to manually invoke the garbage collector or wait for a garbage collection to dispose of your out of scope ActiveMIL control and free your physical system resources. Only then will you be able to access your physical resource (for example your imaging board). For each call to an Allocate method, you should have a call to a corresponding Free method. A control is automatically allocated upon initialization if its AutomaticAllocation property is set to True.

In some cases, even manually invoking the garbage collector will not free your resources; see Invoking the garbage collector.

Free method is skipped due to a thrown error

You should trap errors that might cause your call to the Free method to be skipped. For more information, see Error reporting (Visual Basic .NET and Visual C++ .NET), in the Building an application chapter. Any code between the allocation of the control and the Free method call should be placed in the Try block, while the Free method should be placed in the Finally block.

Invoking the garbage collector

To manually invoke the garbage collector, use the .NET System.GC.Collect method. The garbage collector disposes all objects that are out of scope and that are no longer referenced by any active pointers. For more information on the system.GC class, refer to the .NET documentation available through the MSDN library.

Note that it is good practice to let the garbage collector choose the most appropriate time to run.

There are a few garbage collector peculiarities that you should consider before calling System.GC.Collect.

Freeing an ActiveMIL control created under the AxMatrox.ActiveMIL.* namespace

An ActiveMIL control that is added on a Windows Form at design-time is automatically created under the AxMatrox.ActiveMIL.* namespace. All controls under the AxMatrox.ActiveMIL.* namespace are derived from the AxHost class.

If an AxHost derived control is no longer in use and you want to free it from memory without delay, you should use the Component.Dispose method instead of System.GC.Collect. If you call System.GC.Collect, the control and its associated resources might not be released immediately.

You cannot call the Component.Dispose method to release an ActiveMIL control created under the Matrox.ActiveMIL.* namespace.

Freeing an ActiveMIL control declared in a Try block

An ActiveMIL control that reserves physical resources (for example, the ActiveMIL System control) should not be declared within a Try block. This is because if an error is thrown and the Try block jumps to the Catch block, you will not be able to free your control since it will be out of scope.

In this situation, if you invoke the garbage collector in the Catch or the Finally block, the control and its associated resource are not disposed from memory. This is because an object declared in a Try block is not marked for garbage collection until the end of a procedure (there is still a pointer to that control until the end of the procedure).

For example, if an ActiveMIL System control is declared within the Try block of a procedure and an exception is thrown, you cannot free the System control in the Catch or Finally block since it is out of scope. You can try to release the control by invoking the garbage collector in the Catch or Finally block; however, this would not work because, although the System control is out of scope, it is not marked for garbage collection until the end of the procedure.


Using ActiveMIL with multi-processing and under multi-thread systems

Chapter 22: Using ActiveMIL with multi-processing and under multi-thread systems

This chapter describes how ActiveMIL handles multi-processing and multi-threading.

  • Multi-processing
  • Multi-threading
    • ActiveMIL and multi-threading
      • Using threads with ActiveMIL
      • Thread execution
      • Synchronization
      • ActiveMIL Threading control
      • Error reporting
    • Using Windows' native threads with ActiveX controls (advanced)
      • COM threading models overview
      • Initializing a COM apartment in C++ .NET (unmanaged)
      • Initializing a COM apartment in Visual Basic .NET
      • Accessing an object from another apartment
      • Improving performance in a multi-threaded ActiveMIL application
      • Relaying events to threads


Multi-processing

Next

Multi-processing is the ability to execute various processes (applications) simultaneously.

ActiveMIL applications are autonomous processes (or executables) designed to execute a complete operation or series of operations. Therefore, they can profit from multi-processing by executing independently, without interference from each other.

Not all systems support multi-processing. For example, a simple imaging board with only acquisition capability (such as Matrox Meteor-II) cannot ensure either the response time to a function or the independence of a process necessary for multi-processing. On such boards, two different processes cannot use the same system simultaneously. To use a non-multi-processing system within a multi-processing environment, all processes that need to communicate with the system must do so by sending their requests through a single dedicated process.

Systems that support multi-processing, such as the Host or Matrox Odyssey, have on-board resources (such as, processors) that can be shared by different processes. However, multiple processes running at the same time have to share the available processing time, and cannot share data, since the memory used by one process is protected from all other processes.


Multi-threading

See Also | Previous
Reference:
Threading.Events.Add
Threading.Events.AddFromExternalEvent
Threading.Free
Threading.Threads.Add
Threading.Threads.Item.SynchronizationMode
Threading.Threads.Remove
Threading.Threads.Select
Threading.Wait

ActiveMIL also supports multi-threading. Multi-threading is the ability to perform multiple operations simultaneously in the same process. This is done by creating different threads (execution queues) to ensure sequential execution of operations within the same thread, while allowing simultaneous yet independent execution of operations in other threads.

Threads within a process share the same data. Individual threads can communicate with each other and exchange data.

Multi-threading is most appropriate for applications where independent tasks can be done simultaneously but need to share data or to be controlled and synchronized within a main task.

Multi-threading does not always result in an increase of speed and efficiency. Threads running simultaneously on the same CPU share the same system resources (such as memory). When using a machine with multiple CPUs under Windows, the threads generally run on separate CPUs and provide more processing power. However, since they share the same memory, operations that are I/O intensive and require only simple processing might not be accelerated.

Most applications do not require the use of multiple threads since there are other ways of multi-tasking. Mechanisms such as asynchronous grab and events can be used. Applications resolved by alternative means are often simpler to implement and easier to maintain than multi-threaded applications.

ActiveMIL and multi-threading

When your application contains several independent processing tasks that can be performed in parallel, you can design it so that each part is controlled by a separate thread (or task).

Using threads with ActiveMIL

Under the Windows operating system, you can create as many native threads as you require. These native threads can be created using C++ functions like Win32's CreateThread and MFC's AfxBeginThread, or using the .NET class System.Threading.Thread. For more information on using threads with ActiveX controls, please refer to the Using native Windows threads with ActiveX controls subsection, later in this section.

ActiveMIL provides another kind of thread called selectable thread. Selectable threads are threads executed on an on-board processor that supports multi-threading and can be controlled from a single corresponding thread on the Host. Only ActiveMIL calls will be executed on the on-board processor. You can create a selectable thread by adding it to the collection of selectable threads, using the Threads.Add method of the Threading control.

The first element of the collection of selectable threads is always present and points to the default ActiveMIL thread associated with the Host thread.

Once you have created an ActiveMIL selectable thread and you want to send ActiveMIL calls to that particular thread, you must first select the thread using the Threads.Select method. To return to the default thread or to select a different thread, call the Threads.Select method again.

To delete a specific thread, use the Threads.Remove method. At the end of your application, you should free all selectable threads and threading events, using the Free method.

Thread execution

ActiveMIL calls in any thread are executed as follows:

  • If the target processor is the Host CPU, processing in each thread is determined by the operating system.
  • If the target processor is an on-board processor of a system that supports multi-threading (such as Matrox Odyssey), ActiveMIL automatically creates, and eventually terminates, an on-board thread for each thread that sends commands to the board.

Synchronization

Thread synchronization is generally done using the Host synchronization services (such as, Windows event objects). However, when using a system with an on-board processor, the activities of this processor cannot be synchronized by the Host.

This means that threads continue execution without waiting for the execution of the on-board functions to complete. In most cases, this behavior is acceptable, since it leaves the Host available for other tasks. However, for operations that require sequential execution of commands to return valid results (for example, the Get method of the Image control after the Grab method of the Digitizer control), ActiveMIL automatically synchronizes the threads, forcing the Host to wait for completion of the earlier function(s).

Explicit synchronization of threads is necessary if commands sharing a common resource might conflict with each other. For example, if two threads sharing the same image are not synchronized, and each thread tries to clear the image to a different value, these methods might execute at the same time and the image could be cleared to either value or even to a combination of both. Use the synchronization features of the Threading control to control the flow of such commands.

ActiveMIL Threading control

The Windows operating system is a multi-process and multi-thread operating system. The operating system provides various thread control services, including events (used to synchronize threads).

The ActiveMIL Threading control serves as a link between ActiveMIL and the operating system; the control allows you to create an operating-system-independent ActiveMIL version of these services. ActiveMIL threading events created using the Events.Add method of the Threading control can be used in addition to, or instead of, the events created using commands of the operating system.

You can control and coordinate your threading events, using the Events property. The Threads property manages the ActiveMIL selectable threads. The Wait method synchronizes thread processing by forcing a "wait" state on the specified thread or event. The Free method frees and deletes all selectable threads and threading events that were added to the Threading control.

The OwnerSystem property of the Threading control allows you to specify a particular system on which to create the ActiveMIL selectable threads or threading events. This permits you to synchronize the execution of calls on a specific ActiveMIL system. Also, you can create events whose Native MIL identifiers will be recognized by multiple systems with on-board processors, using the Events.AddFromExternalEvent method.

For an example that uses threads, see MMultiThread.

Error reporting

Some methods in ActiveMIL are asynchronous, that is, they queue their command to the hardware and then immediately return control to the Host. Errors are logged once the call is executed on the processor of the specified system, and are reported immediately after being logged. In this situation, you get asynchronous error messages that do not necessarily specify where the error occurred.

When debugging your application, it is recommended that you set the Threads.Item(ThreadIndex).SynchronizationMode property of the Threading control to thrSynchronous. By doing so, all ActiveMIL calls will be synchronous. Then, if a call generates an error, the error will be thrown immediately.

Using Windows' native threads with ActiveX controls (advanced)

Whereas the previous section explained ActiveMIL selectable threads, this section gives an overview on how Windows' native threads work with ActiveX controls. Since ActiveMIL is a set of ActiveX controls, anyone who needs to use ActiveMIL in a thread other than the main thread of the application should read this section.

Strong knowledge of ActiveX controls and COM (Component Object Model) objects might help you understand the notions presented in this section. It is strongly recommended that you read documentation about apartments for COM objects in the MSDN library or in any other documentation. You should also read the following MSDN library article Descriptions and Workings of OLE Threading Models (Article ID 150777) for a better understanding of COM apartments.

COM threading models overview

ActiveX technology is based on the COM specification. Like most types of variables and objects, COM objects can be used in any thread of a process. Microsoft uses the concept of an apartment to describe how method calls are delivered to an object, the relationship between objects and threads, and the rules for passing interface pointers between threads. An apartment is a virtual box in which one or more threads can exist (live). Calls to a COM object cannot directly cross the boundaries of an apartment. Marshaling must be used to make a call to an object that exists in another apartment.

Two apartment models are currently supported by COM:

  • Single-threaded apartment (STA) model.

    • In this model, only one thread can exist in the apartment, but it can host an unlimited number of objects. You can have multiple single-threaded apartments in one process. Each call to a COM object is transferred to the STA thread in which the object was created. COM synchronizes the calls by sending a message for each call to the STA's message queue. The STA threading model is also known as the "apartment model".

  • Multi-threaded apartment (MTA) model.

    • In this model, one or more threads can exist in the apartment, but there is only one MTA per process. Calls to an object must still processed on the thread in which the object was created. However, no synchronization mechanism is provided when calls are made between threads of the apartment, meaning that two or more threads can simultaneously call the same object. The MTA threading model is also known as the "free-threaded model".

Both models can be used in the same process. This is referred to a "a mixed-model" process.

ActiveMIL controls are a set of MFC-based ActiveX controls that need to run in a single-threaded apartment (STA). A thread using ActiveMIL must be initialized to the STA threading model. Unexpected behaviours and a loss of performance can occur if you run an ActiveMIL control in an MTA, especially in Visual Basic .NET.

Initializing a COM apartment in C++ .NET (unmanaged)

To use a COM object in a thread, you must initialize the COM library at the beginning of the thread; calls to a COM object will fail if the COM library is not initialized. To initialize the library, call CoInitialize() at the beginning of the thread and CoUninitialize() at the end. CoInitialize() will initialize the COM library and specify the type of apartment that the thread will join. The thread will always join an STA apartment when using CoInitialize().

Initializing a COM apartment in Visual Basic .NET

The default setting for a Visual Basic .NET apartment is MTA. To use ActiveMIL in a thread, you must override the default behaviour and change the apartment to an STA. To do so, use the ApartmentState property of the .NET System.Threading.Thread class. The best way to set the ApartmentState property is to change it just before starting the thread. Changing the ApartmentState property in the thread procedure might fail because the apartment state can only be set once for a thread and the .NET framework might set the apartment state just before thread execution.

Accessing an object from another apartment

The rules of COM specify that objects can only be processed on the thread on which they were created. For apartment B to call an object created in apartment A, apartment A's interface pointer must first be marshaled to apartment B. Using the data from the interface pointer, COM creates a proxy/stub pair that allows the two apartments to communicate.

In a single-threaded apartment (STA), all threads exist in a different apartment and the pointer of an object in one apartment needs to be marshaled to be used in another thread. In Visual Basic .NET, the interface pointers of COM objects that exist in an STA are automatically marshaled when used from another apartment. In a C++ application, the CoMarshalInterThreadInterfaceInStream and CoGetInterfaceAndReleaseStream COM functions can be used to marshal the interface pointer.

For example, if you create a worker thread (to do some parallel processing) that requires data from an ActiveX object created in the main thread, you will need to call CoMarshalInterThreadInterfaceInStream from the main thread to marshal the ActiveX object's interface pointer. CoMarshalInterThreadInterfaceInStream writes in a stream the necessary data to initialize a proxy and returns the stream's IStream interface pointer. You should then pass the IStream pointer as an argument to the CoGetInterfaceAndReleaseStream function in the worker thread. CoGetInterfaceAndReleaseStream uses the data on the stream to initialize and point to a proxy through which the worker thread can access the ActiveX object's interface, and then it releases the stream. For more information about CoMarshalInterThreadInterfaceInStream function, refer to the COM reference available through the MSDN library.

Improving performance in a multi-threaded ActiveMIL application

It is a performance costly procedure to perform the thread switches needed to marshal calls to another apartment. To improve the overall performance, ActiveMIL has implemented COM's free-threaded marshaler (FTM) mechanism. An object that implements the FTM can be directly accessed by an object in another single-threaded apartment. In other words, you can directly access an ActiveMIL control in another thread without going through the CoMarshalInterThreadInterfaceInStream function. You just call the required control as if it was in the same thread. For example, to free the Image1 control created in another thread, simply call Image1.Free.

If you do call CoMarshalInterThreadInterfaceInStream to marshal an ActiveMIL control, the function will return the interface pointer to the object (a value that you had supplied as an argument in the first place). This is why we do not use the CoMarshalInterThreadInterfaceInStream function in ActiveMIL examples like MMultiThread.

Note, however, with the free-threaded marshaler, the automatic synchronization mechanism provided by the COM library for single-threaded apartments is bypassed. Therefore, you must provide explicit synchronization code for the threads; to synchronize the threads, you can use the properties and methods of the Threading control or use the thread control services provided by Windows. Consult MSDN to know more about the free-threaded marshaler.

Relaying events to threads

To follow the rules of COM, ActiveMIL needs to send its events to the thread in which the control was created. When an ActiveMIL call generates an event, ActiveMIL internally uses the SendMessage function to pass the event message to the message loop of the thread in which the object exists. For example, if an ActiveMIL control is dropped onto a form at design-time, then the control is created, by default, in the main thread of the application and its events will be received in the main thread. In this case, you must ensure that your synchronization code does not block the main thread from receiving ActiveMIL event messages.