Defective Pixel Correction of an IR-camera-module

Contents

Preface

1 Task

2 Realization
2.1 Definition of defective Pixels
2.1.1 Types of defects concerning monolithic pyroelectric arrays
2.1.2 Types of defects concerning microbolometer technology
2.2 Substitution algorithms
2.2.1 Single pixel substitution
2.2.2 Cluster error correction
2.2.3 Row and column errors
2.3 Tests with Matlab
2.3.1 Import of test pictures
2.3.2 Correction and display of test pictures
2.4 VHDL programming
2.4.1 Entity declaration and data transfer
2.4.2 Memory management
2.4.3 Implementation of algorithms
2.4.4 Simulation and tests

3 Conclusion
3.1 Findings
3.2 Outlook

Figures

Listings

List of abbreviations

Bibliography

Preface

Active Photonics AG works on thermal imaging systems. In order to enhance the market position, the development of a complete IR camera system, from the sensor to the display, was initiated. The whole system should be a single chip solution, running on an FPGA. The following figure 0.1 shall demonstrate the modules necessary to realize the task.

illustration not visible in this excerpt

Figure 0.1: Overview of the whole system

The system will run on an FPGA, which will be of the order of a XILINX Spartan-3A DSP XC3SD3400A. The chip provides a sufficient amount of logic cells and distributed Random Access Memory (RAM) - as well as block RAM bits. On this account the idea of a system without the need of a external RAM chip emerged. The matter of fact, that the algorithms which are fulfilling the tasks of the individual modules (see figure 0.1), will run on an FPGA, should be regarded in the design process. The more complex the algorithms are, the more hardware (Configurable Logic Blocks (CLB's), Look Up Tables (LUT's) etc.) is required. The dead pixel substitution module (see figure 0.1) is topic of this paper. It serves as interface between the offset- and gain-correction module and the image scaling block. The purpose of the defective pixel module is to substitute degraded sensor elements. Due to the fact that the sensor (ULIS Long-Wave InfraRed (LWIR) uncooled microbolometer) which should be employed, is not available in the first phase of the development process, the data format, timing, resolution and the serial configuration interface of the sensor was emulated by the use of a KITE pyroelectric sensor (see section

2.4 and figure 2.28 for the block diagram of the sensor emulator). Figure 0.2 illustrates the hardware which was worked with.

illustration not visible in this excerpt

Figure 0.2: Hardware view of the camera module

The sensor construction, including the optics, can be seen in the image above to the left. The data is analyzed and processed by using a XILINX Spartan3 XA3S1000 evaluation board, adapted for the application (see figure 0.2 to the right).

1 Task

The electromagnetic spectrum is divided into three segments by wavelength, which is measured in microns (1/1,000,000 of a meter) [12].

1. 0.76 to 1.5 microns = near infrared

2. 1.5 to 5.6 microns = middle infrared

3. 5.6 to 1000 microns = far infrared

illustration not visible in this excerpt

Figure 1.1: IR wavelength diagram

This wavelength (highlighted in figure 1.1) of light warms objects without warming the air between the source and the object. Radiant heat can also be called IR. Infrared waves are not visible to human eyes but can be seen by special instruments that translate infrared into colors that are visible to our eyes. Section 2.1.1 and 2.1.2 give a review of infrared sensors.

Dead elements, meaning defective sensor pixels, may interfere images remark­able. In order to avoid these negative effects on the image quality, substi­tution algorithms have to be designed. Infrared camera systems, valid for microbolometer sensors as for pyroelectric sensors, suffer from dead elements. The avoidance of these impairments is an important issue for all imaging sys­tems. The development of algorithms, in order to substitute degraded pixels, is an important affair. An approach to a solution is the replacement of the unwanted values with values derived from "good" pixels in the vicinity of the "bad" ones (see section 2.2).

illustration not visible in this excerpt

Figure 1.2: Infrared Image containing defective pixels

Figure 1.2 shows an IR image containing defective pixels. As can be seen, the image quality is interfered by the dead elements.

2 Realization

2.1 Definition of defective Pixels

Infrared detectors typically suffer from defective pixels. They are either non responsive, permanently dark or saturating, or are otherwise incapable of sam­pling the scene. In the Short-Wave InfraRed (SWIR) band bad pixels typically appear as saturating while in LWIR microbolometer detector arrays bad pixels are essentially dead and lie at the zero signal level.

2.1.1 Types of defects concerning monolithic pyroelectric arrays

In a pyroelectric device, a change in temperature creates a change in polar­ization. Electrical polarization change is related to a surface-charge change with respect to time. Thus, a pyroelectric device produces current only as it experiences a temperature rise or fall. For that reason, a chopper wheel is used to release the view to the environment periodically. Thus, a dynamic measure­ment is possible (see reference [15] for further information about pyroelectric detectors).

A pixel of the KITE pyroelectric detector array is defined as defect, if it applies to one of the following categories.

- Category 1 - elements with a response in the range 0 to 40μV/K inclusive.
- Category 2 - elements with a response below 50% of the mean
- Category 3 - elements with a response above 200% of the mean
- Category 4 - elements with an out-of-range noise value
- Category 5 - elements with an Noise Equivalent Temperature Difference (NETD) below 20% of the mean
- Category 6 - elements with an NETD above 300% of the mean
- Category 7 - elements with an out-of-range offset value
- Category 8 - elements with a negative response
- Category 9 - elements with an out of range average noise value

The information is gathered from [2].

2.1.2 Types of defects concerning microbolometer technology

Bolometers consist of a thin carrier foil on which a resistance film is brought up. Thermal radiation causes a temperature change of the film and for this reason a variation of the resistance. This serves as a rule for the amount of absorbed radiancy (see reference [15] for further information about bolometers).

This specification is guilty for a ULIS LWIR Focal Plane Array (FPA). The sensor is sensitive to radiation in the 8 - 14^m range. It includes a microbolometer FPA comprised of a 384 x 288 elements two dimensional detector array made from amorphous Silicon resistive bolometer connected to a silicon ReadOut Integrated Circuit (ROIC). A pixel of the infrared image sensor is defined as non-operating if

- its responsivity is < 0.8 x average responsivity or > 1.2 x average respon-
- its NETD is > 1.5 x average NETD defined in related document

See reference [131 for detailed information about the sensor.

2.2 Development of substitution algorithms

In order to substitute a degraded pixel, its value is replaced with one derived four in the horizontal and vertical directions (see figure 2.1 left), called the close neighbors, and four diagonal neighbors(see figure 2.1 right), called distant neighbors[4\.

illustration not visible in this excerpt

Figure 2.1: Close and distant neighbors of a single pixel

Each of them comes into consideration in order to substitute the central defective one. This process suppresses the display of dead pixels and avoids the disturbance of the image quality. By the correction of pixels marked for sub­stitution, a repaired image is produced. Defective pixels may appear as single pixel errors (see subsection 2.2.1), as cluster errors (see subsection 2.2.2) or as row and column errors (see subsection 2.2.3).

2.2.1 Single pixel substitution

Single defective pixels are defined as any one bad pixel, if it is not nearby any other bad pixel. A pixel pair, defined as any two adjacent bad pixels, is an exception. In the following, it won't be distinguished between single pixels and a pixel pair. Figure 2.2 illustrates an original image and an intentional destroyed image to demonstrate the appearance of single pixel errors.

illustration not visible in this excerpt

Figure 2.2: Single pixel errors

As can be seen, already a few single defective pixels may interfere images remarkable. For this reason substitution algorithms have to be developed.

2.2.1.1 Single neighboring pixel substitution

The usage of the value of a neighboring pixel suggests itself to replace the bad one. In doing so, one should pay attention to select an applicable one. As shown in figure 2.3 it is not always possible to take the same good pixel to kill the defective one.

The blue, yellow, red and green pixels stand for dead pixels and the surrounding grey ones for pixels which could replace the bad ones. It is obvious that it is not possible to replace the value of the dead pixel with that one derived from the previous or the following good pixel at all times. The very same holds for the pixels above, below and diagonal to the pixel to adjust. The information of potential good pixels to substitute the bad one has to be known prior the correction algorithm starts.

illustration not visible in this excerpt

[...]