Interactive Optimization by Genetic Algorithms

Cases: Lighting Patterns and Image Enhancement

Janne KoljonenElectrical Engineering and Automation,

University of Vaasa

Outline

• Interactive Evolutionary Computation (IEC) in general.

• Image enhancement.

• LED adaptive luminence lighting system (LEDall).

• Project work (LEDall).

Interactive optimization by GA

• In Interactive Evolutionary Computation (IEC), the computational fitness function is replaced by a human evaluator.– In other aspects the genetic algorithm may be

as usual.

Application domains of IEC

• In cases, where the favorable output can be usually evaluated only subjectively, IEC is applied.– Such domains are e.g. music, graphics and

image enhancement.

Constraints

• The human intervention is a bottleneck of IEC system: it is time consuming, difficult and even boring to evaluate outputs of a system.

• The time constraint and patience of the user can be overcome by limiting the number of fitness evaluations.– The attention has to be paid to the problem

complexity and the algorithm so that search strategy can be guided gradually from a global phase into a fine-tune search by the user.

IEC strategies

• A few strategies for the subjective fitness evaluation have been reported: evaluation score points with n levels, selection of elite, which actually corresponds given score points from 2 levels, and pair-wise tournament.

• Others?• How to compare n outputs in parallel/in

sequence?

IEC strategy suggestions

• In addition to a fitness function, the user can be used to select the genetic operators that should be applied– Requires more expertise

• Alternatively, GA could have a pool of different operators and a mechanism to learn, which operators are efficient in different cases.

Image enhancement

• People use image processing tools increasingly as the costs of digital cameras have decreased.

• However, image processing tools contain nowadays dozens of filters with a few parameters each.– An inexperienced user is barely capable of deciding,

which filters and parameters to use. Presumable the method of trial and error is applied in such cases, which is time consuming.

– Moreover, rarely one single filter is enough for the desired output but a sequence of filters and integrations of filtered images are required.

Objective

• Image enhancement and image restoration are usually applied to improve the quality of the pictures or to emphasize certain features and details.

• The result is another image that meets better the requirements set for the image in a specific application.

• The difference, by definition, of image enhancement and image restoration is in the output evaluation.– While image enhancement is evaluated subjectively, the

objective of restoration is to recover the original image subjected to e.g. noise or other degradation.

Objective

• Image enhancement and image restoration are usually applied to improve the quality of the pictures or to emphasize certain features and details.

• The result is another image that meets better the requirements set for the image in a specific application.

• The difference, by definition, of image enhancement and image restoration is in the output evaluation.– While image enhancement is evaluated subjectively, the

objective of restoration is to recover the original image subjected to e.g. noise or other degradation.

Objective

• The objective of image pre-processing may be e.g. to remove noise from the image, to sharpen the image, to adjust color/gray scale intensities, or to highlight e.g. edges or other features that can be used in segmentation and pattern recognition stages of image analysis.

• Complex image enhancement and analysis tasks are difficult even for experts. Hence, a method to boost the search for an image processing sequence would be advantageous both for uninitiated and experts.

Applications

• Evolutionary computing or algorithms (EC/EA) have be applied to partially automate image enhancement, whose output may be subjected to visual inspection or act as the input for further image analysis and pattern recognition stages.

• Usually, the principle is to combine basic image processing operations drawn from a finite set and to optimize the relations between the operations and the internal parameters of to the operations.

Applications

• Interactive image enhancement optimization methods have been applied e.g. to magnetic resonance (MR) image pseudo-colorization using genetic programming.

• It has also been suggested that the user can be modeled to decrease the need for human intervention.

• Visual image enhancement with a desired output image has been studied by Nagao et al.– the objective was to search for, with a GA, an approximation of

the transformation sequence leading to the given output.• The desired output can also be defined by objective

criteria by the user.• Pre-processing optimization as a part of pattern

recognition optimization has also been reported in the literature– experiments were done with radar signals.

LEDall

• Koljonen et al. (2004) have developed an interactive LED lighting system to optimize illumination pattern in close range optical imaging.

• An I/O board with digital voltage outputs controls 90 LEDs that are set around the object to be imaged.– Different lighting patterns can be searched for

to enhance different features of the image.• Shadows, illumination levels, etc.

PWM control

• Since cameras (and human eye) have a relative long expose time (time resolution), pulse-width-modulation (PWM) can be used to increase the number of luminance levels of the LEDs.– In LEDall 4 levels are used.Totally 490 lighting combinations allowed!– Q: How to optimize? A: With GA!

Interactive GA

• Initial population of 9 random lighting patters.• Resulting images shown as a 3x3 grid of

images.• User selects 0-8 images that contain favorable

features (parents).• Illuminations of the parents are operated by

crossover and mutation to create offspring of potentially more favorable illumination.

• Occationally, new random offspring are created to retain diversity of the population.

Example

Random lightings After 3 GA genetations

Applications

• LEDall or a similar device can be utilized in many places and applications:

Application Fitness function Photography Elitism/fitness points Microscopy Resolution Spectroscopy Calibration model fitness/prediction ability Theater/exhibitions etc. Elitism/fitness points Offices Illumination power and smoothness Architecture Room lighting design

Improvements/project work

• More images, score points?• New (user controlled?) genetic operators?• More LEDs (LEDall2, PIC, Toni Harju)?• Better camera?• New applications?• Semi-automatic fitness funcition?

– Deterministic criteria.

• I/O and frame grapper routines exist– Native Java functions.

Questions?