Evolving Glitch ArtEelco den Heijer

Vrije Universiteit Amsterdam

Outline• Introduction

• Research Questions

• Glitch Art

• Genotype for Glitch Art

• Fatality rate

• Visual Impact

• Experiments & Results

• Conclusions

Introduction• What is `Glitch'?

• Electronic music

• Circuit bending

• Alter/ `break' digital circuitry to change sonic output

• Transfer to visual domain

• `Data bending'

Data Bending

• Digital images have specific encoding

• Differs per file format

• Essence of Visual Data Bending

• Changes in digital encoding may change the image in unexpected ways

Glitch Art• Difficult to tell in advance whether

a glitch operation will

• break, or

• change the image

• Like photography

• "Easy to do, but hard to do well"

Research Questions

1. Can we develop a genotype for Glitch Art? What are the main obstacles?

2. Can we measure, control the fatality rate of glitch operations?

3. Can we measure, control the visual impact of glitch operations?

4. Does Evolved glitch art make a new visual contribution to the EvoArt?

Glitch operations• Manual

• Take any HEX editor

• Open a digital image

• Insert random bytes, save

• Select N bytes, delete them, save

• Replace all 'EF12' by 'FA45', save

• Etc.?

Glitch operations

Glitch operations

• position and size

• numbers ∈ [0,..,1]

• relative to image size; make them independent of actual image size

• can be transferred from one image to another (important in our genotype)

Genotype for Glitch• Our genotype is a 'glitch recipe'

1. a selector of one source image

2. a list of one or more glitch operations

Genetic operators

• Tailored for Glitch recipe

• Initialisation

• Crossover

• Mutation

Initialisation• Sample one image from test set of 500

• Test set of famous paintings

• Initialise glitch recipe of 1 to 5 operations

Crossover• Two parents

• Take selector of one of the parents

• One-point crossover on the two lists of glitch operations

Image selector

glitchoperations

parent 1 parent 2

Crossover examples

Mutation (1)

• All parts of the glitch genotype may be subject to mutation

• Select new image (p=0.1)

• Iterate over all glitch operations

• change operator (p=0.1)

Mutation (2)• Iterate over all glitch operations (cont'd)

• change one or more arguments (p=0.1)

• Numeric: +/- n%, where n∈ [0,..,1]

• Byte array; replace byte with random new one (p=0.01)

• Byte; +/- n, where n∈ [1,..,4] (result in [0,..,255]

Mutation (3)

Fatality rate• A glitch operation may 'break' the

image (becomes unreadable)

• Some image formats are more sensitive than others

• Some glitch operations are more crude than others

• Experiment: calculate fatality rate

Fatality Rate: setup• Take image set of 100, convert to all image

formats (bmp,gif,jpeg,png,raw,tiff)

• Iterate over all image formats

• Iterate over all glitch operations (g)

• Iterate over image set (I)

• I'=g(I)

• If I' is not readable, count as fatality

• Rate = #fatality / #total

• Perform 10 runs

Fatality Rate: results

Fatility rate: results• Image Formats

• png is a very sensitive format; unusable for glitch operations

• bmp,gif,jpeg and raw are most robust

• Operations

• delete is most fatal operation; 53% overall, 100% on bmp,png,tiff

• not,insert also have high fatality rate

Visual impact

• Glitch operation may cause visual change in resulting image

• Also depends on image format and glitch operation

• Experiment: calculate visual impact

Visual impact: setup• Setup similar to fatality rate experiment

• Visual impact is calculated using simple image grayscale distance function

•I'=g(I)

• visual impact = d(I,I')If I' is broken, we take d=0

• Calculate avg. d for 100 images, 10 runs

Visual impact: results

Fatality rate: results

• Image format

• gif and raw score high

• png scores lowest (due to high fatality rate)

• Operations

• replace operator has highest visual impact (also see GlitchBot)

Unsupervised EvoArt

• Evolve glitched images without human intervention

• Fitness: Simple aesthetic measure

• Based on Global Contrast Factor (Matkovic et al 2005)

• Calculates difference in hue

• Not specific for Glitch Art

ExperimentPopulation 100Generations 10Runs 10Tournament size 2Image format gifInitialisation Custom Glitch InitCrossover Custom Glitch XOMutation Custom Glitch MutFitness Colour Contrast

Other results

• Fatality rate of glitch operator act as negative selection pressure;

• replace most popular (27%)

• delete, insert least popular (7.8% and 7.1%)

• Over 10 runs, over 10 generations, fatality rate varied between 13 and 20%

Conclusions (1)• It is possible to evolve Glitch images using a

custom Glitch genotype

• Unsupervised, but also for IEC

• The fatality rate in Glitch remains an issue, but is 'tolerable'

• Choose image format and operations wisely

• Ensuring Visual impact of glitch operations also remains an issue

Conclusions (2)• Evolved glitch art :

• Visual addition to EvoArt landscape

• Rough, distorted look 'n feel (punk?)

• Complex image filter

• Image as source

• Pixel displacements

• Not as versatile, or expressive, as S-expressions, SVG, grammars

Future work• Better tailored aesthetic measure for Glitch art

• Proper image distance functions

• Heuristics based on knowledge on image formats

• Tune glitch operations

• Perform experiment with single image

• More glitch operations?

• Integrate with other filter techniques

• MOEA

Thank you!Images and paper(s) at

http://www.few.vu.nl/~eelco Questions?

eelcodenheijer@gmail.com