generative face from random data, on ‘how computers...

Generative Face from Random Data, on ‘How Computers Imagine Humans’

João Martinho Moura CITAR – Research Center for Science

and Technology of the Arts School of Arts, Universidade Católica

Porto, Portugal [email protected]

Paulo Ferreira-Lopes CITAR – Research Center for Science

and Technology of the Arts School of Arts, Universidade Católica

Porto, Portugal [email protected]

ABSTRACT

In1 the recent years, face detection technologies have been widely used by artists to create digital art. Face detection provides new forms of interaction, and allows digital artefacts to detect the presence of human beings, through video capture and facial detection, in real-time. In this paper we explore the algorithm proposed by Paul Viola and Michael Jones, presented in 2001, in order to generate imagined faces from visual randomness.

CCS CONCEPTS • Applied Computing → Arts and Humanities

KEYWORDS Face; Generative; Random; Perception.

ACM Reference format: J. M. Moura, P. Ferreira-Lopes. 2017. Generative Face from Random Data, on ‘How Computers Imagine Humans’. In Proceedings of ARTECH2017 8th International Conference on Digital Arts, September 06-08, 2017, Macau, China (ARTECH 2017), 7 pages. DOI: 10.1145/3106548.3106605

1 Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. ARTECH2017, September 06-08, 2017, Macau, China © 2017 Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-5273-4/17/09$15.00 http://dx.doi.org/10.1145/3106548.3106605

1 INTRODUCTION A first step of any face processing system is detecting the locations in images where faces are present. [1]. In 2001, Paul Viola and Michael Jones introduced a machine learning approach for visual object detection which is capable of processing images extremely fast and achieving high detection rates [2]. The Viola-Jones face detection algorithm is the first ever real-time face detection system [3] in computer vision. OpenCV [4], a widely used free open-source library that enables computers to see [5], implements a version of the Viola-Jones technique, which was extended by Rainer Lienhart and Jochen Maydt [6]. In this technique a cascade function is trained from a vast series of positive and negative images. It is then used to detect objects in other images. These objects can be faces or others, like pedestrians or eyes. Within the inclusion of OpenCV in Processing [7] and Openframeworks [8], many artists approached face detection in their artefacts, creating diverse artworks that explore the face through interaction, tracking users, analysing their emotions, or even drawing and painting portraits. ‘Paul the robot’, a robotic installation that produces observational face drawings of people [9], relies on OpenCV and Viola and Jones detector, to locate the user’s face in a video camera. Major part of generative face related digital artworks use acquired human faces, in real time or previously archived, to create imaginary visual abstractions or interactions related to the person recorded in front of the camera. The preliminary experiences we did with acquired real faces and generative drawing techniques resulted in perceptible face drawings [10], as presented in Figures 1 and 2. Recent research related to artificial intelligence and deep learning showed consistent progress in this matter [11], but requires a large dataset, computing power and more complex algorithms. EvoFIT, presented in 2008, is a method that presents a number of candidate generated faces to a witness for rating of best likeness, and it’s used for criminal investigations [12]. In our research we don’t directly use machine learning or convulsionary deep learning techniques.

ARTECH 2017, September 2017, Macau, China J.M.Moura & P.Ferreira-Lopes

2

Instead, we use compulsive brute force requests into a well-known face detection algorithm, and widely used in research and industry.

Figures 1-2: Generative Face Algorithms from real portraits. Previous work. @João Martinho Moura, 2014

Having examined the Viola-Jones face detection algorithm, we got into a very simple question: can a computer imagine and draw a human face, based only on that proposed algorithm? “How Computer Imagine Humans” is, in fact, a title given to the artwork we are presenting in this paper. Although the title is subjectively questionable, it focuses on a very specific point: we humans have created methods and instructions so that computers can easily detect our faces, and in this work we will use this knowledge to generate abstract pictorial face results. We present an unusual use of the Viola-Jones technique, intended to do the opposite of what it is supposed to achieve: instead of trying to locate and capture faces, we generate facial images ‘imagined’ by a computer through the exploration of hypothetical possibilities. Ruth M. J. Byrne argues that imaginative thought is more rational than scientists have imagined [13] and that counterfactual thoughts rely on thinking about possibilities, just as rational thoughts do. Byrne highlights that people can readily make some inferences based on the possibilities they have kept in their mind. In this publication we present a brief research regarding this stance on imagination, applied to computers. In order to do that, we create visual patterns of randomness, and compulsively use a facial detection algorithm on these random images. We, as human beings, are able to train computers so that they can detect our faces with high precision. We may say that we are, in some way, educating

systems to recognize us through mathematical techniques, training models and supported learning algorithms. An algorithm is the explicit statement of the sequence of operators needed to perform some task [14]. For Donald Knuth ‘a person doesn't really understand something until he can teach it to a computer, i.e., express it as an algorithm’ (as cited in Stiny & Gips, 1978). These face detection models are effective and widely used in various applications, ranging from security, industrial, entertainment and art.

This paper provides an introduction to the face detection, in the context of the development of creative artefacts. We will engage in a brief analysis of the state of the art in the adoption of the face in digital art, and will describe our developments working in a generative face software.

2 THE FACE IN MEDIA ART Since the emergence of technologies that can detect the human face, many artists have created artefacts that interact in multiple ways with the user. Face detection can be useful by simply identifying if a human user is in front of an interactive artefact. We highlight the work ‘Nonfacial Mirror’ [15] from the artistic collective Shinseungback Kimyonghun, which subverts the functionality of a small circular mirror, which runs on its own, through a servo motor, to present the user's face look. Golan Levin also used facial detection for multiple works of reference, as ‘Reface [Portrait Sequencer]’ [16], mixing parts of faces of different users, or ‘Opto-Isolator’ [17] in which a device, with an artificial eye, studies the face, follows and imitates the blink of an user's eyes. Some artists have explored these facial detection algorithms to see images where they do not exist, as in the case of ‘Cloud Face’ [18], which simply points a camera to shoot into the sky, until it finds the shape of a face in the clouds. In the work ‘Faces on Mars’ [19] a software finds false positive results of face detection software while searching for faces on the surface of Mars. Face detection algorithms have evolved considerably in recent years, enabling the identification of the various components of the face, such as the eyes, nose and mouth. The work Faces [20] explores the face substitution, in real time, using the technique of fitting face [21]. Currently, as facial detection and recognition techniques are widespread and more accurate, privacy and civil rights concerns are emerging in society, and, as a result, intensified research work is being developed to avoid automatic face detection [22], as is the case of Adam Harvey's interesting ‘CV Dazzle’ project [23], where a person can wear make-up, different materials or even avant-garde hairstyling to foil face detection through fashion. Concerned with the aggressive overdevelopment of surveillance technology and how this is changing human identity, Sterling Crispin developed ‘Data-Masks’ [24], a production of a series of 3D printed face masks which were algorithmically evolved to satisfy facial recognition algorithms. In 2008, Alan Dunning and Paul Woodrow presented ‘Ghosts in the Machine’, where face tracking algorithms scan each video frame and look for any combination of pixels that forms the basic characteristics of a human face [25], as exemplified in Figure 2. This work directly

Generative Face from Random Data, on “How Computers Imagine Humans” ARTECH 2017, September 2017, Macau, China

3

explores the concept proposed in this publication: presenting artificial face forms from random generated patterns. In the referred work each positive detection will cause a direct artificial face presentation, which may not immediately seem a face to the eyes of the beholder. In the work proposed in this paper we will adopt a more consistent method, in order to have a better understandable generated face form.

3 DEVELOPMENT PROCESS

3.1 Generating the face form This experience can be divided in two distinct phases. The first phase corresponds to the development of a simple algorithm that will create, through random image patterns, a figure that is similar to a human face, without resorting to any picture or video signal. In the second phase, we will overlay all the detections into one single image, expecting an end result similar to a human face. We will use Openframeworks and OpenCV in this task. We start by drawing a set of 500 lines in the screen, inside a rectangular area. Each line will have a random start and end position, as well as a random grey colour. To increase the randomness and diversity, between a random period of about 5 to 15 frames, we will draw 10 opaque triangles in random positions and random grey colours, as demonstrated in Figure 2.

Figure 3: Random generation of lines and triangles on the computer screen. On each frame, we will use the face detection algorithm implemented in OpenCV, with the frontal face HAAR Cascade [26]. If a face detection occurs, we immediately copy the cropped bounding box corresponding to that detection, as we demonstrate in Figure 4.

Figure 4: Random generation. In this case, 2 false positive faces were detected with the Viola-Jones algorithm, marked in the pictures as red squares.

One of the known issues in face detection is related to the rotation of the face [1]. As in this artificial scenario we are sure that it is a false positive detection by default, and to avoid a very inconsistent result, we proceeded with additional iterated detections with very small rotations of -5 to +5 degrees until a maximum range of 20 degrees, and set a minimum threshold of 5 positive matches for each try. This process takes some time, due to the nature of the randomness, which forces the user to wait to see the end result. As this is an experimental approach, time and render speed is clearly not a priority. As we finally find a detection that fits the minimum criteria, we save it in a set of what we call reasonable matches, exemplified in Figure 5.

Figure 5: An example set of 59 matches. Each detection is drawn in a rectangle area, with a small opacity value of 10 percent. In our experiments we found that a set of 20 to 70 mixed images is sufficient for a clear perceived face by the common human eye, although this is not the most important goal, this work intends to present something similar to a human face in its final result.

We set a minimum area of 70 to 150 square pixels for a minimum detection area, ignoring matches that are smaller, increasing, in this way, the detail in the final result. We also added a faded gradient vertical ellipse cropped area, and centred the face in the middle of the final image. The end result can be observed in the following pictures.

In the random generation process, we tried different forms of visual randomness, like points or squares, in different colour or


4

event black and white, and we opted for straight grey lines and triangles, randomly placed in different positions, once the end accumulated result is more analogous to what a hand would do when scribbling a paper.

3.2 Drawing over the generated face form

After having a pictorial reference of a computer-generated face form, we are in conditions to draw abstract scratches into it. We adopted an evolutionary swarm style behaviour with 100 randomly positioned initial points over the previously face form, with three simple rules: curved movement behaviour while over bright areas; straight movement behavior while over dark areas; duplication when it remains in a white area for more than a number of frames. The colour and opacity of each point is also conditioned by the colour of the original face form: the whiter the area, the opaquer the colour will be. Also, threshold limit values will change on each run and also over time. The result in this process can be watched in Figure 6.

Figure 6: Abstract scratches, straight and curved, over the previously generated face form. Detailed view. ©João Martinho Moura

4 CONCLUSIONS This work reveals the hidden visual model proposed in the Viola-Jones algorithm. The main point of this procedure is not an exercise in engineering with the aim of optimizing technical systems, but instead on the artistic result, in the form of a pictorial imagined computer generated portrait. Looking at the end result, we see abstract human faces. We should mention that the hardest work in this research remains in the original exceptional algorithm proposed by Paul Viola and Michael Jones. We are just proposing a kind of visualization of the mathematics behind the face detection technology, in an

opposite direction, through real false positive inputs. In doing so, we tried to answer to the subjective question about how computers imagine humans, even if the output is the result of a set of false detections. This work highlights the random possibilities in artificial generation, despite being an implicit random derived from a previous knowledge, defined by Marcel Conche in his book L´aléatoire, in 1999 (as cited in Caires, 2005). We can say that the outcome result comes from a controlled but unsupervised series of erroneous matches, false engineered detections, that we took, under a process of creative approach, to achieve a final pictorial result. Annemarie Bridy clearly discuss the matter of human agency in creativity when inside a high level of code automation, saying that if we define creativity as a quintessentially human faculty, then computers can never be authors, and we can basically stop there, but if we define creativity alternatively as a set of traits or behaviours, then maybe creativity can be coded [28]. In fact, this work is a set of coded procedures, and technical / artistic options, to obtain a visual result, even if it’s unexpected from the aesthetical point of view, our main interest is to obtain an imagined face form, originated by a computer code. This artwork also reveals the great scientific contributions that Paul Viola and Michael Jones, presented sixteen years ago. We can consider that those contributions are part, not only of the science history, but also of the media art history. Philip Galanter, an artist and theorist in generative art and physical computing wrote an article entitled ‘Shot By Both Sides: Art-Science And The War Between Science And The Humanities’, describing the underlying cause for the lack of acceptance of art-science and technology-based art in the mainstream art world [29] and proposes a new art approach, entitled Complexism, where evolutionary art, especially when offered as an ongoing process rather than a static object, presents the dance of formalism and dynamism and marks a radical shift of emphasis in art away from nouns and towards verbs [30]. The outcome of this research work is an artwork, in a procedural form2. Procedural systems allow unique modes of authorship and singular aesthetic experiences [31]. Thus, we can say that our work will cause a computer to draw an imagined generated face form, on each run, and it is also interesting to note that, looking at the final results, the observer can, perhaps, perceive distinct expressions, head inclinations, age and genres on the different generated faces (Figures 7 to 10).

ACKNOWLEDGMENTS We want to acknowledge the team of Researchers and Professors at Universidade Católica Portuguesa, Porto, and CITAR - Research Center for Science and Technology of the Arts.

IMAGES In this section, we present images that demonstrate the result of the research presented in this paper.

2 Pictures and videos at: http://jmartinho.net/how-computers-imagine-humans/


5


6

Figures 7-10. Four examples of generated faces. Each individual sample is accompanied by the respective random signature, containing a set of false positive face detections. ©João Martinho Moura

Figure 11: A face generation without a face mask. ©João Martinho Moura

Figure 12: A generated face form with a face mask. ©João Martinho Moura

Figure 13: A final result of our experience. ©João Martinho Moura

REFERENCES

[1] M. H. Yang, D. J. Kriegman, and N. Ahuja, 2002. Detecting faces

in images: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 34–58.

DOI: 10.1109/34.982883 ISSN: 0162-8828. [2] P. Viola and M. Jones, 2001. Rapid object detection using a

boosted cascade of simple features, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 1, p. I-511-I-518.

DOI: 10.1109/CVPR.2001.990517 ISBN: 0-7695-1272-0. [3] Y.-Q. Wang, Jun. 2014. An Analysis of the Viola-Jones Face

Detection Algorithm, Image Processing On Line, vol. 4, pp. 128–148.

DOI: 10.5201/ipol.2014.104. [4] G. Bradski, 2000. OpenCV, 2000. [Online]. Available:

http://www.opencv.org. [5] G. Bradski and A. Kaehler, 2008. Learning OpenCV:

Computer Vision with the OpenCV Library, vol. 1. O’Reilly, 2008.

DOI: 10.1109/MRA.2009.933612 ISBN: 0596516134. [6] R. Lienhart and J. Maydt, 2002. An extended set of Haar-like

features for rapid object detection, Proceedings. International Conference on Image Processing, vol. 1, pp. 900–903.

DOI: 10.1109/ICIP.2002.1038171. [7] B. Fry and C. Reas, 2001. Processing.org, Processing, 2001.

[Online]. Available: https://processing.org/. [8] Z. Lieberman, A. Castro, and community, 2004.

Openframeworks, 2004. [Online]. Available: http://openframeworks.cc.

[9] P. Tresset and F. Fol Leymarie, 2013. Portrait drawing by Paul the robot, Computers and Graphics (Pergamon), vol. 37, no. 5, pp. 348–363.

DOI: 10.1016/j.cag.2013.01.012


7

ISSN: 00978493. [10] J. M. Moura, 2014. Generative Face Algorithms, 2014. [Online].

Available: http://jmartinho.net/generative-face-algorithms-2014/.

[11] J. Gauthier, 2014. Conditional generative adversarial nets for convolutional face generation, Class Project for Stanford CS231N: Convolutional Neural Networks for Visual Recognition, Winter semester 2014.

[12] C. D. Frowd and P. J. B. Hancock, 2008. Evolving Human Faces, in The Art of Artificial Evolution A Handbook on Evolutionary Art and Music, J. J. Romero and P. Machado, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 189–210.

DOI: 10.1007/978-3-540-72877-1_9 ISBN: 978-3-540-72877-1. [13] R. M. J. Byrne, Dec. 2007. Précis of The rational imagination:

how people create alternatives to reality., The Behavioral and brain sciences, vol. 30, no. 5–6, pp. 439–476.

DOI: 10.1017/S0140525X07002579 ISBN: 0262025841. [14] G. Stiny and J. Gips, 1978. Algorithmic aesthetics, Computer

Models for Criticism and Design in the Arts. University of California Press, 1978

ISBN: 0520034678. [15] S. S. Back and K. Y. Hun, 2013. Nonfacial Mirror. 2013. [16] G. Levin and Z. Lieberman, 2007. Reface [Portrait Sequencer].

Bitforms gallery, NYC, NYC, 2007. [17] G. Levin and G. Baltus, 2007. Opto-isolator. Bitforms gallery,

NYC, NYC, 2007. [18] S. S. Back and K. Y. Hun, 2012. Cloud Face. Ars Electronica,

Linz, 2012. [19] A. Miharbi, 2010. Detect, Bite, Slam, Virginia Commonwealth

University, 2010. [20] A. Castro and K. McDonald, 2012. Faces, 2012. [Online].

Available: http://arturocastro.net/work/faces.html. [21] J. M. Saragih, S. Lucey, and J. F. Cohn, Jan. 2011. Deformable

model fitting by regularized landmark mean-shift, International Journal of Computer Vision, vol. 91, no. 2, pp. 200–215.

DOI: 10.1007/s11263-010-0380-4 ISSN: 1573-1405. [22] M. J. Wilber, V. Shmatikov, and S. Belongie, Mar. 2016. Can we

still avoid automatic face detection?, arXiv, pp. 1–9. DOI: 10.1109/WACV.2016.7477452 ISBN: 9781509006410. [23] A. Harvey, 2010. CV Dazzle: Camouflage from face detection,

2010. [Online]. Available: https://cvdazzle.com/. [24] S. Crispin, 2014. Data-Masks Biometric Surveillance Masks

Evolving in the Gaze of the Technological Other, University of California, Santa Barbara, 2014.

[25] A. Dunning and P. Woodrow, 2008. Seeing things: Ghosts in the machine, International Journal of Design Sciences and Technology, vol. 15, no. 1, pp. 53–65.

[26] R. Lienhart, 2000. Stump-based 24x24 discrete adaboost frontal face detector, 2000. [Online]. Available: https://github.com/opencv/opencv/blob/master/data/haarcascades/haarcascade_frontalface_default.xml.

[27] C. Caires, 2005. O aleatório na imagem vídeo digital: o projecto experimental [fragments_01], in Artech 2005, 2005, pp. 98–110.

[28] A. Bridy, 2016. The Evolution of Authorship: Work Made by Code, Columbia Journal of Law & the Arts, vol. 39.

DOI: https://ssrn.com/abstract=2836568. [29] P. Galanter, 2011. Shot By Both Sides: Art-Science And The

War Between Science And The Humanities, Artnodes - E-journal on art, science and technology, no. 11.

DOI: 10.7238/a.v0i11.1220 ISSN: 1695-5951. [30] P. Galanter, 2008. Complexism and the Role of Evolutionary

Art, in The Art of Artificial Evolution A Handbook on

Evolutionary Art and Music, J. J. Romero and P. Machado, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 311–334.

DOI: http://link.springer.com/10.1007/978-3-540-72877-1_15 ISBN: 978-3-540-72876-4. [31] M. Carvalhais and P. Cardoso, Nov. 2015. What Then Happens

When Interaction is Not Possible: The Virtuosic Interpretation of Ergodic Artefacts, Journal of Science and Technology of the Arts, vol. 7, no. 1, pp. 55–62.

DOI: 10.7559/citarj.v7i1.144 ISSN: 1646-9798.

generative face from random data, on ‘how computers...

Documents