Introduction
Evolutionary art is a wide and complex area of the field of computer science, it can be defined as the creation of art through evolutionary methods. To understand this you must first interpret two keywords, evolution and art.
Evolution is defined by the oxford dictionary as "The process by which different kinds of living organism are believed to have developed from earlier forms during the history of the earth." This was first discovered by Charles Darwin as he studied animals and developed his theory of evolution. He came to this conclusion from his suggestion of natural selection, the idea that an environment is only able to hold a set number of entities and the innate instinct for those entities to reproduce means that there will be selection of those creatures that are produced. Natural selection favours individuals that contain the most efficient qualities to survive in whatever environment they may be in and so the fittest of the population will survive. [1]
It was first thought to apply this theory to solve problems by Turing in 1948 and was finally completed in the 1960s being on "optimisation through evolution and recombination", although there were attempts at creating art with an autonomous system as far back as the 18th century, with the 'Musikalisches Würfelspiel', a system that used dice to randomly generate music from precomposed options. [2] Most of the evolutionary art that we understand now originated from the book ‘The blind Watchmaker’ written by Richard Dawkins and published in the year 1986 was where the first concept of evolving 'visual creatures' started. The program worked as follows an initial set of creatures was created and the user chose the ones they liked or disliked the most and the system would then generate a new set of creatures that adhered slightly more to the users preferences. This continued until the user decided. they found a suitable creature they wanted to keep. [2]
Genetic Algorithm [3, 4]
Introduction
A genetic algorithm is a type of algorithm that is used to search a solution space of a certain optimisation problem, to either minimize or maximise a function that evaluates the fitness of an individual within a population. They are a part of evolutionary computing because of the likeness it shares with natural selection seen with living organisms. They are designed to exhibit some aspect of randomness that can be controlled whilst also carrying over the best individuals to produce gradually improving results every iteration. They can be compared to algorithms such as hill climber but are usually much more efficient than other methods and can overcome some problems inferior algorithms exhibit. Genetic algorithms have been used for a wide range of problems to be solved for example AI for games, code breaking and the travelling salesman problem, showing how versatile they can be.
Method [3]
1. Create a population of random candidate solutions named pop.
2. Until the algorithm termination conditions are met, do the following (each iteration is called a generation):
a. Create an empty population named new-pop.
b. While new-pop is not full, do the following:
i. Select two individuals at random from pop so that individuals which are more fit are more likely to be selected.
ii. Cross-over the two individuals to produce two new individuals.
c. Let each individual in new-pop have a random chance to mutate.
d. Replace pop with new-pop.
3. Select the individual from pop with the highest fitness as the solution to the problem.
Components
A genetic algorithm consists of a population, this is a set of possible solutions to a certain problem. This population is continually being changed as new solutions are generated and added into the population whilst others are removed. A single entity of this population is known as an individual. The representation of an individual is very problem specific however a common method is binary representations. The size of the population can vary for different problems, generally a larger population set is preferred because it allows greater variation within the population and gives a better chance of finding a good solution, however the larger the population the longer the algorithm will take to run. A fitness function is a function that evaluates an individual within the population and gives it a specific value which should represent how suitable an individual is for solving a specific problem. It is the method of selection used by the algorithm and is aim is to be optimised so that the best solution can be found. The solution space is a set of all possible states an individual can exist in that satisfies any of the problems constraints. This landscape can have different characteristics depending on the representation. The method of terminating the algorithm can be done in a variety of ways, the simplest and most common would be to run for a number of iterations however it is also possible to end after no considerable advances or after hitting a certain fitness threshold however there is a possibility that this may never happen.
The components mentioned above are fairly standard for most genetic algorithms, however there are also more specific parts of the algorithm that can only be explained for the specific problem at hand, mainly due to how the individuals are represented. These include how to select the individuals from the population that will be reproduced to create new offspring. Generally this is based on the fitness function and encourages the better solutions to be selected so that the good characteristics of that solution is carried over. When solutions are selected in pairs the two solutions are crossed over or mated. This means that they produce new offspring by randomly taking some aspects from either parent solution. Generally two children are created for each pair of parents however this can be changed to better suit the problem at hand. In nature some offspring are mutated which can give them advantages over other creatures of the same species. This is represented in a genetic algorithm as a mutation a typical method is to have a very small chance that each bit of a solution will change.
Genetic Programming
In more recent times is has become ever more common for evolutionary art to be done with genetic programming. This is another evolutionary method that shares many similarities with genetic algorithms, however with one key difference, instead of using typical bit strings the algorithm uses a mathematical expression represented as a tree. [5] The nodes of the tree can be divided into two groups, functions as internal nodes and terminals as the leaves. The function set can be any mathematic operators whilst the terminal set consists of variables and constants. [6] One simple method of using this is by processing the pixel coordinates as inputs into the equation and returning the value to be the colour of the pixel at that location, thereby generating an image. [5] Similarly to genetic algorithms, the individuals within a genetic programming algorithm can be crossed-over. A common technique to merge two individuals is to combine two tree structures that have been selected from the population, the method of selection is discussed later in this part. There are several methods to execute this, one way is to take an entire tree and insert it into another or pick a random node from each entity and swap the subtrees. Keeping to this format ensures that the solution will always be syntactically correct. [7] Selection is most commonly done with fitness-proportionate methods, so the better the fitness for that individual the higher chance that it will be selected to be mated. This can be implemented in multiple ways depending on the problem being solved. [7] Mutation is comparable in the sense that it with a low probability is has the chance to make small changes to an individual. Instead of changing large branches of the tree the changes are much more subtle such as inserting or deleting operators or changing constants and variables.[5] To expand on the base version of a genetic algorithm there are certain specific components to the algorithm such as which mathematical functions that should be used this will directly affect the solution space and determine what kind of images can be produced. A greater range of function gives a more complex and intricate image.[5]
Representation [6]
There are several ways to represent artistic artefacts, one that I previously mentioned was using pixel coordinates. In this instance an image can be represented as an equation. To get the final image take each pixel's co-ordinates and put them into the equation and receive a value that corresponds to the value of that pixel. The values would range from zero being black to one which would be white. This is how a grayscale image could be represented in the genetic programming form which could then be evolved to receive different outcomes. Alternatively an analogous representation is by using a 3D vector which means it is possible to contain RGB values and therefore generate colour images.
Another distinct type of representation that has been used is a tree coding that is definitive to a form called Mondriaan Art. This is a variety that was created by Pieter Cornelis Mondriaan which contains black lines separating blocks of colours that was limited to white, black, red, blue and yellow. To represent this each image contains a tree structure and two values for height and width. Compared to the previous representations each node of the tree are not mathematic operators but horizontal and vertical functions telling how the image is split with the terminal set containing colours to fill that part of the image.
Types of Evolutionary Art
Images
Genetic Programming Methods
There are many different types of images that can be created as a result of evolutionary methods. The first and most basic were those thought of by men such as Stephan Latham, William Todd and Steven Rooke. They began by creating images using genetic programming and using functions to represent visual artefacts. They continued their work and started from very simple images in the begging and continued to add more complex function and parameters to create interesting pieces of art.[2]
More recently however Karl Sims released a paper outlining his expression-based approach to evolving images and has influenced many artists and graphic programmers since. [5] He integrated human choice of images that influenced further generations allowing the user to generate an ever better image of their own preferences. Throughout the paper he not only applies these ideas to images but also 3D objects and animations. He implements many of these ideas and shows the results of his work. He uses complex functions by combining simple functions to give surprisingly detailed and intriguing images. [8]
http://scottdraves.com/sheep.html
https://electricsheep.org/#/about
Software in the Market
As well as successful individual projects that have been achieved there has also been software released for users to use as art making tools, examples of these include ArtMatic[] Kandid[] and Softology[]. [5] Each of these pieces of software have a distinct design direction however it is still possible to get a very wide range of possible images generated. The first piece of software ArtMatic Designer allows user to create still images and animations. It is a versatile piece of software that can be used to design your own images or create generative art using their mutation and randomisation features. Kandid is another versatile piece of software that uses human interaction of various iterations to create an image. The final image is easily programmed by going through the step by step instructions that include setting the population type, colour schema, population size and so on. The user then picks the images they like the most and the algorithm creates new images based off those just chosen, when the user finds a design they like they can then download that image. Softology has produced a piece of windows software called 'Visions of Chaos' that is a professional windows application that allows users with little to no experience with evolutionary algorithms to generate pieces of art however can also be tweaked by professional if desired. They have a wide range of program modes that allow many different types of designs to be produced.
http://www.artmatic.com/
http://kandid.sourceforge.net/index.html
http://softology.com.au/
3D Art
The earliest implementation of 3D objects was by William Latham working with Stephen Todd and was able to produce intricate 3D models.[9] Their software went on to heavily influence many future methods and has since been implemented in commercial software and individual projects. The Mutator software was a software package enabling artists and designers to explore objects in three dimensions. Other implementations have been created such as Rowbottom's Form software and also Groboto a application designed to allow children to experiment with different forms[5] In his original paper on 'Artificial Evolution for Computer Graphics'[8] Karl Sims not only demonstrated how to generate 2D images but also expanded into 3D plant structures. He explains that plants could be grown using a set of genetic parameters which would could be made use of during the creation of procedurally generated structures.[8]
Animation
Electric Sheep
Electric sheep is a very successful distributed computing project where short animations are generated by grouping many computers together to form a supercomputer. The artistic direction the animations follow is based on user interaction from people voting on their favourite animations. Each individual of the distributed computing system processes mathematical instructions defined by the Draves' "Flame algorithm", essentially rendering part of the larger work. Being part of the initiative rewards you with the finished result and can be displayed as a screensaver.
The Problem with Evolutionary Art
Within the field of computer technology there has been considerate advancement in many areas however in the field of evolutionary there have not been many leaps forward, in fact progress has been slow and uninspiring. It can be seen within this paper where many of the references are several decades old which in other fields and their pace of improvement would be unheard of, it can be proposed there must be a reason that the material has not kept up a similar pace.
In a general evolutionary system the objective that is designed to be optimised, also known as the fitness, is clearly identified and comparison can be made easily with the results. The population can be run with very large populations if the computing power is available ensuring a broad search within the solution space.
http://www.rednoise.org/pdal/uploads/evostar2010%20-%20galanter%20-%20the%20problem%20with%20evo%20art.pdf
Automatic Fitness
http://eelcodenheijer.nl/publications/E-den-Heijer-and-AE-Eiben-Comparing-aesthetic-Measures-for-Evolutionary-Art-2010.pdf
http://delivery.acm.org/10.1145/1810000/1809957/p147-hertzmann.pdf?ip=144.173.23.80&id=1809957&acc=ACTIVE%20SERVICE&key=BF07A2EE685417C5%2E970CDDC2F9A3BE3E%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35&__acm__=1542723805_651a26940ae62bf3c730e6c920afe166
Similar to my work
Evolutionary search for the renderings of photographs
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.370.7878&rep=rep1&type=pdf
Evolutionary Methods for NPR
Evolutionary Art using the Fly Algorithm
This study is about Evolutionary art such as digital mosaics.
https://research.bangor.ac.uk/portal/files/18290341/Abbood2017EvoIASP.pdf
Non-photorealistic Rendering of Images as Evolutionary Stained Glass
Uses stained glass models to represent images as art
https://ieeexplore.ieee.org/document/1688564
Project Specification
Introduction
Within this project I will be developing a mobile application that is able to take an input image and return a stylised render of that image, that has been specifically generated by using evolutionary algorithms.
I am splitting this project into two main sections that require consideration, the first part is the algorithm and the second part is the mobile application. These two things could be developed independently of each other as the algorithm handles the image processing whilst the app handles implementation of an algorithm and input and output of images.
I will be making the application for the iOS operating system meaning that I will be using XCode and programming with the swift programming language, I believe this to be the best programming environment from experience and am more familiar with iOS than any other mobile operating system.
I plan on using the fly algorithm that I have evaluated within my literature review as it brings many advantages that have been outlined. This algorithm is the core part of this project that will determine how successful the app will be. A poor algorithm will result in a useless application.
Evaluation
Application
Input
One of the key points I will be evaluating within the application is the input of images. The app should be able to accept any images that are stored on the mobile device in the camera roll album of the phone. The format of this image may be any of the standard file formats used in which case it can be converted to a standard format which is the best to be modified with the algorithm. I will evaluate this by using multiple files or different sizes and file formats to test that the input works for a wide range of potential situations.
Output
Another similar and just as crucial aspect is the output that is generated. The app should be able to generate an image and from there give the user the option of how to further distribute that image, for example sharing to social media sites or downloading the image to the camera roll of the device, additionally the user should not be limited to only one sharing feature. To test this I will generate an image and check that every share feature works correctly.
Responsiveness
I will aim to make this application feel responsive by ensuring transitions are smooth and that controllable aspects of the app are fast such as initial load up time and input and output features are done in a good time. One way to test this will be to use the app and evaluate how responsive it feels. Along with measuring the time it takes to perform specific actions. I aim to get values for input and output to be below 10s.
Stability
Correspondingly a good app will be stable, this means few crashes and that they work well for the job they are required to perform. I hope to achieve this by keeping the app simple and fixing any bugs that I encounter. To test this I will use the app and perform certain situations such as inputting an image and saving that produced rendering and ensure that the app works as intended.
Algorithm
Independent
As a result of the analysis of many papers I have studied I encountered numerous instances of both supervised and unsupervised algorithms to generate images. The algorithm I aim to create will be unsupervised, which greatly increases the speed of which images can be generated, which is listed as an aim for the mobile application part of this project. It also gives a better user experience as no manual input is needed from them.
Processing
The algorithm to be created has a specific function, which is to take an input image and using evolutionary methods generate an artistic rendering that is representative of the original image. A vital detail that should be reviewed is the quality of the rendered output. However due to the creative and artistic nature of this work, it is difficult to set a distinct value on this feature and therefore will have to be judged by myself and compared to other algorithms of a similar nature. Although what can be tested is that the algorithm outputs a rendered artistic image.
Speed
Another factor to consider is the speed of the algorithm. The longer that the algorithm takes the less useful it will be when being used in an application. Users will not wait long periods of time for the algorithm to complete. My aim for this component is that the algorithm will take no more than one minute to complete. This seems to be a reasonable time to wait considering the complicated task at hand.
Bibliography
Collomosse, John P. “Evolutionary search for the artistic renderings of photographs.” Romero, Juan and Penousal Machado. The art of artificial evolution. Springer, 2008. 39-62.
Lewis, Mathew. “Evolutionary visual art and design.” Romero, Juan and Penousal Machado. The art of artificial evolution. Springer, 2008. 3-37.
Neufeld, Craig, Brian J. Ross and William Ralph. “The evolution of artistic filters.” Romero, Juan and Penousal Machado. The art of artificial evolution. Springer, 2008. 335-356.
References
Oxford dictionary
https://en.oxforddictionaries.com/definition/evolution
Intro, GA, GP Types
[1] Introduction to Evolutionary Computing
[2] https://beta.vu.nl/nl/Images/werkstuk-muller_tcm235-273863.pdf
[3] http://www.labeee.ufsc.br/~luis/ga/artigos/Introdu%E7%E3o.pdf
[4] https://www.whitman.edu/Documents/Academics/Mathematics/2014/carrjk.pdf
[5] https://www.inf.utfsm.cl/~mcriff/Tesistas/Games/chapter-lewis-evolutionary-art.pdf
[6] http://www.cs.bham.ac.uk/~rjh/courses/NatureInspiredDesign/2009-10/StudentWork/Group4/EvoArt.pdf
[7] http://www.cs.montana.edu/~bwall/cs580/introduction_to_gp.pdf
[8]http://www.karlsims.com/papers/siggraph91.html
[9] (book)Evolutionary Art and Computers.
Non-Photorealistic Rendering and the Science of Art
Describes how artists create imagery and how do observers respond to artistic imagery. Problem of how to evaluate aesthetics.
https://dl.acm.org/citation.cfm?doid=1809939.1809957