Together with the progress made in the field of Artificial Intelligence, a new question arises: will A.I. algorithms manage to be as creative as humans are? Being that we consider art as being one of the most specific human abilities, it confers weight to the issue at hand: Can Artificial Intelligence create art?
I chose the approach of deconstructing the characteristics contained by a good photography collection and teaching them to an image processing Artificial Intelligence algorithm. As a source of photography I used Google’s Street View service, by retrieving various GPS points and processing the surroundings in the view line. Using various techniques, I made it either look around for a good shot or go finding another place.
I then performed a number of test on the processed results and proceeded to automate the script for posting images that passed the quality threshold on Twitter under the name Streetstagram. Based on various feedback I received on the images, together with my own observations, I modified the script into it’s final form which will make the subject of this article.
2
Art, Photography and Artificial Intelligence
Radu Nicoara
Abstract
Together with the progress made in the field of Artificial Intelligence, a new question arises:
will A.I. algorithms manage to be as creative as humans are? Being that we consider art as
being one of the most specific human abilities, it confers weight to the issue at hand: Can
Artificial Intelligence create art?
I chose the approach of deconstructing the characteristics contained by a good photography
collection and teaching them to an image processing Artificial Intelligence algorithm. As a
source of photography I used Google's Street View service, by retrieving various GPS points
and processing the surroundings in the view line. Using various techniques, I made it either
look around for a good shot or go finding another place.
I then performed a number of test on the processed results and proceeded to automate the
script for posting images that passed the quality threshold on Twitter under the name
Streetstagram. Based on various feedback I received on the images, together with my own
observations, I modified the script into it's final form which will make the subject of this
article.
Introduction
Starting from the idea that beauty is all around and that it is the artist's job to carve it out and
put it on display, in conjunction with the fact that Google's Street View (GSV) project
managed to map an important part of the developed world, made me raise the question of
creating an Artificial Intelligence (A.I.) algorithm that could potentially filter through all the
image data. This could likely create aesthetically pleasing pictures out of the multitude of
possible images taken from GSV.
I decided to write the algorithm in Java both because of the familiarity with the language and
for the open source libraries available for this technology. I chose to use OpenCV [4] as the
main image processing library, and used a Google API [5] for the interaction with GSV.
Photography concepts
Following techniques described in the book of Klaus Bonn [1] as a source of inspiration, I
outlined a few key principles that the algorithm would pursue in order for the composition to
be pleasing and in good taste. This is in no way different from what an experienced artist
would teach to his human student, or indeed the author of the book [4] to its human readers.
The concepts extracted and used in the algorithm are as following:
1. The photography should try to contain the main-contrast outline either at the middle
of the picture, or at 1/3 or 2/3 from the top.
3
Fig 1 The Arrangement of Focus Points
2. The image should have the main lines aligned either horizontally or vertically. A
rotated or oblique picture does not give a clear image of the subject. This is held
especially true in outdoor photography.
3. The picture should contain good lighting, leading to an easy to understand and
pleasing contrast. The image should in no way be too dark, making it hard to
distinguish characteristics of the subject.
4. The image should not contain black or blurry patches that usually appear as a result of
digital processing.
5. The subject and the image as a whole should show structured objects, meaning it
should have a general low entropy. A too random, or indeed a too simple image
would be highly inaesthetic.
6. An exception to the previous rule is vegetation, which the image can contain, but not
in a too big of a percentage. As can be seen in Fig 2, a high ratio of trees and grass,
for example, would look mostly green and in general, make a dull photo.
Fig 2 An image showing only vegetation
7. The image should contain a balanced amount of colors. Too much of any of them
would shift the balance beyond the aesthetic threshold, especially for outdoor
photography.
8. The balance between the complexity of the bottom part and the simplicity of the top
part should be easy to recognise. I wanted to give the algorithm a style of spaced and
opened photography. Therefore, it tries to avoid top-heavy images, leaving the
superior part for skyes or a combination of sky and trees.
9. As observed by the new social current, culminating with Istagram, images that were
processed using an image filter give a fresher perspective on the subject, and
generally have a better appeal to the end viewer. As a result I constructed multiple
image filters for the images to be processed under.
4
10. The series of pictures should on one hand look arbitrary, so that they are not
predictable, and do not look like algorithm-generated. On the other hand, certain
similarities should remain, amounting to what we could call the ,,style" of the artist.
Description
The first part of the algorithm generates a random GPS coordinate and tests if it lands on
land. If it doesn't and it falls on water, there is little need to continue, so a new location is
generated. The following steps is to find the closest physical address to the GPS point. We
use a radius of 20 km, as a bigger radius would usually generate duplicate images.
Next, we check to see if the selected point has GSV information associated to it, as some
whole countries were not added into the GSV system due to poor vehicle access or
geopolitical reasons. After the check, we try to retrieve an image in it's brute form.
We have multiple parameters to take into account when calling for GSV's API, as I will
detail in the following paragraph.
· The width and the height of the image are 600 px, the maximum amount allowed for
the free version of the API. In order for the algorithm to cope with this, we use a
smaller zoom factor.
· The longitude and latitude of the location are taken from the process described above.
· The heading is a random number from 1 to 360, controlling the rotation of the camera.
· The pitch sends information about how high or how low the image should be relative
to the ground. Since if it is too high or too low, the image contains too much sky,
vegetation or simply tarmac, we use a random number from -10 to +20. Fig 3 shows
one such type of image, generated by the algorithm.
Fig 3 A simple image of Budapest, generated by the algorithm
5
We next try to fit the photograph under the rules described in the chapter ,,Photography
concepts". I had to chose if after the verification I would send the algorithm to the same
location and try to find a better angle, or if it should simply generate a new location. Since the
latter option was simpler, and involved a more randomised output, appealing to rule number
10, I adopted the new location version.
For ensuring that rule number 1 was enforced, I made up the following test. It would
incidentally test for rule number 8 as well. I cut up the image into various pieces and used an
edge detection algorithm, as can be seen in Fig 4. I used OpenCV [4] library to perform the
mentioned processing, and mostly any image manipulation described in this article as well.
Fig 4 An image after Edge Detection Processing through OpenCV
After running the data, the A.I. algorithm would count the number of edges in each segment,
and then calculate the proportions. The conditions for dropping the image were as follows:
· Top part has above 90 edges and there are no trees in the picture
· The top possesses over 500 edges, and the whole image has under 1100 edges. This
would make it a top heavy image
In order to find if vegetation exists in the picture, we look for patches that have a high edge
density, as can be seen at the bottom part of Fig 4. If such patches exist, we move from a
normal image test to a vegetation image test. This means that there can not be too much
vegetation in the top of the screen, no matter how many leaf edges are detected in the bottom
part.
Rule number 2 is not difficult to enforce as it is mostly controlled by the parameters used in
calling the API. Since no additional issues have been observed, no other filters have been
added for testing this.
For rule number 3 we check the median value each channel. When checking against the
luminosity of the image we use OpenCV to ensure for each channel that the value is between
115 and 175. A value of 90 and below would mean the image is too dark or contains too
many dark patches like building fronts. Also, GSV has multiple indoors locations mapped,
which we want to avoid, and having less natural light, have a lower luminosity value.
6
Rule number 7 is enforced by checking if between any two of the three image channels we
have a difference greater than 23, as that would mean one of the colors is too prevalent and
therefore unbalanced and aesthetically unpleasing.
For rule number 4, we cut the top 20 pixels and bottom part of 40 pixels of the picture. We
then check for the luminosity of both patches. If it falls under 10, this means that the patch is
black, a bug of GSV from projecting the images into spherical estimates. We then take the
bottom part and count the edges contained, after removing about ¼ from each side, so we
only process the middle. If the patch contains 3 or less edges, it means the images has a
blurry bottom.
Fig 5 Example of blurry or dark patches
For the point number 5 to be ensured, the algorithm counts the number of edges in the whole
picture. Experimentally, I determined that a number of edges an image should contain should
be no bigger than 3300.
I then proceeded to create a number of nine different image filters, and used two different
vignette effects, so that the algorithm, which I named Streetstagram, would have a total of 18
different effects for our images to choose from. This is a good addition to point number 10.
Since the subject of this paper is in no way the creation of such filters, I will simply leave an
example of them in action.
Fig 6 An original image and filtered examples
More details about the filters can be found in
Streetstagram's official webpage
[2], together
with examples and documentation. In short, we generate random numbers to control which
7
filter should be used, and another set for deciding whether or not a vignette should be applied,
and how strong it should be.
Results
I will proceed in posting below a number of images extracted by the algorithm. More can be
found on
Streetstagram's Twitter Page
[3] or on
Streetstagram's official website
[2]
Fig 7 Different images that the algorithm considered good enough to publish
As can be seen in the twitter page of the project, the images posted have experienced a
relative success. They indeed show a unitary style of photography, with large variations in
color and subject. Fig 7 completely illustrates that the 10 guidelines outlined at the beginning
of the article have indeed been fulfilled.
Conclusions and Further Research
I am confident that the scope of the article has been completed, that is showing that A.I.
algorithms can be thought photography in the same way a human student can. The relative
success of the images on social networks comes to show that human viewers can come to
appreciate A.I. art.
Since this article's subject is an art project, it would be a pleasant addition to investigate the
reaction of a number of human subjects towards the algorithm's photography. It could be
interesting to see if any of the human subjects notices that the images were not taken by a real
person, and if the style of the photography is visible.
8
References
[1] Klaus Bohn. ,,50 Principles of Composition in Photography" CCB Publishing - British
Columbia Canada, 2006
[2] Streetstagram official webpage
[3] Streetstagram Twitter Page
[4] Bradski, Gary, and Adrian Kaehler. Learning OpenCV: Computer vision with the
OpenCV library. " O'Reilly Media, Inc.", 2008.
[5] Anguelov, Dragomir, et al. "Google street view: Capturing the world at street level."
(2010).
Frequently asked questions
What is the main topic of "Art, Photography and Artificial Intelligence"?
The article explores the potential of Artificial Intelligence (AI) algorithms to create art, specifically focusing on photography. It details an experiment where an AI algorithm is taught principles of good photography and tasked with capturing aesthetically pleasing images from Google Street View.
What are the key concepts of photography used in the AI algorithm?
The algorithm uses several concepts of good photography, including:
- Positioning the main contrast outline at the middle, 1/3 or 2/3 of the image.
- Maintaining horizontal or vertical alignment of main lines.
- Ensuring good lighting and pleasing contrast.
- Avoiding black or blurry patches.
- Presenting structured objects with low entropy.
- Controlling the amount of vegetation in the image.
- Balancing colors within the image.
- Creating a recognisable complexity balance.
- Applying image filters.
- Arbitrariness to prevent algorithm-generated images.
How does the algorithm find suitable images?
The algorithm generates random GPS coordinates, checks if they are on land, finds the nearest physical address, and verifies if Google Street View (GSV) data is available. It then retrieves images using GSV's API, applying filters to meet the outlined photography concepts. The algorithm generates the following images:
- Width and height of 600 px
- Longitude and Latitude extracted from the random GPS coordinate
- Heading of 1 to 360, controlling camera rotation
- Pitch from -10 to +20 controlling image height
What image processing techniques are used?
OpenCV library is used for image processing, including edge detection, luminosity checks, and color balancing. Image filters and vignette effects are also applied.
What is Streetstagram?
Streetstagram is the name of the AI algorithm and the related art project. The algorithm is designed to create visually pleasing images, automatically post these pictures on social media, and is also a social media presence with a twitter page and website.
What are some filters that Streetstagram uses?
Streetstagram has a total of 18 different effects for images to choose from that include image filters and vignette effects. The filters help with balance and composition.
What were the results of the experiment?
The algorithm was able to generate images that demonstrate a unitary style with diverse colors and subjects, and are aesthetically pleasing, suggesting that AI can learn and apply principles of photography.
What further research can be done?
Further research can investigate the reaction of human subjects towards the algorithm's photography. It would be interesting to see if any of the human subjects notices that the images were not taken by a real person, and if the style of the photography is visible.
Excerpt out of 8 pages
- scroll top
- Quote paper
- Radu Nicoara (Author), 2016, Art, Photography And Artificial Intelligence, Munich, GRIN Verlag, https://www.grin.com/document/338038
Look inside the ebook
