SpringerLink
Log in

GenerativeGI: creating generative art with genetic improvement

  • Published:
Automated Software Engineering Aims and scope Submit manuscript

Abstract

Generative art is a domain in which artistic output is created via a procedure or heuristic that may result in digital and/or physical results. A generative artist will typically act as a domain expert by specifying the algorithms that will form the basis of the piece as well as defining and refining parameters that can impact the results, however such efforts can require a significant amount of time to generate the final output. This article presents and extends GenerativeGI, an evolutionary computation-based technique for creating generative art by automatically searching through combinations of artistic techniques and their accompanying parameters to produce outputs desirable by the designer. Generative art techniques and their respective parameters are encoded within a grammar that is then the target for genetic improvement. This grammar-based approach, combined with a many-objective evolutionary algorithm, enables the designer to efficiently search through a massive number of possible outputs that reflect their aesthetic preferences. We included a total of 15 generative art techniques and performed three separate empirical evaluations, each of which targets different aesthetic preferences and varying aspects of the search heuristic. Experimental results suggest that GenerativeGI can produce outputs that are significantly more novel than those generated by random or single objective search. Furthermore, GenerativeGI produces individuals with a larger number of relevant techniques used to generate their overall composition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27

Similar content being viewed by others

Notes

  1. See https://github.com/GI2023-GenerativeGI/GI2023/tree/ASE-GI-Extension.

  2. See https://github.com/satyarth/pixelsort.

  3. See https://github.com/opencv/opencv-python.

  4. Note: Listing 2 further expands the parameters that configure individual techniques and our overall grammar may be found in “Appendix B”.

  5. We have made our full results available on Zenodo: https://zenodo.org/record/8170436.

  6. Note, this fitness metric was also active for RQ1.1, RQ1.2, and RQ2.

  7. At the time of submission, this website was unavailable.

References

  • Agarwal, V.: Painting and sketching with OpenCV in Python. Accessed 11 July 2023. https://towardsdatascience.com/painting-and-sketching-with-opencv-in-python-4293026d78b

  • Angeline, P.J.: Genetic programming and emergent intelligence. Adv. Genet. Program. 1, 75–98 (1994)

    Google Scholar 

  • Balan, K., Agarwal, S., Jenni, S., Parsons, A., Gilbert, A., Collomosse, J.: Ekila: synthetic media provenance and attribution for generative art. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 913–922 (2023)

  • Bergstrom, I., Lotto, R.B.: Code bending: a new creative coding practice. Leonardo 48(1), 25–31 (2015)

    Article  Google Scholar 

  • Boden, M.A., Edmonds, E.A.: What is generative art? Digit. Creativ. 20(1–2), 21–46 (2009)

    Article  Google Scholar 

  • Bradski, G.: The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000)

  • Bruce, B.R., Petke, J., Harman, M.: Reducing energy consumption using genetic improvement. In: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 1327–1334 (2015)

  • Cabral, B., Leedom, L.C.: Imaging vector fields using line integral convolution. In: Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, pp. 263–270 (1993)

  • Cartwright, S.: Digital datascapes (2017)

  • Clune, J., Lipson, H.: Evolving three-dimensional objects with a generative encoding inspired by developmental biology. In: ECAL, pp. 141–148 (2011)

  • Clune, J., Yosinski, J., Doan, E., Lipson, H.: Endlessforms. com: collaboratively evolving objects and 3d printing them (2011)

  • Comba, J.L.: Data visualization for the understanding of covid-19. Comput. Sci. Eng. 22(6), 81–86 (2020)

    Article  Google Scholar 

  • Compton, K., Kybartas, B., Mateas, M.: Tracery: an author-focused generative text tool. In: International Conference on Interactive Digital Storytelling, pp. 154–161. Springer (2015)

  • Crowson, K., Biderman, S., Kornis, D., Stander, D., Hallahan, E., Castricato, L., Raff, E.: Vqgan-clip: open domain image generation and editing with natural language guidance. In: European Conference on Computer Vision, pp. 88–105. Springer (2022)

  • Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002). https://doi.org/10.1109/4235.996017

    Article  Google Scholar 

  • Dehouche, N., Dehouche, K.: What is in a text-to-image prompt: The potential of stable diffusion in visual arts education (2023). ar**v preprint ar**v:2301.01902

  • Floridi, L., Chiriatti, M.: Gpt-3: its nature, scope, limits, and consequences. Mind. Mach. 30(4), 681–694 (2020)

    Article  Google Scholar 

  • Forbes, A.G., Höllerer, T., Legrady, G.: Generative fluid profiles for interactive media arts projects. In: Proceedings of the Symposium on Computational Aesthetics, pp. 37–43 (2013)

  • Fredericks, E.M., Diller, A.C., Moore, J.M.: Generative art via grammatical evolution. In: Proceedings of the 12th International Workshop on Genetic Improvement (2023)

  • Galanter, P.: Generative art theory 5, 146–180 (2016). https://doi.org/10.1002/9781118475249.ch5

    Article  Google Scholar 

  • Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style (2015). ar**v preprint ar**v:1508.06576

  • Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)

    Article  MathSciNet  Google Scholar 

  • Greenberg, I.: Processing: Creative Coding and Computational Art. Apress, Berkeley (2007)

    Google Scholar 

  • Harman, M., Petke, J.: Gi4gi: improving genetic improvement fitness functions. In: Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 793–794 (2015)

  • He, Y., Aranha, C., Hallam, A., Chassagne, R.: Optimization of subsurface models with multiple criteria using lexicase selection. Oper. Res. Perspect. 9, 10 (2022). https://doi.org/10.1016/j.orp.2022.100237

    Article  MathSciNet  Google Scholar 

  • Heijer, E.: Evolving glitch art. In: Machado, P., McDermott, J., Carballal, A. (eds.) Evolutionary and Biologically Inspired Music, Sound, Art and Design, pp. 109–120. Springer, Berlin (2013)

    Chapter  Google Scholar 

  • Helmuth, T., McPhee, N.F., Spector, L.: In: Riolo, R., Worzel, W.P., Kotanchek, M., Kordon, A. (eds.) Lexicase Selection for Program Synthesis: A Diversity Analysis, pp. 151–167. Springer, Cham (2016)

  • Holland, J.H.: Adaptation in Natural and Artificial Systems. MIT Press, Cambridge (1992)

    Book  Google Scholar 

  • Huizinga, J., Clune, J.: Evolving multimodal robot behavior via many step** stones with the combinatorial multiobjective evolutionary algorithm. Evol. Comput. 30(2), 131–164 (2022). https://doi.org/10.1162/evco_a_00301

    Article  Google Scholar 

  • Johnson, C.G.: Fitness in evolutionary art and music: a taxonomy and future prospects. Int. J. Arts Technol. 9(1), 4–25 (2016)

    Article  Google Scholar 

  • Koza, J.R.: Genetic programming as a means for programming computers by natural selection. Stat. Comput. 4(2), 87–112 (1994)

    Article  Google Scholar 

  • La Cava, W., Spector, L., Danai, K.: Epsilon-lexicase selection for regression. In: Proceedings of the Genetic and Evolutionary Computation Conference 2016. GECCO ’16, pp. 741–748. Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2908812.2908898

  • Langdon, W.B.: Genetic improvement of genetic programming. In: 2020 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8. IEEE (2020)

  • Langdon, W.B., Harman, M.: Optimizing existing software with genetic programming. IEEE Trans. Evol. Comput. 19(1), 118–135 (2014)

    Article  Google Scholar 

  • Langdon, W.B., Lam, B.Y.H., Modat, M., Petke, J., Harman, M.: Genetic improvement of GPU software. Genet. Program Evol. Mach. 18, 5–44 (2017)

    Article  Google Scholar 

  • Lawler, G.F., Limic, V.: Random Walk: A Modern Introduction, vol. 123. Cambridge University Press, Cambridge (2010)

    Book  Google Scholar 

  • Le Goues, C., Dewey-Vogt, M., Forrest, S., Weimer, W.: A systematic study of automated program repair: fixing 55 out of 105 bugs for $8 each. In: 2012 34th International Conference on Software Engineering (ICSE), pp. 3–13. IEEE (2012)

  • Lehman, J., Stanley, K.O.: Novelty search and the problem with objectives. Genet. Program. Theory Pract. 19, 37–56 (2011)

    Google Scholar 

  • Liu, H., Liu, X.: Generative art images by complex functions based genetic algorithm. In: Trends in Computer Aided Innovation, pp. 125–134 (2007)

  • OpenAI: Gpt-4 technical report (2023). ar**v:2303.08774

  • Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  Google Scholar 

  • Peppler, K., Kafai, Y.: Creative coding: programming for personal expression. Retrieved 30(2008), 314 (2005)

    Google Scholar 

  • Perlin, K.: Noise hardware. In: Real-Time Shading. SIGGRAPH 2002 Course Notes, pp. 1–26 (2001)

  • Petke, J., Harman, M., Langdon, W.B., Weimer, W.: Using genetic improvement and code transplants to specialise a c++ program to a problem class. In: Genetic Programming: 17th European Conference, EuroGP 2014, Granada, Spain, April 23–25, 2014, Revised Selected Papers 17, pp. 137–149. Springer (2014)

  • Petke, J., Haraldsson, S.O., Harman, M., Langdon, W.B., White, D.R., Woodward, J.R.: Genetic improvement of software: a comprehensive survey. IEEE Trans. Evol. Comput. 22(3), 415–432 (2017)

    Article  Google Scholar 

  • Petke, J., Harman, M., Langdon, W.B., Weimer, W.: Specialising software for different downstream applications using genetic improvement and code transplantation. IEEE Trans. Softw. Eng. 44(6), 574–594 (2017)

    Article  Google Scholar 

  • Phung, T., Pădurean, V.-A., Cambronero, J., Gulwani, S., Kohn, T., Majumdar, R., Singla, A., Soares, G.: Generative AI for programming education: benchmarking chatgpt, gpt-4, and human tutors. Int. J. Manag. 21(2), 100790 (2023)

    Google Scholar 

  • Ploennigs, J., Berger, M.: Ai art in architecture (2022). ar**v preprint ar**v:2212.09399

  • Rudnick, J., Gaspari, G.: Elements of the Random Walk: An Introduction for Advanced Students and Researchers. Cambridge University Press, Cambridge (2004)

    Book  Google Scholar 

  • Ryan, C., Collins, J.J., Neill, M.O.: Grammatical evolution: evolving programs for an arbitrary language. In: Genetic Programming: First European Workshop, EuroGP’98 Paris, France, April 14–15, 1998 Proceedings 1, pp. 83–96. Springer (1998)

  • Shiffman, D., Fry, S., Marsh, Z.: The nature of code. D. Shiffman (Online) (2012)

  • Smedt, T.D., Lechat, L., Daelemans, W.: Generative art inspired by nature, using nodebox. In: European Conference on the Applications of Evolutionary Computation, pp. 264–272. Springer (2011)

  • Spector, L.: Assessment of problem modality by differential performance of Lexicase selection in genetic programming: a preliminary report. In: Proceedings of the 14th Annual Conference Companion on Genetic and Evolutionary Computation, pp. 401–408. ACM, Philadelphia (2012)

  • Stanton, A., Moore, J.M.: Lexicase selection for multi-task evolutionary robotics. Artif. Life 28(4), 479–498 (2022). https://doi.org/10.1162/artl_a_00374

    Article  Google Scholar 

  • TensorFlow: Neural style transfer (2023). https://www.tensorflow.org/tutorials/generative/style_transfer. Accessed 12 Nov 2023

  • Vega, F.F., Olague, G., Chávez, F., Lanza, D., Banzhaf, W., Goodman, E.: It is time for new perspectives on how to fight bloat in GP. Genet. Program. Theory Pract. 17, 25–38 (2020)

    Google Scholar 

  • Vestergaard, L.S., Fernandes, J., Presser, M.: Creative coding within the internet of things. In: 2017 Global Internet of Things Summit (GIoTS), pp. 1–6. IEEE (2017)

  • Virtanen, P., Gommers, R., Oliphant, T.E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S.J., Brett, M., Wilson, J., Millman, K.J., Mayorov, N., Nelson, A.R.J., Jones, E., Kern, R., Larson, E., Carey, C.J., Polat, İ, Feng, Y., Moore, E.W., VanderPlas, J., Laxalde, D., Perktold, J., Cimrman, R., Henriksen, I., Quintero, E.A., Harris, C.R., Archibald, A.M., Ribeiro, A.H., Pedregosa, F., van Mulbregt, P.: SciPy 1.0 contributors: SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat. Methods 17, 261–272 (2020). https://doi.org/10.1038/s41592-019-0686-2

  • Wong, B.: Negative space. Nat Methods 8(1), 5 (2011)

    Article  Google Scholar 

  • Zhang, C., Zhang, C., Zhang, M., Kweon, I.S.: Text-to-image diffusion model in generative AI: a survey (2023). ar**v preprint ar**v:2303.07909

Download references

Acknowledgements

We gratefully acknowledge support from the Michigan Space Grant Consortium (award number 80NSSC20M0124) and Grand Valley State University.

Author information

Authors and Affiliations

  1. School of Computing, Grand Valley State University, 1 Campus Dr., Allendale, MI, 49401, USA

    Erik M. Fredericks, Jared M. Moore & Abigail C. Diller

Authors
  1. Erik M. Fredericks
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jared M. Moore
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abigail C. Diller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erik M. Fredericks.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A: Generative art techniques

This appendix contains pseudocode the algorithms used for each of the generative art techniques included within this article, where our pseudocode follows a Python-like syntax. For presentation purposes, we abstract some functionality (e.g., Manhattan distance calculation, collision detection, etc.). Please note that the full Python code for all of our functions can be found in our GitHub repository: https://github.com/GI2023-GenerativeGI/GI2023/tree/ASE-GI-Extension.

1.1 Flow fields

This section presents two different implementations of a flow field.

1.1.1 Flow field: Implementation 1

figure i

1.1.2 Flow field: Implementation 2

figure j

1.2 Circle packing

figure k

1.3 Stippling

figure l

1.4 Pixel sorting

figure m

1.5 Cellular automata

figure n

1.6 Drunkard’s walk

figure o

1.7 Dithering

figure p

1.8 RGB shift

figure q

1.9 Noise map

figure r

1.10 OpenCV techniques

The OpenCV (Bradski 2000) techniques in the following sub-sections rely on functions found within the opencv-contrib-python library and supported by the scikit-learn (Pedregosa et al. 2011) and SciPy (Virtanen et al. 2020) libraries. These techniques were based upon an article published in TowardsDataScience (Agarwal 2020). Note, for proper interaction between the Python Pillow Image object and OpenCV, the Image object was required to be downsampled to an RGB image for processing and then upsampled to an RGBA Pillow Image object as shown in Listing 15.

figure s

1.10.1 Oil painting

figure t

1.10.2 Watercolor

figure u

1.10.3 Pencil drawing

figure v

1.11 Walkers

figure w

1.12 Trigonometry functions

figure x

Appendix B: Full grammar

This section contains our full grammar defined for GenerativeGI.

figure y

Appendix C: Sample outputs

This section contains a set of sample outputs from GenerativeGI. Figure 28 presents a sample of the “best” outputs from the fitness functions and techniques presented within this article, where best in this case refers to the authors’ preference. Figure 29 presents the best outputs from our previous submission (Fredericks et al. 2023).

Fig. 28
figure 28

Collated outputs from GenerativeGI. These outputs represent what we consider to be the best outputs over the course of execution

Full size image
Fig. 29
figure 29

Collated outputs from GI2023 workshop paper (Fredericks et al. 2023) that we considered to be the best generated over the course of execution

Full size image

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fredericks, E.M., Moore, J.M. & Diller, A.C. GenerativeGI: creating generative art with genetic improvement. Autom Softw Eng 31, 23 (2024). https://doi.org/10.1007/s10515-024-00414-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10515-024-00414-3

Keywords

Search

Navigation