How Crypto Artists are Using AI to Scale and Augment Creative Processes

Editorial is open for submissions: [email protected]

by Playform.io

We developed Playform as an AI Art studio to allow artists to experiment and explore the use of generative AI as part of their creative process. Our goal is to make AI accessible to artists, realizing several challenges that face artists and creatives when approaching this technology. Now with the advent of Cryptocurrency and the expansion of the Crypto art world, artists and creators are using Playform technology to evolve a new kind of art.

With Artificial Intelligence (AI) becoming incorporated into more aspects of our daily lives, from our phones to driving, it’s only natural that artists would also start to experiment with artificial intelligence. However, this is not new. Since the dawn of AI, over the last 50 years, several artists have explored with computer programs to generate art, incorporating intelligent elements in some cases. The most prominent early example of such work is by Harold Cohen and his art-making program AARON, which produced drawings that followed a set of rules Cohen had hard-coded. American artist Lillian Schwartz, a pioneer in using computer graphics in art, also experimented with AI, among others.

But AI has emerged over the past couple of decades and incorporated machine learning technology. This resulted in a new wave of algorithmic art that uses AI in new ways to make art in the last few years. In contrast to traditional algorithmic art, in which the artist had to write detailed code that already specified the rules for the desired aesthetics, in this new wave the algorithms are set up by the artists to “learn” the aesthetics by looking at many images using machine learning technology. The algorithm only then generates new images that follow the aesthetics it has learned.

The most widely used tool for this is Generative Adversarial Networks (GANs), introduced by Goodfellow in 2014 (Goodfellow 2014), which has been successful in many applications in the AI community. It is the development of GANs that has likely sparked this new wave of AI Art.

GAN-Ocean: In the last few years, since the introductions of GAN, there has been explosive interest in the AI community in developing new types of improved GANs, addressing several of its limitations and extending its capabilities as a generative engine for images, language, and music. This makes it impossible for an artist trying to approach this technology to even know where to start. For example, going to the code repository Github, where developers deposit their open source codes, if you search for the term “GAN”, you are destined to find several tens of thousands of GAN variants available. As an artist, you were left clueless in front of this ocean of GAN-like algorithms, wondering where to start and which algorithm would fit your creative process.

Screen shot of code repository GitHub showing over than 33K available open source codes for GAN variants. (screen shot taken in April 2020)

Computational challenge: Even with the availability of open-source codes, several challenges will face artists. If you are not a code developer who is familiar with today’s programming languages and up to date with latest AI libraries, it is very unlikely that you would be able to benefit from existing open-source codes. Moreover, running such sophisticated AI programs requires the availability of GPUs (Graphical Processing Units), specialized hardware boards that accelerate the processing of multiple folds (10 to 100s folds) to make it possible to train AI models in hours or days instead of several weeks. The price of a GPU board that is able to run state-of-the-art AI algorithms ranges at more than $2,000. Some platforms allow users to use cloud-based GPUs to run open-source codes easily, with hourly charges that can accumulate easily to a substantial bill, if you don’t know what you’re doing.

Massive data requirements: Another challenge that face artists when using GAN-like algorithms is that they require huge amounts of images (tens of thousands) for “training” to get reasonable results. Most of these algorithms are trained and tested on available image datasets, typically curated and catered for AI research. Instead, most artists would like to use their own image collections in their projects. At Playform, we found that in most cases artists want to train AI algorithms with collections of less than 100 images. This small number of images will not be sufficient to train off-the-shelf AI algorithms to generate desired results.

The Terminology Barrier: As a non-AI expert, you will be faced with a vast number of technical terms that you will need to navigate through to get the minimum understanding needed to be in control of the process. You will have to understand concepts like: training, loss, over-fitting, mode collapse, layers, learning rate, kernels, channels, iterations, batch size, and lots and lots of other AI jargons. Most artists would give up here, or blindly, play around with the knobs hoping to get interesting results to realize that you are more likely to win the lottery. Given the cost of GPU time and the lengthy process, that would mean hours and hours of wasted time and resources without getting anything interesting.

We built Playform to make AI accessible for artists. We want artists to be able to explore and experiment with AI as part of their own creative process, without worrying about AI terminology, or the need to navigate unguided through the vast ocean of AI and GAN-like algorithms.

Most generative-AI algorithms are mainly developed by AI researchers in academia and big corporate research labs to push the technology boundary. Artists and creatives are not typically the target audience when these algorithms are developed. The use of these algorithms as part of an artist’s work is an act of creativity by the artist who has to be imaginative in how to bend, adopt, and utilize such non-specialized tools to their purpose. In contrast, Playform focused on how to build AI that can fit the creative process of different artists, from the stage of looking for inspiration, to preparation of assets, all the way to producing final works.

At the research and development side, we had to address the problem that GANs require a large number of images and long hours of training. We had to work on developed optimized versions of GANs that can be trained with tens of images, instead of thousands, and can produce reasonable results in a matter of one or two hours.

Workflow in Playform. User chooses a creative process (Top Left). User then upload inspiration images and possible influences (Bottom Left). As the training progresses, user sees and navigates through results

At the design side, we focused on making the user experience intuitive and free of AI jargon. All the AI is hidden under the hood. Users choose a creative process, upload their own images and press a button to start training. Within minutes results will start to pop up and evolve as the training continues. Within an hour, or a bit more, the process is done and you have already generated thousands of images. Users can navigate through all iterations to find their favorite results. Users can also continue the training process as needed to achieve better results.

Some artists used Playform as a mean of looking for inspirations based on AI uncanny aesthetics. Some other artists fed images of their own artworks, training models that learn their own style and then used these models to generate new artworks based on new inspirations. Virtual reality artists used AI to generate digital assets to be integrated in virtual reality experiences. Several artists used Playform to generate imagery that were used in creating videos. Playform was also used to generate works that were upscaled and printed as a final art product.

SuperRare user and Playform artist Travis LeRoy Southworth (@travisleroy) integrated Playform AI inputting hundreds of past works as a data set to create what the artist refers to as “digital blemishes and color adjustments to construct new portraits.” The result of the inputs transform into surrealist and dream-like figures, featured from his series titled “New Beginnings, Old Endings, Secrets Secreting,” on SuperRare. “I use the Playform GAN to explore alternative methods of art creation and digital outsourcing,” LeRoy writes on the exhibition process. The artist’s past work is used to “train the machine in my paintings’ style,” which is then fed into Playform. The process continues beyond Playform into Photoshop and After Effects where LeRoy animates and “gives life” to Playform outputs. The New York City based artist presented these works using Playform on Artsy in October, 2020. 

SuperRare artist @coldie uses Playform capabilities in GANdinsky 3D – Green and Red – Variant 01, inputting Wassily Kandinsky’s original Image with Arrow into AI generation. “Kandinsky is one of my favorite artists of all time and it is an honor to work with his art in a new way using technology not available during his era,” the artist says of the AI process juxtaposed with traditional Kandsinsky content.

Mattia Cuttini, the current Playform artist in residence, is an Italian artist whose interdisciplinary practice is situated at the intersection of graphic design and blockchain technology. At the forefront of Crypto art, Cuttini expands his body of work using Playform AI capabilities. As part of his residency with Playform,  the artist is experimenting with rubber stamps and inks on hundreds of papers, which are then fed into Playform’s AI. Cuttini looks for technological error and glitch, challenging the AI’s aim for replication. 

Mattia Cuttini will be hosting an Artist Talk on Tuesday, December 15, 2020, to discuss how he used Playform in his most recent series of work and his personal experiences navigating the crypto art market. Attendees will get an additional 5 Playform credits ($25 value). You can register for the event at playform.io/superrare.