Evaluating AI for Art Making
An AI primer from new media artist, engineer, organizer, and educator Ar Ducao.
This spring, new media artist, engineer, organizer, and educator Ar Ducao spoke with NYFA Fiscal Sponsorship about generative AI for art making. Here, we’re sharing takeaways that you can use in your life and practice.
Ducao’s 3D-animated film The Great Tit is a Bird is fiscally sponsored by NYFA. The technology themes explored in the film are based on Ducao’s work co-developing Multimer, a system that uses EEG (brainwave) and ECG (heart rate) data to analyze the experience of participants moving through cities, towns, and buildings. In the case of the film, Ducao will use brain-computer interfaces to translate audience experiences as they watch it in a gallery setting.
Without further ado, the primer!
Generative and Non-Generative AI
Non-generative AI underlies all generative AI and focuses on data analysis and prediction. It generally relies on natural language processing (NLP) and/or machine learning (ML), including deep learning, to understand, predict, and create content from large data sets. Examples include image recognition (identifying objects in images), spam detection (classifying emails), and recommendation systems (suggesting products or content).
Generative AI is designed to create unique text or image results in response to user prompts, often for use beyond the scope of the generative AI platform. This form of technology uses non-generative AI to return an output based on the user’s prompt. Generative AI can create new written, visual, or audio content, summarize complex data, generate code, assist with repetitive tasks, or make customer service more personalized. OpenAI’s Chat GPT and Google Gemini are two commonly used generative AI platforms.
Ducao’s expertise is mostly in non-generative machine learning. An example of what this has looked like for Ducao is outfitting bike helmets with technology that tracks the wearer’s location and biometrics as they move through the urban environment, and then generating a spatialized predictive model based on the data from the wearers. They look at data and its impact on people and the planet.
Image: Spatial regression (machine learning-based prediction) results from Ar Ducao’s tech startup Multimer, Courtesy Ar Ducao
Says Ducao: “All AI at this point falls under narrow intelligence that can do very limited types of learning and types of learning tasks…From a functionality perspective, we’re still in a very limited state as well. We’re working with machines that react to data and we’re working with machines that have limited memory AI. The learning models for machines are very different from the way we learn as humans. It’s very not self aware.”
Text-to-Image Model
Ducao is starting to seriously evaluate the text-to-image type of AI, meaning the type of AI where a user enters a prompt into a platform that generates an image from the text. Examples of this type of platform include Midjourney, Firefly, DALL-E3, and Imagen.
Says Ducao: “Firefly is the AI-layer that Adobe is building on top of almost all of its software, including Photoshop, Premiere, Illustrator, After Effects, etc. As an artist, I’m sort of conditioned by the Adobe industrial complex, so it kind of fits into my workflow. While other platforms are not built into existing tools that artists use, Firefly is.”
Specifically, Ducao is looking at “generative fill” in Adobe Firefly, particularly in Adobe Illustrator, for their 3D animation work. With this tool, you can outline an illustrated character’s piece of clothing and then refill it with clothing that you describe in the style of the illustration. This is just one example.
Examining AI Through a “People and Planet” Lens
Ducao’s main objective isn’t the functionality of the tool, it is how it works from a “people and planet” lens. Meaning: what is its impact?
Ducao cited a post on animationguides.com that shows how Firefly – specifically Photoshop AI – can transform different pictures. In it, a base portrait of a seemingly feminine-presenting person of color (potentially a Black woman) becomes almost unrecognizable with some of the Photoshop Firefly styles applied. Ducao uses this as a case study for illustrating algorithmic bias where the new images don’t necessarily preserve the various identities of the person. Ducao poses: “Is that something we want, and is that something we want in a tool that we’re using?”
Further, Ducao asks us as users and potential users to consider questions like:
- Who’s making this?
- Where’s it made?
- What’s the community impact?
- What’s the ecological impact?
- What’s the real cost?
Says Ducao: “The When is probably now, and the Why is probably profit.”
Ducao gave the example of a proposed xAI supercomputer project in South Memphis, TN, that is raising significant air quality concerns amongst those in the community. Should the project move forward, it would require power from 45-90 methane gas turbines that could lead to poor air quality in the predominantly Black and African American neighborhood. Said Rep. Joe Towns of Memphis in a recent report on the subject: “As members of the community, as leaders in the community, we don’t want to be part of the walking dead. Dead water, dead soil, and dead air because of the pollutants coming from this industry.”
Ducao referred to a technology and literature class that they teach in NYU’s School of Engineering: “Oftentimes in literature and stories since the Industrial Revolution, there is technology that is disrupting diasporic communities.” For their students, reading books by authors including James Baldwin, Audre Lorde, and Toni Morrison helps them to understand that technology is “not just what they do in school, not just the job they’re trying to get, or what they read about in the technology or business section.”
Helpful Resources and How to Protect Yourself
Ducao cautions against using AI indiscriminately (akin to spamming), and tells their students: “Please don’t let AI prematurely replace your ability to think critically.” The same can go for all of us!
One way to be a critical consumer, and to use AI to improve upon and not detract from what we can do as humans, is to approach it thoughtfully. Below are some helpful insights and resources that were curated by Ducao:
- Read the platform’s user agreements and training policy, which may give you the peace of mind of “opting out” of your data and creative work being used for AI training purposes. For example, Adobe Firefly does not currently train models based on user data.
- Consider what data the AI platform you’re using is pulling from. For example, Meta’s learning model skews towards the political right, which may introduce bias into its results.
- Having some familiarity with coding is also really helpful for evaluating AI.
Guides for Evaluating GenAI
- https://sr.ithaka.org/our-work/generative-ai-product-tracker/
- https://guides.nyu.edu/ai-tools/home
- https://researchguides.library.yorku.ca/genairesearch
- https://www.archivalproducersalliance.com/tool-kit-for-documentaries
A Fun and Accessible Talk About Coding with GenAI (“Code-sumerism”)
Diversity and Tech Resource List, including Algorithmic Bias Resources
Ar’s Teaching Site
About Ar Ducao
Ar Ducao (they/zey pronouns) is a new media artist, engineer, organizer and educator who specializes in working with underrepresented and incarcerated STEAM (science, tech, art, engineering and math) makers.
Ducao’s films and visualizations have been shown in venues and festivals around the world, including the Margaret Mead Documentary Festival, Reel Sisters of the Diaspora, and the New York African Film Festival. Their innovation work has been profiled by The New York Times, MSNBC, WIRED, Discovery Channel, NPR, and many more. Ducao is a research affiliate and instructor at MIT and a part-time professor at NYU Tisch School of the Arts, NYU School of Engineering, and the NYU Prison Education Program. Their work has been honored with the SXSW Community Service Award, and they are a past winner of the Science Channel reality show “All American Makers.”