FRIDA uses a robotic arm to create physical paintings and is able to ‘riff’ on a mistake

Researchers have built an AI-powered robot capable of painting pictures from simple text prompts.

The FRIDA robot, developed at Carnegie Mellon University’s Robotics Institute, uses generative artificial intelligence in a similar way to the viral chatbot ChatGPT and Dall-E 2.

Feeding off other works of art or photographs for inspiration, the robot can create its own unique work of art through user inputs, though takes it a step further than these other systems by producing a physical work of art.

“FRIDA is a robotic painting system, but FRIDA is not an artist,” said Peter Schaldenbrand, a PhD student in the Robotics Institute.

“FRIDA is not generating the ideas to communicate. FRIDA is a system that an artist could collaborate with. The artist can specify high-level goals for FRIDA and then FRIDA can execute them.

“There’s this one painting of a frog ballerina that I think turned out really nicely. It is really silly and fun, and I think the surprise of what FRIDA generated based on my input was really fun.”

FRIDA, which stands for Framework and Robotics Initiative for Developing Arts, uses a robotic arm with a paintbrush taped to it in order to create paintings on a canvas, all the while using machine learning to evaluate the progress of its work.

Its creations were described by the researchers as “impressionistic and whimsical”, with bold brushstrokes and the ability to “riff” on a mistake.

“People wonder if FRIDA is going to take artists’ jobs, but the main goal of the FRIDA project is quite the opposite,” said Jean Oh, a faculty member at the Robotics Institute.

“We want to really promote human creativity through FRIDA. For instance, I personally wanted to be an artist. Now, I can actually collaborate with FRIDA to express my ideas in painting.”

The project, titled ‘FRIDA: A collaborative robot painter with a differentiable, Real2Sim2Real planning environment’, is set to be presented at the 2023 IEEE International Conference on Robotics and Automation in London this May.