PabloNet

The debate about whether internet-fitted AIs can be creative always seemed besides the point to me. Making art is hard. My view is that art is about surfacing the inner world, and only in part about skill. It’s unfortunate that art selects so strongly for skill. Can we decorrelate the two? It seems so. Cheap interpolative* creativity used by 8 billion non-artists surely surfaces new views of the world.

For these reasons and since I suck at art I’ve been very excited about the various AI-driven art forms popping up. A couple of months ago I started playing with real-time diffusion of my webcam feed using StreamDiffusion. Specifically, with the intent of generating pretty visuals and hoping to elicit new/interesting feelings. Although it’s very fun, the laptop form-factor breaks the illusion. It feels all temporary and geeky. So, I recently built an LCD frame that can be hanged to a wall with minimal illusion breakers. What I really like about this setup is that making it a proper object opens up new channels of interaction. It’s no longer just a screen, it has permanence. You can leave it there, come back to it at a different time, in a different mood, with different lighting, objects, friends, etc.

Plus, it looks pretty:

The main issue with the current setup is the low frame rate. I managed to increase it using TensorRT and compressing the images in and out, but there’s still a lot of room for improvement.

For those of you that are interested in creating your own or contributing, here are the details:

• Code for the client and server here.
• RunPod for hosting the server.
• Client runs on a Raspberry Pi 5.
• This 10.1" Pi screen.
• This infrared light.
• This generic frame.
• This infrared Pi camera.
• I used a puncher to cut a hole in the frame’s cardboard for the camera (drills didn’t make a clean enough cut).
• I spent hours playing with different preprocessing filters, not just prompting. In general I found the two to be equally important. Without preprocessing, img2img often looked too realistic. To get the blue Picasso style seen in this post, I ended up using a mix of canny edge detection, blue coloration and blurring.

Here’s a fun Hacker News thread about the project.

*Technically, learning in high dimension always amounts to extrapolation.