Wouldn’t it be interesting if we could capture the world around us as a vibe, rather than an accurate record of photons hitting a sensor? The ways we use photography are under threat from AI image generators. But what if the whole definition and practice of capturing a moment, maybe especially casual or personal ones, could shift from aiming for accuracy to something else?

Capturing vibes, not photons

I thought this was very interesting, even as an idea. What if we captured our lives in ways more like how our memories work? Around the same time, I was scrolling through the OpenAI API docs and noticed the DALL-E 2 endpoints, which included an option to create variations. I thought, “How hard would it be to make a Vibe Camera?” 

Turns out, it’s quite simple! Prototypes like this are easy to implement quickly these days. I made one and find using it delightful. I’ve been using the Vibe Camera in my everyday life. I’ve used it at conferences, at work, and at home, like when I got a tree for my birthday. The camera gives me feelings. Using it is moving, which I didn’t quite expect

‍

How I made a Vibe Camera

Obviously, the most important part of a Vibe Camera is translating a conventional image into a generated variation. This is accomplished using the OpenAI variation function from their JavaScript library within a Firebase function protected by authentication. There’s a little bit of infrastructure involved in setting this up to ensure it’s secure and stable, including saving the returned base64-encoded image to a Storage bucket.

exports.getVariant = functions.https.onCall(async (data, context) => {
 if (!context.auth) {
 // check for authentication
   throw new functions.https.HttpsError(
     "failed-precondition",
     "The function must be called while authenticated."
   );
 }
 // grab the blob from the frontend
 const base_blob = data;
 // get the variation from OpenAI
 const blob = await openai.getVariation(base_blob);
 if (blob) {
   // save the variation to a storage bucket and return a url
   const url = await uploadImage(blob, crypto.randomUUID());
   return url;
 }
 return null;
});

It’s also important to properly format the image you’re sending. If you’re taking the image from a webcam, you might need to process it. The front end extracts metadata from the base64-encoded image; in this function, it must be replaced with a slightly different approach.

async getVariation(blob) {
   // convert to buffer
   const buff = Buffer.from(blob, "base64");
   // add enough metadata to make openai happy
   buff.name = "image.png";
   // request a variation
   const response = await openai.createImageVariation(
     buff,
     1,
     "512x512",
     "b64_json"
   );
   // pass back the new blob
   const image_url = response.data.data[0].b64_json;
   return image_url;
 }

To capture the initial image, there are pretty simple ways to open and select webcams. I’m using the MediaDevices browser API, which I’ve combined with an off-the-shelf React webcam component.

useEffect(() => {
       navigator.mediaDevices.enumerateDevices().then((devices) => {
           const videoDevices = devices.filter((device) => device.kind === 'videoinput');
           setDeviceList(videoDevices);
       });
   });

Finally, I don’t want people to actually think about the realistic capture of their phone’s camera. That means manipulating the camera view to obscure details. To do that, I’m using some CSS filters. At first, I wanted to use SVG filters. That works fine on desktop, but fortunately/unfortunately, iOS will just turn those off because they’re not performant enough, so I’ve limited myself to this primarily blurry view.

.camera-feed {
     filter: blur(calc($sz*0.05)) contrast(2.43) grayscale(0.72) hue-rotate(219deg);
   }

To be honest, I’m not fully satisfied with the end result being a digital prototype. I think the Vibe Camera really, really wants to be a physical object. Thankfully,  I have coworkers who are really good at creating physical objects. Watch this space for updates…

Want to build on the Vibe Camera prototype yourself? Check out my GitHub project.