Samsung’s new image-generating AI tool is a little too good

AI definitely nailed the right surly energy for this orange cat.

The pirate ship in Elliott Bay was one thing, but it was a little blurry bee that sent me over the edge.

Samsung would very much like us (and its shareholders) to know that its new phones are the AI-est phones that ever AI-ed, and the Fold 6 that I’m testing comes with a new tool called “sketch to image.” Draw a rough sketch on a photo or an empty note page, and it will use generative AI to turn that thing into an image. I shrugged it off as just another AI thing when Samsung announced it onstage at Unpacked — but y’all, it’s really good. So good that it worries me a little.

Using the sketch to image tool in a note is pretty harmless: you draw something, highlight it, and choose from a handful of styles like “3D cartoon” and “illustration” to turn your doodle into something more detailed. Your image is sent to the cloud, and after a few moments, you’ll see a handful of options to choose from. The results are usually cute and fun; I took requests from my two-year-old, and we drew goofy-looking dump trucks and school buses. Sometimes you get a teddy bear with too many arms, but nothing serious.

I mean, it’s definitely better than any truck I can draw.

Using sketch to image on a photo is where things get weird. I’m the world’s worst artist, and this tool translated my very basic sketches into photorealistic images. The AI-generated elements are incorporated into photos convincingly — scaled and matched to the surroundings in a way that makes them hard to spot as fakes.

That’s how I arrived at the bee problem. I took a photo off a dock just south of downtown Seattle with some flowers in the foreground. Because they’re close to the camera and my focus was in the distance, they’re slightly blurred. I drew the world’s worst sketch of a bee on one of those flowers, figuring AI would insert an in-focus image of a bee — giving it away easily as a fake. Wrong!

The AI bee is blurred, just like the flower it’s landing on. If I didn’t know AI bee’s origin story, there’s no way I’d think twice about it if I scrolled past that image on Instagram. I’d assume the photographer snapped the picture at just the right time or hung around waiting for a bee to fly into the frame — things that take skill and patience. Not the case. In fact, I’m not even sure I’d spot the “AI-generated content” watermark in the corner of the image.

It’s convincing at a glance — but if you look for more than a second, you’ll notice that something is off

I messed around with sketch to image a lot over the past week, and the results aren’t always “blurry bee” good. Often, they’ll have the telltale signs of generative AI art — words scrawled in an alien-looking language or strange textures that don’t look quite right. Convincing at a glance, but if you look for more than a second, you’ll notice that something is off.

Sometimes the content itself gives it away — I don’t think anyone believes I saw a massive pirate ship anchored in Elliott Bay or a giant orange cat in an intersection in West Seattle. But even when the images are so outlandish that nobody could mistake them as genuine, they look realistic.

I did not take a picture of a ghostly pirate ship from Olympic Sculpture Park.

Generally, big things will look obviously fake. But it’s very easy to add another car to a photo of a busy road or a sailboat in the distance, and most people will be none the wiser. Aside from that AI watermark — which is easily cropped out — there’s really no way of knowing that there’s anything unusual about the image. That’s weird!

Out-of-focus bee isn’t going to undo the fabric of our society

I don’t want to blow this out of proportion. Using sketch to image is entirely optional, and plenty of people will never even find it in the gallery app. Out-of-focus bee isn’t going to undo the fabric of our society. But I do think we’re at an increasingly weird place with AI. Sure, you’ve been able to add an out-of-focus bee to an image in Photoshop for ages. But putting this capability in the very same device you use to take the picture and distribute it is another thing. The capabilities and accessibility of generative AI tools are outrunning our shared understanding of what might be real and what might be fake when you scroll through Instagram.

Personally, I feel weirdest about this feature when I’m showing it to my toddler. He’ll grow up knowing that, with the press of a button, you can turn a rough sketch into something more polished. Or with a trivial amount of effort, you can spice up a photo of some train tracks by adding a train. Is that a good thing or a bad thing? I have no idea, but I definitely feel the dissonance between how I perceived artistic creation as a kid and how he’s going to see it.

Sometimes AI makes Hulk Bear.

None of this has prevented me from having a lot of fun with sketch to image. There’s an earnestness about generative AI’s output that’s kind of hilarious — like when I tried to add a green monster poking its head out of Puget Sound and it interpreted my drawing into a massive green polar bear with rippling muscles standing on the shore. Or when it turned a stick figure sketch into a life-size stick figure, complete with a shadow on the ground beneath it.

Is the definition of photography changing in front of our eyes? Is our understanding of truth in images transforming at an unbelievably precarious time for our democracy? Yeah, but also, I took a picture of a rabbit and AI let me put a tiny top hat on its little head. What a time to be alive.

AI might tear the fabric of our society apart. But look at this bunny wearing a little hat!

Sketch to image is available on the Galaxy Z Fold 6 and Z Flip 6. Samsung hasn’t publicly stated whether it will make the feature available to other Galaxy phones, but given the company’s track record of aggressively expanding Galaxy AI to previous-gen models, I think it’s extremely likely to happen. Samsung has also committed to bringing AI features to 200 million phones this year alone. If a little blurry bee is any kind of indicator, I’d say things are bound to get a little weird when that happens.

Posted from: this blog via Microsoft Power Automate.