AI generated image detector: what the tests reveal

An AI generated image detector does one thing: it looks at a file and returns a probability score. What it can't tell you is which generator made the image, how the image was edited afterward, or whether the 97% accuracy its homepage advertises was measured on images from that same generator. The gap between lab number and real-world performance matters practically - especially if you're creating holiday card designs with AI painting tools and wondering what happens when a platform scans your work.

Here's what the research shows, and what it means if you're working with AI-generated card artwork.

What does an AI generated image detector actually look at?

The most common approach is a neural classifier trained on pairs of real photographs and AI-generated images. The model learns statistical patterns that differ between the two populations: frequency-domain artifacts from diffusion model noise scheduling, edge rendering inconsistencies in fine hair and fabric, over-repetition of learned textures in backgrounds.

A second layer is metadata analysis. Real photos carry EXIF data - camera model, lens specifications, sometimes GPS coordinates. AI-generated images typically don't, or carry minimal metadata. Some detectors weight this signal heavily. Research published by WebsitePlanet found tools perform about 4% better on average when metadata is present: 87% accuracy with metadata versus 83% without. That 4% difference is real enough to care about, but not large enough to treat metadata as your primary filter.

The third approach - and the one that will matter most within the next two years - is provenance verification. This is a different category altogether. Instead of scanning pixels and inferring origin, provenance tools check whether an image carries a signed credential chain: a C2PA manifest or a SynthID watermark embedded at generation time. These don't degrade the way pixel classifiers do when a new generator releases.

The 83% problem: why the lab number is not the number you get

The WebsitePlanet study tested seven tools against 73 images across six categories from seven different generators. Average accuracy without metadata: 83%. That's the honest real-world number.

Where it breaks down is instructive. Fantasy and mythology images scored 100% average accuracy - they're stylized enough that the AI origin is visually obvious. Society and lifestyle images scored 61% - because a well-composed AI image of two people at a table looks very close, at the pixel level, to a smartphone photo of the same scene.

For holiday card design, this categorization matters. A watercolor-rendered family portrait in front of a fireplace sits firmly in the society category, not the fantasy category. The 83% overall average overstates the accuracy you'd actually get on that specific type of output.

The arXiv benchmark finding makes the performance ceiling explicit: ResNet-50 achieved 99% accuracy when trained and tested on the same generator, but only 58.6% when tested across seven different generators it hadn't seen during training. Which means: if a detector was trained before your generating model released, it will perform below its stated accuracy on your images. This is an architectural consequence of how classifiers work, not a product flaw.

Worth skipping: tools that return a confidence percentage with no information about which generators they were trained on, no confidence range, and no guidance on what to do with the result. A 74% AI probability score from a tool that can't tell you what that number was calibrated against is not information. It is a number presented as information.

Seven generators, one detector: why training data limits every tool

Here's the thing about a detector trained primarily on Midjourney v5 outputs: it learned what Midjourney v5 looks like. It may have limited exposure to Flux 1.1, Imagen 3, or whatever model you're running your card-generation workflow through. The NTIRE 2026 Challenge, held alongside CVPR this year, was specifically designed to evaluate detectors on images from generators the models had not necessarily seen during training. The challenge found no universal winner. Every team that topped the leaderboard on one test set fell on another.

Hive Moderation is the most consistently cited for broad coverage - 96 to 98% on current mainstream generators including Midjourney, DALL-E, Stable Diffusion, Flux, and Imagen 3. That coverage comes with a real cost: it's an API product, priced per image, not a free web tool. If you're running content moderation at scale, the cost per image is trivial. If you're a card designer checking ten files before sending them to a client, the overhead is not worth it.

SightEngine hit 98% in the WebsitePlanet head-to-head and has a well-documented API with transparent confidence thresholds. Similar tier to Hive for professional use.

C2PA credentials and SynthID: the layer that matters more than any scan

The pixel-scanning approach to detection is fundamentally a catch-up game. A new generator releases. Detectors don't have training data for it. Accuracy drops temporarily. Detectors retrain. Accuracy recovers. The generator updates. Repeat.

The provenance approach does not have this problem because it doesn't attempt to infer origin from pixels. It checks a signed chain of custody.

C2PA content credentials work like this: when an image is generated or edited with a C2PA-supporting tool, the application attaches a cryptographically signed manifest to the file. The manifest records what tool was used, when, and by whom. A C2PA reader can verify the chain - or flag exactly where it breaks. Adobe Firefly supports C2PA on every output. Canva has implemented it on AI-generated content. Leica, Sony, and Nikon have committed to embedding C2PA credentials in camera hardware, meaning authentic photographs will soon carry provenance trails by default.

SynthID from Google works differently: an imperceptible watermark embedded in the pixel data at generation time. Unlike EXIF metadata, it's resistant to screenshot attacks and light editing. It degrades under heavy JPEG compression and significant cropping, but survives the kind of social media processing that strips EXIF entirely.

You can verify C2PA credentials at contentcredentials.org - a free browser tool that reads the manifest attached to any image file. If your generator attached provenance data, it will show up there. If it didn't, the absence of credentials is itself informative.

For anyone selling AI-painted card designs commercially - through Etsy, through a print-on-demand service, or directly - understanding what provenance data your generator attaches to outputs is practically useful. It's not about making your work harder to detect. It's about having a documented record of your process, which protects you in a market where disclosure requirements are evolving.

For holiday card designers: when detection actually enters your workflow

You will not typically run your own designs through an AI generated image detector before ordering prints. That's not the practical use case. The situations where it does matter:

You're submitting artwork to a platform or publisher that runs AI content screening. Stock photography services, magazine licensing desks, and some large print-on-demand retailers now require disclosure or have policies around undisclosed AI content. Knowing how their scanners perform helps you comply accurately rather than guessing.

You're sourcing designs from a third party and need to verify origin. Someone selling hand-painted watercolor holiday cards on a marketplace who is actually running a generator through a filter is not a hypothetical scenario. Detection tools give you a baseline for due diligence before paying for exclusive rights.

You're deciding how transparent to be with clients. A corporate gifting buyer who asks whether a card design is AI-generated deserves an accurate answer. Understanding what the tools see in your specific style - watercolor, oil pastel, vintage postcard - gives you the vocabulary to answer precisely.

Three tools worth knowing, and what to do with the results

Hive Moderation: Consistently 96 to 98% across major current generators. API-based, not free, transparent about methodology. The right tool if you're processing volume or need a defensible compliance record. Their documentation specifies which generators are covered - a baseline standard that most tools skip.

SightEngine: 98% in independent testing, API-based, well-documented confidence thresholds. Comparable to Hive for professional use, slightly more accessible for lower-volume use cases.

C2PA reader at contentcredentials.org: Not a pixel detector - it reads provenance metadata. Use it first, before any scanning tool. Five seconds. If the manifest is there and valid, you have more reliable information than any probability score a scanner will give you.

Two patterns to approach with caution: any tool that flagged 27 of 31 real photographs as AI-generated (one tool in the WebsitePlanet study did exactly this), and any tool claiming detection accuracy above 98% without specifying which generators the model was trained on. The number may be technically accurate on a narrow training set. On your images, it may not be.

What the detection landscape looks like from here

The C2PA standard is gaining adoption across the software and hardware ecosystem faster than the scanning tools are improving. Adobe, Apple, Leica, Sony, Nikon, and the BBC have all signed on. Within a couple of years, the default assumption will shift: a credentialed image carries a verifiable provenance trail, and an uncredentialed image carries the burden of establishing origin by other means.

For card designers, this is not a threat. It's a clarification of the rules. The transparency infrastructure the industry is building around AI content gives you the ability to say, clearly and verifiably, what your process was. A watercolor rendering that arrives with a C2PA manifest showing it came from your tool, on a specific date, using a photo you own, is not suspicious. It is documented.

The pixel-scanning tools are useful now, filling the gap between a generator existing and the industry having credentialing infrastructure in place. Once that gap closes, the question of whether a detector can catch your work will matter less than the question of whether your image carries its provenance chain intact.

For now: if you're producing AI-painted holiday card designs and want to understand where your work sits in the detection landscape, run a sample through SightEngine or Hive, check the confidence range rather than just the headline number, and verify what provenance data your generator attached. Those three data points together are more informative than any single percentage score presented as a verdict.