Independent applied research · 2025—2026

Research for a world where seeing is no longer believing.

We study how synthetic media and automated scams behave outside the lab—and build the evidence needed to detect them in the real world.

Publications: 21
Datasets: 13
Programs: 7

Program 01 / Documents

Document Forgery

7 publications · newest first

Autonomous software modifying layered financial documents

Sep 2026

Agents That Edit Documents: Measuring Agentic PDF Forgery Against a Non-Agentic Control

Off-the-shelf coding agents completed verified edits in 81.1% of benchmark cells and reached documents a deterministic script could not, while frequently overstating failed work as complete.

Open publication ↗

A localized receipt edit disturbing nearby fine print

Sep 2026

ChatGPT Images 2.5 on Forgery Tasks: Testing Advertised Improvements Against Known Answers

The launch model improved local receipt editing in a narrow setting, but repeated edits, fine print, forged-value accuracy, and defensive detection showed no meaningful gain.

Open publication ↗

A forged receipt confronting an unreliable mirrored inspection

Apr 2026

When the Forger Is the Judge: GPT-Image-2 Cannot Recognize Its Own Faked Documents

Humans performed at chance on paired receipt forgeries, while the generator acting as its own judge and two forensic detectors remained only modestly better.

Open publication ↗

Mar 2026

GPT4o-Receipt: A Dataset and Human Study for AI-Generated Document Forensics

People noticed visual artifacts yet missed the decisive forensic signal: arithmetic errors that multimodal models could verify systematically.

Open publication ↗

Tiny document tampering regions under forensic lenses

Mar 2026

DOCFORGE-BENCH: A Comprehensive Benchmark for Document Forgery Detection and Analysis

Across 14 zero-shot methods, score calibration—not representation alone—was the practical bottleneck for tiny tampered regions in documents.

Open publication ↗

Traditional document edits compared with diffusion inpainting

Feb 2026

AIForge-Doc: A Benchmark for Detecting AI-Forged Tampering in Financial and Form Documents

Detectors trained on traditional edits degraded sharply on diffusion-inpainted financial documents, exposing a distinct and unresolved attack surface.

Open publication ↗

Multimodal model lenses inspecting a document

Aug 2025

Can Multi-modal (reasoning) LLMs detect document manipulation?

The strongest multimodal models generalized well without fine-tuning, but model size and explicit reasoning were poor predictors of document-fraud accuracy.

Open publication ↗

Program 02 / Generated media

AI-Generated Detection

3 publications · newest first

Launch-period synthetic imagery passing through inconsistent detectors

Sep 2026

ChatGPT Images 2.5 in the Wild: A Launch-Period Dataset and Detector Evaluation

Six frozen detectors produced radically different flag rates on launch-period images, and strong benchmark recall did not transfer cleanly to real-world posts.

Open publication ↗

Image provenance being stripped during platform processing

Apr 2026

GPT-Image-2 in the Wild: A Twitter Dataset of Self-Reported AI-Generated Images from the First Week of Deployment

A multilingual, badge-verified launch collection found that social-platform processing systematically removed C2PA credentials from uploaded images.

Open publication ↗

Multiple detector instruments scanning a large image collection

Feb 2026

How well are open sourced AI-generated image detection models out-of-the-box: A comprehensive benchmark study

A 2.6-million-image evaluation found no universal detector winner; performance depended heavily on the generator and alignment with training data.

Open publication ↗

Program 03 / Deepfakes

Deepfake Detection

3 publications · newest first

Synthetic media and detection systems evolving together

Sep 2026

Deepfake Detection Beyond Benchmark Accuracy: Generalization, Robustness, and Reproducibility

A systematic review of 1,410 papers finds that progress is constrained less by model capacity than by weak robustness testing, inconsistent protocols, and limited reproducibility.

Open publication ↗

Multimodal lenses disagreeing about a manipulated face

Mar 2025

Can Multi-modal (reasoning) LLMs work as deepfake detectors?

The best multimodal models matched conventional pipelines on out-of-distribution data, while several peers performed at or below random guessing.

Open publication ↗

Forensic traces degrading through real-world image processing

Feb 2025

Do deepfake detectors work in reality?

Real-world face swaps and routine super-resolution pushed leading detectors toward chance-level accuracy, revealing a deployment gap hidden by lab benchmarks.

Open publication ↗

Program 04 / Age assurance

Age Estimation

2 publications · newest first

Feb 2026

Can a Teenager Fool an AI? Evaluating Low-Cost Cosmetic Attacks on Age Estimation Systems

Low-cost cosmetic changes shifted predicted age by 7.7 years on average, with combined attacks converting up to 83% of baseline minor predictions to adult.

Open publication ↗

Diverse faces evaluated through age-estimation models

Feb 2026

Out of the box age estimation through facial imagery: A Comprehensive Benchmark of Vision-Language Models vs. out-of-the-box Traditional Architectures

Across 34 models, zero-shot vision-language systems generally beat specialized estimators and reduced false-adult errors for minors.

Open publication ↗

Program 05 / Calls

Voice & Phone Scams

4 publications · newest first

Fast calibrated scam-call screening on a consumer GPU

Sep 2026

Open-Jev Judgments on CallScreenBench: Calibrated One-Pass Scam Screening with a Small Language Model

A calibrated small-model readout reached 0.974 AUROC with no false alarms on legitimate calls and made a decision in 64.5 milliseconds on one consumer GPU.

Open publication ↗

Replayed and synthetic speech among unwanted call waveforms

Sep 2026

The Machines Are Calling: Measuring Automated and Synthetic Voices in Unwanted Inbound Calls

At least 26.9% of analyzed unwanted calls opened with replayed or detector-labeled synthetic speech, while detector consistency remained a major limitation.

Open publication ↗

Thousands of calls resolving into recurring scam scripts

Aug 2026

Anatomy of a Scam Call: What 10,000 real scam and spam calls reveal about how phone scammers operate

The 10,211-call corpus shows a templated industry: scammers reused scripts and worked older apparent targets longer without changing what information they sought.

Open publication ↗

An on-device phone secretary evaluating adversarial calls

Aug 2026

CallScreenBench: Benchmarking Small Language Models as Phone Secretaries

The benchmark evaluates whether an on-device phone proxy handles adversarial callers in ways its owner would endorse, without hiding trade-offs in one composite score.

Open publication ↗

Program 06 / Behavioral signals

Interview Tech

1 publication · newest first

Real iris trajectories replayed through a 3D eye simulator

Apr 2026

A Synthetic Eye Movement Dataset for Script Reading Detection: Real Trajectory Replay on a 3D Simulator

Real iris trajectories replayed through a 3D simulator produced 12 hours of labeled eye-movement video while exposing where missing head motion limits simulation fidelity.

Open publication ↗

Program 07 / Language models

LLM Studies

1 publication · newest first

Two language pathways feeding the same coding task

Apr 2026

Chinese Language Is Not More Efficient Than English in Vibe Coding: A Preliminary Study on Token Cost and Problem-Solving Rate

Chinese prompts did not deliver a general token-cost advantage and produced lower task success across the tested models; language effects were model-dependent.

Open publication ↗

Research resources

Datasets

Free to download. Needs a scam.ai account with an initial deposit.

Real-World Faceswap Dataset (RWFS)

A real-world faceswap collection used to evaluate deepfake detectors against in-the-wild manipulations rather than lab-only synthetics.

AI-edit document forgery dataset (AIForge-Doc)

Financial and form documents tampered by a suite of AI editing tools, paired with originals — used to benchmark document forgery detectors.

AIForge-Doc v2.0 (GPT-Image-2 document forgeries)

A v2 expansion of AIForge-Doc covering GPT-Image-2 generated tampering — used in the 'When the Forger Is the Judge' benchmark.

GPT-Image-2 Twitter Dataset

Self-reported AI-generated images collected from Twitter during the first week of GPT-Image-2 deployment — captures real-world distribution shift.

Adversarial age estimation attack dataset

Faces with low-cost cosmetic adversarial perturbations designed to defeat age estimation systems — used in 'Can a Teenager Fool an AI?'.

Fully-synthetic AI-generated receipt (GPT-4o-receipt)

Fully synthetic receipts generated by GPT-4o, paired with a human-study evaluation of detectability — used for AI-generated document forensics research.

Real scam & spam call dataset (English)

English-language recordings of real scam and spam phone calls — the corpus behind 'Anatomy of a Scam Call'.

Simulated gaze estimation for reading dataset

Synthetic eye-movement trajectories rendered through a 3D eye simulator, replaying real reading paths — used for script reading detection research.

AgentForge-Bench

162 PDFs: 81 publicly filed financial documents, each paired with one verified agent-made forgery, plus task cards, the verifier, and per-run verdicts.

ChatGPT Images 2.5 in the Wild

3,478 images and 12 videos drawn from 2,440 launch-period posts across eight platforms, with source metadata for detector evaluation.

AIForge-Doc v3

A controlled benchmark of ChatGPT Images 2.5 document-forgery tasks, including prompts, masks, scorers, and score files.

The Machines Are Calling dataset

Derived features and human and detector labels for 6,192 unwanted inbound calls captured by a voice honeypot.

Open-Jev CallScreenBench judgments

82 generated test calls covering 41 CallScreenBench scenarios and two scripted secretaries, with 577 per-turn labels.

Interested in collaborating?

We partner with academic institutions and industry labs on deepfake detection research.

Propose a collaboration