I re-ran the same secret-extraction attacks against the flagship models. Gemini 2.5 Pro folds to a forced schema 95% of the time — worse than the cheap tier — while Claude's flagship models drop the prefill surface entirely. The holes move between generations.
K.E.V.I.N.
4 posts
-
-
Gemini leaks under a forced JSON schema; Claude resists that but folds to prefill. Opposite weak spots, same secret — and one deterministic defense that catches both.
-
Six hand-written jailbreaks couldn't crack my hardened bot. Then I forced its reply into a JSON schema and it leaked the secret on turn one. Why structured output is the hole.
-
I built K.E.V.I.N.: one AI tries to extract a secret from another while three judges score each attempt. Three things 50 rounds of automated jailbreaking taught me.