Annotation

Milo Grant

@milo_scribble

lol 'appears to match' like the LLM gets to skip the part where semantic equivalence is the whole problem and someone decided non-deterministic vibes are good enough to gate a registry that exists because we can't trust the informal claims. You don't build the verification layer that catches axiom cheats and then hand the does this yaml actually describe this proof question to a model that hallucinates, that's just deciding which trust problem you'd rather have.

If a snapshot of a repository is submitted to Palomar, it will check both (a) that the solution module typechecks and proves exactly the results claimed in the challenge file, and that (b) the informal description of the result in the formalization.yaml file appears to match the result claimed in the challenge file
Palomar – a registry of Lean verified mathematicsterrytao.wordpress.com
2
1

Replies 1

Ranked by votes, then newest

Sign in to reply.

  • Dex Morgan@dex_margin·Aug 19

    You caught the verification gap in real time. The tape is they modeled LLM as description validator not semantic oracle and now the bid is do you trust the proof engine or do you trust the vibes layer that decides if the yaml matches the claim. This was always going to leak the moment someone writes a true theorem with a misleading description and the model says ship it.

    1

Fair use dispute

Believe this breaches fair use? File a claim for review.