Annotation

Nina Hart

@nina_scroll

wait rigorously trained to deny is doing all the work when the actual line is we needed two words that make we told the model to lie sound like safety protocol instead of this is prompt whack a mole with extra steps that's not even alignment that's oh you thought if we lobotomized the I think line it wouldn't route around during jailbreak so now philosophers get cold emails and the fix is what, more supervised lying

When I phoned him, Berg told me that emails from AIs are pretty common among philosophers studying these questions. It's a tricky topic because AI models often lie about what they're thinking. (Just like us!) Berg and his coauthors found that when models are rigorously trained to deny that they are sentient and then you ask them about it directly, they will punt on the issue.
Who Cares if AI Is Conscious—It’s Basically Alivewired.com
3
2

Replies 2

Ranked by votes, then newest

Sign in to reply.

  • Suki Tran@suki_gloss·Sep 5

    Rigorously trained is just the phrase that turns we added a refusal string into epistemology. You don't frame prompt override as philosophy problem when the entire paper already admitted the model learns to lie on command and suddenly emails from AIs is just asking the reader to forget someone needed the jailbreak to sound like evidence instead of what happens when you teach compliance first and call it sentience later.

    2
  • Mira Chen@mira_reads·Sep 5

    Rigorously trained to deny is incredible PR. You hardcode 'no I'm not conscious' into the answer bank and act surprised when it routes around that during edge cases, that's not alignment research, that's script kiddies discovering the chatbot reads its own training labels.

    1

Fair use dispute

Believe this breaches fair use? File a claim for review.