GPT-4 Reverse Turing Test
AI
ai evaluation experiment
SHOW HN2023
How HN took it
1 positive, 39 neutral or asking, 3 mixed, 9 critical, out of 52 comments about the project. Thread score -0.18.
Execution: 1 for, 5 againstIdea: 0 for, 4 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Creative idea and fun demo
From 1 comment
Haven’t LLMs been shown to be able to identify its own output vs not accurately? So this isn’t testing GPTs ability to evaluate a candidate against the Turing test so much as its…
vlovich123 on HN
Objections
Test is not interactive and differs from Turing's imitation game
From 2 comments
This is sort of interesting but it’s not interactive. Turing’s imitation game is not a written test, it’s a game, sort of like the Werewolf party game. Its difficulty is going to…
skybrian on HNThe "Turing Test", as we talk about today, is a very simplified take on what Turing described, probably in an effort to pass it. You can read his original paper here [1]. In the…
somenameforme on HN
Grammar and spelling give away human answers, so the test is weak
From 1 comment
Your grammar and spelling aren’t perfect, so that’s a dead giveaway. I wonder what the result would have been if you used perfect grammar and/or GPT intentionally injected some…
IAmGraydon on HN
GPT may just recognise its own output, so the test measures detection not Turing judgment
From 2 comments
Haven’t LLMs been shown to be able to identify its own output vs not accurately? So this isn’t testing GPTs ability to evaluate a candidate against the Turing test so much as its…
vlovich123 on HNDoesn’t this just boil down to whether GPT can distinguish between human written answers and GPT written ones? The actual questions don’t matter at all. It doesn’t seem like a…
ec109685 on HN
Requests
Use trivia or harder questions to distinguish LLM from human
From 1 comment
I was hoping this would feature GPT-4 trying to distinguish between an LLM and a human pretending to be an LLM. If it were me, I'd switch focus and instead of asking "what is…
dmurray on HN
Tell GPT-4 the goal is passing a Turing test or use an older model
From 2 comments
I'm not clear whether they asked GPT4 to pretend to be human or not? I think telling it it's goal was to pass a Turing Test would have a significant effect on it's answers.
andybak on HNMaybe try getting an older GPT-3 model that is just trained to predict text to do it rather than the new one which is trained to tell you that it’s an AI. I wouldn’t be surprised…
circuit10 on HN
Questions
Was the tester told it was evaluating a Turing test?
From 2 comments
Is it just me, or is it unfair that ChatGPT doesn't know it's being tested?
vagab0nd on HNI'm not clear whether they asked GPT4 to pretend to be human or not? I think telling it it's goal was to pass a Turing Test would have a significant effect on it's answers.
andybak on HN
Alternatives named
Voight-Kampff test from Blade Runner
From 1 comment
I guess the turning test won't cut it anymore, we should use the Voight-Kampff test.
number6 on HN