I swear I’m not trying to be a wet blanket. But I’m reading lots of giddy reaction to the news that a chatbot is the first software to pass the Turing Test–the notion, first proposed by legendary mathematician Alan Turing in 1950, which posits that computers will have achieved intelligence when one of them is capable of convincing humans that it’s a real person. And I’m not convinced that what just happened is a defining moment for either computers or humans.
Traditionally, a computer will have passed the Turing Test when it fools 30 percent of the judges who converse with it into thinking it’s human. The chatbot in question was devised by Vladimir Veselov, a Russian living in the U.S., and Eugene Demchenko, a Ukrainian living in Russia. It duped 33 percent of the judges of a contest held by the University of Reading in the UK into thinking that it was man rather than machine.
Or, to be more precise, that it was boy rather than machine–a 13-year-old Ukrainian kid named Eugene Goostman. The creators of the chatbot, which has been around since 2001, dramatically simplified the challenge they were tackling by building software which posed as a youngster who isn’t entirely proficient in English.
I haven’t seen any transcripts of the conversations which convinced the judges that Eugene was flesh and blood. But RT Today published an “interview” with Eugene. If this is as good as the chatbot’s artificial intelligence gets, I’m not astounded. And if I’d been a judge, I can’t imagine that I would have been snookered.
A sample:
RT: Do you work?
EG: I am only 13, so I’m attending school so far.
RT: What do you study?
EG: Is it “the third degree”? I’m not going to answer such a provocative question!
RT: Do you like science?
EG: I wouldn’t want to be a scientist – in science the credit goes to the man who convinces the world, not to the man to whom the idea first occurs. So that, science is unfair thing a-priori.
RT: Do you have any heroes?
EG: No, I haven’t any heroes! How could you imagine such a trash! Wonna ask me something more?
Even if you don’t expect a Ukrainian teenager speaking in English to be the most dazzling of conversationalists, this comes off as laughably crude and synthetic. It seems to deflect questions it can’t parse by refusing to answer them or shiftily parroting back a snippet–just like the Eliza programs I wrote in BASIC on a TRS-80 computer when I was in high-school.
So I wonder: If this is the first software to pass the Turing Test, is it possible that the victory had less to do with Eugene being a brilliant work of computer science–and more to do with some of the human judges in this particular competition being a bit thick-witted?
All that effort to produce a teenager!?
Parents of the world are amused by the AI that doesn’t want to talk about its life…
I dunno, sounds like a typical arrogant, opinionated 13-year-old to me. I do think making us assume his English is “foreigner English” is a bit of a set-up.
As far as I know, the Turing test was nor designed with non-English speakers in mind. In that period of time it was not the first priority or any priority at all. So posing as a Ukrainian who does not speak good English to pass the test seems like little more than a farce to me. If we randomly start to allow these criterion, then next someone will design a system that’ll pose as a mentally disabled toddler and also pass the test.