The science of (artificial) intelligence
Herbert Roitblat,
TechTalks,
Nov 16, 2023
I think this post asks the right question: "Do we, can we, have a science of intelligence?" Without this, it would be impossible to have a science of artificial intelligence. The argument offered here brings to mind a paper called Textbooks are all you need, shared again today by George Siemens. The claim is "Despite this small scale, phi-1 (trained on textbook-quality data) attains pass@1 accuracy 50.6% on HumanEval and 55.5% on MBPP." The paper was widely shared but I was at a loss to find any reference online defining what "pass@1 accuracy" was - chatGPT (3.5, the free one) eventually told me it "indicates that the system is evaluated based on whether the correct item or information is present in the top position of the recommended list." But it couldn't (or wouldn't) provide a source. And it's a meaningless measure. What we have here is the "focus on leaderboards" and formalism criticized by Roitblat. And he's not wrong, especially when he criticizes the idea that "Intelligence is the same thing as language ability." It's not, of course, and we'll have to wait for GPTs to have real world experience before we expect real world intelligence. But definitions like this, proposed later in the paper, are just circular: "Successful intelligence is defined as one's ability to set and accomplish personally meaningful goals in one's life, given one's cultural context."
Today: 6 Total: 117 [Direct link] [Share]
