{"id":87523,"date":"2024-02-05T21:02:10","date_gmt":"2024-02-06T03:02:10","guid":{"rendered":"http:\/\/www.kateva.org\/sh\/?p=87523"},"modified":"2024-02-05T21:02:10","modified_gmt":"2024-02-06T03:02:10","slug":"as-these-models-get-bigger-and-are-trained-on-more-data-they-improve-on-individual-language-related-abilities-and-also-develop-new-ones-by-combining-skills-in-a-manner-that-hints-at-understanding","status":"publish","type":"post","link":"http:\/\/www.kateva.org\/sh\/?p=87523","title":{"rendered":"&#8220;as these models get bigger and are trained on more data, they improve on individual language-related abilities and also develop new ones by combining skills in a manner that hints at understanding&#8221;"},"content":{"rendered":"<p><a href=\"https:\/\/www.quantamagazine.org\/new-theory-suggests-chatbots-can-understand-text-20240122\/\">Link<\/a>. &#8220;&#8230;  a mathematically provable argument for how and why an LLM can develop so many abilities&#8230; when Arora and his team tested some of its predictions, they found that these models behaved almost exactly as expected&#8221;<\/p>\n<p>Researchers applied techniques from random graph theory. These models have unexpected behaviors.<\/p>\n<p>We don&#8217;t know if this is how *we* &#8220;understand&#8221; but I suspect it&#8217;s something similar. We are not magical.<\/p>\n<p>\u201cMany people found it a little bit eerie how much GPT-4 was better than GPT-3.5, and that happened within a year. Does that mean in another year we\u2019ll have a similar change of that magnitude? I don\u2019t know. Only OpenAI knows.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Link. &#8220;&#8230; a mathematically provable argument for how and why an LLM can develop so many abilities&#8230; when Arora and his team tested some of its predictions, they found that these models behaved almost exactly as expected&#8221; Researchers applied techniques &hellip; <a href=\"http:\/\/www.kateva.org\/sh\/?p=87523\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[32],"class_list":["post-87523","post","type-post","status-publish","format-standard","hentry","category-share","tag-pinboard"],"_links":{"self":[{"href":"http:\/\/www.kateva.org\/sh\/index.php?rest_route=\/wp\/v2\/posts\/87523","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.kateva.org\/sh\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.kateva.org\/sh\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.kateva.org\/sh\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.kateva.org\/sh\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=87523"}],"version-history":[{"count":1,"href":"http:\/\/www.kateva.org\/sh\/index.php?rest_route=\/wp\/v2\/posts\/87523\/revisions"}],"predecessor-version":[{"id":87524,"href":"http:\/\/www.kateva.org\/sh\/index.php?rest_route=\/wp\/v2\/posts\/87523\/revisions\/87524"}],"wp:attachment":[{"href":"http:\/\/www.kateva.org\/sh\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=87523"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.kateva.org\/sh\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=87523"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.kateva.org\/sh\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=87523"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}