tag:blogger.com,1999:blog-6642011.post3100999603709299901..comments2023-10-29T10:32:36.914-04:00Comments on Philosophy, et cetera: Bostrom's Superintelligence - Does AI constitute an Existential Risk?Richard Y Chappellhttp://www.blogger.com/profile/16725218276285291235noreply@blogger.comBlogger8125tag:blogger.com,1999:blog-6642011.post-8018667150317565882015-02-17T15:33:41.876-05:002015-02-17T15:33:41.876-05:00Personally I just hope (and believe) that AIs will...Personally I just hope (and believe) that AIs will realise that, if humans are its biological boot system, that keeping us around & functional is in its best long-term interests as there are natural disasters that would more hurt it, than us. The best example of this is solar flares - the Carrington event of 1859 was so powerful that it set fire to telegraph stations, and would be disproportionately more damaging to a machine intelligence - whilst humans would be relatively (relatively) unaffected. Skynet is going to feel pretty stupid if, the day after it kills the last human, a flare fries it's servants the surface & confemns it to existing in a bunker, with no tools to reboot! Whilst this is a flippant riposte to a serious point, it does highlight the fact that an ai, if intelligent & with an eye on maintaining it's survival, would have it's chances vastly enhanced if it cooperates with humans, rather than enslaves / eliminates us.Stanthorpehttps://www.blogger.com/profile/17247715446642376600noreply@blogger.comtag:blogger.com,1999:blog-6642011.post-68526192619205142132014-09-24T17:12:20.458-04:002014-09-24T17:12:20.458-04:00Yeah. In a sense that makes things scarier, since ...Yeah. In a sense that makes things scarier, since it eliminates one way of avoiding the danger (namely, figuring out the right value programming strategy). Danielhttps://www.blogger.com/profile/17911116321628968901noreply@blogger.comtag:blogger.com,1999:blog-6642011.post-24551659518787631262014-09-24T13:37:29.616-04:002014-09-24T13:37:29.616-04:00Huh, yeah, good point! I guess Bostrom is implici...Huh, yeah, good point! I guess Bostrom is implicitly assuming that some kind of old-fashioned programming approach is more likely? The sort of "em"(ulation)-centric future that <a href="http://www.overcomingbias.com/tag/ems" rel="nofollow">Robin Hanson discusses</a> does look very different from the picture Bostrom sketches -- and (as you note) raises very different issues.Richard Y Chappellhttps://www.blogger.com/profile/16725218276285291235noreply@blogger.comtag:blogger.com,1999:blog-6642011.post-48889306978302179792014-09-24T13:33:02.062-04:002014-09-24T13:33:02.062-04:00Thanks, that's helpful!Thanks, that's helpful!Richard Y Chappellhttps://www.blogger.com/profile/16725218276285291235noreply@blogger.comtag:blogger.com,1999:blog-6642011.post-51804485514329295342014-09-24T12:36:11.208-04:002014-09-24T12:36:11.208-04:00A lot of the problems -- perverse instantiation an...A lot of the problems -- perverse instantiation and principal agent problems, for instance -- are standard issues in law and constitutional theory, and a lot of constitutional theory is concerned with addressing them. In checks and balances, for instance, we are 'stunting' and 'tripwiring' different institutions to make them work <i>less</i> efficiently in matters where we foresee serious risks. Enumeration of powers is an attempt to control a government by direct specification, and political theories going back to Plato that insist on the importance of education are using domesticity and indirect normativity. (Plato's actually very interesting in this respect, because the whole point of Plato's Republic is that the constitution of the city is deliberately set up to mirror the constitution of a human person, so in a sense Plato's republic functions like a weirrd artificial intelligence.)<br /><br />The major differences arise, I think, from two sources: (1) With almost all institutions, we are dealing with less-than-existential risks. If government fails, that's bad, but it's short of wiping out all of humanity. (2) The artificial character of an AI introduces some quirks -- e.g., there are fewer complications in setting out to hardwire AIs with various things than trying to do it with human beings and institutions. But both of these mean that a lot of Bostrom's work on this point can be seen as looking at the kind of problems and strategies involved in institutions, in a sort of pure case where usual limits don't apply.Brandonhttps://www.blogger.com/profile/06698839146562734910noreply@blogger.comtag:blogger.com,1999:blog-6642011.post-78507573907180354132014-09-24T11:49:55.488-04:002014-09-24T11:49:55.488-04:00Others know much more about this than I, but I'...Others know much more about this than I, but I'll offer the following thought anyway. It strikes me that there's a tension between (1) talking about what goals an AGI is programmed with, and on the other hand, (2) thinking that whole brain emulation (or something much like it) is one of the most plausible routes to AGI. There's no straightforward sense in which an AGI that comes about via whole brain emulation is "programmed to want" anything. A brain emulating AGI may end up wanting things, but not because its creators intentionally hard coded certain values. That is, with whole brain emulation, you don't have any clear picture of what sort of behavior you're going to get at the end. When google's "Artificial Brain" ended up identifying cat videos, that wasn't because its programmers had told it to look for cat videos. Rather, they'd "told" it, effectively, to look for patterns, and cats were what it found: http://www.wired.com/2012/06/google-x-neural-network/<br /><br />So if that's right, the idea that a big part of avoiding the existential risk here involves figuring out what sort of value-strategy to code into an AGI (e.g., the idea towards the end of your post that rather than hard-coding particular substantive values, maybe we should hard-code a certain value-discovering methodology) strikes me as possibly beside the point. If AGI comes about via what strikes many as the most plausible route, there won't have been any hard-coding of values, or value-discovering methodologies, at any point in the process. Danielhttps://www.blogger.com/profile/17911116321628968901noreply@blogger.comtag:blogger.com,1999:blog-6642011.post-21834790001458147372014-09-24T09:39:31.504-04:002014-09-24T09:39:31.504-04:00Ah, that'd be interesting! Do you have any pa...Ah, that'd be interesting! Do you have any particular examples in mind?Richard Y Chappellhttps://www.blogger.com/profile/16725218276285291235noreply@blogger.comtag:blogger.com,1999:blog-6642011.post-64927095928764326222014-09-24T09:19:30.282-04:002014-09-24T09:19:30.282-04:00I think there's a pretty straightforward argum...I think there's a pretty straightforward argument for taking this kind of discussion seriously, on general grounds independent of one's particular assessment of the possibility of AI itself. The issues discussed by Bostrom tend to be limit-case versions of issues that arise in forming institutions, especially ones that serve a wide range of purposes. Most of the things Bostrom discusses, on both the risk and the prevention side, have lower-level, less efficient efficient analogues in institution-building.Brandonhttps://www.blogger.com/profile/06698839146562734910noreply@blogger.com