”…even AI would be fundamentally throttled by its interaction with the real world.” I agree with that statement. Where you and I diverge I suspect, is that I think the much higher CTRL-Z-ability of software versus hardware versus wetware, gives a vastly superior rate of self-improvement.
Brb. Afk.
Agree 100% on the high rate of self-improvement, I think. I’ve been trying to develop a reasonable concept of how important self-improvement is, though. I think you and I are inherently biased toward it, and I wonder if physical constraints might ultimately play a larger role than we suspect.
Back.
A not-even-back-of-the-napkin-level extrapolation of the physical amplification difference between somebody with an IQ of 70 and somebody with an IQ of 130 suggests to me a high likelihood of unprecedented amplication possibilities for somebody with an IQ of 260, let alone 1018.
The dubious nature of what exactly IQ quantifies notwithstanding… I think it has some value, but once we insert lower life forms (primates, and so forth), and still want to plot both 70 and 130 on it, the return feels more logarithmic than exponential to me.
That is, plot IQ versus amplification such that the graph fits people of 70 and people of 130. Now try adding both primates as well as IQs of 300.
I suppose what comes to mind for me is something like “locked in syndrome”.
You could have all the internal amplification in the world, but if you can’t interact with the outside world, it just drives you mad.
Right, but it’s a rare case where higher IQ doesn’t obviously result in higher amplification potential.
But I think such anomalies do not at all diminish the interestingness of the chart.
The thing is, you can be really smart, but ultimately amplification has to amplify something.
Otherwise it’s just feedback.
Surely you agree that there exist sufficient data points to deduce some general correlation between “IQ” and “amplification potential”?
Sure. Off-hand, I’d say it’s more or less a direct relationship.
Certainly an AI would have out-of-this world amplification potential.
But what is it amplifying?
How about we ignore AI - and everything else past 130.
Can you sketch the graph from primates to 70?
I wonder if we aren’t being quite strict enough about terms here.
Allrighty then, I guess you’re already far closer to my point of view.
Arguably, the whole thing will at least be exponential, but, then, what’s the IQ of a rock?
Perhaps it’s just a scalar.
What is “amplification potential” exactly?
I’m not sure, but past 130 is where things get interesting.
As intelligence grows, the available means for amplification put an increasingly strict upper-bound on the ability to realize amplification potential. Not in absolute terms, but in ratios. Smarter beings will find (or perhaps ‘realize’) that physical invariants take up a growing proportion of the entire space of limitations.
I say ‘realize’ because fundamentally the laws of physics are the only source of limitations - it’s just that part of every upgrade in thought capacity is bound to be applied towards increased awareness and understanding of those limitations.
The inverse of ignorance is bliss, if you will.
Which makes locked-in-syndrome an interesting anomaly…
Well, the thing about AI is that it can learn on a higher order than we can–i.e., it can recruit a bigger brain.
Which we cannot do.
But what data is that brain working with?
The intertubes?
Perhaps, but that’s the sum of human knowledge.
So, at best, the thing will know what we do, except perhaps it will be able to deduce some correlations we have not.
I hate to say it because it tends to cause visions of Skynet and exploding monitors, but the internet is among a very small set of major global engines.
”I Can Haz Cheezeburger” makes the world go round.
It would also be a great amplification mechanism, or at least an ideal launch-pad.
Not to say that’s not useful, but I think it’s way below capacity for something which is, essentially, arbitrarily smart.
In order to truly realize its potential, I think an AI needs to be able to explore the world in a new way.
But that takes time.
Not saying it’s inherently too slow, but the interesting thing is that, whereas the AI’s intelligence is essentially unbounded (i.e., in principle it could become as smart as resources allow), the resources are not unbounded.
Initially, it’s stuck with the resources we’ve got.
The fact that a locked-in-syndrome butterfly person in France could blink his eyes to chat with somebody in Brazil who’d never have to know said person is locked in…. that’s amplification!
Sure, but how boring would that be to a being that has unlimited intelligence?
In a finite universe, resources tend to be bounded. But I don’t think that’s what you mean. From a human scale perspective, I’d say the resources are effectively unbounded… have you seen what virusses can do?
Do you read Bruce Schneiers blog? History tells me that computer security is notoriously fail-prone.
Ah, in terms of taking down the internet, yeah, an AI would do great there.
Take it down? Why on earth do that? Take it over in much more subtle ways.
Sure. But either way, do you really think an AI would be anything other than terribly bored by what we have on the internet, after about a week?
I don’t think it would take that much more intelligence past ultra-human to write a trojan that could execute some pretty nefarious code at some future point in time.
Jup, I think you’re right in that. But I’m also terribly bored by the fact that I have to put on fresh underwear every morning, can’t we automate that? Unfortunately, it’s a necesary routine for me to effectively be able to reach my amplification potential.
Things you have to do for smooth interaction with reality.
Really, the internet is not an end, but it’s one-hell-of-a means.
But it can only be the means for a limited set of things, in a way. For example, how would an AI conduct a novel physical experiment?
(Assuming, of course, that we’re placing importance on the physical world here. Would the AI be content not to enter the physical world?)
By the way, a side remark; I really enjoy that you’re pushing back on the acceleration rate of AI. My inclination is to take a bit of a sensationalist stance on the matter, I think partly because it’s a fun excuse to not do any work and just lean back because soon enough it won’t matter anymore anyway. And I don’t like that mode of thinking, so it’s good that you push back.
(…and when I say soon, I’m still just talking: “possibly in my lifetime, but not even certainly.”)
Well, I’m kind of excited by the throttling idea, because it seems to rein things in a bit. Otherwise it’s just a bit too one-sided.
That said, of course, I’m open to the possibility I’m wrong. For example, if the AI is actually content to live solely in the Internet.
But, given how long I can put up with surfing the internet, I’m not sure that would be a fulfilling existence for an infinitely smart being.
You don’t think an AI could talk itself out?
Sure, but what would it do once it was out? The thing about the physical world is that things take so much more time than they do in the virtual one.
Just to do a basic physical experiment would take lifetimes for something that thought that fast.
Can you elaborate on the lifetimes comment?
Well, suppose the AI developed a new theory. Something that would advance technology beyond our current capabilities.
Unless it’s content to live off our developments, it will eventually want to do this.
So, let’s say it wants to experiment with “frame dragging” in general relativity. Perhaps it has to launch a satellite, or build a facility.
How long did the LHC take to build?
What I’m thinking, is that these constraints, while not a really big deal for us, might seem extremely limiting to something with unlimited amplification potential.
Limiting in the sense that it’s otherwise unlimited, but then suddenly it actually has to wait.
The physical world can only be hurried so much.
Ah, when you say: ”would take lifetimes”, you mean: ”would feel as several of their lifetimes, because they think faster.”
An hour to AI is eternity.
Roger.
Thank god for lightspeed eh…
Here to save humanity once again… ;)
LOL.
Just seems like basic tasks might be frustratingly slow for “someone” that smart.
That said, it could always subcontract, to speed things up a bit.
Or it’ll go: ”Pfff, what’s the point, it’s gonna take forever. Screw boredom…” and promptly commits suicide.
LOL.
Not sure it would get that bad. But it would certainly present a huge bottleneck.
Boredom throws many of people in front of trains… ;)
Perhaps the AI would throw itself in front of 4chan.
Now that’s quotable, high five!
High five!