Superintelligence Quotes
Before the prospect of an intelligence explosion, we humans are like small children playing with a bomb. Such is the mismatch between the power of our plaything and the immaturity of our conduct. Superintelligence is a challenge for which we are not ready now and will not be ready for a long time. We have little idea when the detonation will occur, though if we hold the device to our ear we can hear a faint ticking sound.
Nick Bostrom
I know it's tedious, but you aren't SL4 until you appreciate the depth of human irrationality. Yes rationality is there, but at a level that just barely shows up when measured with precise instruments. This is important with respect to non-human intelligences, because super-rationality, almost as much as superintelligence, is potentially overwhelming and because intelligence bootstraping moves a system towards rationality. Appreciate how far humans are from rational and you appreciate how utterly transformed, and essentially recreated, they would be by haphazard bootstrapping. Appreciate how formidable rationality is and you see why a highly rational infrahuman GAI would still be a massive existential threat.
Michael Vassar
