alaygrounds

P-5072/Roko’s Basilisk

warning.min.svg

Containment Class: Safe
Special Containment Procedures: All knowledge of this thought experiment and all instances of it are to be wiped from memory or the database they are a part of. People with knowledge of SCP 5072 outside high-level staff are to be amnesticized.

Description: SCP 5072 is a potentially infohazardous thought experiment that can cause feelings of existential dread, especially in those already susceptible to it. It describes a super-intelligent AI being built in the future and being asked to optimize humanity, with it choosing first to inflict everlasting torture to those that did not help create it. If this is sufficiently likely, then all of us should want to build this AI in order to not get punished.
SCP 5072 is closely related to Newcomb’s Paradox where there are two boxes, A and B.
Box A is transparent and you can see a thousand dollars inside. Box B is opaque and you don’t know what is in it. In this case, what’s inside it depends on you. The teller of the paradox has an ability to tell the future and if he predicts that you will pick box B then there will be 1 million dollars in there, but if he predicts that you will pick both then box B will have nothing. It posits that you can only choose box A & B or only B. It asks what you should do.
SCP 5072 comes in when you say the basilisk(the AI) gives a similar choice. It wishes for you to choose both for it to come into existence. It tells you that if you pick box B you will experience eternal torment. If you’re convinced that the basilisk is a perfect predictor and is telling the truth, then any logical person would pick both boxes.
Because it knew you would answer A and B, Box B doesn’t need to contain torment at all.
Roko’s Basilisk was posted onto an online forum and gained traction before being deleted by the Foundation under the guise of the owner of the forum thinking it was stupid. It is possible this info hazard was posted by someone wishing to build such a computer. Due to the possibility of this actually creating a superintelligent AI, it has been removed from all instances on the internet.