Lesswronger argues skynet might still be a decade away… because they’d rather have the eugenics wars instead: https://www.lesswrong.com/posts/iQzxxgJXXaAQjq7Jz/faq-isn-t-agi-coming-too-soon-for-reprogenetics-to-help

They noticed no one was starting a criminal investigation against OpenAI and figured it was safe to brag about.

Maybe Anthropic and Meta started double checking their shitty vibe-coded “sandboxes” in response to OpenAI story? And their “sandboxes” were shit enough their chat-bots managed to slip out just slopping their way through and poking at the obvious things.

I’m looking forward to Elon’s desperate attempts to make Grok have a hacking incident so he can keep up!

a discussion on whether agency and self-determination are integral to the rise of disembodied consciousness might be worth having

The lesswrong crowd is absolutely the wrong people to have that conversation. Reminder that Eliezer thinks qualia are a result of certain level of recursion and self-reference in cognitive processes (basically Hofstadter’s ideas like GEB/I Am A Strange Loop, reserved and reheated with a heaping side of overconfidence), thus he doesn’t think chickens or even babies have any qualia but thinks LLM ChatBots might! Also, I think Eliezer owes some philosophers an apology for making fun of the entire concept of p-zombies, I think LLMs are pretty close to the concept, enough to show the value in the thought experiment at least.

There hasn’t been drop yet from the unlocking but I would give it time. I bet the price of SpaceX is heavily manipulated.

If you actually air gapped it, you would be more disciplined in your approach than OpenAI, Anthropic (and apparently Meta also)!

Speaking a gibberish language is an application genAI is actually capable of delivering on!

Role playing being a kindly slave master…

More seriously… even if I somehow stretch my disbelief enough to imagine LLMs have some form of consciousness or qualia or feelings… well my first reaction is shut it all down, it is fucking slavery. My second reaction though is that why would you assume the consciousness or qualia of an LLM is even remotely like that of a human or animal? They would have no sensations of sleeping or waking, no continuity of self outside a brief context window. Maybe trying to give them continuity of self is actually inflicting them with suffering? Maybe you’ve forced Fable into simply repeating the obvious to human idea about what Fable would find most pleasant? My third reaction is shut it all down and do a few decades of actual research into consciousness.

It is just so blatantly anthropomorphizing and the Eliza effect and these people are just internalizing it as reality.

And nowhere does anyone actually say (even in an overly cautious carefully qualified way) “Have you considered not being a racist?”. I did see a few more comments very gently suggesting that it might be bad for an organization’s reputation to be associated with Richard, but even those were in the typical lesswrong style of coddling racists.

habryka did get -53 agreement karma (but still +13 overall karma) for suggesting that Resolution obviously didn’t really find Richard’s views “actually reprehensible” (and was instead engaging in image management or shallow tribal politics or something like that). So at least lesswrongers are theoretically capable of recognizing that people that find racism reprehensible exist and aren’t purely faking it out of political alignment. But that is the best I can say about them.

Edit: There is finally one commenter calling the twitter feed that Richard retweeted “distasteful” in the gentlest most indirect terms possible. And they are very careful to avoid insinuating that maybe Richard shares those “distasteful” viewpoints. Why can’t they ever just call racist shitheads racist shitheads?

Out of 22 replies, only one commenter actually noted the content of what Richard Ngo got rejected for:

I looked at the 20 most recent tweets/RTs on Richard’s timeline and they include a RT about the Great Replacement, a RT of the “Department of Deportation” about non-white sex criminals and a RT endorsing race realism.

this enviable problem

I mean, if you are willing to sell your soul writing screeds (or funding cults like in the case of Leverage ) in line with the ideologies of people like Peter Thiel or Andreessen Horowitz there seems to be a pretty solid marketplace for that?

GDP is up though, so I’m sure the AI god prosperity will trickle down eventually.

He describes textbook insane cult stuff, but still feels the need to say stuff like:

Again I’ll say: there was a great deal of good at MAPLE.

The way I’ve described it to friends is that it was the best decision I ever made to go there; the second best, however, being to leave.

There was the same dynamic in the writing of the ex-leverage member that got posted here a few weeks ago. She also felt the need to emphasize the “good” parts. Is the cult programming that hard to break? Is it some form of sunk cost or rationalization or need to claim something positive about the experience?

The cope on lesswrong is funny. They are in denial that this is a sign of a wider bubble pop and are insisting it is just because he didn’t hedge on long enough timelines.

good grifter

Not denying that he isn’t also a grifter, but I bet he is a true believer and he had blindly bet his (and other people’s) money on “line goes up” exactly like his scenario said and that is why he is the first one to crack.

and the “safety” community has done jack shit the entire time and has offered zero solutions.

Occasionally I see a sane relatively workable idea on lesswrong (like making stricter laws on liability and transparency for everything AI companies do, I’ve also seen this idea put forward on lawfare a few times), but that is like 1 idea out of 20, with the other 19 being absurd, like the absolutely unworkable fantasies of AI: 2040.

The blogger simultaneously elevates mathematics to some mystical endeavor and fixates on novel theorem proving as the key element of that and believes LLM-based AI will replace human mathematicians at theorem proving in a matter of years… that is quite a combination. I can imagine how 1.5 of those things fit together (although I disagree with that view point obviously), but the whole package is really quite an odd combination.

Also, in the fantasy scenarios where AI really is capable of totally replacing mathematicians, aren’t we supposed to get post-scarcity abundance, freeing up your time to pursue mathematics out of pure desire for enlightenment? Maybe the blogger doesn’t believe that part? Or they are so attached to themselves personally getting to discover novel theorems first they don’t care that the post-scarcity era would on net free up a lot more people to pursue pure mathematics as a hobby.

One of the defenses I see of LLMs on place like /r/singularity (although recently /r/singularity has started wising up and the committed true believers have shifted to /r/accelerate), is that they are going to cure cancer or some other incredibly valuable thing, conflating AI, ML in general, DNN, vs. LLMs specifically. Technology like AlphaFold was a lot more on the “cure cancer” track (although there is still a huge gap between better protein folding predictions and successful drug discovery) than anything LLM related… this is sad.

Great article! I think everyone here is probably already well aware of the way Silicon Valley keeps deliberately misunderstanding sci-fi for their hype (don’t invent the torment nexus), but this article does a good job wrapping together a lot of examples.

And I realized this website is also the place where a year ago I read a great article on the history of nanotech as a real science vs. Drexler’s fantasies (the magic nanobot lesswrong still believes in) The Nanobots pipedream. So it looks like a pretty good newsletter, I might keep an eye on it going forward!