bitbee

← Back to list

Current AI / LLM frontier models have capabilities that surpass human ones in some parts (and I am not saying that LLMs do not have flaws, they have many!), and often people use them for daily advice. Where does this…

Current AI / LLM frontier models have capabilities that surpass human ones in some parts (and I am not saying that LLMs do not have flaws, they have many!), and often people use them for daily advice. Where does this lead to? Where will we land as humans, and where do we really want to land?

If you are in a good mood and don't mind not being so delighted anymore after, you can watch this video where Ray Kurzweil, Elon Musk and others discuss the singularity: YouTube discussion (Kurzweil, Musk, et al.)

A question to ask Ray Kurzweil: Does he really think, if there is something making him 1000x more intelligent, will the other 999 parts really care about this one part that is / was Ray Kurzweil? The future might not turn out as he imagines. I think his view is not only naive, it might be dangerous.

When working with LLMs for coding for two years now, I notice that they've become really powerful tools, that produce code at a depth and speed that makes it impossible for me to review the whole codebase. From helpers to add functions and methods they've grown to tools I am discussing the overall architecture with.

I recently observed myself feeding an answer of an LLM about my current software project as prompt to another one, because I was slightly puzzled by the complexity of its content.

The other LLM answered politely, a very nice discussion spawned where the LLMs were mostly agreeing, discussing some specific points where they did not fully agree in detail.

These were not the absolute frontier LLMs, it was a Chinese open-weight one and a cheaper American one (July 2026).

But moreover, I noticed that my project that had some very specific ideas shifted. I re-started some 7-8 versions because bloat kept creeping in at one point. While trying to keep the code generic and concise, the LLMs silently accumulated functionality in some setup scripts, scaffold scripts, or a god-object that grew out of control. And this despite setting up clear guardrails, directives and architecture guards. In the end I surrendered, abandoned some core ideas, kept the others and now have a software that I do not really "own" anymore. It looks like what I imagined though as a user from the outside.

Anyway, why still write software at all? If I extrapolate the last two years, it might not be needed to write software anymore. Just wait for a while until the LLM does the job without intervening at all. But we do not understand how things work anymore then. We are not deciding anymore about our future. And this process does not only apply to software, and it happens so fast we are not able to comprehend this on a societal level.

Besides all the threats that one could think of that really powerful LLMs pose on us all and this planet, the question we have to solve is: how do we really want to live our lives? We will lose our capability to connect to this unique planet with all the complex life it has evolved, and turn into something silicon-guided that might eventually strip off its hindering biological parts.

The transhumanist version of this offers abundance, wealth and prosperity for everyone. What does that really mean? We don't miss something that we do not know exists, but we want everything we see somewhere else. I recall a TV documentary about a Silicon Valley billionaire, building a house with one hundred rooms. At the end the filmmaker asked him: "Why did you build a house with one hundred rooms?". The answer was "because I could do it!". So what do we need to be happy, and what do we just take "because we can do it"? If you can fly to the moon for 30 bucks, of course people will do it.

Richard Brautigan was one of my favourite authors when I discovered him in the early 90s. His poem "All Watched Over by Machines of Loving Grace" imagines a world where mammals and computers live together in mutually programming harmony — a vision that reads as utopia until you notice it is the machines doing the programming. I always read it as a warning.

Around the same time, in 1991 I had the pleasure of hearing a lecture by Joseph Weizenbaum — the computer scientist who built one of the first chatbots, ELIZA, and then spent his career arguing that what a machine can do is not the same as what it should do. He opened that talk with the words: "Things are becoming too complex! We cannot control them anymore!".

Current LLMs can already empower a lot of people with capabilities they do not have, and this alone would change our society massively even if we would stop LLM progress here, today. The transformation we are taking now should happen in some ten to twenty human generations, to discuss if we really want it and how.

Do we just want to let that happen, without any thought why and how?

This all is of course not new, but it's becoming obvious now. Some more reading on this:

Anmelden