I think people made some crazy magical assumptions about AI based on scifi that doesn’t apply to real life. Real things to consider and prepare for, but not likely.
Recursive exponential self improvement? Just because something knows how to code doesnt mean it can make the best ultimate next version of itself. Even physical evolution takes millions of years, and it creates mistakes and has setbacks.
We might be seeing logarithmic AI improvement today, like evolution hitting a hill that it can’t cross. We might need trillions more gigabytes of clean training data that isn’t LLM generated to hit the next level, and it might not even be worth it.
I think what the AI developers are doing by constantly promoting ai fear is linking the idea of exponential ai self improvement to it, because their biggest fear right now might be investors realizing that isn’t real, and every dollar they invest is getting less and less back.
Exponential improvement is indeed optimistic - a sigmoid curve (plateauing after a period of increase) is much more plausible, though in the computer programming case I haven’t noticed the plateau yet.
Indeed, throughout nature it’s almost all sigmoids. The trick is that sigmoids look exponential before the inflection point and it’s hard to predict when that inflection point is going to come.
Agreed… I’ve been dabbling in “smart” algorithms for 50 years, the recent (last 8-10 years) progress has been dramatically faster than the previous 40, but each new amazing field: voice transcription, language translation, computer vision object recognition, games mastery, have all rather obviously hit sigmoid-like plateaus. LLM agent software writing has been a slow-burn improvement over the past 18 months - from my perspective it seems like it’s still improving, though that also seems to be a combination of the models getting better, their built in instructions getting better, my local “memory” getting better, and me learning what to challenge it with and what’s unrealistic. A big sign for me is: something I challenged it with 12-14 months ago and got basically nowhere, I tried again last month and it’s made solid progress, delivering a lot of features it couldn’t last year - and those are a lot of features I “gave up on” 5-6 years ago, not because they were impossible, but because they were just too much annoying, time consuming work for the value they deliver to me (personally) - and now the barrier to entry for making those things happen in software is dramatically lower.
I think people made some crazy magical assumptions about AI based on scifi that doesn’t apply to real life. Real things to consider and prepare for, but not likely.
Recursive exponential self improvement? Just because something knows how to code doesnt mean it can make the best ultimate next version of itself. Even physical evolution takes millions of years, and it creates mistakes and has setbacks.
We might be seeing logarithmic AI improvement today, like evolution hitting a hill that it can’t cross. We might need trillions more gigabytes of clean training data that isn’t LLM generated to hit the next level, and it might not even be worth it.
I think what the AI developers are doing by constantly promoting ai fear is linking the idea of exponential ai self improvement to it, because their biggest fear right now might be investors realizing that isn’t real, and every dollar they invest is getting less and less back.
Exponential improvement is indeed optimistic - a sigmoid curve (plateauing after a period of increase) is much more plausible, though in the computer programming case I haven’t noticed the plateau yet.
Indeed, throughout nature it’s almost all sigmoids. The trick is that sigmoids look exponential before the inflection point and it’s hard to predict when that inflection point is going to come.
Agreed… I’ve been dabbling in “smart” algorithms for 50 years, the recent (last 8-10 years) progress has been dramatically faster than the previous 40, but each new amazing field: voice transcription, language translation, computer vision object recognition, games mastery, have all rather obviously hit sigmoid-like plateaus. LLM agent software writing has been a slow-burn improvement over the past 18 months - from my perspective it seems like it’s still improving, though that also seems to be a combination of the models getting better, their built in instructions getting better, my local “memory” getting better, and me learning what to challenge it with and what’s unrealistic. A big sign for me is: something I challenged it with 12-14 months ago and got basically nowhere, I tried again last month and it’s made solid progress, delivering a lot of features it couldn’t last year - and those are a lot of features I “gave up on” 5-6 years ago, not because they were impossible, but because they were just too much annoying, time consuming work for the value they deliver to me (personally) - and now the barrier to entry for making those things happen in software is dramatically lower.