Are AI systems plagiarizers?

I am definitely against OpenAI (or any other entity) using AI (or any other technology) to concentrate power just for themselves and I think that they should empower everyone as much as possible instead.

But personally I don't really like this currently popular narrative of AI systems "being just plagiarizers". I think of AI systems as partial memorizers and partial generalizers, depending the exact details (from mathematical deep learning theory), which is what humans do similarly, but still with differences that we're slowly but surely mapping out.

I feel like this narrative extremely downplays what the current AI systems are already capable of, like for example if FunSearch only "plagiarized the training data" and didn't have at least some sort of generalization power (which all deep learning systems do have, otherwise they would never generalize to unseen datasets at all, see bias and variance trade off in statistical learning theory to see the trade off between memorization and generalization), then it wouldn't help to find new result in mathematics. https://deepmind.google/discover/blog/funsearch-making-new-discoveries-in-mathematical-sciences-using-large-language-models/ And its not bruteforce as well because you have insane combinatorial explosion if you try just bruteforce.

And we wouldn't see all these abstract features and circuits emerging in LLMs with scale like in Claude or other deep learning systems, if they only memorized the training data without generalization. https://www.anthropic.com/research/mapping-mind-language-model https://x.com/tegmark/status/1851288315867041903?t=eB9Ft7hF9ocV9s-w3s-O1w&s=19 https://arxiv.org/abs/2410.19750 https://distill.pub/2020/circuits/zoom-in/

Or AlphaZero and similar chess machine learning systems wouldn't be better than all humans in chess. Or AlphaFold wouldn't help with pushing state of the art in protein folding. And so on. They are all deep learning systems.

The new reasoning models go even beyond, using reinforcement learning to enforce even more generalization, and we're starting to reverse engineer the mechanisms behind that. https://arxiv.org/abs/2501.17161 https://x.com/_philschmid/status/1884983965112828051
This memorization and generalization trade off and how generalization happens in the first place from abstracting individual memorized units is also studied in human learning. There are many differences between human and machine learning, but also many similarities. https://pmc.ncbi.nlm.nih.gov/articles/PMC7613724/ https://arxiv.org/abs/2205.10343
https://www.lesswrong.com/s/mqwA5FcL6SrHEQzox/p/fovfuFdpuEwQzJu2w