Is it mathematically possible for neural networks to outperform traditional CPUs in simple (or not) calculations?

Imagine a future where neural networks have become so advanced that they are much cheaper to operate. Even though neural networks are complex and usually use a lot of energy, could they ever become more cost-effective for doing simple math, like ‘2+2’, or more complex tasks like integrating a function, compared to traditional algorithms running on a CPU?

This might be hard to imagine for basic arithmetic, but what about for more complicated calculations?

Is there any research that looks into whether neural networks can perform certain calculations more cheaply and quickly than traditional CPU-based algorithms, focusing on tasks that both can accomplish?

P.s I understand that one can never be 100% sure in correctness of neural network performed calculations, ignoring that