It’s a bit of both. Infants have more neural plasticity as others have mentioned, so when using the same learning strategy babies are going to learn better. That said, adults have a lot more knowledge and experience and are able to make connections to a lot more things; so there are methods aimed at teaching adults that are more effective than methods used to teach babies.
I think you’re conflating average and mean. When it comes to income average is typically median, which does include billionaires but wouldn’t skew the data due to their inclusion.