Interesting and provocative question. I suppose that there's no way of objectively knowing if the changes are beneficial or even safe. (Come to think of it, how would we even define beneficial?)

That said, I find your caution perhaps unwarranted. Many other circumstances are known to cause changes in brain-structure, e.g. growing up bilingual, but I think it would be rather unwarranted to warn against childhood language learning on the basis of this.

Personally I am content with the proof of my own experience; the changes I see in my own psyche and mental performance are entirely positive.