In my opinion, there's no point in forcing a child to take up something that they haven't shown an interest in. There's no right age but you can notice a child's talent at a very young age.
Otherwise, as you say, a child may end up detesting something you're forcing onto them.