I am not sure if it would "make or break" the character honestly, but voice actors are important to a certain extent. Everyone's taste is different, and while you think Smaug was done well, another person might hate that voice. I think it's subjective.