Using a small cap will lower the amplitude, but the bass response will also suffer. Wayne has a nice idea but if the circuit is too complex or no tools, one could use a more time consuming method of turn on the preamp first, then after some period of time, turn on the amp. When turning off, reverse this order.

Take care Hurdy and hope this helps.
Steve