One is to recess the driver into the baffle, which we already discussed. In your case, you'll need re-enforcement behind the baffle.

The other is to raise the baffle area around the driver up to meet flush with the driver! Cut the driver hole into the baffle and use the full 19 mm to mount the driver (T-nuts recommended). Then use contrasting wood (or other material) to cut a ring that sits flush against the driver edge, framing the driver. Round-over the outer edge of the ring/frame and you have a decorative finished look, and reduce diffraction (if it does in fact apply here).