Filling the service holes is a good start. This can be done with thin sheets of mdf. Any of the adhesive deadning materials (dynamatt ect) can be used on the door skin, also the inner skin. How much area you cover is up to you, and how much you want to spend. Put some sticky tape on bits like the door lock tabs and mechanisisms, it can stop them from rattling.
Fabricating some better pods to house the mids can help the sound along also. Someone here threw some diffuser panels behind the mids on their install, it's the first time i've seen them used in a car. Not sure about the benefits with them though being so close to the drivers, and the many possible reflection points in a door, but for the price you pay you might like to use them.