use the right tool !! ... simple things that will always be helpful in most car are those
phillips screw driver (take every cap off) and drive them off
small straight screw driver (to take the cap off)
door trim fork - (softer than pulling by hand..for the panel too lol)
door handle remover - or slide a piece of clothe between handle and plastic spacer, take the clip out(beware it goes far when it unclips

)
when its unclip, and the plastic over the mirror is out, just lift it off the door