You need to remove the interior door panel. There are screws under it that allow you to remove the triangualar cover just inside the mirror; removing this cover gives you acces to the small bolts that hold the mirror on.
If your mirror still doesn't work, let me know and I'll help you trouble shoot to make sure it isn't the switch. If the mirror is indeed busted I've got a drivers side one that I got for free, so I can sell it to you cheap.
Just dawned on me that you ought to try ascertaining if the problem is at the mirror before you take the door panel off -- I have done the right side three times for various reasons and do not find it an enjoyable experience.
The mirror switch is easy enough to remove and disconnect. I would jury rig some connection to your left mirror wires and see if the mirror motors work -- while still installed.
Applying positive and negative poles ( a hot wire and a ground) to two mirror wires at a time should cause the mirror to move; reversing the connection should move it back the other way.
You'd hate to go thru all the work (and possibly break something) and find out your switch was bad.