Taking off the panel is fairly simple:

Pry off the top of the door handle assembly starting at the rear part of the handle. This will reveal two screws - remove them. Unplug the wire harness(es). There are two on the driver's side, and one on the passenger side. Remove the screw from the door release handle. Using a panel popper (or a couple of stiff putty knives, or screw drivers), start at the bottom and work your way up the sides, popping each door clip. They are pretty easy to find - by pulling on the panel a bit, you'll see where it's still attached. Now you can remove the door release assembly, or just leave it attached and push it through. The whole panel should come completely off now. Pull back the plastic around where you need to work, or remove all of it. Be careful not to tear it, as it rips very easily.

I've tried doing this fix, but wasn't very successful, which is why I've resorted to the silicon spray.