Any trick to getting the sun visor pivot out of the visor?

[mark99]

Any trick to getting the sun visor pivot (the end that attaches to the car) out of the visor?
The drive side slid right out, the passenger turns freely but won't slide out