Zonie - The pivot shaft on mine fell right out soon as I removed the nut. But the bushings were quite worn so there was nothing to hold the shaft. You probably have good bushings and it's a tight fit for the very top (unworn) end of the pivot shaft to slip thru the upper bushing. Red's suggestion to use an extension to pound on the shaft is a good one - just be sure to protect the relatively soft pivot shaft threads so you mushroom them over. Use a piece of brass bar stock, or steel bar stock and stick a piece of softer metal (like a penny) on the end of it to protect the pivot shaft threads.
Suggest putting a jack, a pile of wood block under something under the pivot shaft flange. It weighs 20+ lb - when it comes free you won't want it landing on your toes!
Good luck! ...Bob M
PS - Did you buy a new sector gear?