It could be the bearing in where the shifter fork is to engage the pto is or possibly the bearing on the end off the pto shaft. You can try to take a piece of garden hose and put one end to your ear and the other end, with PTO engaged, to the side of the case near the trans than try near the where the pto shaft comes out. The sound might be hard to tell but you may hear it louder in one spot than the other. I know that it may look dumb to poeple watching and isn't very HiTech but it does work.
The bearing at the pto shifter rail section will involve a split. It might be possible to drop the pump and try to remove it from there but I have never done it that way so I can't say for sure. The one at the end of the pto shaft is an EZ fix just pull the shaft and replace the bearing.
Also if you've been out brushogging and hit sometime you may have bent the pto shaft which could also cause the bearings to wear faster. If you do take out the pto shaft check it in case it's not straight.
You can check your FO4 manual for the pto parts I talked about.