Jake, I had a CA with exactly the same problem. I fixed in 1995. I used the tractor until 2001 when I sold it. It came up for sale on Ebay in August. I liked the tractor so I bought it back - a second time. Third gear still works.
I give you that background as information to substantiate the following comments. When my CA was new, third gear meshed about 15 to 20%. Normal use will cause the corners to wear off the knobs on the synchro type gears. When the engagement is less than 25%, normal use will eventually round off the knobs so that they pop out of gear. The cause is totally the result of poor tolerance buildup in the manufacture of the shifting fork assembly.
I split my tractor and carefully examined every gear synchro in the transmission. Second and third have exactly the same set up. And the wear on the gears and synchro were virtually identical. However, second worked fine and third popped out. I bought a shifting fork assembly from another CA that the owner said was a good transmission. Using digital calipers and micrometers, I determined that the new fork assembly would push third gear into engagement 0.080" more than my original forks. I installed the new fork assembly and tested the tractor. It stayed in gear. The tractor has been in use since 1995. It still stays in gear.
There are several ways to repair your tractor.
1) Split the tractor and replace the gears and leave the fork assembly as is. Careful use may get years of service. Cost will likely approach $1000 or more for parts. I think this is a poor choice for a tractor that may be worth from $1000 to $2000.
2) Buy a known good fork assembly. I think mine cost $25. I don't know what they cost today. Even if the first purchase turned out bad, the cost is low enough that you could buy another.
3) Build up the surface of the third gear shift fork so that it pushes the slider more into gear. A good welder can do this but you'd need a machinist and a milling machine to resurface the fork. This will work only if there is 0.050 to 0.100" of clearance between the fork and the slider. These parts are seldom worn that badly.
4) Build a new set of detent shafts. This requires a machinist with a ball mill to machine the detents. The new shaft will be identical to the old shaft except that the third gear detent will be 0.080" to 0.100" further from neutral position. One gentleman came to this board for advice. He had a D-14 with no forward gears that worked properly. He made new shift rails based on my suggestion. After installation, two of the four gears were fixed. He felt he'd moved the detents in the wrong direction on the other two. He was delighted - nothing to two is a big change. He didn't post back after his second rebuild so I suspect he got them all to work.
5) Buy new rails and forks. I would only consider this if I were absolutely sure that the sliders and mating gears were serviceable. Plus, I don't know if Allis ever resolved their tolerance problem so new parts may be no better than what you have.
It is very difficult to determine serviceability of your existing sliders and gears with the transmission assembled. Get an IT manual and pull the transmission and gears and inspect them. Compare the parts for second and third. If they are worn the same, focus on the fork assembly. If third is worn considerably more than second, you'll need new gears. If you have a late model CA, you may only need two parts. Older models require 4 to fix this because Allis changed the ratios at sometime in the models life and don't stock the old ratios.
Good luck. My email is open.