Just hang a loop of chain under the planter right at the front off the fertilizer tool bar. Let it hang down to just clear the ground when the planter is in the down position. They will break any corn stalks forward so they do not spear the drive chains in the seed transmission. BE glad the chain tightener is coming loose. IF it did not you very well can strip the drive dogs out of the drive clutch.
To fix the tightener slipping in the slot/ Use a longer bolt and put a fender washer on each side of the frame slot. Then use a flange nut on the bolt. I have done this to hundreds of them. In normal use they will not cause any issues.
You need to keep the corn stalks out of the drive chains or they will break some serious things. It happens when you are going the opposite way from the direction the combine did. The stalks are just like spears and go right up into the drive chains. That is what the hanging chain stops. It bends the stalks forward and they then just slide under the planter doing zero damage.