The dampener ring is already falling apart and plugging your injection pump. With the check valve out the pieces will return to the tank for a awhile. It still may get plugged up enough to quit running. The internal passages plug and they will quit pumping.
Also it will more than likely drain off over night. You may have to bleed the pump before each use. That is what the check valve keeps from happening.
The kit to repair it is under $50. You have to have some knowledge on how to completely dis-assemble the injection pump. Then clean it all out and reassemble it. I also usually replace the old style governor cage. The old style has the rubber ring that hooks over two rows of rivets. That ring is what is falling apart. The solid steel governor cage does not have that rubber ring. I have been doing that for 25 plus years and never had an issue with the new steel cage.
If you look on line there are getting to be cheaper places to exchange injection pumps on line.
The local repair shops make them perfect again and that costs you $700-800. I have seen the exchange guys in the $350-450 range. I would repair the one you have or exchange one that has been rebuilt. Getting another used pump might just do the same thing in a short while.