The pump can stay on. I think it was maybe just the 1960 models that have the pipe, but I will have to look in the book to make sure. If your pump has just one small line coming out on the right side of it (when facing forward - 5 Star in picture but early M5 used the same pump) it will have the pipe between the tel-o-flo lift and rear end housing. I think it is about a 1/2 diameter and about 1 or 1 1/4" long. If your hydraulic pump has external plumbing with a large line on each side it wont have the pipe.