There are several bearings and bushings as well as springs in the governor. Any or all of them may be worn and need replacement. They also often wear out the hole case where the shaft goes over to the carburetor, which might need to be drilled out and a bushing installed to correct it.
Make sure that the flyweights pivot freely and none of the parts are binding. Parts are available from a CaseIH dealer.
There is a governor set up procedure to follow in the service manual. A properly set up governor can make a world of difference in governor performance.