A good machinist could put bushings in your carburetor, but that would be a pretty big expense.
Coming up with a new throttle shaft may be a problem. Herschel-Adams (I think it was) used to have a kit with the throttle shaft in it, back in the '90s. but I haven't seen a source for that kit for years.
What I suggest is get the carburtor kit for that TSX 97 from NAPA. Their kit has a rubber boot to take the place of that felt seal. I am pretty sure with one of those in there performance will improve. I don't think you can use the boot and the brass retainer in there together, though.