Sanding primer/filler

I have painted numerous tractors using sandblasted parts and then priming with Expoxy primer that lays down great, and then immediately top coating. I have wanted to avoid high build primer and sanding for a whole bunch of reasons and I have been able to do so until now.
I am doing some fenders and panels that require some high build primer / and sanding.
I have laid some filler down and have several coats of primer on and have been trying to sand that down, all by hand. While its going OK it seems that the sand paper gets loaded up awful fast and is relative to how hard and how long I sand without blowing dust away. I have purchased several different grits of good paper from auto body store, and it wasn't cheap. It seems like I can only go a couple of stokes before needing to clean the dust and paper. If I go a little to much I wind up with loaded paper and need to throw it away.
I have let the primer cure for several days and am currently using 180 -220 to knock it down, and I intended to finish with 400.
I thought there should be a better way so i decided to try the 3M Sanding Sponges and 3M Scotchbrite pads it various grades and for me it seems to work much better than paper. The pads seem to last a lot longer than paper and hardly load up at all. Once I am done with the Sponge I go to the Maroon Scotch Brite for a finish surface. This seems to be a better way, but I don't see anyone recommending to do it that way. The pads are more expensive but I am not sure they will be in the long run with the amount of paper I was throwing away.
So the questions is : Am I doing something wrong that makes the Paper load up so quickly? Is there any reason not to use sanding sponges with Scotchbrite pads to finish ?
Thanks in advance
Alan

no reason not to use the scotchbrite,unless you are using a long pad and guide coat. With sandpaper I always give it a slap (imagen clapping hands) every so often.

Could it be to much down pressure?

What are you holding your sandpaper with? Those sponges will take material off and make it smooth but it will end up wavy.

Only way to go is wet sanding. Ya know that black paper that says "wet-or-dry" on it. One sheet torn into 4 and used on a semi rigid block and a water hose with fine sprayer. The paper will clean itself after each change in direction and water will wash it clean so you don't get any grit under it digging deeper scratches which will show up when painted. Only reason to dry sand at all is if you have some sand throughs and need to spot prime them...then just a little dry scotchbrite pad will do. I don't know anyone who dry sands primer surfacer. Complete waste of time and materials.

Some primers just don't sand well. You probably need to change brands. If you are needing a high build it sounds like there are irregularities you need to work out. It would help if you would use two different colors of the same primer and sand with a block instead of a foam pad. The different colors of primer will highlight the high places and let you know where the low places are that need more attention. I've had good luck with Evercoat Featherfil primer.

also if you are using a primer sealer it will not sand well.

Use wet sand paper, keep a bucket of water next to you to dunk in.

The way you stated it,it sound's like you skipped a few steps and started with too fine grit paper. First is bondo leveled with long grater file and block sanded. Next come's primer wet block sanded to reveal high/low spots. Followed by glazeing putty block sanded,then primer block sanded to make sure no high/low spots remain. NOW you can apply two coats of high build folowed by another coat of high build in contrasting color. When you can wet sand last color off without breaking through first two coats it's time for sander/sealer if sealer required by top coat material. You don't need 400 paper unless you need to sand flaws between top coats. Having said all that,I've never nor do I ever plan putting that fine of paint job on a tractor. Frankly I believe a tractor looks pimped with perfect paint but to each his own.