I have the exact same situation going on, same size trailer and all. I stopped chasing the problem, and just used the trailer. Mine sat for a few years before I bought it, and after using it a few times, son of a gun, they work acceptably. Mine will lock while empty on gravel, but not on pavement. They didn't even work that well when I bought it, or on the first few short hauls around the neighborhood. I can, however, feel the pull of the trailer braking while empty. I'd say I have 75% brake function on my trailer.
So, Has yours sat a while? Maybe it just needs the rust polished up on the inside of the drums and the glaze knocked off of the drums and shoes. Just Sharing my experience. -Andy