You're not missing anything. The big o-ring is stuck and you've got to jar the housing loose from the head & rotor assembly. Screw those banjo bolts in a bit and try to wedge a small pry bar between the heads of those bolts and the end of the aluminum housing.
I don't remove all the parts that you did. I leave the back transfer-pump cover on the end of the head & rotor. You can clamp the end of it in a vise (gently) and have the pump upside down. Then by hand, you can twist the aluminun housing back and forth as you pull up.
When you get ready to put the pump together - put that end cover back on the head & rotor first - before putting the housing back on. After torquing the four screws -rotate the rotor by hand and verify nothing inside that transfer pump is binding.