Compression ratio vs. horsepower?

The down side is that the loads on the piston and cylinder walls and eventually the rods and crankshaft are greatly increased. The high temperatures mean that you have to have better materials and/or more cooling of the piston crown. Since the usual piston crown coolant is the lube oil, higher temperatures mean need a bigger pump to provide more coolant and more coolant means, your parasitic losses increase( takes HP to pump the oil around). The starter size increases because it takes more HP to crank the engine and thus the battery capacity needs to be increased. As you can see, the gains in output power don't come free and the engine manufacturers have to trade all these things off and be able to produce the engine at a price for them to make a profit and still make the tractor price affordable. It's a real balancing act and that's why you see variance in what operating compression ratio are used on various engines.
If you have some proprietary knowledge that gives you an edge over your competitor, you might be able to for example, increase the CR and use lower cost materials and have a more competitive price on your engine. I believe the folks at Cummins have some high temperature piston materials that are proprietary to them that allow them to make a lot of power out of the relatively low deplacement engine they put in the Dodge p/u's.
Another reason is that some diesel engines are turbocharged so they may run with a lower compression ratio and use the blower to increase temperature and pressure in the engine cylinder. I don't think there is a simple answer to your question, but I've tried to offer at least a few reasons for the CR differences between different engine manufacturers.