Short answer to your question is this. First with a mill you can also do milling work should the need ever arise, but that"s kind of obvious. The biggest difference your going to find is that with a mill your going to have a better rigidity than you"ll find with the majority of drill presses. Too you"ll have the ability to clamp the vice to the bed of the machine and have both an x and y axis of movement. Another thing is your spindle speeds. Most drill presses have step pulleys giving you a limited amount of speed selection. With some mills they will also have step pulleys, but most will also have a high and low gear which basically doubles the number or speeds available. Better yet if you find one that has a variable speed drive, it will typically have cone pulleys and a high/low range which will give you an infinite number of spindle speeds available.
Beyond that there are some drill presses made that offer basically the same features in the way of the spindle speeds. Some that are made nearly as heavy and rigid as a mill, and if you need the two axis movement you can buy a setup to give you that by just clamping it to the drill press"s table.
In the end, the biggest differences for what your wanting to do will all depend on the brand of press or mill you buy. However in general, the most critical difference, if you want to call it that, is the fact a mill is going to be built to tighter tolerances and allow you to do precision mill work that a standard drill press can"t.