Milborn Port has nothing to do with Spaniards and port wine - sorry.

Milborn is Old English and is really "Mill-bourne" written that way you can see that it simply means "mill stream". Port is Old English for a town and not necessarily a coastal town. That is why there are so many Newports inland. Milborne Port was a borough and probably evolved from a small settlement by a mill stream into a town and then into a borough.