[Dunx] The Scottish Pittsburghers of the late 19th Century (notably Andrew Carnegie) decided to "Scottify" the city by adding the trailing "h". It looks nicer to me, and it differentiates us from Pittsburg, Kansas.
And then of course there's Loogabarooga! Bruce. Ah, bringing us to the ever-irritating River Nene. We in the valley always said "Nenn", but then I heard Stephen Fry calling it "Neen" so now I don't know what to think!
(rab) Mousehole rhymes with tousle, as in hair, but I don't know about the other two you mention. I do know that Fowey is Foy, more or less. But I'm fascinated by Probus. Sounds disgusting. (ZK) If you called it Nenn, then that's it, despite the estimable Mr Fry. Another variation on the burg, burgh, borough ending is of course Middlesbrough.