The term castle certainly gets used here and there for landscape features and it could be that the shooting house or whatever itn is got named for a local feature, e.g. Red Castle House - but equally ironic names have been applied to such lowly structures before as well - and if the structure has been in a ruinious state that too could have led to the 'nicknaming' of it a 'castle'. I saw brief mention to another local building also named a castle and of even more humble dimensions so perhaps it's something of a local custom to name things this way.
Naming landscape features after buildings, implements etc is of course widespread - note the number of cauldrons (in English, Welsh and Gaelic) across our hills; or 'needles'. There might even be a 'Can Hill' ? Dun Caan on Raasay has a number of explanations given for it's name - among which is that the Norse named it 'can' from their word meaning just that, a can - not that far out when you consider the shape.
(In case anyone feels the urge to correct my interpretation of 'dun' as hill - yeh I know it means fort but is also used for fort-like hills and indeed may have originated as a meaning for hill - then applied to hill-forts).
I've long been interested in place-names for what they can tell us about the past - but with places like your castle or the smelting mill above Grinton in Swaledale (went there yesterday) you don't need the names to conjour up history it's standing there in front of you - fascinating stuff, the uses of our hills, whether it's shooting huts, prison 'stop-overs', smelting mills or ironstone works they litter the hills. How many of us realise that Tan Hill had a working coal mine in the 1930s for instance ?