The first instar was the transformation of the idea of a "dungeon" from the castle keep to a place of confinement, typically subterranean in nature2. Once the term had been thus transformed, memetic mutation assigned it to pretty much any underground structure associated with a castle, regardless of what it was actually used for3.
For the next phase, we have to blame Messers Arneson and Gygax. A long, long time ago4 they were running a medieval skirmish wargame in which part of the plot involved breaking into a castle via its "dungeons". This, apparently, was more fun than the rest of the game so they did it again until it became a tradition. Adding in their fantasy skirmish rules and they more or less had a fantasy tunnel fighting wargame …. and this became that fRPG system which has forever5 cemented into the public mind the idea of a dungeon as an underground labyrinth full of mythological creatures and treasure.
The dungeon, as a plot tool, is relatively useful to the GM since it stops the PCs wandering all over the place and allows them to be steered into a series of planned encounters by the simple facts of geography. In most fRPGs, PCs enter dungeons for much the same reasons as a dog chases something that appears to be running away … and all this makes the game a lot easier. Whole fRPG campaigns have been born, lived and died without anything much happening that wasn't in a dungeon. This works until the PCs get to the sort of power level where solid walls aren't that much of an obstacle any more…
A major problem with the extensive use of dungeons is that of verisimilitude. Digging underground tunnels through solid rock is expensive and time consuming and it can be a challenge to explain why the campaign world is riddled with them. Various explanations have been tried including:
- They are in fact the buried ruins of cities from a previous civilisation.
- They are supernatural "wounds in the earth" with an at least partially extradimensional quality.
- They are something similar to pyramids - grandiose structures created at least partially to adsorb surplus resources.
- They are something like bunker complex fortresses - with all those dragons about, a traditional castle isn't much use.
- Given the extensive natural cave systems under most fantasy worlds, they are a specific sort of fortification designed to protect the frontier between these caves and the surface.
- Dungeon building is a common form of insanity amongst powerful wizards.
- (other explanations exist)
Some of these work better than others, especially in the case of more elaborate dungeons with complicated trap rooms, magic lakes of acid and similar wierd things. Ecology is also a factor - the truly old school dungeon gives very little thought to where the various critters get their food, drink or breathing air from - or where their wastes and cooking smoke goes to. This may not matter in some campaigns, in others verismilitude may demand it be sorted out.