Thanks for the comments, guys.
The difference in a game like Fallout is that it is the opposite of Oblivion in all of these areas. First of all you have the ability to actually make choices; secondly, the quests have a clear influence on the world and how people perceive you, and what kinds of quests you are eligible for based on your decisions.
I very much agree.
But I think a more fair comparison to make would be Fallout 1, which was a masterpiece, and Fallout 3, which was merely a good game. Assuming it's fair to compare these two games, Fallout 3 had a fun combat system, but the choices and consequences were mostly trivial. It came down to this: either help one group and get reward A or help another group and get reward B. It was certainly a better game than Oblivion, but it wasn't a masterpiece, for many of the same reasons. Sure, you could blow up Megaton, which was awesome! But one good example of consequence doesn't make up for a general lack of depth, compared to the original game. It didn't make me feel paranoid that if I made the wrong decision there would be unforeseen consequences that actually mattered to the plot. A good example is that there was no way to fail dialogue in Fallout 3, I simply exhausted every dialogue option to get as much information as I wanted. While Fallout 1, in stark contrast, I felt like every choice mattered, including everything I said.
If every dialogue option and every action really mattered in Fallout 3, it could have been great (or greater).
So I guess to summarize, it has to have meaningful choice and consequence, engaging combat, and an interesting atmosphere and story that you can replay from different angles. I at least like the idea Bethesda repeatedly brings up of "radiant AI" but which horribly fails nearly every time, and one day I think we can create truly emergent game play and AI. But for now I really enjoy a deep, well-crafted story more than a big, stale, repetitive playground where I fight the same types of foes over and over again for no real reason.
I agree with this 100%. I do like Bethesda games, but they aren't as deep as, say, Planescape: Torment.
I'll take a compelling setting and choices that matter over a big AAA title any day.