Nightmare Difficulty Without Finishing The Game:

You will need to edit the DoomConfig.cfg file, located in the /Data/BASE folder. Make sure you backup the file before making any changes. Open the file with Notepad or other text editor. Look for the "g_nightmare" entry. If you haven't finished the game yet, you will see the number 0. Change it to 1, save, and nightmare mode will be unlocked.