The easiest way is to launch Notepad first, and then use the File > Open command to open each file (you may have to switch to "all files" in the file type field in the file selection dialog). I'm guessing you're double-clicking on them. Since the file extension (.htm and/or .html) is mapped to Internet Explorer, that's the program that opens them by default.
You can change this behavior by clicking on one of these files and then right-clicking and choosing Notepad as the application to use by default. If you do this, though, remember to change the association back to Internet Explorer, or you'll then have to manually open web pages by select File > Open in Internet Explorer...
