I felt the first two pretty much lived up to the books (except for some inexplicable dialogue-switching between characters), but POA was wide off the mark and left out important details that GOF then assumed you knew (i.e. Harry referring to Peter Pettigrew as 'Wormtail' - You would only know who he meant if you read the books as POA completely omitted all references to the Marauders except for the map itself.). It's amazing how the same screenplay writer manages some inconsistency between films.
Having said that, I really liked the first three films because they introduced me to the books, but with GOF it was the first time I had watched the film after reading the book and from the complexity of the book, I knew it would be an impossible task to cram all 700 pages into a 2h 30m film when POA couldn't even do it with a 300-page book.
If you view the films as they are - entertainment - then you will enjoy them immensely. If you view them as adaptations of the books, they tend to leave a bitter taste in the mouth afterwarRAB.
Incidentally, I have them all on DVD too.