should my toddler be talking more?

[van girl]

My son is almost 23 months old. He reached all his earlier talking milestones as a younger baby with no problem (babbling, saying simple words, making a variety of sounds, etc). If I count up all the words he knows and says clearly and appropriately, it's probably about 20-25. He also makes lots of sounds (car sounds, animal noises, sounds that letters make). But, for the last several months I have not noticed much of an increase in his vocabulary, and he actually has stopped saying many words that he used to say all the time. There is no doubt in my mind that he understands almost everything. He loves books and follows along/points to things in the book. He understands simple instructions and used to follow them happily, yet now that he is getting older he doesn't always follow them. He's gotten very quiet lately, especially around anyone other than me (his mother). With me he can be very interactive and sweet and is most likely to use some (very limited) language. Around other people (lately - not when he was younger) he doesn't pay much attention to them, at least not for very long. I've been around other kids his age, some even a little younger, and see that sometimes they are talking more, learning new words every day, and even combining words for simple sentences. My son really is not doing that yet. I would say that compared to other kids, he may be advanced in his motor and analytical skills (building blocks, puzzles, figuring out how a toy works, handling objects, climbing/balancing, etc). But I'm starting to worry that his speech is falling behind. Has anyone else had a similar experience? Should I be concerned?