If you don't like to carry the larger firearms, there's going to have to a trade off. There are a lot of more compact firearms out there, but you're going to have to sacrifice accuracy and shot capacity.
I made the decision to carry the firearms that I liked best. I am a left handed shooter and decided to go with a Smith & Wesson M&P because of the ambidextrous slide stop and switchable mag release button. I have both full size and compact in 9mm, 40 S&W and now .45 ACP. I like the compacts a lot, but nothing beats the full size versions. I find myself carrying the full size versions more. That means that I had to adapt to carrying them. I don't like inside the waistband, so I had to get a bunch of different cover garments for my outside the waistband holsters. Winter is easy. It's the spring and summer months that pose a problem. I use a lot of 5.11 tactical shirts to get by.
Good luck!