how do I get rid of leaf cutter bees in my garden? There's so many!?

[NJH]

I have just moved house and the garden was not very well looked after at all! There are literally 20-30 leaf cutter bees and their nests/burrows in a very small area of my flower bed. How do I get rid of them? Is there a trap for them or something? I am trying to weed the garden but every time I go out there there are just so many of these bees that it freaks me out. And when I pull weeds it disturbs the soil and there's always a bee in there too. I know they are unlikely to sting but I'm not overkeen on bees and wasps and I'm really not comfortable with them all over the place. There also seems to be a lot of flies around too - I'm guessing my flowerbed has been used as a cat toilet for a while. How do I get rid of these? Hope someone can help. Thanks