Might have to live with future in-laws, how bad does it get?

[68whisk]

My boyfriend and I live together in an apartment and we've been saving up for a house. His grandpa offered to sell the house his mom and sister live in for just $40,000. It's a good price considering the house is worth more than that. Here is the catch though, his mom and sister might have to live with us. He feels that he NEEDS to buy it to put a roof over his mom and his sister's heads. His sister is broke, doesn't pay for anything (basically a freeloader) and his mom makes minimum wage so all her life, her dad (my bf's grandpa) had to support her. His mom has a degree but she does not use it, I guess she settled because she's been getting money from the grandpa after her divorce. His sister has a kid so if she gets kicked out, we may not see his niece as much as we used to cause the niece is there every other week. I'm only a gf so I keep my mouth shut but I told him that problems will arise when we all live together, considering the fact that both the good and the bad things come out at home. I've decided that I'm not going to live with them cause I'm not willing to risk my relationship with him and his family. I don't wanna lose my patience with them and vice versa and have to feel awkward whenever I come home. I've also decided that if he forces this on me that I will just leave. Am I being unreasonable? Any suggestions? Should I try to compromise with them?