Also, imagine that you were in a group that's being majorly persecuted. You have no way of knowing that the Holocaust is coming of course, just that things are really bad right now.
Think of what immigrating/refugeeing means: you are leaving behind everything/every place you have ever known to go live in a place where you don't speak the language, don't know anybody, don't have any money (because if you own a house you're not going to be able to sell it for anything and even if you do the Nazis will seize the money [they did this constantly, usually claiming "tax debt"- it cost Freud almost $500,000 to leave Austria, which was paid by foreign supporters]). Would you rather risk absolutely certain poverty as a penniless refugee, or hold on and think that things in your country, which has had problems but has ALWAYS BEEN CIVILIZED (the most civilized nation on earth by some counts: German universities were considered the best in the world), will ultimately return to normal? It's not a simple issue.
Obviously if they'd had crystal balls and seen the concentration camps more would have come, but they didn't.