I believe that refugee means having to leave the country whereas an internally displaced person (IDP) is within the country.

Regardless, I think that the people that are labeling this as "racism" are confusing it with classism. Although the majority of the people constantly portrayed by the media during this situation are African-American, these were mainly just the people that did not have the means to evacuate. I realize that classism sometimes holds some dimension of racism, I can't imagine the situation would be too different with an equally poor population of a different race.