The mitzvah of Bikur Cholim, visiting the sick, applies to visiting anyone who is sick, even if the sick person is wealthy. But it especially applies to a poor patient, because when such a person is sick and no one comes to visit him, it could lead to actual danger to his life. Such a person, in general, does not have adequate food for his illness and sometimes he cannot afford a doctor or medicine. And the distress of seeing that no one is coming to visit him only compounds his situation. Such things weaken the psychological strength of the person and make his illness that much worse. This can cause his death, and the sin for letting that happen is too great to bear.
And although the Yetzer Ha-ra incites the person to not enter the sick person’s home at all, so that he will not know the patient’s difficulties and needs, yet the person who does not visit does not realize that not ignoring the patient’s plight is vital for his own soul and that his punishment for neglecting it is very great. Rather, it is proper to be the kind of person who contemplates the needy (Tehillim 41:2), and his reward for his mitzvah will be that much greater.