Judging people by where they live is discrimination the same as judging them by their skin color. You say you don’t see color, but you see location.
Go to work, don’t speak to anyone, don’t make any gestures, and don’t have fun. This is the reality of today’s workplace due to workers and management exploiting sexual-harassment and workplace-violence policies.
Instead of criticizing the woman who feeds stray cats, perhaps you could be part of the solution. These cats were turned out to fend for themselves by irresponsible, uncaring people. Support a neuter/spay program.