There is a difference between giving up and knowing when you've had enough. It is definitely okay to remove yourself from situations that bring you down.