What should i do?please help regarding custody of my son?

Hello to all . I have this problem.My son is 7 months old me and his father are no longer together i was 17 when i had my son he was 18.now we are 18 and 19.i live in the state of Connecticut i have recently decided that i want to call Greenville South Carolina my home and return to Connecticut periodically. OK so the problem is he doesn't want me to take my son and move down there which is understandable but he is in no way shape or form to care for a child he can't even take care of his self.he has no job no regular income no nothing.all he cares about is drinking smoking and hanging with his friends you would think at 19 he would be smarter than that.monday to wednesday i allow his mother to take my son to there house to visit she is the one that stays there and cares for him for those few days he is there.today is monday as you all may know and i just received a message from the father on facebook insinuating that he is not gonna return my son to me because i wanna move away i am very scared right now what actions should i take??please help