Asking for khula

My husband and me live in the same house which is my parents house and my husbands in laws house. My father kept his account card with his permission to save alot of money. People were jelous of our relationship as husband and wife. We had alot of arguments. Later he started to threaten me to leave me and he also use to mention talaq though it wasnt given. One day he left us and went back to his parents house undercover. After two weeks he decides to come back showing fake guiltiness so I didnt let him in the house. After a month he decided to go back to his parents as I never took him back. He is there and has spent almost all the money we saved for him he cant provide for his wife telling her this is my money, I work, dont touch my card. He threatens me of giving talaq and so forth. I feel I cannot fulfill the religious obligations. Would it be correct for me to ask for khula?