I wish I had more information. Is the carry position you are using at home the same as the one you use outside the house? When outside, after wrapping baby, do you stay active? When at home, do you tend to sit down and do sedentary tasks -- like checking your email?
From what you've told me, it doesn't sound like your baby doesn't like the wrap so much as it sounds a little like he just doesn't want to be still. That's pretty common.
While many newborns are happy to sleep or nurse as long as they are touching mom, there are a few who like more motion. If your baby fits this category, you may need to be a little creative in order to get more sedentary tasks accomplished -- at least for the next several months or so.
If you really have something sedentary to do and want to wear him while you do it, wrap him and then walk for a little while, so that he settles back down. THEN try to do sitting-down tasks.
Another issue you may be dealing with is face coverage. Some babies just don't like any cloth touching their faces. There are many wrap carries you can do which protect the back of his neck and half of his head without covering his head or face entirely. He may be much more willing to be worn that way.
I hope this is helpful. If you continue having problems, please let me know.
Tracy Dower