Should I break up with my boyfriend because of this?

I agree with Melora and Julia. If you two have been dating for only 3 weeks and he's upsetting you this much, it's only going to get worse with time. If you're looking for a boyfriend, the best thing to do is be yourself. If they don't like you for who you are or won't respect you for who you are, then they aren't worth your time. There's plenty of good guys out there, so don't think your current boyfriend is the last man you'll ever date. I know it isn't anything profound or radically new, but that's why it's a cliche: it is the truth. Be true to yourself and go with you gut, you'll never be disappointed with yourself, I promise.