Ideally, you meet someone who is a great fit, you connect well with them, and are excited to have found someone with whom you can have a wonderful life. I wish this on everyone. However, a significant portion of people will find that whoever they meet, it is a struggle. This could be for some many different reasons, including a whole range of internal issues that may be getting in the way.
Some people have more complex personalities, and there will always be some aspect of their character that does not fit easily. You need to understand that this is not uncommon, and if this applies to you that is the reality. You need to live with it and deal with it. It is entirely your issue, and walking away from yet another date because it is not going easy will not solve anything.