Some people would say that it depends on what I mean by "I' and "You" and "Know'. That would be a long exposition, including contradictory and often controversial theories and that is not my problem. If you wish to explore that problem you might start with, "Conversations on Consciousness" by Susan Blackmore.
The problem I wish to explore is the problem of otherness, and in order to do so I wish to start with my own simplistic view of you and me. I begin by accepting that I exist to myself as a bundle of sensations and experiences contained in a body, and operated by a brain, and that I think of all this together as "me". All this is what I mean when I say, "I'. Equally I accept that you exist to yourself as your own bundle of sensations and experiences, operated by your own brain and that you call all this together "me" and "I" also. I accept that I view the world through my eyes and perceptions and that you view the world through your eyes and perceptions. This give rise to my problem : how much can I ever know about another person, even if that other person is my dearest intimate? A second problem might be that on some level it is easier to know enough about a person with whom one is not intimate.
Take the sentence "I love you". It is so much easier to know what is meant by "I love your hat." Even if both statements are true in so far as the utterer believes them, in the second case the emotion is likely to be on the level of admiration or envy and would be recognised as such by both parties. In the first case the emotion might be desire, tenderness, empathy, need, possessiveness or affection, or all or none of these.
In our language we accept conventions of meaning. We understand the extent of dislike when someone says, "I loathe that person". We understand also that loathing involves finding the other person repellent in some way. If someone says, "I hate that person" we accept that a passion is felt, the nature of which we cannot know, and we cannot know whether this is a momentary passion towards someone more generally liked.
Trouble arises when we bring our own perceptions to what other people are saying.
I might not even pay attention to what you are saying if I do not like your clothes or your accent, your car or the company you keep. I might pre-judge your opinions negatively because of the tone of your voice, your facial expression, your gender, your manners or your political affiliation.
"Charming" and "smarmy" might be used by different people to describe the same person, while those who know him more intimately might describe him as generous or mean, affectionate or reserved, hard working or lazy, intelligent or stupid, calm or tempestuous, thoughtful or shallow, clever or cunning, impetuous or cautious, principled or stubborn.
The psychologist George Kelly claimed that we "construct" a person from our observations of how much they display the qualities that matter to us, both negative and positive. If I value honesty I will be looking for it in the people I see and I might use the flimsiest of evidence to support my theory about their honesty or lack of it.
It is almost as if we are doomed to misunderstand or misread the people we meet. This is only a problem when it matters.
If I feel disappointed or let down by a friend it is most likely my perception of friendship that has let me down. My mother used to say, "Expect nothing and you will be surprised by what you get". This leads me to think that it is more important to be a friend than to lay expectations of friendship upon others.
Perhaps it is expectations that are the real problem. If I do not expect you to think like me, to value what I value, to give what I am prepared to give, to mean what I mean, then I stand a chance of getting to know something of what you do think, what you do value, what you are prepared to give and what you mean by what you say.
When it comes to love and friendship I have a better chance of knowing another person when I listen to what they think and feel and believe, and to their story about how how it is in their world.
David Hume tells us that we can know enough about another person by how they behave in aspects of family, business, and society. He tells us that we can think of him as a good man if he provides for his family, is kind to his servants, deals fairly in business and is affable with his neighbours.
That is not enough for me. If I am to know you at all I must meet you on a level of understanding not only who are but also what matters to you. It requires an open mind and an open heart on both our parts. If you deceive me then "It is not me, it is you." If I fail to understand you then, "It is not you, it is me."