Some would say that Gemini natives have a curious way of being. Since they can see the positive and negative in everyone, they tend to not take one side or the other in arguments. To you that's not lying, just considering both aspects. But to some signs, it may seem like you're not being loyal.
Your sign is also known for its ability to be the bridge between anyone having a disagreement. You tend to not get involved in office politics, and prefer to not play favorites. If this rubs some the wrong way, you're not about to get involved. That's their stuff.