Disable Automatic Email Tracking?

I'm trying to find out what settings I can change to stop the following:

We have a help desk email address setup that our clients send in support requests to. I then have cases automatically created from these emails. Any email correspondence between the client and my company is then tracked in the case, this is exactly what I want. The issue I am having is that if that email chain is commented on internally between to of MY employees, this is also tracked in the case. I would like to not have internal emails tracked. Is this possible?

I know when the issue started and that is what I moved from the Outlook synchronization method to Server-Side Sync. We did this so we can use the new feature of the Outlook app where Server-side sync is required.

I can provide more details is my description is not clear.

Thanks is advance for any assistance.

Ryan