Cannot update "task" entity via organization service?

I am trying to update a task via the organization service as per below:

Entity Task = new Entity("task");
Task.Id = TaskID;
EntityReference AssociatedContact = new EntityReference("contact", ContactID);
Task.Attributes["associatedcontact"] = AssociatedContact;
IOrganizationService service = serviceProxy(OrgName);
service.Update(Task);

However I get this error back:

ActivityPointer With Id = f0a22cf6-f745-e711-80e2-005056ba77b6 Does Not Exist

I can update the task via Dynamics CRM web interface fine, why would the above occur via the API?