Creating a certain type of contacts within an entity

Hi All, 

Can anyone advice with the following:

Within the contact entity I have an option set where someone can define what type of contact he/she wants to create. In total there are 3 types of contacts (X, Y and Z). The contact entity has a N:1 relationship with 2 entities (A and B). 

What I want to do:

1. When within entity A to be able to create contact type X and Y but not Z

2. When in entity B to be able to create only contact Z

What I have done so far: 

I have added sub-grids within entities A and B and when I am clicking the + button to create a new contact I still have the option to create all type of contacts (X,Y and Z). Where I want for entity A to be able to create X and Y and for entity B only the Z type of contact. 

Thanks