Granting Access to Other Parent

I have full custody of my children, whereas the other parent only has access to them during a certain time period. During this time period, I never restrict their access to the other parent. However, my question now is: I have a concern that the other parent is manipulating my children, trying to turn them against me. Knowing that I should never restrict access to the other parent, does that rule/law pertain to the time period outside the court ordered access time period? For example, if my children only see the other parent one weekend a month am I allowed to restrict their access during the week or off-weekend(for example)?

Thank You for your time.