The threats are becoming really serious.

You would need to consult a family law lawyer to specifically obtain direct advice and guidance as to what power you have to force your son to move with you. The custody order is subject to change, and is intended to ensure the child's best interests are met. As you have sole custody, for the most part, the responsibilities to ensure the child's best interests are your sole responsibility. However, there are also other factors that must be considered. For example, the child's preferences, neighbourhood, routine, involvement in community and school, and the desirability of maximizing contact with his mother, etc. If you were to change the child's circumstances from the move and any of these other factors were impacted, then it may be necessary to revisit the custody and access terms in the order. The threshold for such a review would be a material change in circumstances, and the proposed relocation appears to be something that was not foreseeable, and would materially impact the child's best interests.