The new policy is applied in real time. However, the devices could be caching the IP addresses of the domain names for some time, usually until they expire. This might vary between domain names and between devices. The change in the policy won’t take effect until there is a new DNS request from the device.
In this situation we recommend you clear the DNS cache of your device. How to do that depends on the type of device but can usually be achieved by disconnecting and reconnecting to the network or by rebooting the device.