Your phone should have a setting to let you turn off emergency alerts. This includes severe weather alerts (like tornado warnings) and AMBER alerts (the ones that ask you to keep an eye out for abducted children).
On iPhone, go to Settings, then Notifications, then scroll to the bottom to see government alerts. There are switches to turn off AMBER alerts and “Emergency alerts.”
On most Android phones, go to Settings, and then Sound, then Emergency Alerts.
On Samsung devices, you’ll find options for emergency alerts in your Messages app settings instead.
The whole point of the test is to find out if the system works as designed, so your carrier or your phone’s manufacturer will have to fix the problem.