Stop listening on IPs we haven't had for years