SQL Service Broker 2012: the connection was closed by the remote end, or an error occurred while receiving data: '64(The specified network name is no longer available.)'

Anyone can help with the below issue please? Much appreciated.

1. We have about 2k+ messages in sys.transmission_queue
2. Telnet to the ports 4022 is working fine.
3. Network connectivity has been ruled out.
4. The firewalls are OFF.
5. We also explicitly provided the permissions to the service account on Server A and Server B to the Service broker end points.GRANTCONNECTON ENDPOINT <broker> <domain\serviceaccount>
6. Currently for troubleshooting purposes, the DR node is also out of the Availability Group, which means that we right now have only one replica the server is now a traditional cluster.

Important thing to note is when a SQL Server service is restarted, all the messages in the sys.transmission queue is cleared immediately. After about 30-40 minutes, the errors are continued to be seen with the below

Theconnectionwas closedbytheremoteend,oran erroroccurredwhilereceivingdata: '64(The specified network name is no longer available.)'