Can I use the Listener name with Always on and Event Notification from a replica ?

I have a question about External Activation and AlwayOn Service broker databases.

I have

Listener

SSBS-1 (Primary)
SSBS-2

I send messages to the Listener which cause Event Notification from the Primary. 

My problem is we don't want to couple our EAService.config to the names of the replicas.

  <NotificationServiceList>
    <NotificationService name="ExternalActivatorService" id="100" enabled="true">
      <Description>Notification service</Description>
      <ConnectionString>
        <!-- All connection string parameters except User Id and Password should be specificed here -->
        <Unencrypted>server=Listener;database=ServiceBroker;Application Name=External Activator;Integrated Security=true;</Unencrypted>
      </ConnectionString>
    </NotificationService>
  </NotificationServiceList>

<ApplicationServiceList>
    <ApplicationService name="myMessageApp" enabled="true">
      <OnNotification>
        <ServerName>SSBS-1</ServerName> We want to use Listener here
        <DatabaseName>ServiceBrokerDatabase</DatabaseName>
        <SchemaName>dbo</SchemaName>
        <QueueName>QueueName</QueueName>
      </OnNotification>
      <LaunchInfo>
        <ImagePath>C:\Program Files (x86)\Location\Some.exe</ImagePath>
        <CmdLineArgs>QueueName</CmdLineArgs>
        <WorkDir>C:\Program Files (x86)\Location\Actisure Services</WorkDir>
      </LaunchInfo>
      <Concurrency min="1" max="4" />
    </ApplicationService> Repeat for SSBS-2
  </ApplicationServiceList>

If we use the Listener as the <ServerName> in EATrace.log we get the following error:-

2016-11-23 14:38:08    EXCEPTION    ERROR = 32, No enabled application monitor is on behalf of queue SSBS-1.ServiceBroker.dbo.QueueName.

Any ideas how this can get the replicas to publish notification so we only need external activator to monitor the Listener?