Eureka deregistration fails on graceful shutdown: "Connection pool shut down"
I am experiencing an issue where my Spring Boot microservices fail to deregister from Eureka during a graceful shutdown. Because the deregistration HTTP request fails,
During shutdown, it seems the HTTP connection pool is closed before the DiscoveryClient attempts to send the final DELETE request to the Eureka server, resulting in an IllegalStateException: Connection pool shut down.
Could you help me understand how to configure my applications so that the deregistration completes successfully before the connection pools are destroyed?
Environment
Spring Boot: 3.2.1
Spring Cloud: 2023.0.0 (eureka-client 2.0.2)
Deployment: Docker Swarm (using stop-first update config)
Configuration
Eureka Server (application.yml):
```
yaml
spring:
application:
name: service-registry
cloud:
inetutils:
preferred-networks:
eureka:
server:
eviction-interval-timer-in-ms: 60000
response-cache-update-interval-ms: 10000
client:
register-with-eureka: false
fetch-registry: false
service-url:
defaultZone: http://${eureka.instance.hostname}:${server.port}/eureka/
instance:
hostname: localhost
```
Eureka Client / Microservice (eureka-properties.yml via Config Server):
```
eureka:
client:
service-url:
defaultZone: ${EUREKA_SERVICE_ADDRESS:http://localhost:8761/eureka/}
register-with-eureka: true
fetch-registry: true
instance:
prefer-ip-address: true
instance-id: ${spring.application.name}:${spring.application.instance_id:${random.value}}
spring:
cloud:
inetutils:
preferred-networks:
```
Shutdown Logs
Here are the logs during the shutdown of one of the microservices (order-service). It shows the sequence of events leading up to the failure:
```
13:50:42.773 INFO --- [ionShutdownHook] o.s.c.n.e.s.EurekaServiceRegistry : Unregistering application ORDER-SERVICE with eureka with status DOWN
13:50:42.773 INFO --- [ionShutdownHook] com.netflix.discovery.DiscoveryClient : Saw local status change event StatusChangeEvent [timestamp=1779112242773, current=DOWN, previous=UP]
13:50:42.773 INFO --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient : DiscoveryClient_ORDER-SERVICE/order-service:373b0e4bac...: registering service...
13:50:42.777 INFO --- [nfoReplicator-0] com.netflix.discovery.DiscoveryClient : DiscoveryClient_ORDER-SERVICE/order-service:373b0e4bac... - registration status: 204
13:50:42.781 INFO --- [ionShutdownHook] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
13:50:42.782 INFO --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
13:50:42.786 INFO --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
13:50:42.786 INFO --- [ionShutdownHook] com.netflix.discovery.DiscoveryClient : Shutting down DiscoveryClient ...
13:50:45.787 INFO --- [ionShutdownHook] com.netflix.discovery.DiscoveryClient : Unregistering ...
13:50:45.791 INFO --- [ionShutdownHook] c.n.d.s.t.d.RedirectingEurekaHttpClient : Request execution error. endpoint=DefaultEndpoint{ serviceUrl='http://registry-dev:8761/eureka/} exception=Connection pool shut down stacktrace=java.lang.IllegalStateException: Connection pool shut down
13:50:45.792 WARN --- [ionShutdownHook] c.n.d.s.t.d.RetryableEurekaHttpClient : Request execution failed with message: Connection pool shut down
13:50:45.793 ERROR --- [ionShutdownHook] com.netflix.discovery.DiscoveryClient : DiscoveryClient_ORDER-SERVICE/order-service:373b0e4bac... - de-registration failedCannot execute request on any known server
13:50:45.797 INFO --- [ionShutdownHook] com.netflix.discovery.DiscoveryClient : Completed shut down of DiscoveryClient
```
Is there a specific configuration I am missing to ensure the deregistration happens before the connection pools are closed? Any guidance on how to resolve this would be greatly appreciated.