Hey there! If you’re delving into the depths of ElasticSearch on a virtual machine, you might have stumbled upon a sneaky little issue - it refuses to start after a reboot! 😱 But worry not, my friend, for I have tamed this beast and I'm here to share the secret sauce.
The Mystery Error
I just installed ElasticSearch 7.10 on a virtual machine this morning, and I noticed that every time I reboot, the ElasticSearch service fails due to a timeout.
And here's the cryptic message that would show up in the log:
Warning: The unit file, source configuration file or drop-ins of elasticsearch.service changed on disk. Run 'systemctl daemon-reload' to reload units.
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
Active: failed (Result: timeout) since Tue 2020-12-08 09:18:55 CST; 24min ago
Docs: https://www.elastic.co
Process: 889 ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet (code=killed, signal=TERM)
Main PID: 889 (code=killed, signal=TERM)
Tasks: 0 (limit: 3570)
CGroup: /system.slice/elasticsearch.service
Dec 08 09:17:40 terrylin systemd[1]: Starting Elasticsearch...
Dec 08 09:18:55 terrylin systemd[1]: elasticsearch.service: Start operation timed out. Terminating.
Dec 08 09:18:55 terrylin systemd[1]: elasticsearch.service: Failed with result 'timeout'.
Dec 08 09:18:55 terrylin systemd[1]: Failed to start Elasticsearch.
Solution
Modify ElasticSearch Configuration
Edit the configuration file of the ElasticSearch service.
$ vi /usr/lib/systemd/system/elasticsearch.service
Now, you’ll see a setting named TimeoutStartSec
which is like ElasticSearch's patience level. By default, it’s set to 90 seconds. It seems ElasticSearch needs a bit more time to put on its shoes. So, let’s be generous and crank it up to 500.
Save the file and then reload the systemd manager configuration.
Reload systemctl
$ systemctl daemon-reload
Check the timeout value for startup.
$ systemctl show elasticsearch | grep ^Timeout
TimeoutStartUSec=8min 20s
TimeoutStopUSec=infinity
The problem is now solved.
Keep in mind that ElasticSearch is a complex tool, and sometimes it just needs a bit of patience and care to get it going. Now go out there and conquer your data!
Comments
Why I only have TimeoutStopSec? I use ES version 7.3..freshly installed.. When I add TimeoutStartSec and reload daemon, the changes doesn’t applied.. Is it because the main systemd service overrides or something?
Thank you very much it worked for me. 🙂
OK but why does it take so long to start, can’t something be done about that?
Thank you so much. I was fighting this all weekend trying to get ES going on a Raspberry Pi.