So I recently realized my /var directory was growing but couldn’t find anything using du, then find out about lsof and found the following:
lsof -nP | grep '(deleted)' mongod 23184 23205 mongodb 5w REG 202,1 10765518018 284593 /var/log/mongodb.asd/mongod.log (deleted)
Quite a few lines of those, but I understand they won’t just dissapear until the mongod process is restarted?
Anything I can do about it?
This is under Ubuntu 16.04 and MongoDB 3.6.13.
Looks like somebody deleted the log file while it was used by the mongodb process. Deleting log files is such a manner is not a good idea as the process will keep it locked and that means the file will keep occupying space on the disk. Such deleted files entry in the
lsof will disappear once the process that is locking it is restarted.
The best way to handle logs files is by using proper log rotation by utilities like
logrotate which is available on almost all linux systems.
If you ever face a situation where the log file has grown too big and you want to free up space without restarting the service, the best way would be to truncate the file using –
cat /dev/null > /var/log/my_big_log_file.log or just
echo "" > /var/log/my_big_log_file.log. Just remember that, this is not a proper solution for handling log files and should be used as a temporary fix.