CentOS 7 Non-Docker Server Running out of memory

I am running a VM with 2x3 cores, 8GB of RAM, and 2GB swap; non-docker cms v1.8.11. I have about 40 players in my network, (3 windows, the rest Android).

Setup of my server was done in accordance with this guide: Installation guide of XIBO 1.8 on CentOS 7. I have defaults running server side, though I have made adjustments in the web side of the CMS.

My server will run quite smoothly for about 18-24 hours, then the web console will get locked up due to a DB error. From what I can tell MariaDB freezes up and can’t run because it is out of memory. Rebooting the server corrects the issue. System monitor shows RAM fully utilized as well as my swap file. Looking at my system monitor, it shows a number of PHP processes all running /xibo/bin/xtr.php.

After bootup, the server has 4-6 of these running all at 20MB, give or take. Not an issue, to be expected. My problem comes when the server opens more and more of these processes. When the server runs out of memory, I can see well over a hundred of these tasks all running the same /xibo/bin/xtr.php.

I assume my issue is due to all of these PHP processes, though I imagine MariaDB is eating all the rest of the RAM, though it doesn’t show in resource monitor.

It is becoming quite annoying to deal with this. Anyone else having this issue? Admittedly I am not a Linux superuser by any means. Thanks for your patience and help.

The XTR process should close when the Task has been completed. You will first need to narrow down which Task is causing the issue.

I would recommend selecting the Tasks option in your CMS and Disabling all of the Tasks. You can do this by clicking the down facing arrow at the end of the Task row and selecting Edit from the drop down menu. You will then see a check box called Active, which you will need to uncheck and Save.

Continue to monitor the performance of your Server and gradually activate the Tasks. Make sure to give yourself enough time between activating the Tasks again, so that you can narrow down which Task/Tasks are creating the XTR processes that are not closing.

Once you have this information, please let me know.

Many Thanks.

Thanks Dan.

After doing some testing, it seems that the Widget Sync task seems to be opening XTR processes and not closing them.

Do you have any insight on why this is happening? What can we do to correct this.

What exactly does the Widget Sync task do?

Thank you testing the Tasks and narrowing down the issue to the Widget Sync. This behaviour is quite unusual as multiple XTR processes should not be opened. You will need to monitor this behaviour on your setup to find out why it is taking a long period of time. I would recommend checking the logs for any errors that may help answer this question.

The Widget Sync Task is used to check for any Widgets that are active on any Display you have connected to your CMS. It will then update the Widgets Cache if required.

Many Thanks.