1.8 rc2 layouts dumping widgets - Module already loaded in Unknown on line 0

I am having an issue where all my layouts are purging the widgets from all the regions. I have had this happen twice now. I am not sure exactly when it is occurring as I have not specifically been able to reproduce it. I think it might be happening when I copy a layout.

I am also getting intermittent errors while saving various elements. I hit save and get an “Internal server error”. The items do save…but it is throwing the error. I can’t remember where this has happened specifically. I will try to take better note of it next time.

Thank you for reporting the issue - i’ve not noticed it doing this during any of our testing, but there were some prior issues related to this so it is certainly possible.

I’m not sure we have enough information to actually try and reproduce the issue in this case - but I would recommend upgrading to 1.8.0-rc3 when it is released early next week and then letting us know if it happens again.

I haven’t had the layouts purging issue again. I did however take note of where I was having internal server errors…

Anytime I go to edit a display to change a license and set a default layout when I hit save it throws the error. Not sure what is happening as it actually does save the edit. It just doesn’t close the popup and give the saved alert.

OK, rc3 has just been released and fixes numerous issues (62 to be precise) - it might be worth upgrading to that to see if you still have issues.

I am all updated to rc3. I am still getting some internal server errors. This may be more a server configuration issue on my end.

I checked the error logs and I am getting this:

[Tue Mar 07 22:09:44 2017] [warn] [client] (104)Connection reset by peer: mod_fcgid: error reading data from FastCGI server, referer: https://mydomain.com/display/view
[Tue Mar 07 22:09:44 2017] [error] [client] Premature end of script headers: index.php, referer: https://mydomain.com/display/view

It happens on the “display/view” url as well as the “/layout/designer/” url.

I have looked up some solutions to that error…but anything I changed didn’t seem to help.

Not sure if this is something you have encountered and have any solutions for?

That isn’t an error i’ve encountered before - which environment are you using, Apache?

There seems to be some suggestion that the FastCGI configuration isn’t quite right: http://stackoverflow.com/questions/4336757/500-server-error-premature-end-of-script-headers?answertab=votes#tab-top

Perhaps adjust in the FcgidIOTimeout directive would help.

There are also suggestions that it might be a permissions issue - can you check whether the files are owned by the Apache user? If not you will have to chmod them appropriately so that the Apache user has access. I think this is less likely, as you only get the error on some pages (all routes in 1.8 use the same index.php page)

I have tried that and a variety of the other suggestions I have found on that error and none seem to do the trick. It is definitely not a permission issue. I set everything to apache:psacln and temporarily set permissions to 777 and no resolution.

I am noticing the error only occurs after very specific actions. If I change the default layout of a display I get the error. I do not get it with any other edits to the display. I also get the error if I try to send a change layout via the API. I am also getting the error when I change background on a layout. The actual functions go through and save fine. It is just the returned status that seems to cause the error.

I did notice that when I checked my php version on the server (PHP 5.5.38) I got a warning about the soap module: PHP Warning: Module ‘soap’ already loaded in Unknown on line 0

I got the extra soap initialization turned off though and I am still getting the error so I guess that was unrelated.

It sounds like it happens every time the CMS tries to call XMR to send a push message to the player. This happens over a ZMQ socket - if you have set this up.

We could prove it by disabling the zmq module in your PHP installation (which would therefore disable push messaging). If the error goes away, we know that is the problem.

Can you confirm ZMQ is enabled/disabled on the report fault page.

Ok Dan…

So I disabled zmq and that is definitely where the problem is occurring. Any suggestion as to what may be configured incorrectly?

The short answer is no - the long answer is that i’ve spent quite a lot of time looking for any hint of an answer, but have come up blank.

Can you check your Apache error logs to see if there is a more specific message?

Thanks so much for your help on this…

There is more info being logged in the apache error log when the failure occurs:

Permission denied (src/signaler.cpp:181)
[Fri Mar 10 15:25:46 2017] [error] mod_fcgid: process /var/www/cgi-bin/cgi_wrapper/cgi_wrapper(10868) exit(communication error), get unexpected signal 6
PHP Warning: Module ‘zmq’ already loaded in Unknown on line 0
Permission denied (src/signaler.cpp:181)
[Fri Mar 10 15:25:51 2017] [error] mod_fcgid: process /var/www/cgi-bin/cgi_wrapper/cgi_wrapper(11067) exit(communication error), get unexpected signal 6

It is odd that it is claiming zmq already loaded. It is only initialized once in the php config files and if I comment out that one line zmq gets disabled completely.

Sounds like you have more than one php.ini file being loaded then if you’re sure that soap/zmq aren’t added twice in the one you’re looking at.

Tells you how to get a list from PHP of the files it’s included when looking for it’s configuration. One of those I suspect will have duplicate “extension=” lines to the other file, or you’ll have something like a /etc/php/conf.d directory that includes multiple files to build up the php.ini file, and in there will be duplicate extension lines.

This is one of the reasons we’re strongly suggesting people run 1.8 series in Docker, so that we know the environment the CMS is running in is configured correctly, and specifically to the needs of the CMS.

1 Like

Ok, so after a ridiculous amount of upgrading components and trying various configuration solutions I have finally zeroed in on a solution.

The issue with zmq already being loaded wasn’t as simple as an extra extension line in the php.ini. I searched all the possible config files for duplication. I upgraded php 5.5 and the zeromq module to their latest versions. I then upgraded from Plesk 11.0.9 to Onyx. Finally, this managed to get rid of the errors related to zmq already being loaded. I was hoping that would be the end of the issue but I was still getting the 500 errors…

After a long session of googling I finally found that it was a permissions issue from Selinux that was causing the errors. I still need to figure out exactly what files I need to correct… for right now I just temporarily disabled Selinux.

A couple questions though…

As I had brought up in another thread… When a playlist is edited through the CMS it sets an alert that all displays using the layout that was modified should download the changes next time they check in. If you alter a playlist with the API this change alert does not happen. I had hoped that this was simply because the zmq wasn’t working but that doesn’t seem to be the case. Shouldn’t the displays be getting an alert that content has changed regardless of whether the change came from directly in the CMS or from the API?

I am pleased you are closer to a solution - sorry it has taken you this much time to work it out. It’s useful to have this recorded here in the community, hopefully it will help others. I will change the subject slightly to make it more searchable.

This is by design - notifying displays of a change is the most expensive operation the CMS can do and we therefore want to limit the number of times that happens. The behaviour in the CMS might look like it is immediately notifying, but in fact it is waiting until either; new content is generated inside the preview or the layout status is regenerated.

To mimic this behaviour in the API, you would need to make all of your changes to a particular layouts widgets, and then call layout status.