Verifiy all completed. See screenshot below. I am concerned about the X next to Play Local Video. I found the item int the PHP ini file, and it WAS commented out so I removed the comment character and restarted the webserver and the client, but there is still no discernable change. Is there something specific I should look for in the WAMP logs?
…long shot… run “chdsk /f c:” from a admin command prompt and restart. It may be that the security descriptors on the folder need to be repaired.
You can try that, as for logs, any errors permissions related or other might help point us in the right direction.
That’s fine, the checkmark in ‘Library media’ column there is only for modules that needs files uploaded directly to the Xibo CMS library - ie images, videos etc.
Local videos are not uploaded to the library (it can be, for example, path to the video file on player’s storage, RTSP link etc), that’s why it has X there.
I will have to try the chkdsk command later today. Fingers crossed
The snippet below is from my Apache access.log. The entries are from the client IP that I have been using during this testing process. If I am reading it correctly, it appears as though the webserver thinks that the files were downloaded. Otherwise I do not see anything that appears to be a blatant error.
10.200.240.110 - - [08/Aug/2016:14:11:49 -0400] “POST /xibo/xmds.php?v=4 HTTP/1.1” 200 3145
10.200.240.110 - - [08/Aug/2016:14:11:50 -0400] “GET /xibo/xmds.php?file=c0b99986d2c78ee48e2e2cf2bfe61191 HTTP/1.1” 200 -
10.200.240.110 - - [08/Aug/2016:14:11:50 -0400] “GET /xibo/xmds.php?file=e54ddc9ebe142192a58fe552265015d1 HTTP/1.1” 200 -
10.200.240.110 - - [08/Aug/2016:14:11:50 -0400] “GET /xibo/xmds.php?file=13226ba631f0408a7e72d5dc95c96e13 HTTP/1.1” 200 -
10.200.240.110 - - [08/Aug/2016:14:11:50 -0400] “GET /xibo/xmds.php?file=d2ed70c35dd112ade90a482bb4ca206f HTTP/1.1” 200 -
10.200.240.110 - - [08/Aug/2016:14:11:50 -0400] “GET /xibo/xmds.php?file=570ee81dfe9e2bca12f05116e214d5a3 HTTP/1.1” 200 -
10.200.240.110 - - [08/Aug/2016:14:11:50 -0400] “GET /xibo/xmds.php?file=b7cd95572d76260973658e8de25910eb HTTP/1.1” 200 -
10.200.240.110 - - [08/Aug/2016:14:11:51 -0400] “GET /xibo/xmds.php?file=5c1f7ccc7711bc34514b25cf09452f3d HTTP/1.1” 200 -
10.200.240.110 - - [08/Aug/2016:14:11:51 -0400] “GET /xibo/xmds.php?file=d425c45dad2a25aaecec1ec53aba7a4c HTTP/1.1” 200 -
10.200.240.110 - - [08/Aug/2016:14:11:50 -0400] “GET /xibo/xmds.php?file=3fd53ff846a1b881ffd7c9aada02c4d3 HTTP/1.1” 200 -
10.200.240.110 - - [08/Aug/2016:14:11:51 -0400] “GET /xibo/xmds.php?file=009f97236a14acc90b7b46e1491359e8 HTTP/1.1” 200 -
10.200.240.110 - - [08/Aug/2016:14:11:51 -0400] “GET /xibo/xmds.php?file=421679890512424433cfe39b35217395 HTTP/1.1” 200 -
10.200.240.110 - - [08/Aug/2016:14:11:51 -0400] “POST /xibo/xmds.php?v=4 HTTP/1.1” 200 518
10.200.240.110 - - [08/Aug/2016:14:11:51 -0400] “POST /xibo/xmds.php?v=4 HTTP/1.1” 200 514
The apache_error.log shows nothing of note either. (see below)
[Mon Aug 08 11:21:04.927634 2016] [mpm_winnt:notice] [pid 3180:tid 332] AH00422: Parent: Received shutdown signal – Shutting down the server.
[Mon Aug 08 11:21:06.928415 2016] [mpm_winnt:notice] [pid 6368:tid 160] AH00364: Child: All worker threads have exited.
[Mon Aug 08 11:21:07.053463 2016] [mpm_winnt:notice] [pid 3180:tid 332] AH00430: Parent: Child process 6368 exited successfully.
[Mon Aug 08 11:21:11.258229 2016] [auth_digest:notice] [pid 6452:tid 332] AH01757: generating secret for digest authentication …
[Mon Aug 08 11:21:11.602113 2016] [mpm_winnt:notice] [pid 6452:tid 332] AH00455: Apache/2.4.17 (Win64) PHP/5.6.16 configured – resuming normal operations
[Mon Aug 08 11:21:11.602113 2016] [mpm_winnt:notice] [pid 6452:tid 332] AH00456: Apache Lounge VC14 Server built: Oct 11 2015 11:49:07
[Mon Aug 08 11:21:11.602113 2016] [core:notice] [pid 6452:tid 332] AH00094: Command line: ‘C:\wamp64\bin\apache\apache2.4.17\bin\httpd.exe -d C:/wamp64/bin/apache/apache2.4.17’
[Mon Aug 08 11:21:11.617745 2016] [mpm_winnt:notice] [pid 6452:tid 332] AH00418: Parent: Created child process 6372
[Mon Aug 08 11:21:12.336775 2016] [auth_digest:notice] [pid 6372:tid 160] AH01757: generating secret for digest authentication …
[Mon Aug 08 11:21:12.461824 2016] [mpm_winnt:notice] [pid 6372:tid 160] AH00354: Child: Starting 64 worker threads.
[Mon Aug 08 11:22:08.983882 2016] [mpm_winnt:notice] [pid 6452:tid 332] AH00422: Parent: Received shutdown signal – Shutting down the server.
[Mon Aug 08 11:22:15.158166 2016] [mpm_winnt:notice] [pid 6372:tid 160] AH00364: Child: All worker threads have exited.
[Mon Aug 08 11:22:15.236322 2016] [mpm_winnt:notice] [pid 6452:tid 332] AH00430: Parent: Child process 6372 exited successfully.
[Mon Aug 08 11:22:37.463746 2016] [auth_digest:notice] [pid 6668:tid 332] AH01757: generating secret for digest authentication …
[Mon Aug 08 11:22:38.088990 2016] [mpm_winnt:notice] [pid 6668:tid 332] AH00455: Apache/2.4.17 (Win64) PHP/5.6.16 configured – resuming normal operations
[Mon Aug 08 11:22:38.088990 2016] [mpm_winnt:notice] [pid 6668:tid 332] AH00456: Apache Lounge VC14 Server built: Oct 11 2015 11:49:07
[Mon Aug 08 11:22:38.088990 2016] [core:notice] [pid 6668:tid 332] AH00094: Command line: ‘C:\wamp64\bin\apache\apache2.4.17\bin\httpd.exe -d C:/wamp64/bin/apache/apache2.4.17’
[Mon Aug 08 11:22:38.104621 2016] [mpm_winnt:notice] [pid 6668:tid 332] AH00418: Parent: Created child process 6568
[Mon Aug 08 11:22:39.026856 2016] [auth_digest:notice] [pid 6568:tid 160] AH01757: generating secret for digest authentication …
[Mon Aug 08 11:22:39.245691 2016] [mpm_winnt:notice] [pid 6568:tid 160] AH00354: Child: Starting 64 worker threads.
I came across this error in the php_error.log, My php.ini has the always_populate_raw_data = -1 but it seems that I’m missing a secondary setting:
[09-Aug-2016 10:19:14 America/New_York] PHP Deprecated: Automatically populating $HTTP_RAW_POST_DATA is deprecated and will be removed in a future version. To avoid this warning set ‘always_populate_raw_post_data’ to ‘-1’ in php.ini and use the php://input stream instead. in Unknown on line 0
If I am missing the mark, please give me more explicit instructions for the information. Thanks again for all your help.
Will always be in php.ini the trick is to make sure it’s not commented out (no ; before that line) and will be working fine.
Any luck with permissions?
I ran the chkdsk and restarted the server before I left last night. I just checked the client and restarted it, but it still is not completely downloading the required files.
Could there be some sort of network issue with lost packets? Or a through put problem?
I do not think so. I use RDP to connect to the displays remotely, and I can use SMB to transfer the files manually to the clients. In fact, the clients DO recieve the layout xml files from the CMS, it is only the media files that attempt to copy (read: create a 0 byte file) but do not download. I would think that there would be other anomolous networking issues if the network were a factor.
I am stumped… only thing I can think of is to upload one of your files and let someone else see if it works on their system. But I am almost certian it would work fine.
The symptom really reminds me of a firewall striping the data from a jpg file. I have seen watchguard firewalls do that.
Well now that the school season has started up sucessfully I can revisit this issue… If I were more WAMP savvy I think I may have solved this problem already. I agree with you it does appear to be stripping the data, but only from the library directory. I feel as though that WAMP must have permission to the folder becuase uploading to the CMS sends complete files. But when the client attempts to GET the files, nothing is transmitted. Could the following be in any way related?
; Whether PHP will read the POST data.
; This option is enabled by default.
; Most likely, you won’t want to disable this option globally. It causes $_POST
; and $_FILES to always be empty; the only way you will be able to read the
; POST data will be through the php://input stream wrapper. This can be useful
; to proxy requests or to process the POST data in a memory efficient fashion.
; http://php.net/enable-post-data-reading
;enable_post_data_reading = Off
Again, thanks for all your input.
In attempting to decypher where the failure point is in my problem, I found the following in my access.log:
10.200.241.58 - - [03/Oct/2016:14:36:26 -0400] “GET /xibo/xmds.php?file=e64e5958d0646d304c1f3e6ef76b7f95 HTTP/1.1” 200 -
It is my assumption that this is the request from a client to download some media from the CMS. I also assumed that the file in the request is some MD5 hash of the file name. So I tried to find that hash in the xibo database but did not find any matches. Could this be the root cause of my issue? Is there anything else I can provide to help figure this out?
It does appear to be a problem is the MD5. The client is requesting an MD5 that does not exist in the sql database, Do any of you have any idea why this would be? Below is an error log snippet from a client.
FileAgent_media_Id_37|10/3/2016 4:16:47 PM|Info|FileAgent - Run|Thread alive and Lock Obtained
FileAgent_layout_Id_14|10/3/2016 4:16:47 PM|Info|FileAgent - Run|Thread Started
FileAgent_layout_Id_14|10/3/2016 4:16:47 PM|Info|FileAgent - Run|Thread alive and Lock Obtained
RequiredFilesAgentThread|10/3/2016 4:16:47 PM|Info|RequiredFiles - ReportInventory|Reporting Inventory
FileAgent_media_Id_37|10/3/2016 4:16:47 PM|Error|FileAgent - Run|Downloaded file failed MD5 check. Calculated [d41d8cd98f00b204e9800998ecf8427e] & XMDS [ 3e5ce4cb098ca820c7d49814f87c17e8] . 37.mp4
FileAgent_media_Id_37|10/3/2016 4:16:47 PM|Info|FileAgent - Run|Releasing Lock
FileAgent_layout_Id_14|10/3/2016 4:16:47 PM|Info|FileAgent - Run|File Downloaded Successfully. 14.xlf
FileAgent_layout_Id_14|10/3/2016 4:16:47 PM|Info|Schedule - LayoutFileModified|Layout file changed: 14.xlf
FileAgent_layout_Id_14|10/3/2016 4:16:47 PM|Info|FileAgent - Run|Releasing Lock
Are you connecting to the cms with https? Or a proxy? I am wondering if deep packet inspection on a firewall is not the problem. If it is turned on I think that would cause the MD5 check to fail.
There are no firewalls between my client and the CMS, all are running on the same campus network. Windows firewall is disabled for all domain computers. There are no proxies involved either. When I log into the CMS to manage it everything is done in HTTP. I do not know how to see if the clients are using HTTPS. My assumption is that they are not.
How does the client determine what hash is used for the GET request?
I, unfortunitaly do not know. @Dan, might be able to shed light on that part. But I think they are not matching because the file is not getting written to the client so the hashes will not match.
Also I should of read your posted log better. I see that the clients are talkng to the CMS via http.
Would it be possible to connect a client to the CMS server direct with a unmanaged switch or hub, and both machine having all other network connections turned off? This was we can rule out network security as far as devices.
In the CMS settings there is an option for “File Download Mode” - I would imagine you have that set to either Apache or Nginx ?
In either of those cases Xibo will offload the file downloading directly to the web server, rather than using the SOAP service to download the file. This requires the sendfile module to be configured on the web server such that the X-Sendfile or X-Accel headers are obeyed (causing the web server to get the file directly without involving PHP - better for performance).
Setting this to Off will revert to the SOAP service.
Configuring sendfile on Apache is fairly straight forward:
a2enmod xsendfile
Config file
XSendFile on
XSendFilePath <your library location>
The CMS was set to Apache. I switched it to OFF, as a test. There seems to be updating and file transfers now, but they are painfully slow, to the point of hours to get a 35MB video file downloaded and not all the clients completed getting the new content. This is far too slow and unreliable for our pursposes. So I would like to go back to the Apache method of file copies.
I’ve tried to confirm the settings in the webserver but I’m not a web admin/developer by trade. I do know enough to be dangerous and I can follow directions. It is my assumption that your code snippets are from the httpd.conf file. If not please let me know where to look for these settings. Unfortunately, my httpd.conf has nothing like the lines you provided. The only thing I found that was remotely similar is below:
#
# EnableMMAP and EnableSendfile: On systems that support it,
# memory-mapping or the sendfile syscall may be used to deliver
# files. This usually improves server performance, but must
# be turned off when serving from networked-mounted
# filesystems or if support for these functions is otherwise
# broken on your system.
# Defaults: EnableMMAP On, EnableSendfile Off
#
EnableMMAP off
EnableSendfile off
As always, thank you for your time, and let me know if I can provide any additional detail.
To the best of my ability, it appears that my version of WAMP (3.0.0) does not support the XSendFile module. I have looked for a version of the module that matches my Apache version (2.4.17) but to no avail. It so appears that I will have to, at the very least, update my Apache to the latest build (2.4.23) and then confirm that the XSendFile module is installed and configured. This is not something that I have done before, especially in regards to Xibo (my predecessor did the present installation). What can I expect in the way of pitfalls or caveats? Any advise or encouragement will be appreciated.
I have updated the Apache binary in WAMP to 2.4.23 and followed the instructions for installing the XSendFile module. I set the CMS File Download Mode back to Apache, but still no joy. The client will receive the layouts but none of the media. I have even removed and re-installed the client in an attempt to get them talking to each other. I’m getting pretty frustrated with myself and the software.
What would happen if I mapped a windows network drive on the client machine to the CMS library location and set that as my “local” library?
Progress, sort of. I can certainly say that the XSendFile module is loaded. See Below:
[Tue Oct 25 14:25:01.795057 2016] [:error] [pid 7188:tid 872] (20023)The given path was above the root path: [client 10.200.240.82:49964] xsendfile: unable to find file: c:\XiboData\jquery-1.11.1.min.js
[Tue Oct 25 14:25:01.826306 2016] [:error] [pid 7188:tid 864] (20023)The given path was above the root path: [client 10.200.240.82:49965] xsendfile: unable to find file: c:\XiboData\xibo-layout-scaler.js
[Tue Oct 25 14:25:01.966927 2016] [:error] [pid 7188:tid 872] (20023)The given path was above the root path: [client 10.200.240.82:49964] xsendfile: unable to find file: c:\XiboData\xibo-webpage-render.js
[Tue Oct 25 14:25:01.982551 2016] [:error] [pid 7188:tid 864] (20023)The given path was above the root path: [client 10.200.240.82:49965] xsendfile: unable to find file: c:\XiboData\moment.js
[Tue Oct 25 14:25:02.138796 2016] [:error] [pid 7188:tid 872] (20023)The given path was above the root path: [client 10.200.240.82:49964] xsendfile: unable to find file: c:\XiboData\xibo-text-render.js
[Tue Oct 25 14:25:02.232543 2016] [:error] [pid 7188:tid 864] (20023)The given path was above the root path: [client 10.200.240.82:49965] xsendfile: unable to find file: c:\XiboData\jquery.marquee.min.js
[Tue Oct 25 14:25:02.295041 2016] [:error] [pid 7188:tid 872] (20023)The given path was above the root path: [client 10.200.240.82:49964] xsendfile: unable to find file: c:\XiboData\flipclock.min.js
With that said, you see my current dilemma. I tried to change the library location both in the httpd.conf file and in the CMS to the root of the Apache www directory (c:\wamp64\www\XiboData) and restarted all the services, but the error log still states the above. Is there somewhere else I need to change this info?
Hey everybody! I fixed it. I’m not sure how or why it took so long for me to figure it out, but I learned a lot. Thanks for the ideas and troubleshooting @dan @Peter and @cslaughter it was appreciated.
Just for posterity, the final step was to make sure that my library was physically located within the same directory as the CMS. In my case this was (C:/wamp64/www/Xibo/XiboData). With Xibo being the CMS directory and XiboData being my Library directory. Now XSendFile can find and transmit the media files and do it quickly. I can delete the local library (on the client) and be back to fully functional in less than 5 minutes.
Thanks again.
Brian