Can't upload large files

Hi,

I recently had to upload a lot of large files via the API, and found that files greater than ~2.5 gigs can’t be uploaded.

However, it works fine within Xibo, and all my php.ini settings and config.env settings are good.

Here is the error :

> **Notice** : Undefined offset: 1 in **/var/www/html/wioo-xibo/.public_html/vendor/guzzlehttp/guzzle/src/Handler/StreamHandler.php** on line **104**
> 
> **Notice** : Undefined offset: 1 in **/var/www/html/wioo-xibo/.public_html/vendor/guzzlehttp/guzzle/src/Handler/StreamHandler.php** on line **105**
> :xmedia printed :
> **Notice** : Undefined variable: xMedia in **/var/www/html/wioo-xibo/.public_html/vendor/xibosignage/oauth2-xibo-cms/objects/media.php** on line **63**
> 
> **Fatal error** : Uncaught InvalidArgumentException: Status code must be an integer value between 1xx and 5xx. in /var/www/html/wioo-xibo/.public_html/vendor/guzzlehttp/psr7/src/Response.php:151 
Stack trace: 
#0 /var/www/html/wioo-xibo/.public_html/vendor/guzzlehttp/psr7/src/Response.php(98): GuzzleHttp\Psr7\Response->assertStatusCodeRange(0)
#1 /var/www/html/wioo-xibo/.public_html/vendor/guzzlehttp/guzzle/src/Handler/StreamHandler.php(116): GuzzleHttp\Psr7\Response->__construct(0, Array, Object(GuzzleHttp\Psr7\Stream), NULL, NULL) 
#2 /var/www/html/wioo-xibo/.public_html/vendor/guzzlehttp/guzzle/src/Handler/StreamHandler.php(49): GuzzleHttp\Handler\StreamHandler->createResponse(Object(GuzzleHttp\Psr7\Request), Array, Object(GuzzleHttp\Psr7\Stream), NULL) 
#3 /var/www/html/wioo-xibo/.public_html/vendor/guzzlehttp/guzzle/src/PrepareBodyMiddleware.php(66): GuzzleHttp\Handler\StreamHandler->__invoke(Object(GuzzleHttp\Psr7\Request), Array) 
#4 /var/www/html/wioo-xibo/.public_html/vendor/guzzlehttp/guzzle/src/Middleware.php(30): Guzzle in **/var/www/html/wioo-xibo/.public_html/vendor/guzzlehttp/psr7/src/Response.php** on line **151**

PHP can clearly see the file, as it still is uploaded to the CMS, however it stops the code and I can’t get arround it…

Hi,

The file mentioned here isn’t part of Xibo as far as I am aware, there isn’t even any /objects/ folder in that library.

Also I think StreamHandler is used by Guzzle when curl isn’t available, which is strange as curl is included in our containers.

The error itself implies that the response didn’t contain any headers at all, as for why that might be, I am not sure. Are there any Apache errors that match up with the above one?

All in all something looks a bit strange with the environment?

Thanks,
Dan

Thanks for the quick response.

The errors related to the xMedia being undefined are “normal”

I created the folder “objects” with the media.php file for tests purposes, inside of it there is a “create” function, that call the XiboLibrary->create() function and stores the result in the $xMedia file.


As you can see on the image above, I’ve been trying to work arround the error. So the undefined xMedia variable comes from the “finally” statement. However I don’t think that it has anything to do with the problem, considering the errors still happens when I remove the “finally” part

I’ve looked in the apache error logs, and it’s just a copy of what was written on the web page, with some more logs :

[Mon Jul 20 10:08:39.270539 2020] [mpm_prefork:notice] [pid 648] AH00169: caught SIGTERM, shutting down
[Mon Jul 20 10:08:40.118713 2020] [mpm_prefork:notice] [pid 31740] AH00163: Apache/2.4.38 (Debian) configured – resuming nor
mal operations
[Mon Jul 20 10:08:40.118779 2020] [core:notice] [pid 31740] AH00094: Command line: ‘/usr/sbin/apache2’
[2020-07-20 10:08:43] API LIBRARY.INFO: Getting list of Displays [] []
[2020-07-20 10:08:43] API LIBRARY.DEBUG: Passing GET params to request [] []
[2020-07-20 10:08:43] API LIBRARY.DEBUG: Creating a new request with received parameters [] []
[2020-07-20 10:08:43] API LIBRARY.DEBUG: Getting Authenticated Request [] []
[2020-07-20 10:08:43] API LIBRARY.DEBUG: Checking Access token and requesting a new one if necessary [] []
[2020-07-20 10:08:43] API LIBRARY.INFO: Getting a new Access Token [] []
[2020-07-20 10:08:43] API LIBRARY.DEBUG: Getting parsed response from Abstract Provider [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Hydrating the response [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Preparing media payload [] []
[2020-07-20 10:08:44] API LIBRARY.INFO: Uploading new media file 5f15510c2db207.73324521 [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Passing POST params to request [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Creating a new request with received parameters [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Getting Authenticated Request [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Checking Access token and requesting a new one if necessary [] []
[2020-07-20 10:08:44] API LIBRARY.INFO: Getting a new Access Token [] []
[2020-07-20 10:08:44] API LIBRARY.DEBUG: Getting parsed response from Abstract Provider [] []

Regarding this :

API LIBRARY.INFO: Uploading new media file 5f15510c2db207.73324521 [] []

The numbers comes from the uniqid() PHP built-in function, so there is no duplicated names.

OK that makes sense then.

This means an Apache process has been killed, but appears to be from a previous log message. In fact all of the stuff above

[2020-07-20 10:08:44] API LIBRARY.DEBUG: Preparing media payload [] []

Must come from other parts of your code which you aren’t showing?

This is a hunch only, but if you’re running this script on the same server, processing a big loop of files and making multiple API requests, you might have a memory leak in your script which eventually consumes all the resources on the box.

Can you rule that out by quickly spinning up a php:cli container with your import?

I’m sorry I don’t really understand what you want me to do ?

You’re running the import script and the CMS on the same server, correct?

If yes, then i’d like you to separate those two things so that you run the Import Script on a different server to the Xibo CMS. I think the two things are interacting with each other, namely that there is a memory leak importing multiple files in a loop.

A simple example would be a new folder on your machine called /import, containing your script which does the import.

Open a cli inside that folder and then run:

docker run --interactive --tty --volume $PWD:/app --volume ~/.composer:/tmp composer require xibosignage/oauth2-xibo-cms

docker run -it --rm --name my-bulk-import -v "$PWD":/usr/src/myapp -w /usr/src/myapp php:7.4-cli php import.php

What you’ve got then is a completely separate PHP environment with runs your script. You will need to configure your import to use the Xibo CMS’s public URL.

This is just a hunch, but I can’t think of anything else it might be

No luck with that either, still get the same error messages…

Do you have any way to try uploading a file greater than 2.5 gigs to a CMS via API ?

I am wondering if the problem occures only on my end.

The Xibo Community site uses cookies. What are cookies?