1.8 Alpha - Trouble With Some API Calls

Dan,

No need to dig. Double checked with the last reinstall and the dates are now fine. No idea why. I apologize for sending you on a wild goose chase.

Ah, no problem at all :smiley: we appreciate the work you are doing!

1 Like

Thank you!

If you could, can you tell me what format the API would like multiple variables passed to it? We traced all the API calls, that are failing, back to sending more than one variable.

For example, this works:

function LibraryMediaList() {
$params = array(
‘api_path’ => ‘/api/library’,
‘api_method’ => ‘get’,
‘data_to_pass’ => array(‘ownerId’ => $_SESSION[‘clientID’])
);
callService($params, true);
}
function callService($params, $echo = false) {

$client = new Client();
$request = $client->$params[‘api_method’](SERVER_BASE.$params[‘api_path’],
[‘Authorization’=>[‘access_token’=>'Bearer ‘.$client_access_token’]],
[$params[‘data_to_pass’]]
);

$full_response = $request->send();
$GLOBALS[‘return’] = $full_response->getBody();
}

While this does not:

function statsMedia() {
$params = array(
‘api_path’ => ‘/api/stats’,
‘api_method’ => ‘GET’,
‘data_to_pass’ => array(‘display’ => 1, ‘fromDt’ => ‘2015-01-01 00:00:00’, ‘toDt’ => '2015-12-31 00:00:00)
);

callService($params, true);
}

function callService($params, $echo = false) {

$client = new Client();
$request = $client->$params[‘api_method’](SERVER_BASE.$params[‘api_path’],
[‘Authorization’=>[‘access_token’=>'Bearer ‘.$client_access_token’]],
[$params[‘data_to_pass’]]
);

$full_response = $request->send();
$GLOBALS[‘return’] = $full_response->getBody();
}

Problem seems to be solved by changing our code a bit. Not sure why Guzzle was allowing us to submit only one variable the original way we were using.

Changed this:

function callService($params, $echo = false) {

$client = new Client();
$request = $client->$params[‘api_method’](SERVER_BASE.$params[‘api_path’],
[‘Authorization’=>[‘access_token’=>'Bearer ‘.$client_access_token’]],
[$params[‘data_to_pass’]]
);

$full_response = $request->send();
$GLOBALS[‘return’] = $full_response->getBody();
}

to

function callService($params, $echo = false) {

$client = new Client();
$request = $client->$params[‘api_method’](SERVER_BASE.$params[‘api_path’].‘?’.$params[‘data_to_pass’],
[‘Authorization’=>[‘access_token’=>'Bearer ‘.$client_access_token’]]
);

$full_response = $request->send();
$GLOBALS[‘return’] = $full_response->getBody();
}

Which makes since for GET but not for POST, UPDATE, or DELETE. But in the end, the problem is surely not within the API.

We do appreciate the suggestion of jQuery-File-Upload :grinning:

However, we are finding that integrating this is a little more than expected for us. So, at some point, if someone who has a free moment,(:laughing:) to document how to post a file via the API, it would help us so that we can assess which path we should implement

The API expects all its parameters in the “formData” of the request, but I think you might be making it more complex than needed :smile:

Basically you should be letting Guzzel do the work, as that is what its there for - so create your Client:

$client = new Client([
    // Base URI is used with relative requests
    'base_uri' => SERVER_BASE
]);

Then make a post request with it (or whichever verb you want)

$client->request('POST', 'user', [
    'form_params' => [
        'userName' => 'dan'
    ]
]);

@cslaughter i’ve added some bits to the API today that might help you in your efforts (although I don’t know exactly what your integration does, so I can’t be sure!).

Up until now the Authorization server has only implemented the flow for authorization_code grants, where by the user is directed around the page to authorize the application request. However, i’ve now added support for client_credentials grants where the client_id and secret are enough to obtain an access token. This is intended for machine to machine implementations.

Therefore the application edit form now looks like this:

You tick the client credentials grant type and include grant_type=client_credentials in your request to /api/authorize/access_token to get access to this grant.

I’ve also expanded and updated the documentation in the manual: http://xibo.org.uk/manual-tempel/en/cms_api.html

1 Like

Dan, this is great. It works much better for us to have this option of authentication.

We have tried several ways to try and upload media to the API and each time we get

'error' => string 'Filetype not allowed' (length=20)

Using jQuery-file-upload does not work for us in for all instances. So we have tried this code instead:

function LibraryMediaFileUpload() {
	$fileName = MEDIA_BASE.'/'.$_GET['media'];
	$chunkedFile = '';	
	set_time_limit(0);
	$file = fopen($fileName,"rb");

	while(!feof($file))
	{
		$chunkedFile .= fread($file, 1024);
		ob_flush();
		flush();
	}
	$info = stream_get_meta_data($file);
	$payload = base64_encode($chunkedFile);	
	$status = fclose($file);

	$params = array(
		'api_method' => 'POST',
		'api_path' => 'api/library',
		'api_data' => [
			'paramName' => [
				'attachments' => [
					'file' => $payload
				],
				'multipart' => 'true',
				'autoUpload' => 'true'
			]
		]
        );
    callService($params, true);
}

Which is sent to:

function callService($params, $echo = false) {
	$client = new Client();
	$request = $client->$params['api_method'](SERVER_BASE.'/'.$params['api_path'],
		['Authorization'=>['access_token'=>'Bearer '.$_SESSION['client_access_token']]],
		$params['api_data']
	);
	$full_response = $request->send();
	$GLOBALS['return'] = $full_response->getBody();
}

$info tells us the files are read correctly and the file types are identified correctly. But we must be getting something still incorrect when passing the file. The error appears to come from jquery-file-upload-validate.js. Why is what we are not understanding. We have tried both a small jpg and a small-ish mp4.

We have looked through the documentation on jQuery-file-upload, and it looks like it should be one of the options that is blocking the upload due to the file type. However, in the jQuery-file-upload-validate.js file within the alpha, the options section is commented out. So I would assume that would meant that it will pretty much take whatever you give it.

EDIT: We have always assumed that ‘file’ should be the base64 encoded file. If it is the file path, we have also tried that as well. With and without ‘@’ in front of the path.

jquery-file-upload-validate.js won’t be called at all through the API - the API sits behind that in the PHP layer.

What you are actually hitting is the Library Controller, add method. /lib/Controller/Library.php#323 which is a slightly adapted version of the php server part of the jquery-file-uploader library.

If I get the time I will try and create a file upload example, if I can work it out! :smile:

Dan,

Thank you for the reply.

We have gone back over some things and have changed our code. Now we can at least get a jpg file to upload to the CMS temp folder. We cannot get other media types to upload to the temp folder yet. Here is our latest code.

Function to read and chunk media file:

function LibraryMediaFileUpload() {
	$fileName = 'http://abc123.com/xibo_interface_c/'.$_GET['media'];
	$chunkedFile = '';	
	set_time_limit(0);

	$file = fopen($fileName,"rb");
	while(!feof($file))
	{
		$chunkedFile .= fread($file, 1024*8);
		ob_flush();
		flush();
	}
	$info = stream_get_meta_data($file);
	$status = fclose($file);

    $params = array(
		'api_method' => 'POST',
		'api_path' => 'api/library',
		'api_data' => [
			'paramName' => [
				'attachments' => [
					'file' => $chunkedFile,
					'fileType' => $info['wrapper_data'][8]
				]
			]
		]

	);
    callService($params, true);
}

Which then calls this:

function callService($params, $echo = false) {
	if(isset($_SESSION['user_id']) && isset($params['api_path'])){

		$client = new Client();
		$request = $client->$params['api_method'](SERVER_BASE.'/'.$params['api_path'],
			['Authorization'=>['access_token'=>'Bearer '.$_SESSION['client_access_token']]]
		);
		
		if(!empty($params['api_data']['paramName']['attachments'])) {
			$contentType = explode(': ', $params['api_data']['paramName']['attachments']['fileType']);
			$request->addHeader($contentType[0], $contentType[1]);
			$request->addHeader('Accept', 'application/json');
			$request->addHeader('Accept-Encoding', 'gzip,deflate,sdch');
			$request->addHeader('Cache-Control', 'no-cache');
			$request->addHeader('Connection', 'keep-alive');
					
			$request->setBody($params['api_data']['paramName']['attachments']['file']);
		}else{
			$request->__toString($params['api_data']);
		}
		
		
		
		
		$full_response = $request->send();
		$GLOBALS['return'] = $full_response->getBody();
			
	}
}

I think that the key to getting the other media types uploaded, is in the headers.

UPDATE: We have noticed that if we have a mp4 or webm file that we want to upload, it will be rejected with “Filetype not allowed”. If we force the file type to image/jpg, image/gif, or any of the other accepted image formats, the system will take the file. If we then change the uploaded file’s extension in the temp directory and open it, it opens correctly.

So there seems to be an issue, at least on Windows, with the files being uploaded if they are not an image.

Though this might also, somehow be related to the non-response or return from the API after it accepts a file.

UPDATE 2:

We have managed to figure out how to pass all the needed file types we are wanting.

This is what made all the difference:

$request->addHeader('Content-Type', 'application/octet-stream');
$request->addHeader('Content-Disposition', 'attachment; filename="'.$fileName.'"');

But still we have the problem of a blank or no response after the file is up loaded via the API. :notes:… and the beat goes on… and the beat goes on…

In trying to figure out how the API is designed to work with the file uploads, we have found that the response header does have some information in it, the body does not. The information in the header tells us that the file uploaded ok, how much time it took and what not.

We are not sure where to go from here. In the previous version of the API, it was a two-part upload process. First part uploaded the file to the temp directory. The second part would then copy the file over to Xibo and register the file in Xibo. From what I can tell so far from the new API, this is not the case. So that leaves us with wondering why the file uploaded to the temp directory is not being picked up and registered to Xibo.

This problem is more than likely due to the fact that we are not using 'Content-Type', 'multipart/form-data', but 'Content-Type', 'application/octet-stream'. If we change the Content-Type to multipart/form-data we do get a response from the API, but the file length shows zero and the API deletes the file it created. (Also note we changed “filename” to “file” on the previous post.)

So we changed to multipart/form-data and moved the headers around a bit to be compliant with RFC1867. Now we get this for the headers:

string 'POST /xibo18alpha/web/api/library HTTP/1.1
Host: ABC123.com
Authorization: Bearer FDv1oPSDbrTsz6WGSt2f1RcGBYvctuXzMWjYq0mE
User-Agent: Guzzle/3.9.3 curl/7.37.0 PHP/5.5.12
Accept: video/webm
Content-Type: multipart/form-data
Content-Disposition: form-data; name="big-buck-bunny_trailer.webm"
Cache-Control: no-cache
Connection: keep-alive
Transfer-Encoding: chunked
file: Eߣ�B��B��B��B��B��webm����B��B��S�g!	�M�t�M��S��I�fS��@M��S��T�kS���M��S��M�tS��!	sM��S��S�kS��
AI�f�s��FƝE��'... (length=2165604)

And this for the API response:

array (size=1)
  'files' => 
    array (size=1)
      0 => 
        array (size=4)
          'name' => string 'big-buck-bunny_trailer.webm' (length=27)
          'size' => int 0
          'type' => string 'multipart/form-data' (length=19)
          'error' => string 'File is too small' (length=17)

At this point we think that either the API should be changed to accept Content-Type: multipart/mixed; for library uploads, or we will have to wait to see what Dan can figure out.

Sorry that you’ve had to look at this yourself - it is definitely on my list of things to do.

When I upload through the CMS I get this:

POST /library HTTP/1.1
Host: 192.168.0.28
Connection: keep-alive
Content-Length: 65665
Origin: http://192.168.0.28
X-XSRF-TOKEN: dec8b4aae915861b1dd0c6bac7ced1cd353cca9c
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryNegopDTQBWQsCyrm
Accept: application/json, text/javascript, */*; q=0.01
X-Requested-With: XMLHttpRequest
Referer: http://192.168.0.28/library/view
Accept-Encoding: gzip, deflate
Accept-Language: en-GB,en;q=0.8,en-US;q=0.6
Cookie: PHPSESSID=s7mijeqrieftgb8jt81o2duqs2

Payload

------WebKitFormBoundaryNegopDTQBWQsCyrm
Content-Disposition: form-data; name="name[]"


------WebKitFormBoundaryNegopDTQBWQsCyrm
Content-Disposition: form-data; name="name[]"


------WebKitFormBoundaryNegopDTQBWQsCyrm
Content-Disposition: form-data; name="files[]"; filename="unnamed.jpg"
Content-Type: image/jpeg


------WebKitFormBoundaryNegopDTQBWQsCyrm--

The jquery file uploader is designed to use which ever browser technology is available and pick the best method - including chunked uploads if necessary, so I think the key to this is to understand the options available there, pick the best one for API upload and then implement accordingly.

I haven’t forgotten about it and will look as soon as I can.

Dan,

Thank you for the reply. No worries on not getting back to us. I know you are really busy.

Going through everything has at least taught me a lot about headers when submitting content. But I do have to say, now I know why you prefer jQuery-File-Upload. What a pain it is to try and code the file upload.

We now have what we think is the correct format and headers, however, we just don’t seem to be able to get the system to recognize the form-data Content-Type correctly. We get the proper body size for the payload returned when using chunked files, but the file name and type seem to be missing.

------WebKitFormBoundaryNegopDTQBWQsCyrm
Content-Disposition: form-data; name=“name

{WHAT IS HERE?}

------WebKitFormBoundaryNegopDTQBWQsCyrm
Content-Disposition: form-data; name=“name

{WHAT IS HERE?}

------WebKitFormBoundaryNegopDTQBWQsCyrm
Content-Disposition: form-data; name=“files”; filename=“unnamed.jpg”
Content-Type: image/jpeg

{Raw File Goes Here}

------WebKitFormBoundaryNegopDTQBWQsCyrm–

@cslaughter - i took a step back and thought i’d try a simple example to see if I could do that first (i.e. no chunking, just a straight forward upload).

$response = $guzzle->request('POST', 'http://192.168.0.28/api/library', [
    'headers' => [
        'Authorization' => 'Bearer ' . $token
    ],
    'multipart' => [
        [
            'name' => 'name[]',
            'contents' => 'API upload'
        ],
        [
            'name' => 'files[]',
            'contents' => fopen('files/h264.mp4', 'r')
        ]
    ]
]);

This worked for me, although in the process I did identify a bug in the upload routine that effects single file uploads (i.e. if you were to call your variables name and files): https://github.com/xibosignage/xibo/issues/612

I’m also not sure that I am completely happy with the response (it is the same response provided to the jQuery wrapper

{"files":[{"name":"API upload 2","size":254688,"type":"video\/mp4","url":"\/api\/library?file=h264%20%284%29.mp4&download=1","storedas":"74.mp4","delete_url":"\/api\/library?file=API%20upload%202","delete_type":"DELETE"}]}

My feeling is that this should just be a 201, but at least it is workable

Well… Must be an issue when using a Windows platform.

I tried more or less the same same code and got this:

Any chance of you being able to try that on a Windows machine?

Note: We had to basicly chunk the file because of needing to use fread since fopen produces an error in guzzle about using a resource.

The code we used was:

use Guzzle\Http\Client;
use Guzzle\Http\EntityBody;

function LibraryMediaFileUpload() {
	$fileName = basename($_GET['media']);
	$fileNamePath = CMS_BASE.$_GET['media'];
	$chunkedFile = '';	
	set_time_limit(0);

	$file = fopen($fileNamePath,"rb");
	while(!feof($file))
	{
		$chunkedFile .= fread($file, 1024*8);
		ob_flush();
		flush();
	}

	$status = fclose($file);

    $params = array(
		'api_method' => 'POST',
		'api_path' => 'api/library',
		'api_data' => [
			'multipart' => [
				[
					'name' => 'name[]',
					'contents' => $fileName
				],
				[
					'name' => 'files[]',
					'contents' => $chunkedFile
				]
			]
		],
	);
    callService($params, true);
}

function callService($params, $echo = false) {
	if(isset($_SESSION['user_id']) && isset($params['api_path'])){

		$client = new Client();
		$request = $client->$params['api_method'](SERVER_BASE.'/'.$params['api_path'],
			['Authorization'=>['access_token'=>'Bearer '.$_SESSION['client_access_token']]],
 			$params['api_data'] 
		);
		$full_response = $request->send();

		$GLOBALS['return'] = $full_response->getBody();
	}
}

Update: From what I read on fopen, fopen basically sets up a resource pointer to the file, hence the error about using a resource and not a string.

So I then tried to fread the file after using fopen. This resulted in a string, but still the file name, file size and type did not get passed correctly Same problem I was running into before.

If I var_dump() multipart I get this:

array (size=1)
  'multipart' => 
    array (size=2)
      0 => 
        array (size=2)
          'name' => string 'name[]' (length=6)
          'contents' => string 'images.jpg' (length=10)
      1 => 
        array (size=2)
          'name' => string 'files[]' (length=7)
          'contents' => string '�����JFIF�����������	a( %!1"%)+...383,7(-.+



ee,$ $,,-,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,�������"�����������������a���C�a����!1AQa"q�������a2RBbr��#����3CS4s����������������������-�������!1AQ"2RaBq����������?���$�0D�I(A$�$�g��'*0I8L�)^��+�w�P��t�GR������5/�^�P�24����b�ϯ%Y�S��hn��yJ��M	����C�����bI$�(d�$� �'��,t��P�&)�P�JN�A��p����b�L$qq�G{��'... (length=8517)

var_dump() of the API response:

array (size=1)
  'files' => 
    array (size=1)
      0 => 
        array (size=4)
          'name' => string '1444965748-747' (length=14)
          'size' => int 21029
          'type' => string 'application/x-www-form-urlencoded; charset=utf-8' (length=48)
          'error' => string 'Filetype not allowed' (length=20)

Maybe a Guzzle version mismatch? I tried with the latest version - simply to avoid any issues that have already been fixed. My understanding is that if a multipart item contains a stream it will be handled slightly differently to if it contains a string (i.e not try to encode it)

Basically this is not what you want:

I’ve updated the repository with my example and the updated composer (run composer install).

1 Like

Yes! We updated to the latest version and it worked. May I recommend changing the version on oauth2-xibo-cms-master? Maybe you already did?:

Thank you for taking the time to help us on this. Hopefully others will also find the information useful.

Yes I have done so already.

Glad you have it working - its been fantastic to have you try this, we can release it with some confidence its been tested by a 3rd party :smiley:

Since updating the version of Guzzle we can no longer get a response on stats…??

Using this code:

$params = array(
	'api_method' => 'GET',
	'api_path' => 'api/stats',
	'api_data' => [
		'form_params' =>[
			'fromDt' => $fromDt,
			'toDt' => $toDt,
			'displayId' => intval($displayId),
			'mediaId' => intval($mediaId)
		]
	]
);


$client = new Client();
	$request = $client->request($params['api_method'], SERVER_BASE.'/'.$params['api_path'], [
			'headers' => [
				'Authorization' => 'Bearer '.$_SESSION['client_access_token']
			],
			$params['api_data']
		]
	);
}

$full_response = $request;
$GLOBALS['return'] = $full_response->getBody();

Input on the variables is as follows:

array (size=1)
  'form_params' => 
    array (size=4)
      'fromDt' => string '2015-01-01 00:00:00' (length=19)
      'toDt' => string '2015-10-31 23:59:59' (length=19)
      'displayId' => int 1
      'mediaId' => int 10

Double checked to make sure there should be data with those variables.

Get this when dumping the return body:

array (size=0)
  empty

UPDATE: Also tried mutipart after reading this; same result.