Problem With Remote DataSets

[NewBie_1]

Hello,
I am a new user here. I’m having problems with remote datasets. I’m using xibo cms 3.3.4
I know that there have been many topics on this subject but none of them have solved the problem I am experiencing. My problem is that no data appears in the datasets when I press the view data button in the remote settings, please see the image below:

I’ve already created datasets like the example given by Mr. Peter as in the link below (with my little modification):
Data Set → Remote data source - Get Help - Xibo Community
Here is my remote dataset configuration, see the images below:

[NewBie_1]

[NewBie_1]

Here are the results of the url data test when I press the url test data button:

[NewBie_1]

[NewBie_1]

I have followed the methods from the links below but the result remains the same, namely there is no data when I press the view data button in the remote dataset. Here are the links:

Remote Dataset - Trouble configuring Remote Data Path

Since the processed looks to have correct data when you test it, I’d expect the dataSet to be populated when the fetch remote dataset task runs and if the last sync date in the row with your dataset has current date. If it’s not, then please enable CMS auditing, let the task run, collect logs and send them to me via private message.

Remote datasets not Working: SOLVED

Hi, just sharing a solution which might help somebody having the same problem. I was trying to work with remote datasets, however, even they showed correct results using “Test Data URL”, the results were not fetched by the FetchRemoteDatasets-Task nor displayed in the data table. It showed an error parsing JSON (no valid json) in the notifications. After some debugging, I came up with a solution, being related to the Guzzle HTTP client used for fetching. The function “getContents()” seems to b…

[NewBie_1]

No data in remote datasets

Hello again Ive got a problem with my remote datasets again - this time the data seems to have dropped out of some of my datasets so my charts are blank on my Xibo screens and this has been the case now for about 48 hours when Ive set them up to sync every minute. The displays that use these datasets are all logged in and it was all working fine for about a week and now, 1 by 1 I seem to be loosing the data out of my datasets and I dont know why. They arent syncing either and Im struggling to…

I hope someone can help me to solve this problem, thank you

Best regards,
Ray

[203]

I’m currently searching my own json remote dataset issue, but I believe this is related to a previous roadblock I got to on this journey - after all your testing your file is not being resynced as it hasn’t changed.

It looks like this might just be an example file you setup, so change it and try to sync again and see if it works.

A few little things I overcome that might help lead you or future readers towards the answer if you haven’t already (which I might end up being very wrong about, if so, oops):

• You have to go to Tasks and “Run Now” the Fetch Remote Datasets task
• Run Now doesn’t run now, its runs later (it’ll list the time)
• If your remote dataset hasn’t change it will be ignored (debugging->log look for skipping due to md5
• When you click Test data URL the data listed under “processed” will be based on your columns, so if the data is there, it should be okay.
• Clear cache doesn’t clear cache i.e. it doesn’t reset the “has the file already been read” record (as I type this out I realise its likely linked to a task that needs to then run and I’ve not allowed enough time, but maybe not)

Hope this helps you out.

[NewBie_1]

Hi,
Thank You very much for your answer, I really appreciate that.
Actually, I have changed the json file several times and the files that I have replaced I checked the format in the json validator and all the results are valid and when I test the url data, the results are good but the data still does not appear when I press the view data button. I’ve also tried turning off the fetch remote dataset task and re-enabling it, the result remains the same. You can see it in the 2 pictures below:

And the result is as below screenshot shown:

I don’t know why this is happening and there is no resolution. I also see many times more topics in this community about this problem, maybe I missed the topics or solutions that are on those topics, but I still did not find a solution.
Once again I thank you for the answer

Best regards,

Ray

[203]

I just resolved my last dataset/json issue (for future me’s - wildcards don’t seem possible, but the key is data root is the wildcard/row data) so I guess I’m a glutton for punishment and now I’m invested.

For brevity I missed some of the other troubleshooting I went through in this process.

First up, I guess it depends on the install type Cloud, Docker or CMS, I’ll guess you have one of the last two, else I think personal support tickets are offered. If you have the CMS, XTR needs to be manually configured. Basically setting up the cron job, I’m fairly sure as long as your dataset is listing a “Last Sync” this is in working order as per this screenshot (you didn’t include one of your own, so possibly here is the issue):

The other factor to consider is that in the annals of time I read a comment by some of the guys from Xibo along the lines of “Remote Datasets are not synced unless they are actively consumed by a layout”.

Just ad hoc dumping a dataset ticker/view that looks at this dataset into your active layout and publishing it is enough for this to be true.

Finally, your tasks are running at 13 minutes past - are you running every 1 minute? There was guidance somewhere on here that suggested 5 minutes minimum. I’m fairly sure the tasks take more than a minute to run, so they get stuck. I only had successful syncs after changing from 1 minute to 5, but admittedly I’d just changed it to 1 in the first place due to frustration at failed syncs (which were at one time or another due to one of the points I’ve listed above).

The last thing is to consider turning on debug mode and browsing the log, it wasn’t the most friendly experience I’ve had, but I found searching in advanced for the url of the dataset would help me find the exact time the sync last ran, clearing that filter/search I could then find all the other related log’s and see what xibo thinks its doing with the dataset as it runs the remote fetch task.

When things get frustrating, its really easy to break proper testing procedure and skip things that should be logical or obvious on reflection, I did a whole lot of that. But as I just went through this journey hopefully some of this leads you to a solution.

[NewBie_1]

Hi,
Thank you for your reply. I think you’re right, regarding the last sync in the fetch remote dataset task. I am using xibo cms. I don’t have the data as in your image above. My data as shown in the image below:

The problem is that I don’t know how to manually set xtr? I’ve read the topic on this subject in this community but don’t know how to manually set up xtr. Can you give me instructions on how to configure xtr to fetch?

Once again, thank you very much

Best regards,

Ray

[203]

Fantastic that we might have this solved!

So XTR setup details are here: Xibo on a Web server | Xibo Digital Signage

But I realise depending on your own knowledge those might not be entirely clear, its basically just setting up a cron job to point in the right place.

How you setup a cron job will depend on your host and where the right place is will depend on your host & how you setup the CMS.

That may be enough to set you off, but let me know if you need more help.

You should be able to setup cronjobs from cpanel access (which I’m assuming you will have)

Heres basically what I have

If you want help with how to setup cron jobs and what all those asterisks mean https://crontab.guru/

If you compare mine and the example they provide, I hope it’ll make sense with whatever your host offers.

Again, let me know if its just not clear.

[NewBie_1]

Hello,
First, I apologize for my late response. Secondly, I have tried the advice you gave above according to the link you gave above (Xibo on a Web server | Xibo Digital Signage). Because I use windows 11, then I use the task scheduler as suggested in the link, but the result remains the same, last sync on my dataset shows nothing. You can see the picture below:

Here’s my task scheduler configuration:

I even made this so that xibo maintenance also runs automatically, but the result is the same (see image below):

Here are my logs when I run the fetch remote dataset task:

Approximately from the log above, do you know of any errors in the log that make the sync data not appear in the dataset?
I hope you can help me, thank you

Best regards,

Ray

[NewBie_1]

Hello,
This is just my guess. Perhaps the problem is as follows:

1. Fetch remote dataset task runs normally but is not displayed in the dataset so it is not visible in last sync
2. My remote dataset is correct and when I test the url data, the data appears, but when it is saved and when I create a column, the data does not appear in the data view on the remote dataset it is likely that the data is in the form of objects but cannot be parsed into text
   I tried to see in the log, it always appears invalidate cache (Install Fonts called with options: {“invalidateCache”:false}) then appears cms css font returned from cache and appears Loading 1. All Objects = 0. Then sometimes it appears UserOption navigationMenuPosition not found and Install.php exists and shouldn’t. Frankly I don’t know how to solve this problem. I hope you or xibo developers in this community can help me.
   I apologize if I have disrupted your activities with my problem

Best regards,

Ray

[203]

I don’t use xammpp, but I’m pretty sure you don’t need to enter the path to php in your task, as it already points to the php exe. The only parameter (argument) you’d need is the second part, I do see the guide claims otherwise though

Basically this runs php-win.exe with the parameter/argument php-win.exe which doesn’t make a lot of sense to me. But the parameter -f file/path/to/xtr.php does.

It notes that its based on Moodle Cron, which says the same sort of thing (but doesn’t quite match with what I’m seeing when I attempt to follow the guide myself on my version of windows)

You also need to make sure the task is running as a user with the right permissions.

If you aren’t seeing a sync time, your task isn’t running, so the answer is somewhere here.

Basically this task just runs that php file at the scheduled time, it isn’t publicly accessible, so you can’t just open it in a browser, but I see Moodle also suggests using wget.

I’m pretty sure this is just a task configuration issue. But if I were you and I was convinced it was correctly configured, I’d create a quick php page that sends an email or generates a file and run a task pointed at it to see if your task runs as expected.

That way you can at least be sure the task is setup correctly.

[NewBie_1]

Hello,
Thank you for your reply. I think my scheduler task is running (please see the image below):

For create a quick php page that sends an email or generates a file and run a task pointed at it, I don’t know how to make it, maybe you can help me with it, I’m completely new to this field.
Once again, thank you very much for your help and response

Best regards,

Ray

[203]

Sorry, I should have been clearer - Windows Task Scheduler is running, but the Task in Xibo isn’t (if there is no sync date).

Basically the Windows Task runs the command, reports successfully that it ran the command, but it doesn’t have any way of knowing what happens after - if it actually does anything (did you try without the php-win.exe in the argument?)

PHP Mail is here: PHP: mail - Manual

<?php
mail('your@email.com', 'Email Subject', 'This is a test');
?>```

[NewBie_1]

Hello,
Thank you for your reply. Well, for your question about without the php-win.exe in the argument, I replied that I had tried it before and the results remained the same. I will try your suggestion abaut PHP Mail but before that, I have a question for you. Do I have to connect this PHP Mail to another PHP file in XAMPP? For example I connect with php-win? or can I use an example format like the image below:

That’s what I got from this link:
PHP: pcntl_fork - Manual

And I got that link from the link below (about PCNTL in the XTR Routine section):
Xibo on a Web server | Xibo Digital Signage (xibosignage.com)

Actually I have seen the link above before but I am confused which format is valid and should be used in this case. Once again thank you very much

Best regards,

Ray

[203]

Just put the code I gave you in a text file, save is as “testmail.php” in a folder inside your xampp install.

Run the php file to make sure it actually works and you get a mail (its always possible your install isn’t setup to send mail or it gets blocked or something)

If this works, set up the task in windows to run win-php.exe, with the argument -f path/to/testmail.php and run it and see if you get a mail.

[NewBie_1]

Hello,
Thank you very much, I will try it

Best regards,

Ray

[NewBie_1]

Hello,
I’ve tried your suggestion and it has worked (see image below):

I have created the task scheduler according to your suggestion (see image below):

So what are the next steps I should take? Thank You

Best regards,

Ray