Datasetview Displaying Stale Data

Hey gentlemen (@dan, @Peter, @Alex, et. all),

I need some technical guidance about the inner-workings of Xibo.

At the theater company we have a sign outside of each stage that displays the name of the current show or event as well as the names of the donors who made the production possible. I developed what I thought was a clever solution involving the use of datasets, but we find ourselves needing to reboot to get the latest data.

Here is the setup:

  1. A JSON feed on our website exports the show title and the timestamp when it starts and ends.
  2. This JSON is processed and imported as a Remote DataSet.
  3. The dataset is loaded into a layout with a 5 minute refresh and an advanced filter clause:
    `Venue Location` = 'fydamar' AND `Venue ID` = '49' AND `Event Start` < (UNIX_TIMESTAMP() + 4200) AND `Event End` > (UNIX_TIMESTAMP() - 900)
  4. The necessary layout is scheduled to run on schedule via some API magic, and should display info about the show as long as the timestamp falls within the specified window.

Does it work? Yes, sir! Open the layout in the CMS, and the correct show title is shown. Schedule the layout to run for the first time, and you’re golden.

Everything is great until the client sits there for a day or two showing its default layout and then brings up the layout again. Suddenly no title is shown, presumably because it pulled an empty dataset in the interim and stopped updating. The fix? Rebooting the client causes the data to refresh and display again as expected.

Can you guys help me figure out what might cause this? I don’t know enough about the client-cms communication to debug why this may be happening.

I have been able to successfully reproduce the problem with the following CMS/Client versions:
CMS: 1.8.9
Android Client: 1.8-105
Windows Client: 1.8.8-131

Does that mean the layout has 5 min duration or the dataSet view widget has 'Update Interval ’ set to 5 min or perhaps both?

Also is that the only item on the layout / region it’s in?

Have you changed regarding the fetch remote dataSet task or is it running with default settings? (which is every 30 min I believe).

The remote dataSet itself, what’s the refresh and truncate on it?

Could be good to know if that is utilizing the changeLayout action or is it creating a new schedule event.

Could be a good test to enter something in the A message to display when no data is returned from the source and see if that’s what it shows in this case.

Thanks for the help! So many questions…

For starters, the remote JSON feed always contains the next 14 days of data to ensure that the system works even if the feed breaks for some odd reason. That part is working well enough for now. It truncates on each run.

With regard to the layout, it has a 30 second duration and the widget has a 5 minute update interval. There are also two other regions containing other datasets.

As for the scheduling, we’re just auto-populating the schedule with the appropriate campaigns based on events in our database–no use of changeLayout.

I’ve added the message to display when no data is returned from the source to verify it is displayed as expected. I can confirm that it does show in the layout editor.

Please let me know if you have any more thoughts/questions!

UPDATE: We’ve now tried the following…

  • We’ve reduced the update interval to one minute for shits and giggles
  • We’ve verified that the “message to display when no data is returned” actually appears on the display–not only in the layout editor

Here is where we’re at… Sometimes it works, sometimes it doesn’t, but it always fixes itself after a reboot. We’ve gone though multiple performances over the course of two days and it has displayed as intended for every single show. We’ve also had the layout load with the “no data” message, stayed that way, rebooted, and had it fix itself.

@dan Can you lend any insight as to when the dataset refreshes on the client’s end? I’m starting to develop a theory based on the patterns that I have seen. I feel like the client is only refreshing the dataset when the content of the underlying dataset has changed–even if the filter would produce different a different result set. Is there any truth to that? It could also simply be my imagination.

You might be on to something there - the CMS caches the output of the Widget’s resource call (the HTML downloaded to the local player library). This is cached based on a bunch of rules, which you can read about in the developer docs if you’re interested - widget caching heading in this link.

One of these rules (special for DataSet based Tickers/DataSet Views) says that the modified date should be equal to the widget modified date OR the dataset data modified date, whichever is the later. This causes the cache to be invalidated ahead of its cache duration (which is the update interval). So, in English, when the widget is changed or when the dataset is changed we invalidate the cache, otherwise we invalidate the cache after the update interval.

If your filter criteria is causing data to change faster than your update interval, then what you’re seeing makes sense.

Anything in the above confirm or deny your suspicions?

OMG @dan! That sounds like it explains our problem EXCEPT if the interval is 1 minute then I’d expect it to catch up within a minute. Or am I missing something?

Well in theory - yes, but it could fall foul of timing issues - for example the widget could reload on the Player just before it gets updated from the CMS. I’d expect a few minutes tops though?

Okay, so my team said that they made it through this entire past weekend with no reboots. There are only two changes that I made on Thursday:

  • Reduced refresh interval to one minute
  • Reduced length of layout to one minute

I think it’s the latter that may have helped. I’m wondering if it stops checking for updates once the widget starts looping to accommodate other widgets of longer length.

Will follow up if the changes continue to deliver dividends next weekend.

1 Like