R403: DataAgent: Cache not ready

Okay, thats interesting. I already wrote, that this happens on one display only. But I tested another one with the same layout: I get the same error messages about the cache being not ready.

I leave it as is for further inspection, maybe some of the devs find some time for it :confused:

So… I copied the whole layout with contents. I started to remove some contents but the datasets in question do not come to life.

I removed every widget and started over to create new dataset widgets with the existing datatset assigned to it. Just one widget with one dataset - its not working. It shows the “No data”-text and “cache not ready”.

Another display with another dataset is working. Could it be the dataset?

We really need to get that sorted. If any dev have some time: how we can debug that?

The Xibo web preview is always working just fine!

The layout export is as easy as:

{
  "layout": "BS - Layout 1 KOPIE",
  "description": "",
  "regions": {
    "1131": ""
  },
  "drawers": {
    "1133": "BS - Layout 1-5 - drawer"
  },
  "layoutDefinitions": {
    "layoutId": 398,
    "ownerId": 1,
    "campaignId": 61,
    "parentId": null,
    "publishedStatusId": 1,
    "publishedStatus": "Published",
    "publishedDate": null,
    "backgroundImageId": 267,
    "schemaVersion": 4,
    "layout": "BS - Layout 1 KOPIE",
    "description": "",
    "backgroundColor": "#000000",
    "createdDt": "2024-10-21 14:27:30",
    "modifiedDt": "2024-10-21 14:27:34",
    "status": 1,
    "retired": 0,
    "backgroundzIndex": 0,
    "width": 3840,
    "height": 2160,
    "orientation": "landscape",
    "displayOrder": null,
    "duration": 10,
    "statusMessage": null,
    "enableStat": 0,
    "autoApplyTransitions": 0,
    "code": null,
    "isLocked": null,
    "regions": [
      {
        "regionId": 1131,
        "layoutId": 398,
        "ownerId": 1,
        "type": "playlist",
        "name": "",
        "width": 2576,
        "height": 1245,
        "top": 552,
        "left": 236,
        "zIndex": 1,
        "syncKey": null,
        "regionOptions": [
          {
            "regionId": 1131,
            "option": "loop",
            "value": "0"
          },
          {
            "regionId": 1131,
            "option": "transitionDirection",
            "value": null
          },
          {
            "regionId": 1131,
            "option": "transitionDuration",
            "value": null
          },
          {
            "regionId": 1131,
            "option": "transitionType",
            "value": ""
          }
        ],
        "permissions": [],
        "duration": 10,
        "isDrawer": 0,
        "actions": [],
        "tempId": null,
        "regionPlaylist": {
          "playlistId": 1134,
          "ownerId": 1,
          "name": "",
          "regionId": 1131,
          "isDynamic": 0,
          "filterMediaName": null,
          "filterMediaNameLogicalOperator": "OR",
          "filterMediaTags": null,
          "filterExactTags": 0,
          "filterMediaTagsLogicalOperator": "OR",
          "maxNumberOfItems": 0,
          "createdDt": "2024-10-21 14:27:30",
          "modifiedDt": "2024-10-21 14:27:30",
          "duration": 0,
          "requiresDurationUpdate": 0,
          "enableStat": null,
          "tags": [],
          "widgets": [
            {
              "widgetId": 1937,
              "playlistId": 1134,
              "ownerId": 1,
              "type": "dataset",
              "duration": 10,
              "displayOrder": 1,
              "useDuration": 0,
              "calculatedDuration": 10,
              "createdDt": 1729505978,
              "modifiedDt": 1729513650,
              "fromDt": 0,
              "toDt": 2147483647,
              "schemaVersion": 2,
              "transitionIn": null,
              "transitionOut": null,
              "transitionDurationIn": null,
              "transitionDurationOut": null,
              "widgetOptions": [
                {
                  "widgetId": 1937,
                  "type": "attrib",
                  "option": "dataSetId",
                  "value": "5"
                },
                {
                  "widgetId": 1937,
                  "type": "attrib",
                  "option": "enableStat",
                  "value": "Inherit"
                },
                {
                  "widgetId": 1937,
                  "type": "attrib",
                  "option": "isRepeatData",
                  "value": "0"
                },
                {
                  "widgetId": 1937,
                  "type": "attrib",
                  "option": "name",
                  "value": ""
                },
                {
                  "widgetId": 1937,
                  "type": "attrib",
                  "option": "templateId",
                  "value": "dataset_string_template"
                },
                {
                  "widgetId": 1937,
                  "type": "cdata",
                  "option": "noDataMessage",
                  "value": "<p><span style=\"color:#ffffff;font-size:48px;\">Text eintragen...</span></p>"
                },
                {
                  "widgetId": 1937,
                  "type": "cdata",
                  "option": "template",
                  "value": "<p><span style=\"font-size:200px;\"><span style=\"font-family: Calibri, Candara, Segoe, Segoe UI, Optima, Arial, sans-serif;\"><u>[Titel|32]</u></span></span></p>\r\n\r\n<p><span style=\"font-size:70px;\"><span style=\"font-family: Calibri, Candara, Segoe, Segoe UI, Optima, Arial, sans-serif;\">[Inhalt|31]</span></span></p>\r\n"
                }
              ],
              "mediaIds": [],
              "audio": [],
              "permissions": [],
              "playlist": "",
              "actions": [],
              "tempId": null,
              "tempWidgetId": null,
              "isValid": false,
              "isNew": false,
              "folderId": 1,
              "permissionsFolderId": 1,
              "isDynamic": 0
            }
          ],
          "permissions": [],
          "tempId": null,
          "owner": "xibo_admin",
          "groupsWithPermissions": null,
          "folderId": 1,
          "permissionsFolderId": 1
        }
      }
    ],
    "tags": [],
    "drawers": [
      {
        "regionId": 1133,
        "layoutId": 398,
        "ownerId": 1,
        "type": "drawer",
        "name": "BS - Layout 1-5 - drawer",
        "width": 3840,
        "height": 2160,
        "top": 0,
        "left": 0,
        "zIndex": 0,
        "syncKey": null,
        "regionOptions": [],
        "permissions": [],
        "duration": 0,
        "isDrawer": 1,
        "actions": [],
        "tempId": null,
        "regionPlaylist": {
          "playlistId": 1136,
          "ownerId": 1,
          "name": "BS - Layout 1-5 - drawer",
          "regionId": 1133,
          "isDynamic": 0,
          "filterMediaName": null,
          "filterMediaNameLogicalOperator": "OR",
          "filterMediaTags": null,
          "filterExactTags": 0,
          "filterMediaTagsLogicalOperator": "OR",
          "maxNumberOfItems": 0,
          "createdDt": "2024-10-21 14:27:30",
          "modifiedDt": "2024-10-21 14:27:30",
          "duration": 0,
          "requiresDurationUpdate": 0,
          "enableStat": null,
          "tags": [],
          "widgets": [],
          "permissions": [],
          "tempId": null,
          "owner": "xibo_admin",
          "groupsWithPermissions": null,
          "folderId": 1,
          "permissionsFolderId": 1
        }
      }
    ],
    "actions": [],
    "permissions": [],
    "campaigns": [],
    "owner": "xibo_admin",
    "groupsWithPermissions": null,
    "folderId": 1,
    "permissionsFolderId": 1
  }
}

and the dataset is:

[
    {
        "dataSetId": 5,
        "dataSet": "BB Beitr\u00e4ge",
        "description": "",
        "userId": 1,
        "lastDataEdit": 1622711266,
        "owner": "xibo_admin",
        "groupsWithPermissions": null,
        "code": "",
        "isLookup": 0,
        "isRemote": 1,
        "isRealTime": 0,
        "dataConnectorSource": null,
        "method": "GET",
        "uri": "http:\/\/my-address",
        "postData": "",
        "authentication": "none",
        "username": "",
        "password": "",
        "customHeaders": "",
        "userAgent": "",
        "refreshRate": 0,
        "clearRate": 1,
        "truncateOnEmpty": 1,
        "runsAfter": 0,
        "lastSync": 1729513801,
        "lastClear": 0,
        "dataRoot": "",
        "summarize": "none",
        "summarizeField": "",
        "sourceId": 1,
        "ignoreFirstRow": 0,
        "rowLimit": null,
        "limitPolicy": "stop",
        "csvSeparator": null,
        "folderId": 1,
        "permissionsFolderId": 1,
        "columns": [
            {
                "dataSetColumnId": 31,
                "dataSetId": 5,
                "heading": "Inhalt",
                "dataTypeId": 6,
                "dataSetColumnTypeId": 3,
                "listContent": null,
                "columnOrder": 2,
                "formula": null,
                "dataType": "HTML",
                "remoteField": "inhalt",
                "showFilter": 0,
                "showSort": 0,
                "dataSetColumnType": "Remote",
                "tooltip": null,
                "isRequired": 0,
                "dateFormat": null
            },
            {
                "dataSetColumnId": 32,
                "dataSetId": 5,
                "heading": "Titel",
                "dataTypeId": 1,
                "dataSetColumnTypeId": 3,
                "listContent": null,
                "columnOrder": 3,
                "formula": null,
                "dataType": "String",
                "remoteField": "titel",
                "showFilter": 0,
                "showSort": 0,
                "dataSetColumnType": "Remote",
                "tooltip": null,
                "isRequired": 0,
                "dateFormat": null
            }
        ],
        "data": [
            {
                "id": 1,
                "Inhalt": "Text",
                "Titel": "Herzlich Willkommen"
            },
            {
                "id": 2,
                "Inhalt": "Text2",
                "Titel": "Anstehende Wartungen"
            }
        ]
    }
]

Okay, got a bit further. It seems to affect one dataset. as soon as this is involved, the client freaks out…

Is there any limit for the “data” fields? We have some large html, with <img src="png,base64"> format. Thats a big huge for some contents. The web-view displays everything just fine. But the client does not.

While typing this, I edited the data source to only output some simple words. And voila, the Xibo-client magically displays EVERYTHING just fine.

@dan Can you confirm, that big dataset data fields can cause issues? Can we debug that further?

There’s a default limit of 10,000 rows, but you can increase it in the settings

No, not rows. I speak about a single data field content.

1 Like

How much data is a row?

I still dont understand the issue. As soon as the content for a single field is filled with large data, I get the exdeption: cache not ready. I modified the source code to post full Exception:

getData: Failed to get data cache for widgetId 1950, e = Xibo\Support\Exception\NotFoundException Object
(
    [message:protected] => Cache not ready
    [string:Exception:private] => 
    [code:protected] => 404
    [file:protected] => /var/www/cms/lib/Xmds/Soap7.php
    [line:protected] => 152
    [trace:Exception:private] => Array
        (
            [0] => Array
                (
                    [function] => GetData
                    [class] => Xibo\Xmds\Soap7
                    [type] => ->
                )

            [1] => Array
                (
                    [file] => /var/www/cms/web/xmds.php
                    [line] => 384
                    [function] => handle
                    [class] => SoapServer
                    [type] => ->
                )

        )

    [previous:Exception:private] => 
    [httpStatusCode:Xibo\Support\Exception\GeneralException:private] => 500
    [property:protected] => 
    [help:protected] => 
)

So I enabled the debug log and searched for the debug messages for the Cache but there is nothing obvious. I try to dig deeper…

Okay, I found the issue:

Xibo is using StashPHP with Memcached driver for caching inside Docker.

Any Cache->save() failure seems to be ignored. I always see the Xibo debug message saveToCache: cached /widget/dataset/af0bbf628e7bce0898a5e821871073a9 for 300 seconds. But the truth is another: Memcached reports SERVER_ERROR object too large for cache.

Xibo is also just checking, if the $this->cache->set($object); returned an error but not if the $this->pool->save($this->cache) returned an error. The latter could pointed me eralier to the right direction maybe…

However: The solution here are slight modifications of the memcached service container:

The default object size is 1m. But my object is 1.61m.

So I changed the default memcahced cmdline from command: memcached -m 15 to command: memcached -m 32 -I 10m.

-m is the memory limit, I dont know if some more would be a problem. -I is the item size limit.

Now its working!

I create a PR for the Check, if the ->save command was successful. I DONT create a PR for the memcached cmdline. I want a dev to check my new parameters.

@dan ? Any news? A permanent fix would be awesome.

I also noticed, that if one widget says “cache not ready”, it brings down EVERY dataset related content on one display. That one display does not use playlists. Some widgets are directly inserted. Would this make any difference?

I dont know what Iam doing wrong. I have again one display that does absolutely nothing. If I assign a working layout from another working display it also does not populate any data. Its always these two errors then appearing:

So, the memcached memoyr was one thing (no errors now) but something else is messing with me. And I dont have any further debug messages available :frowning: