Docker upgrade from 4.0.0 to 4.0.1 or 4.0.2 fails with mysql error

pp_browser · September 12, 2023, 3:12pm

Hello,
upgrade from Xibo 4.0.0 (docker) to 4.0.1 or from 4.0.0 to 4.0.2 doesn’t work. Mysql is not reachable

No error during the upgrade process

dan · September 12, 2023, 4:21pm

Hi,

In the docker-compose.yml file we create a link between the cms-db container, and name it mysql so that docker should map that host inside the cms-web container.

It’s this bit:

        links:
            - cms-db:mysql

It isn’t clear why that would not work for you. Are you using the standard docker compose files we provide?

pp_browser · September 12, 2023, 4:38pm

Hi,
I have been updating Xibo regularly since version 1.7.5 without any problems
It is since the transition from version 4.0.0 to 4.0.1 or 4.0.2 that the problem occured.
I use the standard docker compose files that you provide.

I will test again and check if the docker-compose.yml file contains the indicated link

pp_browser · September 12, 2023, 4:50pm

Hi,

Here the details (I removed the links. 2 only allowed)

root@s001linux03:~# cd /opt/xibo
root@s001linux03:/opt/xibo# docker-compose stop
Stopping xibo_cms-web_1        ... done
Stopping xibo_cms-xmr_1        ... done
Stopping xibo_cms-db_1         ... done
Stopping xibo_cms-memcached_1  ... done
Stopping xibo_cms-quickchart_1 ... done
root@s001linux03:/opt/xibo# wget ***
--2023-09-12 18:41:51--  https://github.com/xibosignage/xibo-cms/releases/download/4.0.2/xibo-docker.tar.gz
Resolving github
Connecting to github|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: following]
--2023-09-12 18:41:51--  
Resolving objects
Connecting to objects.
HTTP request sent, awaiting response... 200 OK
Length: 16064 (16K) [application/octet-stream]
Saving to: ‘xibo-docker.tar.gz.1’

xibo-docker.tar.gz.1                                       100%[=====================================================================================================================================>]  15,69K  --.-KB/s    in 0,001s

2023-09-12 18:41:51 (14,3 MB/s) - ‘xibo-docker.tar.gz.1’ saved [16064/16064]

root@s001linux03:/opt/xibo# tar --strip 1 -zxvf xibo-docker.tar.gz
xibo-docker-4.0.0/.gitignore
xibo-docker-4.0.0/LICENSE
xibo-docker-4.0.0/README.md
xibo-docker-4.0.0/cms_custom-ports.yml.template
xibo-docker-4.0.0/cms_remote-mysql.yml
xibo-docker-4.0.0/config.env.template
xibo-docker-4.0.0/config.env.template-remote-mysql
xibo-docker-4.0.0/docker-compose.yml
root@s001linux03:/opt/xibo# nano docker-compose.yml
root@s001linux03:/opt/xibo# nano cms_remote-mysql.yml
root@s001linux03:/opt/xibo# nano docker-compose.yml
root@s001linux03:/opt/xibo# nano cms_remote-mysql.yml
root@s001linux03:/opt/xibo# docker-compose -f cms_custom-ports.yml up -d
WARNING: Found orphan containers (xibo_cms-memcached_1, xibo_cms-quickchart_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Recreating xibo_cms-xmr_1 ... done
Recreating xibo_cms-db_1  ... done
Recreating xibo_cms-web_1 ... done
root@s001linux03:/opt/xibo# docker-compose up -d
Recreating xibo_cms-db_1       ... done
Recreating xibo_cms-xmr_1      ... done
Starting xibo_cms-memcached_1  ... done
Starting xibo_cms-quickchart_1 ... done
Recreating xibo_cms-web_1      ... done
root@s001linux03:/opt/xibo# docker exec -ti xibo_cms-web_1 bash
root@1b178e50101f:/# mysql -u cms -h mysql -p*************
ERROR 2005 (HY000): Unknown MySQL server host 'mysql' (-3)

dan · September 12, 2023, 5:20pm

I wonder if the database isn’t able to start for some reason - or it starts and then stops.

We updated the MySQL container to 8.0 in 4.0.0, but you say that version is working, its just the 4.0.0 → 4.0.1/2 that doesn’t work. If there was a problem with MySQL I would have expected it to fail when you went from 3.3.7 to 4.0.0

Please can you get the output of:

docker-compose ps

And then:

docker-compose logs -f cms-db

pp_browser · September 13, 2023, 12:54pm

Hi,
Here the results :
root@s001linux03:/opt/xibo# docker-compose ps

Name Command State Ports
xibo_cms-db_1 docker-entrypoint.sh mysqld Up 3306/tcp, 33060/tcp
xibo_cms-memcached_1 docker-entrypoint.sh memca … Up 11211/tcp
xibo_cms-quickchart_1 yarn start Up 3400/tcp
xibo_cms-web_1 /entrypoint.sh Up 0.0.0.0:8080->80/tcp,:::8080->80/tcp
xibo_cms-xmr_1 docker-php-entrypoint /ent … Up 50001/tcp, 0.0.0.0:9505->9505/tcp,:::9505->9505/tcp

docker-compose logs -f cms-db
(returns many lines like these)

cms-db_1 | 2023-09-11T04:53:54.536445Z 67143 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ‘‘mysql_native_password’ is deprecated and will be removed in a future release. Please use caching_sha2_password instead’
cms-db_1 | 2023-09-11T04:53:55.136175Z 67144 [Warning] [MY-013360] [Server] Plugin mysql_native_password reported: ‘‘mysql_native_password’ is deprecated and will be removed in a future release. Please use caching_sha2_password instead’

dan · September 13, 2023, 2:54pm

So the cms-db container is up I see no reason why it wouldn’t be linked.

docker exec -ti xibo_cms-web_1 bash

Does the host ping?

ping mysql
ping cms-db

I guess you could open up your config.env file and change the MySQL host to use the unlinked name

MYSQL_HOST=cms-db

And then up -d the whole lot.

pp_browser · September 13, 2023, 3:22pm

I tried this in 4.0.0
ping mysql : ok
ping cms-db : ok

I added MYSQL_HOST=cms-db to config.env then upgraded to 4.0.2
ping mysql : ok
ping cms-db : KO

root@c97185c2bcb0:/# ping cms-db
ping: cms-db: Temporary failure in name resolution

dan · September 13, 2023, 3:41pm

So after upgrading to 4.0.2 you can ping the host, but you can’t connect to the host via the mysql client, but the mysql container is up.

I’ve never seen anything like this before.

@alex any ideas?

alex · September 13, 2023, 4:08pm

You’re downloading the new docker files but they’re being saved as xibo-docker.tar.gz.1 since xibo-docker.tar.gz already exists, and then you’re extracting xibo-docker.tar.gz so you’re not getting any new files there.

You also have orphan containers. I would down the stack, remove the orphans as the help text suggests, and then bring it up cleanly with the 4.0.2 containers. I would remove the MYSQL_HOST you added to config.env as that shouldn’t be necessary.

pp_browser · September 14, 2023, 8:00am

xibo-docker.tar.gz is removed systematically before each upgrade.
I have been applying this procedure since version 1.7.5
i will check orphan containers and remove MYSQL_HOST from config.env
I will get back to you as soon as possible.

Indeed, in the screenshots that I posted it refers to xibo-docker.tar.gz.1

pp_browser · September 14, 2023, 4:29pm

I tried an upgrade with remove- orphans option from 4.0.0 to 4.0.2 and same result

docker-compose -f cms_custom-ports.yml up -d --remove-orphans

root@f01f1972e218:/# ping cms-db
PING cms-db (172.18.0.4) 56(84) bytes of data.
64 bytes from xibo_cms-db_1.xibo_default (172.18.0.4): icmp_seq=1 ttl=64 time=0.075 ms
64 bytes from xibo_cms-db_1.xibo_default (172.18.0.4): icmp_seq=2 ttl=64 time=0.064 ms
64 bytes from xibo_cms-db_1.xibo_default (172.18.0.4): icmp_seq=3 ttl=64 time=0.061 ms

root@f01f1972e218:/# ping mysql
ping: mysql: Temporary failure in name resolution

root@f01f1972e218:/# mysql -u cms -h mysql -p*********
ERROR 2002 (HY000): Can’t connect to MySQL server on ‘mysql’ (115)

dan · September 15, 2023, 10:47am

Can you try downing the containers so that they get recreated from scratch?

docker-compose down

docker-compose -f cms_custom-ports.yml up -d

I am not sure if this is useful, but it perhaps explains why we’re scratching our heads. This is the difference between 4.0.0 and 4.0.2: Comparing 4.0.0...4.0.2 · xibosignage/xibo-docker · GitHub

And this is the difference between 3.3.8 and 4.0.2: Comparing 3.3.8...4.0.2 · xibosignage/xibo-docker · GitHub (click on “Files Changed”)

As you can see, not a whole lot of changes in there, and nothing which could explain the issue you’re facing here.

pp_browser · September 15, 2023, 11:26am

Thank you for this tip. I will try to upgrade once more asap.
Regards

pp_browser · September 15, 2023, 3:50pm

I tried with
docker-compose down
docker-compose -f cms_custom-ports.yml up -d
Then upgraded 4.0.0 to 4.0.2 and same result :

Recreating xibo_cms-db_1 … done
Recreating xibo_cms-xmr_1 … done
Creating xibo_cms-quickchart_1 … done
Creating xibo_cms-memcached_1 … done
Recreating xibo_cms-web_1 … done
root@s001linux03:/opt/xibo# docker exec -ti xibo_cms-web_1 bash
root@f39c25c31604:/# mysql -u cms -h mysql -p********
ERROR 2005 (HY000): Unknown MySQL server host ‘mysql’ (-3)
root@f39c25c31604:/# ping mysql
PING mysql (172.18.0.3) 56(84) bytes of data.
64 bytes from xibo_cms-db_1.xibo_default (172.18.0.3): icmp_seq=1 ttl=64 time=0.110 ms
64 bytes from xibo_cms-db_1.xibo_default (172.18.0.3): icmp_seq=2 ttl=64 time=0.061 ms
64 bytes from xibo_cms-db_1.xibo_default (172.18.0.3): icmp_seq=3 ttl=64 time=0.071 ms
^C
— mysql ping statistics —
8 packets transmitted, 3 received, 62.5% packet loss, time 7126ms
rtt min/avg/max/mdev = 0.061/0.080/0.110/0.021 ms

root@f39c25c31604:/# ping cms-db
ping: cms-db: Temporary failure in name resolution

pp_browser · September 26, 2023, 4:46pm

Hello,
same issue when upgrading from 4.0.0 to 4.0.3
ping: cms-db: Temporary failure in name resolution
or
ping: mysql: Temporary failure in name resolution

natasha · September 29, 2023, 10:04am

Hi, could you provide logs from the MySQL container?

pp_browser · September 29, 2023, 10:30am

Hi,
I will make another attempt to upgrade 4.0.0 to 4.0.3 and will send you the mysql logs as soon as possible

pp_browser · September 29, 2023, 11:38am

Could you please tell me where are mysql logs ?

I can find mysqld.log but the file is empty
I can’t find error.log.

natasha · September 29, 2023, 12:31pm

docker-compose logs cms-db