[ISSUE] MariaDB crashes during install on musl system

[ISSUE] MariaDB crashes during install on musl system

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 7160 bytes --]

New issue by haunma on void-packages repository

https://github.com/void-linux/void-packages/issues/14485

Description:
### System

* xuname:  
Void 5.2.14_1 x86_64-musl GenuineIntel uptodate rF

* package:  
mariadb 10.1.30_6

### Expected behavior
xbps-install mariadb completes successfully

### Actual behavior
The installer runs the configuration script which causes mysqld to crash while setting up tables:
```
[*] Downloading packages

[*] Verifying package integrity
mariadb-10.1.30_6: verifying RSA signature...

[*] Collecting package files
mariadb-10.1.30_6: collecting files...

[*] Unpacking packages
mariadb-10.1.30_6: unpacking ...

[*] Configuring unpacked packages
mariadb-10.1.30_6: configuring ...
Updated mysql (mysql) system user.
Installing MariaDB/MySQL system tables in '/var/lib/mysql' ...
2019-09-15 20:57:42 140153574591752 [Note] /usr/bin/mysqld (mysqld 10.1.30-MariaDB) starting as process 27417 ...
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: innodb_empty_free_list_algorithm has been changed to legacy because of small buffer pool size. In order to use backoff, increase buffer pool at least up to 20MB.

2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: The InnoDB memory heap is disabled
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Compressed tables use zlib 1.2.11
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Using SSE crc32 instructions 2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Completed initialization of buffer pool
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Highest supported file format is Barracuda.
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: The log sequence number 0 in ibdata file do not match the log sequence number 1609389 in the ib_logfiles!
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Restoring possible half-written data pages from the doublewrite buffer...
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: 128 rollback segment(s) are active.
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Waiting for purge to start
2019-09-15 20:57:42 140153574591752 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.36-82.2 started; log sequence number 1609389
2019-09-15 20:57:42 140153243163424 [Note] InnoDB: Dumping buffer pool(s) not yet started
2019-09-15 20:57:42 140153574591752 [Note] Recovering after a crash using mysql-bin
2019-09-15 20:57:42 140153574591752 [Note] Starting crash recovery...
2019-09-15 20:57:42 140153574591752 [Note] Crash recovery finished.
2019-09-15 20:57:42 140153242876704 [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1146: Table 'mysql.gtid_slave_pos' doesn't exist
190915 20:57:42 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

Server version: 10.1.30-MariaDB
key_buffer_size=16777216
read_buffer_size=262144
max_used_connections=0
max_threads=153
thread_count=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 137050 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x5612f0522748
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x5612ee65c920, backtrace may not be correct.
Bogus stack limit or frame pointer, fp=0x5612ee65c920, stack_bottom=0x7f77f8380000, thread_stack=295936, aborting backtrace.

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x0): is an invalid pointer
Connection ID (thread ID): 2
Status: NOT_KILLED

Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

We think the query pointer is invalid, but we will try to print it anyway.
Query:

Segmentation fault

Installation of system tables failed!  Examine the logs in
/var/lib/mysql for more information.

The problem could be conflicting information in an external
my.cnf files. You can ignore these by doing:

    shell> usr/bin/mysql_install_db --defaults-file=~/.my.cnf

You can also try to start the mysqld daemon with:

    shell> /usr/bin/mysqld --skip-grant --general-log &

and use the command line tool /usr/bin/mysql
to connect to the mysql database and look at the grant tables:

    shell> /usr/bin/mysql -u root mysql
    mysql> show tables;

Try 'mysqld --help' if you have problems with paths.  Using
--general-log gives you a log in /var/lib/mysql that may be helpful.

The latest information about mysql_install_db is available at
https://mariadb.com/kb/en/installing-system-tables-mysql_install_db
MariaDB is hosted on launchpad; You can find the latest source and
email lists at http://launchpad.net/maria

Please check all of the above before submitting a bug report
at http://mariadb.org/jira
```

### Steps to reproduce the behavior
Just try to xbps-install mariadb on a fresh Void musl install.

After some googling, I believe this may be the same bug as reported here:
https://gitlab.alpinelinux.org/alpine/aports/issues/9407
and here:
https://jira.mariadb.org/browse/MDEV-17200

If so, it has been fixed in upstream MariaDB as of 10.1.37.  I am somewhat surprised that it has not been reported in Void up to now, as I had exactly the same issue more than a year ago, which eventually caused me to give up as I could not find the cause.  It basically means no mysql on Void musl systems, nor any package which depends on a mysql database...

Re: MariaDB crashes during install on musl system

[voidlinux-github]

[-- Attachment #1: Type: text/plain, Size: 343 bytes --]

New comment by haunma on void-packages repository

https://github.com/void-linux/void-packages/issues/14485#issuecomment-531976354

Comment:
Bug confirmed with another musl user on #voidlinux IRC today.  It appears that no musl system has been able to install MariaDB for the last year and a half.  (Package was last updated in January 2018.)

Re: MariaDB crashes during install on musl system

[Johnnynator]

[-- Attachment #1: Type: text/plain, Size: 182 bytes --]

New comment by Johnnynator on void-packages repository

https://github.com/void-linux/void-packages/issues/14485#issuecomment-772840877

Comment:
This shouldn't be an issue anymore.

Re: [ISSUE] [CLOSED] MariaDB crashes during install on musl system

[Johnnynator]

[-- Attachment #1: Type: text/plain, Size: 7163 bytes --]

Closed issue by haunma on void-packages repository

https://github.com/void-linux/void-packages/issues/14485

Description:
### System

* xuname:  
Void 5.2.14_1 x86_64-musl GenuineIntel uptodate rF

* package:  
mariadb 10.1.30_6

### Expected behavior
xbps-install mariadb completes successfully

### Actual behavior
The installer runs the configuration script which causes mysqld to crash while setting up tables:
```
[*] Downloading packages

[*] Verifying package integrity
mariadb-10.1.30_6: verifying RSA signature...

[*] Collecting package files
mariadb-10.1.30_6: collecting files...

[*] Unpacking packages
mariadb-10.1.30_6: unpacking ...

[*] Configuring unpacked packages
mariadb-10.1.30_6: configuring ...
Updated mysql (mysql) system user.
Installing MariaDB/MySQL system tables in '/var/lib/mysql' ...
2019-09-15 20:57:42 140153574591752 [Note] /usr/bin/mysqld (mysqld 10.1.30-MariaDB) starting as process 27417 ...
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: innodb_empty_free_list_algorithm has been changed to legacy because of small buffer pool size. In order to use backoff, increase buffer pool at least up to 20MB.

2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: The InnoDB memory heap is disabled
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Compressed tables use zlib 1.2.11
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Using SSE crc32 instructions 2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Completed initialization of buffer pool
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Highest supported file format is Barracuda.
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: The log sequence number 0 in ibdata file do not match the log sequence number 1609389 in the ib_logfiles!
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Restoring possible half-written data pages from the doublewrite buffer...
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: 128 rollback segment(s) are active.
2019-09-15 20:57:42 140153574591752 [Note] InnoDB: Waiting for purge to start
2019-09-15 20:57:42 140153574591752 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.36-82.2 started; log sequence number 1609389
2019-09-15 20:57:42 140153243163424 [Note] InnoDB: Dumping buffer pool(s) not yet started
2019-09-15 20:57:42 140153574591752 [Note] Recovering after a crash using mysql-bin
2019-09-15 20:57:42 140153574591752 [Note] Starting crash recovery...
2019-09-15 20:57:42 140153574591752 [Note] Crash recovery finished.
2019-09-15 20:57:42 140153242876704 [Warning] Failed to load slave replication state from table mysql.gtid_slave_pos: 1146: Table 'mysql.gtid_slave_pos' doesn't exist
190915 20:57:42 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

Server version: 10.1.30-MariaDB
key_buffer_size=16777216
read_buffer_size=262144
max_used_connections=0
max_threads=153
thread_count=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 137050 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x5612f0522748
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x5612ee65c920, backtrace may not be correct.
Bogus stack limit or frame pointer, fp=0x5612ee65c920, stack_bottom=0x7f77f8380000, thread_stack=295936, aborting backtrace.

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x0): is an invalid pointer
Connection ID (thread ID): 2
Status: NOT_KILLED

Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on,orderby_uses_equalities=off

The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

We think the query pointer is invalid, but we will try to print it anyway.
Query:

Segmentation fault

Installation of system tables failed!  Examine the logs in
/var/lib/mysql for more information.

The problem could be conflicting information in an external
my.cnf files. You can ignore these by doing:

    shell> usr/bin/mysql_install_db --defaults-file=~/.my.cnf

You can also try to start the mysqld daemon with:

    shell> /usr/bin/mysqld --skip-grant --general-log &

and use the command line tool /usr/bin/mysql
to connect to the mysql database and look at the grant tables:

    shell> /usr/bin/mysql -u root mysql
    mysql> show tables;

Try 'mysqld --help' if you have problems with paths.  Using
--general-log gives you a log in /var/lib/mysql that may be helpful.

The latest information about mysql_install_db is available at
https://mariadb.com/kb/en/installing-system-tables-mysql_install_db
MariaDB is hosted on launchpad; You can find the latest source and
email lists at http://launchpad.net/maria

Please check all of the above before submitting a bug report
at http://mariadb.org/jira
```

### Steps to reproduce the behavior
Just try to xbps-install mariadb on a fresh Void musl install.

After some googling, I believe this may be the same bug as reported here:
https://gitlab.alpinelinux.org/alpine/aports/issues/9407
and here:
https://jira.mariadb.org/browse/MDEV-17200

If so, it has been fixed in upstream MariaDB as of 10.1.37.  I am somewhat surprised that it has not been reported in Void up to now, as I had exactly the same issue more than a year ago, which eventually caused me to give up as I could not find the cause.  It basically means no mysql on Void musl systems, nor any package which depends on a mysql database...