MDEV-39035 IN Subquery String Value Retrieval Failure (val_str)#4800
Open
AestheticAkhmad wants to merge 997 commits intoMariaDB:12.0from
Open
MDEV-39035 IN Subquery String Value Retrieval Failure (val_str)#4800AestheticAkhmad wants to merge 997 commits intoMariaDB:12.0from
AestheticAkhmad wants to merge 997 commits intoMariaDB:12.0from
Conversation
…DEFAULT VALUE() should only use table->insert_values when table->insert_values contains row values. table->insert_values gets row values for the ODKU clause so if VALUE() is used before that it shouldn't use table->insert_values
…sions 10.11.X and beyond my_getcputime() returns "cpu time in 1/10th on a microsecond (1e-7 s)"
otherwise it causes random failures in some later test that lists files in $datadir/test
don't let the parser create ridiculously deep joins that will be rejected later anyway
the "Test that bad value for plugin enum option is rejected correctly" needed multiple fixes: 1. don't set plugin-dir based on $MYSQLTEST_VARDIR, all plugins are in var/plugins, but $MYSQLTEST_VARDIR is var/1/, var/2/, etc if --parallel is used (that is, practically always), thus the ha_example.so cannot be loaded, because cannot be found. Test fails with "unknown option --plugin-example-enum" as the plugin is not loaded 2. force --plugin-maturity=experimental, otherwise even if not parallel the plugin will fail to load because of low maturity, test still fails with "unknown option --plugin-example-enum" 3. don't specify .so extension explicitly otherwise the plugin still doesn't load on windows, even if paths and maturity are fixed 4. set --plugin-example=FORCE otherwise plugin fails to load after reading --plugin-example-enum-var=noexist because of unknown enum value, the server ignores the failure and starts normally. the test hangs. 5. This needs the fix in sql_plugin.cc to detect that the plugin is forced even when some options failed to parse. It used to consider plugin forced only if all options parsed correctly, which was wrong. Now the test passes, testing what it was supposed to test - failure to parse an enum value of a plugin option. Without these fixes the test hanged as in 4 when run on the main branch in non-prarallel (e.g. one test only) mode.
…keys on vault errors * let use_cache_on_timeout apply to other errors * enable use_cache_on_timeout by default and deprecate it * increase cache_timeout to max and deprecate it * change it from long to portable longlong * delete both in 13.3
* put autocommit/commit outside of LOCK/UNLOCK. * use uppercase like all other commands * restore the old value of autocommit
with `ORDER BY number` if the number doesn't refer to a valid result column, use this number in the error message not '???'.
restore OPTIMIZE/ANALYZE replication under @read_only that was disabled in b62101f
apparently a file can be present in *more than one* rpm, e.g. /usr/bin/dtrace on rhel10 is present *both* in systemtap-sdt-dtrace and in systemtap-sdt-devel. Make sure there's a separator between entries.
don't use such a greedy regex_replace pattern
event scheduler was printing a lot of info in [Note] in error log. change to print its startup/shutdown notes only when log_warnings>0. and runtime notes only when log_warnings>2. one note was an abnormal error, change to [Error].
if ((res=item->val_str(str)) != str) is incorrect way to detect whether res can be safely used, because Item_char_typecast::val_str() can return res which is different from str, but shares the same buffer.
…, causes replication discrepancy
number-to-time conversion was too eagerly capping the value. A string "9000090" was invalid time, because of 90 seconds. But number-to-time was capping first, validating later, to 9000090->time worked. Let's fix it. also, let's make invalid time values in a string field include the field name in the warning message, just like invalid time values in a numeric field do.
set collation for AVG, like it's done for MAX
…background_work enable background threads on shutdown, so that they could respond to signals and free resources
columns Issue: - Purge thread attempts to purge a secondary index record that is not delete-marked. Root Cause: - When a secondary index includes a virtual column whose v_pos is greater than the number of fields in the clustered index record, the virtual column is incorrectly skipped while reading from the undo record. - This leads the purge logic to incorrectly assume it is safe to purge the secondary index record. - The code also confuses the nth virtual column with the nth stored column when writing ordering columns at the end of the undo record. Fix: - In trx_undo_update_rec_get_update(): Skip a virtual column only when v_pos == FIL_NULL, not when v_pos is greater than the number of fields. - In trx_undo_page_report_modify(): Ensure ordering columns are written based on the correct stored-column positions, without confusing them with virtual-column positions.
with foundation/corporation/external contribution The MariaDB server repository uses certain pull request labels as queue markers for incoming pull requests as follows: - External Contribution: for all external contributions - MariaDB Corporation: for employees of the MariaDB corporation - MariaDB Foundation: for the employees of the MariaDB foundation Right now, when a new pull request is filed, these tags need to be applied manually. However a large % of these manual assignments can be automated by checking if the author is in https://github.com/orgs/MariaDB/teams/staff or in https://github.com/orgs/MariaDB/teams/developers. If they are in staff "MariaDB Foundation" is assigned. if they are not in staff, but are in developers, "MariaDB Corporation" is assigned. If they are in neither staff nor developers, "External Contribution" is assigned. A github workflow is created to do the assignments. It is being triggered every day at 2AM. Or it can be triggered manually. This is stage1 of the fix: it is read only and only *prints* the changes needed
with foundation/corporation/external contribution Fixed a typo in the pr list filter.
The initial scanning of the GTID pos table in find_gtid_pos_tables_cb() used the wrong call to commit the transaction, ha_commit_trans(thd, TRUE) instead of trans_commit(thd). This could cause an assertion Assertion '!thd->in_active_multi_stmt_transaction() || thd->in_multi_stmt_transaction_mode()' failed in mysql_execute_command and possibly other issues due to incorrect state in the SQL thread THD. Similar problem in rpl_slave_state::truncate_state_table() also fixed. Note there is a sister patch cd88b08 which applied this same pattern which fixed a hang due to locks not being released in the ha_commit_trans(thd, TRUE) call. Test-case-by: Alice Sherepa <alice.sherepa@gmail.com> Reviewed-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com> Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>
compiling mariadb from a git tree Fixed sprintf deprecation warnings compiling on MacOSX. Replaced some sprintf calls with equivalent snprintf calls, enough so that "normal" compile on MacOSX (as documented in the docs) completes without warnings.
…th foundation/corporation/external contribution Further tuned the query to cover only PRs since 2025-01 and to cover all PR states. Removed unused JSON fields from the output and the parsing Enabled writing the labels back to the issue.
This fixes Appveyor build.
…UE call Analysis: When scanning json and reaching a path, the path type is JSON_PATH_ARRAY_WILD. So it should not return any value. But the code does not check that. Fix: Use path_setup_nwc() to check for range queries because json_value() should not accept range queries.
* Compile error GCCism - unportable use of __PRETTY_FUNCTION__ * Compile warning ha_rows to ulong cast missing * Link error Due to unnecessary use of _dig_vec_lower _dig_vec_lower would have to be marked MYSQL_PLUGIN_IMPORT if we really wanted it. * Packaging error (MSI) Newly introduced component videx-engine needs a place in MSI Feature tree
Build with vcpkg and WITH_ZLIB=bundled throws error "Broken installation of vcpkg port zlib" in FIND_PACKAGE(CURL) Set CMAKE_DISABLE_FIND_PACKAGE_ZLIB to workaround. Previously, this workaround was hashicorp specific, now we need this also for videx. Thus setting CMAKE_DISABLE_FIND_PACKAGE_ZLIB was moved to correct place, inside MYSQL_USE_BUNDLED_ZLIB macro.
Many CHANGE MASTER fields typically have the same configurations between multi-source connections – namely: master_connect_retry master_ssl master_ssl_ca master_ssl_capath master_ssl_cert master_ssl_cipher master_ssl_key master_ssl_verify_server_cert master_ssl_crl master_ssl_crlpath master_use_gtid master_retry_count master_heartbeat_period When MDEV-25674 added `master_retry_count` to CHANGE MASTER, it kept the server option `--master-retry-count` to be its default value. This commit back-adds corresponding server options for the defaults of the rest of those fields. With them, the command line or config files can set up common configurations across replication sources (and even replicas). `--autoset-master-use-gtid` and `--autoset-master-heartbeat-period` can also reset their prior corresponding options back to their unset states. CHANGE MASTER can override unset (defaulted) configurations per connection. This commit also adds `DEFAULT` keyword support for all of those fields, so overridden configurations can reset to the default without RESET REPLICA. Supporting passing the `DEFAULT` keyword also enables setting `master_connect_retry` and `master_retry_count` to 0, which was previously disregarded. While here, the commit also increases `master_retry_count` to 64-bit on LLP64 (e.g., Windows) to match LP64 (e.g., Linux). Reviewed-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com> Reviewed-by: Andrei Elkin <andrei.elkin@mariadb.com>
* MDEV-38410: Use array, not `std::initializer_list`
Some environments appear not to retain the backing array of a
static `std::initializer_list` in the MDEV-37530 release candidate,
and eventually crash when reading overwritten data.
This commit resolves the stealth issue by reverting to conventional
arrays, while maintaining convenience through deductive overloads.
* Compile problems
* Some of our platforms (namely SUSE 15, which uses GCC 7.5) support
C++17 syntaxes, but not all libraries, `<charconv>`` among those.
* Update to the current `main` branch
Co-authored-by: Sergei Golubchik <serg@mariadb.org>
Co-authored-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
…`amd64-msan-clang-20` workaround for clang issue llvm/llvm-project#179605 that shows up on clang 20+ with -O2 and msan
…VEPOINT, assertion failure InnoDB was rolling back a transaction internally, while the server thought the transaction stayed open. this was fixed in 10.11 by 387fe5e to rollback the transaction in the server and in 12.3 by d228f23 to not rollback in InnoDB let's keep 12.3 behavior, update test results to match. but combine two nearly indentical test cases into one.
… NULL Fixing the return type of Type_handler::Item_save_in_value() from bool to void. Adding a new method st_value::is_null(). This makes the code less confusing.
Adding support for cursors on prepared statements.
- SQL Standard way:
DECLARE c CURSOR FOR stmt;
PREPARE stmt FROM 'SELECT ?';
OPEN c USING 1;
- Oracle-style way with SYS_REFCURSOR variables:
DECLARE
c SYS_REFCURSOR;
BEGIN
OPEN c FOR 'SELECT ?' USING 1;
…p_lex_instr::get_query, UBSAN member access within null pointer of type 'const sp_assignment_lex'
The patch for "MDEV-38626 Unexpected `Data too long`..." earlier fixed this problem. Adding a test case only.
…::write(THD *, time_t, const char *, size_t, ulonglong, ulonglong, bool, const char *, size_t) The patch for "MDEV-33830 Support for cursors on prepared statements" tried to enable slow log for cursor OPEN statements. However it introduced a crash reported in MDEV-38359. After checking the slog log implementation for sp_instr_stmt, it appeared to be buggy: - MDEV-38611 Procedure statements do not write Explain entries to the slow log - MDEV-38614 Procedure statements write wrong Query_time into slow log It was decided to wait untile these bugs are cloded and enable slow log for OPEN under terms of a separate task: MDEV-38612 Cursor OPEN statements do not write to the slow log This patch reverts the change which enabled slow log for OPEN.
…ool parse_vcol_defs(THD *, MEM_ROOT *, TABLE *, bool *, vcol_init_mode) A new assertion added by MDEV-36290 is correct in trying to identify default_fields that weren't correctly accounted in parse_vcol_defs. It caught a case for INSERT DELAYED on tables using TIMESTAMP and DATETIME types with ON UPDATE default values. That is, the insert delayed logic opens the table in one thread, and then copies the field info to another table in the user thread. This copy loses most flag metadata for these fields though. In this case, the flag ON_UPDATE_NOW_FLAG was dropped, and the parse_col_defs() logic was unable to properly account for that default field, and the assertion fired. This patch fixes this by restoring the ON_UPDATE_NOW_FLAG during the field copy. Also, a comment is added to Field::make_new_field() to explain the state of the returned field. Reviewed-by: Monty <monty@mariadb.com> Signed-off-by: Brandon Nesterenko <brandon.nesterenko@mariadb.com>
XMLTYPE column added. Type_handler::get_column_attributes() added so parser can check if unexpected attributes were specified for the UDT column.
This is achived by changing the 'is_prefix' parameter to strncoll from a my_bool to my_bool*. This parameter is null if not specified by the caller. If the is_prefix parameter is set to point to a my_bool variable, this variable will be set to 1 if the second argument is a prefix of the first. This is needed by TO_DATE to be able to detect partial uniqiue matches of months etc. find_type() did this for latin1 strings, but we did not do this for utf8mb4 strings. This patch fixes this incompatibility.
Syntax:
TO_DATE(string_expression [DEFAULT string_expression ON CONVERSION ERROR],
format_string [,NLS_FORMAT_STRING])
The format_string has the same format elements as TO_CHAR(), except a
few elements that are not supported/usable for TO_DATE().
TO_DATE() returns a datetime or date value, depending on if the format
element FF is used.
Allowed separators, same as TO_CHAR():
space, tab and any of !#%'()*+,-./:;<=>
'&' can also be used if next character is not a character a-z or A-Z
"text' indicates a text string that is verbatim in the format. One cannot
use " as a separator.
Format elements supported by TO_DATE():
AD Anno Domini ("in the year of the Lord")
AD_DOT Anno Domini ("in the year of the Lord")
AM Meridian indicator (Before midday)
AM_DOT Meridian indicator (Before midday)
DAY Name of day
DD Day (1-31)
DDD Day of year (1-336)
DY Abbreviated name of day
FF[1-6] Fractional seconds
HH Hour (1-12)
HH12 Hour (1-12)
HH24 Hour (0-23)
MI Minutes (0-59)
MM Month (1-12)
MON Abbreviated name of month
MONTH Name of Month
PM Meridian indicator (After midday)
PM_DOT Meridian indicator (After midday)
RR 20th century dates in the 21st century. 2 digits
50-99 is assumed from 2000, 0-49 is assumed from 1900.
RRRR 20th century dates in the 21st century. 4 digits
SS Seconds
SYYYY Signed 4 digit year; MariaDB only supports positive years
Y 1 digit year
YY 2 digits year
YYY 3 digits year
YYYY 4 digits year
Note that if there is a missing part of the date, the current date is used!
For example if 'MM-DD HH-MM-SS' then the current year will be used.
(Oracle behaviour)
Not supported options:
- BC, D, DL, DS, E, EE, FM, FX, RM, SSSSS, TS, TZD, TZH, TZR, X,SY
BC is not supported by MariaDB datetime.
- Most of the other are exotic formats does not make sence in MariaDB as
we return datetime or datetime with fractions, not string.
- D (day-of-week) is not supported as it is not clear exactly how it would
map to MariaDB. This element depends on the NLS territory of the session.
- RR only works with 2 digit years (In Oracle RR can also work with 4
digit years in some context but the rules are not clear).
Extensions / differences compared to Oracle;
- MariaDB supports FF (fractional seconds). If FF[#] is used,
then TO_DATE will return a datetime with # of subseconds.
If FF is not used a datetime will be returned.
There is warning (no error) if string contains more digts than what
is specified with F(#]
- Names can be shortened to it's unique prefix. For example January and Ja
works fine.
- No error if the date string is shorter format_string and the next
not used character is not a number.. This is useful to get a date
from a mixed set of strings in date or datetime format.
Oracle gives an error if date string is too short.
- MariaDB supports short locales as language names
- NLS_DATE_FORMAT can use both " and ' for quoting.
- NLS_DATE_FORMAT must be a constant string.
- This is to ensure that the server knows which locale to use
when executing the function.
New formats handled by TO_CHAR():
FF[1-6] Fractional seconds
DDD Daynumber 1-366
IW Week 1-53 according to ISO 8601
I 1 digit year according to ISO 8601
IY 2 digit year according to ISO 8601
IYY 3 digit year according to ISO 8601
IYYY 4 digit year according to ISO 8601
SYYY 4 digit year according to ISO 8601 (Oracle can do signed)
Supported NLS_FORMAT_STRING options are:
NLS_CALENDAR=GREGORIAN
NLS_DATE_LANGUAGE=language
Support languages are:
- All MariaDB short locales, like en_AU.
- The following Oracle language names:
ALBANIAN, AMERICAN, ARABIC, BASQUE, BELARUSIAN, BRAZILIAN PORTUGUESE
BULGARIAN, CANADIAN FRENCH, CATALAN, CROATIAN, CYRILLIC SERBIAN CZECH,
DANISH, DUTCH, ENGLISH, ESTONIAN, FINNISH, FRENCH, GERMAN,
GREEK, HEBREW, HINDI, HUNGARIAN, ICELANDIC, INDONESIAN ITALIAN,
JAPANESE, KANNADA, KOREAN, LATIN AMERICAN SPANISH, LATVIAN,
LITHUANIAN, MACEDONIAN, MALAY, MEXICAN SPANISH, NORWEGIAN, POLISH,
PORTUGUESE, ROMANIAN, RUSSIAN, SIMPLIFIED CHINESE, SLOVAK, SLOVENIAN,
SPANISH, SWAHILI, SWEDISH, TAMIL, THAI, TRADITIONAL CHINESE, TURKISH,
UKRAINIAN, VIETNAMESE
Development bugs fixed:
MDEV-38403 Server crashes in Item_func_to_date::fix_length_and_dec upon
using an invalid argument
MDEV-38400 compat/oracle.func_to_date fails with PS protocol and cursor
protocol (Fixed by Serg)
MDEV-38404 TO_DATE: MTR coverage omissions, round 1
MDEV-38509 TO_DATE: AD_DOT does not appear to be supported
MDEV-38513 TO_DATE: NULL value for format string causes assertion failure
MDEV-38521 TO_DATE: Date strings with non-ASCII symbols cause warnings
and wrong results
MDEV-38578 TO_DATE: Possibly unexpected results upon wrong input
MDEV-38582 TO_DATE: NLS_DATE_LANGUAGE=JAPANESE does not parse values
which work in Oracle
MDEV-38584 TO_DATE: NLS_DATE_LANGUAGE=VIETNAMESE does not parse values
which work in Oracle
MDEV-38703 TO_DATE: Quotation for multi-word NLS_DATE_LANGUAGE leads
to syntax error in view definition
MDEV-38675 TO_DATE: MSAN/Valgrind/UBSAN errors in
extract_oracle_date_time
MDEV-38635 TO_DATE: UBSAN errors in item_timefunc.h upon comparison with
a view column
MDEV-38719 TO_DATE: Assertion `&my_charset_bin != charset()' failed in
String::append_for_single_quote_using_mb_wc
MDEV-38756 TO_DATE: MSAN/Valgrind errors in
Item_func_to_date::fix_length_and_dec upon PREPARE with
parameters
Known issues:
- Format string character matches inside quotes are done
one-letter-to-one-letter, like in LIKE predicate. That means things
like expansions and contractions do not work.
For example 'ss' does not match 'ß' in collations which treat them
as equal for the comparison operator.
Match is done taking into account case and accent sensitivity
of the subject argument collation, so for example this now works:
MariaDB [test]> SELECT TO_DATE('1920á12','YYYY"a"MM') AS c;
+---------------------+
| c |
+---------------------+
| 1920-12-17 00:00:00 |
+---------------------+
Co-author and reviewer: Alexander Barkov <bar@mariadb.com>
removed: * DES encryption * --secure-auth * --old * spider table options: bfz, btt, cmd, ctp, cwg, isa, ilm, ios, smd, stc, stl extended under old-mode: * YEAR(2), still available when old-mode=2_DIGIT_YEAR un-deprecated: * keep_files_on_create, originally (MDEV-23570) the idea was to make it TRUE and deprecate. It cannot be removed when FALSE, but TRUE breaks mariabackup.aria_backup where a table is altered from Aria to InnoDB during a backup, so both t.MAD/t.MAI and t.ibd gets into a backup.
- Scoped partial optimization for MDEV-21543 improving PAD SPACE collations for multibyte character sets in HA_KEYTYPE_VARTEXT1. - This keeps the old behavior for other cases until MDEV-38712 is resolved. - Benchmark: see zulip topic in the PR description - Results: - ~3.1% lower wall-clock time - ~2.1% fewer CPU cycles - ~0.5% fewer retired instructions - ~0.7% fewer branch mispredictions
…r_sql_thread Pure refactoring for clarity. No functional changes.
Master_Server_Id was not cleared after CHANGE MASTER or RESET SLAVE, showing a stale value until the slave reconnected. Reset master_id and prev_master_id to 0 in both code paths. The reset value (0) will be present in SHOW SLAVE STATUS until it is re-evaluated to the id of a new connected master server. Signed-off-by: Varun Deep Saini <varun.23bcs10048@ms.sst.scaler.com>
Add support for reversed executable comments using /*!!version */ and /*M!!version */ syntax. These execute the comment body only when the server version is strictly less than the specified version, which is the inverse of the existing /*!version */ syntax. This enables writing portable SQL that uses newer syntax on new servers while falling back to older syntax on older servers, e.g.: CREATE /*!100000 OR REPLACE */ TABLE /*!!100000 IF NOT EXISTS */ t1 ... On MariaDB >= 10.0 this expands to CREATE OR REPLACE TABLE t1, while on older versions it expands to CREATE TABLE IF NOT EXISTS t1. Implementation: in lex_one_token(), after detecting a versioned comment (/*! or /*M!), check for an additional '!' character. If present, invert the version comparison so the comment body is expanded only when MYSQL_VERSION_ID < version. Tests added to main.comments and plugins.server_audit confirming: - Reversed comments with version <= server version do not execute - Reversed comments with version > server version execute - MariaDB-specific /*M!! variant works correctly - Reversed comments without a version number always execute - Audit plugin correctly logs executed reversed comments - Combined forward + reversed comments in a single statement All new code of the whole pull request, including one or several files that are either new files or modified ones, are contributed under the BSD-new license. I am contributing on behalf of my employer Amazon Web Services, Inc.
There is a lot of more work needed to make this code compatitble with the rest of the MariaDB server. - All std:: usage should be removed - Most templatest should be removed - Usage of <unordered_map>, <string_view>, <optional>, <unordered_set> should be removed and replaced with my_sys, sql_string functions It would be nice of one could also have default values for all the CHANGE MASTER variables in my.cnf. This would allow one to have much shorter CHANGE MASTER commands
…ns (fix) Correct compilation on clang-20 by using a bitwise & rather than the incorrect &&.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The issue was caused by Item_in_subselect::val_str() which contained DBUG_ASSERT(0). To fix the problem, we use val_real() instead of the val_str() method, and then convert the numeric result to string.