[ruby-core:122665] [Ruby Bug#21503] \p{Word} does not match on \p{Join_Control} while docs say it does

[ruby-core:122665] [Ruby Bug#21503] \p{Word} does not match on \p{Join_Control} while docs say it does

[procmarco (Marco Concetto Rudilosso) via ruby-core]

Issue #21503 has been reported by procmarco (Marco Concetto Rudilosso).

----------------------------------------
Bug #21503: \p{Word} does not match on \p{Join_Control} while docs say it does
https://bugs.ruby-lang.org/issues/21503

* Author: procmarco (Marco Concetto Rudilosso)
* Status: Open
* ruby -v: 3.4.4
* Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
in the [docs](https://ruby-doc.org/3.4.1/Regexp.html#:~:text=/%5Cp%7B-,Word,-%7D/%3A%20A%20member) it is mentioned that `\p{Word}` matches the equivalent of: `[\p{M}\p{Nd}\p{Pc}\p{Alpha}\p{Join_Control}]` as it's also defined in the [unicode spec](https://unicode.org/reports/tr18/#word)

the issue is that it does not seem to be the case

```
irb(main):018> REGEX = /\p{Word}/u
=> /\p{Word}/
irb(main):019> "\u200D".gsub(REGEX, "-")
=> "‍"
irb(main):020> REGEX2 = /\p{Join_Control}/u
=> /\p{Join_Control}/
irb(main):021> "\u200D".gsub(REGEX2, "-")
=> "-"
```

There's 2 solutions here, either we change the docs or the code.




-- 
https://bugs.ruby-lang.org/
______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:122666] [Ruby Bug#21503] \p{Word} does not match on \p{Join_Control} while docs say it does

[procmarco (Marco Concetto Rudilosso) via ruby-core]

Issue #21503 has been updated by procmarco (Marco Concetto Rudilosso).


What I mean is that the current implementation of `\p{Word}` does not seem to match `\p{Join_Control}` even though it should and it also says so in the docs

----------------------------------------
Bug #21503: \p{Word} does not match on \p{Join_Control} while docs say it does
https://bugs.ruby-lang.org/issues/21503#change-113944

* Author: procmarco (Marco Concetto Rudilosso)
* Status: Open
* ruby -v: 3.4.4
* Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
in the [docs](https://ruby-doc.org/3.4.1/Regexp.html#:~:text=/%5Cp%7B-,Word,-%7D/%3A%20A%20member) it is mentioned that `\p{Word}` matches the equivalent of: `[\p{M}\p{Nd}\p{Pc}\p{Alpha}\p{Join_Control}]` as it's also defined in the [unicode spec](https://unicode.org/reports/tr18/#word)

the issue is that it does not seem to be the case

```
irb(main):018> REGEX = /\p{Word}/u
=> /\p{Word}/
irb(main):019> "\u200D".gsub(REGEX, "-")
=> "‍"
irb(main):020> REGEX2 = /\p{Join_Control}/u
=> /\p{Join_Control}/
irb(main):021> "\u200D".gsub(REGEX2, "-")
=> "-"
```

There's 2 solutions here, either we change the docs or the code.




-- 
https://bugs.ruby-lang.org/
______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:122669] [Ruby Bug#21503] \p{Word} does not match on \p{Join_Control} while docs say it does

[mame (Yusuke Endoh) via ruby-core]

Issue #21503 has been updated by mame (Yusuke Endoh).


There is already a PR for that: https://github.com/ruby/ruby/pull/7711

Can you take a look? @duerst @naruse 

----------------------------------------
Bug #21503: \p{Word} does not match on \p{Join_Control} while docs say it does
https://bugs.ruby-lang.org/issues/21503#change-113949

* Author: procmarco (Marco Concetto Rudilosso)
* Status: Open
* ruby -v: 3.4.4
* Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
in the [docs](https://ruby-doc.org/3.4.1/Regexp.html#:~:text=/%5Cp%7B-,Word,-%7D/%3A%20A%20member) it is mentioned that `\p{Word}` matches the equivalent of: `[\p{M}\p{Nd}\p{Pc}\p{Alpha}\p{Join_Control}]` as it's also defined in the [unicode spec](https://unicode.org/reports/tr18/#word)

the issue is that it does not seem to be the case

```
irb(main):018> REGEX = /\p{Word}/u
=> /\p{Word}/
irb(main):019> "\u200D".gsub(REGEX, "-")
=> "‍"
irb(main):020> REGEX2 = /\p{Join_Control}/u
=> /\p{Join_Control}/
irb(main):021> "\u200D".gsub(REGEX2, "-")
=> "-"
```

There's 2 solutions here, either we change the docs or the code.




-- 
https://bugs.ruby-lang.org/
______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:122718] [Ruby Bug#21503] \p{Word} does not match on \p{Join_Control} while docs say it does

[naruse (Yui NARUSE) via ruby-core]

Issue #21503 has been updated by naruse (Yui NARUSE).


It looks `\p{Word}` is updated in TR#18 Version 15.
https://www.unicode.org/reports/tr18/tr18-15.html
The fix looks good.

----------------------------------------
Bug #21503: \p{Word} does not match on \p{Join_Control} while docs say it does
https://bugs.ruby-lang.org/issues/21503#change-113995

* Author: procmarco (Marco Concetto Rudilosso)
* Status: Open
* ruby -v: 3.4.4
* Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN
----------------------------------------
in the [docs](https://ruby-doc.org/3.4.1/Regexp.html#:~:text=/%5Cp%7B-,Word,-%7D/%3A%20A%20member) it is mentioned that `\p{Word}` matches the equivalent of: `[\p{M}\p{Nd}\p{Pc}\p{Alpha}\p{Join_Control}]` as it's also defined in the [unicode spec](https://unicode.org/reports/tr18/#word)

the issue is that it does not seem to be the case

```
irb(main):018> REGEX = /\p{Word}/u
=> /\p{Word}/
irb(main):019> "\u200D".gsub(REGEX, "-")
=> "‍"
irb(main):020> REGEX2 = /\p{Join_Control}/u
=> /\p{Join_Control}/
irb(main):021> "\u200D".gsub(REGEX2, "-")
=> "-"
```

There's 2 solutions here, either we change the docs or the code.




-- 
https://bugs.ruby-lang.org/
______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:122720] [Ruby Bug#21503] \p{Word} does not match on \p{Join_Control} while docs say it does

[hsbt (Hiroshi SHIBATA) via ruby-core]

Issue #21503 has been updated by hsbt (Hiroshi SHIBATA).

Status changed from Open to Closed
Backport changed from 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: UNKNOWN to 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: REQUIRED

https://github.com/ruby/ruby/pull/7711 has been merged

----------------------------------------
Bug #21503: \p{Word} does not match on \p{Join_Control} while docs say it does
https://bugs.ruby-lang.org/issues/21503#change-113999

* Author: procmarco (Marco Concetto Rudilosso)
* Status: Closed
* ruby -v: 3.4.4
* Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: REQUIRED
----------------------------------------
in the [docs](https://ruby-doc.org/3.4.1/Regexp.html#:~:text=/%5Cp%7B-,Word,-%7D/%3A%20A%20member) it is mentioned that `\p{Word}` matches the equivalent of: `[\p{M}\p{Nd}\p{Pc}\p{Alpha}\p{Join_Control}]` as it's also defined in the [unicode spec](https://unicode.org/reports/tr18/#word)

the issue is that it does not seem to be the case

```
irb(main):018> REGEX = /\p{Word}/u
=> /\p{Word}/
irb(main):019> "\u200D".gsub(REGEX, "-")
=> "‍"
irb(main):020> REGEX2 = /\p{Join_Control}/u
=> /\p{Join_Control}/
irb(main):021> "\u200D".gsub(REGEX2, "-")
=> "-"
```

There's 2 solutions here, either we change the docs or the code.




-- 
https://bugs.ruby-lang.org/
______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:122772] [Ruby Bug#21503] \p{Word} does not match on \p{Join_Control} while docs say it does

[k0kubun (Takashi Kokubun) via ruby-core]

Issue #21503 has been updated by k0kubun (Takashi Kokubun).


The patch to master modified `CR_Word` on `enc/unicode/16.0.0/name2ctype.h`, but Ruby 3.4 uses `enc/unicode/15.0.0/name2ctype.h` that has a different content in `CR_Word`. I'm not sure how to backport this properly. Could @procmarco or anybody else have a look at making a backport PR to ruby_3_4 branch on GitHub?

----------------------------------------
Bug #21503: \p{Word} does not match on \p{Join_Control} while docs say it does
https://bugs.ruby-lang.org/issues/21503#change-114051

* Author: procmarco (Marco Concetto Rudilosso)
* Status: Closed
* ruby -v: 3.4.4
* Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: REQUIRED
----------------------------------------
in the [docs](https://ruby-doc.org/3.4.1/Regexp.html#:~:text=/%5Cp%7B-,Word,-%7D/%3A%20A%20member) it is mentioned that `\p{Word}` matches the equivalent of: `[\p{M}\p{Nd}\p{Pc}\p{Alpha}\p{Join_Control}]` as it's also defined in the [unicode spec](https://unicode.org/reports/tr18/#word)

the issue is that it does not seem to be the case

```
irb(main):018> REGEX = /\p{Word}/u
=> /\p{Word}/
irb(main):019> "\u200D".gsub(REGEX, "-")
=> "‍"
irb(main):020> REGEX2 = /\p{Join_Control}/u
=> /\p{Join_Control}/
irb(main):021> "\u200D".gsub(REGEX2, "-")
=> "-"
```

There's 2 solutions here, either we change the docs or the code.




-- 
https://bugs.ruby-lang.org/
______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:123015] [Ruby Bug#21503] \p{Word} does not match on \p{Join_Control} while docs say it does

[naruse (Yui NARUSE) via ruby-core]

Issue #21503 has been updated by naruse (Yui NARUSE).


@k0kubun `name2ctype.h` is generated by `tool/enc-unicode.rb`. You can run it on ruby_3_4.

----------------------------------------
Bug #21503: \p{Word} does not match on \p{Join_Control} while docs say it does
https://bugs.ruby-lang.org/issues/21503#change-114319

* Author: procmarco (Marco Concetto Rudilosso)
* Status: Closed
* ruby -v: 3.4.4
* Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: REQUIRED
----------------------------------------
in the [docs](https://ruby-doc.org/3.4.1/Regexp.html#:~:text=/%5Cp%7B-,Word,-%7D/%3A%20A%20member) it is mentioned that `\p{Word}` matches the equivalent of: `[\p{M}\p{Nd}\p{Pc}\p{Alpha}\p{Join_Control}]` as it's also defined in the [unicode spec](https://unicode.org/reports/tr18/#word)

the issue is that it does not seem to be the case

```
irb(main):018> REGEX = /\p{Word}/u
=> /\p{Word}/
irb(main):019> "\u200D".gsub(REGEX, "-")
=> "‍"
irb(main):020> REGEX2 = /\p{Join_Control}/u
=> /\p{Join_Control}/
irb(main):021> "\u200D".gsub(REGEX2, "-")
=> "-"
```

There's 2 solutions here, either we change the docs or the code.




-- 
https://bugs.ruby-lang.org/
______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/

[ruby-core:123096] [Ruby Bug#21503] \p{Word} does not match on \p{Join_Control} while docs say it does

[alanwu (Alan Wu) via ruby-core]

Issue #21503 has been updated by alanwu (Alan Wu).

Backport changed from 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: REQUIRED to 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: DONE

Backport for 3.4 done in commit:5a42d267bfabc86f86cae2e83de24b1b86bc316a

----------------------------------------
Bug #21503: \p{Word} does not match on \p{Join_Control} while docs say it does
https://bugs.ruby-lang.org/issues/21503#change-114410

* Author: procmarco (Marco Concetto Rudilosso)
* Status: Closed
* ruby -v: 3.4.4
* Backport: 3.2: UNKNOWN, 3.3: UNKNOWN, 3.4: DONE
----------------------------------------
in the [docs](https://ruby-doc.org/3.4.1/Regexp.html#:~:text=/%5Cp%7B-,Word,-%7D/%3A%20A%20member) it is mentioned that `\p{Word}` matches the equivalent of: `[\p{M}\p{Nd}\p{Pc}\p{Alpha}\p{Join_Control}]` as it's also defined in the [unicode spec](https://unicode.org/reports/tr18/#word)

the issue is that it does not seem to be the case

```
irb(main):018> REGEX = /\p{Word}/u
=> /\p{Word}/
irb(main):019> "\u200D".gsub(REGEX, "-")
=> "‍"
irb(main):020> REGEX2 = /\p{Join_Control}/u
=> /\p{Join_Control}/
irb(main):021> "\u200D".gsub(REGEX2, "-")
=> "-"
```

There's 2 solutions here, either we change the docs or the code.




-- 
https://bugs.ruby-lang.org/
______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/