Issue #22229 has been updated by mdalessio (Mike Dalessio). I would also like to have this feature. Loofah (used by Rails's sanitization stack) currently has this code to re-implement `escapeHTML` with a custom table: https://github.com/flavorjones/loofah/blob/main/lib/loofah/html5/scrub.rb#L2..., as does the Rouge gem (and probably others). ---------------------------------------- Feature #22229: Allow GCI.escapeHTML to take a custom escape table https://bugs.ruby-lang.org/issues/22229#change-118360 * Author: byroot (Jean Boussier) * Status: Open ---------------------------------------- ### Use case `CGI.escapeHTML` has a fixed escape table: ```c HTML_ESCAPE('\'', "'"), HTML_ESCAPE('&', "&"), HTML_ESCAPE('"', """), HTML_ESCAPE('<', "<"), HTML_ESCAPE('>', ">"), ``` But in some context you may want to escape more or less characters than that, or escape them differently. One example of this is Active Support JSON serialization, which by defaults escape `<>&` as `\u003e\u003c\u0026`, to ensure that the generated JSON can safely be interpolated inside a `<script>` tag without causing XSS vulnerabilities. There's likely other use case as evidenced by the popularity of the `htmlentities` gem, which support several more escape tables. ### Why not `gsub` ? Today, this sort of escaping is performed using `gsub`, it works but is very noticeably slower than `CGI.escapeHTML` (see benchmarks on the PoC PR). This is because the only way to use `gsub` with an escape table is to craft a regexp: ```ruby def escape(string, table) pattern = Regexp.union(table.keys) string.gsub(pattern, table) end puts escape("<script>", "<" => "<", ">" => ">") ``` If `gsub` could be used directly with the escape table it would certainly make it faster: ```ruby puts "<script>".gsub("<" => "<", ">" => ">") ``` But compared to `CGI.escapeHTML` it would still need to deal with noticeably more scenarios (multi character search, etc), so would always be slower than a dedicated escaping method. `CGI.escapeHTML` can be more restrictive (single character search, etc). ### Specification ```ruby GCI.escapeHTML("</script>", "<" => "<", ">" => ">") ``` - The escape table keys must be single characters (currently ASCII only, but could be muti-byte characters if deemed necessary). ### Implementation Pull Request: https://github.com/ruby/cgi/pull/55 (NB: it's at proof of concept / demo stage, if the feature is accepted I can polish it). -- https://bugs.ruby-lang.org/

Issue #22229 has been updated by Eregon (Benoit Daloze). If there is any doubt that `gsub` can be faster, here is my blog post about it: https://eregon.me/blog/2025/03/14/matching-regexps-200-times-faster.html Notably matching could be about 200 times faster. ---------------------------------------- Feature #22229: Allow GCI.escapeHTML to take a custom escape table https://bugs.ruby-lang.org/issues/22229#change-118362 * Author: byroot (Jean Boussier) * Status: Open ---------------------------------------- ### Use case `CGI.escapeHTML` has a fixed escape table: ```c HTML_ESCAPE('\'', "'"), HTML_ESCAPE('&', "&"), HTML_ESCAPE('"', """), HTML_ESCAPE('<', "<"), HTML_ESCAPE('>', ">"), ``` But in some context you may want to escape more or less characters than that, or escape them differently. One example of this is Active Support JSON serialization, which by defaults escape `<>&` as `\u003e\u003c\u0026`, to ensure that the generated JSON can safely be interpolated inside a `<script>` tag without causing XSS vulnerabilities. There's likely other use case as evidenced by the popularity of the `htmlentities` gem, which support several more escape tables. ### Why not `gsub` ? Today, this sort of escaping is performed using `gsub`, it works but is very noticeably slower than `CGI.escapeHTML` (see benchmarks on the PoC PR). This is because the only way to use `gsub` with an escape table is to craft a regexp: ```ruby def escape(string, table) pattern = Regexp.union(table.keys) string.gsub(pattern, table) end puts escape("<script>", "<" => "<", ">" => ">") ``` If `gsub` could be used directly with the escape table it would certainly make it faster: ```ruby puts "<script>".gsub("<" => "<", ">" => ">") ``` But compared to `CGI.escapeHTML` it would still need to deal with noticeably more scenarios (multi character search, etc), so would always be slower than a dedicated escaping method. `CGI.escapeHTML` can be more restrictive (single character search, etc). ### Specification ```ruby GCI.escapeHTML("</script>", "<" => "<", ">" => ">") ``` - The escape table keys must be single characters (currently ASCII only, but could be muti-byte characters if deemed necessary). ### Implementation Pull Request: https://github.com/ruby/cgi/pull/55 (NB: it's at proof of concept / demo stage, if the feature is accepted I can polish it). -- https://bugs.ruby-lang.org/

Issue #22229 has been updated by Eregon (Benoit Daloze). byroot (Jean Boussier) wrote in #note-5:

I made the point that by being limited to single characters, `escapeHTML` can be much easier to optimize much further.

That's something Regexp can optimize for and maybe already does to some extent, it can see which characters are in a character set and if they are all 1-byte, etc. That's a great use case for SIMD, which Regexp should definitely use as it'd make all Regexps faster (if it doesn't already do that). byroot (Jean Boussier) wrote in #note-5:

please see the benchmark at [https://github.com/ruby/cgi/pull/55](https://github.com/ruby/cgi/pull/55), it doesn't compile a regexp every time and there's still a 5x difference.

Interestingly, 2.83x when there is nothing to replace. So from that it sounds a bit like `gsub` is quite a bit slower to replace than `escapeHTML`. Yet it should be about the same since they both need to lookup the passed `Hash` when replacing. That tells me there is something in `gsub` about replacing which is slower than it should be, independent of matching. Looking at the PR diff it creates the escape table every time, that's the part that feels to me like this is not the right approach, because Regexp can build that escape table at compile time vs on every call. `gsub` can't associate the given replacement with the escape table, but either way there are Hash lookups on every call if there are replacements. ---------------------------------------- Feature #22229: Allow GCI.escapeHTML to take a custom escape table https://bugs.ruby-lang.org/issues/22229#change-118366 * Author: byroot (Jean Boussier) * Status: Open ---------------------------------------- ### Use case `CGI.escapeHTML` has a fixed escape table: ```c HTML_ESCAPE('\'', "'"), HTML_ESCAPE('&', "&"), HTML_ESCAPE('"', """), HTML_ESCAPE('<', "<"), HTML_ESCAPE('>', ">"), ``` But in some context you may want to escape more or less characters than that, or escape them differently. One example of this is Active Support JSON serialization, which by defaults escape `<>&` as `\u003e\u003c\u0026`, to ensure that the generated JSON can safely be interpolated inside a `<script>` tag without causing XSS vulnerabilities. There's likely other use case as evidenced by the popularity of the `htmlentities` gem, which support several more escape tables. ### Why not `gsub` ? Today, this sort of escaping is performed using `gsub`, it works but is very noticeably slower than `CGI.escapeHTML` (see benchmarks on the PoC PR). This is because the only way to use `gsub` with an escape table is to craft a regexp: ```ruby def escape(string, table) pattern = Regexp.union(table.keys) string.gsub(pattern, table) end puts escape("<script>", "<" => "<", ">" => ">") ``` If `gsub` could be used directly with the escape table it would certainly make it faster: ```ruby puts "<script>".gsub("<" => "<", ">" => ">") ``` But compared to `CGI.escapeHTML` it would still need to deal with noticeably more scenarios (multi character search, etc), so would always be slower than a dedicated escaping method. `CGI.escapeHTML` can be more restrictive (single character search, etc). ### Specification ```ruby GCI.escapeHTML("</script>", "<" => "<", ">" => ">") ``` - The escape table keys must be single characters (currently ASCII only, but could be muti-byte characters if deemed necessary). ### Implementation Pull Request: https://github.com/ruby/cgi/pull/55 (NB: it's at proof of concept / demo stage, if the feature is accepted I can polish it). -- https://bugs.ruby-lang.org/