Elaborate scripting flag and text encoding.

Author: dmsnell

src/wp-includes/html-api/class-wp-html-tag-processor.php

@@ -288,7 +288,31 @@
  * double-quoted strings, meaning that attributes on input with single-quoted or
  * unquoted values will appear in the output with double-quotes.
  *
- * scripts aren't processed
+ * ### Scripting Flag
+ *
+ * The Tag Processor parses HTML with the "scripting flag" disabled. This means
+ * that it doesn't run any scripts while parsing the page. In a browser with
+ * JavaScript enabled, for example, the script can change the parse of the
+ * document as it loads. On the server, however, evaluating JavaScript is not
+ * only impractical, but also unwanted.
+ *
+ * Practically this means that the Tag Processor will descend into NOSCRIPT
+ * elements and process its child tags. Were the scripting flag enabled, such
+ * as in a typical browser, the contents of NOSCRIPT are skipped entirely.
+ *
+ * This allows the HTML API to process the content that will be presented in
+ * a browser when scripting is disabled, but it offers a different view of a
+ * page than most browser sessions will experience. E.g. the tags inside the
+ * NOSCRIPT disappear.
+ *
+ * ### Text Encoding
+ *
+ * The Tag Processor assumes that the input HTML document is encoded with a
+ * text encoding compatible with 7-bit ASCII. These include but are not
+ * limited to ISO-8859-1 (latin1), CP-1250, UTF-8. If provided a UTF-16 byte
+ * stream or another encoding which does not preserve the 7-bit ASCII
+ * characters as a single byte matching the 7-bit ASCII bytes, then it will
+ * not be able to properly parse the document.
  *
  * @since 6.2.0
  * @since 6.2.1 Fix: Support for various invalid comments; attribute updates are case-insensitive.