Refactor/pagefind search (#2879)

* Add pagefind search result utility files

* Remvove pagefind-ui.css & load pagefind.js dynamically

No longer a need for pagefind-ui.css as we will be referencing our own
custom CSS instead. Likewise, we no longern eed PagefindUI's js.

Dynamically imports pagefind by lazy loading it each time the search
opens.

* Fix import path in page.njk

* Replace PagefindUI with pagefind.js API in Search.vue

Removed deep() overrides with custom css
Update keyboard navigation using selectedIndex state
Remove MutationObserver and innerHTML hacky solution
Implemented custom result rendering

* Update styling for Search.vue

* Update pagefind search utils to process subresult status

Included an additional field subResult fields in FormattedSearchResult
to detect whether the result if a subResult and if it is the last
subResult.

Updated Search.vue results to indicate if a result if a main result or a
search result with svgs. Svgs were taken from algolia's design on the
cs2103 website.

* Fix bug first index sub result and Clean search.css

The occured as the first sub result would be essentially the same as the
main result. This is because our page names and their first heading are
usually the same contributing to this error. For example,
makingTheSiteSearchable.html is the same as
makingTheSiteSearchable.html#making-the-site-searchable.

Hence, by checking of the sub result's title is the same as the main
result's title, we can identify which sub result to not include when
displaying the data.

Also search.css by removing accidentally duplicated code and
restructuring the order of the class selectors in a more logical manner.
Also included more comments explaining and breaking down the classes
into different sections for easier understandability. Word lol

* Update styling for mark & font sizes

* Fix mark component not appearing in excerpt bug

Some of the marked components were not appearing as they were being cut
off within the elipsis due to the length of the excerpt displayed.

Hence, instead, it now processes the excerpt length to display a maximum
of 15 characters before it to ensure that the marked components are
always included.

* Add nowrap for result titles styling

This was mainly discovered because I found a bug where it seems that
pagefind is indexing the headers generated within generated
<question></question>. As a result, since these titles are extremely
long, it ended up completely hiding the excerpt part of the
search-result-item.

Hence, this warrants a change by first ensuring that nowrap on the title
to ensure that it only displays the first part of the header.

And also, I need to raise a issue for the bug as I don't think we should
be letting pagefind index these generated headers.

* Add merging of consecutive marked terms

This will ensure that they are displayed as one segment instead of 2
distinct segements when the search terms are next to each other within
the excerpt.

* Fix bug storing only duplicated sub-results not unique

Previous it was adding duplicated sub-results instead of unqiue ones.
E.g.when searching "refactoring" in 2103 site, it was would store 10
(the count limt) copies of the "What" subresult instead of the other
sub-sections (How, When, etc).

This happened because it was iterating through each weighted location
(i.e. the search term position). And for each, it would select the
sub-result with the most context, after which it would push the it into
the sub array without checking for duplicates. Hence, using the 2103
site example from before, there were >=10 instances of "Refactoring" in
"What" section, it was interating through these instances until it
reached the 10 count limit. After which, the filters to remove the
duplicated entries would result in only mention of the "What" section
being displayed, which meant that it would only display "Refactoring"
with 1 sub result -> "What" section.

Hence, to fix this, I added a duplicate check before pushing subs array.
Now we will only store the FIRST instance of each sub-result. Hence
"What" is stored once, and it will allow the correct storing of the
other sections as well. This ensures that we can display the actual
maximum of 10 sub-results.

* Add more DocSearch styling

* Use score instead of weight when sorting

* Create testcases for searchUtils

* Rehaul tests for Search.vue

* Update functional tests

* Update functional test (again)

* Export as types & used numeric check

* Resolve Search.vue code quality issues

* Create pagefind-lazloader.js & update functional tests

* Fix assets manager bug

* Update functional tests

* Fix weight order issue

Author: MoshiMoshiMochi

.eslintignore

@@ -22,6 +22,7 @@ packages/core/src/lib/markdown-it/highlight/*.js
 packages/core/src/lib/markdown-it/plugins/**/*.js
 packages/core/src/lib/markdown-it/utils/*.js
 packages/core/src/Page/*.js
+packages/core/src/Pagefind/*.js
 packages/core/src/patches/**/*.js
 packages/core/src/plugins/**/*.js
 packages/core/src/Site/*.js

.eslintrc.js

@@ -2,7 +2,11 @@
 /* eslint quotes: ["error", "double"] */
 
 module.exports = {
-  "ignorePatterns": ["docs/_site/**", "**/dist/**", "**/node_modules/**"],
+  "ignorePatterns": [
+    "docs/_site/**",
+    "**/dist/**", "**/node_modules/**",
+    "packages/core-web/asset/js/pagefind-lazyloader.js",
+  ],
   "env": {
     "node": true,
     "es6": true,

.gitignore

@@ -98,6 +98,7 @@ packages/core/src/lib/markdown-it/highlight/*.js
 packages/core/src/lib/markdown-it/plugins/**/*.js
 packages/core/src/lib/markdown-it/utils/*.js
 packages/core/src/Page/*.js
+packages/core/src/Pagefind/*.js
 packages/core/src/plugins/**/*.js
 packages/core/src/Site/*.js
 packages/core/src/utils/*.js

packages/cli/test/functional/test_site/expected/bugs/index.html

@@ -16,7 +16,6 @@
   <link rel="stylesheet" href="/test_site/markbind/glyphicons/css/bootstrap-glyphicons.min.css">
   <link rel="stylesheet" href="/test_site/markbind/css/codeblock-dark.min.css">
   <link rel="stylesheet" href="/test_site/markbind/css/markbind.min.css">
-  <link rel="stylesheet" href="/test_site/markbind/pagefind/pagefind-ui.css">
   <link rel="stylesheet" href="/test_site/plugins/testMarkbindPlugin/testMarkbindPluginStylesheet.css">
   <link rel="stylesheet" href="/test_site/plugins/web3Form/web-3-form.css">
   <link rel="stylesheet" href="/test_site/plugins/markbind-plugin-anchors/markbind-plugin-anchors.css">
@@ -360,6 +359,6 @@ <h1 id="heading-in-footer-should-not-be-indexed">Heading in footer should not be
     });
 
   </script>
-<script src="/test_site/markbind/pagefind/pagefind-ui.js"></script>
+<script src="/test_site/markbind/js/pagefind-lazyloader.js"></script>
 
 </html>

packages/cli/test/functional/test_site/expected/index.html

@@ -16,7 +16,6 @@
   <link rel="stylesheet" href="/test_site/markbind/glyphicons/css/bootstrap-glyphicons.min.css">
   <link rel="stylesheet" href="/test_site/markbind/css/codeblock-dark.min.css">
   <link rel="stylesheet" href="/test_site/markbind/css/markbind.min.css">
-  <link rel="stylesheet" href="/test_site/markbind/pagefind/pagefind-ui.css">
   <link rel="stylesheet" href="/test_site/plugins/testMarkbindPlugin/testMarkbindPluginStylesheet.css">
   <link rel="stylesheet" href="/test_site/plugins/web3Form/web-3-form.css">
   <link rel="stylesheet" href="/test_site/plugins/markbind-plugin-anchors/markbind-plugin-anchors.css">
@@ -1026,6 +1025,6 @@ <h1 id="heading-in-footer-should-not-be-indexed">Heading in footer should not be
     });
 
   </script>
-<script src="/test_site/markbind/pagefind/pagefind-ui.js"></script>
+<script src="/test_site/markbind/js/pagefind-lazyloader.js"></script>
 
 </html>

packages/cli/test/functional/test_site/expected/markbind/js/pagefind-lazyloader.js

@@ -0,0 +1,17 @@
+/**
+ * Lazy loader for Pagefind search functionality.
+ * This script dynamically imports pagefind.js only when needed (when user opens search).
+ * This avoids loading the search library on initial page load.
+ */
+window.__pagefind__ = null;
+
+window.loadPagefind = async () => {
+  if (!window.__pagefind__) {
+    // baseUrl is defined in page.njk script tag
+    const baseUrl = window.baseUrl || '';
+    const module = await import(`${baseUrl}/markbind/pagefind/pagefind.js`);
+    window.__pagefind__ = module;
+    module.init();
+  }
+  return window.__pagefind__;
+};

packages/cli/test/functional/test_site/expected/sub_site/index.html

@@ -15,7 +15,6 @@
   <link rel="stylesheet" href="/test_site/markbind/glyphicons/css/bootstrap-glyphicons.min.css">
   <link rel="stylesheet" href="/test_site/markbind/css/codeblock-dark.min.css">
   <link rel="stylesheet" href="/test_site/markbind/css/markbind.min.css">
-  <link rel="stylesheet" href="/test_site/markbind/pagefind/pagefind-ui.css">
   <link rel="stylesheet" href="/test_site/plugins/testMarkbindPlugin/testMarkbindPluginStylesheet.css">
   <link rel="stylesheet" href="/test_site/plugins/web3Form/web-3-form.css">
   <link rel="stylesheet" href="/test_site/plugins/markbind-plugin-anchors/markbind-plugin-anchors.css">
@@ -367,6 +366,6 @@ <h1 id="heading-in-footer-should-not-be-indexed">Heading in footer should not be
     });
 
   </script>
-<script src="/test_site/markbind/pagefind/pagefind-ui.js"></script>
+<script src="/test_site/markbind/js/pagefind-lazyloader.js"></script>
 
 </html>

packages/cli/test/functional/test_site/expected/sub_site/nested_sub_site/index.html

@@ -15,7 +15,6 @@
   <link rel="stylesheet" href="/test_site/markbind/glyphicons/css/bootstrap-glyphicons.min.css">
   <link rel="stylesheet" href="/test_site/markbind/css/codeblock-dark.min.css">
   <link rel="stylesheet" href="/test_site/markbind/css/markbind.min.css">
-  <link rel="stylesheet" href="/test_site/markbind/pagefind/pagefind-ui.css">
   <link rel="stylesheet" href="/test_site/plugins/testMarkbindPlugin/testMarkbindPluginStylesheet.css">
   <link rel="stylesheet" href="/test_site/plugins/web3Form/web-3-form.css">
   <link rel="stylesheet" href="/test_site/plugins/markbind-plugin-anchors/markbind-plugin-anchors.css">
@@ -358,6 +357,6 @@ <h1 id="heading-in-footer-should-not-be-indexed">Heading in footer should not be
     });
 
   </script>
-<script src="/test_site/markbind/pagefind/pagefind-ui.js"></script>
+<script src="/test_site/markbind/js/pagefind-lazyloader.js"></script>
 
 </html>

packages/cli/test/functional/test_site/expected/sub_site/nested_sub_site/testNunjucksPathResolving.html

@@ -16,7 +16,6 @@
   <link rel="stylesheet" href="/test_site/markbind/glyphicons/css/bootstrap-glyphicons.min.css">
   <link rel="stylesheet" href="/test_site/markbind/css/codeblock-dark.min.css">
   <link rel="stylesheet" href="/test_site/markbind/css/markbind.min.css">
-  <link rel="stylesheet" href="/test_site/markbind/pagefind/pagefind-ui.css">
   <link rel="stylesheet" href="/test_site/plugins/testMarkbindPlugin/testMarkbindPluginStylesheet.css">
   <link rel="stylesheet" href="/test_site/plugins/web3Form/web-3-form.css">
   <link rel="stylesheet" href="/test_site/plugins/markbind-plugin-anchors/markbind-plugin-anchors.css">
@@ -364,6 +363,6 @@ <h1 id="heading-in-footer-should-not-be-indexed">Heading in footer should not be
     });
 
   </script>
-<script src="/test_site/markbind/pagefind/pagefind-ui.js"></script>
+<script src="/test_site/markbind/js/pagefind-lazyloader.js"></script>
 
 </html>

packages/cli/test/functional/test_site/expected/sub_site/testNunjucksPathResolving.html

@@ -16,7 +16,6 @@
   <link rel="stylesheet" href="/test_site/markbind/glyphicons/css/bootstrap-glyphicons.min.css">
   <link rel="stylesheet" href="/test_site/markbind/css/codeblock-dark.min.css">
   <link rel="stylesheet" href="/test_site/markbind/css/markbind.min.css">
-  <link rel="stylesheet" href="/test_site/markbind/pagefind/pagefind-ui.css">
   <link rel="stylesheet" href="/test_site/plugins/testMarkbindPlugin/testMarkbindPluginStylesheet.css">
   <link rel="stylesheet" href="/test_site/plugins/web3Form/web-3-form.css">
   <link rel="stylesheet" href="/test_site/plugins/markbind-plugin-anchors/markbind-plugin-anchors.css">
@@ -364,6 +363,6 @@ <h1 id="heading-in-footer-should-not-be-indexed">Heading in footer should not be
     });
 
   </script>
-<script src="/test_site/markbind/pagefind/pagefind-ui.js"></script>
+<script src="/test_site/markbind/js/pagefind-lazyloader.js"></script>
 
 </html>