Skip to content

Refactored code in controllers/bulk.js#27

Open
Mehulantony wants to merge 14 commits intomainfrom
refactor
Open

Refactored code in controllers/bulk.js#27
Mehulantony wants to merge 14 commits intomainfrom
refactor

Conversation

@Mehulantony
Copy link
Collaborator

Included 4 helper functions to reduce duplications and improve maintainability:

  1. setJsonHeaders - Standardizes JSON response headers by centralizing the "Content-Type" setting. Ensures consistent formatting in bulkCreate and bulkUpdate
  2. fail - A reusable helper function to generate and forward Express errors.
  3. requireNonEmptyArrayBody - Ensures the body is a non-empty array
  4. isValidJsonObject - Ensures each item is a valid non-array JSON object.

thehabes and others added 14 commits October 15, 2025 11:25
* Updates and refactor.  Dang, we cannot see the registered app routes like we used to.

* packages

* cleanup

* old logs

* cleanup for AI

* undiff for AI

* undiff for AI

* Reformat exists tests

* remove unused package

* Add in check for /overwrite exists

* cleanup

* cleanup

* Update __tests__/routes_mounted.test.js

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* cleanup

* cleanup

* cleanup

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Bring in search logic

* It searches!

* idNegotiation on search results

* All the search logic

* Lint, and add support for passing search options into the endpoint

* polish

* Update API documentation

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* polish

* exists test for new routes

* Update public/API.html

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update public/API.html

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update public/API.html

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update controllers/search.js

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update controllers/search.js

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* get rid of utils. prefix from createExpressError

* Update public/API.html

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update public/API.html

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* slop formatting

* Touch ups to API.html as discussed at standup.

* bump version because of new search feature

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Updated guidance to clarify that the file provides instructions for AI assistants rather than Claude Code.
Removed specific Bash command permissions from settings.
…anities#232)

* Initial plan

* Change slug default from empty string to null

Co-authored-by: cubap <1119165+cubap@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: cubap <1119165+cubap@users.noreply.github.com>
…igitalHumanities#234)

* Initial plan

* Optimize queries to use _id instead of @id for root object lookups

Co-authored-by: cubap <1119165+cubap@users.noreply.github.com>

* Add null check validation for primeID before parsing

Co-authored-by: cubap <1119165+cubap@users.noreply.github.com>

* Add null check for rootObj after database query

Co-authored-by: cubap <1119165+cubap@users.noreply.github.com>

* Remove duplicate getAllVersions function and improve error handling

Co-authored-by: cubap <1119165+cubap@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: cubap <1119165+cubap@users.noreply.github.com>
Co-authored-by: Bryan Haberberger <bryan.j.haberberger@slu.edu>
…italHumanities#235)

Updated the create and release controllers to assign the __rerum.slug property only when a slug value is present. This prevents undefined slugs from being set in the object metadata.
* Update packages, npm versions, and node versions

* the force

* the force

---------

Co-authored-by: Claude Code <claude@anthropic.com>
import controller from '../db-controller.js'

router.route('/')
.post(controller.searchAsWords)

Check failure

Code scanning / CodeQL

Missing rate limiting High

This route handler performs
a database access
, but is not rate-limited.
This route handler performs
a database access
, but is not rate-limited.

Copilot Autofix

AI 11 days ago

In general, this problem is fixed by adding a rate-limiting middleware to the affected routes so that each client (or IP) can only make a bounded number of requests over a time window. For Express applications, a standard solution is the express-rate-limit package, configured with sensible limits for the specific endpoints.

For this file, the best minimal fix is to import express-rate-limit, define a limiter specifically for search endpoints, and apply it to both POST routes (/ and /phrase) without changing the existing handler behavior. Concretely:

  • Add an import/require for express-rate-limit at the top of routes/search.js.
  • Define a searchLimiter instance with appropriate windowMs and max values (for example, 100 requests per 15 minutes per IP, mirroring the example from the background).
  • Apply searchLimiter to the route definitions, e.g., router.route('/').post(searchLimiter, controller.searchAsWords), and the same for /phrase.

All changes stay within routes/search.js. No changes to controller or other files are assumed or required.

Suggested changeset 2
routes/search.js

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/routes/search.js b/routes/search.js
--- a/routes/search.js
+++ b/routes/search.js
@@ -1,9 +1,15 @@
 import express from 'express'
 const router = express.Router()
 import controller from '../db-controller.js'
+import rateLimit from 'express-rate-limit'
 
+const searchLimiter = rateLimit({
+    windowMs: 15 * 60 * 1000, // 15 minutes
+    max: 100, // limit each IP to 100 search requests per windowMs
+})
+
 router.route('/')
-    .post(controller.searchAsWords)
+    .post(searchLimiter, controller.searchAsWords)
     .all((req, res, next) => {
         res.statusMessage = 'Improper request method for search.  Please use POST.'
         res.status(405)
@@ -11,7 +15,7 @@
     })
 
 router.route('/phrase')
-    .post(controller.searchAsPhrase)
+    .post(searchLimiter, controller.searchAsPhrase)
     .all((req, res, next) => {
         res.statusMessage = 'Improper request method for search.  Please use POST.'
         res.status(405)
EOF
@@ -1,9 +1,15 @@
import express from 'express'
const router = express.Router()
import controller from '../db-controller.js'
import rateLimit from 'express-rate-limit'

const searchLimiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // limit each IP to 100 search requests per windowMs
})

router.route('/')
.post(controller.searchAsWords)
.post(searchLimiter, controller.searchAsWords)
.all((req, res, next) => {
res.statusMessage = 'Improper request method for search. Please use POST.'
res.status(405)
@@ -11,7 +15,7 @@
})

router.route('/phrase')
.post(controller.searchAsPhrase)
.post(searchLimiter, controller.searchAsPhrase)
.all((req, res, next) => {
res.statusMessage = 'Improper request method for search. Please use POST.'
res.status(405)
package.json
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -37,7 +37,8 @@
     "express-oauth2-jwt-bearer": "~1.7.1",
     "express-urlrewrite": "~2.0.3",
     "mongodb": "^7.0.0",
-    "morgan": "~1.10.1"
+    "morgan": "~1.10.1",
+    "express-rate-limit": "^8.2.1"
   },
   "devDependencies": {
     "@jest/globals": "^30.2.0",
EOF
@@ -37,7 +37,8 @@
"express-oauth2-jwt-bearer": "~1.7.1",
"express-urlrewrite": "~2.0.3",
"mongodb": "^7.0.0",
"morgan": "~1.10.1"
"morgan": "~1.10.1",
"express-rate-limit": "^8.2.1"
},
"devDependencies": {
"@jest/globals": "^30.2.0",
This fix introduces these dependencies
Package Version Security advisories
express-rate-limit (npm) 8.2.1 None
Copilot is powered by AI and may make mistakes. Always verify output.
})

router.route('/phrase')
.post(controller.searchAsPhrase)

Check failure

Code scanning / CodeQL

Missing rate limiting High

This route handler performs
a database access
, but is not rate-limited.
This route handler performs
a database access
, but is not rate-limited.

Copilot Autofix

AI 11 days ago

In general, the problem is fixed by introducing rate limiting middleware in front of any route handler that performs expensive operations like database queries. In Express, this is commonly done using a middleware such as express-rate-limit, configured with an appropriate window length and maximum request count, and then attached either to the whole app, a router, or specific routes.

For this code, the least intrusive and clearest fix is to add a rate limiter specifically for the search routes in routes/search.js. Because we can only modify this file, we will: (1) import express-rate-limit; (2) define a limiter instance with reasonable defaults (for example, a 15-minute window and 100 requests per IP, matching the background example); and (3) apply that limiter to the relevant routes. The most straightforward way, without changing existing behavior of handlers, is to apply router.use(searchLimiter) so it protects both '/' and '/phrase' search routes, or to apply it individually to each route definition. Applying it to the whole router has the benefit of also covering any future search routes defined here without further changes.

Concretely:

  • At the top of routes/search.js, after the existing imports, add import rateLimit from 'express-rate-limit'.
  • Define a const searchLimiter = rateLimit({ ... }) with configuration comments so its behavior is clear and can be tuned later.
  • Attach router.use(searchLimiter) after the limiter definition and before the route definitions, so all routes in this router are rate-limited.
    This preserves all existing route behavior (methods, status codes, error handling) while adding protection against abuse.
Suggested changeset 2
routes/search.js

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/routes/search.js b/routes/search.js
--- a/routes/search.js
+++ b/routes/search.js
@@ -1,7 +1,17 @@
 import express from 'express'
 const router = express.Router()
 import controller from '../db-controller.js'
+import rateLimit from 'express-rate-limit'
 
+// Rate limiter for search routes to mitigate abuse and DoS via expensive DB queries.
+const searchLimiter = rateLimit({
+    windowMs: 15 * 60 * 1000, // 15 minutes
+    max: 100,                 // limit each IP to 100 search requests per window
+})
+
+// Apply rate limiting to all routes in this router.
+router.use(searchLimiter)
+
 router.route('/')
     .post(controller.searchAsWords)
     .all((req, res, next) => {
EOF
@@ -1,7 +1,17 @@
import express from 'express'
const router = express.Router()
import controller from '../db-controller.js'
import rateLimit from 'express-rate-limit'

// Rate limiter for search routes to mitigate abuse and DoS via expensive DB queries.
const searchLimiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // limit each IP to 100 search requests per window
})

// Apply rate limiting to all routes in this router.
router.use(searchLimiter)

router.route('/')
.post(controller.searchAsWords)
.all((req, res, next) => {
package.json
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -37,7 +37,8 @@
     "express-oauth2-jwt-bearer": "~1.7.1",
     "express-urlrewrite": "~2.0.3",
     "mongodb": "^7.0.0",
-    "morgan": "~1.10.1"
+    "morgan": "~1.10.1",
+    "express-rate-limit": "^8.2.1"
   },
   "devDependencies": {
     "@jest/globals": "^30.2.0",
EOF
@@ -37,7 +37,8 @@
"express-oauth2-jwt-bearer": "~1.7.1",
"express-urlrewrite": "~2.0.3",
"mongodb": "^7.0.0",
"morgan": "~1.10.1"
"morgan": "~1.10.1",
"express-rate-limit": "^8.2.1"
},
"devDependencies": {
"@jest/globals": "^30.2.0",
This fix introduces these dependencies
Package Version Security advisories
express-rate-limit (npm) 8.2.1 None
Copilot is powered by AI and may make mistakes. Always verify output.
@joeljoby02
Copy link
Collaborator

  • The new commits are well structured.
  • The changes introduced here improve correctness and robustness, especially around input validation.
  • One area to consider for future improvement is reducing duplicated validation logic across bulk operations. While the current implementation is clear, extracting shared checks into a common helper would improve maintainability and reduce the risk of inconsistencies as additional bulk features are added.

err.message = "All objects in the body of a `/bulkCreate` must be JSON and must not contain a declared identifier property."
err.status = 400
next(createExpressError(err))
fail(next, "All objects in the body of a `/bulkCreate` must be JSON and must not contain a declared identifier property.", 400)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could further enhance the error message by pointing to which object caused the error through indexing or any other valid method

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

Comments