Skip to content

Commit 7193549

Browse files
StoneHubStoneHub
authored
Merge pull request #1 from StoneHub/claude/add-bot-protection-7VGJK
feat: add comprehensive bot protection
2 parents 42733cd + 65ac3ec commit 7193549

13 files changed

Lines changed: 1278 additions & 1072 deletions

‎.env‎

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,11 @@
1-
# Wikipedia API Configuration
2-
# Replace securely in your hosting dashboard (Vercel/Netlify)
3-
4-
# Your contact email or project URL (Required by Wikipedia Policy)
5-
# Example: WikiWebMap/1.0 (https://github.com/your-repo; your-email@example.com)
6-
VITE_WIKI_CONTACT= WikiWebMap/1.0 (https://github.com/StoneHub/WikiWebMap.git; flyingchangesfarm438@gmail.com)
1+
# Wikipedia API Configuration
2+
# Replace securely in your hosting dashboard (Vercel/Netlify)
3+
4+
# Your contact email or project URL (Required by Wikipedia Policy)
5+
# Example: WikiWebMap/1.0 (https://github.com/your-repo; your-email@example.com)
6+
VITE_WIKI_CONTACT= WikiWebMap/1.0 (https://github.com/StoneHub/WikiWebMap.git; flyingchangesfarm438@gmail.com)
7+
8+
# reCAPTCHA v3 Configuration
9+
# Get your site key from: https://www.google.com/recaptcha/admin
10+
# Current key is a TEST key - replace with your actual key for production
11+
VITE_RECAPTCHA_SITE_KEY=6LeFqi0sAAAAAFtOhr-p-WVjbTvKe7XVdwAc_2aR

‎.env.example‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,5 @@
1-
# Optional: contact email to include in the Wikipedia User-Agent.
2-
# VITE_WIKI_API_CONTACT_EMAIL=you@example.com
1+
# Optional: contact email to include in the Wikipedia User-Agent.
2+
# VITE_WIKI_API_CONTACT_EMAIL=you@example.com
3+
4+
# reCAPTCHA v3 Site Key (Get from https://www.google.com/recaptcha/admin)
5+
VITE_RECAPTCHA_SITE_KEY=your_recaptcha_site_key_here

‎index.html‎

Lines changed: 15 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,15 @@
1-
<!doctype html>
2-
<html lang="en">
3-
<head>
4-
<meta charset="UTF-8" />
5-
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
6-
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
7-
<title>WikiWeb Explorer - Interactive Wikipedia Knowledge Graph</title>
8-
</head>
9-
<body class="m-0 p-0 overflow-hidden">
10-
<div id="root"></div>
11-
<script type="module" src="/src/main.tsx"></script>
12-
</body>
13-
</html>
1+
<!doctype html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="UTF-8" />
5+
<link rel="icon" type="image/svg+xml" href="/vite.svg" />
6+
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
7+
<title>WikiWeb Explorer - Interactive Wikipedia Knowledge Graph</title>
8+
<!-- reCAPTCHA v3 for bot protection -->
9+
<script src="https://www.google.com/recaptcha/api.js?render=6LeFqi0sAAAAAFtOhr-p-WVjbTvKe7XVdwAc_2aR" async defer></script>
10+
</head>
11+
<body class="m-0 p-0 overflow-hidden">
12+
<div id="root"></div>
13+
<script type="module" src="/src/main.tsx"></script>
14+
</body>
15+
</html>

‎public/robots.txt‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# WikiWebMap - Robots.txt
2+
# This file helps prevent automated bot abuse
3+
4+
User-agent: *
5+
# Allow general crawling of the site
6+
Allow: /
7+
8+
# Reasonable crawl delay to prevent aggressive scraping
9+
Crawl-delay: 2
10+
11+
# Block known bad bots and scrapers
12+
User-agent: AhrefsBot
13+
User-agent: SemrushBot
14+
User-agent: DotBot
15+
User-agent: MJ12bot
16+
User-agent: BLEXBot
17+
User-agent: PetalBot
18+
Disallow: /
19+
20+
# Sitemap (you can add this later if needed)
21+
# Sitemap: https://wikiconnectionsmap.web.app/sitemap.xml

0 commit comments

Comments
 (0)