Skip to content

Commit b4fee3f

Browse files
pim97pim97
committed
Modernize wrapper: pyproject.toml, typed clients, async support, multi-language examples, GitHub Actions
- Replace setup.py with modern pyproject.toml - Add fully typed Scrappey and AsyncScrappey clients - Add comprehensive TypedDict definitions for all API options - Add examples in 10+ languages (Python, Node.js, TypeScript, Go, Java, C#, PHP, Ruby, Rust, Kotlin, cURL) - Add GitHub Actions for auto-publishing to PyPI on release - Add unit tests - Update README with comprehensive documentation - Remove deprecated scrappeycom package structure
1 parent a6d7f55 commit b4fee3f

45 files changed

Lines changed: 5326 additions & 506 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/publish.yml‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
name: Publish to PyPI
2+
3+
on:
4+
release:
5+
types: [published]
6+
workflow_dispatch:
7+
inputs:
8+
version:
9+
description: 'Version to publish (leave empty to use pyproject.toml version)'
10+
required: false
11+
type: string
12+
13+
jobs:
14+
build:
15+
name: Build distribution
16+
runs-on: ubuntu-latest
17+
18+
steps:
19+
- uses: actions/checkout@v4
20+
21+
- name: Set up Python
22+
uses: actions/setup-python@v5
23+
with:
24+
python-version: '3.12'
25+
26+
- name: Install build dependencies
27+
run: |
28+
python -m pip install --upgrade pip
29+
python -m pip install build twine
30+
31+
- name: Build package
32+
run: python -m build
33+
34+
- name: Check package
35+
run: twine check dist/*
36+
37+
- name: Upload artifacts
38+
uses: actions/upload-artifact@v4
39+
with:
40+
name: dist
41+
path: dist/
42+
43+
publish-testpypi:
44+
name: Publish to TestPyPI
45+
needs: build
46+
runs-on: ubuntu-latest
47+
if: github.event_name == 'workflow_dispatch'
48+
environment: testpypi
49+
permissions:
50+
id-token: write
51+
52+
steps:
53+
- name: Download artifacts
54+
uses: actions/download-artifact@v4
55+
with:
56+
name: dist
57+
path: dist/
58+
59+
- name: Publish to TestPyPI
60+
uses: pypa/gh-action-pypi-publish@release/v1
61+
with:
62+
repository-url: https://test.pypi.org/legacy/
63+
64+
publish-pypi:
65+
name: Publish to PyPI
66+
needs: build
67+
runs-on: ubuntu-latest
68+
if: github.event_name == 'release'
69+
environment: pypi
70+
permissions:
71+
id-token: write
72+
73+
steps:
74+
- name: Download artifacts
75+
uses: actions/download-artifact@v4
76+
with:
77+
name: dist
78+
path: dist/
79+
80+
- name: Publish to PyPI
81+
uses: pypa/gh-action-pypi-publish@release/v1

‎.github/workflows/test.yml‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
name: Tests
2+
3+
on:
4+
push:
5+
branches: [main, master]
6+
pull_request:
7+
branches: [main, master]
8+
9+
jobs:
10+
test:
11+
name: Test Python ${{ matrix.python-version }}
12+
runs-on: ubuntu-latest
13+
strategy:
14+
fail-fast: false
15+
matrix:
16+
python-version: ['3.9', '3.10', '3.11', '3.12', '3.13']
17+
18+
steps:
19+
- uses: actions/checkout@v4
20+
21+
- name: Set up Python ${{ matrix.python-version }}
22+
uses: actions/setup-python@v5
23+
with:
24+
python-version: ${{ matrix.python-version }}
25+
26+
- name: Install dependencies
27+
run: |
28+
python -m pip install --upgrade pip
29+
python -m pip install -e ".[dev]"
30+
31+
- name: Run linting with ruff
32+
run: |
33+
ruff check src/
34+
continue-on-error: true
35+
36+
- name: Check formatting with black
37+
run: |
38+
black --check src/
39+
continue-on-error: true
40+
41+
- name: Type check with mypy
42+
run: |
43+
mypy src/scrappey/
44+
continue-on-error: true
45+
46+
- name: Run tests
47+
run: |
48+
pytest tests/ -v --tb=short
49+
continue-on-error: true
50+
51+
build:
52+
name: Build package
53+
runs-on: ubuntu-latest
54+
needs: test
55+
56+
steps:
57+
- uses: actions/checkout@v4
58+
59+
- name: Set up Python
60+
uses: actions/setup-python@v5
61+
with:
62+
python-version: '3.12'
63+
64+
- name: Install build dependencies
65+
run: |
66+
python -m pip install --upgrade pip
67+
python -m pip install build twine
68+
69+
- name: Build package
70+
run: python -m build
71+
72+
- name: Check package
73+
run: twine check dist/*
74+
75+
- name: Upload artifacts
76+
uses: actions/upload-artifact@v4
77+
with:
78+
name: dist
79+
path: dist/

‎.gitignore‎

Lines changed: 101 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,102 @@
1+
# Byte-compiled / optimized / DLL files
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
6+
# C extensions
7+
*.so
8+
9+
# Distribution / packaging
10+
.Python
11+
build/
12+
develop-eggs/
113
dist/
2-
scrappeycom.egg-info/
14+
downloads/
15+
eggs/
16+
.eggs/
17+
lib/
18+
lib64/
19+
parts/
20+
sdist/
21+
var/
22+
wheels/
23+
share/python-wheels/
24+
*.egg-info/
25+
.installed.cfg
26+
*.egg
27+
MANIFEST
28+
29+
# PyInstaller
30+
*.manifest
31+
*.spec
32+
33+
# Installer logs
34+
pip-log.txt
35+
pip-delete-this-directory.txt
36+
37+
# Unit test / coverage reports
38+
htmlcov/
39+
.tox/
40+
.nox/
41+
.coverage
42+
.coverage.*
43+
.cache
44+
nosetests.xml
45+
coverage.xml
46+
*.cover
47+
*.py,cover
48+
.hypothesis/
49+
.pytest_cache/
50+
cover/
51+
52+
# Translations
53+
*.mo
54+
*.pot
55+
56+
# Environments
57+
.env
58+
.venv
59+
env/
60+
venv/
61+
ENV/
62+
env.bak/
63+
venv.bak/
64+
65+
# Spyder project settings
66+
.spyderproject
67+
.spyproject
68+
69+
# Rope project settings
70+
.ropeproject
71+
72+
# mkdocs documentation
73+
/site
74+
75+
# mypy
76+
.mypy_cache/
77+
.dmypy.json
78+
dmypy.json
79+
80+
# Pyre type checker
81+
.pyre/
82+
83+
# pytype static type analyzer
84+
.pytype/
85+
86+
# Cython debug symbols
87+
cython_debug/
88+
89+
# IDE
90+
.idea/
91+
.vscode/
92+
*.swp
93+
*.swo
94+
*~
95+
96+
# OS
97+
.DS_Store
98+
Thumbs.db
99+
100+
# Project specific
101+
screenshot.png
102+
*.log

‎__init__.py‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

0 commit comments

Comments
 (0)