Skip to content

Commit 4439137

Browse files
author
Max
committed
feat: steam market scraper.
0 parents  commit 4439137

12 files changed

Lines changed: 577 additions & 0 deletions

File tree

‎.flake8‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
[flake8]
2+
max-line-length = 79
3+
import-order-style = google
4+
exclude = venv/, env/, .git, __pycache__, venv
5+
max-complexity = 10
6+
max-expression-complexity = 7
7+
8+
[flake8-quotes]
9+
quote-style = single

‎.github/workflows/lint.yml‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
name: lint
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
pull_request:
8+
branches:
9+
- main
10+
11+
jobs:
12+
black:
13+
runs-on: ubuntu-latest
14+
steps:
15+
- name: Checkout code
16+
uses: actions/checkout@v2
17+
18+
- name: Set up Python
19+
uses: actions/setup-python@v2
20+
with:
21+
python-version: '3.13'
22+
23+
- name: Install dependencies
24+
run: |
25+
python -m pip install --upgrade pip
26+
pip install -r requirements/dev.txt
27+
28+
- name: List installed packages
29+
run: pip list
30+
31+
- name: Check Black version
32+
run: black --version
33+
34+
- name: Run Black
35+
run: |
36+
black . --check --verbose --
37+
38+
isort:
39+
runs-on: ubuntu-latest
40+
steps:
41+
- name: Checkout code
42+
uses: actions/checkout@v2
43+
44+
- name: Set up Python
45+
uses: actions/setup-python@v2
46+
with:
47+
python-version: '3.13'
48+
49+
- name: Install dependencies
50+
run: |
51+
python -m pip install --upgrade pip
52+
pip install -r requirements/dev.txt
53+
54+
- name: List installed packages
55+
run: pip list
56+
57+
- name: Check Isort version
58+
run: isort --version
59+
60+
- name: Run Isort
61+
run: |
62+
isort . --verbose
63+
64+
flake8:
65+
runs-on: ubuntu-latest
66+
steps:
67+
- name: Checkout code
68+
uses: actions/checkout@v2
69+
70+
- name: Set up Python
71+
uses: actions/setup-python@v2
72+
with:
73+
python-version: '3.13'
74+
75+
- name: Install dependencies
76+
run: |
77+
python -m pip install --upgrade pip
78+
pip install -r requirements/dev.txt
79+
80+
- name: List installed packages
81+
run: pip list
82+
83+
- name: Check Flake8 version
84+
run: flake8 --version
85+
86+
- name: Run Flake8
87+
run: |
88+
flake8 . --count --show-source --statistics

‎.gitignore‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# IDEs
2+
.vscode/
3+
.idea/
4+
5+
6+
# Environments
7+
.env
8+
.venv
9+
env/
10+
venv/
11+
ENV/
12+
env.bak/
13+
venv.bak/
14+
15+
16+
# Distribution / packaging
17+
.Python
18+
build/
19+
develop-eggs/
20+
dist/
21+
downloads/
22+
eggs/
23+
.eggs/
24+
lib/
25+
lib64/
26+
parts/
27+
sdist/
28+
var/
29+
wheels/
30+
share/python-wheels/
31+
*.egg-info/
32+
.installed.cfg
33+
*.egg
34+
MANIFEST
35+
36+
# Byte-compiled / optimized / DLL files
37+
__pycache__/
38+
*.py[cod]
39+
*$py.class
40+
41+
42+
# C extensions
43+
*.so

‎.isort.cfg‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
[settings]
2+
profile = black
3+
skip= venv/
4+
default_section = THIRDPARTY
5+
force_sort_within_sections = true
6+
line_length = 79

‎README.md‎

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
# Steam Market Scraper Parser
2+
<p>
3+
<a
4+
href='https://github.com/RandomProgramm3r/Steam-Market-Scraper/actions/workflows/lint.yml/'>
5+
<img
6+
src='https://github.com/RandomProgramm3r/Steam-Market-Scraper/actions/workflows/lint.yml/badge.svg'
7+
alt='pipeline status'
8+
height='30'
9+
width='30'>
10+
</a>
11+
<a
12+
href='https://steamcommunity.com/market/'>
13+
<img
14+
src='https://steamcommunity.com/favicon.ico'
15+
alt='Steam logo'
16+
height='30'
17+
width='30'>
18+
</a>
19+
</p>
20+
21+
22+
## TOC
23+
24+
- [📋 Description](#-description)
25+
- [💻 Requirements](#-requirements)
26+
- [🚀 Project Installation](#-project-installation)
27+
- [📂 Step 1: Clone the Repository](#-step-1-clone-the-repository)
28+
- [🖥 Step 2: Create and activate a virtual environment](#-step-2-create-and-activate-a-virtual-environment)
29+
- [🔃 Step 3: Installing Dependencies](#-step-3-installing-dependencies)
30+
- [🛠️ Dependencies for development (dev)](#%EF%B8%8F-dependencies-for-development-dev)
31+
- [⚙ Linters](#-linters)
32+
- [🧩 Usage](#-usage)
33+
- [🔨 Function Signature](#-function-signature)
34+
- [📤 Example](#-example)
35+
36+
37+
## 📋 Description
38+
39+
This project is a parser for the steam market of such games as CS 2, Team Fortress 2, Dota 2, PUBG, RUST, etc.
40+
41+
42+
This guide describes the steps to install and run the project on Linux and Windows.
43+
44+
45+
## 💻 Requirements
46+
47+
Before you start, make sure that the following components are installed on your system:
48+
49+
- **Python 3** (check with `python3 --version` Linux / `python --version` Windows)
50+
#### I'm using Python 3.13. Other versions will probably work.
51+
- **pip** (check with `pip --version`)
52+
- **Git** (check with `git --version`)
53+
- **Python Virtual Environment** (venv)
54+
- **Linux Shell** (for example, Bash)
55+
56+
## 🚀 Project Installation
57+
58+
### 📂 Step 1: Clone the Repository
59+
60+
Clone the project repository using Git:
61+
62+
```bash
63+
git clone url---------------------------------------
64+
cd team ?????????????????????????????
65+
```
66+
67+
### 🖥 Step 2: Create and activate a virtual environment
68+
69+
Create a virtual environment using the `venv` command, which allows you to isolate project dependencies:
70+
71+
```bash
72+
python3 -m venv venv # Linux
73+
python -m venv venv # Windows
74+
```
75+
76+
Activate the virtual environment:
77+
78+
```bash
79+
source venv/bin/activate # Linux
80+
source venv/Scripts/activate # Windows
81+
```
82+
83+
### 🔃 Step 3: Installing Dependencies
84+
85+
Dependencies are divided into two groups:
86+
87+
- **For development (dev)** – additional dependencies for development.
88+
- **For code analysis (flake8)** – additional dependencies for code analysis.
89+
90+
#### 🛠️ Dependencies for development (dev)
91+
92+
Install the dependencies for local development (they include all **flake8** dependencies)
93+
94+
```bash
95+
pip install -r requirements/dev.txt
96+
```
97+
98+
Now you can use the parser.
99+
100+
101+
## ⚙ Linters
102+
103+
If there are any changes, I recommend using linters:
104+
105+
```bash
106+
flake8 . --count --show-source --statistics # Use to view PEP8 errors.
107+
black . --check --verbose -- # Use to format code.
108+
isort . --verbose # Use to format the order of imports.
109+
```
110+
111+
112+
## 🧩 Usage
113+
##### A market_scraper function retrieves pricing information for a specified item from the Steam Community Market.
114+
##### It constructs the request URL using the item name, Steam application ID, and the desired currency, the fetches and decodes the JSON response.
115+
116+
##### the list of all available currencies is stored in data.py, as well as a list with some games from steam. (you can add your own games)
117+
118+
## 🔨 Function Signature
119+
```python
120+
import data
121+
def market_scraper(
122+
item_name: str,
123+
app_id: int,
124+
currency: int = data.Currency.USD.value,
125+
) -> str:
126+
pass
127+
```
128+
129+
#### Parameters:
130+
##### item_name (str): Full name of the item.
131+
##### app_id (int): Steam Application ID.
132+
##### currency (int, optional): Currency code (default: USD).
133+
134+
#### Returns:
135+
##### A dictionary with the JSON response or a string with an error message.
136+
137+
138+
## 📤 Example
139+
140+
```python
141+
import data
142+
import scraper
143+
144+
# Example usage: Fetch price information for 'Dreams & Nightmares Case' in USD for the CS2 app.
145+
# see more in examples.py
146+
print(
147+
scraper.market_scraper(
148+
'Dreams & Nightmares Case',
149+
data.Apps.CS2.value,
150+
data.Currency.USD.value,
151+
),
152+
)
153+
```
154+
#### Output json data:
155+
```json
156+
{
157+
"success": true,
158+
"lowest_price": "$1.90",
159+
"volume": "77,555",
160+
"median_price": "$1.90"
161+
}
162+
```

‎data.py‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
import enum
2+
3+
4+
class Apps(enum.Enum):
5+
TEAM_FORTRESS_2 = 440
6+
DOTA_2 = 570
7+
CS2 = 730
8+
STEAM = 753
9+
RUST = 252490
10+
PUBG = 578080
11+
12+
13+
class Currency(enum.Enum):
14+
# ISO 4217
15+
USD = 1 # United States dollar
16+
GBP = 2 # Pound sterling
17+
EUR = 3 # Euro
18+
CHF = 4 # Swiss franc
19+
RUB = 5 # Russian ruble
20+
PLN = 6 # Polish złoty
21+
BRL = 7 # Brazilian real
22+
JPY = 8 # Japanese yen
23+
SEK = 9 # Swedish króna
24+
IDR = 10 # Indonesian rupiah
25+
MYR = 11 # Malaysian ringgit
26+
PHP = 12 # Philippine peso
27+
SGD = 13 # Singapore dollar
28+
THB = 14 # Thai baht
29+
VND = 15 # Vietnamese đồng
30+
KRW = 16 # South Korean won
31+
TRY = 17 # Turkish lira
32+
UAH = 18 # Ukrainian hryvnia
33+
MXN = 19 # Mexican peso
34+
CAD = 20 # Canadian dollar
35+
AUD = 21 # Australian dollar
36+
NZD = 22 # New Zealand dollar
37+
CNY = 23 # Renminbi
38+
INR = 24 # Indian rupee
39+
CLP = 25 # Chilean peso
40+
CUP = 26 # Cuban peso
41+
COP = 27 # Colombian peso
42+
ZAR = 28 # South African rand
43+
HKD = 29 # Hong Kong dollar
44+
TWD = 30 # New Taiwan dollar
45+
SAR = 31 # Saudi riyal
46+
AED = 32 # United Arab Emirates dirham
47+
# 33 - None
48+
ARS = 34 # Argentine peso
49+
ILS = 35 # Israeli new shekel
50+
# 36 - None
51+
KZT = 37 # Kazakhstani tenge
52+
KWD = 38 # Kuwaiti dinar
53+
QAR = 39 # Qatari riyal
54+
CRC = 40 # Costa Rican colon
55+
UYU = 41 # Uruguayan Peso

0 commit comments

Comments
 (0)