Develop a "CSGO/CS2 accessories market data collection and analysis system"

Project Title:
Full-Stack Development of a Real-Time CSGO/CS2 Skin Market Data Aggregation & Analysis Platform

Project Overview:
We are seeking an experienced developer or team to build a comprehensive real-time data aggregation and analysis platform for CSGO/CS2 skin markets. This system will serve as a backend data infrastructure for multiple gaming-related products.

Core Objectives:
Real-time data crawling (per second level if possible) from multiple marketplaces.

Smart data aggregation, median filtering, and volume-based weighting.

Generate live price charts, K-line graphs, market indices, and volume analysis.

Admin dashboard + external API output.

Fully documented system for easy integration with other projects.

Module 1: Multi-Source Skin Market Crawler
Target Marketplaces (CN + Global):
Buff.163 (CN)

C5Game (CN)

IGXE (CN)

Ueeshop / ECO (CN)

BitSkins (Global)

CSDeals (Global)

ShadowPay (Global)

Steam Community Market

other relevant marketplaces with real traffic (your suggestions welcome)

Data to be Collected:
Current price, lowest/average/highest price per skin

Recent trade transactions (price + volume + timestamp)

Real-time listing volume (circulation/liquidity)

Historical price data (for trend and charting)

Steam data (player count, item introduction date, etc.)

Technical Requirements:
Per-second or near real-time crawling

Smart handling of anti-bot protection (sliding CAPTCHA, JS challenges, etc.)

Dynamic scheduling, retry on error, alert on failure

Scalable, maintainable crawler framework (e.g., Puppeteer + Node.js / Python Scrapy)

Module 2: Data Aggregation & Smart Analytics
Aggregation Logic:
Merge data from all platforms per skin

Median price filtering, anomaly detection

Volume-weighted pricing

Trend detection algorithms

Real-time market index system (akin to stock indices)

Top-selling skins, most traded, fastest moving

Real-time detection of price manipulation or market anomalies

Visual Outputs:
K-Line charts per skin

Historical price trend graphs

Global Market Index (CSGO/CS2)

Real-time trade volume charts

Leaderboards (Top 10 most traded, volatile, highest gainers, etc.)

Module 3: Admin Panel + External API
Admin Dashboard Features:
Crawler status monitor

Enable/disable marketplaces

Error logs

Config panel for index formulas / thresholds

Real-time market overview charts

API Output (with full documentation):
Get latest skin price (per source or merged)

Fetch historical K-line data

Query market index & trend data

Get Steam player stats

Query skin liquidity and rank

Tech Stack Recommendations (Preferred):
Backend: Node.js / Python + Express / FastAPI

Crawler: Puppeteer, Playwright, or Scrapy (IP rotation + headless)

Storage: MySQL/PostgreSQL + Redis + Elasticsearch (for large queries)

Dashboard Frontend: Vue.js / React

DevOps: Docker (containerized deployment), PM2, Nginx

Deliverables:
Full source code (frontend + backend + crawler)

Deployment manual (Docker + environment variables)

API documentation (Swagger or Postman format)

Admin panel with complete functionality

Optional: K8s deployment if you're experienced
Timeline:
Please propose your expected development timeline. Ideally, we expect a working MVP within 15–30 days.

Proposal Requirements:
Please include links to similar crawler or data analytics projects you've done (especially market price tracking).

Demonstrated experience with anti-bot bypass and dynamic scraping is a big plus.

Bonus if you’ve worked on crypto, financial, or Steam-based data systems.
Notes:
This system will be integrated into our internal CSGO ecosystem, so stability, speed, and scalability are essential. We will have long-term work and maintenance opportunities for the right developer/team.