Samuel Enocsson
74bfb7bd88
- Document database-first architecture and automatic startup population - Add Docker setup instructions and mobile-responsive UI features - Include API endpoint documentation and technical architecture details - Explain optimized PDGA scraping strategy and user-controlled refresh system - Add performance, reliability, and Docker configuration sections 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
97 lines
3.6 KiB
Markdown
97 lines
3.6 KiB
Markdown
# PDGA Ratings Scraper
|
|
|
|
A comprehensive web application that scrapes PDGA player ratings, calculates rating predictions, and displays them with interactive charts and real-time updates.
|
|
|
|
## Setup
|
|
|
|
### Local Development
|
|
1. Install dependencies:
|
|
```bash
|
|
npm install
|
|
```
|
|
|
|
2. Add PDGA numbers to `pdga-numbers.txt` (one per line)
|
|
|
|
3. Start the server:
|
|
```bash
|
|
npm start
|
|
```
|
|
|
|
4. Open http://localhost:3000 in your browser
|
|
|
|
### Docker (Recommended)
|
|
1. Build and run with Docker:
|
|
```bash
|
|
docker build -t pdga-ratings .
|
|
docker run -p 3000:3000 --name pdga-ratings-container pdga-ratings
|
|
```
|
|
|
|
2. Open http://localhost:3000 in your browser
|
|
|
|
## Features
|
|
|
|
### Core Functionality
|
|
- **Database-first architecture**: SQLite database serves as single source of truth
|
|
- **Automatic startup population**: Reads `pdga-numbers.txt` and populates missing players at startup
|
|
- **User-controlled refresh**: Data only updates when user explicitly clicks refresh icons
|
|
- **Optimized PDGA scraping**: Uses `/details` page as baseline + only scrapes NEW tournaments for predictions
|
|
|
|
### Rating System
|
|
- **Official PDGA ratings**: Current player ratings from PDGA website
|
|
- **Rating predictions**: Multi-day tournament support with weighted calculations
|
|
- **Rating history charts**: Interactive SVG charts with hover tooltips
|
|
- **Rating changes**: Shows recent rating changes with color-coded indicators
|
|
|
|
### User Interface
|
|
- **Mobile-responsive design**: Optimized layouts for desktop and mobile
|
|
- **Expandable player rows**: Click to view detailed rating history charts
|
|
- **Individual refresh controls**: Separate refresh icons for ratings, predictions, and charts
|
|
- **Progress tracking**: Real-time progress bars for bulk operations
|
|
- **Error handling**: Comprehensive error messages with retry suggestions
|
|
|
|
### Performance & Reliability
|
|
- **Respectful scraping**: 2-second delays between PDGA requests
|
|
- **Smart caching**: Database persistence with user-controlled freshness
|
|
- **Docker optimization**: Alpine Linux with Chromium for stable Puppeteer execution
|
|
- **Retry logic**: Automatic retries with fallback strategies for network issues
|
|
|
|
## Usage
|
|
|
|
### Basic Operations
|
|
- **View ratings**: Page loads instantly from database
|
|
- **Refresh player data**: Click refresh icon next to player rating
|
|
- **Refresh predictions**: Click refresh icon next to predicted rating
|
|
- **View rating history**: Click on any player row to expand chart
|
|
- **Refresh charts**: Click refresh icon in chart title
|
|
|
|
### Player Management
|
|
- Edit `pdga-numbers.txt` to add/remove players
|
|
- Restart server to automatically populate new players
|
|
- Use "Load All" button to force refresh all player data
|
|
|
|
### API Endpoints
|
|
- `GET /api/ratings` - Get all player ratings from database
|
|
- `POST /api/refresh-player/:pdgaNumber` - Refresh specific player
|
|
- `POST /api/refresh-round-history/:pdgaNumber` - Refresh player predictions
|
|
- `GET /api/rating-history/:pdgaNumber` - Get player rating history
|
|
- `GET /api/database-status` - Check database population status
|
|
|
|
## Technical Details
|
|
|
|
### Architecture
|
|
- **Backend**: Node.js + Express + SQLite
|
|
- **Frontend**: Vanilla JavaScript with Server-Sent Events
|
|
- **Scraping**: Puppeteer with Alpine Linux + Chromium
|
|
- **Charts**: Custom SVG implementation with interactive tooltips
|
|
|
|
### PDGA Integration
|
|
- Uses official PDGA `/details` page for rating calculations
|
|
- Only scrapes tournaments played AFTER last official round
|
|
- Respects PDGA servers with proper delays and retry logic
|
|
- Handles both current ratings and historical data
|
|
|
|
### Docker Configuration
|
|
- Alpine Linux base for minimal footprint
|
|
- Pre-installed Chromium for Puppeteer
|
|
- Non-root user for security
|
|
- Proper volume mounting for data persistence |