Samuel Enocsson
8c977d6624
Introduce new design token set (paper/ink/line/accent + radius/shadow) with backward-compat aliases for legacy --surface/--navy/--text names. Swap DM Sans for Plus Jakarta Sans, add JetBrains Mono with tabular numerics. Replace .app-header with sticky .topbar partial (brand + segmented nav + Next update / Last refresh meta + Refresh all button). Add POST /api/refresh-all that runs refreshAllPlayersInDB() with an in-memory mutex and returns the rendered topbar so HTMX can swap it in. "Next update" is computed as first Tuesday of next month (approximation of PDGA's monthly cycle). "Last refresh" derives from MAX(players.last_updated).
PDGA Ratings Scraper
A comprehensive web application that scrapes PDGA player ratings, calculates rating predictions, and displays them with interactive charts and real-time updates.
Setup
Local Development
- Install dependencies:
npm install
-
Add PDGA numbers to
pdga-numbers.txt(one per line) -
Start the server:
npm start
- Open http://localhost:3000 in your browser
Docker (Recommended)
- Build and run with Docker:
docker build -t pdga-ratings .
docker run -p 3000:3000 --name pdga-ratings-container pdga-ratings
- Open http://localhost:3000 in your browser
Features
Core Functionality
- Database-first architecture: SQLite database serves as single source of truth
- Automatic startup population: Reads
pdga-numbers.txtand populates missing players at startup - User-controlled refresh: Data only updates when user explicitly clicks refresh icons
- Optimized PDGA scraping: Uses
/detailspage as baseline + only scrapes NEW tournaments for predictions
Rating System
- Official PDGA ratings: Current player ratings from PDGA website
- Rating predictions: Multi-day tournament support with weighted calculations
- Rating history charts: Interactive SVG charts with hover tooltips
- Rating changes: Shows recent rating changes with color-coded indicators
User Interface
- Mobile-responsive design: Optimized layouts for desktop and mobile
- Expandable player rows: Click to view detailed rating history charts
- Individual refresh controls: Separate refresh icons for ratings, predictions, and charts
- Progress tracking: Real-time progress bars for bulk operations
- Error handling: Comprehensive error messages with retry suggestions
Performance & Reliability
- Respectful scraping: 2-second delays between PDGA requests
- Smart caching: Database persistence with user-controlled freshness
- Docker optimization: Alpine Linux with Chromium for stable Puppeteer execution
- Retry logic: Automatic retries with fallback strategies for network issues
Usage
Basic Operations
- View ratings: Page loads instantly from database
- Refresh player data: Click refresh icon next to player rating
- Refresh predictions: Click refresh icon next to predicted rating
- View rating history: Click on any player row to expand chart
- Refresh charts: Click refresh icon in chart title
Player Management
- Edit
pdga-numbers.txtto add/remove players - Restart server to automatically populate new players
- Use "Load All" button to force refresh all player data
API Endpoints
GET /api/ratings- Get all player ratings from databasePOST /api/refresh-player/:pdgaNumber- Refresh specific playerPOST /api/refresh-round-history/:pdgaNumber- Refresh player predictionsGET /api/rating-history/:pdgaNumber- Get player rating historyGET /api/database-status- Check database population status
Technical Details
Architecture
- Backend: Node.js + Express + SQLite
- Frontend: Vanilla JavaScript with Server-Sent Events
- Scraping: Puppeteer with Alpine Linux + Chromium
- Charts: Custom SVG implementation with interactive tooltips
PDGA Integration
- Uses official PDGA
/detailspage for rating calculations - Only scrapes tournaments played AFTER last official round
- Respects PDGA servers with proper delays and retry logic
- Handles both current ratings and historical data
Docker Configuration
- Alpine Linux base for minimal footprint
- Pre-installed Chromium for Puppeteer
- Non-root user for security
- Proper volume mounting for data persistence