Update README with comprehensive documentation of new features
- Document database-first architecture and automatic startup population - Add Docker setup instructions and mobile-responsive UI features - Include API endpoint documentation and technical architecture details - Explain optimized PDGA scraping strategy and user-controlled refresh system - Add performance, reliability, and Docker configuration sections 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Samuel Enocsson
@@ -1,9 +1,10 @@
|
|||||||
# PDGA Ratings Scraper
|
# PDGA Ratings Scraper
|
||||||
|
|
||||||
A web application that scrapes PDGA player ratings and displays them in a sorted table.
|
A comprehensive web application that scrapes PDGA player ratings, calculates rating predictions, and displays them with interactive charts and real-time updates.
|
||||||
|
|
||||||
## Setup
|
## Setup
|
||||||
|
|
||||||
|
### Local Development
|
||||||
1. Install dependencies:
|
1. Install dependencies:
|
||||||
```bash
|
```bash
|
||||||
npm install
|
npm install
|
||||||
@@ -18,15 +19,79 @@ npm start
|
|||||||
|
|
||||||
4. Open http://localhost:3000 in your browser
|
4. Open http://localhost:3000 in your browser
|
||||||
|
|
||||||
## Usage
|
### Docker (Recommended)
|
||||||
|
1. Build and run with Docker:
|
||||||
|
```bash
|
||||||
|
docker build -t pdga-ratings .
|
||||||
|
docker run -p 3000:3000 --name pdga-ratings-container pdga-ratings
|
||||||
|
```
|
||||||
|
|
||||||
- Edit `pdga-numbers.txt` to add/remove players
|
2. Open http://localhost:3000 in your browser
|
||||||
- Click "Refresh Ratings" to scrape current ratings
|
|
||||||
- Players are automatically sorted by rating (highest first)
|
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- Web scraping of PDGA player ratings
|
### Core Functionality
|
||||||
- Sortable table display
|
- **Database-first architecture**: SQLite database serves as single source of truth
|
||||||
- Easy player management via text file
|
- **Automatic startup population**: Reads `pdga-numbers.txt` and populates missing players at startup
|
||||||
- Real-time rating updates
|
- **User-controlled refresh**: Data only updates when user explicitly clicks refresh icons
|
||||||
|
- **Optimized PDGA scraping**: Uses `/details` page as baseline + only scrapes NEW tournaments for predictions
|
||||||
|
|
||||||
|
### Rating System
|
||||||
|
- **Official PDGA ratings**: Current player ratings from PDGA website
|
||||||
|
- **Rating predictions**: Multi-day tournament support with weighted calculations
|
||||||
|
- **Rating history charts**: Interactive SVG charts with hover tooltips
|
||||||
|
- **Rating changes**: Shows recent rating changes with color-coded indicators
|
||||||
|
|
||||||
|
### User Interface
|
||||||
|
- **Mobile-responsive design**: Optimized layouts for desktop and mobile
|
||||||
|
- **Expandable player rows**: Click to view detailed rating history charts
|
||||||
|
- **Individual refresh controls**: Separate refresh icons for ratings, predictions, and charts
|
||||||
|
- **Progress tracking**: Real-time progress bars for bulk operations
|
||||||
|
- **Error handling**: Comprehensive error messages with retry suggestions
|
||||||
|
|
||||||
|
### Performance & Reliability
|
||||||
|
- **Respectful scraping**: 2-second delays between PDGA requests
|
||||||
|
- **Smart caching**: Database persistence with user-controlled freshness
|
||||||
|
- **Docker optimization**: Alpine Linux with Chromium for stable Puppeteer execution
|
||||||
|
- **Retry logic**: Automatic retries with fallback strategies for network issues
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
### Basic Operations
|
||||||
|
- **View ratings**: Page loads instantly from database
|
||||||
|
- **Refresh player data**: Click refresh icon next to player rating
|
||||||
|
- **Refresh predictions**: Click refresh icon next to predicted rating
|
||||||
|
- **View rating history**: Click on any player row to expand chart
|
||||||
|
- **Refresh charts**: Click refresh icon in chart title
|
||||||
|
|
||||||
|
### Player Management
|
||||||
|
- Edit `pdga-numbers.txt` to add/remove players
|
||||||
|
- Restart server to automatically populate new players
|
||||||
|
- Use "Load All" button to force refresh all player data
|
||||||
|
|
||||||
|
### API Endpoints
|
||||||
|
- `GET /api/ratings` - Get all player ratings from database
|
||||||
|
- `POST /api/refresh-player/:pdgaNumber` - Refresh specific player
|
||||||
|
- `POST /api/refresh-round-history/:pdgaNumber` - Refresh player predictions
|
||||||
|
- `GET /api/rating-history/:pdgaNumber` - Get player rating history
|
||||||
|
- `GET /api/database-status` - Check database population status
|
||||||
|
|
||||||
|
## Technical Details
|
||||||
|
|
||||||
|
### Architecture
|
||||||
|
- **Backend**: Node.js + Express + SQLite
|
||||||
|
- **Frontend**: Vanilla JavaScript with Server-Sent Events
|
||||||
|
- **Scraping**: Puppeteer with Alpine Linux + Chromium
|
||||||
|
- **Charts**: Custom SVG implementation with interactive tooltips
|
||||||
|
|
||||||
|
### PDGA Integration
|
||||||
|
- Uses official PDGA `/details` page for rating calculations
|
||||||
|
- Only scrapes tournaments played AFTER last official round
|
||||||
|
- Respects PDGA servers with proper delays and retry logic
|
||||||
|
- Handles both current ratings and historical data
|
||||||
|
|
||||||
|
### Docker Configuration
|
||||||
|
- Alpine Linux base for minimal footprint
|
||||||
|
- Pre-installed Chromium for Puppeteer
|
||||||
|
- Non-root user for security
|
||||||
|
- Proper volume mounting for data persistence
|
||||||
Reference in New Issue
Block a user