# PDGA Rating Tracker PDGA rating scraper and display app. Scrapes player ratings and course data from pdga.com, stores in SQLite, serves via Express with EJS templates and HTMX. ## Tech Stack - **Runtime:** Node.js 22 (slim/Debian-based in Docker) - **Hosting:** Gitea (`gitea.shcizo.se/shcizo/pdga-rating`) — use `tea` CLI for issues/PRs, not `gh` - **Server:** Express with EJS templates - **Database:** SQLite3 (file: `ratings.db`, Docker: `/app/data/ratings.db`) - **Frontend:** HTMX + vanilla JS (in `public/js/`) - **Scraping:** Puppeteer (with stealth plugin) + direct HTTP - **Logging:** Pino (JSON in production, pino-pretty in dev) - **CI/CD:** Gitea Actions (tag-triggered docker build/push to `gitea.shcizo.se/shcizo/pdga-rating`) ## Project Structure ``` server.js # Express app entrypoint src/ logger.js # Pino logger instance db.js # SQLite init, migrations, seeding models/ # Data access (player.js, course.js) routes/ # Express routes (players, courses, pages) scrapers/ # PDGA scrapers (HTTP + Puppeteer) services/ # Business logic (player-service, rating-calculator) views/ pages/ # EJS page templates partials/ # EJS partials (shared layout) public/ css/ # Stylesheets js/ # Client-side JS (HTMX interactions) ``` ## Commands - `npm start` — Start production server (port 3000) - `npm run dev` — Start with nodemon (auto-reload) - `LOG_LEVEL=debug npm start` — Enable debug logging - `docker compose up` — Run via Docker **No test framework or lint setup** — `package.json` has only `start` and `dev` scripts. If adding either, document it here. ## Conventions - **Logging:** Use `require('./logger')` (or relative path). Never use `console.log/error` in backend code. Use appropriate Pino levels: `debug` for verbose/diagnostic data, `info` for operational status, `warn` for retries/degraded state, `error` for failures, `fatal` for startup crashes. - **Frontend JS:** `console.error` is fine in `public/js/` — runs in browser, no Pino. - **Commits:** Conventional commits (`feat:`, `fix:`, `refactor:`, `chore:`, `ci:`). - **Releases:** Manual version bump — edit `version` in `package.json` + `package-lock.json`, commit as ``, tag `v`, push commit + tag (`git push origin main v`). Triggers `.gitea/workflows/docker-build.yml` which builds and pushes the image. Auth uses repo secret `PACKAGES_TOKEN` (PAT with `write:package`) — the auto-injected `GITEA_TOKEN` does not have effective registry access. - **Scraping:** Two strategies per entity: direct HTTP (fast, preferred) with Puppeteer fallback (stealth plugin for anti-bot). Rate limiting must be respected. - **Database:** Migrations run automatically on startup in `db.js`. Schema changes go there. - **Templates:** EJS with shared layout in `views/partials/`. Pages use HTMX for dynamic content loading. ## PDGA Domain Notes - **Rating publication cycle:** PDGA officially recalculates ratings on the **second Tuesday of each month**. `getNextPDGAUpdateDate()` in `src/services/rating-calculator.js` computes this — round filtering uses it as cutoff. - **Predicted rating algorithm:** `calculatePredictedRating(roundRatings)` replicates PDGA's formula — 12-mo window (expands to 24 if <8 rounds), outlier removal at ≥7 rounds (2.5σ + 100pt threshold), double-weighting of recent 25% at ≥9 rounds. Returns `{rating, stdDev, debugLog}`. - **Rate limits:** `POST /api/refresh-round-history/:pdgaNumber` enforces a 24h cooldown per player (`src/routes/players.js`). Don't bypass — PDGA's site rate-limits aggressively. - **Round history refresh** uses Puppeteer (stealth plugin), other scraping prefers direct HTTP. Predicted rating is recomputed and stored on each refresh. ## Environment Variables | Variable | Default | Description | |----------|---------|-------------| | `LOG_LEVEL` | `info` | Pino log level | | `NODE_ENV` | — | Set to `production` for JSON logs | | `DB_PATH` | `./ratings.db` | SQLite database path | | `PUPPETEER_EXECUTABLE_PATH` | — | Chromium path (set in Docker) |