PDGA Rating Tracker

PDGA rating scraper and display app. Scrapes player ratings and course data from pdga.com, stores in SQLite, serves via Express with EJS templates and HTMX.

Tech Stack

Runtime: Node.js 22 (slim/Debian-based in Docker)
Hosting: Gitea (gitea.shcizo.se/shcizo/pdga-rating) — use tea CLI for issues/PRs, not gh
Server: Express with EJS templates
Database: SQLite3 (file: ratings.db, Docker: /app/data/ratings.db)
Frontend: HTMX + vanilla JS (in public/js/)
Scraping: Puppeteer (with stealth plugin) + direct HTTP
Logging: Pino (JSON in production, pino-pretty in dev)
CI/CD: Gitea Actions (tag-triggered build + push + deploy via .gitea/workflows/deploy.yml)

Project Structure

server.js              # Express app entrypoint
src/
  logger.js            # Pino logger instance
  db.js                # SQLite init, migrations, seeding
  models/              # Data access (player.js, course.js)
  routes/              # Express routes (players, courses, pages)
  scrapers/            # PDGA scrapers (HTTP + Puppeteer)
  services/            # Business logic (player-service, rating-calculator)
views/
  pages/               # EJS page templates
  partials/            # EJS partials (shared layout)
public/
  css/                 # Stylesheets
  js/                  # Client-side JS (HTMX interactions)

Commands

npm start — Start production server (port 3000)
npm run dev — Start with nodemon (auto-reload)
LOG_LEVEL=debug npm start — Enable debug logging
docker compose up — Run via Docker

No test framework or lint setup — package.json has only start and dev scripts. If adding either, document it here.

Conventions

Logging: Use require('./logger') (or relative path). Never use console.log/error in backend code. Use appropriate Pino levels: debug for verbose/diagnostic data, info for operational status, warn for retries/degraded state, error for failures, fatal for startup crashes.
Frontend JS: console.error is fine in public/js/ — runs in browser, no Pino.
Commits: Conventional commits (feat:, fix:, refactor:, chore:, ci:).
Releases: Manual version bump — edit version in package.json + package-lock.json, commit as <version>, tag v<version>, push commit + tag (git push origin main v<version>). Triggers .gitea/workflows/deploy.yml which (1) builds and pushes the image to gitea.shcizo.se/shcizo/pdga-rating:<tag> + :latest, then (2) calls package-updater-action against updater.shcizo.se/update to roll out the new image. Required secrets: PACKAGES_TOKEN (PAT with write:package, for registry auth — the auto-injected GITEA_TOKEN does not have effective registry access) and UPDATER_API_KEY (for the updater endpoint). The action repo shcizo/package-updater-action is referenced via full Gitea URL (https://gitea.shcizo.se/...) since uses: defaults to GitHub.
Scraping: Two strategies per entity: direct HTTP (fast, preferred) with Puppeteer fallback (stealth plugin for anti-bot). Rate limiting must be respected.
Database: Migrations run automatically on startup in db.js. Schema changes go there.
Templates: EJS with shared layout in views/partials/. Pages use HTMX for dynamic content loading.

PDGA Domain Notes

Rating publication cycle: PDGA officially recalculates ratings on the second Tuesday of each month. getNextPDGAUpdateDate() in src/services/rating-calculator.js computes this — round filtering uses it as cutoff.
Predicted rating algorithm: calculatePredictedRating(roundRatings) replicates PDGA's formula — 12-mo window (expands to 24 if <8 rounds), outlier removal at ≥7 rounds (2.5σ + 100pt threshold), double-weighting of recent 25% at ≥9 rounds. Returns {rating, stdDev, debugLog}.
Rate limits: POST /api/refresh-round-history/:pdgaNumber enforces a 24h cooldown per player (src/routes/players.js). Don't bypass — PDGA's site rate-limits aggressively.
Round history refresh uses Puppeteer (stealth plugin), other scraping prefers direct HTTP. Predicted rating is recomputed and stored on each refresh.

Environment Variables

Variable	Default	Description
`LOG_LEVEL`	`info`	Pino log level
`NODE_ENV`	—	Set to `production` for JSON logs
`DB_PATH`	`./ratings.db`	SQLite database path
`PUPPETEER_EXECUTABLE_PATH`	—	Chromium path (set in Docker)

4.5 KiB Raw Permalink Blame History Unescape Escape