Compare commits

...

9 Commits

Author SHA1 Message Date
Samuel Enocsson 66e892893f 1.2.4
Build and deploy / build-and-push (push) Successful in 1m29s
Build and deploy / deploy (push) Failing after 2s
2026-05-22 15:23:19 +02:00
Samuel Enocsson 5b9138da25 ci: fix deploy.yml and consolidate workflows
Fix YAML indentation (on:/jobs: were nested under name:), correct
image name from shcizo/myapp to shcizo/pdga-rating, switch registry
auth from GITEA_TOKEN to PACKAGES_TOKEN (auto-injected token lacks
effective registry access), and add packages:write permission.

Remove docker-build.yml since deploy.yml now covers build + push +
deploy in one workflow — previously both triggered on tag push and
raced for the same image tags.
2026-05-22 15:22:41 +02:00
Samuel Enocsson b1e8d63a63 added deploy 2026-05-22 15:15:18 +02:00
shcizo c6ac174921 Merge pull request 'fix: preload player rating history to fix first-click chart render (#10)' (#15) from fix/preload-player-history-10 into main 2026-05-22 12:59:26 +02:00
Samuel Enocsson fba1bea247 refactor: address review feedback — extract date helper, rename listener 2026-05-22 11:47:47 +02:00
Samuel Enocsson a63da6f3ca fix: preload player rating history to fix first-click chart render (#10) 2026-05-22 11:41:38 +02:00
shcizo 3cfdc305ec Merge pull request 'fix: header "Next update" uses second Tuesday (closes #12)' (#13) from fix/header-next-update-date into main
Reviewed-on: #13
2026-05-22 10:00:39 +02:00
Samuel Enocsson d336156bbb fix: header "Next update" uses second Tuesday (closes #12)
The topbar showed the first Tuesday of the *next* month instead of
PDGA's actual cycle (second Tuesday of the month). Replace the
duplicated computeNextUpdate() with the central
getNextPDGAUpdateDate() from rating-calculator, keeping only the
formatter ("Tue 9 Jun") here.
2026-05-22 09:54:57 +02:00
Samuel Enocsson e1b9e97484 docs: update CLAUDE.md for Gitea migration and PDGA domain notes
- Fix runtime line (Node 22 slim, not Node 18 Alpine)
- Add Hosting line (Gitea, use tea not gh)
- Reflect new CI/CD flow (Gitea Actions, manual version bump, PACKAGES_TOKEN)
- Add PDGA Domain Notes section (rating cycle, predicted-rating algorithm,
  rate limits) so future sessions don't have to re-derive domain logic
- Note absence of test framework explicitly
2026-05-22 09:35:54 +02:00
12 changed files with 147 additions and 83 deletions
+42
View File
@@ -0,0 +1,42 @@
name: Build and deploy
on:
push:
tags:
- "v*"
jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Login to Gitea registry
uses: docker/login-action@v3
with:
registry: gitea.shcizo.se
username: ${{ gitea.actor }}
password: ${{ secrets.PACKAGES_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: |
gitea.shcizo.se/shcizo/pdga-rating:${{ gitea.ref_name }}
gitea.shcizo.se/shcizo/pdga-rating:latest
deploy:
runs-on: ubuntu-latest
needs: [build-and-push]
steps:
- uses: shcizo/package-updater-action@v1
with:
endpoint: https://updater.shcizo.se/update
image: gitea.shcizo.se/shcizo/pdga-rating
tag: ${{ gitea.ref_name }}
token: ${{ secrets.UPDATER_TOKEN }}
-37
View File
@@ -1,37 +0,0 @@
name: Docker Build
on:
push:
tags:
- 'v*'
jobs:
docker:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- uses: docker/login-action@v3
with:
registry: gitea.shcizo.se
username: ${{ gitea.actor }}
password: ${{ secrets.PACKAGES_TOKEN }}
- uses: docker/metadata-action@v5
id: meta
with:
images: gitea.shcizo.se/shcizo/pdga-rating
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,value=latest
- uses: docker/build-push-action@v6
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
+14 -3
View File
@@ -4,13 +4,14 @@ PDGA rating scraper and display app. Scrapes player ratings and course data from
## Tech Stack ## Tech Stack
- **Runtime:** Node.js 18 (Alpine in Docker) - **Runtime:** Node.js 22 (slim/Debian-based in Docker)
- **Hosting:** Gitea (`gitea.shcizo.se/shcizo/pdga-rating`) — use `tea` CLI for issues/PRs, not `gh`
- **Server:** Express with EJS templates - **Server:** Express with EJS templates
- **Database:** SQLite3 (file: `ratings.db`, Docker: `/app/data/ratings.db`) - **Database:** SQLite3 (file: `ratings.db`, Docker: `/app/data/ratings.db`)
- **Frontend:** HTMX + vanilla JS (in `public/js/`) - **Frontend:** HTMX + vanilla JS (in `public/js/`)
- **Scraping:** Puppeteer (with stealth plugin) + direct HTTP - **Scraping:** Puppeteer (with stealth plugin) + direct HTTP
- **Logging:** Pino (JSON in production, pino-pretty in dev) - **Logging:** Pino (JSON in production, pino-pretty in dev)
- **CI/CD:** Release Please + Docker build/push to GHCR - **CI/CD:** Gitea Actions (tag-triggered docker build/push to `gitea.shcizo.se/shcizo/pdga-rating`)
## Project Structure ## Project Structure
@@ -38,15 +39,25 @@ public/
- `LOG_LEVEL=debug npm start` — Enable debug logging - `LOG_LEVEL=debug npm start` — Enable debug logging
- `docker compose up` — Run via Docker - `docker compose up` — Run via Docker
**No test framework or lint setup**`package.json` has only `start` and `dev` scripts. If adding either, document it here.
## Conventions ## Conventions
- **Logging:** Use `require('./logger')` (or relative path). Never use `console.log/error` in backend code. Use appropriate Pino levels: `debug` for verbose/diagnostic data, `info` for operational status, `warn` for retries/degraded state, `error` for failures, `fatal` for startup crashes. - **Logging:** Use `require('./logger')` (or relative path). Never use `console.log/error` in backend code. Use appropriate Pino levels: `debug` for verbose/diagnostic data, `info` for operational status, `warn` for retries/degraded state, `error` for failures, `fatal` for startup crashes.
- **Frontend JS:** `console.error` is fine in `public/js/` — runs in browser, no Pino. - **Frontend JS:** `console.error` is fine in `public/js/` — runs in browser, no Pino.
- **Commits:** Conventional commits (`feat:`, `fix:`, `refactor:`, `chore:`) — drives release-please. - **Commits:** Conventional commits (`feat:`, `fix:`, `refactor:`, `chore:`, `ci:`).
- **Releases:** Manual version bump — edit `version` in `package.json` + `package-lock.json`, commit as `<version>`, tag `v<version>`, push commit + tag (`git push origin main v<version>`). Triggers `.gitea/workflows/docker-build.yml` which builds and pushes the image. Auth uses repo secret `PACKAGES_TOKEN` (PAT with `write:package`) — the auto-injected `GITEA_TOKEN` does not have effective registry access.
- **Scraping:** Two strategies per entity: direct HTTP (fast, preferred) with Puppeteer fallback (stealth plugin for anti-bot). Rate limiting must be respected. - **Scraping:** Two strategies per entity: direct HTTP (fast, preferred) with Puppeteer fallback (stealth plugin for anti-bot). Rate limiting must be respected.
- **Database:** Migrations run automatically on startup in `db.js`. Schema changes go there. - **Database:** Migrations run automatically on startup in `db.js`. Schema changes go there.
- **Templates:** EJS with shared layout in `views/partials/`. Pages use HTMX for dynamic content loading. - **Templates:** EJS with shared layout in `views/partials/`. Pages use HTMX for dynamic content loading.
## PDGA Domain Notes
- **Rating publication cycle:** PDGA officially recalculates ratings on the **second Tuesday of each month**. `getNextPDGAUpdateDate()` in `src/services/rating-calculator.js` computes this — round filtering uses it as cutoff.
- **Predicted rating algorithm:** `calculatePredictedRating(roundRatings)` replicates PDGA's formula — 12-mo window (expands to 24 if <8 rounds), outlier removal at ≥7 rounds (2.5σ + 100pt threshold), double-weighting of recent 25% at ≥9 rounds. Returns `{rating, stdDev, debugLog}`.
- **Rate limits:** `POST /api/refresh-round-history/:pdgaNumber` enforces a 24h cooldown per player (`src/routes/players.js`). Don't bypass — PDGA's site rate-limits aggressively.
- **Round history refresh** uses Puppeteer (stealth plugin), other scraping prefers direct HTTP. Predicted rating is recomputed and stored on each refresh.
## Environment Variables ## Environment Variables
| Variable | Default | Description | | Variable | Default | Description |
+2 -2
View File
@@ -1,12 +1,12 @@
{ {
"name": "pdga-ratings", "name": "pdga-ratings",
"version": "1.2.3", "version": "1.2.4",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "pdga-ratings", "name": "pdga-ratings",
"version": "1.2.3", "version": "1.2.4",
"dependencies": { "dependencies": {
"ejs": "^4.0.1", "ejs": "^4.0.1",
"express": "^4.18.2", "express": "^4.18.2",
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "pdga-ratings", "name": "pdga-ratings",
"version": "1.2.3", "version": "1.2.4",
"description": "PDGA rating scraper and display", "description": "PDGA rating scraper and display",
"main": "server.js", "main": "server.js",
"scripts": { "scripts": {
+21 -14
View File
@@ -26,23 +26,30 @@ function applyDeltaPill(pillEl, value) {
pillEl.appendChild(numSpan); pillEl.appendChild(numSpan);
} }
function setupTooltipsAfterSwap() { function initChartsIn(rootEl) {
document.body.addEventListener('htmx:afterSwap', function(event) { rootEl.querySelectorAll('.player-chart').forEach(function(container) {
if (event.detail.target.id === 'ratings-table') { if (container.dataset.charted === 'true') return;
initRatingsTooltips(); if (!container.dataset.history) return;
try {
const history = JSON.parse(container.dataset.history);
createRatingChart(container, history);
container.dataset.charted = 'true';
} catch (e) {
console.error('Error rendering chart:', e);
} }
// After player history partial loads, render the chart });
}
function setupAfterTableSwap() {
document.body.addEventListener('htmx:afterSwap', function(event) {
const target = event.detail.target; const target = event.detail.target;
if (target.id === 'ratings-table') {
initRatingsTooltips();
initChartsIn(target);
return;
}
if (target.id && target.id.startsWith('history-content-')) { if (target.id && target.id.startsWith('history-content-')) {
const container = target.querySelector('.player-chart, .chart-container'); initChartsIn(target);
if (container && container.dataset.history) {
try {
const history = JSON.parse(container.dataset.history);
createRatingChart(container, history);
} catch (e) {
console.error('Error rendering chart:', e);
}
}
} }
}); });
} }
+28 -1
View File
@@ -253,6 +253,32 @@ function getAllMonthlyHistoriesFromDB(months = 12) {
}); });
} }
/**
* Fetches the full rating history for ALL players in one query.
* Returns Map<pdgaNumber, {rating, date}[]> ordered chronologically (oldest → newest).
* Mirrors getAllMonthlyHistoriesFromDB but returns every point, not monthly snapshots.
*/
function getAllRatingHistoriesFromDB() {
return new Promise((resolve, reject) => {
db.all(
`SELECT p.pdga_number, rh.date, rh.rating
FROM rating_history rh
JOIN players p ON rh.player_id = p.id
ORDER BY p.pdga_number, rh.date ASC`,
[],
(err, rows) => {
if (err) return reject(err);
const map = new Map();
for (const row of rows) {
if (!map.has(row.pdga_number)) map.set(row.pdga_number, []);
map.get(row.pdga_number).push({ date: row.date, rating: row.rating });
}
resolve(map);
}
);
});
}
function getLastRefresh() { function getLastRefresh() {
return new Promise((resolve, reject) => { return new Promise((resolve, reject) => {
db.get( db.get(
@@ -278,5 +304,6 @@ module.exports = {
savePredictedRatingToDB, savePredictedRatingToDB,
getLastRefresh, getLastRefresh,
getMonthlyHistory, getMonthlyHistory,
getAllMonthlyHistoriesFromDB getAllMonthlyHistoriesFromDB,
getAllRatingHistoriesFromDB
}; };
+5 -7
View File
@@ -5,7 +5,7 @@ const { getPlayerFromDB, savePlayerToDB, getRatingHistoryFromDB, saveRatingHisto
const { fetchPlayerDataHTTP, parsePlayerData, fetchRatingHistory, parseRatingHistory } = require('../scrapers/player-http'); const { fetchPlayerDataHTTP, parsePlayerData, fetchRatingHistory, parseRatingHistory } = require('../scrapers/player-http');
const { getOfficialRatingHistory, getOptimizedPlayerRounds } = require('../scrapers/player-puppeteer'); const { getOfficialRatingHistory, getOptimizedPlayerRounds } = require('../scrapers/player-puppeteer');
const { launchBrowser } = require('../scrapers/browser'); const { launchBrowser } = require('../scrapers/browser');
const { getPlayerDataFromDB, scrapePDGARating, getAllRatingsFromDB, refreshAllPlayersInDB, getPredictedRatingFromDB } = require('../services/player-service'); const { getPlayerDataFromDB, scrapePDGARating, getAllRatingsFromDB, refreshAllPlayersInDB, getPredictedRatingFromDB, formatDisplayDate } = require('../services/player-service');
const { getTopbarLocals } = require('../services/topbar-service'); const { getTopbarLocals } = require('../services/topbar-service');
const { calculatePredictedRating } = require('../services/rating-calculator'); const { calculatePredictedRating } = require('../services/rating-calculator');
const logger = require('../logger'); const logger = require('../logger');
@@ -43,6 +43,8 @@ router.get('/partials/ratings-table', async (req, res) => {
} }
}); });
// Used only by the per-player "refresh rating history" button. The initial table render
// pre-attaches history via getAllRatingsFromDB to avoid the load-then-fetch race.
router.get('/partials/player-history/:pdgaNumber', async (req, res) => { router.get('/partials/player-history/:pdgaNumber', async (req, res) => {
try { try {
const { pdgaNumber } = req.params; const { pdgaNumber } = req.params;
@@ -61,7 +63,7 @@ router.get('/partials/player-history/:pdgaNumber', async (req, res) => {
const formattedHistory = (history || []).map(row => ({ const formattedHistory = (history || []).map(row => ({
date: row.date, date: row.date,
rating: row.rating, rating: row.rating,
displayDate: new Date(row.date).toLocaleDateString('en-US', { day: '2-digit', month: 'short', year: 'numeric' }) displayDate: formatDisplayDate(row.date)
})); }));
const player = await getPlayerDataFromDB(pdgaNumber); const player = await getPlayerDataFromDB(pdgaNumber);
@@ -117,11 +119,7 @@ router.get('/api/rating-history/:pdgaNumber', async (req, res) => {
const formattedHistory = cachedHistory.map(row => ({ const formattedHistory = cachedHistory.map(row => ({
date: row.date, date: row.date,
rating: row.rating, rating: row.rating,
displayDate: new Date(row.date).toLocaleDateString('en-US', { displayDate: formatDisplayDate(row.date)
day: '2-digit',
month: 'short',
year: 'numeric'
})
})); }));
res.json({ res.json({
+18 -3
View File
@@ -1,9 +1,15 @@
const { db } = require('../db'); const { db } = require('../db');
const { getPlayerFromDB, getRoundHistoryFromDB, savePredictedRatingToDB, savePlayerToDB, getMonthlyHistory, getAllMonthlyHistoriesFromDB } = require('../models/player'); const { getPlayerFromDB, getRoundHistoryFromDB, savePredictedRatingToDB, savePlayerToDB, getMonthlyHistory, getAllMonthlyHistoriesFromDB, getAllRatingHistoriesFromDB } = require('../models/player');
const { fetchPlayerDataHTTP, parsePlayerData } = require('../scrapers/player-http'); const { fetchPlayerDataHTTP, parsePlayerData } = require('../scrapers/player-http');
const { calculatePredictedRating } = require('./rating-calculator'); const { calculatePredictedRating } = require('./rating-calculator');
const logger = require('../logger'); const logger = require('../logger');
function formatDisplayDate(dateStr) {
return new Date(dateStr).toLocaleDateString('en-US', {
day: '2-digit', month: 'short', year: 'numeric'
});
}
// Derives previous-month rating and the delta to it. Prefers PDGA's reported // Derives previous-month rating and the delta to it. Prefers PDGA's reported
// rating_change (canonical), falls back to our own monthly snapshots when // rating_change (canonical), falls back to our own monthly snapshots when
// rating_change is missing — common for players whose latest scrape failed. // rating_change is missing — common for players whose latest scrape failed.
@@ -167,6 +173,7 @@ async function getAllRatingsFromDB(progressCallback = null) {
// Fetch all monthly histories in one query so the per-player loop doesn't add N extra queries // Fetch all monthly histories in one query so the per-player loop doesn't add N extra queries
const monthlyHistoryMap = await getAllMonthlyHistoriesFromDB(12); const monthlyHistoryMap = await getAllMonthlyHistoriesFromDB(12);
const ratingHistoryMap = await getAllRatingHistoriesFromDB();
const ratings = []; const ratings = [];
const total = allPlayers.length; const total = allPlayers.length;
@@ -189,6 +196,12 @@ async function getAllRatingsFromDB(progressCallback = null) {
if (playerData) { if (playerData) {
playerData.monthlyHistory = monthlyHistoryMap.get(pdgaNumber) ?? []; playerData.monthlyHistory = monthlyHistoryMap.get(pdgaNumber) ?? [];
const rawHistory = ratingHistoryMap.get(pdgaNumber) ?? [];
playerData.ratingHistory = rawHistory.map(row => ({
date: row.date,
rating: row.rating,
displayDate: formatDisplayDate(row.date)
}));
// Re-derive now that history is attached — bulk path skipped includeMonthlyHistory // Re-derive now that history is attached — bulk path skipped includeMonthlyHistory
const derived = deriveMonthlyDeltas(playerData.rating, player.rating_change, playerData.monthlyHistory); const derived = deriveMonthlyDeltas(playerData.rating, player.rating_change, playerData.monthlyHistory);
playerData.lastMonthRating = derived.lastMonthRating; playerData.lastMonthRating = derived.lastMonthRating;
@@ -218,7 +231,8 @@ async function getAllRatingsFromDB(progressCallback = null) {
stdDev: null, stdDev: null,
lastMonthRating: (errorRating != null && errorRatingChange != null) ? errorRating - errorRatingChange : null, lastMonthRating: (errorRating != null && errorRatingChange != null) ? errorRating - errorRatingChange : null,
deltaPredicted: null, deltaPredicted: null,
monthlyHistory: [] monthlyHistory: [],
ratingHistory: []
}; };
ratings.push(errorData); ratings.push(errorData);
@@ -344,5 +358,6 @@ module.exports = {
getPredictedRatingFromDB, getPredictedRatingFromDB,
getAllRatingsFromDB, getAllRatingsFromDB,
refreshAllPlayersInDB, refreshAllPlayersInDB,
computeKpis computeKpis,
formatDisplayDate
}; };
+9 -12
View File
@@ -1,6 +1,10 @@
const { getLastRefresh } = require('../models/player'); const { getLastRefresh } = require('../models/player');
const { getNextPDGAUpdateDate } = require('./rating-calculator');
const logger = require('../logger'); const logger = require('../logger');
const DAY_NAMES = ['Sun', 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat'];
const MONTH_NAMES = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'];
function formatRelative(isoString) { function formatRelative(isoString) {
if (!isoString) return 'Never'; if (!isoString) return 'Never';
const then = new Date(isoString.replace(' ', 'T') + (isoString.endsWith('Z') ? '' : 'Z')); const then = new Date(isoString.replace(' ', 'T') + (isoString.endsWith('Z') ? '' : 'Z'));
@@ -18,25 +22,18 @@ function formatRelative(isoString) {
return then.toISOString().slice(0, 10); return then.toISOString().slice(0, 10);
} }
// First Tuesday of next month — approximation of PDGA's monthly cycle function formatNextUpdate(date) {
function computeNextUpdate(now = new Date()) { return `${DAY_NAMES[date.getDay()]} ${date.getDate()} ${MONTH_NAMES[date.getMonth()]}`;
const year = now.getUTCFullYear();
const month = now.getUTCMonth() + 1; // next month, may roll over
const candidate = new Date(Date.UTC(month === 12 ? year + 1 : year, month === 12 ? 0 : month, 1));
// 0=Sun, 1=Mon, 2=Tue
const offset = (2 - candidate.getUTCDay() + 7) % 7;
candidate.setUTCDate(1 + offset);
const months = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'];
return `${['Sun', 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat'][candidate.getUTCDay()]} ${candidate.getUTCDate()} ${months[candidate.getUTCMonth()]}`;
} }
async function getTopbarLocals() { async function getTopbarLocals() {
const nextUpdate = formatNextUpdate(getNextPDGAUpdateDate());
try { try {
const lastIso = await getLastRefresh(); const lastIso = await getLastRefresh();
return { lastRefresh: formatRelative(lastIso), nextUpdate: computeNextUpdate() }; return { lastRefresh: formatRelative(lastIso), nextUpdate };
} catch (err) { } catch (err) {
logger.warn({ err }, 'topbar locals fallback'); logger.warn({ err }, 'topbar locals fallback');
return { lastRefresh: 'Unknown', nextUpdate: computeNextUpdate() }; return { lastRefresh: 'Unknown', nextUpdate };
} }
} }
+1 -1
View File
@@ -109,7 +109,7 @@
activePage: 'players', activePage: 'players',
cssFiles: ['players.css'], cssFiles: ['players.css'],
jsFiles: ['tooltips.js', 'chart.js', 'players.js'], jsFiles: ['tooltips.js', 'chart.js', 'players.js'],
initScript: 'setupTooltipsAfterSwap();', initScript: 'setupAfterTableSwap();',
body: body, body: body,
modals: modals modals: modals
}) %> }) %>
+6 -2
View File
@@ -89,8 +89,12 @@ function renderSparkline(values) {
</tr> </tr>
<tr id="history-<%= player.pdgaNumber %>" class="expanded-content"> <tr id="history-<%= player.pdgaNumber %>" class="expanded-content">
<td colspan="5" class="expanded-cell"> <td colspan="5" class="expanded-cell">
<div id="history-content-<%= player.pdgaNumber %>"> <div id="history-content-<%= player.pdgaNumber %>" data-loaded="true">
<div class="loading-chart">Click to load rating history...</div> <%- include('player-history', {
pdgaNumber: player.pdgaNumber,
history: player.ratingHistory || [],
player: player
}) %>
</div> </div>
</td> </td>
</tr> </tr>