Compare commits

...

13 Commits

Author SHA1 Message Date
Samuel Enocsson 46f78b42dc 1.2.6
Build and deploy / build-and-push (push) Successful in 20s
Build and deploy / deploy (push) Successful in 3s
2026-05-22 15:27:42 +02:00
Samuel Enocsson d0c278ea1b ci: use full Gitea URL for package-updater-action 2026-05-22 15:27:30 +02:00
Samuel Enocsson d0040901ab 1.2.5
Build and deploy / build-and-push (push) Successful in 21s
Build and deploy / deploy (push) Failing after 2s
2026-05-22 15:25:51 +02:00
Samuel Enocsson c0f9dd5f33 ci: fix updater secret name (UPDATER_API_KEY) 2026-05-22 15:25:38 +02:00
Samuel Enocsson 66e892893f 1.2.4
Build and deploy / build-and-push (push) Successful in 1m29s
Build and deploy / deploy (push) Failing after 2s
2026-05-22 15:23:19 +02:00
Samuel Enocsson 5b9138da25 ci: fix deploy.yml and consolidate workflows
Fix YAML indentation (on:/jobs: were nested under name:), correct
image name from shcizo/myapp to shcizo/pdga-rating, switch registry
auth from GITEA_TOKEN to PACKAGES_TOKEN (auto-injected token lacks
effective registry access), and add packages:write permission.

Remove docker-build.yml since deploy.yml now covers build + push +
deploy in one workflow — previously both triggered on tag push and
raced for the same image tags.
2026-05-22 15:22:41 +02:00
Samuel Enocsson b1e8d63a63 added deploy 2026-05-22 15:15:18 +02:00
shcizo c6ac174921 Merge pull request 'fix: preload player rating history to fix first-click chart render (#10)' (#15) from fix/preload-player-history-10 into main 2026-05-22 12:59:26 +02:00
Samuel Enocsson fba1bea247 refactor: address review feedback — extract date helper, rename listener 2026-05-22 11:47:47 +02:00
Samuel Enocsson a63da6f3ca fix: preload player rating history to fix first-click chart render (#10) 2026-05-22 11:41:38 +02:00
shcizo 3cfdc305ec Merge pull request 'fix: header "Next update" uses second Tuesday (closes #12)' (#13) from fix/header-next-update-date into main
Reviewed-on: #13
2026-05-22 10:00:39 +02:00
Samuel Enocsson d336156bbb fix: header "Next update" uses second Tuesday (closes #12)
The topbar showed the first Tuesday of the *next* month instead of
PDGA's actual cycle (second Tuesday of the month). Replace the
duplicated computeNextUpdate() with the central
getNextPDGAUpdateDate() from rating-calculator, keeping only the
formatter ("Tue 9 Jun") here.
2026-05-22 09:54:57 +02:00
Samuel Enocsson e1b9e97484 docs: update CLAUDE.md for Gitea migration and PDGA domain notes
- Fix runtime line (Node 22 slim, not Node 18 Alpine)
- Add Hosting line (Gitea, use tea not gh)
- Reflect new CI/CD flow (Gitea Actions, manual version bump, PACKAGES_TOKEN)
- Add PDGA Domain Notes section (rating cycle, predicted-rating algorithm,
  rate limits) so future sessions don't have to re-derive domain logic
- Note absence of test framework explicitly
2026-05-22 09:35:54 +02:00
12 changed files with 147 additions and 83 deletions
+42
View File
@@ -0,0 +1,42 @@
name: Build and deploy
on:
push:
tags:
- "v*"
jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- name: Login to Gitea registry
uses: docker/login-action@v3
with:
registry: gitea.shcizo.se
username: ${{ gitea.actor }}
password: ${{ secrets.PACKAGES_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: |
gitea.shcizo.se/shcizo/pdga-rating:${{ gitea.ref_name }}
gitea.shcizo.se/shcizo/pdga-rating:latest
deploy:
runs-on: ubuntu-latest
needs: [build-and-push]
steps:
- uses: https://gitea.shcizo.se/shcizo/package-updater-action@v1
with:
endpoint: https://updater.shcizo.se/update
image: gitea.shcizo.se/shcizo/pdga-rating
tag: ${{ gitea.ref_name }}
token: ${{ secrets.UPDATER_API_KEY }}
-37
View File
@@ -1,37 +0,0 @@
name: Docker Build
on:
push:
tags:
- 'v*'
jobs:
docker:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- uses: docker/login-action@v3
with:
registry: gitea.shcizo.se
username: ${{ gitea.actor }}
password: ${{ secrets.PACKAGES_TOKEN }}
- uses: docker/metadata-action@v5
id: meta
with:
images: gitea.shcizo.se/shcizo/pdga-rating
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,value=latest
- uses: docker/build-push-action@v6
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
+14 -3
View File
@@ -4,13 +4,14 @@ PDGA rating scraper and display app. Scrapes player ratings and course data from
## Tech Stack
- **Runtime:** Node.js 18 (Alpine in Docker)
- **Runtime:** Node.js 22 (slim/Debian-based in Docker)
- **Hosting:** Gitea (`gitea.shcizo.se/shcizo/pdga-rating`) — use `tea` CLI for issues/PRs, not `gh`
- **Server:** Express with EJS templates
- **Database:** SQLite3 (file: `ratings.db`, Docker: `/app/data/ratings.db`)
- **Frontend:** HTMX + vanilla JS (in `public/js/`)
- **Scraping:** Puppeteer (with stealth plugin) + direct HTTP
- **Logging:** Pino (JSON in production, pino-pretty in dev)
- **CI/CD:** Release Please + Docker build/push to GHCR
- **CI/CD:** Gitea Actions (tag-triggered docker build/push to `gitea.shcizo.se/shcizo/pdga-rating`)
## Project Structure
@@ -38,15 +39,25 @@ public/
- `LOG_LEVEL=debug npm start` — Enable debug logging
- `docker compose up` — Run via Docker
**No test framework or lint setup**`package.json` has only `start` and `dev` scripts. If adding either, document it here.
## Conventions
- **Logging:** Use `require('./logger')` (or relative path). Never use `console.log/error` in backend code. Use appropriate Pino levels: `debug` for verbose/diagnostic data, `info` for operational status, `warn` for retries/degraded state, `error` for failures, `fatal` for startup crashes.
- **Frontend JS:** `console.error` is fine in `public/js/` — runs in browser, no Pino.
- **Commits:** Conventional commits (`feat:`, `fix:`, `refactor:`, `chore:`) — drives release-please.
- **Commits:** Conventional commits (`feat:`, `fix:`, `refactor:`, `chore:`, `ci:`).
- **Releases:** Manual version bump — edit `version` in `package.json` + `package-lock.json`, commit as `<version>`, tag `v<version>`, push commit + tag (`git push origin main v<version>`). Triggers `.gitea/workflows/docker-build.yml` which builds and pushes the image. Auth uses repo secret `PACKAGES_TOKEN` (PAT with `write:package`) — the auto-injected `GITEA_TOKEN` does not have effective registry access.
- **Scraping:** Two strategies per entity: direct HTTP (fast, preferred) with Puppeteer fallback (stealth plugin for anti-bot). Rate limiting must be respected.
- **Database:** Migrations run automatically on startup in `db.js`. Schema changes go there.
- **Templates:** EJS with shared layout in `views/partials/`. Pages use HTMX for dynamic content loading.
## PDGA Domain Notes
- **Rating publication cycle:** PDGA officially recalculates ratings on the **second Tuesday of each month**. `getNextPDGAUpdateDate()` in `src/services/rating-calculator.js` computes this — round filtering uses it as cutoff.
- **Predicted rating algorithm:** `calculatePredictedRating(roundRatings)` replicates PDGA's formula — 12-mo window (expands to 24 if <8 rounds), outlier removal at ≥7 rounds (2.5σ + 100pt threshold), double-weighting of recent 25% at ≥9 rounds. Returns `{rating, stdDev, debugLog}`.
- **Rate limits:** `POST /api/refresh-round-history/:pdgaNumber` enforces a 24h cooldown per player (`src/routes/players.js`). Don't bypass — PDGA's site rate-limits aggressively.
- **Round history refresh** uses Puppeteer (stealth plugin), other scraping prefers direct HTTP. Predicted rating is recomputed and stored on each refresh.
## Environment Variables
| Variable | Default | Description |
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "pdga-ratings",
"version": "1.2.3",
"version": "1.2.6",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "pdga-ratings",
"version": "1.2.3",
"version": "1.2.6",
"dependencies": {
"ejs": "^4.0.1",
"express": "^4.18.2",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "pdga-ratings",
"version": "1.2.3",
"version": "1.2.6",
"description": "PDGA rating scraper and display",
"main": "server.js",
"scripts": {
+17 -10
View File
@@ -26,23 +26,30 @@ function applyDeltaPill(pillEl, value) {
pillEl.appendChild(numSpan);
}
function setupTooltipsAfterSwap() {
document.body.addEventListener('htmx:afterSwap', function(event) {
if (event.detail.target.id === 'ratings-table') {
initRatingsTooltips();
}
// After player history partial loads, render the chart
const target = event.detail.target;
if (target.id && target.id.startsWith('history-content-')) {
const container = target.querySelector('.player-chart, .chart-container');
if (container && container.dataset.history) {
function initChartsIn(rootEl) {
rootEl.querySelectorAll('.player-chart').forEach(function(container) {
if (container.dataset.charted === 'true') return;
if (!container.dataset.history) return;
try {
const history = JSON.parse(container.dataset.history);
createRatingChart(container, history);
container.dataset.charted = 'true';
} catch (e) {
console.error('Error rendering chart:', e);
}
});
}
function setupAfterTableSwap() {
document.body.addEventListener('htmx:afterSwap', function(event) {
const target = event.detail.target;
if (target.id === 'ratings-table') {
initRatingsTooltips();
initChartsIn(target);
return;
}
if (target.id && target.id.startsWith('history-content-')) {
initChartsIn(target);
}
});
}
+28 -1
View File
@@ -253,6 +253,32 @@ function getAllMonthlyHistoriesFromDB(months = 12) {
});
}
/**
* Fetches the full rating history for ALL players in one query.
* Returns Map<pdgaNumber, {rating, date}[]> ordered chronologically (oldest → newest).
* Mirrors getAllMonthlyHistoriesFromDB but returns every point, not monthly snapshots.
*/
function getAllRatingHistoriesFromDB() {
return new Promise((resolve, reject) => {
db.all(
`SELECT p.pdga_number, rh.date, rh.rating
FROM rating_history rh
JOIN players p ON rh.player_id = p.id
ORDER BY p.pdga_number, rh.date ASC`,
[],
(err, rows) => {
if (err) return reject(err);
const map = new Map();
for (const row of rows) {
if (!map.has(row.pdga_number)) map.set(row.pdga_number, []);
map.get(row.pdga_number).push({ date: row.date, rating: row.rating });
}
resolve(map);
}
);
});
}
function getLastRefresh() {
return new Promise((resolve, reject) => {
db.get(
@@ -278,5 +304,6 @@ module.exports = {
savePredictedRatingToDB,
getLastRefresh,
getMonthlyHistory,
getAllMonthlyHistoriesFromDB
getAllMonthlyHistoriesFromDB,
getAllRatingHistoriesFromDB
};
+5 -7
View File
@@ -5,7 +5,7 @@ const { getPlayerFromDB, savePlayerToDB, getRatingHistoryFromDB, saveRatingHisto
const { fetchPlayerDataHTTP, parsePlayerData, fetchRatingHistory, parseRatingHistory } = require('../scrapers/player-http');
const { getOfficialRatingHistory, getOptimizedPlayerRounds } = require('../scrapers/player-puppeteer');
const { launchBrowser } = require('../scrapers/browser');
const { getPlayerDataFromDB, scrapePDGARating, getAllRatingsFromDB, refreshAllPlayersInDB, getPredictedRatingFromDB } = require('../services/player-service');
const { getPlayerDataFromDB, scrapePDGARating, getAllRatingsFromDB, refreshAllPlayersInDB, getPredictedRatingFromDB, formatDisplayDate } = require('../services/player-service');
const { getTopbarLocals } = require('../services/topbar-service');
const { calculatePredictedRating } = require('../services/rating-calculator');
const logger = require('../logger');
@@ -43,6 +43,8 @@ router.get('/partials/ratings-table', async (req, res) => {
}
});
// Used only by the per-player "refresh rating history" button. The initial table render
// pre-attaches history via getAllRatingsFromDB to avoid the load-then-fetch race.
router.get('/partials/player-history/:pdgaNumber', async (req, res) => {
try {
const { pdgaNumber } = req.params;
@@ -61,7 +63,7 @@ router.get('/partials/player-history/:pdgaNumber', async (req, res) => {
const formattedHistory = (history || []).map(row => ({
date: row.date,
rating: row.rating,
displayDate: new Date(row.date).toLocaleDateString('en-US', { day: '2-digit', month: 'short', year: 'numeric' })
displayDate: formatDisplayDate(row.date)
}));
const player = await getPlayerDataFromDB(pdgaNumber);
@@ -117,11 +119,7 @@ router.get('/api/rating-history/:pdgaNumber', async (req, res) => {
const formattedHistory = cachedHistory.map(row => ({
date: row.date,
rating: row.rating,
displayDate: new Date(row.date).toLocaleDateString('en-US', {
day: '2-digit',
month: 'short',
year: 'numeric'
})
displayDate: formatDisplayDate(row.date)
}));
res.json({
+18 -3
View File
@@ -1,9 +1,15 @@
const { db } = require('../db');
const { getPlayerFromDB, getRoundHistoryFromDB, savePredictedRatingToDB, savePlayerToDB, getMonthlyHistory, getAllMonthlyHistoriesFromDB } = require('../models/player');
const { getPlayerFromDB, getRoundHistoryFromDB, savePredictedRatingToDB, savePlayerToDB, getMonthlyHistory, getAllMonthlyHistoriesFromDB, getAllRatingHistoriesFromDB } = require('../models/player');
const { fetchPlayerDataHTTP, parsePlayerData } = require('../scrapers/player-http');
const { calculatePredictedRating } = require('./rating-calculator');
const logger = require('../logger');
function formatDisplayDate(dateStr) {
return new Date(dateStr).toLocaleDateString('en-US', {
day: '2-digit', month: 'short', year: 'numeric'
});
}
// Derives previous-month rating and the delta to it. Prefers PDGA's reported
// rating_change (canonical), falls back to our own monthly snapshots when
// rating_change is missing — common for players whose latest scrape failed.
@@ -167,6 +173,7 @@ async function getAllRatingsFromDB(progressCallback = null) {
// Fetch all monthly histories in one query so the per-player loop doesn't add N extra queries
const monthlyHistoryMap = await getAllMonthlyHistoriesFromDB(12);
const ratingHistoryMap = await getAllRatingHistoriesFromDB();
const ratings = [];
const total = allPlayers.length;
@@ -189,6 +196,12 @@ async function getAllRatingsFromDB(progressCallback = null) {
if (playerData) {
playerData.monthlyHistory = monthlyHistoryMap.get(pdgaNumber) ?? [];
const rawHistory = ratingHistoryMap.get(pdgaNumber) ?? [];
playerData.ratingHistory = rawHistory.map(row => ({
date: row.date,
rating: row.rating,
displayDate: formatDisplayDate(row.date)
}));
// Re-derive now that history is attached — bulk path skipped includeMonthlyHistory
const derived = deriveMonthlyDeltas(playerData.rating, player.rating_change, playerData.monthlyHistory);
playerData.lastMonthRating = derived.lastMonthRating;
@@ -218,7 +231,8 @@ async function getAllRatingsFromDB(progressCallback = null) {
stdDev: null,
lastMonthRating: (errorRating != null && errorRatingChange != null) ? errorRating - errorRatingChange : null,
deltaPredicted: null,
monthlyHistory: []
monthlyHistory: [],
ratingHistory: []
};
ratings.push(errorData);
@@ -344,5 +358,6 @@ module.exports = {
getPredictedRatingFromDB,
getAllRatingsFromDB,
refreshAllPlayersInDB,
computeKpis
computeKpis,
formatDisplayDate
};
+9 -12
View File
@@ -1,6 +1,10 @@
const { getLastRefresh } = require('../models/player');
const { getNextPDGAUpdateDate } = require('./rating-calculator');
const logger = require('../logger');
const DAY_NAMES = ['Sun', 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat'];
const MONTH_NAMES = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'];
function formatRelative(isoString) {
if (!isoString) return 'Never';
const then = new Date(isoString.replace(' ', 'T') + (isoString.endsWith('Z') ? '' : 'Z'));
@@ -18,25 +22,18 @@ function formatRelative(isoString) {
return then.toISOString().slice(0, 10);
}
// First Tuesday of next month — approximation of PDGA's monthly cycle
function computeNextUpdate(now = new Date()) {
const year = now.getUTCFullYear();
const month = now.getUTCMonth() + 1; // next month, may roll over
const candidate = new Date(Date.UTC(month === 12 ? year + 1 : year, month === 12 ? 0 : month, 1));
// 0=Sun, 1=Mon, 2=Tue
const offset = (2 - candidate.getUTCDay() + 7) % 7;
candidate.setUTCDate(1 + offset);
const months = ['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'];
return `${['Sun', 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat'][candidate.getUTCDay()]} ${candidate.getUTCDate()} ${months[candidate.getUTCMonth()]}`;
function formatNextUpdate(date) {
return `${DAY_NAMES[date.getDay()]} ${date.getDate()} ${MONTH_NAMES[date.getMonth()]}`;
}
async function getTopbarLocals() {
const nextUpdate = formatNextUpdate(getNextPDGAUpdateDate());
try {
const lastIso = await getLastRefresh();
return { lastRefresh: formatRelative(lastIso), nextUpdate: computeNextUpdate() };
return { lastRefresh: formatRelative(lastIso), nextUpdate };
} catch (err) {
logger.warn({ err }, 'topbar locals fallback');
return { lastRefresh: 'Unknown', nextUpdate: computeNextUpdate() };
return { lastRefresh: 'Unknown', nextUpdate };
}
}
+1 -1
View File
@@ -109,7 +109,7 @@
activePage: 'players',
cssFiles: ['players.css'],
jsFiles: ['tooltips.js', 'chart.js', 'players.js'],
initScript: 'setupTooltipsAfterSwap();',
initScript: 'setupAfterTableSwap();',
body: body,
modals: modals
}) %>
+6 -2
View File
@@ -89,8 +89,12 @@ function renderSparkline(values) {
</tr>
<tr id="history-<%= player.pdgaNumber %>" class="expanded-content">
<td colspan="5" class="expanded-cell">
<div id="history-content-<%= player.pdgaNumber %>">
<div class="loading-chart">Click to load rating history...</div>
<div id="history-content-<%= player.pdgaNumber %>" data-loaded="true">
<%- include('player-history', {
pdgaNumber: player.pdgaNumber,
history: player.ratingHistory || [],
player: player
}) %>
</div>
</td>
</tr>