Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
f797390
fix(importer): harden AiReviewAnalyzer config, summary, and vote boun…
JOY Oct 24, 2026
ad89f35
fix(importer): enforce integer RAM/storage bounds and asset checks (#…
JOY Oct 24, 2026
f1a1352
fix(importer): clamp fetchAllPages bounds and harden merchant risk (#…
JOY Oct 24, 2026
5cca9e6
fix(importer): harden computeRetryDelayMs, logger redaction, and merg…
JOY Oct 24, 2026
e5b0fc7
fix(web): harden cache-tags targets, api-handler bounds, and cookie t…
JOY Oct 24, 2026
cf37bcc
fix(web): harden structured logger redaction and extension URL/payloa…
JOY Oct 24, 2026
fa86dcf
fix(extension): harden security-guards and detector JSON-LD/ID bounds…
JOY Oct 24, 2026
17518ec
fix(web): harden csv-safe, seo, merchant-directory, and sold-count bo…
JOY Oct 24, 2026
74e603e
fix(web): harden offer-plausibility, competitors, review-display, and…
JOY Oct 24, 2026
26f2ca5
fix(web): harden deal-radar, leaderboard-engine, category-filters, an…
JOY Oct 24, 2026
8aeaa86
fix(web): harden catalog-listing, analyzer-copy, product-detail-copy,…
JOY Oct 24, 2026
e816004
fix(web): harden navigation, merchant-directory, hero suggestions, an…
JOY Oct 24, 2026
c79942a
fix(web): harden server rate deals radar and cached query key bounds …
JOY Oct 24, 2026
3aaf2dd
fix(web): harden review vote eligibility, community review mapper, an…
JOY Oct 24, 2026
f4785af
fix(importer): harden peer floor, merchant risk, and arena/entity bou…
JOY Oct 24, 2026
82b1015
fix(web): harden API handler price/IP bounds, watchlist offers, and c…
JOY Oct 24, 2026
5eaeb5f
fix(web): deal radar counts every offer of a cluster (#867) (#869)
JOY Oct 24, 2026
aaa730f
fix(web): harden inspect observed prices, health TTL clock guard, and…
JOY Oct 24, 2026
afbe1f5
feat(ops): commit agent release helpers and report the live commit (#…
JOY Oct 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,9 @@ jobs:
- name: Test the E2E stale-stack pruning
run: node --test scripts/e2e/stale-stacks.test.mjs

- name: Test the agent release scripts
run: node --test scripts/ops/merge-when-green.test.mjs scripts/ops/board.test.mjs scripts/ops/promote.test.mjs scripts/ops/verify-live.test.mjs

- name: Replay rate migrations on PostgreSQL 17 (Docker)
run: node scripts/test-migration-replay.mjs

Expand Down
7 changes: 7 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@ Tài liệu này tổng hợp đầy đủ **User-Level Hard Rules (`C:\Users\JO
- **TUYỆT ĐỐI KHÔNG** chạy `vercel deploy --prod`, `vercel --prod`, `--prebuilt --prod`, `--skip-domain` rồi `promote`, hay promote một bản Preview của `dev` lên production, kể cả khi JOY đã cho phép "deploy". Lời cho phép deploy nghĩa là merge PR promote `dev` -> `main`, không phải deploy từ máy local.
- Chốt kỹ thuật: `scripts/vercel-production-guard.mjs` (chạy trong `buildCommand` của `vercel.json`) làm build production thất bại nếu không phải GitHub build từ `DOS/Rate@main`; workflow `Validate PR Base` chặn PR vào `main` không từ `dev`; workflow `Branch Divergence` báo lỗi khi `main` có commit mà `dev` không có. Không được sửa hay vô hiệu hóa các chốt này để deploy cho nhanh.
- Ngoại lệ duy nhất: **rollback** về một bản production cũ đã build từ Git (`vercel rollback`), vì không build lại. Sau đó sửa tiếp qua PR hotfix.
- **Script vận hành có sẵn trong repo (`scripts/ops/`, dùng thay cho lệnh gõ tay):**
- `node scripts/ops/merge-when-green.mjs <pr>`: chờ commit head xanh rồi merge, ghim theo SHA head. Xanh nghĩa là mọi check run và mọi commit status (Vercel) đều đạt, và các check mà ruleset bắt buộc đã báo kết quả. PR vào `main` luôn dùng merge commit, PR khác mặc định squash. `--ignore "<check>"` chỉ dùng khi một check không bắt buộc đang hỏng có lý do đã ghi trong PR.
- `node scripts/ops/promote.mjs`: mở hoặc dùng lại PR promote `dev` -> `main`, liệt kê các PR được phát hành, merge bằng merge commit khi CI xanh.
- `node scripts/ops/verify-live.mjs --wait 900`: kiểm tra rate.vn chạy đúng commit của `main` (`/api/health` trả `commit`), database đọc được và các route chính trả 200. Script đọc lại `main` mỗi vòng chờ. Chỉ khi bản live chậm hơn `main`, nó mới in lệnh `vercel promote <deployment của main> --scope metados`; nếu bản live đi trước hoặc lệch nhánh so với `main`, nó dặn không làm gì. Không bao giờ redeploy hay promote một deployment build từ commit khác (`promote` không build lại, và bật lại tự gán domain sau một lần `vercel rollback`).
- `node scripts/ops/board.mjs <issue> --status in-progress|in-review|done --agent <Nick>`: cập nhật Project #9 và nhãn (xem mục 1). Script từ chối issue đang mang nhãn `agent:*` của agent khác; `--take-over` chỉ dùng khi agent đó đã bàn giao hoặc JOY chỉ định.

7. **Chủ động thực thi liên tục & đề xuất việc tiếp theo (Proactive Next-Task):**
- **Khi đang ở chế độ `/goal` hoặc được giao làm hết backlog:** Tự động tạo/nhận và làm liên tục từng task đến khi sạch toàn bộ backlog mà **KHÔNG dừng lại hỏi `Bắt đầu chứ?`**.
Expand Down Expand Up @@ -75,6 +80,8 @@ gh issue list -R DOS/Rate-Services --state open
```
- **Chuyển trạng thái sang `In Progress` & Ghi danh Agent ngay khi bắt đầu (BẮT BUỘC đồng bộ cả Issue lẫn Board Project #9, comment bằng Tiếng Anh):**

Cách nhanh, một lệnh làm cả nhãn lẫn board (tự thêm issue vào Project #9 nếu chưa có, chỉ giữ một nhãn `status:*`): `node scripts/ops/board.mjs <ISSUE_NUMBER> --status in-progress --agent $agentNick`. Các trạng thái khác: `ready`, `in-review`, `done`, `backlog`. Các lệnh `gh` dưới đây là cách làm tương đương bằng tay.

```powershell
$agentNick = "Crawl" # Nickname của Agent hiện tại
$agentLabel = "agent:$($agentNick.ToLower())"
Expand Down
3 changes: 2 additions & 1 deletion Docs/Resources/RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,9 @@ This is the release workflow for `rate.vn`. It follows the same model as DOS/DOS
`gh pr create --repo DOS/Rate --base main --head dev --title "chore(release): promote dev to main"`
List the PRs being released in the body.
4. Wait for the required checks on the promotion PR: `Typecheck & Build Monorepo`, `PR into main comes from dev`, `main must not lead dev`.
`node scripts/ops/promote.mjs` does steps 3 to 5 in one command: it opens the promotion PR from `dev` (or reuses the open one) with the released PRs listed, then merges it with a merge commit once its checks pass. `--no-merge` leaves it open.
5. Promotions are batched and need no per-release question: the agent promotes at the first checkpoint reached: the backlog runs out, 10 or more PRs are waiting on `dev`, the oldest unpromoted change is more than 24 hours old, or the agent is ending a long working session. Promote only when `dev` head CI is green and no PR is mid-way through fixing a production bug. The agent merges it with a merge commit: `gh pr merge <number> --repo DOS/Rate --merge`. JOY does not merge PRs; never ask JOY to click Merge. **Never squash or rebase a promotion.** Squashing creates a commit on `main` that can never exist on `dev`, and the next promotion then diffs from a stale merge base.
6. Vercel builds `main` and assigns `rate.vn`. Verify the live revision (the deployment's Git SHA equals `main`), `/api/health`, the main routes and a browser check before calling the release done. To check a build before it takes traffic, create it from Git with `autoAssignCustomDomains: false` (Vercel API `POST /v13/deployments` with `gitSource` ref `main` and `target: "production"`), verify it with `vercel curl --deployment <id>`, then `vercel promote <id>`. Promoting an existing production deployment does not rebuild.
6. Vercel builds `main` and assigns `rate.vn`. Verify the live revision (the deployment's Git SHA equals `main`), `/api/health`, the main routes and a browser check before calling the release done. `node scripts/ops/verify-live.mjs --wait 900` checks the first three: `/api/health` reports the deployment's `commit`, which must equal `main`, and `checks.database` must be `ok`. It re-reads `main` on every poll. Only when the live commit is behind `main` does it print `vercel promote <main's production deployment> --scope metados`; when live is ahead of or diverged from `main` it says to do nothing. Never redeploy or promote a deployment built from another commit: that rolls the release back. `promote` does not rebuild, and it turns production domain auto-assignment back on after a `vercel rollback`. To check a build before it takes traffic, create it from Git with `autoAssignCustomDomains: false` (Vercel API `POST /v13/deployments` with `gitSource` ref `main` and `target: "production"`), verify it with `vercel curl --deployment <id>`, then `vercel promote <id>`. Promoting an existing production deployment does not rebuild.

## Hotfix

Expand Down
70 changes: 55 additions & 15 deletions apps/extension/src/extractors/ecommerce-detector.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,17 @@ import type {
* Strip all tracking query strings (sp_atk, xptdk, utm_*, fbclid, etc.) and hashes.
*/
export function sanitizeProductUrl(rawUrl: string): string | null {
if (typeof rawUrl !== 'string') return null;
const cleaned = rawUrl.replace(/\p{Cc}+/gu, '').trim();
if (!cleaned || cleaned.length > 2048) return null;
try {
const parsed = new URL(rawUrl);
const parsed = new URL(cleaned);
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') {
return null;
}
if (parsed.username || parsed.password) {
return null;
}
return `${parsed.origin}${parsed.pathname}`;
} catch {
return null;
Expand All @@ -25,12 +31,18 @@ function isDomainOrSubdomain(host: string, domain: string): boolean {
export function detectEcommercePlatform(
url: string
): SupportedEcommercePlatform | null {
if (typeof url !== 'string') return null;
const cleaned = url.replace(/\p{Cc}+/gu, '').trim();
if (!cleaned || cleaned.length > 2048) return null;
let host: string;
try {
const parsed = new URL(url);
const parsed = new URL(cleaned);
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') {
return null;
}
if (parsed.username || parsed.password) {
return null;
}
host = parsed.hostname.toLowerCase();
} catch {
return null;
Expand Down Expand Up @@ -114,20 +126,20 @@ export function extractIdsFromUrl(
const m =
canonicalUrl.match(/(?:a-)?i\.(\d+)\.(\d+)/i) ||
canonicalUrl.match(/\/(?:product|opaanlp)\/(\d+)\/(\d+)/i);
if (m) {
if (m && m[1] && m[2] && m[1].length <= 24 && m[2].length <= 24) {
return {
externalShopId: m[1] || null,
externalItemId: m[2] || null,
externalShopId: m[1],
externalItemId: m[2],
};
}
return { externalItemId: null, externalShopId: null };
}

if (platform === 'lazada') {
const m = canonicalUrl.match(/-i(\d+)(?:-s(\d+))?\.html/i);
if (m) {
if (m && m[1] && m[1].length <= 24 && (!m[2] || m[2].length <= 24)) {
return {
externalItemId: m[1] || null,
externalItemId: m[1],
externalShopId: m[2] || null,
};
}
Expand All @@ -136,9 +148,9 @@ export function extractIdsFromUrl(

if (platform === 'tiki') {
const m = canonicalUrl.match(/-p(\d+)\.html/i);
if (m) {
if (m && m[1] && m[1].length <= 24) {
return {
externalItemId: m[1] || null,
externalItemId: m[1],
externalShopId: null,
};
}
Expand All @@ -153,9 +165,13 @@ export function extractIdsFromUrl(
canonicalUrl.match(/\/product\/(\d+)/i) ||
canonicalUrl.match(/\/view\/product\/(\d+)/i);
const shopMatch = canonicalUrl.match(/\/@([a-zA-Z0-9_.-]+)/);
const itemId =
prodMatch?.[1] && prodMatch[1].length <= 24 ? prodMatch[1] : null;
const shopId =
shopMatch?.[1] && shopMatch[1].length <= 64 ? shopMatch[1] : null;
return {
externalItemId: prodMatch?.[1] || null,
externalShopId: shopMatch?.[1] || null,
externalItemId: itemId,
externalShopId: shopId,
};
}

Expand All @@ -179,6 +195,8 @@ export function extractIdsFromUrl(
const isValidItemSlug =
Boolean(lastSegment) &&
lastSegment!.length >= 2 &&
lastSegment!.length <= 120 &&
/^[a-zA-Z0-9_.-]+$/.test(lastSegment!) &&
!RETAIL_NON_PRODUCT_FIRST_SEGMENTS.has(firstSegLower) &&
!RETAIL_NON_PRODUCT_FIRST_SEGMENTS.has(lastSegLower) &&
!(segments.length === 1 && RETAIL_CATEGORY_ROOT_SLUGS.has(firstSegLower)) &&
Expand Down Expand Up @@ -225,6 +243,9 @@ const JSON_LD_PRODUCT_TYPES = new Set([
'productmodel',
]);

const MAX_JSON_LD_ENTITIES = 100;
const MAX_JSON_LD_OFFERS = 100;

export function flattenJsonLdEntities(
raw: unknown,
depth = 0
Expand All @@ -233,15 +254,24 @@ export function flattenJsonLdEntities(
if (Array.isArray(raw)) {
const out: Record<string, unknown>[] = [];
for (const entry of raw) {
out.push(...flattenJsonLdEntities(entry, depth + 1));
if (out.length >= MAX_JSON_LD_ENTITIES) break;
const nested = flattenJsonLdEntities(entry, depth + 1);
for (const item of nested) {
if (out.length >= MAX_JSON_LD_ENTITIES) break;
out.push(item);
}
}
return out;
}
if (typeof raw === 'object') {
const obj = raw as Record<string, unknown>;
const out: Record<string, unknown>[] = [obj];
if (Array.isArray(obj['@graph'])) {
out.push(...flattenJsonLdEntities(obj['@graph'], depth + 1));
const nested = flattenJsonLdEntities(obj['@graph'], depth + 1);
for (const item of nested) {
if (out.length >= MAX_JSON_LD_ENTITIES) break;
out.push(item);
}
}
return out;
}
Expand Down Expand Up @@ -270,11 +300,18 @@ export function extractPriceFromJsonLdOffers(offersRaw: unknown): {
}
const candidates: Record<string, unknown>[] = [];
const pushOffer = (entry: unknown) => {
if (!entry || typeof entry !== 'object') return;
if (
!entry ||
typeof entry !== 'object' ||
candidates.length >= MAX_JSON_LD_OFFERS
) {
return;
}
const obj = entry as Record<string, unknown>;
candidates.push(obj);
if (Array.isArray(obj.offers)) {
for (const nested of obj.offers) {
if (candidates.length >= MAX_JSON_LD_OFFERS) break;
if (nested && typeof nested === 'object') {
candidates.push(nested as Record<string, unknown>);
}
Expand All @@ -283,7 +320,10 @@ export function extractPriceFromJsonLdOffers(offersRaw: unknown): {
};

if (Array.isArray(offersRaw)) {
for (const entry of offersRaw) pushOffer(entry);
for (const entry of offersRaw) {
if (candidates.length >= MAX_JSON_LD_OFFERS) break;
pushOffer(entry);
}
} else {
pushOffer(offersRaw);
}
Expand Down
62 changes: 56 additions & 6 deletions apps/extension/src/security-guards.ts
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,35 @@ export function selectFreshestSessionSignal<T extends SessionSignalEntry>(
return tabSignal ?? latestSignal ?? null;
}

const SAFE_EXTERNAL_ID_REGEX = /^[a-zA-Z0-9_.-]{1,120}$/;

function sanitizeExternalIdentifier(val: unknown): string | null {
if (typeof val !== 'string') return null;
const cleaned = val.replace(/\p{Cc}+/gu, '').trim();
if (!cleaned || !SAFE_EXTERNAL_ID_REGEX.test(cleaned)) {
return null;
}
return cleaned;
}

function sanitizeCanonicalSignalUrl(rawUrl: unknown): string {
if (typeof rawUrl !== 'string') return '';
const cleaned = rawUrl.replace(/\p{Cc}+/gu, '').trim();
if (!cleaned || cleaned.length > 2048) return '';
try {
const parsed = new URL(cleaned);
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') {
return '';
}
if (parsed.username || parsed.password) {
return '';
}
return `${parsed.origin}${parsed.pathname}`;
} catch {
return '';
}
}

export function sanitizeExtensionProductSignal(
signal: ExtractedPageProductSignal
): ExtractedPageProductSignal {
Expand Down Expand Up @@ -159,14 +188,27 @@ export function sanitizeExtensionProductSignal(
: null;

const trimmedTitle =
typeof signal.title === 'string' ? signal.title.trim().slice(0, 500) : '';
typeof signal.title === 'string'
? signal.title
.replace(/\p{Cc}+/gu, ' ')
.replace(/\s+/g, ' ')
.trim()
.slice(0, 500)
: '';
const trimmedVariant =
typeof signal.selectedVariant === 'string' && signal.selectedVariant.trim()
? signal.selectedVariant.trim().slice(0, 200)
typeof signal.selectedVariant === 'string'
? signal.selectedVariant
.replace(/\p{Cc}+/gu, ' ')
.replace(/\s+/g, ' ')
.trim()
.slice(0, 200) || null
: null;

return {
...signal,
canonicalUrl: sanitizeCanonicalSignalUrl(signal.canonicalUrl),
externalItemId: sanitizeExternalIdentifier(signal.externalItemId),
externalShopId: sanitizeExternalIdentifier(signal.externalShopId),
title: trimmedTitle,
selectedVariant: trimmedVariant,
price: validPrice,
Expand All @@ -181,13 +223,16 @@ export function escapeSafeExternalHref(
fallbackUrl = 'https://rate.vn'
): string {
if (typeof rawUrl !== 'string') return fallbackUrl;
const trimmed = rawUrl.trim();
if (!trimmed) return fallbackUrl;
const trimmed = rawUrl.replace(/\p{Cc}+/gu, '').trim();
if (!trimmed || trimmed.length > 2048) return fallbackUrl;
try {
const parsed = new URL(trimmed, 'https://rate.vn');
if (parsed.protocol !== 'https:' && parsed.protocol !== 'http:') {
return fallbackUrl;
}
if (parsed.username || parsed.password) {
return fallbackUrl;
}
return parsed.href
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
Expand Down Expand Up @@ -315,15 +360,20 @@ export function sanitizeVerdictClassSuffix(
: 'insufficient_data';
}

export const MAX_SPARKLINE_POINTS = 365;

export function filterValidSparklinePrices(
points: ReadonlyArray<{ price: number }> | null | undefined
): number[] {
if (!Array.isArray(points)) return [];
return points
const valid = points
.map((p) => Number(p?.price))
.filter(
(p) => Number.isFinite(p) && p > 0 && p <= MAX_VALID_EXTENSION_VND_PRICE
);
return valid.length > MAX_SPARKLINE_POINTS
? valid.slice(-MAX_SPARKLINE_POINTS)
: valid;
}

export function shouldRenderTrustPill(
Expand Down
Loading
Loading