Skip to content

docs(proposals): design weighted criteria stars, pros and cons, review value - #1533

Merged
JOY (JOY) merged 1 commit into
devfrom
docs/1531-criteria-stars-pros-cons
Oct 31, 2026
Merged

JOY (JOY) merged 1 commit into
devfrom
docs/1531-criteria-stars-pros-cons

Conversation

@JOY

@JOY JOY (JOY) commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Refs #1531

Design proposal for JOY (Vietnamese, plain words): Docs/Proposals/2026-10-31-criteria-stars-pros-cons.html. Proposal only: no code, no SQL and no data changes. Implementation issues are filed after JOY decides.

What it proposes

  • Criteria per category. Each category gets its own list of criteria and weights. The list lives in packages/shared-data and is versioned. The five fixed keys used today map onto the new lists.
  • A Rate review's star is computed. It is the weighted average of the review's criterion stars, as JReviews does. The API computes it on the server and no longer trusts the browser's number. A new rating_exact numeric(2,1) column keeps the decimal.
  • Required fields for Rate reviews. Rate users and Rate experts must give a verdict, at least one pro and at least one con. Pros and cons per criterion are optional, and so is everything for imported reviews.
  • Product pros and cons.
    • Code clusters each review's lines per criterion, deduplicates them and counts distinct writers by weight. AI only rewords the clusters.
    • A line is shown only when at least 2 distinct people support it with a weight sum of 1.5 or more.
    • At least one con always stays visible.
    • Weights halve every 12 months.
  • Review value score (0 to 100).
    • Built from completeness, specificity, verified purchase, helpful votes (Wilson) and author trust.
    • It sorts reviews and drives a public quality tier. The author sees the number; readers see only the tier.
    • Only the completeness part touches the rating: x1.00 to x1.25 on today's helpfulness_weight.
  • fix(web): per-criterion scores are derived from the star rating and saturate at 5.00 (ties, fake ranks) #1143. Criteria with no data return null and read as "Chưa đủ dữ liệu".
    • They are removed from the leaderboard combined score.
    • 1v1 battles skip them.
    • A criterion score is shown from 3 distinct writers. A criterion rank needs 5.
  • ux(web): the same product shows different ratings across blocks (4.8 / 4.98 / 5.0) #1470. One rating everywhere: the Bayesian estimate, stored on master_products by the refresh function (option 2 of PR fix(web): versus scoreboard shows the product's own rating at one decimal #1478). The expert score is shown separately with a label.
  • Data model, all as DOS-Me migrations for JOY.
    • New rate.review_points and rate.product_points tables.
    • New reviews columns: verdict, criteria version, exact star and value score.
    • New master_products rating columns.
    • A v2 of submit_community_review.
  • Plan. Seven phases, P0 to P6, with sizes; P0 and P1 need no database. Ten decisions, D1 to D10, each with a recommendation.

Review

A fresh-context review checked the page against the code. Its findings, all fixed in the page:

  • the integer rating_star column;
  • the six-axis trust weight is not the stored weight yet;
  • the multiplier math and a double count with trust axis C;
  • weights in TypeScript against the RPC;
  • the interim rule for ux(web): the same product shows different ratings across blocks (4.8 / 4.98 / 5.0) #1470;
  • the per-criterion threshold;
  • removing the leaderboard criteriaAvg;
  • §1 facts about the form and API;
  • terms explained and diacritics kept when clustering.

No Codex round, since this PR has no code.

Note: git dropped one body line of the commit message (it began with #1470). The squash commit takes this PR's text.

🤖 Generated with Claude Code


📌 TL;DR

This PR introduces a comprehensive design proposal for a new rating system that includes category-specific criteria, weighted scoring, and aggregated pros/cons. It outlines the data model, UI mockups, and a phased implementation plan to resolve existing issues with rating consistency and data integrity.

🎯 Type of Change

  • 🚀 New feature
  • 🐛 Bugfix
  • 🧹 Refactor
  • ⚡ Performance
  • 📚 Documentation
  • ⚙️ CI / Configuration

🔍 Changes Walkthrough

File Summary of Changes
Docs/Proposals/2026-10-31-criteria-stars-pros-cons.html Adds a detailed HTML design document proposing a new rating architecture. It defines category-specific criteria with weights, a new data model for storing review points and product aggregates, and a phased rollout plan (P0-P6) to implement these changes.

📊 Architectural Flow

flowchart TD
    subgraph "User Input"
        A[User Writes Review] --> B[Rate Criteria Stars]
        A --> C[Write Verdict & Pros/Cons]
    end

    subgraph "Processing & Validation"
        B --> D[API Validation]
        C --> D
        D --> E[Calculate Weighted Star Score]
        D --> F[Store Review Points]
    end

    subgraph "Aggregation Pipeline"
        F --> G[Code Clustering & Counting]
        G --> H[AI Refinement & Summarization]
        H --> I[Store Product Points]
    end

    subgraph "Display & Scoring"
        E --> J[Calculate Bayes Rating]
        I --> K[Display Aggregated Pros/Cons]
        J --> L[Update Master Product Rating]
        L --> M[Display Single Consistent Score]
    end
Loading

…w value

Design proposal for #1531, for JOY to decide. Per-category rating criteria
with weights live in shared-data; a Rate review's star is the weighted
average of its criteria; pros, cons and a verdict are mandatory for Rate
reviews and optional for imported ones; product pros and cons are
clustered by code, then reworded by AI; a 0-100 review value score sorts
reviews and lifts a complete review's weight by at most 25%. It also
resolves #1143 (no derived criterion scores) and the headline question of
#1470 (one Bayesian rating, stored by DOS-Me). Proposal only: no code,
no SQL, no data changes.

Refs #1531

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Code review skipped — your organization has no extra usage available to pay for this review.

If your organization's extra usage balance is empty, an organization admin can add extra usage credits at claude.ai/admin-settings/usage. If its monthly spend limit was reached, an admin can raise it on the same page. If neither applies, contact Anthropic support.

Once extra usage is available, someone with write access to this repository can comment @claude review on this pull request to trigger a review.

@vercel

vercel Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
rate Ready Ready Preview Oct 31, 2026 2:48am UTC

Request Review

@coderabbitai

coderabbitai Bot commented Oct 31, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 52 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0e67cbcf-eed9-40fc-a32e-a18a47aad6ac
Commits

Reviewing files that changed from the base of the PR and between 96e5c41 and 31c4cd7.

Files selected for processing: 1
  • Docs/Proposals/2026-10-31-criteria-stars-pros-cons.html
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions github-actions Bot added the docs Documentation changes label Oct 31, 2026
dos[bot]
dos Bot approved these changes Oct 31, 2026

@dos dos Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⏱️ Code Review completed (1 eligible files · 42,726 chars · 1 model call(s) · 2 PR unit(s))

Whole diff with bounded context at base 96e5c41 / head 31c4cd7.

✅ CLEAN_PASS: Complete diff reviewed

@JOY
JOY (JOY) merged commit 76a033e into dev Oct 31, 2026
18 checks passed
@JOY
JOY (JOY) deleted the docs/1531-criteria-stars-pros-cons branch October 31, 2026 13:22

This branch was successfully deployed

1 active deployment
Preview — 31c4cd7c Deployed Oct 31, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant