Repository navigation
fix(importer): reject used, like-new and refurbished listings at ingest - #1561
Conversation
…st (#1430) rate.products stores no condition, so a non-new listing linked to a published master (exact model_key, variant or a master hint) fed its lower price into the new device's min_price and the deal radar. - ingestProduct throws NonNewListingError (code NON_NEW_LISTING) before anything is written when the normalizer detects a non-new condition, with or without a master hint (option A of #1430, no DDL). - Trade-in offers for the buyer's old device ("nhận thu máy cũ", "thu cũ lên đời", "đổi máy cũ lấy máy mới") no longer read as used, so new listings are not dropped for them. - The error is exported from @repo/crawlers (dist smoke). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#1430 review) - The ingest check reads the title alone. Description boilerplate such as "không bán hàng cũ" or "99% khách hàng hài lòng" would drop new listings, so a description hit still only sets needs_review. - "2nd" counts as used only in "2nd hand": "AirPods Pro 2nd Gen" and "iPhone SE 2nd generation" are new devices. - The trade-in strip also covers "thu máy cũ đổi máy mới" and "thu mua máy cũ". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on (#1430 Codex) - The normalizer sets title_condition from the sanitized title, and the ingest check reads it, so a control character inside "Máy cũ" no longer slips a used listing past the check. - A bare "98%" or "99%" (such as "Chống ồn 98%") no longer reads as like-new; a percentage counts only after máy, ngoại hình, hình thức, keng or pin, besides the existing mới, đẹp and zin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
If your organization's extra usage balance is empty, an organization admin can add extra usage credits at claude.ai/admin-settings/usage. If its monthly spend limit was reached, an admin can raise it on the same page. If neither applies, contact Anthropic support.
Once extra usage is available, someone with write access to this repository can comment @claude review on this pull request to trigger a review.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 54 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
![dos[bot]](https://avatars.githubusercontent.com/in/4711427?s=60&v=4){kind=small}
There was a problem hiding this comment.
⏱️ Code Review completed (5 eligible files · 11,146 chars · 4 model call(s) · 1 PR unit(s))
Whole diff with bounded context at base cc57e2d / head bff1ac7.
✅ No verified defects
1 possible issue(s) could not be grounded in or verified against the snapshot source, so they are not published.
Context investigation was bounded to the explicitly requested snapshot paths.
…Codex) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Codex round 2 (the last; the cap of 2 is reached):
The importer's |
There was a problem hiding this comment.
⏱️ Code Review completed (5 eligible files · 11,246 chars · 3 model call(s) · 1 PR unit(s))
Whole diff with bounded context at base cc57e2d / head 472105c.
🔍 Verified Adversarial Review Findings
🟡 IMPORTANT
apps/importer/src/shopee/product-normalizer.ts:160: Trade-in regex over-strips genuine used-device phrases- Source Side: RIGHT (head)
- Failure Trace: detectProductCondition('iPhone 13 128GB máy cũ giá tốt') → TRADE_IN_PHRASE matches 'máy cũ' (second alternative 'thu\s*(?:mua\s*)?(?:máy|điện\sthoại)\scũ' with optional 'thu'/'mua' absent), replacing it with a space. The remaining text no longer contains 'máy cũ', so the used-device regex fails and the function returns 'new'. The ingest check (shopee-to-supabase.ts:546-548) then reads title_condition='new' and proceeds to upsert the used listing, whose lower price becomes the new device's min_price — exactly the #1430 scenario the guard is meant to prevent. The old regex /thu\scũ\sđổi\s*mới/ required the full 'thu cũ đổi mới' phrase and would not have stripped 'máy cũ' alone.
- Evidence:
apps/importer/src/shopee/product-normalizer.ts:160at472105cb05656dc05d7c869d20383740c094880a,apps/importer/src/shopee/shopee-to-supabase.ts:546at472105cb05656dc05d7c869d20383740c094880a,apps/importer/src/shopee/product-normalizer.ts:190at472105cb05656dc05d7c869d20383740c094880a
Context investigation was bounded to the explicitly requested snapshot paths.
|
|
||
| /** A trade-in offer for the buyer's old device, which says nothing about the listing's own. */ | ||
| const TRADE_IN_PHRASE = | ||
| /(?:thu|đổi)\s*(?:mua\s*)?(?:(?:máy|điện\s*thoại)\s*)?cũ\s*(?:lên\s*đời|(?:lấy|đổi)\s*(?:(?:máy|điện\s*thoại)\s*)?mới)|thu\s*(?:mua\s*)?(?:máy|điện\s*thoại)\s*cũ/gi; |
There was a problem hiding this comment.
🟡 IMPORTANT: Trade-in regex over-strips genuine used-device phrases
Failure Trace:
detectProductCondition('iPhone 13 128GB máy cũ giá tốt') → TRADE_IN_PHRASE matches 'máy cũ' (second alternative 'thu\s*(?:mua\s*)?(?:máy|điện\sthoại)\scũ' with optional 'thu'/'mua' absent), replacing it with a space. The remaining text no longer contains 'máy cũ', so the used-device regex fails and the function returns 'new'. The ingest check (shopee-to-supabase.ts:546-548) then reads title_condition='new' and proceeds to upsert the used listing, whose lower price becomes the new device's min_price — exactly the #1430 scenario the guard is meant to prevent. The old regex /thu\scũ\sđổi\s*mới/ required the full 'thu cũ đổi mới' phrase and would not have stripped 'máy cũ' alone.
Closes #1430
Problem
rate.productsstores no condition. A used, like-new or refurbished listing that keys to an existing master (exactmodel_key, a variant match, or a master hint) is linked to the new device.refresh_master_market_statsdoes not filter on condition, so the listing's lower price can become the new device'smin_priceand feed the deal radar.Decision: option A of the issue (reject at ingest)
Change
ShopeeIngestionPipeline.ingestProductthrowsNonNewListingError(code: 'NON_NEW_LISTING', withcondition) when the listing's title states a non-new condition.@repo/crawlers.title_condition, read from the sanitized title, so control characters cannot split "Máy cũ".needs_review, as before. Description boilerplate would otherwise drop new listings, for example "không bán hàng cũ" or "99% khách hàng hài lòng".detectProductConditiongives fewer false "non-new" results, because they now drop a listing:Not in this PR
min_price.discontinuedand refreshes the touched masters, and it is a prod write that needs JOY.UNKEYABLE_PRODUCT_TITLE.NON_NEW_LISTING: complete the job without ingesting, count it apart, and mark a refresh job's listingdiscontinued.Tests
shopee-to-supabase.test.ts:NonNewListingErrorwith the rightcondition, and write no listing.entity-matcher.test.ts:dist-exports.smokechecks the new export.pnpm test(importer),tscandeslintare clean.Review
🤖 Generated with Claude Code
📌 TL;DR
This PR introduces a strict ingestion guard that rejects used, like-new, or refurbished product listings to prevent incorrect
min_pricecalculations in the deal radar. It also refines the condition detection logic to accurately distinguish between trade-in promotional offers and actual used device indicators.🎯 Type of Change
🔍 Changes Walkthrough
apps/importer/src/shopee/product-normalizer.tsNonNewListingErrorclass andtitle_conditionfield toNormalizedProductCandidate. Expanded theTRADE_IN_PHRASEregex to catch more Vietnamese trade-in variations. Refined condition detection regexes to require context for percentages (e.g., "pin 99%") and to correctly identify "2nd hand".apps/importer/src/shopee/shopee-to-supabase.tsShopeeIngestionPipeline.ingestProduct. The pipeline now throwsNonNewListingErrorif the title-derived condition is not 'new', preventing any database writes for non-new items.apps/importer/src/shopee/__tests__/entity-matcher.test.tsapps/importer/src/shopee/__tests__/shopee-to-supabase.test.tsNonNewListingErroris thrown for non-new listings (even with hints) and that no database upserts occur. Verified that description-only non-new keywords do not trigger rejection.apps/importer/src/shopee/__tests__/dist-exports.smoke.mjsNonNewListingErrorclass.📊 Architectural Flow
flowchart TD A[Shopee Product Ingest] --> B[ProductNormalizer] B --> C{Detect Condition} C -->|Title Analysis| D[title_condition] C -->|Title + Desc| E[condition] D --> F{Is title_condition 'new'?} F -->|No| G[Throw NonNewListingError] F -->|Yes| H[Proceed to Supabase Upsert] G --> I[Reject Listing] H --> J[Write to Database]