The problem
The client needed to expand their Amazon product catalog but faced massive research overhead:
- Hundreds of thousands of potential product categories and niches to evaluate
- Manual research per category took hours: checking demand, competition, pricing trends
- No systematic way to score opportunities against their brand criteria
- Product decisions were delayed by weeks due to research bottlenecks
- High risk of missing profitable opportunities or entering oversaturated markets
They needed an automated system that could rapidly evaluate products and surface the best opportunities based on their specific needs.
The pipeline: Data → Scoring → Export
We built a multi-stage pipeline that automates the entire product research workflow:
Stage 1: Product Signal Collection
- Integrated with Keepa-style APIs to pull historical pricing, sales rank, and demand signals
- Gathered data across target categories: price history, BSR trends, review counts, ratings
- Pulled competitor analysis: who's selling, at what price points, with what margins
- Collected seasonal trends and demand patterns
Stage 2: Opportunity Scoring
Custom scoring algorithm tailored to the client's brand criteria:
- Demand score: Based on sales rank trends, review velocity, and search volume
- Competition score: Number of sellers, price concentration, dominant brands
- Margin potential: Estimated profit after FBA fees, based on typical pricing
- Brand fit: Alignment with client's product line and quality standards
- Risk assessment: Volatility, seasonal dependency, regulatory concerns
Each product received a composite score (0-100) plus detailed breakdowns per dimension.
Stage 3: Competitive Summary
- Identified top competitors per niche
- Analyzed their pricing strategies, review patterns, and feature gaps
- Highlighted differentiation opportunities
Stage 4: Export to Google Sheets
- Automatically exported results to Google Sheets with formatted tables
- Included product links, images, key metrics, and scores
- Added conditional formatting to highlight top opportunities
- Enabled team collaboration and decision-making directly in familiar tools
Custom scoring criteria
The scoring system was customized to match the client's specific needs:
Example scoring formula:
- 40% Demand potential (sales velocity + review growth)
- 25% Margin opportunity (price - estimated cost - fees)
- 20% Competition level (inverse of seller count + price variance)
- 10% Brand alignment (category match + quality threshold)
- 5% Risk factors (seasonality penalty + regulatory concerns)
Weights were adjustable based on strategic priorities. For example:
- During expansion phase: higher weight on demand potential
- When optimizing margins: higher weight on profit calculations
- When entering new categories: higher weight on risk assessment
Tech stack
- Data collection: Python with Keepa API integration
- Scoring engine: Custom Python algorithms with configurable weights
- Storage: PostgreSQL for product data and scoring history
- Export: Google Sheets API for automated report generation
- Orchestration: Scheduled runs per category batch using cron
The system was designed to run autonomously on a schedule, with results delivered to the team's shared Google Sheets workspace.
Outcome
The automated pipeline transformed product research from a bottleneck into a strategic advantage:
Time savings:
- What took weeks of manual research now completed in hours
- Analyst time shifted from data gathering to strategic evaluation
- Faster time-to-market for new product launches
Better decisions:
- Systematic scoring removed gut-feel guesswork
- Consistent evaluation across all opportunities
- Easier to spot trends and patterns across categories
Scalability:
- Could evaluate 10x more product opportunities with the same team
- System extended to new marketplaces (UK, DE, FR) with minimal changes
- Scoring criteria easily adjusted as business priorities evolved
The client now runs this pipeline monthly per category, maintaining a constantly updated view of product opportunities across their target niches.
Want something like this?
Tell me your stack + what you want automated. I’ll reply with a simple plan.