n8n-Based AI Lead Generation & Multi-Channel Outreach System
Client: Marketing Agency
 Objective: Build a scalable AI-powered lead scraping, enrichment, personalization, and multi-channel outreach system controlled via Slack and powered by n8n.

1. Project Overview
Develop an automated outbound acquisition engine that:

Accepts City + Business Category as input
Scrapes ALL relevant Google Maps business listings
Enriches lead data (email, WhatsApp, phone, social, insights)
Creates a dedicated Google Sheet per campaign
Generates AI-personalized outreach using Gemini + RAG
Sends Slack notifications for approvals
Sends personalized emails (multi-step sequence)
Tracks responses and notifies Slack
Supports fallback channels (WhatsApp / LinkedIn / Call)
This system must be modular, scalable, and production-ready.

2. System Architecture
Core Components

Workflow Engine: n8n (self-hosted VPS)
Scraper Service: Node.js microservice (Puppeteer or Scraping API)
Database: PostgreSQL
Queue System: BullMQ / Redis
LLM: Gemini API
Vector DB (RAG): Pinecone / Supabase Vector
Storage: Google Drive (RAG folder)
Email Service: Gmail API / SendGrid
Validation APIs: NeverBounce / ZeroBounce
Slack Integration: Slack Bot API


3. Functional Modules

MODULE 1 – Campaign Input System
Requirements:
• Input form (web form or Slack slash command)
• Fields:
City
Business Category
Country
Max Leads (optional)
Campaign Name (auto-generate if blank)
Output:

Campaign record stored in PostgreSQL
Trigger scraping pipeline


MODULE 2 – Google Maps Full Scraping Engine
Requirements:
Scrape ALL businesses for given City + Category.
Must include:

Business Name
Phone
Website
Address
Rating
Reviews count
Google Maps URL
Business Category
Latitude / Longitude


Scraping Strategy
1. Pagination Handling

Use API supporting offset OR
Loop in batches of 100
Continue until no results
2. Geo Grid Expansion
• Break city into geo-coordinates grid
• Query each grid area separately
• Merge & deduplicate by:
Google Place ID
Business Name + Address
3. Subcategory Expansion
Automatically expand search:
 Example for Dentists:

Pediatric dentist
Cosmetic dentist
Orthodontist
Dental clinic


Technical Constraint
Scraping must NOT run inside n8n directly.
 Implement separate microservice:
n8n → Scraper API → Queue → Worker → n8n Webhook callback

MODULE 3 – Lead Enrichment Pipeline
If email not found → trigger enrichment workflow.
Multi-Step Enrichment:
Step 1 – Website Crawl
• Crawl homepage
• Crawl:
/contact
/about
/privacy-policy
• Extract:
Emails (regex)
Phone numbers
WhatsApp links
Social links
Step 2 – Email Pattern Generation
Generate:

info@domain
contact@domain
hello@domain
firstname@domain (if owner known)
Validate using email verification API.
Step 3 – WHOIS Lookup
Extract domain contact if public.
Step 4 – Third-Party Enrichment APIs
Integrate:

Hunter
Apollo
Clearbit
Step 5 – WhatsApp Detection
Detect:

wa.me links
api.whatsapp.com links
WhatsApp widgets
Extract number.

Store in DB:

Email (verified)
WhatsApp number
Additional phone numbers
Social links
Enrichment confidence score


MODULE 4 – AI Business Insight Analyzer
For each website:
Use Gemini to analyze:

Website quality
Booking system present?
SEO optimized?
Speed issues?
Facebook Pixel detected?
Ads running?
Outdated design?
Generate:

Identified problems
Recommended marketing service
Lead Score (1–100)
Store insights in database.

MODULE 5 – Google Sheets Generator
For each campaign:
Create new Google Sheet:
Naming format:


[Category]_[City]_[Date]Columns:

Lead ID
Business Name
Email
Phone
WhatsApp
Website
Rating
Insight Summary
Lead Score
Email Status
Reply Status
Follow-up Count
Sent Date
Populate automatically.

MODULE 6 – Slack Control System
Slack notifications required for:

Sheet created
Scraping complete
Email draft ready
Replies received
Approval Commands:

START
APPROVE
REGENERATE
CANCEL
Workflow must pause until Slack confirmation.

MODULE 7 – RAG Knowledge Base System
Google Drive Folder Structure:


/RAG/
   Case Studies
   Testimonials
   Services
   Pricing
   Guarantees
   Objection HandlingFlow:
1. Extract documents from Drive
2. Chunk + Embed
3. Store in Vector DB
4. During email generation:
Query relevant chunks
Inject into prompt

MODULE 8 – AI Email Generation (Gemini)
Generate:

3 Subject Lines
Primary Email
Follow-up Email 1
Follow-up Email 2
Breakup Email
Use:

Business insights
Review analysis
RAG case studies
Lead score
Category-specific pain points
Output:

HTML template with dynamic placeholders
Plain text version


MODULE 9 – HTML Preview + Slack Review

Host email HTML on cloud storage
Generate preview link
Send Slack message with link
Wait for APPROVE command


MODULE 10 – Personalized Email Sending Engine
For each lead:
1. Inject:
Business name
Website insight
Review reference
City reference
2. Send via:
SendGrid or Gmail API
3. Respect rate limits
4. Update sheet + DB

MODULE 11 – Multi-Step Follow-Up Automation
Schedule:

Day 0: Initial email
Day 3: Follow-up 1
Day 7: Case study
Day 14: Breakup
Stop sequence if reply received.

MODULE 12 – Response Tracking
Monitor:

Inbox via API
SendGrid webhooks
On reply:
• Classify via AI:
Interested
Not Interested
Referral
OOO
• Notify Slack
• Update CRM status
Optional:
 Auto-generate suggested reply draft.

MODULE 13 – Multi-Channel Fallback
If no email found:
Switch automatically to:

WhatsApp outreach
LinkedIn outreach (if available)
SMS
Call task creation
Channel priority configurable.

MODULE 14 – Lead Scoring System
Score based on:

No website
Poor SEO
Low rating
No ads detected
No social presence
Higher score = higher outreach priority.

4. Non-Functional Requirements

Must support 10+ campaigns simultaneously
Modular microservice architecture
Logging for all actions
Error retry system
Rate limiting to avoid bans
GDPR + CAN-SPAM compliant
Unsubscribe link in all emails
Bounce handling


5. Deliverables

n8n workflows
Scraper microservice
Enrichment engine
RAG integration
Slack bot
Email sending engine
Documentation
Deployment guide
Monitoring setup




















first time: docker compose up -d

if dependency changes:  docker compose up -d --build api

if no dependecy change: docker compose up -d api

to clear image: docker image prune



docker run --rm \
  -v $(pwd)/scripts/migrations:/migrations \
  migrate/migrate \
  -path /migrations \
  -database "po..uire" \
  up