Disclosure: This is my personal blog project. No affiliation with Google or any product mentioned.
This project is built entirely on Google Cloud Platform using Gemini API as the core AI engine — no third-party LLM providers, no external ML platforms. Pure GCP stack.
Full GCP stack: Cloud SQL → Datastream → BigQuery → Dataform → Vertex AI → Gemini API → Looker Studio → Cloud Run → SendGrid
---
Over the past few years, the rapid evolution of modern LLMs has significantly changed how companies use data. Traditional Data Science workflows are no longer isolated systems — businesses are now integrating LLMs into analytics, marketing, recommendation systems, and operational automation. More recently, this evolution has expanded into company-specific AI Agents capable of autonomous decision-making and workflow execution.
Throughout my career, I have worked across the full data stack — from Data Engineering and Data Science to recent AI Agent development. In this project, I wanted to combine all of those experiences into a single practical end-to-end AI Agent system built entirely on GCP, and share not only the final implementation, but also the real-world project lifecycle behind it.
This project focuses on the entire practical process of building enterprise AI systems: project planning, architecture design, database modeling, data pipelines, machine learning, RAG implementation, Gemini API integration, business automation, and deployment considerations.
Rather than presenting only isolated code examples or tutorials, I wanted to recreate how real production projects are actually planned and executed in industry environments — including the technical trade-offs, architectural decisions, and business reasoning behind each choice.
Synthetic/public datasets were used instead of proprietary company data. However, the architecture, engineering decisions, and trade-offs are heavily based on real-world experience.
The main purpose is not simply "how to build an AI Agent on GCP," but: why the project exists, how the system should be designed, why specific GCP services were selected, what compromises were made, and what business outcomes the final system produced.
---
PROJECT OVERVIEW
Walsh Retail is a fictional e-commerce company with 200 customers, 1,163 orders, and 2,000 products (Kaggle Flipkart dataset).
The business problem: every customer receives the same promotional email regardless of purchase history or preferences. Low conversion, wasted budget.
The solution: an autonomous AI Agent on GCP that handles customer targeting, segmentation, personalized recommendation, campaign content generation via Gemini API, and automated email delivery — without human intervention.
The agent was designed to answer a question I kept encountering in production: can a small team — or even a single engineer — build what used to require a full ML platform team, using GCP managed services and Gemini API?
The answer turned out to be yes.
---
PART 1 — Architecture & Database Design
Most AI projects jump straight to the model. This one starts where real projects actually start — infrastructure planning and schema design.
Cloud SQL (MySQL 8.4) as the operational database. Designed a normalized 3NF schema with 8 tables including DML history columns for CDC tracking. BigQuery as the analytics layer with 4 datasets: mart, campaign, vector, log.
---
PART 2 — Real-Time Data Pipeline
A model is only as good as the data feeding it.
Datastream CDC streams every insert, update, and delete from Cloud SQL into BigQuery in real time via binary log — zero impact on the operational database. Dataform SQLX builds 10 RFM features into mart.customer_features. Cloud Workflows + Cloud Scheduler orchestrate the full pipeline daily at 01:00 UTC — fully automated, no manual runs.
DataStream & Dataform Datapipeline
---
PART 3 — Machine Learning + RAG + Gemini API
This is where the system starts thinking.
Vertex AI Colab Enterprise — Random Forest trained on 8 RFM features, top 30 customers selected (avg probability 0.986). Feature importance (XAI) confirms recency_days as the strongest predictor.
LangChain + Pinecone RAG — 500 synthetic brand reviews embedded (768-dim, text-embedding-004) and retrieved at inference time to ground Gemini with real product knowledge and prevent hallucination.
Gemini 2.5 Flash — generates personalized offers per customer: selects the best product from each customer's preferred category, applies segment-based discounts (Premium: 20%, Very High: 15%). Results stored in BigQuery campaign.offer.
---
PART 4 — Looker Studio + Gemini API Email Automation
An agent that stops at offer generation is a recommendation engine, not an agent.
Looker Studio connects directly to BigQuery — no CSV export needed. KPI scorecards, segment distribution, category breakdown, full offer detail table.
Gemini 2.5 Flash generates unique HTML emails per customer: subject line, offer body, product description rewritten as marketing copy, and CTA — all personalized, all different. SendGrid + Cloud Run handle automated delivery. Full campaign.email_log and campaign.email_content stored back in BigQuery.
Gemini Email Generation & Send
The same Gemini-powered pipeline can drive any channel: SMS, outbound call scripts, direct mail, web promotions, push notifications — email is one example.
---
My goal was to share not just the implementation, but the full engineering mindset behind it — the trade-offs, the decisions, and the reasoning that rarely shows up in tutorials.
Full write-up available on my blog: isupernova.io/tech
Happy to answer questions in the comments.