Rahul
Veerapur

Rahul Veerapur
About

I help startups build AI, scale data infrastructure, and grow revenue through data-driven decisions.

I help startups and businesses build AI systems, scale data infrastructure, and make better decisions with data. My work spans data engineering, machine learning, analytics, and AI, where I design end-to-end solutions ranging from real-time data pipelines and predictive models to production-ready LLM applications and autonomous AI agents.

I enjoy solving problems that sit at the intersection of software engineering and data science. Whether it's architecting distributed data platforms, deploying machine learning models, or building intelligent systems that automate complex workflows, I'm driven by creating solutions that are scalable, reliable, and built to make a real impact.

Outside of work, I love exploring new places and experiencing different cultures, especially through food. Whether it's discovering a hidden local restaurant, planning my next trip, or trying cuisines I've never had before, I'm always looking for new experiences. Whenever I get the chance, you'll also find me snowboarding, listening to music, gaming with friends, or getting lost in a good book.

Statistical & ML
Regression, XGBoost, LightGBM, Random Forest, SVM, PCA, K-Means, DBSCAN, t-SNE, SVD, ARIMA, LSTM, A/B Testing, SHAP, GAN, SEM.
Deep Learning & Agentic AI
PyTorch, TensorFlow, HuggingFace, MLflow, LangChain, AutoGen, LangGraph, RAG, Ollama, FastAPI, FAISS, Multi-Agent Systems, Prompt Engineering, scikit-learn.
Data Engineering
Python, SQL, Apache Spark, Kafka, Airflow, dbt, Docker, ETL / ELT, Fivetran, Kubernetes.
Cloud & Databases
AWS S3, Redshift, AWS Glue, Snowflake, GCP / BigQuery, Azure, PostgreSQL, MongoDB, Cassandra, Neo4j, Oracle DB.
Analytics & Visualization
Power BI, Tableau, Plotly, Seaborn, Matplotlib, KNIME, Excel / VBA.
Languages & Tools
Python, SQL, R, Scala, Java, Bash, Git, Jupyter, Hadoop HDFS, Apache Hive.
Selected Work

My Projects

From natural-language data warehouses to real-time streaming pipelines, every build here solves a real data or AI problem end to end.

01 / Featured
Financial Intelligence Platform
AI + Data
Local financial data warehouse with a natural language query interface. Real OHLCV data for 20 US stocks (5yr via yfinance), 500k synthetic transactions, 200k loans, and a fraud layer in DuckDB. Ask business questions in plain English, a local Llama model generates and validates DuckDB SQL.
PythonDuckDBFastAPILangChainOllamayfinance
View project
DATA SOURCES ETL + dbt SNOWFLAKE WAREHOUSE API + AGENT yfinance API Real OHLCV, 20 tickers Synthetic Generators NumPy seed=42, BLS weights INGEST + dbt market_data fetcher yf.download() 5yr daily txn / loan generators 500k txns, 200k loans DQ Validators nulls, ranges, row counts SNOWFLAKE (analytics schema) market_data transactions loans fraud_events data_quality_log lineage.json (append-only) FastAPI POST /query GET /query/schema GET /query/dq GET /query/lineage/{'{'}table{'}'} GET /health NL-SQL Agent LangChain + Ollama sqlglot validation 3-attempt retry loop Llama via Ollama Ollama local Llama GPU
SEC EDGAR Financials 80 large-cap companies 22 Engineered Features growth, margin, leverage XGBoost + LightGBM ensemble, walk-forward CV DCF / CAPM + Monte Carlo 5,000 simulations
02
Automated Equity Research & Valuation Platform
ML, Finance
End-to-end ML pipeline ingesting SEC EDGAR financials for 80 large-caps, engineering 22 features, and training an XGBoost + LightGBM ensemble validated with walk-forward cross-validation.
PythonXGBoostLightGBMSHAPMonte Carlo
View project
03
Scalable Knowledge Graph Intelligence System
Multi-Agent LLM
Production-grade Neo4j knowledge graph replacing expensive LLM-only reasoning. Unstructured data ingested at 10,000+ records per second; agentic queries are 6.5x cheaper than pure-LLM equivalents.
PythonNeo4jAirflowLangGraphOllama
View project
Unstructured Data Sources Batch Files CSV / JSON / Text Streaming APIs Real-time feeds INGESTION Airflow + Docker 10,000+ rec/sec Batch + Streaming NORMALIZE Entity Resolution Schema Alignment Deduplication MULTI-AGENT LLM PIPELINE Extraction Agents Entity / Relation Attribute tagging Voting Session Scoring Agents Multi-vote consensus Optimal score Parent Agent Decision Conflict resolution Domain rules Staging Pending review Conflict queue KNOWLEDGE GRAPH Neo4j E1 E2 E3 E4 50M+ nodes Ontology schema GraphRAG Cypher + Vector DOWNSTREAM AGENT 👤 Multi-hop reasoning Pure lookups skip the LLM ~$0.011 / synthesis
5 Crash Datasets 2.5M+ records integrated Feature Engineering 70+ features, 130+ raw vars LightGBM / XGBoost / NN AUC 0.990 (PA 2022) Risk Scoring $114M+ cost impact
04
Opioid-Influenced Crash Risk Modeling
ML, Risk
Multi-state crash risk intelligence system integrating 2.5M+ records across five datasets, engineering 70+ behavioral and geospatial features. LightGBM achieves AUC 0.990 on Pennsylvania 2022 data.
PythonLightGBMXGBoostSHAPPower BI
View project
05
Real-Time Distributed Trend Analysis System
Streaming
Production-grade streaming pipeline ingesting news via Kafka, routing through Spark ETL into a polyglot storage layer (MongoDB for raw documents, Cassandra for time-series metrics). End-to-end latency under 1 second.
KafkaSparkCassandraMongoDBDocker
View project
Google News API via RapidAPI keywords: Penn State Pennsylvania Univ. JSON Kafka Producer Python parse_timestamp() fetch_news() publish Apache Kafka 172.17.0.4:9092 penn_news durable offsets fault-tolerant horizontal scaling raw store process classify MongoDB news_archive.articles port 27017 | raw JSON Spark ETL keyword extraction entity tagging NLP Analytics NLTK, Bag of Words incident + edu classification Cassandra final_stream.realtime_news port 9042 | write-optimized time-series metrics Power BI Real-time dashboards publisher frequency NLP trend signals SOURCE INGEST BROKER PROCESSING STORAGE ANALYTICS All components run as Docker containers: psu-kafka, psu-cassandra, psu-mongodb, psu-ubuntu
Machine Group 1 4 Mitsubishi EVOLs Speed, Feed Gap, Print Gaps Machine Data Logging #1 Machine Group 2 3 Isowa FALCONs Speed, Feed Gap, Print Gaps Machine Data Logging #2 Speed Link C# data logging software transforms controller formats writes to SQL database Supplier Quality Lab ECT strength measurements ECT Link API SQL Database Machine params + ECT 124,857 clean records (from 125,288 raw) Data Cleaning null removal, type casting binary cols, dummy encoding 34+ engineered features 5 Sub-Datasets Split by no. of printing cylinders used (0-4) per-SKU analysis 0 cylinders 1 cylinder 2 cylinders 3 cylinders 4 cylinders
06
Box Strength Predictive Model
Data Mining
Full data mining pipeline for SCGP (corrugated box manufacturer, Southeast Asia) predicting box compression test results before physical testing, eliminating $2.7M/year in production downtime.
PythonK-MeansFP-GrowthKernel PCAUMAP
View project
07
Electoral Donation Data Insights
Data Engineering
Dual-architecture data engineering project ingesting 56 CSV files of FEC contribution data across CA, FL, NY, PA. Two complete pipelines built side-by-side: PostgreSQL star schema and Hadoop Hive distributed architecture.
PostgreSQLHadoopHivePySparkPig
View project
FEC Data 56 CSV files CA / FL / NY / PA ~5M records total PATH A: RELATIONAL DATA WAREHOUSE KNIME ETL 4 node workflows Transform, clean, join PostgreSQL Star schema (7 tables) ~2 GB per table KNIME Reporting Value Filter + Joiner GroupBy + Aggregation 6 BQ Reports State, city, party occupation, time series PATH B: HADOOP DISTRIBUTED ARCHITECTURE HDFS 4 state subfolders CA / FL / NY / PA PySpark + Apache Pig Custom schema + ETL MapReduce across nodes Hive Table election.main5 (14 cols) 4,502,043 rows KNIME Reports Hive Connector Same 6 BQ results vs Key Difference DWH: structured, slow on large batch queries Hadoop: faster updates via distributed nodes
SOURCE IQVIA Excel 175,625 rows openpyxl INGEST / ETL Python / Pandas normalize + type-cast DOCUMENT STORE MongoDB raw JSON ingestion RELATIONAL + INDEXED PostgreSQL B-tree indexes applied MBA + Clustering Time Series
08
Prescribing Pattern Analysis for Chronic Systolic Heart Failure
ML, Healthcare
End-to-end pharmaceutical intelligence pipeline on 175,625 IQVIA records (2019 to 2025). Three parallel analytical tracks: FP-Growth market basket, TruncatedSVD + KMeans segmentation, and XGBoost time-series forecasting.
PythonXGBoostFP-GrowthTruncatedSVDGAN
View project
09
Product Analytics Playbook
Product Analytics
Full-stack product analytics case study on the Stack Exchange public data dump: activation funnel, 18-month cohort retention, RFM segmentation, Bayesian A/B testing, and AI workflow prototyping with DuckDB.
PythonDuckDBPlotlyPyMCscikit-learn
View project
INGEST METRICS FUNNEL RETENTION ADOPTION 01 Data Ingestion XML parsing → DuckDB TRY_CAST all columns Users / Posts / Comments Badges / Votes (5 tables) 02 Platform Health Monthly growth curves Answer rate + content quality Rep distribution (log scale) Accept rate by score bucket 03 Activation Funnel 5-step: Registered→Activated Step-over-step drop-off PERCENTILE_CONT P25/P50/P75 Time-to-first-question dist. 04 Cohort Retention 18-month heatmap matrix Avg retention + P25/P75 band Best vs worst cohort M6 DATEDIFF('month') periods 05 Feature Adoption Time-to-adopt per feature P25 / P50 / P75 days M3 retention rate by feature Correlation, not causation SEGMENTATION EXPERIMENT DESIGN A/B ANALYSIS AI WORKFLOW ROADMAP 06 User Segmentation RFM quintile scoring NTILE(5) per metric Power / Regular / Casual At Risk / Churned Scatter + pie + box viz Rep bar by segment 07 A/B Test Design Baseline from real data Cohen's h effect size NormalIndPower sample size Power curves 5/10/20% MDE Experiment duration estimate alpha=0.05, power=80% 08 A/B Analysis Frequentist z-test + Wilson CI Bayesian Beta-Binomial 500K Monte Carlo draws P(T>C) + expected loss Posterior lift distribution SHIP / KILL decision 09 AI Workflow LLM hypothesis generation NL-to-SQL for ad hoc work Auto-narrative from metrics Variant copy generation Anomaly investigation loop AI-augmented PM workflows 10 Roadmap 3 data-backed product bets Evidence → Action → Impact Fix activation gap Reduce unanswered questions Protect power users Color-coded evidence table
Ingestion UCI + FHIR R4, 101K+ records Feature Eng. 50 to 37 clean features Custom GD + SMOTE LogisticGD, 11% to 50% 5-Model Ensemble Optuna tuned SHAP Interpretability 43 output figures
10
CRIS: Clinical Readmission Intelligence System
ML, Clinical AI
End-to-end ML system predicting 30-day hospital readmission from FHIR R4 + UCI clinical data. Custom gradient descent, SMOTE resampling, Optuna hyperparameter tuning across 5 models, and SHAP interpretability on 101K+ records.
PythonLightGBMOptunaSHAPSMOTE
View project
11 / Featured
FeedbackIntel: AI-Powered Product Feedback Intelligence
AI Engineering
Production AI platform that ingests user reviews, NPS comments, and support tickets from 8 sources, then answers product questions with structured insight reports: cited verbatim quotes, per-theme breakdowns, and actionable recommendations. Fully offline with Llama3 + nomic-embed-text via Ollama.
PythonFastAPILlama3FAISSDocker
View project
PHASE 1 — INGESTION App Store Play Store G2 / Cap. NPS Support Twitter / In-App Ingestion Pipeline POST /feedback/bulk embed via nomic-embed-text normalize → FAISS.add() embedded=True in PostgreSQL batch size: 50 Storage Layer FAISS IndexFlatIP 768-dim normalized vecs metadata: id, source, rating, date PostgreSQL PHASE 2 — QUERY PIPELINE Product Manager POST /insights/ {question} Agent 1: Planner Llama3 decomposes question → up to 4 investigation themes Agent 2: Retriever FAISS ANN search per theme top-12 per theme, deduplicated [Source, Rating, Date] formatted output Agent 3: Synthesizer Llama3 writes cited report ## Summary / Findings / Recs no claim without evidence Evaluator (LLM-as-Judge) Faithfulness · Relevance · Specificity 3 independent Llama3 scoring calls score clamped 0.0–1.0, JSON parsed PostgreSQL InsightQuery + EvalResult cited_ids_json, contexts_json full history, traceable to source API Response report + themes + eval scores cited_ids + latency_ms full insight in one request search at query time
CourtListener Bulk Data 18,482 cases, 1860-2023 Two Pipelines historical filter + modern join Four Models 2x LEGAL-BERT, structured, hybrid Agent + API classify, retrieve, explain
12 / Featured
Verdict: Legal Case Outcome Prediction
Legal NLPAgentic AI
Two fine-tuned LEGAL-BERT models predict US appellate case outcomes from case text: one on historical syllabi (1860-1935, 55.8% accuracy), one on a fully-recovered 4,399-case 54GB export of full opinion text (2019-2023, 76.5% accuracy). A structured metadata baseline and an Optuna-tuned hybrid fusion model round out four classification experiments, and a three-step agent classifies, retrieves precedent via FAISS, then explains its reasoning with a QLoRA fine-tuned Qwen2.5-3B language model.
PythonPyTorchTensorFlowLEGAL-BERTQLoRAFAISSFastAPI
View project
13
NYC Mobility Data Platform
Data EngineeringCloud Infrastructure
End-to-end data engineering platform processing 2.96M real NYC Yellow Taxi trip records through a PySpark ETL job into governed Delta Lake tables. Event-driven ingestion (S3, Lambda, DynamoDB), the same transform code runs on Databricks or AWS Glue, and all infrastructure, Unity Catalog governance included, is defined in Terraform and deployed through OIDC-authenticated CI/CD.
PySparkDelta LakeDatabricksTerraformAWSGitHub Actions
View project
DATA SOURCE INGEST + TRIGGER TRANSFORM (DATABRICKS / GLUE) STORE + SERVE NYC TLC Trip Data Public parquet, no auth 2.96M rows, Jan 2024 INGEST + TRIGGER S3 raw bucket ObjectCreated event Lambda trigger_etl calls Databricks Jobs API DynamoDB processed_files, idempotency PYSPARK ETL clean_trips enrich_trips zone_hourly_aggregates data_quality_summary runs on Databricks job cluster or Glue DELTA / UNITY CATALOG trips_clean zone_hourly_metrics pipeline_run_log catalog + schema grants storage credential, external location RDS Postgres curated warehouse tables materialized from Delta served to BI / ad hoc SQL private subnet, no public route Monitor CloudWatch CloudTrail SNS alerts
Work History

Experience

Penn State University
Oct 2024 → Dec 2025
Pennsylvania, USA
Research Assistant, Prescribing Pattern Analysis
Funded by Penn State & IQVIA
  • Multi-Source Integration: Ingested IQVIA NPA and FDA Orange Book data into a drug-level panel dataset spanning 50+ branded and generic compounds for longitudinal prescribing analytics.
  • Hybrid SEM + Neural Network: Architected an end-to-end pipeline across 5+ years of cardiovascular drug market data, enabling simultaneous causal pathway estimation and non-linear interaction modeling.
  • Prescriber Segmentation: Increased targeting accuracy by 88% via t-SNE/SVD behavioral features enabling demographic and clinical risk stratification.
  • Time-Series Forecasting: Achieved 92% R² with a custom LSTM model, outperforming Random Forest baselines on non-linear seasonality in prescription demand.
Penn State University
Oct 2024 → Jul 2025
Pennsylvania, USA
Research Assistant, Autonomous Inventory Forecasting
  • Automated ML Pipeline: Built an Airflow + Google Cloud Functions pipeline ingesting Target APIs and Google Trends data for daily stockout probability forecasts with automated retraining triggers.
  • Continuous Training: Deployed a CT workflow with XGBoost + PyTorch managed via MLflow, triggered by concept drift detection, sustaining F1-score above 0.90 across quarterly inventory cycles.
  • NL-to-SQL Agent: Built a tool-augmented LLM agent with schema-aware SQL generation, automatic repair loops, and validation against production BI views for inventory analysts.
Vishwa Holdings LLC
Sept 2022 → Aug 2024
Bengaluru, India
Data Analyst Engineer
  • ETL at Scale: Consolidated 7M+ land parcel records from 15+ sources into Snowflake via Airflow pipelines with fuzzy-matching entity resolution, catching 12% of malformed records before production.
  • Predictive Site Selection: Gradient boosting model with 25+ geospatial features achieved 0.81 AUC, reducing initial screening effort by 70%.
  • Demand Forecasting: 12 to 24 month regional occupancy and rental yield forecasts (ARIMA/Prophet) correctly predicted 3 of 4 regional demand shifts in FY2023.
  • Tenant Churn Prediction: XGBoost churn model on 350+ assets achieved 82% precision, enabling proactive retention that reduced turnover by 18%.
  • Asset Segmentation: K-Means/DBSCAN clustering on 900+ logistics assets identified 5 distinct profiles, reducing appraisal variance by 25%.
CME Group
Jan 2022 → Jul 2022
Bengaluru, India
Data Engineer Intern
  • Federated Metadata Search: Reduced data discovery time by 80% across teams by enabling search across 50K tables in 1,200 databases via a centralized metadata index.
  • Usage Analytics Dashboard: Lowered redundant exploratory queries by 25% by giving teams visibility into dataset usage patterns, connection times, and peak access windows.
  • Dataset Recommendations: Cut average employee search time by 30% through relevant-dataset recommendations based on historical query behavior.
Academic Background

Education

Pennsylvania State University
Master of Data Analytics, Big Data and AI Specialization
University Park, PA Aug 2024 → Dec 2025 GPA 3.64 / 4.0
Deep LearningMachine Learning Predictive ModelingStatistical Methods Time Series AnalysisData Mining Big Data AnalyticsData Pipeline Development
R.V. College of Engineering
Bachelor of Engineering, Computer Science
Bengaluru, India Aug 2018 → Aug 2023
Data StructuresAlgorithms Operating SystemsSoftware Engineering Machine LearningCloud Computing
Get in Touch
LinkedIn GitHub

Let's Talk

Got a question, an opening, or a project that needs a data or AI engineer? I'd like to hear about it.

Send me an email or Call (215) 651-0869