anchit@sevaro : ~/pune ▊
Senior Data Engineer, 5+ years across healthcare, gaming and enterprise. I work three layers of the same stack: the lakehouse that makes data trustworthy, the MLOps that gets models out of notebooks, and the AI / RAG layer that makes any of it answerable in English. M.Tech, IIIT Delhi.
I'm a Senior Data Engineer at Sevaro Health in Pune, building cloud-native data platforms and the GenAI analytics that sit on top of them. Most of my work starts the same way: something upstream is messy, slow, or expensive — and by the time I'm done it's governed, observable, and 40% cheaper to run.
Right now that means a multi-tenant access architecture — per-tenant Databricks service principals, Okta JWT authentication, Unity Catalog scoped grants and AWS Secrets Manager — enforcing client data isolation on shared infrastructure. Alongside it, an OpenMetadata catalog spanning Databricks, DOMO and MongoDB with automated data dictionaries, cross-platform lineage and PII tagging.
Before that: five years at Junglee Games and Axtria resolving 50M+ pharmaceutical records at 87% match accuracy, cutting Spark compute 30% by moving EMR → Databricks, and shipping a LangChain summarisation agent that gave five game studios back 3.5 hours per analyst per day.
Off the clock I write the Medallion & AI series on Medium and build in public — a dbt project with 56 tests, an LLM built from scratch, a KV-cache quantiser, and a browser-native SQL trainer.
A lakehouse nobody trusts is a swamp. A model that never leaves a notebook is a hobby. A RAG system on ungoverned data is a confident liar. I build all three so none of them fails alone.
Raw lands once and never changes. Everything downstream is a rebuildable opinion of it, and each hop earns a stronger guarantee — replayable, then conformed, then trusted.
The Gold layer is where models should start. Same tables, same tests, same lineage — so a feature in training is provably the feature in production.
An LLM is only as honest as its retriever. Ground it in governed Gold tables, cite the chunk, evaluate the answer — otherwise you've shipped a very fluent guess.
EMR → Databricks migration of production Spark workloads
Right-sized clusters, job consolidation and cloud cost audits
Freshness + anomaly detection modules shipped platform-wide
Reusable, config-driven alerting framework with email delivery
Rebuilt KPI dashboards and the pipelines feeding them
NLP-driven master data management across 50M+ records
3.5 hours back per analyst per day, across five games — the LangChain summarisation agent. Five analysts, five days a week, is most of a working month returned every month.
A production RAG pipeline that reads dense clinical documentation and returns the parts a neurologist actually needs. Semantic chunking, retrieval and prompt orchestration tuned for clinical precision — extraction that used to take minutes takes seconds.
A LLaMA-driven opponent for Junglee's Learn & Practice mode — it plays a credible hand and explains why, turning onboarding from a tutorial into a game.
MLOps pipelines at Junglee Games for A/B testing and dynamic recommendation systems — models fed from the same governed tables that back the reporting.
Entity resolution at scale: 50M+ records collapsed into Golden Records with fuzzy matching and NLP, hitting 87% match accuracy on notoriously dirty HCP/HCO data.
A production-grade dbt project for e-commerce analytics — 11 models, 56 tests, CI/CD, BI integration and an interactive GitHub Pages site. Built on dbt + DuckDB, so the whole warehouse runs on a laptop.
A visual ETL platform — connect databases, browse catalogs, build pipelines by drag and drop, stream CDC to S3, then schedule, monitor and export logs. FastAPI + React.
LeetCode-style SQL practice that runs entirely in the browser — every solution validated by DuckDB-WASM, plus a learning hub and admin CMS. Next.js, Prisma, Monaco.
Build an LLM from scratch: 15 runnable Python scripts that walk from a bare neural network to a 2026-architecture, DPO-aligned tiny GPT — with a companion site.
TurboQuant for PyTorch — near-optimal vector quantization for LLM KV-cache compression. The unglamorous inference work that decides whether a model is affordable to serve.
Python vs Rust, honestly measured: an OpenAI-compatible batch-summariser CLI built twice, with a deterministic mock server, cross-language golden checks and benchmarks.
Your LLM compiles the wiki, you read it. A desktop app inspired by Karpathy's LLM-knowledge-base idea — generation as a reading surface, not a chat log.
Implementations of Standard ES and EGGROLL from Evolution Strategies at the Hyperscale (arXiv:2511.16652). Reading a paper properly means running it.
Retrieval before it was fashionable — a Chrome extension that embeds a cited document and surfaces the sentence closest to the claim, so you can check whether the citation says what it claims. IIIT Delhi IR project.
Free online PDF tools — merge, split, compress, convert, OCR. 100% client-side, so files never leave the device. The privacy guarantee is architectural, not a promise.
nothing here yet — try another filter.
I write the series I wanted when I was learning this: how a Medallion actually behaves in production, where MLOps meets data engineering, and what AI genuinely changes about the job.
A practical decision framework for identifying fact and dimension tables — plus exactly how to explain your reasoning in an interview.
2026 · Jul 05Most Medallion tutorials quietly assume a batch world — a nightly job reads yesterday's files and rebuilds a table. That model breaks on a stream.
2026 · Apr 25How today's data engineers are evolving from pipeline builders into platform enablers — and why MLOps is the missing bridge to AI-driven products.
2026 · Apr 18From self-healing pipelines to intelligent schema detection — AI is not replacing data engineers, it is making them considerably faster.
2026 · Apr 16Article 1 of the “AI × Data Engineering” series — a practitioner's guide to smarter pipelines and where automation actually pays off.
2026 · Apr 16From pipeline plumbing to intelligent systems — how AI is rewriting the data engineer's job description.
2026 · Mar 27Any single layer is straightforward in isolation. The complexity — and the craft — lives in wiring them together.
2026 · Mar 27A perfectly modelled Gold table that takes 3 minutes to query will be bypassed within a month. Someone will build a workaround.
2026 · Mar 21A wrong dimension gives you the wrong label. A wrong fact gives you the wrong number — and a convincingly wrong number is worse than an obviously broken one.
2026 · Mar 19Batch SCD2 is a solved problem. Streaming SCD2 is a different animal entirely — and most teams only discover the difference in production.
2026 · Mar 18Your pipeline is only as smart as the decisions baked into the Gold layer — and those decisions start with how you handle a slowly changing dimension.
2026 · Feb 07A data engineer's collection of SQL tricks that changed how I write queries — VALUES as an inline table, GROUPING SETS, LATERAL joins, DISTINCT ON.
2026 · Feb 07A decision process for picking the right model for the problem in front of you, rather than defaulting to whatever shape you used last time.
2026 · Jan 04A case study in root-cause analysis — tracing a fall in orders back through the data until the actual reason surfaces.
2025 · Nov 27A complete guide to trip data modelling and analytics storage — the warehouse design behind delivery at scale, and the interview questions it invites.
2025 · Nov 12When a workload genuinely outgrows conventional tooling — and what actually changes at the point you make the move.
nothing here yet — try another filter.
IIIT Delhi
Specialised in Natural Language Processing and Deep Learning. Research work in semantic analysis and information retrieval — the roots of everything I now call RAG.
GATE 98.14 percentileMoradabad Institute of Technology · AKTU
Foundations in programming, Android development and machine learning — where the Blood Bank Finder and my first models came from.
First DivisionI'm open to senior data engineering and platform roles — especially where the lakehouse meets MLOps and AI. Got a gnarly pipeline, a cost problem, or a RAG system that keeps making things up? My inbox is genuinely open.
$ echo $EMAIL
anchitgupt2012@gmail.com
$ ▊