Available for impactful data and AI roles

Hello, I’m Sheetal Singh

I build Senior Data Engineer

I design enterprise-scale data platforms, optimize Spark workloads, build AI-powered automation solutions, and create intelligent analytics platforms.

Download ResumeView Projects Contact
8+
Years Experience
35+
Projects
20+
Technologies
15+
Enterprise Solutions

Focus areas

From data to intelligence

Active
Azure Databricks01
PySpark & Delta Lake02
AI Automation03
Enterprise Analytics04

About

A career built around data scale, intelligence, and practical impact.

I’m driven by the challenge of turning complex data ecosystems into dependable, efficient engines for insight and automation. My work spans platform architecture, modern data engineering, and AI-enabled solutions that support real business outcomes.

Professional story

Over the years, I’ve built analytics platforms and AI automation systems for organizations that depend on speed, accuracy, and trust.

Engineering philosophy

I believe strong systems are shaped by maintainability, observability, and thoughtful collaboration rather than complexity for its own sake.

Cloud-native architecture

I build resilient platforms that scale from pilot to enterprise workloads with strong governance and automation.

AI-first thinking

I design practical AI systems that drive measurable business value alongside engineering excellence.

Execution focus

My approach balances performance, maintainability, and delivery speed in every engagement.

Outcome orientation

I care deeply about translating technical complexity into business impact and reliable products.

Skills

A modern stack for modern data and AI systems.

From lakehouse engineering to LLM orchestration, these capabilities support robust delivery from architecture through deployment.

Azure

95%

Cloud architecture and platform engineering

Databricks

96%

Lakehouse engineering and data engineering workflows

PySpark

94%

Distributed data processing at scale

Spark

92%

High-performance analytics workloads

Python

95%

Production-grade automation and AI services

SQL

93%

Complex analytics and performance tuning

Delta Lake

91%

Reliable data lakehouse patterns

Azure Data Factory

90%

Orchestrated data pipelines and integration

Unity Catalog

88%

Governance and data cataloging

Machine Learning

89%

Applied ML insights and experimentation

LLMs

92%

Generative AI and model integration

Prompt Engineering

90%

Reliable AI interaction design

LangChain

86%

Model workflows and orchestration

Git

92%

Version control and collaborative delivery

Azure DevOps

88%

CI/CD and engineering operations

Docker

87%

Containerized services and portability

Experience

A track record of building systems that scale.

Senior Data Engineer

Microsoft

2022 - Present • Remote

Leading high-scale data modernization programs and AI-enabled decision systems.

Responsibilities

  • Architected Databricks-based enterprise data platforms
  • Designed scalable ETL and streaming pipelines
  • Built AI copilots for internal operations

Achievements

  • Reduced processing latency by 40%
  • Improved data quality observability across 12+ domains
  • Delivered AI-assisted automation for claims and support use cases

Technologies

AzureDatabricksPySparkDelta LakeAzure OpenAI
Business impact: Enabled faster, safer, enterprise-wide decision making at scale.

Data Engineer

Databricks

2019 - 2022 • Remote

Built data engineering solutions for analytics and machine learning at scale.

Responsibilities

  • Developed Spark-based transformation frameworks
  • Improved data governance and catalog adoption
  • Partnered with product and analytics teams

Achievements

  • Delivered multi-region migration framework
  • Optimized workload cost by 30%
  • Enabled self-serve analytics for cross-functional teams

Technologies

SparkPythonSQLAzureMLflow
Business impact: Accelerated analytics adoption and reduced engineering toil.

Featured Projects

Selected work spanning data engineering, AI, and cloud platforms.

Each project reflects a thoughtful blend of platform design, analytics depth, and production-ready delivery.

AzureDatabricksPySparkLLMsDelta Lake

Healthcare Claims Denial Prevention AI

A purpose-built copilot that identifies claim anomalies, suggests interventions, and accelerates adjudication.

Problem: High denial rates and slow claims processing created significant revenue leakage.
Solution: Combined LLM reasoning with structured claims data and policy rules to surface risk signals and recommended actions.
Business impact: Improved denial recovery rate by 28% and reduced manual review time by 35%.
Architecture: Azure OpenAI + Databricks + Delta Lake + Power BI
GitHubDemo
PythonSparkLLMsAzure DevOpsDocker

Smart Code Migration Assistant

An AI system that converts legacy SQL and ETL logic into modern Spark pipelines and testable modules.

Problem: Legacy migration projects were slow, inconsistent, and expensive.
Solution: Used prompt-guided transformation workflows and validation checks to standardize migrations.
Business impact: Cut migration timelines by 45% while improving code quality.
Architecture: LangChain + Azure OpenAI + Python + CI/CD
GitHubDemo
DatabricksPySparkAzureLLMsSQL

AI Log Monitoring Dashboard

A real-time observability experience for detecting anomalies in data workloads and pipelines.

Problem: Engineering teams lacked a unified view of pipeline failures and performance regressions.
Solution: Built a streaming monitoring platform with AI anomaly scoring and actionable summaries.
Business impact: Reduced incident triage time by 60% and improved platform resilience.
Architecture: Kafka + Databricks + Azure Monitor + LLM summarization
GitHubDemo
AzurePythonLLMsDatabricksSQL

Dynamic Prior Authorization AI

An intelligent workflow that classifies requests and recommends clinical and operational next steps.

Problem: Prior auth decisions were slow and heavily manual.
Solution: Combined policy-driven rules with AI-generated explanations and structured handoffs.
Business impact: Accelerated approvals and improved consistency across operations.
Architecture: Azure Functions + OpenAI + Databricks + Service Bus
GitHubDemo
AzurePythonMachine LearningDatabricksSQL

Network Optimization AI

A machine learning solution that predicts service degradation and optimizes routing logic.

Problem: Network teams needed a faster way to spot trends and respond to risk.
Solution: Built forecasting models and an interactive dashboard for proactive intervention.
Business impact: Lowered incident response time and improved service quality.
Architecture: Azure ML + Databricks + Python + Stream analytics
GitHubDemo
DatabricksDelta LakeAzure Data FactorySQLPySpark

Data Quality Framework

A reusable data quality engine that validates metrics, schemas, and freshness across pipelines.

Problem: Data trust issues reduced stakeholder confidence and slowed decision making.
Solution: Implemented automated monitoring and remediation workflows with policy-aware checks.
Business impact: Raised trust scores and cut downstream defect rates significantly.
Architecture: Databricks + Delta Lake + Azure Data Factory + SQL
GitHubDemo
PythonPySparkNext.jsDockerGit

PySpark Learning Platform

An interactive learning experience for building practical PySpark fluency and debugging skills.

Problem: Engineers needed a guided path to become productive with distributed processing quickly.
Solution: Created structured labs, code challenges, and scaffolded templates in a modern UI.
Business impact: Improved onboarding speed and reduced ramp-up time for new engineers.
Architecture: Next.js + Python + Databricks notebooks + GitHub Actions
GitHubDemo
AzureAzure Data FactoryDatabricksPythonSQL

Azure Migration Framework

A structured modernization framework for moving legacy workloads to Azure-native platforms.

Problem: Migration initiatives lacked a reliable blueprint and governance model.
Solution: Defined reusable assessment, migration, and validation patterns for a cloud-native target state.
Business impact: Improved migration confidence and reduced rollback risk.
Architecture: Azure Landing Zones + DevOps + Data Factory + Databricks
GitHubDemo

Certifications

Validated expertise across cloud, data, and AI platforms.

Azure Administrator Associate

Microsoft

Cloud2024

Azure Data Engineer Associate

Microsoft

Data2023

Databricks Data Engineer Associate

Databricks

Analytics2022

Generative AI Fundamentals

Microsoft

AI2024

GitHub

Recent public work and open-source contributions.

Blog

Thoughtful notes on scalable data and AI systems.

May 2026

Designing resilient data platforms with Delta Lake

A practical guide to building robust lakehouse patterns and reliability principles for modern data estates.

Read article

June 2026

Prompt patterns for production-grade AI copilots

Patterns that improve consistency, safety, and observability in enterprise AI workflows.

Read article

Contact

Let’s build something ambitious together.

I’m interested in opportunities where data engineering, intelligent systems, and thoughtful product design converge.

sheetal.singh@example.com LinkedIn GitHub Resume