Senior Data Engineer
- Lead root-cause investigations across 8+ source systems feeding underwriting, actuarial and regulatory reporting. Two examples: a formatting bug affecting 267,911 reinsurance records, and a hardcoded rule that sent every New York policy to full underwriting.
- Converted full-reload pipelines to watermark-based incremental merges on two production feeds, catching a bug that would have reset the watermark on empty runs.
- Removed about 40% duplicate records from a partner-facing report.
- Asked by the Director of Data Governance to help draft a partner data contract.
- Present findings at sprint reviews to product owners and source-system owners.
- Write runbooks and setup guides used by engineers joining the Databricks platform.
The four projects below are from this role.