JobHabor
Sign inStart free trial

Senior Data Engineer | Offshore

Photon
Location
India
Workplace
—
Employment
—
Salary
—
Apply on the employer’s site

Posted 21d ago

  • Key Responsibilities
  • Assess existing SQL Server (Atlas) and Snowflake environments to identify pipeline gaps, data quality issues, and structural inefficiencies
  • Build and maintain data pipelines supporting the Operational Data Store (ODS), including ingestion, transformation, and loading of key business entities
  • Implement, version-control, and enforce data dictionary standards, logical/physical entity models, and naming conventions established by the Data Architect
  • Design, implement, and maintain CI/CD pipelines (e.g., GitHub Actions, GitLab CI) to automate the deployment of Snowflake schemas, database structures, and pipeline code
  • Manage database schema migrations as code using tools such as dbt, Schemachange, Flyway, or Terraform
  • Execute the batch-to-API migration roadmap — re-engineering Informatica/EDW batch flows to consume FDR APIs
  • Implement automated data quality frameworks, testing validations (e.g., dbt tests, Great Expectations), and data parity reconciliation routines between legacy and new ODS environments
  • Collaborate with domain teams to engineer pipelines for customer, loan, payment, and interaction data entities
  • Apply Snowflake best practices in structuring schemas, managing compute, and separating analytical from operational workloads
  • Partner with the internal data architect and downstream consumers to ensure data accuracy, lineage, and observability across the platform
  • Leverage generative AI tools (e.g., coding assistants, LLMs) to accelerate pipeline development, optimize query performance, write documentation, and generate automated tests
  • Must-Have Skills
  • 7+ years in data engineering roles with a strong focus on pipeline development, CI/CD setup, and database modeling
  • Strong programming skills in SQL (Sql Server and Postgres) and Python for data engineering, automation, and pipeline development
  • Proficiency in Snowflake — including schema design, performance tuning, and data loading patterns
  • Hands-on experience building DevOps/DataOps pipelines (Git, CI/CD tools, automated deployment strategies)
  • Hands-on experience with database migration tools / schema management systems (e.g., dbt, Schemachange, Flyway)
  • Hands-on experience with ETL/ELT tools and batch processing (Informatica experience strongly preferred)
  • Solid understanding of ODS concepts, dimensional modeling, and managing data dictionaries
  • Experience integrating with REST/API-based data sources as part of modernization or migration efforts
  • Demonstrated experience using Generative AI coding assistants (e.g., Copilot, ChatGPT, Gemini) to enhance coding productivity, write tests, and troubleshoot complex SQL/Python scripts
  • Background in financial services or lending domain
  • Nice to Have
  • Experience with modern orchestrators (e.g., Apache Airflow, Prefect, Dagster) or Snowflake-native orchestration (Tasks, Dynamic Tables)
  • Familiarity with deploying semantic layers or metadata-driven catalog systems (Collibra, Alation) to support downstream AI/ML and Natural Language query agents
  • Experience working with FDR or similar loan servicing platforms
  • Familiarity with data mesh principles and domain-oriented data ownership
  • Exposure to AI/ML data pipeline requirements and feature engineering
  • Experience with data governance and cataloguing tools (Collibra, Alation, etc.)
  • Background in student lending or consumer finance

Skills

  • SQL Server
  • Snowflake
  • GitHub Actions
  • GitLab CI
  • dbt
  • Terraform
  • Informatica
  • Great Expectations
  • Generative AI
  • LLM
  • SQL
  • PostgreSQL
  • Python
  • Git
  • ETL
  • ELT
  • GitHub Copilot
  • ChatGPT
  • Gemini
  • Airflow
  • Prefect
  • Dagster

More jobs at Photon

All 822

Similar roles