JobHabor

Senior Data Engineer (Databricks Migration)

Sigma Software
Location
Kraków, Lesser Poland Voivodeship, Poland
Workplace
Remote
Employment
Full Time
Salary
Apply on the employer’s site

Posted 1mo ago

  • Participate in the migration of a large-scale analytical platform from BigQuery to Databricks
  • Design and implement scalable Lakehouse architectures using Databricks and Delta Lake
  • Analyze existing ETL / ELT workloads and define migration approaches
  • Develop and optimize data pipelines processing large volumes of retail and analytical data
  • Implement incremental processing strategies and scalable transformation frameworks
  • Build and maintain Spark-based data processing solutions using PySpark
  • Design and maintain medallion architecture layers including Bronze, Silver, and Gold
  • Implement data governance and security best practices using Unity Catalog
  • Collaborate with Data Science, Analytics, Product, and Customer Engineering teams
  • Participate in architecture discussions and technical solution design
  • Develop reusable data platform components and engineering standards
  • Conduct code reviews and contribute to platform reliability and maintainability
  • Troubleshoot and optimize complex SQL and Spark workloads
  • Support production deployments and platform modernization activities
  • 5+ years of professional experience as a Data Engineer
  • Strong programming skills in Python and advanced SQL
  • Hands-on commercial experience with Databricks
  • Strong knowledge of Apache Spark, primarily PySpark
  • Experience designing and building modern cloud-based data platforms
  • Experience developing ETL / ELT pipelines and large-scale data processing solutions
  • Hands-on experience with Delta Lake
  • Experience with Spark Declarative Pipelines
  • Experience with cluster monitoring, metrics analysis, and performance optimization
  • Strong understanding of distributed data processing architectures
  • Solid understanding of data warehousing concepts and dimensional modeling
  • Experience with Airflow or similar orchestration tools
  • Experience optimizing complex analytical SQL workloads
  • Experience implementing CI / CD practices for data engineering platforms
  • Strong troubleshooting and performance optimization skills
  • Ability to work collaboratively in cross-functional international teams
  • Upper-Intermediate or higher English level

WILL BE A PLUS

  • Experience working with GCP cloud services
  • Experience with AWS or Azure cloud platforms
  • Experience in retail analytics or pricing optimization domains
  • Experience supporting machine learning or AI-related data workloads
  • Experience with platform modernization and cloud migration initiatives

PERSONAL PROFILE

  • Strong analytical and problem-solving mindset
  • Proactive and ownership-driven approach
  • Ability to work independently and collaboratively
  • Good communication and stakeholder collaboration skills
  • Passion for scalable data engineering and modern data platforms
  • Interest in continuous learning and technology innovation

Skills

  • Databricks
  • BigQuery
  • Delta Lake
  • ETL
  • ELT
  • Spark
  • PySpark
  • Unity Catalog
  • SQL
  • Python
  • Airflow
  • GCP
  • AWS
  • Azure
  • Machine Learning

More jobs at Sigma Software

All 101

Similar roles