Senior Data Engineer (Databricks Migration)
Sigma Software- Location
- Warsaw, Masovian Voivodeship, Poland
- Workplace
- Remote
- Employment
- Full Time
- Salary
- —
Posted 1mo ago
- Participate in the migration of a large-scale analytical platform from BigQuery to Databricks
- Design and implement scalable Lakehouse architectures using Databricks and Delta Lake
- Analyze existing ETL / ELT workloads and define migration approaches
- Develop and optimize data pipelines processing large volumes of retail and analytical data
- Implement incremental processing strategies and scalable transformation frameworks
- Build and maintain Spark-based data processing solutions using PySpark
- Design and maintain medallion architecture layers including Bronze, Silver, and Gold
- Implement data governance and security best practices using Unity Catalog
- Collaborate with Data Science, Analytics, Product, and Customer Engineering teams
- Participate in architecture discussions and technical solution design
- Develop reusable data platform components and engineering standards
- Conduct code reviews and contribute to platform reliability and maintainability
- Troubleshoot and optimize complex SQL and Spark workloads
- Support production deployments and platform modernization activities
- 5+ years of professional experience as a Data Engineer
- Strong programming skills in Python and advanced SQL
- Hands-on commercial experience with Databricks
- Strong knowledge of Apache Spark, primarily PySpark
- Experience designing and building modern cloud-based data platforms
- Experience developing ETL / ELT pipelines and large-scale data processing solutions
- Hands-on experience with Delta Lake
- Experience with Spark Declarative Pipelines
- Experience with cluster monitoring, metrics analysis, and performance optimization
- Strong understanding of distributed data processing architectures
- Solid understanding of data warehousing concepts and dimensional modeling
- Experience with Airflow or similar orchestration tools
- Experience optimizing complex analytical SQL workloads
- Experience implementing CI / CD practices for data engineering platforms
- Strong troubleshooting and performance optimization skills
- Ability to work collaboratively in cross-functional international teams
- Upper-Intermediate or higher English level
WILL BE A PLUS
- Experience working with GCP cloud services
- Experience with AWS or Azure cloud platforms
- Experience in retail analytics or pricing optimization domains
- Experience supporting machine learning or AI-related data workloads
- Experience with platform modernization and cloud migration initiatives
PERSONAL PROFILE
- Strong analytical and problem-solving mindset
- Proactive and ownership-driven approach
- Ability to work independently and collaboratively
- Good communication and stakeholder collaboration skills
- Passion for scalable data engineering and modern data platforms
- Interest in continuous learning and technology innovation
Skills
- Databricks
- BigQuery
- Delta Lake
- ETL
- ELT
- Spark
- PySpark
- Unity Catalog
- SQL
- Python
- Airflow
- GCP
- AWS
- Azure
- Machine Learning
More jobs at Sigma Software
All 101Customer Success & Implementation Support Specialist (AI Solutions)
Sigma Software · São Paulo, SP, Brazil · 2d ago
Senior Data Engineer
Sigma Software · Warsaw, Masovian Voivodeship, Poland · 2d ago
Senior MLOps / ML Platform Engineer
Sigma Software · Kraków, Lesser Poland Voivodeship, Poland · 2d ago
Senior ML Engineer / Data Scientist
Sigma Software · Lviv, Lviv Oblast, Ukraine · 3d ago
Senior .NET Developer
Sigma Software · Bucharest, Bucharest, Romania · 3d ago
Similar roles
Principal Data Engineer
Ledgebrook · US · Canada · Poland · USD 180,000–220,000 · today
Senior Full-Stack Engineer
Resourceful Talent Group · Erevan, Erevan, Armenia · Canberra, Australian Capital Territory, Australia · Vienna, Wien, Austria +18 · today
Full Stack Engineer
Ledgebrook · Wrocław, Poland · today
Mid Java Software Engineer
Sportradar · Warszawa, Województwo mazowieckie, Poland · yesterday
Senior Java Software Engineer
Sportradar · Warszawa, Województwo mazowieckie, Poland · yesterday
Engineering Manager
Sportradar · Warsaw, Poland · yesterday