Senior Data Engineer (Databricks Migration)
Sigma Software- Location
- Lisbon, Lisbon, Portugal
- Workplace
- Remote
- Employment
- Full Time
- Salary
- —
Posted 1mo ago
- Participate in the migration of a large-scale analytical platform from BigQuery to Databricks
- Design and implement scalable Lakehouse architectures using Databricks and Delta Lake
- Analyze existing ETL / ELT workloads and define migration approaches
- Develop and optimize data pipelines processing large volumes of retail and analytical data
- Implement incremental processing strategies and scalable transformation frameworks
- Build and maintain Spark-based data processing solutions using PySpark
- Design and maintain medallion architecture layers including Bronze, Silver, and Gold
- Implement data governance and security best practices using Unity Catalog
- Collaborate with Data Science, Analytics, Product, and Customer Engineering teams
- Participate in architecture discussions and technical solution design
- Develop reusable data platform components and engineering standards
- Conduct code reviews and contribute to platform reliability and maintainability
- Troubleshoot and optimize complex SQL and Spark workloads
- Support production deployments and platform modernization activities
- 5+ years of professional experience as a Data Engineer
- Strong programming skills in Python and advanced SQL
- Hands-on commercial experience with Databricks
- Strong knowledge of Apache Spark, primarily PySpark
- Experience designing and building modern cloud-based data platforms
- Experience developing ETL / ELT pipelines and large-scale data processing solutions
- Hands-on experience with Delta Lake
- Experience with Spark Declarative Pipelines
- Experience with cluster monitoring, metrics analysis, and performance optimization
- Strong understanding of distributed data processing architectures
- Solid understanding of data warehousing concepts and dimensional modeling
- Experience with Airflow or similar orchestration tools
- Experience optimizing complex analytical SQL workloads
- Experience implementing CI / CD practices for data engineering platforms
- Strong troubleshooting and performance optimization skills
- Ability to work collaboratively in cross-functional international teams
- Upper-Intermediate or higher English level
WILL BE A PLUS
- Experience working with GCP cloud services
- Experience with AWS or Azure cloud platforms
- Experience in retail analytics or pricing optimization domains
- Experience supporting machine learning or AI-related data workloads
- Experience with platform modernization and cloud migration initiatives
PERSONAL PROFILE
- Strong analytical and problem-solving mindset
- Proactive and ownership-driven approach
- Ability to work independently and collaboratively
- Good communication and stakeholder collaboration skills
- Passion for scalable data engineering and modern data platforms
- Interest in continuous learning and technology innovation
Skills
- Databricks
- BigQuery
- Delta Lake
- ETL
- ELT
- Spark
- PySpark
- Unity Catalog
- SQL
- Python
- Airflow
- GCP
- AWS
- Azure
- Machine Learning
More jobs at Sigma Software
All 101Customer Success & Implementation Support Specialist (AI Solutions)
Sigma Software · São Paulo, SP, Brazil · 2d ago
Senior Data Engineer
Sigma Software · Warsaw, Masovian Voivodeship, Poland · 2d ago
Senior MLOps / ML Platform Engineer
Sigma Software · Kraków, Lesser Poland Voivodeship, Poland · 3d ago
Senior ML Engineer / Data Scientist
Sigma Software · Lviv, Lviv Oblast, Ukraine · 3d ago
Senior .NET Developer
Sigma Software · Bucharest, Bucharest, Romania · 3d ago
Similar roles
Software Development Engineer
Luma Financial Technologies · Lisbon, Portugal · today
Senior Software Development Engineer
Luma Financial Technologies · Lisbon, Portugal · today
AEM Technical Consultant (100% Remote) (India & EMEA)
Allshore Talent · Dolnośląskie, Poland · Bucharest, București, Romania · Blagoevgrad, Bulgaria +17 · today
DevOps Engineer - Data & Machine Learning Platform (f/m/div.)
Bosch Group · Braga, Braga, Portugal · today
Hardware Test Engineer (f/m/div.)
Bosch Group · Ovar, Portugal · today
Senior Data Scientist Reinforcement Learning – Offer Intelligence (m/f/d)
Sixt · Lisbon, Lisbon, Portugal · yesterday