Manager

Role Overview

We are looking for a highly skilled and motivated Senior Data Engineer to join our growing data engineering team.

Key Responsibilities

• In this role, you will build services that will integrate directly with Lending Club’s products and with external vendors. you
• You will work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, etc., modern data platforms such as Databricks and Snowflake and cloud technologies across AWS stack
• Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc.
• Implement processes and systems to monitor Data Quality, Observability, Governance and Lineage.
• Support operations to manage the production environment and help in resolving production issues with RCA
• Write unit/integration tests, adopt Test-driven development, contribute to engineering wiki, and document design/implementation etc.

Skills required

• Strong hands-on experience with distributed systems and big data technologies, including Hadoop, Spark, Hive, DBT, and Airflow/Dagster.
• Minimum 4 years of solid production-quality coding experience in Python focused on data pipeline development.
• Experience working with cloud platforms, preferably AWS.
• Experience with Databricks and/or Snowflake.
• Proficiency in Git, JIRA, Jenkins, and shell scripting.
• Good understanding of Agile methodologies, source control management, test automation, and Test-Driven Development.
• Experience working with cross-functional teams in fast-paced and dynamic environments.
• Strong analytical, collaborative problem-solving, and communication skills.
• Proven track record of building fast, scalable, reliable, and high-quality data pipelines.

Nice to Have

Experience building data pipelines for Digital Marketing use cases.

Required Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
• 6+ years of experience in Data Engineering or related roles
• Self-driven and proactive mindset.
• Strong ownership and accountability and communication skills. Passion for building scalable and maintainable data systems.
• Ability to work independently while collaborating effectively with distributed teams and client.
• In this role, you will build services that will integrate directly with Lending Club’s products and with external vendors. you
• You will work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, etc., modern data platforms such as Databricks and Snowflake and cloud technologies across AWS stack
• Identify, design, and implement automation of manual processes, optimizing data delivery, reducing Cloud cost, etc.
• Implement processes and systems to monitor Data Quality, Observability, Governance and Lineage.
• Support operations to manage the production environment and help in resolving production issues with RCA
• Write unit/integration tests, adopt Test-driven development, contribute to engineering wiki, and document design/implementation etc.
• Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience.
• 6+ years of experience in Data Engineering or related roles
• Self-driven and proactive mindset.
• Strong ownership and accountability and communication skills. Passion for building scalable and maintainable data systems.
• Ability to work independently while collaborating effectively with distributed teams and client.