Senior Data Engineer

Duties & Responsibilities

Job Duties

Senior Data Engineer with GREENSKY MANAGEMENT COMPANY LLC in Atlanta, GA. Remote work permitted.

Identify, design, and implement internal process improvements

automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Work with data and analytics experts to strive for greater functionality in our data systems. Design and develop/build jobs using MSSQL, SQL, Snowflake SQL, Java, Talend, JavaScript, Python and Bash to move data generated by one of our Online Transactional Processing (OLTP) databases and move it into our Data Lake. Build/develop jobs using tools such as Talend or any other ETL tools in order to allow for the data movement. Design Talend, Snowflake, and SQL jobs that will clean, transform, and move the data into the correct places. Make additions to the automated tools that display the data to the end users or simplify the data for consumption by Greensky's own internal processes. Design jobs in such a way that they can be monitored to fix any problems that arise from working on the databases. Build and maintain robust CI/CD pipelines using GitHub to automate the testing and promotion of dbt models, Snowpark procedures, and schema migrations. Implement enterprise-wide security standards using Row Access Policies, Dynamic Data Masking, and Object Tagging to automate compliance. Implement automated data quality checks, schema drift alerts, and lineage tracking to maintain trust and compliance. Mentoring data engineers, code reviews and performance enhancements to maintain engineering standards, governance and cost optimization.

Required Skills/Qualifications

Job Requirements

Master’s degree (U.S. or foreign equivalent) in Computer Information Systems, Information Technology, Computer Science, Computer Engineering, Electrical Engineering or a related field and three (3) years of experience in the job offered or in a related role. Prior experience must include three (3) years of experience with: build and maintain ETL tools to develop jobs for extracting, integrating, and loading data into data warehouse, database environments; using MS-SQL, Snow-SQL, Stored Procedures, Tasks, Warehouse Management, Amazon-AWS/S3, Talend, ETL like SSIS, HTML, XML, JSON and JavaScript, Python; develop data applications, data processing workflows and automated scripts; design and maintain stored procedure, user-defined functions and end-end data pipeline orchestrations across multiple data sources and teams; automate workflows and build complex transformations using automation tools and scheduling; implement incremental loads, CDC patterns, merge based upsert, and end-end data pipelines; build testing strategies including unit tests, data quality checks and integration checks and migrate code using CI/CD pipeline and version control tools like Github. Prior experience must include two (2) years of experience with: conducting Audit processes in cloud warehouses. Prior experience must include one (1) year of experience with: using Data governance tools such as Ovaledge with deep understanding of data catalogs, crawling and profiling meta data, and the data lifecycle process; and using change data capture tools for configuration, management and support of real time data synchronization. Up to 10% travel required. Domestic only.

GreenSky is an equal opportunity employer and will not discriminate against any employee or applicant on the basis of age, color, disability, gender, national origin, race, religion, sexual orientation, veteran status, or any classification protected by federal, state, or local law. For more information, visit us on the web at www.greensky.com.