Big Data Infrastructure Engineer

Job Overview

We are seeking a Big Data Infrastructure Engineer with a strong AI-first and automation-driven mindset to join our

infrastructure team and support the administration and operation of enterprise Big Data platforms and their underlying

infrastructure.

The ideal candidate should have strong hands-on experience in Linux operating system administration and database

administration, with particular focus on configuration, troubleshooting, performance tuning, optimization, monitoring,

and production support.

The candidate is expected to actively leverage AI-assisted tools and automation to improve troubleshooting,

operational efficiency, scripting, documentation, research, and day-to-day engineering activities, while maintaining

the technical judgment required to validate solutions before implementation.

The role also requires practical knowledge of Docker and Kubernetes, along with a good understanding of clustering,

high availability, distributed systems, and Big Data concepts. A strong willingness to continuously learn and stay

current with emerging AI, automation, infrastructure, and Big Data technologies is essential.

Duties and Responsibilities

• Leverage AI-assisted tools to improve troubleshooting, log analysis, scripting, documentation, research, and

operational efficiency while validating outputs before implementation.

• Identify repetitive operational activities and develop automation solutions using Shell/Bash, Python, Ansible,

APIs, or other appropriate technologies.

• Continuously evaluate emerging AI and automation capabilities and identify practical opportunities to improve

infrastructure operations and engineering workflows.

• Administer, configure, manage, troubleshoot, and optimize Linux operating systems supporting production and

non-production environments.

• Monitor and analyze CPU, memory, disk, filesystem, network, processes, and system services, and perform

configuration and performance tuning when required.

• Administer and manage PostgreSQL, MySQL/MariaDB, and Redis, including configuration, access management,

backup and recovery, monitoring, troubleshooting, maintenance, and performance optimization.

• Support database replication, high availability, backup/recovery, and capacity management requirements.
• Support and maintain Docker and Kubernetes environments, including deployment, configuration, monitoring,

troubleshooting, scaling, and cluster administration.

• Support clustered and distributed platforms with focus on high availability, replication, failover, load

balancing, quorum, capacity management, and disaster recovery.

• Support the installation, configuration, monitoring, administration, and upgrade of Cloudera/Hortonworks and

Hadoop-based environments.

• Support and troubleshoot Hadoop ecosystem components such as HDFS, YARN, Hive, Spark, HBase, and

Kafka, as well as related platforms such as Airflow, Superset, and Trino/Presto where applicable.

• Perform production monitoring and support using tools such as Zabbix and Grafana, and participate in incident

management, root cause analysis, and corrective/preventive actions.

• Support security integrations and technologies such as Ranger, LDAP, and Kerberos.
• Collaborate with development, infrastructure, and other technical teams on deployments, upgrades,

infrastructure changes, troubleshooting, and production support.

• Maintain technical documentation, operational procedures, automation, and infrastructure configuration records.

Skills and Qualifications

• 3-5 years of relevant hands-on experience in Linux/System Administration, Database Administration, Big Data

Infrastructure, DevOps, or a related infrastructure role.

• Bachelor's Degree in Computer Science, Computer Engineering, Information Technology, or a related field.
• Strong AI-first and automation-driven mindset, with demonstrated ability to use AI-assisted tools effectively in

technical workflows and critically validate generated recommendations before applying them.

Big Data Infrastructure Engineer - Job Description

Good scripting and automation skills using Shell/Bash; knowledge of Python, Ansible, APIs, or similar

technologies is highly desirable.

• Strong hands-on knowledge of Linux administration, including system configuration, service management,

resource management, storage/filesystems, permissions, networking, troubleshooting, and performance

optimization.

• Good hands-on knowledge of PostgreSQL, MySQL/MariaDB, and Redis administration, including configuration,

backup and recovery, users and privileges, monitoring, maintenance, and performance tuning.

• Good understanding of database concepts including connections, transactions, locks, indexing, query

performance, replication, and high availability.

• Good hands-on understanding of Docker and Kubernetes, including containers, images, pods, deployments,

services, storage, networking, monitoring, resource management, and troubleshooting.

• Good understanding of clustering and distributed system concepts, including high availability, replication,

failover, load balancing, and quorum.

• Good understanding of networking fundamentals, including TCP/IP, DNS, ports, routing, connectivity, and

network troubleshooting.

• Good understanding of Big Data concepts and the Hadoop ecosystem, with familiarity or hands-on experience

in HDFS, YARN, Hive, Spark, Kafka, and HBase.

• Familiarity with Cloudera or Hortonworks platforms is highly desirable.
• Familiarity with Zabbix/Grafana, Ranger/LDAP/Kerberos, CI/CD tools, and Trino/Presto is an advantage.
• Strong troubleshooting, analytical, and problem-solving skills, with the ability to investigate issues

systematically and identify root causes.

• Ability to work effectively in production environments, collaborate across technical teams, take ownership of

assigned activities, and continuously develop technical knowledge.

Preferred Certifications

• Relevant Linux certifications such as RHCSA or RHCE.
• Kubernetes certification such as CKA.
• PostgreSQL or MySQL-related certifications/training.
• Red Hat Ansible or other relevant automation certifications.

Note

Certifications are considered an advantage and are not a substitute for practical hands-on experience.