Data Engineer

Arcgate is a dynamic and rapidly growing team of 2500+ professionals passionate about data and technology. We deliver cutting-edge solutions to some of the world’s most innovative startups to market leaders across application development, quality engineering, AI data preparation, data enrichment, search relevance, and content moderation.

Responsibilities:

  • Design, build, and optimize Python based data pipelines that handle large, complex, and messy datasets efficiently.
  • Develop and manage scalable data infrastructures, including databases and data warehouses such as Snowflake, Azure Data Factory etc. ensuring reliability and performance.
  • Build, maintain, and optimize CDC processes that integrate data from multiple sources into the data warehouse.
  • Collaborate closely with data scientists, analysts, and operations teams to gather requirements and deliver high-quality data solutions.
  • Perform data quality checks, validation, and verification to ensure data integrity and consistency.
  • Support and optimize data flows, ingestion, transformation, and publishing across various systems.
  • Work with AWS infrastructure (ECS, RDS, S3), manage deployments using Docker, and package services into containers.
  • Use tools like Prefect, Dagster and dbt to orchestrate and transform data workflows.
  • Implement CI/CD pipelines using Harness and GitHub Actions.
  • Monitor system health and performance using DataDog.
  • Manage infrastructure orchestration with Terraform and Terragrunt.
  • Stay current with industry trends, emerging tools, and best practices in data engineering.
  • Coach and mentor junior team members, promoting best practices and skill development.
  • Contribute across diverse projects, demonstrating flexibility and adaptability.

Qualifications:

  • Bachelor’s degree in Computer Science, Engineering, Mathematics, Physics, or a related field.
  • 5+ years of demonstrable experience building reliable, scalable data pipelines in production environments.
  • Strong experience with Python, SQL programming, and data architecture.
  • Hands-on experience with data modeling in Data Lake or Data Warehouse environments (Snowflake preferred).
  • Familiarity with Prefect, Dagster, dbt, and ETL/ELT pipeline frameworks.
  • Experience with AWS services (ECS, RDS, S3) and containerization using Docker.
  • Knowledge of TypeScript, React, Node.js is a plus for collaborating on the application platform.
  • Strong command of GitHub for source control and Jira for change management.
  • Strong analytical and problem-solving skills, with a hands-on mindset for wrangling data and solving complex challenges.
  • Excellent communication and collaboration skills; ability to work effectively with cross- functional teams.
  • A proactive, start-up mindset, adaptable, ambitious, responsible, and ready to contribute wherever needed.
  • Passion for delivering high-quality solutions with meticulous attention to detail.
  • Enjoy working in an inclusive, respectful, and highly collaborative environment where every voice matters.

Benefits:

  • Competitive salary package.
  • Opportunities for growth, learning, and professional development.
  • Dynamic, collaborative, and innovation-driven work culture.
  • Work with cutting-edge technologies and leading-edge startups.

Excited to turn data into insights and bring your expertise to the next level?

Click the Apply button below and become an Arcgatian!

Apply