Senior Data Engineer – Remote India

  • Full Time
  • India

Website idctechnologies

Location

India – Remote

Job Category

  • Information Technology (IT) & Software
  • Engineering & Technical
  • Science & Research
  • Remote / Work-from-Home Opportunities

Job Overview

We are seeking an experienced Senior Data Engineer to join a technology-focused data engineering team in a remote position based in India. The ideal candidate will have 7+ years of professional experience and strong hands-on expertise in modern data engineering technologies, including Databricks, PySpark, Spark Structured Streaming, Delta Lake, and Lakehouse Architecture.

The role will involve designing, developing, and supporting scalable data platforms and real-time data processing solutions. Candidates should have practical experience with Databricks Auto Loader, Delta Live Tables (DLT), Unity Catalog, and modern Lakehouse environments, along with exposure to AI and Generative AI solutions.

This position offers an opportunity for experienced data professionals to work on advanced data engineering technologies while expanding their expertise in cloud-based data platforms, real-time processing, Lakehouse Architecture, and AI/GenAI applications. The role can support long-term career growth through continued technical experience and professional development in modern data engineering.

Key Responsibilities

  • Design and develop scalable data engineering solutions using Databricks and PySpark.
  • Build and maintain data processing pipelines using Spark Structured Streaming.
  • Implement efficient data ingestion solutions using Databricks Auto Loader.
  • Develop and manage data workflows using Delta Live Tables (DLT).
  • Work with Delta Lake for reliable and scalable data storage and processing.
  • Implement and manage data governance capabilities using Unity Catalog.
  • Contribute to Lakehouse Architecture and modern data platform development.
  • Develop solutions for batch and real-time data processing requirements.
  • Support the integration of AI and Generative AI capabilities into data solutions.
  • Collaborate with technical teams to develop reliable and scalable data platforms.
  • Apply data engineering best practices for performance, scalability, and maintainability.

Requirements & Qualifications

  • Minimum 7+ years of professional experience in data engineering or a closely related technical field.
  • Strong hands-on experience with Databricks.
  • Advanced practical knowledge of PySpark and Apache Spark.
  • Experience with Spark Structured Streaming.
  • Hands-on experience with Databricks Auto Loader.
  • Experience working with Delta Live Tables (DLT).
  • Strong knowledge of Delta Lake.
  • Experience with Unity Catalog and data governance concepts.
  • Understanding of Lakehouse Architecture.
  • Exposure to AI and Generative AI solutions.
  • Strong analytical, problem-solving, and technical communication skills.
  • Ability to work effectively in a remote professional environment.

Salary, Benefits & Career Growth

Benefits

Specific company benefits were not provided in the original job description. Candidates should confirm the applicable benefits and employment terms during the recruitment process.

Career Growth

This role provides experienced data engineering professionals with the opportunity to strengthen their expertise in Databricks, PySpark, real-time data processing, Lakehouse Architecture, data governance, and AI/GenAI solutions. Continued exposure to modern data technologies can support long-term technical career growth.

Training & Professional Development

The position offers opportunities to expand practical knowledge across modern data engineering platforms and technologies, including Databricks, Spark, Delta Lake, Unity Catalog, Lakehouse Architecture, and AI/GenAI solutions.

Application Process

Application Process (Website)

Apply only through the official job link.

Click Apply Now on the website and follow the application instructions.

HR Email for Application

Send your updated CV directly

Only shortlisted candidates will be contacted.

To apply for this job email your details to mounika.t@idctechnologies.com