Job Description
This a Full Remote job, the offer is available from: Indiana (USA) This role is for one of the Weekday's clients Salary range: Rs 1900000 - Rs 2800000 (ie INR 19-28 LPA) Min Experience: 4 years Location: Remote (India) JobType: full-time We are looking for a highly skilled Big Data Engineer – Streaming to design, build, and maintain large-scale real-time data processing systems. This role is focused exclusively on streaming data pipelines and requires deep expertise in Java-based backend development along with hands-on experience in Apache Spark Streaming or Hadoop streaming ecosystems. The ideal candidate thrives in high-performance environments and has a strong passion for building scalable, low-latency data platforms. Requirements Key Responsibilities • Design, develop, and optimize real-time streaming data pipelines for large-scale, distributed systems. • Build and maintain robust data ingestion and real-time processing frameworks using Apache Spark Streaming and Hadoop-based streaming technologies. • Develop high-performance, production-grade solutions using Java as the core programming language. • Ensure streaming systems meet requirements for scalability, fault tolerance, reliability, and low latency. • Integrate streaming pipelines with downstream applications, platforms, and analytics systems. • Monitor, troubleshoot, and optimize performance bottlenecks across streaming data workflows. • Collaborate closely with platform, infrastructure, and application teams to deliver end-to-end streaming solutions. • Support CI/CD practices for continuous improvement and reliable deployment of data pipelines. Technical Skills • Strong expertise in real-time data processing and streaming architectures. • Hands-on experience with Apache Spark Streaming and/or Hadoop streaming ecosystems (no batch processing). • Advanced proficiency in Java for large-scale, distributed systems development. • Experience with CI/CD pipelines and production-grade deployment practices. • Familiarity with cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform. • Solid understanding of system performance tuning, fault tolerance, and distributed computing concepts. Skills Big Data Engineering Real-Time Streaming Apache Spark Streaming Hadoop Streaming Java Cloud Platforms (AWS / Azure / GCP) CI/CD This offer from "Weekday (YC W21)" has been enriched by Jobgether.com and got a 72% flex score. Apply tot his job