Lead Cloudera Kafka And Spark Streaming Engineer Job In Talent Arabia Fz Llc In Bengaluru / Bangalore, Hyderabad / Secunderabad, Telangana

Participate in requirement gathering, analysis, solution design to build big data engineering application on Cloudera Data Platform Hands on involvement in Low Level Design, Development and architecting of large Data projects leading developer and testing teams Build, maintain and monitor large scale Big Data analytics pipelines on Cloudera Data Platform Build the end to end data engineering pipelines by programming/coding on sqoop, kafka, spark, spark-streaming, hive, hbase , nifi Job Scheduling and Automation.

Data Validation, Quality checks, profiling and data reconciliation testing Leverage Cloudera tools to like Ranger , Atlas, Knox to implement security, data governance, data lineage and compliance best practices Work as an individual contributor as well lead teams when required Mentor junior members in the team by improving their skills and knowledge and have the ability to get things delivered Work with both senior and junior team members like Project Manager, Cloudera Architect, other data engineers, data modelers, report developers, testers to complete the task.

Build batch/stream data ingestion engineering pipelines using Nifi, Sqoop, Spark, Hive, Hbase and related scheduling and automations around it Resource and security management using Kerberos , Ranger, Atlas, Data Encryption Setting up security and governance policies on data , users, data pipelines Troubleshooting application errors and ensuring that they do not occur again Apply agile and CI/CD methodologies and tools for development and deployment in agile mode.

Raise tickets to Cloudera Support for any CDP platform issues and get it resolved in timely manner

Lead Cloudera- Kafka And Spark Streaming Engineer

Talent Arabia Fz Llc is looking for Bachelor Of Technology (b.tech/b.e) profile candidates.