Senior Big Data Engineer
Tata Consultancy Services - tampa, FL
Apply NowJob Description
Job DescriptionSenior Big Data Engineer* Designing and implementing ETL Process using Bigdata Technologies (Pyspark, Spark, Hadoop, Hive, Sqoop, Oozie, Unix Shell Script)* Experience in handling Querycode Optimization & query performance* Managing the Data Lake environment like Compliance issues fix, Upgrades, Data ArchivalRetrieval process tofrom AWS S3 * Developing Automation scripts to fix the Production issues and mentors Team to handle the Prod issues* Managing & Setting up the Ingestion * Developing data extraction and transformations to improve data reliability and quality* Utilizing past experiences to identify and troubleshoot potential issues, ensuring smooth data operations* Advising team members in analyzing relevant data and assessing alternatives* Provide effective solutions to complex technical challenges, applying technical competence to understand and resolve issues.* Develop high-performance ingestion pipelines to a Big Data platforms, ensuring consistent and reliable data flows
Created: 2024-10-13