Big Data Engineer

We are actively hiring for Big data engineer Location : Remote W2 Visa: USC, GC, GC EAD, H4 EAD Looking for candidates with 10+ years experience Must have hands-on experience with Databricks Hands-on experience with high-velocity high-volume stream processing: Apache Kafka and Spark Streaming (A. Experience with real-time data processing and streaming techniques using Spark structured streaming and Kafka B. Deep knowledge of troubleshooting and tuning Spark applications) Hands-on experience with Python and/or Scala i.e. PySpark/Scala-Spark Hands-on experience building, testing, and optimizing 'Big Data' data ingestion pipelines, architectures and data sets Experience in successfully building and deploying a new data platform on Azure/ AWS Strong knowledge of Messaging Platforms like Kafka, Amazon MSK & TIBCO EMS or IBM MQ Series Databricks UI, Managing Databricks Notebooks, Delta Lake with Python, Delta Lake with Spark SQL, Delta Live Tables, Unity Catalog Experience with data ingestion of different file formats across like JSON, XML, CSV Experience with NoSQL databases, including HBASE and/or Cassandra Knowledge of Unix/Linux platform and shell scripting is a must Experience with Cloud platforms e.g. AWS, GCP, etc. Experience with database solutions like Kudu/Impala, or Delta Lake or Snowflake or BigQuery Please share updated profile for *************

Created: 2024-11-19