Skip to main content

Data Engineer

Experience: 4-5
Posted: 18 Nov,2021
Location: India

Param is looking for Data Engineer to join us in our Hyderabad office. 


Data Engineer

 

Qualification:

 •Bachelor/master’s in computer science/IT or equivalent.

•Azure certifications will be added advantage(Certification in Az-900 and/or AZ-204, AZ-303, AZ-304 or AZ-400 , DP200 & DP201).

Experience: 5 - 8 Years

Salary upto 15 LPA


Skills:

•Azure Data lake, Data factory

•Azure Databricks, Azure SQL database, Azure Synapse Database, Azure SQL Datawarehouse.

•Hadoop ecosystem.

•Azure analytics services.

•Programming Scala, Python, R, Spark SQL

•KAFKA, Azure MQ etc.

•MSBI (SSIS, SSAS, SSRS), Oracle, SQL, PL/SQL

•Data Visualization, SAP BI, Power BI

•Data Migration

•SQL Server programming

• Analytic Problem-Solving


Responsiblilities and Duties

• Responsible for implementing robust data pipeline using Microsoft Stack.

• Responsible for creating reusable and scalable data pipelines.

• Responsible for development and deployment of new data platforms.

• Experience with Cloud Platforms such as Azure, AWS , Google Cloud etc.

• Experience on Migrating SQL database to Azure data Lake, Azure data lake Analytics, Azure SQL Database, Azure Synapse, Data Bricks and Azure SQL Data warehouse.

• Experience in Deploying Distributed Messaging Systems using KAFKA, or other messaging System.

• Experience in Developing Spark applications using Spark - SQL in Databricks for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.

• Good understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, Driver Node, Worker Node, Stages, Executors and Tasks.

• Good understanding of Big Data Hadoop and Yarn architecture along with various Hadoop Demons such as Job Tracker, Task Tracker, Name Node, Data Node, Resource/Cluster Manager, and Kafka (distributed stream-processing) .

• Experience in Database Design and development with Business Intelligence using SQL Server 2014/2016, Integration Services (SSIS), DTS Packages, SQL Server Analysis Services (SSAS), DAX, OLAP Cubes, Star Schema and Snowflake Schema.

• Expertise in various phases of project life cycles (Design, Analysis, Implementation and testing).

• Analyze, design and build Modern data solutions using Azure PaaS service to support visualization of data. Understand current Production state of application and determine the impact of new implementation on existing business processes.

• Create Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards.

• Primary responsibilities include using services and tools to ingest, egress, and transform data from multiple sources.

• Create Pipelines in ADF using Linked Services/Datasets/Pipeline/ to Extract, Transform and load data from different sources like Azure SQL, Blob storage, Azure SQL Data warehouse, write-back tool and backwards.

• The Azure data engineer focuses on data-related tasks in Azure.

• Primary responsibilities include using services and tools to ingest, egress, and transform data from multiple sources.

• Azure data engineers collaborate with business stakeholders to identify and meet data requirements.

• Design and implement solutions. Manage, monitor, and ensure the security and privacy of data to satisfy business needs.

• Design and implement data ingestion techniques for real time and batch processes for structured and unstructured data sources.

• Apply cutting-edge software design and cloud migration expertise.

• Responsible for design, development, and architecture of Microsoft Azure Cloud solutions.

• Experience with architecting and designing technical solutions using combination of Azure IaaS and PaaS

 


 

Required Skills

Skill
Years
Months
Azure Data Lake
5
0
Azure Data Factory
5
4
Databricks
6
4
Azure SQL database
5
4
Synapse Analytics
5
4
Azure SQL Datawarehouse
5
4
Big Data Hadoop
5
4
Scala
5
4
Python
4
3
R language
4
3
Spark SQL
5
4
Kafka
4
4
Azure MQ
5
4
MSBI(SSIS, SSAS, SSRS)
5
3
PLSQL
5
4
Oracle
5
4
Oracle Data Visualization
5
4