Bangalore
2 years ago
50 Applied
JOB ID:42122
Job Type
Full Time
Functional Area
Database Administrator/ Data warehousing,Research/Analytics/Business Intelligence/Big data,RnD/Engineering
Industry
IT-Software/Software Services
Education
Not Specified
Experience Required
5 - 9 Yrs
Key Skills Required
azure: adls, databricks, stream analytics, sql dw, cosmos db, analysis services, azure functions, serverless architecture, arm templates, azure, aws ‘big data’ technologies
Other Skills Required
Job Description
Create and maintain optimal data ... See More
Create and maintain optimal data pipeline architecture.
Assemble large, complex data sets that meet functional / non-functional business requirements.
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Azure, AWS ‘big data’ technologies.
Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
Keep our data separated and secure across national boundaries through multiple data centers and Azure, AWS regions.
Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
Work with data and analytics experts to strive for greater functionality in our data systems.Qualifications for Data Engineer
Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
Strong analytic skills related to working with unstructured datasets.
Build processes supporting data transformation, data structures, metadata, dependency and workload management.
A successful history of manipulating, processing and extracting value from large disconnected datasets.
Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
Strong project management and organizational skills.
Experience supporting and working with cross-functional teams in a dynamic environment.
We are looking for a candidate with 6+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:
Experience with Azure: ADLS, Databricks, Stream Analytics, SQL DW, COSMOS DB, Analysis Services, Azure Functions, Serverless Architecture, ARM Templates
Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
Experience with object-oriented/object function scripting languages: Python, SQL, Scala, Spark-SQL etc.
Nice to have experience with big data tools: Hadoop, Spark, Kafka, etc.
Nice to have experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
Nice to have experience with stream-processing systems: Storm
See Less
Shell is an international energy company with expertise in the exploration, production, refining and marketing of oil and natural gas, and the manufacturing and marketing of chemicals. We use advanced technologies and take an innovative approach to help build a sustainable energy future. We invest in power, including from renewable sources such as wind and solar. We also invest in electric vehicle charging and low-carbon fuels for transport, such as advanced biofuels and hydrogen.
Our operations are divided into our businesses: Integrated Gas and Upstream, and Downstream, Renewables and Energy Solutions. Our Projects & Technology organisation manages the delivery of Shell’s major projects and drives our research and innovation.
Our purpose
Shell’s purpose is to power progress together by providing more and cleaner energy solutions. We believe that rising standards of living for a growing global population are likely to continue to drive demand for energy, including o
... See MoreShell is an international energy company with expertise in the exploration, production, refining and marketing of oil and natural gas, and the manufacturing and marketing of chemicals. We use advanced technologies and take an innovative approach to help build a sustainable energy future. We invest in power, including from renewable sources such as wind and solar. We also invest in electric vehicle charging and low-carbon fuels for transport, such as advanced biofuels and hydrogen.
Our operations are divided into our businesses: Integrated Gas and Upstream, and Downstream, Renewables and Energy Solutions. Our Projects & Technology organisation manages the delivery of Shell’s major projects and drives our research and innovation.
Our purpose
Shell’s purpose is to power progress together by providing more and cleaner energy solutions. We believe that rising standards of living for a growing global population are likely to continue to drive demand for energy, including oil and gas, for years to come. At the same time, the need to tackle climate change means there is a global transition under way to a low-carbon energy system.
Our people
Our people are essential to the successful delivery of the Shell strategy and to sustaining business performance over the long term. Performing competitively in the evolving energy landscape requires competent and empowered people working safely together across Shell. We believe that diverse teams led by inclusive leaders deliver better safety and business performance.
See Less
