Data Engineer, Pasay

Qualifications for Data Engineer/Operations

            Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.

            Experience building and optimizing data pipelines, architectures and data sets.

            Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

            Strong analytic skills related to working with structured and unstructured datasets.

            Build processes supporting data transformation, data structures, metadata, dependency and workload management.

            A successful history of manipulating, processing and extracting value from large disconnected datasets.

            Strong project management and organizational skills.

            Experience supporting and working with cross-functional teams in a dynamic environment.

            Experience in retail and its business metrics is a plus.

            We are looking for a candidate with 3+ years of experience in a Data Engineer/Operations role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. They should also have experience using the following software/tools:

            Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.

            Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.

            Experience with AWS cloud services: EC2, EMR, RDS, Redshift

            Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.

            Experience with ETL/ELT tools: AWS Glue, Informatica, Talend, etc.

Optional

            Working knowledge of message queuing, stream processing, and highly scalable big data data stores.

            Experience with big data tools: Hadoop, Spark, Kafka, etc.

            Experience with stream-processing systems: Storm, Spark-Streaming, etc.

Responsibilities:

Designing, developing, and modifying data infrastructure to accelerate the processes of data analysis and reporting

General Duties

1. Analyzing and organizing raw data

2. Building data systems and pipelines

3. Evaluating business needs and objectives

4. Interpreting trends and patterns

5. Preparing data for descriptive, prescriptive and predictive modeling

6. Building algorithms and prototypes

7. Developing analytical tools and programs

8. Collaborating with data analyst, data scientists and architects on projects

Detailed Duties

1. Analysis and profiling in preparation for data ingestion to data lake

2. Develop, maintain and monitor ETL/ELT jobs

3. Analysis and preparation of data models for the data warehouse

4. Develop, maintain and monitor SQL queries

5. Analysis and preparation of descriptive, prescriptive or predictive analytics

6. Document, optimized and maintain existing data pipelines