Big Data Engineer- Mountain View, CA in Mountain View, CA at MDI Group

Date Posted: 9/6/2018

Job Snapshot

Job Description


MDI Group is a premier IT workforce solutions provider with more than 25 years of expertise in finding “best fit” IT talent for mid-sized to Fortune 500 clients. We have established relationships with our clients and work directly with the hiring managers. Why MDI Group?...You can expect that we will learn what is most important to you in your job search and match that to the needs of our clients. We offer career coaching and resume services, skills certifications, interview preparation skills, health benefits and a 401K plan. We are currently interviewing for the following contract position:

Big Data Engineer

Contract

Mountain View, CA

Description:   

Big Data Engineers serve as the backbone of the Strategic Analytics organization, ensuring both the reliability and applicability of the team’s data products to the entire Samsung organization.

They have extensive experience with ETL design, coding, and testing patterns as well as engineering software platforms and large-scale data infrastructures.

Big Data Engineers have the capability to architect highly scalable end-to-end pipeline using different open source tools, including building and operationalizing high-performance algorithms.

Extensive experience working with both 1) big data platforms and 2) real-time / streaming deliver of data is essential.

Background, Experience& Qualifications:

  • 5-8 years of Python or Java/J2EE development experience
  • 3+  years of demonstrated technical proficiency with Hadoop and big data projects
  • 5-8 years of demonstrated experience and success in data modeling
  • Fluent in writing shell scripts [bash, korn]
  • Writing high-performance, reliable and maintainablecode.
  • Ability to write MapReduce jobs
  • Ability to setup, maintain, and implement Kafka topics and processes
  • Understanding and implementation of Flume processes
  • Good knowledge of database structures, theories, principles, and practices.
  • Understand how to develop code in an environment secured using a local KDC and OpenLDAP.
  • Familiarity with and implementation knowledge of loading data using Sqoop.
  • Knowledge and ability to implement workflow/schedulers within Oozie
  • Experience working with AWS components [EC2, S3, SNS, SQS]
  • Analytical and problem solving skills, applied to Big Data domain
  • Proven understanding and hands on experience with Hadoop, Hive, Pig, Impala, and Spark
  • Good aptitude in multi-threading and concurrency concepts.

Additional qualifications:

  • Tune Hadoop solutions to improve performance and end-user experience
  • Proficient in designing efficient and robust data workflows
  • Documenting requirements as well as resolve conflicts or ambiguities
  • Experience working in teams and collaborate with others to clarify requirements
  • Strong co-ordination and project management skills to handle complex projects
  • Excellent oral and written communication skills

Major Responsibilities:  

  • Translate complex functional and technical requirements into detailed design.
  • Design for now and future success
  • Hadoop technical development and implementation.
  • Loading from disparate data sets. by leveraging various big data technology e.g. Kafka
  • Pre-processing using Hive, Impala, Spark, and Pig
  • Design and implement data modeling
  • Maintain security and data privacy in an environment secured using Kerberos and LDAP
  • High-speed querying using in-memory technologies such as Spark.
  • Following and contributing best engineering practice for source control, release management, deployment etc
  • Production support, job scheduling/monitoring, ETL data quality, data freshness reporting
  • This job description is designed to cover the main responsibilities and duties of the role but is not designed to be a comprehensive list of all.

See our new look and learn why more than 25 years of IT focus makes MDI Group different at www.mdigroup.com!


CHECK OUT OUR SIMILAR JOBS

  1. Software Engineer Jobs
  2. Project Engineer Jobs

A GREAT PLACE FOR GREAT PEOPLE TO WORK

Search IT Jobs