Big Data Software Engineer New York, NY
Big Data Software Engineer with broad technical knowledge and depth of experience with production implementations of server side software and big data in particularThis is a key role in our server engineering team. You’ll be building both iterative and substantial revisions to our flagship tracking, reporting, and ad-serving tools and will have an integral role in our engineering organization. You’ll have the opportunity to interact with, teach and learn from many experienced developers working on emerging mobile, analytics, data processing, and ad serving platforms.
We are looking for someone to provide innovative solutions to scale and build rock-solid code for reliable server systems. Our clients expect us to lead and we work hard to deliver reliable and timely information and a phenomenal end user experience.
Key responsibilities are:
Architect and Build Big Data Platform using Hadoop eco system.
Architect and build data collectors and data fabric to collect and transport data to the Hadoop echo system based Big Data Platform.
Build monitoring solution(s) for the Big Data infrastructure to understand the health of the infrastructure
Be the SME (Subject Matter Expert) for Big Data Platform(s).
Requirements:
Bachelors degree in Computer Science, Management Information Systems or related field
4+ years of experience working in Linux/Unix environment.
5+ years of experience building and managing complex products/solutions.
Expert level experience architecting, building and maintaining Enterprise grade Hadoop store.
Expert level understanding of Hadoop HDFS and MapReduce framework internals.
Experience with components in Hadoop echo system (Hive, Pig, Impala, Ambari, Oozie, Sqoop,,Zookeeper, Mahout)
Experience architecting and building data fabric with Flume, Scribe or kafka would be a plus.
You love big data and have a burning desire to do so now.
Big bonus points for experience using scalable data processing tools such as Hadoop, Storm, Spark, Streaming, Shark
Bigger bonus points for experience with building AI or rule engines or other advanced matching algorithms.
Experience with open source frameworks like Puppet or Chef for deployment and configuration a plus
Management experience is a plus.
You possess knowledge and experience in at least 3 different languages (i.e. C++, Python, Java, Scala, Ruby). We are a 100% Linux shop but if you have Windows experience we would see this as another way you can show us how you tackle and solve problems with your code.
While we do all of our data aggregation in Hadoop, ultimately the results end up in SQL databases. Your knowledge and experience with databases (DDL & DML) are a must. If you have used NoSQL systems before that’s great but not a requirement.
EOE
To be considered for this position, you must use the “apply now” button to submit your resume. If you have questions about the position please contact Milly Rivera at 203-325-3544 and/or milly.rivera@adeccona.com |
Job Description