For more than 60 years, the Lawrence Livermore National Laboratory (LLNL) has applied science and technology to make the world a safer place.
We have an opening for an innovative Big Data System Infrastructure Engineer. You will apply your knowledge in the development and improvement of our high performance and Big Data Intensive (e.g. Hadoop/Spark) compute infrastructure and parallel file system environment. You will be a member of the Global Security Application Support (GS-CAD) Division working in the Global Security Directorate.
This position supports geophysical monitoring research and development on a global scale for national security and environmental research. With the ubiquity of IoT, monitoring sensor deployments are increasing exponentially and recording massively more signatures. Our current data ingestion pipelines and analytics platforms are undergoing rapid changes to facilitate research and applied problems impacting academic, commercial, and government customers. This position will directly contribute toward the evolution of our existing infrastructure to a next generation system that leverages Big Data technologies.
This position will be filled at either the SES.3 or SES.4 level depending on your qualifications. Additional job responsibilities (outlined below) will be assigned if you are selected at the higher level.
Essential Duties
- Provide solutions for the design and implementation of multiple Linux-based HPC, Hadoop/ Big Data Infrastructure and Parallel file system servers and clusters and investigate, evaluate, test and recommend technical solutions for future systems.
- Deploy and maintain Hadoop/Big Data and database storage Infrastructures on prem and cloud (e.g. AWS).
- Monitor installation of Hadoop/Spark/Nifi and related software releases, patches of the operating system, third-party utilities with emphasis on overall system performance.
- Monitor and implement Splunk / Nagios and other performance/monitoring technologies.
- Troubleshoot and determine root cause of complex data provenance, metadata issues and user questions that may involve interfacing with various technical staff in multiple organizations and with differing levels of expertise.
- Investigate, evaluate, test and recommend technical solutions for future systems.
- Develop tools and procedures to monitor and automate system tasks on servers and clusters.
- Perform other duties as assigned.
In Addition at the SES.4 Level
- Provide solutions that require in-depth analysis of multiple factors and the creative use of established methods.
- Solve abstract and highly complex data issues and user questions.
Qualifications
- Bachelor’s degree in computer science, computer engineering, or a related field, or the equivalent combination of education and related experience.
- Significant experience with Linux/Unix systems including installation, configuration, networking, backups, updates and patching, and system security.
- Significant experience with andr knowledge of Big Data technologies such Hadoop,, Spark, Nifi, Storm, Spark, HDFS, NFS, Lustre.
- Significant experience with software container technologies such as Docker and Singularity.
- Advanced knowledge of scripting and programming languages such as C/C++, Java, Perl, Python, Expect and bash/csh/ksh.
- Experience with AWS/Cloud computing design, provisioning, and tuning.
- Significant experience supporting multiple independent but inter-related systems and software packages and demonstrated advanced ability to provide innovative solutions to broadly defined tasks and problems and to interact with system developers and vendors.
- Advanced verbal and written communication skills necessary to effectively collaborate in a team environment and present and explain technical information and provide advice to management.
In Addition at the SES.4 Level:
- Effective expert analytical, problem-solving, and decision-making skills to develop creative solutions to complex problems.
- Substantial experience in one or more of the advanced areas: local, parallel and distributed file systems, NAS platforms, or container orchestration framework, SQL/NoSQL database systems and IaaS technologies.
- Expert communication, facilitation, and collaboration skills necessary to present, explain, and advise senior management and/or external sponsors.
Lawrence Livermore National Laboratory (LLNL), located in the San Francisco Bay Area (East Bay), is a premier applied science laboratory that is part of the National Nuclear Security Administration (NNSA) within the Department of Energy (DOE). LLNL's mission is strengthening national security by developing and applying cutting-edge science, technology, and engineering that respond with vision, quality, integrity, and technical excellence to scientific issues of national importance. The Laboratory has a current annual budget of about $1.5 billion, employing approximately 6,000 employees.
LLNL is an affirmative action/ equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, marital status, national origin, ancestry, sex, sexual orientation, gender identity, disability, medical condition, protected veteran status, age, citizenship, or any other characteristic protected by law.
