Responsible for all aspects of data acquisition, data transformation, analytics scheduling and operationalization to drive high-visibility, cross-division outcomes. Investigate, evaluate, test and recommend technical solutions for future systems. Support software developers, database architects, data scientists on data initiatives and will ensure optimal data delivery architecture.
Data Design Management
- Own product data set features from the development phase through to production deployment.
- Support building distributed, scalable, and reliable data pipelines that ingest and process data at scale and in real-time.
- Deploy and maintain Hadoop/Big Data/Spark and database storage Infrastructures in AWS cloud.
- Support managing data environments and/or data sets to serve a wide range of data users, including but not limited to Data Scientists, Data Analysts, Business Analysts etc.
- Perform offline analysis of large data sets using components of a big data software ecosystem.
- Evaluate big data technologies and prototype solutions to improve data processing architecture.
- Monitor installation of HDFS/Hadoop/Spark and related software releases, third-party utilities with emphasis on overall system performance.
- Support the activity of troubleshoot and determine root cause of complex data provenance, metadata issues and engineering questions that may involve interfacing with various technical staff in multiple organizations and with differing levels of expertise.
- Develop tools and procedures to monitor and automate system tasks on servers and clusters
- Author extract, transform and load (ETL) scripts for moving and curating data into data sets for storage and use by a datalake, data warehouse and datamart.
- Collaborate with other teams to deploy data tools and data sets that support both operations and product use cases.
- Evaluate and advise on technical aspects of open work requests in the product backlog for other projects as assigned.
- Knowledge of database concepts, object and data modeling techniques and design principles.
- Detailed knowledge of database architectures, software, and facilities.
- Successful history of manipulating, processing, and extracting value from large disconnected data sets.
- Experience with programming languages – Python (required), Scala, Ruby, R.
- Database technologies - SQL, performance tuning concepts, AWS RDS, RedShift, MySQL.
- Experience with big data batch processing tools: Hadoop MapReduce, ElasticSearch, PIG, Hive, Cascading/Scalding, Apache Spark, AWS EMR.
- Experience with stream-processing systems: Kinesis, Kafika, MQTT.
- Experience with relational NoSQL databases including DyanamoDB.
- Ability to write JSON, XML, YAML and other data definition schemas.
- Good verbal and written communication skills necessary to effectively collaborate in a team environment and present and explain technical information and provide advice to management.
- A seasoned, experienced professional with a full understanding of area of specialization; resolves a wide range of issues in creative ways.
- Work is independent and collaborative in nature.
- Contributes to moderately complex aspects of a project. Provides regular updates to manager on project status.
- Bachelor’s degree in computer science, computer engineering, or a related field, or the equivalent combination of education and related experience.
- 5 years of professional experience as a data software engineer.
1 years of experience with AWS/Cloud Big Data computing design, provisioning, and tuning.
- Previous experience as a Data Engineer / Database Administrator and/or Business Intelligence Analyst.
Panasonic is proud to be an Equal Opportunity/Affirmative Action employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, national origin, disability status, protected veteran status, and any other characteristic protected by law or company policy. All qualified individuals are required to perform the essential functions of the job with or without reasonable accommodation. Pre-employment drug testing is required for safety sensitive positions or as may otherwise be required by contract or law. Due to the high volume of responses, we will only be able to respond to candidates of interest. All candidates must have valid authorization to work in the U.S. Thank you for your interest in Panasonic Corporation of North America.