Application Development Lead - BigData
- Ameriprise
- 7 - 10 years
- Noida, Gurugram
- 2 months ago
- Email to a friend
- Report this job
Job Description
- We are looking for an experienced data engineer to join our team
- You will analyze and organize raw data, build data systems and pipelines
- You would evaluate business needs objectives, conduct complex data analysis and explore ways to enhance data quality and reliability
- Work on data acquisition and collaborate with architects on several projects
- If you are detail-oriented, with excellent organizational skills and experience in this field, we d like to hear from you
Key Responsibilities
- Provides thought leadership with respect to RD of the latest and greatest Big data toolsets available in the market, perform comparative analysis and provides recommendations to choose the best tool as per the current and future needs of the enterprise.
- Act as a strong proponent for promoting cloud technology culture within the organization. Provides thought leadership with respect to the RD of the cloud toolsets available in the market.
- Liaison as an intermediary between Data scientists/Platform and cloud engineering teams to understand the requirements, provide technical guidance/mentorship and consultation towards resolution of their queries.
- Engineering responsibilities of the cloud ecosystem including data enablement and data acquisition support.
- Provide thought leadership in development of automated pipeline, dataops and curation functions for the lake ecosystem.
- Leading the technical discussions across infrastructure teams on tool and infra performance parameters such as CPU, disk IO, memory usage, disk space etc in order to provide an optimal performance experience to the end users.
- Encourage a culture of innovation and research within the team by providing strong thought leadership and promoting out-of-box thinking.
- Diligently teaming with the cloud engineering, delivery, infrastructure (storage and compute), network, security teams to deliver an optimal tool experience to the end users.
- Working closely with various stakeholders in US team with a collaborative approach.
Required Qualifications
- 7 to10 years of overall IT experience, which includes hands on experience in Big Data technologies.
- Mandatory - Hands on experience in Python and Pyspark. Python as a language is practically usable for anything, we are looking for application Development and Extract/Transform/Load and Data lake curation experience using Python.
- Build Pyspark applications using Spark Data frames in Python using Jupyter notebook and PyCharm(IDE).
- Worked on optimizing spark jobs that processes huge volumes of data.
- Hands on experience in version control tools like Git.
- Worked on Amazon s Analytics services like Amazon EMR, Amazon Athena, AWS Glue.
- Worked on Amazon s Compute services like Amazon Lambda, Amazon EC2 and Amazon s Storage service like S3 and few other services like SNS.
- Experience/knowledge of bash/shell scripting will be a plus.
- Has built ETL processes to take data, copy it, structurally transform it etc involving a wide variety of formats like CSV, TSV, XML and JSON.
- Experience in working with fixed width, delimited , multi record file formats etc
- Good to have knowledge of data warehouse concepts - dimensions, facts, schemas- snowflake, star etc
- Have worked with columnar storage formats- Parquet, Avro, ORC etc Well versed with compression techniques - Snappy, Gzip.
- Good to have knowledge of AWS databases (at least one) Aurora, RDS, Redshift, ElastiCache, DynamoDB.
Preferred Qualifications
- Certifications Preferred: aws Solution Architect
- AWS Certified Cloud Practitioner Certification
Job Classification
Industry: Financial ServicesFunctional Area: Financial Services
Role Category: Software Development
Role: Software Development - Other
Employement Type: Full time
Contact Details:
Company: Ameriprise FinancialLocation(s): Noida, Gurugram
+ View Contact
Keyskills: Data analysis XML Shell scripting Network security Application development JSON Data quality Asset management Analytics Python
Fraud Alert to job seekers!
₹ Not Disclosed
Similar positions
Ameriprise
Ameriprise Financial has helped millions of clients feel confident about their financial futures for more than 125 years. Our network of approximately 10,000 financial advisorsdelivers personalized financial advice to help clients reach their goals..
Job Listings
We are in open beta version of website, please let us know if any issues, at: info(at)indigojobs.in