Senior Software Engineer – Data Ingest Engineering(Kafka)

Hyderabad Full-time Technology & Engineering Senior

What You’ll Do

Design and implement high-throughput, event-driven architectures to power near real-time data pipelines for ingesting into Azure + Databricks Lakehouse
Build cloud-native data pipelines to support batch ingest workloads
Configure Unity Catalog by designing Bronze/Silver/Gold medallion architecture and optimize for centralized data governance, fine-grained access control, and end-to-end lineage tracking across Databricks workspaces
Drive engineering excellence by setting standardized DevOps practices and optimizing data engineering code for performance
Identify and design solutions to potential data processes that can be improved with automation
Partner with domain teams to onboard new capability use cases, mentor engineers, and contribute to reusable blueprints and reference implementations.

What You’ll Bring

7+ years of software engineering experience focused on data infrastructure and backend systems.
Hands-on experience with the Microsoft Azure data platform — Azure Data Factory, Event Hubs, Blob Storage, ADLS Gen2, and Azure DevOps for orchestration and CI/CD
Production experience with Azure Databricks and Delta Lake
Production experience with PySpark for distributed data processing and transformation.
Streaming frameworks (Kafka or comparable) for real-time data paths including CDC and incremental batch patterns.
Strong communication skills with the ability to influence senior stakeholders and lead technical decisions across global teams.

Must Have Skills

Data Platform Architecture (Lakehouse, Delta Lake, Azure Databricks)
Kafka / Streaming Frameworks
Strong SQL and Python Proficiency
Experience with Cloud Native Tools

Nice-to-Have

Building data infrastructure for AI use cases
Prior experience in large-enterprise data integration or platform engineering environments
Prior experience in enterprise platform migrations

More open mandates

Related mandates

Full-time

Apply for this mandate

It takes about four minutes. You can complete your application without leaving this page.

Have your CV ready. It goes directly to the consultant leading this search.

Your application is handled with discretion.

First name*

Last name*

Email*

Phone*

Current organisation*

Current roleoptional

Relevant experience*

Current location*

Current compensation*

Your full annual package (your CTC, in India), for example 1800000 or 18 LPA. If you are between roles, give your last one. Picked Other? Write the currency in with the amount.

Expected compensation*

On the same basis, for a move. A cross-border move often changes the currency, so set each one.

Notice period*

Education*

LinkedIn*

CV*

Attach your CV PDF, Word, RTF or text · up to 5 MB

What makes you a good fit for this role?*

I agree that Recruise may hold and process the details on this form, including my compensation and notice period, to consider me for this and other suitable mandates, under the DPDP Act, 2023. Compensation is used to weigh me against the brief, and reaches a hiring client only when Recruise presents me to them, with my knowledge. My application is not added to any marketing list. See the Candidate Data Policy and Privacy Policy.

Application received

Thank you for applying for this role.

As we receive many applications, we’re not able to reply to each one personally. If you’re shortlisted, the consultant running this search will be in touch. We do not share your details with any client before we speak to you.

See other open mandates Career guidance

Not ready to apply? Submit your profile for future and confidential mandates.

Senior Software Engineer – Data Ingest Engineering(Kafka)

What You’ll Do

What You’ll Bring

Must Have Skills

Nice-to-Have

Related mandates

AI/ML Architect

AI/ML Backend Developer

Support Trainee- SQL

Apply for this mandate

Thank you for applying for this role.

Let's build what's next.