Data Engineer

Wilen is driven by industry-leading agency services and cutting-edge direct mail production. We provide strategic direct marketing solutions for national brands, leveraging innovative variable data technology, to turn concepts into finished digital and physical products, all in one seamless process.

We are seeking a skilled Data Engineer to help modernize and operate a hybrid data architecture for our direct mail production and printing business. You will play a critical role in building real-time, event-driven data pipelines that power our microservices, reporting systems, client portals, and production workflows — bridging our on-premises infrastructure with scalable cloud platforms.

This is a hands-on engineering role with a focus on reliability, performance, and scalability. You'll work closely with architects, software engineers, and DevOps teams to bring our hybrid, event-driven vision to life.

RESPONSIBILITIES:

• Design and build scalable data pipelines that move, transform, and enrich data between on-premises systems and cloud services (Azure Data Factory, Databricks, Delta Lake, Azure SQL, etc.).
• Develop event-driven data workflows, consuming and publishing domain events (e.g., Kafka topics, Azure Service Bus) to coordinate microservices and ensure data consistency.
• Create hybrid ingestion systems to handle client-provided data in various forms (files, APIs, streams), validate and cleanse incoming data, and integrate into operational databases or data lakes.
• Optimize data storage and retrieval for both operational needs (low-latency production systems) and analytical needs (cloud-based data warehouse, reporting).
• Ensure high data quality and governance, including master data management (MDM) practices and schema enforcement across services.
• Implement real-time and batch data processing workflows to support direct mail production timelines (e.g., daily mail jobs, regular scheduled mailings).
• Collaborate with microservice developers to define event schemas, database designs, and API contracts that support reliable event-driven systems.
• Monitor, troubleshoot, and optimize data pipelines, ensuring they perform reliably across hybrid environments.
• Contribute to DevOps pipelines, ensuring data infrastructure (databases, pipelines, cloud resources) is deployed and managed through Infrastructure-as-Code.
• Document processes, data flows, and best practices to support maintainability and scaling across teams.

QUALIFICATIONS

Technical Skills:

• Strong experience with data engineering on both on-premises and cloud platforms (preferably Azure: Azure Data Factory, Azure SQL, Azure Databricks, Azure Event Hubs or Service Bus).
• Proficient in building ETL/ELT pipelines, streaming ingestion, and real-time data processing.
• Solid understanding of event-driven architectures and message broker systems (e.g., Kafka, Azure Service Bus, RabbitMQ).
• Hands-on experience with SQL (complex queries, optimization, partitioning) and NoSQL systems (e.g., Cosmos DB, MongoDB).
• Familiarity with data lake concepts (Delta Lake, ADLS) and warehouse modeling (star schema, snowflake schema).
• Programming experience in Python, Scala, or similar for data engineering tasks.
• Experience working with containerized environments (Docker, Kubernetes) is a strong plus.
• Exposure to DevOps practices (CI/CD pipelines, Infrastructure as Code, Terraform or Bicep preferred).

Nice To have:

• Familiarity with Power BI, Microsoft Fabric, or other BI/analytics platforms.
• Understanding security and compliance (e.g., encryption, GDPR) in hybrid data flows.
• Experience with distributed tracing and monitoring of data pipelines and microservices.

Soft Skills:

• Strong problem-solving skills and attention to detail.
• Ability to work cross-functionally with architects, software engineers, and business stakeholders.
• Passion for building reliable, scalable, and observable data systems.
• Ability to adapt in a fast-paced, evolving hybrid cloud environment.
• Excellent communication skills — written, verbal, and documentation.

Preferred Qualifications:

• Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or related field.
• 5+ years of experience in data engineering or hybrid cloud data management role.

What We Offer:

• Opportunity to be part of a strategic modernization of a 50-year-old company into a cloud-ready, event-driven operation.
• Ability to work on critical systems that directly impact production, clients, and operations.
• Competitive salary and benefits package.
• Professional growth in cloud technologies, event-driven systems, and next-generation data platforms.

We’d love to hear your story and learn how your talents and experiences will enhance our team. You can find out more about us at wilengroup.com. We are an equal opportunity employer and a drug-free workplace. E-Verify.

Wilen offers a variety of benefits to eligible employees, including health insurance coverage (Medical, Dental, Vision), paid holiday, paid time off (PTO), 401(k), and many others.