Role description
As our first Senior Data Engineer, you will own the end-to-end data lifecycle of our industrial IoT platform. We collect massive amounts of data across commercial and industrial installations; your mission is to transform this raw "noise" into scalable, actionable intelligence. You will act as the bridge between our raw telemetry and our customer-facing insights, working closely with the Staff Engineer to scale our data infrastructure and with our UI Engineers to power custom BI dashboards for our largest clients.
Technical Requirements
1. Data Architecture & Scalability
- 6-8+ Years of Data Engineering: Proven experience building and scaling data pipelines in a high-volume environment (preferably IoT or Time-Series data).
- Big Data Mastery: Experience moving beyond traditional relational databases to handle "Big Data" scales. You should know when to use partitioning, sharding, or specialized data warehouses (e.g., Snowflake, BigQuery, or AWS Redshift/Athena).
- Advanced SQL: Expert-level SQL skills. You can write complex, highly optimized queries that aggregate millions of rows into lightning-fast results for our BI layers.
2. Infrastructure & Languages
- The AWS Ecosystem: Deep familiarity with AWS data services (RDS, S3, Glue, Athena, Kinesis, or Lambda).
- Python or Go: Proficiency in Python (the industry standard for data pipelines) or Go (our core backend language) to build and maintain data ingestion and transformation tools.
- Database Management: Strong experience with PostgreSQL performance tuning and managing large-scale time-series datasets.
3. Analytics Engineering
- The "Semantic Layer": Experience building the views and materialized tables that power frontend dashboards. You understand how to "clean" data so that it is easily consumable by UI Engineers.
- BI Strategy: Familiarity with modern BI tools and how to architect data to support custom, client-specific visualizations and reporting.
Key Responsibilities
1. Insight Extraction & BI Support
- Custom Insights: Tailor complex queries and data models for our largest clients, providing them with the unique industrial insights they need to run their businesses.
- UI Partnership: Build the queryable endpoints and data structures that our UI Engineers will use to implement high-performance charts and dashboards in our apps.
2. Data Scalability & Integrity
- Scale the Dataset: Take our existing "massive" dataset and lead the effort to make it more scalable and cost-effective as our fleet of devices grows.
- The Feedback Loop: Analyze our current data collection and proactively inform the Backend/Staff Engineers of "blind spots"—identifying exactly what data we aren't collecting that we should be.
3. Pipeline Development
- Ingestion & Transformation: Design and maintain the pipelines that move data from our field devices and cloud APIs into our analytical storage layers.
- Data Health: Implement monitoring and alerting to ensure data quality and consistency across our entire ecosystem.
Nice-to-Haves (Training Provided)
- Industrial Protocols: Knowledge of how data is formatted in LoRaWAN, BACnet, or Modbus environments.
- AI & RAG: Helping the team prepare our dataset to be used as a knowledge base for our upcoming RAG-based AI agent.
- Time-Series DBs: Experience with specialized databases like TimescaleDB or InfluxDB.
The Profile
You are a data enthusiast who isn't satisfied with just "storing" data—you want to find the signal in the noise. You are a proactive communicator who can tell an engineer, "We're missing a timestamp on this event," and tell a client, "Here is exactly why your energy usage spiked last Tuesday." You are looking for a role with high ownership where you can build the data foundation for a rapidly scaling company.
