6 verified briefings on Data Engineering. Each story includes a plain-English summary, why it matters, and the concrete action engineering teams should take.
Java Champion Gunnar Morling shares insights on building high-performance Java applications for data engineering. He discusses experiments with durable execution engines and the development of Apache Hardwood, a new, minimal-dependency Java parser for the Apache Parquet file format, offering lessons for developers and engineering leaders.
ClickHouse announced several major updates at its Open House 2026 event. Key developments include deeper integration with Postgres, new data ingestion tools called ClickPipes and ClickHouse Agents, and a partnership with Langfuse for LLM observability. The updates aim to simplify real-time data analytics.
DuckDB, the popular in-process analytical database, has introduced a new remote protocol called Quack. It uses HTTP to enable a client-server model, allowing multiple users and applications to connect to and query the same database instance over a network, a significant shift from its embedded-only origins.
Meta's engineering team successfully migrated its petabyte-scale data ingestion platform, which processes social graph data from MySQL. The team used advanced techniques like reverse shadowing and continuous checksum monitoring to ensure a seamless transition with zero downtime, improving both reliability and operational efficiency.
dbt Labs has launched dbt Agent Skills, a new feature in dbt Cloud. It allows developers to package data logic into reusable "skills" for AI agents. This helps agents answer data-related questions more reliably and accurately by using pre-defined logic instead of generating SQL from scratch.
NASDAQ handles up to a trillion messages daily across its 26 business lines. To manage this massive scale, the company built a governed intelligence layer using dbt and Databricks. This modern data stack enables them to ensure data quality, security, and accessibility for decision-making.