Collaborating to Handle “Eclectic Data” – University Researchers and Start-up Engineers Develop Novel Solutions

CSE’s Assistant Professor Amy Ousterhout, one of four new faculty members to join UC San Diego’s Center for Networked Systems, presented research at the 2023 Conference on Innovative Data Systems Research (CIDR) held in Amsterdam earlier this year. The paper, Zed: Leveraging Data Types to Process Eclectic Data, was a collaboration between researchers at UC Berkeley, where Ousterhout did her postdoc research, and Brim Data, a start-up founded by Steve McCanne.

The CIDR paper is an early workshop paper on the work Brim Data initiated to analyze logs of network traffic and respond to threats to network security. As the scope and complexity of this goal widened, McCanne reached out to Ousterhout for help tackling some of the more perplexing research questions. Their collaborative efforts resulted in Zed – a novel approach to processing eclectic data.

“Brim Data found that managing all of this heterogeneous data was challenging in itself. Users want to query network logs that contain different types of network traffic, the structure of the logs can change over time, and these tasks were challenging with existing ways of representing and querying data,” said Ousterhout, who believes Zed offers solutions in a variety of data-processing contexts, not only in network security.