» Abstract

Current languages and libraries provide limited support for asynchronous and event-based programming. This forces developers to use explicit continuation passing style by breaking their code into many disjoint event-handlers. The lack of a simple programming model for asynchronous programming is quickly getting problematic for developers because of the inevitable advent of many-core computers, distributed and Cloud computing. Remember that the "A" in AJAX stands for "Asynchronous".

This talks shows that the familiar Subject/Observer design pattern is the mathematical dual of the even more familiar Iterator design pattern. From this abstract duality, we derive a concrete implementation of the LINQ standard sequence operators for "push" based, observable collections based on a pair of IObservable and IObserver interfaces that are the mirror images of the familiar "pull"-based, enumerable collections defined using IEnumerable and IEnumerator pair of interfaces. As a consequence we can elegantly formulate many concurrent asynchronous and event-driven programs using standard LINQ query comprehensions.

» Abstract

The ability to analyze massive-scale datasets has become a critical requirement for industry and sciences alike. Because of the magnitude of the data involved in today's applications, users are increasingly turning toward parallel data processing systems running in shared-nothing clusters. These systems provide efficient query
processing facilities, but the magnitude of input data sets still causes most queries to take from minutes to several hours to complete. At this scale, users need more than efficient processing. They also need effective tools for managing their queries at runtime.

In this talk, we will present our ongoing work developing Parallax, the first non-trivial, time-oriented progress indicator for parallel queries. We developed the current version of Parallax for Pig queries running in a Hadoop cluster, an environment that is a popular open-source parallel data-processing engine under active development. As an initial step, we focused on Pig Latin queries that compile into a series of MapReduce jobs. In this talk, we will present the techniques behind Parallax and our preliminary experimental results.

This work is part of the Nuage project at the University of Washington: http://nuage.cs.washington.edu. We will touch upon the larger vision behind the Nuage project and the other problems that we are currently tackling within this project.

» Abstract

Analysts working with large data sets often use statistical models to "guess" at unknown, inaccurate, or missing information associated with
the data stored in a database. For example, an analyst for a manufacturer may wish to know, "What would my profits have been if I'd increased my margins by 5% last year?" The answer to this question naturally depends upon the extent to which the higher prices would have affected each customer's demand, which is undoubtedly guessed via
the application of some statistical model.

In this talk, I'll describe MCDB, which is a prototype database system that is designed for just such a scenario. MCDB allows an analyst to
attach arbitrary stochastic models to the database data in order to "guess" the values for unknown or inaccurate data, such as each customer's unseen demand function. These stochastic models are used to produce multiple possible database instances in Monte Carlo fashion (a.k.a. "possible worlds"), and the underlying database query is run over each instance. In this way, fine-grained stochastic models become first-class citizens within the database. This is in contrast to the "classical" paradigm, where high-level summary data are first
extracted from the database, then taken as input into a separate statistical model which is then used for subsequent analysis.

» Abstract

Bigtable, Dynamo, Dynomite, Cassandra, HBase, Voldemort, HyperTable, MongoDB, Neptune, ... the list goes on. All of a sudden, it seems like everyone (including Google, Amazon, Facebook, PowerSet, Linkedin, ...) is building distributed key-value stores. Why all the fuss? In
this open-ended talk, we'll discuss the APIs, data model, and use cases (including schema design); then we'll foray into how these systems do and don't handle consistency, indexing, replication, and availability. Finally, we'll talk about how this all relates to frameworks like Apache Hadoop.

» Abstract

Social and communication networks are formed by entities (such as individuals or computer hosts) and their connections (which may be contacts, relationships, or flows of information). Such networks are analyzed to understand the influence of individuals in organizations, the transmission of disease in communities, the operation of computer networks, among many other topics. While network data can now be recorded at unprecedented scale, releasing it can result in unacceptable disclosures about participants and their relationships. As a result, privacy concerns are severely constraining the dissemination of network data and disrupting the emerging field of network science.

Our recent work investigates the properties of a network that can be accurately studied without threatening the privacy of individuals and their connections. We adopt the rigorous condition of differential privacy, and develop algorithms for releasing randomly perturbed statistics about the topology of a sensitive network. This talk will focus on two basic analysis tasks: the estimation of the degree distribution of a network and the study of small structural patterns that occur in a network (sometimes called motif analysis). We show that the degree distribution of a network can be very accurately estimated by a novel technique in which constraints are applied to the noisy output to improve utility. This technique is of general interest, and can be used to boost the accuracy of differentially private output in other tasks as well. We show that studying motifs is fundamentally harder, but can be done with acceptable accuracy if the privacy condition is relaxed.

» Abstract

Streaming continuous analytics systems have emerged as key solutions for dealing with massive data volumes and demands for low latency. These systems have been heavily influenced by an assumption that data streams can be viewed as sequences of ordered data. The reality, however, is that streams are not continuous and disruptions of various sorts in the form of either big chunks of late arriving data or arbitrary failures are endemic. We argue, therefore, that stream processing needs a fundamental rethink and advocate a unified approach providing Analytics over Continuous and DisContinuous (ACDC) streams of data. Our approach is based on a simple insight – partially process independent runs of data and defer the consolidation of the associated partial results to when the results are actually used on an on demand basis. Not only does our approach provide the first real solution to the problem of data that arrives arbitrarily late, it also lets us solve a host of hard problems such as parallelism, recovery, transactional consistency and high availability that have been neglected by streaming systems. In this talk we describe the Truviso ACDC approach and outline some of the key technical arguments and insights behind it.

» Abstract

Data stream processing has found application in many areas including environmental monitoring, object tracking and monitoring, and business analytics. While the foundation for data stream processing has been developed in prior work, recent real-world deployments are raising a host of new challenges.

The first challenge that we address regards uncertain data streams, where data can be incomplete, imprecise, and even misleading. Feeding such data streams to existing stream systems produces results of unknown quality. In the main part of the talk, I present the design of a data stream system that captures data uncertainty from data collection to query processing to final result generation, with a focus on its data model and processing algorithms for complex relational operators. Other challenges to data stream systems include the need to extend the data model from set-based to sequence-based and the need to archive and index data streams to answer continuous queries. In the rest of the talk, I survey two other projects that address these challenges.

» Abstract

In this talk I will discuss the practical challenge of data anaysis at the terrabyte and even petabyte scale. Using examples from MySpace and Fox Audience Network, I will share methods which yield insight with minimal management overhead.

» Abstract

In this talk, I present our recent work on using declarative languages to specify, implement, and analyze secure distributed systems. In the first half of the talk, I first describe Secure Network Datalog (SeNDlog), a declarative language that unifies declarative networking and logic-based access control languages. SeNDlog enables network routing, distributed systems, and their security policies to be specified and implemented within a common declarative framework. I will focus on a specific use-case, based on an extensible platform for Application-Aware Anonymity (A3) that we have developed, and describe ongoing collaborative efforts at integrating our work with LogicBlox, an emerging commercial Datalog-based platform for enterprise software systems.

In the second half of the talk, I introduce the notion of network provenance naturally captured within our declarative framework, and demonstrate its applicability in the areas of network debugging, analysis and trust management. I further discuss ongoing work at optimizing distributed query processors in order to process and maintain network provenance efficiently and securely. Details of this project and our research group are available at http://netdb.cis.upenn.edu/.