No lab wednesday 26th

Still no lab this wednesday, Mar 26th. Theory only.

Slides for Lectures 2 and 3 are available. Of course, they may change at any time without warning. The references will be in the reference page soon.

Note: In Lecture 1 I have corrected a couple of typos in the statements and applications of Chernoff-Hoeffding.

Lecture 1. The Data Stream model. Counting. Probability tools

Today we went through some course logistics and covered the slides for Lecture 1.

There are two exercises in the slides. To get credit, they should be given to me (by any means, in any format) within two weeks, i.e. by the start of the April 2nd class.

All exercises and lab assignments are individual. If you want to solve them collaborating with someone, ask me first.

Welcome to the Spring 2014 (and first) edition of the MIRI Seminar on Data Streams

Logistics

• Instructor: Ricard Gavaldà. Mail: gavalda AT lsi then the UPC domain. 
• Start date: Wednesday March 19th, 2014
• End date: Roughly end of May, 2014
• 3 ECTS credits, to be enrolled and claimed from MIRI at the end of the course. 
• Time and places: 
• Theory and problems: every Wednesday, 10:00 - 12:00, room A4104
• Labs: approx. every two Wednesdays, 14:00 - 16:00, room C6S303
• No lab yet on Wednesday March 19th
• There will be some gaps due to travels or other commitments. Will be announced on time. 
• I will post all the materials here, including pretty detailed descriptions of the lab assignments. It should be possible, though harder, to follow the seminar on your own. 
• Intended audience & requisites: 
• Students enrolled in MIRI, particularly in the Advanced Computing and Data Mining & Business Intelligence specialities. 
• But everybody is welcome. 
• Some familiarity with probabilistic reasoning and algorithmics is assumed. Some familiarity with machine learning / data mining is helpful, but probably not essential. Some programming may be necessary. I will try to give as much freedom in the choice of programming language but MOA is mostly Java.
• Evaluation: based on a few exercise sets and a few deliverable lab assignments. Promised to be not too heavy. I am open to discussion for alternative evaluation methods.
• Website: http://www.lsi.upc.edu/~gavalda/DataStreamSeminar

Overview

Streaming is one of the central aspects of the ``Big Data'' slogan. At a planet scale, we are today generating more data than we can store, and most of it will never be seen by human eyes. It often takes the form of sequences or streams of data items, arriving at high speed, potentially infinite, and evolving over time. The data stream paradigm contrasts with the usual input-compute-output algorithmic paradigm in this sequential nature, and also in the strong computational requirements it imposes: one pass over the data, small memory, small computation time per data item, ability to give answers in real-time and at anytime.

In this seminar we will:

• Describe some of the scenarios where this paradigm is necessary (sensor networks, smart cities, social media, network monitoring, ...).
• Study and experiment with algorithms for computing basic and not so basic queries over data streams.
• Study and experiment with algorithms for mining knowledge from data streams (predictive models, clustering, pattern mining), as an extension of traditional data mining and machine learning.

The seminar will consist of 1) theory/problem sessions, where the instructor will
present the main ideas and where we will go over the solutions of problem sets
distributed in the previous sessions and 2) lab sessions, where we will experiment
or implement some of the methods described; for Part II (data stream mining) the
MOA stream mining framework will be used.

Very preliminary syllabus

Part I: Algorithms for Data Streams

1. The Data Stream Model: Scenarios and definition
2. Computing statistics on streams
3. Matrix and graph sketches
4. Detecting change on streams

 Part II: Data Stream Mining

1. Predictive models
2. Clustering
3. Mining frequent patterns
4. Evaluation

Course material

See the pages with Slides, Bibliography and Links, and Lab material to the right.