All array projections share two operations in common:
- Traverse the source array
- Add each item's projected value to a new array
Why not abstract away how these operations are carried out?
Functional programming provides developers with the tools to abstract common collection operations into reusable, composable building blocks. You'll be surprised to learn that most of the operations you perform on collections can be accomplished with five simple functions:
Here's my promise to you: if you learn these 5 functions your code will become shorter, more self-descriptive, and more durable. Also, for reasons that might not be obvious right now, you'll learn that these five functions hold the key to simplifying asynchronous programming. Once you've finished this tutorial you'll also have all the tools you need to easily avoid race conditions, propagate and handle asynchronous errors, and sequence events and AJAX requests. In short, these 5 functions will probably be the most powerful, flexible, and useful functions you'll ever learn.
This isn't just a tutorial, it's a series of interactive exercises that you can fill out right in your browser! It's easy to finish the exercises. Just edit the code and press "Run." If the code works, a new exercise will appear below. Otherwise an error will appear.
This tutorial may have bugs, so if you get into a weird state or you're sure you've got the right answer but can't move on, just refresh your browser. If you're using a modern browser, and if you're here I assume you are, the state of your exercises will be saved. If you want, you can also restart the lab.
This tutorial is on GitHub, and is asymptotically approaching completion. If you'd like to add an exercise, clarify a problem description, or fix a bug feel free to fork and send us a pull request. We'll try and fit user-contributed exercises into a narrative.
The Array is Javascript's only collection type. Arrays are everywhere. We're going to add the five functions to the Array type, and in the process make it much more powerful and useful. As a matter of fact, Array already has the map, filter, and reduce functions! However we're going to to reimplement these functions as a learning exercise.
This section will follow a pattern. First we'll solve problems the way you probably learned in school, or on your own by reading other peoples code. In other words, we'll transform collections into new collections using loops and statements. Then we'll implement one of the five functions, and then use it to solve the same problem again without the loop. Once we've learned the five functions, you'll learn how to combine them to solve complex problems with very little code.
// Traverse array with for loop
function(str) {
var fun = eval("(" + str + ")");
var items =[];
fun({
log:function(name) {
items.push(name);
console.log(name);
}
});
if(JSON.stringify(items.sort()) === '["Ben","Brian","Jafar","Matt","Priya"]') {
return "Success!"
}
else {
throw 'console.log did not receive all of these values: "Ben","Brian","Jafar","Matt","Priya" (note: order does not matter)'
}
}
function(console) {
var names = ["Ben", "Jafar", "Matt", "Priya", "Brian"],
counter;
for(var counter = 0; counter < names.length; counter++) {
console.log(names[counter]);
}
}
Ask yourself this question: did we need to specify the order in which the names were printed? If not, why do it?
Let's repeat the previous exercise using the forEach function.
// Traverse array with foreach
function(str) {
if (str.indexOf(".forEach") === -1) {
return "You have to use forEach!"
}
var fun = eval("(" + str + ")");
var items =[];
fun({
log:function(name) {
items.push(name);
console.log(name);
}
});
if(JSON.stringify(items.sort()) === '["Ben","Brian","Jafar","Matt","Priya"]') {
return "Success!"
}
else {
throw 'console.log did not receive all of these values: "Ben","Brian","Jafar","Matt","Priya" (note: order does not matter)'
}
}
function(console) {
var names = ["Ben", "Jafar", "Matt", "Priya", "Brian"];
names.forEach(function(name) {
console.log(name);
});
}
Notice that forEach let's us specify what we want to happen to each item in the array, but hides how the array is traversed.
Applying a function to a value and creating a new value is called a projection. To project one array into another, we apply a projection function to each item in the array and collect the results in a new array.
For each video, add a projected {id, title} pair to the videoAndTitlePairs array.
// Projection with with forEach
function(str) {
var fun = eval("(" + str + ")"),
videoAndTitlePairs = fun(),
expected = '[{\"id\":675465,\"title\":\"Fracture\"},{\"id\":65432445,\"title\":\"The Chamber\"},{\"id\":70111470,\"title\":\"Die Hard\"},{\"id\":654356453,\"title\":\"Bad Boys\"}]';
// Sorting by video id
videoAndTitlePairs = videoAndTitlePairs.sortBy(function(video) { return video.id });
if (JSON.stringify(videoAndTitlePairs) === expected) {
return true;
}
else {
throw 'Expected: ' + expected;
}
}
function() {
var newReleases = [
{
"id": 70111470,
"title": "Die Hard",
"boxart": "http://cdn-0.nflximg.com/images/2891/DieHard.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": [4.0],
"bookmark": []
},
{
"id": 654356453,
"title": "Bad Boys",
"boxart": "http://cdn-0.nflximg.com/images/2891/BadBoys.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": [5.0],
"bookmark": [{ id:432534, time:65876586 }]
},
{
"id": 65432445,
"title": "The Chamber",
"boxart": "http://cdn-0.nflximg.com/images/2891/TheChamber.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": [4.0],
"bookmark": []
},
{
"id": 675465,
"title": "Fracture",
"boxart": "http://cdn-0.nflximg.com/images/2891/Fracture.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": [5.0],
"bookmark": [{ id:432534, time:65876586 }]
}
],
videoAndTitlePairs = [];
newReleases.forEach(function(video) {
videoAndTitlePairs.push({id:video.id, title: video.title});
});
return videoAndTitlePairs;
}
All array projections share two operations in common:
Why not abstract away how these operations are carried out?
To make projections easier, let's add a map() function to the Array type. Map accepts the projection function to be applied to each item in the source array, and returns the projected array.
// Implement map()
function(str) {
var fun = eval(str),
arr = [1,2,3],
result;
result = arr.map(function(x) { return x + 1});
if (JSON.stringify(arr) !== "[1,2,3]") {
throw "Whoa! You changed the input array. Map never changes the value of the array passed in. It creates a new array with the results of applying the projection function to every value in the old array."
}
else if(JSON.stringify(result) !== '[2,3,4]') {
throw 'Expected that [1,2,3].map(function(x) { return x + 1}) would equal [2,3,4].'
}
}
Array.prototype.map = function(projectionFunction) {
var results = [];
this.forEach(function(itemInArray) {
results.push(projectionFunction(itemInArray));
});
return results;
};
// JSON.stringify([1,2,3].map(function(x) { return x + 1; })) === '[2,3,4]'
Let's repeat the exercise of collecting {id, title} pairs for each video in the newReleases array, but this time we'll use our map function.
// Projection with map
function(str) {
var fun = eval("(" + str + ")"),
videoAndTitlePairs = fun(),
expected = '[{\"id\":675465,\"title\":\"Fracture\"},{\"id\":65432445,\"title\":\"The Chamber\"},{\"id\":70111470,\"title\":\"Die Hard\"},{\"id\":654356453,\"title\":\"Bad Boys\"}]';
// Sorting by video id
videoAndTitlePairs = videoAndTitlePairs.sortBy(function(video) { return video.id });
if (JSON.stringify(videoAndTitlePairs) === expected) {
return true;
}
else {
throw 'Expected: ' + expected;
}
}
function() {
var newReleases = [
{
"id": 70111470,
"title": "Die Hard",
"boxart": "http://cdn-0.nflximg.com/images/2891/DieHard.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": [4.0],
"bookmark": []
},
{
"id": 654356453,
"title": "Bad Boys",
"boxart": "http://cdn-0.nflximg.com/images/2891/BadBoys.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": [5.0],
"bookmark": [{ id:432534, time:65876586 }]
},
{
"id": 65432445,
"title": "The Chamber",
"boxart": "http://cdn-0.nflximg.com/images/2891/TheChamber.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": [4.0],
"bookmark": []
},
{
"id": 675465,
"title": "Fracture",
"boxart": "http://cdn-0.nflximg.com/images/2891/Fracture.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": [5.0],
"bookmark": [{ id:432534, time:65876586 }]
}
];
return newReleases.map(function(video) { return {id: video.id, title: video.title}; });
}
Notice that map allows us to specify what projection we want to apply to an array, but hides how the operation is carried out.
Like projection, filtering an array is also a very common operation. To filter an array we apply a test to each item in the array and collect the items that pass into a new array.
Use forEach() to loop through the videos in the newReleases array and, if a video has a rating of 5.0, add it to the videos array.
// Filter with forEach
function(str) {
var fun = eval("(" + str + ")"),
videos = fun(),
expected = '[{"id":675465,"title":"Fracture","boxart":"http://cdn-0.nflximg.com/images/2891/Fracture.jpg","uri":"http://api.netflix.com/catalog/titles/movies/70111470","rating":5,"bookmark":[{"id":432534,"time":65876586}]},{"id":654356453,"title":"Bad Boys","boxart":"http://cdn-0.nflximg.com/images/2891/BadBoys.jpg","uri":"http://api.netflix.com/catalog/titles/movies/70111470","rating":5,"bookmark":[{"id":432534,"time":65876586}]}]';
// Sorting by video id
videos = videos.sortBy(function(v) { return v.id; });
if (JSON.stringify(videos) === expected) {
return true;
}
else {
throw 'Expected: ' + expected;
}
}
function() {
var newReleases = [
{
"id": 70111470,
"title": "Die Hard",
"boxart": "http://cdn-0.nflximg.com/images/2891/DieHard.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"bookmark": []
},
{
"id": 654356453,
"title": "Bad Boys",
"boxart": "http://cdn-0.nflximg.com/images/2891/BadBoys.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"bookmark": [{ id:432534, time:65876586 }]
},
{
"id": 65432445,
"title": "The Chamber",
"boxart": "http://cdn-0.nflximg.com/images/2891/TheChamber.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"bookmark": []
},
{
"id": 675465,
"title": "Fracture",
"boxart": "http://cdn-0.nflximg.com/images/2891/Fracture.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"bookmark": [{ id:432534, time:65876586 }]
}
],
videos = [];
newReleases.forEach(function(video) {
if (video.rating === 5.0) {
videos.push(video);
}
});
return videos;
}
Notice that, like map(), every filter() operation shares some operations in common:
Why not abstract away how these operations are carried out?
To make filtering easier, let's add a filter() function to the Array type. The filter() function accepts a predicate. A predicate is a function that accepts an item in the array, and returns a boolean indicating whether the item should be retained in the new array.
// Implement filter()
function(str) {
var fun = eval(str),
arr = [1,2,3],
result;
result = arr.filter(function(x) { return x > 2});
if (JSON.stringify(arr) !== "[1,2,3]") {
throw "Whoa! You changed the input array. Filter never changes the value of the array passed in. It creates a new array that includes only those items in the old array that pass the predicate function."
}
else if(JSON.stringify(result) !== '[3]') {
throw 'Expected that [1,2,3].filter(function(x) { return x > 2}) would equal [3].'
}
}
Array.prototype.filter = function(predicateFunction) {
var results = [];
this.forEach(function(itemInArray) {
if (predicateFunction(itemInArray)) {
results.push(itemInArray);
}
});
return results;
};
// JSON.stringify([1,2,3].filter(function(x) { return x > 2})) === "[3]"
Like map(), filter() let's us express what data we want without requiring us to specify how we want to collect the data.
// Filter with filter()
function(str) {
var fun = eval("(" + str + ")"),
videoids = fun(),
expected = '[675465,654356453]';
// Sorting by video id
videoids = videoids.sortBy(function(v) { return v; });
if (JSON.stringify(videoids) === expected) {
return true;
}
else {
throw 'Expected: ' + expected;
}
}
function() {
var newReleases = [
{
"id": 70111470,
"title": "Die Hard",
"boxart": "http://cdn-0.nflximg.com/images/2891/DieHard.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"bookmark": []
},
{
"id": 654356453,
"title": "Bad Boys",
"boxart": "http://cdn-0.nflximg.com/images/2891/BadBoys.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"bookmark": [{ id:432534, time:65876586 }]
},
{
"id": 65432445,
"title": "The Chamber",
"boxart": "http://cdn-0.nflximg.com/images/2891/TheChamber.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"bookmark": []
},
{
"id": 675465,
"title": "Fracture",
"boxart": "http://cdn-0.nflximg.com/images/2891/Fracture.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"bookmark": [{ id:432534, time:65876586 }]
}
];
// ------------ INSERT CODE HERE! -----------------------------------
// Chain the filter and map functions to select the id of all videos
// with a rating of 5.0.
return newReleases.
filter(function(video) {
return video.rating === 5.0;
}).
map(function(video) {
return video.id;
});
// ------------ INSERT CODE HERE! -----------------------------------
}
Chaining together map() and filter() gives us a lot of expressive power. These high level functions let us express what data we want, but leave the underlying libraries a great deal of flexibility in terms of how our queries are executed.
Sometimes, in addition to flat arrays, we need to query trees. Trees pose a challenge because we need to flatten them into arrays in order to apply filter() and map() operations on them. In this section we'll define a mergeAll() function that we can combine with map() and filter() to query trees.
Let's start by using two nested forEach loops collect the id of every video in the two-dimensional movieLists array.
// Use filter and map to collect video ids with rating of 5.0
function(str) {
var fun = eval("(" + str + ")"),
videos = fun(),
expected = '[675465,65432445,70111470,654356453]';
videos = videos.sortBy(function(v) { return v });
if (JSON.stringify(videos) !== expected) {
throw "Expected " + expected;
}
}
function() {
var movieLists = [
{
name: "New Releases",
videos: [
{
"id": 70111470,
"title": "Die Hard",
"boxart": "http://cdn-0.nflximg.com/images/2891/DieHard.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"bookmark": []
},
{
"id": 654356453,
"title": "Bad Boys",
"boxart": "http://cdn-0.nflximg.com/images/2891/BadBoys.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"bookmark": [{ id:432534, time:65876586 }]
}
]
},
{
name: "Dramas",
videos: [
{
"id": 65432445,
"title": "The Chamber",
"boxart": "http://cdn-0.nflximg.com/images/2891/TheChamber.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"bookmark": []
},
{
"id": 675465,
"title": "Fracture",
"boxart": "http://cdn-0.nflximg.com/images/2891/Fracture.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"bookmark": [{ id:432534, time:65876586 }]
}
]
}
],
allVideoIdsInMovieLists = [];
movieLists.forEach(function(movieList) {
movieList.videos.forEach(function(video) {
allVideoIdsInMovieLists.push(video.id);
});
});
return allVideoIdsInMovieLists;
}
Flattening trees with nested forEach expressions is easy because we can explicitly add items to the array. Unfortunately it's exactly this type of low-level operation that we've been trying to abstract away with functions like map() and filter(). Can we define a function that's abstract enough to express our intent to flatten a tree, without specifying too much information about how to carry out the operation?
Let's add a mergeAll() function to the Array type. The mergeAll() function iterates over each sub-array in the array and collects the results in a new, flat array. Note: the mergeAll() function expects that each item in the array will be another array.
// Flatten movieLists into an array of video ids
function(str) {
var fun = eval(str),
arr = [[1,2,3],[4,5,6],[7,8,9]],
result,
expected = "[1,2,3,4,5,6,7,8,9]";
result = arr.mergeAll();
result = result.sortBy(function(x) { return x; });
if (JSON.stringify(result) !== expected) {
throw 'Expected that [1,2,3].map(function(x) { return x + 1}) would equal [2,3,4].'
}
}
Array.prototype.mergeAll = function() {
var results = [];
this.forEach(function(subArray) {
results.push.apply(results, subArray);
});
return results;
};
// JSON.stringify([ [1,2,3], [4,5,6], [7,8,9] ].mergeAll()) === "[1,2,3,4,5,6,7,8,9]"
// [1,2,3].mergeAll(); // throws an error because this is a one-dimensional array
Merge is a very simple function, so much so that it may not be obvious yet how it can be combined with map() to query a tree. Let's try an example...
Hint: use two nested calls to map() and one call to mergeAll().
function(str) {
var fun = eval("(" + str + ")"),
videos = fun(),
expected = '[675465,65432445,70111470,654356453]';
videos = videos.sortBy(function(v) { return v });
if (JSON.stringify(videos) !== expected) {
throw "Expected " + expected;
}
}
function() {
var movieLists = [
{
name: "New Releases",
videos: [
{
"id": 70111470,
"title": "Die Hard",
"boxart": "http://cdn-0.nflximg.com/images/2891/DieHard.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"bookmark": []
},
{
"id": 654356453,
"title": "Bad Boys",
"boxart": "http://cdn-0.nflximg.com/images/2891/BadBoys.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"bookmark": [{ id:432534, time:65876586 }]
}
]
},
{
name: "Dramas",
videos: [
{
"id": 65432445,
"title": "The Chamber",
"boxart": "http://cdn-0.nflximg.com/images/2891/TheChamber.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"bookmark": []
},
{
"id": 675465,
"title": "Fracture",
"boxart": "http://cdn-0.nflximg.com/images/2891/Fracture.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"bookmark": [{ id:432534, time:65876586 }]
}
]
}
];
// ------------ INSERT CODE HERE! -----------------------------------
// Use map and mergeAll to flatten the movieLists in a list of video ids.
// ------------ INSERT CODE HERE! -----------------------------------
return movieLists.
map(function(movieList) {
return movieList.videos.
map(function(video) {
return video.id;
});
}).
mergeAll();
}
Wow! Great work. Mastering the combination of map() and mergeAll() is key to effective functional programming. You're half way there! Let's try a more complicated example...
You've managed to flatten a tree that's two levels deep, let's try for three! Let's say that instead of a single boxart url on each video, we had a collection of boxart objects, each with a different size and url. Create a query that selects {id, title, boxart} for every video in the movieLists. This time though, the boxart property in the result will be the url of the boxart object with dimensions of 150x200px. Let's see if you can solve this problem with map(), mergeAll(), and filter().
There's just more one thing: you can't use indexers. In other words, this is illegal:
var itemInArray = movieLists[0];
Furthermore, you're not allowed to use indexers in any of the remaining exercises unless you're implementing one of the five functions. There is a very good reason for this restriction, and that reason will eventually be explained. For now, you'll simply have to accept it on faith that this restriction serves a purpose. :-)
function(str) {
var fun = eval("(" + str + ")"),
videos = fun(),
got,
expected = JSON.stringify([
{"id": 675465,"title": "Fracture","boxart":"http://cdn-0.nflximg.com/images/2891/Fracture150.jpg" },
{"id": 65432445,"title": "The Chamber","boxart":"http://cdn-0.nflximg.com/images/2891/TheChamber150.jpg" },
{"id": 654356453,"title": "Bad Boys","boxart":"http://cdn-0.nflximg.com/images/2891/BadBoys150.jpg" },
{"id": 70111470,"title": "Die Hard","boxart":"http://cdn-0.nflximg.com/images/2891/DieHard150.jpg" }
].sortBy(function(v) { return v.id }));
if (str.indexOf('[0]') !== -1) {
throw "You're not allowed to index into the array. You might be creating the object too early. Instead of using an indexer to get the boxart out of the array, try adding a call to map() and creating the object inside the projection function.";
}
videos = videos.sortBy(function(v) { return v.id });
got = JSON.stringify(videos);
if (got !== expected) {
throw "Expected " + JSON.stringify(expected + " Got " + got);
}
}
function() {
var movieLists = [
{
name: "Instant Queue",
videos : [
{
"id": 70111470,
"title": "Die Hard",
"boxarts": [
{ width: 150, height:200, url:"http://cdn-0.nflximg.com/images/2891/DieHard150.jpg" },
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/DieHard200.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"bookmark": []
},
{
"id": 654356453,
"title": "Bad Boys",
"boxarts": [
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/BadBoys200.jpg" },
{ width: 150, height:200, url:"http://cdn-0.nflximg.com/images/2891/BadBoys150.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"bookmark": [{ id:432534, time:65876586 }]
}
]
},
{
name: "New Releases",
videos: [
{
"id": 65432445,
"title": "The Chamber",
"boxarts": [
{ width: 150, height:200, url:"http://cdn-0.nflximg.com/images/2891/TheChamber150.jpg" },
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/TheChamber200.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"bookmark": []
},
{
"id": 675465,
"title": "Fracture",
"boxarts": [
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture200.jpg" },
{ width: 150, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture150.jpg" },
{ width: 300, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture300.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"bookmark": [{ id:432534, time:65876586 }]
}
]
}
];
// Use one or more map, mergeAll, and filter calls to create an array with the following items
// [
// {"id": 675465,"title": "Fracture","boxart":"http://cdn-0.nflximg.com/images/2891/Fracture150.jpg" },
// {"id": 65432445,"title": "The Chamber","boxart":"http://cdn-0.nflximg.com/images/2891/TheChamber150.jpg" },
// {"id": 654356453,"title": "Bad Boys","boxart":"http://cdn-0.nflximg.com/images/2891/BadBoys150.jpg" },
// {"id": 70111470,"title": "Die Hard","boxart":"http://cdn-0.nflximg.com/images/2891/DieHard150.jpg" }
// ];
return movieLists.
map(function(movieList) {
return movieList.videos.
map(function(video) {
return video.boxarts.
filter(function(boxart) {
return boxart.width === 150;
}).
map(function(boxart) {
return {id: video.id, title: video.title, boxart: boxart.url};
});
}).
mergeAll();
}).
mergeAll();
}
Fantastic job! Now you've learned to use mergeAll() alongside map() and filter() to queries trees. Notice that map() and mergeAll() are very commonly chained together. Let's create a small helper function to help us with this common pattern.
Nearly every time we flatten a tree we chain map() and mergeAll(). Sometimes, if we're dealing with a tree several levels deep, we'll repeat this combination many times in our code. To save on typing, let's create a flatMap function that's just a map operation, followed by a mergeAll.
// Implement mergeAll
function(str) {
var fun = eval(str),
spanishFrenchEnglishWords = [ ["cero","rien","zero"], ["uno","un","one"], ["dos","deux","two"] ],
allWords = [0,1,2],
result,
expected = '["cero","rien","zero","uno","un","one","dos","deux","two"]';
var allWords = [0,1,2].
flatMap(function(index) {
return spanishFrenchEnglishWords[index];
});
if (JSON.stringify(allWords) !== expected) {
throw "Expected " + expected;
}
}
Array.prototype.flatMap = function(projectionFunctionThatReturnsArray) {
return this.
map(function(item) {
return projectionFunctionThatReturnsArray(item);
}).
// apply the mergeAll function to flatten the two-dimensional array
mergeAll();
};
/*
var spanishFrenchEnglishWords = [ ["cero","rien","zero"], ["uno","un","one"], ["dos","deux","two"] ];
// collect all the words for each number, in every language, in a single, flat list
var allWords = [0,1,2].
flatMap(function(index) {
return spanishFrenchEnglishWords[index];
});
return JSON.stringify(allWords) === '["cero","rien","zero","uno","un","one","dos","deux","two"]';
*/
Now, instead of using map().mergeAll() to flatten a tree, we can just use flatMap helper function.
Let's repeat the exercise we just performed. However this time we'll simplify the code by replacing the map().mergeAll() calls with flatMap().
function(str) {
var fun = eval("(" + str + ")"),
videos = fun(),
got,
expected = JSON.stringify([
{"id": 675465,"title": "Fracture","boxart":"http://cdn-0.nflximg.com/images/2891/Fracture150.jpg" },
{"id": 65432445,"title": "The Chamber","boxart":"http://cdn-0.nflximg.com/images/2891/TheChamber150.jpg" },
{"id": 654356453,"title": "Bad Boys","boxart":"http://cdn-0.nflximg.com/images/2891/BadBoys150.jpg" },
{"id": 70111470,"title": "Die Hard","boxart":"http://cdn-0.nflximg.com/images/2891/DieHard150.jpg" }
].sortBy(function(v) { return v.id }));
videos = videos.sortBy(function(v) { return v.id });
got = JSON.stringify(videos);
if (got !== expected) {
throw "Expected " + JSON.stringify(expected + "\n\nGot " + got);
}
}
function() {
var movieLists = [
{
name: "Instant Queue",
videos : [
{
"id": 70111470,
"title": "Die Hard",
"boxarts": [
{ width: 150, height:200, url:"http://cdn-0.nflximg.com/images/2891/DieHard150.jpg" },
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/DieHard200.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"bookmark": []
},
{
"id": 654356453,
"title": "Bad Boys",
"boxarts": [
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/BadBoys200.jpg" },
{ width: 150, height:200, url:"http://cdn-0.nflximg.com/images/2891/BadBoys150.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"bookmark": [{ id:432534, time:65876586 }]
}
]
},
{
name: "New Releases",
videos: [
{
"id": 65432445,
"title": "The Chamber",
"boxarts": [
{ width: 150, height:200, url:"http://cdn-0.nflximg.com/images/2891/TheChamber150.jpg" },
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/TheChamber200.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"bookmark": []
},
{
"id": 675465,
"title": "Fracture",
"boxarts": [
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture200.jpg" },
{ width: 150, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture150.jpg" },
{ width: 300, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture300.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"bookmark": [{ id:432534, time:65876586 }]
}
]
}
];
// Use one or more flatMap, map, and filter calls to create an array with the following items
// [
// {"id": 675465,"title": "Fracture","boxart":"http://cdn-0.nflximg.com/images/2891/Fracture150.jpg" },
// {"id": 65432445,"title": "The Chamber","boxart":"http://cdn-0.nflximg.com/images/2891/TheChamber150.jpg" },
// {"id": 654356453,"title": "Bad Boys","boxart":"http://cdn-0.nflximg.com/images/2891/BadBoys150.jpg" },
// {"id": 70111470,"title": "Die Hard","boxart":"http://cdn-0.nflximg.com/images/2891/DieHard150.jpg" }
// ];
return movieLists.
flatMap(function(movieList) {
return movieList.videos.
flatMap(function(video) {
return video.boxarts.
filter(function(boxart) {
return boxart.width === 150;
}).
map(function(boxart) {
return {id: video.id, title: video.title, boxart: boxart.url};
});
});
});
}
It's a very common pattern to see several nested flatMap operations, with the last operation being a map. You can think of this pattern as the functional version of a nested forEach.
Sometimes we need to perform an operation on more than one item in the array at the same time. For example, let's say we need to find the largest integer in an array. We can't use a filter() operation, because it only examines one item at a time. To find the largest integer we need to the compare items in the array to each other.
One approach could be to select an item in the array as the assumed largest number (perhaps the first item), and then compare that value to every other item in the array. Each time we come across a number that was larger than our assumed largest number, we'd replace it with the larger value, and continue the process until the entire array was traversed.
If we replaced the specific size comparison with a closure, we could write a function this handled the array traversal process for us. At each step our function would apply the closure to the last value and the current value and use the result as the last value the next time. Finally we'd be left with only one value. This process is known as reducing because we reduce many values to a single value.
In this example we use forEach to find the largest box art. Each time we examine a new boxart we update a variable with the currently known maximumSize. If the boxart is smaller than the maximum size, we discard it. If it's larger, we keep track of it. Finally we're left with a single boxart which must necessarily be the largest.
// Find largest box art
function(str){
var fun = eval("(" + str + ")"),
boxart = fun(),
got = JSON.stringify(boxart),
expected = JSON.stringify({ width: 425, height:150, url:"http://cdn-0.nflximg.com/images/2891/Fracture425.jpg" });
if (got !== expected) {
throw "Expected " + JSON.stringify(expected) + "\n\nGot " + got;
}
}
function() {
var boxarts = [
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture200.jpg" },
{ width: 150, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture150.jpg" },
{ width: 300, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture300.jpg" },
{ width: 425, height:150, url:"http://cdn-0.nflximg.com/images/2891/Fracture425.jpg" }
],
currentSize,
maxSize = -1,
largestBoxart;
boxarts.forEach(function(boxart) {
currentSize = boxart.width * boxart.height;
if (currentSize > maxSize) {
largestBoxart = boxart;
maxSize = currentSize;
}
});
return largestBoxart;
}
This process is a reduction because we're using the information we derived from the last computation to calculate the current value. However in the example above, we still have to specify the method of traversal. Wouldn't it be nice if we could just specify what operation we wanted to perform on the last and current value? Let's create a helper function to perform reductions on arrays.
Let's add a reduce() function to the Array type. Like map
// Implement reduce
function(str) {
var fun = eval(str),
numbers = [1,2,3],
sum = numbers.reduce(function(acc,curr) { return acc + curr }),
expected = JSON.stringify([6]),
sum2 = numbers.reduce(function(acc,curr) { return acc + curr },10),
expected2 = JSON.stringify([16]);
if (JSON.stringify(sum) !== expected) {
throw "Expected that [1,2,3].reduce(function(accumulated,current) { return accumulated + current; }) === [6]. Instead got " + JSON.stringify(sum);
}
if (JSON.stringify(sum2) !== expected2) {
throw "Expected that [1,2,3].reduce(function(accumulated,current) { return accumulated + current; }, 10) === [16]. Instead got " + JSON.stringify(sum2);
}
}
// [1,2,3].reduce(function(accumulatedValue, currentValue) { return accumulatedValue + currentValue; }); === [6];
// [1,2,3].reduce(function(accumulatedValue, currentValue) { return accumulatedValue + currentValue; }, 10); === [16];
Array.prototype.reduce = function(combiner, initialValue) {
var counter,
accumulatedValue;
// If the array is empty, do nothing
if (this.length === 0) {
return this;
}
else {
// If the user didn't pass an initial value, use the first item.
if (arguments.length === 1) {
counter = 1;
accumulatedValue = this[0];
}
else if (arguments.length >= 2) {
counter = 0;
accumulatedValue = initialValue;
}
else {
throw "Invalid arguments.";
}
// Loop through the array, feeding the current value and the result of
// the previous computation back into the combiner function until
// we've exhausted the entire array and are left with only one function.
while(counter < this.length) {
accumulatedValue = combiner(accumulatedValue, this[counter])
counter++;
}
return [accumulatedValue];
}
};
Let's use our new reduce function to isolate the largest value in an array of ratings.
// Find largest rating
function(str){
var fun = eval("(" + str + ")"),
boxarts = fun(),
got = JSON.stringify(boxarts),
expected = JSON.stringify([5]);
if (got !== expected) {
throw "Expected " + JSON.stringify(expected) + "\n\nGot " + got;
}
}
function() {
var ratings = [2,3,1,4,5];
// You should return an array containing only the largest rating. Remember that reduce always
// returns an array with one item.
return ratings.
reduce(function(acc, curr) {
if(acc > curr) {
return acc;
}
else {
return curr;
}
});
}
Nice work. Now let's try combining reduce() with our other functions to build more complex queries.
Let's try combining reduce() with map() to reduce multiple boxart objects to a single value: the url of the largest box art.
// Find largest box art with reduce
function(str){
var fun = eval("(" + str + ")"),
boxarts = fun(),
got = JSON.stringify(boxarts),
expected = JSON.stringify(["http://cdn-0.nflximg.com/images/2891/Fracture425.jpg"]);
if (got !== expected) {
throw "Expected " + JSON.stringify(expected) + "\n\nGot " + got;
}
}
function() {
var boxarts = [
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture200.jpg" },
{ width: 150, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture150.jpg" },
{ width: 300, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture300.jpg" },
{ width: 425, height:150, url:"http://cdn-0.nflximg.com/images/2891/Fracture425.jpg" }
];
// You should return an array containing only the largest box art. Remember that reduce always
// returns an array with one item.
return boxarts.
reduce(function(acc,curr) {
if (acc.width * acc.height > curr.width * curr.height) {
return acc;
}
else {
return curr;
}
}).
map(function(boxart) {
return boxart.url;
});
}
Sometimes when we reduce an array, we want the reduced value to be a different type than the items stored in the array. Let's say we have an array of videos and we want to reduce them to a single map where the key is the video id and the value is the video's title.
// Reducing with an initial value
function(str){
var fun = eval("(" + str + ")"),
videoMap = fun()[0],
expected = [
{
"65432445": "The Chamber",
"675465": "Fracture",
"70111470": "Die Hard",
"654356453": "Bad Boys"
}
];
if (!(videoMap["65432445"] === "The Chamber" && videoMap["675465"] === "Fracture" && videoMap["70111470"] === "Die Hard" && videoMap["654356453"] === "Bad Boys")) {
throw "Expected " + JSON.stringify(expected);
}
}
function() {
var videos = [
{
"id": 65432445,
"title": "The Chamber"
},
{
"id": 675465,
"title": "Fracture"
},
{
"id": 70111470,
"title": "Die Hard"
},
{
"id": 654356453,
"title": "Bad Boys"
}
];
// Expecting this output...
// [
// {
// "65432445": "The Chamber",
// "675465": "Fracture",
// "70111470": "Die Hard",
// "654356453": "Bad Boys"
// }
// ]
return videos.
reduce(function(accumulatedMap, video) {
// Object.create() makes a fast copy of the accumulatedMap by
// creating a new object and setting the accumulatedMap to be the
// new object's prototype.
// Initially the new object is empty and has no members of its own,
// except a pointer to the object on which it was based. If an
// attempt to find a member on the new object fails, the new object
// silently attempts to find the member on its prototype. This
// process continues recursively, with each object checking its
// prototype until the member is found or we reach the first object
// we created.
// If we set a member value on the new object, it is stored
// directly on that object, leaving the prototype unchanged.
// Object.create() is perfect for functional programming because it
// makes creating a new object with a different member value almost
// as cheap as changing the member on the original object!
var copyOfAccumulatedMap = Object.create(accumulatedMap);
copyOfAccumulatedMap[video.id] = video.title;
return copyOfAccumulatedMap;
},
// Use an empty map as the initial value instead of the first item in
// the list.
{});
}
Nice work. Now let's try combining reduce() with our other functions to build more complex queries.
This is a variation of the problem we solved earlier, where we retrieved the url of the boxart with a width of 150px. This time we'll use reduce() instead of filter() to retrieve the smallest box art in the boxarts array.
// Find the id, title, and smallest box art.
function(str) {
var fun = eval("(" + str + ")"),
videos = fun(),
got,
expected = JSON.stringify([
{"id": 675465,"title": "Fracture","boxart":"http://cdn-0.nflximg.com/images/2891/Fracture120.jpg" },
{"id": 65432445,"title": "The Chamber","boxart":"http://cdn-0.nflximg.com/images/2891/TheChamber130.jpg" },
{"id": 654356453,"title": "Bad Boys","boxart":"http://cdn-0.nflximg.com/images/2891/BadBoys140.jpg" },
{"id": 70111470,"title": "Die Hard","boxart":"http://cdn-0.nflximg.com/images/2891/DieHard150.jpg" }
].sortBy(function(v) { return v.id }));
if (str.indexOf('[0]') !== -1){
throw "You're not allowed to index into the array. You might be creating the object too early. Instead of using an indexer to get the boxart out of the array, try adding a call to map() and creating the object inside the projection function.";
}
videos = videos.sortBy(function(v) { return v.id });
got = JSON.stringify(videos);
if (got !== expected) {
throw "Expected " + JSON.stringify(expected + "\n\nGot " + got);
}
}
function() {
var movieLists = [
{
name: "New Releases",
videos: [
{
"id": 70111470,
"title": "Die Hard",
"boxarts": [
{ width: 150, height:200, url:"http://cdn-0.nflximg.com/images/2891/DieHard150.jpg" },
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/DieHard200.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"bookmark": []
},
{
"id": 654356453,
"title": "Bad Boys",
"boxarts": [
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/BadBoys200.jpg" },
{ width: 140, height:200, url:"http://cdn-0.nflximg.com/images/2891/BadBoys140.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"bookmark": [{ id:432534, time:65876586 }]
}
]
},
{
name: "Thrillers",
videos: [
{
"id": 65432445,
"title": "The Chamber",
"boxarts": [
{ width: 130, height:200, url:"http://cdn-0.nflximg.com/images/2891/TheChamber130.jpg" },
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/TheChamber200.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"bookmark": []
},
{
"id": 675465,
"title": "Fracture",
"boxarts": [
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture200.jpg" },
{ width: 120, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture120.jpg" },
{ width: 300, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture300.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"bookmark": [{ id:432534, time:65876586 }]
}
]
}
];
// Use one or more flatMap, map, and reduce calls to create an array with the following items (order doesn't matter)
// [
// {"id": 675465,"title": "Fracture","boxart":"http://cdn-0.nflximg.com/images/2891/Fracture120.jpg" },
// {"id": 65432445,"title": "The Chamber","boxart":"http://cdn-0.nflximg.com/images/2891/TheChamber130.jpg" },
// {"id": 654356453,"title": "Bad Boys","boxart":"http://cdn-0.nflximg.com/images/2891/BadBoys140.jpg" },
// {"id": 70111470,"title": "Die Hard","boxart":"http://cdn-0.nflximg.com/images/2891/DieHard150.jpg" }
// ];
return movieLists.
flatMap(function(movieList) {
return movieList.videos.
flatMap(function(video) {
return video.boxarts.
reduce(function(acc,curr) {
if (acc.width * acc.height < curr.width * curr.height) {
return acc;
}
else {
return curr;
}
}).
map(function(boxart) {
return {id: video.id, title: video.title, boxart: boxart.url};
});
});
});
}
Sometimes we need to combine two arrays by progressively taking an item from each and combining the pair. If you visualize a zipper, where each side is an array, and each tooth is an item, you'll have a good idea of how the zip operation works.
Use a for loop to traverse the videos and bookmarks array at the same time. For each video and bookmark pair, create a {videoId, bookmarkId} pair and add it to the videoIdAndBookmarkIdPairs array.
// Zip imperatively
function(str) {
var fun = eval("(" + str + ")"),
pairs = fun(),
got,
expected = '[{"videoId":65432445,"bookmarkId":445},{"videoId":70111470,"bookmarkId":470},{"videoId":654356453,"bookmarkId":453}]';
pairs = pairs.sortBy(function(v) { return v.videoId });
got = JSON.stringify(pairs);
if (got !== expected) {
throw "Expected " + JSON.stringify(expected + "\n\nGot " + got);
}
}
function() {
var videos = [
{
"id": 70111470,
"title": "Die Hard",
"boxart": "http://cdn-0.nflximg.com/images/2891/DieHard.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
},
{
"id": 654356453,
"title": "Bad Boys",
"boxart": "http://cdn-0.nflximg.com/images/2891/BadBoys.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
},
{
"id": 65432445,
"title": "The Chamber",
"boxart": "http://cdn-0.nflximg.com/images/2891/TheChamber.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
},
{
"id": 675465,
"title": "Fracture",
"boxart": "http://cdn-0.nflximg.com/images/2891/Fracture.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
}
],
bookmarks = [
{id: 470, time: 23432},
{id: 453, time: 234324},
{id: 445, time: 987834}
],
counter,
videoIdAndBookmarkIdPairs = [];
for(var counter = 0; counter < Math.min(videos.length, bookmarks.length); counter++) {
videoIdAndBookmarkIdPairs.push({videoId: videos[counter].id, bookmarkId: bookmarks[counter].id});
}
return videoIdAndBookmarkIdPairs;
}
Let's add a static zip() function to the Array type. The zip function accepts a combiner function, traverses each array at the same time, and calls the combiner function on the current item on the left-hand-side and right-hand-side. The zip function requires an item from each array in order to call the combiner function, therefore the array returned by zip will only be as large as the smallest input array.
// Implement zip
function(str) {
var fun = eval(str),
left = [1,2,3],
right = [4,5,6],
sum = Array.zip(left, right, function(left, right){ return left + right; }),
expected = '[5,7,9]';
if (JSON.stringify(sum) !== expected) {
throw "Expected that [1,2,3].reduce(function(accumulated,current) { return accumulated + current; }) === [6]. Instead got " + JSON.stringify(sum);
}
}
// JSON.stringify(Array.zip([1,2,3],[4,5,6], function(left, right) { return left + right })) === '[5,7,9]' accumulatedValue + currentValue; }); === [6];
Array.zip = function(left, right, combinerFunction) {
var counter,
results = [];
for(counter = 0; counter < Math.min(left.length, right.length); counter++) {
results.push(combinerFunction(left[counter],right[counter]));
}
return results;
};
Let's repeat exercise 19, but this time let's use your new zip() function. For each video and bookmark pair, create a {videoId, bookmarkId} pair.
// Combine videos and bookmarks
function(str) {
var fun = eval("(" + str + ")"),
pairs = fun(),
got,
expected = '[{"videoId":65432445,"bookmarkId":445},{"videoId":70111470,"bookmarkId":470},{"videoId":654356453,"bookmarkId":453}]';
pairs = pairs.sortBy(function(v) { return v.videoId });
got = JSON.stringify(pairs);
if (got !== expected) {
throw "Expected " + JSON.stringify(expected + "\n\nGot " + got);
}
}
function() {
var videos = [
{
"id": 70111470,
"title": "Die Hard",
"boxart": "http://cdn-0.nflximg.com/images/2891/DieHard.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
},
{
"id": 654356453,
"title": "Bad Boys",
"boxart": "http://cdn-0.nflximg.com/images/2891/BadBoys.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
},
{
"id": 65432445,
"title": "The Chamber",
"boxart": "http://cdn-0.nflximg.com/images/2891/TheChamber.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
},
{
"id": 675465,
"title": "Fracture",
"boxart": "http://cdn-0.nflximg.com/images/2891/Fracture.jpg",
"uri": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
}
],
bookmarks = [
{id: 470, time: 23432},
{id: 453, time: 234324},
{id: 445, time: 987834}
];
return Array.
zip(
videos,
bookmarks,
function(video, bookmark) {
return {videoId: video.id, bookmarkId: bookmark.id};
});
}
This is a variation of the problem we solved earlier. This time each video has an interesting moments collection, each representing a time during which a screenshot is interesting or representative of the title as a whole. Notice that both the boxarts and interestingMoments arrays are located at the same depth in the tree. Retrieve the time of the middle interesting moment and the smallest box art url simultaneously with zip(). Return an {id, title, time, boxart} object for each video.
// Find id, title, smallest box art, and bookmark id
function(str) {
var fun = eval("(" + str + ")"),
videos = fun(),
got,
expected = '[{"id":675465,"title":"Fracture","time":3453434,"url":"http://cdn-0.nflximg.com/images/2891/Fracture120.jpg"},{"id":65432445,"title":"The Chamber","time":3452343,"url":"http://cdn-0.nflximg.com/images/2891/TheChamber130.jpg"},{"id":70111470,"title":"Die Hard","time":323133,"url":"http://cdn-0.nflximg.com/images/2891/DieHard150.jpg"},{"id":654356453,"title":"Bad Boys","time":6575665,"url":"http://cdn-0.nflximg.com/images/2891/BadBoys140.jpg"}]';
videos = videos.sortBy(function(v) { return v.id });
got = JSON.stringify(videos);
if (got !== expected) {
throw "Expected " + JSON.stringify(expected + "\n\nGot " + got);
}
}
function() {
var movieLists = [
{
name: "New Releases",
videos: [
{
"id": 70111470,
"title": "Die Hard",
"boxarts": [
{ width: 150, height:200, url:"http://cdn-0.nflximg.com/images/2891/DieHard150.jpg" },
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/DieHard200.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"interestingMoments": [
{ type: "End", time:213432 },
{ type: "Start", time: 64534 },
{ type: "Middle", time: 323133}
]
},
{
"id": 654356453,
"title": "Bad Boys",
"boxarts": [
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/BadBoys200.jpg" },
{ width: 140, height:200, url:"http://cdn-0.nflximg.com/images/2891/BadBoys140.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"interestingMoments": [
{ type: "End", time:54654754 },
{ type: "Start", time: 43524243 },
{ type: "Middle", time: 6575665}
]
}
]
},
{
name: "Instant Queue",
videos: [
{
"id": 65432445,
"title": "The Chamber",
"boxarts": [
{ width: 130, height:200, url:"http://cdn-0.nflximg.com/images/2891/TheChamber130.jpg" },
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/TheChamber200.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 4.0,
"interestingMoments": [
{ type: "End", time:132423 },
{ type: "Start", time: 54637425 },
{ type: "Middle", time: 3452343}
]
},
{
"id": 675465,
"title": "Fracture",
"boxarts": [
{ width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture200.jpg" },
{ width: 120, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture120.jpg" },
{ width: 300, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture300.jpg" }
],
"url": "http://api.netflix.com/catalog/titles/movies/70111470",
"rating": 5.0,
"interestingMoments": [
{ type: "End", time:45632456 },
{ type: "Start", time: 234534 },
{ type: "Middle", time: 3453434}
]
}
]
}
];
//------------ COMPLETE THIS EXPRESSION --------------
return movieLists.
flatMap(function(movieList) {
return movieList.videos.
flatMap(function(video) {
return Array.zip(
video.boxarts.
reduce(function(acc,curr) {
if (acc.width * acc.height < curr.width * curr.height) {
return acc;
}
else {
return curr;
}
}),
video.interestingMoments.
filter(function(interestingMoment) {
return interestingMoment.type === "Middle";
}),
function(boxart, interestingMoment) {
return {id: video.id, title: video.title, time: interestingMoment.time, url: boxart.url};
})
});
});
}
Now that we've learned the five operators let's flex our muscles and write some powerful queries.
When information is organized in a tree like a JSON expression, relationships point from parent to child. In relational systems like databases, relationships point from children to their parents. Both ways of organizing information are equivalent, and depending on the circumstances, we might get data organized in one way or another. It may surprise you to learn that you can use the 5 query functions you already know to easily convert between these representations. In other words, not only can you query arrays from trees, you can query trees from arrays.
We have 2 arrays each containing lists, and videos respectively. Each video has a listId field indicating its parent list. We want to build an array of list objects, each with a name and a videos array. The videos array will contain the video's id and title. In other words we want to build the following structure:
[
{
"name": "New Releases",
"videos": [
{
"id": 65432445,
"title": "The Chamber"
},
{
"id": 675465,
"title": "Fracture"
}
]
},
{
"name": "Thrillers",
"videos": [
{
"id": 70111470,
"title": "Die Hard"
},
{
"id": 654356453,
"title": "Bad Boys"
}
]
}
]
Note: please make sure when creating objects (both lists and videos) that you add properties in the same order as above. This doesn't impact the correctness of your code, but the verifier expects properties to be created in this order.
// Combine videos and bookmarks
function(str) {
var fun = eval("(" + str + ")"),
pairs = fun(),
got,
expected = '[{"name":"New Releases","videos":[{"id":65432445,"title":"The Chamber"},{"id":675465,"title":"Fracture"}]},{"name":"Thrillers","videos":[{"id":70111470,"title":"Die Hard"},{"id":654356453,"title":"Bad Boys"}]}]';
got = JSON.stringify(pairs);
if (got !== expected) {
throw "Expected " + JSON.stringify(expected + "\n\nGot " + got);
}
}
function() {
var lists = [
{
"id": 5434364,
"name": "New Releases"
},
{
"id": 65456475,
name: "Thrillers"
}
],
videos = [
{
"listId": 5434364,
"id": 65432445,
"title": "The Chamber"
},
{
"listId": 5434364,
"id": 675465,
"title": "Fracture"
},
{
"listId": 65456475,
"id": 70111470,
"title": "Die Hard"
},
{
"listId": 65456475,
"id": 654356453,
"title": "Bad Boys"
}
];
return lists.map(function(list) {
return {
name: list.name,
videos:
videos.
filter(function(video) {
return video.listId === list.id;
}).
map(function(video) {
return {id: video.id, title: video.title};
})
};
});
}
Looks like you figured out that you can use map and filter to join two different arrays by a key. Now let's try a more complex example...
Let's try creating a deeper tree structure. This time we have 4 seperate arrays each containing lists, videos, boxarts, and bookmarks respectively. Each object has a parent id, indicating its parent. We want to build an array of list objects, each with a name and a videos array. The videos array will contain the video's id, title, bookmark time, and smallest boxart url. In other words we want to build the following structure:
[
{
"name": "New Releases",
"videos": [
{
"id": 65432445,
"title": "The Chamber",
"time": 32432,
"boxart": "http://cdn-0.nflximg.com/images/2891/TheChamber130.jpg"
},
{
"id": 675465,
"title": "Fracture",
"time": 3534543,
"boxart": "http://cdn-0.nflximg.com/images/2891/Fracture120.jpg"
}
]
},
{
"name": "Thrillers",
"videos": [
{
"id": 70111470,
"title": "Die Hard",
"time": 645243,
"boxart": "http://cdn-0.nflximg.com/images/2891/DieHard150.jpg"
},
{
"id": 654356453,
"title": "Bad Boys",
"time": 984934,
"boxart": "http://cdn-0.nflximg.com/images/2891/BadBoys140.jpg"
}
]
}
]
Note: please make sure when creating objects (both lists and videos) that you add properties in the same order as above. This doesn't impact the correctness of your code, but the verifier expects properties to be created in this order.
// Combine videos and bookmarks
function(str) {
var fun = eval("(" + str + ")"),
pairs = fun(),
got,
expected = '[{"name":"New Releases","videos":[{"id":65432445,"title":"The Chamber","time":32432,"boxart":"http://cdn-0.nflximg.com/images/2891/TheChamber130.jpg"},{"id":675465,"title":"Fracture","time":3534543,"boxart":"http://cdn-0.nflximg.com/images/2891/Fracture120.jpg"}]},{"name":"Thrillers","videos":[{"id":70111470,"title":"Die Hard","time":645243,"boxart":"http://cdn-0.nflximg.com/images/2891/DieHard150.jpg"},{"id":654356453,"title":"Bad Boys","time":984934,"boxart":"http://cdn-0.nflximg.com/images/2891/BadBoys140.jpg"}]}]';
got = JSON.stringify(pairs);
if (got !== expected) {
throw "Expected " + JSON.stringify(expected + "\n\nGot " + got);
}
}
function() {
var lists = [
{
"id": 5434364,
"name": "New Releases"
},
{
"id": 65456475,
name: "Thrillers"
}
],
videos = [
{
"listId": 5434364,
"id": 65432445,
"title": "The Chamber"
},
{
"listId": 5434364,
"id": 675465,
"title": "Fracture"
},
{
"listId": 65456475,
"id": 70111470,
"title": "Die Hard"
},
{
"listId": 65456475,
"id": 654356453,
"title": "Bad Boys"
}
],
boxarts = [
{ videoId: 65432445, width: 130, height:200, url:"http://cdn-0.nflximg.com/images/2891/TheChamber130.jpg" },
{ videoId: 65432445, width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/TheChamber200.jpg" },
{ videoId: 675465, width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture200.jpg" },
{ videoId: 675465, width: 120, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture120.jpg" },
{ videoId: 675465, width: 300, height:200, url:"http://cdn-0.nflximg.com/images/2891/Fracture300.jpg" },
{ videoId: 70111470, width: 150, height:200, url:"http://cdn-0.nflximg.com/images/2891/DieHard150.jpg" },
{ videoId: 70111470, width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/DieHard200.jpg" },
{ videoId: 654356453, width: 200, height:200, url:"http://cdn-0.nflximg.com/images/2891/BadBoys200.jpg" },
{ videoId: 654356453, width: 140, height:200, url:"http://cdn-0.nflximg.com/images/2891/BadBoys140.jpg" }
],
bookmarks = [
{ videoId: 65432445, time: 32432 },
{ videoId: 675465, time: 3534543 },
{ videoId: 70111470, time: 645243 },
{ videoId: 654356453, time: 984934 }
];
return lists.map(function(list) {
return {
name: list.name,
videos:
videos.
filter(function(video) {
return video.listId === list.id;
}).
flatMap(function(video) {
return Array.zip(
bookmarks.filter(function(bookmark) {
return bookmark.videoId === video.id;
}),
boxarts.filter(function(boxart) {
return boxart.videoId === video.id;
}).
reduce(function(acc,curr) {
return acc.width * acc.height < curr.width * curr.height ? acc : curr;
}),
function(bookmark, boxart) {
return { id: video.id, title: video.title, time: bookmark.time, boxart: boxart.url };
});
})
};
});
}
Wow! That's a large query, but the code is still small relative to the amount of work it's doing. If we rewrote this query with a series of loops our code would be much less self-describing. Loops don't give the reader any information about the kind of operation being performed. Every time you see a loop, you need to carefully read through the code to find out what's being done. Is it a projection? A filter? A reduction? Why use loops for querying data when we've demonstrated that the 5 functions can be used to create virtually any output we want?
Let's try an easier question. Let's say we have a collection of all of the prices for NASDAQ stocks over time. Every time the price of a stock changes on the NASDAQ ticker an entry is added to this collection. Let's say that ten days ago you bought shares in Microsoft, and now you want to print all of the MSFT share prices since then. Filter the collection for MSFT trades starting from ten days ago and print each price record (including the time stamp) using the print() function. Note: this is not a trick question. It's as easy as it seems.
// The pricesNASDAQ collection looks something like this...
var pricesNASDAQ = [
// ... from the NASDAQ's opening day
{name: "ANGI", price: 31.22, timeStamp: new Date(2011,11,15) },
{name: "MSFT", price: 32.32, timeStamp: new Date(2011,11,15) },
{name: "GOOG", price: 150.43, timeStamp: new Date(2011,11,15)},
{name: "ANGI", price: 28.44, timeStamp: new Date(2011,11,16)},
{name: "GOOG", price: 199.33, timeStamp: new Date(2011,11,16)},
// ...and up to the present.
];
Console
// Combine videos and bookmarks
function(str, lesson) {
var output = $(".output", lesson)[0],
fun = eval("(" + str + ")"),
stockSymbols = ["MSFT", "GOOG","NFLX","OSTK"],
input = [{name: "MSFT", price: 32.32, timeStamp: new Date() }, {name: "GOOG", price: 150.43, timeStamp: new Date(2011,11,15)}],
expected = [input[0]],
items = [],
counter = 0;
confirmPrint = function(item) {
items.push(item);
},
print = function(item) {
output.innerHTML += "MSFT " + item.price + " at " + item.timeStamp.toString() + "<" + "br" + ">";
output.scrollTop = output.scrollHeight;
counter++;
if (counter % 100 === 0) {
output.innerHTML = "";
}
},
stocks =
Rx.Observable.
interval(250).
map(function() {
var symbol = stockSymbols[Math.floor(Math.random() * stockSymbols.length)];
return {name: symbol, price: 30 + ((Math.floor(Math.random() * 100))/200), timeStamp: new Date()};
});
fun(input, confirmPrint);
if (JSON.stringify(items) !== JSON.stringify(expected)) {
throw "Got " + JSON.stringify(items, null, 4) + ", expected " + JSON.stringify(expected, null, 4);
}
fun(stocks, print);
}
function(pricesNASDAQ, printRecord) {
var microsoftPrices,
now = new Date(),
tenDaysAgo = new Date( now.getYear(), now.getMonth(), now.getDate() - 10);
// use filter() to filter the trades for MSFT prices recorded any time after 10 days ago
microsoftPrices =
pricesNASDAQ.
filter(function(priceRecord) {
return priceRecord.name === 'MSFT' && priceRecord.timeStamp > tenDaysAgo;
});
// Print the trades to the output console
microsoftPrices.
forEach(function(priceRecord) {
printRecord(priceRecord);
});
}
Notice that the console is changing over time. Now look at the time stamps on the stock prices. We're displaying stock prices sampled after we ran our program! How could our array have contained stock price records from the future? Did we accidentally rip a hole in the space-time continuum?
The solution to the riddle is that pricesNASDAQ is not an array. Unlike an array, which can only store a snapshot of stock prices, this new type can react to changes and update over time.
In the next section I'll reveal the inner workings of this magical type. You'll learn how you can use it model everything from mouse events to asynchronous JSON requests. Finally I'll show you how you can query this type using the 5 query functions you already know. It's about time we gave this magical type a name...
Microsoft's open-source Reactive Extensions library introduces a new collection type to Javascript: Observable. An Observable is a lot like an Event. Like an Event, an Observable is a sequence of values that a data producer pushes to the consumer. However unlike an Event, an Observable can signal to a listener that it has completed, and will send no more data.
Observables can send data to consumers asynchronously. Unlike Array, there's no Javascript literal syntax for creating Observable sequences. However we can build a helper method that visually describes the contents of sequences as well as the times between each item's arrival. The seq function creates an Observable from an array of items, and adds a delay for every empty item encountered. Every ,,, adds up to a second.
// An array of numbers 1,2,3
var numbers123Array = [1,2,3];
// A sequence that returns 1, and then after 4 seconds returns 2,
// and then after another second returns 3, and then completes.
var numbers123Observable = seq([1,,,,,,,,,,,,2,,,3]);
// Like Arrays, Observables can contain any object - even Arrays.
var observableOfArrays = seq([ [1,2,3],,,,,,[4,5,6],,,,,,,,,,,[1,2] ]);
Observables are a sequence of values, delivered one after the other. Therefore it's possible that an Observable can go on sending data to a listener forever just like a mouse move event. To create a sequence that doesn't complete, you can add a trailing ,,, to the end of the items passed to seq().
// The trailing ,,, ensures that the sequence will not complete.
var mouseMovesObservable =
seq([ {x: 23, y: 55},,,,,,,{x: 44, y: 99},,,{x:55,y:99},,,{x: 54, y:543},,, ]);
// No trailing ,,, indicates that sequence will complete.
var numbers123Observable = seq([1,,,2,,,3]);
Querying Arrays only gives us a snapshot. By contrast, querying Observables allows us to create data sets that react and update as the system changes over time. This enables a very powerful type of programming known as reactive programming.
Let's start off by contrasting Observable with Events...
You're probably used to thinking about events as a list of handlers stored in an object. In this example, we subscribe to a button click event and then unsubscribe the first time the button is clicked.
// Combine videos and bookmarks 2
function(str, lesson) {
var fun = eval("(" + str + ")"),
button = $('.button', lesson)[0];
fun(button);
}
Ask yourself this question: How is subscribing to an event different than traversing an array? Both operations involve sending a listener a sequence of items by repeatedly invoking a function. So why can't we traverse Arrays and Events the same way?
Subscribing to an Event and traversing an Array are fundamentally the same operation. The only difference is that Array traversal is synchronous and completes, and Event traversal is asynchronous and never completes. If we convert a button click Event to an Observable object, we can use forEach() to traverse the Event.
// Combine videos and bookmarks 2
function(str, lesson) {
var fun = eval("(" + str + ")"),
button = $('.button', lesson)[0];
fun(button);
}
Notice that Observable's forEach() function returns a Subscription object. Disposing of a Subscription object unsubscribes from the event and prevents memory leaks. Disposing of a subscription is the asynchronous equivalent of stopping half-way through a counting for loop.
Disposing of a Subscription object is basically the same as calling removeEventListener(). On the surface, these two approaches to Event handling don't seem to be very different. Under the circumstances, why should we bother converting Events to Observables? The reason is that if we convert Events to Observable Objects, we can use powerful functions to transform them. In the next exercise we'll learn how we can use one such function to avoid dealing with Subscriptions in many cases...
Have you ever wished that you could listen for the next occurrence of an event and then immediately unsubscribe? For example, developers will often attach an event handler to window.onload, expecting that their event handler will only be called once.
window.addEventListener(
"load",
function()
// do some work here, but expect this function will only be called once.
})
In instances such as this, it's good practice to unsubscribe from the event after it's fired. Failing to unsubscribe is a memory leak. Depending on the circumstances, memory leaks can seriously destablize your application and can be very difficult to track down. Unfortunately unsubscribing from an event after one occurrence can be cumbersome:
var handler = function() {
// do some work here, then unsubscribe from the event
window.removeEventListener("load", handler)
};
window.addEventListener("load", handler);
Wouldn't it be nice if there was an easier way to code this? That's why Observable has a take() function. The take() function works like this...
seq([1,,,2,,,3,,,4,,,5,,,6,,,]).take(3) === seq([1,,,2,,,3]);
An Observable based on an Event will never complete on its own. The take() function creates a new sequence that completes after a discrete number of items arrive. This is important, because unlike an Event, when an Observable sequence completes it unsubscribes all of its listeners. That means that if we use take() to complete our Event sequence, we don't need to unsubscribe!
Let's repeat the previous exercise, in which we listened for a single button click and then unsubscribed. This time let's use the take() function to complete the sequence after the button is clicked.
// Combine videos and bookmarks 2
function(str, lesson) {
var fun = eval("(" + str + ")"),
button = $('.button', lesson)[0];
fun(button);
}
function(button) {
var buttonClicks = Observable.fromEvent(button, "click");
// Use take() to listen for only one button click
// and unsubscribe.
buttonClicks.
take(1).
forEach(function(priceRecord) {
alert("Button was clicked once. Stopping Traversal.");
});
}
The take() function is great way of listening for a discrete number of events and then unsubscribing, but Observable has an even more flexible function that we can use to complete sequences...
Have you ever wanted to unsubscribe from one Event when another Event fires? Observable's takeUntil() function is a convenient way of completing a sequence when another Event occurs. Here's how takeUntil() works:
var numbersUntilStopButtonPressed =
seq( [ 1,,,2,,,3,,,4,,,5,,,6,,,7,,,8,,,9,,, ]).
takeUntil(seq([ ,,, {eventType: "click"},,, ]) ) === seq([ 1,,,2 ])
Earlier we (unknowningly) built a dynamic Microsoft price stock ticker using Observable. The problem with that stock ticker was that it kept going on forever. If left unchecked, all the entries in the log could use up all of the memory on the page. In the exercise below, filter the Observable sequence of NASDAQ prices for MSFT stock prices, use the fromEvent() function to create an Observable .
<-Press this button to complete the sequence of microsoft stock prices.Console
// Combine videos and bookmarks 2
function(str, lesson) {
var fun = eval("(" + str + ")"),
output = $(".output", lesson)[0],
stopButton = $('.stop', lesson)[0],
stockSymbols = ["MSFT", "GOOG","NFLX","OSTK"],
input = [{name: "MSFT", price: 32.32, timeStamp: new Date() }, {name: "GOOG", price: 150.43, timeStamp: new Date(2011,11,15)}],
expected = [input[0]],
items = [],
counter = 0,
print = function(item) {
output.innerHTML += "MSFT " + item.price + " at " + item.timeStamp.toString() + "<" + "br" + ">";
output.scrollTop = output.scrollHeight;
counter++;
if (counter % 100 === 0) {
output.innerHTML = "";
}
},
stocks =
Rx.Observable.
interval(250).
map(function() {
var symbol = stockSymbols[Math.floor(Math.random() * stockSymbols.length)];
return {name: symbol, price: 30 + ((Math.floor(Math.random() * 100))/200), timeStamp: new Date()};
});
fun(stocks, print, stopButton);
}
function(pricesNASDAQ, printRecord, stopButton) {
var stopButtonClicks = Observable.fromEvent(stopButton,"click"),
microsoftPrices =
pricesNASDAQ.
filter(function(priceRecord) {
return priceRecord.name === "MSFT";
}).
takeUntil(stopButtonClicks);
microsoftPrices.
forEach(function(priceRecord) {
printRecord(priceRecord);
});
}
We've learned that Observable sequences are much more powerful than raw events, because they can complete. The take() and takeUntil() functions are powerful enough to ensure that we never have to unsubscribe from another event again! This reduces the risk of memory leaks and makes our code more readable.
Here's what we learned in this section:
In the next section we'll learn how to combine events to create more complex events. You'll be suprised how easily you can solve complex, asynchronous problems!
What's the difference between these two tasks?
You might think of them as different, and might code them very differently, but these tasks are fundamentally the same. Both of these tasks are queries, and can be solved using the functions you've learned in these exercises.
The difference between traversing an Array and traversing an Observable is the direction in which the data moves. When traversing an Array, the client pulls data from the data source, blocking until it gets a result. When traversing Observables, the data source pushes data at the client whenever it arrives.
It turns out that the direction in which data moves is orthogonal to querying that data. In other words, when we're querying data it doesn't matter whether we pull data, or data is pushed at us. In either case the query methods make the same transformations. The only thing that changes is the input and output type respectively. If we filter an Array, we get a new Array. If we filiter an Observable, we get a new Observable, and so on.
Take a look at how the query methods transform Observables and Arrays:
// map()
[1,2,3].map(function(x) { return x + 1 }) === [2,3,4]
seq([1,,,2,,,3]).map(function(x) { return x + 1}) === seq([2,,,3,,,4])
seq([1,,,2,,,3,,,]).map(function(x) { return x + 1 }) === seq([2,,,3,,,4,,,])
// filter()
[1,2,3].filter(function(x) { return x > 1; }) === [2,3]
seq([1,,,2,,,3]).filter(function(x) { return x > 1; }) === seq([2,,,3])
seq([1,,,2,,,3,,,]).filter(function(x) { return x > 1; }) === seq([2,,,3,,,])
// mergeAll()
[ [1, 2, 3], [4, 5, 6] ].mergeAll() === [1,2,3,4,5,6]
seq([ seq([1,,,2,,,3]),,,seq([4,,,5,,,6]) ]).mergeAll() === seq([1,,,2,,,3,,,4,,,5,,,6])
// Depending on when the elements in each
// sequence arrive, the order of the merged
// sequence could be different.
seq([
seq([1,, ,,, ,,,2,,,3])
,,,seq([4,,,5,,,,,,,,6]) ]).
mergeAll() === seq([1,,,4,,,5,,,2,,,3,,,6])
// Notice that as long as at least one sequence being
// merged is incomplete, the merged sequence is also
// incomplete.
seq([
seq([1,, ,,, ,,,2,,,3])
,,,seq([4,,,5,,, ,,, ,,6,,,]) ]).
mergeAll() === seq([1,,,4,,,5,,,2,,,3,,,6,,,])
// reduce()
[ 1, 2, 3 ].reduce(sumFunction) === [ 6 ]
seq([ 1,,,2,,,3 ]).reduce(sumFunction) === seq([,,,,,,6])
// Reduced sequences do not complete until the
// sequence does.
seq([ 1,,,2,,,3,,, ]).reduce(sumFunction) === seq([ ,,,,,,,,,])
// zip()
// In both Arrays and Observables, the zipped sequence
// completes as soon as either the left or right-hand
// side sequence completes.
Array.zip([1,2],[3,4,5], sumFunction) === [4,6]
Observable.zip(seq([1,,,2]),seq([3,,,4,,,5]), sumFunction) === seq([4,,,6])
// take()
[1,2,3].take(2) === [1, 2]
seq([ 1,,,2,,,3 ]).take(2) === seq([ 1,,,2 ])
seq([ 1,,,2,,,3,,, ]).take(2) === seq([ 1,,,2 ])
// takeUntil()
// takeUntil works for Arrays, but it's not very useful
// because the result will always be an empty array.
[1,2,3].takeUntil([1]) === []
seq([1,,,2,,,3,,, ]).takeUntil(
seq([ ,,, ,,4 , })) === seq([ 1,,,2 ])
Remember when I prohibited the use of array indexers? The reason for that restriction should now become clearer to you. Whereas the 5 functions can be used on any collection, indexers can only be used on collections that support random-access (like Array). If you avoid indexers and stick to the functions you've learned in this tutorial, you'll have a unified programming model for transforming any collection. Having a unified programming model makes it trivial to convert synchronous code to asynchronous code, a process which would otherwise be very difficult. As we've demonstrated, you don't need indexers to perform complex collection transformations.
Now that we've seen that we can query asychronous and synchronous data sources using the same programming model, let's use Observable and our query functions to create complex new events.
Remember the exercise we solved earlier? The one in which we retrieved all the movies with a rating of 5.0 from an array of movie lists? If we were to describe the solution in pseudocode it might read something like this...
"For every movie list, retrieve only those videos with a rating of 5.0"
var moviesWithHighRatings =
movieLists.
flatMap(function(movieList) {
return movieList.videos.
filter(function(video) {
return video.rating === 5.0;
});
});
Now we're going to create a mouseDrag event for a DOM object. If we were to describe this problem as pseudocode it might read something like this...
"For every movie list mouse down event on the sprite, retrieve only those videos with a rating of 5.0 mouse move events that occur before the next mouse up event."
// Combine videos and bookmarks 2
function(str, lesson) {
var fun = eval("(" + str + ")"),
output = $(".output", lesson)[0],
container = $(".container", lesson)[0],
sprite = $(".sprite",lesson)[0],
moveSprite = function(point) {
sprite.style.left = point.pageX + "px";
sprite.style.top = point.pageY + "px";
}
fun(sprite, container, moveSprite);
}
function(sprite, spriteContainer) {
var spriteMouseDowns = Observable.fromEvent(sprite, "mousedown"),
spriteContainerMouseMoves = Observable.fromEvent(spriteContainer, "mousemove"),
spriteContainerMouseUps = Observable.fromEvent(spriteContainer, "mouseup"),
spriteMouseDrags =
// For every mouse down event on the sprite...
spriteMouseDowns.
flatMap(function(contactPoint) {
// ...retrieve all the mouse move events on the sprite container...
return spriteContainerMouseMoves.
// ...until a mouse up event occurs.
takeUntil(spriteContainerMouseUps);
});
// For each mouse drag event, move the sprite to the absolute page position.
spriteMouseDrags.forEach(function(dragPoint) {
sprite.style.left = dragPoint.pageX + "px";
sprite.style.top = dragPoint.pageY + "px";
});
}
Now we're really cooking. We just created a complex event with a few lines a code. We didn't have to deal with any subscriptions objects, or write any stateful code whatsoever. Let's try something a little harder.
Our mouse drag event is a little too simple. Notice that when we drag around the sprite, it always positions itself at the top-left corner of the mouse. Ideally we'd like our drag event to offset its coordinates, based on where the mouse was when the mouse down event occurred. This will make our mouse drag more closely resemble moving a real object with our finger.
Let's see if you can adjust the coordinates in the mouse drag event, based on the mousedown location on the sprite. The mouse events are sequences, and they look something like this:
spriteContainerMouseMoves =
seq([ {x: 200, y: 400, offsetX: 10, offsetY: 15},,,{x: 210, y: 410, offsetX: 20, offsetY: 26},,, ])
Each item in the mouse event sequences contains an x, y value that represents that absolute location of the mouse event on the page. The moveSprite() function uses these coordinates to position the sprite. Each item in the sequence also contains a pair of offsetX and offsetY properties that indicate the position of the mouse event relative to the event target.
function(str, lesson) {
var fun = eval("(" + str + ")"),
output = $(".output", lesson)[0],
container = $(".container", lesson)[0],
sprite = $(".sprite",lesson)[0],
moveSprite = function(point) {
sprite.style.left = point.pageX + "px";
sprite.style.top = point.pageY + "px";
}
fun(sprite, container, moveSprite);
}
function(sprite, spriteContainer) {
// All of the mouse event sequences look like this:
// seq([ {pageX: 22, pageY: 3423, offsetX: 14, offsetY: 22} ,,, ])
var spriteMouseDowns = Observable.fromEvent(sprite, "mousedown"),
spriteContainerMouseMoves = Observable.fromEvent(spriteContainer, "mousemove"),
spriteContainerMouseUps = Observable.fromEvent(spriteContainer, "mouseup"),
// Create a sequence that looks like this:
// seq([ {pageX: 22, pageY:4080 },,,{pageX: 24, pageY: 4082},,, ])
spriteMouseDrags =
// For every mouse down event on the sprite...
spriteMouseDowns.
flatMap(function(contactPoint) {
// ...retrieve all the mouse move events on the sprite container...
return spriteContainerMouseMoves.
// ...until a mouse up event occurs.
takeUntil(spriteContainerMouseUps).
map(function(movePoint) {
return {
pageX: movePoint.pageX - contactPoint.offsetX,
pageY: movePoint.pageY - contactPoint.offsetY
};
});
});
// For each mouse drag event, move the sprite to the absolute page position.
spriteMouseDrags.forEach(function(dragPoint) {
sprite.style.left = dragPoint.pageX + "px";
sprite.style.top = dragPoint.pageY + "px";
});
}
Events aren't the only source of asynchronous data in an application. There's also HTTP requests. Most of the time HTTP requests are exposed via a callback-based API. To receive data asynchronously from a callback-based API, the client typically passes a success and error handler to the function. When the asynchronous operation completes, the appropriate handler is called with the data. In this exercise we'll use jQuery's getJSON api to asynchronously retrieve data.
// Combine videos and bookmarks 2
function(str, lesson) {
var fun = eval("(" + str + ")");
fun(jQueryMock);
}
Let's say that we're writing the startup flow for a web application. On startup, the application must perform the following operations:
// Combine videos and bookmarks 2
function(str, lesson) {
var fun = eval("(" + str + ")"),
NOOP = function() {};
fun(
{
addEventListener: function(event, handler) {
window.setTimeout(handler, 200)
},
removeEventListener: NOOP
},
jQueryMock,
function(output) { alert(output) },
function(output) { alert(output) });
}
It's fair to say that sequencing HTTP requests with callbacks is very hard. In order to perform two tasks in parallel, we have to introduce a variable to track the status of each task. Every time one of the parallel tasks completes it must check whether its sibling task has also completed. If both have completed, only then can we move forward. In the example above, every time a task is finished the tryToDisplayOutput() function is called to check if the program was ready to display output. This function checks the status of all tasks and displays the output if possible.
With a callback-based API, asynchronous error handling is also very complicated. In synchronous programs, a unit of work is cancelled when an exception is thrown. By contrast, in our program we had to explicitly track whether an error occurred in parallel to prevent an unnecessary call for the instant queue. Javascript provides us with special support for synchronous error handling with the keywords try/catch/throw. Unfortunately no such support is available for asynchronous programs.
The Observable interface is a much more powerful way of working with asynchronous APIs than callbacks. We'll see that Observables can free us from having to track the status of tasks that are run in parallel, just Observables frees us from having to track Event Subscriptions. We'll also see that Observable gives us the same error propagation semantics in asynchronous programs that we expect in synchronous programs. Finally we'll learn that by converting callback-based APIs to Observables, we can query them along with Events to build much more expressive programs.
If a callback API were a sequence, what kind of sequence would it be? We've seen that UI Event sequences can contain anywhere from 0 to infinite items, but will never complete on their own.
mouseMoves === seq([ {x: 23, y: 55},,,,,,,{x: 44, y: 99},,,{x:55,y:99},,,{x: 54, y:543},,, ]);
In contrast, if we were to convert output from the $.getJSON() function we've been using into a sequence it would always return a sequence that completes after sending a single item.
getJSONAsObservable("http://api-global.netflix.com/abTestInformation") ===
seq([ { urlPrefix: "billboardTest" } ])
It might seem strange to create sequences that contain only one object. We could introduce an Observable-like type specifically for scalar values, but that would make callback-based APIs more difficult to query with Events. Thankfully, an Observable sequence is flexible enough to model both.
So how do we convert a callback API into an Observable sequence? Unfortunately, because callback-based APIs vary so much in their interfaces, we can't create a conversion function like we did with fromEvent(). However there is a more flexible function we can use to build Observable sequences...
Observable.create() is powerful enough to convert any asynchronous API into an Observable. Observable.create() relies on the fact that all asynchronous APIs have the following semantics:
In the following example, we'll use the Observable.create() function to create an Observable that issues a request to getJSON when it's traversed.
// Combine videos and bookmarks 2
function(str, lesson) {
var fun = eval("(" + str + ")"),
NOOP = function() {};
fun(
{
addEventListener: function(event, handler) {
window.setTimeout(handler, 200)
},
removeEventListener: NOOP
},
jQueryMock,
function(output) { alert(output) },
function(output) { alert(output) });
}
Understand that the function passed to Observable.create() is the definition of the forEach() function for this Observable. In other words, all we have to do to define an Observable is to define its traversal function. Notice that although we pass three functions to the Observable's forEach() function, the function we pass to Observable.create() accepts only one value: an Observer. An Observer is just a triple containing three handlers:
The three handlers we pass to forEach() are packaged together into a single Observer object for convenience. Finally Observable.create() returns a function that defines the dispose() method of the Subscription object during Traversal. Like the Observer, the Observable.create() function creates the Subscription object for us and uses our function as the dispose() definition.
Now that we've built a version of the getJSON function that returns an Observable sequence, let's use it to improve our solution to the previous exercise...
Let's use the getJSON function that returns Observables, and the Observable.fromEvent() to complete the exercise we completed earlier.
// Combine videos and bookmarks 2
function(str, lesson) {
var fun = eval("(" + str + ")"),
getJSON = function(url) {
return Observable.create(function(observer) {
var subscribed = true;
jQueryMock.getJSON(url,
{
success:
function(data) {
// If client is still interested in the results, send them.
if (subscribed) {
// Send data to the client
observer.onNext(data);
// Immediately complete the sequence
observer.onCompleted();
}
},
error: function(ex) {
// If client is still interested in the results, send them.
if (subscribed) {
// Inform the client that an error occurred.
observer.onError(ex);
}
}
});
// Definition of the Subscription objects dispose() method.
return function() {
subscribed = false;
}
})
},
NOOP = function() {};
fun(
{
addEventListener: function(event, handler) {
window.setTimeout(handler, 200)
},
removeEventListener: NOOP
},
getJSON,
function(output) { alert(JSON.stringify(output)) },
function(output) { alert(output) });
}
Almost every workflow in an web application starts with an event, continues with an HTTP request, and results in a state change. Now we know how express the first two tasks elegantly.
When dealing with user input, there will be times when the user's input is too noisy, and will potentially clog your servers with extraneous requests. We want the ability to throttle the users's input so that if they interacting for one second, then we will get the user input. Let's say for example, the user clicks a button once too many times upon saving and we only want to fire after they've stopped for a second.
seq([1,,,2,,,3,,,4,,,5,,,6,,,]).throttle(1000 */ms */) ===
seq([,,,,,,,,3,,,,,,,,,,,6,,,]);
<<-- Click this to save your data
function(str, lesson) {
var $inputName = $('.inputName', lesson),
$savedValue = $('.savedValue', lesson);
var counter = 0;
var clicks = Rx.Observable.fromEvent($('.submitInputName'), 'click');
var code = eval("(" + str + ")")
var code = code(clicks, function() { return Rx.Observable.timer(1000) }, $inputName.val());
code
.subscribe(function () {
$savedValue.text('Data Saved!!!');
});
}
function (clicks, saveData, input) {
return clicks.
throttle(1000).
flatMap(function () {
return saveData(input.value);
})
}
Now that we know how to throttle input, let's take a look at another problem where throttling data is important...
One of the most common problems in web development is the autocomplete box. This seems like it should be an easy problem, but is actually quite challenging. For example, how do we throttle the input? How do we make sure we're not getting out or order requests coming back? For example if I type "react" then type "reactive" I want "reactive" to be my result, regardless of which actually returned first from the service.
In the example below, you will be receiving a sequence of key presses, a textbox, and a function when called returns an array of search results.
getSearchResultSet('react') === seq[,,,["reactive", "reaction","reactor"]]
keyPresses === seq['r',,,,,'e',,,,,,'a',,,,'c',,,,'t',,,,,]
function(str, lesson) {
var wordlist = window.wordlist;
wordlist.sort();
var searchText = function (text) {
var matched = wordlist.filter(function (x) {
return x.indexOf(text) === 0;
});
return Rx.Observable.returnValue(
matched.slice(0, 10)
).delay(1000);
};
var $inputName = $('.inputName', lesson),
$searchResults = $('.searchResultsForAutoComplete', lesson);
var clicks = Rx.Observable.fromEvent($inputName, 'keyup');
var code = eval("(" + str + ")")
var code = code(searchText, clicks, $inputName[0]);
code
.subscribe(function (results) {
var s = results.map(function (r) {
return '<li>' + r + '</li>';
});
$searchResults.html(s.join(''));
});
}
function (getSearchResultSet, keyPresses, textBox) {
var getSearchResultSets =
keyPresses.
map(function () {
return textBox.value;
}).
throttle(1000).
flatMap(function (text) {
return getSearchResultSet(text).takeUntil(keyPresses);
});
return getSearchResultSets;
}
Now that we're able to query with our throttled input, you'll still notice one slight problem. If you hit your arrow keys or any other non character key, the request will still fire. How do we prevent that?
You'll notice in the previous exercise that if you pressed your arrow keys while inside the textbox, the query will still fire, regardless of whether the text actually changed or not. How do we prevent that? The distinctUntilChanged filters out successive repetitive values.
seq([1,,,1,,,3,,,3,,,5,,,1,,,]).distinctUntilChanged() ===
seq([1,,,,,,,3,,,,,,,5,,,1,,,]);
function(str, lesson) {
var $inputName = $('.inputName', lesson),
$filtered = $('.filteredKeysByDistinct', lesson);
var keyups = Rx.Observable.fromEvent($inputName, 'keypress');
var isAlpha = function (x) {
return 'abcdefghijklmnopqrstuvwxyz'.indexOf(x.toLowerCase()) !== -1;
};
var code = eval("(" + str + ")")
var code = code(keyups, isAlpha);
code
.subscribe(function (text) {
$filtered.text(text);
});
}
function (keyPresses, isAlpha) {
return keyPresses.
map(function (e) { return String.fromCharCode(e.keyCode); }).
filter(function (character) { return isAlpha(character); }).
distinctUntilChanged().
scan('', function (stringSoFar, character) {
return stringSoFar + character;
});
}
Now that we know how to get only the distinct input, let's see how it applies to our autocomplete example...
In the previous version of the autocomplete box, there were two bugs
The example below is the same as above, but this time, fix the bugs!
getSearchResultSet('react') === seq[,,,["reactive", "reaction","reactor"]]
keyPresses === seq['r',,,,,'e',,,,,,'a',,,,'c',,,,'t',,,,,]
function(str, lesson) {
var wordlist = window.wordlist;
wordlist.sort();
var searchText = function (text) {
var matched = wordlist.filter(function (x) {
return x.indexOf(text) === 0;
});
return Rx.Observable.returnValue(
matched.slice(0, 10)
).delay(1000);
};
var $inputName = $('.inputName', lesson),
$searchResults = $('.searchResultsForAutoComplete', lesson);
var clicks = Rx.Observable.fromEvent($inputName, 'keyup');
var code = eval("(" + str + ")")
var code = code(searchText, clicks, $inputName[0]);
code
.subscribe(function (results) {
var s = results.map(function (r) {
return '<li>' + r + '</li>';
});
$searchResults.html(s.join(''));
});
}
function (getSearchResultSet, keyPresses, textBox) {
var getSearchResultSets =
keyPresses.
map(function () {
return textBox.value;
}).
throttle(1000).
distinctUntilChanged().
filter(function (s) { return s.length > 0; }).
flatMap(function (text) {
return getSearchResultSet(text).takeUntil(keyPresses);
});
return getSearchResultSets;
}
With just this little amount of code, we're able to produce a fully functioning autocomplete scenario. But there are other outstanding questions, such as error handling. How can we handle failure and retry if necessary?
You'll notice in the previous exercise that if you pressed your arrow keys while inside the textbox, the query will still fire, regardless of whether the text actually changed or not. How do we prevent that? The distinctUntilChanged filters out successive repetitive values.
seq([1,,,1,,,3,,,3,,,5,,,1,,,]).distinctUntilChanged() ===
seq([1,,,,,,,3,,,,,,,5,,,1,,,]);
function(str, lesson) {
var $inputName = $('.inputName', lesson),
$filtered = $('.filteredKeysByDistinct', lesson);
var keyups = Rx.Observable.fromEvent($inputName, 'keyup');
var isAlpha = function (x) {
return 'abcdefghijklmnopqrstuvwxyz'.indexOf(x.toLowerCase()) !== -1;
};
var code = eval("(" + str + ")")
var code = code(keyups, isAlpha);
code
.subscribe(function (text) {
$filtered.text(text);
});
}
function (keyPresses, isAlpha) {
return keyPresses.
map(function (e) { return String.fromCharCode(e.keyCode); }).
filter(function (character) { return isAlpha(character); }).
distinctUntilChanged().
scan('', function (stringSoFar, character) {
return stringSoFar + character;
});
}
Now that we know how to get only the distinct input, let's see how it applies to our autocomplete example...
Congratulations! You've made it this far, but you're not done. Learning is an on-going process. Go out and start using the functions you've learned in your day-to-day coding. Over time, I'll be adding more exercises to this tutorial. If you have suggestions for more exercises, send me a pull request!