Everybody likes to predict, but nobody likes being predictable, nor told what to do

The Netflix algorithm is in the news again.
The Science Behind the Netflix Algorithms That Decide What You’ll Watch Next

Netflix finds rating predictions are no longer as important, trumped by current viewing behaviour, i.e. what you are watching now. However, browsing through the comments, and again, you will see a generally negative reaction. Some people really hate being told what to watch, even if it's just a recommendation. Others say Netflix sucks, because it recommends things they've watched elsewhere. That sounds like a lack of understanding: if you don't tell Netflix you've watched something already, then how could it know?

As "big data" gets more media attention, it is reaching a wider audience who don't yet understand how algorithms work, but only know there are algorithms everywhere in their life, and it's scary to them. The lack of understanding seems to create fear and resentment.

LinkedIn and Facebook's recommendation systems for helping people find colleagues or friends they may know are generally well received, yet these film recommendation systems aren't. The difference between them might underline the success criteria of rolling out such recommendation systems.

Machine Learning in Movie Script Analysis Rouses Angry Reactions

An application of Machine Learning is covered in the news lately: movie script analysis.
Solving Equation of a Hit Film Script, With Data

They "compare the story structure and genre of a draft script with those of released movies, looking for clues to box-office success". However, the comments reveal that the general population (at least of the commenters) dislikes the concept for fear of anti-creativity.

Comments like these sum up the overall sentiment:

"Using old data to presage a current idea is both terrible and foolish. It is to writing what Denny's is to fine dining - mediocrity run wild."   

"Data crunchers will take the art out of everything. Paint-by-numbers."  

Ouch.
You be the judge whether this is a good application or not.

I tend to bias towards answers like this from the comments (sadly this was only 1 of 2 positive comments at the time of my reading; the other one was from the CEO of the script analysis business):

"I'm sure people have all sots of assumptions about what audiences like already. This data could be a tool to look deeper into these assumptions. Film makers have always wondered about consumer taste. It is a business. When commerce and art mix, there are inevitable compromises. This tool helps people see possible preferences based on past behavior. Information should never frighten us. It is how this information is applied that most deserves our attention." 

I think it also never helps the image of such machine learning practitioners when the journalist tries to paint him with an antagonist brush, such as "chain-smoking" and "taking a chug of Diet Dr Pepper followed by a gulp of Diet Coke and a drag on a Camel". Reminded me somewhat of another writer's writing style when covering analytics.

Introducing variability, flow and processes in a funny video to anyone

I'm leading on two variability & flow management projects at the hospital right now, and the terms "variability" and "flow" are certainly not something the medics hear much about. I needed a quick way of explaining what the projects are about, what these terms mean, and what kind of problems we are trying to resolve. A colleague suggested this video from the ever popular "I Love Lucy" TV series, episode "Chocolate Factory". It does a wonderful job of making people laugh, as well as acting out some strong parallels to a process, and the variability and flow within the process. Take a look at the video (it's a funny one!) and read on for the parallels to the operation of a hospital. The doctors, nurses and patients on my team all found the video not only hilarious but also made it clear to them what we are trying to do in the variability & flow management project.



The parallels:

• Process: the chocolates can be patients coming into the hospital 'conveyor belt'. Lucy and her friend Ethel can be the nurses, for example, (or the various clerks, doctors, pharmacists, radiographers, etc.) handling the patients, 'dressing' them up or giving them care to make them better so they can go on to the next hospital professionals, i.e. the pharmacists to receive medications in the next room down the conveyor belt. The patient traveling through the conveyor belt is a process. Similarly, Lucy and Ethel picking up the chocolate from the conveyor belt, taking the wrapping paper, wrapping up the chocolate nicely, placing the wrapped chocolate back onto the conveyor belt, and returning to the position to be ready for the next chocolate, is a process. Lucy and Ethel are the 'servers' within the process. The things they do to the chocolate are 'steps' within the process. The girls feeding the chocolate onto the conveyor belt for Lucy & Ethel in the previous room are the servers of the upstream process to Lucy & Ethel's wrapping process. Similarly, the girls boxing the chocolates in the next room, perhaps, are the servers of the downstream process.
  
• Flow: The chocolates going through the Lucy & Ethel's wrapping process is a flow.
• Variability: The speed the chocolates are placed onto the conveyor belt is a source of variability, because the speed changes, and so is the speed that Lucy & Ethel wraps the chocolate, as they have very different styles of wrapping. This results in the variable speed of the wrapped chocolates flowing out of the Lucy & Ethel wrapping process.
• Queuing & waits - When Lucy & Ethel were running behind and when they started to collect the chocolates in front of them and in their hats, so that they can wrap them later, that's queuing the chocolates, and those chocolates are experiencing 'waits'.
  
• Mis-communication: When the supervisor meanie lady shouted to the upstream girls to "let it roll" and nothing happened so she had to go to the previous room to sort it out, that's mis-communication or signal failure. :)

The video also shows some classic examples of problems around processes:

• Isolated processes and working in silos – what is going on 'upstream' and 'downstream' is absolutely unknown to Lucy & Ethel.
  
• Lack of issue escalation procedure - when the chocolates are coming too fast for Lucy & Ethel to handle, they had no way of letting the upstream or the manager know (but of course, the meanie supervisor lady didn't allow them to leave one chocolate behind).
  
• Performance management - the meanie supervisor lady did not have realistic expectations on Lucy & Ethel's performance, or maybe she simply didn't have any clue about the variability of the sometimes very high demand placed on Lucy & Ethel from the upstream.
  
• Reactionary management - When the supervisor lady came into the room and saw that Lucy & Ethel had no chocolates on the belt and therefore ordered the upstream to feed faster is very reactionary. She simply made the decision based on one observation / data point, and did not ask any questions about why it is that way.

Hope you find the video useful in your work as well. I'm sure you can draw parallels to other industries aside from health care. Please feel free to share it with me. Things are often best explained by humour.

Operations Research & Movie Scripts

What has OR got to do with movie scripts? What has the geek world got to do with the Sunset Boulevard in the Hollywood hills? Well well...read on.

Warning to studio readers — two marketing professors at the Wharton School could very well put you out of a job.

Actually, Z. John Zhang and Jehoshua Eliashberg (plus a bevy of co-authors) claim that their goal is merely to augment your special talents, not replace them. But the paper they published in Management Science magazine in June called 'From Story Line to Box Office: A New Approach for Green-Lighting Movie Scripts' establishes a statistical model for analyzing screenplays and predicting whether a resulting movie will be successful at the box office. Which, if accurate, would render your silly personal judgments obsolete.

Greenlighting, or putting a screenplay into active production, relies mostly on the subjective intuition of readers and executives (plus a studio calculus derived from the budget and the past record of the film's genre and potential cast). It's a system that can produce, shall we say, spotty results. Zhang and Eliashberg hope to take some of the guesswork out of it. Their model combines textual analysis (paragraph construction, frequency and distribution of words, etc.) with structural analysis (a clear premise, a surprise ending, and the like) using 22 yes-or-no queries that are posed and then cross-referenced.

— Los Angeles Times