Will I "Really Like" this Movie?

Navigating Movie Website Ratings to Select More Enjoyable Movies

Archive for the category “Cinemascore”

Why Does CinemaScore Leave Out So Many Good Movies When Issuing Grades?

The 2017 Academy Awards will be forever remembered as the year that La La Land was awarded Best Picture for about a minute before they discovered that Moonlight was the actual winner. Those two movies have something else in common. Neither movie received a CinemaScore grade despite, arguably, being the top two movies of 2016.

I’m thinking about this issue this week because three movies with Oscar buzz, StrongerBattle of the Sexes, and Victoria and Abdul,  went into limited release last weekend. None of them were graded by Cinemascore. There is a valid reason for this but that doesn’t make it any less disappointing to movie pre-screeners like myself.

For me, Cinemascore is appealing because it measures only opening night reaction. Most people who go to the opening night of a movie are there because they really want to see that movie. The pre-release buzz has grabbed their attention to such an extent that they can’t wait to see it. They walk into an opening night movie expecting to love it. When they walk out of the movie and respond to CinemaScore’s survey they are probably grading based on expectations. So, when a movie receives an “A” from Cinemascore, it tells us that the movie lives up to the hype. Anything less than that suggests that the movie experience was less than they expected.

CinemaScore gets stuck when it comes to movies that are released in a limited number of theaters prior to them being released widely in most theaters. CinemaScore samples locations throughout the U.S. and Canada to establish a credible unbiased sample. When a movie goes into limited release, it is released in some of their sample locations but not in most of their sample locations. Last weekend, Stronger was released in 573 theaters, Battle of the Sexes was released in 21 theaters, and Victoria and Abdul was released in 4 theaters. To provide some perspective, Kingsman: The Golden Circle opened in 4,003 theaters last weekend and earned a “B+” grade from CinemaScore. When Stronger and Battle of the Sexes goes into wide release tomorrow, does word of mouth reaction from moviegoers who’ve seen the movie in the last week disturb the integrity of any sample taken this weekend? When Victoria and Abdul goes into wide release on October 6, is its release into just 4 theaters last weekend sufficiently small to not taint the sample? I don’t know the answers to these questions. I’ll be looking to see if these movies get graded when they go into wide release. In Box Office Mojo’s article on last weekend’s box office performance they indicate that CinemaScore graded Stronger an “A-” even though it hasn’t been officially posted on their website. Perhaps they are waiting to post it after wide release?

I understand why grades don’t exist on CinemaScore for many limited release movies. I understand the importance of data integrity in the creation of a credible survey. I will just observe, though, that in this age of social media, using limited movie releases to build pre-wide release momentum for quality movies is an increasingly viable strategy. Just this week, A24, the studio behind the rise of Moonlight last year, decided to put their dark horse candidate this year, Lady Bird, into limited release on November 3rd after it emerged from the Telluride and Toronto film festivals with a 100% Fresh grade from Rotten Tomatoes. CinemaScore may be facing the prospect of an even broader inventory of ungraded top tier movies than it does today. It will be interesting to see how they respond to this challenge, if at all.

 

Advertisements

Cinemascore Is a “Really Like” Indicator

Those of  you who checked in on Monday to see the updated Objective Top Ten may have noticed that Cinemascore grades were included in the information provided for each movie. If you were particularly observant, you might have also noticed that the bar at the top of the page, which includes links to the movie ratings websites I use, now includes the link to Cinemascore. All of which means that Cinemascore grades are now officially included in the “really like” algorithm.

As I’ve mentioned before, the folks at Cinemascore have been surveying moviegoers as they leave the theater since 1978. They limit their surveys to the three or four movies each week that they suspect will do the best at the box office. This limited sample of movies represents around 40% of the movies in my database, which is a plenty big enough sample for me to work with.

The other factor in using the data is that the grades seem to line up with their “really like” potential.

Cinemascore Database Results
Grade Database Total Graded “Really Like” %
A+ 51 82%
A 201 80%
A- 212 73%
B+ 156 58%
B 117 50%
B- 52 42%
C+ 21 33%
C 9 11%
C- 4 0%
D+ 1 0%
D 0 0%
D- 1 0%

The “really like” percentages follow a logical progression by grade. Now, because the sample sizes for each grade are relatively small, I’ve had to group the grades into two buckets that represent above average Cinemascore grades and below average grades.

All Grades               825 65%
A+,A, A-               464 77%
All Other               361 50%

This suggests that a good Cinemascore grade is an A- or better (Talk about grade inflation!!). The statistical gap between the two buckets is great enough for it to be an effective differentiator of “really like” movies.

The practical effect of this change is that the Objective Top Ten will be more weighted to mainstream movies. Independent movies are less likely to be surveyed by Cinemascore for example. On the other hand, a movie like Hidden Figures, which already benefitted from high IMDB and Rotten Tomatoes scores, now adds a Cinemascore grade of A+. This makes the model even more confident that this movie is a “really like” movie and as a result the probability % for the movie goes higher, lifting it to the top of the list.

I’m excited about this enhancement and I hope you will be too.

***

I mentioned last week that I had my eye on two movies, The Beguiled and The Big Sick. I jumped the gun a little bit because both of these movies only went into limited release last Friday. The Beguiled goes into wide release tomorrow, while The Big Sick goes into wide release on July 14th. Baby Driver, which went into wide release yesterday, is another new movie that looks good from the early indicators.

Next Monday the Objective Top Ten will become the Objective Top Fifteen (just in case you needed something else to look forward to this weekend). Have a “Really Like” 4th of July weekend at the movies!

Leave Mummy Out of Your Father’s Day Plans

One of the goals of this blog is to make sure that you are aware of the internet tools that are out there to protect you from wasting your time on blockbusters like The Mummy. While it had a disappointing opening in the U.S., moviegoers still shelled out an estimated $32.2 million at the box office last weekend for this bad movie. Overseas it met its blockbuster expectations with a box office of $141.8 million. However, if you were really in the mood for a horror genre movie a better choice, but not a sure thing, might have been It Comes At Night which had a more modest U.S. box office of $6 million.

As a general rule, I won’t go to a movie on its opening weekend. I prefer to get at least a weekend’s worth of data. But if you just have to see a movie on its opening weekend here are a couple of hints. First, if you are seeing the movie on its opening Friday, the most reliable indicator is Rotten Tomatoes. Most critics have released their reviews before the day of the movie’s release. The Rotten Tomatoes rating on the movie’s release date is a statistically mature evaluation of the movie. It won’t change much after that day.

If you are going to the movies on the Saturday of opening weekend, you can add Cinemascore to the mix. I’ve blogged about this tool before. This grade is based on feedback moviegoers provide about the movie as they are leaving the theater. The grade is posted on the Saturday after the Friday release.

Finally, by Sunday IMDB will produce a pretty good, though slightly inflated, average rating for the movie.

The comparison of these three checkpoints for The Mummy and for It Comes At Night might’ve been helpful to those who thought they were in for a “really like” movie experience.

Rotten Tomatoes IMDB Avg. Rating Cinemascore Grade
The Mummy Rotten (17%) 5.9 B-
It Comes At Night Certified Fresh (86%) 7.2 D

While the Cinemascore grade of D for It Comes At Night would keep me away from opening weekend for both movies, if I had to see one, it wouldn’t be The Mummy.

Here’s the data behind my reasoning. For IMDB, the breakpoint between a movie with a good chance that I will “really like” it and one that I probably won’t like is an average rating of 7.2. Movies with a 7.2 IMDB average rating of 7.2 or higher I “really like” 63.3% of the time. Movies with an IMDB rating less than 7.2 I “really like” 43.3% of the time. Turning to Rotten Tomatoes, Movies that are Certified Fresh I “really like” 68% of the time. These “really like” percentages drop to 49.6% for movies that are Fresh and 37.5% for movies that are Rotten. So absent any information based on my own personal tastes, I won’t go to the movieplex to watch a movie that isn’t graded Certified Fresh by Rotten Tomatoes and has an IMDB Rating 7.2 or higher. That doesn’t mean that there aren’t any movies out there that don’t meet that criteria that I wouldn’t “really like”. The movie may be in a genre that appeals to me which might provide some tolerance for a little less quality. That being said, the odds that I’ll “really like” a low rated movie are less than 50/50.

I should probably explore the potential of adding Cinemascore to the objective probability factors I use in developing “really like” probabilities. To date, though, I don’t have any Cinemascore data . I don’t yet have a feel for its “really like” reliability. For now, I just use it as another piece of data that might tip me one way or the other if I’m on the fence about a new movie.

Enjoy Father’s Day but stay away from Mummy.

Cinemascore Is For Opening Weekend, but Beware of Grade Inflation

We have just endured another Presidential Primary season where every tea leaf was micro-analyzed and every phrase parsed to death. One of the primary tools of the political pundits is the exit poll. In key districts across the primary State, pollsters await voters as they exit the polling place to determine who the voters were pinning their hopes on to lead the free world at that very moment and why. The exit poll fills our insatiable desire for instant feedback for what we’re collectively thinking.

The movie industry has its own version of the exit poll, Cinemascore. In the pre-IMDB days of 1978, the movie industry had the same concerns with critics that they have with Rotten Tomatoes today. The industry felt critics had too much influence with the viewing public. Cinemascore filled this perceived need to balance the sway of critics by measuring the opening night reaction to a movie from moviegoers who were walking out of the theater. Like political exit polls, the theaters polled in the survey were specifically selected to provide a cross section, regionally and demographically, of the viewing public in the U.S. and Canada. Participants in the survey answer six questions about the movie they’ve just watched including the assignment of a grade from A to F.

By going to the website linked above you can view the average grade from the surveys given to recent major movie releases. You can also type in a movie title released after 1978 to see that movie’s average grade. With a paid subscription you can enter the website and presumably access results from the other five questions surveyed. Not all movies are surveyed, only those considered major releases.

How useful are these grades? Well, if you absolutely can’t wait to see a movie, but you can hold off until Saturday night, they can be quite useful. The survey sample is representative of moviegoers like you. I would expect that most attendees of an opening night movie have a high degree of interest in the movie, just like you. On the other hand, if your decision to attend an opening weekend movie is more casually made, Cinemascore could be deceiving.

The 24 recent movie releases currently displayed on the Cinemascore Home Page are ranked below by grade with the accompanying IMDB rating results:

Cinemascore
Recent Movie Results
Movie Cinemascore # IMDB Votes IMDB Avg. Rating
ME BEFORE YOU A                6,217 7.9
CAPTAIN AMERICA: CIVIL WAR A            229,127 8.3
GOD’S NOT DEAD 2 A                3,809 3.3
JUNGLE BOOK, THE A              84,945 7.8
TEENAGE MUTANT NINJA TURTLES: OUT OF THE SHADOWS A-                8,013 6.5
X-MEN: APOCALYPSE A-            102,507 7.4
ALICE THROUGH THE LOOKING GLASS A-              12,479 6.4
CONJURING 2, THE A-                7,238 8.4
NOW YOU SEE ME 2 A-                3,696 7.2
BARBERSHOP: THE NEXT CUT A-                2,099 6.1
MY BIG FAT GREEK WEDDING 2 A-                8,083 6.2
ANGRY BIRDS MOVIE, THE B+              13,128 6.4
WARCRAFT B+              45,628 7.8
HUNTSMAN: WINTER’S WAR, THE B+              23,522 6.2
MONEY MONSTER B+              11,349 6.8
MOTHER’S DAY B+                3,643 5.4
BATMAN V SUPERMAN: DAWN OF JUSTICE B            299,641 7.0
KEANU B                7,507 6.6
NEIGHBORS 2: SORORITY RISING B              16,437 6.1
POPSTAR: NEVER STOP NEVER STOPPING B                2,815 7.4
RATCHET AND CLANK B                1,778 6.1
CRIMINAL B-                5,452 6.4
NICE GUYS, THE B-              23,900 7.8
DARKNESS, THE C                1,901 4.1

If an IMDB rating of 7.3 or higher is considered an above average rating, then only a Cinemascore of A is solidly reinforced by the IMDB average ratings. Of the 7 movies  receiving an A- grade only X-Men: Apocalypse and The Conjuring 2 were considered above average by IMDB voters. A Cinemascore of A- may not translate favorably when the more general audience begins to view the film. If on the other hand, you are really into Christian movies and you were really looking forward to God’s Not Dead 2, Cinemascore is going to be a better indicator of the quality of the movie than IMDB, whose voters may not be representative of your taste in movies.

Cinemascore was created before we had sites like IMDB. It still has its use for “must see” opening weekend moviegoers and movies for unique tastes. Once you get past opening weekend, however, IMDB is probably a better tool for word of mouth feedback.

***

6/17/2016

I’ve entered my final estimate for Finding Dory this morning. The early indicators are that this will be a critical and box office success. I’ve forecasted it will be a “really like” of 85%.

Post Navigation