Extracting JSON data from websites and public APIs with R

JSON web scraping NFL espnscrapeR

tidyr + jsonlite are magical.

Thomas Mock https://twitter.com/thomas_mock
12-13-2020

Finding JSON Sources

I’ve covered some strategies for parsing JSON with a few methods in base R and/or tidyverse in a previous blog post. I’d like to go one step up in the chain, and talk about pulling raw data/JSON from sites. While having a direct link to JSON is common, in some situations where you’re scraping JavaScript fed by APIs the raw data source is not always as easy to find.

I have three examples for today:
- FiveThirtyEight 2020 NFL Predictions
- ESPN Win Percentage/play-by-play (embedded JSON)
- ESPN Public API

Web vs Analysis

Most of these JSON data sources are intended to be used with JavaScript methods, and have not been oriented to a “flat” data style. This means the JSON has lots of separations of the data for a specific use/purpose inside the site, and efficient singular representations of each data in JSON storage as opposed to normalized data with repeats in a dataframe. While extreme detail is out of scope for this blogpost, JSON is structured as a “collection of name/value pairs” or a “an ordered list of values”. This means it is typically represented in R as repeated lists of list elements, where the list elements can be named lists, vectors, dataframes, or character strings.

Alternatively typically data for analysis is usually most useful as a normalized rectangle eg a dataframe/tibble. “Under the hood a data frame is a list of equal length vectors” per Advanced R.

One step further is tidy data which is essentially “3rd normal form”. Hadley goes into more detail in his “Tidy Data” publication. The takeaway here is that web designers are optimizing for their extremely focused interactive JavaScript apps and websites, as opposed to novel analyses that we often want to work with. This is often why there are quite a few steps to “rectangle” a JSON.

An aside on Subsetting

Subsetting in R is done many different ways, and Hadley Wickham has an entire chapter dedicated to this in Advanced R. It’s worth reading through that chapter to better understand the nuance, but I’ll provide a very brief summary of the options.

# a VERY basic list of named elements
car_list <- list(manufacturer = "Honda", vehicle = "Civic", year = 2020)

“Subsetting a list works in the same way as subsetting an atomic vector. Using [ always returns a list; [[ and $, … let you pull out elements of a list.”

When working with lists, you can typically use $ and [[ interchangeably to extract single list elements by name. [[ requires exact matching whereas $ allows for partial matching, so I typically prefer to use [[. To extract by location from base R you need to use [[.

purrr functions pluck() and chuck() implement a generalised form of [[ that allow you to index deeply and flexibly into data structures. pluck() consistently returns NULL when an element does not exist, chuck() always throws an error in that case."

So in short, you can use $, [[ and pluck/chuck in many of the same ways. I’ll compare all the base R and purrr versions below (all should return “Honda”).

# $ subsets by name
car_list$manufacturer
[1] "Honda"
# notice partial match
car_list$man
[1] "Honda"
# [[ requires exact match or position
car_list[["manufacturer"]]
[1] "Honda"
car_list[[1]]
[1] "Honda"
# pluck and chuck provide a more strict version of [[
# and can subset by exact name or position
purrr::pluck(car_list, "manufacturer")
[1] "Honda"
purrr::pluck(car_list, 1)
[1] "Honda"
purrr::chuck(car_list, "manufacturer")
[1] "Honda"
purrr::chuck(car_list, 1)
[1] "Honda"

For one more warning of partial name matching with $, where we now have a case of two elements with similar names see below:

car_list2 <- list(manufacturer = "Honda", vehicle = "Civic", manufactured_year = 2020)

# partial match throws a null
car_list2$man
NULL
# exact name returns actual elements
car_list2$manufacturer
[1] "Honda"

An aside on JavaScript

If we dramatically oversimplify JavaScript or their R-based counterparts htmlwidgets, they are a combination of some type of JSON data and then functions to display or interact with that data.

We can quickly show a htmlwidget example via the fantastic reactable R package.

library(reactable)

table_ex <- mtcars %>% 
  select(cyl, mpg, disp) %>% 
  reactable()

table_ex

That gives us the power of JavaScript in R! However, what’s going on with this function behind the scenes? We can extract the dataframe that has now been represented as a JSON file from the htmlwidget!

table_data <- table_ex[["x"]][["tag"]][["attribs"]][["data"]]

table_data %>% class()
[1] "json"

This basic idea, that the data is embedded as JSON to fill the JavaScript app can be further applied to web-based apps! We can use a similar idea to scrape raw JSON or query a web API that returns JSON from a site.

FiveThirtyEight

FiveThirtyEight publishes their ELO ratings and playoff predictions for the NFL via a table at projects.fivethirtyeight.com/2020-nfl-predictions/. They are also kind enough to post this data as a download publicly! However, let’s see if we can “find” the data source feeding the JavaScript table.

rvest

We can try our classical rvest based approach to scrape the HTML content and get back a table. However, the side effect of this is we’re returning the literal data with units, some combined columns, and other formatting. You’ll notice that all the columns show up as character and this introduces a lot of other work we’d have to do to “clean” the data.

library(xml2)
library(rvest)

url_538 <- "https://projects.fivethirtyeight.com/2020-nfl-predictions/"

raw_538_html <- read_html(url_538)

raw_538_table <- raw_538_html %>% 
  html_node("#standings-table") %>% 
  html_table(fill = TRUE) %>% 
  janitor::clean_names() %>% 
  tibble()

raw_538_table %>% glimpse()
Rows: 34
Columns: 16
$ x                               <chr> "", "elo with top qbelo rat…
$ x_2                             <chr> "", "1-week change", "", ""…
$ x_3                             <chr> "", "current qb adj.", "", …
$ x_4                             <lgl> NA, NA, NA, NA, NA, NA, NA,…
$ x_5                             <chr> "", "team", "ChiefsChiefs11…
$ x_6                             <chr> "", "division", "AFC West",…
$ playoff_chances                 <chr> "", "make playoffs", "✓", "…
$ playoff_chances_2               <chr> "", "win division", ">99%",…
$ playoff_chances_3               <chr> "", "1st-round bye", "61%",…
$ playoff_chances_4               <chr> "", "win super bowl", "30%"…
$ playoff_chances_5               <chr> "", "Week 14 matchupWk 14",…
$ playoff_chances_6               <chr> "", "Week 15 matchupWk 15",…
$ playoff_chances_7               <chr> NA, "Week 16 matchupWk 16",…
$ choose_your_own_results_reset   <chr> NA, "Week 17 matchupWk 17",…
$ choose_your_own_results_reset_2 <lgl> NA, NA, NA, NA, NA, NA, NA,…
$ choose_your_own_results_reset_3 <lgl> NA, NA, NA, NA, NA, NA, NA,…

Inspect + Network

Alternatively, we can Right Click + inspect the site, go to the Network tab, reload the site and see what sources are loaded. Again, FiveThirtyEight is very kind and essentially just loads the JSON as data.json.

I have screenshots below of each item, and the below is a short video of the entire process.

Network Tab

We can click over to the Network Tab after inspecting the site

Network Tab Reloaded

We need to reload the web page to find sources

Network Tab Data

We can examine specific elements by clicking on them, which then shows us JSON!



In our browser inspect, we can see the structure, and that it has some info about games, QBs, and forecasts. This looks like the right dataset! You can right click on data.json and open it in a new page. The url is https://projects.fivethirtyeight.com/2020-nfl-predictions/data.json, and note that we can adjust the year to get older or current data. So https://projects.fivethirtyeight.com/2019-nfl-predictions/data.json returns the data for 2019, and you can go all the way back to 2016! 2015 also exists, but with a different JSON structure, and AFAIK they don’t have data before 2015.

Read the JSON

Now that we have a JSON source, we can read it into R with jsonlite. By using the RStudio viewer or listviewer::jsonedit() we can take a look at what the overall structure of the JSON.

library(jsonlite)

raw_538_json <- fromJSON("https://projects.fivethirtyeight.com/2020-nfl-predictions/data.json", simplifyVector = FALSE)

raw_538_json %>% str(max.level = 1)
List of 9
 $ archie                :List of 24
 $ clinches              :List of 31
 $ distances             :List of 32
 $ games                 :List of 269
 $ pageconfig            :List of 20
 $ playoff_qb_adjustments:List of 128
 $ qbs                   :List of 86
 $ urls                  :List of 2
 $ weekly_forecasts      :List of 2

Don’t forget that the RStudio Viewer also gives you the ability to export the base R code to access a specific component of the JSON!

Screenshot of RStudio Viewer

Which gives us the following code:

raw_538_json[["weekly_forecasts"]][["forecasts"]][[1]][["types"]][["elo"]][[1]]

ex_538_data <- raw_538_json[["weekly_forecasts"]][["forecasts"]][[1]][["types"]][["elo"]][[1]]

ex_538_data %>% str()
List of 29
 $ conference           : chr "NFC"
 $ current_losses       : int 6
 $ current_ties         : int 0
 $ current_wins         : int 6
 $ division             : chr "NFC North"
 $ elo                  : num 1536
 $ losses               : num 8.09
 $ make_conference_champ: num 0.0437
 $ make_divisional_round: num 0.14
 $ make_playoffs        : num 0.424
 $ make_superbowl       : num 0.0177
 $ name                 : chr "MIN"
 $ point_diff           : num -13.3
 $ points_allowed       : num 411
 $ points_scored        : num 398
 $ rating               : num 1524
 $ rating_current       : num 1537
 $ rating_top           : num 1537
 $ seed_1               : int 0
 $ seed_2               : num 0.00016
 $ seed_3               : num 0.00486
 $ seed_4               : num 1e-04
 $ seed_5               : num 0.0219
 $ seed_6               : num 0.164
 $ seed_7               : num 0.233
 $ ties                 : num 0.0056
 $ win_division         : num 0.00512
 $ win_superbowl        : num 0.00688
 $ wins                 : num 7.91

We can also play around with listviewer.

raw_538_json %>% 
  listviewer::jsonedit()

Since these are unique list elements, we can turn it into a dataframe! This is the current projection for Minnesota.

data.frame(ex_538_data) %>% glimpse()
Rows: 1
Columns: 29
$ conference            <chr> "NFC"
$ current_losses        <int> 6
$ current_ties          <int> 0
$ current_wins          <int> 6
$ division              <chr> "NFC North"
$ elo                   <dbl> 1536.008
$ losses                <dbl> 8.08856
$ make_conference_champ <dbl> 0.04372
$ make_divisional_round <dbl> 0.1396
$ make_playoffs         <dbl> 0.4239
$ make_superbowl        <dbl> 0.01768
$ name                  <chr> "MIN"
$ point_diff            <dbl> -13.2957
$ points_allowed        <dbl> 410.814
$ points_scored         <dbl> 397.5183
$ rating                <dbl> 1523.734
$ rating_current        <dbl> 1536.777
$ rating_top            <dbl> 1536.777
$ seed_1                <int> 0
$ seed_2                <dbl> 0.00016
$ seed_3                <dbl> 0.00486
$ seed_4                <dbl> 1e-04
$ seed_5                <dbl> 0.02186
$ seed_6                <dbl> 0.16394
$ seed_7                <dbl> 0.23298
$ ties                  <dbl> 0.0056
$ win_division          <dbl> 0.00512
$ win_superbowl         <dbl> 0.00688
$ wins                  <dbl> 7.90584

Parse the JSON

Ok so we’ve found at least one set of data that is pretty dataframe ready, let’s clean it all up in bulk! I’m most interested in the weekly_forecasts data, so let’s start there.

raw_538_json$weekly_forecasts %>% str(max.level = 1)
List of 2
 $ last_updated: chr "2020-12-13T15:13:49.762Z"
 $ forecasts   :List of 15

Ok so last_updated is good to know, but not something I need right now. Let’s go one step deeper into forecasts.

raw_538_json$weekly_forecasts$forecasts %>% str(max.level = 1)
List of 15
 $ :List of 3
 $ :List of 3
 $ :List of 3
 $ :List of 3
 $ :List of 3
 $ :List of 3
 $ :List of 3
 $ :List of 3
 $ :List of 3
 $ :List of 3
 $ :List of 3
 $ :List of 3
 $ :List of 3
 $ :List of 3
 $ :List of 3

Ok now we have a list of 14 lists. This may seem not helpful, BUT remember that as of 2020-12-12, we are in Week 14 of the NFL season! So this is likely 1 list for each of the weekly forecasts, which makes sense as we are in weekly_forecasts$forecasts!

At this point, I think I’m at the right data, so I’m going to take the list and put it in a tibble via tibble::enframe().

raw_538_json$weekly_forecasts$forecasts %>% 
  enframe()
# A tibble: 15 x 2
    name value           
   <int> <list>          
 1     1 <named list [3]>
 2     2 <named list [3]>
 3     3 <named list [3]>
 4     4 <named list [3]>
 5     5 <named list [3]>
 6     6 <named list [3]>
 7     7 <named list [3]>
 8     8 <named list [3]>
 9     9 <named list [3]>
10    10 <named list [3]>
11    11 <named list [3]>
12    12 <named list [3]>
13    13 <named list [3]>
14    14 <named list [3]>
15    15 <named list [3]>

We need to separate the list items out, so we can try unnest_auto() to see if tidyr can parse the correct structure. Note that unnest_auto works and tells us we could have used unnest_wider().

raw_538_json$weekly_forecasts$forecasts %>% 
  enframe() %>% 
  unnest_auto(value)
# A tibble: 15 x 4
    name last_updated              week types           
   <int> <chr>                    <int> <list>          
 1     1 2020-12-13T15:13:49.762Z    14 <named list [2]>
 2     2 2020-12-10T16:21:56.731Z    13 <named list [2]>
 3     3 2020-12-06T15:29:51.523Z    12 <named list [2]>
 4     4 2020-11-25T23:43:57.380Z    11 <named list [2]>
 5     5 2020-11-18T16:35:55.623Z    10 <named list [2]>
 6     6 2020-11-12T20:35:59.178Z     9 <named list [2]>
 7     7 2020-11-05T22:38:58.087Z     8 <named list [2]>
 8     8 2020-10-29T17:24:00.045Z     7 <named list [2]>
 9     9 2020-10-22T19:47:36.133Z     6 <named list [2]>
10    10 2020-10-18T15:20:01.392Z     5 <named list [2]>
11    11 2020-10-08T22:14:12.455Z     4 <named list [2]>
12    12 2020-10-01T13:39:13.805Z     3 <named list [2]>
13    13 2020-09-24T14:00:27.562Z     2 <named list [2]>
14    14 2020-09-15T05:29:29.158Z     1 <named list [2]>
15    15 2020-09-09T18:00:47.381Z     0 <named list [2]>

We can keep going on the types list column! Note that as unnest_auto() tells us “what” to do, I’m going to replace it with the appropriate function.

raw_538_json$weekly_forecasts$forecasts %>% 
  enframe() %>% 
  unnest_wider(value) %>% # changed per recommendation
  unnest_auto(types)
# A tibble: 15 x 5
    name last_updated              week elo         rating     
   <int> <chr>                    <int> <list>      <list>     
 1     1 2020-12-13T15:13:49.762Z    14 <list [32]> <list [32]>
 2     2 2020-12-10T16:21:56.731Z    13 <list [32]> <list [32]>
 3     3 2020-12-06T15:29:51.523Z    12 <list [32]> <list [32]>
 4     4 2020-11-25T23:43:57.380Z    11 <list [32]> <list [32]>
 5     5 2020-11-18T16:35:55.623Z    10 <list [32]> <list [32]>
 6     6 2020-11-12T20:35:59.178Z     9 <list [32]> <list [32]>
 7     7 2020-11-05T22:38:58.087Z     8 <list [32]> <list [32]>
 8     8 2020-10-29T17:24:00.045Z     7 <list [32]> <list [32]>
 9     9 2020-10-22T19:47:36.133Z     6 <list [32]> <list [32]>
10    10 2020-10-18T15:20:01.392Z     5 <list [32]> <list [32]>
11    11 2020-10-08T22:14:12.455Z     4 <list [32]> <list [32]>
12    12 2020-10-01T13:39:13.805Z     3 <list [32]> <list [32]>
13    13 2020-09-24T14:00:27.562Z     2 <list [32]> <list [32]>
14    14 2020-09-15T05:29:29.158Z     1 <list [32]> <list [32]>
15    15 2020-09-09T18:00:47.381Z     0 <list [32]> <list [32]>

We now have a list of 32 x 14 weeks. There are 32 teams so we’re most likely at the appropriate depth and can go longer vs wider now. We can also see that name/week don’t align so let’s drop name, and we can use unchop() to increase the length of the data for elo and rating at the same time.

raw_538_json$weekly_forecasts$forecasts %>% 
  enframe() %>% 
  unnest_wider(value) %>% 
  unnest_wider(types) %>% # Changed per recommendation
  unchop(cols = c(elo, rating)) %>% 
  select(-name)
# A tibble: 480 x 4
   last_updated              week elo               rating           
   <chr>                    <int> <list>            <list>           
 1 2020-12-13T15:13:49.762Z    14 <named list [29]> <named list [29]>
 2 2020-12-13T15:13:49.762Z    14 <named list [29]> <named list [29]>
 3 2020-12-13T15:13:49.762Z    14 <named list [29]> <named list [29]>
 4 2020-12-13T15:13:49.762Z    14 <named list [29]> <named list [29]>
 5 2020-12-13T15:13:49.762Z    14 <named list [29]> <named list [29]>
 6 2020-12-13T15:13:49.762Z    14 <named list [29]> <named list [29]>
 7 2020-12-13T15:13:49.762Z    14 <named list [29]> <named list [29]>
 8 2020-12-13T15:13:49.762Z    14 <named list [29]> <named list [29]>
 9 2020-12-13T15:13:49.762Z    14 <named list [29]> <named list [29]>
10 2020-12-13T15:13:49.762Z    14 <named list [29]> <named list [29]>
# … with 470 more rows

We now have 14 weeks x 32 teams (448 rows), along with last_updated, week, elo and rating data. We can use unnest_auto() on the elo column to see what’s the next step. Rating is duplicated so there’s been name repair to avoid duplicated names. We get the following warning that tells us this has occurred.

* rating -> rating...18
* rating -> rating...32

You’ll see that I’ve done unnest_auto() on both elo and rating...32 (the renamed rating list column). If you look closely at the names, we can also see that there is duplication of the names for MANY of the columns. A tricky part is that elo/rating each have a LOT of overlap, and are most appropriate as separate data frames that could be stacked if desired.

elo_raw <- raw_538_json$weekly_forecasts$forecasts %>% 
  enframe() %>% 
  unnest_wider(value) %>% 
  unnest_wider(types) %>% # Changed per recommendation
  unchop(cols = c(elo, rating)) %>% 
  select(-name) %>% 
  unnest_auto(elo) %>% 
  unnest_auto(rating...32)

elo_raw %>% 
  names()
 [1] "last_updated"               "week"                      
 [3] "conference...3"             "current_losses...4"        
 [5] "current_ties...5"           "current_wins...6"          
 [7] "division...7"               "elo...8"                   
 [9] "losses...9"                 "make_conference_champ...10"
[11] "make_divisional_round...11" "make_playoffs...12"        
[13] "make_superbowl...13"        "name...14"                 
[15] "point_diff...15"            "points_allowed...16"       
[17] "points_scored...17"         "rating...18"               
[19] "rating_current...19"        "rating_top...20"           
[21] "seed_1...21"                "seed_2...22"               
[23] "seed_3...23"                "seed_4...24"               
[25] "seed_5...25"                "seed_6...26"               
[27] "seed_7...27"                "ties...28"                 
[29] "win_division...29"          "win_superbowl...30"        
[31] "wins...31"                  "conference...32"           
[33] "current_losses...33"        "current_ties...34"         
[35] "current_wins...35"          "division...36"             
[37] "elo...37"                   "losses...38"               
[39] "make_conference_champ...39" "make_divisional_round...40"
[41] "make_playoffs...41"         "make_superbowl...42"       
[43] "name...43"                  "point_diff...44"           
[45] "points_allowed...45"        "points_scored...46"        
[47] "rating...47"                "rating_current...48"       
[49] "rating_top...49"            "seed_1...50"               
[51] "seed_2...51"                "seed_3...52"               
[53] "seed_4...53"                "seed_5...54"               
[55] "seed_6...55"                "seed_7...56"               
[57] "ties...57"                  "win_division...58"         
[59] "win_superbowl...59"         "wins...60"                 

Let’s try this again, with the knowledge that elo and rating should be treated separately for now. Since they have the same names, we can also combine the data by stacking (bind_rows()). I have added a new column so that we can differentiate between the two datasets (ELO vs Rating).

weekly_raw <- raw_538_json$weekly_forecasts$forecasts %>% 
  enframe() %>% 
  unnest_wider(value) %>% 
  unnest_wider(types) %>% 
  select(-name) %>%
  unchop(cols = c(elo, rating))

weekly_elo <- weekly_raw %>% 
  select(-rating) %>% 
  unnest_wider(elo) %>% 
  mutate(measure = "ELO", .after = last_updated)

weekly_rating <- weekly_raw %>% 
  select(-elo) %>% 
  unnest_wider(rating) %>% 
  mutate(measure = "Rating", .after = last_updated)

# confirm same names
all.equal(
  names(weekly_elo),
  names(weekly_rating)
)
[1] TRUE
weekly_forecasts <- bind_rows(weekly_elo, weekly_rating)

weekly_forecasts %>% glimpse()
Rows: 960
Columns: 32
$ last_updated          <chr> "2020-12-13T15:13:49.762Z", "2020-12-…
$ measure               <chr> "ELO", "ELO", "ELO", "ELO", "ELO", "E…
$ week                  <int> 14, 14, 14, 14, 14, 14, 14, 14, 14, 1…
$ conference            <chr> "NFC", "AFC", "NFC", "NFC", "NFC", "A…
$ current_losses        <int> 6, 4, 7, 8, 7, 9, 12, 8, 5, 7, 4, 2, …
$ current_ties          <int> 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0…
$ current_wins          <int> 6, 8, 5, 4, 5, 2, 0, 4, 7, 5, 8, 10, …
$ division              <chr> "NFC North", "AFC East", "NFC East", …
$ elo                   <dbl> 1536.008, 1549.242, 1449.720, 1523.17…
$ losses                <dbl> 8.08856, 6.26544, 9.17700, 10.33888, …
$ make_conference_champ <dbl> 0.04372, 0.06642, 0.03702, 0.00084, 0…
$ make_divisional_round <dbl> 0.13960, 0.20540, 0.16254, 0.00274, 0…
$ make_playoffs         <dbl> 0.42390, 0.51814, 0.37400, 0.00746, 0…
$ make_superbowl        <dbl> 0.01768, 0.02526, 0.01000, 0.00034, 0…
$ name                  <chr> "MIN", "MIA", "WSH", "ATL", "DET", "C…
$ point_diff            <dbl> -13.29570, 81.68556, -2.28996, -3.220…
$ points_allowed        <dbl> 410.8140, 296.4016, 342.8474, 388.278…
$ points_scored         <dbl> 397.5183, 378.0872, 340.5575, 385.057…
$ rating                <dbl> 1523.734, 1559.728, 1446.673, 1488.03…
$ rating_current        <dbl> 1536.777, 1511.241, 1481.722, 1472.68…
$ rating_top            <dbl> 1536.777, 1511.241, 1481.722, 1472.68…
$ seed_1                <dbl> 0.00000, 0.00004, 0.00000, 0.00000, 0…
$ seed_2                <dbl> 0.00016, 0.00588, 0.00000, 0.00000, 0…
$ seed_3                <dbl> 0.00486, 0.06646, 0.00022, 0.00000, 0…
$ seed_4                <dbl> 0.00010, 0.04626, 0.35170, 0.00000, 0…
$ seed_5                <dbl> 0.02186, 0.07966, 0.00034, 0.00000, 0…
$ seed_6                <dbl> 0.16394, 0.14156, 0.00320, 0.00118, 0…
$ seed_7                <dbl> 0.23298, 0.17828, 0.01854, 0.00628, 0…
$ ties                  <dbl> 0.00560, 0.00700, 0.00604, 0.00564, 0…
$ win_division          <dbl> 0.00512, 0.11864, 0.35192, 0.00000, 0…
$ win_superbowl         <dbl> 0.00688, 0.01076, 0.00260, 0.00016, 0…
$ wins                  <dbl> 7.90584, 9.72756, 6.81696, 5.65548, 6…

Create a Function

Finally, we can combine the techniques we showed above as a function (and I’ve added it to espnscrapeR). Now we can use this to get data throughout the current season OR get info from past seasons (2015 and beyond). Again, note that 2015 has a different JSON structure but the core forecasts portion is still the same.

get_weekly_forecast <- function(season) {
  
  # Fill URL and read in JSON
  raw_url <- glue::glue("https://projects.fivethirtyeight.com/{season}-nfl-predictions/data.json")
  raw_json <- fromJSON(raw_url, simplifyVector = FALSE)
  
  # get the two datasets
  weekly_raw <- raw_538_json$weekly_forecasts$forecasts %>%
    enframe() %>%
    unnest_wider(value) %>%
    unnest_wider(types) %>%
    select(-name) %>%
    unchop(cols = c(elo, rating))
  
  # get ELO
  weekly_elo <- weekly_raw %>%
    select(-rating) %>%
    unnest_wider(elo) %>%
    mutate(measure = "ELO", .after = last_updated)
  # get Rating
  weekly_rating <- weekly_raw %>%
    select(-elo) %>%
    unnest_wider(rating) %>%
    mutate(measure = "Rating", .after = last_updated)
  # combine
  bind_rows(weekly_elo, weekly_rating)
}

get_weekly_forecast(2015) %>% 
  glimpse()
Rows: 960
Columns: 32
$ last_updated          <chr> "2020-12-13T15:13:49.762Z", "2020-12-…
$ measure               <chr> "ELO", "ELO", "ELO", "ELO", "ELO", "E…
$ week                  <int> 14, 14, 14, 14, 14, 14, 14, 14, 14, 1…
$ conference            <chr> "NFC", "AFC", "NFC", "NFC", "NFC", "A…
$ current_losses        <int> 6, 4, 7, 8, 7, 9, 12, 8, 5, 7, 4, 2, …
$ current_ties          <int> 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0…
$ current_wins          <int> 6, 8, 5, 4, 5, 2, 0, 4, 7, 5, 8, 10, …
$ division              <chr> "NFC North", "AFC East", "NFC East", …
$ elo                   <dbl> 1536.008, 1549.242, 1449.720, 1523.17…
$ losses                <dbl> 8.08856, 6.26544, 9.17700, 10.33888, …
$ make_conference_champ <dbl> 0.04372, 0.06642, 0.03702, 0.00084, 0…
$ make_divisional_round <dbl> 0.13960, 0.20540, 0.16254, 0.00274, 0…
$ make_playoffs         <dbl> 0.42390, 0.51814, 0.37400, 0.00746, 0…
$ make_superbowl        <dbl> 0.01768, 0.02526, 0.01000, 0.00034, 0…
$ name                  <chr> "MIN", "MIA", "WSH", "ATL", "DET", "C…
$ point_diff            <dbl> -13.29570, 81.68556, -2.28996, -3.220…
$ points_allowed        <dbl> 410.8140, 296.4016, 342.8474, 388.278…
$ points_scored         <dbl> 397.5183, 378.0872, 340.5575, 385.057…
$ rating                <dbl> 1523.734, 1559.728, 1446.673, 1488.03…
$ rating_current        <dbl> 1536.777, 1511.241, 1481.722, 1472.68…
$ rating_top            <dbl> 1536.777, 1511.241, 1481.722, 1472.68…
$ seed_1                <dbl> 0.00000, 0.00004, 0.00000, 0.00000, 0…
$ seed_2                <dbl> 0.00016, 0.00588, 0.00000, 0.00000, 0…
$ seed_3                <dbl> 0.00486, 0.06646, 0.00022, 0.00000, 0…
$ seed_4                <dbl> 0.00010, 0.04626, 0.35170, 0.00000, 0…
$ seed_5                <dbl> 0.02186, 0.07966, 0.00034, 0.00000, 0…
$ seed_6                <dbl> 0.16394, 0.14156, 0.00320, 0.00118, 0…
$ seed_7                <dbl> 0.23298, 0.17828, 0.01854, 0.00628, 0…
$ ties                  <dbl> 0.00560, 0.00700, 0.00604, 0.00564, 0…
$ win_division          <dbl> 0.00512, 0.11864, 0.35192, 0.00000, 0…
$ win_superbowl         <dbl> 0.00688, 0.01076, 0.00260, 0.00016, 0…
$ wins                  <dbl> 7.90584, 9.72756, 6.81696, 5.65548, 6…

Other FiveThirtyEight Data

There’s several other interesting data points in this JSON, but they’re also much easier to extract.

QB playoff adjustment values

qb_playoff_adj <- raw_538_json$playoff_qb_adjustments %>% 
  enframe() %>% 
  unnest_wider(value)
Output
qb_playoff_adj
# A tibble: 128 x 4
    name team   week  qb_adj
   <int> <chr> <int>   <dbl>
 1     1 ARI      19   27.2 
 2     2 ATL      19  -15.4 
 3     3 BAL      19    2.07
 4     4 BUF      19   28.4 
 5     5 CAR      19   22.5 
 6     6 CHI      19    7.12
 7     7 CIN      19 -109.  
 8     8 CLE      19    4.61
 9     9 DAL      19  -81.6 
10    10 DEN      19    4.06
# … with 118 more rows

Games Data

This one is interesting, it’s got ELO change as a result of win/loss along with the spread and ratings.

games_df <- raw_538_json$games %>% 
  enframe() %>% 
  unnest_auto(value) %>% 
  select(-name)
Output
games_df
# A tibble: 269 x 39
       id date  datetime  week status team1 team2 neutral score1
    <int> <chr> <chr>    <int> <chr>  <chr> <chr> <lgl>    <int>
 1 4.01e8 2020… 2020-09…     1 post   KC    HOU   FALSE       34
 2 4.01e8 2020… 2020-09…     1 post   DET   CHI   FALSE       23
 3 4.01e8 2020… 2020-09…     1 post   BAL   CLE   FALSE       38
 4 4.01e8 2020… 2020-09…     1 post   MIN   GB    FALSE       34
 5 4.01e8 2020… 2020-09…     1 post   JAX   IND   FALSE       27
 6 4.01e8 2020… 2020-09…     1 post   NE    MIA   FALSE       21
 7 4.01e8 2020… 2020-09…     1 post   BUF   NYJ   FALSE       27
 8 4.01e8 2020… 2020-09…     1 post   CAR   OAK   FALSE       30
 9 4.01e8 2020… 2020-09…     1 post   WSH   PHI   FALSE       27
10 4.01e8 2020… 2020-09…     1 post   ATL   SEA   FALSE       25
# … with 259 more rows, and 30 more variables: score2 <int>,
#   elo1_pre <dbl>, elo2_pre <dbl>, elo_spread <dbl>,
#   elo_prob1 <dbl>, elo_prob2 <dbl>, elo1_post <dbl>,
#   elo2_post <dbl>, rating1_pre <dbl>, rating2_pre <dbl>,
#   rating_spread <dbl>, rating_prob1 <dbl>, rating_prob2 <dbl>,
#   rating1_post <dbl>, rating2_post <dbl>, bettable <lgl>,
#   outcome <dbl>, qb_adj1 <dbl>, qb_adj2 <dbl>, rest_adj1 <int>,
#   rest_adj2 <int>, dist_adj <dbl>, rating1_top_qb <dbl>,
#   rating2_top_qb <dbl>, rating1_current_qb <dbl>,
#   rating2_current_qb <dbl>, prob1 <dbl>, prob2 <dbl>,
#   nocrowd <lgl>, playoff <chr>

Distances

This data has the distances for each team to other locations/stadiums.

distance_df <- raw_538_json$distances %>% 
  enframe() %>% 
  unnest_wider(value) %>% 
  unnest_longer(distances)
Output
distance_df
# A tibble: 1,024 x 6
    name team    lat   lon distances distances_id
   <int> <chr> <dbl> <dbl>     <dbl> <chr>       
 1     1 TEN    36.2 -86.8        0  TEN         
 2     1 TEN    36.2 -86.8      758. NYG         
 3     1 TEN    36.2 -86.8      471. PIT         
 4     1 TEN    36.2 -86.8      339. CAR         
 5     1 TEN    36.2 -86.8      595. BAL         
 6     1 TEN    36.2 -86.8      619. TB          
 7     1 TEN    36.2 -86.8      251. IND         
 8     1 TEN    36.2 -86.8      698. MIN         
 9     1 TEN    36.2 -86.8     1454. ARI         
10     1 TEN    36.2 -86.8      634. DAL         
# … with 1,014 more rows

QB Adjustment

I believe this is the in-season QB adjustment for each team.

qb_adj <- raw_538_json$qbs %>% 
  enframe() %>% 
  select(-name) %>% 
  unnest_wider(value)
Output
qb_adj
# A tibble: 86 x 6
    api_id name            team  priority elo_value starts
     <int> <chr>           <chr>    <int>     <int>  <int>
 1 3040206 Chris Streveler ARI          3         0      1
 2 4240689 Jake Fromm      BUF          3        38      1
 3 4036378 Jordan Love     GB           2       102      1
 4 4035003 Jacob Eason     IND          3        58      1
 5 3124900 Jake Luton      JAX          3        20      1
 6 3926936 Reid Sinnett    MIA          3         0      1
 7 4241479 Tua Tagovailoa  MIA          1       119      1
 8 4038941 Justin Herbert  LAC          1       167      1
 9 4040715 Jalen Hurts     PHI          1        89      1
10 3895785 Ben DiNucci     DAL          3         6      1
# … with 76 more rows

ESPN

ESPN has interactive win probability charts for their games. They also go a step farther than FiveThirtyEight and the JSON is embedded into the HTML “bundle”. They also have a hidden API, but I’m going to first show an example of how to get the JSON from within the page itself.

Example End Function
get_espn_win_prob <- function(game_id){

  raw_url <-glue::glue("https://www.espn.com/nfl/game?gameId={game_id}")

  raw_html <- raw_url %>%
    read_html()

  raw_text <- raw_html %>%
    html_nodes("script") %>%
    .[23] %>%
    html_text()

  raw_json <- raw_text %>%
    gsub(".*(\\[\\{)", "\\1", .) %>%
    gsub("(\\}\\]).*", "\\1", .)

  parsed_json <- jsonlite::parse_json(raw_json)

  raw_df <- parsed_json %>%
    enframe() %>%
    rename(row_id = name) %>%
    unnest_wider(value) %>%
    unnest_wider(play) %>%
    hoist(period, quarter = "number") %>%
    unnest_wider(start) %>%
    hoist(team, pos_team_id = "id") %>%
    hoist(clock, clock = "displayValue") %>%
    hoist(type, play_type = "text") %>%
    select(-type) %>%
    janitor::clean_names() %>%
    mutate(game_id = game_id)

  raw_df

}

Get the data

Let’s use an example from a pretty wild swing in Win Percentage from Week 13 of the 2020 NFL season. The Vikings and Jaguars went to overtime, with a lot of back and forth. Since there is an interactive data visualization, I’m assuming the JSON data is present there as well.


If we try our previous trick from FiveThirtyEight and the Inspect -> Network we get a total of… about 300 different requests! None of them appear big enough to be the “right” data. We’re expecting 4-5 Mb of data.


Another trick is to look for embedded JSON in the site itself. The basic representation of JSON is [{name: item}], so let’s try looking for [{ as the start of a JSON structure.

Inside the Google Chrome Dev tools we can search and find a few JSON files, including one inside some JavaScript called espn.gamepackage.probability! There’s a good amount of data there, but we need to extract it from the raw HTML. This JSON is inside a <script> object, so let’s parse the HTML and get script nodes.


espn_url <-glue::glue("https://www.espn.com/nfl/game?gameId=401220303")

raw_espn_html <- espn_url %>%
    read_html()

raw_espn_html %>%
    html_nodes("script") 
{xml_nodeset (27)}
 [1] <script type="application/ld+json">\n\t{\n\t\t"@context": "htt ...
 [2] <script type="text/javascript" src="https://dcf.espn.com/TWDC- ...
 [3] <script type="text/javascript">\n;(function(){\n\nfunction rc( ...
 [4] <script src="https://secure.espn.com/core/format/modules/head/ ...
 [5] <script src="https://a.espncdn.com/redesign/0.524.4/js/espn-he ...
 [6] <script>\n\t\t\tif (espn && espn.geoRedirect){\n\t\t\t\tespn.g ...
 [7] <script>\n\tvar espn = espn || {};\n\tespn.isOneSite = false;\ ...
 [8] <script src="https://a.espncdn.com/redesign/0.524.4/node_modul ...
 [9] <script type="text/javascript">\n\t(function () {\n\t\tvar fea ...
[10] <script>\n\t\twindow.googletag = window.googletag || {};\n\n\t ...
[11] <script type="text/javascript">\n\tif( typeof s_omni === "unde ...
[12] <script type="text/javascript" src="https://a.espncdn.com/prod ...
[13] <script>\n\t// Picture element HTML shim|v it for old IE (pair ...
[14] <script type="text/javascript">\n\t\t\tvar abtestData = {};\n\ ...
[15] <script type="text/javascript">\n\t\tvar espn = espn || {};\n\ ...
[16] <script type="text/javascript">\n\n    var __dataLayer = windo ...
[17] <script>\n\tvar espn_ui = window.espn_ui || {};\n\tespn_ui.sta ...
[18] <script src="https://a.espncdn.com/redesign/0.524.4/js/espn-cr ...
[19] <script type="text/javascript">\n\t\t\tvar espn = espn || {};\ ...
[20] <script type="text/javascript">jQuery.subscribe('espn.defer.en ...
...

Big oof. There’s 27 scripts here and just parsing through the start of the script as they are in XML is not helpful…So let’s get the raw text from each of these nodes and see if we can find the espn.gamepackage.probability element which we’re looking for!

raw_espn_text <- raw_espn_html %>%
  html_nodes("script") %>% 
  html_text()

raw_espn_text %>% 
  str_which("espn.gamepackage.probability")
[1] 23

Ok! So we’re looking for the 23rd node, I’m going to hide the output inside an expandable tag, as it’s a long output!

example_embed_json <- raw_espn_html %>%
    html_nodes("script") %>% 
    .[23] %>%
    html_text()
Example Embed JSON
example_embed_json
[1] "\n\t\t\t\t(function($) {\n\t\t\t\t\tvar espn = window.espn || {},\n\t\t\t\t\t\tDTCpackages = window.DTCpackages || [];\n\n\t\t\t\t\tespn.gamepackage = espn.gamepackage || {};\n\t\t\t\t\tespn.gamepackage.gameId = \"401220303\";\n\t\t\t\t\tespn.gamepackage.type = \"game\";\n\t\t\t\t\tespn.gamepackage.timestamp = \"2020-12-06T18:00Z\";\n\t\t\t\t\tespn.gamepackage.status = \"post\";\n\t\t\t\t\tespn.gamepackage.league = \"nfl\";\n\t\t\t\t\tespn.gamepackage.leagueId = 28;\n\t\t\t\t\tespn.gamepackage.sport = \"football\";\n\t\t\t\t\tespn.gamepackage.network = \"CBS\";\n\t\t\t\t\tespn.gamepackage.awayTeamName = \"jacksonville-jaguars\";\n\t\t\t\t\tespn.gamepackage.homeTeamName = \"minnesota-vikings\";\n\t\t\t\t\tespn.gamepackage.awayTeamId = \"30\";\n\t\t\t\t\tespn.gamepackage.homeTeamId = \"16\";\n\t\t\t\t\tespn.gamepackage.awayTeamColor = \"00839C\";\n\t\t\t\t\tespn.gamepackage.homeTeamColor = \"240A67\";\n\t\t\t\t\tespn.gamepackage.showGamebreak = true;\n\t\t\t\t\tespn.gamepackage.supportsHeadshots = true\n\t\t\t\t\tespn.gamepackage.playByPlaySource = \"full\";\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\n\t\t\t\t\t\n\n\t\t\t\t\t\n\t\t\t\t\t\tespn.gamepackage.probability = espn.gamepackage.probability || {};\n\t\t\t\t\t\tespn.gamepackage.probability.data = [{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":0,\"start\":{\"distance\":0,\"yardLine\":35,\"team\":{\"id\":\"16\"},\"down\":0,\"yardsToEndzone\":65},\"text\":\"D.Bailey kicks 65 yards from MIN 35 to end zone, Touchback.\",\"clock\":{\"displayValue\":\"15:00\"},\"type\":{\"id\":\"53\",\"text\":\"Kickoff\",\"abbreviation\":\"K\"}},\"homeWinPercentage\":0.828,\"playId\":\"40122030340\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":0,\"start\":{\"shortDownDistanceText\":\"1st & 10\",\"possessionText\":\"JAX 25\",\"downDistanceText\":\"1st & 10 at JAX 25\",\"distance\":10,\"yardLine\":75,\"team\":{\"id\":\"30\"},\"down\":1,\"yardsToEndzone\":75},\"text\":\"(15:00) M.Glennon pass deep middle to J.O'Shaughnessy to JAX 49 for 24 yards (A.Harris; E.Wilson).\",\"clock\":{\"displayValue\":\"15:00\"},\"type\":{\"id\":\"24\",\"text\":\"Pass Reception\",\"abbreviation\":\"REC\"}},\"homeWinPercentage\":0.807,\"playId\":\"40122030355\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":0,\"start\":{\"shortDownDistanceText\":\"1st & 10\",\"possessionText\":\"JAX 49\",\"downDistanceText\":\"1st & 10 at JAX 49\",\"distance\":10,\"yardLine\":51,\"team\":{\"id\":\"30\"},\"down\":1,\"yardsToEndzone\":51},\"text\":\"(14:25) (Shotgun) M.Glennon pass short right to J.Robinson pushed ob at MIN 43 for 8 yards (T.Dye).\",\"clock\":{\"displayValue\":\"14:25\"},\"type\":{\"id\":\"24\",\"text\":\"Pass Reception\",\"abbreviation\":\"REC\"}},\"homeWinPercentage\":0.796,\"playId\":\"40122030379\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":0,\"start\":{\"shortDownDistanceText\":\"2nd & 2\",\"possessionText\":\"MIN 43\",\"downDistanceText\":\"2nd & 2 at MIN 43\",\"distance\":2,\"yardLine\":43,\"team\":{\"id\":\"30\"},\"down\":2,\"yardsToEndzone\":43},\"text\":\"(13:54) J.Robinson left guard to MIN 34 for 9 yards (T.Davis; E.Wilson).\",\"clock\":{\"displayValue\":\"13:54\"},\"type\":{\"id\":\"5\",\"text\":\"Rush\",\"abbreviation\":\"RUSH\"}},\"homeWinPercentage\":0.785,\"playId\":\"401220303108\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":0,\"start\":{\"shortDownDistanceText\":\"1st & 10\",\"possessionText\":\"MIN 34\",\"downDistanceText\":\"1st & 10 at MIN 34\",\"distance\":10,\"yardLine\":34,\"team\":{\"id\":\"30\"},\"down\":1,\"yardsToEndzone\":34},\"text\":\"(13:21) J.Robinson left end to MIN 28 for 6 yards (A.Harris; S.Stephen).\",\"clock\":{\"displayValue\":\"13:21\"},\"type\":{\"id\":\"5\",\"text\":\"Rush\",\"abbreviation\":\"RUSH\"}},\"homeWinPercentage\":0.777,\"playId\":\"401220303129\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"2nd & 4\",\"possessionText\":\"MIN 28\",\"downDistanceText\":\"2nd & 4 at MIN 28\",\"distance\":4,\"yardLine\":28,\"team\":{\"id\":\"30\"},\"down\":2,\"yardsToEndzone\":28},\"text\":\"Laviska Shenault Jr. 28 Yd pass from Mike Glennon (Chase McLaughlin PAT failed)\",\"clock\":{\"displayValue\":\"12:33\"},\"type\":{\"id\":\"67\",\"text\":\"Passing Touchdown\",\"abbreviation\":\"TD\"}},\"homeWinPercentage\":0.742,\"playId\":\"401220303150\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"distance\":0,\"yardLine\":-4,\"team\":{\"id\":\"30\"},\"down\":0,\"yardsToEndzone\":65},\"text\":\"L.Cooke kicks 69 yards from JAX 35 to MIN -4. A.Abdullah to MIN 21 for 25 yards (S.Quarterman).\",\"clock\":{\"displayValue\":\"12:33\"},\"type\":{\"id\":\"12\",\"text\":\"Kickoff Return (Offense)\"}},\"homeWinPercentage\":0.741,\"playId\":\"401220303191\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"1st & 10\",\"possessionText\":\"MIN 21\",\"downDistanceText\":\"1st & 10 at MIN 21\",\"distance\":10,\"yardLine\":21,\"team\":{\"id\":\"16\"},\"down\":1,\"yardsToEndzone\":79},\"text\":\"(12:28) K.Cousins pass short left to A.Thielen pushed ob at MIN 29 for 8 yards (T.Herndon). PENALTY on JAX-M.Jack, Defensive Holding, 5 yards, enforced at MIN 21 - No Play.\",\"clock\":{\"displayValue\":\"12:28\"},\"type\":{\"id\":\"24\",\"text\":\"Pass Reception\",\"abbreviation\":\"REC\"}},\"homeWinPercentage\":0.747,\"playId\":\"401220303213\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"1st & 10\",\"possessionText\":\"MIN 26\",\"downDistanceText\":\"1st & 10 at MIN 26\",\"distance\":10,\"yardLine\":26,\"team\":{\"id\":\"16\"},\"down\":1,\"yardsToEndzone\":74},\"text\":\"(12:11) (Shotgun) K.Cousins pass incomplete deep right to A.Thielen.\",\"clock\":{\"displayValue\":\"12:11\"},\"type\":{\"id\":\"3\",\"text\":\"Pass Incompletion\"}},\"homeWinPercentage\":0.719,\"playId\":\"401220303257\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"2nd & 10\",\"possessionText\":\"MIN 26\",\"downDistanceText\":\"2nd & 10 at MIN 26\",\"distance\":10,\"yardLine\":26,\"team\":{\"id\":\"16\"},\"down\":2,\"yardsToEndzone\":74},\"text\":\"(12:06) D.Cook right tackle to MIN 33 for 7 yards (M.Jack).\",\"clock\":{\"displayValue\":\"12:06\"},\"type\":{\"id\":\"5\",\"text\":\"Rush\",\"abbreviation\":\"RUSH\"}},\"homeWinPercentage\":0.731,\"playId\":\"401220303279\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"3rd & 3\",\"possessionText\":\"MIN 33\",\"downDistanceText\":\"3rd & 3 at MIN 33\",\"distance\":3,\"yardLine\":33,\"team\":{\"id\":\"16\"},\"down\":3,\"yardsToEndzone\":67},\"text\":\"(11:27) (Shotgun) K.Cousins pass incomplete short right to D.Cook.\",\"clock\":{\"displayValue\":\"11:27\"},\"type\":{\"id\":\"3\",\"text\":\"Pass Incompletion\"}},\"homeWinPercentage\":0.683,\"playId\":\"401220303300\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"4th & 3\",\"possessionText\":\"MIN 33\",\"downDistanceText\":\"4th & 3 at MIN 33\",\"distance\":3,\"yardLine\":33,\"team\":{\"id\":\"16\"},\"down\":4,\"yardsToEndzone\":67},\"text\":\"(11:23) B.Colquitt punts 50 yards to JAX 17, Center-A.DePaola. K.Cole Sr. to JAX 30 for 13 yards (T.Conklin).\",\"clock\":{\"displayValue\":\"11:23\"},\"type\":{\"id\":\"52\",\"text\":\"Punt\",\"abbreviation\":\"PUNT\"}},\"homeWinPercentage\":0.699,\"playId\":\"401220303322\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"1st & 10\",\"possessionText\":\"JAX 30\",\"downDistanceText\":\"1st & 10 at JAX 30\",\"distance\":10,\"yardLine\":70,\"team\":{\"id\":\"30\"},\"down\":1,\"yardsToEndzone\":70},\"text\":\"(11:11) J.Robinson left guard to JAX 34 for 4 yards (T.Dye).\",\"clock\":{\"displayValue\":\"11:11\"},\"type\":{\"id\":\"5\",\"text\":\"Rush\",\"abbreviation\":\"RUSH\"}},\"homeWinPercentage\":0.693,\"playId\":\"401220303347\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"2nd & 6\",\"possessionText\":\"JAX 34\",\"downDistanceText\":\"2nd & 6 at JAX 34\",\"distance\":6,\"yardLine\":66,\"team\":{\"id\":\"30\"},\"down\":2,\"yardsToEndzone\":66},\"text\":\"(10:41) (Shotgun) J.Robinson left end to JAX 38 for 4 yards (T.Davis; J.Johnson).\",\"clock\":{\"displayValue\":\"10:41\"},\"type\":{\"id\":\"5\",\"text\":\"Rush\",\"abbreviation\":\"RUSH\"}},\"homeWinPercentage\":0.685,\"playId\":\"401220303368\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"3rd & 2\",\"possessionText\":\"JAX 38\",\"downDistanceText\":\"3rd & 2 at JAX 38\",\"distance\":2,\"yardLine\":62,\"team\":{\"id\":\"30\"},\"down\":3,\"yardsToEndzone\":62},\"text\":\"(10:04) (Shotgun) M.Glennon pass short right to T.Eifert to JAX 44 for 6 yards (J.Gladney).\",\"clock\":{\"displayValue\":\"10:04\"},\"type\":{\"id\":\"24\",\"text\":\"Pass Reception\",\"abbreviation\":\"REC\"}},\"homeWinPercentage\":0.653,\"playId\":\"401220303389\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"1st & 10\",\"possessionText\":\"JAX 44\",\"downDistanceText\":\"1st & 10 at JAX 44\",\"distance\":10,\"yardLine\":56,\"team\":{\"id\":\"30\"},\"down\":1,\"yardsToEndzone\":56},\"text\":\"(9:29) J.Robinson left tackle to MIN 47 for 9 yards (E.Wilson).\",\"clock\":{\"displayValue\":\"9:29\"},\"type\":{\"id\":\"5\",\"text\":\"Rush\",\"abbreviation\":\"RUSH\"}},\"homeWinPercentage\":0.629,\"playId\":\"401220303413\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"2nd & 1\",\"possessionText\":\"MIN 47\",\"downDistanceText\":\"2nd & 1 at MIN 47\",\"distance\":1,\"yardLine\":47,\"team\":{\"id\":\"30\"},\"down\":2,\"yardsToEndzone\":47},\"text\":\"(8:48) M.Glennon pass short middle to J.O'Shaughnessy to MIN 40 for 7 yards (T.Dye).\",\"clock\":{\"displayValue\":\"8:48\"},\"type\":{\"id\":\"24\",\"text\":\"Pass Reception\",\"abbreviation\":\"REC\"}},\"homeWinPercentage\":0.62,\"playId\":\"401220303434\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"1st & 10\",\"possessionText\":\"MIN 40\",\"downDistanceText\":\"1st & 10 at MIN 40\",\"distance\":10,\"yardLine\":40,\"team\":{\"id\":\"30\"},\"down\":1,\"yardsToEndzone\":40},\"text\":\"(8:06) (Shotgun) M.Glennon pass deep left to C.Johnson to MIN 6 for 34 yards (H.Smith) [J.Holmes].\",\"clock\":{\"displayValue\":\"8:06\"},\"type\":{\"id\":\"24\",\"text\":\"Pass Reception\",\"abbreviation\":\"REC\"}},\"homeWinPercentage\":0.688,\"playId\":\"401220303458\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"1st & Goal\",\"possessionText\":\"MIN 6\",\"downDistanceText\":\"1st & Goal at MIN 6\",\"distance\":6,\"yardLine\":6,\"team\":{\"id\":\"30\"},\"down\":1,\"yardsToEndzone\":6},\"text\":\"(7:28) (Shotgun) M.Glennon pass short right to J.Robinson to MIN 5 for 1 yard (T.Dye).\",\"clock\":{\"displayValue\":\"7:26\"},\"type\":{\"id\":\"24\",\"text\":\"Pass Reception\",\"abbreviation\":\"REC\"}},\"homeWinPercentage\":0.523,\"playId\":\"401220303482\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"2nd & Goal\",\"possessionText\":\"MIN 5\",\"downDistanceText\":\"2nd & Goal at MIN 5\",\"distance\":5,\"yardLine\":5,\"team\":{\"id\":\"30\"},\"down\":2,\"yardsToEndzone\":5},\"text\":\"(6:52) (Shotgun) J.Robinson up the middle to MIN 4 for 1 yard (E.Yarbrough; J.Gladney).\",\"clock\":{\"displayValue\":\"6:52\"},\"type\":{\"id\":\"5\",\"text\":\"Rush\",\"abbreviation\":\"RUSH\"}},\"homeWinPercentage\":0.578,\"playId\":\"401220303506\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":6,\"start\":{\"shortDownDistanceText\":\"3rd & Goal\",\"possessionText\":\"MIN 4\",\"downDistanceText\":\"3rd & Goal at MIN 4\",\"distance\":4,\"yardLine\":4,\"team\":{\"id\":\"30\"},\"down\":3,\"yardsToEndzone\":4},\"text\":\"(6:12) (Shotgun) M.Glennon pass incomplete short left to K.Cole Sr..\",\"clock\":{\"displayValue\":\"6:12\"},\"type\":{\"id\":\"3\",\"text\":\"Pass Incompletion\"}},\"homeWinPercentage\":0.621,\"playId\":\"401220303527\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":9,\"start\":{\"shortDownDistanceText\":\"4th & Goal\",\"possessionText\":\"MIN 4\",\"downDistanceText\":\"4th & Goal at MIN 4\",\"distance\":4,\"yardLine\":4,\"team\":{\"id\":\"30\"},\"down\":4,\"yardsToEndzone\":4},\"text\":\"Chase McLaughlin 22 Yd Field Goal\",\"clock\":{\"displayValue\":\"6:04\"},\"type\":{\"id\":\"59\",\"text\":\"Field Goal Good\",\"abbreviation\":\"FG\"}},\"homeWinPercentage\":0.619,\"playId\":\"401220303549\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":9,\"start\":{\"distance\":0,\"yardLine\":65,\"team\":{\"id\":\"30\"},\"down\":0,\"yardsToEndzone\":65},\"text\":\"L.Cooke kicks 65 yards from JAX 35 to end zone, Touchback.\",\"clock\":{\"displayValue\":\"6:04\"},\"type\":{\"id\":\"53\",\"text\":\"Kickoff\",\"abbreviation\":\"K\"}},\"homeWinPercentage\":0.624,\"playId\":\"401220303568\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":9,\"start\":{\"shortDownDistanceText\":\"1st & 10\",\"possessionText\":\"MIN 25\",\"downDistanceText\":\"1st & 10 at MIN 25\",\"distance\":10,\"yardLine\":25,\"team\":{\"id\":\"16\"},\"down\":1,\"yardsToEndzone\":75},\"text\":\"(6:04) D.Cook left end to MIN 31 for 6 yards (J.Schobert).\",\"clock\":{\"displayValue\":\"6:04\"},\"type\":{\"id\":\"5\",\"text\":\"Rush\",\"abbreviation\":\"RUSH\"}},\"homeWinPercentage\":0.625,\"playId\":\"401220303583\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":9,\"start\":{\"shortDownDistanceText\":\"2nd & 4\",\"possessionText\":\"MIN 31\",\"downDistanceText\":\"2nd & 4 at MIN 31\",\"distance\":4,\"yardLine\":31,\"team\":{\"id\":\"16\"},\"down\":2,\"yardsToEndzone\":69},\"text\":\"(5:28) K.Cousins pass short left to J.Jefferson to MIN 36 for 5 yards (T.Herndon).\",\"clock\":{\"displayValue\":\"5:28\"},\"type\":{\"id\":\"24\",\"text\":\"Pass Reception\",\"abbreviation\":\"REC\"}},\"homeWinPercentage\":0.636,\"playId\":\"401220303604\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":9,\"start\":{\"shortDownDistanceText\":\"1st & 10\",\"possessionText\":\"MIN 36\",\"downDistanceText\":\"1st & 10 at MIN 36\",\"distance\":10,\"yardLine\":36,\"team\":{\"id\":\"16\"},\"down\":1,\"yardsToEndzone\":64},\"text\":\"(4:53) J.Jefferson right end pushed ob at MIN 38 for 2 yards (M.Jack).\",\"clock\":{\"displayValue\":\"4:53\"},\"type\":{\"id\":\"5\",\"text\":\"Rush\",\"abbreviation\":\"RUSH\"}},\"homeWinPercentage\":0.553,\"playId\":\"401220303628\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":9,\"start\":{\"shortDownDistanceText\":\"2nd & 8\",\"possessionText\":\"MIN 38\",\"downDistanceText\":\"2nd & 8 at MIN 38\",\"distance\":8,\"yardLine\":38,\"team\":{\"id\":\"16\"},\"down\":2,\"yardsToEndzone\":62},\"text\":\"(4:17) D.Cook left end to MIN 35 for -3 yards (T.Herndon).\",\"clock\":{\"displayValue\":\"4:17\"},\"type\":{\"id\":\"5\",\"text\":\"Rush\",\"abbreviation\":\"RUSH\"}},\"homeWinPercentage\":0.572,\"playId\":\"401220303649\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":9,\"start\":{\"shortDownDistanceText\":\"3rd & 11\",\"possessionText\":\"MIN 35\",\"downDistanceText\":\"3rd & 11 at MIN 35\",\"distance\":11,\"yardLine\":35,\"team\":{\"id\":\"16\"},\"down\":3,\"yardsToEndzone\":65},\"text\":\"(3:36) (Shotgun) K.Cousins sacked at MIN 30 for -5 yards (D.Smoot).\",\"clock\":{\"displayValue\":\"3:36\"},\"type\":{\"id\":\"7\",\"text\":\"Sack\"}},\"homeWinPercentage\":0.464,\"playId\":\"401220303670\",\"tiePercentage\":0,\"secondsLeft\":0},{\"play\":{\"period\":{\"number\":1},\"homeScore\":0,\"awayScore\":9,\"start\":{\"shortDownDistanceText\":\"