Jump to content

Data scraping: Difference between revisions

From The APES Atlas
No edit summary
No edit summary
 
(20 intermediate revisions by the same user not shown)
Line 1: Line 1:
<pre>
<pre>


### APES Atlas Web Scraper
################# APES Atlas Web Scraper #################


# The following R code extracts data from the standardized APES Atlas tables.
# The following R code extracts data from the standardized APES Atlas tables.
# The following code must be loaded into R.
# To execute the code, you need to have R installed and an editor (e.g. R Studio or Notepad++).  
# It will take about two minutes for the results to be scraped and saved.


#----------------------------------------------------------------------------------
# INSTRUCTIONS
rm(list=ls())
# 1. Copy the lines below from # SCRIPT START to # SCRIPT END and paste them into your editor.
# 2. In the last line, replace "path_to_save" with the folder path name where you want to store the scraped data (use only '/' or '\\' as folder separator). For example, replace path_to_save with "D:/users/my_docs/APESATLAS/"
 
# 3. Run the lines in R.
#  Note: Windows may open during the installation of the packages. Confirm all with ‘OK’.
#  It will take about 30 seconds for the tables to be scraped and saved into the folder specified in path_to_save .
 
 
# SCRIPT START
#----------------------------------------------------------------------------------
#----------------------------------------------------------------------------------
# Install & load packages
# Install & load packages
Line 14: Line 21:
install.packages("plyr")
install.packages("plyr")
install.packages("writexl")
install.packages("writexl")
install.packages("jsonlite")
install.packages("data.table")
install.packages("httr2")
install.packages("httr2")
library(httr2)
library(httr2)
library(jsonlite)
library(data.table)
library(writexl)
library(writexl)
library(plyr)
library(plyr)
Line 24: Line 31:
#----------------------------------------------------------------------------------
#----------------------------------------------------------------------------------
source("https://github.com/lakik/apesatlas_scraping/raw/refs/heads/main/script")
source("https://github.com/lakik/apesatlas_scraping/raw/refs/heads/main/script")
#----------------------------------------------------------------------------------
# Define the path to save the results
#----------------------------------------------------------------------------------
path_to_save = "YOUR_PATH"  # change accordingly (e.g. "D:/Users/APES/APESATLAS/")
#----------------------------------------------------------------------------------
#----------------------------------------------------------------------------------
# Run scraper function
# Run scraper function
Line 33: Line 36:
scrape_apesatlas(path_to_save)
scrape_apesatlas(path_to_save)
#----------------------------------------------------------------------------------
#----------------------------------------------------------------------------------
# SCRIPT END

Latest revision as of 20:00, 21 January 2026


################# APES Atlas Web Scraper #################

# The following R code extracts data from the standardized APES Atlas tables.
# To execute the code, you need to have R installed and an editor (e.g. R Studio or Notepad++). 

# INSTRUCTIONS
# 1. Copy the lines below from # SCRIPT START to # SCRIPT END and paste them into your editor.
# 2. In the last line, replace "path_to_save" with the folder path name where you want to store the scraped data (use only '/' or '\\' as folder separator). For example, replace path_to_save with "D:/users/my_docs/APESATLAS/"

# 3. Run the lines in R.
#  Note: Windows may open during the installation of the packages. Confirm all with ‘OK’.
#  It will take about 30 seconds for the tables to be scraped and saved into the folder specified in path_to_save .


# SCRIPT START
#----------------------------------------------------------------------------------
# Install & load packages
#----------------------------------------------------------------------------------
install.packages("plyr")
install.packages("writexl")
install.packages("data.table")
install.packages("httr2")
library(httr2)
library(data.table)
library(writexl)
library(plyr)
#----------------------------------------------------------------------------------
# Load functions from Github
#----------------------------------------------------------------------------------
source("https://github.com/lakik/apesatlas_scraping/raw/refs/heads/main/script")
#----------------------------------------------------------------------------------
# Run scraper function
#----------------------------------------------------------------------------------
scrape_apesatlas(path_to_save)
#----------------------------------------------------------------------------------
# SCRIPT END