Statewide R User Group (ShRUG)

October Meeting agenda

Tuesday October 27th - 3:30-4:30 PST - Virtual
  • Introductions by chatbox - your name, agency, and a link to a recent project you think relates to R.
  • Day to day grind & efficiency:  applying R in a data ecosystem - Jack Janisch, 30 minutes
  • How long does it take---look across 10 years of change (use a metric, everyone has their own)
  • How long does it take---dealing with that folder of data files, continuous data sonde readings
  • How long does it take---calculate centroid of year
  • How long does it take---validate results from electronic data forms that run a calculation remotely
  • How long does it take---are we identifying all R package dependencies with internal mirror
  • Writing HTML reports with RMarkdown - Statewide Telework Survey – Kate Ito – 30 minutes
  • Scraping tables from a web page - Ian Wesley / Will Saunders - 15 minutes
What do you want to see covered in future meetings? (survey to follow)

Day to day grind & efficiency:  applying R in a data ecosystem

  • How long does it take---look across 10 years of change (use a metric, everyone has their own)
  • How long does it take---dealing with that folder of data files, continuous data sonde readings
  • How long does it take---calculate centroid of year
  • How long does it take---validate results from electronic data forms that run a calculation remotely
  • How long does it take---are we identifying all R package dependencies with internal mirror
Jack's daily grind
rmarkdown logo

RMarkdown

Writing HTML reports with RMarkdown - Statewide Telework Survey – Kate Ito – 30 minutes
- knew they wanted to build a report for RPubs
- emphasis on snappy load times and clean design
- took about 40 hours for 2 staff to build the report
View of text of WSDOT future of telework survey from web
View of rmarkdown code in RStudio
charts in the report are static but scalable .svg images (shown in RPubs)

Scraping tables from the web

# Read ATG's breach notification table from website into dataframe Current_Breach_Table
#
# libraries
library(xml2)
library(rvest)
library(dplyr)
#
#Set the URL
AtgUrl <- "https://www.atg.wa.gov/data-breach-notifications"
# read into buffer
HtmlBuffer <- read_html(AtgUrl)
# parse buffer into table and load to df
CurrentBreachTable <- (html_table(HtmlBuffer, header = NA, trim = TRUE, fill = FALSE, dec = "."))
# trim and clean up
CurrentBreachTable <- CurrentBreachTable[[2]]
colnames(CurrentBreachTable) = CurrentBreachTable[2, ]
CurrentBreachTable <- CurrentBreachTable[-c(1, 2, 3), ]
names(CurrentBreachTable) <- make.names(names(CurrentBreachTable), unique = TRUE)
mutate(CurrentBreachTable, Reported.Date= as.Date(Reported.Date, format= "%m/%d/%Y"))
# Check it out
View(CurrentBreachTable)
ATG breach notification page - source material
scaping code from StackExchange (courtesy of Ian)
table view in RStudio

Suggest a future topic

Topics suggested so far

June 2020 Meeting Agenda

-Welcome, tech, recording (Jack)
-Goals & purpose of the group (Will)
-From Excel to R an illustrative example  (Alon)
-Good ways to learn R (Will)
-R + OCR to automate business (Ian)
-Open Discussion: R and next steps for the R user group
R logo

ECY - Jack Janisch
Watershed Health Monitoring Database Coordinator   
jack.janisich@ecy.wa.gov   360-407-6649
whiteboard with boxes for concepts: what is a community of practice?
whiteboard with concept boxes: communities are difficult to cultivate
WSDOT - Ian Wesley
Builder of very cool data things at WSDOT.

WSDOT transit's Github

Ian and Alon work with the Github site linked below to post code for their R projects.  The image is a hyperlink to Github.

snapshot of Github page showing repositories
excel logo with bidirectional arrows to R logo

From Excel to R: an Illustrative example

Alon Bassok

DOL - Will Saunders
Privacy & Data Stewardship Administrator at Department of Licensing. 
wsaunders@dol.wa.gov   360-902-0104 

Goals / Purpose

Will Saunders, DOL
  1. Show off good stuff done in R
  2. Learn how to do new stuff
  3. Put it into practice at work
  4. Discuss with colleagues
  5. Repeat as time allows

Cycle: Show Learn Practice Discuss

Good ways to learn R

Will Saunders, DOL
Suggested resources for learning the basics of R, from (and for) a quasi-technical guy who last coded in 1980's Pascal...

What should I study next?

The table before is an open dataset on Data.wa.gov. Got suggestions? add to the list with this form.

school age child sitting with feet up on a clean desk with a telescope and papers

R + Open Source OCR to automate business

Ian Wesley, WSDOT


Next Steps for the User Group

"discussion..."  uh... does that mean I have to talk?
Well, you're still here, so what would be good to do next?
Meet (virtually) again?
Got a good demo to show?
Got a project or practice idea?
Blackboard with hand and arrows up, straight, down