Scraping Streetview for a Soft Story Structure Catalog
I wrote this script to download google streetview images locally for a list of addresses in a text file, one address per line. It’s an easy (at the least preliminary) way to get images of structures, houses, etc.. We needed this for an inventory of certain structures in a planning effort, but this can be utilized for a wide variety of purposes.
The first thing needed is a text file with addresses that will be turned into local copies of streetview images.
Here’s part of the text file I use in this example. This is the formatting needed for this script, no headers, one line per address. If you want to try this, save the following as a text file locally and call that text file in the script later.
310 E. Monte Vista Ave. Ste A Vacaville CA 95688 3777-B Vaca Valley Parkway Vacaville CA 95688 950 Mason Street Vacaville CA 95688 848 A Alamo Drive Vacaville CA 95688 7428 Paddon Road Vacaville CA 95688 4117 Cricket Trail Vacaville CA 95687 851 La Cruz Lane Vacaville CA 95687 396 West Monte Vista Avenue Vacaville CA 95688 325 Stone Manor Crt. Vacaville CA 95687 4692 Udell Road Vacaville CA 95688 513 Tipperary Drive Vacaville CA 95688 931 Reading Way Vacaville CA 95687 707 Morning Sun Court Vacaville CA 95688 4219 Rolling Hills Lane Vacaville CA 95688 254 Donegal Court Vacaville CA 95688 1943 Marshall Rd Vacaville CA 95687 184 Woodridge Circle Vacaville CA 95687 621 S Orchard Vacaville CA 95688 1201 Burton Drive Vacaville CA 95687 126 Peabody Road Vacaville CA 95687 1707 California Drive Rm 3 Vacaville CA 95687 1625 Alamo Drive Vacaville CA 95687 101 Markham Avenue Room 26 Vacaville CA 95688 100 Padan School Road Vacaville CA 95688 880 Alamo Drive Vacaville CA 95688 581 Peabody Road Vacaville CA 95687 3442 Browns Valley Rd, #100 Vacaville CA 95688 425 Hemlock Vacaville CA 95688 301 N Orchard Ave Vacaville CA 95688 425 Hemlock Vacaville CA 95688 580 Trinity Drive Vacaville CA 95687 1117 Davis Street Vacaville CA 95687 1201 Burton Drive Vacaville CA 95687 880 Alamo Drive Vacaville CA 95688 581 Peabody Road Vacaville CA 95687 3442 Browns Valley Rd, #100 Vacaville CA 95688
The script itself is below, it’s thoroughly commented. Read the comments above each part as a guide to each step. To use it yourself, the things you need to play with are the ‘text’ variable (point to the location on your harddrvie with the address text file) and the ‘suf’ cariable, get a google streetview api key and change it here (google ‘google streetview api’ if you need help or don’t have one).
After those things are in place the script can be run in the python command line. I’d suggest copying the entire script except the last line so get a better feel for what it does.
#import os and urllib modules #os for file path creation #urllib for accessing web content import urllib import os #this is the first part of the streetview, url up to the address, this url will return a 600x600px image pre="https://maps.googleapis.com/maps/api/streetview?size=600x600&location=" #this is the second part of the streetview url, the text variable below, includes the path to a text file containing one address per line #the addresses in this text file will complete the URL needed to return a streetview image and provide the filename of each streetview image text="C:\Users\BrianGreer\Desktop\Test_Address.txt" #this is the third part of the url, needed after the address #this is my API key, please replace the one below with your own (google 'google streetview api key'), thanks! suf="&key=AIzaSyAC9P4Njf_yRogkp0M-cPpWr5Op_r6hvg4" #this is the directory that will store the streetview images #this directory will be created if not present dir=r"C:\Users\BrianGreer\Desktop\imagescrape" #checks if the dir variable (output path) above exists and creates it if it does not if not os.path.exists(dir): os.makedirs(dir) #opens the address list text file (from the 'text' variable defined above) in read mode ("r") with open(text,"r") as text_file: #the variable 'lines' below creates a list of each address line in the source 'text' file lines = [line.rstrip('\n') for line in open(text)] print "THE CONTENTS OF THE TEXT FILE:\n"+str(lines) #start a loop through the 'lines' list for line in lines: #string clean-up to get rid of commas in the url and filename ln = line.replace(",","") print "CLEANED UP ADDRESS LINE:\n"+ln # creates the url that will be passed to the url reader, this creates the full, valid, url that will return a google streetview image for each address in the address text file URL = pre+ln+suf print "URL FOR STREETVIEW IMAGE:\n"+URL #creates the filename needed to save each address's streetview image locally filename = os.path.join(dir,ln+".jpg") print "OUTPUT FILENAME:\n"+filename #you can run this up to this line in the python command line to see what each step does #final step, fetches and saves the streetview image for each address using the url created in the previous steps urllib.urlretrieve(URL, filename)
Once you are ready, run the script including the last line and watch the magic as the imagescrape folder fills with streetview images.
This script was used to help build a catalog of dozens of structures. An important component of the catalog was to have a photograph of the facade of each structure to illustrate its condition.
A sample of the final product can be seen below.