Spatial Data Science @ Berkeley May 2015

Bootcamp participants outside historic Mulford HallOur bootcamp on Spatial Data Science has concluded. We had three packed days learning about the concepts, tools and workflow associated with spatial databases, analysis and visualizations. 

Our goal was not to teach a specific suite of tools but rather to teach participants how to develop and refine repeatable and testable workflows for spatial data using common standard programming practices.

On Day 1 we focused on setting up a collaborative virtual data environment through virtual machines, spatial databases (PostgreSQL/PostGIS) with multi-user editing and versioning (GeoGig). We also talked about open data and open standards, and modern data formats and tools (GeoJSON, GDAL).

Analyzing spatial data is the best part! On Day 2 we focused on open analytical tools for spatial data. We focused on one particular class of spatial data analysis: pattern analysis, and used Python (i.e. PySAL, NumPy, PyCharm, iPython Notebook), and R Studio (i.e. raster, sp, maptools, rgdal, shiny) to look at spatial autocorrelation and spatial regression. 

Wait, visualizing spatial data is the best part! Day 3 was dedicated to the web stack, and visualization. We started with web mapping (web stack, HTML/CSS, JavaScript, Leaflet), and then focused on web-based visualizations (D3).  Web mapping is great, and as OpenGeo.org says: “Internet maps appear magical: portals into infinitely large, infinitely deep pools of data. But they aren't magical, they are built of a few standard pieces of technology, and the pieces can be re-arranged and sourced from different places.…Anyone can build an internet map."

All-in-all it was a great time spent with a collection of very interesting mapping professionals from around the country (and Haiti!). Thanks to everyone!

10-year anniversary for the GIF

I'm musing, contemplating and writing on the decade 2005-2015, as this is the GIF's 10-year anniversary. What a decade it was. Here I'll post and add to some of the key events that helped transform mapping (and the GIF) in the last 10 years.

Key background events

• 1996. Mapquest launched.
• 1997. Skynet becomes self-aware.
• May 2000. Selective Availabilility on GPS turned off, leading the way for GPS in smartphones.
• The Scan Line Corrector (SLC) on the Landsat 7 ETM+ instrument failed May 31, 2003.
• 2004. Open Street Map founded.
• March 2004. Yahoo! maps launched, first slippy maps (click and drag to pan and zoom the map).
• 2004. NASA releases WorldWind.
• October 2004. Google acquires Where 2 allowing AJAX map tiling to a desktop client.
• October 2004. Google acquires Keyhole.

What made 2005 such a crazy year

• Google Maps launches in February, and goes mobile in April.
• The first mashup: Paul Rademacher's Housingmaps.org. His original post on Craigslist asking for feedback: https://forums.craigslist.org/?ID=26638141
• Google Maps API launches in June.
• NASA's Blue Marble Next Generation released.
• Google Earth launches in June.
• Hurricane Katrina hits in August. Simple webmaps for the disaster proliferate, and ESRI and GE get on the scene.
• Kellylab's first blog post in September.
• GIF launches and hosts our first GIS Day in November with Michael Jones, formerly of Keyhole.
• The back-up solar array drive on Landsat 5 began failing and was not able to provide the power needed to charge the batteries. November 26.

Where we are in 2015

We've gone through a number of transitions in the world of mapping:

• Data have transitioned from being siloed, and found in clearinghouses to being open and provided through APIs.
• We’ve moved from desktop computing to cloud computing.
• Webmaps have transitioned from using proprietary stacks to networks with multiple open and proprietary options.
• We’ve moved from imagery gathered monthly or seasonally to daily; footprints are smaller, and our focus has shifted from local focus to global coverage.
• Our planimetric 2D view is changing with lidar and radar sensors.
• Visualization has moved from static cartography or simple animations to dynamic interactive visualization.
• Finally, mapped content is no longer anonymous or regulated, but highly personal and narrative.

Key GIF milestones:

• 2005 GIIF (Geospatial Imaging and Informatics Facility) launches
• 2006 OakMapper changes from ArcIMS to Google Earth API
• 2008 GIIF becomes GIF
• 2008 OakMapper 2.0 launches
• 2008 SNAMP website launches
• 2011 Cal-Adapt goes live
• 2013 EcoEngine/HOLOS goes live
• 2014 LandCarbon launches
• 2014 GIF and Cal-Adapt go to the White House
• 2014 vtm.berkeley.edu goes live, built from the HOLOS API
• 2015 Spatial Data Science bootcamp in May

Onwards and upwards!