blog authors
past blog entries

Welcome to the Kellylab blog

geospatial matters

Please read the UC Berkeley Computer Use Policy. Only members can post comments on this blog.

Entries in open source (55)

Friday
Apr012016

ASTER Data Open - No April Fools!

We know about the amazing success for science, education, government, and business that has resulted from the opening of the Landsat archive in 2008. Now more encouraging news about open data:

On April 1, 2016, NASA's Land Processes Distributed Active Archive Center (LP DAAC) began distributing ASTER Level 1 Precision Terrain Corrected Registered At-Sensor Radiance (AST_L1T) data products over the entire globe at no charge. Global distribution of these data at no charge is a result of a policy change made by NASA and Japan.

The AST_L1T product provides a quick turn-around of consistent GIS-ready data as a multi-file product, which includes a HDF-EOS data file, full-resolution composite images (FRI) as GeoTIFFs for tasked telescopes (e.g., VNIR/SWIR and TIR ), and associated metadata files. In addition, each AST_L1T granule contains related products including low-resolution browse and, when applicable, a Quality Assurance (QA) browse and QA text report.

More than 2.95 million scenes of archived data are now available for direct download through the LP DAAC Data Pool and for search and download through NASA‘s Earthdata Search Client and also through USGS‘ GloVis , and USGS‘ EarthExplorer . New scenes will be added as they are acquired and archived.

ASTER is a partnership between NASA, Japan‘s Ministry of Economy, Trade and Industry (METI), the National Institute of Advanced Industrial Science and Technology (AIST) in Japan, and Japan Space Systems (J-spacesystems ).

Visit the LP DAAC ASTER Policy Change Page to learn more about ASTER. Subscribe to the LP DAAC listserv for future announcements.

Tuesday
Jan122016

Spatial Data Science Bootcamp March 2016

Register now for the March 2016 Spatial Data Science Bootcamp at UC Berkeley!

We live in a world where the importance and availability of spatial data are ever increasing. Today’s marketplace needs trained spatial data analysts who can:

  • compile disparate data from multiple sources;
  • use easily available and open technology for robust data analysis, sharing, and publication;
  • apply core spatial analysis methods;
  • and utilize visualization tools to communicate with project managers, the public, and other stakeholders.

To help meet this demand, International and Executive Programs (IEP) and the Geospatial Innovation Facility (GIF) are hosting a 3-day intensive Bootcamp on Spatial Data Science on March 23-25, 2016 at UC Berkeley.

With this Spatial Data Science Bootcamp for professionals, you will learn how to integrate modern Spatial Data Science techniques into your workflow through hands-on exercises that leverage today's latest open source and cloud/web-based technologies. We look forward to seeing you here!

To apply and for more information, please visit the Spatial Data Science Bootcamp website.

Limited space available. Application due on February 19th, 2016.

Wednesday
Mar112015

print 'Hello World (from FOSS4G NA 2015)'

FOSS4G NA 2015 is going on this week in the Bay Area, and so far, it has been a great conference.

Monday had a great line-up of tutorials (including mine on PySAL and Rasterio), and yesterday was full of inspiring talks.  Highlights of my day: PostGIS Feature Frenzy, a new geoprocessing Python package called PyGeoprocessing, just released last Thurs(!) from our colleagues down at Stanford who work on the Natural Capital Project, and a very interesting talk about AppGeo's history and future of integrating open source geospatial solutions into their business applications. 

The talk by Michael Terner from AppGeo echoed my own ideas about tool development (one that is also shared by many others including ESRI) that open source, closed source and commercial ventures are not mutually exclusive and can often be leveraged in one project to maximize the benefits that each brings. No one tool will satisfy all needs.

In fact, at the end of my talk yesterday on Spatial Data Analysis in Python, someone had a great comment related to this: "Everytime I start a project, I always wonder if this is going to be the one where I stay in Python all the way through..."  He encouraged me to be honest about that reality and also about how Python is not always the easiest or best option.

Similarly, in his talk about the history and future of PostGIS features, Paul Ramsey from CartoDB also reflected on how PostGIS is really great for geoprocessing because it leverages the benefits of database functionality (SQL, spatial querying, indexing) but that it is not so strong at spatial data analysis that requires mathematical operations like interpolation, spatial auto-correleation, etc. He ended by saying that he is interested in expanding those capabilities but the reality is that there are so many other tools that already do that.  PostGIS may never be as good at mathematical functions as those other options, and why should we expect one tool to be great at everything?  I completely agree.

Saturday
Feb142015

Landsat Seen as Stunning Return on Public Investment 

Undersanding the value of Landsat program to the U.S. economy has been the ambitious goal of the Landsat Advisory Group of the National Geospatial Advisory Committee. This team of commercial, state/local government, and NGO geospatial information experts recently updated a critical review of the value of Landsat information that has recently been released to the public.

They found that the economic value of just one year of Landsat data far exceeds the multi-year total cost of building, launching, and managing Landsat satellites and sensors.  This would be considered a stunning return on investment in any conventional business setting.

Full article by Jon Campbell, U.S. Geological Survey found here.

Friday
Nov212014

Map of open source map resources (as of 2012)

From this great paper I just came across (already much has changed in 2 years, but still cool):

Stefan Steiniger and Andrew J.S. Hunter, 2013. The 2012 free and open source GIS software map – A guide to facilitate research, development, and adoption. Computers, Environment, and Urban Systems. Volume 39: 136–150.

From the paper: "Over the last decade an increasing number of free and open source software projects have been founded that concentrate on developing several types of software for geographic data collection, storage, analysis and visualization. We first identify the drivers of such software projects and identify different types of geographic information software, e.g. desktop GIS, remote sensing software, server GIS etc. We then list the major projects for each software category. Afterwards we discuss the points that should be considered if free and open source software is to be selected for use in business and research, such as software functionality, license types and their restrictions, developer and user community characteristics, etc. Finally possible future developments are addressed."

Wednesday
Nov122014

Web mapping of high res imagery helps conservation

One of our collaborators on the Sonoma Vegetation Mapping Project has sent work on how web mapping and high resolution imagery has helped them do their job well. These are specific comments, but might be more generally applicable to other mapping and conservation arenas.

  1. Communicating with partnering agencies.
    • In the past year this included both large wetland restoration projects and the transfer of ownership of several thousands of acres to new stewards.
  2. Articulating to potential donors the context and resources of significant properties that became available for purchase.
    • There are properties that have been identified as high priority conservation areas for decades and require quick action or the opportunity to protect would pass.
  3. Internal communication to our own staff.
    • We have been involved in the protection of over 75 properties, over 47,000 acres. At this time we own 18 properties (~6500 acres) and 41 conservation easements (~7000  acres). At this scale high quality aerial imagery is essential to the size of land we steward and effective broad understanding. The way it is served as a seamless mosaic means it is available to extremely experienced and intelligent people who find the process of searching and joining orthorectified imagery by the flight path and row cumbersome or inefficient.
  4. Researching properties of interest.
    • Besides our own internal prioritization of parcels to protect, I understand that we receive a request a week for our organization’s attention towards some property in Sonoma. Orienting ourselves to the place always includes a map with the property boundary using the most recent and/or high quality imagery for the parcel of interest and its neighbors.  This is such a regular part of our process that we created a ArcGIS Server based toolset that streamlines this research task and cartography. The imagery service we consume as the basemap for all these maps is now the 2011 imagery service.  This imagery is of high enough resolution that we can count on it for both regional and parcel scale inspection to support our decisions to apply our resources.
  5. Orienting participants to site.
    • Our On the Land Program uses the imagery in their introduction maps to help visitors on guided hikes quickly orient to the place they are visiting and start folding their experience and sense of place into their visit.
  6. Complementing grant applications.
    • Grants are an important part of the funding for major projects we undertake. High quality imagery facilitates our ability to orient the grant reviewer and visually support the argument we are making which is that our efforts will be effective and worthy of funds that are in short supply.
  7. Knowing what the resources on a property are is an essential part of thoughtfully managing them.
    • In one example we used the aerial imagery (only a year old at the time) as a base map for botanists to classify the vegetation communities. These botanists are not experts in GIS, but by using paper maps with high resolution prints in the field they were easily able to delineate what they were observing on the ground on features interpreted in the photo.  We then scanned and confidently registered their hand annotations to the same imagery, allowing staff to digitized the polygons that represent the habitat observed. These vegetation observations are shared with Sonoma County and its efforts to map all the vegetation of Sonoma County.
  8. Conservation easement monitoring makes extensive use of aerial imagery.
    • In some cases we catch violations of our easements that are difficult to view on the ground, for example unpermitted buildings by neighbors on the lands we protect, illegal agriculture or other encroachment. It is often used to orient new and old staff to a large property before walking their and planning for work projects that might be part of prescribed management.
  9. The imagery helps reinforce our efforts to communicate the challenge to preserve essential connectivity in the developed and undeveloped areas of Sonoma County.
    • In the Sonoma Valley there is a wildlife corridor of great interest to us as conservation priority. Aerial imagery has been an important part of discussing large land holdings such as the Sonoma Developmental Center, existing conserved land by Sonoma Land Trust and others, and the uses of the valley for housing and agriculture.
  10. Celebration of the landscape cannot be forgotten.
    • We often pair this high quality aerial imagery with artful nature photography. The message of the parts and their relation to the whole are succinctly and poetically made. This is essential feedback to members and donors who need to see the numbers of acres protected with their support and have the heartfelt sense of success.

We look forward to the continued use of this data and the effective way it is shared.
 
We hope that future imagery and other raster or elevation data can be served as well as this, it would benefit many engaged in science and conservation.

Thanks to Joseph Kinyon, GIS Manager, Sonoma Land Trust

Wednesday
Oct222014

Check out the ZevRoss blog: Know Your Data

Technical Tidbits From Spatial Analysis & Data Science. This nice blog highlights many technical aspects of web mapping: Leaflet, D3, R... lots of neat examples.

Friday
Jan242014

Using Social Media to Discover Public Values, Interests, and Perceptions about Cattle Grazing on Park Lands

“Moment of Truth—and she was face to faces with this small herd…” Photo and comment by Flickr™ user, Doug GreenbergIn a recent open access journal article published in Envrionmental Management, colleague Sheila Barry explored the use of personal photography in social media to gain insight into public perceptions of livestock grazing in public spaces. In this innovative paper, Sheila examined views, interests, and concerns about cows and grazing on the photo-sharing website, FlickrTM. The data were developed from photos and associated comments posted on Flickr™ from February 2002 to October 2009 from San Francisco Bay Area parks, derived from searching photo titles, tags, and comments for location terms, such as park names, and subject terms, such as cow(s) and grazing. She found perceptions about cattle grazing that seldom show up at a public meeting or in surveys. Results suggest that social media analysis can help develop a more nuanced understanding of public viewpoints useful in making decisions and creating outreach and education programs for public grazing lands. This study demonstrates that using such media can be useful in gaining an understanding of public concerns about natural resource management. Very cool stuff!

Open Access Link: http://link.springer.com/article/10.1007/s00267-013-0216-4/fulltext.html?wt_mc=alerts:TOCjournals

Wednesday
Nov132013

Differences between Google Maps and Open Street Map

For class discussion: here is one look at the difference between Google Maps and Open Street Map.

From http://www.theguardian.com/news/datablog/2012/mar/28/openstreetmap-google-maps-technologies#

Tuesday
Jul232013

New SOD map from Google Maps Engine

Here is my attempt at a map from Google Maps Engine!

Thanks Karin Tuxen-Bettman for the help!