2020 ESIP Winter Meeting

SpatioTemporal Asset Catalogs is an emerging specification of a common metadata model for geospatial data, and a way to make data catalogs indexable and searchable. We have already seen STAC being adopted for both public data and commercial data. Catalogs exist for several AWS Public Datasets, Landsat Collection 2 data will be published along with STAC metadata, and communities like Pangeo are using STAC to organize data repositories in a scalable way. Commercial companies like Planet and Digital Globe are starting to publish STAC metadata for some of their catalogs. Session talks may cover overviews of the STAC, software projects utilizing STAC, and use cases of STAC in organizations. How to Prepare for this Session: See https://stacspec.org/.

View Recording:https://youtu.be/BdZbJLQSNFE.

Takeaways

• Checkout STAC at https://stacspec.org/ and follow along (or join) the discussion at https://gitter.im/SpatioTemporal-Asset-Catalog/Lobby
• See live STAC cataloges at stac.cloud. Some additional tools worth checking out include the STAC validator https://github.com/sparkgeo/stac-validator and STAC web linter http://staclint.com/.

NASA, NOAA and USGS are in the process of moving data onto the cloud. While they have discussed what types of services are available and future plans of what data can be found, it is not completely clear what datasets users can currently access. This session will go over what datasets are currently up in the cloud and what data to expect in the near future. This way as users are transitioning to the cloud for their compute, they can also know what data are available to them on the cloud as well. There will also be presentations from AWS. Speakers:
Katie Baynes - NASA/EOSDIS
Jon O'Neil - NOAA
Jeff de La Beaujardiere - NCAR
Kristi Kliene - USGS/EROS
Joe Flasher - AWS

Presentations: See attached.

View Recording: https://youtu.be/yssgXB7iaxw

Takeaways

• Petabyte scale data is being moved into the cloud. This is concentrated in AWS, Google Cloud and Microsoft depending on the agency and dataset
• Some concern around partnerships with companies (AWS most discussed) in terms of long term relationships, moving data etc. and how those things might impact access or data use
• Need to make clear the authoritative source of the data, who is stewarding it, and any modifications done when copying to cloud. Users should exercise due diligence in selecting and using data.

We will describe our project to upload a 2.4 PB dataset encapsulated into ~80K fused files from the 5 instruments on the Terra satellite into NASA AWS S3.
We will share the bottlenecks points and lessons learned during this process and expect to share experiences with similar projects in order to understand the best practices and collect guidelines for future projects that are adopting cloud solutions for their data needs.

We'll discuss data volumes, data integrity strategies for migration, S3 bucket organization, metadata curation, transfer rates, transfer pipelines, etc. We will also discuss and share data access patterns, costs, and architectures and how we can construct guidelines for access to these datasets efficiently.

We encourage the discussion among different projects that faced similar processes or are looking to migrate their datasets into the cloud.

https://drive.google.com/file/d/1fts06XDM2dbZxxljBTpplCEMSiTqfp6t/view?usp=sharing

Presentations:
https://doi.org/10.6084/m9.figshare.11553147.v1

View Recording: https://youtu.be/1xVJghJI4Gg

Takeaways

• Project required/used a combination of NSF, NASA and AWS resources. Some interesting discussion around AWS or other cloud services as a stand in or follow on to limited term NSF assets
• Some interesting discussion of tailoring to appropriate end users- wide range of potential users and thus requirements for the dataset. This includes access guidelines, user capabilities etc.
• Project aimed to make a paradigm shift from understanding/observing physical processes to a full climate observing objective

This session provides an overview of the results from the recent OGC Research & Development initiative Testbed-15. The 9-months 5M USD initiative addressed six different topics, Earth Observation Process and Application Discovery, Machine Learning, Federated Cloud Analytics, Open Portrayal Framework, Delta Updates, and Data Centric Security. This session focuses on the results produced by the first three.

Earth Observation Process and Application Discovery developed draft specifications and models for discovery of cloud-provided process and applications. This was achieved by extending existing standards with process and application specific extensions. Now, data processing software can be made available as a service, discovered using catalog interfaces, and executed on demand by customers. This allows to locate the process execution physically close to the data and reduces data transport overheads.

The Machine Learning research developed models in the areas of earth observation data processing, image classification, feature extraction and segmentation, vector attribution, discovery and cataloguing, forest inventory management & optimization, and semantic web-link building and triple generation. Both model discovery and access took place through standardized interfaces.

The Federated Cloud Analytics research analysed how to handle data and processing capacities that are provided by individual cloud environments transparently to the user. The research included how federated membership, resource, and access policy management can be provided within a security environment, while also providing portability and interoperability to all stakeholders. Additionally, the initiative conducted a study of the application of Distributed Ledger Technologies (DLTs), and more specifically Blockchains, for managing provenance information in Federated Cloud.

The other three topics will be briefly introduced in addition. The Open Portrayal Framework provides a fully interoperable portrayal and styling suite of standards. Here, the initiative developed new OGC APIs for styles, maps, images, and tiles. Delta updates explored incremental updates and thus reduced communication payloads between clients and servers, whereas the Data Centric Security thread examined the use of encrypted container formats on standard metadata bindings. How to Prepare for this Session: Al results will be made available as public Engineering Reports that provide full details. These become stepwise available at http://docs.opengeospatial.org/per/

Presentations:
https://doi.org/10.6084/m9.figshare.11551563.v1

View Recording: https://youtu.be/ojMrcIE-SgE

Takeaways

• OGC innovation program: Test fitness for purpose of geospatial community initiatives. TESTBED-15 concluded last November results available soon from document repository. End to end cloud pipeline for data processing and analytics. Call for TESTBED-16 due Feb 9th 2020! 1.6M in funding available. Three major threads: earth observation clouds, data integration and analytics, and modeling and packaging. 
• Way to synergize between needs of user communities competing and collaborating projects, contributing to a more interoperable world. Provides applications, process and catalogues for data processing. 
• Testbeds center around an exploitation/processing platform (for data with relevant applications) like an application market with cloud services. Having some trouble finding application developers. Finding web services with relevant data can be problematic.
  
  
  

Political, economic, social or scientific decision making is often based on integrated data from multiple sources across potentially many disciplines. To be useful, data need to be easy to discover and integrate.
This session will feature presentations highlighting recent breakthroughs and lessons learned from experimentation and implementation of open knowledge graph, linked data concepts and Discrete Global Grid Systems. Practicality and adoptability will be the emphasis - focusing on incremental opportunities that enable transformational capabilities using existing technologies. Best practices from the W3C Spatial Data on the Web Working Group, OGC Environmental Linked Features Interoperability Experiment, ESIP Science on Schema.org; implementation examples from Geoscience Australia, Ocean Leadership Consortium, USGS and other organisations will featured across the entire session.
This session will highlight how existing technologies and best practices can be combined to address important and common use cases that have been difficult if not impossible until recent developments. A follow up session will be used to seed future collaborative development through co-development, github issue creation, and open documentation generation.

How to Prepare for this Session: Review: https://opengeospatial.github.io/ELFIE/, https://github.com/ESIPFed/science-on-schema.org, https://www.w3.org/TR/sdw-bp/, and http://locationindex.org/.

Notes, links, and attendee contact info here.

View Recording: https://youtu.be/-raMt2Y1CdM

Session Agenda:
1.  2.00- 2.10,  Sylvain Grellet, Abdelfettah Feliachi, BRGM, France
'Linked data' the glue within interoperable information systems
“Our Environmental Information Systems are exposing environmental features, their monitoring systems and the observation they generate in an interoperable way (technical and semantic) for years. In Europe, there is even a legal obligation to such practices via the INSPIRE directive. However, the practice inducing data providers to set up services in a "Discovery > View > Download data" pattern hides data behind the services. This hinders data discovery and reuse. Linked Data on the Web Best Practices put this stack upside down and data is now back in the first line. This completely revamp the design and capacities of our Information Systems. We'll highlight the new data frontiers opened by such practices taking examples on the French National Groundwater Information Network”
View Slides: https://doi.org/10.6084/m9.figshare.11550570.v1

2.  2.10 - 2.20,  Adam Leadbetter, Rob Thomas, Marine Institute, Ireland
Using RDF Data Cubes for data visualization: an Irish pilot study for publishing environmental data to the semantic web
The Irish Wave and Weather Buoy Networks return metocean data at 5-60 minute intervals from 9 locations in the seas around Ireland. Outside of the Earth Sciences an example use case for these data is in supporting Blue Economy development and growth (e.g. renewable energy device development). The Marine Institute, as the operator of the buoy platforms, in partnership with the EU H2020 funded Open Government Intelligence project has published daily summary data from these buoys using the RDF DataCube model[1]. These daily statistics are available as Linked Data via a SPARQL endpoint making these data semantically interoperable and machine readable. This API underpins a pilot dashboard for data exploration and visualization. The dashboard presents the user with the ability to explore the data and derive plots for the historic summary data, while interactively subsetting from the full resolution data behind the statistics. Publishing environmental data with these technologies makes accessing environmental data available to developers outside those with Earth Science involvement and effectively lowers the entry bar for usage to those familiar with Linked Data technologies.
View Slides: https://doi.org/10.6084/m9.figshare.11550570.v1

3. 2.20 - 2.30,  Boyan Brodaric, Eric Boisvert, Geological Survey of Canada, Canada; David Blodgett, USGS, USA
Toward a Linked Water Data Infrastructure for North America
We will describe progress on a pilot project using Linked Data approaches to connect a wide variety of water-related information within Canada and the US, as well as across the shared border
View Slides: https://doi.org/10.6084/m9.figshare.11541984.v1

4.  2.30 - 2.40,  Dalia Varanka, E. Lynn Usery, USGS, USA
The Map as Knowledge Base; Integrating Linked Open Topographic Data from The National Map of the U.S. Geological Survey
This presentation describes the objectives, models, and approaches for a prototype system for cross-thematic topographic data integration based on semantic technology. The system framework offers a new perspectives on conceptual, logical, and physical system integration in contrast to widely used geographic information systems (GIS).
View Slides: https://doi.org/10.6084/m9.figshare.11541615.v1

5.  2.40 – 2.50,  Alistair Ritchie, Landcare, New Zealand
ELFIE at Landcare Research, New Zealand
Landcare Research, a New Zealand Government research institute, creates, manages and publishes a large set of observational and modelling data describing New Zealand’s land, soil, terrestrial biodiversity and invasive species. We are planning to use the findings of the ELFIE initiatives to guide the preparation of a default view of the data to help discovery (by Google), use (by web developers) and integration (into the large environmental data commons managed by other agencies). This integration will not only link data about the environment together, but will also expose more advanced data services. Initial work is focused on soil observation data, and the related scientific vocabularies, but we anticipate near universal application across our data holdings.
View Slides: https://doi.org/10.6084/m9.figshare.11550369.v1

6.  2.50 - 3.00,  Irina Bastrakova, Geoscience Australia, Australia
Location Index Project (Loc-I) – integration of data on people, business & the environment
Location Index (Loc-I) is a framework that provides a consistent way to seamlessly integrate data on people, business, and the environment.
Location Index aims to extend the characteristics of the foundation spatial data of taking geospatial data (multiple geographies) which is essential to support public safety and wellbeing, or critical for a national or government decision making that contributes significantly to economic, social and environmental sustainability and linking it with observational data. Through providing the infrastructure to suppo

This working session will follow on the "Advancing Data Integration approaches of the structured data web” session and the Coverage Analytics sprint as an opportunity for those interested in building linked data information products that integrate spatial features, coverage data, and more. As such, inspiration will be drawn from projects like science on schema.org, the Environmental Linked Features Interoperability Experiment, the Australian Location Index, and those that session attendees take part in. Participants will self organize into use-case or technology focused groups to discuss and synthesize the outcomes of the sprint and structured data web session. Session outcomes could take a number of forms: linked data and web page mock ups, ideas and issues for OGC, W3C, or ESIP groups to consider, example data or use cases for relevant software development projects to consider, or work plans and proposals for suture ESIP work. The session format is expected to be fluid with an ideation and group formation exercise followed by structured discussion to explore a set of ideas then narrow on a focused valuable outcome. Participants will be encouraged to work together prior to the meeting to design and plan the session structure. Outcomes of the session will be reported at an Information Technology and Interoperability webinar in early 2020. How to Prepare for this Session: Attend the coverage sprint and the "Advancing Data Integration approaches of the structured data web" session.

Shared document for session here.

Full Notes: https://doi.org/10.6084/m9.figshare.11559087.v1

Presentations:

View Recording: https://youtu.be/u2x3I0cr46A

• Takeaways
  Breakout session information interoperability committee and webinar series. See notes: https://docs.google.com/document/d/1LpcTMwP0mAD4G4Gb8mStI5uSDV61_qWPUkQ9nI1x1cI/edit?usp=sharing
• Foster cross-project consistency via breakouts. Such as dealing with science on schema.org issue of Links to “in-band” linked (meta)data and “out of band” linked data. Content negotiation and in-band and out of band links Use blank nodes with link properties for rdf elements that are URI for out of band content. Identify in band links with sdo @id, out of band links with sdo:URL
• Incorporating Spatial Coverages in Knowledge Graphs; Next Steps? Need to explore more on tessellations as an intermediate index. Will carry forward some of these ideas at the EDR SWG Will represent some of these ideas to the OGC-API Coverages SWG Will mention these ideas to the UFOKN Role of ‘spatial’ knowledge graphs Will spatial data analysis and transformation tools grow to adopt/support RDF as an underlying data structure for spatial information or will RDF continue to be a ‘view’ of existing (legacy) spatial data in GI systems?
• 
  

We will investigate graph-based methods of connecting data with the uses made and the knowledge gained from those data, from science research to applications to strategic planning. We will examine the diverse capabilities enabled by connecting uses with data for a variety of stakeholders, and explore how to connect existing knowledge graphs together to scale out across the ESIP federation and related communities toward an inter-connected mega-graph.

0-5 min: Chris Lynnes (NASA): Documenting how data matters...
5-15 min: Doug Newman (NASA): EOSDIS Knowledge Graph
https://doi.org/10.6084/m9.figshare.11561805.v1
15-25 min: Reid Sherman (GCIS): Global Change Information System
https://doi.org/10.6084/m9.figshare.11560011.v1
25-35 min: Dave Blodgett (USGS): SELFIE
https://doi.org/10.6084/m9.figshare.11559093.v1
35-45 min: Joe Conran (NOAA): Interagency Coordination of Satellite Needs
https://doi.org/10.6084/m9.figshare.11561946.v1
45-55 min: Wil Doane (IDA): Assessing the Impact of Land Imaging
https://doi.org/10.6084/m9.figshare.11561913.v1
55-90 min: The Way Forward:
1 - Got Use Case?
2 - ESIP Cluster? https://www.esipfed.org/get-involved/collaborate
3 - Who's In?

Session Notes

View Recording: https://youtu.be/yi05crW6Ya0\

Takeaways

• How to connect data with the uses of that data = Documenting how data matter.
  Federating knowledge bases is daunting task but possible.
• Connect research and data to place (but gap around using place identifiers in linked data).
  Discussion of potentially make a new cluster or using another one. Decision to recharter/repurpose/rename the data discovery cluster.
• Sin of computer science is giving people impression that things are mostly 1 to 1 relationship, but more accurately life and universe is full of many-to-many relationships, i.e., graph databases > RDBMS
  
  
  
  

This workshop session will follow up on the OGC Coverage Analytics sprint, focusing specifically on advanced services for spatio-temporal datacubes. In the Earth sciences datacubes are accepted as an enabling paradigm for offering massive spatio-temporal Earth data analysis-ready, more generally: easing access, extraction, analysis, and fusion. Also, datacubes homogenizes APIs across dimensions, allowing unified wrangling of 1-D sensor data, 2-D imagery, 3-D x/y/t image timeseries and x/y/z geophysics voxel data, and 4-D x/y/z/t climate and weather data.
Based on the OGC datacube reference implementation we introduce datacube concepts, state of standardization, and real-life 2D, 3D, and 4D examples utilizing services from three continents. Ample time will be available for discussion, and Internet-connected participants will be able to replay and modify many of the examples shown. Further, key datacube activities worldwide, within and beyond Earth sciences, will be related to.
Session outcomes could take a number of forms: ideas and issues for OGC, ISO, or ESIP to consider; example use cases; challenges not yet addressed sufficiently, and entirely novel use cases; work and collaboration plans for future ESIP work. Outcomes of the session will be reported at the next OGC TC meeting's Big Data and Coverage sessions. How to Prepare for this Session: Introductory and advanced material is available from http://myogc.org/go/coveragesDWG

Presentations
https://doi.org/10.6084/m9.figshare.11562552.v1

View Recording: https://youtu.be/82WG7soc5bk

Takeaways

• Abstract coverage construct defines the base which can be filled up with a coverage implementation schema. Important as previously implementation wasn’t interoperable with different servers and clients. 
• Have embedded the coordinate system retrieved from sensors reporting in real time into their xml schema to be able to integrate the sensor data into the broader system. Can deliver the data in addition to GML but JSON, and RDF which could be used to link into semantic web tech. 
• Principle is send HTTP url-encoded query to server and get some results that are extracted from datacube, e.g., sources from many hyperspectral images.