Loading…
Putting Data to Work: Building Public-Private Partnerships to Increase Resilience & Enhance the Socioeconomic Value of Data. Register at the early-bird rate by 12/6!

Let’s keep making data matter!

Tuesday, January 7 • 4:00pm - 5:30pm
Experiences Migrating Mission Scale Data in the Cloud

Sign up or log in to save this to your schedule and see who's attending!

We will describe our project to upload a 2.4 PB dataset encapsulated into ~80K fused files from the 5 instruments on the Terra satellite into NASA AWS S3.
We will share the bottlenecks points and lessons learned during this process and expect to share experiences with similar projects in order to understand the best practices and collect guidelines for future projects that are adopting cloud solutions for their data needs.

We'll discuss data volumes, data integrity strategies for migration, S3 bucket organization, metadata curation, transfer rates, transfer pipelines, etc. We will also discuss and share data access patterns, costs, and architectures and how we can construct guidelines for access to these datasets efficiently.

We encourage the discussion among different projects that faced similar processes or are looking to migrate their datasets into the cloud.
How to Prepare for this Session:

Speakers

Tuesday January 7, 2020 4:00pm - 5:30pm
White Flint
Feedback form isn't open yet.

Attendees (4)