AWS Re:Invent Hackathon For Good 2019

SkyTruth Challenge + PRFAQ

CHALLENGE

SkyTruth hopes to *stop pollution in the ocean* so marine ecosystems and coastal communities can thrive. Your challenge is to build the key components of a platform that will *monitor the whole ocean, all the time*, and empower citizens, law enforcement, NGOs, and even retailers to work in unison to bring polluters to justice. You can *choose *to either build models that reliably detect oil slicks at sea and identify the polluters; demonstrate a global-scale data analytics infrastructure using their data; or design engaging interfaces that make everyone an ocean warrior.

PRESS RELEASE

*Sea It, Change It*
SkyTruth’s ocean-monitoring platform unified international efforts to bring justice to the high seas, and end illegal oil pollution in the ocean

Seattle, January 2, 2024 - Five years ago in 2019, a mysterious oil spill devastated over 2,250 km (1,400 mi) of coastline in northeastern Brazil. It took months of manual data analysis to track down the perpetrators. The incident was a clarion call to address the growing problem of a common illegal practice -- bilge dumping.

Bilge is the dirty, oily wastewater that accumulates below decks in a vessel. Bilge dumping at sea has been illegal under international maritime law since the 1970s. Yet all too many ship operators continue to deliberately dump their untreated, oily wastewater directly into the ocean. This toxic slurry harms marine life, and doesn't always stay out at sea: "Mystery" oil spills affect beaches and seashores around the world, coating, suffocating, and poisoning marine life like sea turtles and shorebirds, while also closing fisheries and driving away tourists.

"It's been a miracle, what's happened on our beach," said Yasmin Sousa, referring to the improvement over the last five years. Sousa operates a small coastal ecotourism business in northeastern Brazil. "The oil in the sand smelled awful, and it killed the turtles. No one knew where it came from." Sousa and her neighbors thought themselves helpless to combat a crisis that started hundreds of kilometers offshore. But that all started to change when SkyTruth, a nonprofit organization dedicated to environmental justice, sponsored a challenge at the 2019 Amazon Web Services (AWS) re:Invent conference.

SkyTruth developed Cerulean, a cloud-based platform that provides real-time visibility into global shipping traffic and detects pollution events; enabling individuals like Sousa to collaborate with NGOs and law enforcement organizations to reduce bilge dumping incidents by 85%.

Cerulean continuously applies deep learning neural networks to satellite images as they are downlinked to Earth and integrates this data stream with automatic identification system (AIS) vessel tracking data, processing an area of 100,000 km² every second. SkyTruth’s platform effectively monitors all the oceans, all the time, and its interface (API) lets software engineers at NGOs, businesses and governments across the globe develop applications that empower both citizens and government officials alike. Cerulean also includes an alerts system that provides real-time notifications to maritime law enforcement agencies, allowing them to get to the scene of the oily crime before perpetrators have disappeared beyond the horizon. Users can sign up directly at skytruth (http://skytruth.org/).org (http://skytruth.org/) to immediately get notified whenever a bilge slick is detected in their area of interest, or browse an interactive map to see patterns of pollution that reveal the locations of bilge-dumping "hotspots" in the ocean.

When an oil slick is detected, Cerulean alerts enforcement agencies with real time notifications and GPS coordinates identifying criminal violations. Additionally, each offense is accompanied by a timestamped image depicting the unlawful act and details about the culprit such as radio call sign, course of navigation, and port of destination. Enforcement officials can leverage this reconnaissance to intercept the perpetrator and collect additional material evidence for use during prosecution. Thus far, 78 countries have subscribed over 5,000 enforcement vessels to Cerulean’s alerts service, resulting in a 20% drop in incidents for every year the alerts are used on patrol, compared to fleets that don’t subscribe to the service.

In addition, in 2020, e-commerce giant Amazon partnered with SkyTruth to develop LightSea, a service that lets everyday consumers track their purchases on certified ships that pass regular safety inspections, maintain zero climate violations and active tracking through their entire journey, and are thus rated with a high Vessel Sustainability Score. This means that consumers can now help keep their coast clean by choosing to shop with businesses using LightSea, powered by the SkyTruth Cerulean platform.

The ocean was once seen as a wild west of lawless activity. “It’s an evergreen truth about human nature, that a person’s behavior changes when they know they’re being watched,” said John Amos, founder of SkyTruth. “That turned out to be true here -- vessel operators started to worry they would be caught, and they began to actually follow the law.”

Today, the future for coastal communities looks bright, as the scourge of bilge dumping nears extinction. As Amazon founder Jeff Bezos put it, “We knew this was a problem that could be solved, and we knew we were the ones who were going to do it. So we did.”

FAQ

*Show me the data!*
BILGE DUMP LOCATIONS
We have documented 124 suspected instances of bilge dumping on 104 different Sentinel-1 satellite images. We have created GeoJSON features that trace each one by GPS coordinate. That database is stored in an S3 bucket here: _https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/Oil_Slick_Database.csv_ (https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/Oil_Slick_Database.csv)

The description of the dataset is stored here:
_https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/Oil_Slick_ReadMe.pdf_ (https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/Oil_Slick_ReadMe.pdf)

BILGE DUMP IMAGES
In addition to pre-generating GeoTIFFs and PNGs for you, we created pixel masks for each satellite image that classify the pixels into two classes: black is background, and white is bilge dump. These images and their masks are available on S3 as well, details on accessing them are found at the bottom of this document:
_https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/Oil_Slick_ReadMe.pdf_ (https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/Oil_Slick_ReadMe.pdf)

*What is bilge dumping, and why is it a problem?*
Bilge dumping is the practice of illegally dumping oily “bilge water” at sea. Most modern cargo and container vessels use heavy bunker oil for fuel, and it produces a thick, oily sludge when burned. The majority of this sludge — along with most of the waste from the ship — drains into the bottom of the vessel, into an area known as the bilge tanks. The bilge tanks need to be emptied regularly. International law specifies the proper handling of bilge water, but the unfortunate reality is that it is usually cheaper and easier to bypass costly pollution prevention equipment by simply flushing the bilge tanks directly into the sea. It’s estimated that as much as _230 million gallons_ (https://www.marinedefenders.org/oil-pollution-facts.html) of oil are dumped into the ocean every year in this way.

*Why hasn’t this problem already been solved? What are the challenges?*
Because the dumping often happens over the horizon, far out at sea. And by the time oil starts washing up on your beach, the perpetrators are long gone.

*Where does bilge dumping happen?*
Bilge dumping happens all over the world. In fact, that’s one of the reasons that we’re so interested in automating the detection process — the problem is simply too widespread for our small team of analysts to look at all of the satellite radar imagery of the oceans we’d need to. But there are a few places in the world where bilge dumping is more common: (1) far out in the ocean where no country is really monitoring, and (2) in major international shipping lanes, and (3) in the waters of coastal countries that can’t afford robust Navy or Coast Guard patrols.

*How often does bilge dumping happen?*
We don’t know, because nobody has been systematically monitoring the ocean to collect that data — until now!

*Who can take action to stop bilge dumping, and what can they do?*
It’s going to take action from a lot of different groups to stop bilge dumping. We think that a few of the most important ones are:

* enforcement authorities, like navies and coast guards, that can use this data to intercept violators in their waters;
* port authorities that can use the data to inspect vessels and fine the operators of poorly behaving vessels when they come into port;
* private companies, like Amazon, that can use the data to monitor their shipping partners and ensure environmentally responsible supply chains;
* commercial vessel insurance providers that can withhold policies from repeat offenders;
* flag state compliance officers that can delist vessels that violate international law; and
* conservation nonprofits that can use the map and data to raise public awareness about bilge dumping and oil pollution in the oceans, and build political pressure to do something about it.

*What does SkyTruth need to make this happen?*
We need help with:

* developing a model from a sparse set of training images (only about 100);
* creating and deploying a data processing pipeline so the model can continuously make predictions on new satellite images as they come into Earth on AWS (about 1,000 new images each day, with potential to grow significantly beyond that);
* Automating the correlation of bilge slicks with AIS data showing vessel movements, to identify the likely perpetrators;
* publishing model detections and polluter identifications onto a web map, in a dashboard, or via some other brilliant front end that you’ll come up with to display global pollution events for the public, the media, and policymakers; and
* notifying enforcement authorities and other actors whenever our system discovers a new pollution event.

*What does bilge dumping look like, and how can we spot it?*
SkyTruth uses satellite radar imagery to identify oil slicks. In this type of imagery, bilge dumps typically take the form of long, slender black streaks on the ocean. In some cases, there is also a bright white spot at one end of the streak. Much of the time, this bright white spot is the vessel responsible for the slick. The image below offers a good example:

Not all slicks visible on radar imagery are bilge slicks. Heavy rain, slack wind, leaking oil platforms and other phenomena in the ocean can also create dark slicks on imagery, although bilge slicks are very distinctive. Also, bilge dumping is intentional, but there are other reasons that a slick might be visible trailing a vessel: there could be a serious mechanical problem and the vessel might be in distress. In any event, it’s important to be able to detect these pollution events and identify the vessels that are causing them.

*What is SAR? Why is it useful, and where can I get it?*
SAR is synthetic aperture radar imagery: an image of the Earth’s surface made by beaming radar energy down at the planet, and measuring the signal that comes back up. We’re particularly interested in SAR imagery collected from two orbiting satellites, called _Sentinel-1A and Sentinel-1B_ (https://sentinel.esa.int/web/sentinel/missions/sentinel-1), operated by the European Space Agency. The radar sensors on the Sentinel-1 satellites operate at a frequency that makes them very sensitive to the roughness of the ocean surface. Under typical conditions, with a light steady wind, small ripples cover the water and scatter radar imagery back up at the satellite, giving the ocean a medium-gray tone. Any flat patches of ocean, with no ripples, appear black (ask us why at the hackathon!). Because oil smoothes the surface of the water, it creates a black slick on a radar image. A moving vessel that is dumping oily bilge creates a long, linear slick that is quite distinctive, as in the image shown above.

Because radar satellites are creating their own source of energy to “light up” the Earth’s surface, they can operate day and night; and radar cuts through clouds, smoke and haze, creating useful images under conditions that would render optical (visible + near-infrared) imagery useless. This makes radar a much more valuable tool for continuous ocean monitoring.

You can gather more Sentinel-1 imagery than we have provided through these links:
_https://roda.sentinel-hub.com/sentinel-s1-l1c/GRD/readme.html_ (https://roda.sentinel-hub.com/sentinel-s1-l1c/GRD/readme.html)
The information page on the satellites has links to "Tools & Applications":
_https://registry.opendata.aws/sentinel-1/_ (https://registry.opendata.aws/sentinel-1/)
_https://search.remotepixel.ca/#3/38.97/-96.45_ (https://search.remotepixel.ca/#3/38.97/-96.45)
Forums also have a lot of good information about browsers and troubleshooting bucket access:
_https://forum.sentinel-hub.com/t/how-to-access-s3-data-from-aws/171/14_ (https://forum.sentinel-hub.com/t/how-to-access-s3-data-from-aws/171/14)

*How can we **identify** the vessels that are polluting?*
Satellite radar imagery doesn’t have enough detail to identify the vessels, which simply appear as very bright spots because metal objects are such strong radar reflectors. So we have to use another dataset to make the identification possible. Most of the bilge-dumping incidents we’ve seen over the years have been caused by large cargo ships and oil/chemical tankers.

Under _international maritime safety law_ (http://www.imo.org/en/OurWork/Safety/Navigation/Pages/AIS.aspx) (SOLAS), these vessels and many others (passenger ships, research vessels, large fishing vessels) are required to continuously broadcast a radio-frequency identification signal when they’re at sea, using a public, open safety system called the Automatic Identification System (AIS).

*What is AIS? Why is it useful, and where can I get it? *
_AIS_ (https://en.wikipedia.org/wiki/Automatic_identification_system) is VHF radio gear installed on a ship to broadcast information packets as frequently as every few seconds, to alert other nearby vessels to your location, course, speed, identity, and other useful information. These broadcasts are public and unencrypted, designed to help ships avoid collision at sea. The broadcasts are also collected by networks of land-based radio receivers (“terrestrial AIS”) as well as AIS receivers carried on a rapidly growing number of orbiting satellites (“satellite AIS”). Companies like Spire and Orbcomm and others package the data for sale (_Spire_ (https://spire.com/en) has generously donated a large AIS sample dataset for use during this hackathon). If you can help us confidently correlate the slicks and vessels that we see on satellite images of bilge dumping, with the AIS broadcasts of vessels passing through the same area during that time, _we can identify the likely polluters_ (https://skytruth.org/2019/03/perkasa-caught-bilge-dumping/).

The dataset and access to it is described here: _https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/ais_data/AIS_ReadMe.pdf_ (https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/ais_data/AIS_ReadMe.pdf)

*I want to train a machine learning model to detect bilge slicks. Where can I get training data?*
Awesome! Accessing the images and labeled dataset is described in the second half of this document: _https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/Oil_Slick_ReadMe.pdf_ (https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/Oil_Slick_ReadMe.pdf)

*I don’t know how to train a machine learning model. Do you have one pretrained that I can play with?*
You bet! A stand-in TensorFlow U-net segmentation model is available here:
_https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/tf_model/TF_Model_Full.zip_ (https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/tf_model/TF_Model_Full.zip)

Some sample images that it accepts can be found here: _https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/tf_model/TF_Model_Samples.zip_ (https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/tf_model/TF_Model_Samples.zip)

Snippets to get you started on inference can are here:
_https://colab.research.google.com/drive/1PtFQLW5ON6Cz8hTGxa_k4vKKV_2-3o2q_ (https://colab.research.google.com/drive/1PtFQLW5ON6Cz8hTGxa_k4vKKV_2-3o2q)

*How can an ML model scale to ingest imagery of the entire ocean, every day, efficiently, and cheaply?*
We’re not really sure yet. That’s why we need your help!

*Exactly what information is in the bilge database? *
We’ve got a lot of good information in our bilge database. It’s also stored in an Amazon S3 bucket that we can help you get access to. The columns in the database include:

* sentinel_image_id → Sentinel-1 radar satellite image ID
* date → Image acquisition date
* polygon_mask → Geometry of the bilge slick
* length → Length of the bilge slick
* ship_lon → longitude of the potentially responsible vessel
* ship_lat → latitude of the potentially responsible vessel

More details and sample fields here: _https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/Oil_Slick_ReadMe.pdf_ (https://skytruth-aws-hackathon.s3-us-west-2.amazonaws.com/Oil_Slick_ReadMe.pdf)

*What projection do the GeoTIFFs use?*
The images are all in UTM projection using the WGS 84 datum. The UTM zones are different based on where the image is located. For instance, some images will be UTM Zone 40N (EPSG:32640), while others may be Zone 48S (EPSG:32748). It can be a bit confusing at times, so if you have questions, just ask.

*What additional datasets might I play with?*
We can point you to a few other really interesting datasets that you can work with, including:

* marine protected areas,
* sea surface winds, and
* major global shipping lanes

*Who generously provided these datasets?*
Sentinel-1 images are made available by the _European Space Agency_ (https://scihub.copernicus.eu/) and AWS. The AIS data is provided by _SPIRE_ (https://spire.com/en), for use during this hackathon. Our preliminary ML model for detecting bilge slicks on Sentinel-1 imagery was built by a team of engineers in the UK: Alex Wilson, Rob Bacon and Chris Bacon.

*Who is SkyTruth?*
_SkyTruth_ (https://skytruth.org/) is a technology-driven nonprofit with a mission to protect the environment by making more of it visible. We do this by using satellite imagery and remote sensing data to identify and monitor threats to the planet’s natural resources such as offshore drilling and oil spills, urban sprawl, fracking, mountaintop removal mining, and overfishing of the oceans.

We are driven by the belief that better transparency leads to better management and better outcomes. By sharing our findings — stunning imagery and robust science-based data — with the public for free, we move policy makers, governments and corporations towards more responsible behavior in the environment. We arm citizen activists with the tools they need to be more effective advocates. We also provide researchers and scientists with critical data that can inform groundbreaking work and, notably, aid in the effort to begin asking a new set of questions. And we’re always _looking for smart people_ (https://skytruth.org/about/careers/) like you!