An Introduction to Geo-Embeddings 

AI’s biggest breakthroughs all share a hidden DNA: embeddings. At LGND, we use embeddings to transform how we map, analyze, and discover changes on our planet. But what exactly is an “embedding” and how do they work in geospatial contexts?

What are embeddings?

The word “red” embedded by a neural network.

Embeddings are the internal representations that machine learning models, like neural networks, use to process inputs. They are vectors, or sequences of numbers, that represent more complex data like images or sentences. One can also interpret the numbers in a vector as a location in a multidimensional space. It turns out that the distance or angle between two embedding vectors in multidimensional space corresponds to the conceptual proximity of their inputs.

Embeddings first rose to popularity in text modeling in the 2010s – they allowed researchers to efficiently capture the relationships between different concepts. Consider the words: “red”, “blue”, “tomato” and “lettuce”: Embeddings of “red” and “blue”, both colors, are more similar than embeddings of “blue” and “lettuce”, a color and a vegetable respectively. Similarly, “red” and “tomato” are closer in embedding space than “blue” and “lettuce”, as tomatoes are (usually) red! These embedding relationships underpin the language models that so many of us use today.

From image embeddings to geo embeddings

Here at LGND we focus on image embeddings. Rather than represent words, image embeddings capture visual concepts and patterns in imagery. Returning to our example above, an image of a tomato and a red painting will be closer in image embedding space than those of a lettuce and blue painting–simply because tomatoes and the red painting share the same color. 

A satellite image embedded using the Clay foundation model.

It turns out we can apply the same logic to satellite images when we embed images of Earth using neural networks to better understand our planet. These geo-embeddings are produced by large earth observation models (LEOMs), like Clay, attuned to the particular structure of satellite imagery across different wavelengths of light. Geo-embeddings from Clay (and other LEOMs), much like text, can tell us about the relationship between different places. For example, geo-embeddings of Shanghai and Chicago will be more similar than geo-embeddings of Chicago and farmland in Illinois, despite their spatial proximity, because Shanghai and Chicago are functionally similar – characterized by the same visual features of skyscrapers, highways, airports, etc. 

The Upshot

Embeddings unlock fast and efficient similarity search. Given a few examples of a feature of interest (e.g. five different airports), the model can generate embeddings for the images of those locations and then query a database of pre-generated embeddings to find similar ones. Simply put, we can find new locations on Earth that resemble our referenced examples. 

The promise of geo embeddings has spurred the development of new models over the last few years, including NASA and IBM’s Prithvi and Google DeepMind’s Alpha Earth Foundations. But just like with raw satellite data, accessing geo embeddings can be tricky. The models are often difficult to work with. Many of them don’t let users generate the embeddings at specific times, locations and spatial resolutions. They may only be compatible with a single satellite sensor. The list goes on. 

At LGND we’re building infrastructure to easily generate and work with geo-embeddings . We want users to be able to query the planet in space and time without the need for them to collect, curate and process raw satellite data. We’re also building applications to easily run similarity searches with an interactive map and chat interface. By turning billions of pixels into searchable insights, we’re making it possible for anyone to ask complex questions of our Earth—and extract insights in seconds.