This patent by Amazon will change the way we search for things online and beyond

Neural networks and artificial intelligence are developing at an extremely high pace nowadays, changing our habits and the way things are done. One of the main directions of these developments is the work with images, both from the aspect of recognition and from the aspect of production, which can have lots of applications in various areas of our lives.

A method described in a patent newly released by Amazon under the name of “System and method for rare object localization and search in overhead imagery”, can be seen as another milestone in the development of image recognition, having a powerful potential for use in various areas. Today we want to have a look at what this patent is about and what it can give us.

What is the patent about?

In simple words, this patent is about a system and method that helps locate and search for rare objects in images taken from above, like satellite imagery. The system uses a neural network to extract features from images. These features help identify and match objects in the images.

The method works by applying a query image, which shows the object to be searched, and a search image, which contains the image where the object needs to be found, to the neural network. The neural network has different parts: one part analyzes the search image to extract spatial features, and another part analyzes the query image to extract embedding features. These features are like unique fingerprints that describe the objects.

After extracting the features, the method generates spatial search features from the spatial feature neural network and a query feature from the embedding network. Then, it applies the query feature to an artificial neural network (ANN) index, which is like a big database that helps match the features with known objects.

Finally, the method determines the best match for the object in the search image by using the ANN index. This helps find rare objects in the images without needing to modify or reprocess the features or the database. The system and method described in this patent have shown good results in locating objects in satellite imagery and can be used in various applications where precise object localization is required.

In summary, this patent introduces a system and method that use a neural network to locate and search for rare objects in overhead images. The system analyzes features from the images, matches them with known objects using a database, and provides accurate results without needing extensive adjustments or modifications.

The principle of operation of the new method

The applications of the method

The method described was primarily developed for searching for objects on the images made by satellites. However, its possible applications may go beyond this field. What is special about this particular method is that it enables finding particular objects on an image that contains a lot of other complex details. While previous methods allowed the recognition of objects that are in the focus of the image, the method discussed in the article allows us to detect something that may be hardly noticeable to the human eye and quickly classify it. One of the most practical applications is in the field of medicine, allowing for quick detection of certain ailments just by looking at the tissue, simplifying the diagnostic process in such areas as histopathology and others.

Locating and searching for an image on an overhead photograph

Beyond the areas discussed in the article, the method can also find its applications in various other fields. To name a few:

1. Environmental Monitoring: The method can be applied to overhead images to identify rare and endangered species or track changes in ecosystems, to provide a better understanding of what needs to be done.
2. Security and Surveillance: In security and surveillance applications, the method can assist in locating suspicious objects or individuals within crowded scenes. It can be utilized in airports, public spaces, or critical infrastructure sites to enhance threat detection and response capabilities.
3. Geology and Mining: The method can be valuable in the geology and mining sectors. It can help identify mineral deposits, geological formations, or potential excavation sites by analyzing high-resolution satellite or aerial imagery.
4. Infrastructure and Urban Planning: Urban planners and engineers can benefit from this method by using overhead imagery to locate underground utilities, assess infrastructure conditions, or plan new developments. It can expedite the process of identifying objects of interest within complex urban environments.

The method, however, can also be beneficial for ordinary users. Let’s consider a few possible applications:

1. Online search: While previous methods worked better in cases when the object in question is in the foreground, the new method allows you to pick up an inconspicuous from an image of a bigger scale. This way, by making a photograph of a crowd of people, you can find a particular t-shirt you liked – which opens great perspectives for the uses in the field of e-commerce as well.
2. Personal Safety: In the context of personal safety, the method can contribute to improved surveillance systems and object recognition. It can aid in identifying potential hazards or suspicious objects in public spaces, thereby enhancing overall security and providing peace of mind.
3. Personal Photography and Image Organization: A big library of photos can be something hard to manage, and looking for a particular photograph can take a lot of time. The method can be helpful here as well, since it can help automatically tag and categorize photos based on the objects or landmarks present in the images, making it easier to search and manage large photo collections. 

Conclusion

The newly introduced patent by Amazon can be considered a new step in the development of AI image recognition, allowing the productive processing and analysis of more complex images. This can find its applications in areas where searching through images containing a lot of tiny details may be required. Introduced in the first place for analyzing satellite images, it can be useful in various fields that have to do with overhead images. Besides that, it can be applied efficiently in the field of medicine, facilitating the diagnosis of various pathologies, or being helpful to ordinary people, making image searches far more efficient.