Conservation X Labs (CXL) is a nonprofit with the goal to leverage interdisciplinary intelligence, resources, and technology to prevent the impending sixth global extinction event. One of the tools CXL has developed for this mission is Sentinel, an award-winning[1] smart camera adapter that takes the information usually available from trail cameras and upgrades them with near real-time processing of the environmental data through use of artificial intelligence. While all of this information is invaluable to the numerous conservation groups currently using the Sentinel hardware, CXL was interested in expanding access to their foundational image recognition models beyond this user base. Data Clinic partnered with CXL to create a user-friendly desktop UI that would increase the usability of the Sentinel innovations, regardless of the user’s hardware and location.
Real-Time Data Processing in the Field
CXL, based in Washington, DC, aims to tackle a wide range of environmental issues through partnerships with many different conservation groups, such as Island Conservation, Osa Conservation, Andes Amazon Fund, The Nature Conservancy, and the New Zealand Department of Conservation. Many of these organizations have upgraded to the Sentinel monitoring tools and have been using the models created by CXL to process their data wherever they are. Being able to efficiently analyze their imagery in a timely manner allows them to respond faster to threats such as invasive species, zoonotic diseases, and changing wildlife behavior.
Out in the field, Sentinels may analyze thousands of photos a day with computer vision models built by CXL. Highlighting images like needles in a haystack, Sentinel then sends critical information over satellite, LoRa wireless technology, or cellular communications. Prior to Sentinel, this same process could take months, as someone would have needed to hike to the camera to retrieve the SD card of photos. By the time the photos were reviewed, action may no longer have been possible.
“Sentinel has accelerated how quickly conservation groups can respond to environmental challenges by providing near real-time data from the field. With its ability to analyze thousands of images in a fraction of the time, Sentinel has already helped partners tackle invasive species and monitor endangered wildlife in ways that were previously impossible,” says Henrik Cox, Head of Product at CXL.
While the Sentinel hardware and models allowed researchers and conservation groups to gain a great deal of information while out in the field, the user interface was not always user friendly. Conservation organizations who wanted to use the models on photos they had already collected on their trail cameras in areas with no internet would require coding skills to run models in a terminal. And whether they had the Sentinel hardware or not, they didn’t necessarily receive much feedback on the progress of the analysis, which could take hours or even days. Additionally, issues could be difficult to debug. To increase the usability of these invaluable innovations, our team volunteered to execute CXL’s vision of a desktop application that would provide a smoother user experience for processing pre-collected images in zero-connection environments using models built by CXL.
According to Henrik Cox, “We recognized that Sentinel’s powerful AI models needed to be accessible to everyone, regardless of their hardware or technical expertise. That’s why partnering with Data Clinic made perfect sense—to help us create a streamlined, offline desktop app that could bring these critical tools into the hands of more conservationists, no matter where they are in the world.”
User Interface Built for the Elements
The needs of conservation organizations, especially those working in very remote areas, pose unique challenges. As with all our partnerships, we met with CXL frequently to keep open communication and make sure the needs of the field were fully understood and integrated into the solution.
Due to the nature of the work being done by these conservation groups, the UI needed to be flexible. Internet connectivity is spotty in many locations, so the app needed to function with no internet connectivity once downloaded and installed by users. The computers in use would have a variety of generations of operating systems and compute power, so it would need to be lightweight and usable on both Windows and Mac. Most importantly, the interface needed to be intuitive. This application was going to be used around the world by people who speak many different languages, so everything needed to be clear and streamlined.
With this all in mind, the decision was to use Electron JS to begin building the application. Electron JS allows developers to create applications that are compatible across the most commonly used platforms using a single codebase. It also allows for automated version pushes whenever an update is made within the GitHub repository, streamlining any future improvement processes for CXL developers.
The Sentinel App
Upon opening the app, the user will be prompted to install and run Docker as required to communicate with the models by a simple pop-up. Once everything is up and running, the application is accessible. Users are prompted to select:
- Which downloaded model they want to use
- Which folder contains the dataset to be analyzed
- The output style they would like the photos to be in
- What confidence threshold to use for the model
- Where to save the analyzed photos
Once these values are entered, clicking “Run Model” will begin the analysis. Instead of a bare load view, users are greeted with a progress bar indicating how many photos have been processed and how many more are yet to go, as well as a grid populated with the marked images. This allows for quick review of the model’s output, even while it is still running.
In addition to the tab for running the models, users can also view past results to see a summary of previous model runs and view the logs from each run to identify issues and send troubleshooting reports back to CXL. These new functionalities provide the user with better agency over their own data processing.
Testing With CXL Partners
One thing that really contributed to the success of this project was how closely our two teams collaborated. We were in constant contact to make sure all of the required features were included in a logical and user-friendly way in the Sentinel App. This helped Data Clinic to zero in on the most important aspects and make sure the app would seamlessly interact with the models already being used by groups working with CXL, such as one used to identify jaguars on trail cameras in Costa Rica through Osa Conservation.
Throughout the course of the project, we also met with representatives of the conservation organizations eager to use the desktop app. During one of our earlier demonstrations of the app, we identified where we could improve usability without adding to the core function of the app: clear feedback to bolster the analysis being done by the models.
With that feedback in hand, we made final tweaks on the app and were finally ready for beta testing. Dedicated partners of CXL are utilizing the app and testing it in all the conditions mentioned previously that could strain the app.
Future of the Sentinel App
With the beta version of the app available now to CXL’s partners, conservation groups across the world will be able to utilize the additional visibility to analyze their photos more effectively and with greater feedback, no matter their internet connectivity. Potential future plans for the Sentinel Desktop App include incorporating additional models from CXL and further improving the interface with features such as additional language support.
“Working with Data Clinic was a phenomenal experience and the desktop application has built on improving the overall system’s user experience and offerings. Moving forward, we hope to continue to build on this by integrating even more features and functionality and making Sentinel a cornerstone of global conservation efforts,” Henrik Cox said.
Our team had an incredibly rewarding time contributing to and learning about wildlife conservation with CXL, and hope the Sentinel Desktop App will prove to be an impactful and useful tool to conservation organizations across the globe.
A special thanks to Leah Fischer for writing this blog.
[1] Sentinel was awarded the 2023 Conservation Technology Award by EarthRanger