Let’s try understanding the scale of the problem.

Realizations on the larger landscape

Due to sensor technology advancements (spectral ie. hx, spatial resolution, sensor type e.g. radars), the data volume has dramatically increased. Modern hyperspectral satellites capture hundreds of wavelength bands at 5m compared to the tens of wavelength bands collected by traditional systems at 30m, providing say 300 times more information. However, groundstation capacities are constrained by the technology (e.g. X band systems giving 100s of Mbps or upto 1 Gbps) with optical communication systems (which are awesome) can give more. Groundstation capacity is increasing at ~13% CAGR but the data generation is growing faster. Basically, the utilization of the onboard sensors is very limited. Even with advanced optical comm and relays, some studies indicate that it can (with some assumptions) increase utilization only from 0.1% to 4%.

In other words:

The rate at which data is being produced in space is far outstripping the capacity to downlink it.

Earth observation data will generate about 2 Exabytes (EB) by 2032, growing at a CAGR of 11%. EO data accounted for about 86% of the cumulative data generated by the Space applications segment. EdgeAI will be a necessary technology to reduce downlink data volume (oftentimes between 50-90%), we ourselves demonstrated 80% reduction. Moreover, the data storage and compute costs on the cloud also will also play a role. Moving to storage of relevant data (enabled by edge), can save 50-90% of the cost according to our case study.

We have CPU, GPU, VPU, FPGA-based machines widely available for spacecraft nowadays as compiled in NASA State-of-the-Art of Small Spacecraft Technology: Small Satellite Avionics report or this 2026 presentation from Kacprzak. Google plans to put TPUs. When you compare the memory (apart from other specs) requirements for such machines, it is clear that we have enough commercial edge devices having capabilities to perform a lot (if not all) of compute work typically done on ground. Multiple premium-grade hardware have long-lifetime guarantees required for say, governmental space missions.

The edge hardware is good enough now that it is possible to do most compute in space earlier reserved for ground.

Considering the plans of development of satellites by different countries, developed countries will be the first ones to realize the advantages of the low latency and intelligent data streaming. Countries having relatively small satellite constellation sizes maybe be more incentivized to have edge technologies due to cost efficiency that edge will bring.

A Speculative Timeline

2026: Edge proves it’s not a gimmick

Given the serious advantage provided by real-time satellite-based intelligence as seen in the war between US and Iran in 2026, the value of edge computing as an essential technology for satellites is felt immediately in US, Europe, China, East Asia and even in countries like India.

This is the year where edge computing in space transitions from demonstrations to selective operational validation for all mature space powers. Early deployments may focus on Earth Observation and ISR (Intelligence, Surveillance, Reconnaissance) with primary objective to reduce downlink volume and latency via onboard filtering and inference.

Subscribe now

If all things go per plan, we may also have a winner in XPRIZE Wildfire Space-Based-Detection and Intelligence Finals 2026. Each team will need to identify all active wildfires within just one minute and provide a detailed analysis of each fire’s behavior, such as its perimeter, direction of spread, and intensity, within 10 minutes. The teams will also be challenged to minimize false positives - incorrectly identifying non-fire heat sources like solar panels or hot surfaces as fires- which is crucial for ensuring that resources are deployed effectively in real-life scenarios. The winning teams will prove some thing essential: wide area low latency fire detection by global teams with a real-world use case like Australian wildfires.

2027-2028: Edge becomes part of the architecture

By 2027, edge computing stops being a payload feature and starts becoming a baseline architectural assumption. Key shifts to be seen are integration with inter-satellite links (ISL) and software-defined payloads and emergence of multi-node processing across constellations. Satellites begin acting as cooperative compute nodes. Task distribution and in-orbit workload sharing becomes feasible. Supporting trend: Space network architectures are increasingly designed for networked, autonomous, software-defined systems.

By 2027, the US is expected to complete Tranche 1 of the PWSA architecture by launching 28 satellites in Tracking layer using atleast IR with edge computing to track ‘interesting’ signals as seen in their graphic below.

The Chinese, per a recent paper, have interesting architecture propositions of their own. Large AI models (LAMs) on a massive supercomputers, “on-board data processing by leveraging the integrated communication-and-computation capabilities in space computing power networks … enhancing the timeliness, effectiveness, and trustworthiness for remote sensing downstream tasks” and “a microservice-empowered satellite edge LAM inference architecture” make it seem like an engineering-first thought process. A lot of the satellites in the architecture map to the recently launched constellations.

I expect Europe to invest in the Virtual Constellation / Sensorweb technology seriously (including into private constellation providers) as well.

2029-30: Autonomy Phase: Satellites start making decisions

This is where things get slightly uncomfortable for traditional ground control.

Key capabilities emerge like event-driven tasking (e.g. detection → onboard decision → retasking) and reduced human-in-the-loop for: prioritization, routing and scheduling. The demonstrators like dynamic tasking done by Ubotica and NASA are recently showing a lot of promise right now in 2026 itself. One can only imagine that such limited autonomous decision making may become common by 2029-2030. Given the trend of increasing constellation density, the demand for for real-time responsiveness in disaster monitoring and defense scenarios will be met much better.

As per a recent paper called Federated Autonomous Operations: A New Paradigm for Large-Scale Observation Systems, satellites could imagined as observing assets belonging to federated entities (e.g. private constellations operators) where federations can communicate with each other but agents from two different federations cannot. As the paper explains:

The scheduling for such a tasks would fall somewhere between decentralized allocation if operators talk to each other or federations bidding for such task requests and then distributing among their observing assets. Constellations begin behaving like loosely coordinated autonomous systems rather than centrally commanded assets.

2031 onwards: Edge dissolves into the space compute economy

Rapid growth trajectory in space-based edge computing and a strong demand for low-latency, real-time analytics services along with adoption across commercial and government users. Market projections indicate sustained expansion driven by AI-enabled onboard processing and autonomous operations.

I expect the emergence and maturity of processing-as-a-service in orbit and mission software licensing models and constellations begin monetizing insights, not imagery. We transition from space infrastructure to space compute economy. By the end of the decade, edge computing in space may stop being discussed as “edge”.

It simply becomes how space systems work. Key characteristics of this system are:

• Dedicated missions for extreme environment monitoring already planned with advanced onboard processing (e.g., next-gen missions targeting 2030 timelines)

• Constellations operate as: persistent compute fabrics tightly integrated with terrestrial cloud

System-level shifts:

• Data flow becomes: selective, prioritized, intent-driven

• Satellites participate in: real-time decision loops like dynamic weather forecasting

Industry convergence: Edge + AI + ISL unified architecture and space becomes part of a 3D compute network (space–air–ground). The distinction between: “satellite system”, “data system” and “compute system”…starts to dissolve.

This article is a application-oriented attempt to wrap my head around recent developments in remote sensing and geospatial foundational models, focusing on research published within the last five years, trying to identify gaps and opportunities in my view and sharing my experiments. The sections are organized into foundational models intro, applications, benchmarks, and key resources.

Foundational Models in Remote Sensing and Geospatial Science

Remote sensing foundation models mostly refer to large models, mostly built through self supervised learning, trained on diverse remote sensing data (RGB,MX,HX, SAR,LiDAR,etc.). A nice, helpful taxonomy is given by Xiao et. al. is shown below.

Thanks for reading Mostly Stable, Slightly Inclined ! Subscribe for free to receive new posts and support my work.

The Visual Foundation Models (VFMs) comprise Pre-trained models learning transferable representations from unlabeled data through contrastive (similarity) and Masked Image Modeling (MIM) and finetuning models to improve performance. Meta’s Segment Anything Model (SAM) has been adapted to Samgeo which can interact through text and location pin prompts save segmented areas with geospatial data like GeoTIFF raster data and GPKG vector data.

The Vision Language models (VLMs) integrate vision and text data and involve captioning, Visual Question Answer, Visual Grounding, Scene Classification, Object Detection, Region Caption and Multi-round conversation as shown by this excellent graphic from generative VLM EarthGPT (utilitizing Vision Transformers, LLAMA-2 and multiple task-specific remote sensing datasets).

Many Visual-text models are extensions of CLIP in the remote sensing domain. We did a test of the CLIP-based RS-ICD image captioning model and reported our learnings in a previous blog. Github user Jack-bo1220 has compiled many Remote Sensing Vision-Language Foundation Models here. Visual-location models like SatCLIP have a location encoder summarize the characteristics of any given location (latitude,longitude) using CNN and ViT inferred visual patterns for a downstream task like predicting air temperature.

SkyCLIP

Tiny Demo 1: I found this demo by RT Wilson based on SkyCLIP model really the first fun thing to try out. Quoting the blogpiece:

An embedding model lets you turn a piece of data (for example, some text, or an image) into a constant-length vector – basically just a sequence of numbers. In this case, I’m using the SkyCLIP model which produces vectors that are 768 elements long. One of the key features of these vectors are that the model is trained to produce similar vectors for things that are similar in some way. For example, a text embedding model may produce a similar vector for the words “King” and “Queen”, or “iPad” and “tablet”. The ‘closer’ a vector is to another vector, the more similar the data that produced it.

Turns our queries like “truck in parking lot” are fairly high precision although I am not sure about the recall:

SAM3 Geo

Tiny Demo 2:

I made a colab notebook for SAM3 Geo, a fresh unified foundation model for promptable segmentation in images and videos for a test location in Bangalore, India. It can detect, segment, and track objects using text or visual prompts such as points, boxes, and masks. These are the results I got:

Clearly, there is a lot of progress still needed to make building segmentation better (e.g. for test datasets with different spatial distribution and packing of buildings) because the images in the example/tutorial looked a bit better than real-world (e.g. from developing countries):

AlphaEarth Foundations Satellite Embeddings

As per AlphaEarth introduction blog, every 10m pixel is know containing a 64-D vector characterizing it:

This first-of-its-kind dataset was generated using AlphaEarth Foundations, Google DeepMind’s new geospatial AI model that assimilates observations across diverse sources of geospatial information, including optical and thermal imagery from Sentinel-2 and Landsat satellites, radar data that can see through clouds, 3D measurements of surface properties, global elevation models, climate information, gravity fields, and descriptive text. Unlike traditional deep learning models that require users to fine-tune weights and run their own inference on clusters of high-end computers, AlphaEarth Foundations was designed to produce information-rich, 64-dimensional geospatial “embeddings” that are suitable for use with Earth Engine’s built-in machine learning classifiers and other pixel-based analysis.

Some of the applications involve change detection and tracking of processeses like urban sprawl, wildfire impacts and recovery, automatic clustering, etc.

The dataset has a super cool feature called similarity search. Similarity searches are an easy way to compare embedding vectors for different locations and quickly identify pixels that have similar environmental and surface conditions to a location of interest. For example, the embedding vector for a 100-square-meter (10 m x 10 m) pixel of an airport runway allows me to find all airports (in the world) bearing a strong similarity to the same.

Tiny Demo 3:

So, I published this app where I put a reference point on an existing airport in Hotan prefecture in China and started eyeballing and using similarity score above a specified threshold found a brand new airport not labelled (as of Dec 9th 2025) on OSM or Google Maps. So here is the Google maps pin which shows this find:

This is unbelievably powerful as you can imagine. My first guess would be that we can easily use it to fill in global missing infrastructure for Open Street Maps for example:

• Airports

• Railway Stations

• Public infrastructure

Prithvi: A Generalist Remote Sensing AI Model

Notable foundation models for “earth monitoring” (basically Earth Observation applications) are being evaluated on the GeoBench benchmark which evaluate models on six classification and segmentation tasks.

Prithvi is a transformer-based family of geospatial foundational models developed by NASA and IBM. It is pre-trained on multispectral satellite imagery from NASA’s Harmonized Landsat Sentinel-2 (HLS) global dataset. We can perform crop classfication (demo), flood detection and more. The crop classification model requires Blue, Green, Red, Narrow NIR, SWIR, SWIR 2 bands so it is clearly utilizing spectral bands which are more suitable compare to traditional models which just take RGB data.

An exhaustive list of foundation models are found here.

Research Question

Can foundational Earth models such as AlphaEarth can be extended to autonomously detect, vectorize, and update new infrastructure from continuous satellite imagery? I wonder where this can naturally extend to. The central question is:

Recent advances like AlphaEarth, SatCLIP, and Segment-Anything Geospatial (SAMGeo) have shown that foundation models can learn rich spatial-semantic embeddings across modalities. Building on this, the proposed framework introduces a closed-loop pipeline for automated infrastructure discovery:

1. Detection of Emerging Infrastructure
   AlphaEarth’s similarty search modeling capacity is harnessed to analyze recent satellite imagery and detect morphological changes indicating human activity—such as new roads, construction, or urban expansion. We can then choose prompt-based inputs for say identifying the infrastructure segmentation mask. This stage could use unsupervised or semi-supervised similarity-search detection to identify genuinely new structures.

2. Conversion to Map Vectors
   Inspired by HPix, SAT2MAP and other image-to-map translation methods, detected contiguous regions that exceed defined spatial thresholds (e.g., >500 m² for buildings, >2 km for roads) are converted into vectorized map features. This step transforms pixel-level detections into GIS-ready data, aligning them with existing cartographic layers. This probably requires some human QA today.

3. Continuous Updating and Validation
   New imagery is continuously compared to prior map states. Detected differences trigger localized updates, forming a self-correcting mapping loop. External data (e.g., OpenStreetMap, construction records) provide validation signals to refine confidence and accuracy.

By merging AlphaEarth’s multimodal perception, SAM-Geo’s segmentation capabilities with HPix/SAT2MAP-style translation and dynamic updating, this system would enable autonomous, continuously evolving digital maps. The implications are vast: near-real-time urban monitoring, disaster response with instant map regeneration, detection of unauthorized development, and improved situational awareness for autonomous systems.

“When hypersonic missiles travel hundreds of kilometers in minutes, and drone swarms reach targets before decisions can be made, real-time or near-real-time surveillance becomes not just helpful, but critical for survival,” stated Air Marshal Ashutosh Dixit, Chief of Integrated Defence Staff to Chairman Chiefs of Staffs Committee, regarding the compressed timelines of modern warfare. His emphasis on the OODA loop (observe, orient, decide, act), which has compressed from hours to minutes, reflects India’s urgent need to transform satellite data into actionable intelligence at the speed of threat evolution.

This gap between data collection and actionable intelligence creates vulnerabilities that adversaries exploit daily. Modern satellites capture terabytes of imagery, but traditional workflows require downloading, processing, and analyzing data at ground stations. This cycle transforms real-time threats into historical records.

When Minutes Matter: The Cost of Delay

India’s border security demonstrates the critical impact of processing delays. Consider the surveillance challenge along the 3,488 km India-China border, where satellite flyovers capture movement patterns at key mountain locations. By the time this imagery reaches ground stations through limited downlink windows, undergoes processing at centralized facilities, and reaches military command centers, tactical opportunities vanish.

Defense Minister Hon’ble Shri Rajnath Singh emphasized in 2025: “Space capacities in the Low Earth Orbit are transforming military intelligence, persistent surveillance, positioning, targeting and communications, thus taking combat to a new high.” Yet frontline forces rely on intelligence that arrives hours after collection.

During Operation Sindoor, satellite data provided crucial intelligence on drone and missile trajectories, but processing delays meant reactive rather than preventive responses. As noted by defense analysts, India’s Rs 27,000 crore SBS-III project aims to deploy 52 satellites with AI-driven analytics and synthetic aperture radar for real-time monitoring of troop movements and missile deployments.

Air Marshal Dixit further elaborated on the strategic imperative: “The decisive advantage in modern warfare lies in the ability to see first, see farthest, and see most accurately. When weapons can strike targets hundreds of kilometres away with pinpoint accuracy, he traditional concepts of front, rear and flanks combat zones and depth areas all become irrelevant. What we call the front and the theatre merge into one. This new reality demands that we extend our surveillance envelope far beyond what previous generations could have even imagined. We must detect, identify and track potential threats not when they approach our borders, but when they are still in their staging areas, airfields and bases, deep within adversary territory. This existed as a concept even earlier but today we have the means to realise it.”

The Technology Gap: Why Edge AI Remained Elusive

Despite urgent operational needs, multiple barriers have prevented widespread edge AI adoption in Indian and global satellite systems:

Data Bottlenecks from Growing Sensor Networks

Modern satellites, drones, and ground sensors generate massive volumes of data, but traditionally this information has to be transmitted to distant data centers for processing a workflow that quickly becomes impractical. Bandwidth limitations, high transmission costs, and latency make it difficult to move such large datasets in real time, especially from remote or hostile environments. This creates a significant technology gap, where valuable insights are delayed or lost entirely, highlighting the growing need for efficient edge processing solutions.

Processing Without Accuracy Loss

The fundamental challenge lies in maintaining intelligence quality despite severe constraints. According to IEEE research, space-qualified processors operate with 10–100 watts of power compared to terrestrial data centers using megawatts. Storage limitations mean satellites can buffer only hours of raw data. Bandwidth restrictions limit downlink to 1–2 Gbps, while sensors generate 10–100 Gbps. Traditional compression destroys critical details needed for threat detection. Edge AI must achieve 95%+ accuracy while operating within these constraints, a challenge that has stymied deployment for decades.

The GEOAI Revolution: Processing at the Point of Collection

Recent breakthroughs demonstrate that GEOAI models (geospatial AI optimized for orbital deployment) can transform satellites from passive collectors into intelligent analysts. These models process imagery directly in orbit, identifying threats, filtering noise, and prioritizing critical information for immediate transmission.

ESA’s Φsat-2 mission proved the concept, running six AI applications simultaneously on a CubeSat. The system detects vessels, identifies wildfires, and generates emergency maps, all within minutes of image capture.

The key to closing the gap lies in two complementary capabilities working in concert: rapid GEOAI model development that adapts to emerging threats, and real-time orbital processing infrastructure that executes these models instantly. Organizations need platforms that can develop and deploy new GEOAI models in weeks rather than years, while simultaneously managing satellite operations to process multiple data streams in real time. This combination creates the competitive advantage that transforms satellites from data collectors into decision enablers.

Operational Integration: Where Speed Meets Intelligence

The transformation occurs when integrated systems combine GEOAI model libraries with real-time tasking and operations management. This fusion enables immediate threat detection, adaptive intelligence gathering, and multi-source correlation that fundamentally changes security operations.

Real-World Applications

Maritime surveillance systems can now detect suspicious vessel activity within minutes. Border forces gain real-time visibility into unauthorized crossings using live satellite feeds. SkyServe enhances these operations with EdgeAI-powered insights delivered directly from orbit, enabling faster response to critical events such as oil spills, illicit shipping, or encroachments.

These edge applications reduce reliance on ground downlink and provide actionable intelligence for defense, disaster response, and infrastructure security where seconds matter.

Implementation Pathways: Becoming Edge Ready

Organizations must systematically prepare for edge AI deployment through seven critical steps:

1. Data Curation and Preprocessing
   Collect diverse, high-quality geospatial datasets from multiple sources including satellites, drones, sensors, and ground truth validation. Indian defense agencies require comprehensive datasets covering varied terrain from desert borders to mountain passes, with annotations for threat patterns unique to regional adversaries.
   
   Key activities:

• Normalize, clean, and annotate data for AI tasks including object classification, threat detection, and terrain segmentation

• Simulate edge conditions with low resolution, noise, or partial data availability to ensure model robustness

• Integrate satellite surveillance with GIS mapping for precise georeferencing and multi-layer analysis critical for border monitoring operations

2. Model Selection and Optimization
   Choose architectures optimized for space constraints while ensuring models meet strict orbital computing requirements. Quantized neural networks reduce memory footprint by 75% while maintaining accuracy.

Optimization requirements:

• Optimize for inference time and memory usage in constrained computing environments

• Ensure models do not exceed RAM limits or generate output beyond allowable size

• Validate that inference completes within milliseconds and memory usage remains within allocated boundaries

• Apply pruning to remove unnecessary connections and knowledge distillation to transfer capabilities from large models to compact versions suitable for orbital deployment

3. Onboard Deployment Readiness
   Ensure that both software and hardware compatibility are aligned with the use case requirements and objectives. Models must operate within the power envelope of the intended use case and adhere to memory constraints (4–16 GB). Testing must validate performance across the full operational temperature range of the target platform.
   

4. Reliability Engineering
   Implement redundancy through ensemble models that cross-verify detections. Error correction codes protect model weights from radiation-induced bit flips. Graceful degradation ensures partial functionality even with component failures.
   

5. Integration with Edge Pipelines
   Develop APIs for seamless data flow from sensors through AI models to communication systems. Standardize interfaces for model updates via ground commands. Enable autonomous operation during communication blackouts.
   

6. Test and Validate in Real Environments
   Run integrated hardware and software tests that simulate real orbital conditions. Focus on key constraints such as limited sensor fidelity, strict memory ceilings, and tight compute budgets. Validate that inference outputs stay within size limits and complete processing within the allocated time windows.
   
   Indian organizations should prioritize test scenarios that reflect regional risks, especially maritime threats in the Indian Ocean Region. Ensure AI models deliver accurate results under these conditions and maintain performance within the resource limits of space-qualified systems.
   

7. Security Implementation
   Protect deployed systems through comprehensive security measures:

• Encrypt model weights and inference results

• Implement authentication for model updates

• Protect against adversarial attacks designed to fool AI systems

• Establish audit trails for all automated decisions affecting security operations
  

SkyServe: Proven Edge AI Leadership

SkyServe has operationalized this vision through its EdgeAI platform, enabling rapid development, deployment, and management of AI models directly on orbit. This approach delivers real-time insights from space, eliminating delays associated with ground-based processing.

Proven Track Record

With four completed missions including partnerships with D-Orbit (Mission Matterhorn) and Loft Orbital (Mission Denali), SkyServe has demonstrated:

• 5x improvement in data transmission efficiency

• Processing intelligence directly in orbit on existing satellite hardware

• Organizations don’t need new satellites to gain edge AI capabilities

• Sensor-agnostic capabilities across multispectral, hyperspectral, SAR, IoT, RF, and other payload types

Dual-Platform Advantage

The company’s approach uniquely addresses both sides of the equation:
SURGE enables development of mission-specific GEOAI models in weeks rather than months, while STORM manages the orbital infrastructure to execute these models in real time.

This combination has delivered tangible results including vegetation analysis, cloud detection, and anomaly identification over strategic regional areas, all processed within seconds of capture rather than hours after ground station download.

The Strategic Path Forward

SkyServe continues advancing its technology through collaborations with leading private and public space agencies like ISRO, NASA, and IN-SPACe, while expanding partnerships across the defense sector. SkyServe focuses on making edge AI accessible to government security agencies by:

• Reducing deployment time from years to months

• Ensuring compatibility with existing satellite infrastructure

• Enhancing automated threat detection algorithms

• Expanding multi-sensor fusion capabilities

• Enabling even faster model deployment cycles to address emerging security challenges

The Imperative for Action

As Air Marshal Ashutosh Dixit emphasized: “Our adversaries are not waiting. Every day we delay implementing real-time space intelligence capabilities is a day they gain advantage.”

The evidence is clear: organizations implementing integrated edge AI platforms gain immediate competitive advantages through:

• Faster response: processing at the speed of collection

• Broader coverage: multiple data streams analyzed simultaneously

• Adaptive capabilities: rapid deployment of new threat detection models

In an environment where seconds determine outcomes, the advantage belongs to those who process at the speed of collection, adapt at the pace of threats, and decide faster than adversaries can act.

Conclusion

The transformation from traditional ground-based satellite intelligence processing to real-time edge AI represents a fundamental shift in national security capabilities.

India’s investment in the SBS-III programme, combined with emerging technologies from companies like SkyServe, positions the nation to close the critical gap between data collection and actionable intelligence.

The compressed timelines of modern warfare, where hypersonic missiles and drone swarms operate in minutes, demand intelligence systems that match the speed of threats. Edge AI and GEOAI technologies have matured from concept to operational reality, proven through missions in orbit and real-world defense applications.

For defense agencies and satellite operators, the path forward is clear: systematic preparation through the seven implementation steps, adoption of integrated platforms that combine rapid model development with orbital operations management, and commitment to continuous advancement as threats evolve.

The competitive advantage in space-based intelligence now belongs to those who process at the point of collection, transform satellites from passive sensors into intelligent analysts, and deliver actionable intelligence measured in seconds rather than hours.

Read

SkyServe is collaborating with scientists at NASA's Jet Propulsion Laboratory (JPL) and D-Orbit to pioneer advancements in Earth observation systems. The collaboration centres on optimizing and deploying AI models developed under NASA’s New Observations Strategies (NOS) initiative on D-Orbit’s ION Satellite Carrier.

As part of NASA’s NOS initiative, JPL aims to create a unified network of space-borne, terrestrial, and airborne sensors. This system envisions near-real-time monitoring of critical phenomena such as wildfires, floods, and urban heat islands, thereby supporting scientific research and disaster response efforts. Achieving these objectives requires harnessing edge computing to integrate and optimize AI models across diverse sensor configurations.

The collaboration leverages SkyServe’s STORM, an edge-computing platform for satellites that enables seamless deployment of advanced AI applications in orbit. Complementing STORM, the SURGE software suite offers an on-ground environment to test, develop, and deploy AI models on a highly configurable architecture.

This software suite envisions a wide range of use cases—from detecting wildfires in Malibu to tracking unregistered ships in the Red Sea—all within the same orbital mission. The rapid adaptability of these systems enhances response times for emergencies and accelerates decision-making.

By simulating the constraints of space embedded environments, SURGE ensures AI models maintain operational consistency across varied satellite platforms and computing environments. This abstraction simplifies the complexities of deployment and management, allowing satellite operators to onboard end users, such as government and commercial entities. SURGE efficiently manages the diverse sensors and computing hardware involved.

“SkyServe’s technology plays a pivotal role in streamlining AI model deployment across diverse satellite platforms, ensuring consistency and efficiency,” said Vishesh Vatsal, CTO of SkyServe. “Our platform bridges the gap between groundbreaking AI models and the hardware diversity of satellite systems, enabling a unified approach to Earth observation.”

D-Orbit’s ION satellite platform enhances these efforts with a robust and adaptable infrastructure foundation, ensuring seamless integration with SkyServe’s STORM and SURGE for superior in-orbit performance.

This collaboration exemplifies the fusion of EdgeAI solutions with satellite platforms to meet the dynamic challenges of Earth observation. SkyServe is pushing the boundaries of space technology and Earth science, redefining the possibilities of space innovation and driving transformative advancements in global monitoring.

Subscribe now

• ‌Ability to uplink STORM to a satellite already in space

• Autonomous execution on live images as they are acquired

• Versatility to leverage VPU to speed up Deep-Learning Models

• Real-time vegetation classification

• Compression ratio of 5 or more

We had a really big bucket list of applications that we could demonstrate, but we finally settled on the following image pipeline tasks:

• ‌Access Satellite Camera

• Image an area of 400km x 300km

• Check if Camera is looking at target

• Correct errors & enhance quality

• Detect clouds & mask

• Segment unmasked pixels into land/water

• Detect vegetation on land area

• Categorize vegetated land based on chlorophyll content

• Compress results to under 1MB

• Transmit data back to Earth

Express schedule

Once our engagement was formalized in the first week of March, we decided to push the pedal on the submitting our onboard app executable in the fastest possible timeline. Given that we had some work done on similar missions before, we were able to share the first version of our app in 10 days. The next steps were tough, particularly for our model where complying to the VPU’s library version was required. We also figured out in the meanwhile how to maximize the outcomes from the onboard run (more about this later). There was a breakthrough in the last week of March when we finally got confirmation that our engineering model runs were successful. We finally got to hear that the satellite uplink was done by April 11th. We had the results downlinked and shared with us by April 18th!

Data exchange

Our data exchange begins with understanding mission and sensor specifications and making the app (particularly models and libraries) compliant to the onboard environment. Finally, the models need to work with onboard imagery, so it is best to get as much onboard sample imagery as possible, or we can always create synthetic imagery.

In this mission, the second step for development of a tested (against onboard sample imagery) application took more than 60% of the overall timeline. I felt this is where a tool like Surge could help to compress timelines while adhering to model guidelines. Fortunately, we got great support during the engineering model (EM) testing and this gave us confidence to give a go ahead for the onboard runs!

Target selection

Based on this cool 20-year average cloud cover reference, we figured out the Egypt-Sinai region is expected to be super clear during April and would be an ideal target (compared to other options available) for us. Given the large swath of the imager, the footprint of the Sinai image looked something like this:

Based on the satellite’s orbit TLEs, we were able to guess approximate time of targeting the area of interest. From the geometry of the footprint, we could estimate off-nadir angles at the time of acquisition of the imagery

Image

The image turned out to be great and pretty clear as expected (around 414km swath and 204 m GSD). Notice the camera baffle covering the top corners of the image. Also notice that the Nile river is much thinner than it seems because of dense/dark transition area (riparian basin) between the river and the land.

Interesting bits and learnings

1. We were able to publish a lot of our data (including performance parameters like runtimes) on the mission marketing page here (kudos to Harmit and Aravinth). This communicated our mission outcomes way better than any presentation or technicality.

2. Squeezing the maximum from on onboard pipeline with downlink constraints can be done in multiple ways. You can bitpack your inference masks depending on the number of classes. You can get some crops at native resolution, prioritize them by their cloud cover stats, and get histogram stats to recover the original image. You can resample the imagery to get a thumbnail. You can zip all your items together and drop items basis your downlink size budget and priority.

   

3. We take the privacy of the models and inference scripts of our customers very seriously. To demonstrate this, we employed techniques of File Encryption and Code obfuscation for software delivery. The specific technique employed for this mission also factored in the software payload size, and time/resource consumption for encryption/decryption.

Contribution to open data

We at SkyServe aim at achieving a user centric approach for mission developments and user interactive products. While we try to achieve this milestone by one stepping stone at a time. One such exercise towards this goal is having our mission data and sample images listed on STAC index using services provided by Ellipsis Drive (checkout our podcast here).

This step allows us to fine tune and refine our data delivery mechanism and quality as per the Open Geospatial Consortium (OGC) standards, while also allowing geospatial developers to access our mission data through APIs to learn the image processing pipeline quality of SkyServe STORM and use the data for their geospatial applications and GeoAI models.

Feel free to try our Matterhorn mission sample data at STAC index here. We will keep adding more products as we go.

Subscribe now

Foundational Layer of smartness

Building upon Part-1: "City to Smart City: 'Smartness Proportion' with Satellite Intelligence", let's explore some activities undertaken by smart city development bodies to demonstrate the utility of satellite edge computing. Before we proceed, let us keep in mind that cities have the autonomy to chart their unique paths towards "smartness", without being rushed or pressured to adopt a specific technology or even the extent of its application.

In his publication (2016), “Smart City Implementation Framework for Developing Countries: The Case of Egypt”, Dr.Karim Hamza suggested the following framework.

Figure 1 illustrates the "Smart City Structure", highlighting the top priorities such as preserving land, water, and forest resources, along with establishing robust infrastructure for sewage systems, water supply and transportation. Maintaining up-to-date digital twins of these features is critical for every city in developing nations, as they aspire to integrate "Smartness" into their urban landscapes, reflecting a holistic approach to urban development.

Without this groundwork, cities may encounter obstacles in subsequent stages of smart city development. This is where satellite edge computing can be a transformative solution, offering an alternate to traditional high-cost approaches.

Unlocking Smartness: Satellite Edge Computing Advantages

Satellite edge computing offers a host of advantages for cities aspiring to embrace smartness. It can seamlessly integrate with existing systems, thereby reducing upfront costs and overhauling legacy infrastructure. Even cities with limited digital infrastructure can benefit during early development stages, accelerating their smart city initiatives. Moreover, by providing actionable insights directly, satellite edge computing reduces the reliance on in-house expertise at the very beginning, enabling swift action and building trust with their citizens for delivering on public goods.

Share

In consonance with our focus on "City-led" approaches (recall the generations of smart cities outlined earlier), technology only serves as an enabler.

1. Understanding city dynamics – Master Plan Compliance

The increased frequency of data collection, facilitated by satellite edge computing, simplifies the identification of illegal land use, a prevalent issue in many developing nations, through classification models onboarded onto satellites. This capability is crucial for city authorities as it allows them to address illegal land use at its beginning stages, supporting various aspects of urban governance. Additionally, frequent monitoring of land use dynamics enhances the efficiency of disaster management activities. Furthermore, supports proactive urban planning by tracking urban sprawl and ensuring compliance with land-use regulations.

2.Infrastructure Development and Monitoring

In a developing city, construction projects span across roads, buildings and bridges dispersed across the urban landscape. Monitoring these projects from an executive-level often requires extensive field visits, creating a significant dependency on reporting officials.

In a city-led approach, where councils directly oversee project progress, the need for frequent field visits must be kept to a minimum. This means that instead of relying heavily on physical inspections, city officials can leverage technologies like satellite-based construction activity monitoring. With this system integrated into the smart city's monitoring infrastructure, a significant portion of the monitoring work is accomplished without the necessity for on-site visits.

Once a model that is trained to meet the requirements is onboarded, the system delivers regular snapshots of project sites, on a daily or weekly basis. These snapshots are accessible through standard web-mapping frameworks, eliminating the necessity for specialized GIS software and reducing the burden of frequent field visits.

This feature not only ensures effective work implementation but also results in cost-effectiveness. Considering that most infrastructure projects stretch beyond planned timeline and budgets, and involve a multitude of departments across administration tenures, satellite edge computing can be specifically enabled to cater to these extended periods, ensuring continuous monitoring and oversight.

3. Encroachment detection

Encroachment detection, a critical aspect of urban land management, relies on routine field visit which may not be timely or effective. However, with satellite edge computing, encroachment attempts can be swiftly detected and alerted upon, preventing the unauthorized use of land parcels. This proactive approach ensures the timely conservation of valuable land resources as well as minimize lost revenue to the municipality. Using satellite images in encroachment detection is a high-value activity, widely used by metro cities like Mumbai. Additionally, in forested areas adjacent to urban zones, early warning systems can be established to monitor deforestation activities, safeguarding vital ecosystems.

For further insights into proactive property monitoring and deforestation tracking using satellite imagery and edge computing, explore our case studies on Monitoring property lines and Deforestation early warning.

4. Natural resources – Management

Satellite-based monitoring is a very powerful tool for natural resource management, tracking water quality and green cover health. Satellite edge computing can be used to provide timely alerts when water quality drops below safe limits, enabling proactive interventions. Additionally, vegetation indices computed continuously can assess green cover health. This integrated approach to natural resource management can enable proactive conservation efforts towards keeping cities sustainable and healthy.

Skyserve conducted a case study illustrating the density of Cyanobacteria, which is known for causing severe social and ecological consequences. These blooms incur significant socioeconomic and environmental costs, affecting various sectors including drinking water, fisheries and habitats. Continuous monitoring of such contamination enables early detection of adverse effects, facilitating timely intervention through appropriate channels.

Subscribe now

5. Public health

Landfills hold complex significance in urban areas, preventing waste accumulation and associated health risks. Continuous monitoring of landfills is essential, not only for managing environmental and social impacts but also to preserve the land resources they occupy.

Satellite image-centric solutions offer vital inputs here. Alongside traditional techniques like land use classification and NDVI-based studies, deep learning models are providing better accuracy nowadays. From the medium to very high-resolution, satellite imageries are being trained with those models.

These models not only monitor landfill sites but also detect illegal landfills within cities, where human access and visibility are limited, with minimal effort. Continuous monitoring of official and illegal landfills through satellite edge computing enables the assessment of their growth rates, the development of elimination plans, and the implementation of measures to prevent new pollution.

6. Diverse AI Models to Solve City Problems

High-resolution satellite images can collect property-level information, allowing for automated monitoring of approved properties and detection of modifications. Tax officers can leverage this technology to identify changes and impose relevant taxes, thereby bolstering the financial health of the city corporation. These capabilities extend to many more applications that can be tailored to address city-specific issues. For instance, detection of potholes and road quality monitoring enables timely repair by highway departments, while city-wide traffic monitoring using satellite images informs traffic management strategies.

Empowering Smartness in Developing Nations: Role of AI/ML Models

AI/ML is already playing a crucial role in our quest to increase the smartness quotient of cities, especially in developing countries. These models provide invaluable support by automatedly analysing satellite images and extracting actionable insights, empowering city authorities to make informed decisions with a smaller workforce.

While training these models requires initial effort, the long-term benefits align with the goal of continuously integrating smartness within existing digital infrastructures. Additionally, the adaptability of AI/ML models allows for seamless upgrades to satellite systems, facilitating ongoing smart city development.

The increase in diversity and access to satellite datasets has not only streamlined the process of training AI/ML models, but has also democratized access to remote sensing solutions. This has catalysed innovation in the field, with many researchers, start-ups and enterprises leveraging these datasets to contribute to smart city objectives

Closing the Global Smart City Gap

Satellite edge computing isn't exclusive to developing nations; it can be a priority-driven strategy. Even cities in developed nations, opting to reduce ground-based data collection, may transition towards utilizing satellite edge computing.

We can see that today's cities can adapt very quickly if technologies like satellite edge-computing are adopted, while being prepared to on-ramp infrastructure modifications at their own pace. This prepares cities for future growth, ensuring resilience and adaptability regardless of their current developmental stage. Ultimately, it is the citizens and their trust in their local bodies that will foster a more livable urban environment for the benefit of society.

Introduction:

As of March 2024, only 142 cities out of 47,868 were recognized in the IIMD's smart city index report for 2024, indicating a significant gap in smart city development globally. This discrepancy becomes even more pressing when considering that, according to the UN, over 68% of the world's population will reside in urban environments by 2050, with a substantial portion concentrated in developing countries. The presence of blue dot clusters on the map further emphasizes the need for smart city advancements to accommodate the burgeoning urban populations in these regions.

What Drives the Attention on Developing Countries?

In the historical narrative of smart city initiatives, developed nations have typically spearheaded the movement. This momentum has been fuelled by substantial investments in Information Communication Technology (ICT) and the established robust infrastructures to align to smart city requirements. Nonetheless, these advantages may not be readily available to developing countries, given their diverse socio-economic conditions. Additionally, the industry's current focus on mature economies widens the gap in technology adoption and outcomes between developed and developing nations, a discrepancy especially noticeable in their smart city initiatives.

The Good News: The Definition of "Smartness" is evolving!

Boyd Cohen's article (2015) signals a significant shift in smart city development, emphasizing a dynamic perception of "smartness." This evolving understanding prioritizes a balanced integration of technology and city governance, emphasizing citizen engagement over excessive technology adoption. In contrast to the earlier Smart City 1.0 model, Smart City 2.0 empowers local municipalities to strategically select technologies aligned with their unique contexts. This evolution holds promise for narrowing the gap between developed and developing nations as they navigate the challenges and opportunities of smart city implementation. 

Balancing Technology Adoption

While technology undoubtedly plays a pivotal role in smart city initiatives, the emphasis should not solely be on adopting technology for the sake of it. Instead, there's a growing recognition of the importance of making strategic choices in technology adoption based on specific needs and circumstances. 

Consider the landscape of smart city initiatives in various developing nations: Roughly 1% of Indian cities, 30% of China's cities, 17% of Russia's cities, and 25% of Indonesia's cities are actively planning, participating in, or achieving smart city initiatives.

Verge of Overload: Rethinking the Sensor Network Paradigm

Amidst the ever-expanding landscape of smart city technologies, can cities, especially those in developing nations, effectively harness the multitude of options available?

Before delving into the specifics of technology, it's imperative to ponder the following:

The "Network of sensors" integrated with the Internet of Things (IoT) stands as a cornerstone of smart city endeavours. Traditionally, these systems heavily rely on sensor networks for real-time data collection, encompassing aspects like traffic flow, air quality, and energy consumption. While this framework presents undeniable opportunities, the implementation of extensive IoT deployments encounters substantial hurdles in developing countries. Challenges stem from the dearth of reliable infrastructure and services, spanning electricity, internet connectivity, roads, and urban structures. Furthermore, the sheer magnitude of sensor deployment poses maintenance complexities and the looming threat of "sensory overload”, a phenomenon Forbes has highlighted in its discussions on smart city challenges.

Additionally, projections from Electronic Specifier indicated that by 2020, one trillion sensors could have been deployed, magnifying the challenges associated with overreliance on sensor networks.

Redefining Smartness

Amidst challenges such as sensory overload and the unique hurdles faced by developing nations, actual “smartness” can greatly enhance daily living. Redefined smartness entails prioritizing basic infrastructure development and leveraging existing digital frameworks to connect administrations with citizens. This approach aims to bring immediate relief to people's lives by addressing fundamental needs like roads and electricity, while maximizing the efficiency and sustainability of smart city initiatives. It involves evaluating new technologies based on criteria such as capital efficiency and suitability for city economics, emphasizing minimal data collection and effective use of existing digital frameworks such as e-governance portals.

Satellite Imagery for Developing Regions

In the context of optimizing data collection and leveraging e-governance portals, remote sensing and Geographic Information Systems (GIS) stand out as indispensable tools for advancing smart city initiatives in developing countries. These technologies have long played a crucial role in urban planning and management, offering valuable insights and facilitating the integration of IoT solutions.

Satellite imagery, a key product of remote sensing and geospatial analysis, serves a multitude of urban planning applications. Its wide coverage and high accuracy make it particularly influential, enabling modern analysis and insight generation. However, challenges such as cloud coverage render satellite imagery unusable at times, while purchasing complexities like minimum order requirements and unclear licensing terms hinder accessibility.

When satellite images are obtained at no cost, the challenge of data storage becomes more pronounced. At its core, as the resolution of the images increases, so does the image size, exacerbating the storage issue.

Both open-source and commercially procured satellite imagery face processing time constraints, leading to latency issues.

Limited budgets mean authorities may have to wait up to three days to gain insights from satellite images after processing, further highlighting the budget constraints!

Overcoming these challenges is essential to fully realize the benefits and allowing cities to unlock the transformative potential of satellite imagery.

Satellite Edge Computing – No longer futuristic

Satellite imagery, previously limited by inherent issues, is now undergoing a paradigm shift with satellite edge computing.

Satellite edge computing is no longer futuristic! Our ongoing missions are listed here. We have completed our first set of demonstrations processing data in space - and it's not just us, there will be more announcements coming in this field from other ventures as well!

Former NASA Administrator, Dan Goldin, recently extended his support to put more powerful computers in space to run AI in real-time. This is the start of an era in which trained AI/ML models can be uploaded to a relevant satellite, which can swiftly applying corrections to captured images and identify specific areas of interest with precision and the inference from the model would be then be downloaded for the user. The game-changing part is the speed with which these insights are delivered. Within a couple of hours, all the information needed on decision-makers' dashboards and public-facing e-government portals is refreshed and ready to be acted upon.

Our study on leveraging edge computing for property encroachment detection vividly demonstrates this workflow to solve a real-world problem, of relevance to metropolitan governance.

It is thus possible to bypass the complexities associated with regular satellite image-centric monitoring solutions. By delivering timely information to city authorities, one can empower informed decision-making and drive effective city management initiatives.

In Part-2, we will dive deeper into a few specific application areas in order to support city management and smart city initiatives in developing nations.

Quick Recap

So what are we trying to do again? As mentioned here, we are building infrastructure which can allow management of onboard applications for different platforms (payload hardware, sensor(s), middleware). We start with three-channel RGB and multispectral sensors for our demonstration. This involves the acquisition of tasked onboard imagery and mapping it to a task graph which defines the imagery pipeline to be executed. Typically this would involve radiometric and geometric corrections to produce a top-of-atmosphere (ToA) product along with quality data (like cloud and cloud shadow masks) and metadata, which can generate ‘Smart Tasked Images’ for consumption by GeoAI models (let’s say a real-time fire detection application output). The model outputs and other configurable data and metadata are then compressed and sent to the ground. A ground-based configurable pipeline is then called to generate quality parameters, perform additional atmospheric corrections (if needed), finer georeferencing, and makes the final outputs and output stats available for access to our users. Over the next few months, we intend to demonstrate our technology progress over the following missions:

Matterhorn: (Q1-Q2 2024)

As per the mission booklet, during this mission,

“The platform will deploy customer spacecraft, perform the in-orbit demonstration of third-party payloads, and validate several innovative features that will be available to customers on future missions”.

In the Matterhorn mission’s satellite, we will be testing our preprocessing and compression algorithms with an RGB imager onboard. The preprocessing algorithms include:

• a smart discard deep learning model,

• radiometric denoising and dehazing,

• cloud detection,

• spectral indices like the Visible Atmospherically Resistant Index (VARI) and visible waterbody detection using a normalized difference index alongside some intelligently cropped regions of the input image.

We maintain a (output insights/input image size) ratio of 5 or better while trying to maximize the information extracted from the input image. This mission requires software image uplink using onboard infrastructure and run it over certain areas of interest. We intend to perform coarse georeferencing on the ground.

Denali: Loft Orbital YAM-6 (Q2-Q3 2024)

Spacenews covered us here! The Loft Orbital YAM6 will be a virtual mission-enabled satellite. As far as Loft YAM-6 capabilities go, here is a snippet from their brochure:

Virtual missions provide the opportunity to radically shorten time to orbit. Customers can deploy their own software apps to our infrastructure to analyze data as it is being collected, enabling all kinds of use cases like tip-and-cue, response and sensor fusion. Software apps that require compute power, like AI and Machine Learning, enable us to use the unique vantage point of space in a variety of important ways.

We are working to deploy the entire onboard pre-processing software for this mission to demonstrate:

• high-quality top-of-atmosphere (ToA) product generated,

• smart tasked imagery generated,

• classical and deep learning models tested on the ToA products,

• commissioning procedures to perform radiometric Cal/Val and general image pipeline performance and quality enhancement,

• configurable pipelines through tasking command templates,

• submission and standardization to ingest new models and tracking model performance over time through a common access portal for model developers

K2: SatRevolution SOWA (Q2-Q4 2024)

The SkyServe-SatRevolution mission will be a hosted hardware and software payload on the SOWA platform. SatRevolution SOWA Universal Platform is a 6U NanoSatellite shared platform aimed at provision of in-orbit demonstration and in-orbit service for SkyServe. This will be launched on a PSLV rocket from India in Q2/Q3 this year.

SkyServe STORM is placed on a 0.25U-sized payload consisting of a high-performance edge processor that is flying on SatRevolution SOWA satellite. This computer runs proprietary firmware to interface with the host satellite, and middleware that allows the installation of enterprise analytics applications, and tasking them to receive programmable insights. Alongside other features, we have built:

• a unique stitching scheme to prepare a ToA scene for the VISION 300 imager,

• a dehazing algorithm optimized for the hardware,

• a custom georeferencing algorithm using onboard sensors,

• communication services with onboard computers which are robust to failures,

• customized Linux-based OS image for the hardware,

• continuation service of a task after recovering from an unscheduled power cycle.

Summary of the tech evolution

A more complete chart of the technology evolution over the three missions can be seen below:

As you can see, we are planning to demonstrate the technology to be matured in an incremental (and sometimes supplemental) manner with each mission. The development requirements grew organically from wishing peak insights generation per day capacity to maximizing compression to the runtime ratio for each platform. There was a growing need to protect third-party models which led us to build encryption and model standardization guidelines for each mission and building automation tools to minimize the manual effort for the same. I wish I could cover more in this blog but maybe another day.

The delivery

It, of course, has been an interesting time building this technology which has gone through long internal and external debates and reviews. In no order of importance, the key folks (some of them no longer with SkyServe) involved in building STORM have been amazing: Shashi, Nayan, Nitai, Sreekar, Harmit, Arvind, Dr. Ramakrishnan, Vijay, and of course other testers and reviewers. The experience of testing and delivering the SatRevolution payload hardware for the export (around mid-2023) was humbling and bittersweet. My apologies to all who are not in the pictures below (but you missed it😉).

Of course, the payload software delivery for the DOrbit and Loft software missions are happening in the really-really soon 🤞.

My learnings

• Build good-quality, functional and robust systems first and then focus on performance optimization.

• Most GeoAI edge models perform very differently for different levels of processing.

• It might be a good idea to be in a team of engineers who don’t necessarily come from a space background and are willing to take more risks. Many are extremely curious about working on space applications but don’t get opportunities.

• In space, perceptions change if you are doing tests in space and not in lab perpetually waiting for a launch.

• Agreeing on product names is the hardest problem 😉.

Want to know more about what we are doing? Click here

Early Detection - VISIBLE TO THE SKY!

According to the 2022 Global Annual Tree Cover Loss by Dominant Driver - Global Forest Watch, 2,28,190 sq. km of deforestation occurred due to anthropogenic activities like urbanization and commodity-driven deforestation.

In Mongabay's investigative piece "Meeting an Illegal Logger" by Robert Eshelman from 2014 , the illegal Indonesian logger revealed that logging to load a complete tractor with wood can span approximately 6-8 hours. Logs are then transported via river using boats. (We recommend Robert Eshelman's article to immerse yourself in the real and tough conditions that push one into this profession.) The signs of logging activity, like sawmills and log rafts, are often clearly visible from the sky!

Early detection of such activities is crucial for authorities to take action and reduce deforestation.

Capacity Building to Combat Deforestation

Increasing the frequency of monitoring and reducing the time between early warning analysis (using satellite images with predictive models) and early warning alerts is crucial to building local capacity to combat deforestation and promoting transparency globally.

Unsurprisingly, forest patrolling has human resource shortfalls and logistical bottlenecks. On the data side, satellite imagery available to teams has temporal gaps, producing an alert has high computational costs and needs specialized experts.

In light of the data limitations, researchers have started modeling deforestation activity to provide a more strategic approach as depending on overhead imagery (drones or satellites) for tactical response becomes expensive very quickly. By analyzing spatio-temporal patterns of illegal logging, we can find proxies for logging activities with the available data (open-access USGS and Copernicus satellites). Correlating these proxies with single data source pixel pattern differences and time-series analysis of multi-source data could be used to trigger alerts, albeit with temporal blind-spots.

Annotating satellite imagery using patterns intrinsic to illegal logging provides a foundation for building effective early warning systems. These systems rapidly detect these signatures in every new image, providing an alarm for illegal and unsustainable activities that might otherwise go unseen.

Temporal gaps in satellite imagery, and prohibitive computational costs have an indirect effect on the models themselves. You can see only what you can afford to - and in that lies an artificial limit to the currently active early-warning systems.

Take the example of Forest Alert a real-world example of the work done in partnership between the University of Leicester and the Kenya Forest Service (KFS). Built upon the data collected by the Sentinel-2 constellation, Forest Alert detects incidents every five days and generates notifications within 24 hours. If we assume a clearing rate of four hectares per day, we can potentially lose 24 hectares of forest area by the time the alert hits the patrol team's device.

The effectiveness of early warning systems depends on the frequency and availability of satellite data, and any compromise will hinder its performance.

Our primary need is timely alerts!

As soon as the system detects deforestation and triggers an alert, the rate of deforestation should decrease.

To detect illegal logging activity more effectively, we require satellite data that is acquired more frequently.

By utilizing a combination of satellites such as Landsat and Sentinel, each with different revisit days, we can align our observation calendar to access more satellite data at no cost.

If our budget permits, we can procure commercial satellite images to guarantee that each day,a new satellite image will be appended to our database.

However, obtaining more satellite data results in a significant need for greater storage and computational capacity since the image capacity increases manifold when the spatial resolution of the satellite image rises.When the resolution of satellite images doubles, the pixel size increases fourfold.

Nonetheless, the requirements for data storage and computational infrastructure remain high.

Even if we receive an image every 24 hours, the analysis required to trigger the alert can take an additional 24 hours. So, we lose up to 2 days from the incident to even thinking about on-ground actions.

Leveraging Space Segment

The two major segments of the deforestation early warning system are ground operations and the imaging satellite. The former serves as the instrument to enforce laws and neutralize threats, while the latter enables rapid monitoring and detection over areas larger than the ability of a ground team. The space segment feeds data to the ground segment, enabling timely warnings and interventions.

On-board edge computing in focus

While the revisit can be improved through access to more satellites, the processing latency cannot be sustainably solved by just increasing raw computing power. In industries like manufacturing and transportation, this has been achieved by bringing compute and algorithms to the sensors, thereby speeding up decision-making. In our scenario, this would mean that our AI model needs to run on the imaging satellite literally! This is actually possible, and the experts call it satellite edge computing.

If trained models are deployed on satellites equipped with on-board edge computing, tailored monitoring of high-risk regions susceptible to deforestation can be achieved. This capability is essential as the net latency in monitoring of deforestation incidents is critical for timely intervention.

By processing data directly on the satellite, the system minimizes delays associated with transmitting raw data to ground stations.

Moreover, this technology allows for customization such as receiving only area of interest images, only the affected area boundary, or just an alert message, ensuring that transmissions from the satellite deliver precisely what is needed for the teams on ground.

In the given scenario, the model deployed onboard is purposefully engineered to generate deforestation early warning alert messages directly from the satellite. With the ability to deal with only boundary files or text alerts, actual procurement of satellite imagery is, for all practical purposes, eliminated. Need for resident experts in data analysis can be reduced and so is the cost of storage and compute while still achieving the critical goal of delivering timely early warning alerts.

Figure 4 (A sample scenario) illustrates how on-board edge processing can empower daily deforestation monitoring by increasing the number of alerts, leading to more field interventions. This approach can reduce temporal gaps to mere hours, while simultaneously addressing data storage, computation, and associated cost limitations.

Value additions from On-board edge computing to deforestation early warning systems

• On-time alerts: Faster analysis enables immediate notification of deforestation activity, allowing for quicker response by field teams and authorities.

• Efficient resource allocation: Field teams can be deployed quickly to areas with confirmed deforestation activity.

• Improved trend monitoring: Continuous analysis allows for early detection of emerging deforestation patterns.

• Empowering local authorities with transparency: Faster alerts enable timely enforcement and build trust with communities.

• Global collaboration: Timely data empowers stakeholders like scientists and NGOs to monitor deforestation trends and track the effectiveness of conservation efforts.

Time to trial Space-AI for combating deforestation?

Deforestation undeniably contributes to global warming, with a cumulative annual release of approximately 4.8 billion tons of carbon dioxide into the Earth's atmosphere.

Carbon offsets, intended to mitigate greenhouse gas emissions, particularly those stemming from deforestation, are facing scrutiny due to their limited effectiveness in combating global warming. only 5.4 million out of 89 million credits (only 6%), actually resulted in carbon reduction through forest preservation,scientists reported in Science (Aug, 2023).

Early warning systems should be viewed as a technology enabler to make programs such as REDD+ work.

As we explore the potential of On-board edge computing in deforestation monitoring, it is time to trial this innovative approach on a larger scale to achieve more sustainable forest conservation outcomes.

We’re thrilled to share that we're teaming up with Loft Orbital for Mission Denali to deploy SkyServe STORM, our in-space edge computing platform on YAM-6. 

Mission Denali will leverage Sensing and Compute Resources on YAM-6 to deliver insights from space. By processing multispectral and hyperspectral data in real-time, SkyServe STORM is defining the paradigm of space-based data processing, delivering insights processed from the edge. 

YAM-6 blasted off to low-earth orbit aboard a Falcon-9, on the SpaceX Transporter-10 dedicated rideshare mission from Vandenberg Space Force Base in California, US on March 4th. The mission carried 52 other satellites that, like other Transporter missions, were composed of innovative new space startups both demonstrating new capabilities and increasing capacity for existing ones.

SkyServe STORM for Geospatial Developers

SkyServe is empowering solution providers and core industries with software infrastructure to deliver real-time insights from space. SkyServe STORM will deploy customers’ AI models on the edge leveraging Loft’s satellites to deliver on-demand predictions for wildfire tracking,  irrigation, mining logistics, maritime surveillance, energy, utilities, and carbon monitoring. By deploying AI models onboard STORM, analytics providers can now accelerate their access to space data and process information in real-time at scale.

On-demand information from space has prompted the need for a reliable partner in space operations, and we are proud to work with Loft to provide the essential space infrastructure and multi-sensor data for SkyServe's ambitious goal to build critical value for geospatial developers.

How Loft Makes This Possible

Loft is pivotal to SkyServe's mission by providing space infrastructure and sensor data to power their platform. While many space companies are constrained to the traditional process of designing, building, integrating, testing, launching, and operating a satellite, Loft manages this challenging, capital-intensive process so customers like SkyServe can directly access the data they need. SkyServe STORM will leverage Loft's compute resources and sensing capabilities including multispectral, hyperspectral, and RF sensors.  

What's Next (TL;DR)

In the coming months, SkyServe will enable geospatial companies to push their AI models to space in a snap. It will help them process and feed multi-sensor data into their models onboard. The shared vision of making space simple, and democratizing satellite data pushes the boundaries in the realm of space-based technologies.

Introduction

Encroachments are value-depreciating events for any parcel of land. Sequences of encroachment can cause not just inconvenience to any property but also have financial impacts for residential as well as commercial land, utility installations like cross-country pipelines and power transmission, mining blocks as well as railway lines.

Arya, a product developer studying the above, saw it as a market opportunity and listed more thrilling details. he came across real-world scenarios that played out in the recent past and the size of this unmet need:

• "Forest land five times Delhi's geographical area under encroachment, govt data shows. The latest data showed that 750,633 hectares or 7,506.33 square kilometers of forest land has been encroached in the country." - Deccan Herald (05 January 2024)

• In the USA, the direct financial impact of vandalism and theft in vacant properties is significant, with average losses ranging from $2,000 to over $20,000 per incident. - The Urgent Need for Security in Vacant Commercial Properties: A Comprehensive Guide - 06 - Feb -2024

• The robust growth (8.27% CAGR 2024-2028) in the security services market highlights an increasing demand for private security. This demand is intricately connected to the issue of property encroachment, as property owners seek security services to defend against boundary violations.- Statista

Objectives for Effective Solutions

Enthused by the numbers he's seen, Arya looks into the services addressing property encroachment issues.

The existing solutions vary from assigning security personnel to installing CCTV cameras and using drones. These have been the top touch-points for most property owners in tackling encroachment.

At the back of his mind, Arya understands that these techniques have limitations staring from the actual encroachment detection to reporting. For example, malpractice is possible while engaging security personnel; monitoring vacant land on a large scale with cameras is not really practical and so goes for drones as a deterrent considering the lack of regulatory clarity and the risk of injury to humans on the ground.

From the survey, Arya derived a few objectives that a typical service should deliver:

• Objective - 1 : Ensure effective monitoring by detecting encroachment in a timely manner.

• Objective - 2 : Communicate the essence of the incident to the stakeholder with the least delay.

• Objective - 3 : Cost-effectiveness - Cover budget aspirations of property owners at different levels and different categories of properties.

Searching for scalable solutions, Arya found that "Remote monitoring" addresses the limitations of the on-site solutions he has seen so far. Remote sensing fused with geospatial technology seems to be already addressing these challenges, and the ability for better encroachment detection is surging with all the new satellites being launched to space!

When digging deeper into this, Arya finds that high-resolution satellite images are a critical input.

Arya encounters Aleks Buczkowski's blog, which discusses the market availability of satellite imagery, particularly emphasizing the "Minimum order area" and pricing based on dollars per km².

Though this solution is technically more effective, Arya found a gap in cost-effectiveness when trying to sketch out a prototype solution.

For monitoring a vacant parcel of land spanning 0.23 km², Arya tries to collect the combined solution cost for the top-rated offering today (CCTV with motion-detection and ability to have an armed security person on call) and compared that with a solution that includes an option to leverage the satellite image-based idea.

He realizes that a high resolution satellite image covering 100 km² has to be acquired from the market, to monitor his precious 0.23  km².

Also, Arya finds that the satellite images will always remain under-utilized and become a pain-point.While visualizing the business scenarios, Arya came up with two.

1. More properties to monitor in a smaller region.

Here, a satellite image was acquired for 100km² (smaller region) in conjunction with more properties to monitor, assuming even a measly three-fold of the security and CCTV customers, then using satellite image for this scenario looks competitive.

2. More properties to monitor at the State/Country level.

If Arya were to service property owners country-wide, then he would certainly get more customers, but he would need to procure a massive number of satellite images.  

For example, if ten properties each spanning one km² are spread across ten different US states, then imagery to be acquired would be 1,000 km², and the utilization ratio would stand at 1% even after the sizable spend.

The existing and traditional methods seem more viable cost-wise in this scenario, but the first two objectives will still need to be addressed.

Arya understands that scaling the encroachment-monitoring service to the state/country level turns the business potential from excellent to vulnerable due to the meager utilization rate of images. If one compares with high-profitability SaaS businesses, this current iteration of the monitoring solution seems to be going in the wrong direction - the cost to maintain a larger customer base just keeps going up!  His conclusion is that the utilization rate of satellite imagery procured has to be high in order to result in sustainable and profitable growth of his business.

Area-of-Interest Matters

Arya, after a week of thrashing out his ideas with friends and over caffeine shots in cafes, comes up with a seemingly obvious question:

"A satellite image is required to detect encroachment. But for 100 km² or the area of interest?"

He blurts out, loudly, "Image of the Area of Interest!".

This epiphany ignites another search, "Is it possible to get an image of just the area of interest?"

Arya calls up his old school friend, now a professor or Remote Sensing at a prestigious university, and tries his luck. Dr. Neil winces at the question with a knowing smile and explains, "I have had to defend satellite imaging so many times, especially when this specific question comes up. The field of remote-sensing has been a boon for us scientists, but it hasn't really changed to cater to commercial needs where costs need to be controlled in relation to the final offering's market potential and end-user preferences". After giving a long pause, Neil has a twinkle in his eye and says, "There is a way, and this is a new development where the satellite, when provided with the area of your interest, can figure out which parts of the image lie within it, and send only that to the user. This is an simplified explanation for a complex process that is being built and tested as we speak by researchers. The costs can come down by 30-40% and over time really become proportional to the area of a customer's interest!"

This means, the answer is YES! Okay, maybe as Neil says it will take a year or more for the engineers and scientsist to perfect this, but the answer remains very ecouraging. Neil points Arya to the keywords "edge computing" and "onboard processing" when looking for a way to bridge the gap in his product.

Onboard Edge computing enables delivery of something called "Smart-tasked AOI Image", which is an image of our area of interest, eliminating the minimum order area as well as unnecessary images that might not have clear visibility of the location due to cloud cover.

So, with edge computing combined with a high resolution imaging satellite, the running costs for data acquisition becomes controllable over time, the only job left to be done is automating Encroachment Detection through Machine Learning.

Once the technology and communication choices are made; the question of cost-effectiveness must be answered.

Arya is redefining the needs once again:

• A satellite image for the AoI has to be acquired at the least possible cost.

• The overall storage and processing costs have to be low.

Images will still have to be stored on the cloud, high-availability compute needs to require cloud compute - all have costs associated, and storage costs will only grow. Larger the data footprint per user per inspection, higher the slope of the recurring costs.

Arriving at the opt-solution

Armed with the understanding of 'edge computing', Arya determines that what his customers need is mostly the notification of encroachment, and image is needed at best for visual confirmation - not a heavy TIFF file, just a JPEG thumbnail. He could bring the encroachment detection model closer to its source: "The satellite itself"!

To harvest the benefits of satellite edge computing and provide a commercial service through the value chain, the model has to be placed on the satellite.

Imagery takes second place. When the encroachment is detected onboard by the satellite, a notification needs to be sent to the user - just some text detailing the nature of the encroachment needs to be conveyed. If the user opts for a visual confirmation, an compressed image can also be linked - maybe this can be a premium feature!

Text and a compressed images take very little storage space, ground-side processing time, thereby reducing the total cost of solution, delivering the optimal goods to meet the objectives: technology, communication, and cost-effectiveness. Now, no need to worry about customer density. It is viable to serve a much more significant customer-base irrespective of geographical spread.

Arya finds that, as a product developer, it's crucial to address the limitations of CCTV & physical monitoring services by bridging gaps rather than replacing it with a single one-size-fits-all solution. Building trust incrementally, and also factoring in the limitations of today's satellites (persistent monitoring is not yet there with high-resolution satellites, and inspecting the interiors of a building is still sci-fi).

Through ample study of customer preferences, we regard text notifications and visual imagery for inspecting encroachment as complementary enhancements to physical security personnel and mobile-connected CCTV offerings in the market. Furthermore, these features hold the potential to serve as standalone products over time.

Arya starts working on his product, with an understandably geeky and un-exciting name "EncroWatch". But hey, the product is exciting!

This is not an imaginary product. The App depicted has the potential to hit the market through the endeavors of many product developers like Arya, ably supported by innovative researchers and startups worldwide working to bring the benefits of edge-computed insights to one and all!

Financial Crisis to AI-Powered Solutions

In California, a renowned insurance company is facing financial instability due to wildfires.

Wildfires are becoming increasingly frequent and widespread, as evidenced by recent data. As summarized in this report by the World Economic Forum, "As climate change causes more natural disasters, the insurance industry is raising rates, reducing coverage and exiting some markets entirely".

The need of the hour is to equip these tough and very real scenarios with tech solutions. Recent advancements, such as AI, ML, and their fusion with satellite imagery and geospatial technology can empower service providers who offer analytical services on assessing wildfire risks, thus serving insurers and communities well. Used judiciously by the data scientists and analytics teams, this can really change the game for insurers!

And this is not all! More recent innovations can further power these tech solutions.

For example, imagine a deep-learning model trained to detect areas at risk from wildfires utilizing satellite imagery, running on every satellite looking at Earth, continually - with assessments delivered directly to the insurer's decision-making tools. Sounds futuristic? Not at all!

This article describes the transformative capabilities of edge-computing paired with satellite imaging and its potential.

On-board processing - Supercharging Wildfire Financial Risk Solutions

The model, trained to assess the property-centric wildfire risk, is on-boarded. The satellite on which this model is on-boarded will possess all relevant tools:

• Platform to accommodate the model

• Capacity for earth imaging

• Provisions for Geo-processing and generating input for the model

• Ability to run the model and extract the final output

• Transmitting the property-centric assessment.

The satellite will directly transmit the property-centric assessment to the insurance company's decision-making tools. Based on this assessment, the premium will be determined according to the current scenario. Similarly, for claims, damage can be assessed using the same method.

The up-to-date details and the holistic process make this a hassle-free, win-win situation for the company and customers.

What is actually happening? - Here, one does not order satellite imagery; rather the insight.

Is this just bells and whistles added to traditional remote sensing? - We have come a long way since the advent of satellite imaging as it began in the 1960s - from manual visual interpretation from printed 1:1 image, to convert insights to decision-making in an automated workflow. While the key technical processes remain largely the same, the speed at which we can arrive at meaning from pixels has increased exponentially. Executing this at the edge, on satellites eliminates the latency and with AI/ML, this process can only get better - better than the human eye!

Feasibility: Coming to the technicalities, it is possible today to connect a real-time feed of satellite images to your model and configure it to run in an automated way, with the key to governing its behavior held by you - the model's creator!

Better Perspectives: Financial resilience: The on-board edge computing is designed to smart-task processes like radiometric correction, image quality enhancement, and geometric correction on the raw satellite image and produce the cloud-free AoI image, then feed this into the model on-boarded to deliver targeted data insights.

Focus on the need - Delivering Focus of Interest: The smart tasking mentioned above helps focus on the area of interest, which could be a minimal portion of the larger image, for example, 10% of the pixel ratio over the larger tile of the high-resolution image.

Critical inputs for financial risk discussions: Delivering the Data of Interest: To make decisions on financial risk-related issues, the risk management team needs key inputs like the quantum of damage, areas to be focused on, and number of property damages. Edge computing platforms can deliver these details to the decision-making tools from the satellite.

Continuous improvements: Configurable outputs: Once the model is operational on the satellite, the platform ensures flexibility in updating the model, which may be necessary due to factors such as policy changes, climate variations and geographical reasons thereby influencing the outputs. This feature underscores the control of the model creator over the model's governance.

On-time details - Time-saving: Edge computing helps to extract pixel subsets of the area of interest, and numeric derivatives, result in fast data transmission to the decision-making tools. This faster transmission is possible by the memory footprint compression.

Value propositions from On -board model and On-satellite Analytics:

• Significant cost reduction in wildfire monitoring and Property/Area of Interest risk assessment, since the whole process concerns areas of interest.

• An insurance company's decision-making tools are updated with timely details due to a faster transmission of data, supporting quicker decision-making.

• Elements like targeted notifications and fire detection models enable proactive mitigation, reducing the financial burdens of an insurance company and reinforcing better administration.

• Remote updating the on-board model, making this process long-running and adaptable for better long-term assessment and continuous improvement.

Advancing financial stability

In response to the escalating threat of wildfires, insurance companies require comprehensive insights and timely data to ensure efficient financial management. On-board edge computing emerges as a pivotal solution, seamlessly integrating data into wildfire-related financial risk management processes. This innovative product fosters resilience within the insurance sector by providing timely insights for informed decision-making and proactive risk mitigation.

Join the innovation

SkyServe laid the foundation for this product, as detailed in this blog, "SkyServe Partners with SatRevolution for the 2023 STORK Launch."

Limiting ourselves to multispectral sensors for this blog, there is enough evidence that model performance degrades if the spatial resolution, spectral response, preprocessing or sensor specifications in testing data are very different from training data. There are extremely sophisticated but costly tools to simulate remote sensing physics and sensors to generate synthetic imagery for testing models. But if you want to quickly start off with model testing with minor accuracy hit, it might be a good idea to build something simpler yourself.

The blog covers the following tools to build your own synthetic data generator:

• Review of the PhiSat-2 Sensor and level of processing modeling notebook

• Evaluation notebook for spectral mapping between two sensors

• Evaluation notebook of the CLIP-RS-ICD model for captioning

Requirements for a simple data generator

A tool is required that can take existing open satellite imagery and turn it into imagery of a desired satellite sensor by mapping spectral response, spatial resolution and viewing geometry (lets say nadir facing) between two sensors and sensor specifications (e.g. noise characteristics) and preprocessing done on the target sensor image. Captioning such an image helps to search images of certain class. For model testing, say, get all coastal images for your coastline monitoring model.

Review of the PhiSat-2 simulator

The European Space Agency (ESA), had launched the OrbitalAI Φsat-2 challenge as a global competition last year. For this competition, the participants use a simulated dataset and create a working prototype model adapted to run onboard Φsat-2 (an upcoming mission at the time of the challenge). The challenge provided the participants with a Φsat-2 simulator jupyter notebook to generate representative L1C onboard imagery from open-access Sentinel-2 L1C imagery.

The L1C imagery referred to Top of Atmosphere Reflectance in sensor geometry, fine geo-referenced, fine band-to-band alignment (<10 m RMSE). The PhiSat-2 simulator's essential steps are shown in workflow above. In the interest of simulating any area of interest and not have any cost constraints, Sentinel-2 Multispectral sensor's L1C reflectance product is chosen as an input to the simulator. The simulator takes the 10m-Ground Sampling Distance (GSD) and implements a bicubic interpolation to get to the representative 4.75m-GSD image output for all the eight bands. The next step involves a band misalignment simulation. Due to the pushbroom acquisition mode, the various bands over a given area are not acquired at the same time and suffer from platform attitude stability during the various acquisition times. Keeping a specific band as reference, a misalignment direction and magnitude is chosen as random number extracted from a normal distribution to shift each of the bands. To account for the system Modulation Transfer Function (payload + platform), the next step consists of the convolution of the input image with a filter function representing the system specific Point Spread Function (PSF). Basis the solar irradiance, Sun-zenith angle and Earth-Sun distance, the multiplicative factor needed to convert radiance to reflectance is computed to simulate the L1C product. The last step involves tiling the image into 4096 x4096-sized grids as the final product available for processing onboard. For more technical details, look at the scientific notice.

Takeaways

• If you know the sensor specifications, you can use functions (and add your own) in the notebook to build a data generator of your own. You can add preprocessing steps to simulate the level of processing at which your model expects the imagery input to be.

• Relies on open data (through SentinelHub)

Limitations

• Spatial resolution simulation involves bicubic interpolation. This can be improved by superresolution (like this) which can retain shape integrity and band ratios.

• Changing view geometry is not a feature here.

Spectral mapping between VENµS and Sentinel 2 sensors

Before a mission, there may not be any directly available comparative imagery to map spectral relation but post-flight, there just might be enough data to build a spectral mapping with open data like Sentinel 2. As covered in my previous blogpost, VENµS datasets at 5m resolution are openly available (consider the mission to be equivalent of your target mission) which can be spectrally mapped to Sentinel 2 MSI images.

We provide a workflow and associated code in a Colab notebook and sample images for your reference.

‌Workflow steps:

1. For a VENµS (L1-equivalent) scene that is derived onboard, a reference Sentinel 2 L1C (pre-atmospheric-correction) Image which just exceeds the area of the VENµS scene and is closest in time w.r.t. Raw image captured onboard for the VENµS scene.

2. Get keypoints for both images using Lightglue’s extractor and matcher (performs better than SIFT+RANSAC) and find homography.

3. Reproject the Sentinel 2 L1C scene onto the L2A scene.

4. Extract corresponding R,G,B pixels of the Sentinel 2 L1C's R,G,B pixels after removing cloud pixels or zero-brightness pixels.

5. Build a linear regressor that maps the R,G,B pixels for each channel between the L2A scene and the Sentinel2 scene.

Notice the distinct variation in the water and ground colour between the two images. Upon closer inspection, there are many sensor artefacts in VENµS imagery that don’t seem to appear in the Sentinel 2 imagery. If you are able to follow through the steps provided in notebook, it would be easy to replicate red-red spectral map between the two sensors and build a linear regressor. As can be seen, there are several outlier points which can be due to different colour patterns in the two images. In images, cloud and cloud shadows, atmospheric effects all play a role in the fit quality.

And look at the beatiful results. For example, here is an image of Sentinel and Venus for Douglas, US and you can see the color and artefacts are very faithfully appearing.

Takeaways

• If you know the view time and geometry and radiance values, you can modify the notebook to build a spectral map for your own mission.

• Lightglue makes the extraction and matching of keypoints reliable for two different sensors.

• Relies on open data (through SentinelHub) and some target mission imagery.

• Expect to improve model performance as soon as you have better spectral response in your simulated imagery dataset using spectral maps for your mission.

Limitations

• Requires a larger sample size of data and cloud/shadow-rejection models to build more robust classifiers. Read this for more ideas.

Captioning: Evaluation of the CLIP-RS-ICD model

Transformer architecture in deep learning has gained immense popularity in computer vision problems as well and one such application in remote sensing is image captioning, i.e. assigning labels to remote sensing images. Technically speaking, it is zero-shot image classification which is a task to classify images into one of several classes, without any prior training or knowledge of the classes. Zero-shot image classification works by transferring knowledge learnt during the training of one model to classify novel classes that were not present in the training data. One of the most popular models for zero shot image classification is the CLIP model from OpenAI which uses transformer based image and text encoders. A version of this model called the CLIP-RSICD is finetuned on image caption pairs from mainly the RSICD dataset which contains more than 10k remote sensing images from Google Earth, Baidu Map, MapABC, Tianditu and 5 sentences descriptions per image with each image having a 224 X 224 image resolution at various spatial resolutions. Other datasets used in training are- UCM dataset and Sydney dataset which too contain images with 5 captions each. This model provides the ability to quickly search large database of such images for specific features i.e. text to image retrieval. Our larger goal of performing inference on this model is to evaluate if model developers can quickly find images of their interest from our database through some relevant search prompts.

Evaluation

The CLIP-RSICD model was used to perform inference on Sentinel2GlobalLULC which is a dataset of Sentinel-2 georeferenced RGB imagery annotated for global land use/land cover mapping with deep learning. It contains over 190k images from 29 LULC classes and each image has a 224 X 224 image resolution at 10 m spatial resolution. During inference, image and the captions corresponding to classes of interest are provided to the model for every image and the model assigns a probability score to each caption (i.e. class) which indicates the degree of relatedness or similarity of that class to the input image according to the model. The evaluation Colab notebook is available fpr details.

Learnings and takeaways

Some learnings from using this model for the inference process:

• Provide full sentences i.e. some visual object description as input captions and not just labels/classes as prompts for the input image. Sometimes images can include features of more than one class like forest (major class) and a waterbody eg: a river in it (minor class), but we want the model to classify it according to the major class, so in those cases the prompts can be framed like- ‘An aerial photograph depicting a forest predominantly’. Example:

• Providing too many captions at once can confuse the model and cause the model to undermine the probability of classes more relevant to the image. As a rule of thumb, consider providing upto 10 captions at a time. Since the above dataset has 29 classes, the classes were grouped into 8 main i.e. super classes as per their high level similarity with each other. Inference for each image was carried out in a 2 step hierarchical manner- in the first step inputting only the super classes for each image and in the second step inputting all the classes under/corresponding to the super class with the highest probability obtained from the first step. The class with the highest probability hence obtained is the one that is considered to be the most similar to the input image. 8 Super classes used were- ‘Urban and built-up area’, ‘Grassland or Barren land’, ‘Snow or Lichen land’, ‘Shrubland’, ‘Wetland’, ‘Cropland’, ‘Forest’, ‘Water body’.

• Understand the image pre-processing used during fine tuning of the CLIP-RSICD dataset and make sure the same is used to pre process images before passing it to the same model for inference. Additionally using histogram equalization like CLAHE (Contrast Limited Adaptive Histogram Equalization) to improve the contrast of the images prior to the pre processing helps improve performance.

• Inferencing speed: Average GPU inference time (from loading image to getting the probs on CPU): 83.9ms is impressive.

• Measure performance of the model mainly as top k accuracy and create a plot of k (from 1 to 29) vs top k accuracy. Use top 10 (or equivalent for your case) labels for captioning and indexing every synthetic image for searching.

• The ultimate way to improve model performance would be to fine tune the CLIP model on the specific dataset, and test the inference performance on a holdout i.e. test set of the dataset.

Apart from the above, there are always fake cloud generator, deepfake imagery generator and similar tools to keep adding to your inventory.

Summary

The blog attempts to cover some open tools and data that help you build a synthetic data generator for free and caption it for summoning all useful data for your model testing. It is recommended that you build your own tools to suit your mission needs.

Spectral Mapping Colab notebook and Captioning Colab notebook are shared for evaluation of the respective tools for reference.

So let the pixels fall, and spectra sing, in every byte, a new discovery spring,

For in the quest to see, to know, to share, the greatest reward is the journey, and care.

Want to know more about what we are doing? Click here

Understanding LISS 4

Background

LISS 4 (aka LISS IV, LISS-IV, LISS-4) is part of the Resourcesat-2 Mission planned to provide integrated land and water resources management with enhanced multi-spectral and spatial coverage. The satellite is designed to provide both multi-spectral and panchromatic imagery of the Earth’s surface. The payload system comprises of three optical remote sensing cameras, viz., LISS-IV, LISS-III and AWiFS. LISS-IV (LISS 4) provides 5.8m resolution in three bands with 70Km swath and is the focus of this piece. The Data Users Handbook which is hosted on a German domain for some reason 😕.

The LISS 4 emerges as a royal sirji, flaunting a resolution that's considered high (or medium as per the 2024 cool kids), rocking the green, red, and near-infrared hues. In the multi-spectral mode, a 23.5 Km swath gets bathed in the three bands. Switch to mono mode, and the full 70 Km swath becomes a canvas for a solo band. It throws in a sassy tilt of ±26° in the across-track direction which makes it fashionably punctual at 5 days revisits.

Comparable global sensors

Here is a few comparable sensor specifications I compiled from the global public database:

Nice FoV, nice FWHMs, looking like a wow . Plus it is heartening to know both LISS 4 and VENµS datasets are open now.

Standard Product Generation Workflow

This is what I could interpret how the processing workflow for the generating the standard Level 2 product of LISS 4 looks like. The radiometric corrections seem to take initially care of detector-to-detector non uniformity through a look-up table made through radiometric calibration process done in a lab. This is followed by area of interest extraction using line count or ground receive time metadata. Failed or degraded detectors, which cause striping effects which can be estimated “only if the failed detectors do not exceed two consecutive CCD elements”, are then corrected. Interestingly, line loss corrections are done through averaging or repetitions depending on number of lines lost. Staggering effect, a function of tilt, speed of the platform and integration time is then sobered. Lastly, anisotropic filtering as a denoising technique is employed to a Level 1 Radiometrically corrected product.

The data is corrected for geometric distortion effects (like Earth’s curvature), band to band misregistration and ground-based fine registration to produce the Level 2 Radiometrically and Geometrically Standard product that we will be laying our hands on.

Historical work on LISS 4 products

The historical work (100+ on Google Scholar) can be categorized as:

Preprocessing

Staggering effect correction, comparison between LISS3, LISS4, AWiFS, atmospheric effects correction, coregistration, etc.

Applications

Crop classification, sugarcane classification and sugar yield prediction, mangrove classification, bacterial leaf blight identification, water quality assessment, land suitability for cashew plantation, land use classification, banana classification, seagrass detection among others.

The study that I was most excited about was “Remote sensing to map the invasive weed, Lantana camara in forest“ by Kimothi et. al. The study covers Lantana camara, an invasive weed species that has destroyed 40% of indigenous plant species coverage of the Indian tiger reserves. The study looks at an area of 3.7 km x 8 km where lantana camara field data was tagged to a fused LISS 4 and Cartosat 1 image and LISS 4 False Coloured Composite (FCC). Using 3x3 pixel windows, the authors identified statistical features to train a maximum likelihood classifier. They found that classifier trained on LISS4 FCC had a user’s accuracy of 95%, better than the fused product with Cartosat 1 at 92%. If there is more spatially diverse field data made public/available, a better classifier could be trained that can map all the lantana plantation in India. This helps us with a superb measuring tool to track efforts against probably the biggest threat to Indian forests. Geospatial community, the challenge is open!

Data Exploration

LISS 4’s FCC (NIR-R-G colour setup) can compare with high resolution images like the one used by Google Earth’s CNES/Airbus imagery (10 times better GSD). Since they maybe slightly offset in time, there are broad assessments that can be made. Here’s one comparison for an image from Bahla, Rajasthan.

More imagery can be found here.

Opportunities

Dataset Development

I would love to see who (if not ISRO) can fund development of:

1. subcontinental yearly RGB mosaics (with simulated blue band) for public use: Google Maps can be upto 5 years old over some areas. Plus it can be useful for many.

2. Cloud Optimized Geotiff datasets of yearly water body maps at LISS 4 resolution.

3. datasets for converting satellite imagery to basemaps for dynamic navigation basemaps.

4. India-wide dataset for Lantana camara shrubs.

5. [copy-paste any dataset idea from this list please…]

Product Quality Improvement (suggestions)

Here is a sample Colab notebook that I made with the intention to explore the dataset and find out what more can be done.

1. Accessible through an API

   Can we please get an API that can allow us to query basis the area of interest, date-time range, cover cover threshold, product level and provides us all the products that we are looking for? It seems like the server capacity could be improved further.. one recent single product order has made me wait (“Product not online, availability will be intimated on your registered email“) for 4 hours and counting while I write this piece. I found Ujaval’s blog for accessing data (and preprocessing) manually helpful. Till then I just put a sample dataset on S3 and you can get it here as below:

   # If you want to run the analysis on your data, simply replace the below path of a sample file with yours: 
   
   !wget https://skyserve-nrsc.s3.us-west-2.amazonaws.com/R2F06OCT2023064684010000061SSANSTUC00GTDA.zip -q

2. Just if there was a blue band..

   There is no blue band in the LISS 4 datasets. But I found something cool in the NRSC’s User Manual page 5. Here’s the script to simulate the blue band:

   band2_data = src1.read(1)
   band3_data = src2.read(1)
   band4_data = src3.read(1)
   
   # Calculate the new band data
   # Reference: https://www.nrsc.gov.in/sites/default/files/pdf/Software_download/UserManual.pdf
   
   new_band_data = 55.6215 + band2_data * 0.873393 - band3_data * 0.138254 - band4_data * 0.100616

   I think the simulated blue band RGB seems to work:

   

3. Maybe we add some more denoising?

   It seems that anisotropic filter used in the preprocessing takes care of the directional noise. But I added OpenCV’s FastNLMeans filter which removes a lot of the random looking noise as seen below (with script snippet):

   # Denoise the image
   
   im_rgb_denoised_nlm = cv2.fastNlMeansDenoisingColored(im_rgb_norm, None,2,2,7,21)

4. Striping artefact improvement

   Failed or degraded detectors cause striping effects which can be estimated “only if the failed detectors do not exceed two consecutive CCD elements”. I also found striping artefacts which are one detector wide or more. For example, you can check out the striping in the green band of the LISS 4 image.

   

   This could be tackled more generally by looking at the 2D fourier transform of the image, filtering the high-frequency stripes without modifying genuine low-frequency image features and running an inverse transform followed by a fine refinement step proposed by Lloyd et. al.. As you can see, the stripe patterns have been mostly cured. I had to run a script in my local machine for this, could not cover it in Colab notebook.

   

5. Some metadata reference would help

   I noticed that NRSC has hosted the KOMPSAT-5 metadata definition under the product specifications here. I am not able to find the equivalent definitions for the LISS 4 metadata (maybe I have to dig deeper? or somebody could help?). For example, I would love to understand the definition (maybe with some illustration) of the ‘Tiltangle‘ parameter used in the metadata file of LISS 4.

6. Coastal georeferencing

   What I have seen in some images (like Product ID: 231986861 thumbnails) is that Red and Green bands align well but NIR band can get misaligned in the coastal scenes. Possibly because of no (or few?) Ground Control Points in the sea.

Onboard Computing

OK, here is some simple math. The satellite is having a ground trace speed of 6.65 km/s according to Table 3.1 of Data Users Handbook. When the satellite is imaging in the multispectral mode, the swath is 23.5 km, which means if the sensor is operated with a setting of (23.5/6.65)=3.5 seconds between two consecutive frame captures, it is possible to cover them with no overlap. If it were possible to perform every step of the computing mentioned in the section Standard Product Generation Workflow under 3.5 seconds, it would lead to a near-realtime generation of products onboard.

As noted in Table 1 of our paper Methods to Leverage Onboard Autonomy in Remote Sensing, the interesting pixel ratio can vary from 2 to 40% depending on the use case. Thereby, a standard onboard LISS 4 product size onboard (1.8 GB uncompressed) could be brought down to upto 36 MB (2% interesting pixels) if there were a model running onboard to extract the interesting pixels within the said 3.5 seconds. The current computing systems can get you this capability if setup cleverly. A satellite-enabled continous monitoring system can thus monitor large areas in a short period of time and report the changes as they occur with spatial context. Extending this ability to satellites with multiple heterogenous sensors enables continuous monitoring of a variety of phenomena at regular intervals. Together with inter-satellite links or low power broadcast messages, a timely disemmination of actionable information to relevant stakeholders can be enabled.

If you are an operator knowing that your customers are looking for just those interesting pixels and metadata or are interested in latency of actionable information, you just might want to consider investing in the onboard computing and infrastructure to save on the downlink costs. And putting this capability on ISRO’s next Resourcesat or equivalent satellite might be a good idea too.

The open data policy is a welcome step and it seems to open doors for all to look at the high-quality data being produced.

Want to know more about what we are doing? Click here

Now, for how I got here specifically working with satellite data: In 2020, I took part in ESA's AstroPi challenge, where our team had to identify a problem that could be solved using satellite data. We received satellite images that had our code on them.

We didn’t win the challenge. But that left me in awe for a while, how I could see our planet from space, and how I could so many things with the data. It got me excited because we could use the data for so many things that can help a lot of communities like finding algae in lakes, dangerous temperature levels, vegetation, and much much more.

Ever since, I have thought on how cool remote sensing and satellite imagery really is. It sounded unreal. When I got the opportunity to do an internship at SkyServe, I was nervous, amazed, and just excited that I would get to learn about how to use satellite data.

The whole opportunity was awesome. Around the same timeframe that I got a “Yes”, fires began burning in Canada. Especially in British Columbia, Alberta, and the Northwest Territories. A lot of families, communities, and even towns had to evacuate because of that. I live much more to the east, but I could smell smoke and the view was slightly disturbed and slowly began to faint. It was all over popular news channels and websites. Some of them showed satellites picking up smoke in their images. That’s when my idea clicked. This seemed like an exciting idea, to use satellite imagery to find fires.

I began with the help of my mentor to learn about all of the layers that can be utilized for this application, the first one that I came across was FIRMS (Fire Information for Resource Management System), FIRMS distributes a near real-time active fire data within three hours of satellite observation. I used a color palette of red, orange, and yellow. Yellow is the biggest fire. It had global coverage, was freely accessible, and had real-time updates. So it was perfect for testing. The first time I saw the mini squares pop up on the map, I was so happy, I had goosebumps and that really wanted me to continue.

I added a lot of layers, with bands mostly focusing on B12, B11, and B9. The different layers would help me understand the different properties, and how to identify a fire. It also helped me understand how to add a layer.

But the three main ones in the program are FIRMS, SWIR composite and a B12-B11 ratio. SWIR standing for short-wave infrared (B12 band of Sentinel-2), can be combined with Near Infrared (B8 band of Sentinel-2) and Red (B4 band of Sentinel-2) to get the`swirLayer` visualization.  This layer is what is creating that map-like layer in the background.

The last layer in the code. The calculated ratio is an indicator for potential fire areas.  This calculated ratio (`b12b11`) is used as an indicator of potential fire activity. Seeing the distribution of values of this ratio, we selected a trheshold value of 1.5 to detected areas where the SWIR-2 to SWIR-1 ratio is a potential active fire burning when the satellite flew overhead.

But it didn’t work perfectly on the first try. I encountered a few major errors:

I saw that there was a fire in a community, on top of a house, so I assumed it was a house fire but it was showing orange and when I zoomed into it, it was showing a rooftop. I searched for fires around the neighborhood in local news, but none were correlated. It showed a gleaming solar panel as fire. That could have multiple reasons for failure. I also had some false detection of fire from the sun glinting from water bodies. So, to confirm if those fires were real, I would check on multiple layers, and my fire detection layer picks up the fires that seem the largest, and will be picked up by almost all of the layers. But FIRMS covers globally and can pick up fires that aren’t picked up by others. For the sun glint problem, using a water mask can mask out the water bodies that were in the Area of Interest!

I truly learned a lot in the process, and I am very thankful to SkyServe for giving me the opportunity. I never knew how much can be achieved with satellite data, especially in solving problems like this. With all this new knowledge, I can’t wait to continue on this satellite imagery/remote sensing journey.

Watch my interview - “From Pixels to Patterns” - with my mentor Adithya from September.

From the Mentor

Adithya Kothandhapani: It was a fun experience working with Bhavishyaa on her project. It is amazing what one can do with satellite data given the tool, imagination and most importantly an infectious passion!

Over the last few days, we have seen another series of events unfold in Punjab where crop stubble is being burnt due post-harvest. I put together a GIF to show how Bhavishyaa’s App helps enable pin-pointing of active fires. The last (mostly green) image is from the high-resolution Google Maps Satellite view - which could in theory be used to even report the address of a farm on fire. If we include a ‘Burn Scar’ detector alongside this visualization, it shows where recently doused fires have left a visible mark on the land. Notice (a) the wispy smoke columns emanating from the active fires detected in yellow, and (b) that some active fires overlap the burn scars :(

The plot above condenses insights from the same area shown in the animation. Intent is to leave you the reader with something to think about. Clue: kharif crops are sown in June & harvested in October (DOY~300); rabi crops are sown in November & harvested in May (DOY~120).

The map problem

Commonly used Navigation Tools (like Google Maps for route plan and GNSS for real-time location) use road network generated from dated Satellite, Mobile and Survey maps. And dated maps used for navigation miss the following dynamic road health parameters: recent road damage, snow cover or mud cover (or landslide), blockages due to repairs, flooded road segments, development of new routes and fog or smog affecting road visibility in a local area. Clearly, it leads to consequences of traffic congestion, accidents, delay in the Average Transit Time for general people, businesses and even emergency medical ambulances.

The loss due to non-availability of maps incorporating dynamic road health is non-trivial. Every driver loses 29 hours a year due to bad maps, which means that just for the United States, a loss of $ 9B (assuming a really conservative 10 mn travellers, 32$/hr hourly wage). Globally, the above number maybe a much, much larger value.

Proposed solution

It seems to make a lot of sense to have a future where dynamic maps (changing features with time) are generated using Onboard Processing of state of the art Road Surface Classification and Segmentation Models on the latest satellite imagery of the user locality. The proposed solution supports the precise localization capability of GNSS with near real-time maps meeting Optimal Route Planning requirements.

Alright! Now, we know that GNSS receiver chips provide precise localization (< 4.5 m) at a high frequency enabling convenient navigation for end users and continuous information about position and velocity to navigate to the desired destination. GNSS-based localization is superior to alternative methods such as inertial navigation as they are prone to drift errors. Also, the location of the user obtained from the receiver enables continuous generation of the optimal route to the destination from the dynamic maps. And onto the onboard processing of satellite imagery to generate the dynamics maps. Optical High Resolution Multispectral Satellite Imagery from satellites like Plaiedes can be used to build state-of-the-art Road Surface Classification and Segmentation Models (Deep Learning models) which can generate road health insights. These small-sized insights can be downlinked to ground servers to update the Road network edge weights based on dynamic Road Surface Classification and Segmentation insights from satellites. This allows us to factor dynamic road health into every Navigation solution to go from point A to B.

A use case for this application for flooded roads is illustrated below:

Dynamic Road Health Maps from Smart Earth Observation can save dozens of hours travel time for 100s of millions of users. This solution does not exist in India and the amount of man-hours saved could significantly contribute to the Indian economy: Assume 30 million workforce (average hourly rate in India INR 96 source) using navigation solution in India saving 20 hours of travel time a year. This translates to saving INR 5760 crores each year. Technology innovation involves building state of the art Road Classification and Segmentation Models for Onboard Processing of Satellite Imagery which is clearly feasible.

Insight generation using remotely sensed satellite imagery is proven to have a short turn around time and low price-point. Adithya et al [1] A road extraction model with an F1 score of 0.947 is presented and tested on varied satellite imagery. V.Yerram et al [2] A road surface classification model with an accuracy of 80% for roads in USA is demonstrated in literature. E.Brewer et al [3] Onboard AI models for cloud detection have been tested by ESA -Giuffrida et al. [4] An inflight accuracy of 96% has been reported. Similar to this application a road surface classification and segmentation model can be deployed onboard.

Market

The Earth Observation Market is Booming: With planned Earth Observing Satellites, the revisit duration over an area will be reduced to between minutes to hours. 8 million Autonomous cars projected to be on the road by 2025. Is it a coincidence that these two sectors will talk to each other more? Guess not.

The demand stems from the complex navigation problems which affect us all and can save at least $ 9B (in US and higher globally) of lost value as highlighted earlier. The supply from the GNSS and Earth Observation market make it a feasible opportunity with Imagery costs projected to go significantly down with the onset of Onboard Processing.

Winding up

With the growing presence of high-resolution optical satellites employing onboard processing, use cases like building dynamic road health maps emerge as practical solutions to get feature-rich, near-realtime maps globally. Road Classification and Segmentation Models to run on multispectral imagery can are shown to have feasibility and will improve in accuracy. With GNSS-based precise localization, navigation can be made much more efficient and realistic than it is today. While it was fun to put forth these ideas, we at SkyServe intend to demonstrate these and similar ideas seriously through technology demonstrators in the coming years.

Want to know more about what we are doing? Click here

References

[1] Kothandhapani, Adithya & Vatsal, Vishesh. (2020). Methods to Leverage Onboard Autonomy in Remote Sensing.

[2] Extraction and Calculation of Roadway Area from Satellite Images Using Improved Deep Learning Model and Post-Processing, V.Yerram et al

[3] Predicting road quality using high resolution satellite imagery: A transfer learning approach, E.Brewer et al

[4] G. Giuffrida et al., "The Φ-Sat-1 Mission: The First On-Board Deep Neural Network Demonstrator for Satellite Earth Observation," in IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-14, 2022, Art no. 5517414, doi: 10.1109/TGRS.2021.3125567.

In order to provide a response to this need, an application programming interface (API)-driven solution for quantifying geospatial risk named GeoRisk is presented for looking at properties spanning a large area of interest. Apart from the risk factors like water availability mentioned before, our algorithms are also detecting property accessibility, road width, administrative level, seismic zone, cyclone hazard, proximity to Archaeological Survey of India monuments, and green zone status of any property in the area of interest. The algorithms are demonstrated to work with known public/open datasets and sample solutions are provided for verification. We are using open-source tools such as Shapely and Django that help us write the required geospatial algorithms which helps us identify and operate between a lot of layers of data. These algorithms take care of doing the spatial operations of overlapping, distance matrix, and all other requirements. A sample API response for a randomly chosen property point is shown below:

Considering the momentum of growth of geospatial technologies in recent times, more data is being made available by the Indian government, organizations and individuals from this sector. As a pre-requisite, we have also worked on generating the required data using the raw datasets from the above-mentioned sources. These datasets are generated by using algorithms and geo-processing tools such as interpolation, change-detection, raster-vector conversions, spatial cropping, land use classification, and precise geospatial intersections. All of which was achieved by using open-source tools. For example, the difference in lake bed water body over a period of two decades can be used to identify areas where encroachment can be checked. Satellite imagery datasets separate from the above-mentioned are also available to be used for further insights being available. The datasets cover Karnataka, Kerala, Andhra Pradesh, Telangana, Tamil Nadu, Maharashtra and Gujarat. The rest of India is planned to be integrated into the GeoRisk API very soon. An interactive interface to test and visualize the solution is also provided for checking the validity of responses over areas of interest. The user interface can be accessed through any browser. The entire functionality is available under the GeoRisk API.

NAVANC’s Nagachethan S M said about the partnership “At Navanc, we are creating India's first comprehensive credit score mechanism to the assets possessed by an Individual and help him/her leverage the same financially. NAVANC uses GeoRisk API as a crucial integration for an asset’s geographic risk along with legal and crime reporting.“

If you find this useful, why not request a demo? Reach out to us at: hello@skyserve.ai

If you’ve read Vishesh’s post on what we are building at SkyServe, your curiosity might have piqued at the mention of ‘insights’. We said that an ‘insight’ is all you need, and that the actual image is only for the ultra-rich wanting to put their acquisition under the hammer at Sotheby’s or Christie’s next auction. Why buy the orchard if all you need is just a pint of apple juice?

To that end, we’ve just launched EarthSIGHTS — a platform to experiment with the language of ‘geospatial insights’. By the book, ‘geospatial Insight’ or its older form ‘geospatial Intelligence’ is information used to describe, assess, and visually depict physical features and geographically referenced activities on Earth. But till today, this information has traditionally been plotted on a map. Given an image, the insights you gain by zooming, panning and clicking on the image are subject to your eyesight, academic qualification, pattern-recognition skills and past experience in looking at pixels and inferring something meaningful.

With EarthSIGHTS, you can study — at no cost — and from your laptop, smartphone or smart TV (yes, that’s a thing) any region through the lenses of land-use, water availability, atmospheric pollution, population and more. You may be a concerned citizen looking to find the source of the thick industrial smoke that is preventing you from going for that prescribed morning jog, or a journalist looking to quantify the loss of green space to convey the crux of your story on [insert concrete jungle of choice].

You can pick a location of interest, choose from a list of insights organized under ‘themes’ and view both spatial as well temporal data. We currently have 26 insight types, under the 3 predefined themes of Agriculture, Climate Change and Risk. We have also included true-color and false-color satellite imagery in case you need to take a step back and appreciate why an insight can speak directly to your brain’s parietal (do the math) and frontal (make the decision) lobes.

Get started

For EarthSIGHTS v1.0, we’ve curated a bunch of locations based on the recommendation of our team in Bengaluru and looked up news stories mentioning these places for which satellite data stitches together a gripping narrative in numbers. We have an area in Punjab on EarthSIGHTS, where stubble burning is apparent throughout the October–November 2021 period, which led to the annual smog-show above the Indian capital. Shanghai — the Chinese financial capital is also featured, around the time COVID-19 hit the world, where one can see the fall (and subsequent recovery) in atmospheric pollutants such as Nitrogen dioxide, Sulfur dioxide and Carbon monoxide.

The data powering EarthSIGHTS is derived from open sources such as the European Copernicus Program, the American NASA and NOAA satellites, the WorldPop research group and the World Resources Institute among others. The data processing has been done using Google’s Earth Engine API for Python, and then rendered for your eyes by our talented front-end team.

This is just v1.0, and we will keep the experiments going and seek your feedback throughout. Notice the little yellow speech bubble on the screen? Use that to let us know your views, issues and suggestions. We are continually improving the experience for users and have a ton of new features lined up for EarthSIGHTS v2.0. What else would you like to see on EarthSIGHTS?

Open Problems 🤔

SkyServe: What are the open problems in onboard processing for remote sensing?

ChatGPT: There are several open problems in the field of onboard processing for remote sensing. Some of these include:

• Developing more efficient algorithms for onboard processing, to reduce the amount of data that needs to be transmitted to the ground.

• Developing new hardware and architectures for onboard processing, to improve the speed and accuracy of the processing.

• Developing methods for dealing with the challenges of real-time processing, such as data latency and limited resources.

• Developing robust error correction and fault tolerance mechanisms to ensure the reliability of onboard processing.

• Developing methods for integrating onboard processing with other sensors and systems, such as cameras and lidar, to improve the accuracy and resolution of the data.

SkyServe: Can you share some more examples?

ChatGPT: Sure, here are a few more examples of open problems in the field of onboard processing for remote sensing:

• Developing algorithms for onboard processing of multispectral and hyperspectral data, to extract more information from the data and improve the accuracy of the results.

• Developing methods for onboard data fusion, to combine data from multiple sensors and improve the accuracy and resolution of the results.

• Developing methods for real-time anomaly detection and classification, to identify and classify objects in the data in real time.

• Developing methods for onboard image registration and stitching, to combine multiple images into a single, high-resolution image.

• Developing methods for onboard feature extraction and recognition, to identify and extract important features from the data in real time.

Background 📖

SkyServe: Why didn't companies build onboard processing for remote sensing earlier?

There are several reasons why companies may not have developed onboard processing for remote sensing earlier. Some of the factors that may have contributed to this include:

• Technology limitations: In the past, the technology available for onboard processing may not have been advanced enough to support the requirements of remote sensing applications. As technology has advanced, it has become possible to develop more powerful and sophisticated onboard processing systems.

• Cost: Developing onboard processing systems for remote sensing can be expensive, and may not have been feasible for companies in the past due to financial constraints. As technology has become more advanced and costs have decreased, it has become more feasible for companies to invest in the development of onboard processing systems.

• Market demand: The demand for remote sensing products and services may not have been as strong in the past as it is today. As the market for remote sensing has grown and the demand for high-quality data has increased, companies have had more incentive to invest in the development of onboard processing systems.

• Regulation: There may have been regulatory barriers or other challenges that made it difficult for companies to develop and deploy onboard processing systems for remote sensing earlier. As regulatory frameworks have evolved, it has become easier for companies to develop and deploy these systems.

Business ideas💡

SkyServe: Can you give some business ideas for use of AI in remote sensing for agriculture?

There are many potential business ideas for the use of artificial intelligence (AI) in remote sensing for agriculture. Some possible ideas include:

• Precision agriculture: AI algorithms could be used to analyze images and data collected by remote sensing platforms, such as satellites or drones, to provide insights and recommendations for precision agriculture practices. This could include identifying optimal planting locations, detecting pests or diseases, or estimating crop yields.

• Irrigation management: AI algorithms could be used to analyze remote sensing data to optimize irrigation practices, such as identifying areas where irrigation is needed and determining the optimal amount of water to apply.

• Crop monitoring: AI algorithms could be used to monitor crops using remote sensing data, providing early warning of potential problems such as pests or diseases, and helping farmers take proactive action to address these issues.

• Weather forecasting: AI algorithms could be used to create more accurate weather forecasts for agriculture, using data from remote sensing platforms and other sources. This information could be used to optimize farming practices and reduce risks.

• Supply chain management: AI algorithms could be used to optimize the supply chain for agricultural products, using data from remote sensing platforms and other sources to forecast demand and optimize production, storage, and distribution.

Are you a geospatial analytics company? 💯

SkyServe: How can geospatial analytics companies utilize onboard processing for remote sensing?

Geospatial analytics companies can utilize onboard processing for remote sensing in several ways. Some possible options include:

• Improved data quality: Onboard processing systems can be used to improve the quality of remote sensing data by enabling real-time processing and analysis of the data. This can help to identify and correct errors or anomalies in the data, resulting in more accurate and reliable insights.

• Enhanced capabilities: Onboard processing systems can enable geospatial analytics companies to perform more advanced and sophisticated analyses of remote sensing data, such as machine learning or predictive modeling. This can help to uncover new insights and opportunities that might not have been possible with traditional processing methods.

• Increased efficiency: Onboard processing systems can help to streamline the processing and analysis of remote sensing data, enabling geospatial analytics companies to more quickly and efficiently deliver products and services to their clients.

• Reduced costs: Onboard processing systems can help to reduce the costs associated with processing and analyzing remote sensing data, by eliminating the need for expensive on-ground processing infrastructure and enabling data to be processed closer to the source.

• Improved performance: Onboard processing systems can help to improve the performance of remote sensing platforms, by enabling real-time processing and analysis of data and enabling the platform to respond more quickly to changing conditions.

Phew, this is pretty well summarized!

Want to know more about what we are doing? Click here

SatRevolution will fly SkyServe’s computer onboard its STORK satellite scheduled for launch in Q2 2023. SkyServe’s computer is designed to deliver Earth Observation insights with low latency, eliminating computation and cloud resources on the ground for organizations. The upcoming STORK mission will carry Vision 300, an optical payload made by SatRevolution that can capture images at 5-m GSD resolution. The imagery is fetched by SkyServe’s computer for further processing using custom analytics models to derive insights in real-time. The mission will pave the way for geospatial companies to deploy their applications on this analysis-ready data platform, onboard STORK. SkyServe and SatRevolution have come together owing to clear synergies towards making space accessible for all and making EO data ‘actionable’ for users.

The planned number of flights, targeted revisit, diversity of optical sensors planned and the rapid turnaround time between contract and flight was another key reason for the decision to partner with SatRevolution.

Hosted Payload Mission service provided by SatRevolution is an excellent opportunity to establish interesting business relations across the world and bring new value to the market. The prospect of leveraging SkyServe platform for on-orbit processing of earth observation data is extremely exciting for SatRevolution.

“When building a satellite constellation, edge computing technology provided by SkyServe gives a chance for better allocation of resources and allows for achieving the final product more efficiently”, says Grzegorz Zwoliński, CEO of SatRevolution.

“‘Space for all’ is a key driver to solving many of the challenges we have on Earth. We are happy to come together with SatRevolution as a partner in building affordable and novel EO solutions”, says Vinay Simha, CEO of SkyServe.

About SatRevolution

SatRevolution, founded in 2016, is a Polish space tech company providing complete end-to-end solutions for nanosatellite systems focusing on Earth observations. The main goal of SatRevolution is to enable easy access to space and Earth Observation data by deploying a Real-Time Earth Observation constellation consisting of 1500 satellites capable to deliver high-resolution images with a revisit time of around 30 minutes from every place on Earth. The first company achievement is the successful deployment of two satellites (Światowid and KRAKsat), launched to the International Space Station and deployed into orbit in July 2019. In 2020, SatRevolution’s UniBus platform was approved by NASA as TRL 9 CubeSat platform in the “State of the Art Small Spacecraft Technology” Report. SatRevolution wants to change the way of building the Earth Observation business by focusing on small, lightweight, and reliable spacecraft.

For more information please visit satrev.space

About SkyServe

SkyServe is building autonomous solutions to democratize Earth Observation data for organizations, bringing down the cost of data processing and latency by an order of magnitude. SkyServe’s SaaS offering allows developers of geospatial solutions to deploy their apps on analysis-ready data on satellites. They are working with multiple Earth Observation businesses, enabling them with insights for remote operations and monitoring with a significant reduction in turnaround time and decision-making.

For more information please visit skyserve.ai