Continuous on-device inference on IoT gateways shifts computation from cloud centers to the network edge, creating distinct energy trade-offs that affect device lifetime, operational cost, and emissions. Research into machine learning energy use highlights that inference workloads, while typically lighter than training, become energetically significant when executed continuously or across fleets. Emma Strubell at the University of Massachusetts Amherst and colleagues have documented the climate and energy impacts of machine learning workloads, underscoring that sustained inference at scale can accumulate nontrivial carbon footprints. Jonathan Koomey at Stanford University has similarly emphasized the broader electricity and emissions implications of computing growth, which extend to edge deployments as they proliferate.
Energy pathways and causes
Several factors drive the energy implications of continuous inference. Model complexity and size directly increase computation, memory access, and the need for specialized accelerators; Norman Jouppi at Google showed how hardware design dramatically alters energy per operation, indicating that naive CPU-based inference on gateways can be far less efficient than purpose-built accelerators. Network behavior matters as well: continuous local inference reduces upstream data transfer but may increase local energy draw if models are not optimized. Workload patterns, ambient temperature, and duty cycles also modulate real-world power use in gateway hardware. Flavio Bonomi at Cisco Systems and Rajkumar Buyya at the University of Melbourne have framed fog and edge computing as ways to balance latency and resource constraints, but they also note the energy implications of adding compute to distributed nodes.
Consequences and mitigations
Consequences range from shortened battery life and higher operational costs in remote or off-grid deployments to larger societal impacts where electricity is carbon intensive. In territories with constrained maintenance infrastructure, frequent battery replacements create environmental and logistical burdens; culturally, this can disproportionately affect rural or low-income communities that host many IoT installations. Mitigation strategies have empirical support: Song Han at the Massachusetts Institute of Technology developed model compression and pruning techniques that reduce inference cost, and design work by Norman Jouppi at Google points to accelerators and quantization as effective energy reducers. Operational tactics such as adaptive sampling, duty cycling, and hybrid offload to cloudonly for complex tasks balance accuracy and power use. Evaluating gateway deployments through lifecycle energy and emissions metrics, and aligning optimization with local grid carbon intensity, yields better environmental outcomes while preserving the latency and privacy benefits that drive edge inference adoption.