02 / Energy systems / RL / EVM Team project / 2025

Autonomous Microgrid Economy & Continuous RL Control

Gridium Protocol

A distributed DePIN microgrid architecture pairing a 15-node Ohm's law physics simulation, continuous-control PyTorch DDPG actor-critic agent, high-frequency 500ms Socket.io gateway, constant-product EVM energy AMM, and off-chain Groth16 zero-knowledge proof verification.

15 NodesOhm's law prosumer network
DDPGContinuous actor-critic control
500 msWebSocket gateway tick rate
Groth16zk-SNARK surplus verification
The engineering thesis

Solve the decentralized energy trilemma by replacing rigid feed-in tariffs with autonomous reinforcement learning and zero-knowledge cryptographic settlement.

Traditional electrical distribution networks are brittle and centralized. As residential solar arrays and home battery systems proliferate, uncoordinated peer-to-peer (P2P) trading introduces physical grid instability (such as synchronized trading triggering localized brownouts) and severe privacy vulnerabilities (public blockchain ledgers exposing private occupancy patterns based on power draw).

Gridium addresses this by orchestrating a closed feedback loop: a custom Python physics environment simulates real-world solar generation curves, residential duck-curve load profiles, and battery state of charge (SoC). A Deep Deterministic Policy Gradient (DDPG) agent continuously steers an Automated Market Maker (AMM) via dynamic swap fees, while Groth16 zk-SNARK circuits mathematically prove surplus energy off-chain without broadcasting raw sensor telemetry.

My architectural contributionPython AI physics engine, backend integration, and React Three Fiber 3D command center

I architected the FastAPI physics simulation loop (AegisEnv), engineered the continuous PyTorch DDPG actor-critic pipeline, integrated the Node.js Socket.io state gateway, and built the hardware-accelerated 3D microgrid topology in React Three Fiber. Yash Pandit spearheaded Solidity smart contracts and Circom zk-SNARK circuits; Sanket Deka engineered the frontend dashboard UI and telemetry charts.

Runtime topology

A synchronized four-runtime control loop

Each runtime is decoupled behind a narrow, fault-tolerant interface. That separation ensures high-frequency physics simulation, browser rendering, proof generation, and EVM contract execution operate without blocking each other.

PY
AI Engine
FastAPI, Gym environment, PyTorch DDPG
→
WS
Gateway
Node.js, REST polling, Socket.io fan-out
→
3D
Command Center
React 18, Zustand, React Three Fiber
→
EVM
Settlement
Solidity AMM, access roles, Groth16 verifier
Physics Simulation

15-node Ohm's Law Microgrid

Models synthetic photovoltaic irradiance curves, residential duck-curve load, and battery State of Charge (SoC bounded between 20% and 80%) across 15 interconnected prosumer nodes.

Continuous RL Control

PyTorch DDPG Actor-Critic

Maps a 7-dimensional observation vector into a continuous swap fee. Experience replay memory and Polyak soft target updates (τ = 0.005) prevent market fee whiplash.

Automated Market Maker

Constant-Product AMM

Energy tokens and USDC reserves adhere to the x × y = k invariant. Non-reentrant swap methods, slippage boundaries, and dynamic fee accrual protect reserve liquidity.

Zero-Knowledge Privacy

Groth16 Surplus Circuit

Compiled via Circom 2.0; proves amount_to_sell = total_solar - total_load off-chain via snarkjs, submitting proofs to SurplusVerifier.sol without revealing raw consumption.

Controller design

Continuous policy gradients for microgrid stability

Observation Vector (7D)[total_load, total_generation, net_imbalance, energy_reserve, usdc_reserve, spot_price, mean_battery_soc]
Continuous Action (1D)swap fee ∈ [0.10%, 5.00%]
Composite Reward FunctionR = -α(imbalance²) - β(fee_volatility) + γ(reserve_depth)
01

Unified physical and market observation

At every 500ms physics tick, AegisEnv normalizes power telemetry into an observation vector St. Market pricing is never analyzed in isolation from physical battery reserves and net generation deficit.

02

Continuous floating-point fee control

Discrete action algorithms (like DQN) cause severe fee oscillation. The DDPG actor outputs a continuous value mapped smoothly via sigmoid scaling to a stable fee between 10 and 500 basis points.

actor(S_t) → μ(S_t) + OU_noise → 0.001 + normalized_action × 0.049
03

Actor-Critic learning & Polyak updates

The Critic network minimizes Mean Squared Bellman Error. Replay buffers decouple temporal correlation, while slow target network tracking (τ = 0.005) guarantees training convergence under volatile power spikes.

04

High-frequency WebSocket fan-out

The Node gateway polls the FastAPI engine /tick endpoint every 500ms, immediately broadcasting serialized telemetry across Socket.io to synchronize Zustand stores and React Three Fiber rendering loops.

Product surfaces

Dual-perspective operator command center

Settlement boundary

Smart contract enforcement & zk-SNARK verification

Solidity AMM

Automated Market Maker (AegisAMM.sol)

Enforces the constant-product invariant ($x \cdot y = k$) for instant liquidity. Features strict slippage protection parameters, emergency pause functionality, and OpenZeppelin ReentrancyGuard.

Controller hook

Role-based RL fee execution

Only the verified RL_OPERATOR_ROLE can push updated swap fees to the smart contract, with on-chain clamping between 0.10% and 5.00% to prevent rogue actor exploitation.

Cryptographic Circuit

Groth16 Zero-Knowledge Proofs

Circom circuit verifies that private solar generation exceeds private home consumption without revealing household telemetry, generating a Groth16 proof validated by SurplusVerifier.sol.

Scientific Boundary

The circuit mathematically validates the arithmetic surplus balance ($E_{surplus} = E_{gen} - E_{load}$). Full industrial deployment would require hardware secure enclaves (TEE) or cryptographic smart meters to attest to physical sensor authenticity.

Implementation map

Subsystem architecture & source code mapping

SubsystemSource PathPrimary ResponsibilityTechnical Implementation Detail
Physics Engineai-engine/aegis_env.pyGrid state, battery dynamics, load simulationCustom Gymnasium environment, Ohm's law impedance, duck curve
DDPG Controllerai-engine/ddpg_agent.pyContinuous actor-critic policy optimizationPyTorch, Ornstein-Uhlenbeck noise, Polyak target updates (τ=0.005)
Realtime Serverai-engine/main.pyState serialization & chaos injectionFastAPI 500ms tick loop, REST endpoints, disturbance triggers
Gateway Bridgebackend/index.jsTelemetry fan-out & proof routingExpress, Socket.io, snarkjs proof generation, event replay buffers
AMM Contractcontracts/AegisAMM.solToken swaps & liquidity pool custodySolidity 0.8.20, constant-product invariant, AccessControl, ReentrancyGuard
ZK Circuitcircuits/energy_proof.circomPrivate surplus arithmetic relationCircom 2.0, Groth16, R1CS constraint generation, snarkjs verification
3D Command Centerfrontend/src/components/MicrogridCanvas.tsxSpatial topology & flow visualizationReact 18, React Three Fiber, GLSL shaders, Zustand reactive store
Engineering judgment

Integrated prototype achievements & research boundaries

Implemented

End-to-end DePIN microgrid prototype

  • Closed-loop physics simulation with continuous reinforcement learning
  • 500ms realtime Socket.io gateway and hardware-accelerated 3D command center
  • Constant-product Solidity AMM with role-bounded fee updates
  • Off-chain Groth16 zero-knowledge proof generation and verification
  • Chaos injection engine for demonstrating resilience under power spikes
Prototype boundary

Deliberately bounded research scope

  • Physical battery chemistry degradation is modeled parametrically rather than electrochemically
  • DDPG performance has not yet been benchmarked against Model Predictive Control (MPC) baselines
  • Hardware meter attestation relies on cryptographic simulation rather than physical TEE smart meters
  • Evaluation under multi-agent adversarial bidding rings remains ongoing work
Next engineering milestone

Deploy a deterministic comparative benchmark pitting DDPG against traditional PID and Model Predictive Control (MPC) across standard IEEE 33-bus test feeder topologies.