rPPG Models for Smart Glasses: Building Wearable Camera Vitals

Smart glasses are evolving from display-only devices into multi-sensor platforms capable of health and wellness monitoring. The small, forward-facing or inward-facing cameras embedded in these devices present a unique opportunity for physiological sensing -- and a unique engineering challenge. Building an rPPG model smart glasses wearable camera hardware can actually use requires rethinking nearly every assumption from conventional desktop or automotive rPPG. The imaging geometry, the available facial region, the motion profile, the power budget, and the compute constraints are all fundamentally different from any other rPPG deployment context.

"The periocular region is one of the richest vascular territories on the face. The signal is there. The challenge is building a model that can find it from two centimeters away, on a processor that draws less than a watt." -- Adapted from Hernandez et al., ACM UbiComp 2022

This post examines the technical foundations of rPPG for smart glasses, the unique constraints of head-worn wearable cameras, and the model-building pipeline required to deliver physiological sensing on this emerging platform.

Analysis: Why Smart Glasses Demand a Different rPPG Approach

Conventional rPPG systems image the full face from a distance of 30-100 cm using a standard camera sensor. Smart glasses fundamentally change this geometry: the camera is mounted on the frame, millimeters to centimeters from the skin, imaging only a small periocular or temporal region. This single architectural difference cascades through every stage of the rPPG pipeline.

Periocular Signal Extraction

The periocular region -- the skin surrounding the eye -- is densely vascularized by branches of the superficial temporal artery and the supraorbital artery. Published research confirms that this region produces a strong blood-volume pulse signal detectable by close-range cameras (Hernandez et al., ACM UbiComp 2022). However, the spatial characteristics differ from full-face rPPG: the ROI is small, the skin surface is curved and prone to micro-wrinkling, and the signal is influenced by blink-related eyelid motion and periorbital muscle contractions.

Ultra-Close Imaging Geometry

At camera-to-skin distances of 1-3 cm, depth-of-field is extremely shallow. Small head movements cause the skin surface to move in and out of focus, creating apparent intensity changes unrelated to hemodynamic fluctuations. The camera's field of view captures a very small skin patch, reducing the spatial averaging that helps suppress noise in full-face rPPG. Custom models for this geometry must learn to distinguish hemodynamic intensity fluctuations from focus-induced artifacts -- a challenge absent from any public rPPG dataset.

Power and Compute Constraints

Smart glasses operate on batteries measured in hundreds of milliamp-hours. The rPPG inference pipeline must share compute with display rendering, audio processing, head tracking, and wireless communication. Typical compute budgets for the physiological sensing workload range from 10-50 MOPS on ultra-low-power processors (ARM Cortex-M55, Ambiq Apollo4, or custom DSP cores). This is orders of magnitude less than the GPU compute available in automotive or desktop deployments, demanding aggressively optimized model architectures.

Motion Profile

Head-worn devices experience continuous, complex motion: walking gait, head turns, conversational gestures, and device repositioning on the nose bridge. The motion profile is fundamentally different from both the seated desktop scenario and the vibration-dominated automotive scenario. Custom models must be trained on data that captures the specific motion characteristics of smart-glasses wear.

Comparison: Smart Glasses rPPG vs. Other Deployment Contexts

Dimension	Desktop/Webcam rPPG	Automotive DMS rPPG	Smart Glasses rPPG
Facial ROI	Full face	Full face (driver)	Periocular only (1-2 cm patch)
Camera-to-skin distance	30-80 cm	60-120 cm	1-3 cm
Imaging channel	RGB (3-channel)	NIR single-channel (940 nm)	RGB or NIR (1-3 channel, sensor-dependent)
Depth of field	Large (full face in focus)	Large	Very shallow (focus artifacts)
Dominant motion	Voluntary head movement	Road vibration + maneuvers	Walking gait + head turns
Compute budget	GPU (TFLOPS)	Embedded SoC (INT8, ~10 TOPS)	Ultra-low-power MCU (10-50 MOPS)
Power budget	Unlimited (mains)	Vehicle power (~10-30W for DMS SoC)	Battery (~50-200 mW for sensing)
Inference model size	5-50 MB (unconstrained)	1-10 MB (quantized)	50-500 KB (heavily compressed)
Continuous operation	Minutes (session-based)	Hours (drive duration)	All-day (8-16 hours)
Public training data	Extensive	Very limited	None

The absence of public training data for periocular close-range rPPG is the most critical gap. No public dataset captures the imaging geometry, ROI characteristics, or motion profile of a head-worn device. Custom data collection and model training are non-negotiable for this platform.

Applications: Physiological Sensing on Smart Glasses

Continuous Wellness Monitoring

Smart glasses uniquely enable passive, continuous heart rate and HRV monitoring throughout the day without requiring the user to interact with the device or wear additional sensors. The periocular camera captures the physiological signal as a background process. This longitudinal data stream enables trend analysis -- resting heart rate patterns, HRV trajectories, and stress-response profiles over days and weeks -- that single-measurement devices cannot provide.

Cognitive State and Attention Assessment

Periocular rPPG combined with eye-tracking data (available on the same sensor in many smart-glasses architectures) enables multi-modal cognitive state assessment. Pupil dynamics, blink patterns, and gaze behavior provide behavioral indicators of attention and cognitive load, while rPPG-derived HRV provides a physiological indicator. Fusing these signals produces a richer attention metric than either modality alone.

Occupational Health and Safety

Workers in industrial, logistics, and field-service environments are increasingly equipped with smart glasses for augmented reality task guidance. Adding physiological monitoring via rPPG enables fatigue and heat-stress detection without additional wearable devices. The model must be robust to the specific motion profiles and environmental conditions of the target occupation -- warehouse walking, outdoor heat exposure, or cleanroom protocols.

Sports and Fitness

Head-worn cameras in sports eyewear or cycling glasses can extract pulse rate during physical activity, complementing or replacing chest-strap heart rate monitors. The motion artifact challenge is extreme in this context (high-amplitude, high-frequency head motion during running or cycling), demanding custom models trained on sport-specific motion data with high-quality reference signals.

Research Foundations

The emerging research base for periocular and wearable-camera rPPG includes:

• Hernandez et al., ACM UbiComp 2022 -- Demonstrated pulse-rate extraction from periocular video captured by a near-eye camera at 1-2 cm distance. Showed that the periocular ROI produces sufficient signal for heart rate estimation when the model is specifically trained on close-range periocular data. Models trained on full-face desktop data failed completely on periocular input.
• Poh et al., IEEE TBME 2011 -- While focused on webcam rPPG, established the ICA-based signal decomposition framework that remains foundational for separating hemodynamic signals from motion artifacts in constrained-ROI applications, including periocular sensing.
• Wang et al., IEEE TBIOM 2023 -- Documented cross-sensor generalization failures in rPPG, with implications for wearable camera deployments where the sensor module differs from public-dataset cameras in every relevant parameter.
• Bousefsaf et al., Biomedical Signal Processing and Control 2023 -- Investigated rPPG under physical activity and head motion, finding that motion-robust models require training data that explicitly includes the target motion profile. Architectures with temporal attention mechanisms showed the strongest motion resilience.
• Liu et al., IEEE TMM 2024 -- Proposed lightweight rPPG architectures suitable for edge deployment, demonstrating that models under 500 KB can achieve competitive pulse-rate estimation when trained on domain-matched data. Their MobilePhys architecture targeted mobile and wearable compute constraints.

Future Directions

On-device self-supervised adaptation. Smart glasses have a unique advantage for self-supervised rPPG learning: the same user wears the device every day, providing hours of consistent periocular video. Self-supervised methods that learn the user's individual pulse signal characteristics from unlabeled data could enable personalized model adaptation without explicit ground-truth collection.

Multi-sensor fusion with IMU. Smart glasses include inertial measurement units (IMU) for head tracking. The IMU signal directly captures the rigid-body motion component of the glasses, enabling motion artifact cancellation in the rPPG pipeline. Custom models that jointly process camera and IMU inputs can disentangle motion-induced and hemodynamic-induced intensity changes more effectively than camera-only approaches.

Spectral optimization for periocular sensing. The periocular vascular anatomy suggests that specific wavelengths may provide superior signal contrast in this region compared to the full face. Custom LED illumination at wavelengths matched to the absorption spectrum of the superficial temporal artery branches could enhance the periocular rPPG signal. This represents a hardware-software co-design opportunity unique to wearable form factors.

Battery-aware inference scheduling. Rather than running rPPG inference continuously, adaptive scheduling could trigger physiological measurement during periods of low motion (desk work, sitting) and pause during high-motion periods (running, active gesturing) where signal quality is lowest and power consumption is wasted. The scheduling logic itself becomes part of the custom model deployment.

Thermal and moisture robustness. Smart glasses in prolonged wear accumulate skin moisture (perspiration) in the periocular region, which can alter skin reflectance and potentially interfere with the rPPG signal. Custom models trained with data spanning dry and moist skin conditions will be necessary for all-day reliability.

FAQ

Is the periocular region sufficient for reliable rPPG?

Yes. The periocular region is richly supplied by the superficial temporal artery and supraorbital artery branches, producing a strong pulsatile signal. Hernandez et al. (ACM UbiComp 2022) demonstrated reliable pulse-rate extraction from periocular video at close range. The key requirement is a model trained specifically on periocular close-range data, as full-face models do not transfer to this ROI.

What frame rate do smart-glasses cameras need for rPPG?

A minimum of 20 fps is required for basic pulse-rate estimation. Higher frame rates (30+ fps) improve waveform quality for HRV analysis. Many smart-glasses camera modules already support 30 fps for eye-tracking purposes, making rPPG feasible without hardware changes. Some architectures time-multiplex the camera between eye tracking and periocular skin imaging.

How small can the rPPG model be for smart-glasses deployment?

Published lightweight architectures demonstrate competitive pulse-rate estimation at model sizes of 200-500 KB (Liu et al., IEEE TMM 2024). With aggressive quantization (INT8 or mixed INT4/INT8), domain-specific architecture pruning, and periocular-only ROI (which reduces spatial input dimensions), sub-200 KB models are achievable for heart rate estimation. HRV-grade waveform extraction requires somewhat larger models.

Does the rPPG model need the same camera used for eye tracking?

Not necessarily. Some smart-glasses designs include separate inward-facing cameras: one optimized for pupil tracking (typically NIR with narrow-band illumination) and one for periocular skin imaging (broader spectral sensitivity). Others time-share a single camera. The rPPG model must be trained on the specific camera and imaging mode it will use in production, regardless of the eye-tracking architecture.

How does ambient light affect periocular rPPG on smart glasses?

The glasses frame partially shields the periocular region from ambient light, creating a semi-controlled illumination environment. Designs with active LED illumination (small NIR or green LEDs mounted on the inner frame) can dominate ambient light and provide consistent signal conditions. Custom models trained with the specific active illumination configuration of the target device perform significantly better than models relying on uncontrolled ambient light.

---

Smart glasses represent a new frontier for rPPG -- one where every parameter from imaging geometry to compute budget differs from prior deployments. If your team is building physiological sensing into a head-worn wearable and needs an rPPG model engineered for your periocular camera and ultra-low-power compute platform, start a custom-build conversation with the Circadify engineering team.