IEEE MASS 2025

DailyLLM: Context-Aware Activity Log Generation Using Multi-Modal Sensors and LLMs

Ye TianXiaoyuan RenZihao WangOnat GungorXiaofan YuTajana Rosing

University of California San Diego · Computer Science and Engineering

Paper Dataset Code
DailyLLM illustration showing multimodal smartphone and smartwatch sensing used to generate context-aware activity logs, summaries, and reminders
DailyLLM turns multimodal mobile and wearable sensor context into detailed activity logs and higher-level summaries.

Abstract

Rich and context-aware activity logs facilitate user behavior analysis and health monitoring, making them a key research focus in ubiquitous computing. The remarkable semantic understanding and generation capabilities of Large Language Models (LLMs) have recently created new opportunities for activity log generation. However, existing methods continue to exhibit notable limitations in terms of accuracy, efficiency, and semantic richness.

To address these challenges, we propose DailyLLM. To the best of our knowledge, this is the first log generation and summarization system that comprehensively integrates contextual activity information across four dimensions: location, motion, environment, and physiology, using only sensors commonly available on smartphones and smartwatches.

To achieve this, DailyLLM introduces a lightweight LLM-based framework that integrates structured prompting with efficient feature extraction to enable high-level activity understanding. Extensive experiments demonstrate that DailyLLM outperforms state-of-the-art (SOTA) log generation methods and can be efficiently deployed on personal computers and Raspberry Pi. Utilizing only a 1.5B-parameter LLM model, DailyLLM achieves a 17% improvement in log generation BERTScore precision compared to the 70B-parameter SOTA baseline, while delivering nearly 10 times faster inference speed.

From Sensor Context to an Activity Log

DailyLLM combines location, motion, environmental, and physiological context into a coherent timeline, following the user from a coffee shop and library to a park and back home.

Input

Sensor observations

Location
Daniel's Coffee Shop → library → nearby park → home
Motion
Sitting for 150 minutes → walking for about 20 minutes → lying down
Environment
Scene and ambient context from microphone, light, and temperature sensing
Physiology
Heart rate 55 bpm · EDA 0.5 μS · body temperature 35.1°C

System

DailyLLM

Extracted features + structured prompts + lightweight LLM

Output

Generated activity summary

“You left Daniel's Coffee Shop, sat in the library for 150 minutes, then walked to a nearby park and took a walk for about 20 minutes, came home, and lay down in your bedroom.”

Key Contributions

Context-rich activity understanding

To the best of our knowledge, DailyLLM is the first sensor-based system designed to automatically generate activity logs and summaries across four key dimensions: location, motion, environment, and physiology.

Constructed multimodal dataset

The project releases JSONL data for activity prediction, scene understanding, location description, and activity-log generation and summarization, supporting classification and natural-language generation research.

Lightweight and edge-ready design

Efficient feature extraction, structured prompting, and LoRA fine-tuning enable DailyLLM's 1.5B model to achieve 17% higher activity-log generation BERTScore precision than a SOTA approach using LLaMA3-70B, while delivering nearly 10× faster inference. The system is also evaluated on Raspberry Pi 5.

Released research resource

The DailyLLM Dataset

We construct and publicly release a multimodal dataset that connects sensor interpretation with context-aware language generation. It is organized as JSONL dialogues for four complementary task groups.

Task 01

Activity Prediction

Smartphone IMU features support activity classification across four public HAR benchmarks.

Task 02

Scene Understanding

Acoustic features support recognition across 15 everyday environmental scenes.

Task 03

Location Description

GPS and Wi-Fi context supports structured natural-language location descriptions.

Task 04

Logs and Summaries

Combined multimodal context supports activity-log generation and longer-term summarization.

View the DailyLLM Dataset on Hugging Face

System Overview

DailyLLM pipeline: smartphone and smartwatch sensors for spatial, motion, environmental, and physiological context are formatted and semantically processed; compact features and structured prompts feed a locally fine-tuned LLM that produces activity logs and multi-hour summaries.
DailyLLM architecture for multimodal data processing, context-aware understanding, and activity-log generation and summarization.

DailyLLM first aligns heterogeneous smartphone and wearable sensor streams and converts them into compact semantic features representing location, motion, environment, and physiology. Structured prompts combine those features with task and output guidance, enabling the locally fine-tuned LLM to generate fine-grained activity logs and higher-level summaries.

  1. 1Sensor data processingAlign streams, localize, and annotate context.
  2. 2Feature extractionCompress spatial, motion, scene, and physiological signals.
  3. 3Context understandingBuild structured prompts for the fine-tuned LLM.
  4. 4Logs and summariesGenerate activity records and periodic summaries.

Key Results

Compared with SOTA methods, DailyLLM achieves 17% higher activity-log generation BERTScore precision while delivering nearly 10× faster inference. On a resource-constrained Raspberry Pi 5, it can summarize two hours of activity data within four minutes. Together, the experiments demonstrate the practical effectiveness of DailyLLM for real-world activity logging.

Three grouped bar charts compare DailyLLM with SVM, KNN, CNN, and LSTM across activity and scene datasets using precision, recall, and F1 score.
Sensor-understanding performance across activity-recognition and acoustic-scene datasets.
Two radar charts compare activity-summary accuracy, coverage, conciseness, compliance, and clarity across model configurations and summary-window lengths.
Activity-summary quality across model configurations and time windows.

Average inference time per sample

DeviceActivity predictionScene understandingLog generationSummarization
NVIDIA RTX 4090
24 GB		0.08 s0.13 s2.22 s1.63 s
Raspberry Pi 5
ARM Cortex-A76, 8 GB		37 s90 s180 s240 s

Frequently Asked Questions

What is DailyLLM?

DailyLLM is a system for generating and summarizing context-aware activity logs from multimodal sensors commonly available on smartphones and smartwatches.

What sensors does DailyLLM use?

The paper organizes mobile and wearable sensing into spatial sensors such as GPS, barometer, Wi-Fi, and Bluetooth; motion sensors such as accelerometers, gyroscopes, and magnetometers; environmental sensors such as microphones, temperature, and light; and physiological sensors such as GSR, PPG, thermistors, and infrared sensors.

What does the DailyLLM Dataset contain?

The released JSONL dataset covers four task groups: activity prediction from smartphone IMU features, acoustic scene understanding from audio features, natural-language location description, and activity-log generation and summarization. It combines derived data from public benchmarks including UCI, HHAR, MotionSense, Shoaib, DCASE, and StudentLife.

What contextual information does DailyLLM capture?

DailyLLM captures four complementary context dimensions: location, motion, environment, and physiology.

Why use an LLM for activity logging?

DailyLLM uses an LLM to interpret compact, structured sensor features and turn them into coherent activity descriptions and higher-level summaries rather than isolated activity labels.

How large is the DailyLLM model?

The main DailyLLM configuration uses a 1.5B-parameter DeepSeek-R1-Distill-Qwen model with LoRA fine-tuning.

Can DailyLLM run on edge devices?

Yes. The paper evaluates DailyLLM on a Raspberry Pi 5 with 8 GB of memory and reports a four-minute runtime for summarizing a two-hour activity-log window.

How does DailyLLM compare with larger LLM baselines?

Against the reported SOTA log-generation approach using LLaMA3-70B, the 1.5B DailyLLM configuration reports BERTScore precision of 81.65% versus 65.0% and average PC log-generation latency of 2.22 seconds versus 22.09 seconds.

Citation

Cite the official IEEE MASS 2025 publication.

@inproceedings{tian2025dailyllm,
  title     = {{DailyLLM}: Context-Aware Activity Log Generation Using Multi-Modal Sensors and {LLM}s},
  author    = {Tian, Ye and Ren, Xiaoyuan and Wang, Zihao and Gungor, Onat and Yu, Xiaofan and Rosing, Tajana},
  booktitle = {2025 IEEE 22nd International Conference on Mobile Ad-Hoc and Smart Systems (MASS)},
  pages     = {372--380},
  year      = {2025},
  publisher = {IEEE},
  doi       = {10.1109/MASS66014.2025.00060},
  url       = {https://doi.org/10.1109/MASS66014.2025.00060}
}