team-3/local_run/SENSOR_README.md

NOI Sensor Management System

This system allows you to manage environmental sensors from the NOI Open Data Hub in c3nav, displaying them as overlay features on different levels/floors.

Overview

The system supports:

• Multiple sensors on the same overlay but on different levels
• Dynamic addition of new sensors through Django management commands
• Automatic data scraping from NOI Open Data Hub APIs
• Real-time display of CO2, temperature, humidity and other environmental data

Architecture

• Single Overlay: All NOI environmental sensors are managed under one DataOverlay
• Multiple Levels: Sensors can be placed on different floors (floor0, floor1, etc.)
• Flexible Configuration: Sensor locations and properties are configurable via the overlay's sensor_config field
• Dynamic Discovery: The system can automatically discover and display any sensor data from the NOI API

Setup

The main setup is handled by the up.sh script, which:

1. Creates a single "NOI Environmental Sensors" overlay
2. Configures initial sensors with their coordinates and levels
3. Scrapes initial data from the NOI Open Data Hub
4. Applies necessary database migrations

Managing Sensors

1. List All Sensors

# Using the helper script
./manage_noi_sensors.sh list

# Or directly
docker compose exec -T c3nav-core python manage.py list_sensors --overlay-id 1

2. Add a New Sensor

# Using the helper script
./manage_noi_sensors.sh add 'NOI:YourSensorID' 'Sensor Display Name' 300.0 250.0 floor1

# Or directly
docker compose exec -T c3nav-core python manage.py add_sensor \
  --overlay-id 1 \
  --sensor-id 'NOI:YourSensorID' \
  --name 'Sensor Display Name' \
  --x 300.0 \
  --y 250.0 \
  --level floor1

3. Scrape Data for All Sensors

# Using the helper script
./manage_noi_sensors.sh scrape

# Or directly
docker compose exec -T c3nav-core python manage.py manage_sensors --scrape-data --overlay-id 1

Configuration Structure

The overlay's sensor_config field contains:

{
  "data_path": "data",
  "mappings": {
    "id_field": "scode",
    "name_field": "sname",
    "x_field": "scoordinate.x",
    "y_field": "scoordinate.y"
  },
  "sensors": [
    {
      "id": "NOI:FreeSoftwareLab-Temperature",
      "coordinates": {"x": 291.0, "y": 241.0},
      "level": "floor1"
    },
    {
      "id": "NOI:NOI-A1-Floor1-CO2",
      "coordinates": {"x": 270.0, "y": 241.0},
      "level": "floor1"
    }
  ]
}

Database Schema

DataOverlay fields:

• data_source_url: URL to scrape sensor data from
• sensor_config: JSON configuration for sensor mapping and processing

DataOverlayFeature fields:

• sensor_id: Unique identifier for the sensor
• sensor_type: Type of sensor (e.g., 'environmental')
• sensor_value: Single sensor value (nullable for multi-measurement sensors)
• sensor_unit: Unit of measurement (nullable for multi-measurement sensors)
• coordinates_x, coordinates_y: Position in c3nav coordinate system
• last_updated: Timestamp of last data update
• sensor_data: Raw sensor data for debugging
• extra_data: Processed sensor readings for display

Data Flow

1. Configuration: Sensors are configured in the overlay's sensor_config
2. Scraping: The manage_sensors command fetches data from NOI Open Data Hub
3. Processing: Data is processed according to sensor configuration
4. Storage: Sensor features are created/updated in the database
5. Display: Sensors appear as interactive points on the map

Adding New Sensor Types

To add a new sensor from the NOI Open Data Hub:

1. Find the sensor ID in the NOI API (usually starts with "NOI:")
2. Determine the coordinates where it should appear on the map
3. Choose the appropriate level/floor
4. Add it using the add_sensor command
5. Run the scrape command to fetch initial data

Troubleshooting

Sensor not appearing on map

• Check if the level exists: docker compose exec -T c3nav-core python manage.py shell -c "from c3nav.mapdata.models import Level; print([l.short_label for l in Level.objects.all()])"
• Verify coordinates are within the map bounds
• Check if the overlay is enabled and visible

No data being scraped

• Verify the sensor ID exists in the NOI Open Data Hub API
• Check the API URL is accessible: https://mobility.api.opendatahub.com/v2/flat/IndoorStation/*/latest
• Review logs during scraping for errors

Data not updating

• Check the last_updated field in the sensor feature
• Verify the scraping command completed successfully
• Consider running the scrape command more frequently

Files

• up.sh: Main setup script
• manage_noi_sensors.sh: Helper script for sensor management
• src/c3nav/mapdata/management/commands/manage_sensors.py: Core sensor management command
• src/c3nav/mapdata/management/commands/add_sensor.py: Command to add new sensors
• src/c3nav/mapdata/management/commands/list_sensors.py: Command to list sensors
• src/c3nav/mapdata/models/overlay.py: Database models
• src/c3nav/mapdata/migrations/0140_add_temperature_fields.py: Migration for sensor fields
• src/c3nav/mapdata/migrations/0141_add_sensor_data_field.py: Migration for sensor_data field