# NOI Sensor Management System

This system allows you to manage environmental sensors from the NOI Open Data Hub in c3nav, displaying them as overlay features on different levels/floors.

## Overview

The system supports:
- Multiple sensors on the same overlay but on different levels
- Dynamic addition of new sensors through Django management commands
- Automatic data scraping from NOI Open Data Hub APIs
- Real-time display of CO2, temperature, humidity and other environmental data

## Architecture

- **Single Overlay**: All NOI environmental sensors are managed under one `DataOverlay`
- **Multiple Levels**: Sensors can be placed on different floors (floor0, floor1, etc.)
- **Flexible Configuration**: Sensor locations and properties are configurable via the overlay's `sensor_config` field
- **Dynamic Discovery**: The system can automatically discover and display any sensor data from the NOI API

## Setup

The main setup is handled by the `up.sh` script, which:

1. Creates a single "NOI Environmental Sensors" overlay
2. Configures initial sensors with their coordinates and levels
3. Scrapes initial data from the NOI Open Data Hub
4. Applies necessary database migrations

## Managing Sensors

### 1. List All Sensors
```bash
# Using the helper script
./manage_noi_sensors.sh list

# Or directly
docker compose exec -T c3nav-core python manage.py list_sensors --overlay-id 1
```

### 2. Add a New Sensor
```bash
# Using the helper script
./manage_noi_sensors.sh add 'NOI:YourSensorID' 'Sensor Display Name' 300.0 250.0 floor1

# Or directly
docker compose exec -T c3nav-core python manage.py add_sensor \
  --overlay-id 1 \
  --sensor-id 'NOI:YourSensorID' \
  --name 'Sensor Display Name' \
  --x 300.0 \
  --y 250.0 \
  --level floor1
```

### 3. Scrape Data for All Sensors
```bash
# Using the helper script
./manage_noi_sensors.sh scrape

# Or directly
docker compose exec -T c3nav-core python manage.py manage_sensors --scrape-data --overlay-id 1
```

## Configuration Structure

The overlay's `sensor_config` field contains:

```json
{
  "data_path": "data",
  "mappings": {
    "id_field": "scode",
    "name_field": "sname",
    "x_field": "scoordinate.x",
    "y_field": "scoordinate.y"
  },
  "sensors": [
    {
      "id": "NOI:FreeSoftwareLab-Temperature",
      "coordinates": {"x": 291.0, "y": 241.0},
      "level": "floor1"
    },
    {
      "id": "NOI:NOI-A1-Floor1-CO2",
      "coordinates": {"x": 270.0, "y": 241.0},
      "level": "floor1"
    }
  ]
}
```

## Database Schema

### DataOverlay fields:
- `data_source_url`: URL to scrape sensor data from
- `sensor_config`: JSON configuration for sensor mapping and processing

### DataOverlayFeature fields:
- `sensor_id`: Unique identifier for the sensor
- `sensor_type`: Type of sensor (e.g., 'environmental')
- `sensor_value`: Single sensor value (nullable for multi-measurement sensors)
- `sensor_unit`: Unit of measurement (nullable for multi-measurement sensors)
- `coordinates_x`, `coordinates_y`: Position in c3nav coordinate system
- `last_updated`: Timestamp of last data update
- `sensor_data`: Raw sensor data for debugging
- `extra_data`: Processed sensor readings for display

## Data Flow

1. **Configuration**: Sensors are configured in the overlay's `sensor_config`
2. **Scraping**: The `manage_sensors` command fetches data from NOI Open Data Hub
3. **Processing**: Data is processed according to sensor configuration
4. **Storage**: Sensor features are created/updated in the database
5. **Display**: Sensors appear as interactive points on the map

## Adding New Sensor Types

To add a new sensor from the NOI Open Data Hub:

1. Find the sensor ID in the NOI API (usually starts with "NOI:")
2. Determine the coordinates where it should appear on the map
3. Choose the appropriate level/floor
4. Add it using the `add_sensor` command
5. Run the scrape command to fetch initial data

## Troubleshooting

### Sensor not appearing on map
- Check if the level exists: `docker compose exec -T c3nav-core python manage.py shell -c "from c3nav.mapdata.models import Level; print([l.short_label for l in Level.objects.all()])"`
- Verify coordinates are within the map bounds
- Check if the overlay is enabled and visible

### No data being scraped
- Verify the sensor ID exists in the NOI Open Data Hub API
- Check the API URL is accessible: https://mobility.api.opendatahub.com/v2/flat/IndoorStation/*/latest
- Review logs during scraping for errors

### Data not updating
- Check the `last_updated` field in the sensor feature
- Verify the scraping command completed successfully
- Consider running the scrape command more frequently

## Files

- `up.sh`: Main setup script
- `manage_noi_sensors.sh`: Helper script for sensor management
- `src/c3nav/mapdata/management/commands/manage_sensors.py`: Core sensor management command
- `src/c3nav/mapdata/management/commands/add_sensor.py`: Command to add new sensors
- `src/c3nav/mapdata/management/commands/list_sensors.py`: Command to list sensors
- `src/c3nav/mapdata/models/overlay.py`: Database models
- `src/c3nav/mapdata/migrations/0140_add_temperature_fields.py`: Migration for sensor fields
- `src/c3nav/mapdata/migrations/0141_add_sensor_data_field.py`: Migration for sensor_data field