funge

local_run/SENSOR_README.md

@@ -0,0 +1,152 @@
+# NOI Sensor Management System
+
+This system allows you to manage environmental sensors from the NOI Open Data Hub in c3nav, displaying them as overlay features on different levels/floors.
+
+## Overview
+
+The system supports:
+- Multiple sensors on the same overlay but on different levels
+- Dynamic addition of new sensors through Django management commands
+- Automatic data scraping from NOI Open Data Hub APIs
+- Real-time display of CO2, temperature, humidity and other environmental data
+
+## Architecture
+
+- **Single Overlay**: All NOI environmental sensors are managed under one `DataOverlay`
+- **Multiple Levels**: Sensors can be placed on different floors (floor0, floor1, etc.)
+- **Flexible Configuration**: Sensor locations and properties are configurable via the overlay's `sensor_config` field
+- **Dynamic Discovery**: The system can automatically discover and display any sensor data from the NOI API
+
+## Setup
+
+The main setup is handled by the `up.sh` script, which:
+
+1. Creates a single "NOI Environmental Sensors" overlay
+2. Configures initial sensors with their coordinates and levels
+3. Scrapes initial data from the NOI Open Data Hub
+4. Applies necessary database migrations
+
+## Managing Sensors
+
+### 1. List All Sensors
+```bash
+# Using the helper script
+./manage_noi_sensors.sh list
+
+# Or directly
+docker compose exec -T c3nav-core python manage.py list_sensors --overlay-id 1
+```
+
+### 2. Add a New Sensor
+```bash
+# Using the helper script
+./manage_noi_sensors.sh add 'NOI:YourSensorID' 'Sensor Display Name' 300.0 250.0 floor1
+
+# Or directly
+docker compose exec -T c3nav-core python manage.py add_sensor \
+  --overlay-id 1 \
+  --sensor-id 'NOI:YourSensorID' \
+  --name 'Sensor Display Name' \
+  --x 300.0 \
+  --y 250.0 \
+  --level floor1
+```
+
+### 3. Scrape Data for All Sensors
+```bash
+# Using the helper script
+./manage_noi_sensors.sh scrape
+
+# Or directly
+docker compose exec -T c3nav-core python manage.py manage_sensors --scrape-data --overlay-id 1
+```
+
+## Configuration Structure
+
+The overlay's `sensor_config` field contains:
+
+```json
+{
+  "data_path": "data",
+  "mappings": {
+    "id_field": "scode",
+    "name_field": "sname",
+    "x_field": "scoordinate.x",
+    "y_field": "scoordinate.y"
+  },
+  "sensors": [
+    {
+      "id": "NOI:FreeSoftwareLab-Temperature",
+      "coordinates": {"x": 291.0, "y": 241.0},
+      "level": "floor1"
+    },
+    {
+      "id": "NOI:NOI-A1-Floor1-CO2",
+      "coordinates": {"x": 270.0, "y": 241.0},
+      "level": "floor1"
+    }
+  ]
+}
+```
+
+## Database Schema
+
+### DataOverlay fields:
+- `data_source_url`: URL to scrape sensor data from
+- `sensor_config`: JSON configuration for sensor mapping and processing
+
+### DataOverlayFeature fields:
+- `sensor_id`: Unique identifier for the sensor
+- `sensor_type`: Type of sensor (e.g., 'environmental')
+- `sensor_value`: Single sensor value (nullable for multi-measurement sensors)
+- `sensor_unit`: Unit of measurement (nullable for multi-measurement sensors)
+- `coordinates_x`, `coordinates_y`: Position in c3nav coordinate system
+- `last_updated`: Timestamp of last data update
+- `sensor_data`: Raw sensor data for debugging
+- `extra_data`: Processed sensor readings for display
+
+## Data Flow
+
+1. **Configuration**: Sensors are configured in the overlay's `sensor_config`
+2. **Scraping**: The `manage_sensors` command fetches data from NOI Open Data Hub
+3. **Processing**: Data is processed according to sensor configuration
+4. **Storage**: Sensor features are created/updated in the database
+5. **Display**: Sensors appear as interactive points on the map
+
+## Adding New Sensor Types
+
+To add a new sensor from the NOI Open Data Hub:
+
+1. Find the sensor ID in the NOI API (usually starts with "NOI:")
+2. Determine the coordinates where it should appear on the map
+3. Choose the appropriate level/floor
+4. Add it using the `add_sensor` command
+5. Run the scrape command to fetch initial data
+
+## Troubleshooting
+
+### Sensor not appearing on map
+- Check if the level exists: `docker compose exec -T c3nav-core python manage.py shell -c "from c3nav.mapdata.models import Level; print([l.short_label for l in Level.objects.all()])"`
+- Verify coordinates are within the map bounds
+- Check if the overlay is enabled and visible
+
+### No data being scraped
+- Verify the sensor ID exists in the NOI Open Data Hub API
+- Check the API URL is accessible: https://mobility.api.opendatahub.com/v2/flat/IndoorStation/*/latest
+- Review logs during scraping for errors
+
+### Data not updating
+- Check the `last_updated` field in the sensor feature
+- Verify the scraping command completed successfully
+- Consider running the scrape command more frequently
+
+## Files
+
+- `up.sh`: Main setup script
+- `manage_noi_sensors.sh`: Helper script for sensor management
+- `src/c3nav/mapdata/management/commands/manage_sensors.py`: Core sensor management command
+- `src/c3nav/mapdata/management/commands/add_sensor.py`: Command to add new sensors
+- `src/c3nav/mapdata/management/commands/list_sensors.py`: Command to list sensors
+- `src/c3nav/mapdata/models/overlay.py`: Database models
+- `src/c3nav/mapdata/migrations/0140_add_temperature_fields.py`: Migration for sensor fields
+- `src/c3nav/mapdata/migrations/0141_add_sensor_data_field.py`: Migration for sensor_data field