2025-summer/team-3
12
Fork 0
Code Issues Pull requests Projects Releases Activity

funge

Browse source
This commit is contained in:
Alessio 2025-08-02 12:03:57 +02:00
parent 789640998a
commit 197264f27a
7 changed files with 488 additions and 59 deletions
Download patch file Download diff file Expand all files Collapse all files

152
local_run/SENSOR_README.md Normal file
View file

@ -0,0 +1,152 @@
# NOI Sensor Management System
This system allows you to manage environmental sensors from the NOI Open Data Hub in c3nav, displaying them as overlay features on different levels/floors.
## Overview
The system supports:
- Multiple sensors on the same overlay but on different levels
- Dynamic addition of new sensors through Django management commands
- Automatic data scraping from NOI Open Data Hub APIs
- Real-time display of CO2, temperature, humidity and other environmental data
## Architecture
- **Single Overlay**: All NOI environmental sensors are managed under one `DataOverlay`
- **Multiple Levels**: Sensors can be placed on different floors (floor0, floor1, etc.)
- **Flexible Configuration**: Sensor locations and properties are configurable via the overlay's `sensor_config` field
- **Dynamic Discovery**: The system can automatically discover and display any sensor data from the NOI API
## Setup
The main setup is handled by the `up.sh` script, which:
1. Creates a single "NOI Environmental Sensors" overlay
2. Configures initial sensors with their coordinates and levels
3. Scrapes initial data from the NOI Open Data Hub
4. Applies necessary database migrations
## Managing Sensors
### 1. List All Sensors
```bash
# Using the helper script
./manage_noi_sensors.sh list
# Or directly
docker compose exec -T c3nav-core python manage.py list_sensors --overlay-id 1
```
### 2. Add a New Sensor
```bash
# Using the helper script
./manage_noi_sensors.sh add 'NOI:YourSensorID' 'Sensor Display Name' 300.0 250.0 floor1
# Or directly
docker compose exec -T c3nav-core python manage.py add_sensor \
--overlay-id 1 \
--sensor-id 'NOI:YourSensorID' \
--name 'Sensor Display Name' \
--x 300.0 \
--y 250.0 \
--level floor1
```
### 3. Scrape Data for All Sensors
```bash
# Using the helper script
./manage_noi_sensors.sh scrape
# Or directly
docker compose exec -T c3nav-core python manage.py manage_sensors --scrape-data --overlay-id 1
```
## Configuration Structure
The overlay's `sensor_config` field contains:
```json
{
"data_path": "data",
"mappings": {
"id_field": "scode",
"name_field": "sname",
"x_field": "scoordinate.x",
"y_field": "scoordinate.y"
},
"sensors": [
{
"id": "NOI:FreeSoftwareLab-Temperature",
"coordinates": {"x": 291.0, "y": 241.0},
"level": "floor1"
},
{
"id": "NOI:NOI-A1-Floor1-CO2",
"coordinates": {"x": 270.0, "y": 241.0},
"level": "floor1"
}
]
}
```
## Database Schema
### DataOverlay fields:
- `data_source_url`: URL to scrape sensor data from
- `sensor_config`: JSON configuration for sensor mapping and processing
### DataOverlayFeature fields:
- `sensor_id`: Unique identifier for the sensor
- `sensor_type`: Type of sensor (e.g., 'environmental')
- `sensor_value`: Single sensor value (nullable for multi-measurement sensors)
- `sensor_unit`: Unit of measurement (nullable for multi-measurement sensors)
- `coordinates_x`, `coordinates_y`: Position in c3nav coordinate system
- `last_updated`: Timestamp of last data update
- `sensor_data`: Raw sensor data for debugging
- `extra_data`: Processed sensor readings for display
## Data Flow
1. **Configuration**: Sensors are configured in the overlay's `sensor_config`
2. **Scraping**: The `manage_sensors` command fetches data from NOI Open Data Hub
3. **Processing**: Data is processed according to sensor configuration
4. **Storage**: Sensor features are created/updated in the database
5. **Display**: Sensors appear as interactive points on the map
## Adding New Sensor Types
To add a new sensor from the NOI Open Data Hub:
1. Find the sensor ID in the NOI API (usually starts with "NOI:")
2. Determine the coordinates where it should appear on the map
3. Choose the appropriate level/floor
4. Add it using the `add_sensor` command
5. Run the scrape command to fetch initial data
## Troubleshooting
### Sensor not appearing on map
- Check if the level exists: `docker compose exec -T c3nav-core python manage.py shell -c "from c3nav.mapdata.models import Level; print([l.short_label for l in Level.objects.all()])"`
- Verify coordinates are within the map bounds
- Check if the overlay is enabled and visible
### No data being scraped
- Verify the sensor ID exists in the NOI Open Data Hub API
- Check the API URL is accessible: https://mobility.api.opendatahub.com/v2/flat/IndoorStation/*/latest
- Review logs during scraping for errors
### Data not updating
- Check the `last_updated` field in the sensor feature
- Verify the scraping command completed successfully
- Consider running the scrape command more frequently
## Files
- `up.sh`: Main setup script
- `manage_noi_sensors.sh`: Helper script for sensor management
- `src/c3nav/mapdata/management/commands/manage_sensors.py`: Core sensor management command
- `src/c3nav/mapdata/management/commands/add_sensor.py`: Command to add new sensors
- `src/c3nav/mapdata/management/commands/list_sensors.py`: Command to list sensors
- `src/c3nav/mapdata/models/overlay.py`: Database models
- `src/c3nav/mapdata/migrations/0140_add_temperature_fields.py`: Migration for sensor fields
- `src/c3nav/mapdata/migrations/0141_add_sensor_data_field.py`: Migration for sensor_data field

46
local_run/manage_noi_sensors.sh Executable file
View file

@ -0,0 +1,46 @@
#!/bin/bash
# Helper script to manage NOI sensors
# Usage: ./manage_noi_sensors.sh [add|list|scrape] [args...]
COMPOSE_EXEC="docker compose exec -T c3nav-core python manage.py"
case "$1" in
"add")
if [ $# -lt 6 ]; then
echo "Usage: $0 add <sensor-id> <name> <x> <y> <level>"
echo "Example: $0 add 'NOI:MyNewSensor' 'My New Sensor' 300.0 250.0 floor1"
exit 1
fi
SENSOR_ID="$2"
NAME="$3"
X="$4"
Y="$5"
LEVEL="$6"
echo "Adding sensor: $NAME ($SENSOR_ID) at ($X, $Y) on $LEVEL"
$COMPOSE_EXEC add_sensor --overlay-id 1 --sensor-id "$SENSOR_ID" --name "$NAME" --x "$X" --y "$Y" --level "$LEVEL"
;;
"list")
echo "Listing all sensors in overlay 1:"
$COMPOSE_EXEC list_sensors --overlay-id 1
;;
"scrape")
echo "Scraping data for all sensors in overlay 1:"
$COMPOSE_EXEC manage_sensors --scrape-data --overlay-id 1
;;
*)
echo "NOI Sensor Management Helper"
echo "Usage: $0 [add|list|scrape] [args...]"
echo ""
echo "Commands:"
echo " add <sensor-id> <name> <x> <y> <level> - Add a new sensor"
echo " list - List all sensors"
echo " scrape - Scrape data for all sensors"
echo ""
echo "Examples:"
echo " $0 add 'NOI:NewSensor' 'My Sensor' 300.0 250.0 floor1"
echo " $0 list"
echo " $0 scrape"
exit 1
;;
esac

62
local_run/up.sh
View file

@ -61,61 +61,57 @@ from c3nav.mapdata.models import DataOverlay, DataOverlayFeature
DataOverlay.objects.filter(titles__en__icontains='Environmental').delete()
DataOverlay.objects.filter(titles__en__icontains='Temperature').delete()
# Create NOI environmental sensor overlay with real data configuration
# Create single NOI environmental sensor overlay with multiple sensors configuration
overlay = DataOverlay.objects.create(
titles={'en': 'NOI Environmental Sensors'},
description='Real-time CO2 and temperature sensors from NOI Open Data Hub - displays current readings with values and units',
default_geomtype='point',
data_source_url='https://mobility.api.opendatahub.com/v2/flat/IndoorStation/*/latest?where=and(scode.eq.%22NOI:FreeSoftwareLab-Temperature%22)',
data_source_url='https://mobility.api.opendatahub.com/v2/flat/IndoorStation/*/latest',
sensor_config={
'data_path': 'data',
'level': 'floor1', # Specify which floor/level to place sensors on
'mappings': {
'id_field': 'scode',
'name_field': 'sname',
'fixed_coordinates': {
'x': 291.0,
'y': 241.0
'name_field': 'sname',
'x_field': 'scoordinate.x',
'y_field': 'scoordinate.y'
},
'sensors': [
{
'id': 'NOI:FreeSoftwareLab-Temperature',
'coordinates': {'x': 291.0, 'y': 241.0},
'level': 'floor1'
},
{
'id': 'NOI:NOI-A1-Floor1-CO2',
'coordinates': {'x': 270.0, 'y': 241.0},
'level': 'floor1'
}
}
},
update_interval=120
)
overlay2 = DataOverlay.objects.create(
titles={'en': 'NOI Environmental Sensors 2'},
description='Real-time CO2 and temperature sensors from NOI Open Data Hub - displays current readings with values and units',
default_geomtype='point',
data_source_url='https://mobility.api.opendatahub.com/v2/flat/IndoorStation/*/latest?where=and(scode.eq.%22NOI:NOI-A1-Floor1-CO2%22)',
sensor_config={
'data_path': 'data',
'level': 'floor1', # Specify which floor/level to place sensors on
'mappings': {
'id_field': 'scode',
'name_field': 'sname',
'fixed_coordinates': {
'x': 270.0,
'y': 241.0
}
}
]
},
update_interval=120
)
print(f"NOI sensor overlay created with ID {overlay.id}")
print(f"NOI sensor overlay 2 created with ID {overlay2.id}")
EOF
# Scrape real NOI sensor data for both overlays
# Scrape real NOI sensor data
echo "Scraping NOI sensor data..."
# Give the database a moment to settle after overlay creation
sleep 2
# Scrape the overlays directly using their expected IDs (1 and 2)
echo "Scraping first overlay (ID: 1)..."
# Scrape the overlay data (should automatically discover all configured sensors)
echo "Scraping overlay data (ID: 1)..."
docker compose exec -T c3nav-core python manage.py manage_sensors --scrape-data --overlay-id 1
echo "Scraping second overlay (ID: 2)..."
docker compose exec -T c3nav-core python manage.py manage_sensors --scrape-data --overlay-id 2
# List all sensors to verify setup
echo "Listing all sensors in the overlay..."
docker compose exec -T c3nav-core python manage.py list_sensors --overlay-id 1
echo "Sensor setup completed!"
echo ""
echo "To add a new sensor to the overlay, use:"
echo "docker compose exec -T c3nav-core python manage.py add_sensor --overlay-id 1 --sensor-id 'NOI:YourSensorID' --name 'Your Sensor Name' --x 300.0 --y 250.0 --level floor1"
echo ""
echo "To scrape data for all sensors:"
echo "docker compose exec -T c3nav-core python manage.py manage_sensors --scrape-data --overlay-id 1"