Standing up Project Norman

Project Carl is a home plant-monitoring system: sensor nodes over BLE, an ESP32-S3 hub, and the part this post is about a cloud backend called Project Norman. Norman is not where the system starts. The hub already keeps its own history and serves it over a REST API on the LAN. Norman is what exists for everything that is not on the LAN: history beyond what the hub keeps, an iPhone app that is not always on the same Wi-Fi as the hub, and eventually a model that looks at the readings and says something about them.

This is the first of what I expect to be several posts about Norman, published whenever there is real work to write up rather than on a schedule. As of 25 August 2026 it is live: a small GCP box taking MQTT from the hub, storing it in Postgres, serving it over HTTPS. This is what it took.

Why the app does not write to the cloud

The iPhone app talks to a CompositeHubClient that tries the hub directly over the LAN first and only falls back to Norman for reads. Every write adding a node, changing Wi-Fi, anything that provisions hardware goes to the hub and only the hub. The cloud half of that client throws an unsupported error for all of them, on purpose: provisioning a physical sensor has to happen on the same network as the sensor, and pretending the cloud could do it too would just be a write path that silently did nothing useful. Norman’s job is narrower than “be the backend.” It is “be the part of the backend that makes sense off the LAN.”

What is actually running

Four services, one docker-compose.yml: Postgres on timescale/timescaledb for the actual time series, Mosquitto for MQTT, a FastAPI app for the REST API, and a separate ingest worker that does nothing but subscribe to MQTT and write rows. Two overlay files pick the environment. docker-compose.prod.yml adds Caddy in front of the API for TLS and locks Mosquitto down to a real public listener plus an internal-only one for the ingest container. docker-compose.home.yml swaps Caddy for a Cloudflare Tunnel, for running the same stack on a machine with no public IP at all.

MQTT, abandoned, then not

Somewhere in the middle of building this, one commit is literally titled “refactor ingest protocol from MQTT to HTTPS batch uploads” an attempt to drop the broker in favour of a plain POST the hub could make on its own schedule. It did not stick. By the time this went live, MQTT was back as the primary path: the hub publishes a small JSON object per node to carl/{site}/{node} at QoS 1 roughly every thirty seconds, and the HTTPS batch endpoint survives as the secondary one manual backfill, and anything that wants to send more than one hub’s worth of data in a single request, not what the firmware actually ships. I do not have a tidy record of exactly why the decision reversed, which is itself a fair thing to admit in a post about a project’s real history rather than its clean version.

The ingest worker is deliberately strict about one thing: it will not create a hub record on the fly. If a message arrives for a site nobody has registered through the API first, it gets dropped the code comments this as forcing “a clean ownership flow.” Accepting readings for any site id that shows up would mean anyone who can guess or sniff one could plant fake history against it, and there would be no clean way to claim it back.

A box, not a cluster

The infrastructure is one e2-micro on GCP, provisioned with Terraform: a static IP, a firewall that only opens SSH to my own address, a GCS bucket for nightly Postgres backups. It runs Debian 12, not the more obvious Ubuntu 24.04 Ubuntu’s guest agent on GCP renames the SSH service in a way that breaks the metadata-based key injection Terraform relies on, which is exactly the kind of thing you find out by losing SSH access to a box you just created. Debian does not have the problem.

It is also not free, and the Terraform README says so directly: compute, disk, storage and egress all sit inside GCP’s always-free tier, but a static IPv4 has cost money since a February 2024 policy change about $3.65 a month, roughly $4 all in. The honest alternative, also documented in the same repo, is running the identical containers on a machine at home behind a Cloudflare Tunnel instead of a public IP: a few pounds a month of always-on laptop power, and no public IP to secure at all.

The TLS split that is easy to get wrong

DNS sits behind Cloudflare, and its two subdomains are not treated the same way. norman.manideepreddy.com, the HTTPS API, is proxied Cloudflare’s orange cloud, ordinary and fine. mqtt.manideepreddy.com is deliberately grey-cloud, DNS-only: MQTT over TLS is a raw TCP connection, and Cloudflare’s proxy terminates and re-wraps HTTP traffic in a way that breaks a bare TCP handshake. The MQTT certificate comes from Let’s Encrypt via a standalone certbot run, independent of Cloudflare entirely. Get the proxy toggle wrong on one subdomain and the hub cannot reach the broker at all, with an error that looks like a firewall problem rather than a DNS setting.

Two bugs that only showed up once this was real

An Alembic revision id was one migration away from breaking every future one. The third migration added a rooms table and reshaped predictions to be per-node instead of per-hub, and its own revision id ran to 36 characters. Alembic tracks the current revision in a bookkeeping table with a 32-character VARCHAR column, so the final UPDATE in the migration failed on a value that did not fit the column meant to hold it a limit that only bites once a revision id happens to be descriptive enough to cross it. The fix was shortening the id, not the migration.

passlib and a bcrypt release stopped agreeing with each other. passlib, which has not been maintained since 2020, probes internals of the bcrypt package that bcrypt 4.1 removed. Every password operation started failing with an error claiming a password was longer than 72 bytes, which was not true and had nothing to do with the actual problem. The fix was pinning bcrypt below 4.1 not a code change at all, just refusing an update that two unmaintained assumptions could no longer survive.

Where this leaves the app

Firmware, cloud and iOS agree on one shape for a node, and Norman’s API layer carries an explicit comment that its field names have to byte-match the model the iPhone app decodes, down to serialising an empty room association as "" rather than null. That is not a big architectural idea. It is just the tax of three repositories describing the same object, and the fix that finally lined the API’s node shape up with what the firmware actually ships went in on the same day as the deployment itself.

What is live now

As of 25 August 2026: MQTT publish, ingest worker, Postgres, GET /v1/nodes checked end to end, not just deployed and hoped for. If you go looking at these repositories on GitHub, the main/master branches you land on by default are years out of date. Everything above is on dev for the firmware and pilot for Norman (the iPhone app’s real work is on pilot too), which is what the projects page now links to instead of the stale default. Next post is whichever of the three has something to say next.