Data Backend

IronFlock provisions a private database for each project, powered by TimescaleDB. Your app defines the data schema; IronFlock creates the tables and starts collecting data the moment a device is added to the app.

How It Works

1. Define your data schema in .ironflock/data-template.yml.
2. Use the IronFlock SDK to publish data from your edge code.
3. IronFlock automatically sets up the database tables in each project where the app is installed.
4. Data flows from devices through the messaging system into the project database.

Each project gets its own physical database — there is no data sharing between projects.

The user has full control over the data collected by your app in their project. As a developer, you do not have access to this data.

Defining the Data Schema

Create a data-template.yml file in the .ironflock/ directory:

data:
  tables:
    - tablename: sensordata
      columns:
        - id: tsp
          name: Timestamp
          description: Timestamp of measurement
          path: args[0].timestamp
          dataType: timestamp
        - id: temperature
          name: Temperature
          description: Temperature reading in Celsius
          path: args[0].temperature
          dataType: numeric
        - id: humidity
          name: Humidity
          description: Relative humidity percentage
          path: args[0].humidity
          dataType: numeric
        - id: device_id
          name: Device ID
          description: Source device identifier
          path: args[0].device_id
          dataType: string

Column Options

Publishing Data from Edge Code

Use the IronFlock SDK to send data from your app:

from ironflock import IronFlock
 
flock = IronFlock()
 
flock.publish_to_table("sensordata", {
    "timestamp": "2025-01-15T10:30:00Z",
    "temperature": 23.5,
    "humidity": 62.1,
    "device_id": "sensor-001"
})

import { IronFlock } from "ironflock";
 
const flock = new IronFlock();
 
flock.publishToTable("sensordata", {
    timestamp: "2025-01-15T10:30:00Z",
    temperature: 23.5,
    humidity: 62.1,
    device_id: "sensor-001"
});

Transform Tables

You can define SQL transforms that automatically aggregate or process your raw data:

data:
  tables:
    - tablename: sensordata
      columns:
        # ... raw data columns ...
 
  transforms:
    - tablename: hourly_averages
      materialize: true
      schedule: "0 * * * *"
      sql: >
        SELECT
          time_bucket('1 hour', tsp) AS hour,
          avg(temperature) AS avg_temp,
          avg(humidity) AS avg_humidity
        FROM sensordata
        GROUP BY hour
      columns:
        - id: hour
          name: Hour
          dataType: timestamp
        - id: avg_temp
          name: Average Temperature
          dataType: numeric
        - id: avg_humidity
          name: Average Humidity
          dataType: numeric

Transform tables are accessible in boards and via the SDK, just like regular tables.

Tracking the Latest State of an Entity

For tables that represent the current state of real-world entities — machines, assets, production orders — IronFlock supports a pattern called latest-flag tracking.

Instead of overwriting a row when something changes, you always append a new row. IronFlock then automatically maintains a special boolean column called latest_flag that marks the most recent row for each unique value in a chosen column. This gives you a full history of every change while also making it easy to query only the current state.

Enable it on a table with maintainLatestFlagFor:

- tablename: machineform
  maintainLatestFlagFor: ['machinename']
  columns:
    - id: tsp
      dataType: timestamp
    - id: machinename
      dataType: string
    - id: machinetype
      dataType: string
    - id: active
      dataType: boolean
    - id: description
      dataType: string

maintainLatestFlagFor takes a list of columns that together identify a unique entity. When a new row arrives with the same machinename, the previous row’s latest_flag is set to false and the new row’s latest_flag is set to true.

To query only current machine states:

SELECT * FROM machineform WHERE latest_flag = true

To view the full history of a specific machine:

SELECT * FROM machineform WHERE machinename = 'Assembly-Line-01' ORDER BY tsp

Widgets on a board that connect to this table can be configured to filter by latest_flag = true automatically, so users always see current values without any extra work.

Soft Deleting Records

IronFlock’s append-only model means records are never physically deleted. Instead, use a deleted boolean column to mark a record as removed. This preserves the full audit trail while hiding deleted records from dashboards.

Add a deleted column to any entity table:

- id: deleted
  name: Deleted
  dataType: boolean

When a user deletes a record (for example via a form on the board), your app publishes a new row for that entity with deleted: true. Combined with maintainLatestFlagFor, this new row becomes the latest state.

To query only active (non-deleted) current records:

SELECT * FROM machineform
WHERE latest_flag = true
  AND (deleted IS NULL OR deleted = false)

Board widgets can have this filter pre-applied, so deleted records disappear from the dashboard immediately after the form is submitted — but remain in the database for history and audit purposes.