Proprietary Software (MyriaCeph)

Open Source Software (IntraCeph)

Software for Sensing and Automation

We can consider interconnected software layers for Sensing and Automation. At the foundation, edge computing runs directly on or near sensing hardware, at a minimum processing and filtering raw data locally to enable fast, low-latency responses.

From the edge, communication and messaging protocols and standards — such as MQTT, AMQP, ZeroMQ (transport layer), MessagePack, Protobuf, BSON, CBOR (encoding) and OPC-UA (protocol-transport-security) — to move data reliably between devices and systems.

User interfaces — across mobile, web, and desktop — for control and to give real-time visibility into system state, trends, and alerts.

A user might interact directly with an edge platform, but also there can be a Centralised data platform, for example Home Assistant, openHAB, (consumer); online social platforms (e.g. Strava – Fitness); SCADA (industrial); cloud based (IoT Core/Hub); or some custom platform for example built on Prometheus, Grafana, and we will also discuss JupyterLab. The role of the platforms is to receive, store, and organize data streams, providing historical records and scalability to support analysis and serve multiple applications.

Finally, the evolving area of autonomous AI analysis and control, continuously monitoring data and potentially issuing control commands without requiring constant human intervention. This decision making, action, could happen both at the edge and/or centrally.

The multitude of communication protocols, encoding standards, platforms is a persistent challenge, and points to the importance of open, well documented standards, to support integration. Given the evolving and wide diversity of requirements and applications this suggests there will be a convergence to a selection of competing interoperable open standards and platforms, rather than a single standard, platform.

MyriaCeph – Applications and Data Platform [May-2026]

In the design of any interconnected application, where we are traversing from any point of sensing and action (edge devices) all the way to interface device (e.g. smart-phone) we need to start with the question of “where” does data reside. Often the simplest, lowest cost and most scalable solution is that it largely resides and is maintained at the edge. As edge devices become more capable, much of any analysis can be performed on the edge, with just the results and/or notifications (e.g. triggers) sent. This approach also does not exclude a connected edge device incorporating “other” data in an analysis, be it from other edge devices or pulling in other published data. And finally, the edge device can always act as a server, answering requests, be it for a subset of the raw data, or some supported derivative of the raw data.

In the storing of data, we also need to address the question of what should be the primary key. Experience has shown that it is best to always have a primary key based on location and datetime. When something critical happens, it is generally at a specific location and corresponding to a specific time interval, hence being able to efficiently find and access all captured data for a time interval, location can be valuable. This also raises the question of accuracy and resolution of time and location. For example, to make use of a sound recording to help locate the origin of a particular sound would benefit from a timing resolution down to milliseconds (time to distance), requiring a GPS receiver and precise timing of the data.

In considering long term storage of data, managing storage, the granularity (sampling) of the data should be considered. One elegant approach is to capture the data at multiple sample rates, where recent data is captured in detail, while only a sampling of older data is kept. PostgreSQL is a good example of this, though this is primarily implemented to support efficient viewing of very large datasets; for example displaying one sample point per-day when displaying a year or more of data; displaying a data point every few hours when looking at one week of data. If data storage is implemented in this way at the time of data capture, data management can be greatly simplified. The higher sampled data discarded, over written after a few hours, days, with the lower sample rate data kept for months or even years.

And finally the “dashboards”, interactive applications. The most most versatile solution here often being hosting the application on the web, which can then be accessed from any device, location, no software instillation necessary. One platform to consider is the open source JupyterNotebooks, JupyterLab, JupyterHub platform, which continues to rapidly evolve, incorporate new capabilities. Further, as a testament to its design and utility, we can see that elements of the platform are being replicated by the new platforms of the major players in the AI space, the likes of Google and Microsoft. In essence, the platforms are evolving to the point that we need simply ask a series of questions with the platforms generating the code, processing the data, and generating and editing a document, or in other format as we might request. And critically, or at least until we can truly trust answers, having a transparent platform, one can view, modify and re-execute code, implement new code, validate results.

InterCeph – Communication and Networking [May-2026]

There are a multitude of protocols, communication standards, in use today, including the publish OPC-UA standards. This begs the question of why propose yet one more? While the proposed Open Source InterCeph code and libraries would include certain new protocols, at its core is the concept of networking, the routing of messages, which could then be interoperable with existing applications and messaging platforms. It starts with the end points, the origin of any message and a target recipient (or recipients in the case of a channel), where the end points are software. These end points correspond to unique address, with the infrastructure responsible for the secure routing of messages. For inter-process communication, this could be through shared memory (to maximize performance); via a local area network; a wide area network (internet), or even through file transfer, such as via a USB drive.

A critical aspect of any messaging platform, is the question of trust. Some aspects in the proposed design come from learnings in developing solutions applicable in the Oil and Gas industry, where raw data volumes are often measured in terabytes; where real-time data, decisions can be critical; and where authentication and security is a necessity.

In addressing security we start with the concept of joining a “network”, where the sentinels of the network maintain the list of authorized members (addresses), can monitor their behavior; can de-unauthorize a member. Only authorized members would be permitted to transmit messages, the sentinels the authority to authenticate the messages. Other than administrative messages, the network would otherwise be agnostic to message content; it would be up to the end points (applications) to secure and encrypt messages as required, with the network maintaining zero knowledge regarding the contents. A network could remain closed (first class members only) or support a bridge, where messages could pass between certain authorized applications on different networks.

IntraCeph – Code, Models, ML, Automation

Open source code related to Sensing and Automation will be published under the IntraCeph banner [Github Link]