TellurideUI collaborates with TelluridePipeline and TellurideSensor to provide an example of a Broadway pipeline consuming a stream of simulated IoT sensor reading messages from a RabbitMQ queue, in batches, computing some simple aggregate metrics over the stream of messages, and then publishing those metrics in a batch-oriented way to a queue on RabbitMQ by way of the BroadwayRabbitMQ producer. The point of this example is not the domain, which is contrived, but the mechanics of Broadway and Rabbit MQ working together.
TellurideUI implements a Phoenix LiveView dashboard that provides configuration and visualization of the Broadway pipeline. Here's an example screenshot:
See Getting Started below for instructions on starting this example.
Phoenix Framework (including LiveView)
with:
- amqp library
- TailwindUI
- Start RabbitMQ.
A docker-compose.yaml that includes RabbitMQ is provided in telluride_pipeline. Start RabbitMQ by executing:
cd telluride_pipeline/
docker-compose up -d- Start TelluridePipeline by executing:
cd telluride_pipeline/
iex -S mixTo run the telluride_pipeline tests:
mix test --only telemetry_broadway- Start TellurideSensor by executing:
cd telluride_sensor/
iex -S mix- Start TellurideUI by executing:
cd telluride_ui/
mix phx.serverand then point your browser at http://localhost:4000
If you need help with your Elixir projects, contact info@brsg.io or visit https://brsg.io.
This project was inspired by Marlus Saraiva's ElixirConf 2019 talk Build Efficient Data Processing Pipelines.
Copyright 2021 - Blue River Systems Group, LLC - All Rights Reeserved
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.