Build Yourself an AI Powered Microscope – DevConf 2024

Thank you everyone who attended my session at both DevConf Cape Town and Johannesburg
I met so many awesome people during DevConf inside and outside of my session! Thank you for making DevConf 2024 the incredibile conference it was!

I’ve included the slides for those interested in building your own DIY Microscope, or something similar to help you with your own accessibility challenges. If you have an imaging device with some intelligence, there’s so many things you can do!

The project is using a Raspberry Pi 3B (4 and 5 works too). It is build using C# and .NET 8. The user interface is created using AvaloniaUI. There’s nothing better that Avalonia if you need to create small, incredibly fast user interfaces on Linux using .NET. I really recommend trying it out!

The backend is using Azure CosmosDB for storage. The MongoDB VCore Implementation. This is chosen for it’s vector search capabilities. Perfect for a conversational solution using OpenAI.

The slides can be found below

The whole premise of the talk was to inspire developers to think about using all these tech components in your toolbox. Both AI and IoT bits and use it for good. There’s so many solutions that could be created especially for accessibility, to make people’s life a little easier. This is also exactly why I created this Microscope, to solve my own accessibility problems when building IoT solutions. Think about what you could do!

Recorded Demos

Demo of analysing a Gigatron board
Unicorn HATs and dynamic lighting!
Using a Conversational Interface
Looking a ZX Spectrum RAM Chips
Helios Speaks Afrikaans!

The Microscope Tech Stack

.NET 8 (C#) https://learn.microsoft.com/en-us/dotnet/core/whats-new/dotnet-8/overview

Avalonia UI https://avaloniaui.net/

SkiaSharp https://github.com/mono/SkiaSharp

Azure CosmosDB https://azure.microsoft.com/en-us/products/cosmos-db

Azure AI Vision https://azure.microsoft.com/en-us/products/ai-services/ai-vision

Azure Speech https://azure.microsoft.com/en-us/products/ai-services/ai-speech

Azure OpenAI https://azure.microsoft.com/en-us/products/ai-services/openai-service

Azure IoT Edge https://azure.microsoft.com/en-us/products/iot-edge

Docker https://www.docker.com/

Mosquito MQTT https://mosquitto.org/

The components used can be found here. You can put this all together yourself!

Some photos of the session at DevConf 2024

I hope to see you next time! Happy building and learning with cool tech!!! 😍

DevConf 2023 – Create the sustainable home farm of your dreams!

Thank you everyone who attended my session at both DevConf Cape Town and Pretoria
I met so many amazing people excited about creating a more sustainable future.

This is a quick blog post which includes the presented slide deck. I hope attendees found the talk useful. It’s a very broad topic and fitting months of content into 40 min is a little challengening. So the presentation is a broad overview of the completed farming project project and there’s links to more deeper content within the slides. This includes content and github repositiories with some source code examples. The plan is to also release ALL the content eventually as a full OSS project when I’m happy with it, including the Mobile App and integrations with the cloud components. I’m currently rounding everything off to do this. So expect a few more posts about this and source code.

The slides can be found below

DevConf YouTube Channel: https://youtu.be/-74owkJbfEQ?list=PLZgTMWtBnKkopU5DZbP7so-f3z7eMFYqq

There is also Agri hacks planned where we can hack and learn about this topic together over a few sessions where you can learn hands on how to build something like this project. This will be a free to attend event held at @CPTMSDUG in Cape Town. Keep an eye out here: https://www.meetup.com/cape-town-ms-dev-user-group/events/

This hack series be similar in format to our current IoT AI hack series currently running. You can see some of the fun had here at the all day iot hack events: https://twitter.com/search?q=%23iotaihackdays&src=typed_query&f=image

Additional Information

I received so many questions on how to get started. I suggest getting the off the shelf kit I mentioned in the talk that I started with. It contains everything you need to grow your first plants. This is also an indoor grower, so you can start with growing from seeds indoor and not worry about weather and protecting the electronics against sun and water. It will also make your home office a lot prettier. 😊

What’s in the kit

Arduino

4 x Solenoids (for 4 plants)

4 x Relays

4 Moisture Sensors

1 Pump

Silicon Piping

The kit is great to learn with as it’s not a “black box” watering system. It’s an arduino where Elecrow provides the source code to flash onto the arduino. It works out of the box, but it also allows you to learn from it and also make changes.

Please don’t start by hacking it like I did. I decided it should to be connected to Azure IoT. That’s definitely not needed especially when starting out. But the instructions are there if you want to do that too. 😊

For the beginner this is great and as you could see in the talk, once plants start multiplying, you are forced to scale from there. 😁

Link to github repository:

https://github.com/apead/SmartWateringKit

Link to blog post on the kit:

Where to buy the kit in South Africa:

https://www.robotics.org.za/AAK90039K?search=smart%20watering%20kit

Once again thanks to everyone that attended both my session and DevConf 2023 and made the conference the amazing conference it was!

The Outdoor IoT Farm Begins

It’s time to start moving the indoor IoT Farm to being an outdoor IoT farm. The idea I have is to do some experimentation and research into creating automated constrained space farms. I’m going to turn this into a hands on hack series for the CPTMSDUG user group and also share the learnings. Hopefully it goes well. 😁

Today was the big shopping day to get everything I need.

Shopping at the nursery with the much needed coffee.
Some nice wooden planters from the Pole Yard.
Preparing the soil
The offerzen Basil that was grown at a hack event was planted
Getting there
The first evening of the indoor crop being outdoors
New crops added to the indoor IoT Grow station

Azure Percept

Woohoo!! So excited my Azure Percept arrived. Time to do some playing. 😊

Learning Resources

Azure Percept DK and Vision device overview https://docs.microsoft.com/en-us/azure/azure-percept/overview-azure-percept-dk

Azure Percept sample AI models https://docs.microsoft.com/en-us/azure/azure-percept/overview-ai-models

AI @ the Edge https://www.youtube.com/channel/UCa8_TsUuotIMuzkLzGNQPIg

Azure Percept Audio Overview- set up and use your voice assistant https://www.youtube.com/watch?v=K9HfwsDGVnU

Azure Percept | Build & Deploy to edge AI devices in minutes https://www.youtube.com/watch?v=zSBNsEqU5NA

Azure Percept showing Edge Computing and AI in the Agriculture https://youtu.be/1H-U8psvxYE

Unboxing the Azure Percept
Unboxing of the Azure Percept Audio

Cool logo
It arrived!!!
Azure Percept Vision

The full Azure Percept unit

Azure Percept Ports

Lets try some Agri models

Microsoft Developer Cloud Summit 2021

I hope you enjoyed the Microsoft Developer Cloud Summit today! Thank you very much for attending my session: Automating your home or office with IoT Central and Power Apps” It was a lot of information in 30 min, so here’s the content, resources and source code.

This whole session was done using Home Automation devices combined with OBS. Camera Scene changes were controlled with a standard home remote control. So automation doesn’t have to be traditional garage doors or lighting, you can control you PC life too! 😎 Controlling your OBS Scenes with a Home Automation device

Link to Presentation: Microsoft Developer Cloud Summit 2021 Presentation

Topics

Connecting a Pimoroni Environment Monitor to IoT Central

Connecting Azure IoT Central to Power Apps and Power Automate

Arduino Smart Watering Kit with Azure IoT Central

Building an Azure IoT Central β€œGateway” using Node-RED

Flashing Sonoff Devices with TASMOTA

Controlling your OBS Scenes with a Home Automation device

Installing Node-RED in a Container

Installing Eclipse Mosquitto in a Container

Other Resources

IoT Central

https://azure.microsoft.com/en-us/services/iot-central/

Power Apps

https://powerapps.microsoft.com/en-us/

Node-RED

https://nodered.org/

Tasmota

https://tasmota.github.io/docs/

Environmental Monitor

https://www.pishop.co.za/store/enviro–for-raspberry-pi?keyword=enviro&category_id=0

Sonoff

https://www.takealot.com/all?filter=Brand:Sonoff

Arduino Smart Watering Kit with Azure IoT Central

I came across this Smart Watering Kit by Elecrow. It was really great, but one thing missing is internet conectivity. We all know that we can’t grow plants without the “I” in IoT! So let’s see what we can do about that!

Electrow Ardunio Smart Plant Watering Kit
Inside the box

The kit comes with everything you need to water four different plants automatically.

The box contains

  • Arduino Leonardo
  • Power Supply
  • Pump
  • 4 Capacitive Moisture Sensors
  • 4 Way valves
  • Pipes

The kit is very comprensive and includes and LCD display which can show at a glance the conditions of your plants.

The board
Serial Port

The board unfortunately does not have built in wifi connectivity. But what it does have is a serial port. That’s perfect, so what we could do is use that to send telemetry out to another device that is internet / wifi enabled.

A good device for that is a good old (and cheap) 8266 board. I had a NodeMCU in my box of tricks, so I decided to use that.

Parts List

  • Arduino Smart Watering Board (Arduino Leonardo)
  • NodeMCU board
  • 2.2 K ohm resistor
  • 1 K ohm Resistor
  • 470 ohm Resistor
  • Breadboard
  • Wires
  • Power Supply

Connecting an Arduino Leonardo to a Node MCU via Serial Port

First step is to make it work on a breadboard.

Parts List

  • Arduino Smart Watering Board (Arduino Leonardo)
  • NodeMCU board
  • 2.2 K ohm resistor
  • 1 K ohm Resistor
  • 470 ohm Resistor
  • Veroboard
  • Wires
  • Power Supply
Connecting an Arduino Leonardo to a Node MCU via Serial Port

Once working, the next step is to build that onto a more permanent solution. I decided to use just veroboard as it was quick and easy.

Two boards up and running

IoT Central

Device Templates

Device Capabilities

Create device capabilities for the moisture sensors as telemetry (Moisture1, Moisture2, Moisture3, Moisture4). [Device Definition json file is in the github repository along with the source code]

Create properties for the 4 relay states for each of the 4 valves. [Device Definition json file is in the github repository along with the source code]

Create a property for the pump state. [Device Definition json file is in the github repository along with the source code]

Publish the template and create an instance of the template as a device.

Instance of a device template

Note the “connection” information for the new device instance.

Device “Connection” information

Take note of the following:

  • Scope ID
  • Device ID
  • Primary Key
iotcserialrelay.ino

Update the iotcserialrelay.ino file with the:

  • WIFI_SSID – The wifi hotspot ssid
  • WIFI_PASSWORD – The wifi hotspot password
  • SCOPE_ID – The Device Instance’s Scope Id
  • DEVICE_ID – The Device Instance’s Device Id
  • DEVICE_KEY 0 The Device Instance’s Key

Deploy iotcserialrelay.ino to NodeMCU.

Deploy watering_kit.ino to the Arduio Smart Watering Board.

Telemetry in IoT Central

Once deployed and connection is made, telemetry starts appearing within IoT Central’s dashboard.

Finished Product with happy plants powered by Microsoft Azure 😎😎

Source code: https://github.com/apead/SmartWateringKit

Product Information: https://www.elecrow.com/arduino-automatic-smart-plant-watering-kit.html

Where to buy? https://www.robotics.org.za/AAK90039K

Controlling your OBS Scenes with a Home Automation device

You can easily set up a standard 433 Mhz remote control that you use every day in your home to control scenes in OBS (Open Broadcast Software) if you don’t own a fancy Stream Deck.

All you need is a remote, a 433 Mhz Wifi Bridge and some Node-RED magic.

Sonoff 433 Mhz Bridge

Flash the bridge using these instructions: https://explorationspace.co.za/2021/02/10/sonoff-rf-bridge-433-tasmota/

Standard Home Automation Remote control
433 Mhz Wifi Bridge to OBS via Websockets Node-RED Flow

Install the OBS Websocket plugin: https://github.com/Palakis/obs-websocket/releases/tag/4.9.0

Node-RED Switch using button codes to control flow

The 433 Mhz Bridge when it detects a signal, it publishes a message with it’s contents via MQTT. These will have codes attached to them and in the case of a remote, that code / data that is sent via the message payload translates to the button pressed. You can “train” your solution with the remotes you have. These can be standard house-hold remotes.

Scene flows per button press

The switch statements routes the flow based on the button data to a specific function.

Scene switch command to OBS Socket Server

The function sends a message with the “scene-name” with the parameter of the scene created in OBS.

Scenes in OBS

For additional commands refer to the protocol document:

https://github.com/Palakis/obs-websocket/blob/4.x-current/docs/generated/protocol.md

Happy streaming! 😎

Connecting a Pimoroni Environmental Monitor to IoT Central

The Pimoroni Environmental monitor is an amazing piece of hardware. I recently purchased a few to monitor air quality around the house. https://shop.pimoroni.com/products/enviro?variant=31155658457171

Packed into the HAT for a Raspberry Pi Zero you will find:

  • BME280 temperature, pressure, humidity sensor
  • LTR-559 light and proximity sensor
  • MICS6814 analog gas sensor 
  • ADS1015 analog to digital converter (ADC)
  • MEMS microphone
  • 0.96″ colour LCD (160×80)
  • Connector for particulate matter (PM) sensor (available separately)
  • Pimoroni breakout-compatible pin header
Device Template

Device Definition json file is in the github repository along with the source code. This can be imported into Azure IoT Central and the views generated.

environ.py
IoT Central Dashboard

Source code: https://github.com/apead/Pimoroni-Env-Monitor-IoT-Central

Product Information: https://shop.pimoroni.com/products/enviro?variant=31155658457171

Where to buy?

https://www.pishop.co.za/store/enviro–for-raspberry-pi

Connecting Azure IoT Central to Power Apps and Power Automate

Azure IoT Central makes it easy to connect, monitor, and manage your IoT devices at scale. With the IoT Central V3 connector, you can trigger workflows when a rule has fired, and take actions by executing commands, updating properties, getting telemetry from devices, and more. Use this connector with your Azure IoT Central V3 application.

Url: https://docs.microsoft.com/en-us/connectors/azureiotcentral/

The connector is available to be used in

  • Power Apps
  • Power Automate
  • Logic Apps

Power Apps

Due to the dynamic nature of Azure IoT Central, using the connector is not quite straight-forward. To use the Azure IoT Central connector in Canvas apps you need to enable two features in the settings of the application.

These are:

  • Enhanced formula bar
  • Dynamic Schema

Enabling these two features allows Power Apps to understand the resulting schema from a call to IoT central. “Capture Schema” will be enabled on the formula bar.

Power Apps Canvas Settings: Enhanced Formula Bar
Power Apps Canvas Settings: Dynamic Schema
Formula bar: Schema Capture

Power Automate

Power Automate is a lot simpler to get to work. It automatically understand the dynamic schemas of IoT Central.