Build Yourself an AI Powered Microscope – DevConf 2024

Thank you everyone who attended my session at both DevConf Cape Town and Johannesburg
I met so many awesome people during DevConf inside and outside of my session! Thank you for making DevConf 2024 the incredibile conference it was!

I’ve included the slides for those interested in building your own DIY Microscope, or something similar to help you with your own accessibility challenges. If you have an imaging device with some intelligence, there’s so many things you can do!

The project is using a Raspberry Pi 3B (4 and 5 works too). It is build using C# and .NET 8. The user interface is created using AvaloniaUI. There’s nothing better that Avalonia if you need to create small, incredibly fast user interfaces on Linux using .NET. I really recommend trying it out!

The backend is using Azure CosmosDB for storage. The MongoDB VCore Implementation. This is chosen for it’s vector search capabilities. Perfect for a conversational solution using OpenAI.

The slides can be found below

The whole premise of the talk was to inspire developers to think about using all these tech components in your toolbox. Both AI and IoT bits and use it for good. There’s so many solutions that could be created especially for accessibility, to make people’s life a little easier. This is also exactly why I created this Microscope, to solve my own accessibility problems when building IoT solutions. Think about what you could do!

Recorded Demos

Demo of analysing a Gigatron board
Unicorn HATs and dynamic lighting!
Using a Conversational Interface
Looking a ZX Spectrum RAM Chips
Helios Speaks Afrikaans!

The Microscope Tech Stack

.NET 8 (C#) https://learn.microsoft.com/en-us/dotnet/core/whats-new/dotnet-8/overview

Avalonia UI https://avaloniaui.net/

SkiaSharp https://github.com/mono/SkiaSharp

Azure CosmosDB https://azure.microsoft.com/en-us/products/cosmos-db

Azure AI Vision https://azure.microsoft.com/en-us/products/ai-services/ai-vision

Azure Speech https://azure.microsoft.com/en-us/products/ai-services/ai-speech

Azure OpenAI https://azure.microsoft.com/en-us/products/ai-services/openai-service

Azure IoT Edge https://azure.microsoft.com/en-us/products/iot-edge

Docker https://www.docker.com/

Mosquito MQTT https://mosquitto.org/

The components used can be found here. You can put this all together yourself!

Some photos of the session at DevConf 2024

I hope to see you next time! Happy building and learning with cool tech!!! 😍

DevConf 2023 – Create the sustainable home farm of your dreams!

Thank you everyone who attended my session at both DevConf Cape Town and Pretoria
I met so many amazing people excited about creating a more sustainable future.

This is a quick blog post which includes the presented slide deck. I hope attendees found the talk useful. It’s a very broad topic and fitting months of content into 40 min is a little challengening. So the presentation is a broad overview of the completed farming project project and there’s links to more deeper content within the slides. This includes content and github repositiories with some source code examples. The plan is to also release ALL the content eventually as a full OSS project when I’m happy with it, including the Mobile App and integrations with the cloud components. I’m currently rounding everything off to do this. So expect a few more posts about this and source code.

The slides can be found below

DevConf YouTube Channel: https://youtu.be/-74owkJbfEQ?list=PLZgTMWtBnKkopU5DZbP7so-f3z7eMFYqq

There is also Agri hacks planned where we can hack and learn about this topic together over a few sessions where you can learn hands on how to build something like this project. This will be a free to attend event held at @CPTMSDUG in Cape Town. Keep an eye out here: https://www.meetup.com/cape-town-ms-dev-user-group/events/

This hack series be similar in format to our current IoT AI hack series currently running. You can see some of the fun had here at the all day iot hack events: https://twitter.com/search?q=%23iotaihackdays&src=typed_query&f=image

Additional Information

I received so many questions on how to get started. I suggest getting the off the shelf kit I mentioned in the talk that I started with. It contains everything you need to grow your first plants. This is also an indoor grower, so you can start with growing from seeds indoor and not worry about weather and protecting the electronics against sun and water. It will also make your home office a lot prettier. 😊

What’s in the kit

Arduino

4 x Solenoids (for 4 plants)

4 x Relays

4 Moisture Sensors

1 Pump

Silicon Piping

The kit is great to learn with as it’s not a “black box” watering system. It’s an arduino where Elecrow provides the source code to flash onto the arduino. It works out of the box, but it also allows you to learn from it and also make changes.

Please don’t start by hacking it like I did. I decided it should to be connected to Azure IoT. That’s definitely not needed especially when starting out. But the instructions are there if you want to do that too. 😊

For the beginner this is great and as you could see in the talk, once plants start multiplying, you are forced to scale from there. 😁

Link to github repository:

https://github.com/apead/SmartWateringKit

Link to blog post on the kit:

Where to buy the kit in South Africa:

https://www.robotics.org.za/AAK90039K?search=smart%20watering%20kit

Once again thanks to everyone that attended both my session and DevConf 2023 and made the conference the amazing conference it was!

BrickPi and EV3 progress update

There has been some nice progress on the pure c# library for the EV3 Lego Mindstorm Brick from Lego and the BrickPi3 from Dexter Industries.  The BrickPi is great as it adds loads of extra processing power and more importantly memory for the EV3 kit using a Raspberry Pi.

 

The library is for both Xamarin.IoT and Android Things.    As soon as it’s ready I’ll post the source to GitHub.   If there is interest, I can also spend some time and port the Android Things implementation to Java or Kotlin for Android Studio.

 

The ultimate plan is to use this in a Lego based robot as a play project, but the experimentation will form basis of an actual larger artificial intelligence project.      It will also be hooked up to Azure IoT for the telemetry.

 

BrickPi Sample

 

BrickPi with 2 sensors

Happy IoT’ing

Azure and the Internet of Things at GXUGSA

Always fun to fly to Johannesburg and speak at GXUGSA.    I hope it was useful.

I’ve made all content available.

 

Event:  https://www.meetup.com/Gauteng-Xamarin-User-Group/events/241783761/

 

Presentation:   https://github.com/apead/XUGSA/tree/master/12092017

 

Android Things Weatherstation (Android Studio):   https://github.com/androidthings/weatherstation

 

Android Things Weatherstation (Xamarin):   https://github.com/apead/Xamarin-AndroidThings-Contrib

 

MX Chip kit Twitter Shake-Shake:  https://microsoft.github.io/azure-iot-developer-kit/docs/projects/shake-shake/

 

MX Chip orders available from DFRobot:  https://www.dfrobot.com/product-1616.html

 

This session featured

 

Azure Functions

Azure Iot Hub

Azure Stream Analytics

Azure Machine Learning

Power BI

Android Things

MX Chip Azure Iot Kit

And other Iot platforms and devices (Arduino, Netduino, Raspberry Pi, Windows 10 Iot, Xamarin Iot)

 

MXChip Iot Kit

 

IoT at GXUGSA

Happy Cross Platform IoT’ing

 

Azure Cape Town – Azure and the Internet of Things

Last night was a fun evening of Azure Iot at the Azure Cape Town user group.   Thanks to all for coming, I hope it was useful.   The turnout was awesome!

 

I’ve made all content available.

 

Presentation:   https://github.com/apead/XUGSA/tree/master/12092017

 

Android Things Weatherstation (Android Studio):   https://github.com/androidthings/weatherstation

 

Android Things Weatherstation (Xamarin):   https://github.com/apead/Xamarin-AndroidThings-Contrib

 

MX Chip kit Twitter Shake-Shake:  https://microsoft.github.io/azure-iot-developer-kit/docs/projects/shake-shake/

 

This session featured

 

Azure Functions

Azure Iot Hub

Azure Stream Analytics

Azure Machine Learning

Android Things

MX Chip Azure Iot Kit

And other Iot platforms and devices (Arduino, Netduino, Raspberry Pi, Windows 10 Iot, Xamarin Iot)

 

 

More Devices!

 

Weatherstation

 

Happy Iot’ing!

Lego Mindstorm EV3 with Xamarin Iot

There’s a lot of fun to be had here!   Xamarin Iot deploying to a Lego Mindstorm EV3 from Visual Studio, isn’t that awesome?

 

Visual Studio 2017 Preview 5 – Xamarin Iot EV3

 

LCD Update via Xamarin Iot

 

I’m now off to make it do something more useful.  I will write a full blog post about it afterwards, including instructions how to set this up yourself.

Have a great day!

Visual Studio: The one IDE to rule them all

This week there was some great news about .NET MF and the return of the Netduino!(http://blog.wildernesslabs.co/netduino-is-back/)

 

I was inspired enough to dust off one of my older projects which included .NET MF and the Xamarin Monkey Robotics project. I then decided to consolidate and rearrange it with some of the newer stuff I’m playing with in the “Iot” space.    It’s also an example of how Visual Studio can handle projects of all types, held together and fully “debuggable” within one solution.     This seems like a good time to do this especially with the recent talks at the Xamarin User Groups in Cape Town (http://explorationspace.co.za/2017/07/20/cross-platform-iot-at-ctxug/)  and Johannesburg (http://explorationspace.co.za/2017/07/12/cross-platform-iot-at-gxugsa/) on the topic.

 

 

Visual Studio Controller of Things Solution

 

I do still have to port all this to Visual Studio 15.3 preview 3 so I can have Xamarin Iot working in the solution as well.

 

Still a lot of on the list to do today, so I’ll end off here.

 

Happy Iot’ing!

Cross Platform Iot at CTXUG

Last night was the Cross Platform Iot Session at CTXUG in Cape Town.    The turnout was absolutely amazing!    Thank you to all for coming it was a really fun evening!

 

Roger Weiss from Aliens kicked off with an overview of Windows 10 Iot Core and a few demos.  These demos featured some awesome use of Microsoft Cognitive services and also some nifty voice controlled home automation.

 

It was then my turn.   Unfortunately Chris van Wyk couldn’t make it this evening.  🙁

 

The session featured

 

Visual Micro for Visual Studio

Visual Studio Code with Arduino Extension which is now Open Source.

Xamarin Forms and Xamarin.Android with Android Things 0.41 Preview

The new Xamarin Iot preview for Linux based devices

Azure Iot Hubs

 

Devices wise 

 

Raspberry Pi 3

Intel Edison

Raspberry Pi Zero W

Latte Panda and Arduino Leonardo

MXChip Azure Kit

Devices everywhere! Latte Panda, Pi With Android Things. NexDock

 

The awesome new MXCHIP Azure Iot kit (Arduino)

 

The presentation can be found here:  https://github.com/apead/XUGSA/tree/master/19072017

Android Things Weather Station Sample with Azure Iot Hubs:    https://github.com/apead/Xamarin-AndroidThings-Contrib

Cross Platform Generic Xamarin Forms sample that ran on Phone, Android Things and Windows 10 Iot (I didn’t demo this, but it works too):   https://github.com/apead/XUGSABuildAutomation

 

Xamarin Android Things Nuget:  https://www.nuget.org/packages/Xamarin.Android.Things/0.4.0-devpreview

 

Xamarin Android Things Contrib Drivers:  https://www.nuget.org/packages/Xamarin.AndroidThings.Contrib.RainbowHat/0.40.0-beta

 

Getting Started with Xamarin Iot:   http://explorationspace.co.za/2017/06/21/xamarin-iot-comes-to-visual-studio-2017-on-windows/

 

MXChip Azure Kit:   https://microsoft.github.io/azure-iot-developer-kit/

 

Android Things Starter Kit with Rainbow Hat:   https://shop.pimoroni.com/products/rainbow-hat-for-android-things

 

Rainbow HAT

 

There’s a Xamagon!

 

Interesting VR Stuff happening in Aliens office

Cross Platform Iot at GXUGSA

Last night was the Cross Iot Session at GXUGSA in Johannesburg.    It’s always fun presenting all the Iot toys, but also what can actually be done with Xamarin and the rest of the Microsoft tooling.

 

It’s also not always clear that you can basically do anything you want with Xamarin, so always nice and exciting to showcase it outside the usual phone type scenarios.

 

 

Gert Talking Windows 10 Iot Core

 

 

The session featured

 

Visual Micro for Visual Studio

Visual Studio Code with Arduino Extension which is now Open Source.

Xamarin Forms and Xamarin.Android with Android Things 0.4 Preview

The new Xamarin Iot preview for Linux based devices

Windows 10 Iot Core

Azure Iot Hubs

 

Devices wise 

 

Raspberry Pi 2 + 3

Intel Edison

Raspberry Pi Zero W

Latte Panda and Arduino Leonardo

MXChip Azure Kit

Devices Devices Everywhere

The presentation can be found here:   https://github.com/apead/XUGSA/tree/master/11072017

 

Android Things Weather Station Sample with Azure Iot Hubs:    https://github.com/apead/Xamarin-AndroidThings-Contrib

Cross Platform Generic Xamarin Forms sample that ran on Phone, Android Things and Windows 10 Iot:   https://github.com/apead/XUGSABuildAutomation

 

Xamarin Android Things Nuget:  https://www.nuget.org/packages/Xamarin.Android.Things/0.4.0-devpreview

 

Xamarin Android Things Contrib Drivers:  https://www.nuget.org/packages/Xamarin.AndroidThings.Contrib.RainbowHat/0.40.0-beta

 

Getting Started with Xamarin Iot:   http://explorationspace.co.za/2017/06/21/xamarin-iot-comes-to-visual-studio-2017-on-windows/

 

MXChip Azure Kit:   https://microsoft.github.io/azure-iot-developer-kit/

 

Android Things Starter Kit with Rainbow Hat:   https://shop.pimoroni.com/products/rainbow-hat-for-android-things

 

Rainbow HAT

 

Behind the Scenes Fun 🙂

First Look at Xamarin IOT

This is a first look at the new Xamarin Iot functionality found within Visual Studio for Mac 7.1 Preview.   I will dig much further into detail in future posts, but lets have a very quick look and it.    I’m really excited about this addition to the already awesome Xamarin tooling!

 

New Iot Project Type

 

A new project type has been added for Iot.    This project type or template will create the solution and project needed to run on the Iot device.

 

Manage devices option

 

What’s immediately apparent is the new run option for the Iot project type.   There’s now a new item called “Manage Devices”

 

Iot device manager

 

The Iot Device Manager allows all the connected devices to be managed.   This setup is very similar to the Mac Agent used for iOS development with the Xamarin Visual Studio for Windows extension, however it will display all the Linux based devices running on the network.  In this case and screenshot it’s a Raspberry Pi Zero running Raspian (Jesse).   Configuring the device will allow remote deploys to the device from your Mac.    This will require a user name and password of a user on the Raspberry Pi.  This user will be used to connect to the Raspberry pi via SSH.    Also note, SSH needs to be enabled on the Raspberry Pi for this to work.

 

Executing remotely on device

 

Now you can start developing your Iot solution.   What’s really awesome is, all the things you expect to work will just work. This includes break points and a full debugging experience.     All this can happen wirelessly to a “headless” device.

 

The build and execution process is very similar to the Mac Agent when you do iOS development from Windows (Visual Studio) to a Mac.  The difference in this case, it is from a Mac (Visual Studio) to a Linux instance (Raspberry Pi).   On the very first execution, the mono run time will be copied over to the device.  This will be used to execute the built app.    The application itself and it’s “built code” is also synchronized with the Raspberry Pi.    This compiled Iot app is then in turn executed on the device remotely.

 

Lets see it in action!    Here is a simple example of the now standard Iot demo of a flashing LED, but running on a tiny Raspberry Pi Zero W.    What’s really nice about this implementation is, it runs on devices that is not supported by other Iot platforms, like Android Things and Windows 10 Iot Core.   It will run on the myriad of Linux based devices out there.

 

 

There is much more to show and tell.  So there’s much more blogs to come on this topic!   This will also include how Android Things and Xamarin Iot can work together to make cross platform Iot simpler.

 

But for now:  Happy Xamarin Iot’ing! 🙂