Using ONNX BERT Model for Text-based Q&A in your Mobile .NET Apps

In the first article of this series, I explained how to load, evaluate and convert a PyTorch-trained BERT QnA model to an ONNX-compatible format. In the second article, I explained how to load an ONNX Model, perform processing tasks on both the input and output and connect the prediction pipeline to the frontend UI. In this article, I will give a high-level overview of how to load the converted ONNX BERT model trained using PyTorch. Getting Started The BERT QnA model aims to recognize the context in each text input by looking in both forward & backward directions and provide a text-based human-readable answer as output. This code base provides a more interactive feel by allowing users to input text as a question and context, and the BERT ONNX model will reply with a text-based answer. NuGet Packages Used The following NuGet packages were installed in the Uno platform application project for this code base. Microsoft ML OnnxRuntime Skia Sharp BERT Tokenizers INFERENCING WITH PyTorchBertQnA The TensorFlowMNISTClassifier C# file houses the class-based code for loading, processing, and prediction tasks. STEP 1: Load Embedded Resources In the 1st article of this series, we mentioned the large .onnx file that was created

Add ML-Assisted Handwritten Digit Recognizer with C# to your Mobile Apps

This article will explore loading a pre-trained ONNX model, trained on the popular MNIST dataset, into an application built with the Uno Platform. By loading a pre-trained ONNX model, we can leverage the power of machine learning to add advanced functionality to our mobile, .NET,  and Uno Platform-built applications, such as image classification or handwriting recognition. We will provide a step-by-step guide on how to load an ONNX model into an Uno Platform application and explain how to use the model to make predictions on new data. By the end of this article, you will have a solid understanding of how to integrate machine learning into your Uno Platform applications. To get a better understanding of training, evaluating and converting an ML model trained with the MNIST dataset to an ONNX-compatible format or how to load an ONNX Model, perform processing tasks on both the input and output and connect the prediction pipeline to the frontend UI, check out Part 1 and Part 2 of this series. Sample App Getting Started The objective of the MNIST classifier model is to recognise handwritten numerical digits from 0-9. The model accepts as input an image of size 14 * 14 and outputs

MOBILENET for On-Device Inference in Uno Platform Applications

The previous article introduced the ONNX, an open standard for exchanging and sharing deep learning models which can allow developers to facilitate on-device inference. The article showcased the conversion of 2 none ONNX models (the first built and trained from scratch and another pretrained) to the ONNX format. It went through a high-level overview of how to train, evaluate, convert and test an ML model in TensorFlow formats, PyTorch format & ONNX format.  This article will go through the key steps from a C# Uno Platform developer’s perspective on leveraging ONNX runtime to achieve local (on-device) inference on an Uno platform application. A bonus is readers will see how to set up and use a control from Uno Platforms Material Toolkit.  Introduction In this article’s sample code base, which is an Uno Platform application, the ML task achieved is image classification using an open-source ONNX format model called Mobile NET. What is MOBILENET MobileNet is a collection of Convolutional Neural Networks designed for efficient on-device image classification and other tasks. Google researchers developed the architecture to enable deep learning models to run efficiently on mobile and edge devices with limited computing resources. MobileNet models are trained to recognize a wide

Building Smarter .NET Applications with ONNX ML Models

One of the most recent and exciting areas in mobile application development is the integration of machine learning models to add intelligent capabilities. With the ability to process large amounts of data, identify patterns, and make predictions, ML models can enhance the functionality and usability of mobile applications in various ways, such as image recognition, natural language processing, and personalized recommendations. Furthermore, with frameworks such as ONNX, it has become easier to integrate machine learning models into mobile .NET applications, making it a viable option for developers who want to create smart and personalized user experiences. This 4 part series will outline how to consume your pre-trained models (Trained using TensorFlow & PyTorch) in a .NET, Uno Platform built application. Each article will explore one of the following machine-learning tasks:  Part 1: Prepping ONNX Models for Uno Platform Part 2: Image classification using ONNX format model called Mobile NET. Part 3: Loading the ONNX model trained with the MNIST dataset. Part 4: How to add Text-based Q&A with ONNX BERT model trained using PyTorch. This series’ primary objective is to illustrate, in steps, how to consume an ML Model in an Uno Platform Application and not to teach you how

Building Apps for MS Teams in C# and XAML with Uno Platform

In this blog, we’ll explore how Uno Platform and WebAssembly make it possible to develop apps for Microsoft Teams using C# and XAML. And how they can help streamline your workflows, automate tasks, and bring new capabilities to your team. By the end of this tutorial, you will have a simple sample app running in web version of Microsoft Teams, similar to our Getting Started tutorial. (Create a Single Page App with Uno Platform) Prerequisites 1. Visual Studio 2022 includes built-in support for Microsoft Teams and provides useful starter templates, but they are not installed by default with any workloads. Instead, we need to select them in the Visual Studio Installer manually. Open the Visual Studio Installer and click Modify on your favourite Visual Studio 2022 instance. Next, select the ASP.NET and web development workload and then find and check the Microsoft Teams development tools option in the Installation details pane. 2. While you can technically test your Microsoft Teams on your production tenant, creating a developer account in the Microsoft 365 Developer Program is preferable. First, go to the official website here and click Join. Next, log in using your Microsoft or company account and fill out the requested information. When successfully logged in, click the Set up E5

Replicating Pet Adoption UI with Uno Platform and Figma

Figma is notably regarded as a design tool. However, it is also easy for developers to learn and use, even with limited design experience. Furthermore, the Uno Platform for Figma plugin allows you to export your designs directly into code, saving time, streamlining the development process and improving the consistency of designs. This tutorial will walk you through creating a Pet UI mobile screen and exporting your Figma designs into code, including setting up your Figma file and designing using Uno Material Toolkit components. By the end of this tutorial, regardless of being a developer or designer, you will have a solid understanding of using Figma to create and export designs into code. Let’s get started! Pet Adopt UI Before Getting Started Install the Uno Platform Plugin Copy the Uno Material Toolkit Set up your Canvas and Screen Start by opening your file, selecting the cover page, and renaming it. 2. Continue to the Getting Started page. This is where you will want to set up all aspects of your app’s theme (Color styles, Font styles). These can always be changed later, but best practice would recommend you do it at the onset of your project. Note: all the

XAML Fundamentals for Web & Mobile: Adding Custom Sorting Logic

Nick Randolph’s earlier post in this blog series covered the basics of sorting and grouping. The CollectionViewSource provides a wrapper around a data source that can indicate if the source is grouped and will maintain the selected item. Suppose you are familiar with CollectionViewSource from WPF. In that case, you may not be aware that the class no longer has functionality for sorting and grouping itself in UWP, but this can be achieved in the underlying source. The WinUI/Uno version follows the UWP functionality. CustomSorting Sample App Sorting Collections .NET has a set of functionalities for querying collections, and they make up the System.Linq namespace. This adds several extension methods to standard .NET collection classes to allow querying, sorting, and grouping. LINQ stands for Language INtegrated Query, a declarative query syntax right in your C# code. You can perform operations using the query syntax or directly using the fluent OrderBy, GroupBy, etc. methods provided. LINQ Fluent When you are using intrinsic types such as Int32 (aliased as “int” in C#), there is a predictable behavior for sorting because these types implement the IComparable<T> interface. This interface defines a single method CompareTo which returns a value indicating whether the other instance is less than,

Building a Profile Page with Uno Platform for Figma

In this tutorial, we’ll build a completely functional Profile Page using Figma and Uno Platform for Figma plugin to generate responsive XAML code that can be built upon and extended quickly. Profile Page – Playground File Breaking Down the UI The screen can be broken down into four sections, each with its own set of components: User Info Image Text Call Log IconButtons TextBlock Action Buttons IconButtons Bottom Menu IconButtons Text Now that we understand the layout of our UI and the components we are using, the next step is picking our theme as an essential design element. It’s best practice to set up your colors at the start of the project. Editing Color Styles in the Figma Toolkit File To edit your colors: In the Styles Panel (right side of the screen when nothing is selected), expand the Colors folder and subfolders to view all color roles used in the document. Select one that needs an update to follow your brand colors. Right-Click and Select Edit. From the Properties section of the Edit Style Panel, input the HEX code you want, which will update throughout the entire document. You can always edit them at any point in the project.

Replicating Food Delivery App UI with Uno Platform

Welcome to this article! 👋 Together, we will develop a mobile user interface using Uno Platform XAML. As you might know, the XAML Uno Platform uses is 100% aligned with XAML used for WinUI and UWP applications. Even if you are more familiar with other XAML variations, such as Xamarin.Forms / MAUI or WPF, you will find it very familiar. It is the default XAML used by modern Windows applications.  Since introducing our Figma plugin, Uno Platform has slowly shifted to introducing Figma as the starting point for new project product design and high-fidelity prototyping. However, using Figma is not a default entry point for developing UI with Uno Platform. On the contrary, we have been deploying consumer-facing mobile apps using only XAML for over 13 years with our parent agency, nventive. With this article, we’d like to show you what you can do with Uno Platform alone. Food Delivery Sample App We will divide this article into different sub-topics, which are as follows: Analyzing the visual structure of the user interface. Code explanation: We will see step by step each one of the components obtained from the analysis, and we will learn how to convert it to XAML code.