Disclaimer: this is an automatic aggregator which pulls feeds and comments from many blogs of contributors that have contributed to the Mono project. The contents of these blog entries do not necessarily reflect Xamarin's position.

June 23

Podcast: WWDC 2017, Fastlane, Xamarin.Forms Embedding, and more!

In this episode of the Xamarin Podcast, I’m joined by James Montemagno, Principal Program Manager for Mobile Developer Tools at Microsoft, to discuss what’s new for Xamarin developers, including our thoughts on WWDC 2017, new features in Visual Studio and Visual Studio for Mac, why Xamarin.Forms Embedding matters, and more. Be sure to download today’s episode from your favorite podcasting app!



Subscribe or Download Today

Keeping up with the latest in .NET, C#, and Xamarin is easier than ever with the Xamarin Podcast! Cohosts Pierce Boggan and James Montemagno cover topics relevant to Xamarin developers, from designing mobile apps to identity management. Be sure to subscribe to the Xamarin Podcast on iTunes, Google Play Music, Stitcher, or your favorite podcast app you use to ensure you don’t miss an episode.

The post Podcast: WWDC 2017, Fastlane, Xamarin.Forms Embedding, and more! appeared first on Xamarin Blog.

June 21

Building Xamarin.Forms Apps with .NET Standard

.NET Standard libraries bring our code to more platforms and give us even more APIs from the Base Class Libraries with which to build amazing apps. Developers have been able to use .NET Standard libraries with Xamarin to share code across platforms, and use Xamarin.Forms to create a shared user interface across iOS, Android, UWP, macOS, for a long time. When combined, .NET Standard and Xamarin.Forms are a match made in heaven for maximizing the amount of code we can share across platforms, both in our underlying business logic as well as our user interface layer.

In this blog post, you’ll learn how to combine .NET Standard libraries and Xamarin.Forms to maximize code sharing and productivity.

What is this .NET Standard thing?

.NET Standard is a formal specification of .NET APIs that are intended to be available on all .NET runtimes (such as .NET Framework, Mono, and .NET Core). In real terms, you can think of this as a simplified, yet expanded, Portable Class Library. Any code added to a .NET Standard library can be used on any runtime that supports the .NET Standard Platform. In addition, we get expanded access to APIs within the .NET base class libraries, support more platforms, and we don’t ever have to deal with the madness that is PCL profiles.

.NET Standard Diagram

Bringing Xamarin.Forms to .NET Standard

The Xamarin.Forms 2.3.5-pre release added compatibility with .NET Standard. In addition to shared projects and Portable Class Libraries, you can now add .NET Standard libraries to your code sharing tool belt.

File -> New

Let’s get started by creating a new Xamarin.Forms app. If you already have an existing Xamarin.Forms app you’re looking to migrate to a .NET Standard, you can skip directly to the next step.

In Visual Studio, click File -> New Project -> Cross-Platform -> Cross-Platform App. Select Master Detail, and click OK. This is going to create a scaffolded mobile app for iOS, Android, and UWP with Xamarin.Forms that displays some data in a list—a common mobile development recipe.

Shows a File -> New Project for a mobile app in Visual Studio.

In Visual Studio for Mac, you can create a similar template by going to File -> New -> Multiplatform -> Forms App.

Adding a .NET Standard Library

Adding a .NET Standard library to your app is super easy; it works just like adding any other library to your project!

If you’re using Visual Studio 2017, right-click the solution and click Add -> New Project. Navigate to the .NET Standard section and add a .NET Standard project.

Add a .NET Standard library to your project.

Developers using Visual Studio for Mac can create a .NET Standard library by navigating to Multiplatform -> Library -> .NET Standard Library. If you’re using Visual Studio 2015, be sure to follow our getting started guide to create and convert a Portable Class Library to a .NET Standard library.

Adding Xamarin.Forms to a .NET Standard Library

Just like Portable Class Libraries, we can add NuGets to .NET Standard libraries. Open the NuGet Package Manager by right-clicking the project and selecting Manage NuGet Packages. Switch to the Browse tab, select Include prerelease, and search for Xamarin.Forms. At the time of this blog post, only the prerelease version of Xamarin.Forms has compatibility with .NET Standard, although it is likely that .NET Standard compatibility will be included in the next stable release.

Add Xamarin.Forms NuGet to a .NET Standard library.

In each of the other projects (such as iOS, Android, UWP, etc.), upgrade to the latest prerelease Xamarin.Forms NuGet, add a reference to the .NET Standard library we created, and remove the reference to the shared project or PCL that contains your Xamarin.Forms UI logic.

Finally, if you’re migrating, copy over files from your shared project or PCL to our .NET Standard library. That’s it! You’re now using a .NET Standard library with Xamarin.Forms to share code:

Using .NET Standard to share code across all the platforms with Xamarin.Forms.

Wrapping Up

In this blog post, we created a brand new cross-platform mobile app and used a .NET Standard library as our code sharing strategy to reach all the platforms. To learn more about .NET Standard for Xamarin (without Xamarin.Forms), check out our blog on .NET Standard support. If you’re still curious as to what .NET Standard is all about, I suggest you check out Immo Landwerth’s introductory video, read up on the documentation on docs.microsoft.com, or follow along on GitHub.

The post Building Xamarin.Forms Apps with .NET Standard appeared first on Xamarin Blog.

June 20

Minnesota Twins Hits a Home Run with Cloud-Powered Scouting Apps

Rapid development cycles and outstanding mobile experiences are a competitive advantage, and we’re always happy to learn how our customers use .NET to deliver five-star apps. With high stakes requirements and an aggressive timeline, the Minnesota Twins recently released its first ever mobile app to field scouts and internal teams—and end users love it. Now, scouts who travel around the world to evaluate players are free to work offline, capture game notes in real time, and automatically sync information with the convenience of mobile devices.

Jeremy Raadt, Senior Developer of Baseball Systems for the Minnesota Twins, joins us to share how his team uses Xamarin for Visual Studio and Visual Studio for Mac, Azure, and Team Foundation Server to streamline development and construct apps that deliver new experiences for any device or internet connectivity.

Tell us a little bit about your company and role.

The Minnesota Twins has a reputation for innovation, on and off the field. I joined a few years ago to collaborate with the Baseball R&D team, which acts as a service group for our entire baseball department. We create technology that helps our business partners make better decisions: delivering and maintaining data driven solutions to evaluate players, improve player performance, and provide a range of statistical analytics. There’s so much data and information from a variety of sources, and we bring it all together in a cohesive platform to produce intelligence, not just facts and figures.

I’ve been a software developer for 20 years, mostly for the web, and I’ve been lucky to learn many languages and platforms, from Cold Fusion to Java to NodeJS to ASP.NET. Fifteen years ago, I got my start in “mobile” building custom applications for Palm Pilots and other devices. At that time, mobile development was incredibly difficult and time consuming.

Tell us about your app and what prompted you to build it. What does it allow users to accomplish?

Minnesota Twins app homescreenWe’ve put a huge amount of effort into building a great web platform that allows the baseball department to store data and to perform research and advanced analytics. However, many of our users, such as scouts and coaches, are often are not in an office; they’re traveling to high schools, colleges, minor league, major league, and international baseball games. Being on the road, they’re also plagued with spotty internet connections. We needed the ability to coordinate schedules, deliver information, and analyze data between scouts, coaches, and front office staff, whenever and wherever they are. Mobile allows us to extend our powerful web-based system, using the same APIs that connect our data store to our web app, to deliver features and functionality tailored specifically for mobile use, such as offline access, real time updates, notifications, and data sync.

We started with a project to give amateur scouts (scouting high school and college players) the ability to view game schedules based on area, importance, and various other factors. Historically, coordinating coverage at key events consisted of a massive email barrage across dozens of scouts. Now, scouts filter the most important games with a simple click and always have the most up-to-date game details so they can ensure they’re watching the best players.

This app is a game changer for our organization, and we plan to continue releasing more game changing features.

Why did you choose Xamarin? What alternatives did you consider? 

Since scouts use a variety of mobile devices, we have to support a wide range of platforms, operating systems, and hardware manufacturers. We wanted a framework that was highly performant today and would allow us to grow for years to come. I was in a similar situation in a prior role, and we initially tried HTML5, but quickly ran into performance issues. Our small bootstrapped team was forced to rewrite in Java and Obj-C, and we were never able to deliver all the features we wanted with the quality we demanded.

Fast forward to the Minnesota Twins: I knew trying to manage separate codebases wouldn’t work for our small team, who also supports our web application (which has an increasing number of features on the roadmap). So, I searched for a better way.

We needed a long term solution, that provided us:
Schedule on the Minnesota Twins app 

  • Native performance: A smooth, performant user experience. Lagging and slow load times would make us look less professional and we’d lose credibility. With hundreds of thousands of datasets, including players, stats, reports, game details, and more, we demanded high performance. To increase adoption from our scout and front office personnel, our mobile experience needed to feel extremely professional and load information quickly.
  • Reliability and security: Users needed to feel safe and confident adding their (highly confidential) scouting information, proprietary algorithms, and all the other data that’s an invaluable part of our competitive advantage. Like any company with highly sensitive data, we needed to ensure the highest levels of security. Xamarin and Microsoft provide not only the framework, but monitoring tools as well.
  • Consistent user experience: We needed high fidelity access to the camera, GPS, notifications, and other device-specific capabilities that felt, looked, and behaved as users expected.

 
 
Xamarin intrigued me right away, due to its ability to provide native support across device platforms while using our C# skills (our web app is built on ASP.NET Core). After exploring and really testing its performance, I was impressed. With our first version, we not only have native performance, we’re also sharing 99% code across Android and iOS thanks to Xamarin.Forms.

How did you get up to speed with cross-platform mobile development?

Xamarin University was invaluable; even though I knew C#, there are nuances and patterns that help you build better apps. I spent three weeks completing the Xamarin Certification courses, and we also had the Xamarin team onsite, working side-by-side to help us build a rock solid foundation. This acted like a jet pack, accelerating us ahead by months.

We (developers, along with our stakeholders, and upper management) were shocked when we had a working prototype in just five days. We knew immediately that we’d made the right decision, and one that would pay dividends in the future.

Describe your development process. 

Minnesota Twins app schedule on iOSWe started prototyping on Monday. By Friday, we showed our stakeholders and management a working prototype. Without Xamarin, we’d still be working on an early prototype. Honestly, I’m not sure we’d have started creating a mobile experience, since we don’t have the resources to learn Obj-C and Java and support two mobile codebases along with a 3rd code base for our web application.

I credit our speed to two things: (1) we share almost 99% code across Android and iOS. This is huge for us, since we’re a small team, and we need to support the app and continuously deliver new features and (2), since we integrate with Azure AD for authentication, the libraries, documentation, and simple set up made it easy to get up and running quickly without worrying about building authentication ourselves.

Currently, our team codes our web and mobile apps and manages our DevOps pipeline. It’s key to automate as much of the process as we can, so we can concentrate on business-specific items and delivering features to users. We can’t afford downtime since our users are all over the world, in various time zones, working 24/7.

To achieve this, our team uses both Visual Studio for Mac (Preview) and Xamarin tools for Visual Studio, which is another nice thing about the platform. Our team has developers on Mac and Windows platforms so we can bring up our favorite environments and it just works. In addition to Xamarin, we store our data in SQL Server, and an ASP.NET Core API connects to both our web and mobile apps. Team Foundation Server (TFS) acts as our source control, build, and deployment platform (currently we build and deploy our web and API from TFS, but when time permits, we’re planning to move mobile). We use Visual Studio Mobile Center (Preview) to distribute our apps to beta testers and internal stakeholders.

Tell us more about how you’re using Microsoft Azure.

Most notably, we use Azure AD, which allows our scouts to use the same credentials, whether they’re accessible via web or mobile. For our upcoming video capture functionality, we’re using Azure Media Services, and we’re looking into Microsoft Cognitive Services for voice recognition, speech-to-text, and more.

What’s the most compelling or exciting aspect of your app? 

For me, it’s creating something that is invaluable to our scouts; making sure they’re at the right games, with the right information. If a top prospect is announced to pitch in a game tomorrow, we need the right scouts to know about it right away.

We’re delivering a high quality, impactful experience, and enabling never-before-possible scenarios, now and with our future releases.

What have your users said about your app? Your boss? 

They absolutely love it! Scouts have the information they need immediately searchable at their fingertips, instead of lost in email, and they’re excited for what’s next.

We’re bombarded with requests and ideas for new features, which tells us that they’re enthusiastic and that our work has been a success. Our leadership team sees the potential and how fast we can build and deploy new features, and they’ve placed a high priority on its continued development.

In fact, the project has gotten so much attention that we’re growing our team to help us continue expanding on our mobile vision. One of the requirements: Xamarin mobile development skills.

We’ve accomplished our initial goal, proving a small team can provide a world class mobile experience, and now we have many new goals that we’re aiming to tackle.

What’s your team planning to build next?

We have many features planned, with our immediate development focused on video capture (allowing scouts to capture and share from anywhere and at any time) and making advanced player stats and game locations available offline. Video capture is extremely powerful for scouting; watching and sharing video clips gives us a feel for players that just isn’t possible with text.

The entire Scouting Department and I are excited for what’s possible and where we can take mobile unique capabilities.

What advice do you have for developers who are just starting out or investigating mobile development? 

Start small. Download and try some Xamarin apps to get a feel for how performant they are, and prepare to be amazed!
 
 
Visit xamarin.com/customers to learn how our customers across industries and geographies deliver amazing mobile apps, and visit xamarin.com/download to start building your own.

The post Minnesota Twins Hits a Home Run with Cloud-Powered Scouting Apps appeared first on Xamarin Blog.

June 19

Add Push Notifications to iOS Apps with Azure Notification Hubs

Push Notifications have become ubiquitous in mobile apps since they were first introduced. One of the major advantages of push notifications is being able to deliver a contextual notification without having the app manually check (or poll) a server; this saves a lot of battery life due to reduced CPU usage. Another key advantage of push notifications is the ability to keep users engaged with the app when it’s not running in the foreground.

Azure Notification Hubs allow you to send push notifications to any platform from any backend service in a scalable manner. Unlike setting up a custom delivery system, combining Azure with Xamarin.iOS is simple.

In this blog post, we’ll run through all the steps required to set up and configure a Notification Hub on Azure and implement the code required in a Xamarin.iOS app.

To complete this tutorial, you must have an active Azure account. If you don’t have an account, you can create a free trial account in only a couple of minutes. You can sign up for an Azure Free Trial here.

Signing and Provisioning

Unlike local notifications, you must register your App ID with Apple and enable Push Notifications in order for them to work; this is also the case even when just running a debug build.

To set up the certificates and provisioning profiles required, follow this part of our guide on the Apple Push Notification System.

Follow the guide to the end until you have a Personal Information Exchange (PKCS12) certificate (.p12 file). You will need this to set up the Azure Notification Hub in the next step.

Azure Set Up

Head over to the Azure Portal and sign in. Once logged in, click on “+ New” and search for “Notification Hub”. Next, we need to fill out some information, such as the name for the hub, a unique namespace, server location, and a name for your resource group. Once done, click create and wait for the hub to be set up by Azure; this normally takes no longer than a couple of minutes.

Once the Notification Hub is ready, go to “Manage” then choose “Notification Services”, and then “Apple (APNS)”. Here, we need to upload the .p12 file that we created earlier; make sure Authentication Mode is set to “Certificate”, select the file, enter the password for the certificate (if set), and choose “Sandbox” for the Application Mode. Click “Save” and that’s it! The Notification Hub is set up and ready to use for development!

While still in the Azure portal, we need to grab the EndPoint string for use later on. You can find this under “Manage” > “Access Policies” and copy the connection string for the DefaultListenSharedAccessSignature policy.

The Code

Now we need to add some code to our app so we can receive push notification from the Notification Hub. With Xamarin.iOS, this is really simple; start by adding the Xamarin.Azure.NotificationHubs.iOS NuGet package to your project by right-clicking the “Packages” folder and selecting “Add Packages…”

Following this, we need to add some code to AppDelegate.cs to register for notifications and handle them when received. At the top of the file add:

using WindowsAzure.Messaging;


Next, we need to declare a SBNotificationHub and the connection parameters at the class level:

private SBNotificationHub Hub { get; set; }


public const string ConnectionString = "Endpoint=sb://…";

public const string NotificationHubPath = "AzurePushHub";


The ConnectionString is the string we copied from Azure previously and the NotificationHubPath is simply the name of your Notification Hub.

Now we need to register our preferred notification settings for the app and register to receive push notifications. Add the following code in FinishedLaunching.

var settings =  UIUserNotificationSettings.GetSettingsForTypes( UIUserNotificationType.Alert |  UIUserNotificationType.Badge | UIUserNotificationType.Sound, new NSSet());


UIApplication.SharedApplication.RegisterUserNotificationSettings(settings);

UIApplication.SharedApplication.RegisterForRemoteNotifications();


Here, we’re specifying that by default, unless overrided by the user, we want to display alerts, a badge on the app icon, and play a sound when a notification is received.

iOS will now go ahead and register the app for remote push notifications. It will let us know this is complete by calling RegisteredForRemoteNotifications; we need to override this to handle the result and register the device with our Azure Notification Hub.

public override void RegisteredForRemoteNotifications(UIApplication application, NSData deviceToken)
{
  // Create a new notification hub with the connection string and hub path
  Hub = new SBNotificationHub(ConnectionString, NotificationHubPath);
  // Unregister any previous instances using the device token
  Hub.UnregisterAllAsync(deviceToken, (error) =>
  {
    if (error != null)
    {
      // Error unregistering
      return;
    }
    // Register this device with the notification hub
    Hub.RegisterNativeAsync(deviceToken, null, (registerError) =>
    {
      if (registerError != null)
      {
        // Error registering
      }
    });
  });
}

Once this is done, iOS will automatically display received alerts for your app when the app is backgrounded. If we want to handle alerts when the app receives a push notification, we can use ReceivedRemoteNotification:

public override void ReceivedRemoteNotification(UIApplication application, NSDictionary userInfo)
{

  // This method is called when a remote notification is received and the

  // App is in the foreground - i.e., not backgrounded


  // We need to check that the notification has a payload (userInfo) and the payload

  // has the root "aps" key in the dictionary - this "aps" dictionary contains defined

  // keys by Apple which allows the system to determine how to handle the alert

  if (null != userInfo && userInfo.ContainsKey(new NSString("aps")))
  {

    // Get the aps dictionary from the alert payload

    NSDictionary aps = userInfo.ObjectForKey(new NSString("aps")) as NSDictionary;


    // Here we can do any additional processing upon receiving the notification


    // As the app is in the foreground, we can handle this alert manually

    // here by creating a UIAlert for example

  }


}


If you want to run code when a notification is received while the app is backgrounded, you can do so by using DidReceiveRemoteNotification

Testing

Deploy your app to your iOS device from Visual Studio. Once deployed, hit the home button to send it to the background. Next, go to the Azure Portal, open your Notification Hub, and go to “Support + Troubleshooting” > “Test Send”.

From here we can send a test notification to our app, so set the platform to “Apple” and click “Send”. It may take up to a minute to see the notification appear on your device, depending on connectivity, but usually it will be received in a matter of seconds. The result should look like this:

Summary

Whether it’s sending updates, reminders, messages or more, using remote notifications in your apps is a powerful way to keep users engaged. Using an Azure Notification Hub with Xamarin.iOS makes this a simple task and allows you to leverage all the other benefits provided by Azure. Azure is able to scale with your app, can quickly send millions of messages, and can be integrated into almost any back-end service, whether it’s Microsoft .NET, PHP, Java, or Node.js (to name just a few).

As well as the standard use cases, you can also use push notifications to update your app in the background by using silent notifications. Find out more about this by reading our documentation.

The full code for this blog post can be found on the GitHub repo for this post.

The post Add Push Notifications to iOS Apps with Azure Notification Hubs appeared first on Xamarin Blog.

June 16

Easy iOS App Provisioning with Fastlane and Visual Studio for Mac

iOS developers know that app and device provisioning is a huge pain. Just to deploy an iOS app to a device for the first time, you must request a development certificate, generate a signing key, add a device in the Apple Developer Center, and create a development provisioning profile for the app that includes your device in the list of approved devices to deploy to. That was a mouthful.

We wanted to make it as easy as possible for you to get started with iOS development by streamlining this key pain point of app and device provisioning. Visual Studio for Mac now integrates with the open source project fastlane, which was created to simplify the app provisioning process from app and device provisioning to team provisioning management. In this blog post, you’ll learn how to quickly provision a new device and app in less than five minutes with the click of a few buttons.

Easy Provisioning with Fastlane

The fastlane toolchain has been integrated into Visual Studio for Mac 15.3. To access these features, simply change the updater channel to Alpha. Install fastlane from GitHub with HomeBrew, the installer script, or via Rubygems. To provision your apps, you’ll need to be a member of the Apple Developer Program.

File -> New Project

The best time to provision apps is at creation, so we’ve integrated fastlane directly into our File -> New Project wizard in order to ensure every app you create is properly provisioned. In Visual Studio for Mac, select File -> New Project -> iOS -> Single View App. Click Add Team.

File -> New Project dialog where you can now add an iOS developer team.

The Apple Developer Accounts tab within the Preferences window will appear. This is where you can manage all of your developer accounts across all of the apps you’re working on. Click the + icon to add your account. Enter your Apple ID and password to enable fastlane to integrate with your iOS developer account.

Sign into your Apple Developer Account using fastlane from Visual Studio for Mac.

From this dialog, you can create certificates and manage provisioning profiles for all teams you’re a member of. Click OK and return to the File -> New dialog. Select your team from the dropdown, click Next, and click Create.

Create a Signing Identity

Right-click the iOS app you’ve just created, and select Options -> iOS Bundle Signing. The team you selected during the File -> New Project wizard is automatically set as the default team. Next to the Signing Identity heading, click the dropdown, and select Create signing identity. This automatically creates a new signing identity for you and associates it with your Apple developer account.

Generating a signing identity with fastlane and Visual Studio for Mac.

It’s worth noting that all setup up until this point only needs to be done once per Visual Studio for Mac installation.

Create a Provisioning Profile

Now that we have a signing identity, it’s time to create a provisioning profile for your app. In the iOS Bundle Signing tab, select the Provisioning Profile dropdown and click Create provisioning profile. In less than ten seconds, we have successfully provisioned our new app.

iOS app provisioning in Visual Studio for Mac.

Provision a Device

To be able to deploy to a device, we must provision the device and “attach” it to the provisioning profile we just created. Normally, we would go to the Apple Developer Portal to do this, but Visual Studio for Mac provides a way to provision new devices directly within the IDE.

Plug your device into your Mac via the Lightning to USB cable. If the device attached is not associated with the provisioning profile selected, Visual Studio for Mac will suggest you provision the device. Click Register, followed by Add, and Visual Studio for Mac will add the device to your Apple developer account and associate it with the selected provisioning profile.

What’s Next?

In this blog post, we successfully generated a development provisioning profile for a new app. Publishing iOS apps (either Ad Hoc for testing or for release in the App Store) requires a separate distribution certificate and provisioning profile. Now in Visual Studio for Mac 15.3, you can easily create distribution signing identities for use in your app publishing workflow without leaving the IDE:

Publishing workflow with fastlane.

Wrapping Up

Visual Studio for Mac’s fastlane integration makes it super easy for you to provision apps and devices directly from the IDE with just a few clicks. For more information, read our documentation on using the fastlane tools with Xamarin. In the future, we will be working to bring easy app and device provisioning to Visual Studio on Windows and continue to integrate additional features of fastlane directly into the IDE.

The post Easy iOS App Provisioning with Fastlane and Visual Studio for Mac appeared first on Xamarin Blog.

June 15

Plastic SCM development cycle - key practices described

So far, we shared how we do DevOps and Trunk Based Development using Plastic SCM and a set of tools around it. But I wanted to make the blogposts short, so I left lots of notes and explanation intentionally out for the sake of brevity.

In this 5th blogpost in the series, I will cover why automated testing is so important for us, but also 2 other practices we adopted long ago: code review and explore test each task.

June 14

5 Ways to Boost Xamarin.Forms App Startup Time

The Xamarin SDKs for iOS and Android provide an extremely performant foundation upon which to build Xamarin.Forms cross-platform applications. As you work to tune the speed and responsiveness of your Xamarin.Forms application, keep in mind that the same principles that improve the experience of your apps built with the Xamarin SDKs for iOS and Android will also improve those going cross-platform with Xamarin.Forms—and that experience begins at startup.

When you say Startup…

I’ll first define startup as everything that happens from the time the user hits your app icon to when the first usable screen appears ready for business. When you compare a blank Xamarin.Android app with a blank Xamarin.Forms app, you’ll notice the Xamarin.Android app loads quicker. What’s going on?

Xamarin.Forms provides not only the ability to share UI across targets, but common application features such as AppLinks for deep linking, Navigation, MessagingCenter, and DependencyService, as well as several pieces needed for cross-platform UI alerts, action sheets, toolbars, status bars, etc. When you start adding comparable implementations to a Xamarin.Android app, you start to see the difference in performance disappearing. This begins to explain what’s going on, so what’s in your control to impact startup? Here are my top five tips to tune your startup experience across iOS, Android, and UWP targets.

1. Load Local Content First

We often want to load fresh data as soon as our application is launched, but this will only slow down the first screen from appearing and populating. If you have state from a previous run then use that, or have content ready to go that is for first run.

If you application is already offline ready, and we recommend it is, then you’re all set to populate the content for your first screen instantly. Azure Mobile Apps and Realm with Azure are great options.

When you must go to the web for your first run content, provide some UI telling the user what you’re doing by using a loading indicator. This is also a perfect opportunity to replace the blank part of your screen with useful placeholder content to reinforce why this screen exists, what the user can expect from the app, or just something fun! Whatever your choice here, just don’t leave a screen empty.

Payload Tips Minimize your web payload to just what you need. Don’t spend time on expensive processes like parsing strings, rather deliver your data as it needs to be represented.

2. Optimize Your Assets

Images and video are heavy and can drastically slow down the initial rendering of a screen. Reduce reliance on heavy assets and optimize any media you use for the necessary dimensions. Don’t rely on the OS to resize the assets for you.

For Android target screens, assets are placed in folders representing their density.

  • ldpi (low) ~120dpi
  • mdpi (medium) ~160dpi
  • hdpi (high) ~240dpi
  • xhdpi (extra-high) ~320dpi
  • xxhdpi (extra-extra-high) ~480dpi
  • xxxhdpi (extra-extra-extra-high) ~640dpi

There’s also a special density of nodpi that tells Android to not scale the resources regardless of screen density.

When you put a large image in the ldpi folder and use that asset on a screen of xxhdpi, Android will scale it up. This is slow and will bloat your runtime memory consumption, sometimes even to the point of crashing your application; I’ve done it.

To manage that many densities and that many assets, it’s highly beneficial to use an application such as Sketch and/or Zeplin to generate multiple sizes from a single design source.

Xamarin.Android 7.0 recently added an experimental option to pre-crunch PNG files called AndroidExplicitCrunch. This further reduces the size of your assets (and application), speeds up build time, and ensures they are optimized for runtime. To enable this, edit the text of your Android csproj and amend the PropertyGroup for your build configuration:

<PropertyGroup>
    ...
    <AndroidExplicitCrunch>true</AndroidExplicitCrunch>
</PropertyGroup>

3. Lazy Load Anything You Don’t Need Immediately

The App.xaml Resources is a convenient place to put styles, fonts, and other resources that you’ll use throughout your application, but it’s also all loaded at startup. If you’re trying to erase every unnecessary millisecond from your startup time, remove anything here that you don’t need and lazy load it by page or in another method.

4. Enable XAML Compilation

XAML is a popular and expressively powerful way to declare your user interface. If you opt to stay in C# and build your UI there, it’s naturally compiled along with the rest of your code, giving you compile time checking and speed. When you want that same benefit while using XAML, you enable XAML Compilation (XAMLC). This will compile your XAML into Intermediate Language (IL) and add it to your compiled assembly, resulting in faster startup and runtime performance. Let’s look at how you can enable this:

At the application level you may declare your XAMLC options and it will impact your entire app.

using Xamarin.Forms.Xaml;
...
[assembly: XamlCompilation (XamlCompilationOptions.Compile)]
namespace PhotoApp
{
    ...
}

When you want to set XAMLC options at the class level, you can!

using Xamarin.Forms.Xaml;
...
[XamlCompilation (XamlCompilationOptions.Compile)]
public class HomePage : ContentPage
{
    ...
}

When XAMLC is enabled you’ll get compile time XAML checking on most all properties and UI will render faster. File size may remain unchanged or grow slightly as you are trading .xaml file size for IL in the assembly.

5. Reduce Number of Assemblies

Convenience comes at a cost, as usual. It might be a very small cost, but you want to leave no stone unturned when tuning for performance. While NuGet packages are awesome for leveraging other libraries, the reality is that the more (and larger) assemblies your mobile application depends on naturally slows down the execution as calls pass across boundaries. Xamarin.Forms for example inspects all assemblies for `[ExportRenderer]` attributes and currently has no method to opt-in or opt-out. This is something we’re working to improve.

Weigh the pros/cons of each dependency and, when possible, bring that code into your main application. Your mileage may vary, so give it a look. This is an often overlooked performance consideration that may pay dividends for you.

Bonus 1: Ahead of Time Compilation (AOT)

I’m calling this bonus because we still classify AOT as experimental on Android and it won’t apply to everyone. Of course, iOS uses AOT and the LLVM compiler by default, so there’s nothing to do there.

Enable the experimental Android AOT and get immediate improvement in startup as well as overall speed by reducing some Just-In-Time compilation overhead. The price paid is an increase in the size of your APK, so you should try this out and weigh the pros and cons for your application. To enable AOT, open your project configuration and check the box.


Note: AOT is available on Xamarin Android 5.1 and 7.0+

Bonus 2: Build Time Improvements

While not directly applicable to startup improvement, one of our awesome mobile solutions architects Brandon Minnick has shared his collection of build optimization configurations. Check it out on GitHub.

What’s Next

These are just a few of my favorite tips, and they can apply to all screens of your application, not just the startup screen. I could go on about flattening layout, optimizing measuring in layout cycles, and more. Much of that and more is covered in our Xamarin.Forms Performance guide.

The community and Xamarin.Forms engineering team have been trading tips and discussing pain points and possible solutions on the Xamarin Forums. Please join that conversation to share your findings, and follow along as we continue to explore improving startup and runtime performance across all platforms in Xamarin.Forms.

The post 5 Ways to Boost Xamarin.Forms App Startup Time appeared first on Xamarin Blog.

Monitoring a Plastic server performance

In this blog post we will explain how you can easily analyze your Plastic SCM Server logs to get very useful information of our server behavior and identify potential issues.

The Plastic SCM installation provides a tool (plasticlogstats.exe) that is able to parse your server logs and generate detailed reports.

June 13

Demystifying Build Configurations

I’m sure everyone is familiar with the Debug and Release configurations in most solution templates in use. It seems obvious to use the Debug configuration when building your project for debugging and to use the Release configuration when building your project for release. What may be less clear, though, is what’s actually happening when you switch configurations, or how to add your own custom configurations. Today, we’re going to dive a little deeper into the world of build configurations to see how they can greatly improve your build times and productivity.

What Is a Configuration?

Simply stated, a configuration is a set of project properties. In the default Debug configuration, for instance, debugging will be enabled, while in a Release configuration it will not be. You can set what linker options to use in each configuration, what architectures (CPUs) to build for, etc. You can also further define platforms so that you can have separate project settings depending on whether you are building for a device or emulator. In other words, you can set separate project settings for each Configuration|Platform combination that you have available in your project.

How Can I Edit a Configuration?

To illustrate how to edit, and later add, a configuration we’ll start with a template Android project:

In VS for Mac create a new “Android App”: New Project > Android > App > Android App.
In VS for Windows create a “Single-View App (Android)”: Create new project… > Android > Single-View App (Android).

The template creates a simple Android app with a button that counts clicks. It has two configurations set up by default, Debug and Release, and one platform, AnyCPU. These are simply names that don’t have any inherent meaning, except being reflective of when the configuration should be used. We’re going to add two new configurations named DebugDevice and DebugEmulator later on. These will allow us to quickly set the desired project options/properties when debugging to a device (ARM) or an emulator (x86). Each will only build for the architectures needed for a device or an emulator, reducing build time during development.

First, let’s see what’s different between the two default Debug and Release configurations. To do so, open the project properties by right-clicking on the project in the solution explorer and selecting “Options” (VS for Mac) or “Properties” (VS for Windows). Once the project options/properties are open, you’ll see configuration and platform fields towards the top with drop down lists to select which configuration and platform you want to see the property settings for.

Project Options

Project Properties

The drop down may not be there (VS for Mac) or disabled (VS for Windows), which means that the settings on the settings page you have selected are global, i.e. they will apply to all configurations and platforms. You can see this for the Build > General settings page in VS for Mac and in the Application settings page in VS for Windows.

Most of the differences in the project settings for the default Debug and Release configurations are in the Android Build / Android Options settings page shown above. Open the project options/properties and select the Android Build (VS for Mac) / Android Options (VS for Windows) on the left. Make sure Debug is selected at the top in the configuration drop down. Note that the “Use Shared Mono Runtime” on VS for Mac or “Use Shared Runtime” in VS for Windows is selected, as well as the “Fast Assembly Deployment” (VS for Mac) / “Use Fast Deployment” (VS for Windows) option.

These are great options when you are debugging as they will decrease build and deploy times when developing an app, but they are not at all good for building a release version as it is unlikely that an end user will have a shared mono runtime installed on their Android devices so the app would fail to run at all since there would be no runtime. For Release builds you would want to disable that option. If you now select Release in the configuration drop down at the top, you will see that the default Release configuration has those options disabled.

You can check the Linker behaviour of the respective configurations:

In VS for Mac: In the Android Build page select the “Linker” tab. You will see the “Linking Behaviour” is set to “Don’t Link” for Debug configuration and “Link SDK assemblies only” for Release configuration.
In VS for Windows: In the Android Options page scroll to the “Linker Properties” section. You will see the “Linking” option is set to “None” for Debug configuration and “SDK assemblies only” for Release configuration.

These default options allow for faster build during debugging and a smaller app bundle for your release build.

The “Advanced” tab (Mac)/button (Windows) allows you to set which CPUs (ABIs) to build for when building for Debug or Release. You will note that by default the Debug configuration builds for all possible CPUs. Every CPU you build for will take time, so it might be nice to streamline this so we are only building for the CPU we will be running the app on. If we are only testing on an x86 emulator, then we can only build for x86 and deselect the other ABIs that are not needed. Go ahead and deselect all but x86 using the Advanced tab/button.

Congratulations, you have now just edited a configuration. Editing a configuration is as simple as opening the Project Options/Properties, selecting the configuration you want to change settings for, and changing those settings.

How Can I Add a Configuration?

Wouldn’t it be nice if you could easily choose which architectures to build for during debugging simply by selecting a configuration? If debugging to a device I choose one configuration, DebugDevice, and for building to an emulator I choose DebugEmulator. Let’s get that set up.

Visual Studio for Mac:

To add a configuration in Visual Studio for Mac:

  1. Open the Solution Options by double clicking on the solution in the solution explorer.
  2. Select Build > Configurations on the left (not Run > Configurations).
  3. Since we want a variant of the Debug configuration, in the configuration list select Debug and click “Copy”. This will copy the current Debug settings to your new configuration.
  4. Enter the name for this configuration, i.e. DebugDevice.
  5. Keep the default Any CPU for the Platform.
  6. Click “OK”.
  7. Repeat steps 4-8 for the DebugEmulator configuration.

Visual Studio (Windows):

To add a configuration in Visual Studio (Windows):

  1. Open the Configuration Manager with the Build > Configuration Manager menu.
  2. Click on the Active Solution Configuration drop down and select “New”.
  3. Enter DebugDevice for the Name field.
  4. Select Debug in the “Copy settings from” drop down list.
  5. Make sure the “Create new project configurations” option is checked.
  6. Click OK.
  7. Repeat steps 2 – 6 using DebugEmulator for the Name field.

Now you will see four configurations in the list, Debug, DebugDevice, DebugEmulator, and Release.


Now you just have to open the Android project options and make the desired changes to these two new configurations:

  1. Open the Android project options.
  2. Select the Android Build (Mac) / Android Options (Windows) > Advanced tab.
  3. Select DebugDevice from the Configuration drop down.
  4. Uncheck all of the Supported ABIs except armeabi-v7a.
  5. Select DebugEmulator from the Configuration drop down.
  6. Uncheck all of the Supported ABIs except x86.

You can now select DebugDevice whenever you want to debug using an actual Android device, and you can select DebugEmulator whenever you want to debug to an x86 emulator. Your debug builds will be faster since you are only building for the one needed architecture.

Congratulations, you have now added two configurations and set them up for specific debugging scenarios.

How Can I Add a Platform?

What we did above works, but it may be better to define platforms instead of configurations to switch between building for device or emulator. You will notice this is done by default for iOS projects. There are two platforms defined for iOS projects, iPhone and iPhoneSimulator. We could do the same for Android if we like. For this exercise, start with a new Xamarin Forms project, PCL or Shared type does not matter for this exercise.

Visual Studio for Mac:

To add a platform in Xamarin/Visual Studio for Mac:

  1. Open the Solution Options by double clicking on the solution in the solution explorer.
  2. Select Build > Configurations on the left (not Run > Configurations).
  3. Select the General tab.
  4. Since we want variants of the Debug configuration, in the configuration list select Debug and click “Copy”. This will copy the current Debug settings to your new configuration.
  5. Enter the name for this configuration, in this case keep it the same, Debug, since we only want to add a platform not a configuration.
  6. Enter AndroidDevice for the Platform field.
  7. Make sure the “Create Configurations for all solution items” option is not checked. We only want this platform for the Android project. If this option were selected, we would be adding this platform to the iOS project as well.
  8. Click “OK”.
  9. Repeat steps 4-8 using AndroidEmulator for the Platform field.
  10. Since we did not add this new platform to the projects (step 7), we will need to repeat steps 4-8 in the Android Project Options > Build > Configurations. Do this for both the AndroidDevice and AndroidEmulator platforms.

You will then have to map the solution custom platform to the project custom platform:

  1. Open the Solution Options > Build > Configurations > Configuration Mappings tab.
  2. Select the AndroidDevice platform for the Platform field at the top.
  3. In Configuration column for the Android app project, select Debug|AndroidDevice.
  4. Select the AndroidEmulator platform for the Platform field at the top.
  5. In Configuration column for the Android app project, select Debug|AndroidEmulator.

Now in the main configuration drop down list you will see Debug|AndroidDevice and Debug|AndroidEmulator.

You can edit the settings for each platform just as you did for the configurations we set up earlier, just select the correct platform that you want to edit the build settings for in the Platform drop down in the Project Options > Android Build (Mac) settings page.

Visual Studio (Windows):

Adding custom platforms is not as straight forward in Visual Studio as it is in Xamarin Studio. You can easily add the custom platform name at the solution level, but adding it at the project level requires some editing of the .csproj file. Here’s how to do all this in Visual Studio:

Add a solution level platform:

  1. Open the Configuration Manager with the Build > Configuration Manager menu.
  2. Click on the Active Solution Platform drop down and select “New”.
  3. Enter AndroidDevice for the Name field.
  4. Select AnyCPU in the “Copy settings from” drop down list.
  5. Make sure the “Create new project configurations” option is not checked. You will get an error if this is checked since the Android app project does not have an AndroidDevice platform yet.
  6. Click OK.
  7. Repeat steps 2 – 6 using Androidmulator for the Name field.

Now for the hard part, adding the new platforms to the Android app project.

  1. Unload the Android app project (right click on the project and select Unload).
  2. Edit the AppName.csproj file (right click on the project and select “Edit .csproj”.
  3. Paste the following XML right below the opening Project tag:
    
      AndroidDevice
      AnyCPU
    
    
      AndroidEmulator
      AnyCPU
    
  4. Look for the following element:
    <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
  5. Select all of the text for the above PropertyGroup including the opening and closing PropertyGroup tag and copy it.
  6. Paste the text twice right below the ending PropertyGroup tag copied above.
  7. Edit the pasted property groups, changing:
    to
    for the first copy and
    for the second copy.
  8. Not required, but best to have a separate OutputPath for each platform, so change:
    bin\Debug
    to
    bin\AndroidDevice\Debug
    for the first copy and
    bin\AndroidEmulator\Debug
    for the second copy.
  9. Save and close the .csproj file.
  10. Reload the project.

You will then have to map the solution custom platform to the project custom platform:

  1. Open the Configuration Manager again.
  2. Select the AndroidDevice platform for the Active Solution Platform.
  3. In Platform column for the Android app project, select AndroidDevice.
  4. Select the AndroidEmulator platform for the Active Solution Platform.
  5. In Platform column for the Android app project, select AndroidEmulator.

Now you will want to open the Android project properties and edit the Configuration|Platform settings to only build for the relevant architectures, i.e. armeabi-v7s for AndroidDevice and x86 for AndroidEmulator, like you did earlier for the new configurations.

Configuration Mapping

When we added the configurations above, we added them at the solution level but also added them to each project in the solution by selecting the “Create Configurations for all solution items” on Mac and “Create new project configurations” in Windows (or added them manually in the .csproj file). This way the new configurations are available at the project and the solution level. You can then map solution level configurations to project level configurations using the Configuration Manager in Windows or on Mac using the Solution Options > Build > Configurations > Configuration Mappings tab. You will see that for each project, you can select which project configuration will build when you select the active solution configuration in the Configuration drop down list.

You can also set which projects build with each solution configuration. If you have a solution with multiple app projects, you can add a configuration to only build the project(s) needed at the time. For a Xamarin Forms app (or any solution with multiple platform projects), you may want to add a configuration that only builds the iOS project (and PCL project if used), and one to only build the Android project (and PCL project if used). This would again speed up build times, so you are only building for the platform you are currently debugging.

Configurations and MSBuild Properties in Your .csproj File

We touched briefly on the .csproj file when adding custom platforms to an app project in Visual Studio since the UI provides no way of doing that as in Xamarin Studio. Mostly you will never have to edit a .csproj file but I feel it is useful to at least touch on the items in the .csproj file that are affected when you switch the active Configuration|Platform in the IDE.

When we added the custom platform above, we copied and pasted a PropertyGroup twice to add the new platforms and set the build output path for the new platforms. The property groups that are used for each configuration is simply a set of MSBuild properties and values that will be used when the corresponding Configuration|Platform combination is selected for build. For instance, the following property group will be used when the Debug|AnyCPU  Configuration|Platform is selected:


...

When the condition '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' is met, then the values for the MSBuild properties in that group will be used. For instance, you will find the below in the above property group:

None

This sets the Linker options to “None” when using Debug|AnyCPU Configuration|Platform combination. Your configurations and platforms are really just a way for you to set a bunch of MSBuild properties to specific values just by choosing a Configuration|Platform for your build. If building from the command line, you can manually set any available MSBuild property using the /p:MSBuildProperty=value syntax. In other words, if you wanted to build the Debug|AnyCPU Configuration|Platform from the command line but wanted to enable the linker option “Link SDK Assemblies Only”, you would do the following (Windows):

MSBuild.exe MySolution.csproj /p:Configuration=Debug /p:Platform=AnyCPU /p:AndroidLinkMode=SdkOnly

…or on Mac:

MSBuild MySolution.csproj /p:Configuration=Debug /p:Platform=AnyCPU /p:AndroidLinkMode=SdkOnly

Any property you see defined in a PropertyGroup in the .csproj file can be set to a specific value in the command line with the /p:PropertyName=PropertyValue switch when doing a command line build. Here is the MSBuild Command Line reference for more information.

Summary

You have now learned how to create custom configurations and platforms and how to edit them so that the build options you want for a particular scenario can all be set just by selecting the appropriate configuration | platform combination.

The post Demystifying Build Configurations appeared first on Xamarin Blog.

June 12

Adding Face Tracking and Live Recognition to your Android App

Camera apps today can do so much more than just take perfect photos. Whether it’s adding filters to your image or letting you adjust focus and exposures manually, apps can essentially transform you into a professional photographer. While numerous apps in the app store let you do many things with the camera, there are others that do clever manipulation of your images in real time, such as adding a custom mask to your face, manipulating your face to look older or younger, and more. Social media apps today have this intelligence baked in, so you don’t have to open another app to make your selfies look extraordinary before you share them with friends and family.

Whether you’re building a consumer app that provides fun filters or an enterprise app that recognizes receipts, adding intelligent tires like these gives you an edge over others. With services like the Microsoft Cognitive Services API, you can quickly analyze and process your photos by adding just few lines of code.

In this post, you’ll learn how to display a camera within the Android app, add face tracking using Google’s Mobile Vision APIs and recognize the person in front of the camera using Microsoft Cognitive Services’ Face API.

Face Detection

Adding Face Tracking to Your App

Google’s Mobile Vision provides a set of APIs for “Detecting Face”, “Recognizing Text”, and “Scanning Barcodes”. For this example, I’ll be using the Face APIs to detect human faces from the live camera stream within the app.

To get started, add the Xamarin.GooglePlayServices.Vision NuGet package.

To access the camera on Android, you need to request the user’s permission and add the camera feature to your Android manifest:

if (ActivityCompat.CheckSelfPermission(this, Manifest.Permission.Camera) == Permission.Granted)
{
    CreateCameraSource();
    //...
}
else
{
Log.Warn(TAG, "Camera permission is not granted. Requesting permission");
var permissions = new string[] { Manifest.Permission.Camera };
if (!ActivityCompat.ShouldShowRequestPermissionRationale(this,
        Manifest.Permission.Camera))
{
    ActivityCompat.RequestPermissions(this, permissions, RC_HANDLE_CAMERA_PERM);
    return;
}
Snackbar.Make(mGraphicOverlay, Resource.String.permission_camera_rationale,
        Snackbar.LengthIndefinite)
        .SetAction(Resource.String.ok, (o) => { ActivityCompat.RequestPermissions(this, permissions, RC_HANDLE_CAMERA_PERM); })
        .Show();
}

Create Camera Source

The FaceDetector.Builder instantiates the FaceDetector with the specified properties. In this example, we use the default face detector settings and associate a processor to it.

private void CreateCameraSource()
{
    var context = Application.Context;
    FaceDetector detector = new FaceDetector.Builder(context)
            .SetClassificationType(ClassificationType.All)
            .Build();
    detector.SetProcessor(
            new MultiProcessor.Builder(this)
                    .Build());
    if (!detector.IsOperational)
    {
        // Note: The first time that an app using face API is installed on a device, GMS will
        // download a native library to the device in order to do detection.  Usually this
        // completes before the app is run for the first time.  But if that download has not yet
        // completed, then the above call will not detect any faces.
        //
        // isOperational() can be used to check if the required native library is currently
        // available.  The detector will automatically become operational once the library
        // download completes on the device.
        Log.Warn(TAG, "Face detector dependencies are not yet available.");
    }
    mCameraSource = new CameraSource.Builder(context, detector)
            .SetRequestedPreviewSize(640, 480)
            .SetFacing(CameraFacing.Front)
            .SetRequestedFps(30.0f)
            .Build();
}

In the above code, a MultiProcessor is used to receive detection results and CameraSource.Builder instantiates a camera source with a specified Preview Size, Camera Facing, and required fps.

Show a Live Stream from the Camera

Now that we have access to the camera and the CameraSource is already built, we can start the preview on a custom SurfaceView; in this case CameraSourcePreview.
GraphicOverlay instance is also passed in so that the face border can be drawn at the time detection.

private void StartCameraSource()
{
    // check that the device has play services available.
    int code = GoogleApiAvailability.Instance.IsGooglePlayServicesAvailable(
            this.ApplicationContext);
    if (code != ConnectionResult.Success)
    {
        var dlg =
                GoogleApiAvailability.Instance.GetErrorDialog(this, code, RC_HANDLE_GMS);
        dlg.Show();
    }
    if (mCameraSource != null)
    {
        try
        {
            mPreview.Start(mCameraSource, mGraphicOverlay);
        }
        catch (System.Exception e)
        {
            Log.Error(TAG, "Unable to start camera source.", e);
            mCameraSource.Release();
            mCameraSource = null;
        }
    }
}

The ideal place to call the StartCameraSource() is in the OnResume() of the Activity, while the OnPause() should stop the preview. It will ensure the camera resources are used only when the user is using the app.

Detect Faces

Every image frame received from the camera source may contain multiple faces, and each face corresponds to a distinct face identity which is represented by the Tracker created by the multiprocessor.

Implement IFactory to capture each Face

MultiProcessor requires an implementation of IFactory to callback when a human Face is detected in the camera. In this example, IFactory is implemented within the MainActivity, that has the Create():

public Tracker Create(Java.Lang.Object item)
{
    return new GraphicFaceTracker(mGraphicOverlay, mCameraSource);
}

In the above code, a new instance of GraphicFaceTracker is created for every detected face, and each of it builds a face box graphic object over the video stream using the GraphicOverlay view passed in the constructor.

Below is the implementation of the GraphicFaceTracker

class GraphicFaceTracker : Tracker
{
    private GraphicOverlay mOverlay;
    private FaceGraphic mFaceGraphic;
    public GraphicFaceTracker(GraphicOverlay overlay)
    {
        mOverlay = overlay;
        mFaceGraphic = new FaceGraphic(overlay);
    }
    public override void OnNewItem(int id, Java.Lang.Object item)
    {
        mFaceGraphic.SetId(id);
    }
    public override void OnUpdate(Detector.Detections detections, Java.Lang.Object item)
    {
        var face = item as Face;
        mOverlay.Add(mFaceGraphic);
        mFaceGraphic.UpdateFace(face);
    }
    public override void OnMissing(Detector.Detections detections)
    {
        mOverlay.Remove(mFaceGraphic);
    }
    public override void OnDone()
    {
        mOverlay.Remove(mFaceGraphic);
    }
}

The FaceGraphic instance is created when a Face is detected the first time, updated as the face changes, and hidden when the face goes out of the frame.

That’s it; we successfully created face tracking on a custom camera surface within your app! Next, we’ll recognize the person within the frame.

Recognize a Face from the Live Stream

To recognize the face within the live video frame, we capture the image whenever a new face is detected and send it to Microsoft Cognitive Services APIs to identify the person. Face Recognition requires artificial intelligence and highly efficient machine learning algorithms, which are provided to you as a service and free to get started. If you’re new to Cognitive Services, I highly recommend reading the blog post Adding Facial Recognition to Your Mobile Apps.

Capture the Detected Face

To capture the newly detected Face, modify the GraphicFaceTracker to implement the CameraSource.IPictureCallback first.

class GraphicFaceTracker : Tracker, CameraSource.IPictureCallback
{
	//...
}

And modify the OnNewItem() to capture the image frame with the Face

public override void OnNewItem(int id, Java.Lang.Object item)
{
    mFaceGraphic.SetId(id);
    if (mCameraSource != null && !isProcessing)
        mCameraSource.TakePicture(null, this);
}

Identify the Person in the Captured Frame

Finally, using the Helper Classes ImageAnalyzer and LiveCamHelper (that abstracts the COGs API call), we successfully identify the person in the video stream.

public void OnPictureTaken(byte[] data)
{
    Task.Run(async () =>
    {
        try
        {
            isProcessing = true;
            Console.WriteLine("face detected: ");
            var imageAnalyzer = new ImageAnalyzer(data);
            await LiveCamHelper.ProcessCameraCapture(imageAnalyzer);
        }
        finally
        {
            isProcessing = false;
        }
    });
}



Wrapping Up

Face recognition is widely used in many scenarios, including security, natural user interface, robotics, and more. By using a combination of these services and platform APIs, you can build unmatchable mobile apps that are intelligent and provide five-star user experiences. In this blog post, we used Google Mobile Vision APIs to detect human faces from the Video Live Stream and Microsoft Cognitive Services to recognize the person within the frame. For building the same sample on iOS, read How To Display Camera in Your iOS apps and Adding Facial Recognition to Your Mobile Apps.

The sample built and explained in this post was inspired by the Google Developer documentation on Face Tracker.

Download the samples for both iOS and Android from my GitHub repo. Note: For this demo, I’ve written the code to recognize a single face in the frame. However, feel free to modify and make it better.

The post Adding Face Tracking and Live Recognition to your Android App appeared first on Xamarin Blog.

Monologue

Monologue is a window into the world, work, and lives of the community members and developers that make up the Mono Project, which is a free cross-platform development environment used primarily on Linux.

If you would rather follow Monologue using a newsreader, we provide the following feed:

RSS 2.0 Feed

Monologue is powered by Mono and the Monologue software.

Bloggers