Friday, 27 January 2017

Firebase Cloud Messaging with Delphi 10.1 Berlin update 2.

A comprehensive step by step guide, covering everything you need to know to receive push notifications to your Android device using Firebase Cloud Messaging and the latest Delphi 10.1 Berlin update 2.

Push notifications let your application notify a user of new messages or events even when the user is not actively using the application (downstream messaging) ( On Android devices, when a device receives a push notification, the application's icon and a message appear in the status bar. When the user taps the notification, they are sent to the application. Notifications can be broadcast to all users, such as for a marketing campaign, or sent to just a subset of users, to give personalised information. To provide this functionality I will rely on Firebase Cloud Messaging which is the new version of GCM (Google cloud messaging) and Delphi to develop the Android application.

1. Create your Firebase project

Create your Firebase project by visiting the console if you still don't have one and in this project create an Android App.

I already have one project so I will use this one for my demo. Once in the project, go to Overview -> Add another app -> Android:

And give it a sensible name. In my case I called the package com.embarcadero.FirebaseCloudMessaging. This package name is important as it will be referenced later on. Once you click Add App, you will receive a google-services.json file which contain information that we will use later.

The package name is defined in your Delphi project:

So make sure that everything matches with the name you give to your Firebase application as the manifest file will contain this information.

2 Request your FCM Token

Now that we have our project configured, we need to request Firebase a unique token for our Android device. You can see the description here as to how to get the FCM token via Android Studio, but I will show the necessary steps to get the same value from our Delphi application.

Basically we are trying to get the same value from FirebaseInstanceId.getInstance().getToken(). We will achieve the same behaviour by using TPushServiceManager which is the unit responsible for handling push notifications.

The following code snippet tries to request the FCM token via TPushServiceManager:

Now, to allow this code to work correctly, we will have to configure few things.

a) Enter the Sender Id.

In the source code snippet above, I'm mentioning the SENDER ID. This sender id, can be found under Firebase -> Project Settings -> Cloud Messaging:

This is the value you have to put here:

PushService.AppProps[TPushService.TAppPropNames.GCMAppID] := 'SENDER ID';

Knowing that the GCMAppId is actually the Sender Id has been a quite a struggle for some users and you can see my answer on Stack overflow.

b) Configure the project to receive push notifications.

In the Delphi IDE, go to your project options -> Entitlement List and set the property Receive push notifications to true.

c) Configure the AndroidManifest.template.xml file.

Before we try to run the code above, we'll have to configure the manifest file to grant our device permissions to connect to Firebase. If you don't configure the permissions, you might run into an exception like the one below:

Error message: EJNIException with message 'java.lang.SecurityException: Not allowed to start service Intent { (has extras) } without permission'.

See the code snipped below for reference:

The full source code of the solution can be found here for reference where you can find the manifest files.

Once everything is configured, we can now test if we can receive the FCM token. Here is a screenshot of my project so you can see that there are two buttons, one to receive the token and the other one to store this token somewhere so the system that sends the notification knows the receiver.

Let's see the project in action here:

As you can see in the image above, I get the DeviceID and the FCM Token. The one we are interested in is the FCM Token. This token is quite large so it does not appear completely on the screen.

Now we need to configure what to do when we receive a notification and how this notification is built.

3 Receive your first FCM Push notification

The following code snipped will configure the OnReceiveNotification event and will display a notification using the TNotificationCenter class.

Notice that the ServiceNotification variable contains a DataKey member which contains a JSON envelope. This object will contain all the information of our push notification. Here you can see how this notification looks like:

Notice that the message is part of the gcm.notification.body property and this one is the one that we will use for our DataObject.GetValue method.

Let's see the application in action:

Here you can see side by side my Android device and Firebase Notification testing system. Once the application is ready to receive notifications, you just need to send the notification using the configured app or the token if you want to target a single device.

Next step is to store these tokens on the cloud and use your own system to deliver those messages.

Please, do not hesitate to contact me if you have any further questions.

Related links:

Embarcadero MVP.

Sunday, 18 December 2016

Creating a publicly accessible REST Web API on a Raspberry Pi3

One of the easiest ways to expose your Raspberry Pi publicly is by using ngrok. Ngrok is a lightweight alternative to VPN that creates introspected tunnels to localhost. If you have been following my articles, you will know that I managed to built a REST Web API to check the status of my Raspberry Pi Grid. The project works on my local network and I also have an Android app also that checks if the Grid is alive or not. But the point of building all this is that I should be able to check the status of it anywhere just to make sure that the system is up and running. The easiest way I could find without having to forward ports on my router is by just running Ngrok. 

Why do you need to expose your Raspberry Pi?

Get the latest version of Ngrok for ARM here and unzip the content of it on your local computer (in my case a Windows 10 machine). Then transfer the unzipped file to your Raspberry Pi using your preferred method (in my case I use WinSCP).

Now that the file is in the Raspberry Pi, we need to give it running permissions using the chmod command (I assume here that you know how to SSH to your RPi using putty):

Now we just need to run the ngrok command: ./ngrok http 3000 and all our http traffic on port 3000 will be automatically forwarded:

Notice that now, Ngrok is giving you a public URL that points to your local Raspberry Pi. This means that Ngrok is providing you with a publicly accessible IP address and domain name to your localhost:

Now that Ngrok is running, I need to run my Web API Service on that port 3000. To do that, I will just launch another session to my Raspberry Pi and start my node.js:

Notice that now I can browse that URL (http:/ and check the status of my Raspberri Pis over the internet:
Notice that every time your stop/start ngrok, a new URL is being generated. As you are exposing your data over the internet you need to make sure that this URL is just known by yourself and that you add extra security on your API if required (Now the Raspberry Pi is able to take requests from anyone on the web!).

Using this approach you can publish your applications easily so other can see them while developing them and you can even set up a home server to share data, etc. Possibilities are endless here.

If you check my mobile application (created with my Delphi 10.1 Berlin update 2) you will see that I just need to point the RestClient component to the exposed URL and voila! we'll be able to access that service over the internet:

All the source code is available here:
  • Mobile application (Delphi 10.1 Berlin update 2).
  • Node.js server (Web Api).
Here is the application in action:

Let me know if you have any questions.

Sunday, 11 December 2016

“Could not load SSL library” Error message on Android 6 Marshmallow with TidHTTP

I have recently upgraded from Android 4.4 KitKat to Android 6 Marshmallow and to my surprise I found out that the some of my applications stopped working. The main issue is related to HTTPS communications using Indy components. As you might know Indy does not implement SSL natively. What the component does is to implement a flexible IOHandler architecture that allows for any SSL implementation to be plugged into Indy. Indy itself implements its own IOHandler class that is based on OpenSSL
The problem is that for Android 6, Google decided not to support OpenSSL anymore. Google replaced OpenSLL with a custom fork called BoringSSL to meet it's needs. BoringSSL includes additional changes and patches to the OpenSSL API and it's not backwards-compatible with OpenSSL. Android 6.0 will use the pre-loaded BoringSSL binaries when any application tries to load the OpenSSL library and it will throw an error (the reason why is because BoringSSL and OpenSSL share the same file names).

If you try to debug your Android application, you will see the following message when trying to use the TIdSSLIOHandlerSocketOpenSSL handler:

How to fix this error?

I found few comments in different forums and in the end the solution was not clear at all and I had to spend few hours fixing the issue myself. The best way to solve this is by deploying a recompiled version of the OpenSSL libraries with your Android application and then tell Indy to use those files. Indy operates at the NDK level so to make Indy avoid BoringSSL you will have to do that.

Get a precompiled version of OpenSSL here:

Place these files in your project and reference them in the deployment section. Go to Project -> Deployment and add the two files that are inside of that zip file and specify the destination folder in your Android device as .\assets\internal.

Now the last thing to do is to tell Indy where to look for these files. To do this, you will have to call the IdOpenSSLSetLibPath method with the TPath.GetDocumentsPath during startup (FormCreate for example). Here is an example:

Once completed, your applications should be able to use the correct version of Indy. You can review my solution and test it here:

Sunday, 27 November 2016

Firebase file streaming with Delphi.

I'm trying different BaaS alternatives as is shutting down by the end of the year. Luckily for us, we have a really cool alternative: Firebase. Firebase is a unified app platform for Android, iOS and mobile web development. It features RealTime Database, user authenticator and hosting to make application development easier. It provides web app and mobile app developers with a way to link their applications to back-end cloud storage. 

In this article I will show you how I interact with the Firebase RealTime Database via the REST API to stream files in JSON format using the latest Delphi 10.1 Berlin update 2.

The following example is just a POC and it will allow you to upload a pdf file via the desktop application, marshall its content into JSON format and then send it via POST to the Firebase Database URL Rest end-point. Then the files are accessed with the (Android) mobile version of it where files are unmarshalled via HTTP Response into JSON format and then transformed back into pdf format. The following diagram explains the process in detail:
The following code shows how to stream the file into JSON format using the TDBXJSONTools.StreamToJSON method:

And to translate it back to a file stream:

The source code of the project can be found on my github repository:
In this repository you will find simple examples using different vendors like, Kinvey and Firebase, exploring different alternatives using HTTP Rest. You can read my previous articles about this topic here:

Firebase will store your documents in the Firebase Database in JSON format:

To use the Firebase Database URL as an end-point you just need to append .json to the end of the URL and send a request from our HTTPS component. For this example, I've created a project in Firebase called delphitestproject. Each request needs to be authenticated and the easiest way to achieve that is by attaching the apiKey from Firebase to your request. There are other options to authenticate your requests but these require exploring in deep firebase service accounts.

To get the apiKey, go to Settings -> Project Settings -> General, and you will see there the web API Key:

You can test your end-point using curl to simulate the requests:

The process to upload a file works as follows:

And the same solution running on my android and then opening the file from there as it has been downloaded from the cloud:

The Delphi app uses the the latest PPL (Parallel programming Library) to avoid freezing the UI. There are separate TTask that are called for the GET/POST request so there is no call in the UI Thread.
When opening files on the Android app, I had to use Android intents to call the correct pdf component. The solution is still incomplete as I need to find a good way to browse files on my Android device and upload this one in the cloud database. From here onward, the sky is the limit. You could streamline the process by storing file Ids so then the whole content is not downloaded every time (which increases bandwidth usage) and  encrypt your pdf files prior to streaming them to the cloud for an additional layer of security.

Remember to explore the source code for more. Enjoy!


Tuesday, 4 October 2016

Improve Code Quality with NDepend

One of the most important things you could do to increase the code quality of your product and bring your team up to speed in terms of code reviews is to leverage the code quality analysis to a third party component. Nowadays I rely on FXCop (a free static code analysis tool from Microsoft that checks .NET managed code assemblies for conformance to Microsoft's .NET Framework Design Guidelines. source: wikipedia) to catch any code smell for me but this time I'm bringing an additional player into the game: NDepend.

NDepend "is a static analysis tool for .NET managed code. This tool supports a large number of code metrics, allows for visualization of dependencies using directed graphs and dependency matrix. The tools also performs code base snapshots comparison, and validation of architectural and quality rules. User-defined rules can be written using LINQ queries. This possibility is named CQLinq. The tool also comes with a large number of predefined CQLinq code rules. Code rules can be checked automatically in Visual Studio or during continuous integration". source: wikipedia.

Review your code!
The best time to find any potential issues in code is as soon as the code is written. "If you let code sit and rot for a while, it won't smell any prettier" by S. Venkat, H Andy on Practices of an Agile Developer.
"Formal code inspections are about as twice as efficient as any known form of testing in finding deep and obscure programming bugs and are the only known methid to top 80% in defect-removal efficiency." by Capers Jones in Estimating Software Costs.
As pointed out, code reviews are the best way to find and solve problems. Make sure you include a code analysis step during your continuous integration/continuous delivery model so issues are spotted earlier rather than later. Below you can see a simple pipeline that I tend to follow for any of my builds. This pipeline only refers to the commit stage and following the best practices to provide fast and provide useful feedback.

When any code is checked-in to git, tfs, subversion, etc your build agent aka TeamCity, Cruise Control, Jenkins, etc. should pick up those changes and start processing that code through the pipeline. During this process, notifications should be sent to the user when something "breaks". In my case, anytime additional code duplicates or code smells increase during the build, the pipeline stops building and reports back to the user with the specific problem. I believe this is crucial to spot any possible issues in the code and that will help the developer to take ownership of the code.

During the Code Analysis step, I rely on FXCop and NDepend to provide a suite of principles and practices that will make the code follow certain standards that are adopted by thousands of other developers. FXCop only provides a fair list of code inspections which are really useful but NDepend can give you more.  Let's see what NDepend can do for you! (note that I'm assuming that you know FXCop quite well).

Integrating NDepend with Visual Studio
NDepend extension is really easy to install and integrate with your VS. Download the latest from here and follow the steps in this video for its integration.

Integrating NDepend with TeamCity
This is the bit that interests me the most. Thanks to the easy integration of NDepend with TeamCity I can leverage the code analysis step to focus on the final code review as the code should reach a common quality standard. To integrate NDepend with TeamCity, you can follow the steps in the following tutorial.

Once configured you should see NDepend under your build runners:
The next step is related to the project configuration itself. NDepend build step expects an NDepend Project file. To create one, you will have to do it in Visual Studio and check it in as part of the solution to your repository. Then TeamCity should see it as part of the source code.

Example of NDepend project as part of the source code:

To create a new project, go to VS and NDepend -> Project -> New Project and create a project under your main solution. Then you'll have to configure the dependencies that you want to attach to the project and once that's done you'll be able to explore the nice report that gets generated out of it.

Example of configuration:

Notice that I had to open the project file to change the paths to relative paths as TeamCity works via relative paths and I was getting an error when running the project file.

Once you have your project configured, run Ndepend from your VS and generate your report:

From this html Report, you can get all the summary metrics about your project and by clicking on them you can drill into the items to expand on those results.

Once you configure NDepend in your TeamCity build step, you will get the same information with the amount of code inspections and errors that are encountered during the build:
The number of inspections appear on the build results so TeamCity can keep track of it overtime:

Once you drill into the build results, you will see the code inspections results (with all the findings from NDepend):
And also the same report you saw in VS:

Now that our system is up and running and the integration of NDepend is done, we can focus on the important stuff...code quality.

One of the things I like the most is the way code rules are set up. It's so easy to enable/disable rules that it's all done in the NDepend Project file. So if you are not agreeing with one particular rule for code quality, just disable it and it won't bother you anymore (try to disable one of the rules for FXCop...and you tell me how you did it and how long it took you..):

Here the full list of code metric definitions.

The main features of NDepend are listed below:
  • Dependency Graph
  • Dependency Matrix
  • Treemap Metric View
  • Abstractness vs. Instability
  • Code Query
Dependency Graph
This very useful diagram allows you to see the relationship between different objects in your solution/s.

Dependency Matrix
The Dependency matrix gives you a coupling score matrix. This will help you identify highly coupled dependencies and if you click on the number, you can see the number of relationships for that particular entry:

Treemap Metric View
In this spectacular view, your source code is mapped into nested rectangles showing methods that are relative to other methods. This will give you a great list of candidates for refactoring. In my case I have few boxes with a cyclomatic complexity of 6 units which tell me that are good candidates for my review and analysis. Once you click on one of this rectangles the source code is shown.

Abstractness vs. Instability
This graph give us a high level overview of our application in terms of where it is heading (too abstract or too unstable).

Here is how to read this chart:

  • If an assembly is very stable (that is, lots of assemblies depend on it) and it's not extensible (no abstract classes, no virtuals, etc) then you're in the lower-left quadrant of the chart in the well-named Zone of Pain.
  • If an assembly is very abstract, very extensible, but no one depends on it (it's not really being used) then it moves towards the Zone of Uselessness.

Code Query
Code Query is NDepend tool that uses its scripting language (CQLinq). This allows users to write their own queries and rules. I like the fact that it is highly customisable and developers love scripting components. I haven't played much with it but I can see the potential for it.

I do believe that NDepend needs to be included in your tool-belt kit. Once you start using it you can't leave without it. Just the amount of information that handles for you is amazing and it gives you that level of confidence that all your developers are writing code with the same standards and code practices out there. It also helps you to spot complexity details that you might miss by just doing an informal review.

Thanks to it's integration with TeamCity it made the decision easy for me as I rely a lot on catching   everything during the commit stage. If any rule is broken, the build agent will inform the developer so he/she can take ownership of the committed changes and there won't be any argument about it. All the rules are set as part of the project and shared as part of the solution. NDepend is about $335 per developer license and about $671 for a build server license. It is highly documented and I found all the help I needed online..the guys from NDepend did really a great job!.

Full Disclosure: I received a complimentary copy of NDepend to provide a review from my point of view. I'm a working professional with loads of years of commercial development experience with hundreds of hours (not to say millions) of code reviews on my shoulders and I don't have time to use useless tools and certainly this is not the case. NDepend is extremely helpful and a great addition to be considered by any development team out there.