Android: Fast Communication with .NET Using Protocol Buffers

Improve the performance of interprocess communication between Android and .NET using fast binary serialization (Protocol Buffers) instead of XML or JSON.

Related article: Android: How to communicate with .NET application via TCP

Introduction

One of challanges in the interprocess communication when communicating across platforms (e.g. between Android and .NET) is how to encode (serialize) messages so that they can be understood by both platforms. Default binary serializers provided by platforms are not compatible and so the common solution is to encode messages into some text format e.g. XML or JSON.
This can be perfectly ok in many cases but for applications with high performance expectations using the text format may not be an optimal solution.

The example bellow demonstrates how to use Protocol Buffers binary serialization in the interprocess communication between Android and .NET applications.

The whole example can be downloaded from here.

You Need to Download

• Eneter.ProtoBuf.Serializer - protocol buffer serializer for Eneter, it also contains protocol buffer libraries and utility applications for 'proto' files.
• Eneter.Messaging.Framework - communication framework that can be downloaded for free for non-commercial use.

If you like here are project pages for Protocol Buffers:

• protobuf - Google implementation of Protocol Buffers for Java, C++ and Python.
• protobuf-net - Protocol Buffers implementation from Marc Gravell for .NET platforms.
• Eneter.ProtoBuf.Serializer - Open source project to integrate Protocol Buffers and Eneter Messaging Framework.

Add Following References into your Project

Into .NET project:

• protobuf-net.dll - protocol buffers serializer for .NET, Windows Phone, Silverlight and Compact Framework developed by Marc Gravell.
• Eneter.ProtoBuf.Serializer.dll - implements serializer for Eneter Messaging Framework using protobuf-net.dll.
• Eneter.Messaging.Framework.dll - lightweight cross-platform framework for interprocess communication.

Into Android project:

• protobuf.jar - protocol buffers serializer for Java and Android developed by Google.
• eneter-protobuf-serializer.jar - implements serializer for Eneter Messaging Framework using protobuf.jar from Google.
• eneter-messaging.jar - lightweight cross-platform framework for interprocess communication.

Important: please follow this procedure to add libraries into the Android project:
(To add a library into the project you need to import it instead of adding it via project properties.)
(Java compliance level must be set to 6.0. Properties -> Java Compiler -> JDK Compliance -> 1.6.)

1. Create a new folder 'libs' in your project. (use exactly name libs)
2. Right click on 'libs' and choose 'Import...' -> 'General/File System' -> 'Next'.
3. Then click 'Browser' button for 'From directory' and navigate to directory with libraries you want to add.
4. Select check boxes for libraries you want to add.
5. Press 'Finish'

Protocol Buffers

Protocol Buffers is a binary serialization originally developed by Google to share data among applications developed in different languages like Java, C++ and Python. It became the open source and was ported to other languages and platforms too.

The biggest advantage of Protocol Buffers is its performance and availability on multiple platforms what makes it an alternative to consider when designing the communication between applications.

If you are interested a simple performance measurement is available at

https://code.google.com/p/eneter-protobuf-serializer/wiki/PerformanceMeasurements.

Working with Protocol Buffers

The following procedure is optimized for defining messages for cross-platform communication:

(If you want to use Protocol Buffers only in .NET you do not have to declare messages via the 'proto' file but you can declare them directly in the source code by attributing classes - same way as using DataContractSerializer.)

1. Declare messages in the 'proto' file.
2. Compile the 'proto' file into the source code (C# and Java). It transforms declared messages to classes containing specified fields and the serialization functionality.
3. Include generated source files into C# and Java projects.
4. Initialize Eneter communication components to use EneterProtoBufSerializer.

Example Code

The example bellow is exactly the same as in my previous article Android: How to communicate with .NET application via TCP.

The only difference is the code in this article uses EneterProtoBufSerializer instead of XmlStringSerializer.

Please, for details about how to use TCP on Android and how to setup the IP address in the emulator refer to Android: How to communicate with .NET application via TCP.

proto File

The 'proto' file represents a contract describing messages that shall be used for the interaction.
Messages are declared in the platform neutral 'protocol buffer' language - for the syntax details you can refer to https://developers.google.com/protocol-buffers/docs/proto.

Messages in our example are declared in MessageDeclarations.proto:

// Request Message
message MyRequest
{
    required string Text = 1;
}

// Response Message
message MyResponse
{
    required int32 Length = 1;
}

The 'proto' file is then compiled to C# and Java source code. Declared messages are transformed to classes containing declared fields and serialization functionality.

The following commands were used in our example to compile the 'proto' file:
protogen.exe -i:MessageDeclarations.proto -o:MessageDeclarations.cs
protoc.exe -I=./ --java_out=./ ./MessageDeclarations.proto

Android Client Application

The Android client is a very simple application allowing user to put some text message and send the request to the service to get back the length of the text.

When the response message is received it must be marshaled to the UI thread to display the result.

The client uses EneterProtoBufSerializer. In openConnection() method it instantiates the serializer and puts the reference to the DuplexTypedMessagesFactory ensuring the message sender will use Protocol Buffers.

Note: To add the Eneter library into the Android project please follow the recommended procedure.

The whole implementation is very simple:

package net.client;

import message.declarations.MessageDeclarations.*;
import eneter.messaging.dataprocessing.serializing.ISerializer;
import eneter.messaging.diagnostic.EneterTrace;
import eneter.messaging.endpoints.typedmessages.*;
import eneter.messaging.messagingsystems.messagingsystembase.*;
import eneter.messaging.messagingsystems.tcpmessagingsystem.TcpMessagingSystemFactory;
import eneter.net.system.EventHandler;
import eneter.protobuf.ProtoBufSerializer;
import android.app.Activity;
import android.os.Bundle;
import android.os.Handler;
import android.view.View;
import android.view.View.OnClickListener;
import android.widget.*;

public class AndroidNetCommunicationClientActivity extends Activity
{
    // UI controls
    private Handler myRefresh = new Handler();
    private EditText myMessageTextEditText;
    private EditText myResponseEditText;
    private Button mySendRequestBtn;
    
    
    // Sender sending MyRequest and as a response receiving MyResponse.
    private IDuplexTypedMessageSender<MyResponse, MyRequest> mySender;
    
    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState)
    {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
        
        // Get UI widgets.
        myMessageTextEditText = (EditText) findViewById(R.id.messageTextEditText);
        myResponseEditText = (EditText) findViewById(R.id.messageLengthEditText);
        mySendRequestBtn = (Button) findViewById(R.id.sendRequestBtn);
        
        // Subscribe to handle the button click.
        mySendRequestBtn.setOnClickListener(myOnSendRequestClickHandler);
        
        // Open the connection in another thread.
        // Note: From Android 3.1 (Honeycomb) or higher
        //       it is not possible to open TCP connection
        //       from the main thread.
        Thread anOpenConnectionThread = new Thread(new Runnable()
            {
                @Override
                public void run()
                {
                    try
                    {
                        openConnection();
                    }
                    catch (Exception err)
                    {
                        EneterTrace.error("Open connection failed.", err);
                    }
                }
            });
        anOpenConnectionThread.start();
    }
    
    @Override
    public void onDestroy()
    {
        // Stop listening to response messages.
        mySender.detachDuplexOutputChannel();
        
        super.onDestroy();
    } 
    
    private void openConnection() throws Exception
    {
        // Instantiate Protocol Buffer based serializer.
        ISerializer aSerializer = new ProtoBufSerializer();
        
        // Create sender sending MyRequest and as a response receiving MyResponse
        // The sender will use Protocol Buffers to serialize/deserialize messages. 
        IDuplexTypedMessagesFactory aSenderFactory = new DuplexTypedMessagesFactory(aSerializer);
        mySender = aSenderFactory.createDuplexTypedMessageSender(MyResponse.class, MyRequest.class);
        
        // Subscribe to receive response messages.
        mySender.responseReceived().subscribe(myOnResponseHandler);
        
        // Create TCP messaging for the communication.
        // Note: 10.0.2.2 is a special alias to the loopback (127.0.0.1)
        //       on the development machine.
        IMessagingSystemFactory aMessaging = new TcpMessagingSystemFactory();
        
        IDuplexOutputChannel anOutputChannel
            = aMessaging.createDuplexOutputChannel("tcp://10.0.2.2:8060/");
            //= aMessaging.createDuplexOutputChannel("tcp://192.168.1.102:8060/");
        
        // Attach the output channel to the sender and be able to send
        // messages and receive responses.
        mySender.attachDuplexOutputChannel(anOutputChannel);
    }
    
    private void onSendRequest(View v)
    {
        // Create the request message using ProtoBuf builder pattern.
        final MyRequest aRequestMsg = MyRequest.newBuilder()
                .setText(myMessageTextEditText.getText().toString())
                .build();
        
        // Send the request message.
        try
        {
            mySender.sendRequestMessage(aRequestMsg);
        }
        catch (Exception err)
        {
            EneterTrace.error("Sending the message failed.", err);
        }
        
    }
    
    private void onResponseReceived(Object sender,
                                    final TypedResponseReceivedEventArgs<MyResponse> e)
    {
        // Display the result - returned number of characters.
        // Note: Marshal displaying to the correct UI thread.
        myRefresh.post(new Runnable()
            {
                @Override
                public void run()
                {
                    myResponseEditText.setText(Integer.toString(e.getResponseMessage().getLength()));
                }
            });
    }
    
    private EventHandler<TypedResponseReceivedEventArgs<MyResponse>> myOnResponseHandler
            = new EventHandler<TypedResponseReceivedEventArgs<MyResponse>>()
    {
        @Override
        public void onEvent(Object sender,
                            TypedResponseReceivedEventArgs<MyResponse> e)
        {
            onResponseReceived(sender, e);
        }
    };
    
    private OnClickListener myOnSendRequestClickHandler = new OnClickListener()
    {
        @Override
        public void onClick(View v)
        {
            onSendRequest(v);
        }
    };
}

.NET Service Application

The .NET service is a simple console application listening to TCP and receiving requests to calculate the length of a given text.

The service uses EneterProtoBufSerializer. It instantiates the serializer and puts the reference to the DuplexTypedMessagesFactory ensuring the message receiver will use Protocol Buffers to deserialize incoming messages and serialize response messages.

The whole implementation is very simple:

using System;
using Eneter.Messaging.DataProcessing.Serializing;
using Eneter.Messaging.EndPoints.TypedMessages;
using Eneter.Messaging.MessagingSystems.MessagingSystemBase;
using Eneter.Messaging.MessagingSystems.TcpMessagingSystem;
using Eneter.ProtoBuf;
using message.declarations;

namespace ServiceExample
{
    class Program
    {
        private static IDuplexTypedMessageReceiver<MyResponse, MyRequest> myReceiver;

        static void Main(string[] args)
        {
            // Instantiate Protocol Buffer based serializer.
            ISerializer aSerializer = new ProtoBufSerializer();

            // Create message receiver receiving 'MyRequest' and receiving 'MyResponse'.
            // The receiver will use Protocol Buffers to serialize/deserialize messages. 
            IDuplexTypedMessagesFactory aReceiverFactory = new DuplexTypedMessagesFactory(aSerializer);
            myReceiver = aReceiverFactory.CreateDuplexTypedMessageReceiver<MyResponse, MyRequest>();

            // Subscribe to handle messages.
            myReceiver.MessageReceived += OnMessageReceived;

            // Create TCP messaging.
            IMessagingSystemFactory aMessaging = new TcpMessagingSystemFactory();
            
            IDuplexInputChannel anInputChannel
                = aMessaging.CreateDuplexInputChannel("tcp://127.0.0.1:8060/");

            // Attach the input channel and start to listen to messages.
            myReceiver.AttachDuplexInputChannel(anInputChannel);

            Console.WriteLine("The service is running. To stop press enter.");
            Console.ReadLine();

            // Detach the input channel and stop listening.
            // It releases the thread listening to messages.
            myReceiver.DetachDuplexInputChannel();
        }

        // It is called when a message is received.
        private static void OnMessageReceived(object sender, TypedRequestReceivedEventArgs<MyRequest> e)
        {
            Console.WriteLine("Received: " + e.RequestMessage.Text);

            // Create the response message.
            MyResponse aResponse = new MyResponse();
            aResponse.Length = e.RequestMessage.Text.Length;

            // Send the response message back to the client.
            myReceiver.SendResponseMessage(e.ResponseReceiverId, aResponse);
        }
    }
}