Building Loosely Coupled and Scalable RESTful Services using Orleans

These days, I’m working on Orleans and Actor-based systems as I mentioned in my post titled “Overview of Orleans“. In this article, I will try to explain how we can build loosely coupled and scalable RESTful services using Orleans as a middle-tier.

 

In addition to the practicality that we have gained from Orleans, it also brings a new approach to the architecture. It gives us location transparency, also everything is handled with Scalable Grains(Virtual Actors), without any Reentrancy and Concurrency problems. Sounds nice, isn’t it?

Anyway, although actor-based systems look very interesting to me, I can say that I gained a different perspective with using the Orleans project in my almost 10 years software development experience.

Using Orleans as a Middle-Tier

It is possible to build distributed, high-scale applications without thinking about any reliability, distributed resource management or scalability bottlenecks using the virtual actor model that Orleans implements.

Let’s continue with an example. Let’s assume, we are developing a vehicle tracking system for users. While drivers driving their cars, we will collect tracking data. When an object is passed to an Orleans Grain, it is serialized and deserialized. We used “[Immutable]” attribute on message contract for better serialization performance since serialization is a performance centric operation.

Basically, we will develop a REST endpoint and a Silo that runs behind like the above picture.

Building Silo

Firstly let’s create a “VehicleTracking.Common” class library. We will create messages in here that will pass between the Grains.

“VehicleInfo” message is defined as below.

using System;
using Orleans.Concurrency;

namespace VehicleTracking.Common
{
    [Immutable]
    public class VehicleInfo
    {
        public long DeviceId { get; set; }
        public string Location { get; set; }
        public string Direction { get; set; }
        public DateTime Timestamp { get; set; }
    }
}

When an object is sent to across node, it is serialized then deserialized with the binary serializer. So Grains cannot access the same object and change their internal state. Also, we used “[Immutable]” attribute on message contract for better serialization performance since serialization is a performance centric operation.

[Immutable] Messages

The serialization process is performed so the objects can access the Grains in the different Silos. On the other hand, deep-copy operations are performed for Grains in the same Silo. This serialization operations can be made slightly more efficient for Grains on the same Silo. It is possible with the using “[Immutable]” attribute, so the serialization operations can be bypass.

Let’s create called “VehicleTracking.GrainInterfaces” class library, then define “IVehicleGrain” interface.

using System.Threading.Tasks;
using Orleans;
using VehicleTracking.Common;

namespace VehicleTracking.GrainInterfaces
{
    public interface IVehicleGrain : IGrainWithIntegerKey
    {
        Task SetVehicleInfo(VehicleInfo info);
    }
}

We defined “SetVehicleInfo” method in the “IVehicleGrain” interface. We will use this method while collecting drivers location info. By the way, if we look at the method name, we can see how it looks like an RPC method name definition. Orleans clients and Grains communicate with each other via PRC, therefore we defined method name an RPC style.

Now, let’s create one more interface called “IVehicleTrackingGrain”.

using System.Threading.Tasks;
using Orleans;
using VehicleTracking.Common;

namespace VehicleTracking.GrainInterfaces
{
    public interface  IVehicleTrackingGrain : IGrainWithIntegerKey
    {
        Task SetVehicleTrackingInfo(VehicleInfo info);
        Task Subscribe(IVehicleTrackingObserver observer);
        Task Unsubscribe(IVehicleTrackingObserver observer);
    }
}

While drivers driving their cars, we will collect the location data then pass with “SetVehicleTrackingInfo” method, over “IVehicleGrain”. When the location data is passed, we will send notifications to client’s subscribers.

Now, we will define an observer to send notifications. Therefore, let’s create another “IVehicleTrackingObserver” interface.

using Orleans;
using VehicleTracking.Common;

namespace VehicleTracking.GrainInterfaces
{
    public interface IVehicleTrackingObserver : IGrainObserver
    {
        void ReportToVehicle(VehicleInfo info);
    }
}

That’s all. Now, we can start Grains implementations.

Firstly let’s implement “IVehicleTrackingGrain” interface for observing operations.

using System;
using System.Threading.Tasks;
using Orleans;
using VehicleTracking.Common;
using VehicleTracking.GrainInterfaces;
using Orleans.Concurrency;

namespace VehicleTracking.Grains
{
    [Reentrant]
    public class VehicleTrackingGrain : Grain, IVehicleTrackingGrain
    {
        private ObserverSubscriptionManager<IVehicleTrackingObserver> _observers;
        private VehicleInfo _vehicleInfo;

        public override Task OnActivateAsync()
        {
            _observers = new ObserverSubscriptionManager<IVehicleTrackingObserver>();

            RegisterTimer(Callback, null, TimeSpan.FromSeconds(5), TimeSpan.FromSeconds(5));

            return base.OnActivateAsync();
        }

        Task Callback(object callbackState)
        {
            if (_vehicleInfo != null)
            {
                _observers.Notify(x => x.ReportToVehicle(_vehicleInfo));

                _vehicleInfo = null;
            }

            return TaskDone.Done;
        }

        public Task SetVehicleTrackingInfo(VehicleInfo info)
        {
            _vehicleInfo = info;

            return TaskDone.Done;
        }

        public Task Subscribe(IVehicleTrackingObserver observer)
        {
            _observers.Subscribe(observer);

            return TaskDone.Done;
        }

        public Task Unsubscribe(IVehicleTrackingObserver observer)
        {
            _observers.Unsubscribe(observer);

            return TaskDone.Done;
        }
    }
}

We performed the observing operations with the “ObserverSubscriptionManager” helper in Orleans. It provides an easy way to process, such as subscribing and sending a notification. The “OnActivateAsync” method is called at the end of the Grain activation process. We used the “RegisterTimer” method here so that we can perform callback operations on Grains as periodic. If we look at the callback method, if the “_vehicleInfo” field is not null, all subscribed clients will get notifications via the “ReportToVehicle” method.

[Reentrant] Attribute

We have used the “[Reentrant]” attribute, that we have defined above to overcome bottlenecks in the network and to apply some performance optimizations. According to Carl Hewitt, as conceptually messages are processed one at a time in the actor model. In Orleans, concurrent processing can be provided with techniques such as the “[Reentrant]” attribute. In this way, where it may be necessary Grains will not block in the face of some costly operations. However, we are advised to be careful at the points we need to use, otherwise, we may face race-conditions situations.

Now, we can implement “IVehicleGrain” interface as follows:

using System.Threading.Tasks;
using Orleans;
using Orleans.Concurrency;
using VehicleTracking.Common;
using VehicleTracking.GrainInterfaces;

namespace VehicleTracking.Grains
{
    [Reentrant]
    public class VehicleGrain : Grain, IVehicleGrain
    {
        private long _currentGrainId;

        public override Task OnActivateAsync()
        {
            _currentGrainId = this.GetPrimaryKeyLong();

            return base.OnActivateAsync();
        }

        public async Task SetVehicleInfo(VehicleInfo info)
        {
            //some business logics...

            var vehicleTrackingGrain = GrainFactory.GetGrain<IVehicleTrackingGrain>(_currentGrainId);

            await vehicleTrackingGrain.SetVehicleTrackingInfo(info);
        }
    }
}

Let’s assume, we are getting vehicle location data with the “SetVehicleInfo” method, then processing it for vehicle tracking operations with some business logics. When the business logics processed, we are passed the message to “VehicleTrackingGrain” for notification step.

Now, we completed all implementations. We can create now Orleans Dev/Test Host as follows:

Then create a new class called “VehicleTrackingObserver”.

using System;
using VehicleTracking.Common;
using VehicleTracking.GrainInterfaces;

namespace VehicleTracking.TestSilo
{
    public class VehicleTrackingObserver : IVehicleTrackingObserver
    {
        public void ReportToVehicle(VehicleInfo info)
        {
            Console.WriteLine($"The vehicle id {info.DeviceId} moved to {info.Direction} from {info.Location} at {info.Timestamp.ToShortTimeString()} o'clock.");
        }
    }
}

We implemented “IVehicleTrackingObserver” interface at the above code block. At this point, when drivers drive their cars, we will write the vehicle tracking notifications on the console screen.

Let’s refactor “Program.cs” as follows:

using System;
using Orleans;
using Orleans.Runtime.Configuration;
using VehicleTracking.GrainInterfaces;

namespace VehicleTracking.TestSilo
{
    /// <summary>
    /// Orleans test silo host
    /// </summary>
    public class Program
    {
        static void Main(string[] args)
        {
            // The Orleans silo environment is initialized in its own app domain in order to more
            // closely emulate the distributed situation, when the client and the server cannot
            // pass data via shared memory.
            AppDomain hostDomain = AppDomain.CreateDomain("OrleansHost", null, new AppDomainSetup
            {
                AppDomainInitializer = InitSilo,
                AppDomainInitializerArguments = args,
            });

            var config = ClientConfiguration.LocalhostSilo();
            GrainClient.Initialize(config);

            // TODO: once the previous call returns, the silo is up and running.
            //       This is the place your custom logic, for example calling client logic
            //       or initializing an HTTP front end for accepting incoming requests.

            Console.WriteLine("Orleans Silo is running.\nPress Enter to terminate...");

            var vehicleTrackingObserver = new VehicleTrackingObserver();
            var vehicleTrackingObserverRef = GrainClient.GrainFactory
                                                    .CreateObjectReference<IVehicleTrackingObserver>(vehicleTrackingObserver).Result;

            var vehicleTrackingGrain = GrainClient.GrainFactory.GetGrain<IVehicleTrackingGrain>(1);
            vehicleTrackingGrain.Subscribe(vehicleTrackingObserverRef).Wait();

            hostDomain.DoCallBack(ShutdownSilo);

            Console.ReadLine();
        }

        static void InitSilo(string[] args)
        {
            hostWrapper = new OrleansHostWrapper(args);

            if (!hostWrapper.Run())
            {
                Console.Error.WriteLine("Failed to initialize Orleans silo");
            }
        }

        static void ShutdownSilo()
        {
            if (hostWrapper != null)
            {
                hostWrapper.Dispose();
                GC.SuppressFinalize(hostWrapper);
            }
        }

        private static OrleansHostWrapper hostWrapper;
    }
}

We performed subscription operations with using “IVehicleTrackingObserver” and “IVehicleTrackingGrain”. In this project, we will initialize Orleans Test Silo also write the notifications that coming from the observer on the console screen.

Defining the REST Endpoint

We are ready to coding REST endpoint. Let’s create an empty Web API project called “VehicleTracking.Api”. Then add “Microsoft.Orleans.Core” package via NuGet Package Manager as follows:

After this, we should initialize Test Silo in the “Global.asax” as follows, so we can communicate with Silo.

using Orleans;
using System.Web.Http;

namespace VehicleTracking.Api
{
    public class WebApiApplication : System.Web.HttpApplication
    {
        protected void Application_Start()
        {
            GlobalConfiguration.Configure(WebApiConfig.Register);

            var config = Orleans.Runtime.Configuration.ClientConfiguration.LocalhostSilo();
            GrainClient.Initialize(config);
        }
    }
}

Now, we can communicate with Silo. Let’s add our first controller called “VehicleTracking”, then coding as follows:

using Orleans;
using System;
using System.Threading.Tasks;
using System.Web.Http;
using VehicleTracking.Common;
using VehicleTracking.GrainInterfaces;

namespace VehicleTracking.Api.Controllers
{
    public class VehicleTrackingController : ApiController
    {
        [Route("api/vehicle-trackings")]
        public async Task Post(long deviceId, string location, string direction)
        {
            var vehicleGrain = GrainClient.GrainFactory.GetGrain<IVehicleGrain>(deviceId);

            VehicleInfo trafficInfo = new VehicleInfo()
            {
                DeviceId = deviceId,
                Location = location,
                Direction = direction,
                Timestamp = DateTime.Now
            };

            await vehicleGrain.SetVehicleInfo(trafficInfo);
        }
    }
}

Now, we have a POST endpoint. At this point, we are passed vehicle tracking info to Grain with using “SetVehicleInfo” method incoming from “VehicleGrain” instance.

We are ready for the test steps. Firstly we have to initialize Silo. Let’s start the “VehicleTracking.TestSilo” project, then the “VehicleTracking.Api” project.

After this, start the “VehicleTracking.Api” project too. Now, let’s send a POST request to “/api/vehicle-trackings?deviceId=1&location=Taksim Square&direction=Bagdat Street” endpoint via Postman as follows:

As a result, we can see that the notification operation of the vehicle tracking info, that we sent via the REST endpoint is written on the console screen via observer.

Conclusion

We have built a system that works loosely coupled and scalable with using the Orleans Silo as a middle-tier behind of the REST endpoint. Also without any thread locking or concurrency problems.

I hope this article would help who needs any information about using the Orleans as a middle-tier. Currently, I’m researching on the Orleans to work with the Docker. At the same time, I will try to share my experience on the Orleans in new articles.

Sample project: https://github.com/GokGokalp/orleans-vehicletracking-sample

References:

https://dotnet.github.io/orleans/Tutorials/Front-Ends-for-Orleans-Services.html

https://dotnet.github.io/orleans/Tutorials/Concurrency.html