In order to follow along with this post, it is important that you have an IoTIFY account. You can sign up for a free account here.

There are two things we need to know before we can start setting up our network simulations. First, the credentials of a device to connect to the Azure IoT Hub. Second, the actual message structure of the simulated device to be sent to the IoT Hub. Let’s begin with the first one.

Go to the Azure Portal and visit your IoT Hub. Click Device Explorer > Add. Fill in a unique ID for your device and save the settings (see figure below).

Once the device is setup, we can use the Device Twin Explorer to retrieve a SAS (Share Access Signature) token that will be used later to connect the device to the IoT Hub. In the management tab you have an overview of the different devices. Select your device and click the SAS Token button. A dialog appears where you again need to select the correct device as well as a Time-To-Live (TTL) for your token. Click the Generate button and your token will appear.

For the actual message structure, we will visit our Intel IoT gateway and go to the Node-RED environment. Double click the Create Payload element in the flow. Here we can see the actual structure of the message we are sending to our IoT Hub. Record this for future reference.

It is a simple structure with only three elements:

{ "deviceId": "client0", "timestamp": "08/18/2017 12:46 PM", "temperature": 36 }

Please note that the above step can be very different in your setup. The way described here assumes you have followed the previous posts as well.

Next, we can start creating our simulations using the IoTify service. Go to the IoTify website and login using your account. In your network tab, you should see something like the following:

Templates are the mechanism used by IoTify for simulating your device. Several samples have been provided to you. However, we will create two templates of our own. One is using the MQTT protocol and one is using the HTTP REST protocol.

MQTT Template

Create a new template and you should see something similar as the figure above. Next, fill in the blanks concerning MQTT parameters and credentials as follows:

1. Connection type: MQTT
2. Protocol: mqtts (TLS 1.2)
3. Endpoint: .azure-devices.net:8883 (replace with the name of your IoT Hub).
4. Client ID: (replace with the ID of the device created above)
5. Credentials:
   1. Username: .azure-devices.net/
   2. Password : Everything that comes after ‘SharedAccessSignature=’ in the SAS Token generated above (see underlining in the figure x).

In the “Specify Message Contents” section we will specify the topic endpoint as well as the message structure.

1. Topic: devices//messages/events/
2. Message Contents : A JavaScript function body returning the actual message. This is the same structure as the one determined above. The message basically contains a device id, a timestamp and a temperature. Numerous expressions can be used to specify this message. Here we opted for a timestamp representing current time and a randomly generated temperature between -10 and 45 degrees Celsius.

HTTP Template

Again create a new template and you should see something similar as the figure above. Next, fill in the blanks concerning the HTTP parameters as follows:

1. Connection type: HTTP
2. Protocol: https (TLS 1.2)
3. Host: .azure-devices.net:443 (replace with the name of your IoT Hub).
4. Path: /devices//messages/events?api-version=2017-06-30
5. Method: POST
6. Additional Header:
   1. Create a new Authorization header with as value everything that comes after ‘SharedAccessSignature=’ in the SAS Token generated above (see underlining in the figure x).

Note: In other blog posts, I have seen people ask about how to retrieve the actual API version. You could use the date that is provided in the documentation. At the time of writing this one says “2016-02-03”. I have retrieved mine from the source code of the Node.js SDK for connecting devices to the Azure IoT Hub (which is available on GitHub). This also worked for me!

In the “Specify Message Contents” section we will specify the message structure in the same way as the we did for the MQTT template.

Simulations

Next, we can start building our simulations. State the number of clients that you want to initialize as well as how many times you want them to repeat sending a message.

Additionally, you can apply a number of network parameters to make your simulation better resemble real-life network conditions e.g. average delay, jitter,… You can do all this using predefined templates as well.

One important thing to note about the simulations is that in the case of MQTT connections IoT Hub only supports one active MQTT connection per device. New MQTT connections using the same credentials will result in dropping the existing connection. In terms of performing simulations using the MQTT protocol this resulted in on average 50% success rate for our simulations (i.e. using 1000 clients through the same device credentials). Performing the same simulation using the HTTP protocol resulted in a 100% success rate.

Conclusion

In this blog post, we covered running simulations on our IoT solution, i.e. IoT Hub, using the IoTify service. We made the connection between IoTify and IoT Hub using the MQTT and HTTP protocol.

The image above visualizes the complete architecture of the hands-on-lab (original figure: link). The necessary schema’s, source code, … can be found in the following github repository.

Planning the Microsoft Azure resources

The right side of the figure above gives you an overview of all the resources required to make our application working the way we want to. Careful planning could possibly save you money as well as make your life a lot easier.

From a very high level the solution we will be developing looks as follows. Our IoT gateway communicates with the Microsoft Azure cloud through the IoT Hub. This is both for sending sensor data to the cloud as well as receiving commands from the cloud. The IoT Hub will forward the sensor readings to a Stream Analytics Job. This service will monitor all incoming sensor readings and on one hand persist them into a SQL database and on the other hand forward them to the Event Hub. There is a Web App that is used to visualize the (most recent) data. There is also a Function App that is used to send alerts back to the IoT Hub (and eventually the gateway).

Creating a Microsoft Azure IoT Hub and a Device Identity

The first two tasks are about setting up the Microsoft Azure IoT Hub. This can be simply done using the Azure Portal, selecting the respective service and filling-in the blanks. One additional thing to do is to create a consumer group which, at a later stage, will be used by the Stream Analytics Job, an instrumented piece of code that specifies data streams to abserve, transform, inspect and what actions to undertake when conditions apply. Each application that will read from your IoT Hub should get its own consumer group, a private state view on the hub. If you are sure there will be only one, you could get away with just using the default one.

For our gateway to send and receive messages from the IoT Hub, it needs to be registered as a device. This is where things have become easier now. A Device Explorer has been added to the Azure Portal. This explorer allows us to create the device identity of the gateway. Something which before was only possible through a graphical tool on Windows or through the iothub-explorer using a terminal.

Just click Device Explorer > Add. Fill in a unique ID for your device and save the settings (see figure above). The connection string for this device can then be retrieved from the device’s details blade. This string is needed in your IoT workflow later.

Publishing Sensor Data to the Microsoft Azure IoT Hub

With the Azure IoT Hub set up, we now can start to publishing the sensor data to the Cloud. To do this, additional software packages need to be installed onto the Intel IoT gateway. In the hands-on-lab, several commands are executed through a secure shell. Note that these steps could also be done using the Cloud command web console as mentioned in our previous post.

A new package is installed to communicate with Microsoft Azure Services. This installs the azure CLI and IoT SDKs for C, Python, and Java. On top of that, a new node in the Intel Gateway hosted Node-RED environment is made available that allows to send messages to the IoT Hub. The only thing that needs to be configured in our Node-RED flow is the connection string for the IoT Hub in the azureiothub node, and more particularly, the device we create in the previous step above.

Processing Sensor Data with Stream Analytics

With our data now streaming from our sensors across the gateway to the Azure Cloud, processing the data stream is done using a Stream Analytics job. This is a continuously running service that takes the output of the IoT Hub as input, performs some queries on this data, and has an SQL Database and an Event Hub as output. In order not having to jump back and forth, the outputs are created first. After both services have been setup, we just need to wire them together. Select the inputs and output and build a query in between. The query is in fact two queries, one to select all the incoming data and put it in the database and another one to only filter readings above a threshold and put those into the event hub.

Displaying Sensor Data with Microsoft Web Apps

The next task is about setting up a Web App in Microsoft Azure (as you probably have done many times before). The source code for the web app is provided and can be found inside a github repository. All you need to do is modify the configuration for your SQL database and the IoT Hub. The web app is a single page app refreshing every 30 seconds and displaying the most recent values from the database on the screen. The app experience is not very fluid due to the automatic refresh interval but it gives you a first idea.

Sending messages from the Microsoft IoT Hub to the Intel IoT Gateway

As mentioned in the beginning. The IoT Hus is also used to send commands back to our gateway. This can be done in two ways. The first, as you might have noticed when completing the task before this one, is to use the test button on the web page to trigger the buzzer in our demo setup. The second is using a so-called Function App. Function Apps are Microsoft’s take on the ‘serverless computing model’, which you might know from Amazon’s Lambda. It allows us to specify a small piece of code which is executed upon a certain trigger. In our case, the Event Hub is our trigger, meaning all alerts (temperature data above our threshold), are triggering our source code. When setting up the Azure Event Hub trigger in the Azure Function, one can now simply select the correct Event Hub to connect to instead of manually having to enter the connection string for the Services Bus (see figure below). Alternatively, you can select an IoT Hub directly, or using a good old connection string directly.

You can also find the source code to be used in the github repository of the hands-on-lab.

Display Sensor Data with Power BI (Embedded)

The final (optional) task is about visualizing our sensor data, inside the Web App we created before, using Power BI. It refers to this section in the lab. Due to some changes in Azure this section could not be completed as described. At least not on our side at the time of writing. What follows are the steps we took to achieve the same outcome.

Note that the following steps are performed in Power BI desktop. First, download and open the TemperatureChart.pbix (link). At the moment, this chart connects to the source used in the lab, so we need to change that. Click Edit Queries > Data Source settings (see figure below).

A dialog window opens where you can manage the data sources. Click Change Source (see figure below). Another dialog window pops up where you need to enter your SQL server and database. Click Ok, and then click Close.

A small notification pops up that allows you to apply the changes just made (see figure below). Click Apply changes (1). A dialog window pops up where you need to specify your credentials for the data source you just entered (2). Click Connect and you are all set (3). You should see your data be shown in the chart now.

For us to embed the chart into our Web App, we need to publish it to Power BI on the web (see figure below). Click Publish (1). Select a destination (2) and click Select (3).

If you now sign in to your online Power BI environment, you should see the Temperature chart appear (see figure below).

Click the TemperatureChart report. You should now see the same data appear as you did in the desktop version of Power BI. Click File > Publish to web. A dialog window pops up that provides you with a snippet of HTML code we can use to embed the report into our Web App.

Next, you to the Web App and open index.html. Comment out the image placeholder for the chart and place the HTML code snippet into the file and you are done (see figure below).

Launching the Web App should now show you a nice Power BI chart embedded into your app.

Conclusion

That’s all for now. In this blog post and also the previous we have covered the Intel IoT Gateway and the Microsoft Azure IoT Suite using a hands-on-lab provided by Intel. Initial experience with the gateway was fulfilling. It for the most part just worked out of the box. Something which is often not the case. Overall, a rather complete gateway worth investigating further.

• Intel IoT Gateway
• Hands-on-Lab

The hands-on-lab can roughly be divided into two parts. The first part covers the local setup, i.e., the Intel IoT Gateway together with the sensor platform. The second part covers the cloud setup, i.e., the Microsoft Azure IoT Suite.
The remainder of this post will discuss the first part. In a follow-up post, we will then discuss the second part. So, let’s get started!

The IoT Commercial Developer Kit

There are many IoT starter kits listed on Microsoft’s catalog of certified IoT devices. We opted for the Grove IoT Commercial Developer Kit. See the figure below for our demo setup. The kit consists of an Intel NUC serving as the local gateway. This is the big black box on the right of the figure. The gateway communicates, using a USB cable, with the sensors/actuators through the Grove system. This is the little Arduino board (+shield) on the left of the figure. The setup as displayed in the figure will be used throughout this series of posts as the running example.

There are in fact two versions of the kit available:

• Intel NUC DE38157TYKE with Bluetooth (but no clear reporting of the Gateway Software suite installed).
• Intel NUC DE3815TYKE, with the Gateway Software Suite installed.

As the latter was on backorder, we ordered the first one with the risk of having to install the gateway software ourselves. Upon arrival, we found the gateway software was pre-installed on this version as well, saving the whole register and download on the Intel site which while possible, would have been a nuisance nonetheless. The gateway was online in a matter of minutes.
The Grove system is an easy prototyping platform. It follows an intuitive plug and play interface. So, no soldering needed. This is a bit similar as the .NET Gadgeteer system people in the .NET world might be acquainted with. It typically consists of a base shield that you place on top of a board such as an Arduino in our case. The shield has several analog and digital ports to which you can connect your Grove modules (sensors/actuators) using special Grove cables. The kit contains about 9 modules but you can find many more online.

One of the downsides we found was the extreme shortness of the Grove cables (max 50cm) which is a problem when putting sensors in more distant locations. However, this can easily be overcome this by using a pair of RJ45 modules and putting a standard UTP cable in between. See the figure on the right. We tested them with cables up to 10m and they are still working fine.

The Intel NUC Developer Hub

The intel IoT Developer Hub is a software suite installed on the Intel NUC. It is the main access point for the IoT Gateway. It allows you firstly to administer your gateway, i.e. keep the OS up to date as well as install/update the different packages installed. Beware that updating the gateway takes some time. The out-of-the-box hardware configuration is rather constrained. It took us over 24 hours to successfully install the updates.

Note: We didn’t do a root cause analysis of the update problems, but the updating process hangs/crashes regularly. We had to leave the gateway portal page continually open in the web browser and refresh-and-restart periodically for the updating to succeed.

The most notable tool that comes pre-installed with the Gateway is a Node-RED. The Node-RED environment allows you to create and run your IoT workflows. Several nodes (building blocks) have been made available, through their UPM library, to interact with the different sensors in the kit. However, the set is very small which makes building alternative solutions on the gateway rather limited. A nice and clear tutorial on how to extend the library with your own nodes would be nice (maybe something to consider for another blog post ). Node-RED itself remains very nice though. We have used it in other setups as well. Be sure to check it out!
A second tool available to you, which is not explicitly mentioned in the Hands-on-Lab, is the ability to register your device (IoT Gateway) on the so-called Wind River Helix App Cloud (see figure below). Once registered, one can develop and deploy C/C++/JavaScript (Node.js) projects from the cloud onto the gateway.

This private cloud also houses a fully functional web-based development environment called “Cloud9”.

Connecting to the gateway using SSH

The following part of the Hands-on-Lab is about connecting to the gateway using a secure shell. However, we completely skipped this step as it was not needed to complete the rest of the lab. In case you do need some command line interaction with your gateway (which was mentioned at some points throughout the lab), you can always resort to the fully functional web based console. Just hit the launch button of the Cloud command web console from within the IoT Developer Hub’s administration section. Next, click the console button on the bottom right of the screen (>_~) and you are there.

Reading the temperature sensor

Finally, we can start to develop some IoT workflows. Having connected the different components as depicted in the first figure in the beginning of this post, we will now use the Node-RED environment to interact with the sensors/actuators.

The basic flow we will be creating is that of a upm-grove-temperature node to a function node to a upm-grove-rgb-lcd node. The first node will read the raw values from the temperature sensor. These raw values need to be converted to actual meaningful temperature values. This is the responsibility of the function node. Then, the values obtained are sent to the last node which is responsible for displaying them on the small LCD display. You can find (and import) the complete flow here. Launch the flow by clicking the Deploy button in the top right corner of the screen. If everything went well, you should see the current temperature on the LCD screen now.

Note: The be realistic, the temperature readings should go through another data conversion function to properly calibrate them. If you zoom in on the figure above, you can see the temperature in our room is said to be 39° Celsius while the actual temperature is about 25° Celsius. From observations, there seems to be a constant difference of about 15° Celsius. However, this could differ in your environment. Things to consider when dealing with real sensors.

Conclusion

OK, that’s enough for now. In this post, we gave you an overview of the Intel IoT Gateway and started developing our first IoT workflow, reading data from a temperature sensor. In the next post, we will start pushing these sensor readings to the cloud and start processing them.

Imagine a television recommender that can sense who you are with, what your day was like, how you feel, why you want to watch some TV, how this influences you, learns from your preferences …

After being unveiled  at the Berlin co-summit on ‘the impact of software innovation’, Spikes Research’s ‘Empathic Media Recommender’ was selected to represent the European ITEA research cluster at Europe’s largest ICT Research Conference in Lisbon. Backstage Spikes’ Research Director Peter Stuer was interviewed by David Wortley, gamification expert, on the application of empathic technologies to gaming. The interview, was published last march as part of a series on the impacts of gamification and immersive technologies on business and society.

Cortana, Siri, Alexa and Assistant are just starting. Whether at home or at work, get ready for machines that come with an advanced psychology degree under their belts, and are not afraid to use it.

ATL#, pronounced as “ATL Sharp”, is an ATL (ATL Transformation Language) parser written in C# for the .NET platform. It is implemented as an assembly and thus can easily be incorporated into other projects (in any .NET language).

The base parser was generated using the Antlr parser generator. Antlr translates a so-called grammar into a parser/lexer. The grammar for the ATL language was created from its BNF grammar. However, some changes were made to this original BNF.

Firstly, the “module” rule was expanded. It is now referencing separate rules for the source model, target model and transformation mode (i.e. sourceModelPattern, targetModelPattern and transformationMode). There might be more of these changes possible in other parts of the grammar, depending on your particular use case.

Secondly, in some places “IDENTIFIER” has been changed to the choice of an identifier or a string as some of the example transformations used during the development of the parser allowed both. This is a bit different than the concept of Identifer in other languages such as Java, C#. It is still unclear whether this applies to every instance though. If so, the grammar should be modified accordingly.

A small wrapper class, the “ATLTransformationParser”, was written to facilitate the parsing of ATL transformations. The parser can take either a filename or a stream as input (i.e. via the Parse method). Alternatively, a string can also be given as input (i.e. via the ParseString method). In both cases, a parse tree (syntax tree) is returned which can be traversed either manually or via a dedicated visitor.

Sidenote: The origin of the ATL# parser lies in the ARTIST EU project where ATL transformations are extensively used. One of the tools developed in the context of ARTIST, the Evolution Support Service. The Evolution Support Service is taking different versions of one particular ATL transformation, tries to find low level changes, and afterwards tries to infer high-level changes/change patterns from these low level changes. ATL# was used during the initial model creation of ATL transformations.

The source code for the ATL parser can be found on GitHub. Please note that the same grammar that was used here could also be used to generate a similar Java parser for ATL.

–

–