Wednesday, June 28, 2017

LabPlot getting "live support"

As I told you guys in my previous post, I'm here now to show you some new, cool stuff about LabPlot. First let me introduce you the synopsis of my project's proposal: 

"Currently, the visualization and analysis of data is only possible on static data that was imported into or generated in one of LabPlot's data containers. The goal of the project is to add support for streaming data. At the moment LabPlot has no support for this kind of data processing even though it is very important for this feature to be available in a scientific data plotting software."

As you can see one of the main tasks here is to be able to handle changing, streaming data. Somebody may plug in their physical device and then watch as their datas are being plotted. For this to work nicely and user friendly, we collected a handful of options to control this kind of data source. The first thing you want to do is to create a new FileDataSource, you can do that this way:

FileDataSource creation

After this you get this nice looking dialog with the new options, controls. We decided to support more types of live data sources, as you can see in the picture below.

FileDataSources options
For "File or named pipe" source type we have the following options:

File or named pipe options

This kind of source consists of reading from a local ascii or binary file or a named pipe. A named pipe is a bit more than a traditional pipe because it has a name and unlike the traditional one it can last till the system is up. In Unix systems one has to explicitly create a named pipe using mkfifo or mknod. After creation one process can write to it and one can read from it by name. Also for "Local socket" live data source type we have the same options, but this consist of reading from a local socket using QLocalSocket. To work with QLocalSocket we just need the name of the server the socket is connected to.

The next type of live data source is the Network socket one. The following options we have right now to configure this kind of live data source:
Network socket options

A network socket is an endpoint for sending/receiving data in a computer network. In our situation we want to read from it. To access a socket we must have the socket host's address and port number. 

The last type of live data source for now is the Serial port one. In the following picture you can see the options for this kind of data source:

Serial port options
This kind of live data source consist of reading from a port to which a physical device is connected. We just have to select the port our device is connected to, set the baud rate and voilá, it should work nicely soon. We use QSerialPort to access, work with serial ports. Using QSerialPortInfo class we can have informations easily about the available ports on our system. Also the supported baud rates can be acquired using QSerialPortInfo.

These options are necessary to set up the wanted kind of live data source. Beside this we must have some control above this kind of data source, right now we have the following options to control the update a FileDataSource:

Update options
Here you can see that we have two types of updates. The "Time interval" one will update the curves on a specified time interval (every second, every five seconds or so) which is a value in milliseconds. The "New data" update type will update the curves only when there's new data available. In the "Update frequency" we set the interval between the updates when using "Time interval" update type. The "Keep last N values" will control how much of the data we want to update. For this kind of data source one usually want to observe how the data is changing, to take a snapshot of their data. 

Another thing we might want is to have control over this kind of data source even after creation. For this we'll have a dock widget, it's not yet used, but currently it looks like this in Creator:

FileDataSource dock widget

With these options we will be able to modify the number of values we want to keep on every update, the time between updates. If the user uses a long interval between the updates then a functionality like the "Update now" button will provide it's very useful because there might be moments we didn't think we'd need to check the actual state of our data. Also the pausing and stopping the updates/data reading helps in the control of this kind of data source.

I have created a demo video which shows the creation of a FileDataSource, setting the X and Y axis data of some xy curves. Then a python script writes Brownians motion data to a file on which the FileDataSource was created. We can see there the curves being updated on new data:

An issue you can see here is that I have to set for every curve separately the X and Y value columns. We will use the PlotDataDialog which is used for plotting spreadsheet columns to solve this issue. PlotDataDialog is a very user friendly dialog which can be accessed from the spreadsheet's context menu. In this dialog we can select the X data and Y data for any curves we might want to plot from spreadsheet column data. This is the PlotDataDialog:


This is for now, hope you guys enjoyed it! The next step is to implement all the logic behind these options and make them do their job. A greater step after implementing the options will be to optimize our code on larger amount of data. Thanks to my mentors, Alexander and Stefan for their great help. Every time I am unsure on something they take their time to talk about it so together we can make LabPlot even cooler than it is right now :).

See you guys soon again! Have fun on your GSoC project!


  1. Any chance of connecting a postgreSQL db?

    1. One of the main developers (Alexander Semke)has already implemented the import from SQL databases. You can check it out, compile for yourself, it will be released in the next release (2.5). Is there a chance you meant live data from a postgreSQL database?