Link to the Information Technology Laboratory Website Link to the Information Access Division Website Link to the NIST Website NIST, IAD Banner

Chapter 7. Tools

The NDFS-II is a complex system. Moreover, it is often used on clusters of machines. Some tools are provided in order to make the development and management of applications easier.

The Control Center

The Control Center is a GUI allowing one to create and manage NDFS-II applications. It gives the user the ability to build a map, i.e. to connect client nodes together and to associate each one of them to a host. Then the user can launch the application: each client node is spawned by the proper data flow server on its host.

The application can also be stopped from the Control Center. It is possible to start client nodes using the command line as well.

Once an application is started, the Control Center can be closed, the application will keep running. If the GUI is launched again, it is possible to discover the current application over the NDFS-II network and to take control of it.

The Control Center displaying an application map composed of four client nodes.

The Control Center displaying an application map composed of four client nodes.

When the Control Center is launched, it tries to connect to its local data flow server, and displays a list of servers sharing the same application domain. If no data flow server is present, you will get a warning message letting you know the Control Center cannot connect to its server. If later on you start a data flow server, you can request the Control Center to connect to the server by pressing on the Picture of the Connect To Server button button.

The Control Center allows one to create and manage a client nodes list. Using this list, it is then possible to create application maps. An application map describes the connection between some or all client nodes and associates each one of them with a machine from the server list.

Creating a client list

Creating a client list: The first thing to do to create an application is to create or load a client list. A list of client nodes can be loaded from the File menu. An example file is provided in the bin/tools folder and named example_client_list.conf.

The Control Center gives you the possibility of creating a list. In order to add a client node in the list, you need to click on the "Add.." button located in the client list. A dialog appears, and you can either enter manually the description file's path of a client node or select it using the "..." button. For example, go to the example_consumer_memcpy’s source folder and select the file example_consumer_memcpy.client file and load its description using the "Apply" button.

You should get the description of the client as shown on the screenshot below.

Screenshot showing the form used to add client nodes in the client list.

Screenshot showing the form used to add client nodes in the client list.

Validate it by clicking on the "OK" button. The client node now appears in the client list. Check that the path of the client node is correct by invoking "Edit" located in the contextual menu of each client node in the list. If the path is not correct, enter the proper one. Do the same for the example_provider_memcpy client.

This list of client nodes can be saved using the File menu.

Creating an application map

Once you have a client node list you can create an application map using them. Click on one of the clients and drag it into the map and drop it there. A dialog appears letting you select on which machine the client node should be launched. Select your current machine within the list. Drop the example_consumer_memcpy and the example_provider_memcpy in the map.

Now the two client are present in the map but not connected yet. To connect them, click on the flow outlet of one the two client nodes, drag your mouse toward the other client node and release your mouse button on the outlet of the other client node.

The two client nodes are now connected. The Control Center checks that the connection is possible. If you want to connect two flow outlets, which are of different kinds, or you try to connect an output flow to an output flow, it will be refused by the Control Center.

You can save your application using from the File menu.

Note: When you save an application map, the client nodes from the list that are used in the map are also saved in the application map file. So you don’t have to reload the client list corresponding to your application each time you load a map.

Starting an application map

At this point, you created a client list and build an application map using the client nodes from the list. You can start the application by clicking on the Picture of the Start Application Button button. In the same manner you can stop the application by clicking on the Picture of the Stop Application Button button. It is also possible to start each client node individually using the contextual menu of the client nodes located in the map.

Using the discover application capability of the Control Center

The Control Center is a client node, which uses special control methods provided by the NDFS-II API. So like any other client node, it can join or leave an application domain. Because this client does not provide or consume data, it does not affect your application when you shut it down. If you have an application map running, you can quit the Control Center and start it again. You will notice that the map is empty, but you can ask the Control Center to display you the application currently running in the application domain. Click on the Discover Map button (Picture of the Discover button) to discover the application currently running. If you had the example_provider_memcpy and example_consumer_memcpy running when you quit the Control Center, you will see them appear in the map and be able to take control of that application. Only one Control Center can be launched per application domain.

Reading the map

The Control Center can be really useful to monitor a NDFS II application. In order to provide an easy way to represent the status of an application in real-time, we use a color code in the map of the Control Center. Following are some screenshots showing the different states that client nodes or flow can have.

Picture showing a producer connected to two consumers. All client nodes are idle.

Picture showing a producer connected to two consumers. All client nodes are idle.

On this screenshot, the provider, the two consumers and the flow are in yellow. This color represents client nodes or flows, which are idle or stopped. They have either not be started yet or have been stopped.

Picture showing a producer connected to two consumers. All client node are running.

Picture showing a producer connected to two consumers. All client node are running.

On this screenshot the client nodes and the flow are green. Green means running, i.e., the client nodes have been started and the provider is sending data to the two consumers.

Picture showing a producer connected to two consumers. All client node are running and a consumer paused a flow.

Picture showing a producer connected to two consumers. All client node are running and a consumer paused a flow.

The grey color for a flow means the flow is paused. Here consumer has paused its flow, meaning that it is still connected to the provider but the data are not send anymore to the consumer. Every piece of data sent by the provider is lost by the consumer during the time it paused its flow.

Picture showing a producer connected to two consumers. All client node are running and the provider paused a flow.

Picture showing a producer connected to two consumers. All client node are running and the provider paused a flow.

On the above diagram, either both consumers have paused the flow or the provider itself has paused. The provider is still producing data, but none of it is sent to the consumers.

Picture showing a producer connected to two consumers. All client node are running and a consumer destroyed a flow.

Picture showing a producer connected to two consumers. All client node are running and a consumer destroyed a flow.

On the above screenshot, all the client nodes are still running but consumer has either destroyed its flow or not created it yet. If consumer has destroyed its flow, it means it doesn't intend to consume data anymore. In order to represent this, the flow outlet of consumer is colored in yellow.

Picture showing a producer connected to two consumers. All client node are running and the producer destroyed a flow.

Picture showing a producer connected to two consumers. All client node are running and the producer destroyed a flow.

In the above diagram, the two consumers have been started and successfully created their input flow, but the output flow of the provider is not running. That means that the provider has not created yet its flow or has already destroyed it.

Picture showing a producer connected to two consumers. Only the provider and a consumer are running. The remaining consumer couldn't be started.

Picture showing a producer connected to two consumers. Only the provider and a consumer are running. The remaining consumer couldn't be started.

On the screenshot above, consumer2 is colored in red. That color means there was a problem starting the client node. Typically that means that the client executable has not been found.

Control Center's Hello World

Some client nodes, whose purpose is to provide a view of how to use the NDFS-II, are provided. This includes a client list and an example map.

The client list is located in Smartflow2/bin/clients/exampleClientList.conf and the example application map located in Smartflow2/bin/clients/exampleClientMap.app