ControlToolbox.com

I joined LinkedIn (www.LinkedIn.com) almost a year ago. Originally, I could not see the value it may have. Gradually, when I was bored, I started looking into the discussion boards. This triggered my interest a bit and prompted me to find technical groups and communities in there. Sure enough, there were lots of groups related to Control and Automation.

In December 2008, just as a test, I posted couple of messages about one of our new products and started watching our website traffic. It looked like people were paying attention. I saw a surge in the traffic. Also, I realized that the traffic is coming to us either directly or by Google search on our name. Very interesting! I also got couple of objections from people that though my post was not related to the subject of the group. For example, I had posted something about our Modbus TCP module on a ZigBee group. The module was designed to collect data from a distributed population of remote units. They could communicate to the server on any link that would allow XML web services. The data is presented to the Modbus client through our Modbus server. I thought ZigBee people may be interested. What I had not realized was that the ZigBee people are more interested in ZigBee itself, not the applications loaded on the link. I responded to the objections reasoning that application people like Substation Engineers need to see this information.

This prompted me to search for forums on Substation Automation. There was none. So, how about if I put together a group for Substation Automation to help people to exchange their ideas and experiences. Moving on, I started three groups:

Substation Automation

Distribution Automation

DNP Users

Soon enough, people started pouring in. It was like a magic. You put something out there and have no idea what happens. I had the same feeling when we sold our Modbus TCP code library for the first time on the Internet.

Anyway, this is looking very promising. I have high hope that this forum can become something helpful to everybody.

Posted by rnafisi at 11:19 PM | Permalink | TrackBacks (0)

Terminal servers are inexpensive devices that are used for converting IP to serial traffic. They have found their way to SCADA application too. They are used where a legacy RTU not being able to cover IP communication must be used with a more modern SCADA or telecomm infrastructure in which IP communication is used as the backbone of the control system. There are several manufacturers already providing good and reliable terminal server type devices for industrial applications such as substation automation or plan automation.

Terminal servers, in general can operate in the following modes: TCP Server, TCP Client, UDP and Com.

In the TCP Server mode, the device is set to open a specific IP port. The device is listening on the IP port for incoming TCP connection calls from the remote host. Obviously, the remote host is responsible for making the connections in this mode. Once the connection between the host and device is made, the device repeats the packets it receives on the TCP connection to the serial port. On the other hand, if data is received on the serial port, the terminal server sends the traffic to the remote host via the open connection. Please note that in this mode, the software running on the remote host trying to communicate to the terminal server must be able to operate in TCP/IP mode. In addition, the host must be able to detect TCP session time out and capable of retrying for establishing the session again. 

In the TCP client mode, the remote host is responsible for opening a specific IP port. The host is listening on the port for incoming TCP connection calls from the terminal server device. The sequence of the events is the same as the TCP Server mode except that the device is responsible for re-establishing the link.

In the UDP mode, both sides open specific ports. In addition, both sides need to be aware of IP address and IP port of the other party. If the remote host has something to send to the terminal server, it wraps the data in one or more UDP packets and sends to the device IP address/port. On the other hand, the device sends to the remote host IP address/port what it receives over the serial link. Again, the software running on the remote host should be able of sending the data via UDP packets.

The last mode the is Com mode. In this mode, a proprietary device driver developed by the terminal server manufacturer is installed on the host machine. The driver communicates over IP to a resident program in the terminal server. Via this communication path, the device is instructed to send data over the serial link. Likewise, the device sends the data received on the serial link to the driver residing in the remote host via the same channel. At the host, the device driver emulates itself as a regular serial port. Therefore, the application software running on the host must be able to drive the local machine’s serial ports

In some cases, the bandwidth is abundant and the remote device is located close to the host machine with no firewall in between. Examples are communicating over a local fiber optic or Ethernet network. In this case, there would be no real difference between the performances of the four modes of operation. Here you need to see what are the requirements of the host application and try to comply with them. The problems start appearing when you have smaller bandwidth, spotty communication or have to go through firewalls. In this case, the Com mode is out of picture since you must have more information about its communication standards and timings to get it to work. Even though UDP is less reliable compared to TCP, it may be the way to go since it requires less overhead. Remember, in most substation data protocols such as DNP or Modbus, the master is already keeping track of the remote device responses. Therefore, you do not need to add another layer on top of it. You also need to determine how the packets at the device side are packaged. For example, if you know that your serial link receives packets with 300 ms lengths, it is better to wait the full period before sending the packets on the UDP port. Otherwise, terminal server needs to send two UDP packets per RTU response. This is something that you find it troubling in wireless environments.

I will talk about ways for putting together more versatile terminal server applications in future posts.

Take care for now,

Rod

Posted by rnafisi at 11:22 PM | Permalink | TrackBacks (0)

When we were working on the design of NetProModServe family of products , we had the following goals in mind:

-          Implement Modbus TCP Server

-          A plug and play DLL that the developer can integrate into his application is three steps

-          Be able to push/pull data into/from the server virtual RTUs from anywhere in the network or outside of the network domain to cover distributed applications

-          Be able to add/remove RTUs to the database seamlessly and dynamically

-          Communicate to the core event base

-          Stateless communication to the remotes so that break in the communication can be covered with minimum impact

-          Allow unlimited Modbus TCP connections to a single RTU on the same IP port

-          Allow all Modbus TCP connections to all the RTUs on the same IP port of a single machine

At the time, we were sure this design covers many features developers may need. Recently we were contacted by an international client. They were using our code library in an interesting setup. The application was receiving data on its holding register from remote terminals. One possible solution was to have the terminals to communicate to different Modbus RTUs with different UIDs. This is necessary because at the receiver side, they must to differentiate between the traffic received from different terminals. However, since they had to manage a large number of terminals, they did not want to have different configuration files in the terminals. So, they came up with a great idea: How about if NetPro interprets the last segment of the terminal IP address as the UID of the destination RTU?

We found the idea very useful. Not only did we implement it for the client, we added it to our standard code since it makes sense that there would be other people looking into this option. A great example as how a product evolves with the users’ feedback.

Take care for now,

Rod

Posted by rnafisi at 04:18 PM | Permalink | TrackBacks (0)

Now, here is the problem with Modbus TCP. Lots of people have Modbus TCP masters and legacy Modbus devices. For them, connecting the master to the device requires more than a simple terminal server that converts IP traffic to serial. The reason is there would be a need for a gateway that can adjust the protocol from serial version to TCP version and vice versa. That’s why you see many manufacturers making devices called Modbus gateway. The situation with DNP is much simpler. Since the DNP packets in IP and serial version are the same, a simple terminal server can do the job. Terminal server gets the IP traffic and sends the DNP payload on the serial side. Conversely, it gets the DNP packets on the serial link and embeds in the IP envelopes to transmit over the IP link.

I’ll talk about using terminal servers and potential issues they may create for data communication in a future blog.

Take care for now, Rod       

Posted by rnafisi at 03:00 AM | Permalink | TrackBacks (0)

I was playing with Triangle Test Harness for DNP and comparing it with the one from ASE. ASE test set requires a higher investment, twice as much as Triangle. So this could be a disadvantage right there. However, if you are an engineering company dealing with legacy protocols such as CDC, Harris, TEJAS, …, ASE is a  better choice since they can be added for a very low price. You do not have that option with Triangle.

 ASE application seems to have a better graphical user interface. It uses a multiple document interface, MDI, for its application that allows a better management of the information. On the other hand, Triangle user interface is more of a dialog based interface where you need to deal with multiple floating windows.

Both applications have an interface to show the data traffic. ASE’s interface seems to be more useful since it uses different colors to differentiate the layers of the packet. What seems to be odd with Triangle is that the packets are not time-stamped. In a recent experience, I was trying to get a sense of the channel delay and could not use Triangle for the very same reason.

You have the option of defining the requests you send to the outstation in both applications. But again, ASE’s arrangement seems to be much more user friendly.

If you like to test DNP over IP applications, you have both TCP and UDP options in ASE test set. I could not get the UDP option to work with Triangle (I was using a packet sniffer, which was showing that the Triangle application is always sending out TCP traffic). This is a big minus for a test set.

One thing that Triangle can score against ASE is how the host IP address is defined in IP communication. In ASE, the host IP address is defined globally. Therefore, even if you define multiple outstation addresses in your data exchange list, still all of them are sent to the same IP address. You do not have that restriction with Triangle.

Take care for now,

Rod

Posted by rnafisi at 01:40 AM | Permalink | TrackBacks (0)

You may have heard it that some people say this or that protocol is or isn’t good for wireless environment. Here is what is different in wireless environment (yes, I admit this is a big generalization) compared to a dedicated line is:

·         Higher level of noise and bit error rate

·         Single channel is shared by many remote devices

·         Higher propagation delay

·         Hidden nodes, meaning that there may be nodes that cannot see each other and as a result step on each other’s toe

·         Wireless signal issues such as fading and bouncing

So a communication protocol needs to have enough ammunition to combat against these issues. I like to see these features in the protocol to overcome the above problems:

·         Moderately small size of data packets so that the probability of packet corruption goes down

·         Ability to fragment the frames so that the longer frames can be sent in smaller independent packets

·         Enough number of CRCs inside and at the end of the packet to prevent accepting the corrupt packets

·         Link acknowledgment to signal the other side that the packet is received

·         Sequence numbers so the consequent packets can be differentiated and also a single packet cannot be accepted more than once

·         Being able to send unsolicited messages to prevent wasting the bandwidth through repeated polls

DNP3 seems to have all of these features. So why do people complain so much? Because:

·         The RTU is not intelligent enough when to send the response, meaning it may send it when the channel is already seized by someone else. The remote device needs to be monitoring its own radio signals before putting data on the line

·         The radio communication is such that the nodes cannot see each other. If we suffice to packet-acks, the throughput would not go beyond 20% of the channel capacity. There are better ways to combat the hidden node issue.

·         Most often DNP3 master drivers are not designed to consider the limitations of the wireless environment. One of the most common mistakes is overloading the channel with many requests at the same time.

·         Timing, Timing, Timing: It is very important to tune your timing parameters at the master so that for example a delayed packet is not rejected due to link timeout setting.

I am planning to put together a DNP crash course in near future for those who like to know more about this popular protocol, stay tuned.

Take care for now, Rod

[Full disclosure: This weblog is published by a service provided by Yahoo]

You may think this is an odd title for a weblog supposedly talking about data comm and automation. So, let me explain.

When I left my former employer more than a year ago and started my own company, I was always thinking about the best strategy that the company can grow with. Like any other classic case, you start by figuring out your strengths, opportunities, weaknesses and threats. But before then, you may want to define yourself and see what is the business you are in. Are you a product developer? A service company? A consultant? ….

Once you are through the identity crisis , you need to answer other questions like:

Who are your clients: are you willing to work in the automation industry per say or electric market as your niche?

What about your cash flow, how the money comes in? A venture capital? Or, you want to bootstrap the company?

What is your growth path? Do you see yourself as a large firm with 1000 employees in 10 years or a small core with very talented people that only tackles top of the line technologies?

And yes, I am sure that you may have seen similar texts in your textbook, internet etc. But these are real questions. The good thing is that we are living is an era that data is abundant. If I erect a wall around what I know or can do, I am only fooling myself. So this abundance of information can be applied to answering business questions too, right? The way you do it, you share it with the right community of people and see what other people have experienced. And besides, whatever comes out of it becomes a part of the public domain and strengthens the same community that helped you.

This is the model that I think Google follows. Google to me is only a facilator that allows the communities to form around its platform. To some extend Facebook and LinkedIn do the same thing. Community gives you something and you give back and through this vicious cycle everybody benefits. On the other hand, Yahoos of industry have positioned themselves at some corner and tell people: come to me I have something great form you. They do not realize that your value can only be to facilitate people to exchange their own great things.

Now, here come how all these things are related to the topic of automation. I posted a set of questions couple of days ago on Control.com that is supposedly one the most famous forums for automation. Please find the link here:

http://www.control.com/thread/1237082739#1237240408

I got two responses: one from a gentleman that was obviously editor of one of the online automation sites and the other from an individual that I respect for his prior posts and seems to be a knowledgeable person. Both were insisting that you need to spend some money and one was suggesting you need to hire a market research consultant. While I do not dispute the idea that you need to spend money to make money and run your business, but I think they both were missing the point. The point is sites like control.com can be the back-bone of a community. Instead of killing the effort right there, try to promote it. You will benefit too, hey, it is public domain, everybody benefits.

I really believe that the days of Yahho-ers are numbered. The days of I have something great in my box, pay me to open it for you are gone. Put the information on the table, get people to your site, give to community and be sure that the community returns your favor.

Take care for now, Rod

Posted by rnafisi at 08:11 PM | Permalink | TrackBacks (0)

Simple M2M Solution?

A couple of months ago, we gave a technical proposal/solution to an international client. Sometimes it is like a magic how the clients find you. It looks like Internet facilitated new communities to form in huge scales. This helps us, small consultants and engineering firms, to let the world know about us through these communities.

Anyway, getting back to the topic of M2M, we gave them a solution that think will work. For some reason, that initiative never became a project. I thought to post the proposed solution here. It may have some values to you as a consultant or developer.

To access the document please click here:

Download file

Take care for now,

Rod.

Posted by rnafisi at 02:24 PM | Permalink | TrackBacks (0)

Hello World!

Sometimes it looks like a funny title, but I guess this is how any book or article about programming starts with. I thought this might not be a bad start. However, typically the author has something under this title to show how easy a program can be put together. The easy stuff first and of course a bit tricky code comes later.

I must confess I do not have the skeleton program nor the intention of put the easy stuff on the front and the tricky ones later. I want to talk about the stuff I like in no particular order. Here is what I like to talk about:

Data communication

Network programming

Real-time programming

Data communication protocols

DNP

IEC 61850

Modbus

ICCP

Utility automation

Substation Automation

Distribution Automation

SCADA systems

Fault detection and recovery

Circuit modeling

Data modeling

Wireless communication

Embedded systems

…

And a bunch of other boring stuff.

To add a bit fun to the blogs, I like to talk about the business of the pushing bits left and right as well. Oh, I forgot to mention: I work for myself. I am right now at the point trying to put together a business from scratch. Sometimes funny things happen. I try to remember and share.

That is it for now, I see you soon with some boring thoughts.

Rod

Posted by rnafisi at 02:08 AM | Permalink | TrackBacks (0)