Linear segments and the use of ring technology are the most striking differences between the way plant floor engineers and IT employees use Ethernet technologies.
Ring topology is a network configuration where the connected devices create a circular path for data to travel. Each device gets connected to two others with the last device connected back to the first, creating a circular communication path. Ethernet packets travel from one device to the next until they reach their intended destination or the initial source device. In the IT world, this kind of Ethernet topology is anathema. It’s unreliable, slow and disruptive when new nodes are added.
However, on the factory floor, Device Level Ring (DLR) topology is advantage. The qualities that make ring technology deficient for corporate IT are actually advantageous for the factory floor and for the specific ring implementation, Device Level Ring (DLR), discussed in this article:
Disruption from new nodes is another difference but a moot point for the factory floor. Unlike IT, factory floor systems are very rarely added or removed.
The remainder of this article describes the topologies used on the factory floor, ring technologies in general and specifically, the Device Level Ring (DLR) standard promoted by the ODVA.
You can find three kinds of networks in a plant (Table 1). Even though much of the infrastructure that you can see in the information and control networks appears to be identical – cables, switches, routers and the like – there is both a philosophical and practical difference to how factory floor networks operate.
|Information Network||Control Network||IO Network|
|Scope||Plantwide Information||Area wide||I/O Devices|
|Response Required||Best Effort||Soft Real Time||Either Soft or Hard Real Time|
|Devices||Servers, End User Computers||PLCs, PPCs1||Either Soft or Hard Real Time|
|How Used||Connect end users to corporate resources and the internet||Coordination of activities between devices||PLC to I/O device(s)|
|Typical Interaction||End user accessing server||PLC to PLCs and PPCs using bidirectional EtherNet/IP implicit messaging||EtherNet/IP implicit message connection from PLC to end device|
Table 1 – List of Different Plant Floor Networks
Philosophically, IT or enterprise networks operate more like a utility. The IT department provides resources and services to an ever-changing set of customers with constantly changing service requirements. IT uses a centralized management philosophy to accomplish their objective of protecting the corporate assets and their customers from each other and the outside world. The IT world is a very dynamic one, managed using a very structured infrastructure set: Microsoft Windows computers, Cisco switches and routers, firewalls and the like.
The operational technology (OT) world is quite the opposite. Factory floor networks across a plant are not only quite different from each other but are an integral component of a specific manufacturing machine or production process. These networks are quite static and composed of a very diverse infrastructure of controllers, actuators and sensors developed by a broad base of vendors. Control devices come with unlimited varieties of capabilities, operating parameters, communications capabilities and functionality. In this more static but yet more diverse world, control engineers build custom networks that must meet specific operating requirements dictated by the product being manufactured.
The operating philosophy in the OT world is quite different than the IT world. The philosophy of OT is to build, deploy and test a control network until it meets the reliability, performance and quality goals, then never touch anything again. OT accomplishes this using special topologies, operating mechanisms, protocols and systems.
There are many ways to connect Ethernet nodes together into a network. Early in the life of Ethernet, the Ethernet Bus topology was all that existed. Every Ethernet device was connected by physically tapping into the bus and connecting a device to that tap. Today, there is a wide variety of physical topologies that can be used including ring, bus, mesh and star, with most installations using a star topology.
Star topology (Figure 1) is the most prevalent topology used in industrial control systems. In a control system connected together using a Star topology, all network devices are individually connected to a central switch, which is then connected to a higher-level switch, a manufacturing router or a plant-wide router.
One significant advantage of a star topology is that a cable or device failure is not catastrophic. Star networks are robust and extensible. A node failure does not affect the entire network, and it’s nearly effortless to add nodes to a star network.
Disadvantages include the limitations imposed by the number of available switch ports, switch cost, the additional cabling required for each node and the trust needed in the switch reliability as it is the single point of failure in a star network.
One of the most consequential developments for industrial control networks was the development of a network switch on an integrated circuit (IC). Not only did this radically reduce the costs of network switches, it led to the development of a specialized three-port IC that could be deployed in end devices and enable “daisy-changed” Ethernet networks (Figure 2). Using this technology, Ethernet networks could be deployed that resembled the RS485 Modbus networks of earlier generations.
Linear topology is very prevalent in manufacturing applications containing a large number of devices as the per switch port cost of a device is radically smaller than other EtherNet/IP network topologies. In linear networks, all devices use an embedded three-port switch with two ports for network connections and one port for an internal connection to the device. The advantages of linear topology are lower-cost deployment and the ease of adding additional device nodes.
The big disadvantage to linear networks is that any cable or device failure makes all downstream devices inaccessible. For that reason, many control engineers avoid connecting critical control equipment using linear networks. Another, less obvious problem is linear segmenting tends to hide performance problems from the control engineer. If the autonegotiation sequences between one pair of devices in a linear segment results in half-duplex or lower baud rate operation, message traffic downstream of that portion of the segment will operate less efficiently. This problem is hard to detect but not difficult to prevent. All devices in a linear segment should be configured for full-duplex with the highest possible baud rate for the network and autonegotiation disabled.
Some manufacturers prefer to use a hybrid topology, a combination of the other topologies (Figure 3). In hybrid type networks, two or more of the other topologies are implemented in different parts of a manufacturing system to take advantage of the different features of each topology. While this provides the control system designer with the various advantages of the different technologies, it adds to the complexity of the system and makes it less maintainable.
Figure 3 – Hybrid topology
Ring topology (Figure 4) is very prevalent in manufacturing applications where network and process availability are key requirements. In ring type networks, devices are daisy-chained to each other with the last device connected back to the first device and messages sometimes traveling in either direction around the ring. Usually, some sort of a ring master manages traffic on the network, preventing messages from circling endlessly and managing device and network failures. Devices lacking the dual Ethernet port required to participate in a ring can use a tap device to participate in a ring network. The significant advantage of a ring topology is that control systems can continue operation after a cable or even device failure. The major disadvantage to rings is significant additional network complexity and the extra cost of ring enabled end-devices and tap devices.
Figure 4 – Ring topology
Control engineers building systems using linear segmenting discovered the one big disadvantage of linear segmenting: when a device failed, all the downstream devices became inaccessible. Any device or cable failure – especially one near the switch – could disable nearly the entire segment. In high availability systems, these failures could be catastrophic.
In normal DLR operation, the ring supervisor continually sends Beacon frames through the ring to verify ring integrity. Beacon frames arriving at the secondary port validate ring integrity. Cable or device failure are detected when the ring supervisor fails to receive Beacon frames on its secondary port. When a failure is detected, a ring supervisor switches into non-DLR operating mode, treating the ring as two linear segments – one operating from its primary port and one operating from its secondary port.
While in linear operation, the ring supervisor continues to send Beacon frames from its primary port. Once the failure is repaired, the Beacon frames again begin to arrive at the secondary port, and the supervisor restores DLR active status to the ring.
One of the advantages of ring networks is the ability to detect the location of a failure. Ring nodes sensing the loss of a next-door neighbor report the failure to the primary port of the ring supervisor. That process works best if both neighbors have DLR capabilities.
In Figure 2, a failure at ‘A’ would be reported to the ring supervisor by DLR Node Y. A break at ‘B’ though would be reported by the Tap at DLR Node Z and could only be located as someplace between node Z and node Y.
DLR technology is single fault tolerant. The network fails on multiple simultaneous faults in the ring. Another disadvantage of DLR is additional complexity. The DLR object must be configured at each ring node.
It is possible to argue that Device Level Ring is irrelevant in discussing the operation of an EtherNet/IP network. Device Level Ring is a physical architecture that describes a topology and organization for a network. It says nothing about how that network is used. EtherNet/IP, on the other hand, is about messaging and data organization. It says nothing about how its messages move and over what physical network.
Where EtherNet/IP and Device Level Ring come together is at the embedded Ethernet switch in the EtherNet/IP device. The Ethernet switch must process the messages it receives. Application layer EtherNet/IP messages are processed as they normally would in a star topology network. DLR protocol messages are processed per the Device Level Ring specification. These messages maintain the ring and are never received by the EtherNet/IP device application layer.
The Ethernet switch is key to the operation of EtherNet/IP on a Device Level Ring network. If you consult the ODVA specifications, there are several Ethernet embedded switches that are qualified for EtherNet/IP operation.
How the embedded switch functions is described by the parameters of the EtherNet/IP Device Level Ring object, the subject of the next section.
DESCRIPTION: The Device Level Ring (DLR) object is the configuration and status interface to a Device Level Ring network. It contains the configuration parameters that describe how the embedded switch functions on the Device Level Ring network. A controller or network configuration tool uses CIP commands to set the DLR Object parameters of each device on the Device Level Ring network.
The Device Level Ring Object is required for devices that supports the Device Level Ring protocol.
INSTANCES: EtherNet/IP devices can be connected to multiple Device Level Ring networks. An instance of the Device Level Ring object is required for each Device Level Ring network.
CLASS ATTRIBUTES: There are no class-specific attributes in the Device Level Ring Object.
INSTANCE ATTRIBUTES: The Device Level Ring Object supports only 19 attributes:
|1||Network Topology||Identifies the current technology mode (Linear or Ring)|
|2||Network Status||Indicates the current status of the Device Level Ring network|
|3||Ring Supervisor Status||Indicates if the device is functioning as a device supervisor|
|4||Ring Supervisor Config||Provides access to the configuration parameters for Device Level Ring operation|
|5||Ring Fault Count||Number of times a ring fault was detected since power on|
|6||Last Active Node 1||Identifies the last reachable node on the 1st port when the device is functioning as the active ring supervisor|
|7||Last Active Node 2||Identifies the last reachable node on the 2nd port when the device is functioning as the active ring supervisor|
|8||Ring Participant Count||Indicates the number of participants in the ring. Set to zero when the device is not the active supervisor|
|9||Ring Participant List||Identifies the list of ring nodes participating in the ring protocol|
|10||Active Supervisor Address||Identifies the IP address of the active ring supervisor|
|11||Active Supervisor Precedence||MISSING|
|12||Capability Flags||Identifies the Device Level Ring capabilities of a device including whether it is announce-based or beacon-based, supervisor capable or redundant gateway capable|
|13||Redundant Gateway Config||Provides access to the configuration parameters for redundant gateway operation|
|14||Redundant Gateway Status||Indicates the device’s status as a gateway|
|15||Active Gateway Status||The IP Address of the active gateway device|
|16||Active Gateway Precedence||The Precedence value of the active gateway device|
|17||Ring Port 1 Link Object Instance||The Link Object instance for Port 1|
|18||Ring Port 2 Link Object Instance||The Link Object instance for Port 2|
|19||Device Level Ring Enable||Identifies if Device Level Ring operations are enabled for the ports associated with the Device Level Ring Instance|
COMMON SERVICES: The Device Level Ring Object supports several common services including Get Attribute All and Get/Set Single Attribute.
1PPCs – Peripheral Process Controllers (PPCs) are intelligent Ethernet devices like pumps, fluid applicators, glue systems, paint controllers…etc.
2ARC Advisory Group, a leading research and advisory firm focusing on technologies and best practices in automation technology, developed the original version of this table.