Important Terms:
SM - System Management
CTK - Conformance Test Kit. In this case, H1 CTK release 4.0.0.
SUT - System Under Test. In this case, the H1 LinkMaster stack being tested for conformance.
LM - an H1 LinkMaster stack.
UTA - Upper Tester Agent. A software layer applied to a candidate H1 stack that allows the CTK to test the conformance of the candidate stack.
LAS - Link Active Scheduler. The H1 stack on a Foundation Fieldbus network segment that controls the network.
FFusb - The Softing USB Test System H1 LAS Stack that communicates to a SUT with an applied UTA.
CT - Compel Time. A system management message from an H1 LM stack requesting current SM Time from the Time Manager on the segment.
Bug Description:
A stack tested for the very first time with H1 CTK 4.0.0 received random "inconclusive" verdicts with several different System Management test cases. The test case that exhibited this "inconclusive" verdict the most was C9011002. When the test cases did not have an "inconclusive" verdict, they received a "passed" verdict.
When a test procedure of all System Management test cases was executed continuously, and then the inconclusive log files were compared to the passing log files of the same test case, a common, unusual behavior from the SUT was identified in the "inconclusive" log files.
The following steps describe the unusual behavior from the inconclusive logs:
1. LM SUT under test is the LAS of an H1 segment that includes only the LM SUT. This is the condition of the segment between each test case in the CTK that is executed against a candidate LinkMaster stack.
2. An SM test case launches. FFusb joins the network with the LM SUT and claims the LAS from the LM SUT.
3. FFusb becomes the LAS on the segment.
4. Test case code begins to execute the purpose of the SM test case.
5. FFusb sends an SM Identify Request to the LM SUT and expects the LM SUT to immediately provide an SM Identify Response.
6. Instead of sending an SM Identify Response, the LM SUT intermittently sends a CT message without setting the final-token-use bit in the CT message to "Final". This means the LM SUT still retains the token for communication.
7. The LM SUT then sends the SM Identify Response to the LAS and returns the communication token to the LAS.
Analysis of Bug:
The behavior of the LM SUT in this case is unusual, but completely legal according to the IEC 61158 Data Link Layer Specification.
The CT message without the final-token-use bit being set by the LM SUT is completely unexpected by the test case logic and the FFusb test system stack in the conditions outlined above in steps 1-7.
As a result, when the the FFusb LAS stack sees the CT message before the expected SM Identify Response, test case logic in the SM Test cases and timing problems in the firmware of FFusb combine to cause the reported "inconclusive" result. The SUT behavior is correct. This is a test system bug.
Only one H1 stack supplier's LM stack code has ever exhibited this unusual, but legal behavior in the FCG lab.
Solution:
Softing AG was able to duplicate this bug and fix it in a firmware patch for FFusb in May 2020. All previously tested H1 CTK 4.0.0 conformant stacks were retested with this firmware patch to verify this solution does not cause any new problems for previously tested, conformant stacks. No new problems were found.
H1 CTK 4.0.0 license holders with SUT samples exposing this bug were contacted and the firmware patch was provided so they can avoid this bug in their future development projects.
Since this bug appears to be isolated to the way FFusb interacts with a specific stack manufacturer's code, and a solution has been provided to that stack manufacturer, FieldComm Group is not currently planning a new release of the H1 CTK at this time.
If a license holder of H1 CTK 4.0.0 believes they see this behavior and they have not been contacted by FCG staff, please open a support ticket for further assistance. If it is determined that the firmware patch is necessary, FCG will supply it freely to licensed product owners in need of this solution.