I have a few Fusion controllers that have been in operation for a few months now, but have developed an odd behavior. I can communicate with the boards, via both the NCD Base software and via my own software, but the boards do not act upon commands sent.
For example, using Base, I can send commands via the Pro XR screen to turn on/off relays, but the relays do not trigger. However, the relay status change is represented in the software, and the commands can be seen transmitted back to the board, as well as back to computer over the network (sniffing lantronix TCP protocol using Wireshark). While connected with Base, the busy light on the Fusion controller blinks rapidly. Additionally, checking the input states of the AD input board is not handled. I do not receive any traffic back on input changes (I have push notifications configured).
I receive the same behavior with my software. I can send a command to the controller and receive an ACK, but no action occurs on the controller.
I am off site, but will have one of my techs able to look at it tomorrow, and will be able to remotely access the boards that are exhibiting this behavior.
The configuration of each Fusion controller is as follows:
Fusion 4 Chnael DPDT (FR41DPDT)
TLEE Module
Lantronix XPORT module
AD GPIO board (using for input only to detect switch contact).
Any questions or advice would be greatly appreciated.
Usually I see this under the 3 following circumstances.
The power supply for the board is too low powered. This allows for enough power for the on-board logic, but not enough to drive relays (usually the voltage is too low). This seems unlikely as this was running properly until recently, but its something to check.
Go into the Fusion Master Control Panel in our Base Station Software (ncd.io/start) and make sure that Taralist is currently on and is set to be on at powerup. Then go into the Taralist Enhanced Event Controller in Base Station. Then click on the MORE button in the top right hand side. Click on Advanced Interface. Click on Midnight Backup Buffer. Make sure that all boxes are unchecked on the Midnight Backup Buff panel and then hit save.
Somehow the manual refresh settings became active. This setting is to allow you to set the status of multiple relays without activating them and then send a separate command to change the Relays real state to what has been sent into the board. Here’s a video that talks more about it: https://www.youtube.com/watch?time_continue=49&v=5twukUiuI_Y
Power most likely is good. Still waiting to hear back on that.
Item #2 is most likely culprit. My dev board that mimics what I have out on my site decided to act up in the same exact manner… no input displayed, and no outputs would trigger. Followed through instructions, verifying taralist was on (Current and Powerup status were both set to on), and went through midnight backup buffer and unchecked all options. However, problem still persisted.
After cleaning out all settings on the board that I had configured (Main screen–>Reactor Gen II Auto Relay Controller–> Restore Factory Settings), and fidling with other settings (in all honestly clicking through multiple screens trying to default settings, and not noting them (bad Nick, bad)), the board came back and I was able to control the relays.
Of important note, the Base software was locking up when I was trying to make changes. Moving the PGM/RUN jumper to PGM resulted in smoother operation.
At this point, I am working on breaking my dev board so I can then figure out what worked to fix the problem.
That was my thoughts as well. Reconfigured back to a sane speed (115200bps) and all is stable while I am trying to replicate my problem. Oh well. Going to sit on it for the evening and come back tomorrow. Thanks for the help. Will update if I can (or can’t) replicate the behavior of not activating the relays.
My site finally got back with me on issue. We ordered a replacement setup (4 relay controller/taralist/ethernet module) to see if a full on replacement would work. After going through configuration and matching a good setup that I have back at my office, ran into same issue where relays would not respond, and push inputs would not work. Spent a few hours troubleshooting and going through configuration differences using NCD Base under my remote control on a board that is working as expected and the busted board, and did not come across anything. I am asking that my field guy send me back the board so I can take a look at it in person.
Going back through the recommendations Jacob made (thanks again!), checked the following:
1: Power supply: Was powered off of a 12VDC power supply other than regular power supply.
Had them switch it to power supply that was originally ordered with the first board. Measured voltage off power supply at barrel jack at 12.24VDC. Tested operation before/after power supply change, with no change in behavior.
2: Taralist set to on at boot and was reported as being on. All other settings (midnight mode/proxr control over relay) was applied to midnight backup buffer. In the midnight backup buffer screen, in checkboxes on right hand side of screen under Taralist/ProXR Control section, all checkboxes were selected with with ProXR has control. Unchecking each item under the section results in Taralist has control. I tested with the setting applied both ways, but did not get any difference in results, other than observing that the relay status was under control of ProXR when the relay boxes were checked (ProXR has control in midnight buffer screen); relays were under control of taralist when same checkboxes were not checked (with text Taralist has control) in buffer screen.
I was able to replicate issue with turning on/off manual refreshing via ProXR advanced command set screen on my local board. This however did not change behavior in field.
I am having the site ship me back the original board so I can test in up close and personally.
Thanks for reading, and look forward to getting this resolved.
It sounds like you have checked all the boxes in troubleshooting the Fusion board. I see no reason why ProXR relay control would not work. I’ll be interested to hear what you find once you receive the board back. Perhaps if you can’t find anything once you have the board back in hand we can have a Remote Desktop meeting and I could take a look.
Finally received back fusion relay card from field.
Got it on bench, and connected to it with Base utility. Was able to control relays (was not able to do that before in the field). Tested board inputs via ADC (they worked), but push notification was not working. Queried Device Information using Base, and Base reported TLEE module was not present. I powered off the controller, checked that the TLEE module was seated well (removing and then reattaching module), and powered up the controller, and then checked again; however, TLEE module was still not present.
Any thoughts? Also what can we do about the bad TLEE module?
Sounds like something is either wrong with the TLEE module itself or the Fusion Processor. I would recommend returning the board for warranty repair by filling out the RMA form here and following the return instructions:
Just note on the RMA that Base Station is not recognizing the TLEE module is attached. They will repair and get it back to you ASAP.