MCP23008 input board stops responding

Currently using this as the mainboard:
With this as the controller:
With the input board attached via I2C cable:
And a 12vdc meanwell power supply

It had been working fine for years running an ESPHome-based software stack. Recently, the input board (which takes 12 volt inputs) has started flaking out by reporting all inputs as high, even those that don’t have anything connected. The relays (also i2c connected) still work just fine when this happens. There are two things that will temporarily fix it for another 1-3 weeks: Either power cycling the whole system, or just power cycling only the input board.

Things I have tried with no luck:

  • rebooting the controller
  • powering it via either the external or bus power
  • swapping the controller for a backup controller.
  • Different power supply for the system

Any ideas here?

Thank you!

That will be a tough one to troubleshoot if the failure only happens every 1-3 weeks.

If you can figure out a way to cause the failure to occur that would be ideal as rapid troubleshooting would be possible. My guess is something on the inputs is causing the anomaly but it’s hard to say for sure.

The only way that I have been able to immediately get the same failure (all pins reading high), is by simply unplugging the input board while the controller is running.

For the record, I’m currently only using two of the inputs, and yet all inputs read high when it fails:

  1. The first is simply going through a current sense switch.
  2. The second is going through an otherwise unused pole of a contactor.

This sounds like a latch-up condition of the I2C chip, often caused by a voltage spike of some kind. This is very difficult to troubleshoot without a scope to monitor the status. One thing to check once it fails is to see if the both I2C lines are reading 5V (you will need to halt all I2C communication attempts). Also, I saw in your comments that you rebooted the CPU, but you might also try a power-down of the CPU (while the I2C chips remain powered). However, from your explanation, it sounds like the CPU experiencing any problems. Another thing to check is to disconnect anything that might be spiking power into the board, anything that could cause an electrical disruption.

I do have access to a 4 channel scope. Where do you recommend I hook it up?

I would try connecting at the power supply first to see if you are getting spikes.
Next, try the I2C SCL and SDA lines to see if there is a disruption on the data lines.

Thanks! I have also setup an automation to alert me next time it acts up. Hopefully then I will be able to reproduce the events that cause a problem.