So for a while now, I noticed that the AirPort Extreme would have this blinking yellow light. Everything is working, but the light keeps blinking.
To solve this, start up the Airport Utility, select the device on the left, and at the bottom left corner of the main panel there would be a text along the lines of “Press Continue to solve detected problem”. In my case it was just a new firmware update available.