Comment 4 for bug 1906280

Revision history for this message
Billy Olsen (billy-olsen) wrote : Re: Charm stuck waiting for ovsdb 'no key "ovn-remote" in Open_vSwitch record'

The charm looks like its been executing the hook for quite awhile:

          octavia-ovn-chassis/1:
            workload-status:
              current: waiting
              message: '''ovsdb'' incomplete'
              since: 27 Nov 2020 23:36:17Z

while the latest entry in the unit log is about 4 hours later:

2020-11-28 03:43:30 DEBUG juju.worker.uniter.remotestate watcher.go:636 update status timer triggered

And the last thing of relevance that the charm was doing was creating the integration bridge 'br-int':

2020-11-27 23:36:55 INFO juju-log ovsdb:195: Creating bridge br-int

Looking at the syslog, it appears that creating the bridge is causing apport to detect a crash:

Nov 27 23:36:54 juju-7f6c1c-1-lxd-8 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl set open . external-ids:ovn-remote=ssl:10.246.64.225:6642,ssl:10.246.64.250:6642,ssl:10.246.65.21:6642
Nov 27 23:36:55 juju-7f6c1c-1-lxd-8 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl -- --may-exist add-br br-int -- set bridge br-int external-ids:charm-ovn-chassis=managed -- set bridge br-int protocols=Ope
nFlow13,OpenFlow15 -- set bridge br-int datapath-type=system -- set bridge br-int fail-mode=secure -- set bridge br-int other-config:disable-in-band=true
Nov 27 23:36:55 juju-7f6c1c-1-lxd-8 networkd-dispatcher[222]: WARNING:Unknown index 2 seen, reloading interface list
Nov 27 23:36:55 juju-7f6c1c-1-lxd-8 systemd[1]: system-apport\x2dforward.slice: Failed to reset devices.list: Operation not permitted
Nov 27 23:36:55 juju-7f6c1c-1-lxd-8 systemd[1]: Created slice system-apport\x2dforward.slice.
Nov 27 23:36:55 juju-7f6c1c-1-lxd-8 systemd[1]: Starting Apport crash forwarding receiver...
Nov 27 23:36:56 juju-7f6c1c-1-lxd-8 systemd[1]: Started Apport crash forwarding receiver.

which is confirmed in the apport log file:

ERROR: apport (pid 71613) Fri Nov 27 23:36:55 2020: called for pid 17618, signal 6, core limit 34359738368, dump mode 1
ERROR: apport (pid 71613) Fri Nov 27 23:36:55 2020: executable: /usr/lib/openvswitch-switch/ovs-vswitchd (command line "ovs-vswitchd unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach")
ERROR: apport (pid 71613) Fri Nov 27 23:36:55 2020: is_closing_session(): no DBUS_SESSION_BUS_ADDRESS in environment
ERROR: apport (pid 71613) Fri Nov 27 23:36:56 2020: wrote report /var/crash/_usr_lib_openvswitch-switch_ovs-vswitchd.0.crash
ERROR: apport (pid 71613) Fri Nov 27 23:36:56 2020: writing core dump to core (limit: 34359738368)
ERROR: apport (pid 71613) Fri Nov 27 23:36:56 2020: writing core dump core of size 27217920

So this appears to be ovs crashing when creating the bridge.