multicast data pauses every 5m with IP530

[wancom]

We are having fits trying to make multicast run reliably (and not pause every 5m) via our IP530 cluster.
H/W: IP530
S/W: Checkpoint R55
Running VRRP, although we've disabled both interfaces on the secondary node to eliminate this option. We've tried the cisco option both on and off, and currently have it off.


We have a sender on our itnernal network. It is sending data to to a multicast channel. It is plugged into an internal cisco 6513 switch. The switch is one of two in a switch cluster. The nokias are plugged into these cats and a set of 6513 DMZ switches with receivers. The RP is statically set to a loopback IP on the internal catalysts. There are static /32 host routes in the Nokias pointing the traffic to each of the local neighbor IPs on the internal catalysts.

This is the type of error we get with set trace pim on during a data pause. Any thought would be appreciated.

Thanks

set trace pim on:
(see below for MFC trace)

May 9 17:24:53 task_timer_dispatch: calling PIM_PIM_SM_MRT_MAINTENANCE, late by 0.005
May 9 17:24:53 PIM MRT: Could not add <*,g> join <s=x.x.203.253,g=224.0.1.40> to j/p list of nbr x.x.102.253
May 9 17:24:53 PIM MRT: Could not add <*,g> join <s=x.x.203.253,g=224.1.1.5> to j/p list of nbr x.x.102.253
May 9 17:24:53 PIM MRT: Could not add <*,g> join <s=x.x.203.253,g=224.2.101.4> to j/p list of nbr x.x.102.253
May 9 17:24:53 PIM MRT: Could not add <*,g> join <s=x.x.203.253,g=224.10.x.x1> to j/p list of nbr x.x.102.253
May 9 17:24:53 PIM MRT: Could not add <*,g> join <s=x.x.203.253,g=239.1.255.252> to j/p list of nbr x.x.102.253
May 9 17:24:53 PIM MRT: Could not add <*,g> join <s=x.x.203.253,g=239.1.255.254> to j/p list of nbr x.x.102.253
May 9 17:24:53 PIM MRT: Could not add <*,g> join <s=x.x.203.253,g=239.1.255.255> to j/p list of nbr x.x.102.253
May 9 17:24:53 PIM MRT: Ignoring <s=x.x.200.120,g=239.1.255.255> entry due to no iif info
May 9 17:24:53 PIM MRT: Could not add <*,g> join <s=x.x.203.253,g=239.11.0.1> to j/p list of nbr x.x.102.253
May 9 17:24:53 PIM MRT: Could not add <*,g> join <s=x.x.203.253,g=239.11.0.2> to j/p list of nbr x.x.102.253
...
May 9 17:24:53 PIM MRT: Could not add <*,g> join <s=x.x.203.253,g=239.12.0.7> to j/p list of nbr x.x.102.253
May 9 17:24:53 PIM MRT: Could not add <*,g> join <s=x.x.203.253,g=239.13.0.1> to j/p list of nbr x.x.102.253
May 9 17:24:53 PIM MRT: Added <S=x.x.200.120,G=239.13.0.1/32>SPT to trigger list of neighbor x.x.101.4
May 9 17:24:53 PIM MRT: Add <s,g> prune <s=x.x.200.120,g=239.13.0.1> to triggered list of nbr x.x.101.4
May 9 17:24:53 PIM MRT: sending join/prune(54) to x.x.101.4 on eth-s1p1c0 (x.x.101.4)
May 9 17:24:53 task_send_packet_iovec: task PIM_IO socket 16 length 54 if_index 3 to 224.0.0.13
PIM SENT x.x.101.4 -> 224.0.0.13 len 34 join/prune:
upstream nbr x.x.101.4 num_groups 1 holdtime 210
group 239.13.0.1/32 num_joins 0 num_prunes 1
prune source x.x.200.120/32 SPARSE
May 9 17:24:53
May 9 17:24:53 PIM RP: active crp x.x.203.253 for group 239.13.0.1 based on highest ip address
May 9 17:24:53 PIM MRT: received a igmp join for group 239.13.0.1 whose rp = x.x.203.253
May 9 17:24:53 PIM MRT: I am not DR on rpf interface towards RP, ignore IGMP membership report for group 239.13.0.1
May 9 17:24:53 PIM MRT: Added group=239.13.0.1 to orphan list on interface eth-s3p1c0
May 9 17:24:53 PIM MRT: Ignoring <s=x.x.200.120,g=239.13.0.1> entry due to no iif info
May 9 17:24:53 PIM MRT: trigger for <S=x.x.200.120,G=239.13.0.1>SPT failed due to no rpf neighbor
May 9 17:24:53 PIM MRT: Ignoring <s=x.x.200.120,g=239.13.0.1> entry due to no iif info
May 9 17:24:53 PIM MRT: Could not add <*,g> join <s=x.x.203.253,g=239.13.0.2> to j/p list of nbr x.x.102.253
...
May 9 17:24:53 PIM MRT: Could not add <*,g> join <s=x.x.203.253,g=239.13.0.6> to j/p list of nbr x.x.102.253
May 9 17:24:53 PIM MRT: Could not add <*,g> join <s=x.x.203.253,g=239.13.0.7> to j/p list of nbr x.x.102.253
May 9 17:24:53 task_timer_dispatch: returned from PIM_PIM_SM_MRT_MAINTENANCE, rescheduled in 29.992
May 9 17:24:55 task_process_sockets: recv ready for PIM_IO
May 9 17:24:55 task_receive_packet: task PIM_IO from x.x.101.253 to 224.0.0.13 socket 16 length 54
May 9 17:24:55 PIM MRT: Processing a join/prune message from x.x.101.253 to 224.0.0.13
May 9 17:24:55 PIM MRT: Processing group address 239.1.255.255/32
May 9 17:24:55 PIM RP: active crp x.x.203.253 for group 239.1.255.255 based on highest ip address
May 9 17:24:55 PIM MRT: Processing <*,g> entry for grp=239.1.255.255/32 rp=x.x.203.253/32
May 9 17:24:55 PIM: Add oif eth-s1p1c0 to src (null) group 239.1.255.255
PIM RECV x.x.101.253 -> 224.0.0.13 len 34 join/prune:
upstream nbr x.x.101.4 num_groups 1 holdtime 210
group 239.1.255.255/32 num_joins 1 num_prunes 0
join source x.x.203.253/32 RPT SPARSE WC


MFC trace during similar event:

May 9 17:50:53 mfc_del_oif: deleting oif eth-s1p1c0 from (239.13.0.1, x.x.200.120)/64
May 9 17:50:53 mfc_krt: CHANGE kernel multicast forwarding entry for (239.13.0.1, x.x.200.120)/64
May 9 17:50:53 mfc_krt: parent IF eth-s3p1c0
May 9 17:50:53 MFCv4 XRSLV SEND Route(1) CHANGE(3) len 204 ver 8 pid 22222 seq 179 timestamp 0.0 table 2 rttype <USER>3 nhtype <MCAST_DIST>8 nhflags <>0 ifindex 0 rd type 0 rib 0 <239.13.0.1,x.x.200.120>/64, mcast type GENERIC(0), iif eth-s3p1c0, oifs:
May 9 17:50:53 mfc_data_driven_prune: deleting kernel forwarding entry for (239.13.0.1, x.x.200.120)/64 to trigger data driven prune
May 9 17:50:53 mfc_get_krt_pktcnt: retrieve usage counts for kernel multicast forwarding entry for (239.13.0.1, x.x.200.120)/64
May 9 17:50:53 MFCv4 CACHE SEND Route(1) GET(5) len 52 ver 8 pid 22222 seq 1081 timestamp 0.0 table 2 rttype <USER>3 nhtype <MCAST_DIST>8 nhflags <>0 ifindex 0 rd type 0 rib 0 <239.13.0.1,x.x.200.120>/64
May 9 17:50:53 MFCv4 CACHE RECV Route(1) GET(5) len 204 ver 8 pid 22222 seq 1081 timestamp 1147193454.213945 table 2 rttype <USER>3 nhtype <MCAST_DIST>8 nhflags <MULTICAST>4 ifindex 0 rd type 0 rib 0 <239.13.0.1,x.x.200.120>/64, mcast type GENERIC(0), iif eth-s3p1c0, oifs:
May 9 17:50:53 mfc_krt: DELETE kernel multicast forwarding entry for (239.13.0.1, x.x.200.120)/64
May 9 17:50:53 MFCv4 XRSLV SEND Route(1) DELETE(2) len 52 ver 8 pid 22222 seq 180 timestamp 0.0 table 2 rttype <USER>3 nhtype <DEAD>0 nhflags <>0 ifindex 0 rd type 0 rib 0 <239.13.0.1,*>/64
May 9 17:50:53 task_process_sockets: recv ready for MFCv4 XRSLV
May 9 17:50:53 mfc_recv:
May 9 17:50:53 task_receive_packet: task MFCv4 XRSLV from *RoutingSocket* socket 14 length 48
May 9 17:50:53 MFCv4 XRSLV RECV Resolve(7) RESOLVE(4) len 48 ver 8 pid 0 seq 76 timestamp 1147193454.221511, resolve type NORMAL(1), pkt src x.x.200.120, pkt dst 239.13.0.1
May 9 17:50:53 mfc_recv: external resolve source x.x.200.120, group 239.13.0.1, pim source (null), pim destination (null), pim border disabled
May 9 17:50:53 mfc_resolve_sg: (re)installing (239.13.0.1, x.x.200.120)/64
May 9 17:50:53 mfc_krt: ADD kernel multicast forwarding entry for (239.13.0.1, x.x.200.120)/64
May 9 17:50:53 mfc_krt: parent IF eth-s3p1c0
May 9 17:50:53 MFCv4 XRSLV SEND Route(1) ADD(1) len 204 ver 8 pid 22222 seq 181 timestamp 0.0 table 2 rttype <USER>3 nhtype <MCAST_DIST>8 nhflags <>0 ifindex 0 rd type 0 rib 0 <239.13.0.1,x.x.200.120>/64, mcast type GENERIC(0), iif eth-s3p1c0, oifs:
May 9 17:50:53 mfc_resolve_sg: installed (239.13.0.1, x.x.200.120)/64 in kernel