1. Home
  2. Networking Firewalls
  3. FW1 NGX R62 on IP530 issue

FW1 NGX R62 on IP530 issue

Hello,

We have a cluster of firewall checkpoint FW1 NGX R62 running on Nokia IP530 platforms. Since, I upgraded to R62 (from R55), every 3/4 days, the master firewall stop running correctly, it doesn't forward the traffic anymore but the hardware is still up. So the failover is not working. I have to switch to the backup one manually. Here, what can I see on the logs of the firewall when it fail:

Jan 7 01:42:51 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:42:55 egb [LOG_CRIT] kernel: cpas_pcb_connect: received NULL pcb as argument, failing Jan 7 01:42:55 egb [LOG_CRIT] kernel: cpas_pcb_connect: received NULL pcb as argument, failing Jan 7 01:42:58 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:42:58 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:42:58 egb [LOG_CRIT] kernel: cpas_pcb_connect: received NULL pcb as argument, failing Jan 7 01:44:01 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:44:06 egb last message repeated 3 times Jan 7 01:44:10 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (90, 0, 0)! Jan 7 01:44:10 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:44:13 egb [LOG_CRIT] kernel: cpas_tcp_input: failed to allocate cpas_pkt_buf for in-sequence data Jan 7 01:44:14 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:44:18 egb [LOG_CRIT] kernel: FW-1: fwtcpstr_init_stream: error allocating kbuf Jan 7 01:44:19 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:44:21 egb [LOG_CRIT] kernel: cpas_tcp_input: failed to allocate cpas_pkt_buf for in-sequence data Jan 7 01:44:26 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:46:25 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:46:25 egb last message repeated 5 times Jan 7 01:46:28 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (91, 0, 0)! Jan 7 01:46:28 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:46:28 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:46:31 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (79, 0, 0)! Jan 7 01:46:31 egb [LOG_CRIT] kernel: FW-1: loghandle_create: failed to allocate memory for the log handle Jan 7 01:46:31 egb last message repeated 2 times Jan 7 01:46:31 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (91, 0, 0)! Jan 7 01:46:34 egb [LOG_CRIT] kernel: fwhandle_get(fwseqvalid.c:914): Table kbufs - Null handle requested Jan 7 01:46:34 egb [LOG_CRIT] kernel: cpas_tcp_input: failed to allocate cpas_pkt_buf for in-sequence data Jan 7 01:46:35 egb [LOG_CRIT] kernel: fwhandle_get(fwseqvalid.c:914): Table kbufs - Null handle requested Jan 7 01:46:37 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (91, 0, 0)! Jan 7 01:47:23 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (79, 0, 0)! Jan 7 01:47:26 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (79, 0, 0)! Jan 7 01:47:32 egb [LOG_CRIT] kernel: FW-1: loghandle_create: failed to allocate memory for the log handle Jan 7 01:47:32 egb [LOG_CRIT] kernel: FW-1: fw_xlate_packet: fw_xlate_log failed Jan 7 01:47:35 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (90, 0, 0)! Jan 7 01:47:35 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:47:38 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (90, 0, 0)! Jan 7 01:47:39 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:47:39 egb last message repeated 4 times Jan 7 01:47:45 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (79, 0, 0)! Jan 7 01:47:46 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:47:58 egb last message repeated 4 times Jan 7 01:48:05 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (29, 0, 0)! Jan 7 01:48:05 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:48:43 egb last message repeated 2 times Jan 7 01:48:48 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (32, 11, 0)! Jan 7 01:48:48 egb [LOG_CRIT] kernel: fwhandle_get(fwseqvalid.c:914): Table kbufs - Null handle requested Jan 7 01:48:48 egb last message repeated 2 times Jan 7 01:48:58 egb [LOG_CRIT] kernel: FW-1: loghandle_create: failed to allocate memory for the log handle Jan 7 01:48:58 egb [LOG_CRIT] kernel: Error: FW-1 failed to generate the log record. Jan 7 01:48:58 egb [LOG_CRIT] kernel: FW-1: fwx_create_xlbuf: error allocating kbuf Jan 7 01:48:58 egb [LOG_CRIT] kernel: FW-1: fwx_get_xlation: fwxl is NULL Jan 7 01:48:58 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:48:58 egb [LOG_CRIT] kernel: FW-1: fw_spii_new_h: failed to put data in kbuf. Jan 7 01:48:58 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:49:17 egb last message repeated 5 times Jan 7 01:49:20 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (29, 0, 0)! Jan 7 01:49:22 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:49:22 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:49:35 egb [LOG_CRIT] kernel: fwhandle_get(fwseqvalid.c:914): Table kbufs - Null handle requested Jan 7 01:49:35 egb [LOG_CRIT] kernel: cpas_tcp_input: failed to allocate cpas_pkt_buf for in-sequence data Jan 7 01:49:36 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:49:38 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (79, 0, 0)! Jan 7 01:49:38 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:49:38 egb [LOG_CRIT] kernel: FW-1: loghandle_create: failed to allocate memory for the log handle Jan 7 01:49:38 egb [LOG_CRIT] kernel: FW-1: fw_xlate_packet: fw_xlate_log failed Jan 7 01:49:38 egb [LOG_CRIT] kernel: fwhandle_get(fwseqvalid.c:914): Table kbufs - Null handle requested Jan 7 01:49:38 egb [LOG_CRIT] kernel: fwhandle_get(fwseqvalid.c:914): Table kbufs - Null handle requested Jan 7 01:49:38 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:49:39 egb last message repeated 2 times Jan 7 01:49:41 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (79, 0, 0)! Jan 7 01:49:41 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:49:41 egb [LOG_CRIT] kernel: FW-1: loghandle_create: failed to allocate memory for the log handle Jan 7 01:49:41 egb [LOG_CRIT] kernel: Error: FW-1 failed to generate the log record. Jan 7 01:49:41 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation Jan 7 01:49:44 egb [LOG_CRIT] kernel: fwhandle_get(fwseqvalid.c:914): Table kbufs - Null handle requested Jan 7 01:49:44 egb [LOG_CRIT] kernel: FW-1: fw_spii_add_inspection: error - failed to write message (4, 0, 0)! Jan 7 01:49:45 egb [LOG_CRIT] kernel: fwlock_call_at_release: no memory available for operation

If anyone have already see those error messages? I found nothing on internet and checkpoint KB. Thanks for your help...

Reply to
guez
Loading thread data ...

hi, have you checked this:

formatting link
do you have enough memory? M

Reply to
mak

Hi,

Yes I'm on Nokia IP530 with 512MB of memory

Thanks

Reply to
guez
1) Monitor your firewall (memory) usage during the 3/4 days while it's working fine using: #fw ctl pstat

You can download a script from the Nokia support site that does this automatically:

formatting link
(Nokia's solution id 11093)

2) When you switch to the standy firewall does this one also has the same problem after 3/4 days?

Br. Robby

Reply to
Robby Cauwerts

Thanks for your response

1) We have already a monitoring tool which monitor the firewalls. And I didn't see any peak or saturation of memory before the problem occurs.

2) Yes, the same problem occurs independently that it's the first or second firewall of the cluster which is master.

Regards David

Reply to
guez

"fw ctl pstat" will show you more information then just general system memory usage. The Check Point kernel only uses a part of the available system memory. This part can be variable or a hard limit (in which case you might need to increase it.)

Do some research on the output of "fw ctl pstat" (hmem, kmem, failed alloc,...) . There's a lot of information in it. Looking at the errors you posted you'll probably find some answers in the output of it.

Br. Robby

Reply to
Robby Cauwerts

Cabling-Design.com Forums website is not affiliated with any of the manufacturers or service providers discussed here. All logos and trade names are the property of their respective owners.