SOLVED after reboot firewall is wide open

chris_c_

Active Member
Joined
Aug 19, 2010
Messages
509
Reaction score
67
Make /etc/rc.local work in CentOS 7
Due to systemd introduced in CentOS 7, the /etc/rc.local nis NOT enabled by default, you should make /etc/rc.local executable by hand

Just run
Code:
~$ chmod a+x /etc/rc.local
~$ reboot

and your CentOS 7 will reboot, see if it works.

** CAUSE OF THIS BUG: THE ABOVE COMMAND ("chmod a+x /etc/rc.local") WAS MISSING FROM THE CENTOS INSTALLER IncrediblePBX13-12R-OAUTH.sh therefore the script was not marked as executable and did not execute... **
 

chris_c_

Active Member
Joined
Aug 19, 2010
Messages
509
Reaction score
67
HOWEVER: Fail2Ban appears to still be failing to ban the robo hackers. I suspect it is just not configured properly.
 

dicko

Still learning but earning
Joined
Oct 30, 2015
Messages
1,607
Reaction score
826
The problem you are having that the

/usr/local/sbin/iptables-restart

script relies on "fully qualified domain" access, that just won't work until your network/name service is working . . .

Fail2ban working properly is an addendum to your extant iptables rules, if you restart iptables without then restarting/force-reloading fail2ban, then . . . .
 
Last edited:

jerrm

Guru
Joined
Sep 23, 2015
Messages
838
Reaction score
405
I agree. IPTables is a service. It's meant to start up all by itself, and load the rules from /etc/sysconfig/iptables all by itself, with zero dependence on further scripting commands from rc.local
Yes, it should, but the iptables restore bombs out if any rule fails.

The system loads iptables BEFORE the network is started. Otherwise, the system could be exposed for some (minimal) length of time.

Without a network active when iptables is loaded, host names can't be resolved and therefore nothing is loaded. Host names should not be in startup rules.

Use a minimal, bullet proof, secure template for startup rules then add customizations after boot using whatever startup mechanism fits the platform.
 

chris_c_

Active Member
Joined
Aug 19, 2010
Messages
509
Reaction score
67
The problem you are having that the

/usr/local/sbin/iptables-restart

script relies on "fully qualified domain" access, that just won't work until your network/name service is working . . .

Fail2ban working properly is an addendum to your extant iptables rules, if you restart iptables without then restarting/force-reloading fail2ban, then . . . .
This is a major design flaw.
Zero security on the server when a hostname, in the iptables rule set, can't resolve for any reason.
FQDN was added following this step on the install blog entry:
Add IPtables WhiteList entries for remote access: /root/add-ip or /root/add-fqdn
Source: http://nerdvittles.com/?p=14183
The FQDN is a dynamic hostname of a mobile admin machine, IP address is subject to change, FQDN is the right way to refer to the machine.
 
Last edited:

jerrm

Guru
Joined
Sep 23, 2015
Messages
838
Reaction score
405
HOWEVER: Fail2Ban appears to still be failing to ban the robo hackers. I suspect it is just not configured properly.
The iptables-restart script should load fail2ban. If you manually issue "service iptables restart" or equivalent you kill the fail2ban rules.

Are the fail2ban rules loaded? What is the output of "iptables -vnL | head?" The first few lines should have "f2b-..." targets.
 

chris_c_

Active Member
Joined
Aug 19, 2010
Messages
509
Reaction score
67
On this Centos 7 server, there is about 1 second between iptables attempts to go up with the FQDNs added by the /root/add-fqdn script, and fails, and networking is up and working.

@jerrm @dicko Yuo're right, bug would be solved if add-fqdn would stop adding FQDNs to the main rule set, and only add FQDNs to a separate file that got loaded after networking came up.
 

jerrm

Guru
Joined
Sep 23, 2015
Messages
838
Reaction score
405
On this Centos 7 server, there is about 1 second between iptables attempts to go up with the FQDNs added by the /root/add-fqdn script, and fails, and networking is up and working.

@jerrm @dicko Yuo're right, bug would be solved if add-fqdn would stop adding FQDNs to the main rule set, and only add FQDNs to a separate file that got loaded after networking came up.
The execution order is intentional, not some quirk of timing.

I had assumed the ipchecker process handled the fqdn's, never looked that closely since my rule setup is the first thing I put in place.
 

chris_c_

Active Member
Joined
Aug 19, 2010
Messages
509
Reaction score
67
Am getting this error with fail2ban 0.9.3 on CentOS 7.
These lines are from the relevant section of /etc/fail2ban/jail.conf
Code:
[asterisk]

enabled  = true
filter  = asterisk
action  = iptables-multiport[name=asterisk-tcp, port="5060,5061", protocol=tcp]
  iptables-multiport[name=asterisk-udp, port="5060,5061", protocol=udp]
  iptables-allports[name=ASTERISK, protocol=all]
  sendmail-whois[name=ASTERISK, dest=root, [email protected]]
logpath  = /var/log/asterisk/full
maxretry = 5
bantime  = 1800
Fail2ban is complaining, Action iptables-multiport already exists
Code:
Sep 03 01:25:01 mydomain.com fail2ban-client[16206]: 2016-09-03 01:25:01,768 fail2ban.server  [16224]: INFO  Starting Fail2ban v0.9.3
Sep 03 01:25:01 mydomain.com fail2ban-client[16206]: 2016-09-03 01:25:01,768 fail2ban.server  [16224]: INFO  Starting in daemon mode
Sep 03 01:25:02 mydomain.com fail2ban-client[16206]: ERROR  NOK: ('Action iptables-multiport already exists',)
Sep 03 01:25:02 mydomain.com systemd[1]: Started Fail2Ban Service.
And so fail2ban only adds the asterisk rule for udp, and not the rule for tcp.
Code:
~$ iptables -nL | grep f2b-
f2b-ASTERISK  all  --  0.0.0.0/0  0.0.0.0/0
f2b-asterisk-udp  udp  --  0.0.0.0/0  0.0.0.0/0  multiport dports 5060,5061
f2b-BadBots  tcp  --  0.0.0.0/0  0.0.0.0/0  multiport dports 80,443
f2b-SSH  tcp  --  0.0.0.0/0  0.0.0.0/0  tcp dpt:22
Chain f2b-ASTERISK (1 references)
Chain f2b-BadBots (1 references)
Chain f2b-SSH (1 references)
Chain f2b-asterisk-udp (1 references)
Anyone else experience this bug??
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
19,170
Reaction score
5,199
If you had only provided the information asked for, we could have resolved this about 20 messages ago. The timing delays in rc.local are intended to assure that IPtables is restarted AFTER your network comes on line. If you have FQDNs in your IPtables rules, that is essential with the current design. The bug, if you want to call it that, is in IPtables for crashing and burning whenever any individual rule fails. That is a major shortcoming that the developers have been unwilling to address. It doesn't do that on failed IP addresses, just FQDNs.

If you had provided the requested info on what you observed during the boot process, we would have known that rc.local was not firing, and that would have prompted a suggestion to check the executable flag on the rc.local file. We've added the fix to the installers for CentOS 7.

Just as an aside, this is the kind of "OS improvement" that drives application developers crazy. There was absolutely no reason for the RedHat developers to change the rc.local default setting, and any developer should have known it would have terrible consequences.
 
Last edited:

jerrm

Guru
Joined
Sep 23, 2015
Messages
838
Reaction score
405
Great way to shoot the messenger.

A MAJOR flaw in the IPBX Centos7 install is discovered and you finger point to the reporter's bug reporting skills, decades old OS behavior and a well documented v7 change.
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
19,170
Reaction score
5,199
Wasn't picking on @chris_c_. Just observing that responding to questions as they are posed tends to move things along towards a solution much quicker.

We can agree to disagree as to whether this was a major flaw. First, it only applies to CentOS 7. Second, it only occurs if the administrator has remote devices. And third, it only occurs if the remote devices use dynamic DNS addressing for access. But, rest assured, we will be pushing out a security fix to every CentOS server shortly.
 

chris_c_

Active Member
Joined
Aug 19, 2010
Messages
509
Reaction score
67
@chris_c_ please name the hosted provider, I had the same issue with cloud@cost's base centos7 vm
The bug wasn't in centos 7 nor with the kvm host provider, the bug was in the IncrediblePBX13-12R-OAUTH2.sh CenOS installer script. Someone who wrote the installer script *cough* an imperfect human capable of making mistakes just like everyone *cough* forgot to set the rc.local as executable and it totally failed to run, and was exposing the server wide open to robo hack attack from the entire planet of evil mastermind asterisk toll fraud hackers, with absolutely no fail2ban protections.
 
Last edited:

chris_c_

Active Member
Joined
Aug 19, 2010
Messages
509
Reaction score
67
First, it only applies to CentOS 7. Second, it only occurs if the administrator has remote devices. And third, it only occurs if the remote devices use dynamic DNS addressing for access.
1. CentOS 7 = has been out for 2 years, is the latest and greatest version of the recommended OS technology distribution for FreePBX and by extension IncrediblePBX.
2. Remote devices = Probably 99.999% of administrators have remote devices these days.
3. FQDN in iptables whitelist = Probably all remote devices are mobile needing a FQDN dynamic hostname to compensate for changing IP. Laptop, tablet, phone.
Must support all 3 in order to provide a positive smooth experience that is not a pain in the neck ie vulnerable to hackers and not alerting you!
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
19,170
Reaction score
5,199
56496333.jpg
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
19,170
Reaction score
5,199
We've only been supporting CentOS 7 deployments for about a month. Most still don't use it in production. Patch already has been pushed out. Existing iptables-restart script will remove FQDN entries from active rules IF network is down. For that to work, rc.local needs to be functioning just as it always has with CentOS 6, Ubuntu 14, and Raspbian builds. There are plenty of other options out there if free, open source Incredible PBX code doesn't meet your requirements.
 

chris_c_

Active Member
Joined
Aug 19, 2010
Messages
509
Reaction score
67
@wardmundy Let me be probably the 1000th or 10,000th one to say I appreciate all you've done with IncrediblePBX. Bug reports we make are in the spirit of improving it so it runs better. As for receiving bug reports on an open source project, it's important to not take things quite so personally, because that discourages people from reporting bugs, which in turn decreases the quality as further bugs don't get reported in order to avoid the awkwardness.

Everyone makes mistakes from time to time, we're all human, we make the fix, and move forward. The important part is to learn how did the bug actually make its way in, so as to hopefully prevent it from reoccurring in the future. I have to say it's due to not enough people reviewing the scripts, plus the avoidance of the backlash which occurs when reporting bugs here. "Thousands of people have downloaded this, and you're the only only one reporting this bug, so you must be an idiot, or your server must be defective" is the wrong attitude to display, most people will decline to report bugs with the expectation of that type of reaction.
 

chris_c_

Active Member
Joined
Aug 19, 2010
Messages
509
Reaction score
67
Another bug: in iptables-restart, it's trying to use a non-existant /etc/iptables/rules.v4 which appears to be the wrong path on CentOS 7.
It looks like these are the rules files it should be looking for
Code:
/etc/sysconfig/iptables
/etc/sysconfig/ip6tables
/etc/sysconfig/rules.v4.ubuntu
/etc/sysconfig/rules.v6/ubuntu
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
19,170
Reaction score
5,199
Thanks, @chris_c_. It was worse than that. All sorts of issues with that script on the CentOS 7 platform. Try this one and we'll push it out:
Code:
cd /usr/local/sbin
wget http://incrediblepbx.com/iptables-restart-centos7.tar.gz
tar zxvf iptables-restart-centos7.tar.gz
rm -f  iptables-restart-centos7.tar.gz
 

chris_c_

Active Member
Joined
Aug 19, 2010
Messages
509
Reaction score
67
That version, while it refers to the right paths on CentOS 7 systemd for the iptables rules files, still does not fix the bug from post #29 above, where the `iptables` rule which supposed to be generated by `fail2ban`, `f2b-astrerisk-tcp`, is missing from `iptables -nL`, due to an error in `/etc/fail2ban/jail.conf` section `[asterisk]`.
Also see https://github.com/EspaceNetworks/IncrediblePBXCentOS7/issues/7
 

Members online

Forum statistics

Threads
25,782
Messages
167,513
Members
19,203
Latest member
frapu
Get 3CX - Absolutely Free!

Link up your team and customers Phone System Live Chat Video Conferencing

Hosted or Self-managed. Up to 10 users free forever. No credit card. Try risk free.

3CX
A 3CX Account with that email already exists. You will be redirected to the Customer Portal to sign in or reset your password if you've forgotten it.
Top