Hotfix 160301 - FINAL

CONTENTS DELETED

Typo in rc.local within the hotfix - change

/etc/itus/detect_mode.sh 	# Detects the shield mode

to

sh /etc/itus/detect_mode.sh 	# Detects the shield mode

Or just run it manually within the CLI.  You might need to then force-refresh your browser's cache of the page.

CONTENTS DELETED

Thanks for the 160309 update Hans, and your help with it Roadrunnere42 - very much appreciated!  
Looking good so far, just the rc.local typo with detect mode mentioned previously.  

I think ideally this would only write the file a single time after a factory reset or fw_upgrade, so I wonder if instead of writing every reboot it could just go in a cron or Command Line config to be run manually..  Alternatively maybe the script could check for mismatch before writing.

Also, I suspect most users won't be tweaking their Backup Config file lists - could be worth including it with hotfixes as they grow?

Trying out the snort/trojan tweaks recommended here - no issues so far, possible candidate for a future hotfix.

Gnomad wrote

Thanks for the 160309 update Hans, and your help with it Roadrunnere42 - very much appreciated!  
Looking good so far, just the rc.local typo with detect mode mentioned previously.  

I think ideally this would only write the file a single time after a factory reset or fw_upgrade, so I wonder if instead of writing every reboot it could just go in a cron or Command Line config to be run manually..  Alternatively maybe the script could check for mismatch before writing.

Also, I suspect most users won't be tweaking their Backup Config file lists - could be worth including it with hotfixes as they grow?

Trying out the snort/trojan tweaks recommended here - no issues so far, possible candidate for a future hotfix.

Hi Gnomad

A way to combine both is to script the changes made to /etc/config/dhcp and /etc/config/network using the uci command sequences.
It will also make changes to the /etc/snort/snort.conf file - this is where people have been tweaking things.

As for future improvements for hotfixes, let me know what you've found and I will do my best integrating them.

Hans

Great work with the hotfixs,

After installing the hotfixs, in the system log I'm now getting this error

daemon.err uhttpd[4627]: cat: can't open '/.shield_mode': No such file or directory

is this to do with the hotfix as i've never seen before?


roadrunnere42

Roadrunnere42 wrote

Hans

Great work with the hotfixs,

After installing the hotfixs, in the system log I'm now getting this error

daemon.err uhttpd[4627]: cat: can't open '/.shield_mode': No such file or directory

is this to do with the hotfix as i've never seen before?


roadrunnere42

There was a bug in rc.local. Please run sh /etc/itus/detect_mode.sh or echo "Bridge" > /.shield_mode  if you are in bridge mode.

Hi Hans,

Hans wrote

A way to combine both is to script the changes made to /etc/config/dhcp and /etc/config/network using the uci command sequences.
It will also make changes to the /etc/snort/snort.conf file - this is where people have been tweaking things.

just wondering what you mean by "combine both"?  Not quite sure I follow how dhcp or network configs affect any of what I mentioned there - detect mode, backup file list, or the snort/trojan tweaks..

Hans
 
I just started having a look over the hotfix as I’m starting to have niggling things happen and  I would like to discover why, the first file I have come across /etc/itus/update_blacklist.sh I’m have trouble understanding

can you explain if I’m reading this write as the code after  then will never get run?

# check if blockdomain_ip and blacklist_ip and blockdomain_ip is nor equal to blacklist_ip

if [[ `echo $blockdomain_ip | grep -o $ip_regex` && `echo $blacklist_ip | grep -o $ip_regex` && "$blockdomain_ip" != "$blacklist_ip" ]]
then    

 roadrunnere42

Hi Hans

Started to look over hotfix code in etc/snort/snort.conf what have you changed? if it's ipvar HOME_NET I don't think snort7 /8 conf file has been updated for router mode with same fix.

also have changed the following files which i believe is an improvement ( have trouble uploading to site will just cut and paste.

 write-categories.sh

###################################################################
# update_blacklist.sh                            #
# By: ITUS                            #
# version 2                            #
# Modified: 14th March 2016                    #
# called by: /etc/init.d/dnsmasq                            #
# Purpose: To Goes through the web filter rules and the only one ticked will    #
# be copied into ram, sorted and duplicate one deleted, then copied to    #
# /etc/ITUS_DNS.txt. The tmp file is then removed            #
#                                    #
# changes: roadrunnere42 Added checks for ramdisk, error checking for missing#
#   or blank files, added comments.                    #
# changes: Hans added ram disk feature, orginal code left in.    #
#################################################################################


# Clear files
> /etc/ITUS_DNS.txt

##########################################################################################
# Check to see if there is a mount point in /mnt/ramdisk and if there is'nt it will creat one.
##########################################################################################

# This is used the first time you run this script on the Shield to created the mount point.
        if [ ! -d "/mnt/ramdisk" ] ; then
        mkdir /mnt/ramdisk
        fi

##########################################################################################
# Check to see for /mnt/ramdisk is mounted, if not will create the ramdisk in memory
# should have been created from dnsmasq but just a check
##########################################################################################

if mount | grep /mnt/ramdisk > /dev/null ; then
    echo "yes mounted"
        else
    echo "Creating Ramdisk"
                mount -t tmpfs -o size=50000k tmpfs /mnt/ramdisk
        fi

##########################################################################################
# Goes through and check which rules are ticked from gui and then copies to ramdisk.
##########################################################################################

FILTERS=`grep content_ /etc/config/e2guardian | grep \'1\' | cut -d "_" -f 2 | cut -d ' ' -f 1`
for filter in $FILTERS
do
#        cat "/etc/itus/lists/$filter" >> /etc/ITUS_DNS.tmp
  cat "/etc/itus/lists/$filter" >> /mnt/ramdisk/ITUS_DNS.tmp
done

#########################################################################################
# Check to see if ITUS_DNS,tmp is blank or missing and if yes skip, the file can be     #
# empty if no rules are ticked in the gui causlsing error. #
#########################################################################################
#########################################################################################
# Sorts rules in memory, then delectes duplicate one and then copies back to #
# /etc/ITUS_DNS.txt #
#########################################################################################

[ ! -f "/mnt/ramdisk/ITUS_DNS.tmp" ] && { echo "Error: /mnt/ramdisk/ITUS_DNS.tmp file not found."; exit ; }

if [ -s "/mnt/ramdisk/ITUS_DNS.tmp" ] ; then  
# cat /etc/ITUS_DNS.tmp | sort | uniq > /etc/ITUS_DNS.txt
# rm /etc/ITUS_DNS.tmp
                cat /mnt/ramdisk/ITUS_DNS.tmp | sort | uniq > /etc/ITUS_DNS.txt
                rm /mnt/ramdisk/ITUS_DNS.tmp
else  
  echo "Error file appears to be  empty"
 
fi  




xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
next file  which i#ve changed update_blacklist.sh
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx



#!/bin/sh
#set +x
#################################################################################
# update_blacklist.sh                                                    #
# created By: Hans                                                            #
# Modified: 14th March 2016                                                    #
# called by: /etc/init.d/dnsmasq                                                    #
# Purpose: To retreive blockdomain ip and blacklist ip, compare and if changed                    #
#   update all rules with new ip.                                                    #
#  changes:roadrunnere42 Added checks for ramdisk, error checking for missing                    #
#   or blank files, corrected loading errors, added comments.                                    #
# changes: Hans created                                                                #
#################################################################################

############################################################################################################################
# Gets the blockdomain ip from uci and assigns to blockdomain. added echo $blockdomain_ip just to check # That they is an ip
############################################################################################################################
ip_regex="[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+"
blockdomain_ip=$(uci get network.blockdomain.ipaddr)
echo $blockdomain_ip " this is the blocked domains ip"  # added as display point for checking only

############################################################################################################################
# Check to see if the file ads is present and not empty
############################################################################################################################

        if [[ -f "/etc/itus/lists/ads" && -s "/etc/itus/lists/ads" ]]
                then
# Get the ip address from the first entry in the ads list, added echo $blacklist_ip just to display ip
                blacklist_ip=`head -1 /etc/itus/lists/ads | cut -d'/' -f3`
                echo $blacklist_ip " this is the blacklist ip" # added as display point for checking only
        else
                echo "Error file appears to be missing or empty"
        exit
        fi

############################################################################################################################
# check if blockdomain_ip and blacklist_ip and blockdomain_ip is nor equal to blacklist_ip
# think this is used when the ip of the blocked domain changes and all the rules have to
# be updated with new ip
############################################################################################################################

if [[ `echo $blockdomain_ip | grep -o $ip_regex` && `echo $blacklist_ip | grep -o $ip_regex` && "$blockdomain_ip" != "$blacklist_ip" ]]
then    

# Process blacklist in parallel to increase performance
# blacklist=`echo "porn drugs gambling proxies dating blasphemy racism malicious piracy social ads illegal"`
# blacklist is now pulled from /etc/config/e2gaurdian so allowing only the ones that are select to be downloads.
# & at end of list alowing process to run in background

        blacklist=`grep content_ /etc/config/e2guardian | grep \'1\' | cut -d "_" -f 2 | cut -d ' ' -f 1`
        for list in ${blacklist}
         do
                if [ ! -d "/mnt/ramdisk/$list " ] ; then # check if the rule folder is in ramdisk,if not copy over.
                cp   /etc/itus/lists/$list /mnt/ramdisk/$list      
                fi
                           
    # sed -i -E "s/\/[0-9]+.[0-9]+.[0-9]+.[0-9]+$|\/$/\/$blockdomain_ip/g" /etc/itus/lists/$list &
                echo /mnt/ramdisk/$list  # added as display point for checking only
                sed -i -E "s/\/[0-9]+.[0-9]+.[0-9]+.[0-9]+$|\/$/\/$blockdomain_ip/g" /mnt/ramdisk/$list &

        done

        # Wait for the last process to complete before exiting
        wait

############################################################################################################################
# Run through rule list and copy back to /etc/itus/lists/$list   #
############################################################################################################################

        for list in ${blacklist}
        do
                mv /mnt/ramdisk/$list /etc/itus/lists/$list
        done

echo "finished"
        logger -s "update_blacklist" -t "Updated redirect ip address: $blockdomain_ip"
fi



roadrunnnere42

Hi Roadrunner,

sorry, planned:

20)     = IPS - LOG PROBLEM
        > /etc/snort/snort.conf
        - disabled preproc_rules for preprocessor, decoder and sensitive date
        - ref http://itus.accessinnov.com/Speed-issue-due-to-log-size-too-big-SOLUTION-td189.html

but I did not commit, see

include $RULE_PATH/local.rules
include $RULE_PATH/snort.rules
include $PREPROC_RULE_PATH/preprocessor.rules
include $PREPROC_RULE_PATH/decoder.rules
include $PREPROC_RULE_PATH/sensitive-data.rules
# include $SO_RULE_PATH/so1.rules
# include $SO_RULE_PATH/so2-misc.rules

Please use the < raw> and < /raw>  (remove first space) commands to display code, it makes the posting easier to read.

do you want me to change this in snort.conf, snort7.8 too?

Hi,

It would actually be snort_bridge, snort7, and snort8. I'll put up updated ones shortly which fixes a few bugs and has optimizations.

Please rollback the hotfix and fw_upgrade to V5 until we find the bug in V6. For some reason several users are having duplicate rules in their rulefile which causes snort to not load. Since snort handles the bridging of the interfaces, if it doesn't load you have no WAN connection.

All,

I haven't run the Hotfix Script, but I do have some cherry picks from the script that I ran when I first received the hotfix and a couple of cherry picks I've run by hand from this latest script.

Q: Given that I already have some of these fixes, but not all, if I run the script again will it cause any problem? TIA

Running the hot-fix will just over write the files on the shield, with the one in the hot-fix, it should work ok unless you have modified the files that the hot-fix is over writing.


roadrunnere42

Hi Wisywig, all,

as Roadrunner42 said, the hotfix will only overwrite files. These files are all scripts; they do not impact localized settings.