How long will the eMMC flash memory last on our shields?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

How long will the eMMC flash memory last on our shields?

user8446
Administrator
As we all know, flash memory has a finite number of write cycles before bad blocks start to appear. I have no idea if the controller on the shield has wear leveling or not. That being said, if you are not using the web filter on your box I would comment out the updates for that as the whole thing gets downloaded and overwritten every time the update runs, decreasing the life of the memory. The whole IPS ruleset gets downloaded too instead of just the changes. I've attached the /sbin/fw_upgrade with the web filtering commented out saving memory writes and it runs much quicker too, about a minute and a half to finish and Snort is only offline for about 10 seconds (in bridge).

fw_upgrade.fw_upgrade
Running in bridge mode, 1.51 SP1 fw
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

Wisiwyg
This post was updated on .
Thank you for this... I'm not using the web filter functions on mine in bridge - relying on built-in functions on the router making use of Yandex free web filter services.

edit: I ran this last night - replaced the standard fw_upgrade with this modified script. The Status pages indicates IPS signatures updated Feb 10, Web Filter updated Feb 9, and Shield Update Last Run Feb 9. So it looks like the IPS and WF date edits are working, but the last update run isn't. I'll see if I can figure it out.

Otherwise, thank you again for these edits!  

On another note, I haven't cracked it open to look, without opening it - does anyone know if the eMMC is soldered or socketed?
Shield Pro v1, Chaos Calmer, FW 1.51 SP1, v8.3.2, Bridge Mode
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

Roadrunnere42
In reply to this post by user8446
Hi user8446

That's a good point, what we need a someone who's good at scripting  to change the fw_upgrade script to,

1) download say the snort rules but only to ram
        curl -ko /tmp/botcc.portgrouped.rules https://rules.emergingthreats.net/open/snort-edge/rules/emerging-botcc.portgrouped.rules
        curl -ko /tmp/botcc.rules https://rules.emergingthreats.net/open/snort-edge/rules/emerging-botcc.rules
        curl -ko /tmp/ciarmy.rules https://rules.emergingthreats.net/open/snort-edge/rules/emerging-ciarmy.rules
        curl -ko /tmp/compromised.rules https://rules.emergingthreats.net/open/snort-edge/rules/emerging-compromised.rules
        curl -ko /tmp/dshield.rules https://rules.emergingthreats.net/open/snort-edge/rules/emerging-dshield.rules
        curl -ko /tmp/emerging-exploit.rules https://rules.emergingthreats.net/open/snort-edge/rules/emerging-exploit.rules
        curl -ko /tmp/emerging-malware.rules https://rules.emergingthreats.net/open/snort-edge/rules/emerging-malware.rules
        curl -ko /tmp/emerging-mobile_malware.rules https://rules.emergingthreats.net/open/snort-edge/rules/emerging-mobile_malware.rules
        curl -ko /tmp/emerging-user_agents.rules https://rules.emergingthreats.net/open/snort-edge/rules/emerging-user_agents.rules
        curl -ko /tmp/emerging-web_client.rules https://rules.emergingthreats.net/open/snort-edge/rules/emerging-web_client.rules
        curl -ko /tmp/emerging-worm.rules https://rules.emergingthreats.net/open/snort-edge/rules/emerging-worm.rules
        curl -ko /tmp/emerging-current_events.rules https://rules.emergingthreats.net/open/snort-edge/rules/emerging-current_events.rules

then run the 3 command below on the file that s in ram

sed -i 's/alert /drop /' alert.list
sed '/^\#/d' alert.list >> temp.rules
sed '/^$/d' temp.rules > snorttemp.rules

then check if the line exists in snort.rules that’s saved on the shield,if it does check the next line and so on. If the line does not then add it to snort.rules.

this could also be done for the ads and malicious rules, at present it have
206857 asd rules
76843 mailcious rules
4388 snort rules
these change daily sometimes going up then sometimes going down.
In the long team it will surely help save the eMMC.

Also the fw_update script should have a version number added so people would know what version they are using and what was changed, again the the short team it's fine but as time goes on it becomes hard to track with a version number.

Andy
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

Ronniem1
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

Roadrunnere42
Hi Ronniem1

All that script does is not download any rules for Ads and Malicious rules, It still downloads the snort rules as i believe user8446 uses something else  to block ads and malicious site.

Andy
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

Hans
Administrator
I've made a small change to this fw_upgrade script, see http://itus.accessinnov.com/Update-script-fw-upgrade-td43.html

Also my cron job to update these rules are not daily anymore, I've set it to weekly for now.
Using Shield Pro v1, Chaos Calmer, FW 1.51 SP1, Bridge Mode

2nd Shield as Sandbox, Chaos Calmer, FW 1.51 SP1 + hotfixes
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

Roadrunnere42
Hans
Don't now if you have seen the massive drop in ads rules in last nights download, i have been keeping track of how many lines in the rules,

Ads
206857 going up and down slightly but last night it drop to 157555, the malicious and snort rules say about the same.

I use
wc -l /etc/itus/lists/ads
wc -l /etc/itus/lists/malicious
wc -l /etc/snort/rules/snort.rules

Have run fw_upgrade thinking maybe it stop half way through downloading but no it the right amount of rules.

Andy
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

Hans
Administrator
If I run
wc -l /etc/itus/lists/ads /etc/itus/lists/malicious /etc/snort/rules/snort.rules

then i get

   157555 /etc/itus/lists/ads
    27778 /etc/itus/lists/malicious
     4400 /etc/snort/rules/snort.rules
   189733 total

same as you
Using Shield Pro v1, Chaos Calmer, FW 1.51 SP1, Bridge Mode

2nd Shield as Sandbox, Chaos Calmer, FW 1.51 SP1 + hotfixes
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

user8446
Administrator
This post was updated on .
I have:

0 /etc/itus/lists/ads
0 /etc/itus/lists/malicious
4421 /etc/snort/rules/snort.rules
4421 total

Go into /etc/itus/lists/  and clear out social, racism, proxies, porn, piracy, malicious, illegal, gambling, drugs, dating, blasphemy, and ads to free up memory
Running in bridge mode, 1.51 SP1 fw
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

user8446
Administrator
In reply to this post by Roadrunnere42
Another good GUI command line that I use is: /etc/init.d/log restart

Clear out your system log.
Running in bridge mode, 1.51 SP1 fw
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

Ronniem1
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

user8446
Administrator
The update script or the command line commands?
Running in bridge mode, 1.51 SP1 fw
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

mbohlmann
In reply to this post by user8446
Regarding the eMMC flash memory, It is just possible that Rhino Labs could be helpful in some way. It seems that Rhino manufactured the Shield for ITUS. They also offer for sale a big brother to the Shield, an enterprise-class version:
http://www.rhinolabsinc.com/rhino-utm8-networking-appliance/
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

user8446
Administrator
And check out the one sitting on the top. It's the shield in a new skin!  http://www.rhinolabsinc.com/sdna-7890/
Running in bridge mode, 1.51 SP1 fw
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

Hans
Administrator
user8446 wrote
And check out the one sitting on the top. It's the shield in a new skin!  http://www.rhinolabsinc.com/sdna-7890/
It is a RED shield - what does this mean for us?
Using Shield Pro v1, Chaos Calmer, FW 1.51 SP1, Bridge Mode

2nd Shield as Sandbox, Chaos Calmer, FW 1.51 SP1 + hotfixes
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

user8446
Administrator
This post was updated on .
It would be nice to get a hold of the OpenWRT image on it that takes advantage of the processor offloading. We can then load snort on there and all of our changes and hotfixes on top of it.
Running in bridge mode, 1.51 SP1 fw
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

mbohlmann
user8446,

Since Rhino Labs' contact page appears to want enquiries from companies, not individuals, would you be willing to contact them -- perhaps wearing your company hat?

Determine what it would take for Rhino to adopt us Shield orphans under their software support umbrella.
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

user8446
Administrator
Done, email sent. I'll post what they respond.
Running in bridge mode, 1.51 SP1 fw
Reply | Threaded
Open this post in threaded view
|

Re: How long will the eMMC flash memory last on our shields?

mbohlmann
In reply to this post by user8446
user8446,

Possible answer to your question: "...I have no idea if the controller on the shield has wear leveling or not..."

The Wikipedia entry for OpenWrt says this about the file system:

"A writable root file system, enabling users to add, remove or modify any file. This is accomplished by using overlayfs[23] to overlay[24] a read-only compressed SquashFS file system with a writable JFFS2 file system in a copy-on-write fashion. JFFS2 supports flash wear leveling."

https://en.wikipedia.org/wiki/OpenWrt