[FIRMWARE] Itus Networks Shield Firmware Upgrade *WIP*
Locked
12345
... 12
12345
... 12
Re: [FIRMWARE] Itus Networks Shield Firmware Upgrade *WIP*
 
|
Hi Here's the code that Itus used to determine the switch position if that helps. Putting only one image and then adjusting the the config files does make sense but i have to ask why didn't Itus do that? I will send you the tgz files. Roadrunnere42 ###################################################################### ###### Function To Determine Shields Operating Mode (All Modes) ###### ###################################################################### determine_shield_mode () { if [ `df -h | grep mmcblk* | awk '{ print substr( $0, 6, 14 ) }'` ]; then disk_partition=`df -h | grep mmcblk* | awk '{ print substr( $0, 6, 14 ) }'` if [ $disk_partition = mmcblk0p2 ]; then shield_mode=Router elif [ $disk_partition = mmcblk0p3 ]; then shield_mode=Gateway elif [ $disk_partition = mmcblk0p4 ]; then shield_mode=Bridge else echo -e "Unable to Determine the Shields Operation Mode (Router, Bridge, Gateway) -- \e[31m**SCRIPT FAILED**\e[39m (determine_shield_mode)" >> $script_log_file fi else echo -e "Unable to Determine the Shields Operation Mode (Router, Bridge, Gateway) -- \e[31m**SCRIPT FAILED**\e[39m (determine_shield_mode)" >> $script_log_file fi } On Mon, 20 May 2019 at 20:06, Grommish [via Itus Networks Owners Forum] <[hidden email]> wrote: Ok.. I need some feedback. |
Administrator
|
The Itus script only sets the "mode" based on the mmcblk0p block that is loaded as the /extroot. They went lazy about it, i guess. I was going to use the GPIOs to determine the switch position
GPIO16=0 is Gateway GPIO16=1 and GPIO17=1 is Router GPIO16=1 and GPIO17=0 is Bridge For the record, GPIO9/10/11=0 are the Front "M" LED (Orange/Green/Cyan) You can see it yourself by scripting out the following (taken from my test image rc.local - it sets the LED on at the tail end to signify boot complete):
|
|
Administrator
|
Well, I took the next step and started from scratch on the OpenWrt image. I hosed my external Linux SSD when I installed it and didn't install Grub on it, and it was being such a PITA to get it installed there, i just decided to take the chance to start over.
I'll put up an updated image later on and a fresh Changelog, but I added Snort.
|
|
Administrator
|
This post was updated on .
In reply to this post by Grommish
For anyone who is decent with bash scripting, I'm trying to do the following, but it keeps returning empty in the init script.
|
Administrator
|
This is Roadrunnere42's post... it was causing a redirect on the thread so I had to delete it....
Hi I tried following what you said about running the new gatewayimage but i can't get it to work. I don't have a dev device called mmcblk01 in the dev folder, can you confirm the device name? Roadrunnere42 mount /dev/mmcblk01 /overlay then on your PC scp ~/Downloads/ItusgatewayImage root@10.10.10.10:/overlay As an alternative, I've loaded gkermit on the image so you could use minicom to console transfer. Back to the Shield. Switch the Front-switch to Gateway (Outer) mode umount /overlay reboot -f LuCI - Lua Configuration Interface On Sat, 25 May 2019 at 20:58, Grommish [via Itus Networks Owners Forum] <[hidden email]> wrote: For anyone who is decent with bash scripting, I'm trying to do the following, but it keeps returning empty in the init script. CUR_FS=$(blkid /dev/mmcblk1p3 | awk -F= '{print $3}'); echo "[DEBUG] Shield is in ${SHIELD_MODE} Mode - Using /dev/${MNT_POINT} as ${CUR_FS}" > /dev/kmsg When I run the command on the box in the shell itself, I get the following: root@OpenWrt:/# $(blkid /dev/mmcblk1p3 | awk -F= '{print $3}'); /bin/ash: "f2fs": not found "f2fs" is what it SHOULD report, so that's right (although any suggestions on stripping those double-quotes would be very helpful) However, what appears in the kernel log: [ 34.828073] [DEBUG] Shield is in Gateway Mode - Using /dev/mmcblk1p3 as Eventually, I'll be replacing the hardcoded mmcblk1p3 with another variable that's set by reading the GPIO settings. GPIO=0 GPIO16=1 GPIO17=2 GPIO=16 echo $GPIO > /sys/class/gpio/export echo in > /sys/class/gpio/gpio$GPIO/direction GPIO16=$(cat /sys/class/gpio/gpio${GPIO}/value) echo $GPIO > /sys/class/gpio/unexport GPIO=17 echo $GPIO > /sys/class/gpio/export echo in > /sys/class/gpio/gpio$GPIO/direction GPIO17=$(cat /sys/class/gpio/gpio${GPIO}/value) echo $GPIO > /sys/class/gpio/unexport if [ $GPIO16 -eq 0 ]; then # Gateway Mode SHIELD_MODE="Gateway" MNT_POINT="mmcblk1p3" else if [ $GPIO17 -eq 0 ]; then # Bridge Mode SHIELD_MODE="Bridge" MNT_POINT="mmcblk1p4" else # Router Mode SHIELD_MODE="Router" MNT_POINT="mmcblk1p2" fi fi This allows us to read the Front-switch state and load the config files as appropriate (and/or just load the appropriate mmcblk for the mode) Getting there! My next steps are to ensure the file system is correct (which is why I really need to get the awk command working), format the block if it isn't setup for f2fs, and copy a seed tgz package to the mount point to get things started. This should let anyone who is coming from the stock Itus images to easily convert over. Anyway, I know the couple people who are actually following this are away on travel or other things, but maybe they can spot the error I'm having. Cheers!
Running the latest OpenWrt stable release
|
|
Administrator
|
I've updated the original post and second post.
I'm at a stand-still at the moment because I can't get the block device (mmcblk) to mount any earlier than after init. I can't figure out where in OpenWrt to change the default /init file. Currently, the mmcblk isn't mounted or even available in /dev until AFTER /etc/rc.local finishes.. It is literally the last thing to come up. Without being able to bring this up sooner means I have no way to set the rootfs_data and no changes to the filesystem will ever be saved. I'm missing something and it's frustrating. Anyone that can offer suggestions, I'd be more than happy to try them.
Running Itus Shield v2 Firmware
|
|
Administrator
|
Update 6/14/2019:
I finally got the blasted MMC block to mount! I can finally start the real fun stuff. The good news is that the *core* image seems to be about finished and changes to the extroot survive reboots! *happy dance*. Now i just have to get the image set for multi-boot. Serious question time - I need to get info from anyone with an opinion on what to add. Also, I'm going to start thinking on ways to make it user proof. Updates to follow. Also, for anything NOT kernel related, the posted image will work for tests. It still isn't destructive, but that means nothing survives reboots. There is way more setup I've got to do in the scripts to make it an all-in-one.
Running Itus Shield v2 Firmware
|
Re: [FIRMWARE] Itus Networks Shield Firmware Upgrade *WIP*
|
|
Great work Grommish, you must be spending sometime on this project. On the question of things to add I would say intrusion prevention (snort or equivalent) this was one of the advantages of using the shields cpu, Web Filters, vpn, not sure if it's possible to have traffic management not qos but speed throttling seen it on tomato firmware routers, you just select the ip address and assign a download speed, not sure if this is part of openwrt or not. keep up the great work. Roadrunnere42 On Sat, 15 Jun 2019 at 03:12, Grommish [via Itus Networks Owners Forum] <[hidden email]> wrote: Update 6/14/2019: |
|
Administrator
|
Snort is already loaded, but while it's called, I need to tweak the command line (--daq-dir /usr/lib/daq for instance). Also, I've not really loaded any rulesets or baked in update scripts for the rules. This is where I really, really could use assistance from folks like user8446 and the rest of the really smart folks to make this work well.
Anything I bake into the main image will be available in all three modes. Differences can be handled down the line, like network setups, configurations, etc. I've gotten full mode detection by the front switch (the system reads the front switch setting to figure out what mode it SHOULD be in during boot), and the system will do a half-assed initial setup on the mmcblk partition used by the mode (it copies the rootfs from the image to the block device and sets a /extroot/.gribbits text file to show it's been done). I'm thinking of making "setup packages" to include with the image that are .tgz'd for a given mode (one for router.tgz, bridge,tgz, gateway.tgz) that self-contain any configuration settings and whatnot and just extracting them as an overlay to the extroot. Anyone have any thoughts on it? Or anyone have any suggestions on the best way to get what we want? With the base about done, everything we do is on us and we aren't restrained by OpenWrt or the Itus images. I am also thinking about trying to update uboot, since it's 5 years old, but I don't know if it's worth the hassle since its works. It would allow a wider range of things, but, eh, maybe for down the line. I should mention that full opkg support is already in the system, so people CAN add extras as they see fit (and will be saved now in the extroot). I will look for the bandwidth throttle you were talking about, but if you find a name, let me know since it makes it easier. As for non-standard OpenWrt packages, that's alright too, since I can just compile whatever from source to include it. That shouldn't be an issue. I find it comical that this image will probably never be seen by more than a dozen people, if that. People got rid of their Shields or stopped using them. Maybe once we get things settles, I can convince user8446 to see if there is a way to send out a site announcement email to the registered users inviting them to take a look.
Running Itus Shield v2 Firmware
|
|
Administrator
|
This post was updated on .
Worked through some Snort! config issues. It should be noted that this is almost the stock configuration file, BUT I'm using the ENTIRE Emerging Threats rules (https://rules.emergingthreats.net/open/snort-2.9.0/emerging-all.rules) because it's only a single file and was easy to wget. Not sure this would be the best solution long term, but i figure it should be alright for testing.
I did not see a decrease in speed or latency times (see below), and the system load is reading 0.00, 0.10, 0.14 - which is fantastic considering.
Running Itus Shield v2 Firmware
|
|
In reply to this post by Grommish
Wow - nice work Grommish!
You've gone well past my capabilities with any of this.. I'm a .NET & Angular dev, so am comfortable following & debugging existing code - but doing all this from scratch? not a chance! RE: suggestions for extras, perhaps: * some of the other (non emergingthreats) blocklists covered by the fw_ugprade script that "legacy" installs run weekly - e.g. shallalist, sslipblacklist, zeus, etc. * is it feasible to get the OpenWrt distro & other installed packages to auto-update? Too much risk of breakage? * not sure if it's mature enough, but luci2 might also be worth a look. Keep me posted once you think the router mode install is "user-proof" enough for a Windows user ;)
OpenWrt SNAPSHOT, r10391-3d8d528939
|
|
Administrator
|
This post was updated on .
The Emerging Threats rules were just the first ones I found. Any Snort rules can be used. Updates can be scheduled via Cron job as long as the URL is reasonably scriptable.
By all accounts, once I get everything sorted, we should be able to do a few things. First, opkg has an upgrade option, if I recall correctly. So you could schedule the Cron job for that. More importantly, turning out system updates for use in the luCi interface should be doable, as well. As far as testing, the "Gateway" image I'm building is Gateway in name only. The first post has info on how to install. It's called Gateway because uboot looks for Itus<mode>Image (ItusgatewayImage, ItusrouterImage, etc) depending on the front switch at the end of Stage 2 into Stage 3 boot. The test image is set to act like the router image, it just goes in the Gateway slot because no one ever used that slot. It's also non destructive, so it doesn't mess with the /dev/mmcblk partitions (at this point. My current test builds I have locally do.). So you're perfectly safe in loading the test image in post 1 and just backup your existing ItusgatewayImage file before hand. The test file doesn't do Snort by default, and won't save anything through a reboot of the device. Snort is present, but the /etc/snort/snort.conf isn't right and it dies when it tries to start (silently). You can still call it manually from the command line to test it. I've got it working on the local test builds, but again, it'll overwrite your Gateway stuff (you can restore it later, if you decide to). It doesn't effect the Router image, at all. If you really want to play with it, I'm always willing to walk you through it in something like Google Hangouts. Just send the request to my email (grommish@gmail.com) and we can set aside some time. Any additional eyes I can get on this would be extremely helpful since I never played with the Shield during the hayday, and almost immediately decided to start this little adventure after only like 1 or 2 days of use. I never even hooked it inline on my network, only console and stub use. I can look at luCi2. I didn't see a config option in the source, but I've not synced in a few weeks. Edit: luCi2 can be installed, I suppose, but the dev on it looks kind of janky. Last commit was 2 months ago. While I could put it in the image, it's beyond me to make the pages. I'm no good with JSON or even Js. I also don't know what issues and conflicts luCi and luCi2 might have. I'm all for bleeding edge, but for now, we should probably stick to tried and true. Besides, the luCi limitations in space and RAM don't apply to us.
Running Itus Shield v2 Firmware
|
|
Administrator
|
I was correct in opkg having an update/upgrade options:
I'm sure we can script it to auto-upgrade
Running Itus Shield v2 Firmware
|
|
Administrator
|
This post was updated on .
It seems that every time I get on a tear, the posts just come back and back.
First, I finally managed to get Snort to more or less work properly, although I'm reasonably sure it's just alerting, rather than dropping, matching packets. I'm sure there is a setting for that somewhere. I didn't realize the map files were actually needed for things  . Snort comes with gen_msg.map, but the sid_msg.map from ET was also needed - silly me..
Luckily, it seems to have fixed the detection issues for the most part. Of course, since I'm using ALL the rules, it picks up on more than just threats (which I was surprised to see). It's surprisingly difficult to trigger Snort when you're testing it 3 layers deep in a private network. NAT within NAT within NAT means not much is going to "go rogue" and hit the Shield. First thing I tried was a DNS query on a questionable domain. Then I did a So, that seemed to verify it was working. Then, I got the following, which was completely unexpected. Happened when my Ubuntu laptop went and did a Software Update. So, yeah, seems to be working (at least the detection part, and at least to Console). And in case anyone is wondering, Snort is using both ALL the Emerging Threats AND the Snort Community rules AND my one local rule without issue..
I have no preprocessors, no so_rules, no reputation monitor, and it's not running inline. These are just straight-up .rules files. (I'd love to get the rest of it working, if I understood what they did and how to do it). **** Issue 2: I saw on the Itus firmware they used e2guardian to web-filter. Is this something we want to put back in? I included it, or at least tried to, on a local build. It says it put it in there, but I'll be damned if i can find it. I was looking at the fw_update file Gnomad mentioned and saw it was the site restrictions were being updated by it, so I thought I'd ask. **** Issue 3: Anyone know LUA who wants to try and make a luCi page for Snort/e2guardian rules as far as active/updated? I doubt anyone, including me, wants to use the ET all ruleset. I can work with you on the directory structure and we could include all of the individual files. Perhaps the luCi page can let you toggle which rule files you want to use (read from the directory) and then update a .conf file (which we can call from the main snort.conf). This allows granularity for each user. But, I don't know LUA, so.....
Running Itus Shield v2 Firmware
|
|
Administrator
|
This post was updated on .
Update 6/19/19:
I figured out the e2guardian issue. It WAS being baked in, however it wasn't showing until I iron'ed out the external root issues. Currently, the system is setup like this: Stage 1 uboot, Stage 2 uboot, Stage 3. Stage 3 decides which of the three files (ItusrouterImage, ItusbridgeImage, ItusgatewayImage) to load based on the front-panel switch. Image loads. The init script figures out which mode the Shield is in by reading the GPIO values, sets an environmental export. I had to stall out init until /dev/mmcblk gets mounted, it then mounts the correct for the mode partition (/dev/mmcblk1pX, X=2-4, 2=router, 3=gateway, 4=bridge). I have it checking to see if a /.norwits file exists in the ExtRoot, and if it does, it hands root off to the external mount. If it doesn't, it initiates a "First boot" condition. This does a couple of things.. First, it copies the rootfs to the external mount, and then adds the .norwits file. Then it adds an export FIRSTBOOT flag. Finally, it hands root over to the external partition. I've added a rc.d/init.d file at S13 that checks the FIRSTBOOT flag, and if it is an initial boot, Runs the following structure.
|
|
Administrator
|
I was looking into OpenWrt's update system, and I don't think it's going to be able to be used for what we need to do. Meh.
On the flip side, I might be able to create an update system, although it looks like it would have to be two parts. One chain for the kernel, one chain for the filesystem. The update .tar file that the system generates is broken into those two sections, kernel and root. The kernel file is the standard ELF image, and I can't find where in the rootfs that would go. I doubt it would go in the rootfs at all, but would need to be in the boot image. The root file, however, is a squashfs file, so it can be unsquashfs'd and mounted. Question for the crowd: Does anyone have any scripting suggestions on how to compare two file-trees (I was thinking diff), figuring out which files are new or changed, but excluding things like .conf? I am thinking diff might be able to handle it, but I'm not sure about excluding .conf files that aren't NEW. Anyone with suggestions would be most appreciated.
Running Itus Shield v2 Firmware
|
|
Administrator
|
I put up my stuff on my GitHub, in case anyone wants to either look/suggest/play with their own version.
I pushed the entire OpenWrt repo and then committed my changes on top of that.. OpenWrt offers a "mirror" of their git, but I'm not sure how current it is. I'd rather just make sure I was up-to-date, and I can commit other changes down the line when I pull them from the upstream. https://github.com/Grommish/Itus_Shield_v2
Running Itus Shield v2 Firmware
|
|
|
In reply to this post by Grommish
Hi Grommish,
see commit history under https://github.com/ItusShield/Shield-Master/commits/master/usr/lib/lua/luci for LUA page changes from the original Shield, which includes a tab for users to specify which block lists they want applied. I can't remember the specifics of how that was saved, but if you clone the repo and diff against 2016-03-30 you should find the relevant files. Then the fw_upgrade.sh file referenced earlier picks up those settings to apply them. Similarly under this folder is a tab that allows users to backup or restore their configs via downloadable zip - might be able to replicate this mechanism with your repo.
OpenWrt SNAPSHOT, r10391-3d8d528939
|
|
Administrator
|
Thanks! So, you do the Lua *peer* haha.
One of the advantages to being able to actually work on-device now for a lot of things is that I can rob the bones of the Itus images in place. I can just mount the overlay for the other images and walk through things. I'm working on the Snort Lua stuff at the moment. I pulled the stuff, although interesting enough, it was either being setup for Suricata, or borrowed from the Suricata package.. I looked into getting Suricata in the build, but finally put it aside for now. Suricata doesn't support cross-compiling for mips, meh.. If they ever get it working, I can add it easy enough. I already have the package hooks for it. The rule files for the Itus image has Snort dropping rather than alerting like the public rules do. Do we want to continue with that? I'm down to learn new things like Lua, OpenWrt, Snort, etc, but if I have to sit down and learn functional sed and regex, it's going to slow my down considerably. I'm pretty sure you're the one that is tagging my repo for this, so if you feel froggy, let me know and I'll do the pull request :D if i had been smart, I'd have set up the git repo from the start, but eh. the important bits are there. It's been a long time since I used git (as you can tell from my other repos), so I'm still feeling my way around it again.. In any case, I'll work through it. I figure I'll work Snort first, since it is the main reason for the device. Then, maybe move on to e2guardian, and then go from there. I know the FIREWALL needs to be tweaked something fierce. I DMZ'd the shield and scanned it, and so many closed-but-not-stealthed ports.. Can't let this stand. And yes, the 80:443 open ports are not the Shield, which shouldn't be exposed anyway by default. Standard Scan  DMZ'd Shield 
Running Itus Shield v2 Firmware
|
|
|
Hi Grommish, I'm afraid your peer, tagging and froggy references have gone over my head! I've add your repo to my github favourites list, but not submitted any PRs (or even cloned it yet).
RE: interest in learning lua/sed etc, I've previously been okay tweaking working examples when my Shield was running & my head was in the game, but none of this is in my wheelhouse either m'afraid. Most of the commits I made to the old github repo were me pasting in the work of Roadrunner, user 8446 & others to track them for posterity. You're already well advanced beyond anything I've ever done with the hardware. As it stands before your efforts, I lost enthusiasm when my shield stopped working for the umpteenth time on me nearly a year ago, after one of the weekly fw_upgrade runs. I might be encouraged to plug it in again as a test user given a working image, but will need to find a clear weekend or two.. Sorry I'm not your man for anything more involved! Hopefully one or more of the other users can step up with other assistance.
OpenWrt SNAPSHOT, r10391-3d8d528939
|
«
Return to Technical Discussion
|
1 view|%1 views
| Free forum by Nabble | Edit this page |