Shield - custom build environment

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Shield - custom build environment

hans2
This is a backup of the packetinspector.org forum about making a custom build environment / installing own software in Shield. (thanks Breda for the backup!).

ITUS - Garrett wrote
As you are aware what you are trying to do it outside the scope of support for the Shield Pro as it comes from Itus but I wanted to provide as much info as possible to help you in your endeavors.

"You shouldn't need to recompile the entire kernel, just the relevant package & any dependencies.  Once compiled you should be able to just copy the files onto the box & run them.

Yes, kernel was compiled as big endian with cavium octeon mips64 toolchain

How to build OpenWrt - http://wiki.openwrt.org/doc/howto/build
There's support for the octeon toolchain already but you may need the one from cnusers.

Guide on cross compiling node.js - https://github.com/netbeast-co/docs/wiki/Cross-Compile-Nodejs-for-OpenWrt
Other guides.. http://wiki.openwrt.org/doc/howto/nodejs  (looks like this project is abandoned)"
I would like to do this but I can't do it alone.

My thoughts are:
1) Install OpenWRT in virtual box (link)
2) Rebuild the environment based on notes above
3) Start adding other packges (speedtest cli ...
4) Update the 151SP1 restore image with latest corrections/tools


Who has done something similar before?



No more: Shield Pro v1, Chaos Calmer, FW 1.51 SP1
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

user8446
Administrator
I wish I had experience here. I can't even get DNSCrypt working because of a missing dependency. OpenWRT forums have been no help.
Running in bridge mode, 1.51 SP1 fw
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

Wisiwyg
I think this starts with getting access to cnusers.org and obtaining the cavium Octeon III optimized tools for OpenWRT and IPS (Snort & Suricata) that are mentioned in the Cavium website. I tried but my request wasn't approved.

I'd like to upgrade Snort to the latest 3.0 version if possible.

Also, Snort rules are available directly from snort.org in 3 tiers - 1) public, free, 2) registered, free and 3) subscription. It shouldn't be too difficult to setupthe fw_upgrade script to pull from the registered, free ruleset. I'm assuming that the 1) tier is similar to or a duplicate of emergingthreats rules.
Shield Pro v1, Chaos Calmer, FW 1.51 SP1, v8.3.2, Bridge Mode
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

user8446
Administrator
I know there is  "pulled pork" and "oinkmaster" rules managers if you want to pull rules from snort. From what I've read they are pretty close as data is shared just like with antivirus.
Running in bridge mode, 1.51 SP1 fw
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

mbohlmann
In reply to this post by hans2
Hans,

I guess we would need access to the special Cavium Octeon distribution of OpenWrt which is, “'performance-optimized' for the CN71XX’s acceleration engines..." This is the OpenWrt used by ITUS for Shield:

http://linuxgizmos.com/cavium-adds-openwrt-support-to-octeon-iii/
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

wallaby13


mbohlmann wrote
Hans,

I guess we would need access to the special Cavium Octeon distribution of OpenWrt which is, “'performance-optimized' for the CN71XX’s acceleration engines..." This is the OpenWrt used by ITUS for Shield:

http://linuxgizmos.com/cavium-adds-openwrt-support-to-octeon-iii/
I set up a build environment tonight it looks like it includes Octeon support. I am planning to build a replica of the shield software (in x86 mode for vm testing) using the build environment (it looks to have most of the packages). Then i will need to figure out if or how to inject the current configs (like the update script, html pages and alike) so it matches 1.51sp1 even more. I think that will be as close as i will get to the same build as what we currently have then we can begin modding it. Maybe start simple by removing the branding. I have already spotted some packages that are pre-built and ready to add that may be useful too. Sadly i only have one device (but was meant to get a second) so much of my testing is going to be in a VM. I kind of don't want to turn my device into a brick though. anyone game to be a lab rat?
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

hans2
This post was updated on .
Hi Wallaby

Cavium has not released any SDK (http://www.cavium.com/css_ids_ips_stk.html) yet - how did you setup your VM?
i have a 2nd Shield that I am using for testing my scripts - happy to help out.

update: if i can get it to work again, even Console is not producing data ATM. Will try tomorrow.

cheers, Hans
No more: Shield Pro v1, Chaos Calmer, FW 1.51 SP1
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

user8446
Administrator
When setup, would you mind trying to load Dnscypt? It's already in the repo. I've tried loading it on the shield and it needs the libssp library but I don't know what dependency it's from.
Running in bridge mode, 1.51 SP1 fw
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

Roadrunnere42
In reply to this post by wallaby13
Hi

i have a second shield which  broke through beta testing sp1, managed to bring it back to life and so what the hell lets start testing.

roadrunnere42
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

Roadrunnere42
In reply to this post by user8446
Hi Hans

Sorry to hear about trouble with your second Shield, if you need a hand to get sorted just ask.

roadrunnere42
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

wallaby13
This post was updated on .
In reply to this post by hans2
@Hans
like you say it looks like there is an SDK coming soon it looks like the SDK is for performance improovement which will be great but for now lets at least start by trying to mimic what we have got then move to making it better.

As for setting up a build environment i started with a ubuntu server base with lxdm (for the prittyness) and followed the build environment setup guide on the openwrt website(OpenWrt build system – Installation). The environment  allows me to select Octeon as the target operating system though as noted above not all Cavium Octeon processors will work with this target and we currently don't know if our processor is. This is going to be the bricking point.  I didn't delve into this target option too much last night as it was getting on midnight by the time i had everything running as i wanted but maybe we will have a direct CN7020 sub-target to get things compiled just right (i suspect we might have to work with the lgeneric Octeon though).

I note the cpu is at least partly based on a MIPS64 instruction set so tonight i plan to set up a vm environment using Qemu as it appears to be the only virtual i can get my hands on that will do MIPS64. By running the compiled image through x86 then MIPS64 i hope to be able to work out the majority of issues before we endanger any blue boxes. I suspect we might be able to port some of the scripts and code to some routers as well although they won't have a cpu designed for inspecting packets as well inside i guess. This is of course a little ways down the track.

I plan to attempt to run the current image in Qemu before trying a custom compile too.

@Roadrunnere42
Do you remember how you got your device working again. This could help figure out how to un-brick a device should we run into problems.

@everyone
This is going to be a slow and challenging task it is bigger than any of the kernel modding i have done before. I have no idea if we will even be successful but i hope it will be and want to make this work or something similar (like porting to another router).

Oh and i want dnscrypt too. I did a glance over the build system package inclusions while installing but didn't see it off the bat. It is probably still there but hidden away in a sub directory somewhere.
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

hans2
Hi wallaby

i'm checking the Cavium OpenWRT reference board image to see what I can do. However this is a first for me too and while scripting is not too difficult, building a complete image is going to be a steep learning curve for me. You may want to join cnursers.org (acceptance may take a while).

cheers, hans
No more: Shield Pro v1, Chaos Calmer, FW 1.51 SP1
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

MAHDTech
I managed to get access to cnusers.

I have downloaded the latest OpenWRT version Cavium has released which can be used as a Base OS (v1.7)

I am in the process of uploading the files now, and its shared via my OneDrive for the people who don't have access to CNUsers yet.

https://onedrive.live.com/redir?resid=FF3D3666ED1A2CD%21180773

I don't have experience building OpenWRT images althought I have built multiple FreeBSD images, and thought I would try building one for the Shield, unfortunately the Octeon III support isn't in FreeBSD Current yet and adding that support is outside my skill set :(

Otherwise NanoBSD would have been a great fit.
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

MAHDTech
Getting closer, I have posted where I'm at in the customisations section of the forum!
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

user8446
Administrator
I see that starting in v1.6 the Cavium offloading module is supported which according to Daniel @ Itus this would double throughput and performance. I wish I could help but I have no experience in this area.
Running in bridge mode, 1.51 SP1 fw
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

MAHDTech

We will need plenty of testers, so you can definitely help there!
Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

Roadrunnere42
In reply to this post by MAHDTech
Hi MAHDTECH

Have you got a copy of OpenWRT version Cavium as i have tried your link but the file is no longer available

https://onedrive.live.com/redir?resid=FF3D3666ED1A2CD%21180773

roadrunnere42

Reply | Threaded
Open this post in threaded view
|

Re: Shield - custom build environment

NetNoggin
In reply to this post by MAHDTech
MAHDTech wrote
We will need plenty of testers, so you can definitely help there!
New guy here.  I've got access to several Shields that I can test with.  I had originally planned to keep one w/my laptop, and to partition my home network with the rest.   I want to get these little dudes into action.  Only one of them has 1.5 sp1.  The others are cherry.  

I'm no programmer, but I've been working on corporate networks longer than I care to say.  I'm familiar with UTMs.  How can I help?

I'll spend some time tonight reading to catch up with the state of things.